Suppose we are doing ordinary least-squares linear regression with a fictitious dimension. Which of thefollowing changes can never make the cost function’s value on the training data smaller?  A: Discard the fictitious dimension (i.e., don’t append a 1 to every sample point).  B: Append quadratic features to each sample point.  C: Project the sample points onto a lower-dimensional subspace with PCA (without changing the labels) andperform regression on the projected points.  D: Center the design matrix (so each feature has mean zero).

We've built a logistic regression model in RapidMiner, and would like to use it to make predictions for some new data points. Which operator do we need: Performance Apply Model Cross Validation O Nominal to Numerical.

• Implement and test: • Logistic regression (LR) with L1 regularization • LR is differentiable • But L1 norm is not • Use proximal gradient descent • For L1 norm, that’s soft-thresholding • Use tensorflow library • Dataset – the same as in HW2: • Classify two digits from MNIST dataset • See: tensorflow_minimizeF.py • Performs projected gradient descent on a simple function • The function has global minimum at • w1=-0.25, w2=2 • But the feasible set Q is: w1>=0, w2>=0 • For this function, the best solution is w1=0, w2=2 • The code does the following, in a loop: • Gradient step on the function, followed up by proximal step • Here, the proximal step is just “make w nonnegative” by replacing negative values with 0, the closest non-negative value • Feasible set Q is set of all vectors with nonnegative coordinates, i.e., for 2D, w1>=0, w2>=0 • In your actual code, you should use soft-thresholding instead • See: tensorflow_leastSquares.py • Performs gradient descent on a function…

This is a binary classification problem, y has two values (0 or 1), and X (feature) has three dimensions. • Use a logistic regression model to project X to y (classify X into two categories: 0 or 1). • The initialization is: w1 = 0, w2 = 0, w3 = 0, b = 0, Learning rate is 2. • You must use Gradient Descent for logistic regression in this question. • The regression should stop after one iteration.       Calculation process and formulas must be included in your answer! You must answer this question by manual calculation, but not programming.