oblem of local minima in EM for Bayes nets and in training of deep neural networks.  Why is it a problem in for one of them and not t

Explain in mathematical terms the process of supervised learning in neural networks and the results achieved by using such learning.

Give an example of a difficulty that develops when a neural network has several layers.Discuss overfitting and how to prevent it in the following paragraphs.

Give an example of a situation when a recurrent neural network might be preferable to a non-recurrent one, and explain

Compare and contrast the ways that Evolutionary Computation algorithms and Simulated Annealing escapes from local optima, i.e. • What do they have in common? • How are they different?

The figure below shows a fully connected neural network, with two hidden layers. a. What are the dimensions (ie. the size of the matrix, nxm) of the weights matrix (W) for the node highlighted in red? illustrate your answer. b. Answer the same question for the node highlighted in blue. X₂ X3 X4 X₂

Autoencoders are special type of neural networks trained to estimate identity functionTrueFalse

Consider the very small neural network below, with 2 inputs and no biases, and with the activation function being the identity function, f(z)=z. The input (1, -1) is presented to the network, as illustrated. The layer 2 activations have been calculated, and are shown. Now, what are the layer 3 activations, i.e., the outputs? inputs (1) a₁ = 1 x₁=1 X₂=-1 (1, -2) (1) az = -1 (-2, 1) (1, 2) W12 (2, 1) ‚(1)_ W11 (1) W22 W21 4 a₁ (2) az = 2 W21 (2) W 12 = 1 (2) Wil =-1 = 1 W22 = -1 - outputs a₁ =? (3) az

The following figure is a diagram of a small convolutional neural network that converts a 16x16 image into 4 output values (16x16 - 4x12x12 → 4x6x6 6x1). The network has the following layers/operations from input to output: convolution with 4 filters, max pooling, ReLu (after pooling), and finally a fully-connected layer (with no hidden layer). For this network we will not be using any bias/offset parameters. Please answer the following questions about this network. 16x16 Convolution 4 filters 5x5 Stride 1 4@12x12 max pooling 2x2 Stride 2 4@6x6 6x1 fully- connected

10. Describe the convolutional layer in neural networks. How is it related to regularization? 11. Describe how batch normalization is performed and in which way it helps with training multilayer neural networks with ReLU activation function. 12. What is the significance of the Representer Theorem for learning using kernels (e.g. Gaussians).