Pacman is in an unknown MDP where there are three states [A, B, C] and two actions [Stop, Go]. We are given the following samples generated from taking actions in the unknown MDP. We have Q-learning samples shown below in the order of running. Find the estimates for Q-values of each episode as obtained by Q-learning. Assume that all Q-values are initialized to 0, g= 1, and a = 0.5. Episode s s' a r 1 A Go B 4 2 Stop Stop B A 2 3 B A -4 4 C Stop Go 1 5 C A 2 A Go A -2

Pacman is in an unknown MDP where there are three states [A, B, C] and two actions [Stop, Go]. We are given the following samples generated from taking actions in the unknown MDP. We have Q-learning samples shown below in the order of running. Find the estimates for Q-values of each episode as obtained by Q-learning. Assume that all Q-values are initialized to 0, g= 1, and a = 0.5. Episode s s' a r 1 A Go B 4 2 Stop Stop B A 2 3 B A -4 4 C Stop Go 1 5 C A 2 A Go A -2

Database System Concepts

7th Edition

ISBN:9780078022159

Author:Abraham Silberschatz Professor, Henry F. Korth, S. Sudarshan

Publisher:Abraham Silberschatz Professor, Henry F. Korth, S. Sudarshan

Chapter1: Introduction

Section: Chapter Questions

Problem 1PE

See similar textbooks

Similar questions

Appendix A 10-Fold Cross Validation for Parameter Selection Cross Validation is the standard method for evaluation in empirical machine learning. It can also be used for parameter selection if we make sure to use the training set only. To select parameter A of algorithm A(X) over an enumerated range d E [A1,..., A] using dataset D, we do the following: 1. Split the data D into 10 disjoint folds. 2. For each value of A e (A1,..., Ar]: (a) For i = 1 to 10 Train A(A) on all folds but ith fold Test on ith fold and record the error on fold i (b) Compute the average performance of A on the 10 folds. 3. Pick the value of A with the best average performance Now, in the above, D only includes the training data and the parameter A is chosen without the knowledge of the test data. We then re-train on the entire train set D using the chosen A and evaluate the result on the test set.
Write the updating rule at iteration k+1 for momentum gradient descent learning with a momentum value of 0.9. Explain how this method improves convergence properties of gradient descent learning.
Assume that your hypothesis function is of the form f(x) = w0 + w1x and that the current values of w0 and w1 are 1 and 2 respectively. Further assume that you are using a learning rate (alpha) of 0.001 What is the gradient update for w0 (only the change) associated with the point (1, 12)?
Implement a passive learning agent in a simple environment, such as the 4 x 3 world. For the case of an initially unknown environment model, compare the learning performance of the direct utility estimation, TD, and ADP algorithms. Do the comparison for the optimal policy and for several random policies. For which do the utility estimates converge faster! What happens when the size of the environment is increased? (Try environments with and without obstacles.)
You have a data set which contains Ethno, HbA1c and SBP, indicators in blood. Ethno is a categorical variable and HbA1c and SBP are continuous. They are not evenly distributed. Explain which one of the supervised learning algorithms and why they would be most appropriate for predicting the response Ethno using HbA1c and SBP as the feature variables. The learning algorithms are: Linear Discriminant Analysis, Quadratic Discriminant Analysis and the Naïve Bayes Classifier
We are given the following training examples in 2D ((-3,5), +), ((-4, –2), +), ((2,1), -), ((4,3),–) Use +1 to map positive (+) examples and -1 to map negative (-) examples. We want to apply the learning algorithm for training a perceptron using the above data with starting weights wo = w1 = w2 = 0 and learning rate 7 = 0.1. In each iteration process the training examples in the order given above. Complete at most 3 iterations over the above training examples. What are the weights at the end of each iteration? Are these weights final
Computer Science A way to avoid overfitting in Deep Neural Networks is to add an additional term R to the loss function L (for example L can be the cross entropy loss) as follows: L(w) + λR(w). (1) You know that one choice for R is the L2 norm, i.e. R(w) = ||w||2 2 . One friend of yours from the School of Maths told you however that there’s no need to use squares (i.e. powers of two) and that you can achieve the same effect by using absolute values, i.e. the L1 norm: R(w) = ||w||1. Would you agree with him? i.e. is the use of the L2 norm equivalent to using the L1 norm for regularization purposes? Justify your answer
The learning problem is to find the unknown (functional) relationship hy between objects x X and targets y y based solely on a sample z = (x, y) = ((x1, Y1), ..., (xms Ym)) = (X × y)m of size meN drawn iid from an unknown distribution Pxy. If the output space Y contains a finite number Y of elements then the task is called a classification learning problem.
A model is used to obtain a prediction by accepting features as inputs and returning a prediction in the is.... subset of machine learning. The following are some new cutting-edge models that have become feasible:
Answer this questions True or False : Q1:lf the learning rate is too small, then gradient descent may take a very long time to converge. Q2: If 00 and 01 are initialized at a local minimum, then one iteration will not change their values Q3: If 00 and 01 are initialized so that 00=01, then by symmetry (because we do simultaneous updates to the two parameters), after one iteration of gradient descent, we will still have 00=01
1. The impulse response of a causal system is: h(t) = A cos(wt) e¯¹/¹u(t) where u(t) is the Heaviside step function. The response is measured experimentally with a sampling interval of T. a. Write an expression for the sampled impulse response h[n]. b. Calculate the z transform of h[n] and write an expression for H[z]. Use the tables provided below as necessary. c. Does the system have an infinite impulse response (IIR) or finite impulse response (FIR)? Justify your answer. d. What is the DC gain of H[z]? e. Write a difference equation that describes the output y[n] in terms of input x[n].
Linear regression aims to learn the parameters 7 from the training set D = {(f(),y(i)), i {(x(i),y(i)),i = 1,2,...,m} so that the hypothesis ho(x) = ēr i can predict the output y given an input vector š. Please derive the least mean squares and stochastic gradient descent update rule, that is to use gradient descent algorithm to update Ô so as to minimize the least squares cost function JO).