
Sample-based Learning Methods
University of Alberta · View on Coursera ↗
Or see the score while you browse.
Taught by Martha White, Adam White
Data ScienceComputer Science
About the course
In this course, you will learn about several algorithms that can learn near optimal policies based on trial and error interaction with the environment---learning from the agent’s own experience. Learning from actual experience is striking because it requires no prior knowledge of the environment’s dynamics, yet can still attain optimal behavior.
Read more
We will cover intuitively simple but powerful Monte Carlo methods, and temporal difference learning methods including Q-learning. We will wrap up this course investigating how we can get the best of both worlds: algorithms that can combine model-based planning (similar to dynamic programming) and temporal difference updates to radically accelerate learning. By the end of this course you will be able to: - Understand Temporal-Difference learning and Monte Carlo as two strategies for estimating value functions from sampled experience - Understand the importance of exploration, when using sampled experience rather than dynamic programming sweeps within a model - Understand the connections between Monte Carlo and Dynamic Programming and TD. - Implement and apply the TD algorithm, for estimating value functions - Implement and apply Expected Sarsa and Q-learning (two TD methods for control) - Understand the difference between on-policy and off-policy control - Understand planning with simulated experience (as opposed to classic planning strategies) - Implement a model-based approach to RL, called Dyna, which uses simulated experience - Conduct an empirical study to see the improvements in sample efficiency when using Dyna
The dissent
The most-upvoted critical review (7 votes, 2.0★).
“I spend 1 hour learning the material and coding the assignment while 8 hours trying to debug it so that the grader will not complain. The grader sometimes insists on a particular order of the coding which does not really matter in the real world. Also, grader inconsistently gives 0 marks to a particular part of the problem while give a full mark on other part using the same function. (Like numpy.max) However, the forum is quite helpful and the staff is generally responsive.”
Coursera's 4.8 comes from 1,260 ratings. Of those, 1.7% are 1–2★.
Starisk reads the reviews learners voted most helpful, and weights each by the square root of its votes: a review with 80 upvotes counts about five times one with 2. Unvoted reviews are not skipped out of laziness — by this model they carry no evidence that other students endorsed them.
That measures what the most-endorsed reviews say, not “true quality.” The rating distribution above is the whole population, so you can see the sample next to it.
Reviews read on 22 Oct 2026.Read them on Coursera ↗