05Research Assistant, University of Alberta · Jun–Aug 2026

Q-Learning Grid World

8steps in the shortest path, which the agent learned · the minimum possible on this grid

Training log

  1. Exit found in 9 steps!
  2. Exit found in 8 steps!
  3. Exit found in 9 steps!
  4. Exit found in 10 steps!
  5. Exit found in 10 steps!
  6. Exit found in 8 steps!
  7. Exit found in 9 steps!
  8. Exit found in 8 steps!
  9. Exit found in 10 steps!
  10. Exit found in 9 steps!
  11. Exit found in 10 steps!
  12. Exit found in 10 steps!
  13. Exit found in 8 steps!
  14. Exit found in 8 steps!
  15. Exit found in 8 steps!
  16. Exit found in 10 steps!
  17. Exit found in 8 steps!
  18. Exit found in 10 steps!
  19. Exit found in 8 steps!
  20. Exit found in 8 steps!
  21. Exit found in 8 steps!
  22. Exit found in 10 steps!
  23. Exit found in 9 steps!
  24. Exit found in 9 steps!
  25. Exit found in 8 steps!
  26. Exit found in 9 steps!
  27. Exit found in 8 steps!
  28. Exit found in 8 steps!
  29. Exit found in 8 steps!
  30. Exit found in 8 steps!
  31. Exit found in 8 steps!
  32. Exit found in 10 steps!
  33. Exit found in 8 steps!
  34. Exit found in 8 steps!
  35. Exit found in 8 steps!
  36. Exit found in 11 steps!
  37. Exit found in 8 steps!
  38. Exit found in 8 steps!
  39. Exit found in 8 steps!
  40. Exit found in 8 steps!
  41. Exit found in 11 steps!
  42. Exit found in 8 steps!
  43. Exit found in 8 steps!
  44. Exit found in 8 steps!
  45. Exit found in 9 steps!
  46. Training Completed.
The last episodes of training, as MATLAB printed them. Transcribed from the screenshot in the repo.

Illustration

A 5 by 5 grid. The agent starts at the top left, (1,1). The goal is at the bottom right, (5,5). Two obstacles are in row 3, at columns 3 and 4.start (1,1)goal (5,5)

Learned route: 8 steps.

A re-run of the same method in your browser, with a fixed random seed. It is not a recording of my MATLAB training runs, so its routes differ from mine. My real output is the training log on this page.

Screenshot

MATLAB's plot of the grid, titled Grid World Map, with rows and columns numbered 1 to 5. A dot marks the start at row 1, column 1, top left. Two cells are blocked in row 3, at columns 3 and 4. The goal is the cell at row 5, column 5, bottom right.
MATLAB's plot of the grid.

What I implemented

As a research assistant in reinforcement learning at the University of Alberta, I implemented a grid world in MATLAB, with Q-learning written from scratch.

The environment is a 5×5 grid: start (1,1), goal (5,5), obstacles at (3,3) and (3,4). The rewards are +100 for the goal, −10 for a wall or edge, and −1 for each step.

I implemented epsilon-greedy action selection, a Q-table and Bellman updates.

I tuned the learning rate, the discount factor and the exploration rate to improve how quickly and reliably training converged.

I fixed a bug where the agent repeated the same move in untrained states, by adding random tie-breaking.

How I verified it

Over 500 episodes of training, the agent learned the shortest path: 8 steps, the minimum possible on this grid.

The training log on this page is MATLAB's own output for the last episodes. Individual training episodes still vary, between 8 and 11 steps.

I analyzed the learned Q-table to evaluate and improve the agent.

Reporting

I wrote weekly reports to the supervising professor, and a technical report.

Tools