Reinforcement learning: examples

• Maze
  
  γ = .8, η = .5 and all values initialized at 0. In the chart, "new" means the reinforcement received plus the discounted maximum value of the next state. The "new" value is combined with the "old" using the learning rate to give the updated Q-value appearing in the next line of the chart.
  
  

  x	Q							new	u
  	1,r	1,s	2,r	2,l	3,r	3,l	G,s
  1	0	0	0	0	0	0	0	0	r
  2	0	0	0	0	0	0	0	0	r
  3	0	0	0	0	0	0	0	1	r
  G	0	0	0	0	.5	0	0	0	s
  1	0	0	0	0	.5	0	0	0	r
  2	0	0	0	0	.5	0	0	.4	r
  3	0	0	.2	0	.5	0	0	1	r
  G	0	0	.2	0	.75	0	0	0	s

• Smarts
  A touch sensor distinguishes soft (green) from hard (yellow) from squishy (red) things in front of the Critter. Three actions are possible: MOVE, TURN to the right, and EAT.

  TURN

  MOVE

  MOVE

  EAT

  EAT

  MOVE

  TURN

  EAT

  EAT

  MOVE

  EAT

  

© 2004. Indiana University and Michael Gasser. www.indiana.edu/~gasser/Q530/Notes/rl3.html 4 Oct 2004