Dr. Mark Humphrys

School of Computing. Dublin City University.

Home      Blog      Teaching      Research      Contact


CA216      CA249      CA318

CA400      CA651      CA668

Learning from Rewards (Reinforcement Learning, RL) as Pattern Classification

Object = state x.
e.g. x = state of robot and its environment.

Features describing the object = Values of each dimension defining the point x = (x1,..,xn).
e.g. xi = infra-red sensor reading to front of robot.

Feature space = State space. May be multi-dimensional, where each dimension takes continuous values. x is point in this space.

Classes to assign object into = Action a to take when state x is seen.
e.g. Move left, move right, stop.
Note classes normally small finite set. Actions often small, finite (discrete), but may be continuous, infinite set (e.g. real number output, move at angle).

Agent or actor (robot, program) learns to map x to a.


Does each x map to unique or multiple a's?

Can multiple x map to same a?

Is whole space covered? Does each x map to some a? Can we return an a for a new x, never seen before?

Does each action a map to some state x?

Noise / Probabilistic worlds / inaccurate / incomplete sensors

From the start we will allow our world to be probabilistic rather than necessarily deterministic.
i.e. In state x, you take action a. Sometimes this leads to state y. Sometimes it leads to state z.

Probabilistic reasoning / Stochastic control policy

Our action taker, instead of linking each x to a single a, may say instead something like:
"In state x, take action a with probability 0.9, action b with probability 0.1."

  1. Deterministic control policy in a deterministic world.
  2. Deterministic control policy in a probabilistic world.

  3. Probabilistic control policy in a probabilistic world.
  4. Probabilistic control policy in a deterministic world.

Thought-controlled machines

Much work in thought-controlled machines involves separating a statespace into large regions, each of which can be associated with a discrete action.

Thought-controlled wheelchair.

Monkey controls a robotic arm with its brainwaves.

Feeds      w2mind.org

On Internet since 1987.