Dr. Mark Humphrys School of Computing. Dublin City University. My big idea: Ancient Brain Search:

# CA425 - Artificial Intelligence

 Quickest way to find this web page: Google ca425 dcu Think about it: All other ways (starting at the CA site, Googling my name, etc.) will take much longer to get here. Just Google module code and DCU and it is the first hit.
```
```

```

```

# How to contact me

See How to contact me.
```
```

# Notes

 My notes contain many hyperlinks to background material. Some students get confused about what is the core course. The core course is anything that is linked to directly on this front page. All other links are just background material.

 THE SECTIONS IN YELLOW ARE NOT ON THE COURSE THIS YEAR.

1. Background

2. State-space control

3. Reinforcement Learning - Intro

4. Movie demo
1. Movie demo of W-learning contains within it a demo of basic Q-learning.

5. Program code (for practical)
1. Coding the state-space as a lookup-table
2. Sample code for lookup-table Q-learning (Includes Boltzmann "soft max" option)

```

```
[PRACTICAL CAN NOW BE LAUNCHED]
```

```
6. Reinforcement Learning - More

7. Reinforcement Learning with Neural Networks (Pre-requisite needed.) - NOT ON COURSE THIS YEAR

8. Multiple Minds

```
```

# Notes on Assignment Notation

I often use   :=   for assignment to distinguish from   =   for equality.
```
```

# Labs

 Not WWM-based this year: If the practical is based on the WWM server, I will hold one or two hands-on labs. Dates will be announced.
```
```

# Practical

Practical - Play "X's and O's" with RL

Deadline - last lecture in week 12.

```

```

Experiments in Adaptive State-Space Robotics, Clocksin and Moore, 1989. A simple introduction to the very idea of state-space robotic or agent control.

How to Make Software Agents Do the Right Thing: An Introduction to Reinforcement Learning, Singh et al, 1996. A simple introduction to the idea of RL.

"Reinforcement Learning: A Survey", Kaelbling et al, Journal of Artificial Intelligence Research, 4:237-285, 1996. A survey.

Action Selection methods using Reinforcement Learning. My PhD thesis, 1997, has an intro to RL.

```
```

### Books

Reinforcement Learning: An Introduction, Sutton and Barto, 1998. Also here.

Reinforcement Learning: State-of-the-Art, Marco Wiering and Martijn Otterlo (Editors), 2012.

```
```

# Library categories

• 006 - Special computer methods
• 006.3 - Artificial Intelligence

• 519 - Probabilities & applied mathematics
• 519.7 - Programming
```

```

# Wikipedia

Sometimes I link to Wikipedia. I need to write something in defence of this.

On the one hand, Wikipedia is deeply flawed, so you should use all links to Wikipedia with extreme caution. Many people refuse to link to it.

On the other hand, it is often clearly the best thing to link to on a topic. I say: Link to it, but use with caution and scepticism.

```
```

# Mark calculator

Exam (70): Practical (30):

Total mark:

```

```

# Exam results

The notes are online, but you need to go to every lecture. You will not understand this course from the notes alone.

2014 exam results summary:

• 0 to 10 percent: 0
• 10 to 20 percent: 0
• 20 to 30 percent: 2
• 30 to 40 percent: 0
• 40 to 50 percent: 3
• 50 to 60 percent: 2
• 60 to 70 percent: 3
• 70 to 80 percent: 2
• 80 to 90 percent: 0
• 90 to 100 percent: 0
```
```
```
```

Feeds      w2mind.org      ancientbrain.com

On the Internet since 1987.