---
jupytext:
  formats: content_md///md:myst,content///ipynb
  text_representation:
    extension: .md
    format_name: myst
    format_version: 0.13
    jupytext_version: 1.14.4
kernelspec:
  display_name: Python 3 (ipykernel)
  language: python
  name: python3
---

# Data Science IV (course at WHZ)

The last part of the data science lecture series is devoted to reinforcement learning. Next to very basic techniques we also discuss state-of-the-art deep reinforcement learning with artificial neural networks.

## Reinforcement Learning

### Week 1 (Overview and Stateless Tasks)

::::{grid}
:gutter: 4

:::{grid-item-card}
:columns: 6
**Lectures**
^^^
* [](reinforcement:overview-examples)
  * [](reinforcement:overview-examples:what-is-ri)
  * [](reinforcement:overview-examples:examples)
  * [](reinforcement:overview-examples:maximization-return)
* [](reinforcement:stateless)
:::

:::{grid-item-card}
:columns: 6
**Self-study**
^^^

:::

:::{grid-item-card}
:columns: 12
**Projects and Exercises**
^^^
* [](exercises:reinforcement:modeling) (exercises)
* [](projects:online-advertising) (project)
:::

::::

### Week 2 (Markov Decision Processes)

::::{grid}
:gutter: 4

:::{grid-item-card}
:columns: 6
**Lectures**
^^^
* [](reinforcement:markov)
  * [](reinforcement:markov:basic-notions)
  * [](reinforcement:markov:bellman-equations)
:::

:::{grid-item-card}
:columns: 6
**Self-study**
^^^

:::

:::{grid-item-card}
:columns: 12
**Projects and Exercises**
^^^
* [](exercises:reinforcement:markov) (exercises)
:::

::::

### Week 3 (Dynamic Programming)

::::{grid}
:gutter: 4

:::{grid-item-card}
:columns: 6
**Lectures**
^^^
* [](reinforcement:dynamic-programming)
:::

:::{grid-item-card}
:columns: 6
**Self-study**
^^^

:::

:::{grid-item-card}
:columns: 12
**Projects and Exercises**
^^^
* [](projects:frozen-lake)
  * [](projects:frozen-lake:dynamic-programming) (project)
:::

::::

### Week 4 (Monte Carlo Methods)

::::{grid}
:gutter: 4

:::{grid-item-card}
:columns: 6
**Lectures**
^^^
* [](reinforcement:monte-carlo)
:::

:::{grid-item-card}
:columns: 6
**Self-study**
^^^

:::

:::{grid-item-card}
:columns: 12
**Projects and Exercises**
^^^
* [](projects:frozen-lake)
  * [](projects:frozen-lake:monte-carlo) (project)
:::

::::

### Week 5 (Temporal Difference Learning)

::::{grid}
:gutter: 4

:::{grid-item-card}
:columns: 6
**Lectures**
^^^
* [](reinforcement:temporal-difference)
:::

:::{grid-item-card}
:columns: 6
**Self-study**
^^^

:::

:::{grid-item-card}
:columns: 12
**Projects and Exercises**
^^^
* [](projects:frozen-lake)
  * [](projects:frozen-lake:sarsa) (project)
:::

::::

### Week 6 (Tic-Tac-Toe)

::::{grid}
:gutter: 4

:::{grid-item-card}
:columns: 6
**Lectures**
^^^

:::

:::{grid-item-card}
:columns: 6
**Self-study**
^^^

:::

:::{grid-item-card}
:columns: 12
**Projects and Exercises**
^^^
* [](projects:tic-tac-toe)
:::

::::

### Week 7 (Approximate Methods)

::::{grid}
:gutter: 4

:::{grid-item-card}
:columns: 6
**Lectures**
* [](reinforcement:approximate-methods)
  * [](reinforcement:approximate-methods:principal-approach)
  * [](reinforcement:approximate-methods:policy-evaluation)
  * [](reinforcement:approximate-methods:policy-improvement)
^^^

:::

:::{grid-item-card}
:columns: 6
**Self-study**
^^^

:::

:::{grid-item-card}
:columns: 12
**Projects and Exercises**
^^^
:::

::::

### Week 8 (Cart Pole SARSA)

::::{grid}
:gutter: 4

:::{grid-item-card}
:columns: 6
**Lectures**
^^^

:::

:::{grid-item-card}
:columns: 6
**Self-study**
^^^

:::

:::{grid-item-card}
:columns: 12
**Projects and Exercises**
* [](projects:cart-pole)
  * [](projects:cart-pole:environment)
  * [](projects:cart-pole:sarsa)
^^^
:::

::::

### Week 9 (Deep Q-Learning)

::::{grid}
:gutter: 4

:::{grid-item-card}
:columns: 6
**Lectures**
* [](reinforcement:approximate-methods)
  * [](reinforcement:approximate-methods:deep-q-learning)
^^^

:::

:::{grid-item-card}
:columns: 6
**Self-study**
^^^

:::

:::{grid-item-card}
:columns: 12
**Projects and Exercises**
* [](projects:cart-pole)
  * [](projects:cart-pole:deep-q-learning)
^^^
:::

::::

### Week 10 (Policy Gradient Methods)

::::{grid}
:gutter: 4

:::{grid-item-card}
:columns: 6
**Lectures**
* [](reinforcement:policy-gradient-methods)
^^^

:::

:::{grid-item-card}
:columns: 6
**Self-study**
^^^

:::

:::{grid-item-card}
:columns: 12
**Projects and Exercises**
^^^
:::

::::

### Week 11 (Policy Gradient Methods)

::::{grid}
:gutter: 4

:::{grid-item-card}
:columns: 6
**Lectures**
^^^

:::

:::{grid-item-card}
:columns: 6
**Self-study**
^^^

:::

:::{grid-item-card}
:columns: 12
**Projects and Exercises**
* [](projects:cart-pole)
  * [](projects:cart-pole:policy-gradient)
^^^
:::

::::
