Lernmaterialien für Reinforcement Learning an der University of Zurich

• A value function estimates how good in terms of expected return it is to be in a certain state or to perform a certain action in a given state. 
• Value functions are parametrized by the policy applied by the agent.

• Learner and decision maker (excluding sensations and any “internal states”) 
• The agent’s goal is to maximise the total reward it receives from the environment 
  • Deterministically or stochastically selects an action given a certain state of its environment

• What the agent interacts with
  • includes everything “outside” the agent (including sensations and any “internal states”)
• The information about the environment accessible to the agent at time t is encoded in a state variable St 
• At each time step, the environment responds to the agent’s action by providing the agent with a reward.

• Real valued number with negative rewards being interpreted as punishments. 
• Given an action a applied in a given state s, the reward can be deterministic, or stochastic.
• A reward is a real valued signal Rt ∈ ℛ ⊂ ℝ passing from the environment to the agent at each time step. 
• The agent’s goal is to maximize (a monotonically increasing function of) the total amount of rewards it receives, not the immediate reward.

• Complete specification of an environment 
  • instance of the RL problem

RL methods aim to maximize the expected return.

A state signal that encodes all relevant information from past interactions with the environment (including past states, actions and received rewards)

A reinforcement learning task that fulfills the Markov property

Express the recursive properties of value functions

Special case of the reinforcement learning problem: single state

• Optimistic initial values method:
• Upper-confidence-bound action selection method: