In the context of artificial intelligence and machine learning, the term “return” is most commonly used in reinforcement learning (RL). Here, return refers to the total accumulated reward that an agent receives from a certain point in time, often starting from the current step until a terminal state is reached in an episode. It’s a crucial concept because it provides a measure of how well an agent is performing in an environment, not just based on immediate rewards but on long-term outcomes.
To break it down, imagine a game where an AI agent makes a series of moves. Each move may yield a reward—sometimes positive, sometimes negative, and sometimes zero. The return is the sum (or sometimes the discounted sum) of all these rewards starting from a particular time step. For example, if an agent receives rewards of +1, +2, and -1 over three steps, the return from the first step would be +1 +2 -1 = +2. However, in most real-world scenarios, especially in infinite or ongoing environments, future rewards are discounted by a factor (called the discount rate, usually represented by gamma, γ) to give more weight to rewards received sooner rather than later. This reflects the idea that immediate rewards are typically more valuable than distant ones.
Mathematically, the return (often denoted by G_t) at time t is expressed as:
G_t = r_{t+1} + γ*r_{t+2} + γ²*r_{t+3} + …
Where γ is the discount factor between 0 and 1, and r_{t+i} is the reward received at time t+i. If γ is 0, the agent only cares about immediate rewards. If γ is close to 1, the agent considers long-term rewards nearly as valuable as immediate ones.
Understanding and calculating the return is essential for training RL agents, as most algorithms aim to maximize the expected return. The return helps the agent evaluate how good a particular policy (a strategy for choosing actions) is. When designing RL algorithms like Q-learning or policy gradients, the return is the key quantity that gets optimized.
Outside reinforcement learning, “return” can have different, less technical meanings in programming, such as the value output by a function. However, in machine learning and AI, especially in RL, the term almost always refers to accumulated rewards.
Return is closely linked to other core RL concepts such as reward, policy, and value functions. While reward is the immediate feedback, return is the total feedback over time. Value functions estimate expected return under a given policy, making return the backbone of most learning objectives in RL frameworks.
Understanding return is fundamental in building agents that can learn from and adapt to their environments, whether playing games, controlling robots, or optimizing complex systems. By focusing on maximizing return, RL agents learn to make decisions that are beneficial not only in the short term but in the long run as well.
