Two agents chasing a ball can look like a tiny game and still contain a surprisingly difficult learning problem. Each action changes the position of the ball and the choices available to everyone else. A strategy that succeeds against one opponent may fail as soon as that opponent learns something new.
Unity's SoccerTwos demonstration is an approachable way into this problem. The ML-Agents soccer code from this period makes the environment's assumptions visible: agents receive ray-based observations, choose from movement actions and interact with the ball through the physics system.
The first design decision is what an agent can observe. Supplying the exact positions of every object gives it a different task from supplying limited local sensing. A policy trained with privileged information may behave impressively without solving the perception problem a viewer assumes it solved. The observation interface defines what knowing the game actually means.
The action interface matters just as much. If the agent can move left or right, rotate and push forward, it must learn a sequence that brings it into useful contact with the ball. If it could instead command a perfect kick toward a chosen target, much of the physical control problem would already be solved by code outside the policy.
Rewards specify another layer of authorship. A goal is a natural outcome to reward, but goals may be rare early in training. Additional signals can make learning easier while also changing what the agent is encouraged to do. Reward touching the ball too strongly, for example, and an agent may learn to maximize contacts rather than help its team score. That is a hypothetical design failure, but a useful one to watch for.
The period's soccer example also distinguishes roles such as striker and goalie, with different movement and time-reward behavior. The roles aren't spontaneous proof that the agents invented team organization. Some structure is supplied by the environment. Reading the code makes the boundary between designed structure and learned behavior much clearer.
Multiple learning agents introduce a moving target. The environment seen by one agent includes the policies of the others. As those policies change, a previously successful action can have a different outcome. Evaluation against a fixed set of opponents or situations can help distinguish broad improvement from adaptation to the most recent partner.
I would inspect awkward episodes as carefully as successful goals. Does an agent recover when it starts in an unusual position? Does it become stuck against a wall? Does cooperation survive a changed teammate? A highlight reel answers none of those questions by itself.
The appeal of this small game is that its rules are easy to understand while its behavior is difficult to prescribe. It provides a concrete place to learn how much intelligence is in the policy, and how much the designer quietly placed in the observations, actions and rewards.