Agent Strategies and the Shadow of the Future

Optimal Strategy Depends on the Shadow of the Future

Note: in order to run the simulation referred to in this slide, go here, to the Java Applet version. You will be directed to download the latest version of the Java plug-in.

The simulation set up to the left will answer the question posed in the last slide: which is the more advantageous strategy for the right agent, given that left Agent's strategy is Tit-for-Tat: Tit-for-Tat or All-D? Furthermore, how does the answer to this question depend on the shadow of the future parameter? We have set up two possibilities one under the other in the simulation graphic: in the top one, the right agent uses the Tit-for-Tat strategy, and in the bottom one - All-D strategy instead. Which version of the right Agent will accumulate more points in the same amount of time?

Pressing the "Go" button will run the Society for 100 steps. At δ=0.5, the All-D agent will do about as well overall as its Tit-for-Tat version. On the other hand, try reducing or increasing δ, and running the Society again. You should notice that low δ makes All-D the more successful strategy, while high δ favors Tit-for-Tat instead.

Apparently, when δ is high, it may be advantageous for an agent to cooperate now in order to elicit cooperation from the partner later in the current instance of the Iterated Prisoner's Dilemma. When it is low, such attempts might not be worth it for the agent, and it may be advantageous to reap the immediate benefits of defection. An important conclusion that follows is that if the shadow of the future is not known, there is no best strategy for an agent to follow in any given situation. More specifically, a nice1 strategy like Tit-for-Tat is favored for high δ - a result that turns out to hold in a wide variety of situations.

1. A nice strategy never defects first. For instance, All-D is not a nice strategy, while Tit-for-Tat is.

Previous Slide                                                           Next Slide