Reflecting on the past is often a critical ingredient for successful learning. The current research investigated how counterfactual thinking, reflecting on how prior experiences might have been different, motivates effective learning from these previous experiences. Specifically, we explored how the structure of counterfactual reflection — their additive ("If only I had") versus subtractive ("If only I had not") nature — influences performance in dyadic-level strategic interactions. Building on the functionalist account of counterfactuals, we found across two experiments that generating additive counterfactuals about a previous negotiation produced an advantage for negotiators over their previous performance compared to subtractive counterfactuals, both in terms of obtaining value for oneself and conceiving creative agreements. Additive counterfactuals enabled negotiators to more effectively extract lessons from past experiences to improve their current negotiation performance.