How can economic mechanisms remain effective in a world of constant change?
New research from Columbia Business School tackles the “limited commitment” problem, developing a new approach to mechanism design that accounts for new information and changing rules — in other words, for the real world.
CBS Professor Laura Doval shares how this framework could reshape everything from government policies to corporate pricing strategies, allowing for more adaptive systems better suited to an evolving economy.
Key Takeaways:
- Principles of mechanism design inform various economic systems: Companies like Google use them to organize online ad auctions, e-commerce platforms apply them to pricing algorithms, and governments rely on them to design tax policies.
- In all these cases, the traditional approach to mechanism design assumes that the foundational rules, once set, remain fixed indefinitely. But in reality, business and government decision-makers can — and do — change policies as they gain new information.
- To better mirror real-world scenarios, researchers have developed a new mathematical framework that accounts for “limited commitment” in mechanism design. This approach recognizes that as decision-makers learn more about their users over time, they may be tempted to change previously established rules to their advantage.
In a simpler world, economic systems would operate based on fixed rules that everyone adheres to indefinitely. A ride-sharing company would stick to its original pricing strategy regardless of what it learns about an individual customer's willingness to pay. A government would maintain its tax structure even after learning detailed information about citizens’ income patterns.
But reality is more complicated. As institutions interact repeatedly with the same users — whether they’re customers, taxpayers, or market participants — they learn valuable information about them. This creates a temptation to change the original rules. This challenge, known as the limited-commitment problem, has long vexed economists, policymakers, and business leaders alike.
New research is shedding light on how to address this conundrum. Laura Doval, the Chong Khoon Lin Professor in the Economics Division at Columbia Business School, recently coauthored a paper with Vasiliki Skreta, the Leroy G. Denman Regents Professorship in Economics at the University of Texas at Austin, outlining a mathematical framework for more effective mechanism design for when long-term promises are impractical or impossible. Titled “Mechanism Design with Limited Commitment,” the paper was awarded the inaugural Arrow Prize by The Econometric Society.
Such solutions are especially relevant as businesses and governments grapple with vast amounts of real-time data, which impacts established systems in unprecedented ways, explains Doval. “In the digital age, with the wealth of information being accumulated about our behavior, this problem has become even more evident,” she says. “This tool unleashes a way to think about optimal institutions in settings where we couldn’t before.”
The Art and Science of Mechanism Design
Often called the engineering side of economics, mechanism design refers to the process of creating optimal rules or systems to achieve specific economic goals. “In general, economics researchers take an existing institution and try to analyze some property of it — for instance, what are the repercussions to the economy if I introduce a minimum price in a market setting or if I introduce a tax or a tariff? Mechanism design takes a step back,” explains Doval. “It asks, ‘What are the optimal instruments for designing the best institution we can create?’”
But there’s an inherent flaw in traditional mechanism design. Historically, the discipline assumed that whoever sets the contractual terms — be it a government setting fiscal policy or a company establishing pricing rules — wouldn’t change it in the future. This assumption simplified the math, but it didn’t reflect real-world use cases.
Consider, for instance, a government confronting a financial event akin to the 2008 crisis, breaking its prior commitment to avoid bailouts by invoking the “too big to fail” rationale to rescue banks. Or, think of an e-commerce platform that sets prices: As it gathers data on customer behavior, it may want to update its pricing algorithm. An added challenge is that people tend to anticipate these potential changes and adjust their behavior accordingly. For instance, anticipating it would be bailed out, a bank may take upon even riskier investments. Or an online shopper might abandon their cart, knowing their actions could influence future policies or prices.
Rethinking Economic Institutions for the Information Age
Classical mechanism design, developed in the 1970s and 1980s, provided a mathematical framework to represent and optimize economic institutions. Previous breakthroughs allowed economists to determine the best way to achieve a goal (like maximizing a seller’s revenue) among many possible options (like running an auction or setting a fixed price). Doval explains that companies like Google use versions of these frameworks to organize their marketplaces today.
But this approach breaks down when limited commitment comes into play. To address this limitation, Doval and her co-author, Vasiliki Skreta, developed a new mathematical representation that captures the dynamic interplay between institutional rules and information flow. This innovative approach to mechanism design opens up new possibilities for addressing real-world challenges across various domains, including:
- Taxation: The research suggests that tax policies should balance revenue generation with information gathering. Doval notes the trade-offs involved: For instance, a flat tax of $100 per person tells the government very little about individual incomes, while a proportional tax reveals much more. The optimal policy might lie somewhere in between, gathering enough information to be effective without creating overwhelming incentives for evasion.
- Corporate pricing strategies: For businesses, especially in e-commerce, this new framework provides guidance on designing pricing algorithms. It suggests that companies should consider not just how prices affect current sales but also how the information gathered through transactions will shape future pricing decisions — and how consumers might respond.
- Product line design: The researchers studied how companies should design product lines when purchase histories are observable. Contrary to conventional wisdom, they found that limited commitment could lead to both price discrimination and reduced product variety. This insight challenges the common assumption that personalized pricing necessarily leads to greater product variety, highlighting the complex relationship between privacy concerns and market offerings.
The Future of Dynamic Mechanism Design: Real-World Applications and Emerging Technologies
The implications of this research extend beyond these limited use cases. For policymakers, Doval’s proposed framework provides a more nuanced approach to designing regulations in a world of big data. For business leaders, particularly those dealing with dynamic pricing or personalized offerings, it offers a way to create strategies that account for both short-term profits and long-term customer behavior.
Doval is also intrigued by the possibility of applying this framework to emerging technologies, particularly in the realm of online auctions. “When algorithms use past bidding behavior to optimize on future reserve prices, the same feedback problem occurs between the algorithm rules — which will tend to raise reserves when bidders bid high — and the bidders, who will distort down their bids to keep reserve prices low," she says. "The tools we propose are better suited than the traditional approaches to answer the question of which algorithm would be best.”
Ultimately, this research highlights a growing need for adaptive approaches to institutional design in a quickly changing world. “What our paper shows is that once you understand that the person running an institution will be tempted to change the rules as they learn, then you have to carefully design both elements — the rules of the institution and how the institution stores information over time or manages the flow of information,” says Doval.
Adapted from “Mechanism Design with Limited Commitment” by Laura Doval of Columbia Business School and Vasiliki Skreta of the University of Texas at Austin and University College London.
