A new class of online services allows internet media sites to direct users from articles they are currently reading to other content they may be interested in. This process creates a "browsing path'' along which there is potential for repeated interaction between the user and the provider, giving rise to a dynamic optimization problem. A key metric that often underlies this recommendation process is the click-through rate (CTR) of candidate articles. While CTR is a measure of instantaneous click likelihood, we analyze the performance improvement that one may achieve by some lookahead that accounts for the potential future path of users. To that end, using a large data set of user path history at major media sites, we introduce and derive a representation of content along two key dimensions: clickability, the likelihood to click to an article when it is recommended; and engageability, the likelihood to click from an article when it hosts a recommendation. We then propose a class of heuristics that leverage both clickability and engageability, and provide theoretical support for favoring such path-focused heuristics over myopic heuristics that focus only on clickability (no lookahead). We conduct a live pilot experiment that measures the performance of a practical proxy of our proposed class, when integrated into the operating system of a worldwide leading provider of content recommendations. We estimate the aggregate improvement in clicks-per-visit relative to the CTR-driven current practice. The documented improvement highlights the importance and the practicality of efficiently incorporating for the future path of users in real time.

Omar Besbes, Yonatan Gur, and Assaf Zeevi
Journal Article
Publication Date
Manufacturing & Service Operations

Full Citation

Besbes, Omar, Yonatan Gur, and Assaf Zeevi
. “Optimization in Online Content Recommendation Services: Beyond Click-Through-Rates.”
Manufacturing & Service Operations
, (January 28, 2016):