Efficient online ad serving in a display advertising exchange

We introduce and formalize a novel constrained path optimization problem that is the heart of the real-time ad serving task in the Yahoo! (formerly RightMedia) Display Advertising Exchange. In the Exchange, the ad server's task for each display opportunity is to compute, with low latency, an optimal valid path through a directed graph representing the business arrangements between the hundreds of thousands of business entities that are participating in the Exchange. These entities include not only publishers and advertisers, but also intermediate entities called "ad networks" which have delegated their ad serving responsibilities to the Exchange. Path optimality is determined by the payment to the publisher, and is affected by an advertiser's bid and also by the revenue-sharing agreements between the entities in the chosen path leading back to the publisher. Path validity is determined by constraints which focus on the following three issues: 1) suitability of the opportunity's web page and its publisher 2)suitability of the user who is currently viewing that web page, and 3) suitability of a candidate ad and its advertiser. Because the Exchange's constrained path optimization task is novel, there are no published algorithms for it. This paper describes two different algorithms that have both been successfully used in the actual Yahoo! ad server. The first algorithm has the advantage of being extremely simple, while the second is more robust thanks to its polynomial worst-case running time. In both cases, meeting latency caps has required that the basic algorithms be improved by optimizations; we will describe a candidate ordering scheme and a pre-computation scheme that have both been effective in reducing latency in the real ad serving system that serves over ten billion ad calls per day.

[1]  Jon Feldman,et al.  Auctions with intermediaries: extended abstract , 2010, EC '10.

[2]  Clifford Stein,et al.  Introduction to Algorithms, 2nd edition. , 2001 .

[3]  Divesh Srivastava,et al.  Holistic twig joins: optimal XML pattern matching , 2002, SIGMOD '02.

[4]  Ossama Younis,et al.  Constraint-based routing in the internet: Basic principles and recent research , 2003, IEEE Communications Surveys & Tutorials.

[5]  Ashwin Machanavajjhala,et al.  Scalable ranked publish/subscribe , 2008, Proc. VLDB Endow..

[6]  Alex C. Snoeren,et al.  Mesh-based content routing using XML , 2001, SOSP.

[7]  Ronald L. Rivest,et al.  Introduction to Algorithms, Second Edition , 2001 .

[8]  Marcos K. Aguilera,et al.  Matching events in a content-based subscription system , 1999, PODC '99.

[9]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[10]  Yang Xiang,et al.  3-HOP: a high-compression indexing scheme for reachability query , 2009, SIGMOD Conference.

[11]  Marwan Krunz,et al.  Multi-constrained optimal path selection , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[12]  S. Muthukrishnan,et al.  Ad Exchanges: Research Issues , 2009, WINE.

[13]  Jeffrey F. Naughton,et al.  On the integration of structure indexes and inverted lists , 2004, Proceedings. 20th International Conference on Data Engineering.

[14]  Sergei Vassilvitskii,et al.  Indexing Boolean Expressions , 2009, Proc. VLDB Endow..

[15]  Edith Cohen,et al.  Reachability and distance queries via 2-hop labels , 2002, SODA '02.

[16]  Gerhard Weikum,et al.  Efficient creation and incremental maintenance of the HOPI index for complex XML document collections , 2005, 21st International Conference on Data Engineering (ICDE'05).

[17]  Dennis Shasha,et al.  Filtering algorithms and implementation for very fast publish/subscribe systems , 2001, SIGMOD '01.

[18]  Alexander L. Wolf,et al.  A routing scheme for content-based networking , 2004, IEEE INFOCOM 2004.

[19]  Hans-Arno Jacobsen,et al.  Predicate matching and subscription matching in Publish/Subscribe systems , 2002, Proceedings 22nd International Conference on Distributed Computing Systems Workshops.