Jul 11, 2012 - term growth or shrinkage of traffic at network, site, placement or user segment levels. ... requires to have the ad server delivery algorithm, which is implemented .... AdForecaster platform via FTP or through Amazon S3 push.
PREDICT YOUR CAMPAIGN AD IMPRESSIONS IN SECONDS
TECHNICAL WHITEPAPER VERSION 1 2012 JULY 11
TH
TABLE OF CONTENTS 1! INTRODUCTION
3!
1.1! WHAT IS ADFORECASTER 1.2! WHO IS IT FOR 1.3! FEATURES
3! 3! 4!
2! HOW IT WORKS
5!
2.1! OVERVIEW 2.2! FUTURE AD INVENTORY PREDICTION 2.3! FORECASTING EXECUTION
5! 5! 6!
3! INTEGRATION
8!
3.1! OVERVIEW 3.2! LOG FILES
8! 8!
3.3! CAMPAIGNS
8!
3.4! AD SERVER ALGORITHMS
9!
3.5! ADFORECASTER API
9!
2.3.1! AD SERVER REQUEST SIMULATION
3.2.1! FORMAT 3.2.2! DATA PROVISION
8! 8!
3.3.1! FORMAT 3.3.2! DATA PROVISION
8! 9!
3.4.1! AD SELECTION ALGORITHM 3.4.2! SPIDER AND ROBOTS FILTERING
9! 9!
3.5.1! API REQUEST EXAMPLE 3.5.2! API RESPONSE EXAMPLE
Private and Confidential
7!
9! 10!
2/11
1 INTRODUCTION 1.1 WHAT IS ADFORECASTER The AdForecaster is the next-generation forecasting engine for online advertising campaigns. It overcomes various important limitations of existing forecast engines in ad servers, such as the number of targeting variables, scaling of data samples or the speed of delivering forecast results. The AdForecaster allows you to accurately predict future ad impressions traffic levels and campaign inventory availability using an unlimited number of targeting variables, including geo, keywords, key-values, cookies and multiple frequency capping groups at banner, booking, line item or campaign level. It is operated as a Software-as-a-service, minimising integration and deployment times as well as operational maintenance costs.
1.2 WHO IS IT FOR The AdForecaster is designed for online advertising platforms that require accurate estimations on the number of ad impressions and unique users when booking new targeted campaigns. The platforms that can take advantage of the AdForecaster need to have the ability to: • • •
provide ad request log files provide the information about existing campaigns booked in the platform use the AdForecaster API for making the forecasting queries
Custom-built Ad Servers and Exchanges, Sell-side platforms (SSPs) and Demand-side platforms (DSPs) are the most common type of platforms in this situation that can easily integrate with the AdForecaster.
Private and Confidential
3/11
1.3 FEATURES
PROGRESSIVE FORECASTING
ADAPTATIVE DECISION ENGINE
HANDLES UNLIMITED TARGETING VARIABLES
First result within seconds, refined results progressively available with time. This allows higher responsiveness for your application’s GUI whilst ensuring high accuracy results are available when response times are not critical
Customised to mirror how your Ad Server works, from campaign priority calculation, pacing mechanisms and all the way down to how the infrastructure setup affects ad delivery decision logic
Capable of taking into account any combination of targeting parameters you use in your Ad Server such as Geo, Keywords, Key-Values, Cookies, Segments, etc.
MULTIPLE FREQUENCY CAPPING GROUPS
API INTEGRATION
COST-SAVING INFRASTRUCTURE
Capable of forecasting the most complex frequency capping scenarios with multiple restrictions set at the Campaign, Booking, Banner or any other level set in your Ad Server
Seamless integration with your and third-party platforms via APIs
Automated management turns processing nodes on and off when required, saving everyone money in the process
Private and Confidential
4/11
2 HOW IT WORKS 2.1 OVERVIEW The AdForecaster works in two separate processes: 1. Predict future inventory, using historical Ad Server request logs as the basis. Tomorrow’s weather is not yesterday’s weather: advanced machine learning algorithms spot short, medium and long-term trends, shortening the gap between predicted and real ad inventory, especially for quick changes in inventory sources such as new sites or ad placements which traditionally require up to 28 days before a forecast is built 2. Run real-time forecasts every time it is requested, by a GUI or back-end service for example, responding within seconds with the number of unique users and ad impressions available for the campaign. This evaluation is executed simulating the Ad Server decision logic on the future inventory, observing the resulting outcome banner and campaign. This process takes into account all other campaigns booked in the Ad Serving platform
2.2 FUTURE AD INVENTORY PREDICTION Historical inventory
Future Ad Inventory Prediction
Forecasted inventory
Unique Users Long term trends
Ad Impressions Fast operational changes
July 2011
December 2011
May 2011
Now
October 2012
March 2013
Once a day, historical data is used to predict future inventory traffic levels, generating “future traffic logs” with all the details of what the future ad requests will look like. Short, medium and long term trends are taken into account, detecting weekly traffic patterns, seasonal traffic variations, operational changes such as adding or removing ad placements or entire sites, and longterm growth or shrinkage of traffic at network, site, placement or user segment levels.
Private and Confidential
5/11
2.3 FORECASTING EXECUTION 2
1
Previously generated Future Inventory is loaded for the given dates Inventory which the Ad Server will deliver to other campaigns is removed
Received forecast request for new Campaign during the month of August with medium rectangles on sites categorised as News/Finance
3 5
Inventory that does not conform to the requested targeting criteria, including frequency cap counting, is excluded from the forecast
336 000 Ad Impressions 71 000 Unique Users
4 Forecasted inventory
Remaining inventory is aggregated and added up
Inventory taken by existing Campaigns
Unusable inventory due to targeting or pacing restrictions
Running a forecast works by simulating the ad server algorithm over the predicted future inventory, together with the information of the existing campaigns in the ad server. For this, the AdForecaster also requires to have the ad server delivery algorithm, which is implemented separately for each client in the AdForecaster platform. The execution of the forecast process is distributed across several processing nodes (machines), minimising the time it takes to produce a result. The forecast request can specify the size(s) of the data sample(s) to use, allowing for quick responses using smaller data samples but opening the possibility of having more accurate results when time is not of the essence.
Private and Confidential
6/11
2.3.1 AD SERVER REQUEST SIMULATION The Ad Server ad selection algorithm from the client platform is applied during each forecast execution to simulate the calculation of which campaign and banner is delivered for each future ad inventory request. For this to work, the client provides the ad selection algorithm, which is implemented, specifically for the client in the AdForecaster platform. The AdForecaster supports unlimited algorithm configuration capabilities, thanks to its implementation as a generic rule-engine system, supporting the following campaign filtering settings out-of-the-box:
● ● ●
●
● ● ●
Campaigns with multiple bookings (line items) Start and end dates at campaign level Unlimited key-value pair targeting with Boolean operators at booking level: • Equality (=) • Greater than, Less than • And • Or • Not • Precedence, through the use of parentheses () Frequency capping • N impressions per X minutes • Configurable frequency cap behaviour for adaptation with ad server algorithm: • Rolling time frame, e.g. N impressions within a rolling period of X minutes • Fixed time frame on first view, e.g. N impressions between first impression time and X minutes afterwards. Resets after X minutes. • Implements frequency groups, where frequency is counted at a group level, allowing simulation of any ad servers frequency capping implementation, e.g. Frequency at the banner, multiple banners, campaign, multiple campaigns, advertiser, etc and any combination of the above. • Multiple frequency caps per campaign Priority at campaign level Booked campaign impression goal Campaign weight (e.g. used in pacing)
Private and Confidential
7/11
3 INTEGRATION 3.1 OVERVIEW SSP / DSP / Ad Exchange / Ad Server Online Advertising Platform AdServer front-ends Platform GUI AdServer back-end
Inventory and Yield Manager Campaigns
Real-Time Forecasts
The AdForecaster sits at the centre of your Online Advertising Platform, answering questions from your campaign management GUI/Services when they require the knowledge of how many unique users and ad impressions are available for a campaign.
3.2 LOG FILES 3.2.1 FORMAT The AdForecaster is ready to read in some of the most popular ad server log data formats, and will be easily configured to read in any log file format specified by the client. It is crucial that the log files contain all necessary and relevant information about the ad request and the user that performed it, namely any profiling, registration or behavioural segment information from the user that can be used for ad targeting. Only data provided in the historical log files will be used for producing forecasts.
3.2.2 DATA PROVISION The client can provide the ad request log files using a number of different methods, namely: • Amazon S3 push: The client places daily log files on the Amazon S3 infrastructure, using credentials provided by ShiftForward (preferred method). ShiftForward provides scripts for easier integration with the Amazon S3 API. • FTP Push: The client places daily log files on a FTP server specified by ShiftForward • FTP Pull: The client specifies an FTP location for the AdForecaster to fetch daily log files
3.3 CAMPAIGNS 3.3.1 FORMAT The format of the campaign information is configurable and agreed between the client and ShiftForward during the integration phase. Normally XML or JSON formats are the most appropriate for this type of data. It is important that all information about the campaign is available here, including setup settings (targeting, start and end dates, delivery goals, etc) as well as current campaign status (e.g. delivered impressions so far).
Private and Confidential
8/11
3.3.2 DATA PROVISION Ideally the AdForecaster is given access to real-time APIs from the client’s platform for it to fetch the information on booked campaigns. As each ad server’s API is different, this step requires some customisation and mapping in the AdForecaster platform, performed during the integration phase with the client. If the ad server does not have APIs or is not able to produce the necessary information, then alternative means may be available, such as daily export of campaign information to a file that is delivered to the AdForecaster platform via FTP or through Amazon S3 push.
3.4 AD SERVER ALGORITHMS 3.4.1 AD SELECTION ALGORITHM The ad selection algorithm from the Ad Server of the client’s platform is implemented in the AdForecaster platform specifically for the client. This is required for it to be able to produce actionable and realistic results. The AdForecaster has been designed to make the implementation of a new ad server decision algorithm as extensible and quick as possible. For testing purposes there are generic ad selection algorithms available as well as some from major ad serving vendors, when this information has been made public.
3.4.2 SPIDER AND ROBOTS FILTERING The AdForecaster has an internal list of known Robots and Spiders, which it uses to filter out ad requests produced by these types of applications. When possible, the client should provide the same list of Robots and Spiders used by the Ad Server to replace AdForecaster’s default list, as this will increase the coherence between the forecasted ad impressions and the ad impressions as measured by the Ad Server.
3.5 ADFORECASTER API The client’s platform is able to request forecasts through the AdForecaster API. The complete description of this API is outside the scope of this document, but typical use-cases are covered by the following request / response parameters:
3.5.1 API REQUEST EXAMPLE Example of request parameters: • Type: availability • Accuracy: 1 • Start-date: [yyyy-mm-dd], e.g. “2012-08-01” • End-date: [yyyy-mm-dd], e.g. “2012-08-31” • ResponseBy: [Key-Value], e.g. “SiteID” • Targeting: [Targeting operator list] o Targeting operator list: a list of targeting expressions, using global variables or any of the Key-Values present in the historical log files, e.g.: URL.domain contains .media.com Key.GeoCountry = UK or IE Key.SiteID = 1 or 2 or 4 Key.Size = 468x60 or 120x600 Frequency.Campaign = 3 per 1440 or 10 per 10080 Priority = 5
Private and Confidential
9/11
3.5.2 API RESPONSE EXAMPLE Example of possible XML formatted response: 14843 195721 89184 649106 583996 747881 698451 1396839 1057395 8691343 6891857 10598375 9486712 10284051 8594817 12644320 10968354
Private and Confidential
10/11
Private and Confidential
11/11
