Feb 25, 2013 - Exploiting Big Data for Statistics: Some Implications for Policy. Andrew Wyckoff*. Directorate for Scienc
Exploiting Big Data for Statistics: Some Implications for Policy Andrew Wyckoff* Directorate for Science, Technology & Industry Organisation for Economic Co-operation and Development 25 February F b 2013 * The views expressed in this presentation are those of the authors and do not necessarily reflect the opinions of the OECD or its Membership.
Overview • Some applications today • Implications for public policy: – Economic & social policies; – Statistical policy
• Discussion
Mesh Networks / Ambient Computing Q
Supply Chain Management
Q
S Security & Access C Control
Q
Work In Process Tracking
Q
Consumer Applications
Q
Asset Management
Q
Environmental Applications
3
Internet traffic flows continue to grow 100,000
Monthly global IP traffic (Petabytes), 2005-2014
90,000
MOBILE DATA MOBILE DATA
80,000
Consumer Internet video
70,000
Consumer Voice over IP (VoIP)
60,000
Consumer Online gaming
50,000
Consumer Video calling
40,000
Consumer Web, email, and data
30,000
Consumer file sharing
20,000
CONSUMER VOD CONSUMER VOD
10,000
BUSINESS INTERNET / INTRANET
0 2010
2011
2012
2013
2014
2015 Source: CISCO VNI 2011
A lot of big data buzz • “Data
is the new oil.” Andreas Weigend, Stanford (ex Amazon) • “The future belongs to companies and people that turn data into products”, Mike Loukides, O’Reilly Media
“Why big data is a big deal” InfoWorld – 9/1/11
“Keeping Afloat in a Sea of 'Big Big Data” ITBusinessEdge – 9/6/11
“The The challenge– and opportunity– of big data” McKinsey Quarterly—5/11 Quarterly 5/11
“Getting a Handle on Big Data with Hadoop” Businessweek-9/7/11
“Ten reasons why Big Data will change the travel industry” Tnooz -8/15/11 8/15/11
“The The promise of Big Data” Intelligent Utility-8/28/11
IT has always had an impact on Statistics
see www.abs.gov.au
Price Statistics MIT Billion Price Project
Demand for Jobs / Skills Help Wanted Statistics from the Conference Board
New Job Starts / Job Changes LinkedIn
OECD Output Forecasts SWIFT
Some economic policy implications
Benefits • Timeliness & now-casting now casting • Robustness & granularity • Affordability & access • Democratisation & creativity
Unknown properties of Web Data
Source: www.nature.com www nature com
Some economic policy implications
Ch ll Challenges •Unknown bias •Potential Instability •Quality
Some implications for NSOs: Will they get by by-passed passed by “Big Big Data Data” ?
Some policy implications for NSOs
Good news • Big Data tools are becoming widely available • The Cloud can address infrastructure needs • The “statistical statistical commons” commons grows
Some policy implications for NSOs
Challenges C a e ges to o Address dd ess •Access A &O &Ownership hi •Privacy Pi •Liability •Skills
Access to and ownership of proprietary data
Privacy Issues
•
image from http://wwwalthdatainnovation.com/sites/datawork.drupalgardens.com/files/styles/large/public/target.jpg
Liability
Skills “…the sexy job in the next 10 years will be statisticians.”
Source: NYT, 5 August 2009
Possible new NSO roles • Take on a new mission as a trusted 3rd party whose role would be to certifyy the statistical q quality y of these new sources? • Issue statistical “best practices” in the use of non-traditional sources and the mining g of “big g data”? • Use non-traditional sources to augment g ((and perhaps replace) their official series?
Going Forward • G Groves: “A bl blended d dd data t world” ld” – building b ildi on-top of existing surveys – Calibrating lib i web bd data to survey d data
• Use of relative vs. absolute measures, now-casting • Develop new methodologies • Active Experimentation, Experimentation extracting lessons, devising “best practices”
![[Ebook] Statistics for Big Data For Dummies Download ... - Google Sites](https://m.moam.info/img/260x300/ebook-statistics-for-big-data-for-dummies-download_64777e77097c4786708bc598.jpg)
