Industrial and Organizational Psychology

Industrial and Organizational Psychology http://journals.cambridge.org/IOP Additional services for Industrial

and Organizational

Psychology: Email alerts: Click here Subscriptions: Click here Commercial reprints: Click here Terms of use : Click here

Big Data Recommendations for Industrial–Organizational Psychology: Are We in Whoville? Christopher T. Rotolo and Allan H. Church Industrial and Organizational Psychology / Volume 8 / Issue 04 / December 2015, pp 515 - 520 DOI: 10.1017/iop.2015.76, Published online: 17 December 2015

Link to this article: http://journals.cambridge.org/abstract_S1754942615000760 How to cite this article: Christopher T. Rotolo and Allan H. Church (2015). Big Data Recommendations for Industrial– Organizational Psychology: Are We in Whoville?. Industrial and Organizational Psychology, 8, pp 515-520 doi:10.1017/iop.2015.76 Request Permissions : Click here

Downloaded from http://journals.cambridge.org/IOP, IP address: 54.210.20.124 on 18 Jan 2016

b i g data : h o rt o n h e a r s a n i – o

515

can and should play a leading role in helping organizations to capitalize on the benefits big data can reap. References Bosco, F. A., Aguinis, H., Singh, K., Field, J. G., & Pierce, C. A. (2015). Correlational effect size benchmarks. Journal of Applied Psychology, 100(2), 431–449. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. Guzzo, R. A., Fink, A. A., King, E., Tonidandel, S., & Landis, R. S. (2015). Big data recommendations for industrial–organizational psychology. Industrial and Organizational Psychology: Perspectives on Science and Practice, 8(4), 491–508. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton-Mifflin. Watson, A. M., Thompson, L. F., Rudolph, J. V., Whelan, T. J., Behrend, T. S., & Gissel, A. L. (2013). When big brother is watching: Goal orientation shapes reactions to electronic monitoring during online training. Journal of Applied Psychology, 98(4), 642–657.

Big Data Recommendations for Industrial–Organizational Psychology: Are We in Whoville? Christopher T. Rotolo and Allan H. Church PepsiCo, Inc., Purchase, New York

Guzzo, Fink, King, Tonidandel, and Landis’s (2015) focal article was intended to be not a set of standards but instead “stepping stones” to “raise awareness and provide direction” (p. 492) to our field for working with big data. We believe that the work done by the authors successfully achieved those objectives, and we encourage the Society for Industrial and Organizational Psychology (SIOP) Executive Committee to keep advancing this work toward greater clarity and guidance. Having clear alignment among members of our field and guidelines for handling nebulous issues such as big data is an important aspect of the scientist–practitioner model. As such, we have no substantial debate over the content being proposed in the focal article. In fact, we would advocate for a full set of standards in this area. We do, however, propose that, as important as it is, this work as commissioned by the SIOP Executive Committee is a shortsighted endeavor and does not take Christopher T. Rotolo and Allan H. Church, Global Talent Assessment and Development, PepsiCo, Inc., Purchase, New York. Correspondence concerning this article should be addressed to Christopher T. Rotolo, PepsiCo, Inc., 700 Anderson Hill Road, Purchase, NY 10577. E-mail: [email protected]

516

c h r i s t o p h e r t. r o t o l o a n d a l l a n h . c h u r c h

enough of a stand on the area to make a significant impact on practice for several reasons: 1. Big data is not a “movement,” nor is there a “world of big data” as commonly referenced in the focal article. Big data is just data. Although the volume, variety, velocity, and veracity might be staggering and unlike what we have been taught in graduate school, we are really just witnessing a reflection of our world today. In an increasingly digitized world, data become more prevalent, precise, and available (Manyika et al., 2011; McAfee & Brynjolfsson, 2012). As industrial–organizational (I–O) researchers and practitioners, we need to change the way we research, design, and use tools and processes to take advantage of these dynamics. 2. Relatedly, big data is much bigger than I–O psychology. It has implications for other areas of applied practice including organization development (OD; Church & Dutta, 2013), learning (Saunderson, 2014), and human resources in general (Bersin, 2013; Ferrar, 2014). If we don’t recognize this, we are essentially just talking to ourselves. We don’t “own” this space as we do for other areas such as employment testing, assessment, or job analysis. So although we certainly need to be aligned as a field as to how we treat anything new that impacts how we do research and practice, we wonder whether the focal article’s effort is akin I–O representing the “Whos of Whoville” (Seuss, 1954), and no one will hear us outside of our own speck of a profession. 3. In fact, given the prevalence of big data outside of our field, we believe that our efforts to bring a common set of standards and guidelines for use in organizational research and practice should extend to (if not be primarily focused on) audiences beyond I–O. We know from SIOP’s recent branding research that those aware of our field associate it with scientific rigor, ethics and integrity, and evidenced-based practices. With the focal article’s effort to work toward a set of standards for big data, we believe that SIOP has a prime opportunity to bolster our brand as workplace research experts and advocates who set theory, research, and practice. We would like to see us continue this important work but with a greater aperture than our own membership. We need to think more broadly and holistically and treat the dynamic that we call big data as an opportunity if we are to make a meaningful impact on organizational applications. This means expanding our vision and partnering with other associations, consortia, and professional societies (e.g., Human Resources People and Strategy, Society of Human Resource Management, Organization Development Network) to create a set of unified standards and guidelines. Otherwise we will have as much impact as


517

we have had on the practice of coaching over the years—that is, very little. 4. Perhaps most importantly, big data has the potential to displace the fundamental way we conduct our work. Employment testing is a prime example. This has been an area that has been traditionally associated with I–O psychology. Our field has been at the forefront of construct identification (via job analysis and competency modeling), selection instrument design (e.g., personality, cognitive ability), legal issues, and validation. We create tests, we validate them, we administer them to candidates, and we make hiring recommendations using evidencebased criteria. Big data is disrupting this paradigm. Today there are companies (typically without I–O psychologists on board, which is an entirely different issue altogether) using big data and sophisticated algorithms to r assess personality through mundane everyday interactions like smartphone games, Facebook profiles, and e-mail content; r determine job requirements and organizational culture from social media; and r calculate “fit scores” and making hiring recommendations based on a candidate’s publically available information. All of these issues result in a need to rethink the way we are approaching big data and, as a consequence, the recommendations from the authors to address these issues in research and practice. Let’s take an example. The focal article discusses principles such as informed consent and concerns about data privacy and core identification of information when dealing with big data. The authors then present a list of strategies for data protection, which are the same as those found in many survey research methods books dating back to the 1990s, when confidentiality versus anonymity concerns first became a major debate (Church & Rotolo, 2011; Kraut, 1996). Rather than just tinkering with the construct of informed consent and privacy, however, we propose that big data is forcing us to reexamine some of these fundamental principles entirely. The sheer variety of data, coupled with the velocity with which they are compiled for any given individual makes it nearly impossible to apply current consent and data privacy standards. For example, Guzzo et al. (2015) discuss the “flack” that Facebook and Uber received for conducting research with their user data without informed consent. We wonder whether the flack is really appropriate. After all, U.S. Census data are used for a very broad research agenda, and yet as citizens we do not provide informed consent for our data to be used. In fact, it is required by law that we participate in the Census, and although the government informs us about the purpose of the

518


data collection, we do not have the ability to “opt out.” In many ways, the outrage over the use of data for research and lack of informed consent as in the case of Facebook and Uber is ironic. Essentially, they caught flack for their experimental design. In other words, the uproar that they received revolved mainly around the fact that some users experienced something that other users didn’t, and the users weren’t asked which group (or whether) they wanted to join (obviously, the research wouldn’t have been valid had they done this). However, how much flack would Facebook have received if, for example, they implemented the first treatment to everyone for a period of time and then implemented the second treatment to everyone for a similar period time? Most likely, it would not have even hit our radar. In fact, we are surrounded by such manipulation based on our involvement, or lack thereof, all of the time. The entire function of marketing for example, does exactly this. Advertisements, promotions, and store placement, just to name a few, are all being driven by big data—that is, whether these actions drive us to buy these products or services (Brueur, Forina, & Moutlon, 2013; Duhigg, 2012). Clearly, we as I–O psychologists need to think beyond our own myopic view. There will come a time, we believe, when these big data applications will actually be quite effective at what they are attempting to do. When that time comes, our need for test design, setting cuts scores, conducting validation studies, and so on may even cease to be! Our field will need to adapt to these new approaches or cease to be relevant. That said, we are not yet close to approaching the deterministic and gloomy big data applications we see in science fiction films like Gattaca, Minority Report, or Divergent. We have an opportunity today to shape the use of these data in a positive, productive way, and Guzzo et al. (2015) is a step in that direction. In order to achieve what might be called “big data nirvana,” however, we need three things to happen beyond what has already been discussed. We position them as additional Vs to the four already mentioned above (volume, variety, velocity, and veracity): The first is validity. Organizations must have a better understanding than they have today of what the word validity actually means and its true relevance to the use of the massive quantities of data they are collecting. In other words, they must not be blinded to the lure of prediction at all costs and lose sight of the need for job relevance. As data become more granular, algorithms will get more and more sophisticated. Organizations will need to ask themselves not whether they can but whether they should. The second is valence. For big data to really be effective at predicting the right outcomes, organizations need to have professionals in place with the right level of data skills (something not even all I–O professionals have, let alone other types of organization learning and development practitioners,


519

and human resource professionals) to deliver the insights. People often assume that just because someone has a doctorate they can do this work well, when they really can’t. It is important to also note, however, exemplar data skills are not the be-all and end-all of big data either. Data-analytics professionals can and do come from many other fields (e.g., economics, business, finance, mathematics) where there is often no inherent humanistic values basis for the application of their work compared with a field such as OD or I–O (Church & Dutta, 2013). In fact, many organizations have purposefully decided to build breadth of capability in their analytics groups by including these diverse perspectives. This worries us some and leads us to the final missing V in the equation. The last is values. Using big data to drive positive organizational outcomes requires that the insights from big data are being delivered to decision makers by practitioners who have a set of values and ethics that dictate how they approach data. Although Guzzo et al. (2015) discuss the ethics of the data themselves, we also need to consider the ethics of the way in which data-driven insights are brought forward. Although “cherry picking” is certainly bad practice, it is also important to avoid the “stone soup” method of predictive modeling, where non-job relevant variables are part of the model because the statistical procedure said they should be. We have seen too much of this tendency in organizations when it comes to predicting key criteria such as turnover and high-potential leader “characteristics.” Unless we apply the appropriate OD and I–O frame of reference, values, and common sense (Church & Dutta, 2013) to the work, we may end-up with really bad decisions based on really big data. This is one of the fundamental differences between big data as an end unto itself and intentionally using data-driven methods to drive positive organization change (Burke, 1982; Nadler, 1977; Waclawski & Church, 2002). Thus, the idea of big data isn’t new, but how we chose to use it could be. In summary, although the recommendations outlined in the focal article represent a step in the right direction, we feel they do not push the argument far enough, nor do they tackle some of the broader and more systemic issues raised here. We believe that as the authors continue their work in this area, their scope should be increased to include a broader perspective and outreach to address issues that could have such a disruptive impact to our field. In the end, if we only talk to ourselves and abide by our own set of guidelines and standards, but no one else really listens or cares, aren’t we just the Whos in Whoville (Seuss, 1954)? References Bersin, J. (2013, October 7). Big data in human resources: A world of haves and have-nots. Forbes. Retrieved from http://www.forbes.com/sites/joshbersin/2013/ 10/07/big-data-in-human-resources-a-world-of-haves-and-have-nots/

520


Brueur, P., Forina, L., & Moutlon, J. (2013). Beyond the hype: Capturing value from big data and advanced statistics. In M. Toriello (Ed.), Perspectives on retail and consumer goods (pp. 4–9). Detroit, MI: McKinsey. Burke, W. W. (1982). Organization development: Principles and practices. Glenview, IL: Scott, Foresman. Church, A. H., & Dutta, S. (2013). The promise of big data for OD: Old wine in new bottles or the next generation of data-driven methods for change? OD Practitioner, 45(4), 23– 31. Church, A. H., & Rotolo, C. T. (2011). Revisiting the great survey debate: Aren’t we past that yet? Industrial and Organizational Psychology: Perspectives on Science and Practice, 4(4), 455–469. Duhigg, C. (2012, February). How companies learn your secrets. The New York Times. Retrieved from http://www.nytimes.com/2012/02/19/magazine/shopping-habits. html?pagewanted=all&_r=0 Ferrar, J. (2014, December 19). Predictive analytics: What big data means for the future of HR. Recruiting Daily. Retrieved from http://recruitingdaily.com/big-dataanalytics-hr/ Guzzo, R. A., Fink, A. A., King, E., Tonidandel, S., & Landis, R. S. (2015). Big data recommendations for industrial–organizational psychology. Industrial and Organizational Psychology: Perspectives on Science and Practice, 8(4), 491–508. Kraut, A. I. (Ed.). (1996). Organizational surveys: Tools for assessment and change. San Francisco, CA: Jossey-Bass. Manyika, J., Chui, M., Brown, B., Bughin, J., Dobbs, R., Roxburgh, C., & Byers, A. H. (2011, May). Big data: The next frontier for innovation, competition, and productivity. Retrieved from http://www.mckinsey.com/insights/business_technology/big_ data_the_next_frontier_for_innovation McAfee, A., & Brynjolfsson, E. (2012). Big data: The management revolution. Harvard Business Review. Retrieved from http://hbr.org/2012/10/big-data-themanagement-revolution Nadler, D. A. (1977). Feedback and organization development: Using data-based methods. Reading, MA: Addison-Wesley. Saunderson, R. (2014, March). What happens when you add big data to learning? Discovering how performance metrics, analytics, and learning can elevate business results. Training. Retrieved from http://www.trainingmag.com/trgmag-article/ what-happens-when-you-add-big-data-ld Seuss, D. (1954). Horton hears a Who. New York, NY: Random House. Waclawski, J., & Church, A. H. (2002). Introduction and overview of organization development as a data-driven approach for organizational change. In J. Waclawski & A. H. Church (Eds.), Organization development: A data-driven approach to organizational change (pp. 3–26). San Francisco, CA: Jossey-Bass.