Governance by Algorithms: Reality Construction by Algorithmic ...

Just, N. & Latzer, M. Governance by Algorithms: Reality Construction by Algorithmic Selection on the Internet. Accepted manuscript forthcoming in Media, Culture & Society 2016

Governance by Algorithms: Reality Construction by Algorithmic Selection on the Internet Natascha Just & Michael Latzer University of Zurich, IPMZ, Media Change & Innovation Division [email protected], [email protected], www.mediachange.ch Accepted manuscript: Forthcoming in Media, Culture & Society 2016 This research was supported by a research grant of the Swiss National Science Foundation (SNF). Abstract This paper explores the governance by algorithms in information societies. Theoretically, it builds on (co-)evolutionary innovation studies in order to adequately grasp the interplay of technological and societal change, and combines these with institutional approaches to incorporate governance by technology or rather software as institutions. Methodologically it draws from an empirical survey of Internet-based services that rely on automated algorithmic selection, a functional typology derived from it, and an analysis of associated potential social risks. It shows how algorithmic selection has become a growing source of social order, of a shared social reality in information societies. It argues that – similar to the construction of realities by traditional mass media – automated algorithmic selection applications shape daily lives and realities, affect the perception of the world, and influence behavior. However, the co-evolutionary perspective on algorithms as institutions, ideologies, intermediaries and actors highlights differences that are to be found first in the growing personalization of constructed realities, and second in the constellation of involved actors. Altogether, compared to reality construction by traditional mass media, algorithmic reality construction tends to increase individualization, commercialization, inequalities and deterritorialization, and to decrease transparency, controllability and predictability. Introduction The growing societal significance of algorithms on the Internet is now widely acknowledged (Salvin, 2011; Mager, 2012; Steiner, 2012; Anderson, 2013; Latzer et al., 2014; Gillespie, 2014; Pasquale, 2015). Wide ranges of daily activities in general and media consumption in particular are increasingly shaped by automated algorithmic selection. The selection of online news via search engines and news aggregators or the consumption of music and video entertainment via recommender systems are prominent examples. This paper focuses on the governing effects of Internet algorithms in information societies. In other words, it scrutinizes the governance by algorithms as opposed to the

1


governance of algorithms (Saurwein, Just and Latzer, 2015). How, what and to what extent do algorithms on the Internet govern? What are the peculiarities of algorithmic governance and its impact on reality construction and social order? The paper first presents a conceptual framework of how to grasp the phenomenon of algorithms on the Internet, its magnitude, its basic principles, and the societal functions of applications that build on algorithmic selection. Section two focuses on algorithmic selection on the Internet and discusses algorithmic governance, starting from the more general question of if and how technology – in this case software – governs modern societies. Theoretically, this section builds on (co-)evolutionary innovation studies in order to grasp the interplay of technological and societal change, and combines these with institutional approaches to incorporate governance by technology, or rather software, as institutions. Section three discusses the governing effects of algorithmic selection on the Internet as a specific form of algorithmic reality construction. It starts from the observation that the market for attention – the central scarce resource in information societies – is increasingly being co-produced and allocated by automated algorithmic selection in many life domains. Section four compares algorithmic reality construction with the construction of realities by traditional mass media. It identifies major characteristics and differences, and discusses their possible societal consequences. The final section summarizes the results and draws conclusions. 1. Algorithmic selection on the Internet: a wide and rapidly growing phenomenon To discuss algorithmic governance it is first necessary to define the vague phenomenon of algorithms on the Internet. Algorithms are problem-solving mechanisms. In order to empirically grasp their role, this paper focuses on Internet-based services that build on algorithmic selection. Algorithmic selection is essentially defined by the automated assignment of relevance to certain selected pieces of information. A rapidly growing number of Internet applications build on algorithmic selection and can be categorized according to their central societal functions (table 1). This categorization is the result of an empirical survey of over 80 Internet-based services that were screened using a basic analytical grid. Ranging from search, forecasting and surveillance to filtering, recommendations and content production (Latzer et al., 2014), it permits a rough appraisal of the magnitude and high societal significance of the phenomenon, and forms the basis for the discussion of the governing role of algorithms in information societies. Such societies are characterized by a growing flood of (big) digital data that provide the basis and create an

2


equally rising demand for automated algorithmic selection in order to handle and make sense of these massively collected data. Big data (a new economic asset class) and algorithmic selection (a new method of extracting economic and social value from big data) are coevolving. Table 1: Functional typology of algorithmic selection applications Types search

aggregation observation/surveillance

prognosis/forecast

filtering recommendation scoring

content production allocation

Examples general search engines (e.g., Google search, Bing, Baidu) special search engines (e.g., Mocavo, Shutterstock, Social Mention) meta search engines (e.g., Dogpile, Info.com) semantic search engines (e.g., Yummly) question & answer services (e.g., Ask.com) news aggregators (e.g., Google News, nachrichten.de) surveillance (e.g., Raytheon’s RIOT) employee monitoring (e.g., Spector, Sonar, Spytec) general monitoring software (e.g., Webwatcher) predictive policing (e.g., PredPol), predicting developments: success, diffusion etc. (e.g., Google Flu Trends, scoreAhit) spam filter (e.g., Norton) child protection filter (e.g., Net Nanny) recommender systems (e.g., Spotify; Netflix) reputation systems: music, film, etc. (e.g., ebay’s reputation system) news scoring (e.g., reddit, Digg) credit scoring (e.g., Kreditech) social scoring (e.g., Klout) algorithmic journalism (e.g., Quill; Quakebot) computational advertising (e.g., Google AdSense, Yahoo!, Bing Network) algorithmic trading (e.g., Quantopian)

Source: Latzer et al. (2014).

The functioning of these algorithmic selection applications can be best described and explained with a basic input-throughput-output model (figure 1), a starting point for further research on the specifics of the above-mentioned service categories. Algorithms form the centerpiece of the throughput stage where they operate. Starting from a user request and available user characteristics they apply statistical operations to select elements from a basic data set (DS1) and assign relevance to them. Accordingly, algorithmic selection on the Internet is defined as a process that assigns (contextualized) relevance to information elements of a data set by an automated, statistical assessment of decentrally generated data signals. In detail, input, throughput and output vary for different applications and services. Often, big data serve as input, but there is a wide spectrum of input sources, depending on the field of application. The throughput process is characterized by the assignment of relevance

3


(A2) and respective selections (A1), based on a multitude of different codes and operating modes (e.g., matching, sorting or filtering algorithms). Finally, the output (DS3) also assumes different forms (e.g., rankings, recommendations, biddings, text, music). Often it also serves as additional input for subsequent algorithmic selection processes. Figure 1: Input-Throughput-Output model of algorithmic selection on the Internet

Source: Latzer et al. (2014).

2. Algorithmic governance as governance by software Governance can broadly be understood as institutional steering (Schneider and Kenis, 1996), as the horizontal and vertical extension of traditional government (Engel, 2001; Latzer et al., 2002). Horizontally, this requires looking beyond public actors (e.g. governments) – including private actors (e.g. industry self-regulation), but also considering the active, governing role of technology adequately. Vertically, it suggests focusing on multilevel-governance, looking beyond mechanisms and instruments like multi-stakeholderism and multilateralism in Internet governance (Christou and Simpson, 2011), also including multilevel/global aspects and technological governance strategies.

4


2.1 Growing importance of and attention to software Algorithms on the Internet are software. Hence governance by algorithms is, in essence, an example of governance by technology. Over the course of time, technology and in particular software have been playing a growing role in the media sector. This can be illustrated by the distinction between primary, secondary and tertiary media (Pross, 1970). Primary media like speech do not need technological artifacts (devices), secondary media like books and newspapers need technology only on the sender’s side, and tertiary media like telephones, radio and television need technology on the recipient’s side too. Digital, converged media (e.g., Internet-based media) can be considered quaternary media, characterized by a further step of technologization. The dependence on technological interfaces and devices (especially on software) now extends to all digitized media formats (also to secondary media). Recipients of digital books and newspapers need interfaces (hardware and software) too. This additional technological intermediation on the user side changes the use and the effects of media. Information in the form of text, data, sound, speech, pictures and videos can only be utilized efficiently via software applications, and is consequently shaped by this software. As Manovich (2013) argues, digital media have no attributes per se, because these are more or less assigned by the software used. An important part and special form of this software-based imprinting of digital media is the automated assignment of relevance by algorithmic selection on the Internet. This not only influences what is found but also the reputation and trust in it. Generally, the significance of technology and its analysis is boosted by a central characteristic of the convergence trend in communications (Latzer 2013c), by the disentanglement of media technology and media content. Related to this is the emerging awareness that technology can have effects, and can be conceived as an actor or agency – as something that can create meaning by itself – or as institution with effects on individual/collective behavior and social order. This situation calls for and leads to an intensified discussion about the role and characteristics of different technologies. Consistent with the growing awareness of the significance of (software) technology in the evolution of communication systems, technological issues are increasingly seen and treated as policy issues, also in the case of algorithms. This directs attention away from a merely functional/instrumental understanding of technology to an understanding that technology (software) is design/reality construction (Floyd, 1992), law (Reidenberg, 1998; Lessig, 1999), able to evoke certain behavior and to shape and reshape activities and meaning

5


(Winner, 1986). Currently discussed examples are certain architectural features of search algorithms like transparency or openness (Shah and Kesan, 2003; 2011). If Internet governance is defined as how a multitude of actors and their governance mechanisms/instruments shape the evolution and the use of the Internet (WGIG, 2005), then (software) technology needs special attention – both as a governance mechanism/instrument and as an actor. 2.2 A co-evolutionary and institutional perspective on algorithmic governance Assessments of the role of technology as a governance instrument and actor in general, and of algorithmic selection on the Internet in particular, depend on and differ with the choice of analytical lenses. The long debate on the right approach is characterized by an antagonism between technical and social determinism. In most cases, also in communications, technology is nowadays understood as being primarily shaped by social forces. However, it can be argued that this perspective systematically underestimates the role of technology (Latzer, 2013a). Research is confronted with the situation that theories, classifications and research findings have been elaborated on the basis of a techno-economic reality that no longer exists, based on a pre-Internet reality that does not reflect media convergence and the evolution of quaternary media, represented, for example, by algorithmic selection applications. An innovation-coevolution-complexity approach (Latzer, 2013a) allows the appropriate integration of the role of technology, and of technological change. Innovations like novel algorithms are the nucleus of change. An advantage of innovation approaches in technology studies is their finding that technology is not just formed by society, but that it can also be active as a structure, institution and even as actor (Dolata and Werle, 2007). Innovation theories are combined with evolution theories towards an evolutionary economics of innovation (Frenken, 2006). According to this perspective, technological innovation processes are understood and assessed as evolutionary processes (Nelson and Winter, 1982). There are similarities between biological and technological evolution, but they are not the same. There is no natural selection in the technical field; technologies are the product of design efforts, and there is nothing like genes in technology. Nevertheless, biology and technology can both be considered part of the ‘family’ of complex systems (Ziman, 2000). Common features like interdependencies of elements, non-linear developments, emergence and feedback-loops characterize complex systems. Further, they are both systems

6


where big networks of interdependent components without central steering and with only simple rules on the individual level develop sophisticated collective behavior, characterized by highly developed information processing and a capability to adapt via trial-and-errorlearning, which is characteristic of evolution (Mitchell, 2009). These complexity characteristics, including the resultant low predictability and controllability in policy-making (Latzer, 2013b), are helpful in understanding the evolution of the Internet in general and the effects of algorithms on the Internet in particular. Another advantage of this approach should be underlined: first, evolutionary approaches are not only characterized by variation and selection and adaptation, but also by cooperation as a success factor in selection processes. This focus on cooperation is helpful as – in contrast to self-interest – it is often emphasized as a special feature of the Internet economy, in combination with the growing role of sharing in the Internet economy and the networked public sphere (Benkler, 2011). Moreover, cooperation is an important factor of social order. Second, complexity economics (Beinhocker, 2005) challenges strong rationality assumptions of neoclassical and neo-institutional economics, focuses on selection processes, and in particular refers to corporations as those who select technical and social innovations. Alongside this selection within a company, there are also selection and search processes through user choice (Frenken, 2006), which are of particular importance in automated algorithmic selection. Co-evolution is a helpful concept to analyze media change and (Internet) governance. While evolution can be characterized as design without central designer, co-evolution means simultaneously designing and being designed. This concept thus overcomes the antagonism between technological and social determinism and focuses on the interrelation and interdependencies of technical, economic, political and cultural driving forces in governance processes. An assessment from this combined theoretical perspective presents the Internet as a complex, adaptive system, characterized by interdependencies, non-linear developments, emergence and decentralized structures. It highlights not only the content of the Internet but in particular its infrastructure, i.e. its architecture, which allows innovations at every node of the network, in other words by any user. This specific architecture makes the Internet an innovation machine (Whitt and Schultze, 2009), a modular, open system, a digital construction kit, which offers great flexibility for innovatively assembled services. The previously rigid combination of technology and services (content) is dissolved and

7


complexity increases. These attributes are essential preconditions for the rapid spread of algorithmic selection applications in many life domains, because they allow decentralized innovations. They are also vital for the further development of algorithmic selection applications. In general, it focuses attention on the growing importance of software that shapes all Internet-based content, whether text, audio, pictures or video. As mentioned above, co-evolutionary innovation studies conceptualize technology not only as being shaped by social forces but also as being active as structure, institution or even as actor. So the co-evolutionary approach, which primarily helps to reflect the role of technology adequately, is here combined with an institutional perspective (Reidenberg, 1998; Lessig, 1999; Shah and Kesan, 2011; Napoli, 2014) to better reflect the governing role of algorithms. The focus is on the role of algorithms as institutions (as governance instruments) and also as (governing) key actors themselves. An institutional perspective identifies algorithms as institutions, as norms and rules that affect behavior on the supply and demand side, as a set of rules and routines that both limit activities and create new room for maneuver. Some of them stem from the outcome of evolutionary (trial-and-error) design processes, from technical code and architecture. Several authors underline the role of technologies as institutions that impose certain rules and influence behavior (Reidenberg, 1998). The governing effect of software on social behavior, alongside and in interplay with norms, laws and markets is highlighted with propositions like ‘architecture is politics’ and ‘code is law’ (Lessig, 1999). Shah and Kesan (2003, 2011) specifically emphasize the governing role of software. They provide case studies of its manipulable characteristics, like transparency, defaults and (open) standards, which offer starting points for governing the behavior of technology. On the basis of Shah and Kesan (2011), a basic co-evolutionary governance framework of interrelated technological (algorithmic) and societal change can be described as follows (figure 2): in line with the understanding that co-evolution has no beginning or end, developers design software, software shapes software (self-learning systems), software changes and is changed by users, users form and are formed by societies, societies influence developers and users via institutional imprinting, by social, economic, political and regulatory forces. Altogether, from an institutional perspective, technologies like algorithms are both instruments and outcome of governance (Katzenbach, 2012), they are part and result of a coevolutionary process.

8


Figure 2: Basic co-evolutionary governance framework of (software) technological and societal change

Source: Adapted from Shah and Kesan (2011: 127).

According to the combined theoretical approach applied here, algorithms are generalpurpose technologies (Bresnahan, 2010) with impacts on a wide range of societal fields, they are enabling technologies with a certain impact in connection with and contingent upon social-use decisions. As institutions they operate and govern (Orwat et al., 2010; Napoli, 2014) both supply and demand, in line with Reidenberg’s (1998) and Lessig’s (1999) insight that code is law, or as an example of what Braman (2002) calls posthuman law – information technology not only as the subject of laws but also as part of policy-making and as policymaker. Accordingly, algorithms on the Internet can be seen as governance mechanisms, as instruments used to exert power and as increasingly autonomous actors with power to further political and economic interests on the individual but also on the public/collective level. Algorithmic governance is more evidence-based and data-driven than traditional governance. Based on big data analysis, as for example in the case of predictive policing, huge data sets are analyzed to predict crimes (what, when and where) and to take suitable countermeasures. These predictions are not based on the analysis of causes but on various (historic) patterns of evidence. In line with the argument that big data analysis is more about correlations than causes (Boyd and Crawford, 2012), algorithmic governance focuses more on the governance of effects than on causes (Morozow, 2014).

9


From a media perspective, algorithmic selection significantly influences both media production and media consumption (Napoli, 2014). Algorithmic selection on the Internet not only influences what we think about (agenda-setting), but also how we think about it (framing) and consequently how we act. From a more general perspective – looking beyond the media sector – the market for attention is increasingly being produced and allocated by applications that build on algorithmic selection. Attention is drawn to certain things at the expense of others. Algorithmic selection shapes the construction of individuals’ realities, i.e. individual consciousness, and as a result affects culture, knowledge, norms and values of societies, i.e. collective consciousness, thereby shaping social order in modern societies. This makes algorithms a highly strategic factor in information societies and serves as an argument that the design of these institutions needs democratic legitimation, e.g. in the form of stakeholder participation, like open-source, democratized standardization processes or public certifications (Orwat et al., 2010). 3. Algorithmic reality construction and the formation of social order The governing character of algorithms, their role in reality construction, makes them a source and factor of social order. Social order is needed in societies to bridge the individual and the social, and can be judged on the coordination of actions and cooperation between individuals to achieve common goals (Hechter and Horne, 2003). The logic of the Internet, of social media in general (van Dijck and Poell, 2013) and algorithmic selection in particular, influences this coordination and cooperation, whether in the form of datafication, connectivity, automation, virtual communities, social capital or weak ties. Fullerton and Ettema (2014), for example, illustrate how the collaborative process of creating Wikipedia entries is a cognitive and normative exercise that constitutes a way of knowledge construction, in their terms worldmaking. Van Dijck (2013) points to the important role of automated bots in this context, both as administrators performing policing tasks, such as blocking spam and detecting vandalism, and as co-authors. This makes Wikipedia an example of an institutionalized interaction between human and nonhuman actors, of a sociotechnical ensemble that engineers social order. In general, social order is based on a shared social reality (Scott, 1987; Berger and Luckmann, 1967) – whether it is objective (e.g. governments), symbolic (e.g. media content) or subjective reality (e.g. preferences) – which emerges from real or symbolic interactions (Adoni and Mane, 1984). Social reality is now increasingly shaped and constructed by

10


algorithmic selection on the Internet in various life domains. Like the mass media, institutionalized services that centrally build on algorithmic selection contribute to objective and subjective reality constructions, but also to symbolic realities, for example with automated algorithmic journalism (Dörr, 2015) or when journalists use these services to collect information. In line with Winner’s (1986) argument that technological innovations have politics and co-establish a framework for public order in combination and interaction with coevolving political (e.g. law) and economic institutions (e.g. markets), relevance-assigning algorithms govern as (quasi-social) institutions. They embody values and can organize and impose order on society by both affording and impeding certain practices, behaviors and activities (Nissenbaum, 2011). Whereas a special function of the traditional mass media is its contribution to a shared social reality, the question arises of what impact the increasingly personalized nature of algorithmic reality construction has on this ‘shared’ social reality, especially regarding potential detrimental effects on democracy. According to empirical evidence, automated algorithmic selection on the Internet governs a wide spectrum of individual action, and is heavily used for various societal functions (table 1). Algorithms co-govern or co-determine what: can be found on the Internet (search applications, e.g. what is indexed by search engines/crawlers); is seen and found (search, filtering and aggregation applications); is produced (content production applications like algorithmic journalism); is considered relevant (search and scoring applications; ranking); is anticipated (prognosis/forecast applications); and is chosen and/or consumed (recommendation, scoring and allocation applications; both for economic and social choices – ranging from commercial goods to friends and partners.) Taken together, algorithmic selection essentially co-governs the evolution and use of the Internet by influencing the behavior of individual producers and users, shaping the formation of preferences and decisions in the production and consumption of goods and services on the Internet and beyond. The individual behavior of Internet users at the micro level, for example, has emergent, hence unexpected effects on the web structure at the macro level. It co-determines the evolution of the Internet and should be considered an influential factor in Internet governance. Moreover, algorithmic selection contributes to reality construction, a kind of governance marked by the selection or omission of information. The result, an algorithmically formed reality, again governs – this time behavior and action, i.e. various choices in daily lives.

11


4. Differences between reality construction by algorithms and the mass media Various societal functions of algorithmic selection on the Internet contribute to new dimensions and forms of reality mining and reality construction in information societies. This mix of mining and construction by selection – a special kind of data mining and data interpretation – increasingly forms the basis for more or less intentional governance processes. Consequently, the realities shaped by automated algorithmic selections codetermine individuals’ coordination and cooperation on and beyond the Internet and consequently constitute social order. The question of how traditional mass media contribute to the construction of social realities by selecting or omitting certain information (e.g. gatekeeping, agenda-setting, framing) has always been prominent (McQuail, 2010). However, reality construction by automated algorithmic selection on the Internet differs from traditional reality construction by the mass media. A major difference is (1) the personalization of reality construction that contributes to further individualization in societies, and (2) the constellation of actors that are a constituent part of the Internet’s ecosystem. 4.1 Personalization A major goal and feature of algorithmic selection applications is the personalization of processes and results. In a nutshell, governance by algorithmic selection is automated, instantaneous (real time), predominantly based on big data, partially self-learning and always context-related/personalized, applying customized selection criteria. Personalization happens on the basis of one’s own user characteristics (socio-demographics) and own (previous) user behavior, others’ (previous) user behavior, information on user-connectedness, and location. Algorithmic selection differs from traditional mass-media selection mechanisms, which operate with a time delay and are mostly targeted at well-defined general publics and (mass) markets whose characteristics are known from limited data sources (representative polls, socio-demographic patterns, TV meters in selected households) compared to big data in the case of algorithmic reality mining and construction. The database for algorithmic selection applications consists of both active consumer input (e.g. feedback) and passive data (e.g. location-based, clickstream, social contacts). A major difference is the special role of users in algorithmic reality-construction processes. They are no longer simply the addressees of media messages, with limited or no

12


interaction between sender and receiver, but – together with algorithms – they assume an important role as secondary gatekeepers (Singer, 2014; Wallace, 2015), data providers and inputs into selection processes, which are increasingly based on previous and predicted behavioral patterns generated from real-time data. Consequently, personalized results of algorithmic selection procedures sum up to different individual realities. Altogether this amplifies existing audience fragmentation and individualization trends, resulting from the proliferation of media outlets triggered by liberalization, privatization and digitalization. In general, both the fragmentation and individualization of audiences have been discussed as being detrimental to democracy (Katz, 1996; Mancini, 2013), and personalized reality constructions by algorithmic selection exacerbate these concerns. Dangerous and endangered individuals According to Schroer (2008) the debate on individualization distinguishes between various lines of reasoning and emphasizes the dangerous and the endangered individual or combinations of both. While the debate on the dangerous individual focuses on the threat to social cohesion and order due to increased individualization and the subsequent erosion of common norms and institutions, the discussion about the endangered individual highlights the controlled individual. More effective control and disciplining take the place of increasing autonomy and freedom of action that individualization actually promises. The individual is consequently not liberated but more strictly controlled. Algorithmic reality construction as result of a continuing evolutionary differentiation of information and media systems further enables and fosters individualization in the form of dangerous individuals. Individual autonomy, alternatives and scope are further expanding, resulting in a further liberation from common societal bonds. At the same time, however, new forms of connectivity (Van Dijck, 2013), new bonds and (weak) ties (Kavanaugh et al., 2005) are created. In addition, algorithmic selection also results in endangered individuals due to increased control, stemming, among other things, from highly customized personalization efforts. The discussions on data protection, privacy, data ownership, access to information and surveillance highlight the control aspects entailed in increasing automation, reality mining and algorithmic selection. Deliberate individualized media use of dangerous individuals may result in the weakening of bonds with traditional media institutions, as seen especially in the debates on the future role of public service media in the light of changes in media usage patterns and

13


available media outlets. Moreover, specific forms of algorithmic personalization may lead to isolating, echo-chamber effects (Sunstein, 2007), also discussed as filter bubbles (Pariser, 2011). Accordingly, concerns are raised regarding the general impact of algorithmic selection on people’s access to information via search, social media and aggregators, and amplified by trends towards further customization enabled by services such as Google Now, Cortana (Microsoft) and Proactive Assistants (Apple). Altogether this may lead to situations where people only access information that confirms their own opinion, or communicate with likeminded people, with potentially negative democratic consequences for societies, such as endangering two preconditions for democratic systems: unplanned encounters and shared experience (Sunstein, 2007). Networked, secondary, non-journalistic filtering The fact that algorithmic selection is assuming (secondary) agenda-setting and (secondary) gatekeeping roles via news aggregators, via ranking algorithms in discussion forums, and in social online networks like Facebook, leads to the assessment that algorithms considerably affect the way public opinion is formed, that they govern the public agenda (Bakker, 2012; Van Dalen, 2012; Machill et al., 2013). Filtering by algorithmic selection should at least be seen combined or networked with the filtering by mass media (Barzilai-Nahon, 2008). It may be primary or secondary filtering (Singer, 2014), the latter refers to the situation where, e.g. regarding political information, these algorithmic processes are based on the results of traditional agenda-setting and gatekeeping by the mass media, and then a second ‘filtering’ (algorithmic selection) takes place based on automated combinations of user behavior, established connections, and non-journalistic, customized filtering criteria. 4.2 Special actors’ constellation in algorithmic reality construction Major features of the actors’ constellation in algorithmic reality construction include the dominance of private companies and interests, rising platformization, and algorithms as actors and policy-makers. The question of who determines the criteria for these socially sensitive selection processes is paramount. Traditionally the selection of news was a manual human or institutionalized privilege of journalists and their (national) media organizations on the basis, among other things, of news factors. The selection was targeted at a (geographically) distinct mass audience and at least considered to be in accord with high standards of social

14


responsibility, especially in the case of public service media. Now, in the case of algorithmic reality construction, the selection happens automatically through customized software and services mostly developed and dominated by global IT companies. Dominance of private governance by global companies A distinction can be made between private and public algorithmic governance. Empirical research shows that global IT companies dominate in markets where algorithmic selection is applied either as core or ancillary service (table 2). This is an indicator of the wide societal spread and impact of these technologies in general, and of their essential governing influence on the evolution of the Internet in particular. Table 2: Algorithmic selection in top 10 websites worldwide Company and country of origin

Algorithmic Selection as Core Service general search engine computational advertising

Algorithmic Selection as Ancillary Service

Ranking

Website

1

google.com

2

facebook.com Facebook (USA)

computational advertising

3

youtube.com

Google (USA)

computational advertising

4

baidu.com

Baidu (CHN)

5

yahoo.com

Yahoo (USA)

6

amazon.com

Amazon (USA)

special search (products) recommendations (products) reputation (marketplace sellers)

7

wikipedia.org

Wikimedia Foundation (USA)

special search engine

8

qq.com

Tencent (CHN)

Google (USA)

general search engine computational advertising general search engine computational advertising

general search engine computational advertising

autocomplete filtering (EdgeRank) social search (GraphSearch) recommendations (contacts) variety of recommendations special search engine autocomplete autocomplete

autocomplete

special search (products) recommendations (products) reputation (marketplace sellers) aggregations/recommendations 10 twitter.com Twitter (USA) computational advertising (Twitter Trends, Who to Follow) Source: Ranking based on alexa.com, 09-07-2015. Note: Core function basically means that the results of algorithmic selection is the product demanded (e.g. search results), and ancillary functions are used to support the core service of a company in order to gain competitive advantage (e.g. algorithmic recommendations in ecommerce services). 9

taobao.com

Alibaba Group (CHN)

Public algorithmic selection services also exist but are employed less often, for example, for public security objectives (e.g. predictive policing, public surveillance). Private

15


algorithmic selection services, e.g. Google Flu, which predicts disease outbreaks (Lazer et al., 2014), could be used for public governance purposes as well. This in turn raises additional (black-box) challenges, if findings cannot be replicated and are based on proprietary private datasets of unknown quality. With the notion of ‘Blackbox Society’ Pasquale (2015) highlights two general features of rising algorithmic selection: (1) to gain basic datasets for algorithmic selection, everything is recorded as by a black box in an airplane, and (2) like a black box the throughput stage and its algorithms remain secret. The fact that the big datasets often remain under exclusive control of global companies (e.g. social-media companies), leads to a new form of digital inequality in algorithmic reality production, and to nonreplicable results, an indication of decreasing quality. Limited transparency causes basic algorithmic accountability challenges and encourages the use of methods like reverse engineering to shed light on the detailed interplay of input, throughput and output (Diakopoulos, 2015). From a public policy perspective the question emerges of if and how this private algorithmic form of governance, which is mainly designed to further individual profit maximization, could be legitimized. In its current form, algorithmic selection is hardly used for social-political governance purposes but mostly for purely commercial goals. Companies like Google concentrate their business models on various, but closely connected puzzle pieces of reality mining and construction. The perspective on algorithms as ideologies (Mager, 2012) reflects this situation. Technologies in the form of algorithms convey, reproduce and reinforce beliefs and values (Nissenbaum, 2011). In general, however, technologies – in this case algorithms – should be seen as amplifiers of existing trends, of dominant ideologies and not so much as creators of new social trends. As such it can be argued that algorithms predominantly convey and reinforce commercialization as the dominant value. Another governance issue is that algorithms are designed on a global level, i.e. national effects on social ordering are decided at that level, mostly by global IT companies. This may advance processes of deterritorialization and dissociation (Tomlinson, 1999), and result in a further disembedding of societies from discrete national contexts and media systems, with detrimental democratic consequences (Mancini, 2013).

16


Rising intermediation and platformization Nowadays, reality-construction effects by the mass media are (usually) coupled with or intermediated by algorithmic selection, e.g. by news aggregators, scoring applications or intelligent personal assistants. Many of the key Internet businesses such as Google, Facebook, or Amazon can be described as platform markets, often characterized as two- or multisided markets (Rochet and Tirole, 2003), i.e. markets serving two or more distinct customers with interdependent demand. Accordingly, providers of algorithmic selection services are usually active as intermediaries, as market-makers between two demand sides (Latzer et al., 2014). This changing constellation of actors modifies power structures and focuses attention on the interdependencies and relationships between the platforms and different national and global media players. For example, these platforms often establish themselves as ‘bottleneck monopolists,’ by controlling access to their own services as well as access to other services and products via their platforms (Shelanski, 2013). This has in particular raised competitive concerns and intellectual property rights questions, as in the case of the Google News shut down in Spain in 2014. A similar incident was avoided in Germany because publishers eventually conceded use of their content at no charge to Google. Altogether, this increasing intermediation may result in a reduced role and power of national mass media in the construction of realities. Algorithms as autonomous actors and policy-makers Compared to realities traditionally constructed by the mass media, a further difference in the constellation of actors for automated algorithmic reality construction is the potential role of technology as an actor (Braman, 2002; Napoli, 2014). Do algorithms only solve problems on behalf of humans? Are algorithms non-human actors that are inherently connected with human actors? Or are algorithms independent, autonomous actors themselves? In particular, the question of the power of algorithms, indicated by the degree of autonomy in their decision-making, needs to be discussed, their use as instruments to exert power but also as themselves having power to enforce objectives against others’ interests. This includes issues of algorithms’ agency, responsibility and predictability, whether they are only problem-solving, computerized human instructions, if their activities and output are and always remain foreseeable. Predictability may be compromised, for example with advances in machine-intelligence, and leads to a situation where manual computer programming becomes

17


inferior to computers’ self-learning processes regarding the optimal exploitation of existing (big) data. Applications based on algorithmic selection (see table 1) are characterized by the coevolution of humans (developers, user) and non-humans (algorithms) within a networked information society (see figure 2). Humans shape algorithms and are simultaneously shaped by them. According to ANT – Actor-Network-Theory – (Latour 2005) humans and algorithms are equal actants within this network. Mitcham (2014) shows, how the contested assessment of agency and intentions of artifacts has changed over time. His overview helps to tackle the question of if and how (moral) human agency is transferred to algorithms. Accordingly, human agency is either imposed or delegated to algorithms. The first results in a secondary/imposed agency of intentionless algorithms used to extend human (political) agency (Winner, 1986). Examples would be comparably simple and controllable algorithmic selections based on small datasets. This form of imposed agency of technology is also found in reality construction via traditional mass media. In the second case, algorithms act with delegated agency in a trustee-mode within a range of possible actions, comparable to public broadcasters or human delegates/representatives like parliamentarians. This delegated agency appropriately reflects the majority of current and anticipated algorithmic reality constructions. Technological progress in machine learning, including data mining, computational statistics and neuronal networks, increasing availability of big data sets in combination with the potential to discover unexpected similarities between old and new data that make it possible to solve non-routine tasks, and a growing automation and division of labor in software engineering (Frey and Osborne, 2013) support this assessment. Accordingly, algorithms can be considered as agent-like artifacts or actors, and – depending on their use – even as policymakers. They are autonomous within a pre-defined – but barely controllable – scope of action. There is distributed agency (Rammert 2008) between humans and algorithms, and also between algorithms, characterized by an increasingly complex and contingent interactivity. Alongside relative autonomy, these algorithmic agents are characterized by reactivity, proactiveness and sociability (Rammert 2003). Altogether, reality construction, both via the mass media and algorithmic selection, is a co-production of humans and technology. However, for algorithmic reality construction, agency is not only imposed, as in traditional mass-media technology, but predominantly delegated, allowing limited algorithmic intentions and autonomy. Overall, the (delegated) power of algorithms is rising. A high degree of complexity in the cooperation between

18


algorithmic agents and humans results in low transparency (not only for users, as in the mass media, but also for producers), controllability and predictability compared to reality construction by traditional mass media. Agency and accountability problems become more important (Chopra and White, 2011) as well as the moral significance (Verbeek, 2014) of algorithms. Even programmers and software engineers increasingly do not know what ‘their’ algorithmic selection produces (Auerbach, 2015). Hence the discussed disclosure of source codes might often not be a sufficient solution for more democratic control, and there are hardly any other public measures to control the morality in algorithmic reality construction. 4.3 Technology as problem and solution Algorithmic governance not only contributes to economic welfare gains that increase subjective well-being (e.g. by boosting economic circulation and by reducing complexity), but also produces considerable social risks that compromise potential welfare gains. Against the background of far-reaching algorithmic governance effects, this raises the question of whether public policy measures are needed (e.g. public-interest search engines). These questions are discussed under the term governance of algorithms (Saurwein, Just and Latzer, 2015) and based on identified risks, including manipulation, bias, heteronomy, threats to privacy, intellectual property rights and freedom of expression (Latzer et al., 2014). Taken together, three categories of risk are discernible: the impacts on the mediation of reality, the threats to basic rights and liberties, and the challenges to the future development of the human species. Governance by technology, more precisely by its architecture and design, is considered to be the problem for all of the above risks, but it is also technology that offers solutions. Governance by design is taking place more or less intentionally. It is predominantly applied for special interests, but could be also used to further public policies and public goals. The example of a decentralized architectural design for P2P video streaming (Musiani, 2013) shows how governance by design could be used to further public goals, e.g. the protection of privacy. For algorithmic design it can be demonstrated how design both creates and could eliminate filter bubbles (e.g. by special widgets) that affect the democratically sensitive issue of public-opinion formation (Munson et al., 2013).

19


5. Summary and conclusions This paper explores the governance by algorithms on the Internet. It focuses on Internet-based services that build on algorithmic selection, i.e. the automated assignment of relevance to selected pieces of information. In order to grasp this fast growing phenomenon and its societal relevance the paper provides an empirically based, functional typology of the rapidly growing number of services that build on automated algorithmic selection on the Internet (table 1). This typology demonstrates the broad scope of algorithmic selection applications in information societies, with algorithmized societal functions ranging from search and recommendations to forecasts and content production. The functioning of these services can be understood by applying and specifying a basic input-throughput-output model (figure 1). Algorithmic selection applications are a prime example of the growing importance of software – not only for the media sector as part of quaternary media, but also far beyond. Moreover, it is a perfect example of the governing role of software. As an institution, software affects societies similarly to laws, contracts and values that are imprinted in algorithms. A basic governance model of technological and societal change (figure 2) shows the co-evolutionary interplay and highlights the role of technology as simultaneously designing and being designed. Algorithms are active as governance tools but also as selflearning and relatively autonomous actors in increasingly complex ecosystems, characterized by non-linear developments, emergence and feedback loops. The consequences of growing complexity are declining predictability and controllability, as well as increasingly unintended consequences of private and public attempts to govern via algorithms. Based on empirical-analytical findings, it is argued that algorithmic selection on the Internet has become a growing source of and factor in social order, in a shared social reality in societies, which is increasingly being co-constructed by automated algorithmic selection on the Internet. Automated algorithmic selection applications shape realities and daily lives, increasingly affect the perception of the world and influence behavior. They not only influence what we think about, but also how we think about it and consequently how we act, thereby co-shaping the construction of individuals’ realities, structurally similar but essentially different to mass media. Algorithmic reality construction has various peculiarities and differs from reality construction by the traditional mass media, recoining realities and consequently the social order in contemporary information societies. In general these differences lie first in the growing personalization of constructed realities and the subsequent individualization effects.

20


Second, there are major differences in the constellation of actors, a constituent part of the Internet’s ecosystem. Personalization as a formative feature of algorithmic reality construction happens in essence on the basis of user characteristics, behavior and location. It furthers individualization in societies, both in the form of dangerous and endangered individuals: dangerous in the sense of fragmentation, fewer unplanned encounters and less shared experience, and decreasing social cohesion; endangered in the sense of more controlled individuals, with less privacy and freedom. Nonetheless, technology not only causes these problems but at the same time offers solutions by the way it is designed. The special constellation of actors in algorithmic reality construction is, in essence, marked by the dominance of private governance by global companies, increasing platformization, and algorithms as actors and policy-makers. The prevalence of private algorithmic governance based on proprietary big data tends to strengthen selection criteria oriented on special interests concerned with profit maximization, thus weakening public interest goals and social responsibility in the construction of reality and eventually consolidating and creating new social inequalities. Algorithms as intermediaries push the platformization of markets and modify power structures, leading the mass media to lose ground in the construction of realities. Moreover, the increasing role of algorithms as relatively autonomous actors with delegated (moral) agency, driven by rising machine intelligence, raises agency and accountability challenges for complex ecosystems that produce less controllable and predictable outcomes compared to reality constructions by the mass media. Altogether, this comparative exploration of governance by algorithms demonstrates how and in which direction (increasing individualization, commercialization, inequalities, deterritorialization, and decreasing transparency, controllability, predictability) algorithmic selection on the Internet tends to shape individuals’ realities and consequently social order. From a public-policy perspective, formative features of algorithmic reality construction highlight several risks. This calls for adequate democratic legitimation of this form of governance by algorithms, for a co-evolutionary mutual interplay with governance of algorithms. Together with the proposed input-throughput-output model and the functional typology of algorithmic selection, the overview of features and risks identified here can form the basis for further investigation in this direction.

21


References Adoni H and Mane S (1984) Media and the Social Construction of Reality. Communication Research 11(3): 323-340. Anderson CW (2013) Towards a Sociology of Computational and Algorithmic Journalism. New Media & Society 15(7): 1005-1021. Auerbach D (2015) The Code We Can’t Control. Available at: http://www.slate.com/articles/technology/bitwise/2015/01/black_box_society_by_fran k_pasquale_a_chilling_vision_of_how_big_data_has.html Bakker P (2012) Aggregation, Content Farms and Huffinization. In: Journalism Practice 6(56): 627-637. Barzilai-Nahon K (2008) Toward a theory of network gatekeeping: A framework for exploring information control. Journal of the American Society for Information Science and Technology 59(9): 1493-1512. Beinhocker ED (2006) The Origin of Wealth. London: Random House. Benkler Y (2011) The Unselfish Gene. In: Harvard Business Review 89(7-8): 76-85. Berger PL and Luckmann T (1967) The Social Construction of Reality. New York: Anchor Books. Boyd D and Crawford K (2012) Critical Questions for Big Data. Information, Communication & Society 15(5): 662-679. Braman S (2002) Posthuman Law: Information Policy and the Machinic World. First Monday 7(12). Available at: http://firstmonday.org/ojs/index.php/fm/article/view/1011/932 Bresnahan T (2010) General Purpose Technologies. In: Hall BH and Rosenberg N (eds) Handbook of the Economics of Innovation. Oxford: Elsevier, 761-791. Chopra S and White LF (2011) A Legal Theory for Autonomous Artificial Agents. Ann Arbor: University of Michigan Press. Christou G and Simpson S (2011) The European Union, multilateralism and the global governance of the Internet. Journal of European Public Policy 18(2): 241-257. Diakopoulos N (2015) Algorithmic Accountability. Digital Journalism 3(3): 398-415. Dolata U and Werle R (2007) Bringing Technology Back In. In: Dolata U and Werle R (eds) Gesellschaft und die Macht der Technik. Frankfurt: Campus, 15-43. Dörr K (2015) Mapping the field of Algorithmic Journalism. Digital Journalism. Epub ahead of print DOI: 10.1080/21670811.2015.1096748.

22


Engel C (2001) A Constitutional Framework for Private Governance. Available at: http://papers.ssrn.com/paper.taf?abstract _id=269310. Floyd C (1992) Software Development as Reality Construction. In: Floyd C, Züllighoven H, Budde R and Keil-Slawik, R (eds) Software Development and Reality Construction. Berlin: Springer, 86-100. Frenken K (2006) Innovation, Evolution and Complexity Theory. Cheltenham: Elgar. Frey CB and Osborne MA (2013) The Future of Employment: How Susceptible Are Jobs to Computerisation? Available at: ttp://www.oxfordmartin.ox.ac.uk/downloads/academic/The_Future_of_Employment.p df. Fullerton L and Ettema J (2014) Ways of worldmaking in Wikipedia: reality, legitmacy and collaborative knowledge making. Media, Culture & Society 36(2): 183-199. Gillespie T (2014) The Relevance of Algorithms. In: Gillespie T, Boczkowski P and Foot K (eds) Media Technologies. Cambridge: MIT Press, 167-194. Hechter M and Horne C (eds) (2003) Theories of Social Order. Stanford: Stanford University Press. Katz E (1996) And Deliver Us from Segmentation. Annals of the American Academy of Political and Social Science 546(1): 22-33. Katzenbach C (2012) Technologies as Institutions. In: Just N and Puppis M (eds) Trends in Communication Policy Research. Bristol: Intellect, 117-137. Kavanaugh AL, Reese DD, Carroll JM and Rosson MB (2005) Weak Ties in Networked Communities. The Information Society 21(2): 119-131. Latour (2005) Reassembling the Social. Oxford: Oxford University Press. Latzer M (2013a) Medienwandel durch Innovation, Ko-Evolution und Komplexität. Medien und Kommunikationswissenschaft 61(2): 235-252. Latzer M (2013b) Towards an Innovation-Co-evolution-Complexity Perspective on Communications Policy. In: Löblich M and Pfaff-Rüdiger S (eds) Communication and Media Policy in the Era of Digitization and the Internet, Baden-Baden: Nomos, 15-27. Latzer M (2013c) Media Convergence. In: Towse R and Handke C (eds) Handbook of the Digital Creative Economy. Cheltenham: Edward Elgar, 123-133. Latzer M, Hollnbucher K, Just N and Saurwein F (2014) The economics of algorithmic selection on the Internet. Available at:

23


http://www.mediachange.ch/media//pdf/publications/Economics_of_algorithmic_selec tion_WP.pdf Latzer M, Just N, Saurwein F and Slominski P (2002) Selbst- und Ko-Regulierung im Mediamatiksektor. Wiesbaden: Westdeutscher Verlag. Lazer et al. (2014) The Parable of Google Flu: Traps in Big Data Analysis. Science 343(6176): 1203-1205. Lessig L (1999) Code and Other Laws of Cyberspace. New York: Basic Books. Machill M, Beiler M and Krüger U (2013) Das neue Gesicht der Öffentlichkeit. Düsseldorf: LfM. Mager A (2012) Algorithmic Ideology: How Capitalist Society Shapes Search Engines. Information, Communication & Society 15(5): 769-787. Mancini P (2013) Media Fragmentation, Party System, and Democracy. International Journal of Press/Politics 18(1): 43-60. Manovich L (2013) Software takes command. New York: Bloomsbury Academic. McQuail D (2010) Mass Communication Theory. 6th edition. London: Sage. Mitcham C (2014) Agency in Humans and Artifacts: A Contested Discourse. In: Kroes P and Verbeek P-P (eds) The Moral Status of Technical Artefacts. Dordrecht: Springer, 1129. Mitchell M (2009) Complexity. Oxford: Oxford University Press. Morozow E (2014) The rise of data and the death of politics. Available at: http://www.theguardian.com/technology/2014/jul/20/rise-of-data-death-of-politicsevgeny-morozov-algorithmic-regulation. Munson SA, Lee SY and Resnick P (2013) Encouraging Reading of Diverse Political Viewpoints with a Browser Widget. Available at: https://dub.washington.edu/djangosite/media/papers/balancer-icwsm-v4.pdf Musiani F (2013) Network architecture as internet governance. Internet Policy Review 2(4): 1-9. Napoli PM (2014) Automated Media: An Institutional Theory Perspective on Algorithmic Media Production and Consumption. Communication Theory 24(3): 340-360. Nelson RR and Winter SG (1982) An Evolutionary Theory of Economic Change. Cambridge: Belknap Press.

24


Nissenbaum H (2011) From Preemption to Circumvention: If Technology Regulates, Why Do We Need Regulation (And Vice Versa)? Berkeley Technology Law Journal 26(3): 1367-1386. Orwat C et al. (2010) Software als Institution und ihre Gestaltbarkeit. Informatik_Spektrum 33(6): 626-633. Pariser, E. (2011) The Filter Bubble. London: Penguin. Pasquale, F (2015) The Black Box Society. Cambridge: Harvard University Press. Pross H (1970) Publizistik: Thesen zu einem Grundcolloquium. Neuwied: Luchterhand. Rammert (2008) Where the action is: Distributed agency between humans, machines, and programs. TUTS-WP-4-2008. Available at: https://www.ts.tuberlin.de/fileadmin/fg226/TUTS/TUTS_WP_4_2008.pdf Rammert W (2003) Technik in Aktion: Verteiltes Handeln in soziotechnischen Konstellationen. TUTS-WP-2-2003. Available at: https://www.ts.tuberlin.de/fileadmin/fg226/TUTS/TUTS_WP_2_2003.pdf Reidenberg JR (1998) Lex Informatica: The Formulation of Information Policy Rules through Technology. Texas Law Review 76(3): 553-593. Rochet JC and Tirole J (2003) Platform Competition in Two-sided markets. Journal of the European Economic Association 1(4): 990-1029. Salvin K (2011) How Algorithms Shape Our World. TED Talks. Available at: http://www.ted.com/talks/kevin_slavin_how_algorithms_shape_our_world.html Saurwein F, Just N and Latzer M (2015) Governance of Algorithms: Options and Limitations. Info – The journal of policy, regulation and strategy for telecommunications, information and media, 17 (6), 35-49. Schneider V and Kenis P (1996) Verteilte Kontrolle: Institutionelle Steuerung in modernen Gesellschaften. In: Schneider V and Kenis P (eds) Organisation und Netzwerk. Institutionelle Steuerung in Wirtschaft und Politik. Frankfurt: Campus, 9-43. Schroer M (2008) Individualisierung. In: Baur N, Korte H, Löw M and Schroer M (eds) Handbuch Soziologie. Wiesbaden: VS-Verlag, 139-161. Scott WR (1987) The Adolescence of Institutional Theory. Administrative Science Quarterly 32(4): 493-511. Shah RC and Kesan JP (2003) Manipulating the governance characteristics of code. info 5(4): 3-9.

25


Shah RC and Kesan JP (2011) Software as Governance. Available at: http://papers.ssrn.com/abstract=1799431. Shelanski HA (2013) Information, Innovation, and Competition Policy for the Internet. University of Pennsylvania Law Review 161: 1663-1705. Singer JB (2014) User-generated visibility: Secondary gatekeeping in a shared media space. New Media & Society 16(1): 55-73. Steiner C (2012) Automate This. New York: Penguin. Sunstein CR (2007) Republic.com 2.0. Princeton: Princeton University Press. Tomlinson J (1999) Globalization and Culture. Cambridge: Polity Press. Van Dalen A (2012) The Algorithms Behind the Headlines. Journalism Practice 6(5-6): 648658. Van Dijck J (2013) The Culture of Connectivity. Oxford: Oxford University Press. Van Dijck J and Poell T (2013) Understanding social media logic. Media and Communication 1(1): 2-14. Verbeek (2014) Some Misunderstanding About the Moral Significance of Technology. In: Kroes P and Verbeek P-P (eds) The Moral Status of Technical Artefacts. Dordrecht: Springer, 75-88. Wallace J (2015) Towards a Theory of Online Gatekeeping. IPMZ-Working Paper, Zurich. WGIG – Working Group on Internet Governance (2005) Report of the Working Group on Internet Governance. Available at: http://www.wgig.org/docs/WGIGREPORT.pdf. Whitt R and Schultze S (2009) The New ‘Emergence Economics’ of Innovation and Growth, And What It Means for Communications Policy. Journal on Telecommunications and High Technology Law 7(2): 217-316. Winner L (1986) The Whale and the Reactor. Chicago: University of Chicago Press. Ziman J (2000) Evolutionary models for technological change. In: Ziman J (ed.) Technological innovation as an evolutionary process. Cambridge: Cambridge University Press, 3-11.

26

Governance by Algorithms: Reality Construction by Algorithmic ...

Governance by Algorithms: Reality Construction by Algorithmic ...

Suggest Documents

Governance by Algorithms: Reality Construction by Algorithmic ...

Governance by Algorithms: Reality Construction by Algorithmic ...

Developing Algorithmic Thinking by Inventing and Playing Algorithms

Developing Algorithmic Thinking by Inventing and Playing Algorithms

GOVERNANCE BY INTERNATIONAL ORGANIZATIONS ...

Understanding Algorithms by Design Them by New

Algorithmic Themes by Donald E. Knuth

Algorithmic Governance in Smart Cities: the

TOWARD CONSTRUCTION JIT by - Lean Construction Institute

Experiments in Algorithmic Governance - Quinn DuPont

Algorithmic governance: Developing a research ... - SAGE Journals

Quantum Hidden Subgroup Algorithms: An Algorithmic Toolkit

Algorithms and their others: Algorithmic culture in

Algorithmic construction of the subdifferential from

Algorithmic Construction of Optimal and Load

The Narrative Construction of Reality

Estimating Building Construction Costs by

SENTENCE CONSTRUCTION BY A ... - MPG.PuRe

Knowledge Ascription by Grammatical Construction

Abductive Coreference by Model Construction

Algorithmic Aspects of Golomb Ruler Construction

FRACTAL CONSTRUCTION BY ORTHOGONAL ... - CiteSeerX

mathematical learning processes supported by augmented reality

BRAND REALITY: THE JAPANESE PERSPECTIVE by ... - CiteSeerX