Use the Scientific Method in Computer Science - ACM - Computers in ...

0 downloads 349 Views 470KB Size Report
Feb 2, 2017 - physics, the foundations of com- puter science are logically verifiable; hypotheses play no part. So, whil
letters to the editor DOI:10.1145/3032965

Use the Scientific Method in Computer Science

I

The Theory of Evolution Under the Lens of Computation” (Nov. 2016), Adi Livnat and Christos Papadimitriou argued eloquently that the extraordinary success of sexual evolution has not been adequately explained. Somewhat paradoxically, they concluded that sex is not particularly well suited to the task of generating “outstanding individuals.” They also said that genetic algorithms are similarly ill suited to this task. It should be noted that this critique of genetic algorithms—widely used derivative free optimization heuristics modeled on recombinative evolution—stands in counterpoint to a voluminous empirical record of practical successes. It also speaks to the long-standing absence of consensus among evolutionary computation theorists regarding the abstract workings of genetic algorithms and the general conditions under which genetic algorithms outperform local search. A consensus on these matters promises to shed light on the question the authors originally aimed to answer: Why does recombinative evolution generate populations with outstanding individuals? Generative hypomixability elimination1 is a recent theory that addresses this question, positing that genetic algorithms efficiently implement a decimation heuristic that generates fitter populations over time by iteratively eliminating the joint entropy of small collections of “hypomixable loci,” or loci in which alleles do not mix well. Recombination, or mixing, allows such loci to go to fixation even as it safeguards the marginal entropy of non-interacting loci. Taking a step back, one might ask how this theory and the theory proposed by Livnat and Papadimitriou might be evaluated. Proof of soundness, wherever possible, is always desirable, but end-to-end proof can be elusive when analyzing computation in biological systems like brains and evolving populations. We must N “S E X A S A N ALG ORITH M :

8

COMMUNICATIO NS O F THE ACM

instead use the scientific method,2 an approach undergirded by the following rule: hypothesis ==> prediction ≡ ¬prediction ==> ¬hypothesis Unlike the foundations of, say, physics, the foundations of computer science are logically verifiable; hypotheses play no part. So, while computer scientists have seen engineering revolutions aplenty, they have seen nothing like the transition from a Newtonian universe to an Einsteinian universe or from the phlogiston theory of combustion to Lavoisier’s oxygen-based theory or any of the other foundational shifts described in Thomas Kuhn’s Structure of Scientific Revolutions. Theoretical physicists, chemists, and biologists trained informally, if not formally, in the application of the scientific method know how to evaluate and work with competing hypotheses. The same cannot be said of theoretical computer scientists today. For them, the scientific method is unfamiliar terrain, with different rules and alternate notions of rigor. For example, assumptions must be weak, and hypotheses testable. For all computer science as a field has to contribute to the natural sciences, it also has much to learn. References 1. Burjorjee, K.M. Hypomixability elimination in evolutionary systems. In Proceedings of the 13th Foundations of Genetic Algorithms Conference (Aberystwyth, U.K., Jan. 17–20). ACM Press, New York, 2015, 163–175. 2. Popper, K. The Logic of Scientific Discovery. Routledge, London, U.K., 2007.

Keki M. Burjorjee, Berkeley, CA

While Adi Livnat and Christos Papadimitriou’s article (Nov. 2016) provided the rationale for a provocative magazine cover, the article itself began with a false claim and ignored a much simpler explanation for the success of sexual evolution. Shortly after life appeared on Earth, approximately 3.8 billion years ago, evolution began diversifying lifeforms in a very pragmatic way, with mutations that increased

| F EBR UA RY 201 7 | VO L . 60 | NO. 2

the ability of individuals to survive and reproduce being passed along to future generations, whereas those that were disadvantageous were naturally dropped. This process soon discovered that sexual reproduction worked better than simply subdividing, in that it allows advantageous mutations that occur in different families to be combined, allowing evolution to proceed more rapidly, whereas subdividing does not allow it. Sexual reproduction thus became dominant. Nevertheless, the article said, “What is the role of sex in evolution? Reproduction with recombination is almost ubiquitous in life (even bacteria exchange genetic material), while obligate asexual species appear to be rare evolutionary dead ends. Yet there is no agreement among the experts as to what makes sex so advantageous.” How can there be no agreement when the reason for sexual evolution is so obvious? In order for sexual evolution to work, each generation must die, which some people view as inconvenient, prompting them to imagine an afterlife. Subdividing, on the other hand, produces potential immortals who are naturally less diverse because they mutate less radically than the sexy species. P.S. I do not hold any of this against Christos Papadimitriou, who I have known for 50 years.

Lester Earnest, Stanford, CA

Authors Respond: Earnest’s idea, first proposed by R.A. Fisher (1930) and H.J. Muller (1932), does not solve the problem and is referenced in our online appendix where the interested reader can begin to explore this fascinating topic. The debate among experts is ongoing, and our recent article contributed a fresh idea to it. Burjorjee did not back up with evidence his claim of empirical success of genetic algorithms, compared to, say, simulated annealing. And a propos philosophy of science, he may refer to Papadimitriou’s 1995 article “Database Metatheory: Asking the Big Queries” (http://dl.acm.org/citation. cfm?id=211547) with its sections on T. Kuhn,

letters to the editor K.R. Popper, and P. Feyerabend, and their relevance to computer science. Adi Livnat, Haifa, Israel, and Christos Papadimitriou, Berkeley, CA

Dismayed by ‘Sex’ Cover I am writing to express my dismay and disappointment at the cover of the November 2016 issue introducing the article “Sex as an Algorithm: The Theory of Evolution Under the Lens of Computation” by Adi Livnat and Christos Papadimitriou, finding it offensive and attention-grabbing in a way that is inconsistent with ACM’s public mission. While I would guess that most readers either do not care or thought the cover “funny” or “cute,” I have talked to enough of my colleagues, who describe their reaction as “shocked,” “appalled,” “offended,” and “embarrassed,” to believe it is a serious issue that warrants further reflection. Specifically, is it really appropriate for ACM, a professional organization that purports to represent and support all its members and all members of the computing discipline, to distribute an issue that some are embarrassed to receive in our mailbox, display on our desks or conference tables, or look at on our computers if somebody might be looking over our shoulders? First, the research in question is not about sex but about sexual reproduction and its effect on diversity in populations. There is a major difference, and conflating the two in this way comes across as juvenile. I cannot help think of “locker room talk.” Second, placing the huge, boldfaced word “Sex” on a hot pink cover creates an obvious and immediate association with women. Given the underrepresentation of women in the field, this kind of message is completely counterproductive and particularly reminds young women, who may be less certain about how welcome they are in the field, that they are to be associated with sex, not science. Third, the unfortunate timing of this issue, which arrived during National Breast Cancer Awareness Month, was undoubtedly unintentional, but to those of us who have lost loved ones to breast cancer, the hot pink cover felt disrespectful and insensitive.

This may not seem like a big deal, and I am sure some readers are thinking I am overly sensitive and humorless. But quite honestly, it is tough enough being a woman in an extremely male-dominated field without feeling embarrassed and awkward about displaying my own professional organization’s magazine in public. In the end, I dropped it into the recycling bin without reading it. Marie desJardins, Baltimore, MD

Editor-in-Chief Responds: The cover in question, for which I am ultimately responsible, was meant to be humorous. Since several readers were offended by it, it is clear in retrospect the humor was misguided. For that, I sincerely apologize. This has been discussed by the design team, and we hope to learn from this mistake. Moshe Y. Vardi, Editor-in-Chief

No Revolution Yet for Blockchain Sarah Underwood’s news article “Blockchain Beyond Bitcoin” (Nov. 2016) was yet another disappointing read on blockchain, offering an (incomplete) summary of publicly available information on the technology and its proposed application areas. Many claims, including the key one that “Blockchain technology has the potential to revolutionize applications and redefine the digital economy,” were neither discussed nor backed up with evidence. From a scientific point of view, this is insufficient. Worse, like many blockchain proponents, Underwood failed, in my opinion, to raise the right questions. Instead of focusing on “what blockchain could do,” one should address “what blockchain can do better than other technologies.” In this context, blockchain is often compared to existing solutions rather than to existing technologies, as in the proverbial comparison of apples and oranges. There may be any number of reasons, including operational, economic, or social, why an existing solution (as inadequate as it may be) has not been replaced in the marketplace. However, this does not mean per se there is no existing, better-understood technology than blockchain available to address a given problem.

Moreover, blockchain is often credited with the ability to solve tough longstanding problems. For example, Underwood mentioned “digital identity.” Various attempts to address this challenge, including well-established approaches (such as Public Key Infrastructure and Web of Trust) fail in various ways due to nontechnical aspects of human relationships, including trust, social, cognitive, economic, and even physical. So far, moreover, no evidence has been produced that shows how blockchain outperforms existing technologies in addressing the problem of digital identity. It is time to ask the right questions about blockchain if we want to understand its actual properties, strengths, and weaknesses, as well as its promise. Ingo Mueller, Melbourne, Australia

Keep Human Judgment in Remote Warfare Though Keith Kirkpatrick’s news article “Can We Trust Autonomous Weapons?” (Dec. 2016) was thoughtful and well balanced, we must still ask how we should be identifying targets. From the prisoners in the U.S. military prison at Guantánamo Bay, Cuba, to the fighters targeted by autonomous vehicles in Pakistan or Yemen, we depend on human intelligence on the ground to choose the ones to target, even though that intelligence is sometimes faulty or false. In practical terms, we have shown we cannot get our boots off the ground. We need to embed our forces in and work with the populations we wish to protect. Remote warfare—unless separated completely from ethics, responsibility, and long-term consequences—is likely to remain a fantasy. It thus raises the perennial question of where to draw the line between computing intelligence and human reason, as explored by MIT professor Joseph Weizenbaum in his classic 1976 book Computer Power and Human Reason: From Judgment to Calculation on automation and human decision making. Where computerized warfare is concerned, human judgment remains the supreme arbiter. Andy Oram, Boston, MA Communications welcomes your opinion. To submit a Letter to the Editor, please limit yourself to 500 words or less, and send to [email protected]. ©2017 ACM 0001-0782/17/02

F E B R UA RY 2 0 1 7 | VO L. 6 0 | N O. 2 | C OM M U N IC AT ION S OF THE ACM

9