HUMAN COMPATIBLE

A leading computer scientist urges a “radically new direction” for artificial intelligence research to ensure that superhuman AI benefits humankind.

“The march towards superhuman intelligence is unstoppable, but success might be the undoing of the human race,” warns Russell (Computer Science and Engineering/Univ. of California), co-author of Artificial Intelligence (1995), a widely used textbook. In this authoritative trade debut, he examines the idea of human and machine intelligence, the problems involved in creating extremely intelligent machines, and the need to retain human control of them. Although more scientific breakthroughs are needed, he writes, AI will certainly be the dominant technology of the future. We must have a serious discussion of “the implications of introducing a second intelligent species onto Earth.” After all, “making something smarter than yourself could be a bad idea.” Examining the enormous potential of AI, from its evolving benefits (home robots for the elderly, tutoring for children) to its misuses (automated blackmail, autonomous weapons), Russell writes that AI researchers must refocus their work if humans are to remain in charge. Rather than developing machines that optimize fixed, known objectives—a driving idea of 20th-century technology—they must design systems that defer to human preferences and intentions. As the author writes, “they will ask humans questions or ask for permission when appropriate; they will do ‘trial runs’ to see if we like what they propose to do; they will accept correction when they do something wrong.” An accessible writer, Russell is addressing nonspecialist readers, and he largely succeeds, although some will find his detailed explications challenging. (Refreshingly, he appends the most technical text.) The author is strongly optimistic that increasingly powerful machines that achieve our objectives are feasible. There is still time to change course: If a team of AI’s leading experts, with unlimited resources, was charged today with creating “an integrated, human-level intelligent system by combining all our best ideas, the result would be failure.”

A strong case for planning for the day when machines can outsmart us.