"I would argue that, however intelligent machines may be made to be, there are some acts of thought that ought to be attempted only by humans." So Weizenbaum himself summarizes his purpose in writing Computer Power and Human Reason -- to re-introduce the notion of moral obligation into the field of computer science, whose power in the world has increasingly moral implications. The fundamental assumption Weizenbaum is combating is the assumption that "man really is merely a species of the genus 'information-processing system'" and therefore that it would be possible for a sufficiently advanced computer to fill any role a human can play.
To oppose this view, he first describes the inner workings of computers, and where their power comes from -- not just computing power, but power to shape how we view the world. His purpose is to unlock the mystery behind computers that leaves them, for most of the world even today, shrouded in a semi-mystical aura. It's this lack of understanding that is the foundation for what the author sees as a flawed view of computers, and so he educates. More than this, he analyzes the way computers have made themselves indispensable, by first enhancing the processes we currently use, but by then changing the way we think about those processes so that our thoughts conform to a computer-centric worldview.
Having pulled back the curtain of the computer, Weizenbaum then delves into the world of artificial intelligence, exploring its lofty aspirations and exposing the significant but small progress that has given rise to such "hubris." He corrects many popular misconceptions about artificial intelligence, its power and its progress. For example, to counter people who, upon interacting with an electronic conversational program such as AIM's SmarterChild, he argues that no matter how well an artificial intelligence can mimic human "input and output," there is no reason to believe that what is going on inside the two is anything alike, and therefore whether the computer could be considered "human."
After this, Weizenbaum delivers the crux of his argument as to the limitations of computer intelligence. The main premise of AI idealism is that a computer can be made to learn anything that a human being can learn. Learning comes in large degree through "socialization," through being modified by one's experiences with the world. But because a human being experiences the world as a human being -- facing unique human problems and being treated by others in a unique way as a human being -- he or she gains knowledge that cannot be acquired any other way than through the human experience. Therefore, the most intelligent computer still experiences the world as a computer, and cannot comprehend fundamental human concepts as security, affection, fear, and love. That's the limit of a computer's intelligence: it cannot learn what can only be learned by being human.
He then applies this conclusion to the field of computer science and how it should affect the way we exercise our discipline. We should seek, as people who are more than simply information processors, to bring ethics back into the world of technology, which has influenced us to think of problems in terms of simply finding the right algorithm to accomplish the task. As we bring this moral focus, Weizenbaum encourages two specific areas we should avoid. The first is anything that, in his words, things "whose very contemplation ought to give rise to feelings of disgust in every civilized person." He refers especially to substituting a machine for any role that involves a human being acting as a human and not a robot -- interacting with "interpersonal respect, understanding, and love." Secondly, he cautions us when pursuing projects that could have large-scale social implications, such as the development of voice-recognition software, which could facilitate large-scale processing of wiretapped phone conversations.
Weizenbaum's bias is worn on his sleeve, especially in the first and last chapters. He makes it clear that he is attempting to convince the readers to change the way they look at computers philosophically and ethically. His standpoint is self-admittedly unique in that few other computer professionals even think of their role in ethical terms. He discusses ideals such as truth and justice, offering no metaphysical framework for them but simply arguing that their absence in technological and scientific circles is simply ridiculous. It is therefore unclear what exactly the author believes, but this is no great loss. His ethical views, on issues such as the dignity of human life and the importance of social responsibility, are all strongly in line with Christian values -- and his ideas go a long way toward starting up conversation that could lead to Christian influence and witness in the computer world.
Several questions still remain, however -- and although they are largely Hollywood-inspired, they should not be totally discounted. Weizenbaum wrote in 1975, before the advent of robotics, when the closest one could get to a computer was by typing back and forth at a terminal. Since then, rapid advancements in technology are suggesting ideas such as that of the movie AI, where a robotic child was brought into a family and actually socialized to the point where he became almost human. If a machine could be made to look and act more and more like a human, what is to prevent it from being socialized like a human and therefore coming to a deep understanding of what it means to be human? Perhaps the distinction between man and machine cannot be reached by purely logical reasoning and scientific evidence, as Weizenbaum has attempted.