21 Lessons for the 21st Century(23)
On 16 March 1968 a company of American soldiers went berserk in the South Vietnamese village of My Lai, and massacred about 400 civilians. This war crime resulted from the local initiative of men who had been involved in jungle guerrilla warfare for several months. It did not serve any strategic purpose, and contravened both the legal code and the military policy of the USA. It was the fault of human emotions.24 If the USA had deployed killer robots in Vietnam, the massacre of My Lai would never have occurred.
Nevertheless, before we rush to develop and deploy killer robots, we need to remind ourselves that the robots always reflect and amplify the qualities of their code. If the code is restrained and benign – the robots will probably be a huge improvement over the average human soldier. Yet if the code is ruthless and cruel – the results will be catastrophic. The real problem with robots is not their own artificial intelligence, but rather the natural stupidity and cruelty of their human masters.
In July 1995 Bosnian Serb troops massacred more than 8,000 Muslim Bosniaks around the town of Srebrenica. Unlike the haphazard My Lai massacre, the Srebrenica killings were a protracted and well-organised operation that reflected Bosnian Serb policy to ‘ethnically cleanse’ Bosnia of Muslims.25 If the Bosnian Serbs had had killer robots in 1995, it would likely have made the atrocity worse rather than better. Not one robot would have had a moment’s hesitation carrying out whatever orders it received, and would not have spared the life of a single Muslim child out of feelings of compassion, disgust, or mere lethargy.
A ruthless dictator armed with such killer robots will never have to fear that his soldiers will turn against him, no matter how heartless and crazy his orders. A robot army would probably have strangled the French Revolution in its cradle in 1789, and if in 2011 Hosni Mubarak had had a contingent of killer robots he could have unleashed them on the populace without fear of defection. Similarly, an imperialist government relying on a robot army could wage unpopular wars without any concern that its robots might lose their motivation, or that their families might stage protests. If the USA had had killer robots in the Vietnam War, the My Lai massacre might have been prevented, but the war itself could have dragged on for many more years, because the American government would have had fewer worries about demoralised soldiers, massive anti-war demonstrations, or a movement of ‘veteran robots against the war’ (some American citizens might still have objected to the war, but without the fear of being drafted themselves, the memory of personally committing atrocities, or the painful loss of a dear relative, the protesters would probably have been both less numerous and less committed).26
These kinds of problems are far less relevant to autonomous civilian vehicles, because no car manufacturer will maliciously program its vehicles to target and kill people. Yet autonomous weapon systems are a catastrophe waiting to happen, because too many governments tend to be ethically corrupt, if not downright evil.
The danger is not restricted to killing machines. Surveillance systems could be equally risky. In the hands of a benign government, powerful surveillance algorithms can be the best thing that ever happened to humankind. Yet the same Big Data algorithms might also empower a future Big Brother, so that we might end up with an Orwellian surveillance regime in which all individuals are monitored all the time.27
Indeed, we might end up with something that even Orwell could barely imagine: a total surveillance regime that follows not just all our external activities and utterances, but can even go under our skin to observe our inner experiences. Consider for example what the Kim regime in North Korea might do with the new technology. In the future, each North Korean citizen might be required to wear a biometric bracelet that monitors everything you do and say – as well as your blood pressure and brain activity. By using our growing understanding of the human brain, and using the immense powers of machine learning, the North Korean regime might be able for the first time in history to gauge what each and every citizen is thinking each and every moment. If you look at a picture of Kim Jong-un and the biometric sensors pick up the telltale signs of anger (higher blood pressure, increased activity in the amygdala) – you’ll be in the Gulag tomorrow morning.
Granted, due to its isolation the North Korean regime might have difficulty developing the required technology by itself. However, the technology might be pioneered in more tech-savvy nations, and copied or bought by the North Koreans and other backward dictatorships. Both China and Russia are constantly improving their surveillance tools, as are a number of democratic countries, ranging from the USA to my home country of Israel. Nicknamed ‘the start-up nation’, Israel has an extremely vibrant hi-tech sector, and a cutting-edge cyber-security industry. At the same time it is also locked into a deadly conflict with the Palestinians, and at least some of its leaders, generals and citizens might well be happy to create a total surveillance regime in the West Bank as soon as they have the necessary technology.
Already today whenever Palestinians make a phone call, post something on Facebook or travel from one city to another they are likely to be monitored by Israeli microphones, cameras, drones or spy software. The gathered data is then analysed with the aid of Big Data algorithms. This helps the Israeli security forces to pinpoint and neutralise potential threats without having to place too many boots on the ground. The Palestinians may administer some towns and villages in the West Bank, but the Israelis control the sky, the airwaves and cyberspace. It therefore takes surprisingly few Israeli soldiers to effectively control about 2.5 million Palestinians in the West Bank.28