21 Lessons for the 21st Century(24)

In one tragicomic incident in October 2017, a Palestinian labourer posted to his private Facebook account a picture of himself in his workplace, alongside a bulldozer. Adjacent to the image he wrote ‘Good morning!’ An automatic algorithm made a small error when transliterating the Arabic letters. Instead of ‘Ysabechhum!’ (which means ‘Good morning!’), the algorithm identified the letters as ‘Ydbachhum!’ (which means ‘Kill them!’). Suspecting that the man might be a terrorist intending to use a bulldozer to run people over, Israeli security forces swiftly arrested him. He was released after they realised that the algorithm made a mistake. But the offending Facebook post was nevertheless taken down. You can never be too careful.29 What Palestinians are experiencing today in the West Bank might be just a primitive preview to what billions will eventually experience all over the planet.

In the late twentieth century democracies usually outperformed dictatorships because democracies were better at data-processing. Democracy diffuses the power to process information and make decisions among many people and institutions, whereas dictatorship concentrates information and power in one place. Given twentieth-century technology, it was inefficient to concentrate too much information and power in one place. Nobody had the ability to process all the information fast enough and make the right decisions. This is part of the reason why the Soviet Union made far worse decisions than the United States, and why the Soviet economy lagged far behind the American economy.

However, soon AI might swing the pendulum in the opposite direction. AI makes it possible to process enormous amounts of information centrally. Indeed, AI might make centralised systems far more efficient than diffused systems, because machine learning works better the more information it can analyse. If you concentrate all the information relating to a billion people in one database, disregarding all privacy concerns, you can train much better algorithms than if you respect individual privacy and have in your database only partial information on a million people. For example, if an authoritarian government orders all its citizens to have their DNA scanned and to share all their medical data with some central authority, it would gain an immense advantage in genetics and medical research over societies in which medical data is strictly private. The main handicap of authoritarian regimes in the twentieth century – the attempt to concentrate all information in one place – might become their decisive advantage in the twenty-first century.

As algorithms come to know us so well, authoritarian governments could gain absolute control over their citizens, even more so than in Nazi Germany, and resistance to such regimes might be utterly impossible. Not only will the regime know exactly how you feel – it could make you feel whatever it wants. The dictator might not be able to provide citizens with healthcare or equality, but he could make them love him and hate his opponents. Democracy in its present form cannot survive the merger of biotech and infotech. Either democracy will successfully reinvent itself in a radically new form, or humans will come to live in ‘digital dictatorships’.

This will not be a return to the days of Hitler and Stalin. Digital dictatorships will be as different from Nazi Germany as Nazi Germany was different from ancien régime France. Louis XIV was a centralising autocrat, but he did not have the technology to build a modern totalitarian state. He suffered no opposition to his rule, yet in the absence of radios, telephones and trains, he had little control over the day-to-day lives of peasants in remote Breton villages, or even of townspeople in the heart of Paris. He had neither the will nor the ability to establish a mass party, a countrywide youth movement, or a national education system.30 It was the new technologies of the twentieth century that gave Hitler both the motivation and the power to do such things. We cannot predict what will be the motivations and powers of digital dictatorships in 2084, but it is very unlikely that they will just copy Hitler and Stalin. Those gearing themselves up to refight the battles of the 1930s might be caught off their guard by an attack from a totally different direction.

Even if democracy manages to adapt and survive, people might become the victims of new kinds of oppression and discrimination. Already today more and more banks, corporations and institutions are using algorithms to analyse data and make decisions about us. When you apply to your bank for a loan, it is likely that your application is processed by an algorithm rather than by a human. The algorithm analyses lots of data about you and statistics about millions of other people, and decides whether you are reliable enough to give you a loan. Often, the algorithm does a better job than a human banker. But the problem is that if the algorithm discriminates against some people unjustly, it is difficult to know that. If the bank refuses to give you a loan, and you ask ‘Why?’, the bank replies ‘The algorithm said no.’ You ask ‘Why did the algorithm say no? What’s wrong with me?’, and the bank replies ‘We don’t know. No human understands this algorithm, because it is based on advanced machine learning. But we trust our algorithm, so we won’t give you a loan.’31

When discrimination is directed against entire groups, such as women or black people, these groups can organise and protest against their collective discrimination. But now an algorithm might discriminate against you personally, and you have no idea why. Maybe the algorithm found something in your DNA, your personal history or your Facebook account that it does not like. The algorithm discriminates against you not because you are a woman, or an African American – but because you are you. There is something specific about you that the algorithm does not like. You don’t know what it is, and even if you knew, you cannot organise with other people to protest, because there are no other people suffering the exact same prejudice. It is just you. Instead of just collective discrimination, in the twenty-first century we might face a growing problem of individual discrimination.32