The tyranny of algorithms is part of our lives: soon they could rate everything we do
For the past couple of years a big story about the future of China has been the focus of both fascination and horror. It is all about what the authorities in Beijing call “social credit”, and the kind of surveillance that is now within governments’ grasp. The official rhetoric is poetic. According to the documents, what is being developed will “allow the trustworthy to roam everywhere under heaven while making it hard for the discredited to take a single step”.
As China moves into the newly solidified President Xi Jinping era, the basic plan is intended to be in place by 2020. Some of it will apply to businesses and officials, so as to address corruption and tackle such high-profile issues as poor food hygiene. But other elements will be focused on ordinary individuals, so that transgressions such as dodging transport fares and not caring sufficiently for your parents will mean penalties, while living the life of a good citizen will bring benefits and opportunities.
Online behaviour will inevitably be a big part of what is monitored, and algorithms will be key to everything, though there remain doubts about whether something so ambitious will ever come to full fruition. One of the scheme’s basic aims is to use a vast amount of data to create individual ratings, which will decide people’s access – or lack of it – to everything from travel to jobs.
The Chinese notion of credit – or xinyong – has a cultural meaning that relates to moral ideas of honesty and trust. There are up to 30 local social credit pilots run by local authorities, in huge cities such as Shanghai and Hangzhou and much smaller towns. Meanwhile, eight ostensibly private companies have been trialling a different set of rating systems, which seem to chime with the government’s controlling objectives.
The most high-profile system is Sesame Credit – created by Ant Financial, an offshoot of the Chinese online retail giant Alibaba. Superficially, it reflects the western definition of credit, and looks like a version of the credit scores used all over the world, invented to belatedly allow Chinese consumers the pleasures of buying things on tick, and manage the transition to an economy in which huge numbers of people pay via smartphones. But its reach runs wider.