An Orwellian Nightmare Come True:
China’s Social Credit System
By Cosima Lumley
“If you want a picture of the future, imagine a boot stamping on a human face—for ever.” George Orwell, 1984
In 1949 George Orwell published his dystopic novel 1984 picturing a world of mass surveillance where Big Brother rules supreme and ‘nothing was your own except the few cubic centimetres inside your skull’. The novel contains eerie similarities to features of 21st century society. Orwell’s intrusive telescreens which provide the fictional ruling Party with a live feed of its citizens actions resemble today’s CCTV cameras monitoring our every action. Winston’s concern with personal privacy and freedom of thought evokes our modern age of cyber security and online content.
Yet while Orwell’s dystopic novel sees citizens robbed of their privacy by a totalitarian state, our generation have voluntarily turned the lens upon themselves, offering up individual privacy on social media sites, and commodifying their lives as a visual spectacle for universal consumption. The telescreen of the modern day is a laptop through which we willingly share our lives. Our audience is not ‘The Party’ but society as a whole.
This trend towards offering up our privacy for validation takes a more sinister turn when one looks at those who control our data. In Europe, concern over the use of data by private companies and technological giants such as FANG (Facebook, Amazon, Netflix, Google) has led to the General Data Protection Regulation (GDPR), which gives individuals ownership and control over their own data, as companies now have to seek individuals’ consent to store their users’ data and individuals have a right to ask that their data be deleted. You may have noticed a flurry of GDPR emails before the Regulation’s implementation on 25th May 2018, as companies desperately sought customers’ consent before their much-valued mailing lists were depleted.
Even with such steps being taken, Facebook’s Cambridge Analytica scandal demonstrate not only the ease with which people give their data but the uses it can be put to. In this case data was harvested through a seemingly harmless personality quiz on Facebook and used to manipulate the Brexit and US electoral vote.
While European governments are striving to help individuals control the use and dissemination of their data through GDPR, China is seizing that control for itself and using it as a tool for mass surveillance under the guise of the ‘Social Credit System’. Set to become compulsory in 2020, the Social Credit System is already in operation. It is a system which uses social and economic data, collected by the government in partnership with private corporations such as Alibaba, to generate a ‘trustworthiness’ score for every citizen. While in the UK, credit scores are used purely to judge whether you should be granted a loan or not, in China these scores’ consequences will be, and already are, far-reaching.
While high scorers are rewarded with perks such as discounted energy bills, rents without deposit and skipping health care waiting lists, low scorers risk getting travel bans, their kids not being allowed into the best schools and public naming and shaming. The system is part of a government objective to ‘build a harmonious Socialist society’ and stimulate ‘the progress of civilisation’. It will, as the Chinese government states, ‘allow the trustworthy to roam freely under the heaven while making it hard for the discredited to take a single step’.
While the workings of the algorithms which process the data is strictly confidential, examples of the type of data being used to determine individual scores includetimely payment of bills, loans and parking tickets and compliance with court orders. More worryingly they also include social data such as shopping habits, dating profiles, even the respective scores of friends on social media. A report by Australia’s Foreign Correspondent further showed that the scheme would harness the footage of 200 million CCTV cameras over China equipped with facial recognition software to further monitor citizens and include the information within the score.
It seems that China has taken the telescreens of Orwell’s dystopia, the all-seeing eye of the Party, and shown how they can be used not only to monitor but also to define each citizen’s worth in a numerical value: to create a hierarchy based on conformity. As the algorithm cannot or will not explain its workings in coming to the score, in passing its ‘numerical judgement’ on each individual, it cannot be challenged or broken down. Its biases cannot be publicly exposed or deconstructed. In essence, by attempting to reduce people to numbers, to define them in terms of the data they generate, the Social Credit System achieves Orwell’s vision of a ‘boot stamping on a human face’.
GDPR seminars are held all over London regularly; Privacy Calendar hosts purely privacy events from all over the globe.
The Art of Invisibility on Audible gives enlightened, reasoned advice on how to protect yourself from ‘Big Brother’ from the World’s most famous hacker Kevin Mitnick.
The techno-sociologist giant Zeynep Tufekci gives a fascinating TED talk on tech dystopia and the political and social algorithms used in the digital age.
