Are robots bigots?

The more you talk, the smarter Tay gets.  Or not.

Remember Tay? The Twitbot that took less than an Elon Musk working day to morph from a sweet Millennial teenager to a racist and sexist bigot? And remember when it was taken offline for a week, only to return minus the racism but plus what looked like a very bad drug trip, culminating in tweeting “You are too fast, please take a rest” several times a second?

Oh, the problems we had back in March 2016!

But while few bots have turned ugly quite as spectacularly as Tay (or as political life since 2016), there’s growing concern that AI not only isn’t fair, but goes beyond ordinary human bias to discriminate in multiple and serious ways.

Are robots bigots?

There’s definitely little sign so far of AI’s moral superiority. Three years on from Gorillagate, facial recognition algorithms still make false identifications more often with darker-skinned faces. Language vector databases – multidimensional patterns of how words connect to each other, used as training data for most talky machine learning applications – have shown deep sexism (e.g. “man is to computer programmer as woman is to homemaker”). Amazon’s in-house AI recruiting tool developed multiple ways to mark down female candidates. Facebook’s algorithms have promoted bigotry in multiple forms. Think you can trust AI when it’s used in court? Reoffending-prediction systems used by judges across the US have displayed systematic racial bias, with black defendants 45% more likely to be tagged as likely reoffenders than white defendants, even after controlling for such highly-predictive variables such as prior crimes, age and gender.

Bad, bad bots!

But, hang on a minute, is this bigotry really the robots’ fault? Or are the problems due to those pesky humans behind the sloppy programming, inaccurate labelling of data sets and clickbait business strategies?

As Kate Bush sang in the context of gender absolutism, the truth lies somewhere in the middle.

Some AI is thunkingly biased (see the links above for a start) and much appears to have been under-checked for discrimination – I write appears to because most systems are proprietary and their originators have been rather coy about letting researchers poke around inside their clever tech. Solutions are starting to emerge to eliminate or at least reduce bias – including data points and analytical approaches that can serve as proxies for bias – such as postcodes for racial discrimination, clichéd-accoutrements such as pinafores or axes for gender discrimination. Some companies have gone back to the human, paying armies of Mechanical Turks to bias-check their databases and outputs. New York City has enacted a new law to block algorithmic discrimination in government agencies. The University of California has developed – you guessed it! – an AI training model to clean up databases which claims to reduce bias by up to 37%.

37% less biased. Well, it’s a start.

But there’s a bigger problem.

If machine learning systems are rewarded for the degree of certainty with which they identify group-members (picking out a senior computer programmer from a sea of random jobseekers, say), they will default to the most common differentiators. Sad but true: women make up just 18% of today’s computer science graduates (down from 37% in 1984, if you can believe it). A factor – or proxy factors – that picks up 82% of the possible candidates is not likely to be ignored, especially since robots aren’t such lazy thinkers as humans: whereas we are designed to conserve brain energy and make brutal trade-offs between calculation-effort and likely improvement in results, silicon-based neural networks do whatever it takes, at whatever cost and for however long, to optimise the solution.

A possible remedy, though tricky to implement because of the difficulty (ie cost) of getting the right data, is refined training data. This is not just a matter of removing biased and proxy-biased data, but of inserting relevant non-biased data. For our senior computer programmer, a database that focuses on measures or proxy-measures of the capabilities that drive superior performance in that role (hint: technical competence is pretty far down the list) will produce better results. But just try getting that training data – whatever the fast-talkers in Silicon Valley coffeeshops might tell you, it can’t be scraped from GitHub or social media.

And there’s an even bigger problem!

The offline world is a pretty biased place. Like small children with marshmallows, we find it hard to resist cutting through complex problems with dramatic and simple solutions: Worried about the long-term prospects of the rust-belt economy? Kick out the immigrants! There is overwhelming evidence for systematic discrimination – albeit mostly unconscious – in in everything from recruitment to venture funding, in the criminal justice system and in everyday life. The Berg (as I like to think of Facebook’s Mark and Sheryl), might plead ignorance, shock, regret, good intentions and other warm human factors in the face of accusations of promoting everything from foreign power election trolling to fake news to antisemitism, but they must both be well aware that their revenue model relies on advertisers paying for user attention, and that polarised clickbait works better than nuanced analysis. The Berg have never talked about debating the merits of income-generation versus bigotry-promotion. Is that because those conversations never took place? Or because they did, but everyone around the table voted for the money?

Ay, me. What to do?

Luckily, I think the robots can help us out here. It is nigh-on impossible to look inside the mind of The Berg, but we can interrogate the artificial neural networks, we can get a complete view of their training data, we can ask them straight questions and get straight (okay, convoluted but transparent) answers. Algorithms might misinterpret, but they do not lie.

So maybe instead of shooting the robot messengers, we should listen to what they are telling us about bias and how it adversely impacts results.

And then we should do something about it.