“An algorithm that scans resumes might say for example, ‘Oh, I notice when people use this kind of font, it has a high correlation with being productive, so this is the important feature.’ Is it? I don’t know, maybe it is, maybe it isn’t, but [the algorithm] could do things like that and it’s hard to understand why,” Venkatasubramanian says.

Letting an algorithm make hiring decisions leads to strange biases.

“You are being judged for things that you’re probably not even thinking about in your resume, like for example your address. There was one HR department that has been using an algorithmically driven system that gives people extra credit if they live within a close radius of the workplace because the data showed that if you had a longer commute, you were more likely to to quit or to be fired within a year,” says Crawford. “So what that also means is that they’re just starting to hire people who live nearby, behind which there is a whole range of other discriminatory functions.”

Many are just now beginning to wake up to the discriminatory problems associated with algorithms. Experts like Crawford and Venkatasubramanian are starting to look for solutions.

“I’m really interested in what we do about it because I’m concerned about the kind of discrimination we’re seeing against entire groups, be they African-American, be they women, be they people who live in rural areas — you name it. And we’re seeing a form of group discrimination often occur in these kinds of systems. But there are things we can do about it,” Crawford says. “How do you have sort of internal systems that are checking for discriminatory outcomes? A lot of technology companies are looking into that. Another thing you can do is external audits.”

“Education is incredibly important. I’ve been educated myself just by looking at this,” says Venkatasubramanian. “Essentially we’re trying to formulate a mathematical way of of describing bias and describing how to be fair - how algorithms could be fair, and trying to implement that into the algorithms. So there are lots of things we can do. And I think we need a lot more study of this and there is more of a growing interest in the technical side of things and how to do this.”

http://www.pri.org/stories/2015-12-06/are-algorithms-racist-and-can-we-fix (via kenyatta)