Skynet Will Be Confused

The government is now allowing our local peeping Tom's  at Palantir to use Anthropic’s initiative AI model to shift through government data for purposes not entirely known.  This is both nuts and confusing.  That they would use a model that has hallucinations built into its foundation is, frankly, disturbing.

Palantir, named after an evil all seeing artifact from the Lord of the Rings because apparently we don't do subtle in this country anymore, is a glorified peeping Tom.  Now, they do other things as well but their primary business started out as things like facial recognition and predictions. Palantir says it intends to use the model for sifting through large amounts of data, making predictions based on that data, and helping in communications.  The last is the usual write crappy emails that all the imitative AI companies sell, but the first two should scare you.

One of the defining features of imitative AI is that it bullshits.  Hallucinations is the polite term but it is more like bullshitting to me. Because these systems are just statistical models they have no view of the world and hence have no idea of truth.  Because of that, if their model says it's most likely that bullshit comes next then it will put bullshit next.  That means that even something as simple as a summary of a document can have false or misleading information in it.  

Now imagine making predictions based on that model or information.  Predictive AI is already difficult. The field is rife with examples of predictive AI learning that cancerous tumors have rulers next to them or that woman aren't going to be good employees.  Now add imitative AI bullshit to that mix - predictions based not on data but whatever an imitative AI invents. Some monkies and a dartboard might be worse but maybe by not that much.

The main purpose of predictions by a company like Palantir is that they can be used to hide guilt.  Bomb a wedding?  Sorry, but the AI told us it was a good target. Try to have your next nuptials look a little less terrorist-y.  What makes a wedding look terrorist-y?  Why, whatever the AI tells us looks terrorist-y of course.  No one is blamed, except maybe the terrorist-y looking wedding party and careers can safely continue.

Imitative AI is not going to make that process better. So why add them to an already unstable mix?  I suspect it is just Silicon Valley groupthink.  People like Peter Theil, the founder of Palantir, have a lot of money and a lot of their sense of the world tied to the idea that imitative AI is useful, reliable, and a step on the path to general artificial intelligence. That none of that has proven true has yet to penetrate the toxic mix of financial incentives and blinkered world view that dominated the Silicon Valley bosses.  But there may be a colder, more evil reason.

Imitative AI is not only prone to bullshit, it is also biased against pretty muxh anyone not white and male.  Since these models require an enormous amount of data they are largely trained on the Internet. And the Internet is a festering hive of scum, villainy, and bigotry.  These models have shown a distinct slant toward bigotry. And maybe that is fine as far as Palantir is concerned. 

My deepest, most cynical thoughts are that Palantir doesn't mind.  Since the US government mainly targets minorities at home and non-whites abroad, maybe a system that skews towards bias against those populations is useful. It makes it easier to target those groups while preserving the idea that the decision was impartial and therefore there is no one to blame

Again, that's my life cynical self speaking.  It's likely more connected to the aforementioned groupthink and financial incentives. But even if the cynical little bastard that lives inside me is wrong about motives, he’s not wrong about effect.  Mixing imitative AI into an already difficult prediction process is likely to make the predictions more biased. And is more likely to create a situation that increases harm to innocent people in targeted populations while insulating the perpetuators from accountability.  

Even if the motives of the people doing this are pure as the driven snow - and no one seriously believes that - the system is almost inevitably going to be more harmful than what came before.