Imitative AI and the Damage Done: The New Yorker Profiles Doomed Students

By now, you have all probably seen the New Yorker piece on how students are using imitative AI to cheat. It is a depressing, if well done, piece, and many people have already spoken on it. The best piece I saw was probably this (via Lyz at Men Yell At Me, which you should be reading), but I am sure that there are others. What struck me most, however, was just how screwed these students are going to be.

First, let us get this out of the way: yes, this is cheating. The point of writing essays and research papers is not the finished product. The point is learning how to think about a subject and research that subject. Outsourcing those tasks to imitative AI, especially given how much it bullshits, is cheating. It is an attempt to pretend that you learned the point of the lesson without doing so. I once got an A on a paper that “proved” the witches in Macbeth were actually angels. It is a dumb idea, of course, but I could back it up with citations from the text and research from Shakespeare’s life and time. The point was not to regurgitate some talking point I found in a book but to make an argument. I proved I could do that, as dumb as the argument may be. Students who use imitative AI could never make that kind of leap, and if they did, they wouldn’t know how to support it (and yes, I did think about law school. Why do you ask?)

I am also not overly concerned with the student who wants to build a cheating app. He is an amoral grifter, the kind of walking condom ad that has been with use since before condoms. If he wasn’t trying to con people with this, he would be trying to con people with something else. Yes, it is disturbing he has raised millions of dollars for his scam, but he is not alone, and he is not ahistorical.

No, what really bothers me is the one young woman, Wendy, in the piece, who feels she has to use these tools, even though she says, “I really like writing.” These tools, and the hucksters like Sam Altman who push them, are destroying people like Wendy. The specific skills you lick up in college, yes even in technical fields like engineering or medicine, are not sufficient to do a job. Most of what you learn about your chosen profession, you will learn by doing. What matters is curiosity and the ability to think. The measure of your ability to do the job is how you handle things you don’t know, the unexpected, the edge cases. It matters more how you figure out what is wrong with a script that is beeping every 5 seconds at everyone using it (an invisible control character that caused the beep got added to the script by some idiot who now has the gall to think he can write about tech bullshit.) rather than your ability to produce a textbook script. Wendy had that, and imitative AI is taking that away.

We know that the use of imitative AI degrades peoples’ ability to think. Schools exacerbate this tendency. High schools are entirely focused on standardized tests. Colleges, bereft of state monies and dependent upon funding from students and corporate grants, focus on grades and pleasing their students above all else. In each case, students are pushed away from learning, from thinking, and toward maximizing outcomes. It is very me first, very short sighted, and very business outlook. And it is destroying these students.

Imitative AI systems are not a good business. There is a very good chance that these things don’t exist in two to five years as a going concern. Once the fad passes —and this is a fad, it will not deliver on the promises its hype-men make — what will these students do? The best of them will learn, or relearn, how to think. But that is more difficult once you have depended upon a crutch, and not everyone will be able to do so easily without the structure and guidance of school. Our insistence on making schools nothing more than business prep has landed us with a generation of people that will be useless to businesses.

Fixing this is going to be hard. All the fixes — ending almost all standardized testing in public schools, expressing grades only in pass/fail modes, paying more for teachers so we can have smaller classes that allow for in class writing assignments — are either impossible in this present environment, have too long a time frame to matter, or both. We are going to have to deal with an entire generation that has outsourced its thinking to a bullshit machine. All for the profit of greedy bastards who don’t care a whit about anyone other than themselves.

Well done tech bros. Well done indeed.