The Internet Is Dying

You may be familiar with "Dead Internet Theory," which states that "activity and content on the internet, including social media accounts, are predominantly being created and automated by artificial intelligence agents" and that "[m]any of the accounts that engage with such content also appear to be managed by artificial intelligence agents."

I don't know if the Internet actually is dead yet, but it certainly seems to be dying. Facebook is always pushing pages on me when I just want to see what my friends are up to; a lot of those are clearly AI-written prose. But even some of my friends who should know better often share AI-written posts!

image courtesy Stackery

Reddit, however, is even worse. So many subs have, of late, been taken over by bots that post LLM-generated content. The problem I have run into is that many posters don't seem to recognize this—these posts often get a ton of engagement and upvotes. I am pretty good at recognizing it, because of course I read hundred of pages of LLM-generated prose every semester. But a stranger on the Internet only has my word for it. Though one of the things I am coming to learn from all of this is that I think people who are bad at reading don't understand that other people can actually be good at it.

I wouldn't say I am perfect at it, but I am pretty well calibrated. I usually check by using Pangram; Pangram claims to have a false-positive rate below 1%. Of course they're incentivized to say that, but if you poke around on Google Scholar, the trend of independent analyses of Pangram seems to put the false-positive rate below 3%. But even if you link people to the Pangram result, they'll still be all, "But how do you know?" There are some bad LLM detectors out there which have given the good ones a bad rap. (And, in any case, I suspect most of the Ai DeTeCtOrS aRe So UnReLiAbLe people are probably the ones themselves using LLMs to write.)

It's infuriating to me that I have to deal with this on Reddit, because if I want to read crappy LLM-generated prose, I can just do it at work! Reddit is supposedly where I go to relax and have fun. On the other hand, there is a bit of satisfaction I get from getting a post deleted—I am certainly the number one bot hunter on a couple of subs.

I am getting better at making my case, though. The key is often to be able to link to a pattern of behavior—and this is also the key to understand what's going on, because that's probably the question I get the most. What is a bot's incentive to post a heartwarming parenting story on Daddit or a broadly worded book recommendation request on r/printSF? As the Conversation article I linked to above says, "This creates a vicious cycle of artificial engagement, one that has no clear agenda and no longer involves humans at all." That article discusses propaganda, but there are other reasons at play when it comes to the Dead Internet.

At least when it comes to Reddit, it's about slipping in product recommendations. Take this user, for example. Aside from all being LLM-generated the posts are pretty innocuous, but read every single one of them, and a pattern begins to emerge:

• A post in r/PostConcussion is supposedly about memory issues but links to an online concussion test they took.
• A post in r/eldercare is about their parent getting scammed by AI, and includes a link to a for-profit service that helps people get their money back. 
• A post in r/Entrepreneurs is supposedly about getting recommendations on how to fight fake reviews, and links to one specific service that will help. 
• A post in r/PaintByNumbers asks if other posters have painted places they want to go to, but slips in a mention of a specific kit from a specific company. 
• A post in r/family supposedly asks for advice about telling her sister she could have bought her engagement ring for $6,000 less online, and makes sure to mention the site. 

Note that of these five, three include a link, though two do not. 

(These bots make it tricky by hiding their posting history, but you can usually find the posts via Google, or the very helpful Arctic Shift tool. Arctic Shift reveals another one by this poster that got deleted on r/OldHomeRepair, where the poster asks about a quote they got for modernization.)

Obviously links help SEO, but what's up with the nonlinked posts? And some are even critical of the services mentioned, like the one on r/OldHomeRepair?

The answer turns out to be... other LLMs! There's a good 404 Media article on it (here's a link to an archived version, because it's paywalled). Lots of people ask LLMs for product recommendations these days. What's the best place for people to get production recommendations? Well, Reddit, because it is of course a compilation of real human experiences! Reddit posts get scraped and searched by LLMs and absorbed into their training data. So if you ask ChatGPT to recommend good paint-by-number kits, it's going to recommend one that lots of people on Reddit mentioned that they liked!

The posts that don't mention products, in between them, are to build up karma—hence heartwarming and/or ragebait stories on Daddit, for example. In fact, I've observed a pattern: often the bots build up comment karma with short, random comments in popular subs, then move on to making posts to farm karma, then move on to product recs.

Of course there's a tragedy-of-the-commons problem here. If everyone markets their products on Reddit via bots, then Reddit will cease to be the repository of real human recommendations that made the bots want to post there to begin with.

Although... I'm becoming increasingly skeptical that real human experience is what people want, anyway. Isn't it just easier to talk to bots? Maybe it's not the Internet that's dying, but the whole human social experience. 

We Are Now Firmly in the ChatGPT Era of Academic Writing

It seems unlikely that if you teach college writing, that you haven't been dealing with the impact of ChatGPT and other LLMs. I of course have had students using this next technology since Spring 2023. But the impact it's had on me and and my teaching is probably most starkly represented by this chart:

According to my syllabus policies, as well as our program policies, use of ChatGPT or similar technology is prohibited. Though I guess I can see how it might be useful in other courses, I don't see how it's useful in a writing course; my class (I would argue) is trying to use writing as a mode of thinking. We don't write to record what we already know, we write to come into the knowing of something.

The thing about ChatGPT use is that it's difficult to "prove" in some kind of "objective" sense—and this is the kind of thing academic integrity panels supposedly want. But the whole reason I am a professor of academic writing is that I supposedly have some kind of expertise in academic writing, and I do. It's an expertise honed by reading student writing for years, almost decades. I first taught an academic writing course in Fall 2008. Without methodically counting it all up, I would estimate that in that time I've taught sixty-five sections of first-year writing course, which means I've read the work of approximately thirteen hundred students, each of whom ought to write at a minimum of ten pages per semester, if we're going to be conservative. That means I've read at least (and certainly much more than) thirteen thousand pages of college-student papers in my life.

image generated, of course, by ChatGPT

You read a genre that much, you get to know it. I know what college students write like. What I'm reading now isn't it. But frustratingly, I don't know how much the academic integrity panel goes for "it just doesn't sound right" as evidence. A colleague of mine has suggested we should file more charges on this basis, though, and see what happens. Another colleague of mine has suggested that how college students write may be shifting as a result of ChatGPT, in that even if they're not actually using it your class, they read its output so much that it's influencing how they write. Now that's scary.

But anyway, clearly lots of them are using it. (Though, admittedly, some are clearly not! I have a lot of fondness for the crappy C paper now.)

The thing you can actually prove, though, is the veracity of sources. That is the basis on which all of my charges were filed this semester. Of my fourteen cases, I think three were about sources that did not exist. This, for me, means failure of the course. If you don't see that a basic part of research writing is that the sources you cite have to actually exist, then I don't know that you really belong in my class at all. I can't teach you this. Sure, use ChatGPT to find sources (in my personal experiments I have not found it to be very good at this, but I am sure it could be), but then make sure they are real! And of course, if you're summarizing fabricated sources on, say, an annotated bibliography, you are lying, because how could you have read a source that doesn't exist?

All of my other charges were about fabricated quotations from real sources, students claiming that direct quotations existed that did not actually exist. My institution requires that all academic integrity filings be accompanied by a formal meeting that is witness by a neutral faculty member; to make up for all the colleagues that had to witness mine, I witness a lot of other people's. This was the thing most of them saw as well. Frustratingly, a lot of students had this weird defense: that they didn't know quotation marks were reserved for direct quotations. One might tempted to believe this, except that very few students were making this mistake over a year ago, and though I think that the teaching of writing at the pre-college level has probably got worse since I started teaching in 2008, I don't believe that just a couple years ago, teachers stopped explaining what quotations marks are for.

So, the only explanation is that they are using ChatGPT to "find quotations"... but again, not confirming the existence of the quotations. Some of my students have admitted this when confronted, others have come up with not very compelling explanations. I have seen this across the board, in both my research writing class and in my text-based humanities course. (I can see why students might think I might not know that they made up a quotation from a source they found that I've never read; I don't see why they don't realize I won't catch them when they make up quotes from stories we all read together!)

Thankfully (I guess) it doesn't matter. At my institution, fabricated quotations or sources fall under the category of Deception and misrepresentation; I don't have to prove the fabrications come from anywhere in particular, I just have to prove they don't exist.

In my research writing course, fabrications in the final paper mean failing the final paper. It seems to me there's a basic parameter of the assignment you've failed to reach. And failing the final paper means failing the course. It seems to me there's something this course was supposed to teach you that you just didn't learn, and thus you need to take it again, if you can't write at least a D-level research paper by the end of an entire semester. In the past, I haven't had to enforce this policy very much, but I did at least three or four times this semester.

On top of all this, one has to remember—these are just the students I caught fabricating. There are also all the students using LLMs that I recognized but couldn't "prove," and thus just dinged them on points (I have become fond of marking assignments with a "0" and writing, "This is so vague an AI could have written it"), and then all the students who are actually "good" at using LLMs to write and turning in work that seems human-written... even when it's not. 

Even though ChatGPT has been around a couple years, it's clear that something is shifting in the way students use it. You can see that even just last semester, I only filed one charge (though I didn't teach research writing that time.) Many of my colleagues had similar experiences this semester in particular (though I don't think anyone filed as many charges as I did). As one of my colleagues has said, it seems like there's a shared ethos we used to assume the existence of that just doesn't exist anymore. And how can you teach people if that's the case? I can't teach someone that they want to be ethical.

So anyway, it's been a depressing semester. I even had two students file appeals that persisted beyond the end of the semester (and another just not respond to my communication attempts, which is for some reason an automatic appeal), which meant that it didn't stop even when the semester was over! But finally about two weeks ago, I heard about my last outstanding appeal.

I can't just keep doing everything the same way, clearly. But more on that in another post.