March 4, 2013 - Since the launch of ChatGPT in late 2022, there has been an ongoing discussion about how AI language models will broadly impact the world. Now, with a study byStanford UniversityThe picture is becoming clearer with the release of new research led by the company. The study found that AI language models are now assisting with up to a quarter of professional communication tasks across multiple industries, with a particularly significant impact in less-educated areas of the United States.
"Our study reveals a new reality that businesses, consumers, and even international organizations have come to rely heavily on generative AI in their communications," the researchers noted in their report, which tracked the adoption of large-scale language models (LLMs) across a variety of industries from January 2022 to September 2024. The study dataset covers 687,241 consumer complaints received by the U.S. Consumer Financial Protection Bureau (CFPB), 537,413 corporate press releases, 304.3 million job postings, and 15,919 United Nations press releases.
Using a statistical testing system to track vocabulary usage patterns, the researchers found that about 18% of financial consumer complaints (including 30% of all complaints in Arkansas) showed signs of AI assistance during this period, and that 24% of corporate press releases, 15% of job postings, and 14% of UN press releases also show signs of AI involvement.
The study also found that while AI adoption was higher overall in urban areas (18.21 TP3T), theHowever, areas with lower levels of education used AI writing tools more frequently (19.91 TP3T), higher than in areas with higher levels of education (17.41 TP3T). This phenomenon runs counter to the usual pattern of technology adoption, in which more educated populations typically adopt new technologies more quickly.
"In the area of consumer complaints, LLM's adoption of geographic and demographic patterns exhibits very different characteristics from historical technology diffusion trends. In the past, technology adoption has typically been concentrated in urban areas, higher income groups, and more educated populations." The researchers noted.
The researchers note that AI writing tools may provide a "boost" to those with less educational experience. They wrote: "Although the urban-rural digital divide seems to persist, we find that less educated areas show slightly higher LLM adoption rates in consumer complaints," they said.This suggests that these tools may become 'equalization tools' in consumer advocacy. "
The reliability of AI writing detection services has previously been questioned, and this study confirms that view. At the individual document level, AI detectors are not reliable, but when analyzing data from millions of documents as a whole, a number of characteristic patterns begin to emerge that hint at the impact of AI language models on text.
The researchers developed a methodology based on a previously published statistical framework to analyze changes in lexical frequency and language patterns before and after the release of ChatGPT. By comparing large volumes of text before and after the release of ChatGPT, the researchers estimated the proportion of AI-assisted content at the population level. The hypothesis is that large-scale language models tend to select certain specific vocabulary, sentence structures, and language patterns that are subtly different from typical human writing.
To validate the reliability of this approach, the researchers created test sets with known proportions of AI content (from 0% to 25%).and found that its method predicted these proportions with an error rate of less than 3.3%. This statistical validation gives them confidence in their population-level estimates.
1AI notes that while the researchers specifically state that theTheir estimates may only represent a minimum level of AI use, but the actual level of AI engagement is likely to be much higher.. Because of the difficulty of detecting content that has been heavily edited or generated by increasingly sophisticated AI, the researchers say that theTheir reported adoption rates may significantly underestimate the true level of generative AI usage.
According to the researchers, all of the domains they analyzed (consumer complaints, corporate communications, job postings) showed similar patterns of adoption: a sharp rise in adoption starting three to four months after ChatGPT's launch in November 2022, followed by a stabilization at the end of 2023.1 The researchers also found that the adoption of ChatGPT was not as high as it should have been.
The researchers acknowledged that there were limitations to the analysis because the study focused primarily on English-language content. In addition, as mentioned earlier, they found that it was not possible to reliably detect AI-generated text that had been manually edited or text generated by an updated model that mimicked human writing styles. Therefore, the researchers believe that their findings represent only a lower bound for the adoption rate of actual AI writing tools.
The researchers note that stagnant AI writing adoption in 2024 could reflect market saturation or the ability of increasingly complex LLM-generated text to evade detection methods. They conclude.Today we live in a world where human writing is becoming increasingly indistinguishable from AI writing, and this has profound implications for social communication.
"Increasing reliance on AI-generated content may create challenges for communication." The researchers write, "In sensitive areas, overreliance on AI may result in information that fails to address real-world problems or reduces the credibility of the information when released to the public. Overreliance on AI may also trigger public distrust in the veracity of corporate information."