Introduction
In this article, inspired by the essay "ChatGPT is Bullshit"* we explore the nature of inaccuracies generated by large language models like ChatGPT. Far from being simple "hallucinations," we argue, following the ideas of philosopher Harry Frankfurt, that it is more appropriate to understand these responses as "bullshit." This distinction is crucial for adjusting our expectations about the veracity of AI responses and improving their responsible use in sensitive fields such as medicine and communication, among others.
Hallucinations
The text we are discussing today deals with the nature and accuracy of artificial intelligence (AI) technologies, specifically ChatGPT, a type of AI based on large language models (LLMs). While these models can create text that appears human-generated, their ability to produce false statements raises questions about how we interpret these falsehoods.
In the AI world, the type of incorrect or false responses produced by ChatGPT and other LLMs are called "hallucinations." The article argues that treating these as "hallucinations" or even as "lies" is inappropriate and that it would be more accurate to call them "bullshit."
Bullshitters
In his essay "On Bullshit," published in 1986, American philosopher Harry Frankfurt explores this concept in depth. For Frankfurt, bullshit and lies are very different because the "bullshitter" is not concerned with the truth or falsehood of their statements; they simply seek to create a particular impression.
What characterizes "bullshit" is a fundamental indifference to the truth. Unlike the liar, who needs to know the truth to hide it, the "bullshitter" does not care whether what they say is true or false. Applied to the context of AI, ChatGPT's bullshit refers to the production of text without any concern for its veracity.
Functioning of LLMs
LLMs generate text based on statistical models that predict the likelihood of the next word in a text. They are not designed to represent the truth but to produce responses that appear human-like.
Describing the false statements of LLMs as "hallucinations" can be misleading because it implies that the models are misperceiving reality. Instead, what happens is that these models simply generate plausible text without an underlying truth commitment.
The article concludes that it is more precise and useful to describe ChatGPT's inaccuracies as "bullshit" rather than "hallucinations" because this approach provides a clearer understanding of these systems' functioning and limitations and helps users and developers better guide their applications and manage expectations more realistically.
The Importance of Naming
Although it may seem that way, the problem is not just terminological; the way we define things influences how we handle them. Let's look at an example:
Example of Hallucination
Imagine an AI system used by doctors to quickly obtain information about diagnoses and treatments. A doctor enters a patient's symptoms into the AI system to get a treatment recommendation.
The AI, based on erroneous or misinterpreted patterns during its training, generates a treatment recommendation that includes a nonexistent medication or a combination of symptoms that are not medically related.
If we describe this as a "hallucination," it would imply that the AI is incorrectly perceiving information, as if it had a distorted view of medical data. This suggests that the system is trying to accurately represent medical reality, even if it fails.
Based on this idea, developers might focus on improving training data or adjusting the model to avoid these "perception errors." However, users could continue to trust the system under the false assumption that, while it may occasionally make "errors," it generally provides accurate results.
Example of Bullshit
Now, consider the same scenario from this perspective: the AI generates a recommendation based on statistical patterns without concern for the truthfulness of the information. In other words, it only produces a response that seems medically plausible but has no basis in truth.
This implies that the AI has no commitment to the veracity of its responses and that its goal is to generate text that appears appropriate, regardless of its accuracy, making it clear that the system is not designed to represent the truth.
By recognizing that the AI is producing "bullshit," developers and users would assume from the outset that the system is unreliable for accurate medical recommendations. This would lead to greater caution in using AI in specific scenarios and could motivate the development of additional fact-checking mechanisms before presenting any recommendation to the user.
The Importance of the Difference
If doctors believe the AI is "hallucinating," they might think these are rare errors and trust most of the recommendations. However, if they understand that the AI "produces bullshit," they will be more cautious and skeptical about any output without rigorous verification, which can be crucial in certain scenarios.
Identifying the problem as "bullshit" instead of "hallucination" helps shift developers' focus from improving the system's perception to implementing stricter truth and external validation controls.
Thus, the difference between "hallucinating" and "bullshitting" in this example is crucial because it affects how the reliability of AI is perceived and managed. In especially sensitive environments like medicine, as well as in other areas such as journalism, education, communication, etc., this distinction is key.
Where Do Social Researchers Come In?
These insights open many areas of interest and possible action for research, such as:
- Designing interfaces that clearly communicate the limitations and capabilities of AI. This includes alerting users to the lack of verification of the truth in the generated responses.
- Monitoring how users interact with AI and respond to possible instances of "bullshit" to gain valuable insights that help adjust both the model and the user interface.
- Developing more effective communication strategies to educate the public and stakeholders about these technologies' real capabilities and limitations.
- Designing systems that limit decision-making based solely on AI recommendations to mitigate the risks associated with misinformation, collaborating with experts in the field, especially in critical contexts such as health, security, and finance, to minimize the risk of AI "bullshit" causing harm.
Any more ideas?
*Ref: Hicks, M.T., Humphries, J. & Slater, J. ChatGPT is bullshit. Ethics Inf Technol 26, 38 (2024). https://doi.org/10.1007/s10676-024-09775-5
Michael Townsen Hicks is a professor of Philosophy of Science and Technology at the University of Glasgow.
