Hallucinations of large models refer to phenomena in artificial intelligence, particularly in large language models, where the content generated by the model is inconsistent with real-world facts or the user’s input instructions. These hallucinations can be classified into factual hallucinations and fidelity hallucinations: the former refers to content that does not match verifiable facts, while the latter refers to content that does not align with the user’s instructions or context. This phenomenon may arise from data defects, insufficient training, or issues with the model architecture, leading to inaccurate or unreliable information being output.
How do Hallucinations of Large Models Work?
Hallucinations in large language models originate from data compression and inconsistencies. During the training process, models need to handle and compress large amounts of data, which leads to information loss, causing the model to "fill in the blanks" and generate content inconsistent with real-world facts. Issues in the quality of pre-training data can also contribute to hallucinations. There may be outdated, inaccurate, or missing critical information in the dataset, leading the model to learn incorrect information. In the training phase, the model uses real labels as input, while in the inference phase, it relies on its own generated labels for subsequent predictions, creating inconsistency that may lead to hallucinations.
Large models predict the next token based on the previous one, processing input only from left to right. This unidirectional modeling limits the model's ability to capture complex contextual dependencies, increasing the risk of hallucinations. The softmax operation in the model’s final output layer restricts the expressiveness of the output probability distribution, preventing the model from generating the expected distribution, which contributes to hallucinations. Techniques like temperature, top-k, and top-b, which introduce randomness during inference, can also lead to hallucinations. When processing long texts, models focus more on local information and may lack attention to global context, leading to forgetting instructions or failing to follow them, thus causing hallucinations. There is inherent uncertainty in the meaning of the model’s generated outputs, which can be measured by prediction entropy. The higher the entropy, the more uncertain the model is about possible outputs. These factors together contribute to hallucinations, where the model generates content that seems plausible but contradicts known facts.
Main Applications of Hallucinations of Large Models
Text Summarization Generation: In text summarization tasks, large models may generate summaries that do not align with the original content. For example, the model may incorrectly summarize the timing of an event or the people involved, distorting the summary.
Dialogue Generation: In conversational systems, hallucinations may lead to responses that contradict the conversation history or external facts. This could include introducing non-existent characters or events or providing incorrect information when answering questions.
Machine Translation: In machine translation tasks, hallucinations may result in translations that do not match the original content. The model may introduce information not present in the source text or omit important details.
Data-to-Text Generation: In data-to-text generation tasks, large models might produce text that is inconsistent with the input data. This could include adding information not present in the data or failing to accurately reflect key facts from the data.
Open-Ended Language Generation: In open-ended language generation tasks, large models may generate content that contradicts real-world knowledge.
Challenges Posed by Hallucinations of Large Models
Data Quality Issues: The generated text may contain inaccurate or false information, such as producing content that does not match the original in summarization tasks or providing incorrect advice in dialogue systems.
Challenges During Training: Models may excessively rely on certain patterns, such as proximity or co-occurrence statistics, causing outputs that do not match actual facts. In tasks that require complex reasoning, models may fail to provide accurate answers.
Randomness During Inference: Randomness can lead to model outputs diverging from the original context, such as generating inconsistent translations in machine translation tasks. In long text generation tasks, this may result in inconsistent information between the start and end.
Legal and Ethical Risks: In high-risk scenarios, such as judicial judgments or medical diagnoses, hallucinations could have severe consequences. Users may lack vigilance about the model’s output, leading to misplaced trust in incorrect information.
Challenges in Evaluating and Mitigating Hallucinations: Inadequate evaluation methods may lead to misjudging model performance, affecting optimization and improvement. Insufficient mitigation strategies may allow hallucinations to persist in practical applications, impacting user experience and model credibility.
Limited Applicability: The hallucination problem limits the application of models in fields requiring high accuracy. Domain specialization may cause models to generate more hallucinations in cross-domain tasks, affecting their broader applicability.
System Performance Issues: Performance problems in models may lead to user loss of trust, affecting their competitiveness in the market. Reduced credibility may limit the model’s use in critical tasks, such as financial analysis or policy-making.
Future Prospects of Hallucinations in Large Models
With the continuous development of deep learning techniques, especially the optimization of pre-trained models like Transformers, large language models (LLMs) have shown strong potential in terms of understanding and creativity. Research on hallucinations in large models is not limited to natural language processing but has extended to multimodal fields such as image captioning and visual storytelling, showing broad application prospects. Researchers are exploring more effective ways to evaluate and mitigate hallucinations, aiming to improve the trustworthiness and reliability of these models. As large models are applied in high-risk domains like healthcare and law, the legal and ethical risks posed by hallucinations are gaining increasing attention, prompting the development of related regulations and ethical guidelines. Solving the hallucination problem requires collaboration across various fields such as natural language processing, knowledge graphs, and machine learning, and we can expect to see more interdisciplinary research and solutions in the future. Addressing hallucinations in large models will require collective efforts from the entire industry, including data providers, model developers, and application developers, to promote the healthy development of AI technology.
