Hallucinations, Limits, and Verification

Understand why generative AI can invent facts, what situations increase that risk, and how to design safer workflows.

Explanation

Hallucinations happen when the model produces plausible but unsupported or incorrect information.

Risk increases when the prompt asks for obscure facts, recent facts, or answers without evidence.

Verification layers such as retrieval, human review, and schema checks are essential in serious applications.

Why this topic matters in practice

In generative AI products, the model is only one part of the system. The surrounding workflow determines whether the output is useful, safe, and maintainable. This lesson matters because it helps you connect the idea to tasks such as tutoring, search, copilots, business assistants, and production automation.

Examples

Require evidence before final answers

The code below is intentionally concise so the underlying pattern stays clear. It focuses on the application logic you can reuse, even if you later switch model providers or deployment environments.

def answer_with_evidence(question, evidence):
    if not evidence.strip():
        return "I do not have enough verified evidence to answer reliably."
    return f"Answer the question using only this evidence:\n\n{evidence}\n\nQuestion: {question}"

print(answer_with_evidence(
    "What is the refund period?",
    "Refunds may be requested within 14 days of purchase."
))

How the coding section works

• A simple refusal rule is better than a confident but unsupported answer.
• Evidence-bounded prompting is a core pattern in retrieval-augmented systems.
• This approach does not eliminate all errors, but it reduces unsupported guessing.

Implementation advice

When turning this lesson into a real feature, think beyond the code snippet itself. Decide what inputs should be allowed, how you will validate outputs, how you will recover from errors, and how you will measure whether the feature is actually helping users. Those surrounding choices often determine whether an AI feature feels polished or unreliable.