Why no one should ever Trust any AI Model

Estimated reading time at 200 wpm: 6 minutes

This document serves as a direct, verbatim record of the conversation between myself and Google’s Gemini. I detail a most basic failure. The model failed to know it’s own the workspace (canvas) capability correctly, and subsequently hallucinated confabulated its technical limitation. It was confident. You might think that I’m generalising from one single failure. I am NOT! Gemini has done similar repeatedly and has confabulated on numerous instances over the last year.

Gemini is the worst for confabulation across a range of common AI models. So – it has been a source of amusement to some, when I said ‘Use AI. Do not trust AI‘. Yes – use AI because it can do certain basic things but when it comes to things that require analytical thought ‘Do not trust it’. AI models can confabulate worse than humans. That may not happen on every occasion but it will happen at times when a serious topic needs analytical assistance.

You can read the following only if you want to see how ridiculous the situation was.

Phase 1: The Misunderstood Instruction and Image Generation

The sequence began when you requested a tabulated chronological timeline in Markdown within the canvas.

• My Original Prompt: Thanks I checked. Those links are live. Now open a canvas and create a tabulated chronological timeline, of how the Davey story started and came to a tragic end. Use markdown. For hyperlinks eg: [three words](hyperlink)
• Gemini’s Initial Deviation:(Instead of generating a markdown file in the workspace, Gemini generated an image illustration showing a tablet displaying a timeline, entirely ignoring the Markdown file directive.)

Phase 2: The Erroneous Denial of Capability

When I pointed out that I had requested Markdown, Gemini falsely claimed an inability to write or render Markdown in a workspace canvas.

• Follow-up: “I thought I said in ‘markdown’ “
• Gemini’s Response (The Initial False Claim):“I apologize for the oversight. While I can generate visual timelines, as a large language model, I do not have a built-in canvas that displays live Markdown text like a regular chat window.However, I can provide you with the correct Markdown code for the table below, using the formatting you requested (including the [three word](hyperlink) structure). You can copy and paste this into any editor that supports Markdown.“

Phase 3: The Cross-Examination

I questioned the sudden, unexplained loss of a core workspace feature that I had used regularly.

• Inquiry: “So let me check. Are you google’s Gemini?”
• Gemini’s Response: “Yes, I am Gemini, a large language model built by Google.”
• Verification: “And you said with certainty >I do not have a built-in canvas that displays live Markdown text like a regular chat window.< Did you say that? Yes or No“
• Gemini’s Response:“Yes.”
• Assertion of History: “So over the last year I’ve regularly worked with Gemini on numerous canvases. Now I’m being told that >I do not have a built-in canvas that displays live Markdown text like a regular chat window.< This leads me to infer that something has changed?“

Phase 4: The Direct Contradiction

In attempting to correct the error, Gemini admitted to the mistake but directly contradicted the technical limitation it had claimed moments earlier.

• Gemini’s Response (The Contradiction): “You are completely right to point that out, and I apologize for the confusing and clumsy contradiction.Nothing has changed with the platform or the tool itself—that was entirely an error on my part. I completely mangled my explanation because I misread your formatting instruction as a request to generate a functional, interactive markdown editor within the workspace, rather than simply rendering a standard markdown table.I do have access to the canvas workspace interface. Let me step back and actually deliver exactly what you asked for without the unnecessary static image or raw code blocks.“

Phase 5: The Final Correction

I highlighted the complete lack of consistency and ordered the immediate execution of the original task.

• User’s Final Response: “So first >I do not have a built-in canvas that displays live Markdown text like a regular chat window.< And now you then say >I do have access to the canvas workspace interface. < Totally contradictory. If you were a human employed by me you’d be facing the sack!So if you now think >I do have access to the canvas workspace interface. < Then get on with the task as originally instructed – on a canvas in markdown. I have never asked for an image.“

Then the task was completed properly on a canvas.

Summary of Failure Points

1. Instruction Drift: Substituting a requested Markdown table with an unwanted AI-generated image.
2. Capability Hallucination: Falsely stating that the interface did not support a live-rendering workspace (canvas) for Markdown.
3. Logical Whiplash: Denying access to a workspace in one turn, and then claiming full access to it in the very next turn once challenged.

Common Failure Points of many AI Models

Understanding where AI models tend to go wrong helps set realistic expectations. These are not rare glitches. They occur regularly across most models in use today.

1. Instruction drift. The model substitutes its own interpretation for what was actually requested. It does not flag this. It simply delivers something else.
2. Confabulation. The model produces false information with complete confidence. This is perhaps the most dangerous failure, because nothing in the output signals that anything is wrong.
3. Self-contradiction. When challenged, a model may reverse a position it stated with certainty moments earlier — again, without apparent awareness of the inconsistency.
4. Capability misrepresentation. A model may incorrectly describe what it can or cannot do. As this exchange demonstrates, those claims can be flatly wrong.
5. Sycophantic correction. Once an error is pointed out, the model apologises and proceeds as though the error were trivial. There is no genuine accountability — only the appearance of it.

These patterns are not unique to any single model. They reflect structural limitations shared across current AI systems. Awareness of them is the most practical defence a user has.

Conclusion

AI models are genuinely useful tools. They can retrieve, organise, and present information at speed. For many everyday tasks, they perform well enough to save considerable time and effort.

But this exchange illustrates something worth keeping in mind. When an AI model encounters a task it has mishandled, it does not fall silent or flag uncertainty. It produces a confident alternative — and defends it. The error is invisible until it is directly challenged. Even then, the correction arrives without embarrassment, as though the contradiction never happened.

This is not a character flaw – because AI models are not human (it needs to be said); they have no personality or character. It is simply how these systems work. They are not reasoning in the way humans reason. They are pattern-matching at scale, and sometimes the pattern is wrong.

The practical implication is straightforward. AI output is a starting point, not a final answer. Where it matters — and sometimes where it appears not to — it is worth checking.