Why AI Tasks Keep Going Off Track? Check Your Unknowns First

Deep thoughts on AI and aspirations

—— ByteDance Deep Thinking Circle

Have you ever had this experience: you write a detailed prompt, but the result is still wrong, and it feels like the AI keeps going off track?

You used to blame the model for not being smart enough. But today’s models are strong enough to execute most tasks. When the result is wrong, it’s probably not the model’s fault—it’s that your instructions are too far from reality.

Thariq from Anthropic’s technical team broke down this gap into a framework called “map and territory.”

The prompts, context, and instructions you give the AI are the map. Where the task actually happens—the codebase, real constraints, the actual environment—is the territory. The gap between map and territory is the unknown. When AI encounters unknowns, it can only guess, and the quality of those guesses determines output quality.

The stronger the model, the more expensive your unknowns become. This sentence is worth reading twice.

Four Types of Unknowns

Known knowns. Content written into the prompt that the AI can faithfully execute.

Known unknowns. You know the problem exists but haven’t solved it yet: feature boundaries aren’t clear, approaches aren’t decided. These can still be resolved before starting.

Unknown knowns. You have preferences and intuitions, but they’re so obvious you don’t write them down, and you only know if something’s right when you see the result. This manifests as constantly saying “adjust it a bit more,” feeling it’s still not right—because you don’t know what “right” looks like.

Unknown unknowns. Completely unexpected pitfalls. They were never on your map to begin with, and by the time you discover them, you’ve usually already gone far in the wrong direction.

Excellent AI users don’t necessarily write better prompts—more often, they have fewer unknowns before starting. This is a trainable skill, not a gift.

Pre-Implementation: The Step Most People Skip Entirely

Many people start writing prompts as soon as they get a task, then rework halfway through when they realize something’s wrong. Spending ten extra minutes clearing unknowns upfront can save ten hours later.

Five methods, same logic whether you’re writing code, articles, making product decisions, or editing videos:

Blindspot detection. For “don’t know what to ask.” Have the AI do a blindspot pass directly, while telling it your background—who you are, how much you know about this domain. It needs to know your starting point to find blindspots valuable to you, rather than listing things you already know.

Brainstorming and prototypes. For “know what I want but can’t articulate it.” Have the AI give several radically different directions, then you give feedback on which is right, which is wrong. Your reactions are information. For small needs, changes in sketches take seconds; changes after deep implementation might require starting over.

AI reverse questioning. For “know there are fuzzy areas but don’t know where to start asking.” Give it the background, let it ask one question at a time, prioritizing questions where “your answer will change the overall direction.” You’re not asking the AI—you’re letting the AI interview you, delegating the job of finding questions to the party with a clearer view of the whole picture.

Provide reference material. For “can’t describe it.” For developers, the best reference is source code: point to a library that implements the behavior you want, let the AI understand it, then reimplement. Charts, documentation, screenshots all work, but source code contains the richest information.

Implementation plan. Have the AI draft a plan first. The point isn’t the full picture, but separating decisions that need your sign-off from execution it can handle itself—put the former up front (data structures, interfaces, article structure), completely hand off the latter.

During Implementation: Maintain a Deviation Log

No matter how thorough the preparation, unforeseen situations will still emerge during execution. The core of this phase isn’t eliminating unknowns—it’s not panicking when you encounter them.

Have the AI maintain a deviation log file: when it must deviate from the plan due to edge cases, prioritize conservative handling, note it in the log, and keep moving forward without stopping to wait for your confirmation.

This solves two problems: AI becoming inconsistent over time; getting stuck waiting for your instructions at every small issue. With a log, it can move forward on its own, with all decisions traceable. This record is also raw material for “better maps” for the next task.

Post-Implementation: Proposals and Quizzes

Task complete, two wrap-up actions.

First, package the output, prototype, and implementation log into a document others can understand. This both helps others understand what you did and forces you to understand what you had the AI do.

Second, have the AI quiz you. First provide a report that clearly explains what was done and the reasoning behind it, ending with a quiz you must pass. Only if you answer correctly does it prove you truly understand, rather than just skimming. In many cases we skim, think it looks fine, and merge, but if actually asked what changed and why, we can’t answer.

A Complete Example

Thariq used Claude to edit Fable’s launch video from scratch. He doesn’t understand color grading, and the video came out with muted colors. He had Claude make several versions to choose from, but after finishing realized he still didn’t know which was right—he had no judgment criteria. This was an unknown unknown. So he went back to blindspot detection, first having Claude teach him to understand color grading and establish judgment criteria, then came back to do the work.

Unknowns aren’t cleared all at once before starting. They continue emerging throughout the task. Each time you get stuck, first stop and think: which type of unknown am I not handling well right now?

Cheap mistakes upfront beat expensive rework later. The math is simple.

Note: Framework from Thariq Shihipar (Anthropic) public articles and interviews; viewing data and project details in the article are from video transcription and represent the original author’s views.