Working with long documents and code

A conversation that started well and went sideways an hour later is not the model getting worse. It is the beginning of the conversation falling out of the context-window.

What actually happens

Everything the model sees on a given call - system prompt, conversation history, attached files, the answer being written - shares one budget measured in tokens. When the budget is exceeded, the oldest turns are dropped. Your carefully written setup goes first, and nothing tells you it happened.

Two fixes

Put the rules somewhere that does not scroll away. That is what a system-prompt is for, and it is the durable fix.

Restate the constraints when you notice drift. One line - "as before: minimal edits, no reformatting" - costs nothing and pulls the session back.

A large window is not perfect recall

Models retrieve reliably from the start and the end of a long input, and less reliably from the middle. So a million-token window does not mean you should paste a million tokens. Put the instruction and the thing you actually care about at the edges.

Handing over code

Paste the file, not the repository. The model does not need your whole project to review one function, and the extra context costs money, latency and accuracy.

Where a file depends on something you have not shown, say so. A prompt that ends with "if part of this depends on code you cannot see, say which part" gets you a question instead of an invented implementation.

Reference files by their real path. config/functions.php:41 is something you can act on; "the helper function" is not.

Chunking long documents

When the input genuinely does not fit, chunk on structure - chapters, sections, agenda items - not on character count. A chunk cut mid-argument produces a summary of half an argument.

Then run the same prompt per chunk and combine the outputs in a second pass. Summarising summaries loses less than one call that saw only a truncated document.

When it still drifts

Start a new conversation and paste in a short handover: the goal, the decisions already made, the current state. A fresh window with a good brief beats a long one carrying forty turns of dead discussion.