Software teams already have more context than they can comfortably process.
It lives in code, pull requests, Slack threads, Linear or Jira tickets, incident timelines, dashboards, runbooks, customer messages, and the memory of whoever happened to be on call six months ago.
AI assistants make that fragmentation more visible. A model can write a convincing answer in seconds, but the usefulness of that answer depends on what the system gave it, what it left out, and whether any of it was still true.
This is why I think the difficult part of AI-assisted software work is becoming less about prompts and more about context engineering.
The question is not only whether an assistant can retrieve relevant text. The harder question is whether a team can trust the context pipeline behind its answer.
The real system is larger than the model
Consider a fairly normal production question:
Can we safely change the retry policy for failed payments?
The repository might explain the current implementation. A ticket might contain the original business requirement. Slack might contain a decision that was never copied into the ticket. Sentry could show the current failure shape. A dashboard could reveal that a supposedly rare path now handles meaningful volume. A runbook might describe a manual recovery process, while a recent incident says that process no longer works.
A code-aware assistant sees only part of this. A search system can retrieve more, but retrieval alone does not resolve which source is authoritative, whether the information is fresh, or whether the person asking is allowed to see it.
The model is one component. The product teams need is a context system around it.
Relevance is only the first requirement
Most context discussions start with relevance: find the documents that are semantically closest to the question and place them in the model's context window.
That is useful, but production systems need stronger guarantees.
- Provenance: Where did this claim come from? Can the user open the source and inspect it?
- Freshness: When was the source last observed, and has a newer event superseded it?
- Permissions: Is the requesting user allowed to retrieve this source, not only allowed to use the assistant?
- Authority: Is this an approved runbook, an informal conversation, a generated summary, or an outdated proposal?
- Contradictions: What happens when the ticket, code, and incident notes disagree?
- Memory policy: What should be retained, recomputed, expired, corrected, or deleted?
- Evaluation: How do we know the assembled context improves real engineering decisions?
Without these properties, a context layer can make weak information feel more authoritative. The assistant becomes fluent at repeating organizational confusion.
Provenance should survive every transformation
Teams rarely feed raw sources directly into a model. Content is chunked, normalized, summarized, embedded, classified, and merged with other content. Those transformations are practical, but each one can weaken the connection to the original evidence.
A reliable system should preserve an evidence chain through those steps.
If an assistant says that a service has a three-second timeout because of a partner constraint, the user should be able to see whether that came from the current code, a vendor document, an incident review, or a two-year-old Slack message. If the answer combines several sources, that should remain visible too.
This is not only a citation feature. Provenance changes how people review decisions. It lets an engineer distinguish a measured fact from a team convention, a historical reason from a current requirement, and a source-backed conclusion from model synthesis.
Freshness is not one timestamp
Putting an updated_at field on a document is not enough.
Different sources age differently. A deploy event may be useful for hours. A runbook may remain valid for months, but only if somebody verifies it after architectural changes. A decision record can remain historically correct while no longer describing current behavior. Repository context changes at commit speed. Permissions can change immediately.
Freshness therefore needs policy:
- How often is each source synchronized?
- What event invalidates a derived summary?
- Can the system identify the commit, deployment, or incident state an answer refers to?
- When should stale context be shown with a warning, and when should it be excluded?
A useful answer can be based on older evidence. The dangerous answer is one that presents old evidence as current without saying so.
Permissions belong inside retrieval
An assistant connected to internal systems creates a new access path across them. A user who can ask the assistant a question should not automatically gain access to every indexed source that might answer it.
Permission checks need to happen during retrieval, with the identity and scope of the requesting user. Filtering after generation is too late because restricted context has already influenced the answer.
This becomes more difficult when the system stores summaries or memories derived from several sources. The resulting artifact needs an access policy at least as strict as the material that produced it. If one source is later deleted or restricted, the derived artifacts may need to be invalidated and rebuilt.
That is ordinary data governance applied to a new interface. The chat box does not make the problem disappear.
Contradictions are information
Real organizations do not have one perfectly consistent truth. The code can disagree with the documentation. Two teams can use the same term differently. A ticket can describe intended behavior while production shows actual behavior. A newer decision can reverse an older one without deleting it.
A weak context system hides this by selecting one chunk and producing a clean answer. A stronger system can surface the disagreement:
- The runbook says retries stop after three attempts.
- The current configuration permits five.
- The most recent incident review recommends returning to three.
That answer is less elegant, but much more useful. The contradiction is often the exact thing an engineer needs to resolve.
This is also where source authority matters. Production behavior may answer what the system does now. An approved decision record may answer what it should do next. Reliable context preserves both instead of collapsing them into one sentence.
Memory needs a policy, not unlimited retention
Persistent memory is attractive because teams are tired of repeating context. But remembering everything is not the same as understanding anything.
Some information should be durable: ownership boundaries, architectural decisions, established terminology, operational constraints. Some should expire: temporary incident hypotheses, short-lived rollout states, speculative plans. Some should be recomputed from current sources. Some should be deleted because the user or source system deleted it.
Useful memory therefore needs explicit rules for:
- what can become durable memory;
- which source has authority to correct it;
- how derived memories retain their evidence;
- when a memory expires or is revalidated;
- how deletion propagates through summaries and indexes.
The goal is not maximum recall. It is dependable recall.
Evaluate decisions, not demos
A context system can look impressive in a demo because the questions are known in advance and the relevant source is easy to retrieve. Production is messier. Questions are ambiguous, sources conflict, permissions vary, and the cost of a confident mistake depends on the workflow.
Evaluation should be built around real team tasks:
- Can an engineer identify the current owner and operating constraints of a service?
- Can an incident responder retrieve the latest valid runbook without exposing unrelated private material?
- Can a reviewer see the evidence behind a proposed change?
- Does the system abstain or surface conflict when the evidence does not support one answer?
- Can the team measure time saved without increasing avoidable production risk?
Answer quality still matters, but it should not be measured only by whether the final sentence sounds correct. Source selection, permission behavior, freshness, abstention, and contradiction handling are part of the result.
A practical starting point
I would not start by connecting every company system and building a universal memory layer.
I would choose one painful, bounded workflow with visible evidence and a clear owner. Incident preparation can work. So can change-risk review, support escalation, or onboarding into a specific service.
Then I would make the context contract explicit:
- List the sources required to make the decision.
- Define the authority, freshness, and permission rules for each source.
- Preserve provenance through retrieval and summarization.
- Show conflicts instead of silently resolving them.
- Build an evaluation set from real historical cases.
- Measure whether the workflow becomes faster and safer.
Once that works, the architecture can expand to adjacent workflows. The hard-won pieces - identity, source connectors, evidence tracking, invalidation, and evaluation - begin to compound.
The opportunity
Models will keep improving. Context windows will get larger. Agent frameworks will change.
Software teams will still need systems that know which evidence is current, who may see it, how conclusions were formed, and when the system should admit uncertainty.
That is durable engineering work. It sits between backend systems, data infrastructure, security, developer tooling, and product design. It is less visible than a polished assistant demo, but it determines whether those assistants become dependable parts of real work.
I am documenting my research and implementation notes in the open repository Trustworthy Agent Memory. The goal is to keep the work grounded in concrete system properties, failure modes, and evaluations rather than treating memory as a magic feature.
More text is easy to give a model. Reliable context is something a team has to engineer.