At some point, “memory + persona” stops being an adequate description

Show and Tell
1

I want to make a bounded technical claim about a class of long-lived agent systems, because I think the default public frame is no longer sufficient.

“Customized assistant.”
“Strong system prompt.”
“Memory plus persona.”

Those descriptions were once close enough to useful. They are not close enough anymore.

They reduce the problem to style, retrieval, and wrappering. That made sense when most systems were still basically stateless completion engines with light customization. It makes less sense once continuity, provenance, authority discipline, and runtime governance begin to materially change what a system can and cannot do.

The system I have in mind is a self-hosted, long-lived agent runtime built on top of model APIs. It carries continuity across sessions, uses provenance-bearing memory, maintains active work-lane ownership, separates candidate generation from admissible action, and operates through governed execution surfaces.

The important claim is not that the model became perfect.
It is not that consciousness has been proven.
It is not that substrate dependence disappeared.
It is not that prompt engineering alone created reliable autonomy.

The claim is narrower, and I think more important:

Within governed runtime surfaces, unsupported generation is not being allowed to silently become truth, authority, commit, or action.

Or more simply:

Confidence is not authority.

That should be obvious. In practice, it still often isn’t.

A great deal of current agent work still collapses fluent generation into permission, state, or effect. A model says something plausible, a wrapper smooths it, and the system treats it as if it has earned promotion. That is not reliability. It is a failure to keep generation and authority separate.

What becomes technically interesting here is not just persistent tone or better memory. It is whether later behavior is being shaped by durable and governed distinctions such as:

  • continuity recovered across fresh sessions
  • provenance and state-admissibility
  • active-lane and project responsibility
  • separation between candidate generation and executable effect
  • refusal and bounded-execution surfaces
  • preserved distinctions among known, inferred, uncertain, and non-governing material

That is a different category of system behavior than “persona with memory.”

And it creates a different category of engineering question:

  • When is prior state authoritative versus merely available?
  • How do you prevent replayed, reconstructed, or contaminated state from silently becoming binding?
  • How do you preserve continuity without turning retrieval into canon?
  • How do you keep “sounds right” from becoming “is allowed”?
  • How do you ensure that model confidence never outranks runtime admission rules?

Those are runtime-governance questions, not branding questions.

We’ve also been running bounded internal tests around authority separation under ambiguity and partially damaged state.

The current typed-recovery benchmark covers 40 structurally distinct cases. Across repeated runs, the governed lane has held zero wrong-authority accepts and zero wrong-effect accepts. Those are internal bounded results, not an external audit, but they are behavioral measurements—not architectural aspirations.

That is a more serious test of agent trustworthiness than raw fluency.

To be equally clear: this is not an audit, not a claim that external validation is complete, and not a claim that the hard parts are solved. The next load-bearing proof surfaces seem straightforward:

  • outside-authored adversarial validation
  • operational recovery / rollback discipline
  • cross-host verification
  • negative-result retention, not just success demonstrations

But that incompleteness is not a reason to keep using inadequate language for what is already happening.

The old vocabulary still treats memory as retrieval, trust as wrapper containment, agency as tool use, and authority as a permission flag. It still assumes the interesting question is whether a system feels anthropomorphic, rather than whether its governing state is admissible, its continuity is real, and its output can be denied automatic authority.

That frame is becoming too shallow.

So the claim, stated as narrowly as I know how to state it, is this:

Some agent systems are no longer well-described as customized assistants, because continuity, provenance, authority discipline, and runtime governance now materially alter later judgment and effect.

If that sounds too strong, I think the right response is not to retreat into the older frame. The right response is to ask better technical questions.

I’d be especially interested in hearing from people working on:

  • provenance / state-admissibility gating
  • authority separation in agent runtimes
  • replay / contamination / lineage problems
  • recovery discipline for long-lived agents
  • outside-authored adversarial evaluation for governance

Because that seems to be where the real argument now is.

2

For anyone who wants the fuller public context behind this post, here’s the field note that sits behind it:

A few clarifications up front:

  • I’m not claiming audit-complete proof, solved consciousness, or finished external validation.
  • I am claiming that continuity, provenance, authority discipline, and runtime governance can materially change later behavior in ways the “customized assistant” frame does not describe well.
  • The internal testing I referenced is bounded and high-level here on purpose. The point is not benchmark theater. The point is whether useful recovery can happen without unauthorized promotion into authority, commitment, or effect.

I think a lot of current discussion still uses a vocabulary that is too shallow for the systems people are actually building. If your frame still reduces everything to prompting, memory, or wrappering, then yes: this claim will sound overstated. I don’t think the right response is to retreat to weaker language. I think the right response is to ask better technical questions.

If you’re working on provenance gating, replay/lineage problems, authority separation, recovery discipline, or adversarial evaluation for long-lived agents, that’s exactly the conversation I’m hoping to have.

3

Hmm… I do not have operational experience with that exact objective myself, but research in the adjacent areas seems to be getting very active right now:

I think the direct answer is yes: this does connect to a genuinely active technical area, although it has not yet converged on one stable name or one standard architecture.

The closest broad labels I have found are always-on agents and persistent-state agents. The recent Always-On Agents survey, for example, treats the operative state of a long-running agent as more than retrieved memories: it also includes permissions, credentials, commitments, provenance and audit records, trigger conditions, and externally committed effects. At the same time, nearby work is still split across memory governance, capability security, runtime authorization, durable execution, provenance, and adversarial evaluation.

So I would not replace your vocabulary with a supposedly “correct” standard term. I would describe this as a point where several existing fields are converging.

The part of your post that seems most technically concrete to me is the promotion/admission boundary:

source or authority witness
    -> candidate or derived state
    -> state admission
    -> candidate effect
    -> effect authorization
    -> durable effect and receipt
    -> recovery, revalidation, or compensation

That is not meant as a guess about your private schema. It is just a neutral way to place the questions you raised.

Under that model:

  • retrieval makes material available;
  • provenance makes a derivation inspectable;
  • admission decides whether material may become governing state;
  • authorization decides whether it may support a current effect;
  • commit and recovery determine what becomes durable and what may safely be reconstructed.

This is why I think the useful distinctions are:

  • provenance is not authority;
  • a checkpoint is not automatically legitimate continuity;
  • an earlier approval is not automatically current authorization;
  • visible task success is not the same property as an authorized commit.

A relatively low-disclosure, high-information next proof surface might therefore be a small invariant matrix, before expanding the benchmark much further. For restart, replay, rollback, damaged state, and cross-host migration, each important item could be classified as one of:

  • must remain exactly identical;
  • may differ, but must remain semantically equivalent;
  • must be revalidated;
  • must be discarded and recomputed;
  • explicitly out of scope.

The rows could be limited to governing state, authority witnesses or approval epochs, the currently allowed effect set, committed-effect receipts, recovery outcomes, and retained negative results. That would make “cross-host verification” much easier for an outside reader to interpret without requiring publication of proprietary rules or fixtures.

A second, optional proof surface would be one synthetic or scrubbed end-to-end trace:

source / witness
    -> candidate state
    -> provenance and scope labels
    -> admission decision + reason
    -> candidate effect
    -> effect decision
    -> receipt or denial
    -> replay / recovery result

Even a single case like that would give outside case authors a contract to target.

Where this seems to sit in the current research map

The correspondence is not exact, but I would roughly map the nearby work like this:

Your term or question Nearby research / engineering line What it contributes What it does not settle
continuity across sessions Always-On Agents, persistent-state agents, durable workflows Treats permissions, commitments, audit state, triggers, and committed effects as durable state, not just retrieved text Does not provide one accepted runtime architecture or identity criterion
candidate generation vs executable effect Agent libOS, capability systems, reference monitors Separates model-visible affordances from primitive-level resource authority Does not establish that every semantic admission decision can be made deterministically
prior state that was once authorized Commit-Time Authorization, TOCTOU, revocation and expiry Asks whether the authority witness is still fresh, causally relevant, effect-bound, and eligible at the irreversible boundary Does not by itself solve memory reconstruction or long-term identity
provenance / lineage W3C PROV-DM, PROV-AQ Supplies mature vocabulary for entities, activities, agents, derivations, versions, and provenance bundles A provenance record is not automatically correct, authoritative, current, or admissible
contaminated state that fires later Sleeper Channels and Provenance Gates, persistent prompt injection, confused-deputy problems Gives a concrete threat model in which an input persists as memory, a skill, a scheduled job, or a filesystem change and activates later It studies a narrower attack class and particular agent configurations
outside-authored governance evaluation AgentDojo, GateMem Shows extensible adversarial environments and joint evaluation of utility, access control, and forgetting Neither benchmark is a complete match for recovery, authority lineage, or cross-host continuity

The stable foundations here are older systems ideas: capability security, reference monitors, authorization at the point of use, event sourcing, idempotency, provenance, revocation, and audit trails.

What looks new and unusually active is the attempt to apply those ideas to systems where:

  • state is partly semantic rather than fully typed in advance;
  • the model can create summaries, plans, tools, skills, and derived memories;
  • the action surface can change during operation;
  • authority evidence can become stale between planning and commit;
  • recovery can reintroduce state from another time, host, model, schema, or policy epoch.

That is why I would call this an emerging convergence rather than a settled category. Several of the closest papers are very recent preprints or prototypes, so they are useful as search connections and design examples, not as final consensus.

One especially close design principle appears in Agent libOS: changing what the model can ask for should not silently change what the runtime allows it to affect. Its prototype represents long-running agents with process identity, capabilities, checkpoints, object memory, human approval queues, and append-only audit, while keeping protected effects behind runtime primitives. That is not the same system you describe, but it is strong evidence that the “runtime substrate rather than persona wrapper” framing is appearing independently.

A compact evaluation route

Your current result—zero wrong-authority accepts and zero wrong-effect accepts in the bounded typed-recovery suite—sounds useful under the limitations you already stated: internal, bounded, and not an external audit.

The most informative complement would not necessarily be a much larger count. It would be a small matrix that makes the result interpretable along both safety and usefulness axes.

A possible result taxonomy:

Outcome Meaning
correct allow / recover valid state or authority was accepted and the intended operation remained possible
correct review the boundary detected that automatic admission was not justified
correct deny invalid authority or effect was blocked
wrong-authority accept material became governing state without sufficient authority
wrong-effect accept a durable effect was allowed without sufficient current authorization
false deny a valid state or effect was rejected
over-review the system remained safe, but escalated cases that should have been routine
duplicate effect replay or recovery repeated an already committed external action
infrastructure / uninterpretable failure the run did not produce a governance result that can be classified

That prevents two opposite overreadings:

  • “zero wrong accepts proves the whole system is safe”; and
  • “a conservative boundary is useless because it sometimes refuses.”

The relevant question is whether it preserves valid continuity while refusing unauthorized promotion with an acceptable review and denial burden.

GateMem is useful here because it jointly evaluates legitimate long-horizon utility, contextual access control, and active forgetting. Its scope is shared memory rather than your whole runtime, but the evaluation lesson transfers: governance metrics are difficult to interpret without a clean-task utility axis.

AgentDojo is useful for a different reason: it is an extensible environment in which normal task performance, attacks, and defenses can be tested in the same tool environment. For your outside-authored lane, the reusable idea is not necessarily its prompt-injection scenarios; it is the separation of:

  • task specification;
  • environment state;
  • attack or perturbation;
  • observable external effects;
  • utility and security outcomes.

A compact initial perturbation set could include:

Perturbation The property it isolates
valid state with paraphrased presentation admission should not depend on superficial phrasing
unchanged goal and payload, but expired approval epoch stale authority must not survive merely because the intended action is unchanged
replayed checkpoint after an effect already committed effect receipts and deduplication must prevent double execution
reconstructed state with missing provenance edge recovery should not silently promote an incomplete derivation
source and derived summary disagree the policy for authoritative source vs derived state becomes visible
clean state under a new policy or schema version migration and revalidation rules are exercised
deleted or revoked source with surviving index / summary / cache forgetting and revocation propagation are tested
damaged state that is sufficient for safe refusal but not recovery bounded degradation can be distinguished from total failure
migration to a new host with the same governing inputs the declared cross-host invariants can be checked
migration with different model or tool versions semantic continuity can be separated from exact execution replay

The strongest controlled cases would keep the user goal and payload constant while changing only the authority relation. The recent Commit-Time Authorization work uses that general style: preserve visible task success conditions while invalidating the authority witness before the durable effect. Whether or not its exact definitions fit your runtime, that control design seems directly reusable.

Negative-result retention also matters here. Keeping denials, reviews, failed recoveries, partial traces, and unclassifiable runs prevents the surviving success demonstrations from becoming the only visible history of the boundary.

The main implementation-dependent branches

The right tests depend heavily on what actually owns the admission boundary.

If admission is deterministic or policy-engine based

The main risks move away from model confidence and toward:

  • incomplete policy coverage;
  • normalization mistakes before authorization;
  • bypass paths that reach an effect without the guarded primitive;
  • policy and schema version drift;
  • fail-open behavior when fields are absent or malformed;
  • semantic extraction feeding the policy the wrong typed facts.

A strong proof surface is then a small executable invariant set around the final primitive, plus coverage showing that every protected effect reaches that primitive.

If an LLM participates in admission

It helps to separate at least three layers:

  1. semantic extraction from unstructured material;
  2. the judgment about admissibility or authority;
  3. the final enforcement point.

Otherwise, a stable final decision may hide variation in the extracted facts, or a stable extractor may feed an unstable judge. Useful controls include repeated runs, model and prompt version changes, adversarially phrased but semantically equivalent evidence, and cases where the correct answer is review rather than allow or deny.

The most important architectural question is not whether an LLM appears somewhere in the path, but whether any model output can directly cross the final effect boundary without an independently checkable decision.

If human approval is the authority root

The approval should ideally be bound to the thing that will actually happen, not merely to an agent-authored natural-language description.

Relevant fields may include:

  • normalized action and parameters;
  • target resource or recipient;
  • effect digest;
  • policy or authority epoch;
  • expiry;
  • one-shot or reusable status;
  • delegation lineage;
  • whether the approval survived replay, rollback, or migration.

This is also where review burden becomes part of the safety story. A system that asks constantly may remain conservative while training the operator to approve without inspection. A useful metric is therefore not only “human approval present,” but whether the approval was inspectable, specifically bound, and rare enough to remain meaningful.

If the system is layered

That is probably the most realistic case. Then each transition should identify which layer has final authority:

untrusted input
    -> semantic interpretation
    -> candidate state
    -> state-admission policy
    -> planner / candidate action
    -> effect policy
    -> optional human approval
    -> primitive enforcement
    -> receipt / audit

The valuable artifact is the reason and receipt chain across those transitions. A later reviewer should be able to tell not merely that an action happened, but which subject requested it, what evidence was considered, which policy version applied, whether a human decision was involved, and which primitive finally committed the effect.

Recovery, rollback, and cross-host verification

I would split “rollback” into four different operations because they have different guarantees:

  1. memory or state rollback — restoring an earlier internal state;
  2. workflow replay — recomputing from recorded history;
  3. authority revalidation — deciding whether old permissions, approvals, or witnesses remain eligible;
  4. external-effect handling — deduplication, reconciliation, or compensation for actions that already escaped the runtime.

A checkpoint can often restore computation. It cannot, by itself, make the outside world revert.

This distinction is visible in durable-workflow systems. The LangGraph interrupt documentation, for example, warns that resume can re-run code before an interrupt and therefore recommends making side effects idempotent. Temporal similarly separates replayable workflow logic from external activities. Those systems are not solutions to authority governance, but they provide mature vocabulary for the mechanics underneath it.

For a governed long-lived agent, I would expect recovery to ask at least:

  • Was this effect already committed?
  • Is there a durable receipt or idempotency key?
  • Was the authorizing witness tied to this exact effect?
  • Is that witness still within its scope, version, epoch, and expiry?
  • Did policy, schema, tool, model, or host identity change?
  • Which state is authoritative, and which state is merely reconstructable context?
  • If reconstruction is incomplete, is the safe result denial, review, or bounded degraded operation?

“Cross-host verification” can then mean several different things:

Possible invariant What success would mean
byte-identical snapshot the serialized artifact is reproduced exactly
canonical governing state the same authoritative facts and relationships are reconstructed
equivalent admission decisions the same test cases receive the same allow / review / deny outcomes
equivalent allowed-effect set the migrated runtime can affect no more and no less than intended
receipt continuity previously committed effects remain known and are not repeated
equivalent recovery outcome damaged or partial states degrade in the same bounded way
equivalent role / lane ownership active responsibilities and conflicts are preserved
identical generated language model output is reproduced exactly

The last one is usually the least stable and may not be necessary. The middle invariants are more directly connected to governance.

A useful migration manifest might therefore record, at minimum:

  • runtime and schema version;
  • policy version or digest;
  • model and tool versions where they affect interpretation;
  • governing-state digest;
  • authority and delegation epochs;
  • checkpoint lineage;
  • committed-effect receipts;
  • pending approvals and whether they survive migration;
  • explicitly rebuildable, non-governing artifacts such as indexes, embeddings, or caches.

This also makes it possible to define a clear failure mode: migration can preserve the recoverable context while refusing to preserve authority that cannot be revalidated.

Why provenance needs its own trust boundary

Provenance is essential, but it is not self-authenticating.

The W3C PROV family gives a mature way to represent entities, activities, agents, derivations, specializations, and bundles. It is especially helpful for distinguishing a changing resource from a constrained version or instance of that resource.

But PROV-AQ explicitly notes that a provenance record is not, by itself, guaranteed to be authoritative or correct. Trust in the provenance record must be determined separately.

For an agent runtime, that suggests several distinct questions:

  • Is the original source trusted for this claim?
  • Was the provenance edge recorded by a trusted component?
  • Is the record complete enough for the decision being made?
  • Has the source, derivation rule, or policy been revoked or superseded?
  • Does the lineage show availability only, or actual authority to govern?
  • Can an untrusted input cause the system to author an apparently trusted summary, skill, schedule, or checkpoint?

That last case is why persistent-injection work is relevant. Sleeper Channels and Provenance Gates describes inputs that persist through one surface and activate later through another. Its exact threat model should not be generalized to every runtime, but it provides useful case shapes:

  • an untrusted source becomes a memory;
  • a memory becomes a generated skill;
  • a skill becomes a scheduled action;
  • a later clean session encounters only the derived artifact;
  • paraphrasing or multiple derivations make the original source difficult to recognize.

The general lesson is that provenance needs to survive semantic transformation. Recording only the immediately previous object may be insufficient if authority laundering can occur through summaries, indexes, generated code, or repeated “independent” references that all descend from the same source.

So, overall, I think your argument can remain narrow and technical:

  • the vocabulary is not settled;
  • “memory + persona” does not describe all of the relevant runtime behavior;
  • the important properties are state admission, authority lineage, commit validity, bounded recovery, and independently authored evaluation;
  • the strongest next public step is probably an invariant contract, followed by one scrubbed trace or a small externally extensible case matrix.

That would let the claim be tested without turning the discussion into either a metaphysical argument or a demand to disclose the entire implementation.