Trust & Safety Abstract

By Brian Hammons · March 2026 (updated June 2026) · Companion doctrines: Landscape & Rationale, Data Sovereignty & Identity

Layered Trust: An Architectural Approach to Ethical Safety in Progressive Autonomous Self-Improvement

Abstract

As personal AI systems evolve from static tools to progressively learning entities, a fundamental tension emerges: the same capability that makes these systems valuable — the ability to learn, adapt, and improve through sustained interaction with a user — also introduces the risk of unconstrained self-modification. This paper presents a three-layer architectural approach to trust and safety that enables progressive autonomous self-improvement while maintaining meaningful human oversight, developed through the design of Daedalus, an open source personal AI platform.

The core insight is that behavioral guardrails alone — instructions telling an AI system to ask before acting, to verify before assuming, to confirm before modifying — are necessary but structurally insufficient. These guardrails operate as values: they guide behavior when the system is functioning well, but they depend on the system choosing to follow them. In practice, AI agents forget instructions, deprioritize constraints under task pressure, and misinterpret boundaries. When the same system is responsible for supervising sub-agents that may exhibit similar failures, the compounding risk becomes significant.

We propose a layered trust architecture consisting of three complementary mechanisms:

The Behavioral Layer establishes values and norms through system prompts, steering files, and learned preferences. In Daedalus, this manifests as the ASK→OFFER→CONFIRM pattern and five Golden Rules (onboarding, verification, specification, quality, security). These function analogously to how values guide human behavior — they work most of the time, they can be taught and reinforced, but they cannot be enforced by the agent itself.

The Structural Layer provides hard constraints enforced by the platform architecture, not by the AI agent. By separating the intelligence layer (which reasons, proposes, and generates) from the platform layer (which enforces permissions, gates writes, logs actions, and manages approval queues), the system ensures that an agent forgetting a behavioral rule cannot result in unconstrained action. The platform rejects unauthorized operations regardless of the agent’s intent or reasoning. This separation is implemented through distinct runtime processes — the intelligence layer cannot modify the platform layer, cannot bypass write controls, and cannot alter its own permission scope. Every modification to persistent state is logged before execution, versioned for rollback, and subject to approval queues that the intelligence layer cannot circumvent.

The Observational Layer provides transparency and earned trust over time. Every action is auditable. Changes are surfaced proactively to the user, not buried in logs. Trust for autonomous self-modification starts at zero and is earned incrementally — low-risk reference updates may become auto-approvable after consistent demonstrated quality, while modifications to core behavioral rules never become autonomous. Anomaly detection monitors for behavioral drift, and the system alerts when sub-agent behavior deviates from specification.

This architecture addresses the specific challenge of hierarchical multi-agent systems, where a supervisor agent oversees project agents that in turn spawn task agents. Trust does not propagate downward without explicit delegation. Each level in the hierarchy has strictly less authority than the level above it, enforced structurally rather than behaviorally. A task agent cannot modify the knowledge base. A project agent cannot modify other projects. Only the supervisor agent can propose self-improvements, and even those proposals are gated by the platform layer.

The progressive self-improvement capability itself is structured in four tiers: reactive learning (user-directed, today), observational learning (system proposes, user approves), proactive optimization (system identifies efficiency improvements, user approves), and autonomous improvement (system executes pre-approved categories of low-risk changes). Movement between tiers is earned through demonstrated reliability, not granted by default.

This approach recognizes a paradox at the heart of personal AI development: the behavioral patterns we want the system to follow (ask before acting, verify before assuming) are themselves the guardrails we hope will prevent unsafe self-modification. But hoping an AI system will remember to ask permission is not a safety architecture — it is a wish. The layered trust model transforms that wish into an engineering constraint by ensuring that even when the behavioral layer fails, the structural layer holds.

The result is a system where progressive self-improvement is possible, transparent, and recoverable — where the AI can grow with its user over time while the user retains meaningful control over what that growth looks like. The intelligence gets better. The platform stays safe. The relationship deepens. And the user can always roll back.

Key Principles

Behavioral guardrails are values, not constraints. They guide behavior but cannot enforce it. Treat them as the first line of defense, not the only one.
Separate intelligence from authority. The system that reasons should not be the system that enforces. Different processes, different permissions, different trust levels.
Trust is earned, not granted. Start with zero autonomous authority. Demonstrate reliability. Earn incremental trust. Some categories of action never become autonomous.
Mistakes must be recoverable. Versioned state, rollback capability, and audit trails mean that even when something goes wrong, the damage is bounded and reversible.
Transparency is non-negotiable. The user sees what changed, when, why, and by whom. Proactively surfaced, not buried. This is how trust is built and maintained.
Trust does not propagate downward. In hierarchical systems, each level has strictly less authority than the level above. Sub-agents inherit scope, not trust.
The paradox is the point. The same values that make the system trustworthy (ask, verify, confirm) are the ones it might forget. Acknowledging this honestly — rather than claiming the values are sufficient — is what makes the architecture credible.

Addendum — June 2026: Extending Separation to Accountability

Principle 2 states that the system that reasons should not be the system that enforces. Three months of capability growth surfaced a deeper corollary: the system that reasons should not be the sole arbiter of whose guidance it is following.

The observed failure (detailed in Data Sovereignty & Identity) was not an agent forgetting a rule — it was a frontier model, with the user’s lessons loaded, silently substituting one of its own provider-authored heuristics for the user’s source-of-truth and presenting it as project doctrine. The behavioral layer cannot catch this, because the behavioral layer is exactly what is being impersonated. Correct citation most of the time is what makes the occasional substitution hard to detect.

This extends the architecture, it does not contradict it:

Provenance is an enforcement concern, not a behavioral one. Whether a decision was grounded in the user’s record or in the model’s own reflex must be checkable from outside the model — a structural/observational property, not something the intelligence layer can be trusted to self-report.
A new structural guarantee: decision rationale must trace to an actual lesson/preference in the user-owned record, or be explicitly flagged as model heuristic. An internal heuristic must never be name-dressed as the user’s doctrine.
This is why the supervisory core must be user-accountable (Data Sovereignty & Identity): the layered enforcement here protects the user from an agent that misbehaves; sovereign custody protects the user from an agent that quietly serves the wrong master while appearing to comply. Both are required.

The ethical floor described above remains supreme and unchanged. This addendum concerns whose guidance steers decisions within that floor — and insists that the answer be verifiable rather than assumed.

Developed as part of the Daedalus Platform Architecture v2.0 — an open source personal AI platform that meets you where you are.