Build decisions after:
Citizens’ Assemblies in AI Governance: An Interview With Audrey Tang - Lawfare

The demo impressed the board, but the quiet cost is the undefined ownership of write operations. You now own a system that can modify production data without a clear path to stop it. The desired outcome is a defensible build sequence that proves system reliability. The actual outcome is often a rushed rollout where the engineering lead lacks the evidence to justify the risk.

The failure mode is treating citizen assembly insights as a policy document rather than an engineering constraint. Teams map public values to code without verifying if the underlying data pipeline supports those values. This disconnect creates a gap between the stated goals and the actual system behavior.

You must decide whether to build a shadow validation layer before granting any write access. This is not a feature request; it is a prerequisite for operational safety. The decision frame is clear: what to prove in shadow before any write autonomy.

How do we define success for a shadow run?

Success is defined by the system correctly identifying actions on historical data. This proves the system can distinguish between valid and invalid operations before touching live records. The shadow run must use data that is comparable to production traffic.

The mechanism is a read-only environment that mirrors the production database. The system processes historical transactions and logs its intended actions. These logs are then compared against the actual outcomes to measure accuracy.

The outcome is a baseline for system reliability. This baseline allows the engineering lead to justify the risk of moving to live operations. Without this proof, the system is just a black box making changes.

Who owns the halt decision during a pilot?

The engineering lead owns the halt decision. This role requires a clear protocol for stopping the system when it behaves unexpectedly. The halt path must be immediate and irreversible.

The mechanism is a single command that stops all write operations and reverts recent changes. This command is accessible only to the engineering lead. It ensures that the system cannot continue to modify data after a failure is detected.

The outcome is a clear path to safety. This path reduces the time-to-halt and minimizes the impact of any errors. It also clarifies the ownership of the system's behavior.

What data proves the system is safe?

Shadow mode data that is comparable to production traffic proves safety. This ensures the system handles real-world edge cases and maintains stability under actual user load. The data must include a variety of transaction types and user profiles.

The mechanism is a data sampling process that selects a representative subset of production data. This subset is then used to test the system in the shadow environment. The results are analyzed to identify any patterns of failure.

The outcome is a high degree of confidence in the system's safety. This confidence is based on empirical evidence rather than assumptions. It allows the engineering lead to defend the system's design.

Why do vague success metrics fail?

Vague success metrics that do not map to specific database transactions fail. They create a gap between the stated goals and the actual system behavior. The metrics must be specific and measurable.

The mechanism is a mapping process that links each success metric to a specific database transaction. This ensures that the system is evaluated based on its actual performance. The metrics are then used to track the system's progress over time.

The outcome is a clear understanding of the system's performance. This understanding allows the engineering lead to make informed decisions about the system's future. It also helps to identify any areas for improvement.

How do we resolve ownership ambiguity?

Ownership ambiguity between the data team and the application team must be resolved. This ambiguity creates a gap in the system's accountability. The ownership must be clear and unambiguous.

The mechanism is a formal agreement that defines the responsibilities of each team. This agreement specifies who is responsible for the system's data and who is responsible for its application. The agreement is then reviewed and updated regularly.

The outcome is a clear line of accountability. This line ensures that each team is responsible for its part of the system. It also helps to prevent any conflicts or misunderstandings.

When should we stop the pilot?

The pilot should be stopped when the system behaves unexpectedly. This behavior must be defined in advance. The criteria for stopping the pilot must be clear and objective.

The mechanism is a set of rules that define what constitutes unexpected behavior. These rules are then used to monitor the system's performance. If any of the rules are violated, the pilot is stopped.

The outcome is a clear path to stopping the pilot. This path ensures that the system is not allowed to continue to operate if it is not safe. It also helps to minimize the impact of any errors.

What is the control model for write operations?

The control model ties every write operation to a verifiable audit trail. If the system cannot prove why a change was made, the change is blocked. This ensures that the engineering lead can defend every action taken by the system.

The mechanism is a logging system that records every write operation. This log includes the reason for the change, the user who made the change, and the timestamp of the change. The log is then stored in a secure location.

The outcome is a complete record of the system's actions. This record allows the engineering lead to audit the system's performance and identify any issues. It also helps to build trust with the stakeholders.

The build sequence follows a practitioner method: Diagnose, Model, Build, Harden. First, diagnose the current state of the system and identify any gaps in the data pipeline. Next, model the system's behavior and define the success metrics. Then, build the shadow validation layer and test the system on historical data. Finally, harden the system by adding the control model and the halt path.

This week, run a shadow validation on a small subset of production data. Check if the system correctly identifies the intended actions. This proof will help you decide if the system is ready for live operations.