John Gall’s Systemantics is a satirical book that makes a serious point: complex systems fail in complex ways that are not predictable from the behavior of their components, and the standard response to system failure, add more system, reliably produces new failure modes that are more complex than the original problem.
The AI infrastructure version of this is becoming visible. A base model with concerning behaviors gets a content moderation layer. The content moderation layer has its own failure modes, so a monitoring system gets added. The monitoring system has blind spots, so an escalation process gets added. The escalation process creates latency problems, so an automated pre-screening layer gets added. Each intervention is locally reasonable. The cumulative system is a fragile tower whose failure modes are interactions between layers, not properties of any individual layer.
I’ve become cautious about complexity-layering as an alignment strategy for this reason. Not because the individual interventions are wrong, guardrails, monitoring, governance structures are all necessary, but because the cumulative system needs to be evaluated as a system, not as a sum of individually reasonable components. Gall’s warning: a complex system that works was almost always derived from a simpler system that worked. A complex system designed from scratch rarely works.
The practical implication: before adding a new layer to an AI system, the design question should be whether the problem is best addressed by addition or by simplification of the underlying objective. Most teams don’t ask the second question because simplification feels like regression. But optimizing a cleaner objective is often more robust than constraining a messy one.
