What Building Taught Me That Reading Didn’t

I’ve been writing about AI and design for two years. I’ve also been building — a VQA workflow, a capability framework, internal education infrastructure. The writing and the building have not produced the same knowledge, and the differences are worth being explicit about.

Reading and writing produced conceptual frameworks. I could explain interaction contracts, capability surfaces, calibration as a design discipline. These frameworks are useful. They’re also cheap to produce and easy to hold at a level of abstraction that insulates them from falsification.

Building produced something different: specific, unwelcome surprises that forced revision.

The surprise from the VQA workflow: the bottleneck was shared definition of failure, not model capability. I would not have arrived at that through reading. Every paper I read about AI evaluation assumes the evaluation criteria are given. In practice, establishing the criteria is most of the work, and it’s work that can’t be automated.

The surprise from the maturity framework: AI capability isn’t linear and incentives matter more than clarity. I would not have predicted either of those from first principles. I designed the framework with a reasonable model of how people develop skills and a naive model of what determines organizational behavior.

The pattern: my conceptual frameworks were generated through reading and were reasonably coherent. My specific beliefs — the ones that make contact with how things actually work — were mostly generated through building and were frequently wrong in the first version.

This has practical implications for how I think about AI design work going forward. Frameworks developed without operational grounding will be coherent and wrong in specific ways that only show up when someone tries to use them. The design discipline I’m most interested in developing is the one that connects conceptual frameworks to operational realities fast enough that the revision cycle is short.

That’s not a reading practice. It’s a building practice with writing attached.