Evaluation design protocol v1
Coding Agents can explore repositories, break work into tasks, write code, and revise it until the tests pass. As implementation becomes faster, it becomes harder for developers to follow every change directly. In Agent-Driven Development, clearly defining what counts as a correct result becomes increasingly important.
LLMs are good at connecting different forms of representation, including natural language, documents, code, and tests. They can turn user needs into features, translate domain language into data models, and decompose large tasks into implementable units. They are also useful in the opposite direction: reading existing code and tests to infer their purpose.
These abilities can be understood as translation within the development process. Software development repeatedly moves through the following questions:
- What change is needed?
- What must be true in the system?
- How should that state be implemented?
I understand this through Why–What–How. Why is the underlying need or purpose. What describes the state or capability that must exist when that purpose is fulfilled. How is the method used to create it.
For example, the Why might be:
Users should not lose the flow of analysis while waiting for the system.
A corresponding What could be:
Response latency should not increase sharply under concurrent requests.
Caching, request coalescing, and distributed locks are possible Hows.
LLMs are useful for expanding one Why into several Whats and proposing possible Hows for each What. At the same time, they tend to fill ambiguity with common patterns and naturally settle on one interpretation among several possibilities. The more coherent the result appears, the harder it can be to notice that the original intent has shifted.
With Coding Agents, this translation leads directly to execution. When success criteria are vague, an Agent may adjust the meaning of success together with the implementation. It may simplify a difficult requirement or change a test criterion instead of correcting the implementation, while still presenting the task as complete.
E-H-P and E-H-T provide an evaluation interface through which this translation can be reviewed.
E-H-P, the Evaluation Helix Protocol, defines how users and Agents design, realize, and revise evaluations. E-H-T, the Evaluation Helix Tree, is an intermediate representation that translates evaluation intent into technical evaluation units.
WHY represents a broad purpose close to the project vision. Several WHATs translate it into evaluable states. Under each WHAT, CASEs define how the statement will be judged under specific conditions.
A CASE is itself a How, but it is divided again into Why–What–How:
HOW_why: why this method can evaluate the statementHOW_what: the evidence to observe, such as logs, metrics, or state changesHOW_how: the rule that turns the evidence into a judgment
This decomposition connects test code to evaluation intent. It makes it possible to trace why an assertion exists, what it observes, and what it actually guarantees.
In practice, the Agent first translates the user’s evaluation request into E-H-T and presents it for review. The user confirms the meaning of the evaluation, and the approved CASEs are then implemented as tests or evidence chains. The completed evaluation system is reviewed again to confirm that it behaves as intended.
Once an evaluation is established, everyday development can proceed much like TDD: run the relevant CASEs and revise the implementation. E-H-T becomes relevant again when a new evaluation is needed, the implementation direction changes, or the relationship between the evaluation and the original purpose needs to be reconsidered.
In Agent-Driven Development, evaluation determines the direction in which an Agent iterates and the range within which it can work autonomously. E-H-P and E-H-T use the translation and decomposition strengths of LLMs while keeping purpose, evaluation, and implementation within the same semantic structure.