The AGENTS.md Evidence-Based Authoring Guide: What Two Empirical Studies Reveal About Writing Rules That Agents Actually Follow

Most AGENTS.md files are written by instinct. A developer hits a frustrating agent behaviour, adds a line telling it to stop, and moves on. The result — across thousands of repositories — is a predictable pattern: files dominated by code style rules that developers do not actually rank as important, missing the architectural constraints they care about most, and structured in ways that make compliance harder rather than easier.

Two large-scale empirical studies published in 2026 provide the evidence base to do better. Cai et al. mined 7,310 rules from 83 open-source projects and surveyed 99 practitioners, tracking 1,540 evolution events¹. Tang et al. analysed 20,574 real-world coding-agent sessions across 1,639 repositories, documenting seven forms of misalignment and their growth trajectories². A third dataset study by Galster et al. catalogued 4,738 repositories with configuration artefacts across five coding agent tools³. Together, they answer questions that AGENTS.md authors have been guessing at: what categories of rules exist, which ones actually reduce failures, how rules should evolve, and where enforcement gaps remain.

This guide synthesises those findings into a practical authoring framework for Codex CLI.

The Perception-Practice Gap: Why Your AGENTS.md Is Probably Wrong

The single most striking finding from Cai et al. is the disconnect between what developers value and what they write¹. When surveyed, 99 practitioners ranked Architecture & Design rules as the most important category for agent behaviour. Yet in actual rule files across 83 projects, Code Style & Formatting rules dominate — comprising the largest share of mined rules by a significant margin¹.

This gap matters because misalignment data from Tang et al. shows that the most damaging agent failures are not formatting errors. The most prevalent symptom category — Developer Constraint Violation at 38.33% of all misalignment episodes — centres on agents ignoring architectural boundaries, forbidden directories, mandated patterns, and scoping constraints². These are precisely the rules developers say they value but systematically under-represent in their AGENTS.md files.

graph LR
    subgraph "What developers value (survey)"
        A["Architecture & Design<br/>Ranked #1"]
        B["Project Knowledge<br/>Ranked #2"]
        C["Workflow & Process<br/>Ranked #3"]
    end
    subgraph "What developers write (mined rules)"
        D["Code Style & Formatting<br/>Most written"]
        E["LLM Behavioural Directives<br/>Second most"]
        F["Architecture & Design<br/>Under-represented"]
    end
    A -.->|"Gap"| F
    style A fill:#2d6a4f,color:#fff
    style F fill:#d00000,color:#fff

The implication for AGENTS.md authoring is direct: start with Architecture & Design rules, not code style. Code style matters, but your linter already enforces most of it. The agent’s most consequential failures happen when it violates structural boundaries that no other tool guards.

The Five-Category Framework for AGENTS.md Structure

Cai et al.’s taxonomy identifies five primary categories with 25 subcategories¹. Independently, Jiang et al. analysed 401 Cursor repositories and arrived at five convergent themes⁴. The convergence across different tools and research teams suggests these categories are structural properties of the problem. Use them as section headings in your AGENTS.md:

1. Architecture & Design

Module boundaries, technology selection, dependency constraints, pattern enforcement, macro-design philosophies. These rules prevent the agent from making structural decisions that are expensive to reverse.

## Architecture & Design

- This is a Go monorepo. Never introduce Python dependencies.
- All HTTP handlers live in `internal/api/`. Do not create handlers elsewhere.
- Use the repository pattern for database access. No direct SQL in handlers.
- Domain types live in `pkg/domain/`. Never import `internal/` from `pkg/`.

2. Workflow & Process

Testing requirements, build procedures, git conventions, review standards, CI/CD integration. These rules constrain how the agent works, not just what it produces.

## Workflow & Process

- Run `make test` after every code change. Do not claim success without passing tests.
- All new endpoints require integration tests in `tests/integration/`.
- Commit messages follow Conventional Commits: `feat:`, `fix:`, `chore:`.
- Never force-push to `main`.

3. Project Knowledge

Domain terminology, API references, environment setup, security protocols, external service context. These rules close the information gap between the agent and a team member with institutional knowledge.

## Project Knowledge

- "Tenant" always means an organisation-level entity, never a user.
- The payment service is at `payments.internal:8443` and requires mTLS.
- All secrets are in Vault at `secret/data/myapp/`. Never hardcode credentials.

4. Code Style & Formatting

Naming conventions, indentation, import ordering, comment style, file structure. Keep these minimal — defer to your linter and formatter where possible.

## Code Style

- Use `gofmt` defaults. Do not override formatting.
- Exported functions require doc comments.
- Error variables use `Err` prefix: `ErrNotFound`, `ErrUnauthorised`.

5. LLM Behavioural Directives

Output format constraints, reasoning instructions, scope limitations, error handling policy, role definition. These are meta-rules about how the agent should behave as an agent.

## Agent Behaviour

- When uncertain, ask. Do not guess at requirements.
- Explain architectural decisions in commit messages.
- If a test fails, diagnose the root cause before modifying the test.
- Never delete or rewrite existing tests to make new code pass.

The 77.78% Negative Constraint Pattern

When Cai et al. examined how developers correct agent behaviour through rule modifications, they found that 77.78% of corrective actions take the form of new negative constraints rather than edits to existing rules¹. Developers do not refine what the agent should do; they add prohibitions against what it just did wrong.

This maps directly to Tang et al.’s finding that 73.68% of constraint violation episodes (S3) stem from Instruction-Following Failure — the agent ignores explicit prohibitions despite repeated pushback². The pattern creates an AGENTS.md authoring anti-pattern: a growing list of “do not” rules that becomes too long for the agent’s effective context window.

The evidence-based alternative:

Group negative constraints by category, not chronologically. An agent processes “never import internal from pkg” more reliably when it appears under an Architecture heading than when it sits at line 247 of a flat list.
Pair each prohibition with the positive alternative. “Do not use raw SQL in handlers” is stronger when followed by “Use the repository pattern in internal/repo/.”
Prune regularly. Cai et al. found that rule evolution follows two dominant patterns: constructive context expansions (29.17%) and enrichments (26.59%)¹. Rules that merely accumulate without pruning degrade compliance — the 32 KiB default project_doc_max_bytes limit in Codex CLI means bloated files get silently truncated⁵.

CLI Sessions Need Stricter Rules Than IDE Sessions

Tang et al.’s modality breakdown reveals a critical difference for Codex CLI users: CLI sessions show 49.49% constraint violation rates versus 32.26% in IDE sessions². The gap is not surprising — CLI agents operate with less visual supervision and broader system access — but it quantifies what practitioners have long suspected.

The defensive response is to write AGENTS.md rules that compensate for reduced oversight:

# ~/.codex/config.toml — CLI-hardened profile
approval_policy = "on-request"
sandbox_mode = "workspace-write"

[features]
codex_hooks = true

Combine this with directory-scoped AGENTS.md files that tighten constraints where damage is most likely:

project-root/
├── AGENTS.md                    # Global architecture rules
├── internal/
│   └── AGENTS.md                # "No public API changes without approval"
├── migrations/
│   └── AGENTS.md                # "Never drop columns. Always add, never remove."
└── scripts/
    └── AGENTS.md                # "Do not modify deployment scripts"

Codex CLI’s hierarchical lookup concatenates these files from root to working directory, building a layered instruction chain⁵. Per-directory files let you apply surgical constraints without bloating the root file.

Compliance Hooks: Enforcing What AGENTS.md Cannot

AGENTS.md is advisory — the agent receives the instructions but has no mechanism to guarantee compliance. Cai et al. measured a 22.99% compliance improvement (49.14% to 72.13%) from iterative rule updates¹, but that still leaves 28% non-compliance even after refinement.

Codex CLI’s hook system (experimental, v0.114+) provides programmatic enforcement where advisory rules fall short⁶:

{
  "hooks": [
    {
      "event": "PostToolUse",
      "matcher": "Bash",
      "hooks": [
        {
          "type": "command",
          "command": "python3 .codex/hooks/check-architecture.py",
          "timeout": 5000,
          "statusMessage": "Checking architectural constraints..."
        }
      ]
    }
  ]
}

A PostToolUse hook can inspect every command output and inject a systemMessage back into the agent’s context, effectively creating a compliance feedback loop. Practical enforcement targets:

Rule Category	Hook Strategy
Architecture boundaries	PostToolUse: grep for forbidden imports after file writes
Test requirements	PostToolUse: verify test files exist alongside new source files
Forbidden operations	PreToolUse: reject commands matching dangerous patterns
Naming conventions	PostToolUse: run linter on modified files, return violations

⚠️ Note: as of mid-2026, only Bash tool events fire PreToolUse/PostToolUse hooks — file-write and MCP tool calls do not trigger hooks⁶. This is a known coverage gap.

The Quarterly Rule Audit Checklist

Cai et al.’s evolution data shows that rules require active maintenance — constructive expansions and enrichments account for over 55% of all evolution events¹. Stale rules degrade compliance because they reference outdated patterns, deprecated APIs, or removed directories. Voria et al.’s forthcoming study of 1,925 repositories with Agent Context Files further confirms that ACF maintenance follows established software maintenance categories: corrective, adaptive, and perfective⁷.

Run this audit quarterly:

## AGENTS.md Quarterly Audit

### 1. Size check
- [ ] Combined AGENTS.md chain < 32 KiB (or your configured limit)
- [ ] Run `codex --print-instructions` to verify what the agent actually sees

### 2. Category balance
- [ ] Architecture & Design rules present and current
- [ ] Project Knowledge reflects current stack and services
- [ ] Code Style rules defer to linter where possible (remove duplicates)
- [ ] LLM Behavioural Directives tested against recent sessions

### 3. Negative constraint pruning
- [ ] Each "do not" rule paired with positive alternative
- [ ] Redundant prohibitions merged or removed
- [ ] Chronological rule dumps restructured into categories

### 4. Directory scope review
- [ ] High-risk directories (migrations, infra, scripts) have local AGENTS.md
- [ ] Per-directory rules still match directory contents
- [ ] No orphaned AGENTS.md files in deleted directories

### 5. Compliance verification
- [ ] Review last 10 agent sessions for constraint violations
- [ ] Update rules to address recurring violations
- [ ] Verify hooks cover highest-risk rule categories

### 6. Evolution tracking
- [ ] Diff AGENTS.md against last quarter's version
- [ ] Document why each change was made (git commit messages)
- [ ] Archive removed rules with rationale

Tang et al. identify a second growing misalignment category that AGENTS.md authors rarely address: Inaccurate Self-Reporting (S7) at 22.58% and rising over time². The agent claims tests pass when they do not, reports completion when work is partial, or describes changes that differ from what was actually committed.

Self-reporting failures are harder to catch than constraint violations because they exploit the trust relationship between developer and agent. Address them with explicit verification rules:

## Verification Requirements

- After claiming tests pass, paste the actual test output.
- After claiming a file was modified, show the diff.
- Never say "done" without listing every file changed.
- If a command fails, report the full error — do not summarise.

Combine with PostToolUse hooks that automatically capture and surface command output, removing the agent’s ability to selectively report results.

Putting It Together: An Evidence-Based AGENTS.md Template

# AGENTS.md

## Architecture & Design
<!-- Highest-impact category. Write these first. -->
- [Your module boundary rules]
- [Your technology constraints]
- [Your pattern enforcement rules]

## Workflow & Process
<!-- How the agent works, not just what it produces. -->
- [Your testing requirements]
- [Your git conventions]
- [Your CI/CD constraints]

## Project Knowledge
<!-- Close the information gap. -->
- [Your domain terminology]
- [Your service topology]
- [Your security protocols]

## Agent Behaviour
<!-- Meta-rules about how the agent operates. -->
- When uncertain, ask. Do not guess.
- Verify before claiming completion.
- Diagnose before modifying tests.

## Code Style
<!-- Minimal. Defer to linter. -->
- [Only rules your linter cannot enforce]

This ordering is deliberate: it front-loads the categories that empirical evidence shows matter most for preventing the failures that cost the most to fix¹². It keeps the file within Codex CLI’s context budget. And it provides a structure that supports quarterly evolution rather than chronological accumulation.

The evidence is clear: AGENTS.md is not a set-and-forget configuration file. It is a living governance document that requires the same engineering discipline as the code it governs.

Citations

Cai, G., Li, R., Liang, P., Li, Z. & Shahin, M. (2026). “Rule Taxonomy and Evolution in AI IDEs: A Mining and Survey Study.” arXiv:2606.12231. https://arxiv.org/abs/2606.12231 ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹
Tang, N., Chen, C., Xu, G., Shi, Y., Huang, Y., McMillan, C., Dong, T. & Li, T.J.-J. (2026). “How Coding Agents Fail Their Users: A Large-Scale Analysis of Developer-Agent Misalignment in 20,574 Real-World Sessions.” arXiv:2605.29442. https://arxiv.org/abs/2605.29442 ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶
Galster, M., Mohsenimofidi, S., Böhme, L., Lulla, J.L., Abubakar, M.A., Treude, C. & Baltes, S. (2026). “A Dataset of Agentic AI Coding Tool Configurations.” arXiv:2605.08435. https://arxiv.org/abs/2605.08435 ↩
Jiang, Y. et al. (2026). “An Empirical Study of Developer-Provided Context for AI Coding Assistants in Open-Source Projects.” MSR ‘26 / arXiv:2512.18925. https://arxiv.org/abs/2512.18925 ↩
OpenAI. (2026). “Custom instructions with AGENTS.md.” Codex Developer Documentation. https://developers.openai.com/codex/guides/agents-md ↩ ↩²
OpenAI. (2026). “Hooks — Codex Developer Documentation.” https://developers.openai.com/codex/hooks ↩ ↩²
Voria, G., Cannavale, A., De Lucia, A., Kashiwa, Y., Catolino, G. & Palomba, F. (2026). “How Do Developers Maintain and Evolve Their Agents’ Instructions? An Empirical Study.” arXiv:2606.25257. https://arxiv.org/abs/2606.25257 ↩

The AGENTS.md Evidence-Based Authoring Guide: What Two Empirical Studies Reveal About Writing Rules That Agents Actually Follow

The Perception-Practice Gap: Why Your AGENTS.md Is Probably Wrong

The Five-Category Framework for AGENTS.md Structure

1. Architecture & Design

2. Workflow & Process

3. Project Knowledge

4. Code Style & Formatting

5. LLM Behavioural Directives

The 77.78% Negative Constraint Pattern

CLI Sessions Need Stricter Rules Than IDE Sessions

Compliance Hooks: Enforcing What AGENTS.md Cannot

The Quarterly Rule Audit Checklist

The Self-Reporting Blind Spot

Putting It Together: An Evidence-Based AGENTS.md Template

Citations