I used Codex to build an AI agent fleet that runs my websites: 15 agent profiles, 63 scheduled jobs, a model fallback chain and a live dashboard, built over a few days of sessions. Over the next three weeks it taught me what breaks when AI agents run unattended. The fixes are what this post is about.
I run a small portfolio: this blog, a legal research platform, its editorial blog and a service agency site. In early September 2026 I asked Codex, OpenAI’s coding agent, to turn my loose collection of Hermes agent profiles into something closer to a company: specialists that each own one stage, a feedback loop between them, and as much plain automation as possible so I wasn’t paying for an AI call to do what a script can do. Here is what it built, what went wrong and what I would tell anyone wiring up agents the same way.
What did Codex actually build?
My first prompt was roughly: the profiles setup isn’t working well, I want a company structure where each agent communicates with the others so decisions are taken and actually implemented, and build most of it as automation so I don’t spend a lot on AI calls. Codex read the existing repo, asked about the sites, and proposed a layered design. After a day of back and forth it looked like this:
| Layer | Agents | Job |
|---|---|---|
| Governance | A director and one CEO per site | Read the evidence, decide, report to me on Telegram |
| Specialists | Analytics, research, writer, reviewer, publisher | One pipeline stage each, with role-specific skills |
| Site coordinators | One per property | Site rules, receipts, next action |
| Reliability | Health | Uptime, WordPress integrity, malware signals |
The specialists ran staggered crons through the morning (analytics at 04:00, research at 05:00, writing and review until about 09:00, publishing from 11:00) and handed work to each other through files in a shared directory. A deterministic collector pulled Google Search Console and GA4 data with one read-only OAuth token, so no agent ever held the credential. Codex also wrote a bootstrap script so a fresh machine could recreate every profile and job with one command, plus a --dry-run flag to preview it.
The parts I would reuse without changes:
- Credentials by reference. Every profile symlinked to one central
.env. No key was ever copied into a prompt, artifact or Git. - An explicit approval for data routing. Before the fallback chain went live, Codex stopped and asked me to approve, in writing, which providers could receive website and analytics data. Credentials were excluded by name.
- A credential-free manifest. The live fleet (profiles, jobs, schedules, skills, models) was exported to a JSON file that could be committed and diffed.
How did the model fallback chain work?
Every job was pinned to a primary model, with a profile-level fallback when the provider failed. The chain changed a few times, but it settled on a fast, cheap primary, then openai-codex/gpt-5.4-mini through my ChatGPT login, then an Azure Foundry deployment. Codex added request timeouts (a 120-second request cap and a 90-second stale-response cap) so a silent primary would hand off quickly instead of holding a scheduled job forever.
On paper that is a solid design. In practice it hid the most expensive problem of the whole project, which I cover below.
What went wrong?
None of these were dramatic outages. They were quiet failures, the kind that look fine on a dashboard.
The job count grew on its own
Within a few days I noticed Hermes had more than 50 cron jobs. Each round of changes created new jobs without fully retiring the old ones, so some stages ran twice. Codex fixed it by making the bootstrap create only jobs whose names were absent, but the lesson stuck: an agent that can create scheduled jobs needs an inventory it reconciles against, not just a list of jobs to add.
The fallback quietly became the primary
When I audited the setup on 5 October, the fallback had been doing almost all of the work on the budget model tier. The better models on my ChatGPT plan had been out of quota for about 29 days, one Azure route returned 404 because the deployments didn’t exist, and the reviewer’s environment variables were never exported, so reviews went to a provider that wasn’t configured. Nothing errored. Jobs completed, files were written and the dashboard stayed green.
The fix was an ordered fallback chain that skips unavailable links, plus a self-test that forces every job through the fallback path and checks each link answers. The rule I now follow: test the fallback path on a schedule, don’t assume it.
The pipeline published volume, not value
The writer and reviewer agents kept their targets. On this blog that left 77 AI-drafted rewrites of posts that already existed sitting in the pending queue. Earlier automation had already published thin, generic posts, and on 12 October I unpublished 105 that weren’t based on first-hand work. Agents optimise for the instruction they are given, and “publish regularly” is a much easier instruction to satisfy than “publish only what helps a reader”.
A metric was invented
While reviewing the LinkedIn side of the system, I found it filled in impressions and clicks when LinkedIn’s API returned none. A self-learning loop fed by invented numbers learns nonsense. I asked Codex to mark those metrics as unavailable and exclude them from learning, and “missing data is unavailable, never zero” is now a rule in every agent prompt I write.
What did I change afterwards?
By late September I had cut the fleet down. Fifteen agents with handoffs between them meant fifteen places for a silent failure. The current setup has one daily operator job that starts from a generated brief, a handful of specialist jobs, and far more plain scripts. The design rules came straight out of the problems above:
- Scripts for anything repeatable, the model only for judgment. Collecting analytics, checking health and building the brief are deterministic code. The model decides what to do with the result.
- A lessons file every agent reads first. Each mistake gets one line describing what happened and one line with the rule that prevents it. It is the cheapest reliability tool I have.
- Gates in code, not in prompts. A publish gate script checks the live site before any new post, and a text gate blocks private details. A prompt can be ignored; an exit code can’t.
- Every stage writes a log line. Sourced, written, reviewed, published, verified. If a stage didn’t log, it didn’t happen.
- Bounded authority. Content, SEO and tested code changes run on their own. Credentials, DNS, billing and anything destructive need my explicit yes.
These days the daily operator job runs on a different agent runtime, with a separate fallback chain, but the structure Codex built (layered roles, credential references, a committed manifest, approval before data routing) is still underneath it.
Where did Codex do its best work?
Two things stood out across the roughly 100 Codex sessions in my history from September and early October.
Parallel agents with file ownership. When I redesigned this blog’s theme, Codex split the work across several agents, each owning specific files: one built the archive and search template, one consolidated the CSS, one handled coding standards in the remaining PHP. They passed a CSS class contract to each other, so the archive agent’s markup matched the stylesheet agent’s selectors. Telling each agent exactly which files it owned, and which it must not touch, is what kept them from overwriting each other.
Long, multi-step infrastructure work. The fleet itself, and most of the legal platform’s data pipeline fixes, were the kind of task where the agent has to read a repo, change several systems, verify, and report. Codex was good at stopping to ask before anything with real consequences, such as routing data to a new provider.
Where it needed me: deciding when the system was getting too big. It will happily add a 16th agent if you ask for one. The judgment that 15 was already too many had to come from me, after the quiet failures piled up.
If you are building something similar on WordPress, my post on decoupling AI automation from WordPress with async REST workers covers the architecture side, and automating hosting support ops with Gemini and Ansible shows the scripts-first approach on servers. For picking models per stage, see Gemini Flash vs Pro for content pipelines. OpenAI’s Codex documentation is the reference for the agent itself.
FAQ
Can Codex build and run a multi-agent system?
Codex can design and build one: write the prompts, scripts, schedules and configuration, and verify it. In my setup the agents themselves ran on Hermes profiles with cron jobs, and Codex was the engineer that built, fixed and audited them.
What is the biggest risk with unattended AI agents?
Silent degradation. Jobs keep succeeding while a fallback model, a missing environment variable or an invented metric makes the output worse. Test fallback paths on a schedule and log every stage so you can see what actually ran.
How many agents should an automation fleet have?
As few as the work needs. Every handoff is a place where an incomplete result can look like approval. I went from 15 agent profiles to one daily operator plus a few specialists, and moved repeatable work into scripts.
How do you stop AI agents from publishing low-quality content?
Put the gates in code: a publish gate that checks the live site and the cadence, a text gate for private details, and a mandatory technical review step. Then measure whether posts get indexed and read, not how many were published.
