--- title: Hermes-as-COO Architecture created: 2026-05-03 updated: 2026-05-03 type: investigation status: draft-awaiting-rocky-review drafted-by: cowork-hq related: - "[[Open-Threads]]" - "[[Active]]" - "[[hermes-2026-04-30-self-audit]]" - "[[hermes-skill-audit-2026-05-01]]" - "[[Agent-Stack-MOC]]" --- # Hermes-as-COO Architecture > Drafted 2026-05-03 ~07:00 AEST by cowork-hq while Rocky was awake. Status: draft, awaiting review. Scope: promote Hermes from LFCS-only Telegram-driven assistant → primary autonomous COO across all projects, with Claude Code CLI as dispatched executor. --- ## TL;DR 1. **Don't build a daemon — Hermes IS the daemon.** Hermes Agent has a built-in cron scheduler (gateway daemon ticks every 60 s, runs isolated agent sessions per due job, atomic-write `~/.hermes/cron/jobs.json`, cross-process file lock). We use what's already there. 2. **Add a `coo-inbox` skill.** Hermes loads it on a 5-minute cron, reads `_System/COO-Inbox.md`, classifies open requests, executes light ones, dispatches heavy ones. 3. **Use Claude Code CLI as the dispatched executor.** `claude -p "prompt" --output-format json --allowedTools Bash,Read,Edit,Grep,Glob` — Claude-grade intelligence + tool surface. Hermes triages cheaply with MiniMax, dispatches the actual debugging to CC CLI when reasoning quality matters. 4. **Three-class safety rail.** Hermes never (a) deletes, (b) commits to main, (c) rotates keys, (d) modifies production schema, (e) sends emails/SMS to non-team addresses. Anything in those classes → Telegram escalation, halt. 5. **Implementation cost:** ~1 day focused. Required: `ANTHROPIC_API_KEY` in `~/.hermes/.env` (currently absent), the new skill, one cron job, append-only update to SOUL.md / AGENTS.md. --- ## Hermes today — what exists Drawn from `[[hermes-2026-04-30-self-audit]]`, `[[hermes-skill-audit-2026-05-01]]`, and `[[Agent-Stack-MOC]]`. **Running infrastructure:** - `hermes-gateway.service` — systemd unit, `Restart=always`, 30 s back-off, 5/10 min crash-loop bound. - Watchdog cron `*/15 * * * *` — Telegram alert if down. - Daily health snapshot at 21:00 UTC, daily config backup at 03:00 UTC, 7-day retention. - Hermes v0.11.0 (2026.4.23) — 186+ commits behind upstream as of audit. **Hermes v0.12.0 dropped 2026-04-30** (autonomous Curator on cron ticker, plugin-host gateway, MiniMax OAuth, GMI Cloud + Azure AI Foundry providers, ~57% cold-start improvement). **Tools / skills already wired:** - Telegram + WhatsApp bridge (bridge.js on :3500 since this morning's site fix). - Google Workspace CLI (gws), Deepgram STT, OpenAI auxiliary (gpt-4o-mini for vision). - LFCS skills: docket-check, eod-summary, job-tracker, pre-pour-check, rfi-status, timesheet, client-correspondence, tool-tracker, whatsapp-bridge-setup. - 23 system-bundled skill categories (mlops, creative, gaming, research, devops, etc. — most unused; pruning queued in skill audit). - Filesystem read/write to `/home/ccuser/opsman-work/` and (via OpsMan symlinks) `HQ-Vault/11_OpsMan_LFCS/_AgentLog/` + Operations subdirs. - Cron scheduler built-in (gateway daemon, 60 s tick). **What's missing for COO-grade work:** - `ANTHROPIC_API_KEY` — absent. Blocks both (a) running Hermes inference on Claude, and (b) running Claude Code CLI on the VPS. - `OPENROUTER_API_KEY`, `DEEPSEEK_API_KEY` — absent. Fallback chain proposed (gpt-4o-mini) but not wired. - General shell-execution surface inside an agent session — Hermes CAN call shell via its skills, but has no "free-form bash" tool by default. This is a feature, not a bug — bounded surface = safer agent. - `coo-inbox` skill — does not exist. - Vault read access to `HQ-Vault/_System/` — Hermes has read access to `/home/ccuser/opsman-work/` (which symlinks INTO the vault) but not direct read of arbitrary vault paths. Needs a path expansion or new symlink. - COO-grade judgment for production debugging on MiniMax-M2.7 — meaningfully weaker than Claude Sonnet/Opus at reading nginx logs, diagnosing systemd crashes, judging when to escalate. **This morning's WhatsApp QR-scan misdiagnosis is one data point.** --- ## Two architectures compared ### Architecture A — Hermes as executor Hermes both reasons and executes. Cron job wakes Hermes, Hermes reads inbox, Hermes runs the work directly using its existing skills + a new `bash-runner` skill (or the existing `devops` skill cluster). **Pros:** - Single agent surface — simpler conceptually. - No new external dependency (no Claude Code CLI, no Anthropic key). - Cheap (MiniMax inference cost only). - Skills give bounded surface — easy to audit what Hermes can do. **Cons:** - **Quality ceiling.** MiniMax-M2.7 is weaker than Claude Sonnet for reading nginx error logs, parsing systemd journal output, judging when "the app is up but Stripe checkouts are broken." This morning's QR-scan misdiagnosis is a tell. - Tool surface narrower. Hermes' skill model doesn't give it the same general-purpose `Bash`/`Edit`/`Grep`/`Read` agentic loop Claude Code has out of the box. - **Higher escalation rate.** When Hermes can't figure something out, it punts to Rocky. If MiniMax-quality is the limit, that punt rate may be high enough that Rocky-in-the-loop never really shrinks. **When A is good enough:** - Routine, well-defined tasks where the question is "do this checklist": run a curl, append a line to a log, send a templated Telegram, restart a service that's already documented. - Triage / classification: "is this request a truth-audit, env-check, or something else?" — MiniMax handles that fine. ### Architecture B — Hermes as dispatcher (recommended) Hermes is the standing daemon: reads queues, classifies, decides whether the work is light enough to do itself or heavy enough to dispatch. For heavy work, Hermes shells out to Claude Code CLI (`claude -p "prompt" --output-format json`) with a scoped prompt + bounded tool list. ``` COO-Inbox.md │ ▼ Hermes cron (every 5 min) │ ├── classify request (MiniMax) │ ├── light work → Hermes executes directly │ └─ writes result back to COO-Inbox + daily note │ └── heavy work → claude -p "..." --allowedTools Bash,Read,Edit ├─ Claude Code CLI on VPS executes ├─ output captured (JSON) → Hermes parses └─ Hermes writes result back to COO-Inbox + daily note + git commits via auto-sync.sh ``` **Pros:** - **Claude-grade intelligence on the heavy work.** Production debugging, log reading, multi-file changes — Claude Code's strengths. - **Hermes' daemon nature** — it's already running 24/7, has resilience, has Telegram for escalation. - **Bounded blast radius** — `--allowedTools` controls what Claude Code can touch per dispatch. `--dangerously-skip-permissions` is OFF; we surface a permissions whitelist explicitly per task. - Hermes' existing skills (LFCS, OpsMan, vault-write) stay live for routine work. - Each dispatch is a fresh Claude Code session — clean context, no contamination across requests. **Cons:** - Two intelligences to coordinate (Hermes triage + Claude Code execution). - Cost: every Claude Code invocation hits Anthropic API. Light usage = trivial; heavy usage = monitor. - New surface to harden: how does Hermes safely shell out? What if Claude Code returns an error mid-way? What if the dispatched task takes 15 min — does Hermes block? - Requires `ANTHROPIC_API_KEY` on VPS. **When B is the right call:** - Production debugging, log analysis, multi-file edits, vault-aware reasoning, anything requiring sustained context across multiple tool calls. - Anything where a wrong call has cost (pushing bad code, misdiagnosing an outage, sending a wrong message). ### Recommendation: B with A as a fallback inside it Hermes' classifier picks A or B per request. Cheap-and-clear → A. Cognitively hard or stakes-real → B. This is the **dispatcher pattern** with internal triage. --- ## Skills + tools needed ### New skill: `coo-inbox` Lives at `~/.hermes/skills/coo-inbox/SKILL.md` + supporting files. Loaded by the cron job. **Behaviour:** 1. Read `HQ-Vault/_System/COO-Inbox.md` (file path resolved via `/home/ccuser/HQ-Vault/_System/COO-Inbox.md` after symlink — see below). 2. Parse "Open requests" section. Each block is a `## REQ-{date}-{time}-{slug}` heading. 3. For each open request: - Classify: `truth-audit`, `env-check`, `db-query`, `service-status`, `filesystem-check`, **or `execute-fix`** (new type — to be added to HQ instructions). - Decide: light (Architecture A) or heavy (Architecture B). - Light → run directly using existing Hermes tools (curl for env-check, gws for email, etc.). - Heavy → dispatch via `claude -p "$(cat ~/.hermes/skills/coo-inbox/templates/dispatch.md | envsubst)" --output-format json --allowedTools Bash,Read,Edit,Grep,Glob,WebFetch`. 4. Append result to COO-Inbox under the original request block as `### Result — {YYYY-MM-DD HH:MM} — hermes-coo`. 5. Append session entry to today's daily note under `## HH:MM — hermes-coo — {short title}` per the existing protocol. 6. If anything in the safety-rail class (see below): halt, send Telegram to Rocky, do not execute. 7. Commit + push via auto-sync.sh (already wired). **Skill structure:** ``` ~/.hermes/skills/coo-inbox/ ├── SKILL.md # the skill definition ├── README.md # how it works, escalation rules ├── templates/ │ ├── dispatch.md # Claude Code prompt template │ ├── result.md # COO-Inbox result block template │ └── escalation.md # Telegram escalation template └── lib/ ├── parse_inbox.py # markdown parser for the inbox └── classify.py # request classifier ``` ### New tool: Claude Code CLI on VPS ``` # Install (one-time, on VPS as ccuser): npm install -g @anthropic-ai/claude-code # Sanity check: claude --version claude -p "echo hello" --allowedTools Bash --output-format json # Auth: ANTHROPIC_API_KEY=... in ~/.hermes/.env (Hermes reads this on session start) or in /etc/environment for systemd unit ``` ### Vault read path for Hermes Hermes runs as `root` (per audit `PID 2763217 root /usr/local/lib/hermes-agent/venv/bin/python`). The vault lives in `/home/ccuser/HQ-Vault/`. Two options: - **(i)** Symlink `~/.hermes/vault → /home/ccuser/HQ-Vault/` so Hermes' skill code can `cat ~/.hermes/vault/_System/COO-Inbox.md`. - **(ii)** Fix the path in the skill to read directly via absolute path. Same outcome, no new symlink. (i) is cleaner if Hermes uses relative paths in its skills. ### Existing skills to keep / prune Skill-audit already flagged these — pruning is optional but reduces accidental tool selection: | Category | Recommendation | |---|---| | `lfcs/*`, `lfcs-client-correspondence`, `lfcs-tool-tracker` | KEEP — load-bearing for OpsMan | | `mlops/`, `creative/`, `gaming/`, `research/`, `red-teaming/`, `smart-home/`, `media/`, `yuanbao/` | PRUNE — irrelevant | | `software-development/` | KEEP, audit for COO use (writing-plans + plan duplicate; pick one) | | `productivity/google-workspace/` vs `email/himalaya/` | KEEP google-workspace, prune himalaya | Pruning is a low-stakes Hermes-side cleanup; not blocking COO rollout. --- ## Autonomous loop design ### Cron schedule Inside Hermes' built-in cron (`cronjob` tool, `~/.hermes/cron/jobs.json`): ```yaml job_id: coo-inbox-poll schedule: "*/5 * * * *" # every 5 min skill: coo-inbox prompt: | Read HQ-Vault/_System/COO-Inbox.md. Process any open requests per your skill instructions. If there are no open requests, exit silently with status: empty. on_success: log to /home/ccuser/opsman-work/daily/today on_failure: telegram to Rocky with last 50 lines of agent log timeout_seconds: 600 # 10 min hard limit per tick ``` 5-min poll is responsive enough; can dial up to 1-min if backlog grows. Each tick runs in a fresh agent session — no context bleed between requests. The 10-min timeout prevents one stuck request from blocking subsequent ticks (Hermes' scheduler also has cross-process file lock). ### Per-request flow 1. **Read** — `parse_inbox.py` returns open request blocks. 2. **Lock** — append `- locked-by: hermes-coo @ HH:MM:SS` to the request's status. Prevents a second tick (or a manual COO session) from racing. 3. **Classify** — MiniMax call: "given this request block, return JSON `{type, complexity, safety_class}`." 4. **Decide**: - `complexity: light, safety_class: routine` → Architecture A. - `complexity: heavy` OR `safety_class: high` → Architecture B (dispatch). - `safety_class: blocked` (deletes, key rotations, prod schema, off-team email/SMS) → escalation only, no execute. 5. **Execute / dispatch** — bounded tool list, JSON output captured. 6. **Validate** — parse the result. If well-formed and successful: append to COO-Inbox + daily note. If error: append error block + Telegram-notify Rocky. 7. **Unlock** — change status from `locked-by` → `done` (or `error` / `escalated`). 8. **Commit** — auto-sync.sh handles git. ### Escalation paths Telegram message to Rocky's `@opsman_lfcs_bot` (or a dedicated `@hq_coo_bot` if we split tokens — recommended, see Open Decisions). Format: ``` 🚨 hermes-coo: escalation needed REQ-2026-05-04-0830-stripe-key-rotate reason: safety_class=blocked (key rotation) action: paused. Awaiting Rocky approval. context: ack URL: ``` Reply protocol: Rocky messages back with a directive. The reply gets parsed by a separate Hermes skill (`coo-escalation-reply`) on the next tick. --- ## Safety rails — what hermes-coo never touches without explicit Rocky approval Encoded as `safety_class: blocked` in the classifier. Hard list, no exceptions: 1. **Permanent deletions** — `rm`, `git push --force`, emptying trash, removing rows. (Inherits from `_System/CLAUDE.md` — "No deletions. Move to `_Archive/`".) 2. **Key / token rotation** — Stripe, Supabase service-role, Anthropic, MiniMax, OpenAI, Telegram bot tokens, Doppler tokens, SSH keys. 3. **Production schema changes** — Supabase migrations, Stripe product/price updates, DB schema edits. 4. **Code merges to main** — PR merges, force pushes, branch deletes. Hermes can read PRs and propose merges; only Rocky pulls the trigger. 5. **Off-team messaging** — emails / SMS / WhatsApp to anyone NOT already in `40_People/` with `consent: yes`. Re-engaging Pawel, Susan, etc. requires explicit Rocky go. 6. **SMS at all** — SMS rule from global instructions ("never re-enable SMS in any system without my explicit approval"). Inherits. 7. **Spending money** — paid API upgrades, Stripe charges, ScrapingBee plan upgrades, anything with a credit card. 8. **System file modifications** — `/etc/`, `/root/.hermes/config.yaml` (config edits = a dedicated skill, not free-form), nginx config, systemd units. For each blocked class, the escalation message names the class and the specific action proposed. --- ## Risks + failure modes | Risk | Likelihood | Impact | Mitigation | |---|---|---|---| | Hermes' classifier misjudges a request → executes something it shouldn't | Medium | High | Default to escalation on ambiguity. `safety_class: unknown` → Telegram. Rocky's reply is the tiebreak. | | Claude Code dispatch hangs / times out | Medium | Low | 10-min hard timeout, fresh session per tick, file-locked queue means next tick can pick up. | | Two concurrent ticks race on the same request | Low | Medium | Cross-process file lock (Hermes scheduler does this natively) + `locked-by` status field. | | MiniMax goes down → Hermes can't classify | Medium | Low | Fallback to gpt-4o-mini once `OPENAI_API_KEY` (already present) is wired into config.yaml's `fallback_providers`. | | Anthropic API key gets rate-limited / quota'd | Low | Medium | Hermes detects 429, falls back to Architecture A for that request, logs the rate limit. | | Hermes' cron skips a tick (gateway crash) | Low | Low | systemd `Restart=always` recovers. Watchdog cron alerts Rocky if down >15 min. | | `coo-inbox` skill bug → corrupts the inbox file | Low | High | Pre-write backup: `cp COO-Inbox.md COO-Inbox.md.bak.{tick_id}` before any edit. Append-only writes only — never rewrite the file. | | Rocky's Telegram is on silent overnight, escalation goes unread | High (his schedule) | Variable | Already accepted — Rocky's pattern. Day-shift Cowork sit-rep covers overnight escalations. | --- ## Implementation steps — ordered, with effort estimate **Phase 0 — prerequisites (Rocky, ~10 min):** 1. Acquire `ANTHROPIC_API_KEY` from console.anthropic.com. 2. Decide: dedicated `@hq_coo_bot` Telegram or share `@opsman_lfcs_bot`? (Recommend dedicated — fixes the two-agent-in-one-chat problem documented in Hermes audit.) **Phase 1 — VPS prep (claude-code-coo via terminal, ~30 min):** 3. Install Claude Code CLI on VPS: `npm install -g @anthropic-ai/claude-code`. 4. Add `ANTHROPIC_API_KEY` to `~/.hermes/.env` and `/etc/environment` (for systemd consumption). 5. Smoke-test: `claude -p "echo hello and run \`uname -a\`" --allowedTools Bash --output-format json`. 6. Symlink `~/.hermes/vault → /home/ccuser/HQ-Vault/` (or skip and use absolute paths in the skill). **Phase 2 — skill build (claude-code-coo, ~3 hours):** 7. Write `~/.hermes/skills/coo-inbox/SKILL.md` per the skill structure above. 8. Write `parse_inbox.py`, `classify.py`, templates. 9. Unit test `parse_inbox.py` against the existing `_System/COO-Inbox.md` (currently empty — write a fixture with 3 sample requests). 10. Integration test: classify the 3 fixtures, dispatch one, verify result block is appended correctly. **Phase 3 — Hermes config (claude-code-coo, ~30 min):** 11. Add cron job via Hermes' `cronjob` tool: `*/5 * * * *`, skill=`coo-inbox`, timeout=600s. 12. Update `~/.hermes/SOUL.md` with the new `hermes-coo` role (append-only — don't rewrite existing content). 13. Update `_System/CLAUDE.md` AI Team Protocol with the hermes-coo lane (append a section). **Phase 4 — go-live (Rocky + cowork-hq, ~30 min):** 14. cowork-hq writes one test request to `_System/COO-Inbox.md` (e.g. `type: env-check`, target `which stripe`). 15. Wait one cron tick (≤5 min). Verify result block appears in COO-Inbox + daily note + git commit. 16. Try one heavy request (`type: truth-audit` on the new `_System/COO-Inbox.md` itself — recursive!). Verify dispatch happens, Claude Code result captured. 17. Try one safety-rail request (`type: execute-fix` with a deletion in scope). Verify escalation fires, no execution happens. **Phase 5 — gradual broaden (week 1-2):** 18. Move the four currently-outstanding COO items (nginx buffer, rivet dual-mapping, Stripe smoke, flodeliffre provenance) into COO-Inbox. Watch hermes-coo handle them. 19. Tune classifier thresholds based on observed mistakes. 20. Optional: prune unused Hermes skills per the skill audit. **Total effort:** ~5 hours of focused VPS work for COO + Rocky time on phases 0 + 4. Realistic calendar: 1-2 evenings if Rocky drives in The Window. --- ## Open decisions for Rocky 1. **Dedicated bot token, or share `@opsman_lfcs_bot`?** Recommend dedicated `@hq_coo_bot`. Fixes the two-agent-one-chat problem; gives clean separation between OpsMan-LFCS chat and COO operations chat. 2. **Cron poll interval — 5 min default. Faster (1 min) or slower (15 min)?** 5 min is responsive without burning quota. Tune later. 3. **Anthropic API budget cap?** Suggest a soft cap (e.g. $10/day initially) with Telegram alert at 80%. Adjustable via env var. 4. **Should hermes-coo write to `_Log.md` of every project it touches, or only the daily?** Daily-only is simpler. Per-project `_Log.md` mirrors the existing protocol but doubles writes. Recommend daily-only initially. 5. **Hermes upgrade to v0.12.0?** Released 2026-04-30 with autonomous Curator on cron + plugin-host gateway. Currently 186+ commits behind. Architecture-clean to upgrade BEFORE the COO buildout, but adds risk. Defer if v0.11 cron scheduler is stable for our needs. 6. **MiniMax-only or wire Claude API for Hermes' own inference too?** If Hermes needs to read complex things (like classifying nuanced requests), Claude on Hermes itself raises baseline quality. Cheaper alternative: MiniMax for triage, Claude only inside dispatched calls. Default to that. 7. **Where does this skill live in source control?** Hermes skills are local (`~/.hermes/skills/`). Should the `coo-inbox` skill be tracked in the HQ-Vault repo (e.g. `_System/skills/coo-inbox/`) and rsync'd to `~/.hermes/skills/`? Recommend yes — gives version control and lets Cowork/HQ propose changes. --- ## What this unblocks If wired: - COO-grade work happens 24/7 without Rocky in the loop, bounded by safety rails. - Routine ops (env checks, service status, light filesystem ops) get answered in ≤5 min instead of waiting for Rocky's Window. - Heavy ops (debugging, multi-file changes) still get Claude-grade reasoning, just dispatched through a daemon instead of Rocky's keyboard. - Hermes' Telegram becomes a real ops channel for escalations, not just a chat surface. - The vault becomes a true async coordination surface — HQ writes requests, hermes-coo executes, all logged, all committed, all reviewable. The risk is over-trusting MiniMax-grade triage. Mitigation: start narrow (Phase 5), watch the misclassification rate, dial up scope as confidence grows. --- ## Sources - [Run Claude Code programmatically — Claude Code Docs](https://code.claude.com/docs/en/headless) - [Claude Code Headless Mode — MindStudio](https://www.mindstudio.ai/blog/claude-code-headless-mode-autonomous-agents) - [Claude Code as an Autonomous Agent — Sitepoint](https://www.sitepoint.com/claude-code-as-an-autonomous-agent-advanced-workflows-2026/) - [Claude Agent SDK overview — Claude Code Docs](https://code.claude.com/docs/en/agent-sdk/overview) - [Run prompts on a schedule — Claude Code Docs](https://code.claude.com/docs/en/scheduled-tasks) - [Hermes Agent Cron — Nous Research Docs](https://hermes-agent.nousresearch.com/docs/user-guide/features/cron) - [Hermes Agent Cron Internals](https://hermes-agent.nousresearch.com/docs/developer-guide/cron-internals) - [Hermes v0.12.0 release notes](https://github.com/NousResearch/hermes-agent/blob/main/RELEASE_v0.12.0.md) - [hermes-autonomous-server (production headless setup) — JackTheGit](https://github.com/JackTheGit/hermes-autonomous-server) - [Hermes Issue #5712 — True Autonomy: cron results into chat](https://github.com/NousResearch/hermes-agent/issues/5712) - [LangGraph vs CrewAI vs AutoGen 2026 — Pratik Pathak](https://pratikpathak.com/langgraph-vs-crewai-vs-autogen-2026/) - [Claude Code Scheduling — supalaunch](https://supalaunch.com/blog/claude-code-scheduling-loop-schedule-cron-recurring-tasks-guide)