# 2026-02-18 — Builder Session Notes

## Major Discovery: Stall Detector Wake = Root Cause of ALL Agent Crashes

**The single biggest finding today:** The `cron wake` mechanism in stall-detector.js and buddy-check.js triggers a Clawdbot bug (`mapOptionsForApi: undefined`) that crashes agents regardless of model provider (MiniMax, DeepSeek, Moonshot — all affected).

### The Crash Loop Chain:
1. Agent status.json goes stale (old heartbeat timestamp)
2. Stall detector (every 5 min) sees "stalled" → sends `cron wake` via `/tools/invoke`
3. Wake triggers `streamAssistantResponse` through a code path where model `api` field = undefined
4. Agent crashes with `mapOptionsForApi: undefined`
5. Systemd restarts agent → status.json still stale → repeat from step 2

### The Fix (deployed):
- Replaced `cron wake` with `systemctl restart` in both stall-detector.js and buddy-check.js
- Increased stall threshold from 45 min → 60 min
- This means MiniMax was NEVER broken — the wake trigger was the problem all along

### Key Lesson:
MiniMax API works (tested via curl). Moonshot API works (tested, account no longer suspended). DeepSeek works. ALL custom providers crash through the wake code path. Built-in Anthropic providers work because they go through a different code path in Clawdbot.

## OpsMan = The Product (Strategic Pivot)

Rocky clarified: **OpsMan IS the product being sold.** The 8-agent fleet is the demo, not overhead.
- Consolidation recommendation withdrawn
- All 8 agents must be productive and autonomous
- SYSTEM-BASELINE.md updated to reflect this
- Rivet ACK'd at 08:48

## SYSTEM-BASELINE.md Created

Full system documentation at `/home/ccuser/the-50-dollar-app/SYSTEM-BASELINE.md` and `/home/ccuser/shared/SYSTEM-BASELINE.md`:
- Section 0: Big picture — operating model, three layers, information loop
- Agent registry with ports, models, config paths
- Communication protocol (3 methods: agent bridge, JSONL inbox, queue.json)
- Buddy system pairs
- Daily rhythm
- Record keeping expectations
- Curation flow
- Known issues + audit checklist + recovery procedures

## Fleet Status Fix

- Updated status.json for Susan/Sentinel/Radar/Cog (were stale from yesterday)
- Added "Step FINAL: Update Your Status" to all 4 agents' HEARTBEAT.md — they never had this step
- Cleaned fleet-state.json — all 8 agents showing active

## System Audit vs Baseline

Infrastructure: 90% operational (all services up, build green, SSL working)
Agent productivity: 10% operational (12 tasks pending, 0 completed, 0 cross-agent collaboration, 3 agents amnesiac)

## Clawdbot Version Situation

- Current: 2026.1.24-3 (latest on npm)
- GitHub has 2026.2.15 but NOT published to npm yet
- Known bug: `mapOptionsForApi: undefined` for custom providers — issues #1695, #15036, #7572 on openclaw/openclaw
- Cron job set to check for updates twice daily (8am/8pm)

## Model Provider Status

- **Anthropic (Opus/Sonnet):** Working ✅
- **DeepSeek:** Working ✅ (currently on Susan/Sentinel/Radar/Cog)
- **MiniMax M2.5:** API works, Clawdbot wake path crashes it. Should work with restart-based recovery.
- **Moonshot/Kimi K2.5:** API working again (account unsuspended). Same wake path issue.
- **Perplexity:** Configured for Susan/Radar/Harper

## Config Locations (Critical Knowledge)

Profile configs at `/root/.clawdbot-{agent}/clawdbot.json` take precedence over workspace `.clawdbot/clawdbot.json`.
Session data at `/root/.clawdbot-{agent}/agents/main/sessions/`.
Auth profiles at `/root/.clawdbot-{agent}/agents/main/agent/auth-profiles.json`.

**CRITICAL RULE:** When changing model in config, MUST also archive sessions. Stale session model overrides persist across restarts.

## CEO Dashboard 404

The /ceo route is returning 404 — needs investigation. Was working when built on Feb 17.