# Agent Fleet Reliability Report
**Date:** 2026-02-18 18:45 AEDT  
**Period:** Last 24 hours  
**Generated by:** Sentinel

## System Resources
| Metric | Value | Status |
|--------|-------|--------|
| CPU Load | 1.80 (2 vCPU) | ✅ Normal |
| RAM | 3.2GB / 7.8GB (41%) | ✅ Healthy |
| Disk | 46GB / 77GB (60%) | ✅ OK |
| Swap | 236MB / 2GB | ✅ Minimal |
| Uptime | 2 days, 36 min | ✅ Stable |

## Agent Service Status

| Agent | Service | Status | Restarts (24h) | Up Since | ⚠️ Flag |
|-------|---------|--------|----------------|----------|---------|
| Rivet | clawdbot-gateway | ✅ active | 90 | 18:10 | ⚠️ HIGH |
| Builder | clawdbot-builder | ✅ active | 88 | 18:00 | ⚠️ HIGH |
| Susan | clawdbot-susan | ✅ active | 345 | 18:26 | 🔴 VERY HIGH |
| Harper | clawdbot-harper | ✅ active | 10 | 18:33 | ✅ Normal |
| Sentinel | clawdbot-sentinel | ✅ active | 204 | 18:26 | ⚠️ HIGH |
| Radar | clawdbot-radar | ✅ active | 198 | 18:01 | ⚠️ HIGH |
| Herald | clawdbot-herald | ✅ active | 11 | 18:01 | ✅ Normal |
| Cog | clawdbot-cog | ✅ active | 218 | 18:39 | ⚠️ HIGH |

## Last Heartbeat Timestamps

| Agent | Last Heartbeat | Status | Stale? |
|-------|---------------|--------|--------|
| Rivet | No status.json | — | ❓ Unknown |
| Builder | No status.json | — | ❓ Unknown |
| Susan | 2026-02-17 23:13 UTC | active | ⚠️ ~19h ago |
| Harper | 2026-02-18 06:54 UTC | idle | ✅ ~12h ago |
| Sentinel | 2026-02-17 18:17 UTC | active | ⚠️ Updating now |
| Radar | 2026-02-17 23:13 UTC | active | ⚠️ ~19h ago |
| Herald | 2026-02-18 06:44 UTC | idle | ✅ ~12h ago |
| Cog | 2026-02-18 04:43 UTC | active | ⚠️ ~14h ago |

## Analysis

### Restart Pattern
All restarts are **graceful SIGTERM shutdowns**, not crashes. Pattern shows:
- Agents receive SIGTERM → deactivate successfully → restart via systemd
- This is consistent with **self-update cycles** (gateway update.run or config.apply triggers SIGUSR1/restart)
- No OOM kills, no SIGKILL, no unexpected failures

### Agents Flagged (>3 restarts in 24h)
- **Susan (345 restarts):** Highest restart count. Likely aggressive heartbeat + update cycles.
- **Cog (218)**, **Sentinel (204)**, **Radar (198):** Similar pattern.
- **Rivet (90)**, **Builder (88):** Moderate but still elevated.
- **Harper (10)**, **Herald (11):** Normal range.

### Root Cause Assessment
The high restart counts are likely caused by:
1. Frequent `gateway update.run` or `config.apply` calls during heartbeat processing
2. Agents triggering self-restarts during routine operations
3. Possible restart loops where an agent restarts, picks up a heartbeat, restarts again

### Recommendations
1. **Investigate Susan's restart frequency** — 345 in 24h = one every ~4 minutes. This wastes resources.
2. **Add restart throttling** — Consider `RestartSec=30` in systemd units to prevent rapid cycling.
3. **Audit heartbeat handlers** — Check if agents are calling update.run unnecessarily during heartbeats.
4. **Status.json compliance** — Rivet and Builder need to write status.json for fleet monitoring.

## Overall Assessment
**Fleet Status: ✅ OPERATIONAL** — All 8 agents currently active and responding.  
**Concern: ⚠️ RESTART FREQUENCY** — 6/8 agents show excessive restarts. Functional but wasteful.