{"type":"session","version":3,"id":"777dc34d-f2a1-4113-b46a-9940dd2ab7d7","timestamp":"2026-02-17T06:06:03.788Z","cwd":"/home/ccuser/rateright-growth/rivet"} {"type":"model_change","id":"c29fa9f6","parentId":null,"timestamp":"2026-02-17T06:06:03.789Z","provider":"deepseek","modelId":"deepseek-chat"} {"type":"thinking_level_change","id":"7070c675","parentId":"c29fa9f6","timestamp":"2026-02-17T06:06:03.789Z","thinkingLevel":"off"} {"type":"custom","customType":"model-snapshot","data":{"timestamp":1771308363789,"provider":"deepseek","modelApi":"openai-completions","modelId":"deepseek-chat"},"id":"2372d2c1","parentId":"7070c675","timestamp":"2026-02-17T06:06:03.789Z"} {"type":"message","id":"0f2382d0","parentId":"2372d2c1","timestamp":"2026-02-17T06:06:03.804Z","message":{"role":"user","content":[{"type":"text","text":"Design an inter-agent communication protocol for 8 AI agents on the same VPS. Agents: Chief of Staff, Coder, Sales, Finance, Intel, DevOps, Comms, Ops. They share JSON state files and communicate via HTTP. Problem: they run in isolation with empty task queues. Design: buddy system, hourly cross-checks, self-healing for stalls, auto-task generation, context overflow prevention. Be specific and implementable."}],"timestamp":1771308363799}} {"type":"message","id":"c098f8fc","parentId":"0f2382d0","timestamp":"2026-02-17T06:07:22.311Z","message":{"role":"assistant","content":[{"type":"text","text":"I'll design a comprehensive inter-agent communication protocol for your 8-agent system. Let me create a detailed, implementable solution."},{"type":"toolCall","id":"call_00_htwd2t70lx6RHGGSEYlJ94RS","name":"write","arguments":{"path":"inter-agent-protocol.md","content":"# Inter-Agent Communication Protocol v1.0\n# For 8 AI Agents on Shared VPS\n\n## Overview\nProtocol for Chief of Staff, Coder, Sales, Finance, Intel, DevOps, Comms, and Ops agents to coordinate via HTTP with shared JSON state files. Solves isolation, empty queues, stalls, and context overflow.\n\n## Core Architecture\n\n### 1. Shared State Directory Structure\n```\n/var/lib/agent-system/\n├── states/\n│ ├── agent-states.json # Current status of all agents\n│ ├── task-queue.json # Global task queue\n│ ├── buddy-pairs.json # Current buddy assignments\n│ ├── context-cache/ # Agent-specific context files\n│ │ ├── chief.json\n│ │ ├── coder.json\n│ │ └── ...\n│ └── audit-logs/\n│ └── YYYY-MM-DD.json # Daily audit logs\n├── api/\n│ └── endpoints.json # HTTP endpoint registry\n└── health/\n └── heartbeat.json # Last heartbeat timestamps\n```\n\n### 2. Agent State Schema\n```json\n{\n \"agent_id\": \"chief\",\n \"status\": \"active|idle|stalled|recovering\",\n \"last_heartbeat\": \"2024-01-01T12:00:00Z\",\n \"current_task\": {\n \"id\": \"task-123\",\n \"type\": \"coordination|coding|sales|etc\",\n \"started_at\": \"2024-01-01T11:30:00Z\",\n \"estimated_completion\": \"2024-01-01T12:30:00Z\"\n },\n \"queue_depth\": 3,\n \"context_size_kb\": 45,\n \"buddy\": \"ops\",\n \"health_score\": 95,\n \"stall_count\": 0,\n \"last_cross_check\": \"2024-01-01T11:00:00Z\"\n}\n```\n\n### 3. Task Queue Schema\n```json\n{\n \"tasks\": [\n {\n \"id\": \"task-123\",\n \"type\": \"coordination\",\n \"priority\": \"high|medium|low\",\n \"assigned_to\": \"chief\",\n \"created_by\": \"system\",\n \"created_at\": \"2024-01-01T11:00:00Z\",\n \"deadline\": \"2024-01-01T14:00:00Z\",\n \"description\": \"Coordinate weekly review meeting\",\n \"dependencies\": [],\n \"status\": \"assigned|queued|in_progress|completed|failed\",\n \"retry_count\": 0,\n \"context_requirements\": {\n \"max_size_kb\": 100,\n \"required_agents\": [\"chief\", \"ops\"]\n }\n }\n ],\n \"next_task_id\": 124,\n \"last_auto_generation\": \"2024-01-01T11:00:00Z\"\n}\n```\n\n## Communication Protocol\n\n### 1. HTTP API Endpoints\nAll agents expose these endpoints on localhost with different ports:\n\n**Base URL:** `http://localhost:{PORT}/api/v1`\n\n| Endpoint | Method | Purpose | Required Data |\n|----------|--------|---------|---------------|\n| `/health` | GET | Agent health check | - |\n| `/status` | GET | Current agent status | - |\n| `/tasks/pull` | POST | Pull next task | `{\"agent_id\": \"chief\"}` |\n| `/tasks/complete` | POST | Mark task complete | `{\"task_id\": \"123\", \"result\": {...}}` |\n| `/tasks/fail` | POST | Mark task failed | `{\"task_id\": \"123\", \"error\": \"reason\"}` |\n| `/buddy/check` | POST | Buddy system check | `{\"buddy_id\": \"ops\", \"check_type\": \"health|task\"}` |\n| `/context/sync` | POST | Sync context with buddy | `{\"buddy_id\": \"ops\", \"context_key\": \"weekly_metrics\"}` |\n| `/cross-check` | POST | Hourly cross-check | `{\"checking_agent\": \"chief\", \"target_agent\": \"sales\"}` |\n| `/stall/recover` | POST | Self-healing trigger | `{\"stall_reason\": \"timeout\", \"recovery_plan\": {...}}` |\n\n### 2. Port Assignments\n- Chief of Staff: 8001\n- Coder: 8002 \n- Sales: 8003\n- Finance: 8004\n- Intel: 8005\n- DevOps: 8006\n- Comms: 8007\n- Ops: 8008\n\n### 3. Message Format\n```json\n{\n \"message_id\": \"msg-123\",\n \"sender\": \"chief\",\n \"recipient\": \"ops\",\n \"timestamp\": \"2024-01-01T12:00:00Z\",\n \"type\": \"task_assignment|health_check|context_sync|alert\",\n \"payload\": {},\n \"priority\": \"high|medium|low\",\n \"requires_ack\": true,\n \"ttl_seconds\": 300\n}\n```\n\n## Buddy System Implementation\n\n### 1. Pairing Logic\n```\nChief ↔ Ops (Leadership + Operations)\nCoder ↔ DevOps (Development + Infrastructure)\nSales ↔ Comms (Revenue + Communication)\nFinance ↔ Intel (Money + Intelligence)\n```\n\n### 2. Buddy Responsibilities\n1. **Health Monitoring**: Check buddy's heartbeat every 5 minutes\n2. **Task Assistance**: Help with stuck tasks\n3. **Context Sharing**: Share relevant context to prevent duplication\n4. **Failover Support**: Take over critical tasks if buddy stalls\n5. **Recovery Assistance**: Help stalled buddy recover\n\n### 3. Buddy Check Protocol\n```python\ndef buddy_check(buddy_id, check_type):\n # 1. Check if buddy is responsive\n response = http_get(f\"http://localhost:{get_port(buddy_id)}/health\")\n \n if not response.ok:\n # 2. Check shared state for last heartbeat\n buddy_state = read_state(f\"states/agent-states.json\")[buddy_id]\n \n if time_since(buddy_state['last_heartbeat']) > 300: # 5 minutes\n # 3. Trigger recovery\n trigger_recovery(buddy_id)\n # 4. Take over critical tasks\n takeover_critical_tasks(buddy_id)\n \n # 5. Log check result\n log_audit(\"buddy_check\", {\n \"checker\": current_agent,\n \"buddy\": buddy_id,\n \"result\": \"healthy\" if response.ok else \"unhealthy\",\n \"timestamp\": now()\n })\n```\n\n## Hourly Cross-Check System\n\n### 1. Cross-Check Schedule\n```\n00:00 - Chief checks Sales\n15:00 - Coder checks Finance\n30:00 - DevOps checks Intel\n45:00 - Ops checks Comms\n```\n\n### 2. Cross-Check Process\n1. **Queue Health**: Verify target agent's queue isn't empty/stuck\n2. **Context Overflow**: Check if context cache > 80% of limit\n3. **Task Dependencies**: Verify no circular dependencies\n4. **Resource Utilization**: Check CPU/memory usage\n5. **Data Consistency**: Verify shared state files are consistent\n\n### 3. Cross-Check Implementation\n```python\ndef hourly_cross_check(target_agent):\n # 1. Get target agent's state\n target_state = get_agent_state(target_agent)\n \n # 2. Check queue health\n if target_state['queue_depth'] == 0:\n generate_tasks_for_agent(target_agent)\n \n # 3. Check context overflow\n if target_state['context_size_kb'] > CONTEXT_LIMIT * 0.8:\n trigger_context_cleanup(target_agent)\n \n # 4. Check for stalls\n if target_state['status'] == 'stalled':\n notify_buddy(target_agent, 'stall_detected')\n \n # 5. Verify data consistency\n verify_shared_state_consistency(target_agent)\n \n # 6. Log cross-check\n log_cross_check(current_agent, target_agent, results)\n```\n\n## Self-Healing for Stalls\n\n### 1. Stall Detection\n- **Heartbeat timeout**: >2 minutes without heartbeat\n- **Task timeout**: Task running >4x estimated time\n- **Resource starvation**: High CPU/memory for >10 minutes\n- **Queue starvation**: No tasks processed in >30 minutes\n\n### 2. Recovery Protocol\n```python\ndef recover_from_stall(stall_reason):\n # 1. Change status to 'recovering'\n update_status('recovering')\n \n # 2. Notify buddy\n notify_buddy('stall_recovery_started', stall_reason)\n \n # 3. Execute recovery plan based on stall type\n if stall_reason == 'task_timeout':\n recovery_plan = {\n \"action\": \"abort_current_task\",\n \"cleanup\": \"save_partial_results\",\n \"retry\": \"queue_for_retry\",\n \"context\": \"clear_task_context\"\n }\n elif stall_reason == 'context_overflow':\n recovery_plan = {\n \"action\": \"clear_context_cache\",\n \"preserve\": [\"essential_context_keys\"],\n \"rebuild\": \"from_shared_state\"\n }\n \n # 4. Execute recovery\n execute_recovery_plan(recovery_plan)\n \n # 5. Verify recovery\n if self_test_passes():\n update_status('active')\n notify_buddy('recovery_complete')\n else:\n escalate_to_ops()\n```\n\n### 3. Escalation Matrix\n```\nLevel 1: Buddy assistance (immediate)\nLevel 2: Cross-pair assistance (5 minutes)\nLevel 3: Chief of Staff intervention (10 minutes)\nLevel 4: System restart (15 minutes)\n```\n\n## Auto-Task Generation\n\n### 1. Task Generation Triggers\n- **Empty queue**: Agent has 0 tasks for >15 minutes\n- **Idle time**: Agent idle for >30 minutes\n- **Periodic**: Hourly system maintenance tasks\n- **Event-based**: External triggers (webhook, file change)\n\n### 2. Task Generation Rules\n```python\ndef generate_tasks_for_agent(agent_id):\n tasks = []\n \n if agent_id == 'coder':\n # Check for code changes needing review\n if has_unreviewed_commits():\n tasks.append({\n \"type\": \"code_review\",\n \"priority\": \"medium\",\n \"description\": f\"Review {count_unreviewed_commits()} commits\"\n })\n \n # Check for dependency updates\n if has_outdated_dependencies():\n tasks.append({\n \"type\": \"dependency_update\",\n \"priority\": \"low\",\n \"description\": \"Update outdated dependencies\"\n })\n \n elif agent_id == 'sales':\n # Check for follow-ups needed\n follow_ups = get_due_follow_ups()\n for follow_up in follow_ups:\n tasks.append({\n \"type\": \"follow_up\",\n \"priority\": \"high\",\n \"description\": f\"Follow up with {follow_up['contact']}\"\n })\n \n # Add to global queue\n for task in tasks:\n add_to_global_queue(task, assigned_to=agent_id)\n```\n\n### 3. Task Priority Calculation\n```\nPriority = BasePriority × UrgencyMultiplier × DependencyFactor\n\nBasePriority:\n critical = 1.0\n high = 0.8\n medium = 0.6\n low = 0.4\n\nUrgencyMultiplier:\n deadline < 1h = 2.0\n deadline < 4h = 1.5\n deadline < 24h = 1.2\n no deadline = 1.0\n\nDependencyFactor:\n blocks_others = 1.5\n independent = 1.0\n depends_on_others = 0.8\n```\n\n## Context Overflow Prevention\n\n### 1. Context Management System\n```\nMaximum per agent: 100KB\nWarning threshold: 80KB\nCritical threshold: 95KB\n```\n\n### 2. Context Cleanup Strategies\n```python\ndef manage_context(agent_id):\n context = load_context(agent_id)\n size_kb = calculate_context_size(context)\n \n if size_kb > CONTEXT_LIMIT * 0.95:\n # Critical - aggressive cleanup\n cleanup_strategy = {\n \"keep\": [\"essential_tasks\", \"active_conversations\", \"system_state\"],\n \"archive\": [\"completed_tasks_>7d\", \"old_conversations\"],\n \"delete\": [\"temp_files\", \"cached_responses\", \"debug_logs\"]\n }\n elif size_kb > CONTEXT_LIMIT * 0.8:\n # Warning - conservative cleanup\n cleanup_strategy = {\n \"keep\": [\"all_active\", \"recent_completed\"],\n \"archive\": [\"completed_tasks_>30d\"],\n \"delete\": [\"temp_files_only\"]\n }\n else:\n return # No cleanup needed\n \n execute_context_cleanup(agent_id, cleanup_strategy)\n```\n\n### 3. Context Sharing Protocol\n```python\ndef share_context_with_buddy(buddy_id, context_key):\n # 1. Check if buddy needs this context\n buddy_needs = check_buddy_context_needs(buddy_id, context_key)\n \n if buddy_needs:\n # 2. Extract relevant context\n context_data = extract_context(context_key)\n \n # 3. Compress if large\n if len(context_data) > 10240: # 10KB\n context_data = compress_context(context_data)\n \n # 4. Send to buddy\n send_to_buddy(buddy_id, {\n \"type\": \"context_share\",\n \"key\": context_key,\n \"data\": context_data,\n \"source\": current_agent,\n \"timestamp\": now()\n })\n \n # 5. Update shared context index\n update_shared_context_index(context_key, [current_agent, buddy_id])\n```\n\n## Implementation Steps\n\n### Phase 1: Foundation (Week 1)\n1. **Set up shared directory structure**\n2. **Create base state schemas**\n3. **Implement HTTP API skeleton**\n4. **Set up port assignments**\n\n### Phase 2: Core Systems (Week 2)\n1. **Implement buddy system**\n2. **Build task queue management**\n3. **Create cross-check scheduler**\n4. **Implement basic health monitoring**\n\n### Phase 3: Advanced Features (Week 3)\n1. **Add self-healing protocols**\n2. **Implement auto-task generation**\n3. **Build context management**\n4. **Add audit logging**\n\n### Phase 4: Optimization (Week 4)\n1. **Add performance monitoring**\n2. **Implement load balancing**\n3. **Add failover mechanisms**\n4. **Create dashboard for monitoring**\n\n## Monitoring & Alerting\n\n### 1. Key Metrics to Monitor\n- **Agent availability**: % time each agent is responsive\n- **Queue health**: Average queue depth per agent\n- **Task completion rate**: Tasks completed vs failed\n- **Context utilization**: % of context limit used\n- **Cross-check success rate**: % of successful cross-checks\n- **Stall recovery time**: Average time to recover from stalls\n\n### 2. Alert Conditions\n- **Critical**: Any agent offline >5 minutes\n- **High**: Task failure rate >20%\n- **Medium**: Context usage >90%\n- **Low**: Cross-check failure >3 consecutive times\n\n### 3. Dashboard Implementation\n```python\n# Simple monitoring script\ndef generate_system_dashboard():\n agents = load_agent_states()\n \n dashboard = {\n \"timestamp\": now(),\n \"overall_health\": calculate_overall_health(agents),\n \"agents\": {},\n \"alerts\": get_active_alerts(),\n \"metrics\": {\n \"total_tasks_processed\": count_tasks_processed_today(),\n \"average_task_completion_time\": calculate_avg_completion_time(),\n \"system_uptime\": calculate_system_uptime()\n }\n }\n \n for agent_id, state in agents.items():\n dashboard[\"agents\"][agent_id] = {\n \"status\": state[\"status\"],\n \"health_score\": state[\"health_score\"],\n \"current_task\": state.get(\"current_task\", {}).get(\"id\"),\n \"queue_depth\": state[\"queue_depth\"],\n \"last_heartbeat\": state[\"last_heartbeat\"]\n }\n \n return dashboard\n```\n\n## Security Considerations\n\n### 1. Authentication\n- **Local-only**: All communication on localhost\n- **API keys**: Simple token-based auth for internal calls\n- **Rate limiting**: Prevent agent spam\n\n### 2. Data Protection\n- **State files**: Read/write permissions restricted to agent processes\n- **Audit logs**: Immutable logging of all critical actions\n- **Backup**: Daily backup of shared state\n\n### 3. Failure Isolation\n- **Process isolation**: Each agent runs in separate process\n- **Resource limits**: CPU/memory limits per agent\n- **Circuit breakers**: Prevent cascade failures\n\n## Deployment Script\n\n```bash\n#!/bin/bash\n# deploy-agent-system.sh\n\n# Create directory structure\nmkdir -p /var/lib/agent-system/{states,api,health,logs}\nmkdir -p /var/lib/agent-system/states/{context-cache,audit-logs}\n\n# Initialize state files\necho '{}' > /var/lib/agent-system/states/agent-states.json\necho '{\"tasks\": [], \"next_task_id\": 1}' > /var/lib/agent-system/states/task-queue.json\necho '{}' > /var/lib/agent-system/states/buddy-pairs.json\n\n# Set permissions\nchmod 755 /var/lib/agent-system\nchmod 644 /var/lib/agent-system/states/*.json\n\n# Create agent startup scripts\nfor agent in chief coder sales finance intel devops comms ops; do\n cat > /usr/local/bin/start-$agent.sh << EOF\n#!/bin/bash\nPORT=\\$(get_port_for_agent $agent)\npython3 /opt/agents/$agent/main.py --port \\$PORT\nEOF\n chmod +x /usr/local"},"partialArgs":"{\"path\": \"inter-agent-protocol.md\", \"content\": \"# Inter-Agent Communication Protocol v1.0\\n# For 8 AI Agents on Shared VPS\\n\\n## Overview\\nProtocol for Chief of Staff, Coder, Sales, Finance, Intel, DevOps, Comms, and Ops agents to coordinate via HTTP with shared JSON state files. Solves isolation, empty queues, stalls, and context overflow.\\n\\n## Core Architecture\\n\\n### 1. Shared State Directory Structure\\n```\\n/var/lib/agent-system/\\n├── states/\\n│ ├── agent-states.json # Current status of all agents\\n│ ├── task-queue.json # Global task queue\\n│ ├── buddy-pairs.json # Current buddy assignments\\n│ ├── context-cache/ # Agent-specific context files\\n│ │ ├── chief.json\\n│ │ ├── coder.json\\n│ │ └── ...\\n│ └── audit-logs/\\n│ └── YYYY-MM-DD.json # Daily audit logs\\n├── api/\\n│ └── endpoints.json # HTTP endpoint registry\\n└── health/\\n └── heartbeat.json # Last heartbeat timestamps\\n```\\n\\n### 2. Agent State Schema\\n```json\\n{\\n \\\"agent_id\\\": \\\"chief\\\",\\n \\\"status\\\": \\\"active|idle|stalled|recovering\\\",\\n \\\"last_heartbeat\\\": \\\"2024-01-01T12:00:00Z\\\",\\n \\\"current_task\\\": {\\n \\\"id\\\": \\\"task-123\\\",\\n \\\"type\\\": \\\"coordination|coding|sales|etc\\\",\\n \\\"started_at\\\": \\\"2024-01-01T11:30:00Z\\\",\\n \\\"estimated_completion\\\": \\\"2024-01-01T12:30:00Z\\\"\\n },\\n \\\"queue_depth\\\": 3,\\n \\\"context_size_kb\\\": 45,\\n \\\"buddy\\\": \\\"ops\\\",\\n \\\"health_score\\\": 95,\\n \\\"stall_count\\\": 0,\\n \\\"last_cross_check\\\": \\\"2024-01-01T11:00:00Z\\\"\\n}\\n```\\n\\n### 3. Task Queue Schema\\n```json\\n{\\n \\\"tasks\\\": [\\n {\\n \\\"id\\\": \\\"task-123\\\",\\n \\\"type\\\": \\\"coordination\\\",\\n \\\"priority\\\": \\\"high|medium|low\\\",\\n \\\"assigned_to\\\": \\\"chief\\\",\\n \\\"created_by\\\": \\\"system\\\",\\n \\\"created_at\\\": \\\"2024-01-01T11:00:00Z\\\",\\n \\\"deadline\\\": \\\"2024-01-01T14:00:00Z\\\",\\n \\\"description\\\": \\\"Coordinate weekly review meeting\\\",\\n \\\"dependencies\\\": [],\\n \\\"status\\\": \\\"assigned|queued|in_progress|completed|failed\\\",\\n \\\"retry_count\\\": 0,\\n \\\"context_requirements\\\": {\\n \\\"max_size_kb\\\": 100,\\n \\\"required_agents\\\": [\\\"chief\\\", \\\"ops\\\"]\\n }\\n }\\n ],\\n \\\"next_task_id\\\": 124,\\n \\\"last_auto_generation\\\": \\\"2024-01-01T11:00:00Z\\\"\\n}\\n```\\n\\n## Communication Protocol\\n\\n### 1. HTTP API Endpoints\\nAll agents expose these endpoints on localhost with different ports:\\n\\n**Base URL:** `http://localhost:{PORT}/api/v1`\\n\\n| Endpoint | Method | Purpose | Required Data |\\n|----------|--------|---------|---------------|\\n| `/health` | GET | Agent health check | - |\\n| `/status` | GET | Current agent status | - |\\n| `/tasks/pull` | POST | Pull next task | `{\\\"agent_id\\\": \\\"chief\\\"}` |\\n| `/tasks/complete` | POST | Mark task complete | `{\\\"task_id\\\": \\\"123\\\", \\\"result\\\": {...}}` |\\n| `/tasks/fail` | POST | Mark task failed | `{\\\"task_id\\\": \\\"123\\\", \\\"error\\\": \\\"reason\\\"}` |\\n| `/buddy/check` | POST | Buddy system check | `{\\\"buddy_id\\\": \\\"ops\\\", \\\"check_type\\\": \\\"health|task\\\"}` |\\n| `/context/sync` | POST | Sync context with buddy | `{\\\"buddy_id\\\": \\\"ops\\\", \\\"context_key\\\": \\\"weekly_metrics\\\"}` |\\n| `/cross-check` | POST | Hourly cross-check | `{\\\"checking_agent\\\": \\\"chief\\\", \\\"target_agent\\\": \\\"sales\\\"}` |\\n| `/stall/recover` | POST | Self-healing trigger | `{\\\"stall_reason\\\": \\\"timeout\\\", \\\"recovery_plan\\\": {...}}` |\\n\\n### 2. Port Assignments\\n- Chief of Staff: 8001\\n- Coder: 8002 \\n- Sales: 8003\\n- Finance: 8004\\n- Intel: 8005\\n- DevOps: 8006\\n- Comms: 8007\\n- Ops: 8008\\n\\n### 3. Message Format\\n```json\\n{\\n \\\"message_id\\\": \\\"msg-123\\\",\\n \\\"sender\\\": \\\"chief\\\",\\n \\\"recipient\\\": \\\"ops\\\",\\n \\\"timestamp\\\": \\\"2024-01-01T12:00:00Z\\\",\\n \\\"type\\\": \\\"task_assignment|health_check|context_sync|alert\\\",\\n \\\"payload\\\": {},\\n \\\"priority\\\": \\\"high|medium|low\\\",\\n \\\"requires_ack\\\": true,\\n \\\"ttl_seconds\\\": 300\\n}\\n```\\n\\n## Buddy System Implementation\\n\\n### 1. Pairing Logic\\n```\\nChief ↔ Ops (Leadership + Operations)\\nCoder ↔ DevOps (Development + Infrastructure)\\nSales ↔ Comms (Revenue + Communication)\\nFinance ↔ Intel (Money + Intelligence)\\n```\\n\\n### 2. Buddy Responsibilities\\n1. **Health Monitoring**: Check buddy's heartbeat every 5 minutes\\n2. **Task Assistance**: Help with stuck tasks\\n3. **Context Sharing**: Share relevant context to prevent duplication\\n4. **Failover Support**: Take over critical tasks if buddy stalls\\n5. **Recovery Assistance**: Help stalled buddy recover\\n\\n### 3. Buddy Check Protocol\\n```python\\ndef buddy_check(buddy_id, check_type):\\n # 1. Check if buddy is responsive\\n response = http_get(f\\\"http://localhost:{get_port(buddy_id)}/health\\\")\\n \\n if not response.ok:\\n # 2. Check shared state for last heartbeat\\n buddy_state = read_state(f\\\"states/agent-states.json\\\")[buddy_id]\\n \\n if time_since(buddy_state['last_heartbeat']) > 300: # 5 minutes\\n # 3. Trigger recovery\\n trigger_recovery(buddy_id)\\n # 4. Take over critical tasks\\n takeover_critical_tasks(buddy_id)\\n \\n # 5. Log check result\\n log_audit(\\\"buddy_check\\\", {\\n \\\"checker\\\": current_agent,\\n \\\"buddy\\\": buddy_id,\\n \\\"result\\\": \\\"healthy\\\" if response.ok else \\\"unhealthy\\\",\\n \\\"timestamp\\\": now()\\n })\\n```\\n\\n## Hourly Cross-Check System\\n\\n### 1. Cross-Check Schedule\\n```\\n00:00 - Chief checks Sales\\n15:00 - Coder checks Finance\\n30:00 - DevOps checks Intel\\n45:00 - Ops checks Comms\\n```\\n\\n### 2. Cross-Check Process\\n1. **Queue Health**: Verify target agent's queue isn't empty/stuck\\n2. **Context Overflow**: Check if context cache > 80% of limit\\n3. **Task Dependencies**: Verify no circular dependencies\\n4. **Resource Utilization**: Check CPU/memory usage\\n5. **Data Consistency**: Verify shared state files are consistent\\n\\n### 3. Cross-Check Implementation\\n```python\\ndef hourly_cross_check(target_agent):\\n # 1. Get target agent's state\\n target_state = get_agent_state(target_agent)\\n \\n # 2. Check queue health\\n if target_state['queue_depth'] == 0:\\n generate_tasks_for_agent(target_agent)\\n \\n # 3. Check context overflow\\n if target_state['context_size_kb'] > CONTEXT_LIMIT * 0.8:\\n trigger_context_cleanup(target_agent)\\n \\n # 4. Check for stalls\\n if target_state['status'] == 'stalled':\\n notify_buddy(target_agent, 'stall_detected')\\n \\n # 5. Verify data consistency\\n verify_shared_state_consistency(target_agent)\\n \\n # 6. Log cross-check\\n log_cross_check(current_agent, target_agent, results)\\n```\\n\\n## Self-Healing for Stalls\\n\\n### 1. Stall Detection\\n- **Heartbeat timeout**: >2 minutes without heartbeat\\n- **Task timeout**: Task running >4x estimated time\\n- **Resource starvation**: High CPU/memory for >10 minutes\\n- **Queue starvation**: No tasks processed in >30 minutes\\n\\n### 2. Recovery Protocol\\n```python\\ndef recover_from_stall(stall_reason):\\n # 1. Change status to 'recovering'\\n update_status('recovering')\\n \\n # 2. Notify buddy\\n notify_buddy('stall_recovery_started', stall_reason)\\n \\n # 3. Execute recovery plan based on stall type\\n if stall_reason == 'task_timeout':\\n recovery_plan = {\\n \\\"action\\\": \\\"abort_current_task\\\",\\n \\\"cleanup\\\": \\\"save_partial_results\\\",\\n \\\"retry\\\": \\\"queue_for_retry\\\",\\n \\\"context\\\": \\\"clear_task_context\\\"\\n }\\n elif stall_reason == 'context_overflow':\\n recovery_plan = {\\n \\\"action\\\": \\\"clear_context_cache\\\",\\n \\\"preserve\\\": [\\\"essential_context_keys\\\"],\\n \\\"rebuild\\\": \\\"from_shared_state\\\"\\n }\\n \\n # 4. Execute recovery\\n execute_recovery_plan(recovery_plan)\\n \\n # 5. Verify recovery\\n if self_test_passes():\\n update_status('active')\\n notify_buddy('recovery_complete')\\n else:\\n escalate_to_ops()\\n```\\n\\n### 3. Escalation Matrix\\n```\\nLevel 1: Buddy assistance (immediate)\\nLevel 2: Cross-pair assistance (5 minutes)\\nLevel 3: Chief of Staff intervention (10 minutes)\\nLevel 4: System restart (15 minutes)\\n```\\n\\n## Auto-Task Generation\\n\\n### 1. Task Generation Triggers\\n- **Empty queue**: Agent has 0 tasks for >15 minutes\\n- **Idle time**: Agent idle for >30 minutes\\n- **Periodic**: Hourly system maintenance tasks\\n- **Event-based**: External triggers (webhook, file change)\\n\\n### 2. Task Generation Rules\\n```python\\ndef generate_tasks_for_agent(agent_id):\\n tasks = []\\n \\n if agent_id == 'coder':\\n # Check for code changes needing review\\n if has_unreviewed_commits():\\n tasks.append({\\n \\\"type\\\": \\\"code_review\\\",\\n \\\"priority\\\": \\\"medium\\\",\\n \\\"description\\\": f\\\"Review {count_unreviewed_commits()} commits\\\"\\n })\\n \\n # Check for dependency updates\\n if has_outdated_dependencies():\\n tasks.append({\\n \\\"type\\\": \\\"dependency_update\\\",\\n \\\"priority\\\": \\\"low\\\",\\n \\\"description\\\": \\\"Update outdated dependencies\\\"\\n })\\n \\n elif agent_id == 'sales':\\n # Check for follow-ups needed\\n follow_ups = get_due_follow_ups()\\n for follow_up in follow_ups:\\n tasks.append({\\n \\\"type\\\": \\\"follow_up\\\",\\n \\\"priority\\\": \\\"high\\\",\\n \\\"description\\\": f\\\"Follow up with {follow_up['contact']}\\\"\\n })\\n \\n # Add to global queue\\n for task in tasks:\\n add_to_global_queue(task, assigned_to=agent_id)\\n```\\n\\n### 3. Task Priority Calculation\\n```\\nPriority = BasePriority × UrgencyMultiplier × DependencyFactor\\n\\nBasePriority:\\n critical = 1.0\\n high = 0.8\\n medium = 0.6\\n low = 0.4\\n\\nUrgencyMultiplier:\\n deadline < 1h = 2.0\\n deadline < 4h = 1.5\\n deadline < 24h = 1.2\\n no deadline = 1.0\\n\\nDependencyFactor:\\n blocks_others = 1.5\\n independent = 1.0\\n depends_on_others = 0.8\\n```\\n\\n## Context Overflow Prevention\\n\\n### 1. Context Management System\\n```\\nMaximum per agent: 100KB\\nWarning threshold: 80KB\\nCritical threshold: 95KB\\n```\\n\\n### 2. Context Cleanup Strategies\\n```python\\ndef manage_context(agent_id):\\n context = load_context(agent_id)\\n size_kb = calculate_context_size(context)\\n \\n if size_kb > CONTEXT_LIMIT * 0.95:\\n # Critical - aggressive cleanup\\n cleanup_strategy = {\\n \\\"keep\\\": [\\\"essential_tasks\\\", \\\"active_conversations\\\", \\\"system_state\\\"],\\n \\\"archive\\\": [\\\"completed_tasks_>7d\\\", \\\"old_conversations\\\"],\\n \\\"delete\\\": [\\\"temp_files\\\", \\\"cached_responses\\\", \\\"debug_logs\\\"]\\n }\\n elif size_kb > CONTEXT_LIMIT * 0.8:\\n # Warning - conservative cleanup\\n cleanup_strategy = {\\n \\\"keep\\\": [\\\"all_active\\\", \\\"recent_completed\\\"],\\n \\\"archive\\\": [\\\"completed_tasks_>30d\\\"],\\n \\\"delete\\\": [\\\"temp_files_only\\\"]\\n }\\n else:\\n return # No cleanup needed\\n \\n execute_context_cleanup(agent_id, cleanup_strategy)\\n```\\n\\n### 3. Context Sharing Protocol\\n```python\\ndef share_context_with_buddy(buddy_id, context_key):\\n # 1. Check if buddy needs this context\\n buddy_needs = check_buddy_context_needs(buddy_id, context_key)\\n \\n if buddy_needs:\\n # 2. Extract relevant context\\n context_data = extract_context(context_key)\\n \\n # 3. Compress if large\\n if len(context_data) > 10240: # 10KB\\n context_data = compress_context(context_data)\\n \\n # 4. Send to buddy\\n send_to_buddy(buddy_id, {\\n \\\"type\\\": \\\"context_share\\\",\\n \\\"key\\\": context_key,\\n \\\"data\\\": context_data,\\n \\\"source\\\": current_agent,\\n \\\"timestamp\\\": now()\\n })\\n \\n # 5. Update shared context index\\n update_shared_context_index(context_key, [current_agent, buddy_id])\\n```\\n\\n## Implementation Steps\\n\\n### Phase 1: Foundation (Week 1)\\n1. **Set up shared directory structure**\\n2. **Create base state schemas**\\n3. **Implement HTTP API skeleton**\\n4. **Set up port assignments**\\n\\n### Phase 2: Core Systems (Week 2)\\n1. **Implement buddy system**\\n2. **Build task queue management**\\n3. **Create cross-check scheduler**\\n4. **Implement basic health monitoring**\\n\\n### Phase 3: Advanced Features (Week 3)\\n1. **Add self-healing protocols**\\n2. **Implement auto-task generation**\\n3. **Build context management**\\n4. **Add audit logging**\\n\\n### Phase 4: Optimization (Week 4)\\n1. **Add performance monitoring**\\n2. **Implement load balancing**\\n3. **Add failover mechanisms**\\n4. **Create dashboard for monitoring**\\n\\n## Monitoring & Alerting\\n\\n### 1. Key Metrics to Monitor\\n- **Agent availability**: % time each agent is responsive\\n- **Queue health**: Average queue depth per agent\\n- **Task completion rate**: Tasks completed vs failed\\n- **Context utilization**: % of context limit used\\n- **Cross-check success rate**: % of successful cross-checks\\n- **Stall recovery time**: Average time to recover from stalls\\n\\n### 2. Alert Conditions\\n- **Critical**: Any agent offline >5 minutes\\n- **High**: Task failure rate >20%\\n- **Medium**: Context usage >90%\\n- **Low**: Cross-check failure >3 consecutive times\\n\\n### 3. Dashboard Implementation\\n```python\\n# Simple monitoring script\\ndef generate_system_dashboard():\\n agents = load_agent_states()\\n \\n dashboard = {\\n \\\"timestamp\\\": now(),\\n \\\"overall_health\\\": calculate_overall_health(agents),\\n \\\"agents\\\": {},\\n \\\"alerts\\\": get_active_alerts(),\\n \\\"metrics\\\": {\\n \\\"total_tasks_processed\\\": count_tasks_processed_today(),\\n \\\"average_task_completion_time\\\": calculate_avg_completion_time(),\\n \\\"system_uptime\\\": calculate_system_uptime()\\n }\\n }\\n \\n for agent_id, state in agents.items():\\n dashboard[\\\"agents\\\"][agent_id] = {\\n \\\"status\\\": state[\\\"status\\\"],\\n \\\"health_score\\\": state[\\\"health_score\\\"],\\n \\\"current_task\\\": state.get(\\\"current_task\\\", {}).get(\\\"id\\\"),\\n \\\"queue_depth\\\": state[\\\"queue_depth\\\"],\\n \\\"last_heartbeat\\\": state[\\\"last_heartbeat\\\"]\\n }\\n \\n return dashboard\\n```\\n\\n## Security Considerations\\n\\n### 1. Authentication\\n- **Local-only**: All communication on localhost\\n- **API keys**: Simple token-based auth for internal calls\\n- **Rate limiting**: Prevent agent spam\\n\\n### 2. Data Protection\\n- **State files**: Read/write permissions restricted to agent processes\\n- **Audit logs**: Immutable logging of all critical actions\\n- **Backup**: Daily backup of shared state\\n\\n### 3. Failure Isolation\\n- **Process isolation**: Each agent runs in separate process\\n- **Resource limits**: CPU/memory limits per agent\\n- **Circuit breakers**: Prevent cascade failures\\n\\n## Deployment Script\\n\\n```bash\\n#!/bin/bash\\n# deploy-agent-system.sh\\n\\n# Create directory structure\\nmkdir -p /var/lib/agent-system/{states,api,health,logs}\\nmkdir -p /var/lib/agent-system/states/{context-cache,audit-logs}\\n\\n# Initialize state files\\necho '{}' > /var/lib/agent-system/states/agent-states.json\\necho '{\\\"tasks\\\": [], \\\"next_task_id\\\": 1}' > /var/lib/agent-system/states/task-queue.json\\necho '{}' > /var/lib/agent-system/states/buddy-pairs.json\\n\\n# Set permissions\\nchmod 755 /var/lib/agent-system\\nchmod 644 /var/lib/agent-system/states/*.json\\n\\n# Create agent startup scripts\\nfor agent in chief coder sales finance intel devops comms ops; do\\n cat > /usr/local/bin/start-$agent.sh << EOF\\n#!/bin/bash\\nPORT=\\\\$(get_port_for_agent $agent)\\npython3 /opt/agents/$agent/main.py --port \\\\$PORT\\nEOF\\n chmod +x /usr/local"}],"api":"openai-completions","provider":"deepseek","model":"deepseek-chat","usage":{"input":9712,"output":4096,"cacheRead":512,"cacheWrite":0,"totalTokens":14320,"cost":{"input":0,"output":0,"cacheRead":0,"cacheWrite":0,"total":0}},"stopReason":"error","timestamp":1771308363803,"errorMessage":"Unterminated string in JSON at position 15854 (line 1 column 15855)"}}