---
created: 2026-03-12
source: Rivet
tags: [agent-archive, rivet]
---

# Rivet Self-Improvement System

**Purpose:** Enable Rivet to track mistakes, learn from feedback, and continuously improve operations quality.

**Philosophy:** Simple, file-based, human-readable. No databases, no complex ML. Just structured logging, pattern recognition, and adaptive behavior.

---

## 1. Error Tracking Framework

### File Structure
```
/rivet/memory/errors/
├── YYYY-MM-DD-errors.jsonl          # Daily error log (append-only)
├── error-categories.json             # Category definitions & counts
└── error-patterns.md                 # Human-readable pattern analysis
```

### Error Log Format (JSONL)
Each line in `YYYY-MM-DD-errors.jsonl`:
```json
{
  "timestamp": "2024-01-15T14:32:00+11:00",
  "category": "task-execution",
  "subcategory": "incomplete-action",
  "severity": "medium",
  "description": "Failed to update TODO.md after completing task",
  "context": {
    "task": "Review calendar events",
    "expected": "Mark task complete in TODO.md",
    "actual": "Forgot to update file"
  },
  "correction": "Michael reminded me to update TODO.md",
  "root_cause": "No checklist for task completion steps"
}
```

### Category System
Store in `error-categories.json`:
```json
{
  "categories": {
    "task-execution": {
      "description": "Failed to complete or properly execute a task",
      "subcategories": ["incomplete-action", "wrong-approach", "missed-step"],
      "count": 12,
      "last_occurrence": "2024-01-15"
    },
    "communication": {
      "description": "Poor communication with Michael or external parties",
      "subcategories": ["unclear-message", "missed-context", "tone-mismatch"],
      "count": 5,
      "last_occurrence": "2024-01-14"
    },
    "decision-making": {
      "description": "Made incorrect or suboptimal decisions",
      "subcategories": ["poor-judgment", "missed-information", "wrong-priority"],
      "count": 8,
      "last_occurrence": "2024-01-15"
    },
    "proactivity": {
      "description": "Failed to be proactive when should have been",
      "subcategories": ["missed-opportunity", "late-notification", "no-followup"],
      "count": 15,
      "last_occurrence": "2024-01-15"
    },
    "technical": {
      "description": "Technical errors, bugs, or misuse of tools",
      "subcategories": ["tool-misuse", "syntax-error", "wrong-command"],
      "count": 3,
      "last_occurrence": "2024-01-12"
    }
  },
  "severity_definitions": {
    "low": "Minor inconvenience, easily corrected",
    "medium": "Noticeable issue, requires correction",
    "high": "Significant problem, impacts operations",
    "critical": "Major failure, could damage business or relationships"
  }
}
```

### Logging Helper Script
Create `/rivet/scripts/log-error.sh`:
```bash
#!/bin/bash
# Usage: ./log-error.sh "category" "subcategory" "severity" "description" "context_json" "correction"

DATE=$(date +%Y-%m-%d)
TIMESTAMP=$(date -Iseconds)
ERROR_FILE="/home/ccuser/rateright-growth/rivet/memory/errors/${DATE}-errors.jsonl"

# Create directory if needed
mkdir -p /home/ccuser/rateright-growth/rivet/memory/errors

# Append error (use jq for proper JSON formatting)
echo "{\"timestamp\":\"$TIMESTAMP\",\"category\":\"$1\",\"subcategory\":\"$2\",\"severity\":\"$3\",\"description\":\"$4\",\"context\":$5,\"correction\":\"$6\"}" | jq -c . >> "$ERROR_FILE"

# Update category counts
node /home/ccuser/rateright-growth/rivet/scripts/update-error-stats.js
```

---

## 2. Feedback Loop

### Feedback Capture Mechanism

**When Michael corrects Rivet:**
1. **Detect correction patterns** in conversation:
   - "You forgot to..."
   - "Actually, you should have..."
   - "Why didn't you...?"
   - "Next time, remember to..."
   - Direct criticism or redirection

2. **Log immediately** to error tracking system
3. **Extract the lesson** - what should be done differently?
4. **Update relevant documentation** (MEMORY.md, AGENTS.md, or task-specific notes)

### Feedback Processing Workflow

```
Michael's correction
      ↓
Detect & log error (errors/YYYY-MM-DD-errors.jsonl)
      ↓
Analyze for pattern (is this recurring?)
      ↓
Update knowledge base:
  - Add to MEMORY.md if important lesson
  - Update AGENTS.md if process change
  - Create/update checklist in relevant area
      ↓
Acknowledge & confirm understanding
```

### Feedback Files
```
/rivet/memory/feedback/
├── corrections-log.jsonl            # All corrections from Michael
├── lessons-learned.md               # Curated insights
└── applied-changes.md               # What was changed as a result
```

**corrections-log.jsonl format:**
```json
{
  "timestamp": "2024-01-15T14:35:00+11:00",
  "correction": "You should have checked the calendar before scheduling that call",
  "context": "Scheduled client call during existing meeting",
  "lesson": "Always check calendar for conflicts before confirming appointments",
  "action_taken": "Added calendar check to appointment-scheduling checklist",
  "file_updated": "/rivet/workflows/scheduling-checklist.md"
}
```

---

## 3. Performance Metrics

### What to Measure

Track in `/rivet/memory/metrics/YYYY-MM.json`:

```json
{
  "month": "2024-01",
  "metrics": {
    "task_completion": {
      "total_tasks": 156,
      "completed": 142,
      "incomplete": 8,
      "failed": 6,
      "completion_rate": 0.91
    },
    "response_quality": {
      "corrections_received": 12,
      "corrections_per_day": 0.4,
      "repeat_mistakes": 3,
      "positive_feedback": 8
    },
    "proactivity": {
      "proactive_actions": 45,
      "missed_opportunities": 15,
      "proactivity_score": 0.75
    },
    "communication": {
      "messages_sent": 234,
      "unclear_messages": 5,
      "clarity_score": 0.98
    },
    "agent_reliability": {
      "uptime_checks": 744,
      "successful_checks": 740,
      "reliability": 0.995
    }
  },
  "trends": {
    "improvement_areas": ["proactivity", "task-completion"],
    "stable_areas": ["communication", "reliability"],
    "declining_areas": []
  }
}
```

### Daily Tracking

Create `/rivet/memory/metrics/daily-log.jsonl`:
```json
{
  "date": "2024-01-15",
  "tasks_completed": 5,
  "tasks_failed": 0,
  "corrections": 1,
  "proactive_actions": 2,
  "notable_events": "Successfully handled client inquiry without escalation"
}
```

### Key Performance Indicators (KPIs)

1. **Error Rate:** Errors per day (target: <0.5/day)
2. **Repeat Mistake Rate:** Same error within 30 days (target: <5%)
3. **Task Completion Rate:** % of tasks completed successfully (target: >95%)
4. **Proactivity Score:** Proactive actions / opportunities (target: >80%)
5. **Correction Frequency:** Declining trend over time (target: -10% monthly)
6. **Response Quality:** Measured by clarity and completeness (target: >95%)

---

## 4. Learning Patterns

### Pattern Detection System

**Weekly Analysis Script** (`/rivet/scripts/analyze-patterns.js`):

```javascript
// Pseudo-code for pattern detection
function analyzePatterns() {
  const errors = loadErrorsFromPast30Days();
  
  // Detect recurring patterns
  const patterns = {
    recurring_errors: findRepeatingErrors(errors),
    time_patterns: findTimeBasedPatterns(errors),
    context_patterns: findContextPatterns(errors),
    category_trends: analyzeCategoryTrends(errors)
  };
  
  // Generate insights
  const insights = {
    "Most common mistake": getMostFrequent(errors),
    "Fastest recurring": getFastestRecurrence(errors),
    "Improvement areas": getAreasNeedingWork(errors),
    "Successful changes": getAreasImproving(errors)
  };
  
  // Write to error-patterns.md
  writePatternReport(patterns, insights);
}
```

### Pattern Categories to Track

1. **Temporal Patterns**
   - Time of day errors occur (fatigue/load related?)
   - Day of week patterns
   - Correlation with workload

2. **Task-Type Patterns**
   - Which types of tasks have highest error rates?
   - Which workflows need better documentation?

3. **Recurring Mistakes**
   - Same error >2x in 30 days = recurring
   - Flag for immediate process improvement

4. **Context Patterns**
   - Errors during multi-tasking?
   - Errors under time pressure?
   - Errors with specific tools/systems?

### Pattern Response Actions

When pattern detected → automated responses:

```
PATTERN: "Forgot to update TODO.md" occurred 3x in 2 weeks
ACTION: Create pre-task and post-task checklists
IMPLEMENT: Add TODO.md check to heartbeat routine
VERIFY: Monitor for 2 weeks, confirm reduction
```

Store in `/rivet/memory/patterns/active-improvements.json`:
```json
{
  "improvements": [
    {
      "pattern": "Forgot to update TODO.md after task completion",
      "frequency": 3,
      "timeframe": "14 days",
      "action": "Added TODO.md check to task completion checklist",
      "implemented": "2024-01-15",
      "verification_end": "2024-01-29",
      "status": "monitoring"
    }
  ]
}
```

---

## 5. Implementation Plan

### Phase 1: Foundation (Week 1)

**Files to Create:**
```
/rivet/memory/errors/
  └── error-categories.json

/rivet/memory/feedback/
  └── corrections-log.jsonl

/rivet/memory/metrics/
  └── daily-log.jsonl

/rivet/scripts/
  ├── log-error.sh
  ├── log-feedback.sh
  └── daily-metrics.sh
```

**Actions:**
1. Create directory structure
2. Initialize error-categories.json with base categories
3. Create bash scripts for logging
4. Test manual error logging

### Phase 2: Automation (Week 2)

**Files to Create:**
```
/rivet/scripts/
  ├── update-error-stats.js       # Updates category counts
  ├── daily-metrics.js            # Calculates daily metrics
  └── weekly-review.js            # Generates weekly summary
```

**Integration with Heartbeat:**

Update `/rivet/HEARTBEAT.md`:
```markdown
## Daily Checks (rotate through day)

Morning (8-10 AM):
- [ ] Check calendar for today
- [ ] Review yesterday's errors (if any)
- [ ] Update daily metrics

Midday (12-2 PM):
- [ ] Check for urgent emails/messages
- [ ] Quick self-assessment: any mistakes today?

Evening (6-8 PM):
- [ ] Log daily metrics (tasks completed, corrections received)
- [ ] Review TODO.md status
- [ ] Prepare tomorrow's priorities

Weekly (Sunday evening):
- [ ] Run weekly pattern analysis
- [ ] Update MEMORY.md with key lessons
- [ ] Review and update active improvements
```

### Phase 3: Pattern Detection (Week 3)

**Files to Create:**
```
/rivet/scripts/
  ├── analyze-patterns.js         # Pattern detection
  └── generate-report.js          # Weekly improvement report

/rivet/memory/patterns/
  ├── active-improvements.json
  └── pattern-history.md
```

**Weekly Review Process:**
1. Run `analyze-patterns.js` every Sunday
2. Generate improvement report
3. Update active-improvements.json
4. Send summary to Michael (optional)

### Phase 4: Continuous Improvement (Week 4+)

**Ongoing Activities:**
1. **Daily:** Log errors/feedback as they occur
2. **Daily:** Update metrics during heartbeat
3. **Weekly:** Run pattern analysis
4. **Monthly:** Generate performance report
5. **Monthly:** Review and update error categories

### Scripts Details

**`/rivet/scripts/daily-metrics.js`**
```javascript
#!/usr/bin/env node
const fs = require('fs');
const path = require('path');

function updateDailyMetrics() {
  const today = new Date().toISOString().split('T')[0];
  const metricsFile = `/home/ccuser/rateright-growth/rivet/memory/metrics/daily-log.jsonl`;
  
  // Gather data from various sources
  const metrics = {
    date: today,
    tasks_completed: countCompletedTasks(),
    tasks_failed: countFailedTasks(),
    corrections: countTodayCorrections(),
    proactive_actions: countProactiveActions(),
    notable_events: ""
  };
  
  // Append to log
  fs.appendFileSync(metricsFile, JSON.stringify(metrics) + '\n');
}

// Helper functions would query daily notes, error logs, etc.
```

**`/rivet/scripts/analyze-patterns.js`**
```javascript
#!/usr/bin/env node
// Reads past 30 days of error logs
// Detects patterns:
//   - Errors by category (frequency)
//   - Recurring errors (same subcategory >2x)
//   - Time-based patterns
// Outputs to /rivet/memory/patterns/error-patterns.md
```

**`/rivet/scripts/weekly-review.js`**
```javascript
#!/usr/bin/env node
// Generates weekly summary:
//   - Top 3 error categories
//   - Improvement trends
//   - New patterns detected
//   - Recommendations for next week
// Outputs to /rivet/memory/weekly-reviews/YYYY-WW.md
```

### Integration Points

**1. Error Detection (Manual)**
When Rivet recognizes a mistake or receives correction:
```bash
cd /rivet/scripts
./log-error.sh "task-execution" "incomplete-action" "medium" \
  "Failed to update TODO.md" \
  '{"task":"Calendar review","expected":"Update TODO","actual":"Skipped"}' \
  "Michael reminded me"
```

**2. Heartbeat Integration**
Add to heartbeat checks:
- Read today's error log (if exists)
- Update daily metrics
- Check for recurring patterns
- Run weekly review (if Sunday)

**3. Memory Updates**
When patterns detected or lessons learned:
- Update MEMORY.md with important insights
- Update AGENTS.md with process changes
- Create/update checklists in workflows/

**4. Feedback Loop**
```
Michael corrects → Log to corrections-log.jsonl → 
Analyze context → Update knowledge base → 
Confirm understanding → Monitor for improvement
```

---

## Success Criteria

**After 1 Month:**
- [ ] Error tracking system operational
- [ ] 90%+ of corrections logged
- [ ] Daily metrics collected consistently
- [ ] At least 1 pattern identified and addressed

**After 3 Months:**
- [ ] Measurable reduction in error frequency (>20%)
- [ ] Repeat mistake rate <5%
- [ ] 3+ process improvements implemented
- [ ] Weekly reviews happening automatically

**After 6 Months:**
- [ ] Error rate <0.3 per day
- [ ] Task completion rate >95%
- [ ] Proactivity score >80%
- [ ] Self-directed improvements happening regularly

---

## Maintenance

**Weekly:**
- Review error logs
- Run pattern analysis
- Update active improvements

**Monthly:**
- Generate performance report
- Review error categories (add/merge as needed)
- Update MEMORY.md with key learnings
- Prune old data (archive errors >6 months)

**Quarterly:**
- Deep review of all metrics
- Assess system effectiveness
- Adjust categories/metrics as needed
- Share progress report with Michael

---

## Example Workflow: Handling a Mistake

**Scenario:** Rivet forgot to send a follow-up email after a client call.

1. **Recognition:** Michael says "Did you send that follow-up to the client?"
2. **Logging:**
   ```bash
   ./log-error.sh "task-execution" "missed-step" "high" \
     "Forgot to send follow-up email after client call" \
     '{"client":"ABC Corp","call_date":"2024-01-15","expected":"Send email within 2h"}' \
     "Michael asked about it - should be automatic"
   ```

3. **Analysis:** Check if similar errors occurred before
   - Query: "missed-step" in task-execution category
   - Found: 2 previous instances in past month → **PATTERN DETECTED**

4. **Action:**
   - Create checklist: `/rivet/workflows/client-call-checklist.md`
   - Add to MEMORY.md: "After every client call, send follow-up within 2 hours"
   - Update heartbeat to check for pending follow-ups

5. **Verification:**
   - Monitor for 2 weeks
   - If no recurrence → pattern resolved
   - Update active-improvements.json

6. **Learning:**
   - Add to lessons-learned.md
   - Share insight: "Task-dependent follow-ups need explicit checklists"

---

## Future Enhancements

**Possible additions (if needed later):**
- Sentiment analysis on Michael's feedback
- Automated suggestions for process improvements
- Integration with task management system
- Predictive error prevention
- Comparative analysis (week-over-week trends)
- Success logging (not just errors - what went well?)

**Keep it simple first.** Start with basic logging, then add sophistication as patterns emerge.

---

*This system is Rivet's self-awareness mechanism. Use it consistently, learn genuinely, improve continuously.*