# VAPI Transcript → To-Do Pipeline Specification

## Overview
This pipeline automatically processes VAPI call transcripts after each phone call ends, extracts action items and commitments, and integrates them into Rivet's task management system.

## 1. VAPI Webhook Payload Structure

### End-of-Call Report
VAPI sends a webhook POST request to the configured server URL with:

```json
{
  "message": {
    "type": "end-of-call-report",
    "timestamp": 1757596207237,
    "call": {
      "id": "call-123",
      "customer": {
        "number": "+61400123456",
        "name": "John Smith"
      },
      "assistantId": "asst-abc123",
      "startedAt": "2026-02-06T10:00:00Z",
      "endedAt": "2026-02-06T10:15:30Z",
      "duration": 930,
      "status": "ended",
      "cost": 0.05
    },
    "artifact": {
      "recording": {
        "url": "https://storage.vapi.ai/recordings/call-123.mp3",
        "duration": 930
      },
      "transcript": "AI: Hi John, this is Rivet from RateRight. How can I help you today?\nJohn: I need to hire 3 electricians for a commercial job next week.\nAI: I'd be happy to help. What dates are you looking at?\nJohn: Probably Tuesday to Thursday, so 18th to 20th February.\nAI: Great. And what's the job location?\nJohn: 123 Construction St, Sydney CBD.\nAI: Perfect. I'll make sure to follow up with you on Monday to confirm the details and send you matched workers.",
      "messages": [
        {"role": "assistant", "message": "Hi John, this is Rivet from RateRight. How can I help you today?"},
        {"role": "user", "message": "I need to hire 3 electricians for a commercial job next week."},
        {"role": "assistant", "message": "I'd be happy to help. What dates are you looking at?"},
        {"role": "user", "message": "Probably Tuesday to Thursday, so 18th to 20th February."},
        {"role": "assistant", "message": "Great. And what's the job location?"},
        {"role": "user", "message": "123 Construction St, Sydney CBD."},
        {"role": "assistant", "message": "Perfect. I'll make sure to follow up with you on Monday to confirm the details and send you matched workers."}
      ]
    },
    "analysis": {
      "summary": "Customer John Smith needs to hire 3 electricians for a commercial job in Sydney CBD from February 18-20. Rivet committed to following up on Monday with matched workers.",
      "successEvaluation": "Call went well, clear requirements captured, follow-up scheduled"
    }
  }
}
```

### Key Fields for Processing
- `message.type`: Must be "end-of-call-report"
- `message.artifact.transcript`: Full conversation text
- `message.artifact.messages`: Structured conversation (alternative to transcript)
- `message.call.customer`: Caller information
- `message.call.startedAt/endedAt`: Call timing
- `message.analysis.summary`: AI-generated summary (supplementary)

## 2. Action Item Extraction Approach

### LLM-Based Extraction
Use Claude Opus with a carefully crafted prompt to extract structured action items:

```python
ACTION_ITEM_PROMPT = """You are an expert at extracting commitments and action items from phone call transcripts.

Analyze this call transcript and extract:
1. Specific commitments made by Rivet/assistant
2. Follow-up tasks that need to be completed
3. Time-sensitive actions with deadlines
4. Contact information or details gathered
5. Any decisions made that require action

For each action item, provide:
- Task description (clear and actionable)
- Due date/time (if mentioned)
- Priority (High/Medium/Low based on urgency)
- Category (Follow-up/Research/Action/Reminder)
- Related context (who, what, when)

Transcript:
{transcript}

Respond with a JSON array of action items."""
```

### Extraction Criteria
- **High Priority**: Time-sensitive commitments (same day, next day), urgent customer needs
- **Medium Priority**: Follow-ups within a week, research tasks, planning activities
- **Low Priority**: General information gathering, non-urgent updates

### Example Extracted Items
```json
[
  {
    "task": "Follow up with John Smith about electrician hiring",
    "dueDate": "2026-02-09T09:00:00Z",
    "priority": "High",
    "category": "Follow-up",
    "context": "Customer needs 3 electricians for Feb 18-20 job in Sydney CBD",
    "contact": "John Smith - +61400123456"
  },
  {
    "task": "Search for available electricians in Sydney CBD area",
    "dueDate": "2026-02-08T17:00:00Z",
    "priority": "High",
    "category": "Action",
    "context": "Need 3 workers for commercial electrical job"
  }
]
```

## 3. Integration with TODO.md

### Task Format
Action items are added to TODO.md in this format:

```markdown
### 📞 VAPI Calls - Action Items
- [ ] [HIGH] Follow up with John Smith about electrician hiring (due: Mon 9am) - From call 2026-02-06
- [ ] [HIGH] Search for available electricians in Sydney CBD area (due: Sun 5pm) - From call 2026-02-06
```

### Processing Rules
1. **Deduplication**: Check for similar recent tasks before adding
2. **Date Parsing**: Convert relative dates ("Monday", "next week") to absolute dates
3. **Contact Info**: Append phone numbers for follow-up tasks
4. **Context**: Include brief context from the call
5. **Priority Markers**: Use [HIGH]/[MED]/[LOW] tags

### Integration Script
```bash
# Add to daily memory file
echo "## VAPI Call Action Items - $(date +%Y-%m-%d)" >> memory/$(date +%Y-%m-%d).md
echo "- Action items extracted from call with $CUSTOMER_NAME" >> memory/$(date +%Y-%m-%d).md
echo "- Call duration: $DURATION seconds" >> memory/$(date +%Y-%m-%d).md
echo "- Action items: $ITEM_COUNT" >> memory/$(date +%Y-%m-%d).md
```

## 4. Notification Logic

### When to Notify Michael
Send immediate notification via message tool when:
1. **High-value leads**: Commercial jobs >$1000 or ongoing contracts
2. **Urgent timeframes**: Jobs needed within 24-48 hours
3. **Multiple hires**: Requests for 5+ workers
4. **Repeat customers**: Previous customers calling back
5. **Complaints or issues**: Any negative sentiment detected

### Notification Format
```
🚨 VAPI Call Alert
From: John Smith (+61400123456)
Time: 10:15am today
Summary: Needs 3 electricians for commercial job (Sydney CBD, Feb 18-20)
Action: Follow-up scheduled Monday 9am
Value: ~$150 (3 x $50 hire fees)
```

### Notification Channels
- Telegram: Rocky & Rivet group (topic 4 - Sales)
- Daily summary: Added to morning brief if items exist

## 5. Implementation Plan

### Phase 1: Basic Pipeline (Week 1)
1. **Webhook Receiver**: Create endpoint to receive VAPI webhooks
2. **Transcript Storage**: Save transcripts to `/memory/calls/YYYY-MM-DD-call-ID.md`
3. **Basic Extraction**: Simple regex/pattern matching for commitments
4. **TODO Integration**: Append items to TODO.md

### Phase 2: AI Enhancement (Week 2)
1. **LLM Integration**: Connect to Claude Opus for extraction
2. **Smart Parsing**: Improve date/contact extraction
3. **Deduplication**: Prevent duplicate tasks
4. **Priority Scoring**: Better priority assignment

### Phase 3: Advanced Features (Week 3)
1. **Context Awareness**: Link to previous calls from same number
2. **CRM Integration**: Update customer records
3. **Analytics**: Track conversion rates from calls
4. **Auto-Follow-up**: Send SMS reminders for due tasks

### Technical Stack
- **Runtime**: Node.js/Express webhook server
- **AI Model**: Claude Opus via Anthropic API
- **Storage**: Append to TODO.md and daily memory files
- **Notifications**: Telegram via message tool
- **Security**: Webhook signature verification

### Deployment
1. Deploy webhook server to Railway/Fly.io
2. Configure VAPI assistant with webhook URL
3. Set up environment variables for API keys
4. Test with sample calls
5. Monitor and refine extraction accuracy

### Monitoring
- Log all processed calls with action item count
- Track extraction accuracy (manually review samples)
- Monitor webhook delivery success rate
- Alert on processing failures

## Error Handling

### Webhook Failures
- Retry 3 times with exponential backoff
- Log failures to error log
- Alert if >5 failures in 1 hour

### Extraction Failures
- If LLM fails, fall back to simple pattern matching
- Log failed extractions for manual review
- Save raw transcript regardless

### File System Issues
- Create backup of TODO.md before modifications
- Write to temp file first, then atomic move
- Log all file operations

## Security Considerations

1. **Webhook Authentication**: Verify VAPI signatures
2. **Rate Limiting**: Prevent abuse of endpoint
3. **Data Sanitization**: Clean phone numbers/emails from logs
4. **Access Control**: Restrict webhook endpoint by IP
5. **Encryption**: Use HTTPS for all webhook communications

## Success Metrics

- **Processing Rate**: 100% of calls processed within 5 minutes
- **Extraction Accuracy**: >90% of commitments captured
- **False Positives**: <5% irrelevant tasks created
- **Notification Response**: Michael responds to >50% of high-priority alerts
- **Task Completion**: >80% of extracted tasks completed on time