# Self-Improving Intelligence System > **Vision:** A CRM that analyzes itself, identifies friction, suggests improvements, measures impact, and evolves its own analysis capabilities monthly. > > **Goal:** 0.1% of 0.1% - The system that makes the system better, then makes itself better. --- ## System Overview ``` ┌─────────────────────────────────────────────────────────────────┐ │ SELF-IMPROVING INTELLIGENCE │ ├─────────────────────────────────────────────────────────────────┤ │ │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │ │ │ REAL-TIME │ │ WEEKLY │ │ MONTHLY │ │ │ │ ALERTS │ │ EXCELLENCE │ │ DEEP DIVE + │ │ │ │ │ │ REVIEW │ │ SELF-IMPROVEMENT │ │ │ │ < 1 min │ │ Monday 7am │ │ 1st of month │ │ │ └──────┬───────┘ └──────┬───────┘ └──────────┬───────────┘ │ │ │ │ │ │ │ └────────────┬────┴──────────────────────┘ │ │ ▼ │ │ ┌────────────────────────┐ │ │ │ INTELLIGENCE CORE │ │ │ │ ─────────────────────│ │ │ │ • Usage Analytics │ │ │ │ • Pattern Detection │ │ │ │ • Friction Scoring │ │ │ │ • Impact Measurement │ │ │ │ • Self-Evolution │ │ │ └────────────────────────┘ │ │ │ │ │ ▼ │ │ ┌────────────────────────┐ │ │ │ ACTION & LEARNING │ │ │ │ ─────────────────────│ │ │ │ • Slack Reports │ │ │ │ • Suggestion Tracking │ │ │ │ • Impact Analysis │ │ │ │ • Meta-Improvement │ │ │ └────────────────────────┘ │ │ │ └─────────────────────────────────────────────────────────────────┘ ``` --- ## Part 1: Real-Time Critical Alerts ### Purpose Catch critical issues immediately - don't wait for weekly review when something breaks. ### Trigger Conditions | Alert Type | Condition | Severity | Action | |------------|-----------|----------|--------| | **Error Spike** | >10 errors in 5 minutes | CRITICAL | Immediate Slack | | **API Failure** | External API down (Twilio, OpenAI) | CRITICAL | Immediate Slack | | **Feature Break** | Key flow completion drops >50% | HIGH | Immediate Slack | | **Performance Degradation** | Response time >5s average | HIGH | Immediate Slack | | **Conversion Anomaly** | 0 conversions in 2+ days (when normally >0) | MEDIUM | Slack within 1hr | | **SMS Delivery Failure** | >20% delivery failure rate | HIGH | Immediate Slack | | **AI Quality Drop** | AI responses getting thumbs-down >30% | MEDIUM | Slack within 1hr | ### Data Collection (Continuous) ```javascript // Every API request logs: { endpoint: '/api/leads/123', method: 'GET', user_id: 'uuid', duration_ms: 145, status_code: 200, error: null, timestamp: '2026-01-20T10:30:00Z' } // Every user action logs: { action: 'button_click', element: 'call_now_button', page: 'call_list', user_id: 'uuid', context: { lead_id: '123', position_in_list: 2 }, timestamp: '2026-01-20T10:30:00Z' } // Every error logs: { error_type: 'API_ERROR', message: 'Failed to fetch lead', stack: '...', user_id: 'uuid', page: 'lead_profile', context: { lead_id: '123' }, timestamp: '2026-01-20T10:30:00Z' } ``` ### Alert Format (Slack) ``` 🚨 CRITICAL ALERT: SMS Delivery Failing What: 23 of 30 SMS messages failed delivery in last 15 minutes Impact: Sales team cannot reach leads Likely Cause: Twilio API returning 429 (rate limit) Suggested Action: 1. Check Twilio dashboard for account status 2. Verify phone number validity 3. Check if we hit daily limit Timeline: • 10:15am - First failures detected • 10:20am - Failure rate exceeded 50% • 10:22am - This alert triggered [View Error Logs] [Silence for 1hr] ``` --- ## Part 2: Weekly 0.1% Excellence Review ### Schedule **Every Monday at 7:15am AEST** (after the "What's Working" report at 7:00am) ### Analysis Categories #### 2.1 Friction Detection AI analyzes user flows to find friction: ``` FRICTION ANALYSIS PROMPT: Analyze this week's usage data and identify the top friction points: Usage Data: - Page views: {page_view_counts} - Flow completions: {flow_completion_rates} - Time on page: {avg_time_per_page} - Rage clicks: {rapid_repeat_clicks} - Back button usage: {back_navigation_counts} - Abandoned flows: {started_but_not_completed} - Error encounters: {errors_by_page} Previous friction points addressed: {last_month_fixes} Identify: 1. Where users struggle most (high time, low completion) 2. Where users give up (abandoned flows) 3. Where users get frustrated (rage clicks, back buttons) 4. What's taking too many steps (could be simplified) For each friction point, provide: - Location (page, feature, flow) - Evidence (specific metrics) - Impact score (users affected × frequency × severity) - Root cause hypothesis - Suggested fix (specific, actionable) - Estimated effort (Low/Medium/High) ``` #### 2.2 Unused Feature Detection ``` UNUSED FEATURE ANALYSIS: Features and their usage this week: {feature_usage_matrix} Features usage trend (4-week): {feature_usage_trend} Identify: 1. Features with <5% adoption (never caught on) 2. Features with declining usage (losing relevance) 3. Features used once then abandoned (confusing?) 4. Features only power users use (hidden gems or too complex?) For each, recommend: - PROMOTE: Feature is valuable but hidden → improve discoverability - SIMPLIFY: Feature is useful but complex → reduce steps - REMOVE: Feature adds clutter with no value → deprecate - INVESTIGATE: Need more data → add specific tracking ``` #### 2.3 Pattern Opportunities ``` PATTERN OPPORTUNITY ANALYSIS: High performers (top 20% by conversions): {top_performer_behaviors} Average performers: {average_performer_behaviors} Identify behavioral differences: 1. What do top performers do that others don't? 2. What sequence of actions leads to conversions? 3. What time patterns correlate with success? 4. What features do winners use more? Generate actionable insights: - "Top performers call within 3 minutes of SMS reply" → Suggestion: Add "Hot Reply" notification with one-tap call - "Winners use Intel Brief before 80% of calls" → Suggestion: Make Intel Brief default before every call ``` #### 2.4 AI Quality Assessment ``` AI QUALITY ANALYSIS: AI feature usage and feedback: - Message drafts: {count} generated, {thumbs_up}% positive - Objection responses: {count} generated, {thumbs_up}% positive - Intel briefs: {count} generated, {avg_time_viewed}s viewed - Script suggestions: {count} generated, {used_rate}% used - Copilot suggestions: {count} shown, {accepted_rate}% accepted Identify: 1. Which AI features are trusted (high acceptance)? 2. Which AI features are ignored (generated but not used)? 3. What patterns in rejected AI suggestions? 4. Where is AI adding friction instead of removing it? Suggestions: - Improve prompt for low-acceptance features - A/B test different AI approaches - Remove AI from places where it's ignored ``` #### 2.5 Quick Wins Identification ``` QUICK WIN ANALYSIS: Based on all analysis above, identify the TOP 3 improvements that: 1. Have HIGH impact (affects many users, frequently) 2. Have LOW effort (can be built in <4 hours) 3. Have CLEAR solution (not ambiguous) Format each as: ┌────────────────────────────────────────┐ │ QUICK WIN #1: [Title] │ ├────────────────────────────────────────┤ │ Problem: [Specific friction] │ │ Evidence: [Metrics] │ │ Impact: [Users × Frequency] │ │ Solution: [Specific change] │ │ Effort: [Hours estimate] │ │ Files: [Specific files to modify] │ └────────────────────────────────────────┘ ``` ### Weekly Report Format (Slack) ``` 📊 WEEKLY 0.1% EXCELLENCE REVIEW Week of January 20-26, 2026 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 🎯 TOP FRICTION POINT CallPrep page abandonment: 34% leave before calling • Avg load time: 4.2 seconds (target: <2s) • Users wait, then tap back → FIX: Pre-cache intel for top 10 call list leads → EFFORT: Medium (4-6 hours) → IMPACT: ~40 fewer abandoned preps/week ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ⚡ QUICK WIN OPPORTUNITY "Mark Hot + Call" could be one button • 67% of "Mark Hot" taps followed by "Call" within 10s • Currently: 2 taps, page refresh between → FIX: Add "🔥 Call Now" button that marks hot + dials → EFFORT: Low (2 hours) → IMPACT: Save 134 taps/week ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 🏆 WINNER PATTERN DETECTED Tony converts 3.2x more when calling <5 min after SMS reply Other users average: 18 min response time → FIX: "Hot Reply" push notification with one-tap call → EFFORT: Medium (6-8 hours) → IMPACT: Potential 40% conversion lift for team ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 😴 UNUSED FEATURE ALERT Voice Commands: Used 3x this week (was 24x/week in December) • No errors detected • Users just stopped using it → INVESTIGATE: Survey users or remove from UI? ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 🤖 AI QUALITY CHECK • Message Drafts: 89% accepted ✅ • Objection Responses: 72% accepted ⚠️ (down from 81%) • Intel Briefs: Avg 23s viewed (healthy) → NOTE: Objection quality declining - review prompts ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 📈 LAST WEEK'S SUGGESTION IMPACT Suggested: "Add loading skeleton to Dashboard" Status: ✅ Implemented Jan 15 Result: Perceived load time ↓ 60%, bounce rate ↓ 12% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ PRIORITY RECOMMENDATION: 1. [LOW EFFORT] Add "🔥 Call Now" button 2. [HIGH IMPACT] Pre-cache CallPrep intel 3. [INVESTIGATE] Voice Commands decline [View Full Report] [Mark Suggestion as Planned] ``` --- ## Part 3: Monthly Deep Dive + Self-Improvement ### Schedule **1st of each month at 8:00am AEST** ### 3.1 Strategic Analysis ``` MONTHLY STRATEGIC ANALYSIS: Performance Trends (4-week rolling): - Conversion rate: {trend} - Calls per day: {trend} - SMS response rate: {trend} - Avg deal value: {trend} - Time to conversion: {trend} Feature Adoption Trends: {feature_adoption_over_time} User Satisfaction Signals: - AI suggestion acceptance: {trend} - Feature complaints (inferred from rage clicks): {trend} - Session duration: {trend} - Return user rate: {trend} Generate: 1. WINS: What improved this month and why? 2. CONCERNS: What's declining and why? 3. OPPORTUNITIES: What patterns suggest new features? 4. RISKS: What could break if we don't address? ``` ### 3.2 Competitive Intelligence ``` MONTHLY COMPETITOR SCAN: Using web search, analyze: 1. Top 5 sales CRM product updates this month 2. Features competitors launched that we don't have 3. Industry trends in sales tech 4. User complaints about competitors (Reddit, G2, Capterra) Generate: - Features we should consider adding - Features we have that competitors don't (our moat) - Pricing/positioning insights - Emerging technologies to watch ``` ### 3.3 Suggestion Effectiveness Review ``` SUGGESTION EFFECTIVENESS ANALYSIS: All suggestions from past month: {suggestions_list} For each implemented suggestion: - Before metrics: {baseline} - After metrics: {current} - Calculated impact: {improvement_percentage} - Verdict: SUCCESS / PARTIAL / FAILED / INCONCLUSIVE For suggestions NOT implemented: - Reason: {not_implemented_reason} - Still relevant? {yes_no} - Re-prioritize? {yes_no} Meta-analysis: - What types of suggestions had highest success rate? - What types were ignored? - How accurate were our effort estimates? - How accurate were our impact predictions? ``` ### 3.4 META: Self-Improvement of the System This is the 0.1% of 0.1% - the system improves its own analysis. ``` SELF-IMPROVEMENT ANALYSIS: Review of Intelligence System Performance: 1. ALERT QUALITY - Alerts triggered: {count} - True positives: {count} (real issues) - False positives: {count} (noise) - Missed issues: {count} (should have alerted) → Adjust: {specific_threshold_changes} 2. WEEKLY ANALYSIS QUALITY - Suggestions made: {count} - Implemented: {count} - Successful: {count} - Prediction accuracy: {percentage} → Adjust: {prompt_improvements} 3. FRICTION DETECTION ACCURACY - Friction points identified: {count} - Validated by user behavior change: {count} - False friction (users fine actually): {count} → Adjust: {friction_scoring_changes} 4. PATTERN RECOGNITION QUALITY - Patterns identified: {count} - Patterns that led to improvements: {count} - Patterns that were noise: {count} → Adjust: {pattern_detection_changes} 5. NEW METRICS NEEDED Based on this month's analysis, we should track: - {new_metric_1}: Because {reason} - {new_metric_2}: Because {reason} 6. OBSOLETE METRICS Stop tracking: - {old_metric}: Because {reason} 7. PROMPT EVOLUTION Current prompts that need refinement: - {prompt_name}: {specific_improvement} 8. NEW ANALYSIS CATEGORIES Based on emerging patterns, add these analyses: - {new_analysis_type}: Because {reason} ``` ### Monthly Report Format (Slack) ``` 🧠 MONTHLY INTELLIGENCE DEEP DIVE January 2026 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 📈 WINS THIS MONTH • Conversion rate: 12% → 15% (+25%) Attributed to: Faster SMS response (avg 8min → 3min) • CallPrep adoption: 45% → 78% (+73%) Attributed to: Pre-caching fix from Week 2 • AI acceptance: 71% → 84% (+18%) Attributed to: Prompt improvements from Week 3 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ⚠️ CONCERNS • Voice Commands usage: ↓ 85% from peak Hypothesis: Novelty wore off, not habit-forming Recommendation: Either improve or deprecate • Weekend activity: ↓ 40% vs weekdays Hypothesis: No push notifications on mobile Recommendation: Implement PWA push (in backlog) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 🎯 STRATEGIC OPPORTUNITIES 1. "Speed to Lead" Feature Pattern: Leads contacted <5 min convert 4x better Opportunity: Real-time new lead alerts + one-tap call Estimated impact: +20% conversions 2. "Similar Lead" Campaigns Pattern: When one plumber converts, nearby plumbers 2x likely Opportunity: Auto-suggest "Contact similar leads" after win Estimated impact: +15% pipeline ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 🔍 COMPETITOR INTEL • HubSpot launched "AI Sales Coach" - scores calls, suggests improvements → We have Live Copilot, but no post-call scoring → Consider: AI Call Scoring feature • Salesforce added "Einstein Activity Capture" - auto-logs emails → We don't have email integration → Consider: Email integration (P2) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 📊 SUGGESTION SCORECARD Weekly suggestions made: 12 Implemented: 7 (58%) Successful: 6 (86% of implemented) Impact prediction accuracy: 73% Best suggestion: "Pre-cache CallPrep" Predicted: 30% less abandonment Actual: 42% less abandonment ✅ Worst suggestion: "Add tooltips to Dashboard" Predicted: Reduce confusion Actual: No measurable change, removed ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 🤖 SELF-IMPROVEMENT APPLIED 1. ALERT TUNING • Raised error spike threshold 10 → 15 (Too many false alarms from retry storms) • Added "SMS queue depth" alert (Missed a delivery backup last week) 2. ANALYSIS IMPROVEMENTS • Added "rage click" detection to friction analysis • Improved pattern prompt to weight recency • Removed "page scroll depth" metric (not predictive) 3. NEW TRACKING ADDED • Time from SMS reply to callback • Feature discovery path (how users find features) • AI suggestion edit distance (how much users modify AI text) 4. PROMPT EVOLUTION • Friction detection prompt v3 → v4 Added: Consider mobile vs desktop separately • Quick win prompt v2 → v3 Added: Verify fix doesn't break other flows ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 🎯 NEXT MONTH FOCUS Based on all analysis, the system recommends: PRIORITY 1: Speed to Lead notification system PRIORITY 2: AI Call Scoring (match competitor) PRIORITY 3: Deprecate or revamp Voice Commands [View Full Report] [Export to PDF] [Schedule Review Meeting] ``` --- ## Part 4: Database Schema ```sql -- ============================================ -- SELF-IMPROVING INTELLIGENCE SYSTEM SCHEMA -- ============================================ -- 1. Usage Analytics (Raw Events) -- PARTITIONED BY MONTH for performance at scale -- Retention: 90 days raw events, aggregated metrics kept forever CREATE TABLE IF NOT EXISTS usage_events ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), event_type VARCHAR(50) NOT NULL, -- 'page_view', 'button_click', 'api_call', 'error' event_name VARCHAR(100) NOT NULL, -- 'call_list_view', 'mark_hot_click', etc. user_id UUID REFERENCES auth.users(id), page VARCHAR(100), element VARCHAR(100), context JSONB, -- { lead_id, position, previous_page, etc. } duration_ms INTEGER, timestamp TIMESTAMPTZ DEFAULT NOW(), session_id UUID, device_type VARCHAR(20), -- 'mobile', 'desktop', 'tablet' created_at TIMESTAMPTZ DEFAULT NOW() ) PARTITION BY RANGE (created_at); -- Create monthly partitions (auto-create new ones via cron job) CREATE TABLE IF NOT EXISTS usage_events_2026_01 PARTITION OF usage_events FOR VALUES FROM ('2026-01-01') TO ('2026-02-01'); CREATE TABLE IF NOT EXISTS usage_events_2026_02 PARTITION OF usage_events FOR VALUES FROM ('2026-02-01') TO ('2026-03-01'); -- ... continue for each month -- Retention policy: Delete raw events older than 90 days -- Run daily via cron job: -- DELETE FROM usage_events WHERE created_at < NOW() - INTERVAL '90 days'; -- Note: Aggregated daily_metrics table keeps summaries forever -- Indexes for fast querying (on each partition) CREATE INDEX idx_usage_events_timestamp ON usage_events(timestamp); CREATE INDEX idx_usage_events_type ON usage_events(event_type); CREATE INDEX idx_usage_events_user ON usage_events(user_id); CREATE INDEX idx_usage_events_page ON usage_events(page); -- 2. Aggregated Metrics (Computed Daily) CREATE TABLE IF NOT EXISTS daily_metrics ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), date DATE NOT NULL, metric_name VARCHAR(100) NOT NULL, -- 'call_prep_abandonment_rate', 'avg_response_time', etc. metric_value DECIMAL(10,4) NOT NULL, metric_context JSONB, -- { breakdown_by_user: {...}, breakdown_by_page: {...} } created_at TIMESTAMPTZ DEFAULT NOW(), UNIQUE(date, metric_name) ); CREATE INDEX idx_daily_metrics_date ON daily_metrics(date); CREATE INDEX idx_daily_metrics_name ON daily_metrics(metric_name); -- 3. Friction Points (Detected) CREATE TABLE IF NOT EXISTS detected_friction ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), detection_date DATE NOT NULL, friction_type VARCHAR(50) NOT NULL, -- 'high_abandonment', 'slow_load', 'rage_clicks', 'error_prone' location VARCHAR(100) NOT NULL, -- page or feature name severity_score DECIMAL(5,2) NOT NULL, -- 0-100 affected_users INTEGER, frequency_per_week INTEGER, evidence JSONB, -- { metrics, user_paths, error_logs } root_cause_hypothesis TEXT, suggested_fix TEXT, estimated_effort VARCHAR(20), -- 'low', 'medium', 'high' status VARCHAR(20) DEFAULT 'detected', -- 'detected', 'acknowledged', 'fixing', 'resolved', 'wont_fix' resolved_at TIMESTAMPTZ, created_at TIMESTAMPTZ DEFAULT NOW() ); -- 4. Intelligence Suggestions CREATE TABLE IF NOT EXISTS intelligence_suggestions ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), suggestion_type VARCHAR(50) NOT NULL, -- 'quick_win', 'friction_fix', 'pattern_opportunity', 'feature_idea' title VARCHAR(200) NOT NULL, description TEXT NOT NULL, evidence JSONB, -- supporting data predicted_impact VARCHAR(100), predicted_effort VARCHAR(20), priority_score DECIMAL(5,2), -- calculated: impact / effort source VARCHAR(50), -- 'weekly_review', 'monthly_deep_dive', 'real_time' status VARCHAR(20) DEFAULT 'suggested', -- 'suggested', 'planned', 'in_progress', 'implemented', 'rejected', 'deferred' -- Implementation tracking implemented_at TIMESTAMPTZ, implemented_by UUID, -- Impact measurement baseline_metrics JSONB, post_implementation_metrics JSONB, actual_impact VARCHAR(100), impact_measured_at TIMESTAMPTZ, success_verdict VARCHAR(20), -- 'success', 'partial', 'failed', 'inconclusive' -- Learning learnings TEXT, created_at TIMESTAMPTZ DEFAULT NOW(), updated_at TIMESTAMPTZ DEFAULT NOW() ); CREATE INDEX idx_suggestions_status ON intelligence_suggestions(status); CREATE INDEX idx_suggestions_type ON intelligence_suggestions(suggestion_type); -- 5. Pattern Library (Discovered Patterns) CREATE TABLE IF NOT EXISTS discovered_patterns ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), pattern_type VARCHAR(50) NOT NULL, -- 'winner_behavior', 'conversion_signal', 'churn_risk', 'feature_combo' pattern_name VARCHAR(200) NOT NULL, description TEXT, evidence JSONB, confidence_score DECIMAL(3,2), -- 0.00 to 1.00 times_validated INTEGER DEFAULT 0, last_validated_at TIMESTAMPTZ, actionable_insight TEXT, auto_action_enabled BOOLEAN DEFAULT FALSE, -- can system act on this automatically? auto_action_config JSONB, status VARCHAR(20) DEFAULT 'active', -- 'active', 'deprecated', 'testing' created_at TIMESTAMPTZ DEFAULT NOW(), updated_at TIMESTAMPTZ DEFAULT NOW() ); -- 6. Alerts Log CREATE TABLE IF NOT EXISTS intelligence_alerts ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), alert_type VARCHAR(50) NOT NULL, severity VARCHAR(20) NOT NULL, -- 'critical', 'high', 'medium', 'low' title VARCHAR(200) NOT NULL, description TEXT, evidence JSONB, suggested_action TEXT, -- Response tracking acknowledged_at TIMESTAMPTZ, acknowledged_by UUID, resolved_at TIMESTAMPTZ, resolution_notes TEXT, was_true_positive BOOLEAN, -- for learning created_at TIMESTAMPTZ DEFAULT NOW() ); CREATE INDEX idx_alerts_created ON intelligence_alerts(created_at); CREATE INDEX idx_alerts_severity ON intelligence_alerts(severity); -- 7. System Self-Improvement Log CREATE TABLE IF NOT EXISTS system_evolution ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), evolution_type VARCHAR(50) NOT NULL, -- 'threshold_change', 'prompt_update', 'metric_added', 'metric_removed', 'analysis_added' component VARCHAR(100) NOT NULL, -- 'real_time_alerts', 'weekly_analysis', 'friction_detection', etc. change_description TEXT NOT NULL, old_value JSONB, new_value JSONB, reason TEXT NOT NULL, performance_before JSONB, -- metrics showing why change was needed performance_after JSONB, -- filled in after change takes effect effective_date DATE NOT NULL, created_at TIMESTAMPTZ DEFAULT NOW() ); -- 8. Analysis Prompts (Versioned) -- DUAL STORAGE: Database for runtime + performance tracking, Git for evolution history -- Git location: src/intelligence/prompts/*.md (version controlled) -- Database stores: active version, performance scores, usage stats -- This lets you see prompt evolution alongside performance scores CREATE TABLE IF NOT EXISTS analysis_prompts ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), prompt_name VARCHAR(100) NOT NULL, -- 'friction_detection', 'pattern_discovery', 'quick_win_identification' version INTEGER NOT NULL, prompt_text TEXT NOT NULL, git_commit_sha VARCHAR(40), -- links to git history for full evolution context is_active BOOLEAN DEFAULT TRUE, performance_score DECIMAL(5,2), -- how well suggestions from this prompt perform suggestions_generated INTEGER DEFAULT 0, suggestions_successful INTEGER DEFAULT 0, avg_implementation_rate DECIMAL(5,2), -- % of suggestions that get implemented created_at TIMESTAMPTZ DEFAULT NOW(), deprecated_at TIMESTAMPTZ, deprecation_reason TEXT, -- why this version was replaced UNIQUE(prompt_name, version) ); -- Git-tracked prompt files structure: -- src/intelligence/prompts/ -- friction_detection_v1.md -- friction_detection_v2.md -- pattern_discovery_v1.md -- quick_win_identification_v1.md -- CHANGELOG.md (links versions to performance outcomes) -- 9. Weekly Reports Archive CREATE TABLE IF NOT EXISTS weekly_excellence_reports ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), week_start DATE NOT NULL, week_end DATE NOT NULL, -- Report content top_friction JSONB, quick_wins JSONB, patterns_detected JSONB, unused_features JSONB, ai_quality_assessment JSONB, -- Summary metrics friction_score DECIMAL(5,2), -- overall friction level health_score DECIMAL(5,2), -- overall system health improvement_opportunities INTEGER, -- Full report full_report_json JSONB, -- SLACK THREADING: Keep message_ts to thread follow-ups under the weekly report -- When weekly report posts → save message_ts -- When suggestion gets implemented → reply in thread: "✅ Quick Win #1 implemented!" -- When mid-week alert relates to friction → thread under report -- When impact measured → thread: "Quick Win #1 result: 23% improvement" slack_channel_id VARCHAR(20), -- e.g., 'C0123456789' slack_message_ts VARCHAR(50), -- e.g., '1234567890.123456' - Slack's unique message ID slack_thread_reply_count INTEGER DEFAULT 0, -- track engagement created_at TIMESTAMPTZ DEFAULT NOW() ); -- Slack threading helper: find the current week's report for threading CREATE INDEX idx_weekly_reports_week ON weekly_excellence_reports(week_start DESC); -- 10. Monthly Reports Archive CREATE TABLE IF NOT EXISTS monthly_deep_dive_reports ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), month DATE NOT NULL, -- first of month -- Strategic analysis wins JSONB, concerns JSONB, opportunities JSONB, risks JSONB, -- Competitor intel competitor_analysis JSONB, -- Suggestion effectiveness suggestion_scorecard JSONB, -- Self-improvement applied system_evolutions JSONB, -- Full report full_report_json JSONB, created_at TIMESTAMPTZ DEFAULT NOW(), UNIQUE(month) ); -- ============================================ -- VIEWS FOR COMMON QUERIES -- ============================================ -- Active friction points needing attention CREATE OR REPLACE VIEW v_active_friction AS SELECT * FROM detected_friction WHERE status IN ('detected', 'acknowledged') ORDER BY severity_score DESC; -- Pending suggestions CREATE OR REPLACE VIEW v_pending_suggestions AS SELECT * FROM intelligence_suggestions WHERE status IN ('suggested', 'planned') ORDER BY priority_score DESC; -- Recent alerts needing response CREATE OR REPLACE VIEW v_unresolved_alerts AS SELECT * FROM intelligence_alerts WHERE resolved_at IS NULL ORDER BY CASE severity WHEN 'critical' THEN 1 WHEN 'high' THEN 2 WHEN 'medium' THEN 3 ELSE 4 END, created_at DESC; -- Suggestion success rate by type CREATE OR REPLACE VIEW v_suggestion_success_rate AS SELECT suggestion_type, COUNT(*) as total, COUNT(*) FILTER (WHERE status = 'implemented') as implemented, COUNT(*) FILTER (WHERE success_verdict = 'success') as successful, ROUND( COUNT(*) FILTER (WHERE success_verdict = 'success')::DECIMAL / NULLIF(COUNT(*) FILTER (WHERE status = 'implemented'), 0) * 100, 1 ) as success_rate FROM intelligence_suggestions GROUP BY suggestion_type; ``` --- ## Part 5: API Endpoints ``` # Real-Time Tracking POST /api/intelligence/track - Log usage event - Body: { event_type, event_name, page, element, context } # Alerts GET /api/intelligence/alerts - List recent alerts - Query: ?severity=critical&resolved=false POST /api/intelligence/alerts/:id/acknowledge - Acknowledge an alert POST /api/intelligence/alerts/:id/resolve - Resolve an alert with notes # Suggestions GET /api/intelligence/suggestions - List suggestions - Query: ?status=suggested&type=quick_win POST /api/intelligence/suggestions/:id/status - Update suggestion status - Body: { status: 'planned' | 'in_progress' | 'implemented' | 'rejected' } POST /api/intelligence/suggestions/:id/measure-impact - Record post-implementation metrics - Body: { metrics, verdict, learnings } # Reports GET /api/intelligence/weekly-report - Get latest weekly report - Query: ?week=2026-01-20 GET /api/intelligence/monthly-report - Get latest monthly report - Query: ?month=2026-01 POST /api/intelligence/run-weekly-analysis - Manually trigger weekly analysis POST /api/intelligence/run-monthly-analysis - Manually trigger monthly analysis # Friction GET /api/intelligence/friction - List detected friction points - Query: ?status=detected&min_severity=50 # Patterns GET /api/intelligence/patterns - List discovered patterns - Query: ?type=winner_behavior&min_confidence=0.7 # System Evolution GET /api/intelligence/evolution-log - View system self-improvements - Query: ?component=weekly_analysis&since=2026-01-01 ``` --- ## Part 6: Frontend Integration ### 6.1 Tracking Hook ```javascript // hooks/useIntelligenceTracking.js import { useCallback, useEffect, useRef } from 'react'; import { v4 as uuidv4 } from 'uuid'; export function useIntelligenceTracking() { const sessionId = useRef(uuidv4()); const pageStartTime = useRef(Date.now()); // Track page view const trackPageView = useCallback((page, context = {}) => { fetch('/api/intelligence/track', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ event_type: 'page_view', event_name: `${page}_view`, page, context, session_id: sessionId.current, device_type: window.innerWidth < 768 ? 'mobile' : 'desktop' }) }).catch(() => {}); // Silent fail }, []); // Track button click const trackClick = useCallback((element, page, context = {}) => { fetch('/api/intelligence/track', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ event_type: 'button_click', event_name: `${element}_click`, page, element, context, session_id: sessionId.current }) }).catch(() => {}); }, []); // Track flow completion const trackFlowComplete = useCallback((flowName, duration_ms, context = {}) => { fetch('/api/intelligence/track', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ event_type: 'flow_complete', event_name: `${flowName}_complete`, duration_ms, context, session_id: sessionId.current }) }).catch(() => {}); }, []); // Track flow abandonment const trackFlowAbandon = useCallback((flowName, step, context = {}) => { fetch('/api/intelligence/track', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ event_type: 'flow_abandon', event_name: `${flowName}_abandon`, context: { ...context, abandoned_at_step: step }, session_id: sessionId.current }) }).catch(() => {}); }, []); // Detect rage clicks (3+ clicks in 1 second on same element) const clickTimes = useRef({}); const trackPotentialRageClick = useCallback((element, page) => { const now = Date.now(); const key = `${page}_${element}`; if (!clickTimes.current[key]) { clickTimes.current[key] = []; } clickTimes.current[key].push(now); clickTimes.current[key] = clickTimes.current[key].filter(t => now - t < 1000); if (clickTimes.current[key].length >= 3) { fetch('/api/intelligence/track', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ event_type: 'rage_click', event_name: `${element}_rage_click`, page, element, context: { click_count: clickTimes.current[key].length }, session_id: sessionId.current }) }).catch(() => {}); clickTimes.current[key] = []; } }, []); return { trackPageView, trackClick, trackFlowComplete, trackFlowAbandon, trackPotentialRageClick, sessionId: sessionId.current }; } ``` ### 6.2 Intelligence Dashboard Page (Admin) ``` /intelligence - Admin-only page showing: - Active alerts with severity badges - Pending suggestions with implement/reject buttons - Friction points heatmap - This week's quick wins - System health score - Pattern library - Evolution log (what the system changed about itself) ``` --- ## Part 7: Implementation Order ### Phase 1: Foundation (Week 1) - [ ] Create database schema - [ ] Create usage_events tracking endpoint - [ ] Add tracking hook to frontend - [ ] Instrument key pages (CallList, CallPrep, Inbox, LeadProfile) - [ ] Create daily metrics aggregation job ### Phase 2: Real-Time Alerts (Week 1-2) - [ ] Create alert detection service - [ ] Configure alert thresholds - [ ] Create Slack alert formatting - [ ] Add alert management endpoints - [ ] Test with simulated issues ### Phase 3: Weekly Analysis (Week 2-3) - [ ] Create friction detection analysis - [ ] Create unused feature analysis - [ ] Create pattern opportunity analysis - [ ] Create AI quality analysis - [ ] Create quick win identification - [ ] Create weekly report generator - [ ] Create Slack report formatting - [ ] Schedule Monday 7:15am job ### Phase 4: Monthly Deep Dive (Week 3-4) - [ ] Create strategic analysis - [ ] Create competitor intelligence (Perplexity integration) - [ ] Create suggestion effectiveness review - [ ] Create self-improvement analysis - [ ] Create monthly report generator - [ ] Schedule 1st of month job ### Phase 5: Self-Improvement Loop (Week 4) - [ ] Create prompt versioning system - [ ] Create threshold auto-adjustment logic - [ ] Create metric evaluation system - [ ] Create evolution logging - [ ] Test self-improvement with mock data ### Phase 6: Admin Dashboard (Week 4-5) - [ ] Create /intelligence admin page - [ ] Add alert management UI - [ ] Add suggestion management UI - [ ] Add friction visualization - [ ] Add evolution timeline --- ## Part 8: Success Metrics ### For the Intelligence System Itself: | Metric | Target | Measurement | |--------|--------|-------------| | Alert accuracy | >90% true positive | Manual review of alerts | | Suggestion implementation rate | >50% | Suggestions implemented / total | | Suggestion success rate | >70% | Successful / implemented | | Impact prediction accuracy | >60% | Actual vs predicted impact | | Time to detect friction | <7 days | New friction → detection time | | Self-improvement frequency | 3-5 changes/month | Evolution log entries | ### For the App (Improved by Intelligence): | Metric | Current | 3-Month Target | |--------|---------|----------------| | CallPrep abandonment | TBD | -30% | | Feature adoption (avg) | TBD | +20% | | User-reported friction | TBD | -50% | | AI suggestion acceptance | ~75% | >85% | | Conversion rate | TBD | +15% | --- ## Part 9: The Meta-Loop (0.1% of 0.1%) Every month, the system asks itself: ``` META-IMPROVEMENT QUESTIONS: 1. DETECTION QUALITY - Am I finding the right friction points? - What friction did users report that I missed? - What did I flag that wasn't actually friction? → Adjust: friction detection thresholds, signals 2. SUGGESTION QUALITY - Which suggestions got implemented? - Which implemented suggestions worked? - What made the difference? → Adjust: suggestion ranking, effort estimation, impact prediction 3. ANALYSIS DEPTH - What questions am I not asking? - What data am I not looking at? - What patterns am I missing? → Add: new analysis types, new metrics, new data sources 4. COMMUNICATION QUALITY - Are reports being read? (Slack reaction tracking) - Are suggestions being acted on? - Is the format clear? → Adjust: report format, priority presentation, actionability 5. SPEED VS ACCURACY - Am I alerting too early (noise) or too late (damage done)? - Am I suggesting too many things (overwhelm) or too few (missing opportunities)? → Adjust: alert thresholds, suggestion limits 6. LEARNING LOOPS - What worked last month that I should do more of? - What didn't work that I should stop? - What should I try that I haven't? → Evolve: entire approach based on outcomes ``` The system literally rewrites its own prompts, adjusts its own thresholds, adds new metrics, and deprecates useless analyses - all based on measured outcomes. --- ## Part 10: Architecture Decisions Key design choices that make this system scalable and maintainable: ### 1. Usage Events: Partitioning + Retention **Problem:** Raw events table will grow to millions of rows fast. **Solution:** - **Partition by month** - Each month is its own table partition - **90-day retention** - Raw events deleted after 90 days - **Aggregates forever** - `daily_metrics` keeps summaries indefinitely **Why:** Query last week's data without scanning years of history. Drop old partitions instantly vs slow DELETE. ### 2. Analysis Prompts: Dual Storage (DB + Git) **Problem:** Need runtime performance tracking AND historical evolution context. **Solution:** - **Database:** Active prompts, performance scores, usage stats - **Git:** `src/intelligence/prompts/*.md` - full version history with commit messages - **Link:** `git_commit_sha` column connects DB record to git history **Why:** See "friction_detection_v3 has 82% success rate" AND "v3 was created because v2 missed mobile-specific friction (see commit abc123)". ### 3. Slack Threading: Persistent Conversations **Problem:** Weekly report, mid-week updates, and impact measurements are disconnected. **Solution:** - **Save `slack_message_ts`** when weekly report posts - **Thread all related updates** under that message: - "✅ Quick Win #1 implemented!" - "📊 Quick Win #1 result: 23% improvement" - "⚠️ New friction detected related to this week's patterns" **Why:** One place to see: suggestion → implementation → impact. Not scattered across Slack history. ### 4. Retention Policy Implementation ```javascript // src/jobs/usageEventCleanup.js // Runs daily at 2am AEST async function cleanupOldEvents() { // Delete raw events older than 90 days const { count } = await supabase .from('usage_events') .delete() .lt('created_at', new Date(Date.now() - 90 * 24 * 60 * 60 * 1000).toISOString()); console.log(`Cleaned up ${count} old usage events`); // Create next month's partition if within 7 days of month end await ensureNextPartitionExists(); } ``` ### 5. Slack Threading Implementation ```javascript // When sending weekly report const result = await slack.chat.postMessage({ channel: INTELLIGENCE_CHANNEL, text: weeklyReport, unfurl_links: false }); // Save for threading await supabase.from('weekly_excellence_reports').update({ slack_channel_id: result.channel, slack_message_ts: result.ts }).eq('id', reportId); // Later, when suggestion implemented: await slack.chat.postMessage({ channel: savedReport.slack_channel_id, thread_ts: savedReport.slack_message_ts, // This makes it a reply text: "✅ Quick Win #1 implemented! Measuring impact..." }); ``` --- ## Summary This is a **self-improving intelligence system** with three tiers: | Tier | Frequency | Purpose | Output | |------|-----------|---------|--------| | **Real-Time** | Continuous | Catch breaks | Slack alerts | | **Weekly** | Monday 7:15am | Find friction & quick wins | Excellence report | | **Monthly** | 1st of month | Strategic + self-improve | Deep dive + evolution | The monthly self-improvement loop is what makes this **0.1% of 0.1%** - the system doesn't just analyze the app, it analyzes its own analysis and gets better at getting better. --- *Plan created: January 2026* *Status: MVP IN PROGRESS* --- ## MVP Implementation Status ### Completed: - [x] Database schema (`supabase/intelligence-schema.sql`) - [x] API endpoints (`src/routes/intelligence.js`) - [x] Frontend tracking hook (`admin/src/hooks/useIntelligenceTracking.js`) - [x] CallList page instrumented - [x] Weekly analysis service (`src/services/weeklyIntelligence.js`) - [x] Slack report job (`src/jobs/weeklyExcellenceReport.js`) - [x] Monday 7:15am AEST schedule ### Pending: - [ ] Run database migration in Supabase - [ ] Deploy to Railway - [ ] Instrument more pages (CallPrep, Inbox, LeadProfile) - [ ] Accumulate 1 week of data - [ ] First real weekly report