# Scout Agent - AI News Monitoring System

Scout is RateRight's dedicated AI news intelligence agent, designed to monitor the rapidly evolving AI landscape and deliver actionable insights relevant to construction technology and marketplace innovation.

## Overview

Scout continuously monitors AI news sources and filters for information relevant to:
- Construction AI applications
- Marketplace technology developments
- Pricing algorithm innovations
- Competitive intelligence
- Regulatory changes affecting AI in construction

## Features

- **Automated Monitoring**: Checks sources every 2 hours
- **Intelligent Filtering**: Keyword-based relevance scoring
- **Multi-Source Coverage**: Hacker News, ArXiv, Twitter, TechCrunch
- **Smart Alerts**: Immediate notifications for urgent developments
- **Daily Briefings**: 5-bullet summaries delivered to Telegram
- **Weekly Analysis**: Trend identification and strategic recommendations

## Installation

### Prerequisites
- Clawdbot instance running
- Telegram bot token with access to Rocky & Rivet group
- API keys for news sources (optional, improves coverage)

### Setup Steps

1. **Clone Scout Configuration**
   ```bash
   cd /home/ccuser/rateright-growth/rivet/agents/
   # Scout directory and files already created
   ```

2. **Configure Telegram Bot**
   - Add bot to Rocky & Rivet group (ID: -1003505625266)
   - Ensure bot has message posting permissions
   - Update `config/telegram.json` with bot token (via Clawdbot secrets)

3. **Set Up API Keys (Optional)**
   ```bash
   # Store in Clawdbot secrets management
   # Twitter API (for enhanced monitoring)
   # ArXiv API (for paper search)
   # Hacker News API (built-in, no key needed)
   ```

4. **Configure Keywords**
   Edit `config/keywords.json` to adjust:
   - Priority keywords for alerts
   - Competitor list to monitor
   - Filter exclusions
   - Relevance scoring weights

5. **Test Deployment**
   ```bash
   # Run manual check
   clawdbot agent scout check --sources=all --dry-run
   
   # Test Telegram integration
   clawdbot agent scout test-message --type=daily
   ```

## Configuration

### News Sources (`config/sources.json`)
Configure monitoring frequency and thresholds for each source:
- Hacker News: AI/ML posts with >50 points
- ArXiv: Daily scan of AI categories
- Twitter: 30-minute checks of key accounts
- TechCrunch: AI section monitoring

### Keywords (`config/keywords.json`)
Three-tier priority system:
- **High**: Immediate alerts + daily briefing
- **Medium**: Daily briefing if space available
- **Competitors**: Track all mentions

### Telegram Settings (`config/telegram.json`)
- Group configuration
- Message formatting
- Rate limiting
- Topic routing (Research topic ID: 5)

## Operation

### Heartbeat Schedule
- Every 2 hours: Source monitoring
- 5:00 PM AEDT: Daily briefing
- Sunday 6:00 PM AEDT: Weekly summary
- Immediate: Urgent alerts (score >0.9)

### Output Formats

**Daily Briefing Example:**
```
🤖 Scout Daily Brief - 2024-01-15

• OpenAI releases GPT-4 Vision API with 90% cost reduction. Why it matters: Could enable affordable computer vision for construction site monitoring.

• New paper demonstrates AI-powered structural defect detection with 95% accuracy. Why it matters: Potential integration with RateRight's quality assessment.

• Procore acquires AI safety startup for $50M. Why it matters: Validates construction AI market, may signal pricing pressure.

• Anthropic publishes marketplace pricing optimization research. Why it matters: Directly applicable to RateRight's pricing algorithms.

• Building robotics startup raises Series B with hiring platform features. Why it matters: Emerging competition in construction marketplace space.
```

**Urgent Alert Example:**
```
🚨 Scout Alert - Major Model Release

OpenAI just announced GPT-5 with native construction document understanding

Impact: Could revolutionize RFP processing and project estimation
RateRight relevance: Opportunity to automate contractor matching

Source: https://openai.com/blog/gpt-5

@channel
```

## Monitoring and Maintenance

### Logs
- Check `logs/scout.log` for errors
- Monitor API rate limits
- Track alert accuracy

### Performance Metrics
- Reports per day (target: 1+)
- Source diversity (minimum 5 sources)
- Response time for urgent news (<15 min)
- Relevance feedback from team

### Updates
- Weekly: Review keyword effectiveness
- Monthly: Add new sources, remove dead ones
- Quarterly: Update competitor list
- As needed: Adjust alert thresholds

## Troubleshooting

**No messages appearing:**
- Check Telegram bot permissions
- Verify group chat ID
- Review source API status

**Too many alerts:**
- Increase keyword specificity
- Raise relevance thresholds
- Add exclusion filters

**Missing important news:**
- Add new sources
- Lower thresholds for high-value sources
- Expand competitor monitoring

## Security Considerations

- API keys stored in Clawdbot secrets (never in files)
- Source credibility scoring to prevent misinformation
- No sensitive RateRight data in reports
- Public sources only (no private information)

## Future Enhancements

- Sentiment analysis on construction AI news
- Competitor product launch tracking
- Integration with RateRight's internal data
- Custom dashboard for trend visualization
- Slack/Discord integration options

## Support

For issues or feature requests:
- Check logs in `logs/scout.log`
- Review configuration files
- Test with dry-run mode
- Escalate to development team if needed

---

*Scout Agent v1.0 - AI News Intelligence for RateRight*