Real-Time vs Batch Recommendations

**How it works:**

• Train models on historical data
• Generate top-N recommendations per user
• Store in fast cache (Redis, DynamoDB)
• Serve directly from cache at request time

**Advantages:**

• Simple serving infrastructure
• Can use complex, slow models
• Predictable costs
• Easy to debug and validate

**Best for:**

• Email campaigns and digests
• Users with stable preferences
• Long-term personalization
• Situations where freshness matters less

**Limitations:**

• Can't respond to in-session behavior
• Recommendations may be stale
• Storage costs for all user-item pairs

**When you need real-time:**

• E-commerce browse sessions (interests shift rapidly)
• Short or anonymous sessions
• Content platforms where mood changes
• When recent behavior is most predictive

**Requirements:**

• Streaming infrastructure (Kafka, Kinesis)
• Feature store for low-latency features
• Fast model inference (<50ms p99)
• Session management and state

**Real-time features:**

• Current session clicks and views
• Time since last interaction
• Cart contents
• Search queries in session

**Challenges:**

• More complex infrastructure
• Higher serving costs
• Harder to debug
• Need robust fallbacks

**Example flow:**

• Nightly job computes 1000 candidate items per user
• User starts session, fetch their candidates
• As they browse, update candidate scores with session features
• Apply business rules (in-stock, already purchased, diversity)
• Return top 20

**Benefits:**

• Personalization depth from batch models
• Responsiveness from real-time signals
• Manageable infrastructure complexity
• Graceful degradation (batch-only if real-time fails)

**Measure the value of recency:**

• A/B test batch-only vs. session-aware
• Is the lift worth the infrastructure cost?
• Which user segments benefit most?

**Start simple:**

• Begin with batch recommendations
• Add real-time re-ranking as you scale
• Invest in real-time only where it provides measurable lift

**Feature store is key:**

• Central challenge is low-latency feature access
• Pre-compute what you can, stream what you must
• Consider managed solutions (Feast, Tecton, Databricks)

**Latency budgets:**

• Total recommendation latency: <100ms
• Feature retrieval: <20ms
• Model inference: <30ms
• Business rules: <10ms
• Network overhead: <40ms