Caching Strategies

Caching Patterns

Store frequently accessed data in fast-access storage to reduce latency and backend load

Core Idea

Caching stores copies of frequently accessed data in fast-access storage layers (memory, SSD) to reduce latency, decrease database load, and improve application performance. Different caching strategies offer different trade-offs between consistency, performance, and complexity.

When to Use

Use caching when you have read-heavy workloads, expensive computations, frequently accessed data, or need to reduce database load and improve response times.

Recognition Cues
Indicators that this pattern might be the right solution
  • Database queries are slow or repetitive
  • Read-to-write ratio is high (90%+ reads)
  • Expensive computations are performed repeatedly
  • API rate limits require reducing external calls
  • Need to handle traffic spikes without scaling database

Pattern Variants & Approaches

Cache-Aside (Lazy Loading)
Application checks cache first; on miss, loads from database and populates cache. Most common pattern.

Cache-Aside (Lazy Loading) Architecture

1. Check2. On Miss3. Return Data4. Populate⚙️ApplicationCache💾Database

When to Use This Variant

  • Read-heavy workload with unpredictable access patterns
  • Want to cache only frequently accessed data
  • Can tolerate occasional cache misses
  • Data access patterns are not known in advance

Use Case

General-purpose caching for web applications, user profiles, product catalogs, and session data

Advantages

  • Only caches data that's actually requested
  • Simple to implement and understand
  • Cache failures don't break the application
  • Works well with unpredictable access patterns

Implementation Example

# Cache-Aside Pattern Implementation
def get_user(user_id):
    # Try to get from cache first
    cache_key = f"user:{user_id}"
    user = cache.get(cache_key)
    
    if user is not None:
        return user  # Cache hit
    
    # Cache miss - fetch from database
    user = database.query("SELECT * FROM users WHERE id = ?", user_id)
    
    if user:
        # Populate cache for future requests
        cache.set(cache_key, user, ttl=3600)  # 1 hour TTL
    
    return user

def update_user(user_id, data):
    # Update database
    database.update("UPDATE users SET ... WHERE id = ?", user_id, data)
    
    # Invalidate cache to ensure consistency
    cache_key = f"user:{user_id}"
    cache.delete(cache_key)
Write-Through Cache
Data is written to cache and database simultaneously. Ensures cache is always consistent with database.

Write-Through Cache Architecture

1. Write2. UpdateRead⚙️ApplicationCache💾Database

When to Use This Variant

  • Need strong consistency between cache and database
  • Write performance is acceptable with dual writes
  • Read-heavy workload with predictable data
  • Cannot tolerate stale data

Use Case

Financial systems, inventory management, booking systems where consistency is critical

Advantages

  • Cache is always consistent with database
  • No cache misses for recently written data
  • Simpler consistency model
  • Good for read-heavy workloads after writes

Implementation Example

# Write-Through Cache Pattern
def save_user(user_id, data):
    cache_key = f"user:{user_id}"
    
    # Write to database first
    database.update("UPDATE users SET ... WHERE id = ?", user_id, data)
    
    # Then update cache (write-through)
    cache.set(cache_key, data, ttl=3600)
    
    return data

def get_user(user_id):
    cache_key = f"user:{user_id}"
    user = cache.get(cache_key)
    
    if user is None:
        # Cache miss - load from database
        user = database.query("SELECT * FROM users WHERE id = ?", user_id)
        if user:
            cache.set(cache_key, user, ttl=3600)
    
    return user
Write-Behind (Write-Back) Cache
Data is written to cache immediately and asynchronously written to database later. Optimizes write performance.

Write-Behind (Write-Back) Cache Architecture

1. Write (Fast)2. Queue3. Async Flush⚙️ApplicationCache📬Write Queue💾Database

When to Use This Variant

  • Write-heavy workload with performance requirements
  • Can tolerate eventual consistency
  • Need to batch database writes for efficiency
  • Write latency is critical

Use Case

Analytics systems, logging, metrics collection, social media likes/views counters

Advantages

  • Extremely fast write operations
  • Can batch writes to database for efficiency
  • Reduces database write load
  • Good for write-heavy workloads

Implementation Example

# Write-Behind Cache Pattern
import asyncio
from collections import defaultdict

class WriteBehindCache:
    def __init__(self):
        self.cache = {}
        self.write_queue = defaultdict(dict)
        self.flush_interval = 5  # seconds
        
    async def set(self, key, value):
        # Write to cache immediately
        self.cache[key] = value
        
        # Queue for async database write
        self.write_queue[key] = value
        
        return value
    
    async def flush_worker(self):
        """Background worker to flush writes to database"""
        while True:
            await asyncio.sleep(self.flush_interval)
            
            if self.write_queue:
                # Batch write to database
                batch = dict(self.write_queue)
                self.write_queue.clear()
                
                try:
                    await database.batch_update(batch)
                except Exception as e:
                    # On failure, re-queue writes
                    self.write_queue.update(batch)
                    logger.error(f"Failed to flush cache: {e}")

# Usage
cache = WriteBehindCache()
asyncio.create_task(cache.flush_worker())

async def increment_view_count(post_id):
    key = f"post:{post_id}:views"
    current = cache.cache.get(key, 0)
    await cache.set(key, current + 1)
Tradeoffs

Pros

  • Dramatically reduced latency (microseconds vs. milliseconds)
  • Reduced database load and costs
  • Better handling of traffic spikes
  • Improved user experience
  • Can serve stale data during outages

Cons

  • Increased complexity in maintaining consistency
  • Additional infrastructure and memory costs
  • Potential for serving stale data
  • Cache invalidation is challenging
  • Cold start problems after cache flush
Common Pitfalls
  • Cache stampede - multiple requests fetch same data simultaneously
  • Stale data served due to improper invalidation
  • Cache penetration - queries for non-existent data bypass cache
  • Not setting appropriate TTL values
  • Caching data that changes frequently
  • Not monitoring cache hit rates
Design Considerations
  • Choose appropriate cache eviction policy (LRU, LFU, FIFO)
  • Determine optimal TTL (Time To Live) values
  • Decide on cache invalidation strategy
  • Plan for cache warming and preloading
  • Consider cache size and memory constraints
  • Implement cache key design and namespacing
  • Handle cache failures gracefully
Real-World Examples
Facebook

Uses Memcached for caching user data, reducing database queries by 90%

Trillions of cache requests per day
Twitter

Caches timelines and tweets using Redis, serving millions of reads per second

500+ million tweets per day
Pinterest

Multi-layer caching with Redis and Memcached for pins and boards

Billions of pins, 400+ million users
Complexity Analysis
Scalability

Horizontal - Add more cache nodes with sharding

Implementation Complexity

Medium - Cache invalidation and consistency challenges

Cost

Low to Medium - Memory costs, but reduces database costs