There is a moment in every backend engineer’s life when their database starts refusing connections.

Picture 2 a.m. on a Tuesday. The app is operating under normal traffic, nothing unusual. But the database connection pool is saturated. Queries are timing out. The monitoring dashboard shows 50,000 database operations per second, far beyond what the system should be handling.

When someone pulls the slow-query logs, the pattern is immediately clear. The same query appears thousands of times:

1SELECT * FROM user_profiles WHERE user_id = ?

The individual query is fast and simple. The problem is scale. Thousands of concurrent requests are running this exact same query every few seconds. Each request makes the call, gets back the same data, discards it, and repeats the process.

The fix is straightforward: cache the result with a five-minute TTL. One line of code. The database load drops from saturated to 500 QPS and the incident is over.

This illustrates a larger point. This post is not about the sophisticated architecture of building distributed cache systems. It is about something more fundamental: cache is not a performance optimization you add later. It is a load shield built into the system from the start. Your database without proper caching is not a resilient system. It is a ticking timer.

The Scale Problem

Let’s do the math.

Say you have 10,000 users online. Each user’s request does one database call that it doesn’t have to do: maybe a lookup that gets cached, or a precomputation that could be stored. It’s a small inefficiency. Your database can handle it.

Now imagine that query is slow. It takes 100 milliseconds. So 10,000 users, each doing that query once, generates 10,000 queries per second. At 100ms each, you’ve used up 1,000 seconds of database CPU per second. You don’t have 1,000 CPUs.

Or worse: imagine that query is cached, but cached poorly. The TTL is short. Every 10 seconds, all 10,000 users’ caches expire at once, and they all hit the database simultaneously. This thundering herd causes a spike of 10,000 queries in 1 second. The connection pool fills. Queries queue. Latency spikes. Users retry. The system fails.

This is not hypothetical. It happens in production at 3 a.m. And it happens because someone thought “we can fetch this on demand, it’s fine.”

The rule is simple: every database call at request time that could have been cached multiplies by your concurrent user count. If your users are making 10 requests per second, and each request does 1 uncached lookup, you are generating 10 uncached lookups per second per user. At 10,000 users, that is 100,000 uncached lookups per second.

Add a cache with a 5-minute TTL, and it drops to the cost of refreshing 10,000 users’ worth of cached data across 5 minutes. That is 33 lookups per second. A reduction of three orders of magnitude.

Cache is not optional. It is infrastructure.

Cache Strategies

Before diving into what to cache, you need to understand how to cache. There are two main strategies, and they work in opposite directions.

Write-Through Cache

In a write-through cache, whenever data changes, you update the cache at the same time as the database. The application code looks like:

1// Update the database
2err := db.Update("user_profiles", userID, newData)
3if err != nil {
4  return err
5}
6
7// Simultaneously update the cache
8cache.Set("user:"+userID, newData, 24*time.Hour)
9return nil

The cache is always correct. If data changes, the cache changes immediately. Reads are fast because they hit the cache.

Trade-off: Write operations are slower. You are doing two writes instead of one. If the cache write fails but the database write succeeds, they become inconsistent: the database has the new data but the cache still has the old data. You have to handle that scenario.

When it wins: When writes are rare and reads are frequent. A user profile is written once, read hundreds of times. Update it once, serve it from cache for days. The cost of the extra write is paid back instantly.

Read-Through Cache

In a read-through cache, you don’t pre-populate. When a request needs data, it checks the cache first. On a miss, it fetches from the database and then stores the result in the cache for the next request.

 1// Try cache first
 2if cached := cache.Get("user:" + userID); cached != nil {
 3  return cached, nil
 4}
 5
 6// Cache miss, fetch from DB
 7data, err := db.Get("user_profiles", userID)
 8if err != nil {
 9  return nil, err
10}
11
12// Store for next time
13cache.Set("user:"+userID, data, 5*time.Minute)
14return data, nil

The cache fills up as requests come in. You don’t pre-populate anything.

Trade-off: The first read is slow. Subsequent reads are fast. If the cache TTL expires and nobody has read the data in that window, you are right back to a database query on the next read.

When it wins: When you don’t know what data will be accessed, or when the dataset is too large to precompute. A user searches for something. The first search query is slow. But if 10 users search for the same thing in the next 5 minutes, the next 9 are fast.

Hybrid: Write-Through Cache, Read-Through Fallback

In practice, most systems use both. Write-through for things you control and write frequently. Read-through for everything else. Precompute the data you access constantly (the hot path), and for less frequently accessed data, compute it on the first request and cache the result for subsequent requests.

Types of Caches

Now that you know how to cache, you need to know what to cache. There are many kinds. They solve different problems.

1. Precomputed Tables

The simplest cache is a table. You compute something expensive once, store the result, and point reads to that table instead of recomputing.

Example: a user’s follower count. Computing it from scratch means counting rows in a followers table. At scale, that is slow. Instead, maintain a user_stats table that stores a copy of the follower count alongside other user statistics. Update this copy whenever someone follows or unfollows. Now reads are instant (no counting needed).

1SELECT follower_count FROM user_stats WHERE user_id = ?

This is write-through cache in database form. You update it when data changes, and reads are instant.

Cost: Storage and maintenance overhead. You are duplicating data. If the count gets out of sync with the source, you have to repair it.

When to use: When you read frequently and writes are predictable. Aggregations, counts, rankings. Things that are expensive to compute and updated only when specific events happen (someone follows, a post gets a like).

2. Function-Result Cache

Cache the output of expensive functions. If a function is deterministic (same input always gives same output), cache the result.

Example: converting markdown to HTML. You have a blog post in markdown. You could render it every time someone views the page. Or cache the HTML result. When the post is edited, invalidate the cache.

 1// Hash the function input and use it as the cache key
 2inputHash := md5.Sum([]byte(markdownContent))
 3cacheKey := "md_render:" + hex.EncodeToString(inputHash[:])
 4
 5if cached := cache.Get(cacheKey); cached != nil {
 6  return cached.(string)
 7}
 8
 9html := renderMarkdownToHTML(markdownContent)
10cache.Set(cacheKey, html, 0) // No expiration, invalidate on write
11return html

This is common with image processing, PDF generation, or any computation that is deterministic but expensive.

Cost: Cache invalidation. If you change how the function works, old cached results are wrong. You need a versioning strategy.

When to use: Any function you call frequently with the same inputs. Expensive computations that don’t change unless the input changes.

3. Route Cache

Cache entire HTTP responses. If a route is expensive (multiple database queries, rendering), cache the HTML result.

 1// Check if we have a cached response
 2if cached := cache.Get("route:/blog/" + postID); cached != nil {
 3  w.Header().Set("Content-Type", "text/html")
 4  w.Write(cached.([]byte))
 5  return
 6}
 7
 8// Generate the response
 9data, err := db.GetPost(postID)
10// ... render HTML ...
11responseBody := renderTemplate(data)
12
13cache.Set("route:/blog/"+postID, responseBody, 24*time.Hour)
14w.Write(responseBody)

Popular on static or semi-static pages. Blog posts, documentation, product pages. Cache the entire rendered output.

Cost: Cache invalidation. When a post is edited, you have to invalidate the cached response.

When to use: Routes that are read much more than they are written. Static content, public pages, anything that does not change per-user.

4. CDN Cache

A CDN (Content Delivery Network) caches static files and HTTP responses at the edge, close to users. Requests never hit your origin server.

Your infrastructure looks like:

1User in London
2    ↓
3CDN edge server in London (has cached copy of your JS/CSS/images)
4    ↓
5User in Singapore
6    ↓
7CDN edge server in Singapore (also has cached copy)
8    ↓
9Origin server (only used on cache misses or refreshes)

This is handled by HTTP headers. You tell the CDN “this asset can be cached for 30 days” using the Cache-Control header.

1Cache-Control: public, max-age=2592000

Cost: You are not in control of cache invalidation. If you set max-age=30d on a JavaScript file and it has a bug, users will serve the old version for days.

When to use: Versioned assets (JS/CSS with content hashes in the filename), images, and anything that doesn’t need to change instantly.

5. Database Document Cache (Result Cache)

Some queries are expensive but produce large, well-defined result sets. Instead of running the query every time, compute it once, store it as a document, and serve from there.

Example: a user’s feed. Computing a feed involves joining posts from 100 followed accounts, ordering by relevance, and filtering. That is three or four queries. Cache the entire result as a document.

 1// Try to get the cached feed document
 2if cached := cache.Get("feed:user:" + userID); cached != nil {
 3  return json.Unmarshal(cached, &feed)
 4}
 5
 6// Expensive query: get all relevant posts, join, order, filter
 7feed, err := db.GetUserFeed(userID)
 8if err != nil {
 9  return nil, err
10}
11
12// Cache the entire result set
13feedJSON, _ := json.Marshal(feed)
14cache.Set("feed:user:"+userID, feedJSON, 5*time.Minute)
15return feed, nil

This is read-through caching. The cache fills as users request their feeds.

Cost: Memory usage can be high. You are storing entire result sets. If the feed is 500 posts per user and you have a million users, that is gigabytes of memory.

When to use: Results that are expensive to compute but referenced frequently. User-specific data like feeds, recommendations, or personalized lists.

6. Temp Expiring Cache (TTL-Based Cache)

The simplest cache. Store data with a short expiration time. When it expires, the next request recomputes it.

 1// Set with a 5-minute TTL
 2cache.Set("expensive_stat", computeExpensiveValue(), 5*time.Minute)
 3
 4// Later, when you need it:
 5value := cache.Get("expensive_stat")
 6if value == nil {
 7  value = computeExpensiveValue()
 8  cache.Set("expensive_stat", value, 5*time.Minute)
 9}
10return value

No invalidation logic. No coordination. Data expires automatically.

Cost: Stale data. Between the time the cache expires and the next request, the value is stale. If data changes and the cache has 4 minutes left, a user might see the old value for up to 4 minutes.

When to use: When staleness is acceptable. Metrics, statistics, things that don’t need to be fresh to the second. The 5-minute user count on your dashboard doesn’t need to be exact.

7. Context-Aware Cache (In-Request Cache)

In Go, the context package is often used to pass values through a request chain. You can use it as a cache to avoid repeated database queries within a single request.

 1func (s *Service) GetUser(ctx context.Context, userID string) (*User, error) {
 2  // Check if we already fetched this user in this request
 3  if cached := ctx.Value("user:" + userID); cached != nil {
 4    return cached.(*User), nil
 5  }
 6
 7  // First fetch, hit the database
 8  user, err := s.db.GetUser(userID)
 9  if err != nil {
10    return nil, err
11  }
12
13  // Store in context for this request chain
14  ctx = context.WithValue(ctx, "user:"+userID, user)
15  return user, nil
16}

This prevents the same request from hitting the database twice for the same user, even if the request logic is complex and makes multiple calls to GetUser().

Cost: Memory per request. If a request creates a million users in context, that is a million users in memory for that request. Also, the cache is thrown away after the request ends.

When to use: Within a single request, when you might fetch the same thing multiple times. API requests that do bulk operations. Queries inside loops.

Understanding the Scale Multiplier

The math is simple, and brutal.

Say you have 1000 concurrent users. Each makes one request per second. And each request does one query it shouldn’t do: maybe a count, maybe a lookup that should have been cached.

11000 users × 1 request/sec × 1 extra query/request = 1000 extra queries/sec

If that query takes 50ms, your database needs 50 seconds of CPU per second just to handle those. You don’t have 50 CPU cores.

But here’s the real problem: most systems don’t start with 1000 users. They scale up. When you hit 10,000 users, the same code suddenly does 10,000 extra queries per second. Your database goes from stressed to dead.

This is not hypothetical. It happens because one engineer, one time, wrote a query that was “fast enough” without thinking about the multiplier.

With caching, the math flips:

1Uncached version: 1000 users × 1 uncached query/request = 1000 queries/sec
2Cached version (5 min TTL): 1 cache refresh per user per 5 minutes = 3.3 queries/sec

That is three orders of magnitude. The difference between your database staying alive and your pager going off at 3 a.m.

Here’s what it looks like in practice:

1Without cache:
210,000 users → 10,000 queries/sec → database CPU maxed → timeouts → cascading failures
3
4With cache (5 min TTL):
510,000 users → 33 queries/sec → database barely notices → users happy

The rule: Before deploying, ask yourself: does every request hit the database for something it could have cached? If yes, you will regret it at scale.

Cache Technologies

Different tools solve caching at different layers. Pick the right one.

In-Memory Cache (Application)

Store data in memory within your application process. When memory fills up, the cache removes the least recently used items to make room for new ones.

 1import "github.com/patrickmn/go-cache"
 2
 3// Create a cache that expires items after 5 minutes
 4c := cache.New(5*time.Minute, 10*time.Minute)
 5
 6// Store data
 7c.Set("user:123", userData, cache.DefaultExpiration)
 8
 9// Retrieve data
10if data, found := c.Get("user:123"); found {
11  user := data.(User)
12  // use user
13}
14
15// Manual invalidation
16c.Delete("user:123")

Pros: Extremely fast. No network latency. Simple to debug.

Cons: Not shared across servers. If you have 10 application instances, each has its own cache, 10x the memory overhead. Cache invalidation has to be coordinated across all instances.

Use for: Request-level caching (context cache), warm-up data that doesn’t change often, local state.

Redis

In-memory key-value store accessible over the network. Supports expiration, lists, sets, sorted sets, and more.

 1import "github.com/redis/go-redis/v9"
 2
 3client := redis.NewClient(&redis.Options{
 4  Addr: "localhost:6379",
 5})
 6
 7// Set with 5-minute expiration
 8err := client.Set(ctx, "user:123", jsonMarshal(userData), 5*time.Minute).Err()
 9
10// Get
11val, err := client.Get(ctx, "user:123").Result()
12if err == redis.Nil {
13  // Cache miss, fetch from DB
14} else if err != nil {
15  // Redis error
16} else {
17  // Cache hit, unmarshal val
18  var user User
19  json.Unmarshal([]byte(val), &user)
20}
21
22// Increment (useful for counters)
23client.Incr(ctx, "post:view-count:456")
24
25// Sorted set (useful for leaderboards)
26client.ZAdd(ctx, "leaderboard", &redis.Z{Score: 100, Member: "user:123"})
27
28// Manual invalidation
29client.Del(ctx, "user:123")

Pros: Shared cache across all servers. Rich data structures (lists, sets, sorted sets for leaderboards). Built-in TTL. Persistent snapshots to disk.

Cons: Extra network latency (~1-5ms). Single point of failure. If the Redis instance goes down, all cached data is lost and every request hits the database simultaneously (you can use Redis Cluster for redundancy, but that adds complexity). Memory cost grows with data size.

Use for: User sessions, rate limiting, feature flags, leaderboards, any cached data you want visible across servers.

Memcached

Similar to Redis but simpler. Key-value store, expiration, done. No lists or sorted sets.

 1import "github.com/bradfitz/gomemcache/memcache"
 2
 3mc := memcache.New("localhost:11211")
 4
 5// Set with 5-minute expiration (300 seconds)
 6err := mc.Set(&memcache.Item{
 7  Key: "user:123",
 8  Value: jsonMarshal(userData),
 9  Expiration: 300,
10})
11
12// Get
13item, err := mc.Get("user:123")
14if err == memcache.ErrCacheMiss {
15  // Cache miss, fetch from DB
16} else if err != nil {
17  // Memcached error
18} else {
19  // Cache hit
20  var user User
21  json.Unmarshal(item.Value, &user)
22}
23
24// Delete
25mc.Delete("user:123")

Pros: Very fast, very simple. Lower overhead than Redis. Excellent for pure key-value access patterns.

Cons: No persistence, no complex data types, no TTL per key granularity (uses seconds globally).

Use for: High-throughput caching where you just need basic key-value, and you don’t need persistence or rich data structures.

Nginx / Varnish (HTTP Cache)

Sit in front of your application and cache HTTP responses.

Nginx example:

 1http {
 2  proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=my_cache:10m;
 3
 4  server {
 5    listen 80;
 6
 7    location /api/posts {
 8      proxy_pass http://backend;
 9      proxy_cache my_cache;
10      proxy_cache_valid 200 5m;
11      proxy_cache_bypass $http_pragma $http_authorization;
12      add_header X-Cache-Status $upstream_cache_status;
13    }
14
15    location /api/user/ {
16      proxy_pass http://backend;
17      # Don't cache authenticated requests
18      proxy_cache_bypass $http_authorization;
19    }
20  }
21}

Varnish example:

 1backend default {
 2  .host = "localhost";
 3  .port = "8080";
 4}
 5
 6sub vcl_backend_response {
 7  if (bereq.url ~ "^/api/posts") {
 8    set beresp.ttl = 5m;
 9  } else if (bereq.url ~ "^/api/user/") {
10    set beresp.ttl = 0s; // Don't cache
11  }
12}

Requests for cached routes go through Nginx/Varnish. If a response is cached, it returns immediately without hitting your app. If not, it forwards to your backend, caches the response, and returns it.

Pros: Transparent caching. Your application doesn’t need to know it’s happening. Scales extremely well (thousands of requests per second). Can handle spike traffic without hitting backend.

Cons: Only works for HTTP. Hard to invalidate (you have to purge the cache explicitly). Not suitable for dynamic, user-specific, or authenticated content.

Use for: Public APIs, semi-static endpoints, content that doesn’t change per-user, protecting your origin from traffic spikes.

Database Built-In Caching

Modern databases like PostgreSQL have built-in caching (shared_buffers, OS page cache). You don’t control it directly, but you can tune it.

1-- In postgresql.conf
2shared_buffers = 256MB

This is the first line of defense. Your OS also caches disk pages automatically (page cache). For most applications, this is enough for small datasets.

Pros: Zero application code. Automatic.

Cons: Limited to the database machine. If you add a second database for backup and failover, it has its own separate cache. Each database instance caches independently, so you’re not sharing the benefit across servers. Not useful at massive scale.

Use for: Small datasets (< 10GB), when you don’t want to deploy separate infrastructure.

Putting It Together: A Real Example

A common pattern: an analytics endpoint that runs three expensive aggregates. Without caching, at 1000 concurrent users refreshing every 10 seconds, this is 100+ complex queries per second against a database designed for 50 QPS.

Here’s the uncached approach:

 1func GetDailyStats(w http.ResponseWriter, r *http.Request) {
 2  userID := r.URL.Query().Get("user_id")
 3  
 4  // Three expensive queries
 5  count := db.Query(`SELECT COUNT(*) FROM events WHERE user_id = ? AND date(created_at) = ?`, userID, today)
 6  revenue := db.Query(`SELECT SUM(amount) FROM events WHERE user_id = ? AND date(created_at) = ?`, userID, today)
 7  topEvents := db.Query(`SELECT event_type, COUNT(*) FROM events WHERE user_id = ? AND date(created_at) = ? GROUP BY event_type ORDER BY COUNT(*) DESC LIMIT 10`, userID, today)
 8  
 9  w.Header().Set("Content-Type", "application/json")
10  json.NewEncoder(w).Encode(map[string]interface{}{
11    "count": count,
12    "revenue": revenue,
13    "topEvents": topEvents,
14  })
15}

With layered caching:

 1func GetDailyStats(w http.ResponseWriter, r *http.Request) {
 2  userID := r.URL.Query().Get("user_id")
 3  ctx := r.Context()
 4  
 5  // 1. Check context cache first (avoid refetching in a single request)
 6  cacheKey := "stats:" + userID + ":" + today
 7  if cached := ctx.Value(cacheKey); cached != nil {
 8    w.Header().Set("X-Cache", "context")
 9    return writeStats(w, cached)
10  }
11  
12  // 2. Check Redis (shared across all servers, 5-minute TTL)
13  if cached, err := redis.Get(ctx, cacheKey).Result(); err == nil {
14    w.Header().Set("X-Cache", "redis")
15    ctx = context.WithValue(ctx, cacheKey, cached)
16    return writeStats(w, cached)
17  }
18  
19  // 3. Cache miss, but before hitting DB, check if another request is computing
20  // Use a lock to ensure only one request hits the DB for this key
21  once, _ := computeOnce.LoadOrStore(cacheKey, &sync.Once{})
22  once.(*sync.Once).Do(func() {
23    // This block runs once, other requests wait for it to finish
24    stats := computeStats(userID, today) // Hits DB
25    statJSON, _ := json.Marshal(stats)
26    
27    // Store in Redis for next 5 minutes
28    redis.Set(ctx, cacheKey, statJSON, 5*time.Minute)
29    
30    // Store in context for this request
31    ctx = context.WithValue(ctx, cacheKey, stats)
32  })
33  
34  // 4. At the HTTP layer, tell CDN to cache public dashboards
35  if r.Header.Get("Authorization") == "" { // Public request
36    w.Header().Set("Cache-Control", "public, max-age=60")
37  }
38  w.Header().Set("X-Cache", "db")
39  
40  // Get stats from context (now populated by the once.Do block)
41  stats := ctx.Value(cacheKey)
42  writeStats(w, stats)
43}

What each layer does:

1. Context cache: Within a single request, avoid refetching the same stats if called multiple times. Zero network latency.
2. Redis cache: Shared across all servers with a 5-minute TTL. Most requests hit this.
3. sync.Once lock: When Redis expires, only one request recomputes. Others wait for it to finish and grab the result. Prevents thundering herd.
4. CDN headers: For public dashboards, tell the CDN to cache the response. Removes traffic before it reaches your servers.

The math:

• Without cache: 100 queries/sec hitting the database.
• With cache: ~3 queries/sec (one per 5 minutes per user across 10,000 users).

This pattern scales. The caching layers handle load at different points: within the request, across servers, and at the edge.

Cache in Distributed Systems

When you move from one server to many, caching changes.

Local Cache Problems

In-memory caches don’t synchronize across servers. If you cache data locally on server A, server B doesn’t know about it. Users get inconsistent views depending on which server handles their request.

1User A makes request to server 1 → cache hit, sees data version 1
2User A makes request to server 2 → cache miss, sees data version 2
3User is confused.

Solution 1: Use a shared cache (Redis) instead of local cache.

Solution 2: Use versioning. Include a version number in the cache key, and bump the version when data changes.

1// Use version in the key
2versionedKey := "user:" + userID + ":v" + user.Version
3cache.Set(versionedKey, userData, 24*time.Hour)
4
5// All servers see the same version
6// No coordination needed

Cache Invalidation Across Servers

When data changes on one server, how do you tell all the other servers to invalidate their cache?

Option 1: Event-driven invalidation. When a user updates their profile, publish an event to a message queue. All servers listen to that queue and invalidate the relevant cache key.

1// When profile is updated
2profileUpdated := ProfileUpdatedEvent{UserID: userID, NewVersion: newVersion}
3eventBus.Publish("profile.updated", profileUpdated)
4
5// On all servers
6eventBus.Subscribe("profile.updated", func(e ProfileUpdatedEvent) {
7  cache.Delete("user:" + e.UserID + ":v*") // Delete old versions
8})

Option 2: Time-based invalidation (TTL). Don’t try to coordinate. Just set a short TTL and let caches expire naturally.

1cache.Set("user:"+userID, data, 5*time.Minute)
2// After 5 minutes, all servers refresh independently

Option 3: Hybrid. Set a medium TTL, but also support explicit invalidation when data changes.

1// Set with 1-hour TTL as a safety net
2cache.Set("user:"+userID, data, 1*time.Hour)
3
4// But also listen for updates and invalidate immediately
5eventBus.Subscribe("profile.updated", func(e ProfileUpdatedEvent) {
6  cache.Delete("user:" + e.UserID)
7})

Cache Warming

On startup, servers have empty caches. The first requests are slow (cache misses, database hits). To prevent this, you can “warm” the cache by preloading hot data.

 1func (s *Server) Start() error {
 2  // On startup, preload the top 1000 users from the database
 3  topUsers, err := s.db.GetTopUsers(1000)
 4  if err != nil {
 5    return err
 6  }
 7  
 8  for _, user := range topUsers {
 9    key := "user:" + user.ID
10    s.cache.Set(key, user, 24*time.Hour)
11  }
12  
13  return s.ListenAndServe()
14}

This ensures that as soon as the server comes online, the hot data is ready. Users get cache hits immediately instead of waiting for cache misses to populate it.

Cascading Cache Failures

In a distributed system, if a shared cache (like Redis) goes down, all servers lose their cache simultaneously. If you don’t handle this gracefully, your database gets hammered.

 1// Bad: crash if Redis is unavailable
 2value := redis.Get("user:" + userID) // panics if Redis down
 3
 4// Good: fall back to database on cache error
 5value, err := redis.Get("user:" + userID)
 6if err != nil {
 7  // Cache error, fall back to DB but don't cache the result
 8  // to avoid hitting the DB repeatedly
 9  return db.GetUser(userID), nil
10}
11return value, nil

Better: use a circuit breaker. A circuit breaker watches for repeated failures (cache is down, timeouts, etc.). Once failures exceed a threshold, it “opens” and stops attempting to use the cache, letting requests go straight to the database. Once the cache recovers, it slowly re-enables it.

 1if cacheCircuitBreaker.IsOpen() {
 2  // Circuit is open because cache has been failing
 3  // Skip the cache and hit the database directly
 4  return db.GetUser(userID), nil
 5}
 6
 7value, err := redis.Get("user:" + userID)
 8if err != nil {
 9  cacheCircuitBreaker.RecordFailure()
10  // If failures pile up, the circuit breaker will open
11  return db.GetUser(userID), nil // Fallback
12}
13cacheCircuitBreaker.RecordSuccess()
14// Successful read means the cache is recovering
15return value, nil

The Gotchas

Cache is powerful, but easy to get wrong.

Cache Invalidation Is Hard

The old saying: “There are only two hard things in Computer Science: cache invalidation and naming things.”

When data changes, you have to invalidate the cache. Miss this, and users see stale data. Example:

1// Update user profile
2db.UpdateUser(userID, newData)
3
4// Oops, forgot to invalidate the cache
5// Users still see the old profile for 24 hours

Solution: write invalidation at the same time as the write.

1db.UpdateUser(userID, newData)
2cache.Delete("user:" + userID) // Invalidate cache

Or, use a pattern where cache keys change with data versions:

1// Include a version in the key
2cacheKey := "user:" + userID + ":v" + user.Version
3cache.Set(cacheKey, newData, 24*time.Hour)

Thundering Herd

When a cache entry expires and multiple requests try to refresh it simultaneously, they all hit the database at once.

Solution 1: Use leases. Only one request recomputes the value, others wait.

 1lease, err := cache.Lease("expensive:" + key)
 2if lease != nil {
 3  // This request won the lease and recomputes
 4  result := expensiveComputation()
 5  cache.Set(key, result, 5*time.Minute)
 6  lease.Release()
 7} else {
 8  // Other requests wait for the winner to populate the cache
 9  result := cache.WaitFor(key, 1*time.Second)
10  return result
11}

Solution 2: Extend TTL before expiry. Refresh the cache in the background before it expires.

Hot Keys

Some keys are accessed far more than others. A celebrity’s profile, a viral post, a feature flag that controls your entire app. If that key is cached on one Redis instance, that instance gets hammered while others are idle.

Solution: replicate hot keys across multiple cache instances. Or use a local cache in front of Redis.

Storage Bloat

Caching everything leads to memory exhaustion. You cache the entire feed for every user, suddenly you need 10TB of Redis memory.

Solution: be selective. Cache only what is expensive and frequently accessed. Use smaller TTLs for less critical data.

Rules of Thumb

1. Profile data: Write-through cache (update cache when profile is updated). 24-hour TTL.
2. Counts and aggregates: Precomputed table. Update when the underlying data changes (a user follows, a post gets a like).
3. User-specific data (feed, recommendations): Read-through cache. 5-10 minute TTL.
4. Static routes (public pages): Route cache or CDN. 24+ hour TTL.
5. Expensive computations: Function cache with input-based keys. No expiration, invalidate on code change.
6. Hot data within requests: Context cache. Automatic cleanup after request.

Why This Matters

At 2 a.m., when your database is on fire and your monitoring is screaming, you will not be thinking about the theoretical purity of your system. You will be thinking about what is hitting the database and why.

The engineer who understands cache sees the problem immediately: “Which queries are running that don’t need to? What can I cache?”

The engineer who doesn’t understand cache sees chaos. They add more database replicas. They tune connection pools. They blame the database. Hours pass. The problem is still there.

Cache is not something you add as a performance optimization. Cache is something you build into the architecture from day one, because without it, your database doesn’t survive.

Start simple. Cache one query. See the impact. Cache another. Your database will thank you.

Further Reading

• Scaling Memcache at Facebook. How Facebook keeps a billion QPS alive with intelligent caching.
• Redis documentation. Reference and tutorials.
• Nginx caching guide. HTTP caching at the reverse proxy layer.
• HTTP caching (MDN). Browser and CDN cache headers.