Memcached is a high-performance, distributed memory caching system designed for simplicity and speed. Originally developed to accelerate dynamic web applications by reducing database load, Memcached remains a popular choice for applications needing straightforward, efficient caching without complex features.
Design Philosophy
Memcached follows a “keep it simple” philosophy. It provides a simple key-value store with basic operations: get, set, delete, and increment/decrement. This simplicity translates to exceptional performance and reliability. There are no complex data structures, no query language, and no persistence—just fast in-memory caching.
The lack of features is a strength, not a weakness. By focusing solely on caching, Memcached achieves minimal overhead and maximum throughput. For many applications, especially those with straightforward caching needs, Memcached’s simplicity is ideal.
Architecture
Memcached uses a client-server architecture where cache data resides in server memory and clients communicate via a simple text protocol over TCP or UDP. Multiple Memcached servers form a distributed cache, with clients responsible for distributing keys across servers using consistent hashing.
No Inter-Server Communication distinguishes Memcached from some distributed caches. Servers don’t know about each other or coordinate. This eliminates complex cluster management but means clients must handle sharding logic. Libraries in most languages provide consistent hashing implementations, making this transparent to application code.
Multi-Threaded Architecture allows Memcached to leverage multiple CPU cores efficiently. Each thread can handle multiple concurrent connections, maximizing throughput on modern multi-core hardware.
Operations and API
The API is intentionally minimal. GET retrieves values by key, SET stores key-value pairs with optional expiration, DELETE removes items, and ADD stores only if the key doesn’t exist. INCREMENT and DECREMENT provide atomic counter operations.
Multi-Get retrieves multiple keys in a single operation, essential for performance when fetching many values. Rather than 100 sequential gets, a single multi-get retrieves all values with one network round trip per involved server.
CAS (Check-and-Set) provides optimistic locking for implementing atomic updates without explicit locks. Read a value along with its CAS token, modify it, and write back with the token. The write succeeds only if the value hasn’t changed since reading.
Memory Management
Memcached allocates memory in slabs of fixed sizes to reduce fragmentation. Items are stored in slabs matching their size class, improving memory efficiency. Once the memory limit is reached, Memcached uses LRU (Least Recently Used) eviction to make room for new items.
Memory Limits are configured per server. Memcached won’t use more than the configured limit, making resource usage predictable. This contrasts with Redis, which can use swap space if physical memory is exhausted, potentially degrading performance dramatically.
Eviction Policy is strictly LRU within each slab class. Least-recently-used items are evicted when space is needed. This works well for caches where recently accessed items are more likely to be accessed again.
Distributed Caching
Clients distribute keys across Memcached servers using consistent hashing. Each key is hashed, and the hash determines which server stores that key. Libraries like libmemcached handle this transparently, providing a unified interface to the distributed cache.
Adding or Removing Servers requires rehashing, potentially invalidating many cached items. Consistent hashing minimizes this impact: adding a server invalidates only keys reassigned to the new server, not the entire cache. However, cache warming becomes necessary when scaling the cluster.
No Replication means each item exists on exactly one server. If a server fails, all cached items on that server are lost, resulting in cache misses until they’re repopulated. This keeps architecture simple but requires applications to handle cache failures gracefully.
Use Cases
Memcached excels at caching database query results. Store results of expensive queries, reducing database load and improving response times. The simple key-value model maps naturally to query caching: hash the query as the key, store results as the value.
Session storage is common, though requires handling session loss gracefully since Memcached provides no persistence guarantees. For non-critical session data, this is acceptable. For critical data, consider Redis with persistence or a database.
Full-page caching speeds up dynamic sites by caching rendered HTML pages. Web requests check Memcached first; cache hits serve instantly, while misses render the page and populate the cache.
API response caching reduces backend load for frequently requested data. API gateways or application code can cache responses based on request parameters, dramatically improving throughput and latency.
Limitations
No Persistence means data is lost when servers restart. For pure caching this is acceptable—caches warm up from database queries. For data requiring durability, use a database or Redis with persistence.
No Complex Data Structures limits Memcached to simple values. Need a list or set? You must serialize/deserialize yourself or use Redis. This simplicity is fine for basic caching but limiting for advanced use cases.
No Built-In Replication or failover means operations teams must implement high availability externally if needed. Redis Sentinel or Cluster provides built-in HA; Memcached requires external solutions or accepting cache loss on failures.
Performance Characteristics
Memcached is exceptionally fast. Operations complete in microseconds, with throughput easily reaching hundreds of thousands of operations per second per server. The multi-threaded architecture scales well on multi-core hardware.
Network efficiency is critical. Batch operations with multi-get, use persistent connections, and consider UDP for reduced connection overhead when losing occasional packets is acceptable.
Binary protocol offers slight performance improvements over the text protocol and is supported by most modern clients. However, the text protocol’s human-readability aids debugging.
Memcached vs Redis
Memcached is simpler, focusing solely on caching. Redis offers complex data structures, persistence, pub/sub, and more. For straightforward caching needs, Memcached’s simplicity is attractive. For applications needing advanced features, Redis is more appropriate.
Performance is comparable for simple operations; both are extremely fast. Redis’s single-threaded architecture can be a bottleneck for CPU-intensive operations, while Memcached’s multi-threading leverages more cores.
Operational complexity is lower for Memcached due to its simplicity. No persistence configuration, no cluster management, just start servers and connect. Redis requires more configuration but provides more capabilities.
Best Practices
Size your values appropriately. Memcached handles small values (a few KB) most efficiently. Storing multi-MB values reduces throughput and increases eviction likelihood.
Use consistent key naming to organize cached data and avoid collisions. Include version numbers in keys to invalidate entire categories when data models change.
Monitor cache hit rates to ensure caching is effective. Low hit rates suggest TTLs are too short, cache size is insufficient, or data access patterns aren’t cache-friendly.
Handle failures gracefully. Cache operations should never make your application fail. If Memcached is unreachable, log the error and continue without caching.
Warm up caches after deploying new servers or clearing caches. Pre-populate frequently accessed items to avoid overwhelming databases with cache misses.
Memcached remains relevant decades after its creation because it does one thing exceptionally well: fast, distributed caching. For applications with straightforward caching needs, Memcached provides excellent performance with minimal operational overhead. Understanding when to choose Memcached’s simplicity over Redis’s features is key to effective cache architecture.