Cache (GitHub System Design)

14 Cache (GitHub System Design Primer) Cache *[ Source: Scalable system design patterns ]()* Caching improves page load times and can reduce the load on your servers and == == == == == == == == == == == databases . In this model, the dispatcher will first lookup if the request has been == == == *** *** == == made before and try to find the previous result to return, in order to save the actual == == execution. (Otherwise the Dispatcher forwards the request to a worker ) == == Databases often benefit from a uniform distribution of reads and writes across its == == == partitions . Popular items can skew the distribution , causing bottlenecks . == == == == == == == — Putting a cache in front of a database can help absorb uneven loads and spikes == in traffic . == == == == == == == == == == Client caching == Caches can be located on the client side ( OS or browser ), server side, or in a == == == == == [ ]() *** *** == CDN caching == CDNs are considered a type of cache. ]() == [ == distinct cache layer . == [ == == Web server caching == Reverse proxies and caches such as Varnish can serve static and dynamic content == ]() [ == == ]() == == == == directly. Web servers can also cache requests , returning responses without having to == == == == contact application servers . == == == Database caching == Your database usually includes some level of caching in a default configuration , == == optimized for a generic use case. Tweaking these settings for specific usage patterns == == can further boost performance. == Application caching == In-memory caches such as Memcached and Redis are key-value stores between == == == == == == your application and your data storage . Since the data is held in RAM , it is much == == == == == == == faster than typical databases where data is stored on disk . RAM is more limited than == == == == disk , so cache invalidation algorithms such as least recently used (LRU) can help == [ == == ]() [ == == invalidate 'cold' entries and keep 'hot' data in RAM. == == == == == Redis has the following additional features: == == == ⠀ Persistence option == Built-in data structures such as sorted sets and lists == == == == == ]() There are multiple levels you can cache that fall into two general categories: == ** == == database queries and objects : == == == Row level ** ** == == ** == Query-level == Fully-formed serializable objects == == == Fully-rendered HTML == ⠀ *** Generally, you should try to avoid file-based caching , as it makes cloning and auto*** *** *** == == == == == scaling more difficult. == Why does file based caching make cloning and auto-scaling more difficult? File-based caching makes cloning and auto-scaling more difficult due to the ** ** ** ** following key reasons: ** 1. Lack of Shared State Across Instances ** ** File-based caches are stored locally on the filesystem of a single server ** instance. In a distributed system: When new instances are cloned or added during auto-scaling, they do not have access to the cached data stored on other instances. Each instance ends up with its own isolated cache, leading to cache ** fragmentation. ** ** 2. Cache Inconsistency ** Since the cache is stored locally: Different instances might have different versions of cached data. ** ** This leads to cache inconsistency, where requests handled by ** ** different instances may yield inconsistent results. Synchronizing file-based caches across multiple instances is non-trivial and requires additional mechanisms, such as file replication or a shared storage solution. ** 3. Increased Latency for New Instances ** When new instances are cloned during auto-scaling: They start with an empty cache because the cached files are not ** ** shared. This results in a "cold start", where the new instances must fetch all ** ** data from the original source (e.g., database) again. This degrades performance until the cache warms up. ** ** ** 4. Storage Limitations ** In auto-scaling environments, the local disk space available for caching on ** ** each instance might become a bottleneck: If the cache grows too large, the system risks running out of disk space on some instances. This makes file-based caching harder to manage at scale. ** 5. Difficulty in Load Balancing ** In environments with multiple clones of an application: A load balancer directs requests to different instances. ** ** Since each instance has its own isolated file-based cache, there is no ** guarantee that a cache hit on one instance will be valid on another. ** This reduces the overall cache hit rate, increasing the load on the backend systems. ** 6. Challenges with Deployment and Consistency ** When deploying updates or scaling, maintaining the state of the cache ** across instances becomes a challenge: ** ** **