Skip to content

Cache Eviction Policies | System Design

Last Updated : 23 Jul, 2025

The process of clearing data from a cache to create space for fresh or more relevant information is known as cache eviction. It enhances system speed by caching and storing frequently accessed data for faster retrieval. Caches have a limited capacity, though, and the system must choose which data to delete when the cache is full. The cache eviction policies provide the criteria for choosing which data to replace.

Cache-Eviction-Policies-(1)

Table of Content

What are Cache Eviction Policies?

Algorithms or techniques known as cache eviction policies decide which data should be deleted from a cache when its capacity is reached. By keeping the most relevant and often requested data in the cache, these strategies seek to optimize its effectiveness. In systems with limited cache space, effective cache eviction procedures are essential for preserving peak performance while guaranteeing that important data is kept for easy access.

Cache Eviction Policies

Some of the most important and common cache eviction strategies are:

1. Least Recently Used(LRU)

When the cache hits its capacity limit, the Least Recently Used (LRU) cache eviction policy is designed to eliminate the item that has been accessed the least recently. Items that have not been accessed for a longer period of time are assumed to be less likely to be used in the near future. When the cache is full, LRU evicts the item that hasn't been accessed in the longest time since it keeps track of the order in which items are accessed.

Least-Recently-Used

For Example:

Let us consider a cache with a maximum capacity of 3, initially containing items A, B, and C in that order.

If a new item, D, is accessed, the cache becomes full, and the LRU policy would evict the least recently used item, which is A. The cache now holds items B, C, and D.
If item B is accessed next, the order becomes C, D, B.
If another item, E, is accessed, the cache is full again, and the LRU policy would evict C, resulting in the cache holding items D, B, and E. The order now is B, E, D.

LRU ensures that the most recently accessed items are retained in the cache, optimizing for scenarios where recent access patterns are indicative of future accesses.

Advantages of Least Recently Used(LRU)

Easy Implementation: LRU is a simple option for many caching applications due to its ease of understanding and implementation.
Efficient Use of Cache: When current accesses are a reliable indicator of future accesses, LRU works well. It guarantees that items that are accessed frequently are more likely to remain in the cache.
Adaptability: LRU is adaptable to various types of applications, including databases, web caching, and file systems.

Disadvantages of Least Recently Used(LRU)

Strict Ordering: LRU assumes that the order of access accurately reflects the future usefulness of an item. In certain cases, this assumption may not hold true, leading to suboptimal cache decisions.
Cold Start Issues: When a cache is initially populated, LRU might not perform optimally as it requires sufficient historical data to make informed eviction decisions.
Memory Overhead: Implementing LRU often requires additional memory to store timestamps or maintain access order, which can impact the overall memory consumption of the system.

Use Cases of Least Recently Used(LRU)

Web Caching:
- LRU is commonly employed to store frequently accessed web pages, images, or resources. This helps in reducing latency by keeping the most recently used content readily available, improving overall website performance.
Database Management:
- LRU is often used in database systems to cache query results or frequently accessed data pages. This accelerates query response times by keeping recently used data in memory, reducing the need to fetch data from slower disk storage.
File Systems:
- File systems can benefit from LRU when caching file metadata or directory information. Frequently accessed files and directories are kept in the cache, improving file access speed and reducing the load on the underlying storage.

2. Least Frequently Used(LFU)

The least frequently accessed entries are eliminated first under the LFU cache eviction policy. It is based on the idea that things that are used the least are less likely to be needed later. When the cache is full, LFU removes the item with the lowest access frequency after keeping track of the amount of times each item is accessed.

Least-Frequently-Used

For Example:

Consider a cache with items X, Y, and Z. If item Z has been accessed fewer times than items X and Y, the LFU policy will retain the items X and Y and potentially evict item Z when the cache reaches its capacity.

Advantages of Least Frequently Used(LFU)

Adaptability to Varied Access Patterns: LFU is effective in scenarios where some items may be accessed infrequently but are still essential. It adapts well to varying access patterns.
Optimized for Long-Term Trends: LFU can be beneficial when the relevance of an item is better captured by its overall frequency of access over time rather than recent accesses.
Low Memory Overhead: Since LFU doesn't need to keep timestamps, it might have less memory overhead than some LRU implementations.

Disadvantages of Least Frequently Used(LFU)

Sensitivity to Initial Access:
- LFU may not perform optimally during the initial stages when access frequencies are still being established. It relies on historical access patterns, and a new or less frequently accessed item might not be retained in the cache until its long-term frequency is established.
Difficulty in Handling Changing Access Patterns:
- LFU can struggle in scenarios where access patterns change frequently. Items that were once heavily accessed but are no longer relevant might continue to be retained in the cache.
Complexity of Frequency Counters:
- Implementing accurate frequency counting for items can add complexity to LFU implementations.

Use Cases of Least Frequently Used(LFU)

Database Query Caching: In database management systems, LFU can be applied to cache query results or frequently accessed data.
Network Routing Tables: LFU is useful in caching routing information for networking applications. Items representing less frequently used routes are kept in the cache, allowing for efficient routing decisions based on historical usage.
Content Recommendations: In content recommendation systems, LFU can be employed to cache information about user preferences or content suggestions. It ensures that even less frequently accessed recommendations are considered over time.

3. First-In-First-Out(FIFO)

First-In-First-Out (FIFO) is a cache eviction policy that removes the oldest item from the cache when it becomes full. In this strategy, data is stored in the cache in the order it arrives, and the item that has been present in the cache for the longest time is the first to be evicted when the cache reaches its capacity.

First-In-First-Out

For Example:

Imagine a cache with a capacity of three items:
A is added to the cache.
B is added to the cache.
C is added to the cache.
At this point, the cache is full (capacity = 3)

If a new item, D, needs to be added, the FIFO policy would dictate that the oldest item, A, should be evicted. The cache would then look like:

D is added to the cache (A is evicted).
The order of items in the cache now is B, C, and D, reflecting the chronological order of their arrival.

Advantages of First-In-First-Out(FIFO)

Simple Implementation: FIFO is straightforward to implement, making it an easy choice for scenarios where simplicity is a priority.
Predictable Behavior: The eviction process in FIFO is predictable and follows a strict order based on the time of entry into the cache.
Memory Efficiency: Since FIFO eliminates the need for extra tracking of timestamps and access frequency, it has a comparatively minimal memory overhead when compared to some other eviction strategies.

Disadvantages of First-In-First-Out(FIFO)

Lack of Adaptability: FIFO may not adapt well to varying access patterns. It strictly adheres to the order of entry, which might not reflect the actual importance or relevance of items.
Inefficiency in Handling Variable Importance: FIFO might lead to inefficiencies when newer items are more relevant or frequently accessed than older ones.
Cold Start Issues: When a cache is initially populated or after a cache flush, FIFO may not perform optimally, as it tends to keep items in the cache based solely on their entry time, without considering their actual usage.

Use Cases of First-In-First-Out(FIFO)

Task Scheduling in Operating Systems: In task scheduling, FIFO can be employed to determine the order in which processes or tasks are executed.
Message Queues: FIFO guarantees that messages are handled in the order that they are received in message queuing systems. In applications that depend on message-based communication, this is essential for preserving the order of processes.
Cache for Streaming Applications: For some streaming applications where preserving the data's order is crucial, FIFO may be appropriate. For instance, FIFO guarantees that frames are displayed in the proper order in a video streaming cache.

4. Random Replacement

Random Replacement is a cache eviction policy where, when the cache is full and a new item needs to be stored, a randomly chosen existing item is evicted to make room. Unlike some deterministic policies like LRU (Least Recently Used) or FIFO (First-In-First-Out), which have specific criteria for selecting items to be evicted, Random Replacement simply selects an item at random.

Random-Replacement

For Example:

Consider a cache with three slots and the following data:
Item A
Item B
Item C

Now, if the cache is full and a new item, Item D, needs to be stored, Random Replacement might choose to evict Item B, resulting in:

Item A
Item D
Item C

The selection of Item B for eviction is entirely random in this policy, making it a straightforward but less predictable strategy compared to others.

Advantages of Random Replacement

Simplicity: Random replacement is a straightforward and easy-to-implement strategy. It does not require complex tracking or analysis of access patterns.
Avoids Biases: Since random replacement doesn't rely on historical usage patterns, it avoids potential biases that may arise in more deterministic policies.
Low Overhead: The algorithm involves minimal computational overhead, making it efficient in terms of processing requirements.

Disadvantages of Random Replacement

Suboptimal Performance: Random replacement may lead to suboptimal cache performance compared to more sophisticated policies. It doesn't consider the actual usage patterns or the likelihood of future accesses.
No Adaptability: It lacks adaptability to changing access patterns. Other eviction policies, like LRU or LFU, consider the historical behavior of items and adapt to evolving patterns, potentially providing better cache performance over time.
Possibility of Poor Hit Rates: The random nature of eviction may result in poor hit rates, where frequently accessed items are unintentionally evicted, leading to more cache misses.

Use Cases of Random Replacement

Non-Critical Caching Environments:
- In scenarios where the impact of cache misses is minimal or where caching is employed for non-critical purposes, such as temporary storage of non-essential data, random replacement can be sufficient.
Simulation and Testing:
- In testing situations and simulation environments where simplicity and convenience of use are more important than complex eviction policies, random replacement is helpful.
Resource-Constrained Systems:
- In resource-constrained environments, where computational resources are limited, the low overhead of random replacement may be advantageous.

Conclusion

In conclusion, cache eviction policies play a crucial role in system design, impacting the efficiency and performance of caching mechanisms. The choice of an eviction policy depends on the specific characteristics and requirements of the system. While simpler policies like Random Replacement offer ease of implementation and low overhead, more sophisticated strategies such as Least Recently Used (LRU) or Least Frequently Used (LFU) take into account historical access patterns, leading to better adaptation to changing workloads.

Cache Eviction Policies | System Design

J

janardan333

Improve

Article Tags :

System Design

Similar Reads

System Design Tutorial

System Design is the process of designing the architecture, components, and interfaces for a system so that it meets the end-user requirements. This specifically designed System Design tutorial will help you to learn and master System Design concepts in the most efficient way, from the basics to the

Must Know System Design Concepts

We all know that System Design is the core concept behind the design of any distributed system. Therefore every person in the tech industry needs to have at least a basic understanding of what goes behind designing a System. With this intent, we have brought to you the ultimate System Design Intervi

`; $(commentSectionTemplate).insertBefore(".article--recommended"); } loadComments(); }); }); function loadComments() { if ($("iframe[id*='discuss-iframe']").length top_of_element && top_of_screen articleRecommendedTop && top_of_screen articleRecommendedBottom)) { if (!isfollowingApiCall) { isfollowingApiCall = true; setTimeout(function(){ if (loginData && loginData.isLoggedIn) { if (loginData.userName !== $('#followAuthor').val()) { is_following(); } else { $('.profileCard-profile-picture').css('background-color', '#E7E7E7'); } } else { $('.follow-btn').removeClass('hideIt'); } }, 3000); } } }); } $(".accordion-header").click(function() { var arrowIcon = $(this).find('.bottom-arrow-icon'); arrowIcon.toggleClass('rotate180'); }); }); window.isReportArticle = false; function report_article(){ if (!loginData || !loginData.isLoggedIn) { const loginModalButton = $('.login-modal-btn') if (loginModalButton.length) { loginModalButton.click(); } return; } if(!window.isReportArticle){ //to add loader $('.report-loader').addClass('spinner'); jQuery('#report_modal_content').load(gfgSiteUrl+'wp-content/themes/iconic-one/report-modal.php', { PRACTICE_API_URL: practiceAPIURL, PRACTICE_URL:practiceURL },function(responseTxt, statusTxt, xhr){ if(statusTxt == "error"){ alert("Error: " + xhr.status + ": " + xhr.statusText); } }); }else{ window.scrollTo({ top: 0, behavior: 'smooth' }); $("#report_modal_content").show(); } } function closeShareModal() { const shareOption = document.querySelector('[data-gfg-action="share-article"]'); shareOption.classList.remove("hover_share_menu"); let shareModal = document.querySelector(".hover__share-modal-container"); shareModal && shareModal.remove(); } function openShareModal() { closeShareModal(); // Remove existing modal if any let shareModal = document.querySelector(".three_dot_dropdown_share"); shareModal.appendChild(Object.assign(document.createElement("div"), { className: "hover__share-modal-container" })); document.querySelector(".hover__share-modal-container").append( Object.assign(document.createElement('div'), { className: "share__modal" }), ); document.querySelector(".share__modal").append(Object.assign(document.createElement('h1'), { className: "share__modal-heading" }, { textContent: "Share to" })); const socialOptions = ["LinkedIn", "WhatsApp","Twitter", "Copy Link"]; socialOptions.forEach((socialOption) => { const socialContainer = Object.assign(document.createElement('div'), { className: "social__container" }); const icon = Object.assign(document.createElement("div"), { className: `share__icon share__${socialOption.split(" ").join("")}-icon` }); const socialText = Object.assign(document.createElement("span"), { className: "share__option-text" }, { textContent: `${socialOption}` }); const shareLink = (socialOption === "Copy Link") ? Object.assign(document.createElement('div'), { role: "button", className: "link-container CopyLink" }) : Object.assign(document.createElement('a'), { className: "link-container" }); if (socialOption === "LinkedIn") { shareLink.setAttribute('href', `https://www.linkedin.com/sharing/share-offsite/?url=${window.location.href}`); shareLink.setAttribute('target', '_blank'); } if (socialOption === "WhatsApp") { shareLink.setAttribute('href', `https://api.whatsapp.com/send?text=${window.location.href}`); shareLink.setAttribute('target', "_blank"); } if (socialOption === "Twitter") { shareLink.setAttribute('href', `https://twitter.com/intent/tweet?url=${window.location.href}`); shareLink.setAttribute('target', "_blank"); } shareLink.append(icon, socialText); socialContainer.append(shareLink); document.querySelector(".share__modal").appendChild(socialContainer); //adding copy url functionality if(socialOption === "Copy Link") { shareLink.addEventListener("click", function() { var tempInput = document.createElement("input"); tempInput.value = window.location.href; document.body.appendChild(tempInput); tempInput.select(); tempInput.setSelectionRange(0, 99999); // For mobile devices document.execCommand('copy'); document.body.removeChild(tempInput); this.querySelector(".share__option-text").textContent = "Copied" }) } }); // document.querySelector(".hover__share-modal-container").addEventListener("mouseover", () => document.querySelector('[data-gfg-action="share-article"]').classList.add("hover_share_menu")); } function toggleLikeElementVisibility(selector, show) { document.querySelector(`.${selector}`).style.display = show ? "block" : "none"; } function closeKebabMenu(){ document.getElementById("myDropdown").classList.toggle("show"); }