Skip to content
- Tutorials
- Courses
- Go Premium
-
Google File System
Last Updated :
04 Jan, 2025
Google Inc. developed the Google File System (GFS), a scalable distributed file system (DFS), to meet the company's growing data processing needs. GFS offers fault tolerance, dependability, scalability, availability, and performance to big networks and connected nodes. GFS is made up of a number of storage systems constructed from inexpensive commodity hardware parts. The search engine, which creates enormous volumes of data that must be kept, is only one example of how it is customized to meet Google's various data use and storage requirements.
The Google File System reduced hardware flaws while gains of commercially available servers.
GoogleFS is another name for GFS. It manages two types of data namely File metadata and File Data.
The GFS node cluster consists of a single master and several chunk servers that various client systems regularly access. On local discs, chunk servers keep data in the form of Linux files. Large (64 MB) pieces of the stored data are split up and replicated at least three times around the network. Reduced network overhead results from the greater chunk size.
Without hindering applications, GFS is made to meet Google's huge cluster requirements. Hierarchical directories with path names are used to store files. The master is in charge of managing metadata, including namespace, access control, and mapping data. The master communicates with each chunk server by timed heartbeat messages and keeps track of its status updates.
More than 1,000 nodes with 300 TB of disc storage capacity make up the largest GFS clusters. This is available for constant access by hundreds of clients.
Components of GFS
A group of computers makes up GFS. A cluster is just a group of connected computers. There could be hundreds or even thousands of computers in each cluster. There are three basic entities included in any GFS cluster as follows:
- GFS Clients: They can be computer programs or applications which may be used to request files. Requests may be made to access and modify already-existing files or add new files to the system.
- GFS Master Server: It serves as the cluster's coordinator. It preserves a record of the cluster's actions in an operation log. Additionally, it keeps track of the data that describes chunks, or metadata. The chunks' place in the overall file and which files they belong to are indicated by the metadata to the master server.
- GFS Chunk Servers: They are the GFS's workhorses. They keep 64 MB-sized file chunks. The master server does not receive any chunks from the chunk servers. Instead, they directly deliver the client the desired chunks. The GFS makes numerous copies of each chunk and stores them on various chunk servers in order to assure stability; the default is three copies. Every replica is referred to as one.
Features of GFS
- Namespace management and locking.
- Fault tolerance.
- Reduced client and master interaction because of large chunk server size.
- High availability.
- Critical data replication.
- Automatic and efficient data recovery.
- High aggregate throughput.
Advantages of GFS
- High accessibility Data is still accessible even if a few nodes fail. (replication) Component failures are more common than not, as the saying goes.
- Excessive throughput. many nodes operating concurrently.
- Dependable storing. Data that has been corrupted can be found and duplicated.
Disadvantages of GFS
- Not the best fit for small files.
- Master may act as a bottleneck.
- unable to type at random.
- Suitable for procedures or data that are written once and only read (appended) later.
Similar Reads
DevOps Tutorial DevOps is a combination of two words: "Development" and "Operations." It’s a modern approach where software developers and software operations teams work together throughout the entire software life cycle.The goals of DevOps are:Faster and continuous software releases.Reduces manual errors through a
7 min read
Introduction
What is DevOps ?DevOps is a modern way of working in software development in which the development team (who writes the code and builds the software) and the operations team (which sets up, runs, and manages the software) work together as a single team.Before DevOps, the development and operations teams worked sepa
10 min read
DevOps LifecycleThe DevOps lifecycle is a structured approach that integrates development (Dev) and operations (Ops) teams to streamline software delivery. It focuses on collaboration, automation, and continuous feedback across key phases planning, coding, building, testing, releasing, deploying, operating, and mon
10 min read
The Evolution of DevOps - 3 Major Trends for FutureDevOps is a software engineering culture and practice that aims to unify software development and operations. It is an approach to software development that emphasizes collaboration, communication, and integration between software developers and IT operations. DevOps has come a long way since its in
7 min read
Version Control
Continuous Integration (CI) & Continuous Deployment (CD)
Containerization
Orchestration
Infrastructure as Code (IaC)
Monitoring and Logging
Microsoft Teams vs Slack Both Microsoft Teams and Slack are the communication channels used by organizations to communicate with their employees. Microsoft Teams was developed in 2017 whereas Slack was created in 2013. Microsoft Teams is mainly used in large organizations and is integrated with Office 365 enhancing the feat
4 min read
Security in DevOps
`;
$(commentSectionTemplate).insertBefore(".article--recommended");
}
loadComments();
});
});
function loadComments() {
if ($("iframe[id*='discuss-iframe']").length top_of_element && top_of_screen articleRecommendedTop && top_of_screen articleRecommendedBottom)) {
if (!isfollowingApiCall) {
isfollowingApiCall = true;
setTimeout(function(){
if (loginData && loginData.isLoggedIn) {
if (loginData.userName !== $('#followAuthor').val()) {
is_following();
} else {
$('.profileCard-profile-picture').css('background-color', '#E7E7E7');
}
} else {
$('.follow-btn').removeClass('hideIt');
}
}, 3000);
}
}
});
}
$(".accordion-header").click(function() {
var arrowIcon = $(this).find('.bottom-arrow-icon');
arrowIcon.toggleClass('rotate180');
});
});
window.isReportArticle = false;
function report_article(){
if (!loginData || !loginData.isLoggedIn) {
const loginModalButton = $('.login-modal-btn')
if (loginModalButton.length) {
loginModalButton.click();
}
return;
}
if(!window.isReportArticle){
//to add loader
$('.report-loader').addClass('spinner');
jQuery('#report_modal_content').load(gfgSiteUrl+'wp-content/themes/iconic-one/report-modal.php', {
PRACTICE_API_URL: practiceAPIURL,
PRACTICE_URL:practiceURL
},function(responseTxt, statusTxt, xhr){
if(statusTxt == "error"){
alert("Error: " + xhr.status + ": " + xhr.statusText);
}
});
}else{
window.scrollTo({ top: 0, behavior: 'smooth' });
$("#report_modal_content").show();
}
}
function closeShareModal() {
const shareOption = document.querySelector('[data-gfg-action="share-article"]');
shareOption.classList.remove("hover_share_menu");
let shareModal = document.querySelector(".hover__share-modal-container");
shareModal && shareModal.remove();
}
function openShareModal() {
closeShareModal(); // Remove existing modal if any
let shareModal = document.querySelector(".three_dot_dropdown_share");
shareModal.appendChild(Object.assign(document.createElement("div"), { className: "hover__share-modal-container" }));
document.querySelector(".hover__share-modal-container").append(
Object.assign(document.createElement('div'), { className: "share__modal" }),
);
document.querySelector(".share__modal").append(Object.assign(document.createElement('h1'), { className: "share__modal-heading" }, { textContent: "Share to" }));
const socialOptions = ["LinkedIn", "WhatsApp","Twitter", "Copy Link"];
socialOptions.forEach((socialOption) => {
const socialContainer = Object.assign(document.createElement('div'), { className: "social__container" });
const icon = Object.assign(document.createElement("div"), { className: `share__icon share__${socialOption.split(" ").join("")}-icon` });
const socialText = Object.assign(document.createElement("span"), { className: "share__option-text" }, { textContent: `${socialOption}` });
const shareLink = (socialOption === "Copy Link") ?
Object.assign(document.createElement('div'), { role: "button", className: "link-container CopyLink" }) :
Object.assign(document.createElement('a'), { className: "link-container" });
if (socialOption === "LinkedIn") {
shareLink.setAttribute('href', `https://www.linkedin.com/sharing/share-offsite/?url=${window.location.href}`);
shareLink.setAttribute('target', '_blank');
}
if (socialOption === "WhatsApp") {
shareLink.setAttribute('href', `https://api.whatsapp.com/send?text=${window.location.href}`);
shareLink.setAttribute('target', "_blank");
}
if (socialOption === "Twitter") {
shareLink.setAttribute('href', `https://twitter.com/intent/tweet?url=${window.location.href}`);
shareLink.setAttribute('target', "_blank");
}
shareLink.append(icon, socialText);
socialContainer.append(shareLink);
document.querySelector(".share__modal").appendChild(socialContainer);
//adding copy url functionality
if(socialOption === "Copy Link") {
shareLink.addEventListener("click", function() {
var tempInput = document.createElement("input");
tempInput.value = window.location.href;
document.body.appendChild(tempInput);
tempInput.select();
tempInput.setSelectionRange(0, 99999); // For mobile devices
document.execCommand('copy');
document.body.removeChild(tempInput);
this.querySelector(".share__option-text").textContent = "Copied"
})
}
});
// document.querySelector(".hover__share-modal-container").addEventListener("mouseover", () => document.querySelector('[data-gfg-action="share-article"]').classList.add("hover_share_menu"));
}
function toggleLikeElementVisibility(selector, show) {
document.querySelector(`.${selector}`).style.display = show ? "block" : "none";
}
function closeKebabMenu(){
document.getElementById("myDropdown").classList.toggle("show");
}