Posts from all blogs: open source

Twitter is a global real-time communications platform that processes many billions of events every day. Aggregating these events in real time presents a massive challenge of scale. Classic time-series applications include site traffic, service health, and user engagement monitoring; these are increasingly complemented by a range of analytics products and features such as Tweet activity, Followers, and Twitter Cards that surface aggregated time-series data directly to end users, publishers, and advertisers.Read more…

It’s been just over two years since we open sourced Scalding and today we are very excited to release the 0.9 version. Scalding at Twitter powers everything from internal and external facing dashboards, to custom relevance and ad targeting algorithms, including many graph algorithms such as PageRank, approximate user cosine similarity and many more.

There have been a wide breadth of new features added to Scalding since the last release:Read more…

Finagle is our fault tolerant, protocol-agnostic RPC framework built atop Netty. Twitter’s core services are built on Finagle, from backends serving user profile information, Tweets, and timelines to front end API endpoints handling HTTP requests.Read more…

With the latest Mesos 0.15.0 release, we are pleased to report that we’ve added initial authentication support for frameworks (see MESOS-418) connecting to Mesos. In a nutshell, this feature allows only authenticated frameworks to register with Mesos and launch tasks. Authentication is important as it prevents rogue frameworks from causing problems that may impact the usage of resources within a Mesos cluster.Read more…