Tumblr’s Finagle Redis Implementation

Tumblr uses redis as a cache/store for a number of our internal applications. Some of the things that depend on redis include:

• dashboard notifications (via staircar)
• Tiny URL’s (via gob)
• gearman persistent storage (via george)
• note counts (also via staircar, coming soon)
• dashboard data (coming soon via ira)

Over the past six months we’ve started shifting internal services from being C/libevent based to being Scala/Finagle based. Finagle is a wonderful little library for building asynchronous RPC servers and clients. We’ve built a number of services (besides the ones above) with it, some of our usage I talked about here.

Back in October we created a native redis implementation for finagle, and as of yesterday Marius (one of the finagle leads at Twitter) had pulled it into master. Although I’m sure this will take a slightly different shape over the coming months as Twitter engineers have a chance to work with it, I’m proud to have helped shape a lot of that code along with Bennett, Dallas and Wiktor.

If you’re passionate about massively scalable services like this, we’re hiring.