Tumblr Engineering

Tumblr SDK updates

Alongside yesterday’s Tumblr for iOS 3.3.1 release, I’m happy to also announce a couple of enhancements to the Tumblr iOS SDK (version 1.0.2 is now available in the CocoaPods repository).

Tumblr for iOS now exposes URL schemes that can be invoked in order to create text, quote, link, and chat posts. Additionally, developers can specify callback URLs that will send the user back to their apps once the Tumblr post is either created or cancelled. These endpoints are now wrapped and exposed by the TMTumblrAppClient class, or the URLs can be accessed directly if you’d prefer (they’re documented here).

Lastly, Tumblr for iOS has been able to create photo and video posts using media shared from other apps for some time now, via Apple’s standard UIDocumentInteractionController. As of 3.3.1, third-party applications can now also specify captions and tags to accompany the photos and videos being shared. The annotations used to do so are documented here.

If you’re using the Tumblr iOS SDK in your app or plan to integrate going forward, please get in touch and let me know if there’s anything I can do to make this even easier.

April 25, 2013

Posted by bryan

♥ 29

My Philosophy On Alerting

robewaschuk:

I wrote some stuff while I was at Google about writing clean alerts and keeping an oncall rotation sane; after some cleanup they’ve allowed me to make it public. Of course, this represents my opinions and not Google’s. They do reflect what we think are best practices at Tumblr, though. We’re hiring Site Reliability Engineers.

Check out My Philosophy On Alerting.

Summary

When you are auditing or writing alerting rules, consider these things to keep your oncall rotation happier:

Pages should be urgent, important, actionable, and real.

They should represent either ongoing or imminent problems with your service.

Err on the side of removing noisy alerts.

You should almost always be able to classify the problem into one of
availability & basic functionality

latency

correctness (completeness, freshness and durability of data)

and feature-specific problems.

Symptoms are a better way to capture more problems more comprehensively and robustly with less effort.

The further up your serving stack you go, the more problems you catch in a single rule. Balance this with being able to distinguish what’s going on.

April 25, 2013

Posted by robewaschuk

♥ 25

TMCache: fast object caching for iOS & OS X

A simple write-through cache can be a powerful optimization for any system needing fast, frequent access to the same resource (i.e., almost everything ever). It’s especially true for a limited memory environment like iOS, which is also partially bound by processor and bandwidth constraints. Something as common as downloading and displaying an image touches all these pain points, and that makes it a great place to explore the potential of caching, especially if it’s something your app does as often as the Tumblr mobile apps.

Cocoa itself does a fair bit of caching on its own without intervention, but it has many limitations. In the case of downloading an image using NSURLConnection, the entire server response is stored in a shared NSURLCache, but the inner details are infamously murky and entirely dependent on HTTP headers from the server. After you’ve loaded the image into a UIImageView it’s one memory warning away from being lost, or even sooner if it gets slated for recycling from a table cell scrolling off screen.

Today we’re proud to announce the open sourcing of TMCache, which is designed for this and any other situation where there’s a need to persist “expensive” objects and access them rapidly. TMCache is an object cache for iOS and OS X, suitable for any object conforming to the NSCoding protocol (including the basic Foundation data types, collections, and many UIKit objects like UIImage). It consists of two parallel caches, one in memory and one on disk, both coordinated locklessly by GCD. Like NSCache, TMCache will automatically remove objects from memory when the app receives a warning or goes into the background. Unlike NSCache, TMCache can do things like transparently restore objects from disk, access objects asynchronously using blocks, and automatically limit the size of the cache by age or size.

TMCache is available under the Apache 2.0 license on GitHub or as a CocoaPod, including full documentation in HTML and docset format. Let us know if you find it useful!

April 24, 2013

Posted by jstn

♥ 79

Open Source - Memcache Top

mobocracy:

We rely on memcache pretty heavily at Tumblr, with over 10TB of cache memory available across the stack. One of the things we’ve historically had a challenging time with at Tumblr is finding hot keys. A hot key is a memcache key getting dramatically more activity than other keys. This can have a significant performance impact on your cache backend.

We spent the past few days working on a C++ implementation of mctop*, which we’re happy to release today as memkeys. We do some pretty interesting stuff in memkeys to keep from dropping packets, some of which is documented in the wiki. I’m particularly proud of the striped lock-free queue implementation. In some basic benchmarks I found that memkeys dropped less than 2% of packets when seeing 1Gb/s of traffic. Additionally, the latency between a packet being picked up, parsed, processed, and reported on averages less than 1ms. Here is a screenshot of memkeys in action.

Interested in stuff like this? We’re hiring.

Footnote: Etsy created the excellent mctop tool which aims to be like unix top for memcache, showing you which keys are getting the most activity. Unfortunately (as noted in the known issues), mctop drops packets. It drops a lot of packets. This can be really problematic because depending on the packets being dropped, you’re getting a really incomplete view of your cache story.

April 23, 2013

Posted by mobocracy

♥ 84

Tumblr Engineering @ Percona Live MySQL Conference

We’re pleased to announce that Tumblr’s Database Engineering team will be attending the Percona Live MySQL Conference next week in Santa Clara, CA!

We’ll be giving a talk on our open source automation software, Jetpants, which has helped us scale to over 175 billion distinct rows of relational data to date. We’re also looking forward to attending a number of amazing sessions from our friends at Percona, Facebook, Oracle, Palomino, Etsy, and more.

If you haven’t registered yet, use code SpeakMySQL to save 15%. Hope to see you there!

April 18, 2013

Posted by evan

♥ 20

Last week, two of our engineers, Wolf and JC, got to stay up all night eating pizza with some really talented young hackers at two different hackathons — 2013 hackNY Student Hackathon at Columbia University and Photo Hack Day at Facebook HQ.

After a long day of hacking, pizza, sleep deprivation, more pizza, and some DIY tacos, participants demoed their hacks, including a few for the Tumblr API. We were absolutely blown away by the creativity and quality of the projects and look forward to seeing what you’ll build at the next hackathon!

April 11, 2013

Posted by codingjester

♥ 4270

The spring 2013 hackNY Student Hackathon is this weekend at Columbia University! For 24 hours, students will collaborate on creative coding challenges, and then present their hacks to a panel of judges. Prizes and bragging rights will be awarded to the winners.

One of our engineers, The Wolf, will be demoing the Tumblr API, helping students build awesome stuff, and eating leftover pizza.

Details

When: Saturday, 4/6 at 2pm – Sunday, 4/7 at 2pm

Where: Fu Foundation School for Engineering and Applied Science, Columbia University

How: Sign up here

If all-night hacking isn’t your thing, be sure to check out all the student hacks at DemoFest (tickets are free!) this Sunday from 12-2 pm.

See you soon!

April 3, 2013

Posted by codingjester

♥ 75

Core Data as a cache

Core Data is great but automatic migrations can be tricky. Migrations can take a long time, which could result in your app being terminated if it is happening on the main thread during application launch. Performing migrations on a background thread is also a bad idea, meaning your application really needs to be able to fully launch without a Core Data stack whatsoever in order to safely migrate. This can be a huge change to make to an existing app.

If you’re really only using Core Data as a cache, you don’t actually need to perform a migration. Simply check if the existing store is compatible with your managed object model and if not, delete and recreate it. Here’s my approach for doing so.

Many thanks to Marcus Zarra for being extremely helpful both over Twitter and on Stack Overflow. You should totally buy his Core Data book.

March 25, 2013

Posted by bryan

♥ 24

Over the past few weeks we’ve open-sourced quite a few official client libraries for the Tumblr API. Today I’m proud to announce the Tumblr iOS SDK, an Objective-C library for easily integrating Tumblr data into your iOS (or OS X) applications, however you see it.

The Tumblr SDK for iOS contains a few different components to start:

Authentication (OAuth and xAuth implementations)
A full wrapper around all of our API endpoints
Inter-app communication (this is pretty limited at the moment but we plan to expand it quite a bit going forward)
A UIActivity stub for easy inclusion of a Tumblr button in a standard Apple UIActivityViewController

We’ve been using this SDK in production for quite some time now and are thrilled to finally be able to share it with you.

If you’re interested in integrating with Tumblr on iOS or OS X in a way that the SDK doesn’t currently facilitate, please get in touch. I’m very interested in hearing any and all feedback on how we can make this as easy as possible.

March 22, 2013

Posted by bryan

♥ 106

seejohnrun:

tumblr.php

I’m here to announce another new addition to our list of official API clients. This week we have tumblr.php - it’s available on GitHub and composer via packagist. It is tested, is PSR-2 compatible, and is well documented.

Like the other clients we’ve been announcing (most recently JS and Java) it has full support for all of the Tumblr v2 API endpoints.

Time to make something cool!

March 15, 2013

Posted by codingjester

♥ 124