Rany Keddo: Starling + Workling: simple distributed background jobs
Hinweis: Der Vortrag wird auf deutsch gehalten.
Getting started with background jobs should be really simple. With
Workling, it is. When you’ve written your code, you can configure it
to run over any number of backgrounding systems. I will show how
Workling and Starling can be used to build a backgrounded addressbook
crawler with an ajax progress indicator wich runs the worker code on a
remote machine.
Web applications often need to do work which does not belong in the
request / response cycle. Photo resizing or API calls to other
services block the request, slowing things down uncessarily. Long
running tasks will make your user application unresponsive and your
users impatient, or even destabilize your environment. The solution is
to take these tasks out of the request cycle.
Getting started with backgrounding should be really simple.
Traditionally, backgrounding was done in Rails applications through
use of BackgroundRB. But BackgroundRB has gained a bad reputation over
time, because of Drb issues and its general weightiness. Although
BackgroundRB was recently completely re-written, it is still only one
of many options available at the moment. Unfortunately, it’s not
really clear which option should be used.
In this talk I will survey what’s currently available. I will look at
the queue based systems Starling and BeanstalkD, and also at systems
that achieve a similar effect by using the database and threading or
forking, BackgroundJob and spawn. Finally I will look at Workling,
wich can combine all these approaches under a very simple roof.
The main focus will be on Starling, Twitters inhouse message queue,
released by Blaine Cook and team a few months ago. Starling is a
lightweight persistent message queue which speaks memcache. It is used
at Twitter to distribute work such as sending SMS. I will look at how
to use Starling, as well as differences in the evented version of
Starling.
Next, I will discuss Workling, which seamlessly integrates Starling
into your Rails app. I will show how Workling and Starling can be used
to build a backgrounded addressbook crawler with an ajax progress
indicator wich runs the worker code on a remote machine.
Finally I will demonstrate how the same code can be configured so that
it runs locally with Spawn or also with BackgroundJob.
