Babar K. Zafar

Think Different

Thu, 06 Oct 2011 01:01:00 GMT

Here's to the crazy ones.

The misfits. The rebels. The troublemakers.

The round pegs in the square holes.

The ones who see things differently.

They're not fond of rules.

And they have no respect for the status quo.

You can quote them, disagree with them, glorify or vilify them.

About the only thing you can't do is ignore them.

Because they change things.

They push the human race forward.

And while some may see them as the crazy ones, we see genius.

Because the people who are crazy enough to think they can change the world, are the ones who do.

Product Development

Tue, 20 Sep 2011 01:01:00 GMT

"There are two ways to build a product. The first: a company starts with their strengths and builds to the needs of the consumer. The second: a company starts with the needs of the consumer and builds (into) the strengths of the company." -- Jeff Bezos

We did neither and so we failed: http://curictus.se

R.I.P Curictus AB.

Share Dropbox

Sat, 10 Sep 2011 01:01:00 GMT

So, I finally got sick and tired of manually typing in paths to files in shared Dropbox folders. Even worse is when you manually have to click your way trough to a deep hierarchy to find a specific document.

Why can't you securely share a link with people who have access to the shared folder?

Well, now you can.

Visit http://www.sharedropbox.com for more information and to download Windows/Mac extensions for Dropbox which provide this functionality.

The project is open-source and feedback is always welcome.

BTW, please let people at Dropbox know that this a highly useful feature, especially for people using their Dropbox for Teams product.

EC2 vs S3 for storing compressed data

Thu, 25 Aug 2011 01:01:00 GMT

Problem: Compress logfiles on EC2 and backup them to S3.

Should you trade CPU time (high compression rate) for S3 storage costs?

# 720 GB stored for 6 months
#
# cmdline      cpu (s/MB)  size (%)    time (h)    ec2 ($)     s3 ($)      total ($)
# -----------------------------------------------------------------------------------
# raw          0.000013    100.000000  0.002560    0.000973    604.800000  604.800973
# gzip -1      0.015814    14.600000   3.238656    1.230689    88.300800   89.531489
# gzip -5      0.023426    12.100000   4.797696    1.823124    73.180800   75.003924
# gzip -9      0.032528    11.300000   6.661632    2.531420    68.342400   70.873820
# bzip2 -1     0.191470    11.200000   39.213056   14.900961   67.737600   82.638561
# bzip2 -5     0.246373    8.700000    50.457088   19.173693   52.617600   71.791293
# bzip2 -9     0.303029    7.900000    62.060288   23.582909   47.779200   71.362109
# pbzip2 -1    0.105028    11.200000   21.509632   8.173660    67.737600   75.911260
# pbzip2 -5    0.113369    8.900000    23.217920   8.822810    53.827200   62.650010
# pbzip2 -9    0.141710    7.900000    29.022208   11.028439   47.779200   58.807639
# lzma -1      0.060559    11.100000   12.402432   4.712924    67.132800   71.845724
# lzma -5      0.759580    5.600000    155.561984  59.113554   33.868800   92.982354
# lzma -9      1.692910    3.500000    346.707968  131.749028  21.168000   152.917028
# lzop -1      0.002903    20.900000   0.594432    0.225884    126.403200  126.629084
# lzop -5      0.003011    20.700000   0.616704    0.234348    125.193600  125.427948
# lzop -9      0.086239    14.400000   17.661696   6.711444    87.091200   93.802644

Suprisingly, gzip -9 is almost as cost effective as bzip -9 with the difference being that the same operation takes 10x as long.

The table above was calculated for a standard large EC2 intance (0.38$/h) storing S3 using the first non-reduced tier (0.14$/GB/month).

# ----
# S3
# ----
#                      Normal          Reduced
# First  1 TB / month  $0.140 per GB   $0.093 per GB
# Next  49 TB / month  $0.125 per GB   $0.083 per GB
# Next 450 TB / month  $0.110 per GB   $0.073 per GB
# Next 500 TB / month  $0.095 per GB   $0.063 per GN
#
# ----
# EC2
# ----
#
# Large                 $0.38 per hour
# Extra Large           $0.76 per hour
#
# Hi-Memory On-Demand Instances
#
# Extra Large           $0.57 per hour
# Double Extra Large    $1.14 per hour
# Quadruple Extra Large $2.28 per hour
#
# Hi-CPU On-Demand Instances
#
# Medium                $0.19 per hour
# Extra Large           $0.76 per hour

The code below was uses the calculate the results, pretty interesting to plug in values to simulate various scenarios (thank @theo for the initial measurements):


MEASUREMENT_SAMPLE_SIZE_MB = 800
MEASUREMENTS = [
    # setup        cpu s     ratio %
    # ------------------------------
    ("raw"   ,     0.01,     100.0),
    ("gzip -1"   , 12.651,   14.6),
    ("gzip -5"   , 18.741,   12.1),
    ("gzip -9"   , 26.022,   11.3),
    ("bzip2 -1"  , 153.176,  11.2),
    ("bzip2 -5"  , 197.098,  8.7),
    ("bzip2 -9"  , 242.423,  7.9),
    ("pbzip2 -1" , 84.022,   11.2),
    ("pbzip2 -5" , 90.695,   8.9),
    ("pbzip2 -9" , 113.368,  7.9),
    ("lzma -1"   , 48.447,   11.1),
    ("lzma -5"   , 607.664,  5.6),
    ("lzma -9"   , 1354.328, 3.5),
    ("lzop -1"   , 2.322,    20.9),
    ("lzop -5"   , 2.409,    20.7),
    ("lzop -9"   , 68.991,   14.4)
]

EC2_COST_SEC = 0.38  / 3600.0

S3_MB_COST_PER_MONTH = 0.140 / 1024.0

DATA_SIZE_MB = 30*24*1024 # 1 month of data @ 1 GB/h

STORED_FOR_MONTHS = 6

print "%d GB stored for %d months" % (DATA_SIZE_MB/1024, STORED_FOR_MONTHS)
print "cmdline".ljust(12), "".join([
        h.ljust(12) for h in ["cpu (s/MB)",
                              "size (%)",
                              "time (h)",
                              "ec2 ($)",
                              "s3 ($)",
                              "total ($)"]])

for (cmdline, cpu_time, compress_rate) in MEASUREMENTS:
    cpu_time = cpu_time / float(MEASUREMENT_SAMPLE_SIZE_MB)
    compress_time = (DATA_SIZE_MB * cpu_time)
    ec2_cost = EC2_COST_SEC * compress_time
    s3_cost = (DATA_SIZE_MB * compress_rate / 100.0) * S3_MB_COST_PER_MONTH * STORED_FOR_MONTHS
    print cmdline.ljust(12), "".join([
            ("%.6f" % v).ljust(12)
            for v in [cpu_time,
                      compress_rate,
                      compress_time / 3600.0,
                      ec2_cost,
                      s3_cost,
                      ec2_cost+s3_cost]])

Git Notes

Mon, 22 Aug 2011 01:01:00 GMT

Don’t be another one of those lame programmers who’ll do anything but read the documentation. You don’t need books, tutorials , screencasts or conference tickets.

Here are the magic steps required to learn Git:

Install Git.
Read the online Git manual and/or the man pages.
Experiment.

Let me digest it some more: read the manual.

That’s it. You don’t need anything else.

Start with the included tutorial, move on the everyday usage guide, peek under the hood and finally use the manual on a day to day basis.

Avoid the urge to have opinions and be productive instead.

Social Jukebox Experiment

Sun, 21 Aug 2011 01:01:00 GMT

Note: work in progress (offline)

Try it out: link

Download: https://github.com/bkz/social-jukebox

A shared musical experience, everybody is the DJ, adding and skipping songs on the go.

The jukebox will resort to playing classic Prince hits should it ever runs out of songs.
The Jukebox automatically pauses playback when there's no one listening.
Music is streamed as 192 kbit/s MP3, quality should be fine.

Technical

The backend currrently compresses the audio stream as MP3 which limits browser support to Chrome/IE9/Safari. Firefox only supports Ogg/Vorbis which isn't suitable for our needs (requires us to transmit metadata frames manually since the format isn't designed for streaming).

Ogg/Vorbis is also not supported by Flash which in practice is the best audio streaming technology available across all browsers and platforms.

HTML5 audio support is completely broken for our needs, don't even bother. For the the adventurous amongst you who want to give it a go anyway, the MP3 stream is only a right click away :)

Bloom Filter Notes

Mon, 08 Aug 2011 01:01:00 GMT

For the basics read up on the Wikipedia article and the Scalable Bloom Filters paper. Make sure you understand the false positive rate p and the relationship to m (bits) and k (hashes/key).

Hints:

The capacity n is reached when around 50% of bits are set.
Hashing k can be optimized to near constant time - it's better to try to minimize m/n - especially if you are doing network traffic or dealing with huge datasets. Don't blindly set a expected p, you might be able to shave of a bit or so per key by simply relaxing p by a small amount.
If using a third-party bitvector implementation as storage for your own custom solution, verify that it actually clears out the bits on allocation.
Use separate slices of the bitvector (i.e. m is divied into k elements) to get better distribution and predictable "linear scan" lookup patterns.
Build a growable bloom filter by combining Q filters, start out with Q1 and add keys until to you reach full capacity, add Q2 and use it until you need to add Q3 and so on. To check for a key you simply lookup the key in Q1..N.

Common calculations:

def optimal_bloomfilter(n, p):
    """
    Calculate optimal m (vector bits) and k (hash count) for a bloom filter
    with capacity `n` and an expected positive failure rate of `p`. Returns a
    tuple (m, k).
    """
    n, p = float(n), float(p)
    m = -1 * (n * log(p)) / pow(log(2), 2)
    k = (m / n) * log(2)
    return int(ceil(m)), int(ceil(k))


def bloomfilter_capacity(m, p):
    """
    Calculate max capacity n for bloom filter with size `m` bits and expected
    `p` false positive rate.
    """
    n = m * (pow(log(2), 2) / abs(log(p)))
    k = log(1/p, 2)
    return (int(round(n)), int(ceil(k)))

The relationship between p and m/n scale linearly as n goes up so you can use the following table as quick reference guide:

|--------------|--------------|--------------|--------------|--------------|
|   capacity   |    max p     |   mem (kB)   |   bits/key   |   hash/key   |
|--------------|--------------|--------------|--------------|--------------|
|        1000  |         0.1  |        0.59  |        4.79  |           4  |
|        1000  |        0.05  |        0.76  |        6.24  |           5  |
|        1000  |        0.01  |        1.17  |        9.59  |           7  |
|        1000  |       0.005  |        1.35  |       11.03  |           8  |
|        1000  |       0.001  |        1.76  |       14.38  |          10  |
|        1000  |      0.0005  |        1.93  |       15.82  |          11  |
|        1000  |      0.0001  |        2.34  |       19.17  |          14  |
|--------------|--------------|--------------|--------------|--------------|

Here is another table which optimizes (i.e. minimizes) m/n (bits/key) and k hashes since we have to deal with even bits in the real world:

|--------------|--------------|--------------|--------------|
|    max p     |  expected p  |   bits/key   |   hash/key   |
|--------------|--------------|--------------|--------------|
|       0.100  |      0.0918  |           5  |           3  |
|       0.050  |      0.0423  |           7  |           3  |
|       0.010  |      0.0094  |          10  |           5  |
|       0.005  |      0.0046  |          12  |           5  |
|       0.001  |      0.0009  |          15  |           8  |
|       0.001  |      0.0005  |          16  |          10  |
|       0.000  |      0.0001  |          20  |          10  |
|--------------|--------------|--------------|--------------|

There are a lot of implementations floating around on the net, some perfectly fine but most seem to break down on two things:

Hash algorithm h.
Not verifying the false positive rate p.

Dealing with hashing is pretty straight-forward, don't bother with cryptographic hashing (SHA/MD5) or CRC32 - simply find a Murmur hash implementation for your platform/language combination and you're set. Next step is to read the Less Hashing, Same Performance: Building a Better Bloom Filter paper and understand how you can turn the hashing k operation into an almost constant time operation:

gi(x) = h1(x) + i * h2(x) mod m

h1 = murmur_hash(key, 0)
h2 = murmur_hash(key, h1)
for n in 1..k:
    g(n) = (h1 + i * h2) % bits_per_slice

The second issue, verifying that the implementation you're intending to use actually holds up to the expected p is extremely important. If you're using a library which doesn't contain a test suite for checking the false positive rate you absolutely should to write something like this:

Retrieve or calculate the filter capacity n.
Fill upp the filter with n keys.
Now run a large series of lookups against the filter with random keys. Make sure the keys are true false positives and keep track of the errors so that you can calculate the fp/n.

We expect fp to hover around the expected p value since we're dealing with a probabilistic data-structure but large deviations need to be further investigated. Also look out for implementations where the fp is very low as this a good sign that the filter is not correctly tuned, m is probably way higher than it needs to be.

I've a written a small Python package to document most of my findings and to serve as a future reference for doing bloom filter calculations. Let me know if you find any bugs or if its helpful.

Source: https://github.com/bkz/bloom

Resources:

Almeida, P. S., Baquero, C., Preguiça, N., and Hutchison, D. 2007. Scalable Bloom Filters. Inf. Process. Lett. 101, 6 (Mar. 2007), 255-261. DOI= http://dx.doi.org/10.1016/j.ipl.2006.10.007 link

Kirsh, A., and Mitzenmacher, M. 2006. Less Hashing, Same Performance: Building a Better Bloom Filter. In Proceedings of the 14th Annual European Symposium on Algorithms (ESA), pp. 456-467. link

Hybrid Web Applications

Mon, 25 Jul 2011 01:01:00 GMT

I'm really hoping HTML5 makes it big on the client side. Working with CSS+DOM+JS is an incredibly broken and frustrating experience but it's still the best cross-platform solution we've had in computing. Even mobile devices are catching up nicely, my Samsung Galaxy S II offers a browsing experience almost equal to the one you get on a laptop or a workstation. Native applications will always be king but for most applications needs the lower implementation and support costs for a set of web applications adapted to work on a wide range of platforms are infinitely more attractive.

One interesting approach which I've experimented with using Flash/Flex is a hybrid approach where the application is split into three parts:

Front-end client UI (HTML+JS or Flash/Flex).
Localhost HTTP backend for client-side processing and access to local resources.
Web HTTP backend for credentials, DB, sharing data, social components, etc.

If you write (2) in a cross-platform language (Python, C++, etc) which can be packaged into a single executable which runs a tiny HTTP server in the background your applications is almost guaranteed to work on Linux, OSX, Windows and mobile devices while exposing platform specific API:s to the client. The (2) components also allows you to design your application to work offline - connecting to (3) as needed and when possible allows for a smooth experience even on mobile devices.

The problem so far has been tools, Firefox and FireBug has been a nice combo but the lack of deep integration with the browser does sometimes offer a shallow experience. Luckily the Chrome team has acknowledged this and offer their own variation of developer tools which are deeply integrated with both the V8 JS VM and the network stack of the browser.

Great introducton to Chrome Dev Tools from Google I/0 2011:

Tech Bubble

Wed, 20 Jul 2011 01:01:00 GMT

Notes from Economist debate:

Definition of a tech bubble: rapid valuation inflation which exceeds company fundamental values (profit expectations aren't in sync the financial reality of the security) followed by a crash in value.

Four phases of bubbles: Stealth, awareness, mania, blow-off.

Stealth: New industry segments are invested in which generate real profit and value. Early investors who "understand" how the new markets could result in "off the cart" revenues and profits increase their investments. These people help build the high-profile companies of the bubble but always cash out before the world enters the next phase since they understand the game being played and move on to find the next thing instead.
Awareness: Later investors and different kinds of middle men (brokers, salespeople) start to notice and bring investment money resulting in higher prices. When valuations start going to the roof (i.e. Facebook going from $10 billion to $50 billion in the same year) the mantra "this time - everything is different" is inevitably preached. Everything in this phase is about keeping the bubble alive and making it bigger.
Mania: The public jumps in with their money for the "investment opportunity of a lifetime". More money leads to even higher prices pushing IPO valuations to insane levels. The goal is to maximize the bubble using investments from people who aren't domain experts by showing them how early investors in this phase are making extraordinary amounts of money as the bubble grows. The process repeats until the bubble pops and the result is basically a wealth redistribution, transferring value from "marks" to "smart money" and "promotors" (see the chart above).

Warren Buffet on the housing bubble:

The only way you get a bubble is when a very high percentage of the population buys into some originally sound premise...that becomes distorted as time passes and people forget the original sound premise and start focusing solely on the price action. People overwhelmingly came to believe that house prices could not fall significantly. And since [property] was the biggest asset class in the country and it was the easiest class to borrow against it created, you know, probably the biggest bubble in our history.

Spotify Notes

Sat, 16 Apr 2011 01:01:00 GMT

NOTE: If you haven't used Spotify most of this won't make any sense meaning that those of you in the US should probably find ways to play around with the product instead of reading this. Sorry for any factual errors, I prepared this for a different context...

As a startup person myself and fellow Swede I've been following the company closely since their early days. Even though there are a lot of players in the music streaming market I strongly believe Spotify has the best offering, great taste combined with solid technology gives for a very responsive and well-thought out user experience.

Business Model

First of all, if you're even slightly interested in the music industry and how modern music streaming ventures are trying to "solve" the music problem you need to research the story of imeem. The easiest and probably most interesting way is to watch one the founders talk at YC Startup School. So why is imeem interesting in the context of Spotify? Well both of them tried to solve a hard problem only to realize that ad-supported business models don't scale and that the music industry is highly territorial and fragmented. Dealing with people who see their revenues falling something like 8.4% just last year ($1.5 billion globally) is probably not the easiest thing in the world.

So why isn't Spotify dead like imeem? In 2009 it seems that Spotify and others finally understood the power of the iTunes experience. You buy music, play it on your computer and then bring it along on your iPod/iPhone. What most people fail to realize is that the problem with streaming is not renting versus owning. It's that you can't consume music in the same way, the streaming solution locks your music to computers (and require a working internet connection). To make matters worse the music industry seems to insist on a model of $10/month per user which requires a lot of added value if you're going to get someone to pay when they're used to free.

Timing is everything and with Apple allowing native music apps on the iPhone they kind of gave away the keys to iTunes kingdom to companies like Spotify. The entire iPod ecosystem is controlled by Apple and you can't easily put music on them but with iOS a new music platform opened up. Spotify put out their mobile app which synced music over WiFi/3G and allowed for offline playing. And the customers responded. I believe the Spotify mobile app is the single most important reason for driving sales and securing a profitable future. In retrospective it's almost scary how much of being a successful startup depends on timing and sheer luck. Had Spotify not been able to provide a mobile offering we'd instead be analyzing another imeem and how their big bet on ads and a business model where people would pay to skip them didn't scale.

So mobile seems to be key for making the streaming business models work. What's even more fun is that it looks like the mobile carriers are the ones who are going to scale the business model even further. The Swedish version of AT&T is called Telia and since last year most of their phone offerings have had 6 months of Spotify Premium included. The carrier basically subsidies Spotify just like they do with the phones. This setup shovels customers into the Spotify funnel (with zero cost) and gives them plenty of time to let the product and the social aspects of it (Facebook sharing, collaborative playlists) get people hooked. User retention is a great problem to have when you user acquisition cost is zero.

OK, so the model seems to working out. But they're still stuck in Europe. Rumors about launching in the US have been floating around for almost a year now. My bet is that Spotify probably won't launch in the US without a partner since Apple is dominating the online music market and can leverage that against the music labels. Remember when Apple bought Lala? Just to kill it off? Understand that Apple makes money selling hardware. Not music or software, iTunes is high revenue but very low profit-margin. So if you in any way threaten sales of the Apple ecosystem which you indirectly do when you compete with iTunes (since it drives iPod sales) you end up with Apple working against you. You might think that with the iPad/iPhone Apple shouldn't care since they still make money of the hardware. Sure, but then you're forgetting the 20 million iPods they sold in Q1 2011 alone...

(As a side note, a US launch does seem pretty close though, Spotify recently opened offices in New York which unlike other offices outside of Sweden seem to include an engineering staff.)

Technology

Spotify uses a modified (proprietary) version of Ogg Vorbis which support seeking in streams. Music is normally streamed at 160kbs but paying customers can select higher quality streams at 320kbs. Music is streamed to clients using a hybrid P2P model which is tuned to minimize latency, in most cases playing a song is instant. The focus on speed is not merely a gimmick, it is well known as shown by Google and others that speed and low latency change usage behavior. People will use your product more if it's reponsive since they feel more in control.

The Spotify client maintains a local cache which minimizes network traffic. Together with the P2P components Spotify probably has the best distribution model of all the players HTML/Flash solutions are much harder to scale and the costs are drastically higher when you can't cache or use P2P.

One of the hardest technical problems which are overlooked by almost everyone is the playlists component. Since playlists can be shared and collaborated on with others in real-time and need to be synced across different devices you end with up a form of distributed revision control. The real-time ascept make it even harder since you have to use a push-model and maintain persistent connections to all clients.

A cute trick to make it easy to share songs and playlists is to give them unique a URI. Spotify uses both normal HTTP links and a shorter form like spotify:track:2cJz1loJp5EZM6shmQpLZN which are handled by OS protocol handlers (Windows: via HKEYCLASSESROOT, OSX: CFBundleURLTypes).

Spotify provides a binary-only library for Windows, OSX, Linux and ARM which can be used to write your own player. The SDK is a bit quirky but you can make it to work with something like SDL pretty easily, see my hackish Win32 demo. For an interesting code reading session I also recommend despotify which is the result of reverse engineering the Spotify client protocol. Be sure to check out the authentication trick using computational puzzles which makes the Spotify servers DoS-resistant.

Thoughts on software design

Mon, 10 Jan 2011 01:01:00 GMT

As we are in the middle of crunch on our big project at work I thought I'd write down what I (currently) believe is the correct way of designing and implementing software. My views are heavily influenced by Emacs and dynamic languages, although both of the major projects I've worked on so far have been implemented using a statically typed language (C++). As I'm mostly interested in writing software that serves a purpose, all of the professional projects I've worked on so far have been consumer-oriented products.

People

This factor alone will make or break a project. If a majority of the team hasn't worked on a project together things get dangerous. My advice to management in this case would be to let the team free-roam for a month or so and let them work on writing tools and libraries needed for the project. It might look like a folish waste of time but for the team this period is critical. First of all, senior people get the chance to see new hires in action and secondly they can easily spot people who are not a fit for the team for this specific project.

Why is this important? I believe that the state of your software is determinted by the least competent developer on the team. This is the key if you want to run a successful software development project and in general if you want make money in this business. Good programmers are well worth paying for as the value you can extract from their work far exceeds what you have to spend to make them stay happy.

Team dynamics

The team should consist of people with one or more of the following traits:

High-energy, creative
Detail-oriented, guru
Customer and usability-oriented
Gets-anything-done

If you don't have any team members with the high-energy trait, suspend the project and wait until you get the right people instead of wasting time and money. The team should not consist of more than 3 or 4 people, larger projects should be done with multiple teams. Well motivated smart people will do the right thing and manage themselves. Project managers should manage projects, not people.

Software

If you don't have a clear picture of the user, don't bother unless you're doing R&D work.
Unit-test or die.
Analyze and design top-down. Buzzwords like Agile and Scrum don't remove the need for thinking and knowing what you're doing.
Implement bottom-up, short delieverable iterations.
Design composable software and libraries, because you'll end up doing X when Y isn't Z if Q is P.
Make the software configurable at runtime.
Make the internal data structures of the software inspectable at runtime.
Make the system composable (pluggable) at runtime.
Always start by attacking high-risk problems with prototypes. Plan to throw the first version away.
Experiment, observe, adapt...

Lessons learned in 2010: (2), (4), (9)