Ok, first I must confess, I am not a active twitter user, but I am a regular follower about their System Architecture which always fascinates me. Also they have done some great contribution to the OSS world with products like :
Bootstrap , CloudHopper , Twitter-Server etc.
Its amazing to see what started as a ruby on rails application turns into this Mega-Structure and does things at a scale that I wouldn’t even believe is possible.
I was watching this excellent presentation in infoQ:
http://www.infoq.com/presentations/real-time-twitter
And I’v been wondering, why Twitter needs to be so fast! I have always had a interest in HighFrequency-LowLatency applications and I understand their importance in Telecom / HFT banking market, but does twitter really need it? Working in broadcast industry, I realised when we say a LIVE Game, its actually a 7 sec delayed FEED, and I don’t hear anyone complaining about it. So would it make a big difference if I got my tweets updates 30 sec later rather than sub-second? I can only imagine how much extra energy / development / money had to be spent to gain sub-second or millisecond level latency.
So the question is, was it actually a business requirement or they did it just because they can ? Or the sub-second latency came as a bonus of their superbly scalable design?
Lazy Programmer's Shortcut 