design

Why twitter needs to be so fast???!!!

sajid2045 design, productivity, programming Leave a comment

Ok, first I must confess, I am not a active twitter user, but I am a regular follower about their System Architecture which always fascinates me. Also they have done some great contribution to the OSS world with products like :

Bootstrap , CloudHopper , Twitter-Server etc.

https://github.com/twitter

Its amazing to see what started as a ruby on rails application turns into this Mega-Structure and does things at a scale that I wouldn’t even believe is possible.
I was watching this excellent presentation in infoQ:

http://www.infoq.com/presentations/real-time-twitter

And I’v been wondering, why Twitter needs to be so fast! I have always had a interest in HighFrequency-LowLatency applications and I understand their importance in Telecom / HFT banking market, but does twitter really need it? Working in broadcast industry, I realised when we say a LIVE Game, its actually a 7 sec delayed FEED, and I don’t hear anyone complaining about it. So would it make a big difference if I got my tweets updates 30 sec later rather than sub-second? I can only imagine how much extra energy / development / money had to be spent to gain sub-second or millisecond level latency.

So the question is, was it actually a business requirement or they did it just because they can ? Or the sub-second latency came as a bonus of their superbly scalable design?

Lessons learned in Software Development (2) – Abstraction

sajid2045 design, OOAD, productivity, programming 1 Comment

Long ago during our weekly tech meeting @ Therap, we had this intriguing discussion, “What is the most crucial skill for a software developer?”. It was quite interesting to see how different people at different skill level had different opinion about the topic. I remember my answer was “Algorithm” (Back then I was into solving ACM style problem). Other inputs were “Language”, “Communication”, “Optimization”, “Clean Code” etc. However, we all agreed finally that it was “Design skills” that matters most.

My view on this topic also changed from time to time as I grew older and played different roles. I tried to master java (with two java certification under my belt) when I thought language was the most important, I memorized all design patterns from GOF books and tried to apply them when I thought design was the most important one.

Well now a days, I feel like I got the bigger picture. I think it is the power of “Abstraction” that can turn you from a good Software engineer to a Great one. You can find the importance of abstraction everywhere. When we were tought object oriented design at our schools, we first relate the word “Abstraction” as one of the three attributes of Object Oriented Programming. Well, it has much bigger meaning than that. Abstraction is not only part of  detailed design or Object design, Its the most important part when you want to architect big systems.

When I started my career, I used to get overwhelmed by big problems, big tasks, big projects. I tried to address them all together and made a mess of things. Now a days, I try to apply abstraction in such scenarios. Whenever I get big problems, I try to divide them into smaller pieces. “Divide and conquer”–Thats the trick. If you can abstract away the smaller problems and focus on the big picture, at the end of the day, you are going to achieve the big goal. If you are working in a team, you can put your team members to work with each of the abstractions.

I am not going to talk about the Abstraction @ code level. There are enough materials on them. But here is the key, If you can abstract away the components/responsibility/code, you will always be safe from big disaster and achieve the bigger goal.

Designers design, Coders Code

sajid2045 books, design Leave a comment

Taken from the famous book “Domain Driven Design” —

Manufacturing is a popular metaphor for software development. One inference from this metaphor: highly skilled engineers design; less skilled laborers assemble the products. This metaphor has messed up a lot of projects for one simple reason—software development is all design.

How true! I have heard of organizations where designers design and coders code! I would hate to work with such organization either as a Designer or as a coder.

Few words about “Comments”

sajid2045 books, design, programming, Refactoring Leave a comment

I’v been reading this great book “Clean Code: A Handbook of Agile Software Craftsmanship” By Robert C. Martin. This book is going to be another classic like the “Code Complete” or “Refactoring“. Anyway, I liked and agreed so much with the author about what he wrote about comments that I couldn’t resist sharing a few lines from the book —-

“The proper use of comments is to compensate for our failure to express ourself in code. Note that I used the word failure. I meant it. Comments are always failures. We must have them because we cannot always figure out how to express ourselves without them, but their use is not a cause for celebration.
So when you find yourself in a position where you need to write a comment, think it through and see whether there isn’t some way to turn the tables and express yourself in code. Every time you express yourself in code, you should pat yourself on the back. Every time you write a comment, you should grimace and feel the failure of your ability of expression.”

When we learned programming in our college, we were told by our teachers that you should always write comment! Even when we joined job, our experienced seniors also told us to write comments! It took me a long time to realize that comments should only be used to document some domain knowledge, writing api/library documentation, not to describe code. If you use comments too frequently in every piece of code you write, you should read all three books I mentioned earlier.

Your code should speak for itself. Period.

Lessons learned from Software Development – Part 1

sajid2045 design, productivity, programming Leave a comment

When I started my career, the first thing I made up my mind about is I have to “Stand on the shoulders of giants”. From the very beginning, I tried to learn from the best coding practices, design patterns, case studies, books etc etc. In fact, there are so much materials around that one has to be overwhelmed! But there are some lessons that I had to learn the hard way. Although they are in the books, I didn’t realize them until I made the mistakes over and over again.

In this series of blog post, I’ll try to share them with the rest of the world. Note that these are not my philosophy, I just think they are the most important among all other principles.

Lesson 1 : Pareto’s Law – The 80/20 principle:

While the 80/20 principle is applicable to a lot of areas, I am only talking about the software modules and codes. If you are involved in a big project, You will see only 20% of the code is performing the core task, the rest 80% is just supporting codes. If you have 10 components in the system, only 2 of them is actually performing the task, the rest are just there to help the 20% (This applies if you group your functionality in a proper way rather than scattering them all around).

We software developers often overlook this, we often put our focus in the wrong place. One thing you need to think about is the Return of Investment. If you claim to be an engineer, one thing that an engineer does is Optimization and Trade Off. Say you are developing a plugin framework – the framework itself is much more important than the plugin. If you have limited time and you try to put equal effort on both of them, you will get none of them right. We always work with deadlines, we never get infinite time, so we have to effectively use that time.

Perfection takes a lot more time than getting something done. You have to make wise decision about which areas you need to perfect and which areas just need to work. Abstraction is the key here. If you have good abstraction, you can always go back and improve your code. I was watching this presentation by Eric Evans about “Strategic Domain Driven Design” for large project where he talked about the same thing. I was quite surprised how our view seemed kind of same! Well, Great men think alike 😉

Anyways, the point I am trying raise is that , we developers become too much obsessed with our code and technology and forget to think about the Deadline, estimation and all these stuff. If you don’t think about ROI (Return of Investment), your project is going to suffer from it. This will even make more sense when you think about Agile development, Changes is requirement and “Throw-away System” from The Mythical Man-Month(Another lesson I’ll talk about).

In the next post, I will talk about the Lesson 2: Abstraction