Why twitter needs to be so fast???!!!

Ok, first I must confess, I am not a active twitter user, but I am a regular follower about their System Architecture which always fascinates me. Also they have done some great contribution to the OSS world with products like :

Bootstrap , CloudHopper , Twitter-Server etc.

https://github.com/twitter

Its amazing to see what started as a ruby on rails application turns into this Mega-Structure and does things at a scale that I wouldn’t even believe is possible.
I was watching this excellent presentation in infoQ:

http://www.infoq.com/presentations/real-time-twitter

And I’v been wondering, why Twitter needs to be so fast! I have always had a interest in HighFrequency-LowLatency applications and I understand their importance in Telecom / HFT banking market, but does twitter really need it? Working in broadcast industry, I realised when we say a LIVE Game, its actually a 7 sec delayed FEED, and I don’t hear anyone complaining about it. So would it make a big difference if I got my tweets updates 30 sec later rather than sub-second? I can only imagine how much extra energy / development / money had to be spent to gain sub-second or millisecond level latency.

So the question is, was it actually a business requirement or they did it just because they can ? Or the sub-second latency came as a bonus of their superbly scalable design?

Some taste of functional programming with groovy – 2

Input file:

Ver Duration	House Number	Ver Description
00:15.00	BP0211T0	FS: NRL: BP: NRL Monday Night With Matty Johns TON 2100 (18/3)
00:30.00	CP32336W	CI-April Highlights
01:00.00	CP32119W	LIF-Property Month 60
00:30.00	CP32134W	EV-RioVAlv SunMar31
00:15.00	CP32139W	OD-Argo Sting15
.......... line break.......................................
01:00.00	CP32144W	OD-Argo 60
00:30.00	CP32145W	OD-Argo
01:00.00	CP32149W	OD-Taken 2 60
00:30.00	CP32151W	OD-Taken 2 30B
01:00.00	CP32153W	OD-End Of Watch 60
00:30.00	CP32154W	OD-End Of Watch
00:30.00	CP32159W	OD-Premium Rush VODPrem
00:15.00	CP32224W	OD-Taken 2 STING 15
00:15.00	CP32225W	OD-End Of Watch STING 15
......................erp----------------------------------------
01:00.00	CP32257W	COM-Good God 27/3 60
01:00.00	CP32263W	SHC-GameThrones 1/4 60
00:30.00	CP32269W	A&E-March Highlights
00:15.00	CP32270W	A&E-Toy Hunters 1/4 15
00:45.00	CP32273W	BI-Gene Simmons 27/3
02:00.00	CP32275W	FX-DaVinciDemon 16/4 2min
01:00.00	CP32276W	FX-DaVinciDemon 16/4 60
00:30.00	CP32277W	FX-DaVincis Demons 16/4
^^^^^^^^^^XXXXXXXXXXX^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
00:30.00	CP32278W	NGC-DoomsdayPreppr 8/4
00:30.00	CP32280W	NGC-ToyotaMegafactr8/4
00:30.00	CP32284W	CSA-Aus Podiatry Assoc
00:30.00	CP32285W	CSA-Aus Medicare LocalAll
00:30.00	CP32286W	CSA-RedCross QLD FloodAp
00:45.00	CP32288W	SOHO-Wentworth 1/5 45
00:15.00	CP32324W	OD-The Master
00:30.00	CP32333W	OD-Nitro Circus
01:00.00	CP32335W	CI-April Highlights 60
00:30.00	CP32337W	A&E-TOy Hunters 2/4
02:00.00	CP32341W	FX0-DaVincis Demons 16/4 2m
01:00.00	CP32343W	FX-DaVincis Demons 16/4 60
00:30.00	CP32349W	NGC-WarHerosSkies 10/4
00:30.00	CP32358W	EV-DonVRig Sunday 14/4
00:30.00	JA0086G2	"^FS: FTB: FSD: A League ""iPad  App"" GEN 2012/13 30"" "
01:00.00	JA0106G1	FTV: GEN: Fuel TV is You 2012-13 V2
00:30.00	JA0249D0	SPD: DRA: Andra Drag Racing WED 2130 EDT 2013
00:15.00	JA0251D1	SPD: SPR: World Series Sprintcars WED 1930 (16/1)
00:30.00	JA0447G0	FTV: SUR: ASP Surfing World Tour (2013)
00:30.00	JA0474G1	^FS: RUN: GEN: Rugby Wrap 2013
00:15.00	JA0528G1	"^FS: FSD: NRL: Supercoach ""Prize"" Generic 2013 15"" "
00:15.00	JA0530G1	"^FS: FSD: GEN: Tipping Generic 2013 15"""
01:00.00	JA0574G1	"FS: AFL Who Ever You Love Image 2013 60"" V2"
00:30.00	JA0575G1	"FS: AFL Who Ever You Love Image 2013 30"" V2"
00:15.00	JA0576G1	"FS: AFL Who Ever You Love Image V1 2013 15"" V2"
00:15.00	JA0577G1	"FS: AFL Who Ever You Love Image V2 2013 15"" V2"
01:00.00	JA0582G1	"FS: AFL: Launch Image Games 60"" V2"
00:15.00	JA0583G0	FTV: SUR: ASP Surfing World Tour (2013) 15'  
00:30.00	JA0586G1	"FS: AFL: Launch Image Games 30"" V2"
01:00.00	JA0587G1	"FS: AFL: Image 2013 ""Great Moments"" 60"" V2"
00:30.00	JA0588G1	"FS: AFL: Image 2013 ""Great Moments"" 30"" V2"
00:15.00	JA0619G1	"FS: AFL Who Ever You Love Image V3 2013 15"" V2"
00:15.00	JA0620G1	"FS: AFL Who Ever You Love Image V4 2013 15"" V2"
00:15.00	JA0621G2	"FS: AFL Who Ever You Love Image V5 2013 15"" V3"
00:30.00	JA0622T0	"FS: AFL: On The Couch Launch TON 2030 (18/03) 30"""
00:30.00	JA0627G3	"^FS: FSD: NRL: Supercoach ""Ennis"" 2013 30"" "
01:00.00	JA0648G0	FTV: GEN: March Highlights GEN (2013) 60' 
00:30.00	JA0652T0	"FS: AFL: Open Mike ""Launch"" MON 2130 (18/03) 30"""
01:00.00	JA0681G0	"FS: AFL: Fox Footy Image ""TALENT"" 2013 60"""
00:30.00	JA0682G0	"FS: AFL: Fox Footy Image ""TALENT"" 2013 30"""
00:30.00	JA0699G1	"FS: AFL Who Ever You Love V2 Image 2013 30"" V2"
00:30.00	JA0702T0	"SPD: NAS: Nascar Sprint Cup ""Bristol"" Rply TON 1930 (18/3)"
00:30.00	JA0704D0	"FS: AFL: 2012 GF Recall Syd v Haw SUN 2030 (24/03) 30"""
00:15.00	JA0705D0	"FS: AFL: 2012 GF Recall Syd v Haw ""GOODES"" SUN 2030 (24/03) 15"" "
00:15.00	JA0706D0	"FS: AFL: 2012 GF Recall Syd v Haw ""MALCESKI"" SUN 2030 (24/03) 15"""
00:15.00	JA0707G1	"FS: AFL Who Ever You Love Image V6 ROO 2013 15"" V2"
00:24.00	JA0710D0	^SPD: MAG: Mobil 1 The Grid WED 1900 (20/3)
00:30.00	JA0750G2	FS: CRK: GEN: Cricket in 2013
00:05.00	JA0751G0	"^FS: NRL: NRL Mag Show 5 seconders GEN ""Matty Johns"" 2013"
00:05.00	JA0753G0	"^FS: NRL: NRL Mag Show 5 seconders GEN ""NRL 360 2013"
00:05.00	JA0754G0	"^FS: NRL: NRL Mag Show 5 seconders GEN ""Sterlo"" 2013"
00:05.00	JA0755D0	^FS: GOL: The Golf Show TUE 1930 5 seconders 2013
00:05.00	JA0769G0	"^FS: GEN: Digital Breath  ""HAL"" 2013 "
00:05.00	JA0770G0	"^FS: GEN: Digital Breath  ""CRK"" 2013"
00:05.00	JA0771G0	"^FS: GEN: Digital Breath  ""EPL"" 2013"
00:05.00	JA0772G0	"^FS: GEN: Digital Breath  ""EPL2"" 2013"
00:05.00	JA0773G0	"^FS: GEN: Digital Breath  ""SOCIAL"" 2013 "
00:05.00	JA0774G0	"^FS: GEN: Digital Breath  ""NRL"" 2013"
00:05.00	JA0775G0	"^FS: GEN: Digital Breath  ""NRL2"" 2013 "
00:05.00	JA0776G0	"^FS: GEN: Digital Breath  ""RUN"" 2013"
00:05.00	JA0777G0	"^FS: GEN: Digital Breath  ""SRUN"" 2013 "
00:05.00	JA0778G0	"^FS: GEN: Digital Breath  ""WEB"" 2013 "
00:05.00	JA0779G0	"^FS: GEN: Digital Breath  ""WEB"" 2013"
00:45.00	JA0780G0	"^FS: GEN: FoxSports Relaunch ""Talent"" 2013"
00:05.00	JA0781G0	"^FS: GEN: Digital Breath  ""Foxtel Go 1"" 2013"
00:05.00	JA0782G0	"^FS: GEN: Digital Breath  ""Foxtel Go2"" 2013 "
01:00.00	JA0786G0	"FS: AFL: Friday Night Ritual 60"""
00:30.00	JA0787G0	"FS: AFL: Friday Night Ritual 30"""
00:30.00	JA0788G0	"FS: AFL: Friday Night Ritual ""Ricciuto"" 30"""
00:15.00	JA0789G0	"FS: AFL: Friday Night Ritual 15"""
00:20.00	JA0790G0	"FS: AFL: Friday Night Ritual ""Healy"" 20"""
00:30.00	JA0820C0	"FS: FTB: Socceroos v Oman CSN (26/3) 30"""
00:10.00	JA0821C0	"FS: FTB: Socceroos v Oman CSN (26/3) 10"""
00:30.00	JA0844G0	FS: RUN: FSD: GEN: Rugby Union  Match Centre 2013
00:30.00	JA0850D0	FS: FTB: HAL WK 26 SAT 1700 (23/3)
00:15.00	JA0851D0	FS: FTB: HAL WK 26 SUN 1630 (24/3)
00:30.00	JA0855D0	FS: FSN: AFL: AFL Pre-Season Show WED 1730 (20/3)
00:30.00	XP006929	FX Ultimate Fighter S17 - Wed 1630
00:30.00	XP006930	LSF-AskTheButcher26/3REV

Requirements:

  • Group the lines by 3 category from Description. Description starts with FS / starts with ^FS / starts with ^OD
  • Next group by time for each description group
  • Output with Line number
  • My beginner Groovy code , I am sure the experts will write it even better, comment for improvement will be very much appreciated!

    def out = new File('c:/dev/test/output.txt')
    out.delete()
    new File('c:/dev/test/input.txt').readLines()
                            .findAll { it ==~ /^\d{2}:.+/ } //only select lines that start with number:
                            .collect { it.split ( /\t/ )} //convert string into array of token
                            .collect { lineItems ->           //add an extra column to determine grouping criteria                                 
                                lineItems + [{
                                        switch(lineItems[2]) {
                                            case ~/"FS:.*/ : 
                                                "FS:"
                                                break
                                            case ~/^"\^FS:.*/ : 
                                                "^FS"
                                                break
                                            case ~/^OD-.*/ :
                                                "OD-"
                                                break
                                            default: 
                                                "IGNORE" 
                                        }
                                    }()]
                            }
                            .findAll {!it[3].startsWith("IGNORE")} //drop the rows which doesn't meet our criteria
                            .groupBy { it[3]}  //group by the grouping category (output -> map[string : list[array]]
                            .collect { 
                                       it.value.groupBy{it[0]}  //for each group of Sports Code, perform another grouping on value list based on the duration and sort by keys
                                      .values() //get the values of the groupped by time map
                                      .collectMany{it} //flattens all the result into a single list preserving the order
                            }
                           .collectMany { it } //flatten to a single list of array
                           .eachWithIndex { item , count ->  out << count + '\t' + item[0..2].join('\t') + '\n\r'}
                          
                          
                           println out.text
    

    I think for people familiar with functional coding concepts like Map/Fold/Reduce, the comments are unnecessary and self explanatory. Again the idea is to transform the input file into the output by applying the functions on them. List / Map are the two very important data structure in functional language and you can do magic with them ! And of course all these wouldn’t be possible without the function block/closures. I can’t wait for Java 8 support for closures.

    Anyway, here is the output:

    0	00:15.00	CP32139W	OD-Argo Sting15
    
    1	00:15.00	CP32224W	OD-Taken 2 STING 15
    
    2	00:15.00	CP32225W	OD-End Of Watch STING 15
    
    3	00:15.00	CP32324W	OD-The Master
    
    4	01:00.00	CP32144W	OD-Argo 60
    
    5	01:00.00	CP32149W	OD-Taken 2 60
    
    6	01:00.00	CP32153W	OD-End Of Watch 60
    
    7	00:30.00	CP32145W	OD-Argo
    
    8	00:30.00	CP32151W	OD-Taken 2 30B
    
    9	00:30.00	CP32154W	OD-End Of Watch
    
    10	00:30.00	CP32159W	OD-Premium Rush VODPrem
    
    11	00:30.00	CP32333W	OD-Nitro Circus
    
    12	00:30.00	JA0086G2	"^FS: FTB: FSD: A League ""iPad  App"" GEN 2012/13 30"" "
    
    13	00:30.00	JA0627G3	"^FS: FSD: NRL: Supercoach ""Ennis"" 2013 30"" "
    
    14	00:15.00	JA0528G1	"^FS: FSD: NRL: Supercoach ""Prize"" Generic 2013 15"" "
    
    15	00:15.00	JA0530G1	"^FS: FSD: GEN: Tipping Generic 2013 15"""
    
    16	00:05.00	JA0751G0	"^FS: NRL: NRL Mag Show 5 seconders GEN ""Matty Johns"" 2013"
    
    17	00:05.00	JA0753G0	"^FS: NRL: NRL Mag Show 5 seconders GEN ""NRL 360 2013"
    
    18	00:05.00	JA0754G0	"^FS: NRL: NRL Mag Show 5 seconders GEN ""Sterlo"" 2013"
    
    19	00:05.00	JA0769G0	"^FS: GEN: Digital Breath  ""HAL"" 2013 "
    
    20	00:05.00	JA0770G0	"^FS: GEN: Digital Breath  ""CRK"" 2013"
    
    21	00:05.00	JA0771G0	"^FS: GEN: Digital Breath  ""EPL"" 2013"
    
    22	00:05.00	JA0772G0	"^FS: GEN: Digital Breath  ""EPL2"" 2013"
    
    23	00:05.00	JA0773G0	"^FS: GEN: Digital Breath  ""SOCIAL"" 2013 "
    
    24	00:05.00	JA0774G0	"^FS: GEN: Digital Breath  ""NRL"" 2013"
    
    25	00:05.00	JA0775G0	"^FS: GEN: Digital Breath  ""NRL2"" 2013 "
    
    26	00:05.00	JA0776G0	"^FS: GEN: Digital Breath  ""RUN"" 2013"
    
    27	00:05.00	JA0777G0	"^FS: GEN: Digital Breath  ""SRUN"" 2013 "
    
    28	00:05.00	JA0778G0	"^FS: GEN: Digital Breath  ""WEB"" 2013 "
    
    29	00:05.00	JA0779G0	"^FS: GEN: Digital Breath  ""WEB"" 2013"
    
    30	00:05.00	JA0781G0	"^FS: GEN: Digital Breath  ""Foxtel Go 1"" 2013"
    
    31	00:05.00	JA0782G0	"^FS: GEN: Digital Breath  ""Foxtel Go2"" 2013 "
    
    32	00:45.00	JA0780G0	"^FS: GEN: FoxSports Relaunch ""Talent"" 2013"
    
    33	01:00.00	JA0574G1	"FS: AFL Who Ever You Love Image 2013 60"" V2"
    
    34	01:00.00	JA0582G1	"FS: AFL: Launch Image Games 60"" V2"
    
    35	01:00.00	JA0587G1	"FS: AFL: Image 2013 ""Great Moments"" 60"" V2"
    
    36	01:00.00	JA0681G0	"FS: AFL: Fox Footy Image ""TALENT"" 2013 60"""
    
    37	01:00.00	JA0786G0	"FS: AFL: Friday Night Ritual 60"""
    
    38	00:30.00	JA0575G1	"FS: AFL Who Ever You Love Image 2013 30"" V2"
    
    39	00:30.00	JA0586G1	"FS: AFL: Launch Image Games 30"" V2"
    
    40	00:30.00	JA0588G1	"FS: AFL: Image 2013 ""Great Moments"" 30"" V2"
    
    41	00:30.00	JA0622T0	"FS: AFL: On The Couch Launch TON 2030 (18/03) 30"""
    
    42	00:30.00	JA0652T0	"FS: AFL: Open Mike ""Launch"" MON 2130 (18/03) 30"""
    
    43	00:30.00	JA0682G0	"FS: AFL: Fox Footy Image ""TALENT"" 2013 30"""
    
    44	00:30.00	JA0699G1	"FS: AFL Who Ever You Love V2 Image 2013 30"" V2"
    
    45	00:30.00	JA0704D0	"FS: AFL: 2012 GF Recall Syd v Haw SUN 2030 (24/03) 30"""
    
    46	00:30.00	JA0787G0	"FS: AFL: Friday Night Ritual 30"""
    
    47	00:30.00	JA0788G0	"FS: AFL: Friday Night Ritual ""Ricciuto"" 30"""
    
    48	00:30.00	JA0820C0	"FS: FTB: Socceroos v Oman CSN (26/3) 30"""
    
    49	00:15.00	JA0576G1	"FS: AFL Who Ever You Love Image V1 2013 15"" V2"
    
    50	00:15.00	JA0577G1	"FS: AFL Who Ever You Love Image V2 2013 15"" V2"
    
    51	00:15.00	JA0619G1	"FS: AFL Who Ever You Love Image V3 2013 15"" V2"
    
    52	00:15.00	JA0620G1	"FS: AFL Who Ever You Love Image V4 2013 15"" V2"
    
    53	00:15.00	JA0621G2	"FS: AFL Who Ever You Love Image V5 2013 15"" V3"
    
    54	00:15.00	JA0705D0	"FS: AFL: 2012 GF Recall Syd v Haw ""GOODES"" SUN 2030 (24/03) 15"" "
    
    55	00:15.00	JA0706D0	"FS: AFL: 2012 GF Recall Syd v Haw ""MALCESKI"" SUN 2030 (24/03) 15"""
    
    56	00:15.00	JA0707G1	"FS: AFL Who Ever You Love Image V6 ROO 2013 15"" V2"
    
    57	00:15.00	JA0789G0	"FS: AFL: Friday Night Ritual 15"""
    
    58	00:20.00	JA0790G0	"FS: AFL: Friday Night Ritual ""Healy"" 20"""
    
    59	00:10.00	JA0821C0	"FS: FTB: Socceroos v Oman CSN (26/3) 10"""
    

    Groovy is the new Perl for me, I am not writing any linux scripts with Perl ever again.

    Pragmatic Thinking & Learning

    I just started reading the book “Pragmatic Thinking & Learning” published from the Pragmatic Programmers. Although I only read only the first few chapters so far, I am so excited about it that I thought I should write a line or two about it. If you are in Software development and want to learn the dynamics of software development and software development teams, you must read this book. This is by far the most impressive non technical book I have read so far.

    Here is a snippet to give you some test:


    Dreyfus at Work: Herding Racehorses and Racing Sheep :

    In one of the Dreyfus studies, the researchers did exactly that. They took seasoned airline pilots and had them draw up a set of rules for the novices, representing their best practices. They did, and the novices were able to improve their performance based on those rules. But then they made the experts follow their own rules. It degraded their measured performance significantly.

    – How true! Can you have the same set of rules for your top developer who is at “Expert/Proficient” level and the average developer who is @ “Novice/Beginner” level ? How can you explain intuition to the developer who needs a recipe? (If you don’t know what I am talking about, go read the book). This is one of the top challenges I have faced adopting SCRUM coming from a strict hierarchical team management. In SCRUM everyone from Novice to Expert seems to have the same “voice” and sometimes it just holds you back. But hey, the efficiency of a SCRUM team is often measured by the efficiency of the weakest node of the team! So thats fair enough 🙂

    Where was I ???

    Ok, its been long since I had my last blog post. Where was I? Well, I have been busy! Last few months have been the most happening months of my life.

    First of all, I am a proud father of my 1 month old son now. Although I haven’t managed to stop watching cartoons and start acting like a dad yet, its really amazing how you feel!

    Again, the product “Vantage Contact Center” that I have been working for last 1.5 years is in market now!

    The product is built on top of Broadwork’s 3rd party call control api (OCI-P / CCC2) and Broadworks SIP stack.This is the biggest project that I know of built on top of broadworks 3rd party api. We had a product owner who had spent last 10 year with callcenter solutions. So you can say it was a smooth sail. Once again It proves that for any successful software, most important thing is ‘Domain Knowledge’. We emphasized heavily on Domain Driven Design, unit testing , functional automation testing and we didn’t forget that none of these can replace the manual testing. At the end, it all paid off. Except for some minor integration issues at deployment time (which is natural because the solution depends on a number of servers talking to each other, a typical pattern for telephony apps) , we didn’t have a single major bug that is a showstopper! We had to change some labels on customer demand and add some more validation, but thats about it!

    So here I am , feeling all good about the year 2009 and praying it would go the same for year 2010! And hoping I will update my blog regularly! Wish me luck!

    Lessons learned in Software Development (2) – Abstraction

    Long ago during our weekly tech meeting @ Therap, we had this intriguing discussion, “What is the most crucial skill for a software developer?”. It was quite interesting to see how different people at different skill level had different opinion about the topic. I remember my answer was “Algorithm” (Back then I was into solving ACM style problem). Other inputs were “Language”, “Communication”, “Optimization”, “Clean Code” etc. However, we all agreed finally that it was “Design skills” that matters most.

    My view on this topic also changed from time to time as I grew older and played different roles. I tried to master java (with two java certification under my belt) when I thought language was the most important, I memorized all design patterns from GOF books and tried to apply them when I thought design was the most important one.

    Well now a days, I feel like I got the bigger picture. I think it is the power of “Abstraction” that can turn you from a good Software engineer to a Great one. You can find the importance of abstraction everywhere. When we were tought object oriented design at our schools, we first relate the word “Abstraction” as one of the three attributes of Object Oriented Programming. Well, it has much bigger meaning than that. Abstraction is not only part of  detailed design or Object design, Its the most important part when you want to architect big systems.

    When I started my career, I used to get overwhelmed by big problems, big tasks, big projects. I tried to address them all together and made a mess of things. Now a days, I try to apply abstraction in such scenarios. Whenever I get big problems, I try to divide them into smaller pieces. “Divide and conquer”–Thats the trick. If you can abstract away the smaller problems and focus on the big picture, at the end of the day, you are going to achieve the big goal. If you are working in a team, you can put your team members to work with each of the abstractions.

    I am not going to talk about the Abstraction @ code level. There are enough materials on them. But here is the key, If you can abstract away the components/responsibility/code, you will always be safe from big disaster and achieve the bigger goal.

    Lessons learned from Software Development – Part 1

    When I started my career, the first thing I made up my mind about is I have to “Stand on the shoulders of giants”. From the very beginning, I tried to learn from the best coding practices, design patterns, case studies, books etc etc. In fact, there are so much materials around that one has to be overwhelmed! But there are some lessons that I had to learn the hard way. Although they are in the books, I didn’t realize them until I made the mistakes over and over again.

    In this series of blog post, I’ll try to share them with the rest of the world. Note that these are not my philosophy, I just think they are the most important among all other principles.

    Lesson 1 : Pareto’s Law – The 80/20 principle:

    While the 80/20 principle is applicable to a lot of areas, I am only talking about the software modules and codes. If you are involved in a big project, You will see only 20% of the code is performing the core task, the rest 80% is just supporting codes. If you have 10 components in the system, only 2 of them is actually performing the task, the rest are just there to help the 20% (This applies if you group your functionality in a proper way rather than scattering them all around).

    We software developers often overlook this, we often put our focus in the wrong place. One thing you need to think about is the Return of Investment. If you claim to be an engineer, one thing that an engineer does is Optimization and Trade Off. Say you are developing a plugin framework – the framework itself is much more important than the plugin. If you have limited time and you try to put equal effort on both of them, you will get none of them right. We always work with deadlines, we never get infinite time, so we have to effectively use that time.

    Perfection takes a lot more time than getting something done. You have to make wise decision about which areas you need to perfect and which areas just need to work. Abstraction is the key here. If you have good abstraction, you can always go back and improve your code. I was watching this presentation by Eric Evans about “Strategic Domain Driven Design” for large project where he talked about the same thing. I was quite surprised how our view seemed kind of same! Well, Great men think alike 😉

    Anyways, the point I am trying raise is that , we developers become too much obsessed with our code and technology and forget to think about the Deadline, estimation and all these stuff. If you don’t think about ROI (Return of Investment), your project is going to suffer from it. This will even make more sense when you think about Agile development, Changes is requirement and “Throw-away System” from The Mythical Man-Month(Another lesson I’ll talk about).

    In the next post, I will talk about the Lesson 2: Abstraction

    Every Software Developer must have an Ultra portable Laptop, Period.

    I am obsessed with computers. I spend at least 12-14 hours a day in front of a screen, that includes even the weekends. With all the blog posts stored in my google reader, with all the e-books piled in my reading list, with all the news letters marked unread in my gmail, its really hard to stay away. But many a times I couldn’t be online just because I was too lazy/too tired to sit in front of my desktop or my laptop was too heavy to bring into the bed. But gone are those days with my Samsung NC10. This must be the coolest piece of gadget I’v had so far. Now I can power on my pc and go online in about 60 seconds and can do that right after I open my eyes in the morning(my nc10 sleeps right beside my pillow). I can even run my eclipse with terracotta clustering and do the experimental codes!!! What else can you ask from a 1.3 kg laptop with < 500$ price!

    Now I realize what I missed all these days. I am hooked with the ultra portable concept and can’t live without one for the rest of my life. If you are someone like me who just can’t keep your hands off a computer, you must get one of these. I am looking forward to get another one with a better processor , May be the Sony VAIO TX series but that is when I can afford them.

    The Productive Programmer

    I’v been reading the book “The Productive Programmer“. Being the lazy programmer myself, I always look for more efficient ways of doing things. There are tricks I’v learned from my experience over the years and still learning new things every now and then. But it looks like this book is the summery of all that! Any developer who wants a better way of doing things must have go through this book. How many times have you come across a niche tool and said “Wow! I only wish I knew about this earlier!”. Do you remember first time you used cygwin on Windows! This book is the compilation of all those tools and tricks!

    UPDATE: How to Hotswap classes in the running jvm

    If you have read my previous blog post about How to Hotswap classes in the running jvm, I am sure you will find Java Rebels even more interesting. It supports full class reloading support at runtime with method refactoring and all the stuffs that hotswap.jar could not do. Trust me, Java development has never been more fun. And if you are into spring development, it even supports spring configuration reloading without restarting the whole spring application context! Could it be better!!! I think its a must have feature for every java developer. I don’t know why sun is not incorporating this in the JVM? Just because java is not a scripting based language like php/ruby , doesn’t mean we can’t have fun like those guys! Speaking of fun, you can have some from this Java Rebel cartoon also

    Only downside is JavaRebel is not free. There is a small fee that you have to pay. But let me tell you its worth it. And if you don’t trust me, you can always use the 30 Day trial version (although it will only take 3 days for you to become addicted to it).

    NOTE: I don’t work for java rebel.I wish they took me in 🙂 it would be fun to develop such cool features that eases the life of so many developers.