Evidence Based Scheduling, Estimation and Scrum

People who know me would be surprised to see me talk about Estimation, Scheduling and all the related stuff. To be frank I consider that quite boring and better be left alone for managers! However, I am so loving the new process called Scrum we are following for last 6 months, I thought I should write a line or two about it.

Long ago I was reading this article by Joel which talked about doing your scheduling/estimation based on the evidence. What evidence you may ask? If I understood it right, the evidence is your team’s efficiency! Being used to have the people who has no Idea about what the “NEW FEATURE” is and doing the estimation, I kind of liked the idea. How many times have you stayed late in the office to finish the “Cool and Easy” feature that your manager Promised to deliver the next release? I always felt the manager, no matter how experienced or skilled he is, should not be solely responsible for the estimation. We are in the business of programming, and programming is art! Its not the construction business, its not the military. For any work of creativity, you need freedom and comfort. Until we adopted scrum, I was always wondering how the developers can be a part of the whole management thing specially given the lack of interest of the developers in such things.

I am not going to describe what scrum is and what are the common benefits of following scrum, you will find plenty of reference in the net. I’ll just point out what are the things that I think scrum is so effective from estimation point of view and lucrative from a developer’s perspective, So here is my two cents:

1. The Team does the estimation! Your team is composed of people with diverse skill sets. When you throw a new story to a team, at least 2 of the team members would have some kind of idea about how to do it. For some other story, may be some other team members will put their input. So the estimation will be as good as it gets. And after two or three sprints with the similar kind of stories, the estimation should be perfect!

2. The Developers make the Promises! If you are some investor in any software business, and want to have some feature done within a specific time frame, here is a piece of advice from a man in the tranches, ask your developers, not the manager! If a developer commit he will get something done within a specific deadline, he will. We developers are so dumbly proud that we will even sacrifice our personal life to meet the deadline that we promised. On the other hand, if you ask your manager, he will just keep pushing the developers because he can’t do the things himself even he wanted to. That way, you will end up with a buggy and messy product which is the outcome of all the hatred and frustration that your developers produced during the overtime that they did. Scrum does not have a project manager! Scrum has a Scrum master whose responsibility does not include estimation! It is the team who did the promise during your sprint planning that they will accomplish the features. Any self respecting and efficient team will thrive to achieve that. You will be surprised to see the team requesting you to keep the workspace open during the weekend so they can polish the stories for the reviews! Self-management– this is what scrum is all about. In a manager led project, not all the team members share the same responsibility as the manager. Since the manager is solely responsible for all the planning and estimation, other team member doesn’t event feel responsible for the project because they are not at the gun point! This is specially true when the manager tries to accomplish some impossible deadline.

3. The Estimation is based on Evidence! Right after 2 months of following scrum, we kind of knew our team capacity is 80 points per 15 day sprint. After doing 10 sprints, we can now almost 90% accurately tell that if we plan 100 points, we are going to do overtime and we plan 60 points, we will be relaxing! How do we calculate the 80 point? The stories or the tasks that we estimate are constantly compared with the stories or tasks that we did in the previous sprints. We will say, it took us 2 days to do that “user registration form” with all the unit tests and automated tests which was a 5 point story. So we think the “user account detail collection” is a 8 point story which will take 3 days. Simple yet so efficient! One thing about estimation we don’t give much importance is our hunch or intuition! But we develop our ability to “guess” over so many trials and errors! Our subconscious mind is constantly collecting and processing information and our “guess” is the outcome of that. When 6-7 people are guessing something, its the fruit of years of experience that should not be overlooked. It is as close an estimation as you could ever get.

4. Scrum is sustainable! Scrum is by the developers , for the developers :). Forgive my cheap joke, but couldn’t help it ! We are making the estimation, we are making the promises, we decide whats best for the product, we choose the technology, we take the pride! We don’t have a manager to micromanage us, instead we have a scrum master to lead us. We work for ourselves, we work for the team, we work for the product. The satisfaction that this brings us can’t be compared to anything else, no amount of money or facilities can bring this. We don’t feel sick every week, we don’t think we are underpaid, we don’t feel the constant anger, we don’t think about skipping the office each morning, in a word, we don’t burn out! We are happy developers who have fun in the workplace. Believe me, if you want a good product, you don’t need fancy office or fancy furniture or worlds best salary package or even worlds best programmers. You need happy and motivated developers who are smart enough to get the job done. I believe scrum provides the best way to get there.

Having said all these, I must warn that scrum is not for everyone. Scrum is all about self-managed team. That requires highly skilled and seasoned developers who are matured enough to self manage. Scrum requires honesty which is usually not a problem with programmers. Most of all, scrum requires teaming. If your developers can’t jell together, you will end up with highly inefficient team. One of the main reason I feel we could adopt scrum so easily is because each and every developer of the team is quite experienced and has a minimum level of competency. I wonder how we would adopt scrum if we were a bunch of amateurs.

Asterisk as Sip User Agent emulator

Most common use case of Asterisk is that of a Sip Server. User Agents register to it and can call each other. But have anyone though of using it as a sip user agent emulator? This is exactly what we are doing ! Our main product is written in java which needs to register to a Sip Service Provider, receive calls, play automated answers/forward calls and all kind of play around with the Call. We looked into various open source sip stack written in C / java. We kind of overlooked Asterisk as its known as a sip server! We were just exploring asterisk sip stack to see if we can somehow reuse it, but we were surprised to find out we can use Asterisk as it is without any modifications! Asterisk is written as a B2B UA (Back to Back User Agent) model (not so good for sip servers from performance point of view) but more than fine for us! You can register to your sip service provider as a user agent , answer calls, play messages, collect dtmf, make outgoing call and many other cool stuff! Asterisk real-time even takes this in another level where you can do the all these things dynamically! The best point is, you can write your logic in any language (in our case java) with AGI and AMI which is just TCP message based programming ! Wait, if you are using java, life is even better for you! Asterisk-java is a wonderful abstraction written on AGI/AMI so you don’t even need to bother about TCP programming. Hats off to open source community, It would take us a Year to accomplish what we have accomplished in 3 months with asterisk.

The free book “Asterisk: The Future of Telephony” is a great resource but I am surprised to see how little information is there on net about using Asterisk as a Sip User Agent emulator! Are we the only one who is using asterisk this way !!!???

Spring Session Scoped Bean

I have always hated direct usages of “HttpSession”. You know how tricky it becomes when you want to do advanced things like session replications / caching etc. I have previously used home grown session factory abstraction. But just recently I came across Spring “Session Scoped Bean”. So far, we have only used dependency injection of spring beans which had only 2 scopes – singleton & prototype. But now spring supports custom scopes for beans and one of them is “Session”. It means that this bean is created when a new HttpSession is created and preserved as long as the HttpSession is valid. Spring uses AOP to extract the sessionId from the httprequest and manage the lifecycle of the bean. So instead of putting your object directly into the session, you get to put them in an injected POJO based placeholder. This makes it easy to do the testing and getting rid of HttpSession dependency. So when time comes, you can use solutions like “terracotta” to distribute your session without changing your code! Isn’t that wonderful! Another niche Spring trick!

The details of how to setup the configurations can be found here. I followed the exact steps and it worked flawlessly.

“Enso”—where were you all these days!

Surfing the net is like a hobby/passion/addiction for me, And I mostly surf technology / computer related stuff! I work with other software professionals who are geeks. So anything new, useful comes to my attention early. Yet I am very disappointed to know that Its only today I came to know about Enso! I’v been using it for a few hours now but I am sure its going to be my most useful windows utility forever! It might even become a reason why I prefer Windows over Linux as my development environment [ They don’t have the linux version yet 😦 ]. Just look at the demo and try using it. Using Computer will never be the same again!!!!! I pity myself for not using it earlier 😦

The Productive Programmer

I’v been reading the book “The Productive Programmer“. Being the lazy programmer myself, I always look for more efficient ways of doing things. There are tricks I’v learned from my experience over the years and still learning new things every now and then. But it looks like this book is the summery of all that! Any developer who wants a better way of doing things must have go through this book. How many times have you come across a niche tool and said “Wow! I only wish I knew about this earlier!”. Do you remember first time you used cygwin on Windows! This book is the compilation of all those tools and tricks!

Asterisk-java + Spring

Those who don’t know about Asterisk Java, its a wonderful java library to talk to a asterisk server through AGI & AMI. We are using it heavily to control our Asterisk box from our “main application”. Our main application is a full fledged spring application running on tomcat. At first we were running the asterisk java components (AGIServer , AgiScripts etc) as a normal java application but soon we wanted our AgiScripts to talk to our spring beans Or even better, we want our agi scripts to be Spring beans.

The beauty of the spring and asterisk-java library is that they were made for each other! Asterisk java library was written in wonderful object-oriented way with clear separation on dependencies. So it was very easy to define the AGIServer and all its dependencies as Spring Bean. So We can inject the MappingStrategy with our own implementation of “BeanNameAwareAGIMappingStrategy” which implements ApplicationContextAware looks like this

 @Override
 protected AgiScript createAgiScriptInstance(String beanName) {
        Object bean = applicationContext.getBean(beanName)
        if(bean == null) {
           throw new IllegalArgumentException(\"No bean with name: [\" + beanName +\"] found. Make sure that you have all beans defined which are there in your fastagi-mapping.properties\");
        }

        if(!(bean instanceof AgiScript)) {
          throw new IllegalArgumentException(\"spring bean : \" + beanName + \" must implement org.asteriskjava.fastagi.AgiScript interface\");
        }

        return (AgiScript) bean;
}

So now, instead of defining fully qualified classnames in your “fastagi-mapping.properties”, you only give the bean name and our mapping strategy looks it up from the application context. So your plain agi scripts can talk to your dao, services and do all sort of fancy things! Isn’t that wonderful!

UPDATE: How to Hotswap classes in the running jvm

If you have read my previous blog post about How to Hotswap classes in the running jvm, I am sure you will find Java Rebels even more interesting. It supports full class reloading support at runtime with method refactoring and all the stuffs that hotswap.jar could not do. Trust me, Java development has never been more fun. And if you are into spring development, it even supports spring configuration reloading without restarting the whole spring application context! Could it be better!!! I think its a must have feature for every java developer. I don’t know why sun is not incorporating this in the JVM? Just because java is not a scripting based language like php/ruby , doesn’t mean we can’t have fun like those guys! Speaking of fun, you can have some from this Java Rebel cartoon also

Only downside is JavaRebel is not free. There is a small fee that you have to pay. But let me tell you its worth it. And if you don’t trust me, you can always use the 30 Day trial version (although it will only take 3 days for you to become addicted to it).

NOTE: I don’t work for java rebel.I wish they took me in 🙂 it would be fun to develop such cool features that eases the life of so many developers.

Which dynamic language to learn

Learning a dynamic language has long been on my to-do list. I have tried a little bit of everything , ruby, python even lisp. But I never could proceed much. After playing around with them for a while, I loose interest and find a new framework/tool in java or j2ee more interesting to learn and move on. Well, this time I am very serious and determined and I think it is high time to learn one as even Sun embraced the dynamic language pool into JVM.

Now comes the BIG question, which one to learn??? I know LISP should be my #1 choice according to Paul Graham. It is his book “Hackers And Painters” that convinced me to learn a dynamic language in the first place. But I just can’t bear the braces of lisp, I tried, but its too much for me to digest. Anyway, I guess its quite challenging for anyone to choose a dynamic language to learn specially when he has no idea what he is going to do with it(I don’t see myself moving away from java/j2ee in near future). After a lot of googling/reading blogs/taking suggestions, ruby & python are on the top list. But I have something else in my mind: JavaScript!

I don’t understand why JavaScript is only considered for client side scripting! I can see all the features a dynamic language can offer in JavaScript. May be I’m too naive to tell right now but I am surprised why JavaScript is not mentioned more often in the Dynamic Language Shootouts.

Its not like I haven’t used javascript before to write client side scripts but I have never looked at it as a language seriously. I only used it as a DOM manipulation tool. But the more I’m digging it, the more I am surprised to see its power. And probably, its one of the most used language ever.

So anyone got any advice for me? Any feedback to make me think otherwise? Should I switch to Python/Ruby?

How to mount windows shared network drive as linux directory

My development environment in office is CentOS and quite often I have to access a windows network share directory. I hate to open nautilus samba file system access every time to copy/paste into those directory. I can’t use my bash console to do that. After some googling, I mounted the windows network share folder to my /home/sajid/team_members directory with this command:

mount -t smbfs -o username=sajid,password=sajid  //fileserver/team_members /home/sajid/team_members

Here the username/password is my userid password to access the shared network drive, “//fileserver/team_members” is the location of the network directory.

Now that I could mount the network folder to my local drive, I wanted to automate this task every time I started my computer. So, I added this line to my /etc/fstab file.

//fileserver/team_members /home/sajid/team_members   smbfs username=sajid,password=sajid 0 0

So from now on, I can use the network drive just as my local filesystem. Now that I have it fixed, I have to be extra careful not to do rm -rf /home/sajid/team_members/.. 😉