Lessons learned in Software Development (2) – Abstraction

Long ago during our weekly tech meeting @ Therap, we had this intriguing discussion, “What is the most crucial skill for a software developer?”. It was quite interesting to see how different people at different skill level had different opinion about the topic. I remember my answer was “Algorithm” (Back then I was into solving ACM style problem). Other inputs were “Language”, “Communication”, “Optimization”, “Clean Code” etc. However, we all agreed finally that it was “Design skills” that matters most.

My view on this topic also changed from time to time as I grew older and played different roles. I tried to master java (with two java certification under my belt) when I thought language was the most important, I memorized all design patterns from GOF books and tried to apply them when I thought design was the most important one.

Well now a days, I feel like I got the bigger picture. I think it is the power of “Abstraction” that can turn you from a good Software engineer to a Great one. You can find the importance of abstraction everywhere. When we were tought object oriented design at our schools, we first relate the word “Abstraction” as one of the three attributes of Object Oriented Programming. Well, it has much bigger meaning than that. Abstraction is not only part of  detailed design or Object design, Its the most important part when you want to architect big systems.

When I started my career, I used to get overwhelmed by big problems, big tasks, big projects. I tried to address them all together and made a mess of things. Now a days, I try to apply abstraction in such scenarios. Whenever I get big problems, I try to divide them into smaller pieces. “Divide and conquer”–Thats the trick. If you can abstract away the smaller problems and focus on the big picture, at the end of the day, you are going to achieve the big goal. If you are working in a team, you can put your team members to work with each of the abstractions.

I am not going to talk about the Abstraction @ code level. There are enough materials on them. But here is the key, If you can abstract away the components/responsibility/code, you will always be safe from big disaster and achieve the bigger goal.

Automate SIP Flow testing with Sipper

If you are into VOIP development and want to automate your sip call flows for testing, you might be interested in this project SIPr.

I have been involved in a project for a few months now which provides an advanced callcenter service leveraging the Broadworks Platform. Like any callcenter, the customer calls the CallCenter , Callcenter Queues the Call and Selects an agent and delivers the Call to the Agent. The agent is a registered user of Broadworks.

I have been looking for the right tool to automate the SIP Call Flows. After playing a while with sippr, I end up writing these test cases:

The CUSTOMER controller:
It justs sends the INVITE to a broadworks number which is the QUEUE. After the call gets received, it waits and sends BYE to terminate the call.

require 'sip_test_driver_controller'

class CustomerController < SIP::SipTestDriverController

  transaction_usage :use_transactions=>false
  start_on_load false

  def initialize
    logd('Controller created')
  end

  def on_provisional_res(session)
  end

  def start
    session = create_udp_session(SipperConfigurator[:DefaultRIP], SipperConfigurator[:DefaultRP])
    session.request_with('INVITE', 'sip:sajidqueue1@bwas.broadworkslab.com')
    puts 'Sending INVITE Dest: ',  'sip:sajidqueue1@bwas.broadworkslab.com';
 end

 def on_invite(session)
 puts 'customer got re-invite : '
 session.respond_with(200)
 session.schedule_timer_for("send 200!", 15000)
 end

 def on_timer(session , task)
 puts 'sending bye'
 session.request_with('BYE')
 session.invalidate(true)
 end

 def on_success_res(session)
 puts 'customer ---> on_success_res'
 session.request_with('ACK')
 end
end

Here is the Agent Controller:
It registers with Broadworks and waits for calls from the queue and receives it after 10 sec Ring and then waits for the BYE from the customer.

require 'sip_test_driver_controller'

class AgentController < SIP::SipTestDriverController

  transaction_usage :use_transactions=>false

  # change the directive below to true to start after loading.
  start_on_load false

  def specified_transport
    [SipperConfigurator[:LocalSipperIP],SipperConfigurator[:LocalSipperPort][1]]
  end   

  def initialize
    logd('Controller created')
  end

  def on_invite(session)
    
 
    if !session['invite']
      puts 'Agent got call from Queue '
      session.respond_with(100)
      session.respond_with(180)
      session['invite']=1
      session.schedule_timer_for("send 200!", 5000)
    else
      puts 'agent got re-invite'
      session.respond_with(200)
    end
    
  end
  
  def on_timer(session , task) 
    puts 'Agent is going to accept the call from customer!', task
    session.respond_with(200)
  end

  def on_bye(session)
    puts 'Agent received BYE from Customer!'
    session.respond_with(200)
    session.invalidate(true)
    session.flow_completed_for("CustomerAgentTest")
  end

  def on_provisional_res(session)
  end

  def start
    contact = SipperConfigurator[:LocalSipperIP]+":"
    r = Request.create_initial("REGISTER", "sip:" + "bwas.broadworkslab.com'",
         :from=>"sip:2401120075@bwas.dhaka.vantage.com", :to=>"sip:2401120075@bwas.broadworkslab.com'",
         :contact=>"sip:sajidagent1@"+ SipperConfigurator[:LocalSipperIP]+":"+SipperConfigurator[:LocalSipperPort][1].to_s(),
         :expires=>"300",
         :cseq=>"1 REGISTER",
         :p_session_record=>"msg-info")
    r.contact.q="0.9"       
    session = create_udp_session(SipperConfigurator[:DefaultRIP], SipperConfigurator[:DefaultRP])
    #session.create_register_request('sip:bwas.dhaka.vantage.com', '<sip:sajidagent1@bwas.broadworkslab.com'>')
    session.send(r)    
    #print "Sent a new REGISTER from "+name+"\n"
  end
  
  def on_success_res_for_register(s)
    s.invalidate(true);
  end

  def on_success_res(session)
    puts 'on_success_res->'
  end

  def on_ack(session)
  end

  def on_failure_res(session)
    if session.iresponse.code == 401
     r = session.create_request_with_response_to_challenge(session.iresponse.www_authenticate, false,"a", "b")     
     session.send r
     session[:auth] = r.authorization
    end
  end
end

And Here is the Test Case that runs both controller and performs Validation/Assertion for the call flow.

$:.unshift File.join(ENV['SIPPER_HOME'],'sipper_test')
require 'driven_sip_test_case'

class CustomerAgentTest < DrivenSipTestCase
  def setup
    super
    SipperConfigurator&#91;:SessionRecord&#93;='msg-info'
    puts '------------------------------------'
  end
  
  def test_case1
    start_named_controller_non_blocking("AgentController")
    sleep 1
    start_named_controller_non_blocking("CustomerController")    

    #it waits for Agent & Customer flow complete. The agent controller triggers the completion with session.flow_completed_for invocation. 
    wait_for_signaling()
            
    self.expected_flow = &#91;'< INVITE', '> 100', '> 180', '> 200', '< ACK ' , '< INVITE', '> 200', '< ACK',  '< INVITE', '> 200', '< ACK' , '< BYE', '> 200']
    verify_call_flow(:in,0)

    self.expected_flow = ['> INVITE', '< 100 {0,}', '< 200', '> ACK', '< INVITE', '> 200', '< ACK', '< INVITE', '> 200', '< ACK', '> BYE'  ]
    verify_call_flow(:out, 1)   
    puts "done tests"
  end
end

Ain’t this cool! Just a few lines of Code and you get a test case with Both UAC & UAS simulation!. I wish the Sipper Guys posted some more examples. We are planning to integrate this testcases with Watir & FunFX to automate the Full VOICE + WEB Mash up.

Designers design, Coders Code

Taken from the famous book “Domain Driven Design” —

Manufacturing is a popular metaphor for software development. One inference from this metaphor: highly skilled engineers design; less skilled laborers assemble the products. This metaphor has messed up a lot of projects for one simple reason—software development is all design.

How true! I have heard of organizations where designers design and coders code! I would hate to work with such organization either as a Designer or as a coder.

Few words about “Comments”

I’v been reading this great book “Clean Code: A Handbook of Agile Software Craftsmanship” By Robert C. Martin. This book is going to be another classic like the “Code Complete” or “Refactoring“. Anyway, I liked and agreed so much with the author about what he wrote about comments that I couldn’t resist sharing a few lines from the book —-

The proper use of comments is to compensate for our failure to express ourself in code. Note that I used the word failure. I meant it. Comments are always failures. We must have them because we cannot always figure out how to express ourselves without them, but their use is not a cause for celebration.
So when you find yourself in a position where you need to write a comment, think it through and see whether there isn’t some way to turn the tables and express yourself in code. Every time you express yourself in code, you should pat yourself on the back. Every time you write a comment, you should grimace and feel the failure of your ability of expression.

When we learned programming in our college, we were told by our teachers that you should always write comment! Even when we joined job, our experienced seniors also told us to write comments! It took me a long time to realize that comments should only be used to document some domain knowledge, writing api/library documentation, not to describe code. If you use comments too frequently in every piece of code you write, you should read all three books I mentioned earlier.

Your code should speak for itself. Period.

GNU Screen Essentials

Long ago, a very historic unix hacker was surprised to see I don’t know what “Screen” is! Seeing him so excited, I looked into screen. Basically it is a terminal multiplexer/emulator which gives you the facility of multiple terminal over a single terminal screen. Back then, I had just started my career and the tool didn’t seem very essential to me. If I need multiple terminal, I can just open multiple terminal, right! Huh! How naive I was. I didn’t realize the beauty of screen until I had to maintain a production system over a very slow internet connection where I had to first ssh into one machine, from there, ssh into another machine, then do the work. I had to do this for 3-4 terminal just to get going, But that is  until I started to use Screen properly. Now I open a single connection and use Screen to emulate multiple terminal. The added advantage is, You don’t loose your ssh session even when your connection goes down, you simply reattach to that session after reconnecting. This is specially helpful when you don’t want to stop your process even if the terminal that spawned the process dies. (You can also use nohup for that).

First install screen:

$ yum install screen

Anyway, here are the screen commands that I use frequently and can help you get started.

1. Start Screen:

$ screen

You will see some greeting message. You can set it off by putting this line in your $HOME/.screenrc file

startup_message off

OK, now you can see the same old terminal screen where you can do anything that you used to do in your main terminal. But keep in mind that, its not your regular terminal, you are actually in the “Screen” application. So this is your base screen terminal with index 0. You can continue your work with it.

Now, If you want another terminal, on the “Screen” terminal, hit

CTRL-a CTRL-c

. You are now in another terminal(this with index 1). To go back to previous terminal, hit

CTRL-a '

(that is press CTRL-a and then press ). you will see an emacs style prompt asking your the index: .

CTRL-a CTRL-a

will switch between most resent two terminal.

CTRL-a 1

will directly take you to terminal 1 .  Cool! Now you can start as many screen as you want and play around! If you want to list all active screen terminals :

CTRL-a "

Now that you have the very basics of screen, here comes the most powerful feature: Attaching to a previous screen session and Detaching from an existing screen session.

When you are inside screen, press :


CTRL-a d

This will detach you from an existing screen session. To connect back to the screen session use:


$ screen -r

It will connect you to the screen session, But what if you have multiple screen? You can see the screen list like this:

$ screen -ls

Now you can use the screen name to reattach:

$ screen -r mysession

But you surely don’t like the screen name which is provided by default. You can provide a screen session name when you created a screen session like this:

$ screen -S mysession

Now that you get the basics and have the feel for it, you can explore more advanced options like cut/copy/paste , history etc.
I’ll leave that on you to explore.

Lessons learned from Software Development – Part 1

When I started my career, the first thing I made up my mind about is I have to “Stand on the shoulders of giants”. From the very beginning, I tried to learn from the best coding practices, design patterns, case studies, books etc etc. In fact, there are so much materials around that one has to be overwhelmed! But there are some lessons that I had to learn the hard way. Although they are in the books, I didn’t realize them until I made the mistakes over and over again.

In this series of blog post, I’ll try to share them with the rest of the world. Note that these are not my philosophy, I just think they are the most important among all other principles.

Lesson 1 : Pareto’s Law – The 80/20 principle:

While the 80/20 principle is applicable to a lot of areas, I am only talking about the software modules and codes. If you are involved in a big project, You will see only 20% of the code is performing the core task, the rest 80% is just supporting codes. If you have 10 components in the system, only 2 of them is actually performing the task, the rest are just there to help the 20% (This applies if you group your functionality in a proper way rather than scattering them all around).

We software developers often overlook this, we often put our focus in the wrong place. One thing you need to think about is the Return of Investment. If you claim to be an engineer, one thing that an engineer does is Optimization and Trade Off. Say you are developing a plugin framework – the framework itself is much more important than the plugin. If you have limited time and you try to put equal effort on both of them, you will get none of them right. We always work with deadlines, we never get infinite time, so we have to effectively use that time.

Perfection takes a lot more time than getting something done. You have to make wise decision about which areas you need to perfect and which areas just need to work. Abstraction is the key here. If you have good abstraction, you can always go back and improve your code. I was watching this presentation by Eric Evans about “Strategic Domain Driven Design” for large project where he talked about the same thing. I was quite surprised how our view seemed kind of same! Well, Great men think alike 😉

Anyways, the point I am trying raise is that , we developers become too much obsessed with our code and technology and forget to think about the Deadline, estimation and all these stuff. If you don’t think about ROI (Return of Investment), your project is going to suffer from it. This will even make more sense when you think about Agile development, Changes is requirement and “Throw-away System” from The Mythical Man-Month(Another lesson I’ll talk about).

In the next post, I will talk about the Lesson 2: Abstraction

Domain Driven Design : Sample Application Released!

If you haven’t heard about Domain Driven Design, dude! Its high time you catch up. If you liked the concept, and wonder how to implement it in your spring app, here comes the Sample Application implementing the famous Cargo application from the DDD Book. And the best part is, the DDD guru Eric Evans is the main developer! I have been looking for it for a long time and can’t wait for my maven to finish compiling the code!

Every Software Developer must have an Ultra portable Laptop, Period.

I am obsessed with computers. I spend at least 12-14 hours a day in front of a screen, that includes even the weekends. With all the blog posts stored in my google reader, with all the e-books piled in my reading list, with all the news letters marked unread in my gmail, its really hard to stay away. But many a times I couldn’t be online just because I was too lazy/too tired to sit in front of my desktop or my laptop was too heavy to bring into the bed. But gone are those days with my Samsung NC10. This must be the coolest piece of gadget I’v had so far. Now I can power on my pc and go online in about 60 seconds and can do that right after I open my eyes in the morning(my nc10 sleeps right beside my pillow). I can even run my eclipse with terracotta clustering and do the experimental codes!!! What else can you ask from a 1.3 kg laptop with < 500$ price!

Now I realize what I missed all these days. I am hooked with the ultra portable concept and can’t live without one for the rest of my life. If you are someone like me who just can’t keep your hands off a computer, you must get one of these. I am looking forward to get another one with a better processor , May be the Sony VAIO TX series but that is when I can afford them.

How to get the jar file location of a loaded class?

When there are two versions of the same class (same class & package name) in the classpath (may be from two different jars), there is no way to tell the JVM which one to load at runtime (JVM loads the one which it finds first). And there is no direct way of knowing from which jar the class was loaded! This causes enormous difficulties for a j2ee developer when working with different containers.

You can use the following code to find which jar was used to load the class

	public static String getClassLocation(Class clazz) {
		if(clazz == null) {
			return null;
		}
		
		try {
			ProtectionDomain protectionDomain = clazz.getProtectionDomain();
			CodeSource codeSource = protectionDomain.getCodeSource();
			URL location = codeSource.getLocation();
			return location.toString();
		} catch (Throwable e) {
			e.printStackTrace();
			return "Not found";
		}
	}

For example, The if you run the following code,

	public static void main(String[] args) {
		System.out.println("class location of Log: " + getClassLocation(org.apache.commons.logging.Log.class));
	}

It will give you :

class location of Log: file:/C:/dev/tools/maven2/repository/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar

This simple code helped me a lot to resolve a lot of deployment problem in different containers specially when working with third party libs.

Thinking in Objects

Long ago, I had to write some code which do this:

Given two list 1. oldList 2. newList you have to find out:
1. The elements which are common in both list.
2. The elements which are new in the newList (means they were added)
3. The elements which are in oldList and not in newList (means they were deleted)

Definitely not the hardest kind of problem! I quickly grabbed my keyboard and wrote something like this :

public class CollectionModificationHelper {
public static List getCommonElements(List oldCollection, List newCollection) {
//iterate on oldCollection and find the common elements
}
public static List getNewElements(List oldCollection, List newCollection) {
//iterate on newCollection and find the new elements
}
public static List getRemovedElements(List oldCollection, List newCollection) {
//iterate on oldCollection and find the deleted elements
}
}

Simple solution and works fine. Only problem is, its iterates over the same collection again and again to find the old/common/new elements, not a big deal if you are not worried about performance.

Yesterday, I had to write the same code! Almost 4 years after I wrote the first version, seeing a hell lot of code, reading a hell lot of materials about Object Oriented Programming and design, this is what I wrote:

import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;

import org.apache.commons.collections.CollectionUtils;

public class CollectionModificationHelper<T> {
	private Collection<T> oldCollection;
	private Collection<T> newCollection;
	
	private Collection<T> newElements = new ArrayList<T>();
	private Collection<T> commonElements = new ArrayList<T>();
	private Collection<T> removedElements = new ArrayList<T>();
	
	public CollectionModificationHelper(Collection<T> oldCollection,
			Collection<T> newCollection) {
		super();
		this.oldCollection = oldCollection;
		this.newCollection = newCollection;
		
		init();
	}
	
	public CollectionModificationHelper(T[] oldArray, T[] newArray) {
		this((oldArray ==  null ? null : Arrays.asList(oldArray)), (newArray == null ? null : Arrays.asList(newArray)));
	}

	private void init() {
		if(CollectionUtils.isEmpty(oldCollection) && CollectionUtils.isEmpty(newCollection)) {
			//nothing to do!
		} else if(CollectionUtils.isEmpty(oldCollection)) {
			if(!CollectionUtils.isEmpty(newCollection)) {
				this.newElements = new ArrayList<T>(newCollection);
			}
		} else if(CollectionUtils.isEmpty(newCollection)) {
			if(!CollectionUtils.isEmpty(oldCollection)) {
				this.removedElements = new ArrayList<T>(oldCollection);
			}
		} else { 
			
			for(T item : oldCollection) {
				if(newCollection.contains(item)) {
					commonElements.add(item);
				} else {
					removedElements.add(item);
				}
			}
			
			newElements.addAll(newCollection); //first add all new Collection in new element. 
			newElements.removeAll(commonElements); //then remove the common elements.
		}
	}

	public Collection<T> getCommonElements() {
		return commonElements;
	}

	public Collection<T> getNewElements() {
		return newElements;
	}

	public Collection<T> getRemovedElements() {
		return removedElements;
	}
}

Its interesting to see how your thought process changes over time. As far as I my opinion goes, the 2nd solution is much elegant than the first one! In the first one, I was just not thinking in Objects!

Over the years, as you are involved in serious development, you will surely learn many technologies, tricks, techniques. But I think you must thrive to learn how to think in Objects. Once you can achieve that, you will find a whole new world of idea about how the objects/components should interact. Once you are there, you will love it. I am not saying I am there already, but I sure did come a long way from where I started and trying to master it everyday. How do you prepare yourself? Practice & Read. My two best guru in this area are both named “Martin”! “Robert C. Martin” & “Martin Fowler” 🙂 Do have a look at their work if you haven’t already.

Lastly, what code is good code without some unit tests!


import java.util.Collection;

import junit.framework.TestCase;

public class CollectionModificationHelperTest extends TestCase { public void testWithNullCollections() {  Integer[] old = null;  Integer[] newC = null;

  CollectionModificationHelper&lt;Integer&gt; collectionModificationHelper = new CollectionModificationHelper&lt;Integer&gt;( old, newC);

  assertEquals(0, collectionModificationHelper.getCommonElements().size());  assertEquals(0, collectionModificationHelper.getNewElements().size());  assertEquals(0, collectionModificationHelper.getRemovedElements().size());

  old = new Integer[]{1,2,3,4};  newC = null;

  collectionModificationHelper = new CollectionModificationHelper&lt;Integer&gt;( old, newC);  assertEquals(0, collectionModificationHelper.getCommonElements().size());  assertEquals(0, collectionModificationHelper.getNewElements().size());  assertEquals(old.length, collectionModificationHelper.getRemovedElements().size());

  old = null;  newC = new Integer[]{1,2,3,4};

  collectionModificationHelper = new CollectionModificationHelper&lt;Integer&gt;( old, newC);  assertEquals(0, collectionModificationHelper.getCommonElements().size());  assertEquals(newC.length, collectionModificationHelper.getNewElements().size());  assertEquals(0, collectionModificationHelper.getRemovedElements().size()); }

 public void testCollectionModificationHelper() {  Integer[] old = new Integer[]{1,2,3,4,5,6,7};  Integer[] newC = new Integer[]{1,4,10,11,12,13};

  CollectionModificationHelper&lt;Integer&gt; collectionModificationHelper = new CollectionModificationHelper&lt;Integer&gt;( old, newC);

  Collection&lt;Integer&gt; commonElements = collectionModificationHelper.getCommonElements();  Collection&lt;Integer&gt; newElements = collectionModificationHelper.getNewElements();  Collection&lt;Integer&gt; removedElements = collectionModificationHelper.getRemovedElements();

  System.out.println("commonElements: "   commonElements);  System.out.println("newElements: "   newElements);  System.out.println("removedElements: "   removedElements);

  assertEquals(2, commonElements.size());  assertEquals(4, newElements.size());  assertEquals(5, removedElements.size()); }}