Will scala become obsolete with java 8?

I have been investing a lot of my free time in Scala lately. Although I have been injecting groovy heavily in all my projects whenever I get a chance, I miss the safety of Type system (too much java does that to you).

Even though I had some previous Erlang experience and background knowledge of Actor Model, I must admit Scala has a steeper learning curve than I thought.It requires significant time and energy invested before you can do anything useful with it. I can imagine I’d have a hard time if I wanted to push it to a team of java devs. On the other hand, Groovy is too easy to adopt, you write your java code in a script and in no time fall in love with the power of functional language with Java goodies in your hand.

That brings the question in my mind, will scala be obsolete with java 8? A lot of scala goodies will face serious competition with java 8. Developers will eventually adopt those feature sooner or later if they want to keep their job but I don’t see mass adoption of scala. Don’t get me wrong, I am loving scala and no doubt its an elegant language but I don’t see it as a successful complement language of Java with java 8 in the picture. On the other hand, I can imagine myself doing all my scripting / hacking jobs in Groovy as a replacement of my broken perl skills.

Some taste of functional programming with groovy

In my day job, we have to do a lot of system integration, a lot of it consists consuming external feeds , process it and feed it to some other system. We use Mule a lot for this. I was writing a groovy script in mule to parse a xml feed and extract some data, here is how my functional programming investment helped me do it in an elegant way. The XML to parse

<sport reportName="aflRoundMatches" jobId="72451003">

  <aflRoundMatches roundNumber="3">
    <match matchId="10370301" matchStatus="complete" matchName="Dockers vs Bombers" matchNumber="1" isPreseason="false" isFinal="false" homeSquadId="60" homeSquadName="Fremantle" awaySquadId="50" awaySquadName="Essendon" venueId="70" venueName="Patersons Stadium" venueTimezone="Australia/Perth" utcStartTime="2013-04-12T10:40:00+00:00" localStartTime="2013-04-12T18:40:00+08:00" period="4" periodCompleted="4" periodSeconds="1877" homeSquadGoals="9" homeSquadBehinds="14" homeSquadScore="68" awaySquadGoals="10" awaySquadBehinds="12" awaySquadScore="72"/>
    <match matchId="10370302" matchStatus="complete" matchName="Kangaroos vs Swans" matchNumber="2" isPreseason="false" isFinal="false" homeSquadId="100" homeSquadName="North Melbourne" awaySquadId="160" awaySquadName="Sydney Swans" venueId="2" venueName="Blundstone Arena" venueTimezone="Australia/Hobart" utcStartTime="2013-04-13T03:45:00+00:00" localStartTime="2013-04-13T13:45:00+10:00" period="4" periodCompleted="4" periodSeconds="1873" homeSquadGoals="13" homeSquadBehinds="14" homeSquadScore="92" awaySquadGoals="20" awaySquadBehinds="11" awaySquadScore="131"/>
    <match matchId="10370303" matchStatus="complete" matchName="Demons vs Eagles" matchNumber="3" isPreseason="false" isFinal="false" homeSquadId="90" homeSquadName="Melbourne" awaySquadId="150" awaySquadName="West Coast Eagles" venueId="40" venueName="MCG" venueTimezone="Australia/Melbourne" utcStartTime="2013-04-13T04:10:00+00:00" localStartTime="2013-04-13T14:10:00+10:00" period="4" periodCompleted="4" periodSeconds="1878" homeSquadGoals="13" homeSquadBehinds="5" homeSquadScore="83" awaySquadGoals="27" awaySquadBehinds="15" awaySquadScore="177"/>
    <match matchId="10370304" matchStatus="complete" matchName="Giants vs Saints" matchNumber="4" isPreseason="false" isFinal="false" homeSquadId="1010" homeSquadName="GWS Giants" awaySquadId="130" awaySquadName="St Kilda" venueId="150" venueName="Manuka Oval" venueTimezone="Australia/Canberra" utcStartTime="2013-04-13T06:40:00+00:00" localStartTime="2013-04-13T16:40:00+10:00" period="4" periodCompleted="4" periodSeconds="1949" homeSquadGoals="10" homeSquadBehinds="8" homeSquadScore="68" awaySquadGoals="21" awaySquadBehinds="14" awaySquadScore="140"/>
    <match matchId="10370305" matchStatus="playing" matchName="Cats vs Blues" matchNumber="5" isPreseason="false" isFinal="false" homeSquadId="70" homeSquadName="Geelong Cats" awaySquadId="30" awaySquadName="Carlton" venueId="190" venueName="Etihad Stadium" venueTimezone="Australia/Melbourne" utcStartTime="2013-04-13T09:40:00+00:00" localStartTime="2013-04-13T19:40:00+10:00" period="3" periodCompleted="2" periodSeconds="421" homeSquadGoals="13" homeSquadBehinds="6" homeSquadScore="84" awaySquadGoals="10" awaySquadBehinds="8" awaySquadScore="68"/>
    <match matchId="10370306" matchStatus="playing" matchName="Suns vs Lions" matchNumber="6" isPreseason="false" isFinal="false" homeSquadId="1000" homeSquadName="Gold Coast Suns" awaySquadId="20" awaySquadName="Brisbane Lions" venueId="81" venueName="Metricon Stadium" venueTimezone="Australia/Brisbane" utcStartTime="2013-04-13T09:40:00+00:00" localStartTime="2013-04-13T19:40:00+10:00" period="3" periodCompleted="2" periodSeconds="773" homeSquadGoals="7" homeSquadBehinds="8" homeSquadScore="50" awaySquadGoals="7" awaySquadBehinds="14" awaySquadScore="56"/>
    <match matchId="10370307" matchStatus="scheduled" matchName="Tigers vs Bulldogs" matchNumber="7" isPreseason="false" isFinal="false" homeSquadId="120" homeSquadName="Richmond" awaySquadId="140" awaySquadName="Western Bulldogs" venueId="190" venueName="Etihad Stadium" venueTimezone="Australia/Melbourne" utcStartTime="2013-04-14T03:10:00+00:00" localStartTime="2013-04-14T13:10:00+10:00" period="0" periodCompleted="0" periodSeconds="0" homeSquadGoals="0" homeSquadBehinds="0" homeSquadScore="0" awaySquadGoals="0" awaySquadBehinds="0" awaySquadScore="0"/>
    <match matchId="10370308" matchStatus="scheduled" matchName="Magpies vs Hawks" matchNumber="8" isPreseason="false" isFinal="false" homeSquadId="40" homeSquadName="Collingwood" awaySquadId="80" awaySquadName="Hawthorn" venueId="40" venueName="MCG" venueTimezone="Australia/Melbourne" utcStartTime="2013-04-14T05:20:00+00:00" localStartTime="2013-04-14T15:20:00+10:00" period="0" periodCompleted="0" periodSeconds="0" homeSquadGoals="0" homeSquadBehinds="0" homeSquadScore="0" awaySquadGoals="0" awaySquadBehinds="0" awaySquadScore="0"/>
    <match matchId="10370309" matchStatus="scheduled" matchName="Power vs Crows" matchNumber="9" isPreseason="false" isFinal="false" homeSquadId="110" homeSquadName="Port Adelaide" awaySquadId="10" awaySquadName="Adelaide Crows" venueId="10" venueName="AAMI Stadium" venueTimezone="Australia/Adelaide" utcStartTime="2013-04-14T06:40:00+00:00" localStartTime="2013-04-14T16:10:00+09:30" period="0" periodCompleted="0" periodSeconds="0" homeSquadGoals="0" homeSquadBehinds="0" homeSquadScore="0" awaySquadGoals="0" awaySquadBehinds="0" awaySquadScore="0"/>
  </aflRoundMatches>
</sport>

The Groovy script:

new XmlSlurper().parseText(payload).aflRoundMatches.match
		.findAll { ['playing','postmatch'].contains(it.@matchStatus.text()) }
		.collect { match -> 
			[
			'matchId': match.@matchId.text() , 
			'homeSquadId' : match.@homeSquadId.text() , 
            'matchStatus' : match.@matchStatus.text(),
            'awaySquadId' : match.@awaySquadId.text(), 
		    'HomeTeam/ScoreTotal_C' : match.@homeSquadScore.text(),         
            'AwayTeam/ScoreTotal_C' : match.@awaySquadScore.text() ,            
            'Period_Active' : match.@period.text().toInteger() >= match.@periodCompleted.text().toInteger() ? 0 : 1
            ]
		}

Here, new XmlSlurper().parseText(payload).aflRoundMatches.match gives me all the <match> elements of the xml, on that I run a filter for only the ‘playing’ & ‘postmatch’ games (incase you are wondering, its for pushing match scores on Live Television for the playing/postmatch games). After that I run a ‘collect’ method to convert the <match> elements into a Map<attribute, value> for that match. At the end of process I get a List<Map<attribute, Value>> Imagine doing this in Java! This is pure transition of data from one state to another through functions. Coming from years of Object Oriented background, I had to really struggle with this concept as we are so used to think in terms of Nouns aka Classes. Here is a really interesting read about it: http://steve-yegge.blogspot.com.au/2006/03/execution-in-kingdom-of-nouns.html I have some more advanced example which I will save for the next blog post.

Currying with Groovy

I was writing some functional test case with Cucumber for JVM and opted to use Groovy for my step definitions. While I am not yet comfortable enough to start any big project with an untyped language like groovy,  its super cool for writing test cases. Specially when we write functional test cases, we want to write as less code as possible and achieve as much as possible. I think groovy is a perfect candidate for that. I am also writing my shell/perl scripts with Groovy now. Call me old school but I still like my java 🙂

Anyway, I was fetching the XML from the server and was asserting various attributes on it. Here is how the power of functional programming made it elegant.

def xmlFetchedFromWebservice = '''
<SchedRemote> <!-- Long Form: Room, Service, and Resource -->
 <WorkOrderDetails ConflictLevel="0" SEQNUM="232323">
 <Fields>
 <Field STAT="10" />
 <Field STATUSEX='0'/>
 </Fields>
 </WorkOrderDetails>
 </SchedRemote>
'''

def xmlFind = { xmlFields, fieldName ->
 def field = xmlFields.find {
 return it['@' + fieldName] != null
 }
 return field?.( '@' + fieldName)
}
def selectXmlFields = {xmlText -> new XmlParser().parseText(xmlText).WorkOrderDetails.Fields.Field}

def workOrder = xmlFind.curry(selectXmlFields(xmlFetchedFromWebservice))

assert workOrder('STAT') == 10
assert workOrder('STATUSEX') == 0

//..... many more assersions like this

I refactored some of the implementation details , In a nutshell We have two part task, 1st select the correct section from xml and then perform a search on that section to select the field value.

So once I get the xml from webservice, I select the right fields with selectXmlFields and curry the result in the xmlFind method.

Now, the resulting function can be used almost as object like fashion. The core concept of functional programming which took me quite a while to grasp is that functions are the building blocks, not the objects. If I wanted to do the same thing with java, may be I would create a WorkOrder class, parse the XML in the constructor and use workOrder object. In functional language, I am using the function composition from the currying to perform the same thing. In other words, I am using curry here to bind xmlFind to the perticular workorder instance. How cool is that?

Bug free Software!

Last month we rolled the System Automation tool into production that we have been working for last 8 months. I would rank it as a fairly complex application with integration between Various System and low level network components. Since the roll out there has been some minor cosmetic changes and feature enhancement but overall no Bugs! Yes, I mean it, not a single Issue filed on Jira with ‘Bug’ category!

I remember about 8 years ago when I started my career, Production Release was a big deal. We would often do it on the weekend, prepare for it intensively and after the release we would always work nights to fix the critical bugs that came up after release. With great joy I am feeling the wind of change of that trend. Among the 5+ release that I was involved in for last 4 years with different products in different companies, Bug free (at least major bug free) software seems to be the new Trend! Production Release has become the boring activity that someone just do in the Lunch time!

What happened? Did we hire more QA? Did we all suddenly become super programers who don’t produce any bug? No! I think what changed is the mindset of how we are working now a days. What triggered the mindset you ask? Its the agile movement. After doing Scrum/Agile for last 5 years, I am so confident to say its the best (if not only) way to build software. Agile forced us to care about the quality, craftsmanship, value of unit testing, value of functional testing, Continuous Integration, automated tests.

As we are getting more and more matured in the software development process, I can see we are getting past all the debates around best practices and growing a common consensus. I always felt software development is too much of a dynamic process and its unlikely to become as structured as the other engineering disciplines like Architecture or Civil. But may be that is not true, may be we will be just like them after 50 years ! Just my random thoughts 🙂

 

 

Diplomatic programmer

Linus Torvalds (creator of linux kernel)  on reply to why git is not written in C++

———————————————————————————————————————————————–

>

> When I first looked at Git source code two things struck me as odd:

> 1. Pure C as opposed to C++. No idea why. Please don’t talk about portability,

> it’s BS.

 

*YOU* are full of bullshit.

 

C++ is a horrible language. It’s made more horrible by the fact that a lot

of substandard programmers use it, to the point where it’s much much

easier to generate total and utter crap with it. Quite frankly, even if

the choice of C were to do *nothing* but keep the C++ programmers out,

that in itself would be a huge reason to use C.

 

In other words: the choice of C is the only sane choice. I know Miles

Bader jokingly said “to piss you off”, but it’s actually true. I’ve come

to the conclusion that any programmer that would prefer the project to be

in C++ over C is likely a programmer that I really *would* prefer to piss

off, so that he doesn’t come and screw up any project I’m involved with.

 

C++ leads to really really bad design choices. You invariably start using

the “nice” library features of the language like STL and Boost and other

total and utter crap, that may “help” you program, but causes:

 

– infinite amounts of pain when they don’t work (and anybody who tells me

that STL and especially Boost are stable and portable is just so full

of BS that it’s not even funny)

 

– inefficient abstracted programming models where two years down the road

you notice that some abstraction wasn’t very efficient, but now all

your code depends on all the nice object models around it, and you

cannot fix it without rewriting your app.

 

In other words, the only way to do good, efficient, and system-level and

portable C++ ends up to limit yourself to all the things that are

basically available in C. And limiting your project to C means that people

don’t screw that up, and also means that you get a lot of programmers that

do actually understand low-level issues and don’t screw things up with any

idiotic “object model” crap.

 

So I’m sorry, but for something like git, where efficiency was a primary

objective, the “advantages” of C++ is just a huge mistake. The fact that

we also piss off people who cannot see that is just a big additional

advantage.

 

If you want a VCS that is written in C++, go play with Monotone. Really.

They use a “real database”. They use “nice object-oriented libraries”.

They use “nice C++ abstractions”. And quite frankly, as a result of all

these design decisions that sound so appealing to some CS people, the end

result is a horrible and unmaintainable mess.

 

But I’m sure you’d like it more than git.

 

The Starter, the Architect, the Debugger and the Finisher — which one are you?

I liked this post so much I couldn’t help writing a post about it.

http://jacquesmattheij.com/The+Starter+the+Architect+the+Debugger+and+the+Finisher

–So, which one are you? I think I can perform all of them but not so sure about the “Finisher” 🙂 . I tend to get bored and move to the next Starter/Debugger role. Writing code is much more fun than finishing up that last piece of user doc, although I know the user doc is much more important than the code itself. But I’m just a mere mortal.

My thoughts on upfront architecture and iterative development

Long ago, I was attending this training program on Software design and architecture. A very respected Architect was talking about upfront design. He was trying to illustrate the fact that upfront design was very critical and you should always design upfront keeping performance & extensibility in mind. He was giving analogy with the foundation of building which goes like this:

“If you build a five story building, you will never be able to extend it by another 5 story. So you should always prepare for worst”

… At that time, I agreed completely with him. After all, you can’t really extend a building that doesn’t have enough foundation for it, right !

About 5 years later and a couple of completely failed and successful project experience under my belt, I beg to differ. I have seen so many analogy with building houses to developing software! Is it because of Christopher Alexander and his influence in “Design Patterns”? Now a days, I find many of these analogy to be irrelevant, Because, “Software Development is not the same thing as Developing high rise Buildings! Period. “.

Have you ever been in a situation where you did foundation for a 10 story Building and after finishing the first two floor, you realize its not a House you want to develop, its an Airplane that is actually required ! Or even better, after developing a 10 story building, no one wants to live in that area?

Unless you are working on projects like “Facebook-Chat” where you know on the first day of release, you are going to get a million hit, It is always best to take the simple approach first. This is why I love Agile and Lean process. You do only what is necessary and when its necessary. In agile, we are always prepared to refactor, re-evaluate architecture, scrap the existing foundation if it doesn’t work.