Steampunk Holmes

I've got a buddy named Richard Monson-Haefel who's so obsessed with touch screen technology that he abandoned his consulting career to start a company making interactive books for the iPad, Kindle, etc. He's working on a really cool project making a series of Sherlock Holmes books right now. He's gotten some really high-end writers, actors & illustrators to produce the book and funding is tight, so he's using kickstarter (a web site that helps people fund start ups by giving away prizes for contributions) to help fund his project. I checked to see how things were going for him today and I noticed he could use a little help, so if you're into interactive books check it out. I really admire his ambition and would love to see the project succeed.

Framework Misuse: Killed By Friendly Fire

These days there are a lot of really great development tools & technologies at our disposal. I’ve talked about a few of them in the past and how important I think selectively choosing and slowly evolving your technology stack is. While choosing technologies wisely is important, unfortunately it’s not enough. In reality, selecting strong tools & frameworks is meaningless if those tools aren’t utilized correctly. In fact, the right tool used incorrectly is often worse than not using it all. Some of the largest anger tax I’ve paid can be attributed to the misuse of some of my favorite frameworks.

I’ve used weapons as a metaphor for technology selection in the past. That analogy is just as well suited to technology usage. If JDBC is a surface to air missile, Hibernate is the virtual equivalent of an A-bomb. It’s relatively easy to decide that Hibernate is a powerful, industry standard ORM solution and thus identify it as the new standard to replace JDBC on your project. Just don’t make the mistake of thinking this decision constitutes a free upgrade to the quality of your data access layer. Putting that WMD into the hands of developers comes with the cost of making sure they have direction on how to wield its power. The more powerful the technology, the more damage it can do when poorly used.

There’s nothing worse than anxiously arriving on a project only to find that my productivity is constantly being stifled by a misguided attempt to use a great framework. I find myself dealing with things like a "customized" sub class of Spring context that's being used as a Service Locator. Really? We had to create our own version of the Spring context? At this point I'm still in denial, thinking "wow, we must have some really interesting problems to solve on this project". But no, I search relentlessly for the reason regular dependency injection wasn't sufficient and there's none to be had. Someone made a good technology decision and then checked out before doing some research on how to use it and establishing some architectural guidelines. This is a really tough pill to swallow. All of the pieces seem to be in place but somewhere along the way things went awry. You have the right weapons at your disposal but somehow they keep backfiring on you.

In these situations I repeatedly find myself evaluating the opportunity cost of large refactoring efforts. I can’t stop thinking about how great things could be. After some analysis and considerable venting the refactoring approach loses out. I come back to my senses and realize there’s no time in the budget for such changes and that the best possible outcome from a business perspective is that no one will notice the changes. All signs point toward staying the course despite the less than optimal circumstances. That is, until later when I’m once again feeling pain induced by framework misuse and burning mass quantities of time on things that should be simple – things that the framework should make easier. Perhaps I would save more time AND be returned to my happy coding Zen if I just bit the bullet and fixed some of these annoying problems. Wait! I’ve been down this road before. So it goes, back and forth, around and around in a downward spiral toward insanity.

In order to avoid this cerebral tug-of-war, allocate adequate time to investigate your chosen frameworks and identify their correct usage on your project. Most frameworks are very flexible. There are often many ways to accomplish the same task within a given framework. Indeed, extremely "clever" ways even the framework development team hasn’t envisioned. That type of flexibility usually gives developers more than enough rope to hang themselves. In my experience, there tends to be one decidedly right way of solving any given problem with a framework that can be applied about 80% of the time. This is what I like to call “the normal way of doing things”. Take the time upfront to determine the normal way of doing things for your project. This often requires wading through reams of documentation. This may be painful but it’s part of the cost involved with adopting new technology. Identify a standard approach to using the frameworks you’ve chosen and establish some governance to ensure those principles aren’t being compromised unnecessarily. The alternative is to allow development to ensue without direction, naïvely assuming good framework selection decisions will solve more problems than they create. This is a dangerous assumption that is far more likely to have you cursing tools you’d otherwise praise.

Rock Star Developers Wanted

I’m not sure if anyone in the blogosphere has noticed but I’ve been off on hiatus for a while. I wish I could say that’s because I’ve been placed at a client where my rants have ceased and I am now experiencing software development nirvana. Sadly that’s not the case. Things have been busy, good busy and bad busy. On the good front I finished up a contract of over 3 years with a great company and have moved on to a new contract with a client that looks to have some fun problems to solve. On the not so good front my mother has been diagnosed with cancer in her lung, bones & adrenal gland. Hence, stuff has been taking priority over the old blog and I’ve been given clear vision of how unimportant my rants actually are. Don’t get me wrong, I still hate 400 line queries riddled with business logic – I just don’t hate them as much as cancer.

Despite my absence from cyberspace, as usual I’ve managed to have quite a few conversations about software design and have accumulated some blog-worthy thoughts. I’m hoping to find some time to get those ideas out of my head and onto the screen in the near future but they will be topics for another post. This post is about finding good developers and the hope that there are some out there who follow this blog.

The global services division of the company I work for has been granted an exclusive on about a dozen developer spots, an architecture spot and some PM/BA work for the client where they have placed me. Part of the value my company delivers in this type of engagement over traditional staffing is a promise to provide high quality candidates to cut down on the noise involved with the normal staffing process, i.e. hundreds of resumes from dozens of vendors. Because recruiters and sales people have difficulty sorting through the alphabet soup on a developer’s resume I often get injected into the candidate-requirement matching process to screen candidates and have conversations with hiring managers to see what they’re really working on. This is especially effective for clients I’ve actually worked for. So, as you can probably deduce, client with many spots to fill + Chad working for said client = Chad screening candidates during every free moment. Since I like designing and writing software a lot more than talking on the phone I’m anxious to get this distraction off my plate and return to my happy coding Zen. However, despite the tough economy it’s still relatively difficult to find really stellar developers.

So this is a casting call to anyone in the Minneapolis area (or anyone willing to fly in) that would consider themselves a senior Java developer or architect. There is work on several different projects and I believe all of the spots are 12 month. The client is in particular need of people with strong desktop Java experience. There are also spots on the web team but only a few. We’re also looking for an architect with good i18n experience that includes non-latin based. I’ll be screening for all of these spots and I can tell you that the client is fairly particular about who they bring in. I can also tell you that once they place someone they have no problem keeping a contractor on for 5 years. If you’re interested just send me your gear at chad_lavigne@yahoo.com.

Anger Tax: The even darker side of technical debt

The concept of technical debt gets a lot of air time from architecture gurus like Ward Cunningham, Martin Fowler and Neal Ford. The cost incurred by sacrificing design for other project constraints such as time to market does indeed resemble financial debt in many ways. Like paying interest for an item bought on credit, the complexity created by shortcuts taken now impose an additional cost every time we are forced to work around them in the future. Unless we go back and “pay down” that technical debt, we get to enjoy that interest payment for the life of the project.

Technical debt isn’t necessarily viewed in the most positive light. Still, I think the world has been looking at it through rose colored glasses. The picture is always painted as though the architect or developer taking on the debt were making a conscious decision. As though they carefully weighed their options and came to the conclusion that the prudent decision was to take on some technical debt in order to capitalize on a business opportunity. If that were the case the cost of technical debt would be much easier to swallow because its existence would at least have been recognized and justified by someone. Let’s get real here; a lot of the people holding these debts are software development’s version of a college kid that thinks a credit card is free money. They’re not salty software veterans making calculated choices based on risks and rewards. They’re eager neophytes haphazardly throwing applications together using one hack after another.

This is the breed of technical debt that I truly despise. It may not be technical debt in the sense that someone knowingly took a shortcut based on some external force but ignorance doesn’t change the fact it is technical debt just the same. Perhaps those who coined the term would disagree with that, but it sure feels like I’m paying the same “interest” when I’m forced to deal with the resulting bad design.

Along with making future changes more difficult (interest) and the cost of refactoring to eliminate poor design (paying down the principle) that comes with all technical debt, this type of technical debt comes with a special hidden expense. Since it seems to be fashionable to talk about intangible project costs in terms of financial metaphors, I like to call this the Anger Tax. This is the time you spend being angry when you encounter this type of technical debt in the future. When design is compromised based on some sort of tactical business decision it’s a little more forgivable. When you’re called on to help establish a bailout strategy for a project that is failing under the weight of shameless, unmanaged technical debt you tend to look at the debtor like they're the president of AIG. Paying down their technical debt is angry work and that anger isn’t free. Every roll-your-own hack and framework misuse that irritates you is an interruption that negatively impacts productivity. These distractions make it difficult to stay pragmatic about refactoring as you are easily drawn into idealist thought tangents about how things could have been if only you were there before things went awry.

With all of that ranting out of the way, I must admit that I don’t really have any great answers for avoiding technical debt created via programmer negligence - I just like to complain about it. Perhaps requiring some sort of board exam or apprenticeship for architects would help but that's probably not very realistic. I probably shouldn’t be so angry about it anyway since a solution would be financially crushing for me; resuscitating poorly designed applications is a large part of what keeps guys like me working. While I may not have a cure, I do find ranting, sharing war stories and commiserating tends to lighten my mood on the subject - hence this post. I’m always up for hearing a good anger tax story so if you have one, feel free to vent here.

...and JiBX shall reign supreme

There are a lot of choices out there when it comes to XML data binding tools. JAXB, XMLBeans, Castor, JiBX and Xstream are a few that come to mind. I’ve used quite a few of these tools and I like different ones for different reasons. I’m a big fan of tools that focus on doing one thing really well. Serializing and deserializing XML seems like a problem space that is about the right size for such a tool but for some reason none of the technologies I’ve worked with in this area have been able to meet all of my needs.

JiBX is one such tool that comes as close as any that I’ve worked with to being the silver bullet for XML marshalling. For a few years now JiBX has been my marshalling tool of choice when I need to deserialize a lot of XML because it offers the best performance I’ve seen by far. However, this performance has always come at the cost of having to deal with running an XML binding compiler that embeds the binding code directly into your existing class files. This can put a hitch in your development giddy up because you have to run this binding compiler every time you change one of the classes that gets marshalled/unmarshalled. JiBX does support limited runtime binding for development but it is very limited - it’s not designed to work in a servlet engine or app server. That means the vast majority of us who work with web apps will find ourselves restarting our server every time we change a JiBX-aware class and forget to run the binding compiler. The small pain of having to re-run the binding compiler or re-start your server when you forget to run it is one of those annoying little issues that has a cumulative negative effect over the life of a project that is easy to underestimate. Because of this, JiBX has always left a little something to be desired for me and if performance isn’t an issue I opt for something that doesn’t require the extra step.

All of that changed a couple of weeks ago when I was browsing the JiBX documentation hoping that perhaps this last hurdle had been leaped. To my delight I found that it had. There is now an Eclipse plug-in that detects changes to the classes referenced by your JiBX binding definition and re-compiles the binding automatically. The only downside to this discovery is that I didn’t make it sooner; the plug-in has been available since late May of last year. Unfortunately this isn’t the type of advancement that gets a lot of publicity, you have to get annoyed enough to go looking for it. Although it’s a relatively small feature it is a really satisfying addition that adds a lot of value in my opinion. I’ve always found it a little frustrating that none of the XML marshalling tools I’ve worked with were quite good enough to become my de facto standard. By removing this last chink in the armor I can now say JiBX has claimed that title.

Tools Of The Trade: Spring Web Services

In general I think web services are way oversold. Especially when it comes to those of the enterprise SOA/ESB flavor. It seems like it has taken longer than normal for the hype surrounding web services to die down. This is probably because it’s a good area for vendors to push large “solution in a box” tools. The SOA vendors will probably be pushing their products for a while but I think the development community is reaching the point where web services have found their space of usefulness. The days of putting web services in front of every piece of software are fading away. We’ve landed in a place where web services are the right tool for certain jobs, particularly large-scale integration work. Since there are plenty of systems out there that need to talk to each other I think it’s safe to say that web services, in one form or another, are here to stay.

I thought web services would be a good topic for a “tools of the trade” post because it’s another area where there are many tools and technologies available. In addition to that, it’s a very complicated topic. The security alone is enough to make anyone crazy. In my experience, the best way to handle a complicated solution such as this is to avoid it all together. However, there are times when avoidance is not an option and web services are the right solution. In those situations I find the use of good tools can go a long way in helping to mitigate the necessary complexity involved with such a solution.

When it comes to traditional SOAP/WSDL based web services, most of the tools available have fallen short of my expectations. Just when I was beginning to think the terms web service and painful development were synonymous, along came Spring Web Services to save the day. First of all, it’s exclusively contract first. With Spring-WS your web service journey beings and ends at the main integration point – the XSD. This is a big selling point for me because the only situation where I really need web services is when I have to support multiple disparate clients. In those situations getting the service interface right is the most important part. Tools created to bolt web services on to an existing application by generating the contract from existing code have always seemed backwards to me. They create a contract that is tightly coupled to the implementation code. Such a contract is very likely to be difficult for your service clients to consume; those clients don’t speak the same language and that’s why you’re going through all the work of communicating in XML. In the contract first world you are forced to think about how clients interact with your service before it’s implemented. Focusing on the interface first tends to bring important contract issues to the surface quickly.

In addition to the advantage of being contract first Spring-WS also provides good support for the things you expect to find. The usual suspects like message based routing and security are all accounted for and it allows you to use just about any tool under the sun (no pun intended) to deal with the XML. It also passes my “as expected” rule. Things have reasonable names that explain what they do. Objects and configuration are where you expect them to be. The methods you hope to find are usually there and behave as expected. I think these simple things are a large part of what makes a framework easy to use and the software created using that framework easy to maintain. When you’re working with a complicated technology this aspect of a framework is even more important. Staying on top of these simple things takes diligence. It’s surprising how many frameworks don’t do a good job of it. Many seem to be more concerned with functionality and flexibility but what I’m really after is usability.

When it comes down to it, I think the decision of whether or not to use web services is probably more important than what technologies you choose to implement them with. If you don’t need to support remote clients it’s probably not a good idea to add web services “just in case”. If all parties involved talk Java you probably don’t need the overhead and complexity of web services. If you only need to support one remote client things could probably be kept simpler with a POX over HTTP approach. In many cases I think the RESTful web service approach, which Spring 3 MVC supports nicely, is a good alternative to the traditional approach. So there are a lot of things to consider before taking the web service plunge. However, if you’ve analyzed requirements closely and there’s no getting around the fact that you have to support multiple clients making RPC style calls, Spring Web Services is the best approach I’ve seen.

Reuse Abuse

We hear a lot about this elusive creature called reuse that the entire software development world seems to be in pursuit of. I won’t argue against the idea that reuse is a good thing because in theory it is. However, I think views regarding code reuse are often distorted. Reuse is often perceived as a concept that is so inherently good that it should be pursued at all costs. Because of this, many a software complexity crime has been committed in the name of reuse.

One of my favorite examples of reuse abuse is the “common” project. We’ve all been tempted by the allure of “commonizing” or “genericizing” a large code base into a reusable library at one time or another. On the surface this seems like a reasonable proposal. After all, the code is already written. It’s simply a matter of creating the new project, moving the code and updating existing applications to use the common code. This is a classic “good idea”. Given infinite time and resources it’s probably the right call. Even a separate project plan would suffice. However, it seems these common projects and the refactoring they require are usually appended to the timetable of another project plan or cobbled together in someone’s copious free time. They tend to be side projects, sold under the ruse of being simple matters that will take little time. Perhaps this is because the people selling them believe this ploy. More likely, the salespeople know the primary benefit of such a solution is that it promotes code reuse and that’s a pretty hard sell to stakeholders. When was the last time you got a budget approved for a project that was strictly refactoring? You know you’ll never get dedicated time for the project so you try to squeeze time in for it elsewhere. This is where things go awry. This type of endeavor is a project in its own right. As enticing as it is, if you don’t have the time and resources to commit it’s probably more pragmatic to leave working code alone.

But what about the benefits of removing all of the duplication and only having one code base to maintain? Surely such an admirable goal warrants a rogue pet project. Ah, the temptation of the common project is great. It’s fun work and you can see the grand vision at the end of the tunnel. The draw of designing an elegant, reusable masterpiece tends to magnify perceived benefits and shroud costs. The benefits are real but the end doesn’t justify all means.

Common projects like to create issues of the cross-dependency and build variety. It’s surprising how quickly these non-sanctioned issues can start burning time that was allocated for your “on the books” project. It’s really no fun to explain that you have new issues or are behind schedule because you refactored working code so that all of the applications would share a common code base. Suddenly replacing the old data access code that was working for two years doesn’t seem like such a no-brainer.

From a business perspective, the best possible outcome is that all of the applications will work the same as they did before you started. That should be a red flag. If the biggest upside for the users is that no one will notice you should probably stop right there. Remember: the largest potential downside is something along the lines of “no one can log in”.

You may be betting on the fact that there will be new development in the future that can take advantage of your common code. You could be right, and depending on your level of certainty, creating the common project may be the right call. However, I would tend to defer implementing the common code until I know it’s going to be reused. Gambling your current project’s success on your ability to predict the company’s business needs is a risky bet. Even if you’re correct it’s not very likely you’ll get kudos for being Karnack the architect. It is highly probable however that PM types will be upset when they find out that developers are battling issues with functionality that’s not on their project plan.

At the end of the day, code reuse is good. Removing duplication and refactoring existing code into a common code base is good. However, these efforts aren’t free. Dissecting code and reassembling it into reusable pieces takes diligence. The flexibility it offers adds complexity to overall architecture, especially in the area of build and deployment. Sometimes it’s too idealistic to think that we can implement these concepts the instant we identify an opportunity to do so.