Anger Tax: The even darker side of technical debt

The concept of technical debt gets a lot of air time from architecture gurus like Ward Cunningham, Martin Fowler and Neal Ford. The cost incurred by sacrificing design for other project constraints such as time to market does indeed resemble financial debt in many ways. Like paying interest for an item bought on credit, the complexity created by shortcuts taken now impose an additional cost every time we are forced to work around them in the future. Unless we go back and “pay down” that technical debt, we get to enjoy that interest payment for the life of the project.

Technical debt isn’t necessarily viewed in the most positive light. Still, I think the world has been looking at it through rose colored glasses. The picture is always painted as though the architect or developer taking on the debt were making a conscious decision. As though they carefully weighed their options and came to the conclusion that the prudent decision was to take on some technical debt in order to capitalize on a business opportunity. If that were the case the cost of technical debt would be much easier to swallow because its existence would at least have been recognized and justified by someone. Let’s get real here; a lot of the people holding these debts are software development’s version of a college kid that thinks a credit card is free money. They’re not salty software veterans making calculated choices based on risks and rewards. They’re eager neophytes haphazardly throwing applications together using one hack after another.

This is the breed of technical debt that I truly despise. It may not be technical debt in the sense that someone knowingly took a shortcut based on some external force but ignorance doesn’t change the fact it is technical debt just the same. Perhaps those who coined the term would disagree with that, but it sure feels like I’m paying the same “interest” when I’m forced to deal with the resulting bad design.

Along with making future changes more difficult (interest) and the cost of refactoring to eliminate poor design (paying down the principle) that comes with all technical debt, this type of technical debt comes with a special hidden expense. Since it seems to be fashionable to talk about intangible project costs in terms of financial metaphors, I like to call this the Anger Tax. This is the time you spend being angry when you encounter this type of technical debt in the future. When design is compromised based on some sort of tactical business decision it’s a little more forgivable. When you’re called on to help establish a bailout strategy for a project that is failing under the weight of shameless, unmanaged technical debt you tend to look at the debtor like they're the president of AIG. Paying down their technical debt is angry work and that anger isn’t free. Every roll-your-own hack and framework misuse that irritates you is an interruption that negatively impacts productivity. These distractions make it difficult to stay pragmatic about refactoring as you are easily drawn into idealist thought tangents about how things could have been if only you were there before things went awry.

With all of that ranting out of the way, I must admit that I don’t really have any great answers for avoiding technical debt created via programmer negligence - I just like to complain about it. Perhaps requiring some sort of board exam or apprenticeship for architects would help but that's probably not very realistic. I probably shouldn’t be so angry about it anyway since a solution would be financially crushing for me; resuscitating poorly designed applications is a large part of what keeps guys like me working. While I may not have a cure, I do find ranting, sharing war stories and commiserating tends to lighten my mood on the subject - hence this post. I’m always up for hearing a good anger tax story so if you have one, feel free to vent here.

...and JiBX shall reign supreme

There are a lot of choices out there when it comes to XML data binding tools. JAXB, XMLBeans, Castor, JiBX and Xstream are a few that come to mind. I’ve used quite a few of these tools and I like different ones for different reasons. I’m a big fan of tools that focus on doing one thing really well. Serializing and deserializing XML seems like a problem space that is about the right size for such a tool but for some reason none of the technologies I’ve worked with in this area have been able to meet all of my needs.

JiBX is one such tool that comes as close as any that I’ve worked with to being the silver bullet for XML marshalling. For a few years now JiBX has been my marshalling tool of choice when I need to deserialize a lot of XML because it offers the best performance I’ve seen by far. However, this performance has always come at the cost of having to deal with running an XML binding compiler that embeds the binding code directly into your existing class files. This can put a hitch in your development giddy up because you have to run this binding compiler every time you change one of the classes that gets marshalled/unmarshalled. JiBX does support limited runtime binding for development but it is very limited - it’s not designed to work in a servlet engine or app server. That means the vast majority of us who work with web apps will find ourselves restarting our server every time we change a JiBX-aware class and forget to run the binding compiler. The small pain of having to re-run the binding compiler or re-start your server when you forget to run it is one of those annoying little issues that has a cumulative negative effect over the life of a project that is easy to underestimate. Because of this, JiBX has always left a little something to be desired for me and if performance isn’t an issue I opt for something that doesn’t require the extra step.

All of that changed a couple of weeks ago when I was browsing the JiBX documentation hoping that perhaps this last hurdle had been leaped. To my delight I found that it had. There is now an Eclipse plug-in that detects changes to the classes referenced by your JiBX binding definition and re-compiles the binding automatically. The only downside to this discovery is that I didn’t make it sooner; the plug-in has been available since late May of last year. Unfortunately this isn’t the type of advancement that gets a lot of publicity, you have to get annoyed enough to go looking for it. Although it’s a relatively small feature it is a really satisfying addition that adds a lot of value in my opinion. I’ve always found it a little frustrating that none of the XML marshalling tools I’ve worked with were quite good enough to become my de facto standard. By removing this last chink in the armor I can now say JiBX has claimed that title.

Tools Of The Trade: Spring Web Services

In general I think web services are way oversold. Especially when it comes to those of the enterprise SOA/ESB flavor. It seems like it has taken longer than normal for the hype surrounding web services to die down. This is probably because it’s a good area for vendors to push large “solution in a box” tools. The SOA vendors will probably be pushing their products for a while but I think the development community is reaching the point where web services have found their space of usefulness. The days of putting web services in front of every piece of software are fading away. We’ve landed in a place where web services are the right tool for certain jobs, particularly large-scale integration work. Since there are plenty of systems out there that need to talk to each other I think it’s safe to say that web services, in one form or another, are here to stay.

I thought web services would be a good topic for a “tools of the trade” post because it’s another area where there are many tools and technologies available. In addition to that, it’s a very complicated topic. The security alone is enough to make anyone crazy. In my experience, the best way to handle a complicated solution such as this is to avoid it all together. However, there are times when avoidance is not an option and web services are the right solution. In those situations I find the use of good tools can go a long way in helping to mitigate the necessary complexity involved with such a solution.

When it comes to traditional SOAP/WSDL based web services, most of the tools available have fallen short of my expectations. Just when I was beginning to think the terms web service and painful development were synonymous, along came Spring Web Services to save the day. First of all, it’s exclusively contract first. With Spring-WS your web service journey beings and ends at the main integration point – the XSD. This is a big selling point for me because the only situation where I really need web services is when I have to support multiple disparate clients. In those situations getting the service interface right is the most important part. Tools created to bolt web services on to an existing application by generating the contract from existing code have always seemed backwards to me. They create a contract that is tightly coupled to the implementation code. Such a contract is very likely to be difficult for your service clients to consume; those clients don’t speak the same language and that’s why you’re going through all the work of communicating in XML. In the contract first world you are forced to think about how clients interact with your service before it’s implemented. Focusing on the interface first tends to bring important contract issues to the surface quickly.

In addition to the advantage of being contract first Spring-WS also provides good support for the things you expect to find. The usual suspects like message based routing and security are all accounted for and it allows you to use just about any tool under the sun (no pun intended) to deal with the XML. It also passes my “as expected” rule. Things have reasonable names that explain what they do. Objects and configuration are where you expect them to be. The methods you hope to find are usually there and behave as expected. I think these simple things are a large part of what makes a framework easy to use and the software created using that framework easy to maintain. When you’re working with a complicated technology this aspect of a framework is even more important. Staying on top of these simple things takes diligence. It’s surprising how many frameworks don’t do a good job of it. Many seem to be more concerned with functionality and flexibility but what I’m really after is usability.

When it comes down to it, I think the decision of whether or not to use web services is probably more important than what technologies you choose to implement them with. If you don’t need to support remote clients it’s probably not a good idea to add web services “just in case”. If all parties involved talk Java you probably don’t need the overhead and complexity of web services. If you only need to support one remote client things could probably be kept simpler with a POX over HTTP approach. In many cases I think the RESTful web service approach, which Spring 3 MVC supports nicely, is a good alternative to the traditional approach. So there are a lot of things to consider before taking the web service plunge. However, if you’ve analyzed requirements closely and there’s no getting around the fact that you have to support multiple clients making RPC style calls, Spring Web Services is the best approach I’ve seen.

Reuse Abuse

We hear a lot about this elusive creature called reuse that the entire software development world seems to be in pursuit of. I won’t argue against the idea that reuse is a good thing because in theory it is. However, I think views regarding code reuse are often distorted. Reuse is often perceived as a concept that is so inherently good that it should be pursued at all costs. Because of this, many a software complexity crime has been committed in the name of reuse.

One of my favorite examples of reuse abuse is the “common” project. We’ve all been tempted by the allure of “commonizing” or “genericizing” a large code base into a reusable library at one time or another. On the surface this seems like a reasonable proposal. After all, the code is already written. It’s simply a matter of creating the new project, moving the code and updating existing applications to use the common code. This is a classic “good idea”. Given infinite time and resources it’s probably the right call. Even a separate project plan would suffice. However, it seems these common projects and the refactoring they require are usually appended to the timetable of another project plan or cobbled together in someone’s copious free time. They tend to be side projects, sold under the ruse of being simple matters that will take little time. Perhaps this is because the people selling them believe this ploy. More likely, the salespeople know the primary benefit of such a solution is that it promotes code reuse and that’s a pretty hard sell to stakeholders. When was the last time you got a budget approved for a project that was strictly refactoring? You know you’ll never get dedicated time for the project so you try to squeeze time in for it elsewhere. This is where things go awry. This type of endeavor is a project in its own right. As enticing as it is, if you don’t have the time and resources to commit it’s probably more pragmatic to leave working code alone.

But what about the benefits of removing all of the duplication and only having one code base to maintain? Surely such an admirable goal warrants a rogue pet project. Ah, the temptation of the common project is great. It’s fun work and you can see the grand vision at the end of the tunnel. The draw of designing an elegant, reusable masterpiece tends to magnify perceived benefits and shroud costs. The benefits are real but the end doesn’t justify all means.

Common projects like to create issues of the cross-dependency and build variety. It’s surprising how quickly these non-sanctioned issues can start burning time that was allocated for your “on the books” project. It’s really no fun to explain that you have new issues or are behind schedule because you refactored working code so that all of the applications would share a common code base. Suddenly replacing the old data access code that was working for two years doesn’t seem like such a no-brainer.

From a business perspective, the best possible outcome is that all of the applications will work the same as they did before you started. That should be a red flag. If the biggest upside for the users is that no one will notice you should probably stop right there. Remember: the largest potential downside is something along the lines of “no one can log in”.

You may be betting on the fact that there will be new development in the future that can take advantage of your common code. You could be right, and depending on your level of certainty, creating the common project may be the right call. However, I would tend to defer implementing the common code until I know it’s going to be reused. Gambling your current project’s success on your ability to predict the company’s business needs is a risky bet. Even if you’re correct it’s not very likely you’ll get kudos for being Karnack the architect. It is highly probable however that PM types will be upset when they find out that developers are battling issues with functionality that’s not on their project plan.

At the end of the day, code reuse is good. Removing duplication and refactoring existing code into a common code base is good. However, these efforts aren’t free. Dissecting code and reassembling it into reusable pieces takes diligence. The flexibility it offers adds complexity to overall architecture, especially in the area of build and deployment. Sometimes it’s too idealistic to think that we can implement these concepts the instant we identify an opportunity to do so.