Archive for 2005

Every Developer Architects Every Day

{Wednesday, May 11th, 2005}

Who holds the architectural vision? Who implements architectural infrastructure? Should infrastructure development be segregated from application development? Successful projects have employed this approach. Successful projects also have dictatorial architects who hide the architectural vision and solely implement infrastructure. Are these bad approaches if so many projects successfully employ them?
Every developer architecting a little every day is a lofty concept I’ve encountered frequently in texts and been fortunate enough to practice on some teams. It answers some of these questions and presents some sound, democratic alternatives to segregated development and isolated, dictatorial architectural teams.

A Poisonous Distinction.
Beck’s White Book is one prominent text that explains the approach. Evans in Domain Driven Design also explores it, seeing a division of developers and architects as a ‘poisonous distinction’. I’ve been on many projects with this division and witnessed some of ailments it’s poison brings. Lets explore the severity of these ailments by considering Pro’s and Con’s of the division:

Pro’s

  1. In a large team, a vertical slice of functionality can often be developed faster using a sub-team of developers.
  2. Too many decision makers can often slow a process.

Con’s

  1. Those not developing infrastructure are inhibited from exposure to tools and technologies. Their experiences and rate of professional development is constrained. Typically these developers end-up being skilled in proprietary frameworks, even if they are built on open source frameworks.
  2. Those fortunate enough to perform architectural development are considered technically superior to other developers. Psychological divisions start to form complementing the usual physical divisions that ensue. Developers outside the group become envious.
  3. The architectural group tends not to seek advice from outside it borders. A totalitarian mentality becomes prevalent, attempts at democracy often fail. Legitimate, often more rational alternatives and advice can be ignored in favor of personal preferences.
  4. Pain experienced through client use of infrastructural API’s is often not felt until a heavy investment has been made and the cost of change is higher.
  5. The bus factor rises. Knowledge is not spread, silos develop and dependencies on individuals grow.

Natural Tendencies Easily Prevail.
As much as management may try to limit the effect of the Con’s when they divide teams and employ a hard-line divide and conquer approach, natural tendencies prevail that instigate the Con’s:

  1. Management takes a short-term approach to developing infrastructure and assigns the best available developers to complete the infrastructure in the shortest period of time. This usually flows from infrastructure being mistakenly considered best tackled through a “one-off, upfront task”.
  2. Designing and developing infrastructure is perceived a fun job. It’s unfortunate technologists are traditionally viewed as geeks satisfied by the mental challenge of solving technical problems. Consequently developers strive to address technical problems above the problems of the business employing them.
  3. Teams are naturally defensive of their collective opinions and decisions. A culture of division will cause these defensive walls to propagate. Cross-team communication can become strained on many levels discouraging important feedback, and making the task of integration that much more difficult.

Clearly, it’s hard to police human nature. Admittedly I’ve been fairly pessimistic, but with good reason. Too often I’ve seen team morale shattered by this seemingly logical division. So, by employing this approach you risk developing a poorer solution in a longer period of time, while creating physical and mental barriers and shattering team morale. All significant inputs to poor staff retention. But you don’t have to venture into this tenuous realm of human emotion – I say clear division, especially in terms of bodies, is not the answer. Sure clear leadership in decision making is necessary, and that is a division, but all developers should have a say in deciding and realizing architecture. I’ll get into the details of alternate approaches shortly. But before I get to these approaches I’d like to explore some problems arising from the natural tendencies of technicians.

Who Decides Infrastructure Features And Requirements?
Without accountability developers naturally tend to solve problems that interest them, not the business that employs them. If they can decide their own requirements, the business risks allowing them to develop unnecessary, unused features - they venture into the land of YAGNI. It is natural for technicians to become engrossed in technical challenges, striving to tackle what they consider interesting requirements or designs, losing sight of the business requirements their infrastructure should be fulfilling. Autonomous infrastructure groups certainly suffer from this aliment. Conscientious technicians should derive their technical requirements from business requirements in collaboration with the application development group. But this is not an easy thing to do and often they do not. A complex, co-ordinated effort is required that is even harder when infrastructure groups service large teams, or worse, many teams. Therefore business’ should never assume infrastructure groups act in their best interests. They should strive for ways to account for the activities of infrastructure groups, policing the legitimacy of their requirements, ensuring they address real business needs. But all this management overhead can be avoided in a unified approach. Without the division infrastructure is automatically more accountable to the business requirements – development will not start unless a business requirement fuels it.

Business Problems Are More Challenging.
I’ll ponder one more natural tendency before presenting alternative approaches. Why aren’t developers more concerned with addressing the needs of the business that employs them? In Brooks’ terms, infrastructure development is mostly focused on developing the accidental as opposed to the essential. Tackling the essence of software, business requirements, adds the most value. Tackling technical problems does less for the longevity of technicians, domains are guaranteed to exist longer than any technology. So why do it? Do you consider it fun? Perhaps more mentally challenging? I say understanding domains and developing expressive models and their conceptual contours must be at least equally challenging. I used to think developing frameworks was the pinnacle of development. They were the ultimate challenge. Thankfully my mindset has shifted to more holistic values – developing business functionality in as short as possible time employing a Model Driven approach that usually relies on infrastructure and application logic being developed in parallel. I find the challenges still abound, the greatest now come in developing an those expressive, deep models and discovering those contours I referred to earlier. The industry is after all increasingly demanding multi-faceted developers, competent in development, analysis and testing. So I say challenge that industry misnomer and strive to solve business problems.

An Alternative.
So, onto that alternative I’ve been mentioning. For those knowledgeable with XP this will be nothing new:

  1. Sure technical leadership is important, I’m all for that. I see their primary responsibility as ultimate decision makers. They also communicate the architectural vision and ensure it is upheld. They should not, however, dictate architectures, designs or implementations. They should seek out ideas and input, sure they’ll have plenty themselves but they should be extremely open to others. Just like any other developer, they should also be developers of both infrastructure and application logic. They should not be the sole developers of infrastructure, on a covert operation to design and implement its functionality as they deem fit.
  2. As much as possible, developers using infrastructure should also contribute to its development. As they use it they will develop refined ideas on the infrastructures most effective API. They should have the opportunity to feedback these ideas into the infrastructure. Their changes could fuel deeper insights into the infrastructure. Just like normal application development, developers can pair with technical leads or other developers to complete and refine features of infrastructure.
  3. When work is about to begin on scheduled infrastructure requirements hold a meeting requesting input from all developers. Flesh-out designs, start discussing API’s, get everyone aware of and thinking about the infrastructure. Make it an open invite – those that don’t wish to attend have at least had the opportunity to attend. Importantly, don’t make concrete decisions about the API yet, and don’t code the complete API in one big-bang. Let the API grow as use demands it - when application functionality demands the use of some aspect of the API only then develop it. This is what I and others term ‘organic development’. It is an excellent means of ensuring development efforts are accountable to business requirements. If infrastructure is being added to an existing application, again let the infrastructure grow organically as new requirements or bugs demand code changes.

Conclusion.
So we still have a division of leadership, but there’s a strong concept of one-team. We’ve engendered buy-in, explored a wider range of solutions, spread the knowledge and flattened the organizational hierarchy somewhat by spreading the decision making abilities of the leadership group (I’ll explore this aspect in my next blog). We’ve also made infrastructure development efforts more accountable to business requirements. All top-notch goodness. So I say fight the division trend, spread infrastructure knowledge, let all contribute to it’s design and development in parallel to application development.

Exception Cause or Effect?

{Monday, May 9th, 2005}

I’ll have to relent on my previous post, exceptions is not a dead topic after all. I’ve recently encountered some mis-uses and nuances that warrant discussion. Consider the exception handling construct:

reserveResources();
try {
    performLogic();
} finally {
    releaseResources();
}

It’s a classic that is particularly common when performing JDBC operations, the resources in this case being PreparedStatements, Connections and Transactions. Granted, the catch block can play an important role, but let’s ignore that for now.

Consider this scenario; logic failure resulting in an exception, then resource release failure. What exception is propagated – logic failure or the more incidental releasing of resources? Mr.Finally will always win that battle unless you declare otherwise.
It’s staggering the amount of times I’ve seen this oversight in practice. This approach is, of course, very acceptable if the finally exception has greater importance, but that is a rare situation. Too often, especially with non-TDD approaches, developers place exception handling in the back of there mind, exception flows are usually not considered until normal flows are codified. Staggeringly, in some cases I’ve seen code realized in this order as well – code the normal stuff, then add exception handling later. Resist this urge! Exceptions can and should be considered more fundamental aspects of an objects behavior.

So what’s the best approach. Let’s see….

reserverResources();
Throwable originalException = null;
try {
    performLogic();
} catch (Throwable throwable) {
    originalException = throwable;
} finally {
    try {
        releaseResources();
    } catch (Throwable throwable) {
        if (originalException == null) {
            originalException = throwable;
        } else {
            log.error(“Exception releasing resources.”, throwable);
        }
    }
}
if (originalException != null) {
    throw originalException;
}

Or perhaps this:

reserveResources();
try {
    performLogic();
} finally {
    try {
        releaseResources();
    } catch (Throwable throwable) {
        log.error(“Exception releasing resources.”, throwable);
    }
}

In each case the logic exception is preserved. Before I explore the pro’s and con’s of each approach, lets revise some best practices:

  • Process exceptions once and only once. Processing typically involves logging and presenting a message to the user.
  • Minimize generic catch blocks, such as catch (Exception exc) or catch (Throwable throwable), reducing the probability of gobbling exceptions that would otherwise be propagated.
  • Treat exceptions consistently to facilitate maintainability, just like any cross-cutting concern. At its best consistency is achieved through centralized logic, so centralize exception handling - a simple example is web application error handling via the web.xml element declarations. I’ve seen plenty of violations of this rule, logging exceptions at multiple points – the classic violation being logging exceptions at their origin, promoting code scattering.
  • Propagate at most one checked exception (see my previous blog on this principle). The exception should have meaning in the context of the method – akin to Domain Driven Design Intention-Revealing Interface principles.

With these practices in mind let’s consider the revised approaches:
Approach One

  1. Multiple violations of minimizing generic catch blocks.
  2. Code is relatively verbose – a minor point, but a point nonetheless.
  3. The exceptions are not re-thrown from a line indicating where the exception occurred (they are now both thrown from the lower if block), so the stack trace will slightly less communicative.

Approach Two

  1. Violation of minimizing generic catch blocks.
  2. The exception in the finally block is never propagated. Clients to this code will not be aware of any resource release failures, potentially losing critical information influencing subsequent processing or presentation to the user.

The first approach certainly seems the better of the two evils, but we’ve violated the generic catch block principle in each case. Mr. Checkstyle will explode with the appropriate settings.

IllegalCatch is not legal in all circumstances.

Let’s consider this check in more detail, it minimizes the chance of inadvertent exception consumption. It’s a safeguard for silliness. It is by no means a safeguard for something there is no good reason for doing, like not declaring a class with purely static behaviour as final.
It’s more compelling to violate this principle when developing infrastructure, but it holds as a hard and fast rule for standard application logic. In light of this I’ve changed my stance on this check:
Prevent inadvertent exception consumption in application logic using Checkstyle. For infrastructure development, rely on the pair or a review to prevent inappropriate generic catch blocks. This could be better realized by limiting the applicability of this check to certain packages – unfortunately Checkstyle’s IllegalCatch check does not support this.

I’ve seen a work around to this problem (aka hack) by employing a control flag approach:

int success = 0;
try {
    doSomething();
    success = 1;
} finally {
    if (success == 0) {
        // Exception occurred
    }
}

Clearly this code is not as communicative and ventures to the days of languages void of exception constructs. Still, it’s another evil that you need to weigh into the equation determining which approach is best, but it’s an option that wouldn’t be considered if IllegalCatch was more flexible.

Excessive Exceptions

{Thursday, March 24th, 2005}

From my experience, plenty of developers are still confused about how to nicely use exceptions. I find this staggering given the mass of flame wars over the years on kosher exception use. Simply put, some people just need to be force fed - the drone community is alive and well. But, back to the big debate. When to check and when to uncheck? Given the confusion I’ve seen reign recently, it’s about time I at least started a tiny spot fire in the war.

Once upon a time I used to think this it was a clear good vs. evil battle (in that order) – no exceptions (pardon the pun). But Spring somewhat twisted my point of view. Unchecked’s are an excellent alternative for offering clients the option to handle an exception. As Spring suggests, this is especially useful in the case of fatal, unrecoverable exceptions. Remember, unchecked exceptions can be declared in throws clauses too, so clients can somewhat expect the exception by perusing the method signature. That’s something to consider, but I don’t recommend it. It is a very human process requiring too much discipline to enforce and can yield too much clutter to warrant the effort, especially in highly cohesive code with deep invocation chains, and goes against Checkstyle’s Throws Count check. As a parting word on the war of checked vs. unchecked, perhaps it draws parallels to another old war on strongly typed versus dynamic languages, with checked similar in nature to strongly typed and unchecked similar to dynamic. Some want the immediate feedback of the compiler, others consider this step more of an unnecessary burden. In the case of exceptions the burden is being forced to handle exceptions at layers that can’t do anything about it.

But the big war I’ve been fighting recently is on the number of acceptable declared exceptions per method. My preference is one per method, Checkstyle’s default. In recent times I’ve cringed at code with as much as 10 checked exceptions per method. I’m accustomed to complaining about standard JDK methods with what I consider excessive exception contracts. Reflection exceptions are a good example with one method invocation potentially causing a IllegalAccessException, IllegalArgumentException or an InvocationTargetException. I never care which exception occurred – I handle these exceptions the same way. But that’s only 3 exceptions. 10 is a completely different ball-game. The side-effects are repetitive, messy client code, confusion reigning on kosher use of the method, and nightmarish unit testing as it’s difficult to exercise all exception conditions. Indeed, this type of exception misuse has all the evils that using control flags to govern execution flow brings. As a client forced to use this code, you can pray for an alternative or write one, but that violates the Once and Only Once rule. Better is writing a proxy insulating you from the exception handling cruft. Better still is refactoring, but I find these smells emanate on the bio-hazard scale where, without a significant investment, all you are likely to achieve is scratching the surface of the code with a vein hope of someday achieving your refactoring goals.

As always, it’s best to educate and fix the problem at it’s source somewhere in the mix. So how do you argue the nuance’s of kosher exception practices with someone that doesn’t know any better? Surely key to any sound argument on this is general OO modeling best practice. Strive to create classes with very distinct responsibilities and logical methods with clear single causes of failure. Having 10 exceptions in a method is certainly possible if you’ve written an OO script thats a couple of hundred lines long. Yeah, sure it’s possible. But if that’s your way of thinking, OO is not for you and you probably should be using a language that doesn’t have the concept of an exception. Try C instead. Better still, try joining these clowns.

As an example of a nice OO approach, consider the needs of a graphics application that allows the user to draw shapes. Consider this method in the ShapeRenderer interface;

void render(Shape shape) InvalidShapeException, CircleOutOfScreenBoundsException, ShapeOutOfScreenBoundsException, IllegalShapeDimensionException, ShapeAlreadyRenderedException, UnrenderableShapeException;

vs. this method;

void render(Shape shape) throws RenderingException;

It’s blatantly obvious which one is more understandable, decoupled, and which one clients will take a liking to. A common argument I hear defending masses of exceptions is ‘I want the user to receive a different error for each exception’. Fine - there are far cleaner ways of achieving this goal. For instance centralized converter infrastructure responsible for converting failures to meaningful messages is a good example. What you use as the key to your conversion certainly does not have to be the exception class. Sure it can play a role of great importance, especially with logical exception hierarchies, but exceptions are objects that can contain any information. Use that to your advantage.

Another argument I often hear is ‘Why not just throw Exception or Throwable? That’s one exception’. Consider the side-effects; clients lose context as it communicates the method can fail for any reason and exceptions can be inadvertently gobbled. Indeed this strategy has lead to Checkstyle’s Illegal Catch check being invented.