To manage should not imply authority. Authority should land on the role who practices the decisions most.

Managers should be accessed on the accuracy of their reports and their ability to encourage collaboration when necessary.

To managers:
Set a good example for the team. Trust in the skills of your colleagues. Encourage collaboration. Involve others in decision making. Make informed choices. Innovative IT shops don’t base decisions solely on what worked yesterday; they continually scan the market for new technologies and ideas. Why shouldn’t you?

When I purchased Organizational Patterns of Agile Software Development I had a couple of things I wanted to sponge:

• Understand more about the structures of organizations and how they can nicely evolve.
• Learn more about the world of software development as managers see it.
• Gain an appreciation for the origins of XP.

It delivered on all points.

For my first two goals it provided reams of useful information on organizational healing and growth, and about the principles and practices of highly successful teams. For those wanting to know more, the book bases its core (a series of pattern languages) strongly on empirical evidence gained from analyzing highly efficient and successful organizations. It focuses on the values of these organizations and the product of these values – their structure and communications. It stresses the importance of organizational learning, the key to allowing organizations to remedy their dysfunctions. It presents some strategies organizational introspection and learning. As with most books I read the theory all seems sound, but this book goes deeper proving to you it works in practice by presenting successful use of the patterns. Often the authors provide insights that transcend software development, but that often seems to be the case for organizational/management theory. They also present plenty of antropological references that substantiate their theories and patterns, demonstrating stong background knowledge (for example Maslow’s heirarchy of needs).

As I read through the patterns, shortcomings of organizations I’ve worked for became very obvious. The patterns made paths to healing these organizations very clear. To me thats a sure sign this book is good value. While I’m usually a developer lacking the organizational influence to implement these patterns the book stresses you can plant the seed and play your part with your immediate team and colleagues.

On the origins on XP, it did provide some teasers. It cites Ward Cunningham’s Episodes Pattern Language work as providing ‘much of the foundation material for XP’. An early reviewer of the first published version of the patterns ‘would later go on to be one of the founders of Extreme Programming’ (anyone know why no name was mentioned here?). An early publication of the book ‘was an early influence on SCRUM’, a process familiar to all the ThoughtWorkers out there. So it certainly points you in directions where you can get more information about XP’s inception.

For those of you who have been disturbed with my passion for software design patterns in recent years, expect it worsen as our pattern conversations will now surely touch software and organizational design patterns (i.e. code and politics).

For those of you who have a passion for being a technician and staying that way, rest assured the book is targeted for developers, not just managers. The authors acknowledge their developer audience saying, ‘we predict that many that read this book will not be managers, but developers’. They even present some software design patterns that strongly align with the organizational patterns (although I wouldn’t recommend them).

Keeping true to the critical nature of my blogs though, I do have a couple of gripes. I raised on eyebrow when reading the authors ‘chose “Agile” for the title out of marketing concerns’. The patterns promoting an architect role did little to counter my animosity towards the role. The software design patterns presented seemed largely out-of-place and were very questionable designs, almost shoved in to convince the reader that the authors have been developers.

They also had a dig at XP on a few occasions, points I could see some merit in but largely disagree with:
Quote: ‘….if the organization adopts a policy of daily status meetings, it perpetuates the crisis mode or even creates a crisis mode…’. They go on to explain, ‘Even when not at work, the people will have work on their minds so they can be prepared for the next day’s status meeting.’
Rebuttal: Status/stand-up meetings streamline communications, allowing the team to engage in short and sharp conversations. The team also gets a collective picture of status allowing them to immediately identify and resolve blockages. These benefits are valuable to any collaborative team effort, crisis or no crisis. Regarding work remaining on the mind, reporting status becomes part of your natural rhythm of activities. I haven’t found it adds more stress to report on progress, any conscientious developer should know where they are at without expending any additional brain processing cycles.
Quote: ‘Other aspects of XP – such as the inability to work alone and the need to always have your thought processes open to a pair programmer – help sustain crisis mentality.’
Rebuttal: I see pairing as offering more opportunity for the collective knowledge of the team to be expressed in the code and to learn from others. I do not see it as a negative activity, a subtle way of assessing the adequacy of my domain knowledge or programming. Granted the less experienced are usually more suspicious of its merits.
Quote: ‘…focus on organizational learning distinguishes organizational patterns from other organizations such as XP, which, though rooted in principles, imposes those principles…’.
Rebuttal: XP has feedback, learning and evolution at the heart of its principles. XP doesn’t ‘impose’ principles, you progressively take what works most effectively for your context and culture, much like organizational patterns. Also, much like organizational patterns, they have individual benefits - benefits that strengthen and expand in symphony with other practices.
(Perhaps my instinct to defend XP indicates some close-mindedness on my part)

Regardless of these criticisms, I recommend anyone whose sponge wants similar information to what I craved to make the mental investment.

Who holds the architectural vision? Who implements architectural infrastructure? Should infrastructure development be segregated from application development? Successful projects have employed this approach. Successful projects also have dictatorial architects who hide the architectural vision and solely implement infrastructure. Are these bad approaches if so many projects successfully employ them?
Every developer architecting a little every day is a lofty concept I’ve encountered frequently in texts and been fortunate enough to practice on some teams. It answers some of these questions and presents some sound, democratic alternatives to segregated development and isolated, dictatorial architectural teams.

A Poisonous Distinction.
Beck’s White Book is one prominent text that explains the approach. Evans in Domain Driven Design also explores it, seeing a division of developers and architects as a ‘poisonous distinction’. I’ve been on many projects with this division and witnessed some of ailments it’s poison brings. Lets explore the severity of these ailments by considering Pro’s and Con’s of the division:

Pro’s

1. In a large team, a vertical slice of functionality can often be developed faster using a sub-team of developers.
2. Too many decision makers can often slow a process.

Con’s

1. Those not developing infrastructure are inhibited from exposure to tools and technologies. Their experiences and rate of professional development is constrained. Typically these developers end-up being skilled in proprietary frameworks, even if they are built on open source frameworks.
2. Those fortunate enough to perform architectural development are considered technically superior to other developers. Psychological divisions start to form complementing the usual physical divisions that ensue. Developers outside the group become envious.
3. The architectural group tends not to seek advice from outside it borders. A totalitarian mentality becomes prevalent, attempts at democracy often fail. Legitimate, often more rational alternatives and advice can be ignored in favor of personal preferences.
4. Pain experienced through client use of infrastructural API’s is often not felt until a heavy investment has been made and the cost of change is higher.
5. The bus factor rises. Knowledge is not spread, silos develop and dependencies on individuals grow.

Natural Tendencies Easily Prevail.
As much as management may try to limit the effect of the Con’s when they divide teams and employ a hard-line divide and conquer approach, natural tendencies prevail that instigate the Con’s:

1. Management takes a short-term approach to developing infrastructure and assigns the best available developers to complete the infrastructure in the shortest period of time. This usually flows from infrastructure being mistakenly considered best tackled through a “one-off, upfront task”.
2. Designing and developing infrastructure is perceived a fun job. It’s unfortunate technologists are traditionally viewed as geeks satisfied by the mental challenge of solving technical problems. Consequently developers strive to address technical problems above the problems of the business employing them.
3. Teams are naturally defensive of their collective opinions and decisions. A culture of division will cause these defensive walls to propagate. Cross-team communication can become strained on many levels discouraging important feedback, and making the task of integration that much more difficult.

Clearly, it’s hard to police human nature. Admittedly I’ve been fairly pessimistic, but with good reason. Too often I’ve seen team morale shattered by this seemingly logical division. So, by employing this approach you risk developing a poorer solution in a longer period of time, while creating physical and mental barriers and shattering team morale. All significant inputs to poor staff retention. But you don’t have to venture into this tenuous realm of human emotion – I say clear division, especially in terms of bodies, is not the answer. Sure clear leadership in decision making is necessary, and that is a division, but all developers should have a say in deciding and realizing architecture. I’ll get into the details of alternate approaches shortly. But before I get to these approaches I’d like to explore some problems arising from the natural tendencies of technicians.

Who Decides Infrastructure Features And Requirements?
Without accountability developers naturally tend to solve problems that interest them, not the business that employs them. If they can decide their own requirements, the business risks allowing them to develop unnecessary, unused features - they venture into the land of YAGNI. It is natural for technicians to become engrossed in technical challenges, striving to tackle what they consider interesting requirements or designs, losing sight of the business requirements their infrastructure should be fulfilling. Autonomous infrastructure groups certainly suffer from this aliment. Conscientious technicians should derive their technical requirements from business requirements in collaboration with the application development group. But this is not an easy thing to do and often they do not. A complex, co-ordinated effort is required that is even harder when infrastructure groups service large teams, or worse, many teams. Therefore business’ should never assume infrastructure groups act in their best interests. They should strive for ways to account for the activities of infrastructure groups, policing the legitimacy of their requirements, ensuring they address real business needs. But all this management overhead can be avoided in a unified approach. Without the division infrastructure is automatically more accountable to the business requirements – development will not start unless a business requirement fuels it.

Business Problems Are More Challenging.
I’ll ponder one more natural tendency before presenting alternative approaches. Why aren’t developers more concerned with addressing the needs of the business that employs them? In Brooks’ terms, infrastructure development is mostly focused on developing the accidental as opposed to the essential. Tackling the essence of software, business requirements, adds the most value. Tackling technical problems does less for the longevity of technicians, domains are guaranteed to exist longer than any technology. So why do it? Do you consider it fun? Perhaps more mentally challenging? I say understanding domains and developing expressive models and their conceptual contours must be at least equally challenging. I used to think developing frameworks was the pinnacle of development. They were the ultimate challenge. Thankfully my mindset has shifted to more holistic values – developing business functionality in as short as possible time employing a Model Driven approach that usually relies on infrastructure and application logic being developed in parallel. I find the challenges still abound, the greatest now come in developing an those expressive, deep models and discovering those contours I referred to earlier. The industry is after all increasingly demanding multi-faceted developers, competent in development, analysis and testing. So I say challenge that industry misnomer and strive to solve business problems.

An Alternative.
So, onto that alternative I’ve been mentioning. For those knowledgeable with XP this will be nothing new:

1. Sure technical leadership is important, I’m all for that. I see their primary responsibility as ultimate decision makers. They also communicate the architectural vision and ensure it is upheld. They should not, however, dictate architectures, designs or implementations. They should seek out ideas and input, sure they’ll have plenty themselves but they should be extremely open to others. Just like any other developer, they should also be developers of both infrastructure and application logic. They should not be the sole developers of infrastructure, on a covert operation to design and implement its functionality as they deem fit.
2. As much as possible, developers using infrastructure should also contribute to its development. As they use it they will develop refined ideas on the infrastructures most effective API. They should have the opportunity to feedback these ideas into the infrastructure. Their changes could fuel deeper insights into the infrastructure. Just like normal application development, developers can pair with technical leads or other developers to complete and refine features of infrastructure.
3. When work is about to begin on scheduled infrastructure requirements hold a meeting requesting input from all developers. Flesh-out designs, start discussing API’s, get everyone aware of and thinking about the infrastructure. Make it an open invite – those that don’t wish to attend have at least had the opportunity to attend. Importantly, don’t make concrete decisions about the API yet, and don’t code the complete API in one big-bang. Let the API grow as use demands it - when application functionality demands the use of some aspect of the API only then develop it. This is what I and others term ‘organic development’. It is an excellent means of ensuring development efforts are accountable to business requirements. If infrastructure is being added to an existing application, again let the infrastructure grow organically as new requirements or bugs demand code changes.

Conclusion.
So we still have a division of leadership, but there’s a strong concept of one-team. We’ve engendered buy-in, explored a wider range of solutions, spread the knowledge and flattened the organizational hierarchy somewhat by spreading the decision making abilities of the leadership group (I’ll explore this aspect in my next blog). We’ve also made infrastructure development efforts more accountable to business requirements. All top-notch goodness. So I say fight the division trend, spread infrastructure knowledge, let all contribute to it’s design and development in parallel to application development.