Offshore Agile Software Development: Does It Work?

Due to the ever-rising demand for seasoned software developers in the nineties, offshore software development became a compelling alternative to in-house development for many organizations. Despite the cultural, language and time differences and the geographical distance involved, more and more projects were executed with offshore development and testing, benefiting from lower rates of cost and the high availability of people, and where necessary, the ability to have teams working around the clock.

Early offshore

Particularly during the early years of offshore software development, the majority of projects were executed using rather traditional, Waterfall-style approaches. Projects were characterized as fixed-price and fixed-date. The requirements and the design were compiled onshore, after which coding and testing was done in another part of the world, be it Eastern Europe, India or South America.

The Waterfall model typically involves executing each of the activities in software development consecutively, and only once – requirements, analysis, design, code, test and deployment – to deliver a product based on the completion of each of these project milestones. Interestingly, this model has led to high failure rates in projects, even without offshoring some of the activities. At each milestone knowledge is lost and testing is executed very late – only when development is done – with exponentially rising costs of repairing bugs. Moreover, achieving completeness in requirements early on in a project is difficult and changes to requirements are costly.

Oddly enough, even with these anomalies, and even with failing local projects, many organizations still ventured into offshore software development applying the Waterfall model. Not surprisingly, many of these projects failed to deliver on time or on budget and did not deliver the required functionality. Despite the hoped-for benefits of lower costs and high availability of skilled people, offshore projects add another level of complexity due to more complex control and coordination, and because of language, cultural and time zone discrepancies.

Is Agile more suited for offshore than Waterfall?

So with Agile approaches, such as Scrum and Kanban, reaching the peaks of their popularity, an interesting question is: can Agile approaches and techniques overcome some of the shortcomings of offshore Waterfall development?

In short, Agile approaches are characterized by working in short iterations, where during each iteration a number of continuously re-prioritized work items are fully realized by a multi-disciplinary team, usually applying a similar life cycle per work item – requirements, analysis, design, code, test and deployment – as Waterfall uses over a whole project.

At the start of an Agile project the requirements are only identified, and not compiled into full detail. This list of requirements is known as the backlog, and is not designed to be complete. Rather, items from the backlog are elaborated on during iterations (also known as sprints). So all the real work is done during iterations. As a consequence, Agile teams are required to be multidisciplinary, and work together on a daily basis to implement functionality work item by work item. As such, co-location of teams, quite often at the client, works best in Agile.

So is Agile more suited than Waterfall for offshore software development? Clearly there are huge benefits. Requirements, analysis and design needn’t be finalized until a work item is actually implemented. So there is no grand, fixed, inflexible design decided upfront. Testing and deployment also take place immediately after coding the individual work items, not only at the end of the project. The total life cycle of a work item is usually no longer than a couple of days, instead of the whole project duration. That is why Agile works well in domains where it is accepted that requirements are never complete and might change, which is the case in the vast majority of projects worldwide.

So what happens when Agile approaches and techniques are applied to offshore software development to overcome Waterfall shortcomings? Apart from the apparent benefits, applying Agile to offshore also comes with consequences. Applying an Agile approach will involve close collaboration between all roles, whether on-site or offshore. It will also involve daily communication, and the ability to work on the same work item at the same time. Communication is therefore key in Agile projects, and as we all know, distance makes communication harder. Therefore offshore Agile teams need to be able to rely on other means of communication than on-site teams. Moreover, it is key that information is clear and  unambiguous, which is difficult as this is exactly the bottleneck that Agile is trying to overcome, as work items are not elaborated on until implemented.

But despite some of the difficulties involved in offshore Agile projects, particularly in highly complex domains or around regulatory sensitivities, Agile approaches have the potential to offer many benefits. In my opinion, it almost goes without saying that offshore Agile is going to be more effective to most organizations than offshore Waterfall, but only when key issues around communication, overheads and language issues are overcome.  I will explore some of the key strategies for making offshore Agile software development work in my next post.


This post was also published on the ISD Connect website at:

Offshore Agile Software Development: Does It Work?

Agile anti-patterns. Yes you agile projects can and will fail too

Over the years I have noticed a lot of agile anti-patterns during projects. Wrongly used agile approaches, dogmatic use of agile approaches, agile-in-name-only. Recently I have presented a talk at a number of agile and software development conferences that demonstrates patterns of agile misuse. These conferences include Agile Open Holland (Dieren), Camp Digital (Manchester), GIDS (Bangalore), ACCU (Oxford) and Jazoon (Zurich). Anyway, here’s the slide deck. Enjoy.

Evolving agile

Without any doubt agile is the biggest evolution in software development approaches since the introduction of waterfall back in the early seventies. And yes. Agile is an evolution rather than a revolution. The best practices and techniques in agile didn’t just pop-up. Rather they emerged from years of hard-working, real-life experience in succeeding and failing in projects.

So working in short iterations, in multi-disciplinary teams, prioritizing our work items regularly, and testing and delivering early and frequently using simplified communication, as we could capture agile in a single sentence, are improvements we’ve all introduced in our software development projects over the past forty years. Agile, and all of it flavors, such as Extreme Programming, Scrum, Smart or Kanban, have merely evolved from teams trying to do a little better every day. And have succeeded in doing that.

In the second decade of this century, due to the overall success of agile approaches, the majority of organizations executing projects will move towards agile. Either quickly or slowly. And for a variety of reasons. To raise quality. To increase predictability. To execute on-time, on-budget. To minimize time-to-market. Or simply because Gartner says so.

But with the success of agile come new challenges. Agile projects require agile coaches. Whether certified as Scrum Master, Lean Champion, or Pokémon Trainer, there is a lack of sufficiently experienced agile coaches. As a result an increasing number of projects is guided by ill-experienced coaches, often leading to a dogmatic and rigor application of agile. More over large organizations will feel the need to standardize, industrialize or institutionalize their agile efforts, resulting in ill-digestible, comprehensive agile manuals.

Although adaption and creativity are fundamental to agile, these growing pains will slow down the enormous progress we are making in the field of software development approaches. Agile is meant to evolve. Always. Continuously. Improving and professionalizing our software development is not a frog-leap process, proceeding with one giant leap every fifteen years. It is an fluid, never-ending process of many, many little steps taken on a daily basis. We learn from what we do. Yesterday, today, tomorrow, next year.

This post was written as a contribution to the upcoming book “Shaping Apps Germany”, published by Capgemini, Germany.

Death by Dogma versus Agile Assembly

On November 3, 2011 I presented the keynote of the Agile Open Holland Conference in Dieren. During this challenging talk I discussed the current state of affairs in agile organizations and projects and the effects of the recent strong rise in popularity of agile approaches. Let’s put it mildly: there’s a lot of work to be done.

Death by dogma

Almost all organizations, large and small, are turning towards agile to escape failing traditional software development projects. Due to this strong increase in popularity of agile approaches and techniques, many newcomers will enter the field of agile coaching. Of course without the very necessary real-life experience but proudly waving their Certified Professional Scrum Master Sensei Trainer Certificate proving they at least had two days of training.


Going through the hardship of two whole intense days of training becoming a Certified Agile Jedi Knight is worthwhile!

In my opinion, as a result many organizations and projects in the next couple of years will be coached why well-willing consultants who have barely made it through boot camp, and in the famous Shu-Ha-Ri learning model haven’t yet made it beyond copying their teacher. This will lead to very dogmatic applications of the more popular agile approaches, mostly Scrum, especially when the so-called leaders in the field themselves turn to dogma. This dogmatic thinking will block the use of more mature techniques and technology in agile projects, even when these would really improve projects, or would prevent agile projects from failing. “No, you can not do modeling in Scrum” and “Burn-down charts are mandatory” are two such simple real-life example statements that I’ve witnessed some certified agile Jedi Knight make. Due to this lack of experience and the growing dogmatism in the agile beliefs, more and more agile projects will fail. Death by dogma.

During my keynote I discussed many examples of dogmatic Scrum implementations and the drawbacks from being dogmagile, building the story up from my previous posts Scrumdamentalists and Crusaders and Flower-Power Agile Fluffiness.

But maybe even more important during the keynote I also show that there is no such thing as one-size-fits-all agile. Different organizations and different projects require different agile approaches. Sometimes lightweight agile, for instance implemented using Scrum, user stories, simple planning, simple estimation, and one co-located team using post-its on a wall is just fine. But in many projects the way of working used should rather be built up from slightly more enterprise ready approaches, for example using Smart, smart use cases, standardized estimation, multiple distributed teams and on-line dashboards.

What is agile anyway?

Implementing agile in your projects starts with establishing what it means to be in an agile project. As I demonstrate in the keynote, I consider short iterations, collaborative teams, a small unit of work, continuous planning, delivering early and often, and simplifying communication to be crucial to considering a project working in an agile way.

From there you can pick and choose from a wide variety of approaches, techniques and technology. Most of them stemming from the agile era, but some of them can also be related to older (or more mature) era’s. Concluding you might say that to be successful in implementing agile in your organization, you will need to assemble your agile approach from everything that will help you implement these six agile bare necessities. Anyway, enjoy!

Flower-Power Agile Fluffiness

To all the dear people in the agile community and to the faint-hearted: this will not be an easy blog post. There was a time when being a software developer was a decent craft, requiring decent craftsmanship and yes also a lot of creativity, some communication, some collaboration. Still it was a decent craft. The waterfall-ish methodologies we used weren’t extremely optimal, but at least software development was a craft. Similar to a carpenter who uses his tools to craft new furniture, or a industrial designer using his tools to come up with a new model Toyota – I know this is not the best example, but at least I now have the attention of the kind folks in the lean community. And then came agile.

Now believe me, I don’t have anything against agile. I’ve been promoting agile and iterative approaches to software development since the mid-nineties, and haven’t done traditional projects ever since. Agile used to be about engineering. We were improving ourselves by using better techniques, continuous integration or continuous deployment, writing unit test, pair programming, writing smart use cases or user stories, using a bug tracker, burn-down charts and even post-its on the wall. So far, it’s still all in a days work. There was a time that as an analyst, a developer, or a tester you could be proud of being in an agile project.

But these days if I look at what’s going on at agile conferences, on twitter, in blog posts, literature and discussions on agile, Scrum, Lean, Kanban and whatever new flavors of the month are passing by, I get the feeling I’m no longer talking about craftsmanship but rather ending up in Disneyland or in San Francisco in the late sixties. I’ve got a feeling were not in Kansas anymore.

image
Agile coach at work.

Agile community anti-patterns

Certainly it’s a good thing everybody can join the agile community. But I witness a lot of repetitive behavior I strongly discourage. Let’s name this repetitive behavior agile community anti-patterns – not to confuse with agile anti-patterns. The latter merely describe failures in agile projects, and yes these do occur, while the former describe community failures. Let me sum some up for you – while on the fly breaking my first anti-pattern:

  • Metaphorizing
  • Zenify
  • Kindergarten Agile
  • Open Door Wisdom
  • Scrumdamentalism

Allow me to elaborate a bit on these agile community anti-patterns.

Metaphorizing

Although I’m not sure metaphorizing is a even good English word, I’m quite sure you get the meaning of it. Everything anybody comes up with these days about agile – or about what people think is agile – is immediately turned into a metaphor or is given a silly name.

Can we please stop talking about the Gemba Walk when we mean that it’s a good thing our manager stops by every now and then? This shouldn’t even have a name.

image
Japanese manager stopping by.

What does it mean when an agile specialist states that “you should verify the five why’s with the reverse loop”? And what about Feature Injection? According to a recent tweet “feature Injection is more about using examples to discover the relationships you need and missed or got wrong.” Call me old-fashioned but I totally miss what this is about.

Zenify

Yes, I know lean manufacturing started in Japan at Toyota. So there is a link between agile and Japan. But is that an argument to zenify software development?

image
Our new Feng Shui office space.

Why do we need to explain roles in a software development project as samurai, sensei or roshi. I thought product owner and agile coach were already abstract enough. What about re-arranging our office in a Feng Shui manner? Also the word kaizen seems to become very popular. Quoting a recent tweet: “Just write down small things on small papers. It’s your kaizen.” Although I’m all for small things what does this mean and why do I need to introduce this in my project?

Kindergarten Agile

Not sure about the average level of maturity of people in agile projects around the world, but in the projects I’ve been in the last decade or so, people where pretty mature. So why is it groups of sensible people at unconferences are discussing the Happiness Index of projects? To me this sounds much more like a weekly column in a women’s magazine than a solid engineering practice in a software development project.

And I for one certainly don’t want to have to pass a baton if the flow in my stand-up meeting or scrum isn’t achieved automagically. Still this was seriously recommended by one of the authorities in the agile community during a recent presentation at a conference.

Some tweets to further illustrate my point? If someone says “Add Ready for Celebration before the Done column on your wall board” should I start decorating the office? It makes you wonder how often his projects get things done. Even worse is “Make sure you don’t miss the agile elephant versus the waterfall elephant in the lobby.” which was tweeted from a recent agile conference. Where was this conference held? At the Toys-R-Us?

image
Participants at a recent agile conference strolling down the exhibition hall.

Open Door Wisdom

Often I see quotes coming from the agile community that are no more than open doors, and have been open doors in projects for decades and perhaps centuries, but that are treated by others in the community as sources of new and ultimate wisdom.

Recently a speaker at an agile conference claimed that “if your retrospectives don’t add value to your project, you should change your retrospectives.” Duh. The speaker got a loud applause from the audience, he is now considered an absolute guru, and his quote got tweeted and re-tweeted all over the world. This is not even an open door, it’s an open gate.

Scrumdamentalism

Now I’ve blogged about Scrumdamentalism before, but with the newer generations of agile converts, some communities are getting more and more populated by religious zealots who will treat their newly gained faith with deep fundamentalism. Any best practice from their own belief is treated as mandatory, while followers from other beliefs are often considered heretics. A recent blog post states that: “A technical project manager can be a good product owner if he sticks to managing the product backlog and abiding by the rules of Scrum.” I wasn’t aware that Scrum even had rules. You learn something new everyday.

And if Scrumdamentalism alone isn’t bad enough already, it is even enforced by the so-called leaders themselves, proven by the following horrible quote “the team needs to listen to god talk and follow the commandments” from one of the Great Leaders in this particular community. Dear agilists there is is no one-true-belief. There’s value in all of them. And also can we please abolish the function title Agile (or Scrum or Kanban or Lean) Evangelist. Moreover, people calling themselves Agile Sensei should be banned from conferences and projects if you ask me.

Flower-Power Agile Fluffiness

Please people can we stop adding all this new age Flower-Power fluffiness to agile. In my opinion the agile community with all it’s great ideals and best practices is slowly degrading into a worldwide Luna park. My guess is I that it won’t be long before someone somewhere will suggest to add a psychotherapist to every software development project. “How do you feel about not being able to get your user story to the Ready for Celebration column?” Or plan a clown’s visit during retrospectives to increase the Happiness Index.

image
Agile retrospective with product owners present.

We are slowly becoming the laughing stock of the engineering world. I long for the time that we re-introduce engineering in our trade, and all go back to work again.