CHAPTER 3 (continued) : 3.4 Collaborating

3.4 Collaborating

Introduction

One of the values of the agile manifesto states: “Customer collaboration over following a contract” being completed then in the principles by: “Business people and developers must work together daily throughout the project”. It is very clear that day by day collaboration, between team members or between programmers and customers can deliver better software and, as a result, close collaboration stands at the base of all agile methodologies.

Why is collaboration important?

Building software is a continuous learning process between the customers that know the business constraints and the programmers who can develop the software. The customers must learn about software, about how software can be built, about what can be done and what can’t, about the cost of the different pieces of functionality because he is the one to “steer” the software direction to what he needs for his business. To build the right software, the programmers must learn about the business domain of the customer. This learning can only be done by working together daily.

Robert C. Martin said in “Agile Software Development, Principles, Practices and Patterns” [] that there is this tendency for managers to tell the programmers what needs to be done then go away for as long as they think it is necessary for the software to be built, then come back and find everything working. This tendency is in human nature, and it exists also in the case of the customers; however it was proven over and over that having everything written down in a functional specification document, signed off at the beginning can lead to misunderstanding the software’s purpose, from the client point of view.

Fear

Delivering software that fails to comply with the business needs of the customer contributes to building an atmosphere of distrust between software developers and the customers. This lack of trust is then tried to be compensated by very detailed contracts on what can and what cannot be done in future projects.

Customers are afraid that

• They won't get what they asked for.

• They'll ask for the wrong thing.

• They'll pay too much for too little.

• They must surrender control of their career to techies who don't care.

• They won't ever see a meaningful plan.

• The plans they do see will be fairy tales.

• They won't know what's going on.

• They'll be held to their first decisions and won't be able to react to changes in the business.

• No one will tell them the truth.

Developers are afraid, too. They fear that

• They will be told to do more than they know how to do.

• They will be told to do things that don't make sense.

• They are too stupid.

• They are falling behind technically.

• They will be given responsibility without authority.

• They won't be given clear definitions of what needs to be done.

• They'll have to sacrifice quality for deadlines.

• They'll have to solve hard problems without help.

• They won't have enough time to succeed.

Unacknowledged Fear Is the Source of All Software Project Failures

If these fears are not put on the table and dealt with, then developers and customer each try to protect themselves by building walls.

[Fowler , Beck – Planning Extreme Programming, 2001]

Contracts

Protecting by building walls, usually means contracts. Trying to replace trust and close collaboration with contracts is not solving the problem, but amplifying it. Mary and Tom Poppendieck describe why trust cannot be replaced very well with samples from car manufacturing and from software, enumerating the different types of contracts and showing the flaws they have when it comes to “enforcing” trust.

Can the parties involved in a software project as buyer and as vendor collaborate efficiently without each following their own interests? Because of this great fear, contracts are signed and expected later to replace trust between the companies involved on each side. If as quoted in Lean Software Development, trust means: “one party’s confidence that the other party … will fulfill its promises and will not exploit its vulnerabilities”, then contracts are considered by very many people, a way to enforce this.

Let’s study a little the contract types, used mostly between software companies and examine how trust can be enforced by them or how they can obtain exactly the opposite because of the contracts:

1. Fixed price contracts: The most commonly used type of contract, seems to give the customer the faith that the cost won’t be bigger then the one planned, but this means that the burden is moved on the shoulders of the software vendor, who will try to find different ways to compensate its eventual losses (if any) by overcharging for changes. They tend to push an atmosphere of distrust between the parties, each trying to follow their interest and thus favoring the failure of the software projects
2. Time and materials contracts: this type of contract deals better with unpredictability and uncertainty but simply shifts the risk from the vendor to the buyer. The longer the contract takes to complete the more the customer will pay, encouraging in a way the vendor to charge more for the work. The customer is always on the pressure that the things might be done faster and this favors ,just as the fixed price contract, an atmosphere of distrust
3. Multistage contracts: these types of contracts try to merge together the fixed price and time and materials contract types benefits, minimizing the risks. Usually a big fixed priced contract is divided into a multiple succeeding stage contracts governed by another contract. There is a certain risk in this approach also: if after each small stage contract is delivered, the contract is fully renegotiated, then it can become very expensive and it will not bring the two parties close together.
4. Target cost contracts: this type is the hardest to put in practice, but the most fair for both parties as well, which means that it favors trust. They are structured in such a way that the total cost, including changes – is a joined responsibility of the customer and vendor. This enforces them to collaborate and take decisions together, negotiating so that the target cost is met.

Although, some types of contracts can be seriously flawed when it comes to software development and to building programs that have value for the customers, it has been proven that they can be adapted to work well in software as long as they do not try to replace trust. Agile methodologies tend to favor the types of contracts that are aimed at building trust rather than building distrust. Robert C. Martin gives a very good example of an agile contract:

As an example of a successful contract, in 1994 I negotiated a contract for a large, multi-year, half-million-line project. We, the development team, were paid a relatively low monthly rate. Large payouts were made to us when we delivered certain large blocks of functionality. Those blocks were not specified in detail by the contract. Rather the contract stated that the payout would be made for a block when the block passed the customer’s acceptance test. The details of those acceptance tests were not specified in the contract.

During the course of this project we worked very closely with the customer. We released the software to him almost every Friday. By Monday of Tuesday of the following week he would have a list of changes for us to put into the software. We would prioritize those changes together, and then schedule them into subsequent weeks. The customer worked so closely with us that acceptance tests were never an issue. He knew when a block of functionality satisfied his needs because he watched it evolve from week to week.

The requirements for this project were in a constant state of flux. Major changes were not uncommon. There were whole blocks of functionality that were removed, and others that were inserted. And yet the contract, and the project, survived and succeeded. The key

to this success was the intense collaboration with the customer; and a contract that governed that collaboration rather than trying to specify the details of scope and schedule for a fixed cost.

[Robert C. Martin 2005]

The agile methodologies come with a series of practices that are aimed at building trust between the vendor and the buyer of the software, frequent delivery being the most important of them, because the customer can see the software, at small intervals, with the planned features, built and working. Risks are also addressed by having the most important features (from the client’s point of view) developed first and by allowing him to make mistakes, to change his mind, to figure out how things are really done later after he sees what the software would look and feel.

Customer and programmer rights

Ron Jeffries, in “Extreme Programming Installed” [] and Martin Fowler and Kent Beck in “Planning Extreme Programming” [], built a list of rights for all the three parties usually involved in a software project, which, if known and respected from the beginning, can enhance collaboration.

Customer Bill of Rights

• You have the right to an overall plan, to know what can be accomplished when and at what cost.

• You have the right to get the most possible value out of every programming week.

• You have the right to see progress in a running system, proven to work by passing repeatable tests that you specify.

• You have the right to change your mind, to substitute functionality, and to change priorities without paying exorbitant costs.

• You have the right to be informed of schedule changes, in time to choose how to reduce the scope to restore the original date. You can cancel at any time and be left with a useful working system reflecting investment to date.

Programmer Bill of Rights

• You have the right to know what is needed, with clear declarations of priority.

• You have the right to produce quality work at all times.

• You have the right to ask for and receive help from peers, managers, and customers.

• You have the right to make and update your own estimates.

• You have the right to accept your responsibilities instead of having them assigned to you.

Customer collaboration

In “Extreme Programming Explained: Embrace Change” [], Kent Beck says:

XP planning assumes that the customer is very much part of the team, even if the customer works for a different company and the rest of the team works for a contractor. The customer must be part of the team because his role is far too important to leave to an outsider. Any (XP) project will fail if the customer isn't able to steer.

So the customer's job is a huge responsibility. All the best talent and technology and process in the world will fail when the customer isn't up to scratch. Sadly, it's also a role on which we can't offer particularly sage advice. After all, we're nerds, not business people. But here's what we know for sure.

[Kent Beck, 1999]

This very much shows the role of the customer in an agile project, and how close collaboration is actually being done, with each party having their responsibilities and rights, and collaborating to “steer” and implement the project successfully.

The most obvious way in which the customers and the programmers collaborate, is agile planning, which is done together. SCRUM iteration or sprint planning summarizes in a way the whole concept behind agile planning that is also used in all the other agile “flavors”. The customers and the programmers sit together, starting with the customer explaining in the first part of their meeting where he wants to go, showing the “shopping” list he wants for the next sprint and explaining what each item means, so that the programmers can come up with the cost (estimated time) for those items in the second part of the meeting. In the case that the cost is over the budget (30 days in a sprint) then based on the priorities of the customer some of the low priority items are dropped, so that the cost matches the budget. In case the cost is under, then the customer is asked to come up with other items, that are the next most important to him, so the gap between cost and budget is filled.

One unwanted possibility (but, since agile methodologies are very pragmatic, it is taken into account) is when the team realizes that they won’t be finishing in time, with the next iteration. Since honesty sits at the base of agile principles, then the customer is informed right away and shown the situation. He is then asked to pick one or more of the planned features, and “drop” it, so that the remaining can be delivered successfully. In this way, the crisis is solved fast and together, whereas the alternative: lying would be much more costly. It is important to remember that in this “negotiating scope” session, for example , 8, 100% finished features will be delivered out of 10 planned, rather then 10, 80% finished features that would be unusable. The “dropped” features will be then planned for the next iteration.

The two methods described above are incredibly efficient at reducing the risk of a project failure since the top most important features are built and delivered first, thus showing how close collaboration between the customer and the programmers can lead to success.