TDDing the sales report

Download: Sources

You're starting a new project. You are talking with the customer, defining a few user stories, to make a release plan, when he says the most important thing for him would be a weekly sales report. Your immediate reaction is to say: you can't have the sales report before, you actually can put data into the system, the data entry forms should be developed first. Then he says, no, I need that first, and the data needs to come from our old system, and I will get one of my IT employees to write a data transformation code that transfers the data into your system, so that I can have the report after the first iteration.

Ok, then...so lets do that first. After a few more discussions, you and the customer write down a few acceptance tests for the sales reports as:

I have, 3 orders from 2 customers (one makes two orders as in the first forgets something).

|| order id || product || qty || unit price || customer || order total ||
|| order #123 || xxy mobile phone || 2 || 200 || company #1 limited || 400 ||
|| order #123 || xxy mobile charger || 2 || 15 || company #1 limited || 430 ||
|| order #124 || xxy mobile charger || 1 || 15 || company #1 limited || 15 ||
|| order #125 || bluetooth adapter || 1 || 20 || company #2 limited || 20 ||
|| order #125 || xxy mobile charger || 3 || 15 || company #2 limited || 65 ||

so, seeing my sales report, I would like to see how much has been sold of each product I have (sorted by product):

|| product name || qty || total ||
|| bluetooth adapter || 1 || 20 ||
|| product #no name || 0 || 0 ||
|| xxy mobile charger || 6 || 90 ||
|| xxy mobile phone || 2 || 400 ||

If I also want to include the customer I would get:

|| product name || qty || total || customer ||
|| bluetooth adapter || 1 || 20 || company #2 limited ||
|| product #no name || 0 || 0 || - ||
|| xxy mobile charger || 3 || 45 || company #1 limited ||
|| xxy mobile charger || 3 || 45 || company #2 limited ||
|| xxy mobile phone || 2 || 200 || company #1 limited ||

Ok, now let's make it work. TDD with a database. Testing with a database is not considered unit testing (Michael Feathers [1] ), but there are situations when this kind of tests could prove very good to have.

Over the years I discovered that there are three techniques that can be applied when testing code that interacts with a database:

• mock testing


• rollback changes technique


• reset database technique.

The best technique is to avoid direct interaction with the database, by using mock objects. That can be a very good technique in many cases, as mock objects are very fast, and quite easy to use, but in this case using mock objects, might mean that out tests could cover very little of our actual functionality, so we can leave mock objects for a better situation.

Now if we really need to touch the database, and the automated tests need to be independent from one another, it is important that the state (data) of a database is the same before running each tests. For this there are two techniques: either you recreate the database in a specific state before each test is run, or any eventual changes made by a test when being run are rolled back after its execution. For the latter technique, you could use database or application transactions that can facilitate it. See Roy Osherove's: ....

The reset database technique, can be done in several ways, however speed in TDD is essential as the faster you get feedback from you tests the faster you learn what decisions to take next, so the most common technique is an in memory database, like HypersonicSQL or Cloudscape for Java, or SqlLite or Firebird for everyone. However this technique can be just as misleading as the mocking of the database in some cases, as database engines are different, and you could end up with your tests working but deployed on the real server the code to fail. In order to avoid this, you can write your tests directly against the database engine you will deploy, sacrificing speed for safety.

In order to recreate a database's state, you can clean/re-create the database and populate it from code (using an ORM/ DataSets etc) or using a sql script that does all this. The second approach might prove faster but, it is even less maintainable then the first so great caution must be imposed. However, there is a tool, that can really help us with this latter technique, who's usage in this context will be proven today.

Back to work

In TDD the first step is to write a test, make it fail then make it work (red/green factor - see: ....). Let's write the test:

[TestFixture]
public class TestSalesReport
{
SalesReportDAO dao = new SalesReportDAO();

[SetUp]
public void ResetDatabase()
{
}

[Test]
public void TestProductsAndQtySold()
{
DataSet reportDataSet = dao.GetSalesReport();
DataTable report = reportDataSet.Tables[0];

Assert.AreEqual(4, report.Rows.Count, "we should have 4 rows");
}
}
As you can see we will need to add the code to the test that resets the database state. Now let's create the database and put data the test data into it, then using the amazing Tychotic Database Scripter [2] we will export the whole database state as a stored procedure that can be run in our test's setup.



and put the values in the acceptance test into it (OrderLines table):



Now let's export the script with the database scripter and create the stored procedure. The stored procedure's code is:


if exists (select * from dbo.sysobjects where id = object_id(N'[dbo].[SETUP]') and OBJECTPROPERTY(id, N'IsProcedure') = 1)
drop procedure [dbo].[SETUP]
GO

CREATE PROCEDURE [dbo].[SETUP]
AS

BEGIN

-- DELETES
DELETE FROM [dbo].[OrderLines];
DELETE FROM [dbo].[Orders];
DELETE FROM [dbo].[Customers];
DELETE FROM [dbo].[Products];

-- Products
SET IDENTITY_INSERT [dbo].[Products] ON;
INSERT INTO [dbo].[Products]
(
ID
,Name
,Price
)
SELECT 1,'xxy mobile phone',200.0000 UNION
SELECT 2,'xxy mobile charger',15.0000 UNION
SELECT 3,'bluetooth adapter ',20.0000 UNION
SELECT 4,'product #no name ',50.0000
SET IDENTITY_INSERT [dbo].[Products] OFF;

-- Customers
SET IDENTITY_INSERT [dbo].[Customers] ON;
INSERT INTO [dbo].[Customers]
(
ID
,Name
,Address
)
SELECT 1,'company #1 limited',NULL UNION
SELECT 2,'company #2 limited',NULL
SET IDENTITY_INSERT [dbo].[Customers] OFF;

-- Orders
SET IDENTITY_INSERT [dbo].[Orders] ON;
INSERT INTO [dbo].[Orders]
(
ID
,CustomerID
,OrderNo
,Total
)
SELECT 1,1,'order #123',430.0000 UNION
SELECT 2,1,'order #124',15.0000 UNION
SELECT 3,2,'order #125',65.0000
SET IDENTITY_INSERT [dbo].[Orders] OFF;

-- OrderLines
SET IDENTITY_INSERT [dbo].[OrderLines] ON;
INSERT INTO [dbo].[OrderLines]
(
ID
,OrderID
,ProductID
,UnitPrice
,Quantity
,Total
)
SELECT 1,1,1,200.0000,2,400.0000 UNION
SELECT 2,1,2,15.0000,2,30.0000 UNION
SELECT 3,2,2,15.0000,1,15.0000 UNION
SELECT 4,3,3,20.0000,1,20.0000 UNION
SELECT 5,3,2,15.0000,3,45.0000
SET IDENTITY_INSERT [dbo].[OrderLines] OFF;

END;
GO

As you can see the scripter preserves the original primary key values in the database, thus making testing easier. If you run it manually you will notice that it is very fast. Now it is time to add the code to our test that runs this stored procedure and resets the database state.

[TestFixture]
public class TestSalesReport
{
SalesReportDAO dao = new SalesReportDAO();


[SetUp]
public void ResetDatabase()
{
//we run the clean and populate with data, stored procedure
using (SqlConnection conn = new SqlConnection(ConfigurationSettings.AppSettings["connection_string"]))
{
SqlCommand command = new SqlCommand("SETUP", conn);
command.CommandType = CommandType.StoredProcedure;

conn.Open();
command.ExecuteNonQuery();
}
}


[Test]
public void TestProductsAndQtySold()
{
DataSet reportDataSet = dao.GetSalesReport();
DataTable report = reportDataSet.Tables[0];

Assert.AreEqual(4, report.Rows.Count, "we should have 4 rows");
Assert.AreEqual("bluetooth adapter", report.Rows[0]["product_name"].ToString().Trim(), "the first product should be a bluetooth adapter");
Assert.AreEqual(DBNull.Value, report.Rows[1]["qty"], "no product #no name has been sold");
Assert.AreEqual("90.0000", report.Rows[2]["total"].ToString(), "6 xxy mobile charger sold results in 90");
}
}

To make it compile we need:

public class SalesReportDAO
{
public DataSet GetSalesReport()
{
return null;
}
}

Now we need to see if our code fails as it should since the data is not extracted from the database at this time:

------ Test started: Assembly: TDDSalesReport.dll ------

TestCase 'TDDSalesReport.Tests.TestSalesReport.TestProductsAndQtySold' failed: System.NullReferenceException : Object reference not set to an instance of an object.
D:\Projects\Dan Bunea\TDDSalesReport\TDDSalesReport\TDDSalesReport\Tests\TestSalesReport.cs(37,0): at TDDSalesReport.Tests.TestSalesReport.TestProductsAndQtySold()


0 succeeded, 1 failed, 0 skipped, took 1.02 seconds.



---------------------- Done ----------------------

Running the code will make it fail, but if we alter the database's state then rerun the test we notice that it is in the previous state, so our stored procedure works. Now all we need to do is write the code to pass the test:

public class SalesReportDAO
{
public DataSet GetSalesReport()
{
DataSet results = new DataSet();
results.Tables.Add("SalesReport");
using (SqlConnection conn = new SqlConnection(ConfigurationSettings.AppSettings["connection_string"]))
{
SqlDataAdapter adapter = new SqlDataAdapter(GetCommandByProduct(), conn);
adapter.Fill(results.Tables[0]);
}
return results;
}

private string GetCommandByProduct()
{
StringBuilder builder = new StringBuilder();
builder.Append("select ");
builder.Append(" Products.Name as product_name,");
builder.Append(" Sum(OrderLines.Quantity) as qty,");
builder.Append(" Sum(OrderLines.Total) as total ");
builder.Append("from Products ");
builder.Append("left outer join OrderLines ");
builder.Append(" on Products.Id = OrderLines.ProductID ");
builder.Append("group by Products.Name");
return builder.ToString();
}
}

And running it:

------ Test started: Assembly: TDDSalesReport.dll ------


1 succeeded, 0 failed, 0 skipped, took 1.19 seconds.



---------------------- Done ----------------------

Excellent, we have written a report that is accompanied by an automated test that is independent and fast enough (unfortunately my computer is rather degenerated, but on a more powerful computer the times might be decreased dramatically).

Now we could improve o by adding a few more asserts but in my opinion asserts should be handled with care, as if their number increases, they become harder to maintain so there should always be a balance between the number of asserts and the coverage a test should do, as I tried to show above.

For the second test, exactly the same approach will be used, writing the test, making it compile, making it fail, and then developing the code to make it work as in any TDD scenario. So we start with this test:

[TestFixture]
public class TestSalesReport
{
SalesReportDAO dao = new SalesReportDAO();


[SetUp]
public void ResetDatabase()
{
//we run the clean and populate with data stored procedure
using (SqlConnection conn = new SqlConnection(ConfigurationSettings.AppSettings["connection_string"]))
{
SqlCommand command = new SqlCommand("SETUP", conn);
command.CommandType = CommandType.StoredProcedure;

conn.Open();
command.ExecuteNonQuery();
}
}


[Test]
public void TestProductsAndQtySold()
{
DataSet reportDataSet = dao.GetSalesReportByProduct();
DataTable report = reportDataSet.Tables[0];

Assert.AreEqual(4, report.Rows.Count, "we should have 4 rows");
Assert.AreEqual("bluetooth adapter", report.Rows[0]["product_name"].ToString().Trim(), "the first product should be a bluetooth adapter");
Assert.AreEqual(DBNull.Value, report.Rows[1]["qty"], "no product #no name has been sold");
Assert.AreEqual("90.0000", report.Rows[2]["total"].ToString(), "6 xxy mobile charger sold results in 90");
}

[Test]
public void TestProductsQtySoldAndCustomer()
{
DataSet reportDataSet = dao.GetSalesReportByProductAndCustomer();
DataTable report = reportDataSet.Tables[0];

Assert.AreEqual(5, report.Rows.Count, "we should have 5 rows");
Assert.AreEqual("bluetooth adapter", report.Rows[0]["product_name"].ToString().Trim(), "the first product should be a bluetooth adapter");
Assert.AreEqual(DBNull.Value, report.Rows[1]["qty"], "no product #no name has been sold");
Assert.AreEqual("45.0000", report.Rows[2]["total"].ToString(), "3 xxy mobile charger sold to company #1 limited results in 45");
Assert.AreEqual("company #2 limited", report.Rows[3]["customer"].ToString().Trim(), "company #2 limited bought these");
}
}

Doing a small refactoring to the names of the methods that extract the report data from the database. Of course it doesn’t compile, but we fix that:

public class SalesReportDAO
{
public DataSet GetSalesReportByProduct()
{
DataSet results = new DataSet();
results.Tables.Add("SalesReport");
using (SqlConnection conn = new SqlConnection(ConfigurationSettings.AppSettings["connection_string"]))
{
SqlDataAdapter adapter = new SqlDataAdapter(GetCommandByProduct(), conn);
adapter.Fill(results.Tables[0]);
}
return results;
}

public DataSet GetSalesReportByProductAndCustomer()
{
throw new Exception("The method or operation is not implemented.");
}

private string GetCommandByProduct()
{
StringBuilder builder = new StringBuilder();
builder.Append("select ");
builder.Append(" Products.Name as product_name,");
builder.Append(" Sum(OrderLines.Quantity) as qty,");
builder.Append(" Sum(OrderLines.Total) as total ");
builder.Append("from Products ");
builder.Append("left outer join OrderLines ");
builder.Append(" on Products.Id = OrderLines.ProductID ");
builder.Append("group by Products.Name");
return builder.ToString();
}

}

then we start by running all the tests and making sure the second test fails:

------ Test started: Assembly: TDDSalesReport.dll ------

TestCase 'TDDSalesReport.Tests.TestSalesReport.TestProductsQtySoldAndCustomer' failed: System.Exception : The method or operation is not implemented.
D:\Projects\Dan Bunea\TDDSalesReport\TDDSalesReport\TDDSalesReport\DataLayer\SalesReportDAO.cs(26,0): at TDDSalesReport.DataLayer.SalesReportDAO.GetSalesReportByProductAndCustomer()
D:\Projects\Dan Bunea\TDDSalesReport\TDDSalesReport\TDDSalesReport\Tests\TestSalesReport.cs(48,0): at TDDSalesReport.Tests.TestSalesReport.TestProductsQtySoldAndCustomer()


1 succeeded, 1 failed, 0 skipped, took 1.85 seconds.



---------------------- Done ----------------------

Now let’s write the proper code to make it run:

public class SalesReportDAO
{
public DataSet GetSalesReportByProduct()
{
return GetReportResults(GetCommandByProduct());
}



public DataSet GetSalesReportByProductAndCustomer()
{
return GetReportResults(GetCommandByProductAndCustomer());
}

private DataSet GetReportResults(string sql)
{
DataSet results = new DataSet();
results.Tables.Add("SalesReport");
using (SqlConnection conn = new SqlConnection(ConfigurationSettings.AppSettings["connection_string"]))
{
SqlDataAdapter adapter = new SqlDataAdapter(sql, conn);
adapter.Fill(results.Tables[0]);
}
return results;
}

private string GetCommandByProduct()
{
StringBuilder builder = new StringBuilder();
builder.Append("select ");
builder.Append(" Products.Name as product_name,");
builder.Append(" Sum(OrderLines.Quantity) as qty,");
builder.Append(" Sum(OrderLines.Total) as total ");
builder.Append("from Products ");
builder.Append("left outer join OrderLines ");
builder.Append(" on Products.Id = OrderLines.ProductID ");
builder.Append("group by Products.Name");
return builder.ToString();
}


private string GetCommandByProductAndCustomer()
{
StringBuilder builder = new StringBuilder();
builder.Append("select ");
builder.Append(" Products.Name as product_name, ");
builder.Append(" Sum(OrderLines.Quantity) as qty, ");
builder.Append(" Sum(OrderLines.Total) as total, ");
builder.Append(" Customers.Name as customer ");
builder.Append("from Products ");
builder.Append("left outer join OrderLines ");
builder.Append(" on Products.Id = OrderLines.ProductID ");
builder.Append("left outer join Orders ");
builder.Append(" on Orders.Id = OrderLines.OrderID ");
builder.Append("left outer join Customers ");
builder.Append(" on Orders.CustomerId = Customers.Id ");
builder.Append("group by Products.Name,Customers.Name ");
builder.Append("order by Products.Name ");
return builder.ToString();
}

As you may see several refactorings have been made but let’s see what happens when we run it:

------ Test started: Assembly: TDDSalesReport.dll ------


2 succeeded, 0 failed, 0 skipped, took 1.32 seconds.



---------------------- Done ----------------------


I guess, we can call our customer now (. Although it might not be the most optimized code, the code does what the customer wants, backed up by automated tests that prove it and if there is need for optimizations and refactorings it is always easier to do them protected by the automated tests then without.

Conclusion

By writing just a few lines of code and with the help of a brilliant tool we managed to TDD our sales report as requested by the client, fast and in a convenient way. However, there should be caution in using this technique, as the database script might become rapidly a test maintenance nightmare if the database structure changes, so having as few scripts like this as possible as well as combining this technique with other database testing techniques, will make this maintenance work easier.

[1] Michael Feathers – A Set of Unit Testing Rules http://www.artima.com/weblogs/viewpost.jsp?thread=126923
[2] Thycotic DatabaseScripter - http://www.thycotic.com/dotnet_dbscripter.html

Test a little, code a little - a practical introduction to TDD

Introduction

Test Driven Development is one of the most well known practices in and outside agile communities. However, the level of understanding is in my opinion low as the term seems to be surrounded by lots of misunderstanding and misuse.

Biggest confusions regarding TDD

Many people that do not know too many things about agile practices, or beginners tend to think that test driven development, means having written automated test for your code or even worse that test driven development is a method to test the code, rather then develop it.

1. Test Driven Development is for testers

A lot of confusion comes from the name: Test Driven Development, which to some developers means a method of writing automated tests for software, something that should be the QA team’s responsibility. Having the test word in the name tends to confuse a lot of developers, who presume as explain above that TDD should be ignored as it is a practice for the QA. This comes from a partial read of the title, imagining the rest. The title was intended to be quite clear: automated tests that drive the development of the code

2. I do TDD because I have automated tests that test my code

Beginners in agile practices, have the tendency to believe that having automated tests that cover parts of the whole code, means they are doing TDD. In fact it is right, only of those tests were developed before the code was developed and not after. If the test code did not influence the development of the code, then it is not TDD, it is just having automated tests, which is a good thing, anyway.Having automated tests can say that is partially TDD even if the tests were not developed before the code, at a high or better said project level, as the biggest purpose of the tests in a project as a whole is detecting when something is broken by a change a developer makes, which influences the decisions he makes later in the design and in the code he writes, thus driving the way the software is written.

Theory

Test Driven development is a software development technique, targeted for software developers, and not software testers, that uses automated tests that are written at first, to influence and drive the code that is being written. In the end the code passes the tests, which means that it has been driven towards passing a specific number of tests.

The technique has been widely described, so I will try to keep my introduction as simple as possible. The technique is also called red-green factor, test a little, code a little or test, fail, code , pass and consists from a list of steps that are followed in order to obtain the code we want.

So it is a coding technique, one a little bit different, as it presumes the following steps:

- Test: Write a small test (one, not more)
- Compile: Make the code compile, and make the test fail
- Fail: running the test will show it failing (red)
- Code: write the code, just enough to make the test pass, not more
- Pass: run the test , it should pass. If it doesn’t code a little more (step 4). If it passes, you get the red (red-green factor)
- Refactor: refactor the code written so far (both test and code)
- Pass again. If it doesn’t go to step 4.
- Repeat: Go to step 1 and do it again, until you have the desired code, passing all the tests it should pass.

This would be the simplified list of steps that need to be done in test driven development. In fact, the list is a little bit longer as the first step is always think: about what you need and then the second is to think about making a simple test, then following the 1-8 steps and going back sufficient times to make the code good. This is an iterative process that presumes making a test (test a little) and building the code that passes it (code a little), and repeating this step as much as needed.

Practice

Suppose we have a order processing system, requested by the customer and I am asked to write the part where the clients are given a discount if the order total is over a certain limit. In Extreme Programming, this would be a programming task of a user story .

At this time, I have absolutely no code written. First I think I should have an Order class and maybe also an OrderLine class. Let me write a test about that (step 1):

[TestFixture]
public class OrderTests
{
[Test]
public void EmptyOrder()
{
Order order = new Order();
Assert.AreEqual(0,order.Total);
}
}


I have no such class Order. So at step 2 I should make the code compile and them and at the same time make the test fail. So I write the following code:

public class Order
{
public decimal Total
{
get{throw new NotImplementedException();}
}
}

I can compile the code now, so I should see if the code fails:



It seems steps 1-3 were simple, let’s move on and add the code that passes the test. You might be surprised:

public class Order
{
public decimal Total
{
get{return 0;}
}
}

The fact that I intentionally wrote return 0, makes the code be bad but still pass the test. However this is very good way of understanding that the code needs to pass the tests, as they are influencing the development and that it is important not to write code I advance.And now let’s if it passes as it should (step 5):




Wow, now we have implemented a passing test, we should see if any refactoring is needed. So far I don’t see any so since the code still passes the test, I can consider the first cycle completed. Back to the first step, adding a new test, for a few order lines.

[TestFixture]
public class OrderTests
{
[Test]
public void EmptyOrder()
{
Order order = new Order();
Assert.AreEqual(0,order.Total);
}

[Test]
public void TwoOrderLinesTotal()
{
Order order = new Order();

OrderLine ol1 = new OrderLine();
ol1.Product = "Laptop";
ol1.Quantity = 1.0;
ol1.Price = 1000.0;

OrderLine ol2 = new OrderLine();
ol2.Product = "Monitor";
ol2.Quantity = 2.0;
ol2.Price = 200.0;

order.AddOrderLine(ol1);
order.AddOrderLine(ol2);

Assert.AreEqual(1400.0, order.Total);
}
}

Let’s make it compile and fail, by adding the OrderLine class and AddOrderLine method:

public class OrderLine
{
public string Product
{
get {throw new NotImplementedException();}
set { }
}
public decimal Quantity
{
get { throw new NotImplementedException(); }
set { }
}
public decimal Price
{
get { throw new NotImplementedException(); }
set { }
}
}

And:

public class Order
{
public decimal Total
{
get{return 0;}
}

public void AddOrderLine(OrderLine ol)
{
}
}

Let’s see if it fails (we run all the tests):



Now it is time to code to make the test pass.We add a little code, then if it compiles, run the tests. If it fails, then we need to code some more, and run again and so on until the code passes the tests. The code becomes:

public class Order
{
private IList orderLines = new ArrayList();

public decimal Total
{
get
{
decimal sum = (decimal)0;
foreach(OrderLine ol in this.orderLines)
{
sum += ol.Total;
}
return sum;
}
}

public void AddOrderLine(OrderLine ol)
{
this.orderLines.Add(ol);
}
}

And OrderLine:

public class OrderLine
{
private decimal quantity;
private decimal price;
private string product;

public string Product
{
get { return product; }
set { product = value; }
}
public decimal Quantity
{
get { return quantity; }
set { quantity = value; }
}
public decimal Price
{
get { return price; }
set { price = value; }
}

public decimal Total
{
get
{
return this.quantity * this.price;
}
}
}



Wow, this is the second time in the last 10 minutes, now let’s look for some refactoring possibilities. The code seems ok, but maybe we can do something about the test, as it seems the order initialization code is duplicated. We move it into the SetUp method, as it is run before every method is run, then the OrderLine initialization code could be moved to another method:

[TestFixture]
public class OrderTests
{
Order order = null;

[SetUp]
public void SetUp()
{
order = new Order();
}

[Test]
public void EmptyOrder()
{
Assert.AreEqual(0,order.Total);
}

[Test]
public void TwoOrderLinesTotal()
{
OrderLine ol1 = CreateOrderLine("Laptop",1,1000);
OrderLine ol2 = CreateOrderLine("Monitor", 2, 200);

order.AddOrderLine(ol1);
order.AddOrderLine(ol2);

Assert.AreEqual(1400.0, order.Total);
}

private OrderLine CreateOrderLine(string product, decimal quantity, decimal price)
{
OrderLine ol1 = new OrderLine();
ol1.Product = product;
ol1.Quantity = quantity;
ol1.Price = price;
return ol1;
}
}

Now see if it still passes:



It does, so we have just finished the second cycle, and we have an almost good code, pretty much tested if it works ok. Now let’s add the part with the discount, using another “test a little, code a little” cycle. First we add the test:

[TestFixture]
public class OrderTests
{
Order order = null;

[SetUp]
public void SetUp()
{
order = new Order();
}

[Test]
public void EmptyOrder()
{
Assert.AreEqual(0,order.Total);
}

[Test]
public void TwoOrderLinesTotal()
{
OrderLine ol1 = CreateOrderLine("Laptop",1,1000);
OrderLine ol2 = CreateOrderLine("Monitor", 2, 200);

order.AddOrderLine(ol1);
order.AddOrderLine(ol2);

Assert.AreEqual(1400.0, order.Total);
}

[Test]
public void Discount10PercentOver2000()
{
IRule discountRule = new DiscountRule(10, 2000);

OrderLine ol1 = CreateOrderLine("Laptop", 1, 1000);
OrderLine ol2 = CreateOrderLine("Monitor", 2, 200);
OrderLine ol3 = CreateOrderLine("MimiMac", 2, 500);

order.AddOrderLine(ol1);
order.AddOrderLine(ol2);

order.ApplyBusinessRule(discountRule);

Assert.AreEqual(2400.0, order.Total);
Assert.AreEqual(2160.0, order.TotalAfterDiscount);

}

private OrderLine CreateOrderLine(string product, decimal quantity, decimal price)
{
OrderLine ol1 = new OrderLine();
ol1.Product = product;
ol1.Quantity = quantity;
ol1.Price = price;
return ol1;
}
}


We ensure everything is compilable and then that the test fails, then we struggle to make it pass, fixing the code, then we realize something very ugly: the test should work but it doesn’t.





We have a bug in the test. Now we see that in the test we forgot to add the order line #3 at the order (totals 1400), so we add it, then rerun the test. It seems the relationship between tests and code makes the bugs easy to find in both parts of the code:



Well, it seems that we have a few simple tests that helped us build a piece of code. Now these tests helped us so far, and by adding them to a test suite on the project, and running them regularly, can really tell us when they have been broken and where.

Hopefully, the above lines will clear a little the "fog" around what TDD means and how it can be applied practically.

Model View Presenter - is testing the presenter enough?

Source code: Download


Lately, I have noticed that the Humble Dialog Box or Model View Presenter are gaining more and more acceptance among software developers, especially in agile communities, because of its benefits regarding the very good separation between the view and the behavior and because it can be very easily unit tested, on a problematic field: user interface.

Let's meet Model View Presenter

In model view presenter just as Martin Fowler or Michael Feathers [2] say, the logic of the UI is separated into a class called presenter, that handles all the input from the user and that tells the "dumb" view what and when to display. The special testability of the pattern comes from the fact that the entire view can be replaced with a mock object and in this way the presenter, which is the most important part, can be easily unit tested in isolation. Let's take a small sample:

A client writes in a user story:

User management

The system will support managing users. Each user can have a username and a password. The system will present a list of user and when one is selected, it can be edited. Also there will be functionality to add new users and to be able to select a user from the list and delete it.

Ok, now let's get to work:

Our view should need the following data, which will be put into an interface:



The UsersList represents the list of users, and the User is the current user selected at a certain time.

When the UI is displayed, we want to be able to see the list of available users, and by default the first in the list to be selected. Let’s write a small test for that. Having no view, we will make a simple implementation of the IUsersView, called UsersMockView. Using it we will be able to see if the presenter when it is initialized sets the needed list and the first is selected in it.



where we have:



And for the data we use a separate model class, like:



If I want to make my test compile, I must write my presenter, see the test failing then keep working on it until the test is working:



and now it is time for a new test, and a small refactoring in the test code:




Now we can go further and add the missing code ... until it works. We then follow the same procedure until we have quite a suite that test selecting a user in the list, updating its details, deleting a user, deleting all the users, and others. Now we know we have a Presenter that works fine. At least where it is tested.



Could we have used a dynamic mock?



I like writing in some cases the mock objects by hand. However there are a few dynamic solutions that can prove quite helpful, like NMock [3] for instance. NMock can create mock objects, using Reflection.Emit and your interfaces “on the fly”. Then it is also very useful when it comes to seeing what methods were invoked in the mock object by your class under test. You can also set predefined responses, when the class under test invokes your mock object, but that is another matter. Let’s make a small sample, just as we did before manually. For the TestInit, using NMock it will be like:






It works. The presenter has invoked the properties of the mock object, once as expected and set the expected values. This way we can throw away our previous tests, and can build a new presenter based on testing with dynamic mock objects.

Implementing the real view

If you read Michael Feather’s “Humble Dialog Box” article, you’ll be able to see very well how the real view should be built. The presenter sets the values into the view which are the propagated to the properties of the controls, much like:



but once .NET was released, it comes with the concept of data binding, and there are some developers that do not want to ignore it and use the “old and safe” ways as above.

How does MVP cope with data-binding?

.NET 2.0 comes with a new and much better concept for data binding then 1.0 or 1.1 did, the BindingSource. This now has the possibility to be connected to different kinds of data sources, including business objects. For our interface we create two: bsUser for User and bsUserList for the list of Users. Then for binding we can use, the designer as:




Now, we go back to our properties that are used by the presenter to “push” data into the view to be displays, and modify the code to use the binding sources as follows:



As you can also see, user actions on the view are delegated to the presenter, which is now fully in control of what behavior of out user management interface.

If we run the tests again, they are all green, all succeed. This is great, but this green is false if you think that it works with data-binding because the tests do not touch in any way the actual view. So what do we do?

A first option would be to leave it like that. But since a view for a desktop application is not exactly as “dumb” as we would like, and can quickly become a source of error, we need to add some tests.

Using a functional testing framework like NUnitForms or SharpRobo could be a solution but this can result in tests that test the same thing, and that is not exactly maintenance friendly. But wait, what if our stub is not exactly a stub but an extension of the view itself, to which we can add testing code. For this we inherit, the real view and create ExtendedUsersView. Now we want to modify the testing code in such a way, that the existing code is still reused. For this we inherit the manual test we have written, and just override the SetUp method:



If we run it, it will run the initial tests but on the actual view. It needs to be shown as data-binding to the controls is done when the controls become visible, but that is not a problem. The problem can be speed, but if you run the code you’ll be able to see that the difference is not that significant. If speed is a problem, then maybe you can run the tests that use the actual view, more rarely, as you can get quite enough feedback from the mock tests.

The process

About the process of building destop application pieces, the way I have constructed the application above, might seem wrong, as using TDD, I built the presenter, but I did not have the view yet, and that was constructed later on the exact same set of tests. In practice I wouldn't recommend this, as the first step, building the presenter without the view, using TDD might lead to a code that will need big refactorings (both presenter and tests) when the view is built. A better approach would be to construct, a two test cases, one inheriting teh other, with two setup methods, one with the mock view, and one with the actual extended view, then build a test inside the first, and build some code on it in both the presenter and the view, then add a new test and so on, until we have tested both the presenter and the view, having both tests that are fast and isolate the presenter's bahavoius and tests that also cover the view.

Conclusion

I have tried to present a solution, that can cover more of the code with unit tests, then only the presenter, as for desktop applications this can be an important issue. I have also tried to find a solution that works with data-binding, and what I have found seems ok so far, but only time and more research and tests can really tell if this was a good choise….

1. Martin Fowler - Model View Presenter
2. Michel Feathers - The humble dialog box
3. Peter Provost - TDD in .NET
4. Jeremy Miller - A simple example of "humble dialog box"
5. Ben Reichelt - Learning teh model view presenter pattern

The problem of communication

1. Introduction

A lot of waste and rework is done because of bad communication. In fact, communication is the main factor in a project success or failure. And that does not refer only to software. Alistair Cockburn, wrote an entire book about the importance of communication and builds a methodology around improving communication: the crystal family

2. The big misunderstanding: improving means increasing quantity

The great misunderstanding: improving communication means increase quantity: more meetings, more documentation, more emails and more phone calls. This is profoundly wrong: Communication can only be improve by quality and in many cases improving quantity means increasing the communication problem and not solving it.

One of the most important principles in software is in my opinion looking back to what you did , so you can learn what was good and what was bad, and find ways to improve your activity. This is emphasized in the principles behind the agile manifesto as:

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

In a project we did recently we looked back with the customer and we noticed that the bad things in a project developed were mostly caused by bad communication. By bad communication I mean much time needed to get questions answered, ambiguous requirements, not completely specified parts of the system, etc. The immediate answer was that we need to improve communication. For that the answer was that more emails will be sent, more meetings will be help, more conference calls will be held, the specification document will be even larger.

The immediate problem was overcommunication: we held meetings for the sake of having meetings, without a real need, which was in my opinion a great waste of valuable time. We wrote more emails, we held more conference calls, where we discussed about the weather or other pointless things and usually the discussions were very high management level, like are you confident with the plan, with statistics, with waste of time. There were gains, but unfortunately they were shadowed by the new problems.
I realized that people , no matter where they are from, treat the problem of communication not seriously enough. They just say we will do more, when more means quantity instead of quality. But quality is not an easy matter when it comes to communication.


Can communication be improved? What is quality in communication?

What is quality in communication? In seems that Alistair Cockburn discovered a way, based on his long and great experience, to measure communication quality. He says:

Interactive, face-to-face communication is the cheapest and fastest channel for exchanging information (see 1.)

Continuing with a communication top:



Following Alistair’s advice we need to speak more in front of a whiteboard. Why in front of a whiteboard? Because sometimes, a picture is worth a 1000 words, and this applies to software very well.

Going even further, I have found that a very practical way to improve the quality of communication is visual communication. I noticed that when describing bugs and ways to reproduce then, we or our clients have trouble describing then and describing a step by step process to reproduce them. For this one of the things I did was to ask that customers and others, when reporting bugs even changes, enhancements to include screenshots.

A single screenshot can show even the context in which the bug appeared and give you a great deal of information about the user actions. Screenshots are very cheap, and communicate a lot of information. They can even be automated into the software, so whenever you have a bug, the software takes a screenshot and sends it to you by email along with the last x lines in the log.

Many times, I am just lazy and I don’t want to read. Joel (www.joelonsoftware.com) says in his book: (Users Don't Read the Manual). Well that is very true. I want to use a new technology but I am too lazy to read the help files or the documentation. But wait, a new way of improving communication, emerged: small flash movies, showing you , visually what you need. They seem to pop everywhere. Just take a look at Ruby On Rails: [link] This can be done for bugs, for help, for a lot of things. And they are easier to do and send then writing a step by step document.

3. Communication divided by direction.

If we consider ourselves and the team the center of the communication, and we consider where the source and target of communication is, we can divide communication in:

- Communication within organization, team etc
- Communication outside with other stakeholders


3.1 Communication inside the team and organization

Depending on what needs to be communicated, the agile methodologies have ways to deal with communication inside the organization. In my opinion communication inside the team can be divided into two categories: project related and team performance related.

3.1.1 Project related

The project related information must go very fast inside the team. Crystal calls this osmotic communication and it is at the very core of the methodology. It says that people working on the same project need to be able to use the best communication technique: face to face communication , with and without a whiteboard. For this the best technique is to have the entire team collocated, in the same room as much as possible , all facing each other, to be able to ask each other questions and to enable others to hear discussions etc. The team must also have a whiteboard close.

Extreme Programming goes even further, addressing the communication problem mainly with user stories written on pieces of paper and stuck to the wall, and with writing all the business code by pair programming. This means that two programmers, work together at the same computer, understanding 100% of what’s done, and not 50% as if they were doing it separately.

The techniques above and others mainly address communication when time is crucial, but there are other problems of communication within the organization, where time is not that crucial. This is best described by a situation, where you have a team developing a program for a few months, delivering it to the client. Now the client wants to continue the program, add extensions to it etc, but the team that built it is not available anymore or it is available partially. How will the new programmers that come in contact with the program understand what was required, what was done, and how it was done.

Extreme Programming comes up and says that the best way to document a computer program is the code itself and the automated tests, it has and not documents describing what was done, as these documents need to be written by someone who is costly and become obsolete very soon. In most cases, I find this approach very good, as I noticed that recently whenever I download an open source framework or look at code written in the past, the first thing I instinctually do when I want to learn how to use it, I look at the unit tests. They show me how the software can be used, and which are the limitations, in the same language I want to use it: code.

3.1.2 Team performance related

We all want to improve our skills. Many times, we don’t have the information or the time to explore it. A learning environment has been clearly specified as a way to keep programmers motivated on the long term. In fact, this is one of the 7 principles of lean software development (see 3)

It has been proven that pair programming can help a less experienced programmer that is paired with a more experienced one move very fast ahead. The same result is achieved when the team is collocated and the less experienced can hear and see at the whiteboard the more experienced doing something, being able to ask and be given answers very fast.

One great technique, described by Alistair Cockburn is information radiators. These are things you would like to be communicated to others. These can be of help both in project, as you might have the stories stuck on the wall, divided in iterations and in sections: done, in progress and not done. Whomever goes by can see which are done, which are not, very easily without the need of asking anyone about the progress. Ron Jeffries wrote a great article about how to use the walls to communicate about progress, called Big Visible Charts. (see 4)

The information radiator are usually characterized by being pieces of paper with information that are put in places that readers might find them and read them. Usually, after reading a book, in order to get the others to read it, I cut some of the paragraphs I liked most, print it on pieces of paper and put it all over the working environment: on the walls, where people take breaks and smoke cigarettes, in places they feel comfortable. Usually after sticking them on the walls I hear the others referring to them in our conversations. It is also a very good way to speak to managers, with notes stuck on the walls.

Another technique, I found very valuable, to get people to move ahead, improve fast is what I call school like meetings. I have described them in : [Self training … ]. This is mainly , an hour where we take a subject and all read it, then based on some questions we discuss it so we can make sure everyone understands it. This is done within working hours, usually at the first hour in the morning. We have read even a book, chapter by chapter in this manner. It has proven an enormously healthy practice. Let me give you an example.

We started using an ORM framework. I started using it in a project where I teamed up with another developer. I had more experience with ORMs and I showed him a few tricks. At some time, I moved to another project and he was on his own. Then a new project was started and the ORM was used, then another all by other people. After about 4 months I looked into the code of one and I was very badly surprised that is was completely misused. People even were complaining about using this ORM because it is not of any real help. I decided to go to the bottom of the problem and I noticed that the second project was started, having the first project as an example. The mistakes from the first project that my colleague did were moved to the second and taken from granted. The third even more. A lot of people were using the ORM, but the level of knowledge after 4 months was very low.

I had to do something. I immediately gathered everyone, in that afternoon, took the documentation of the ORM and we stopped work and everyone had to read the documentation. I established a set of questions, so I can discover if the technology was reasonably well understood and we started reading. To read, answer the questions it took 4 hours. It took 4 hours to get everyone to understand how this technology. Four hours of no work, a time that could be used to build software. But those for hours proved later one of the best well spent hours in our activity. By understanding the technology in 4 hour they avoided hundred of hours of misusing it and building code that was just reinvented or badly written. Countless hours of debugging were avoided by understanding the ORM and using it right, which decreased the quantity if code by up to 75% and by having a smaller code, less bugs were produced and debugging was a lot easier.

3.2 Communication outside with other stakeholders

The usual stakeholders in a software project are: the customer, the management and the programmers. The management and the programmers can be considered either inside or outside the same circle, so both techniques are applicable. One of the very effective, silent techniques of communicating with management inside is by using the big visible charts that Ron Jeffries describes. Management can be considered depending on specific contexts both inside the programming environment and outside.

Communicating outside the programming environment is divided in my opinion in 2 sections:

1. From outside the team to programmers
2. To outside by programmers

The first can be further divided between:

1. From outside at the beginning, when code is to be written according to what is communicated
2. From outside at the end or after some code is delivered, commonly known as feedback

The software industry seems to consider that the first forms of communication from outside, about what needs to be coded are the requirements document or functional specifications.

Are functional requirements a good way to tell programmers what to build?

In a project we did before, we joined another company to develop it, having developers from the two companies developing the project. With the other company not being an agile oriented company, but more like waterfall process oriented, they came to us with a big functional specification of more then 100 pages. This was later estimated by us to be included in the first 3 week iteration.

The interesting part is that usually when people see such a nicely layout with so much information about screens, user interaction , use cases, their first thought is that this is such a good document with all the information captured and put where it belonged. Everyone looks up to those that did it. It makes some people just say: now this is how things get done properly. But are they?

My first thought was, if we develop this part of the software in 3 weeks, how many weeks was the functional spec written on? I just hoped it was clear so it does not need even more time to be invested in it. Another reaction I saw in my colleagues was that because the document was so big the tended to overlook some of the content. It was too much text. What a mistake that was, because we missed some well hidden big requirements, that were missed in the estimate, but were required at the end of the iteration.

So, now the conclusion: although big functional specification documents seem very good to the untrained eye:
- They are very costly to build (how much time do you need to write 100 pages of text?),
- They might miss important aspects of the program. As Alistair already pointed out, they are not very good at communicating things. They are indirection. The writer cannot capture whatever he needs to capture and the one that reads tends to imagine the rest. Imagination is good, but not in this case because in this case is very costly. This is a problem with alternative means of capturing requirements like direct talk, stories and acceptance tests but the main difference is the amount of time needed to spot those misunderstandings.
- They tend to push the customer proxy, which is the root of all evil. The more indirection, the more misunderstandings. When the information needs several nodes to be transmitted from the ones that knows the business to the one that builds the program, important time is lost, and important precision.
- Big documents are hard to read. Someone might say it is important to capture everything there otherwise programmers would complain about not being complete, but over completing it is just as bad
- What if something changes? What if at the time the doc was written there was one situation and it just changed? What do you do? Throw up the part or the entire document? If you wrote it in 1 month, and it cost you x, you have just lost a 10-20-30-...100% of x and need to pay again for the new one.

My belief is that big functional documents are written to replace trust between the parties involved. We need to write everything in the spec and cast it in stone, so it can't change and we can't be attacked by the customer later. My belief is that where there isn't trust, there isn't good business. Lack of trust is usually compensated with enormous overhead and cost. But after all, not everyone is to be trusted.

Although they are not evil, I am trying to show that big requirements documents are just not the answer in most projects, anymore. We need to move faster and for that the communication needs to be efficient and light. All agile methodologies say that you should have the customer (or customer representative) with you while developing. That enables the most efficient way to communicate. Direct, face to face talk, in front of the running program and in front of the whiteboard. But in order to be organized and not to replace big heavy, beaurecratic methods to communicate to chaos, we still need to have something written so we can organize, plan light. This text that is being written needs only to remind us of a thing that the customer needs implemented. It needs to be able to be written fast, so in case changes come up can be thrown away with no remorse, can be estimated and have business value. This is what is known as a user story.

In the wonderful book ‘User Stories Applied’ of Mike Cohn he defines these characteristics of a user story:
- Independent
- Negotiable
- Valuable to users or customers
- Estimable
- Small
- Testable

To demonstrate a way of improving quality vs quantity in communicating what needs to be done, agile methodologies (re)invented acceptance tests. Instead of making the customer describe what he wants you ask him: “how will I know that what I did is according to what you want?”. And he will start saying what needs to be done to be able to know if what was requested was build and finished or not. Instead of heavy descriptions, two purposes are achieved: you know how you can test and what needs to be done at the same time. Also, ambiguity tends to be dissolved by acceptance tests, because a man in order to know how to test something he must have a very intimate knowledge and understanding of that thing.

Feedback

One of the Extreme Programming values is feedback. If the customer knows how to specify how you can test that you’re done building what he requested, after delivering how can you really be sure? You need his input after seeing and testing the program.

Improvement can only be done as a result of feedback. You write a program, the customer sees it and tells you if that program is good or not. He can also tell you what would be even better to improve the program’s quality.

The virtual nature of software makes it very hard for customers to understand how it’s build and done, so it is very hard for them to communicate you what they really want. To address this extremely important problem, software should be shown very often to the customer. This has two very important benefits:
- he can see where you are, the real progress, and this increases his trust that you’re doing what you were planning to do and that his money is not wasted
- he can see if something is not what he wanted and tell you, so you can fix it or after seeing it he can start to ‘feel’ the software and be able to give you much better information about what he wants. The smaller the delivery cycles, the better.

The small delivery cycles are incredibly important in a project’s communication. They are called within agile communities, iterations and it has been proven that it is best to be between one and 4 weeks in length. As previously stated they give you one very good technique to avoid derailing the project and they embrace changing requirements.

In the past there have been projects, in which we worked for 4-6-8 months and then one day, it was the big day: the software was presented to the client. That day was one of the most stressful situations I and the other programmers could be in. It feels just like waiting in the court of law for the verdict to be pronounced. And the stress is as big as the risk is. What if the customer says that that is not what he wanted? The losses for your company’s 4 months of development are big, and redoing the software is something any business can very hardly cope with. With iterative development, showing the customer the result of the latest iteration, can mean in the worst scenario, that 2 weeks of work are useless. But 2 weeks is much easier to solve then 4 months or more. It is much easier to put one wagon on the rails again, then the whole train.

When the first requirements are being gathered from the customer, you explain him how something can be done, but it is very easy for him to imagine a completely different thing. It is also very hard for him to understand what can be and what cannot be done. What costs more or what costs less. After you deliver the first result of an iteration, he can see a computer program, and everything seems to take shape in his mind. He can use this as a basis for future requirements, and can see about how much time one thing or another takes to be implemented. The second iteration is even better and as the project progresses, the dark foggy unknown that software seemed for him in the beginning is stating to become more easy to control, more clear and he becomes less and less stressed. He starts to understand how things are done and he sees real progress. There is nothing more honest then showing the customer the real program from which he can see the progress.

If feedback is the base for all improvement, how about the internal quality of a project. What we discussed earlier about feedback is the external quality that can be improved, the part that the customer understands and helps improving, but what about internal quality. But what is internal quality in a software project?

A project in my opinion is of high internal quality if it can be tested and debugged easily and it can changed with very few effort. Being able to be changed easily assures the program a much longer life in production, which makes it more cheaper. If the customer pays $1.000.000 for a software and the software is used for 5 years, it will cost $200.000 every year. If the same software can be fixed, readapted and maintained longer in usage, for instance for 10 years, it costs $100.000 per year. Twice cheaper. Well, customers love that.

But how can you make the software give you feedback about its health? In agile methodologies the best technique for receiving feedback from the code itself about its health are automated tests. They can detect problems very early and report them to you so you can act very fast on them. I call my automated tests, break detectors. They help me by giving me fast feedback about things that broke in my software. If I can detect them fast, I can fix them fast and that allows me a great flexibility. I make a change in the software, even in code I didn’t write. Then I run the tests. If they detect a break, I know and I can choose either to reverse the change or repair the affected part. A big problem in a program’s flexibility is that if it becomes big, it is very rigid, and making a change can affect parts of it that you cannot know of. So you are afraid to change it. So the project starts to be doomed. The hardest it is to be changed, the hardest it is for it to cope with the market requirements so the shorter its life will be.

If I were to make a comparison, I would say that If I were the general and my code the army fighting the requirements, my automated tests would be my intelligence center. The bigger my army would get, and the larger the war zone, the hardest it would be for me to know what’s happening in all areas and be able to adapt my strategy to the new conditions. But If I have the break agents placed with the different parts of the army they can report to me about problems and defeat situations very fast. The more agents, the largest the areas of coverage, from where I can receive information from and act upon it.

Feedback is the one thing that corrects and improves the software.

Communicating with the outside

Before starting a project, teach your customer about the importance of good communication and how that is achieved in software and with you in particular. Show a very honest plan, of how the project will be build.

On misunderstanding with agile methodologies, is that it is believed that it is not well planned and to the uninformed this seems like cowboy programming, where everyone takes arbitrary decisions and wait to see what happens. In fact, agile software methodologies are very well planned. They plan light, having the possibility of change in their minds, making the plans simple and very easy to change. The plans are built together with the customers, management and the programmers, on very honest and proved facts, both at a high level, called release planning and at low level called iteration planning.

Projects in Extreme Programming for instance, are divided into releases, which are to be delivered to the real users, or that are going ‘live’ and usually take one to four months to develop and they are also divided into iterations that are between one and four weeks in length and which need to deliver a tested, production ready system to the customer that is in charge of developing the software.

When making the big, release plans, the customer tells the programmers about what he wants as small sentences written on small paper cards, called user stories. All these are gathered from the customer and then estimated roughly by the customer. After being estimated, the customer can see their cost, and then be able to prioritize them. After the prioritization is done, the user stories are put in the iterations, to be developed. Well, this isn’t very different from traditional planning, excepting the level of detail of the user stories. But wait, I forgot to tell you that the plans can be updated after each iteration. This gives great flexibility in what is being built, and it also gives the customer great control on how and what is being built. He is in charge. After all, if you planned to release the software after three months with the customer, and after one month he realized that he misspecified what he wanted, or the business conditions changed, he can turn the direction very easily to the new one. This gives agile project great adaptability which can deliver the customer a much better product, as it can be easily changed and the control is in his hands.

After each iteration, the customer sees the software planned and developed in that iteration, and can readapt according to what he sees. He can reprioritize, give up some features and add new ones. If the plan is light, it can be easily changed and readapted. He receives the best feedback about what you’re doing for him by seeing the software and this gives him the possibilities to tell you how to improve it, by giving you feedback.

Showing the customer, the product instead of reports, in the middle of the production, is a great way to communicate and build a product of much bigger value to the customer.

Conclusion

Although, maybe what I have said earlier might need a level of knowledge about agile technologies, I hope it can also open an interest in those that have not used them , by showing different strategies on how the problem of communication is addressed by agile methodologies. This really can improve products, customer satisfaction, can decrease costs and in all improve the way we do business.

Charging the customer for miscommunication, is the worst way to address the communication problem, however it is still extensively used. This makes software more expensive, and generates new problems and new stresses that can be pushed to programmers or customer. Just as a simple example, if it costed me x+something to build the detailed requirements documents, and the customer doesn't want to pay more then I'll just get the programmers to build the software in less time with less people. Guess how much programmers under stress cost?

1. Agile Software Development, Alistair CockBurn
2. User Interface Design for Programmers, Joel Spolsky
3. Lean Software Development: An Agile Toolkit, Mary and Tom Poppendiek
4. Big Visible Charts, Ron Jeffries

I fought XP and XP won

One day I heard about eXtreme Programming. Wow, I was a student, eager to know, and I tried to live on the edge. Extreme was just what I looked for. Soon after I got Kent Beck's - Extreme Programming Explained, the first edition. I read it very fast and that was a decisive moment in my programming life. I thought everthing in there was good but not appliable. I said , you can never have 100% unit tested code. In time I saw I was wrong. I said PP costs double. I was wrong. Every practice was for me a new impossible and as the time got by, I found it less and less imposible but as the book said very beneficial.

I see lately more and more people hearing about XP and then trying to resist to it after seing that it is really extreme. My advices, a few years later, in a much better context as there are much more people that can help, much more tools, more proven cases and more openness.

I want to adopt XP. What is the plan?

When I first started with agile methodologies, as a developer, I had to think about
what to do to be able to make the team,the company, the business, become agile.

Sell XP to yourself

In order to do that I had to sell XP to myself in the first step. I thought that if I'm both the vendor and the client, I must think what would I, as a developer, like to achieve by moving to another way of working. Since I was selling XP to a developer, I had to show myself how I could go home being confident that the code runs, be able to work 40 hours every week, be able to apply my new ideas very easily, without fear, that I might break what I already did. I discovered that TDD and automated tests can give me the confidence and from that I was be able to move faster and add ideas to my project, motivation started going high. Refactoring and simple design, allowed me to minimize rework (smaller and simpler code is easier to debug and enhances flexibility). Continuous integration is very important, as it lets me go home happy and find in the morning if I broke something in the worst case. Continuous integration maximizes the feedback from the code to me. It is a bell telling me that I broke something. It is by running the tests, the break detector.

After a few months, I was able to have reasonable enough confidence in myself to go to the practices skeptics. I could confront them, not always very well, but as in Alistair Cockburn's - 'shu ha ri' the tree levels in aikido, things must go ahead ,slowly but ahead, to the next level of expertise and knowledge.

After I've convinced myself, I could convince other developers.

In convincing other developers about agile/xp practices, I use the 'self training'
technique, where we take a book/article etc and every morning we all stop work for an hour or so and read a chapter, the answer a few questions to see what each understood then talk until the things described there are understood by everyone. More at: http://danbunea.blogspot.com/2005/05/self-training.html. The others could see that I did not invent the things, but they will read them there and I was able to explain how they work and answer their questions.

Slowly I was convincing the developers. It was time to move upper - management

Sell XP to your management

In convincing managers you have to remember that they are managers and they are
running a business. They don't care about better code design and more automated tests. They care about money, and you need to show them how these practices can maximize profits. I wrote a simple sample about how TDD minimizes costs here: http://
danbunea.blogspot.com/2005/09/how-tdd-improves-development-speed-and.html.

I started small, but I tried to explain the advantages of iterations and customer on site (if that's not possible in direct contact with no proxis). If I were to explain again I would expain that these two eliminate communication gaps and show examples how in the past commujication gaps costed money. A lot of money. Then go further to user stories. Another communication problem. I tried to show that writing very detailed spec is very costly and has a high risk of being thrown away when changes occured. When management sees that costs can be reduces and profits improves (don't overpromise!) they start to be convinced. They like statistics.

Selling to the customer

The customer must be informed from the beginning about what you're planning to do and the iterative circle of life (see Ron Jeffries's XP Installed). Convince them about the benefits, using the same methods as for managers. Less time, less money, better quality, running, tested features after just a couple of months, at maximum.

I needed about 2-3.5 years so far and I am not ready. I am still becoming agile, just like everyone else. But the situation is changing. More and more people hear about the good results of XP and that might bring down the resistance wall faster. The most time was ,needed to sell agile to myself. I was slow, eating more then half the time just to see that it might be possible to work. But when I saw that there was a chance, I was just agile infected.

Conclusion - XP won, slow but I'm XP infected now and don't regret a thing

I tried to share a little about my experiences in becoming agile. We
are very small (as a company), and that's a huge advantage, but I have seen good results even in monster companies. Just determination is needed. And XP will win, as it did with me.