When it comes to automated tests the first impression everyone (including myself) thinks that that would certainly increase the amount of time a certain feature is developed in. However as I started to practice TDD, Test First Programming, increasing the number of tests I came to the conclusion that automated tests actually allow you and the team to develop faster then without.
Move faster ahead in development
A simple comparison of programming vs walking on a rope. When walking on a rope, at a height, with no safety net, walking is much harder , much slower, as the risk is higher. Only very experienced people can walk fast. When having the safety net below, the rist of falling suddenly doesn’t really concern you that much. If you fall, you just go back and try again. You do not jeopardize your life.
With programming, the bigger the code base gets, the hardest is to move further, to change all code, being very afraid, that you could mess it up really good. If you have automated tests, they are your safety net, allowing you to see if one mistake you did affects the system, and you have to roll it back just like falling off the rope makes you go back to the start. A little time is lost, but not the system, you do not need weeks to restore it. The fact that you can see the mistakes you do very fast, makes you a lot braver, having the courage to change a lot, adapt, improve, refactor a lot, then when you don't know what you're doing. The safety net, composed of tests gives you that very fast feedback that something is broken. These tests are also called change detectors. When you do a change, the tests will detect very soon if it breaks something.
Faster development by automating the process
In a recent project, we had a business process on a web application that required the user to go, like in a wizard, trough 9 screens to be able to ship an order. When we were developing it, we had to go 9 pages, filling 9 pages of data to see if the order can or cannot be shipped. We had a lot of times, the order being rejected by reasons that we forgot to fill in some field. This process of filling in all the fields until screen 9 took usually 2-3 minutes, so after 2-3 minutes to see an Exception is not really nice because it just means you have to take another 2-3 minutes to refill and hope this time it will be ok. After a few attempts, I realized this was not the way to go I said what about we automate the whole filling process, , and being a web application we used Selenium (http://selenium.thoughtworks.com) a great functional testing tool for web applications, which did the whole 2-3 minutes manual process in under 4 seconds. Now instead of starting the first page, we ran the test each time until we got the process of shipping right, with all the validations. It took 15 minutes to write the test but it probably spared us of a lot more manual testing. And when it worked, it could be retested whenever we felt like it, in 6 seconds. It was added to the test suite and we made sure that it always succeeded, so we knew all the time that even with our 'big new changes' it still worked.
Minimizing debugging time and cost
Usually people have the tendency to think that by adding automated tests the development time increases. They say that if you have something to implement in 4 hours and doing the tests would take 2 hours, you have just lost 2 hour writing the tests. Coming back with our feet on the ground we can soon realize, that that thinking is superficial as it presumes that the code done in the 4 hours of development is bug free. That is bug free and will stay like that even if the system around it changes, gets bigger etc. The sad truth is that most code is not bug free for the first time, and a lot of time is lost debugging it. In many cases the time to debug is bigger then the time it was developed in and this time usually increases as the time goes. The biggest problem is maybe that usually debugging time is not measurable or it is hard to predict how much time it will take. So 4h+??? = ???
If the bug is discovered by the client, solving it is usually not paid, so it's cost just doubles or grows even more. Many times you end up with a negative balance when saying 4h developing=10$ then fixing the bug 4h=10$. The company ends up with 10-10=0$. And this is a good case.
Automated tests, with a good coverage of possibilities, will reduce even eliminate the debug time and if they are kept in a test suite that is run very frequently will make sure they remain bug free. In this case you use 6h, but you're very sure you've done it right.
Automated tests help you get things right the first time, costing less
A similar sample was with my wife. She went to buy paint for our doors. She bought two cans, one with 0.85 the other with 1.55. When we started painting we quickly realized that the 0.85 paint , needed to be put twice on each door, so is actually costed double: 1.7. Now if she bought two cans of 1.55 we would have paid less money and less effort. The very same goes with automated tests. They seem to cost more then developing without them but they usually end up costing less, as you get the things right the first time and not after the deployment, when you fix the bugs from your own pocket.
I hope the 3 reasons above gave you an idea, why I think developing being backed up by automated tests actually help you develop faster, then developing without them .
Thursday, September 22, 2005
Thursday, September 01, 2005
TDD without FIT or NUnit: PocketPC and .NET CF
Can you be agile and "test-infected" in environements that provide no or little similarity to the platforms used for doing unit and functional tests? Can you still do test first programming in an development environement where you don't have NUnit or FIT?
Test Driven Development and Test First Programming
Let's start from the roots of Test Driven Development and Test First Programming. I noticed that around these two concepts there is a little missundestanding, some people condidering them to be the same, others considering them different. My opinion is that the two are slightly different, TDD being a method of programming where you think about how you are going to implement a feature before implementing it. The second, TFP is a more concrete approach, that says to write a test before you write the code. So where's the difference. The difference in my opinion is that TDD does not really force you to write the test before writing the code or writing the test at all. It just says to think about how you could automate the testing of the feature you want to implement, how to design a testable piece of software. TFP forces you to write the test first, whereas TDD forces you to design thinking about testing, but giving you the option to write the test when you want to.
So, as a conclusion, in my opinion TFP is a particular case of TDD whichis more generic. Don't get me wrong I do like very much to write the tests before.
Were there automated tests before xUnit and/or FIT?
Coming back to our problem with TDD on environements where you have no or little documentation, where you're a pioneer, I would say that we need to go back to the basics. Automated tests do not mean NUnit (xUnit) or Fit. These two frameworks and the ones deived were build to help making automated tests easier, but what was before these arrived?. Did people not write automated tests for their software? I doubt that. But how did they do automated tests it?
Well I belive that there were several techniques but the most popular were either adding a test button on the UI (desktop apps , we are talking about early internet or before internet times) or by adding a main method to the class (java). This were probably the most populat methods of building both functional and unit tests.
The 'test button' functional tests actualy put values into the controls on the form then simulated the click of a button then checking the results of that action. As a dear friend of mine said, this had the drawback that you needed to remove the test button when going into production, and more complex testing scenarios were increasingly hard to do.
How do you build functional tests on Windows.Forms apps for .NET CF?
You have no NUnitForms, no NUnit not a thing like this. What do you do? In order to be able to automate the testing of my forms, I thought how about adding the test button, but I remebered I wasn't particularily fond of it. Then I had the idea that for each form, I could add a "region" where I can have my test helper methods, each of them starting with Test. This meant, that in my application designed for PocketPC and used to collect measurements for medical equipment on daily, weekly, monthly and quarterly basis, having for each of these at least two forms, one dispalying the
list of available measurements and the add/edit/delete methods and the other to be able to add or edit a record, with ok/cancel buttons.
In each form, I added a region "Testers" where I added my testing helper methods like:
#region testers
public void TestTypeQuantity(double q)
{
TestingHelper.Type(this.txtQuantity, q);
}
public void TestSelectMeasurementUnit(int index)
{
TestingHelper.SelectMeasurementUnit(this.cbxQuantity, index);
}
public void TestClickAddDailyMeasurement()
{
TestingHelper.Click(btnAddDailyMeasurement);
}
#endregion
In order to be able to interact easier with the controls I have a TestingHelper class, that helps me work with the controls as if I were the user. For instance in order to click on a button, I have:
public virtual void Click()
{
if (Control.Visible)
{
FireEvent( "Click" );
}
else
{
throw new Exception("Control " +control+" is not visible." );
}
}
public void FireEvent( string eventName )
{
MethodInfo minfo = Control.GetType().GetMethod( "On" + eventName, BindingFlags.Instance BindingFlags.Public BindingFlags.NonPublic );
ParameterInfo[] param = minfo.GetParameters();
Type parameterType = param[0].ParameterType;
minfo.Invoke( Control, new object[]
{
Activator.CreateInstance( parameterType )
} );
}
or for typing something in a TextBox:
public static void Type(Control control, string text)
{
control.Focus();
control.Text = text;
}
The test and handling modal windows
For the test, I have a simple singleton class, that tests like:
public class TestingInstance
{
#region members
private static TestingInstance testingInstance = null;
private Hashtable expectedModals = new Hashtable();
#endregion
private TestingInstance()
{
//specify what method to be invoked when DailyChecksForm modal is shown
ExpectModal(typeof(DailyChecksForm),"RunDailyChecksForm");
}
private void ExpectModal(Type formType, string param2)
{
this.expectedModals[formType] = param2;
}
public static TestingInstance Instance
{
get
{
if(testingInstance==null)
{
testingInstance = new TestingInstance();
}
return testingInstance;
}
}
public void OnModalShown(Form form)
{
string methodName = expectedModals[form.GetType()] as string;
MethodInfo mi = this.GetType().GetMethod(methodName);
object[] parameters = new object[]{form};
mi.Invoke(this,parameters);
}
public void Run()
{
MainForm mf = new MainForm();
mf.Show();
mf.TestSelectUnitByIndex(2);
mf.Close();
}
public void RunDailyChecksForm(Form form)
{
DailyChecksForm mf = form as DailyChecksForm;
//mechanical checks
mf.TestSelectMechanicalChecksTab();
mf.TestTypeSignature("Dan Bunea");
//x-ray tube
mf.TestSelectXRayTubeTab();
mf.TestSelectTargetFilter1(1);
mf.TestTypeReading1("1.0");
mf.TestSelectTargetFilter2(1);
mf.TestTypeReading2("2.0");
mf.TestSelectTargetFilter3(1);
mf.TestTypeReading3("3.0");
mf.TestSelectTargetFilter4(1);
mf.TestTypeReading4("4.0");
//4cm
mf.TestSelect4cmTab();
mf.TestTypekV("1");
mf.TestTypeMAS("2");
mf.TestSelectTargetFilter4cm(1);
mf.TestTypeDensity("3");
//click ok and save
mf.TestClickOK();
}
...
Since my Win32 knowledge level is low, what could I do to see when a modal form is being shown. Well as always there is a solution, you just have to use your head a little.
All my forms extended a TestableForm, which on load notifies the testing instance that it has been shown . For instance in DailyMeasurementForm I have:
private void DailyChecksForm_Load(object sender, System.EventArgs e)
{
TestingInstance.Instance.OnModalShown(this);
}
which when is loaded and shown tells my TestingInstance that it has been shown, then my testing instance looks in the list of expected modal forms for the type and invokes the specified method, in our case, we have in the tesing instance constructor:
ExpectModal(typeof(DailyChecksForm),"RunDailyChecksForm");
which means that when DailyChecksForm is shown, it will invoke OnModalShown in TestingInstance and that will invoke RunDailyChecksForm. Simple , isn't it?
How do you start the test on the mobile device?
public static void Main()
{
if(IsTestingMode())
{
TestingInstance.Instance.Run();
}
else
{
Application.Run(new MainForm());
}
}
public static bool IsTestingMode()
{
return true;//this can be modified very easely
}
Maybe I could have manipulated VS.NET and would have found a more elegant way of doing it, but let's not forget that I needed to develop software that could be automatically tested not really invent the new all good testing framework for PDAa and respecting the two agile priciples of simplicity and evolutionary design, I think the code is good enough and can be refactored and improved later.
Conclusion
The bad thing about my solution is that it mixes testing code with production code, but I think that even with this drawback, it is still better then testing everything manually. And maybe the test code could be removed from the production code using preprocessor directives, but that is just a supposition.
So do we really need NUunit?...TDD can be done without but apps shouldn't be done without TDD in my opinion. But I guess that I am just "test infected" ... :)
Test Driven Development and Test First Programming
Let's start from the roots of Test Driven Development and Test First Programming. I noticed that around these two concepts there is a little missundestanding, some people condidering them to be the same, others considering them different. My opinion is that the two are slightly different, TDD being a method of programming where you think about how you are going to implement a feature before implementing it. The second, TFP is a more concrete approach, that says to write a test before you write the code. So where's the difference. The difference in my opinion is that TDD does not really force you to write the test before writing the code or writing the test at all. It just says to think about how you could automate the testing of the feature you want to implement, how to design a testable piece of software. TFP forces you to write the test first, whereas TDD forces you to design thinking about testing, but giving you the option to write the test when you want to.
So, as a conclusion, in my opinion TFP is a particular case of TDD whichis more generic. Don't get me wrong I do like very much to write the tests before.
Were there automated tests before xUnit and/or FIT?
Coming back to our problem with TDD on environements where you have no or little documentation, where you're a pioneer, I would say that we need to go back to the basics. Automated tests do not mean NUnit (xUnit) or Fit. These two frameworks and the ones deived were build to help making automated tests easier, but what was before these arrived?. Did people not write automated tests for their software? I doubt that. But how did they do automated tests it?
Well I belive that there were several techniques but the most popular were either adding a test button on the UI (desktop apps , we are talking about early internet or before internet times) or by adding a main method to the class (java). This were probably the most populat methods of building both functional and unit tests.
The 'test button' functional tests actualy put values into the controls on the form then simulated the click of a button then checking the results of that action. As a dear friend of mine said, this had the drawback that you needed to remove the test button when going into production, and more complex testing scenarios were increasingly hard to do.
How do you build functional tests on Windows.Forms apps for .NET CF?
You have no NUnitForms, no NUnit not a thing like this. What do you do? In order to be able to automate the testing of my forms, I thought how about adding the test button, but I remebered I wasn't particularily fond of it. Then I had the idea that for each form, I could add a "region" where I can have my test helper methods, each of them starting with Test. This meant, that in my application designed for PocketPC and used to collect measurements for medical equipment on daily, weekly, monthly and quarterly basis, having for each of these at least two forms, one dispalying the
list of available measurements and the add/edit/delete methods and the other to be able to add or edit a record, with ok/cancel buttons.
In each form, I added a region "Testers" where I added my testing helper methods like:
#region testers
public void TestTypeQuantity(double q)
{
TestingHelper.Type(this.txtQuantity, q);
}
public void TestSelectMeasurementUnit(int index)
{
TestingHelper.SelectMeasurementUnit(this.cbxQuantity, index);
}
public void TestClickAddDailyMeasurement()
{
TestingHelper.Click(btnAddDailyMeasurement);
}
#endregion
In order to be able to interact easier with the controls I have a TestingHelper class, that helps me work with the controls as if I were the user. For instance in order to click on a button, I have:
public virtual void Click()
{
if (Control.Visible)
{
FireEvent( "Click" );
}
else
{
throw new Exception("Control " +control+" is not visible." );
}
}
public void FireEvent( string eventName )
{
MethodInfo minfo = Control.GetType().GetMethod( "On" + eventName, BindingFlags.Instance BindingFlags.Public BindingFlags.NonPublic );
ParameterInfo[] param = minfo.GetParameters();
Type parameterType = param[0].ParameterType;
minfo.Invoke( Control, new object[]
{
Activator.CreateInstance( parameterType )
} );
}
or for typing something in a TextBox:
public static void Type(Control control, string text)
{
control.Focus();
control.Text = text;
}
The test and handling modal windows
For the test, I have a simple singleton class, that tests like:
public class TestingInstance
{
#region members
private static TestingInstance testingInstance = null;
private Hashtable expectedModals = new Hashtable();
#endregion
private TestingInstance()
{
//specify what method to be invoked when DailyChecksForm modal is shown
ExpectModal(typeof(DailyChecksForm),"RunDailyChecksForm");
}
private void ExpectModal(Type formType, string param2)
{
this.expectedModals[formType] = param2;
}
public static TestingInstance Instance
{
get
{
if(testingInstance==null)
{
testingInstance = new TestingInstance();
}
return testingInstance;
}
}
public void OnModalShown(Form form)
{
string methodName = expectedModals[form.GetType()] as string;
MethodInfo mi = this.GetType().GetMethod(methodName);
object[] parameters = new object[]{form};
mi.Invoke(this,parameters);
}
public void Run()
{
MainForm mf = new MainForm();
mf.Show();
mf.TestSelectUnitByIndex(2);
mf.Close();
}
public void RunDailyChecksForm(Form form)
{
DailyChecksForm mf = form as DailyChecksForm;
//mechanical checks
mf.TestSelectMechanicalChecksTab();
mf.TestTypeSignature("Dan Bunea");
//x-ray tube
mf.TestSelectXRayTubeTab();
mf.TestSelectTargetFilter1(1);
mf.TestTypeReading1("1.0");
mf.TestSelectTargetFilter2(1);
mf.TestTypeReading2("2.0");
mf.TestSelectTargetFilter3(1);
mf.TestTypeReading3("3.0");
mf.TestSelectTargetFilter4(1);
mf.TestTypeReading4("4.0");
//4cm
mf.TestSelect4cmTab();
mf.TestTypekV("1");
mf.TestTypeMAS("2");
mf.TestSelectTargetFilter4cm(1);
mf.TestTypeDensity("3");
//click ok and save
mf.TestClickOK();
}
...
Since my Win32 knowledge level is low, what could I do to see when a modal form is being shown. Well as always there is a solution, you just have to use your head a little.
All my forms extended a TestableForm, which on load notifies the testing instance that it has been shown . For instance in DailyMeasurementForm I have:
private void DailyChecksForm_Load(object sender, System.EventArgs e)
{
TestingInstance.Instance.OnModalShown(this);
}
which when is loaded and shown tells my TestingInstance that it has been shown, then my testing instance looks in the list of expected modal forms for the type and invokes the specified method, in our case, we have in the tesing instance constructor:
ExpectModal(typeof(DailyChecksForm),"RunDailyChecksForm");
which means that when DailyChecksForm is shown, it will invoke OnModalShown in TestingInstance and that will invoke RunDailyChecksForm. Simple , isn't it?
How do you start the test on the mobile device?
public static void Main()
{
if(IsTestingMode())
{
TestingInstance.Instance.Run();
}
else
{
Application.Run(new MainForm());
}
}
public static bool IsTestingMode()
{
return true;//this can be modified very easely
}
Maybe I could have manipulated VS.NET and would have found a more elegant way of doing it, but let's not forget that I needed to develop software that could be automatically tested not really invent the new all good testing framework for PDAa and respecting the two agile priciples of simplicity and evolutionary design, I think the code is good enough and can be refactored and improved later.
Conclusion
The bad thing about my solution is that it mixes testing code with production code, but I think that even with this drawback, it is still better then testing everything manually. And maybe the test code could be removed from the production code using preprocessor directives, but that is just a supposition.
So do we really need NUunit?...TDD can be done without but apps shouldn't be done without TDD in my opinion. But I guess that I am just "test infected" ... :)
Subscribe to:
Posts (Atom)