Step 3 Add unit tests¶
Now that we know the code compiles, it would also be very neat to know that the code actually does what we expect it to do, and this is what unit tests do.
They tell us that the code (still) does what's it supposed to do. Do not think that when you have unit tests, your code is perfect, unit tests test your code,
but if the test is bad, the code could be bad also. The benefit of unit tests is that when the code changes (refactoring) you know that there are no new bugs introduced,
at least none detected by the tests. Article on wikipedia about unit testing.
Unit tests should be fast, do not confuse them with scenario tests, which may take longer.
A good piece of advice¶
When a customer reports a bug, do NOT start fixing it!
First write a test that simulates the problem, and than fix it. This way you're sure that a future change in the code will not re-introduce the bug, what is called regression.
Updating CCNet.config¶
What we need to do, is tell CCNet to do unit testing, what is easily to do. We only need to add a call to the nant target which will do the unit testing.
So the only needed change is 1 line : <target>unit_test</target>, add this in the nant_target_CI pre-processor section, after the compile target.
With this 1 line change, we just instructed ALL our CI builds to do unit testing!
And a project definition is still only 29 lines.
The nant_target_CI block looks now like this :
1<!-- the nant targets for a CI build -->
2<cb:define name="nant_target_CI">
3 <targetList>
4 <target>clean</target>
5 <target>compile</target>
6 <target>unit_test</target>
7 </targetList>
8</cb:define>
We also need to merge the output, so that's also a small change. Include the File Merge Task in the common publisher section, and instruct it to load the test output file.
The following has to be inserted before the <xmllogger> on line 28.
1<merge>
2 <files>
3 <file>MStest_UnitTestResults.xml</file>
4 </files>
5</merge>
You can find the full CCNet.Config at the bottom of this page.
Updating the build script¶
The real action is off course in the build script. There are many testing frameworks around : NUnit, MBUnit, MS-Test, XUnit, ...
As long as there is a commandline interface, CCNet can work with it. If the tool can output to XML, so much the better. This makes visualizing later on a lot easier.
- we need to know the test file (vsmdi file)
- clean up temporary files, so that we do not accidentally merge wrong test results
- we need to instruct mstest to run only unit tests (no integration tests, or other kinds)
The test file¶
This is an easy change, just add a property containing the file name. Default the name of the vsmdi file is the same as the solution name, but with the vsmdi extension.
No need to change standards, so we can add the property as a calculation on the solution name. This way we still only need to change the solution name in the script file if we want to use it for another solution (or overwrite the value of that property when calling it via Nant)
needed change somewhere on line 10 :
1 <property name="mstest_metadatafile" value="${string::replace(solution, '.sln', '.vsmdi')}" />
You may off course fill in the full vsmdi file name, but I like to enforce standards as much as possible. This makes it a lot easier to maintain hundreds of solutions.
And when people switch team, the build environment and its settings are the same!
Clean up old test files¶
Just to be sure that we do not merge old files, delete any test result files, before we do any action. So we need to update the clean task in the build script.
For ease of referencing we define a variable (property in Nant terms) which holds the unit-test results.
1 <property name="mstest_unit_test_file" value="MStest_UnitTestResults.xml" />
Be sure that the name is the same as you typed in the merge task of CCNet.config.
Next update the clean task, so it deletes this file also :
1<target name="clean" description="deletes all created files">
2 <delete >
3 <fileset>
4 <patternset >
5 <include name="**/bin/**" />
6 <include name="**/obj/**" />
7 <include name="${mstest_unit_test_file}" />
8 </patternset>
9 </fileset>
10 </delete>
11</target>
Instruct to run unit tests only¶
The easiest way to do this is to create a Test List. You can do this via the Test List Editor. Give it the name UnitTests.
Next drag all your unit-tests into this test list.
Now add the unit-test target :
1<target name="unit_test" description="runs the tests" depends="compile">
2 <if test="${file::exists(mstest_metadatafile)}">
3 <exec program="${mstest_exe}">
4 <arg value="/testmetadata:${mstest_metadatafile}" />
5 <arg value="/resultsfile:${mstest_unit_test_file}" />
6 <arg value="/testlist:UnitTests" />
7 </exec>
8 </if>
9</target>
Full build script¶
1<project default="help">
2 <property name="solution" unless="${property::exists('solution')}" value="ProjectX.sln" />
3 <property name="configuration" unless="${property::exists('configuration')}" value="Debug" />
4 <property name="CCNetListenerFile" unless="${property::exists('CCNetListenerFile')}" value="listen.xml" />
5 <property name="msbuildverbose" unless="${property::exists('msbuildverbose')}" value="normal" />
6 <property name="CCNetLabel" unless="${property::exists('CCNetLabel')}" value="0.0.0.0" />
7 <property name="mstest_metadatafile" value="${string::replace(solution, '.sln', '.vsmdi')}" />
8 <property name="mstest_unit_test_file" value="MStest_UnitTestResults.xml" />
9
10 <property overwrite="false" name="msbuildlogger" value="C:\Program Files\CruiseControl.NET\server\MSBuildListener.dll" />
11 <property overwrite="false" name="versionInfofile" value="VersionInfo.cs" />
12
13 <!-- custom scripts -->
14 <script language="C#" prefix="RuWi">
15 <references>
16 <include name="System.Xml.dll" />
17 <include name="System.dll" />
18 </references>
19 <imports>
20 <import namespace="System.Text" />
21 </imports>
22 <code>
23 <![CDATA[
24 [Function("UpdateVersionFile")]
25 public static bool UpdateVersionFile(string inputFile, string newVersion, bool debugMode)
26 {
27 bool ok = true;
28 try
29 {
30 System.IO.StreamReader versionFile = new System.IO.StreamReader(inputFile, System.Text.Encoding.ASCII);
31 string line = "";
32 System.Text.StringBuilder result = new StringBuilder();
33 string searchPatternVersion = @"(\d+\.\d+\.\d+\.\d+)";
34 string searchPatternAssemblyProduct = string.Format(@"AssemblyProduct\({0}(.*?)\{0}", "\"");
35 string replacePatternAssemblyProduct = string.Format(@"AssemblyProduct({0}(Debug)${1}1{2}{0}", "\"", "{", "}");
36
37 while (!versionFile.EndOfStream)
38 {
39 line = versionFile.ReadLine();
40
41 if (System.Text.RegularExpressions.Regex.IsMatch(line, searchPatternVersion) && (line.Contains("AssemblyFileVersion")))
42 {
43 line = System.Text.RegularExpressions.Regex.Replace(line, searchPatternVersion, newVersion);
44 }
45
46 if (debugMode && System.Text.RegularExpressions.Regex.IsMatch(line, searchPatternAssemblyProduct))
47 {
48 line = System.Text.RegularExpressions.Regex.Replace(line, searchPatternAssemblyProduct, replacePatternAssemblyProduct);
49 }
50
51 result.AppendLine(line);
52 }
53
54 versionFile.Close();
55
56 System.IO.StreamWriter updatedVersionfile = new System.IO.StreamWriter(inputFile);
57 updatedVersionfile.Write(result.ToString());
58 updatedVersionfile.Close();
59 }
60 catch (Exception ex)
61 {
62 ok = false;
63 Console.WriteLine(ex.ToString());
64 }
65 return ok;
66 }
67 ]]>
68 </code>
69 </script>
70 <!-- end custom scripts -->
71
72 <target name="help" >
73 <echo message="Removed for keeping the file shorter." />
74 </target>
75
76 <target name="clean" description="deletes all created files">
77 <delete >
78 <fileset>
79 <patternset >
80 <include name="**/bin/**" />
81 <include name="**/obj/**" />
82 <include name="${mstest_unit_test_file}" />
83 </patternset>
84 </fileset>
85 </delete>
86 </target>
87
88 <target name="adjustversion" description="Adjusts the version in the version.info file">
89 <if test="${not file::exists(versionInfofile)}">
90 <fail message="file: ${versionInfofile} which must contains the version info was NOT found" />
91 </if>
92
93 <echo message="Setting version to ${CCNetLabel}" />
94
95 <property name="debugMode" value = "False" />
96 <property name="debugMode" value = "True" if="${configuration=='Debug'}" />
97 <if test="${not RuWi::UpdateVersionFile(versionInfofile,CCNetLabel,debugMode)}">
98 <fail message="updating file: ${versionInfofile} which must contains the version info failed" />
99 </if>
100 </target>
101
102 <target name="compile" description="compiles the solution in the wanted configuration" depends="adjustversion">
103 <msbuild project="${solution}" >
104 <arg value="/p:Configuration=${configuration}" />
105 <arg value="/p:CCNetListenerFile=${CCNetListenerFile}" />
106 <arg value="/v:${msbuildverbose}" />
107 <arg value="/l:${msbuildlogger}" />
108 </msbuild>
109 </target>
110
111 <target name="unit_test" description="runs the unit tests" depends="compile">
112 <if test="${file::exists(mstest_metadatafile)}">
113 <exec program="${mstest_exe}">
114 <arg value="/testmetadata:${mstest_metadatafile}" />
115 <arg value="/resultsfile:${mstest_unit_test_file}" />
116 <arg value="/testlist:UnitTests" />
117 </exec>
118 </if>
119 </target>
120 </project>
121Full CCNet.config¶
1 <cruisecontrol xmlns:cb="urn:ccnet.config.builder">
2 <!-- preprocessor settings -->
3 <cb:define WorkingDir="D:\WorkingFolders\" />
4 <cb:define WorkingMainDir="D:\ArtifactFolders\" />
5 <cb:define ArtifactsDir="\Artifacts" />
6
7 <!-- TFS settings for a CI build -->
8 <cb:define name="vsts_ci">
9 <server>http://tfs-server:8080/tfs/default/</server>
10 <username>cruise</username>
11 <password>**********</password>
12 <domain>tfs-server</domain>
13 <autoGetSource>true</autoGetSource>
14 <cleanCopy>true</cleanCopy>
15 <force>true</force>
16 <deleteWorkspace>true</deleteWorkspace>
17 </cb:define>
18
19 <!-- all projects will have at least these publishers -->
20 <cb:define name="common_publishers">
21 <merge>
22 <files>
23 <file>MStest_UnitTestResults.xml</file>
24 </files>
25 </merge>
26 <xmllogger />
27 <statistics />
28 <modificationHistory onlyLogWhenChangesFound="true" />
29 <artifactcleanup cleanUpMethod="KeepLastXSubDirs" cleanUpValue="2" />
30 <artifactcleanup cleanUpMethod="KeepLastXBuilds" cleanUpValue="25000" />
31 <!-- email the breakers on a broken build and when it's fixed -->
32 <email from="CruiseControl@TheBuilder.com"
33 mailhost="TheMailer.Company.com"
34 includeDetails="TRUE">
35 <groups/>
36 <users/>
37 <converters>
38 <ldapConverter domainName="Company" />
39 </converters>
40 <modifierNotificationTypes>
41 <NotificationType>Failed</NotificationType>
42 <NotificationType>Fixed</NotificationType>
43 </modifierNotificationTypes>
44 </email>
45 </cb:define>
46
47 <!-- the Nant build arguments for a CI build -->
48 <cb:define name="nant_CI">
49 <executable>c:\Tools\nant\bin\nant.exe</executable>
50 <nologo>true</nologo>
51 <buildTimeoutSeconds>1800</buildTimeoutSeconds>
52 <buildArgs>-D:useExtraMsbuildLogger=true -D:isCI=true -listener:CCNetListener,CCNetListener -D:configuration=Debug</buildArgs>
53 </cb:define>
54
55 <!-- the nant targets for a CI build -->
56 <cb:define name="nant_target_CI">
57 <targetList>
58 <target>clean</target>
59 <target>compile</target>
60 <target>unit_test</target>
61 </targetList>
62 </cb:define>
63
64 <!-- end preprocessor settings -->
65
66 <!-- Projects -->
67 <cb:scope ProjectName="ProjectX">
68 <cb:define ProjectType="_CI" />
69 <project name="$(ProjectName)$(ProjectType)" queue="Q1" queuePriority="901">
70 <workingDirectory>$(WorkingDir)$(ProjectName)$(ProjectType)</workingDirectory>
71 <artifactDirectory>$(WorkingMainDir)$(ProjectName)$(ProjectType)$(ArtifactsDir)</artifactDirectory>
72
73 <labeller type="defaultlabeller" />
74
75 <intervalTrigger name="continuous" seconds="30" buildCondition="IfModificationExists" initialSeconds="30" />
76
77 <sourcecontrol type="vsts">
78 <workspace>$(ProjectName)</workspace>
79 <project>$/$(ProjectName)/Trunk</project>
80 <cb:vsts_ci/>
81 </sourcecontrol>
82
83 <tasks>
84 <nant>
85 <cb:nant_CI/>
86 <cb:nant_target_CI />
87 </nant>
88 </tasks>
89
90 <publishers>
91 <cb:common_publishers />
92 </publishers>
93
94 </project>
95 </cb:scope>
96
97 <!-- End Projects -->
98 </cruisecontrol>