Automated Testing with BizTalk Server

Writing and executing unit tests for a BizTalk Server application can be challenging. How do you test your code when BizTalk Server is executing part of it? All the various artifacts (e.g., schemas, maps, orchestrations) are run inside BizTalk Server. Furthermore there are dependencies on input data (e.g., files). And you’re likely to be updating something on the other end like a database or ERP system. What control do you have of these input and outputs? Or can you mock them out? At what level do you write these tests? Small unit tests or larger integration tests? Some may found the logistical issues so daunting so as to pass up on the effort of automated testing with BizTalk altogether. Well take heart, it can be done!        

There are a number of articles and tools available. Here are the ones we’ve examined: BizUnit 2006, BizUnit v3.1 (was 3.0 RC1 at the time we examined it), and Excellence. Excellence looked like it needed to have the DebugView tool up and running during test executing so didn’t qualify for our requirement of ‘hands-off’ test runs on a build server. BizUnit certainly looks like a worthy tool and in the end we’ve opted for a similar system but leveraging BizTalk Server itself to pick up files, run orchestrations, etc. So it’s important to note that these are not strictly unit tests. They run larger chunks of the application and are designed for executing at build time to validate the system. Perhaps they’re closer to integration tests. Regardless, the ultimate goal was to prove that orchestrations can run, and produce a consistent set of output given a sample input.        

In our case we’re dealing with an application built in Visual Studio 2005 that runs inside BizTalk Server 2006. But the process described applies equally to the latest versions of these products (Visual Studio 2008, BizTalk Server 2006 R2, and BizTalk Server 2009). For reasons described below, you will need both Visual Studio and BizTalk Server on the build machine. The solution we’ve come up mocks out the database. This is done by using Send Ports that are of type File instead of Database. So all writing done by the orchestrations ends up in XML files being produced. These output files can then easily be compared with expected versions to determine if the tests succeed.        

        

The above illustration describes the end-to-end process. CruiseControl.NET is used to drive all the steps and report on progress including success or failure. Here are details of each step:        

  1. Get source
    1. This is done using CruiseControl’s Source Control Block. Source can be retrieved from both Visual SourceSafe and Team Foundation Server (among others).
  2. Compile using dev env
    1. It’s not possible to compile BizTalk Server Projects using msbuild. So you need to run the IDE with a command line switch to compile. Here’s an example:

      C:\Program Files\Microsoft Visual Studio 8\Common7\IDE\devenv.com
         “C:\BuildArea\mySystem\Source\mySolution.sln” /build Release    

    2. Note that mstest (used to run unit tests in last step below) requires a full Visual Studio install anyway as Microsoft doesn’t ship it separately. So you will need Visual Studio on your build server.
  3. Undeploy BizTalk app
    1. This solution completely removes the BizTalk Application along with all its artifacts. However the various artifacts need to be stopped and unenlisted before app removal is possible – this required the use of a custom console app that made use of the Microsoft.BizTalk.ExplorerOM library
    2. Use the BTSTask command line utility to remove the BizTalk Application. Here’s an example:

      call BTSTask RemoveApp /ApplicationName:myApp
         /Server:myServer /Database:BizTalkMgmtDb
         

    3. Note that if the Application is the Default one, it cannot be removed – add a dummy application making it the default, then remove, then after deploying the new app as default, remove the dummy one
  4. Delete out files
    1. Just to be sure that any new files output are indeed produced by the new build, all existing output files are deleted
  5. Deploy BizTalk app
    1. Follow the steps of Undeploy above in reverse
    2. First add the new application. Here’s an example:

      call BTSTask AddApp /ApplicationName:myApp /Default
         /Description:”myBizTalk Application for Test”
         /Server:myServer /Database:BizTalkMgmtDb
         

    3. Then add in all the resources (assemblies). Note that there are at least two types: those that are BizTalk specific assemblies and other standard .NET referenced assemblies. The referenced assemblies must be added before the BizTalk ones referencing them. Here’s an example of both:call BTSTask AddResource /ApplicationName:myApp
         /Type:System.BizTalk:Assembly /Overwrite
         /Source:”myCompany.myProduct.myHelper.dll”
         /Server:myServer /Database:BizTalkMgmtDb
         /Options:GacOnAdd
      call BTSTask AddResource /ApplicationName:myApp
         /Type:System.BizTalk:BizTalkAssembly
         /Overwrite /Source:”myCompany.myProduct.myBTlibrary.dll”
         /Server:myServer /Database:BizTalkMgmtDb
         /Options:GacOnAdd
        
    4. Next set all the bindings. This can be done by importing a bindings XML file (which was produced by an export in the first place). Here’s an example:call BTSTask ImportBindings /Source:”bindings.xml”
         /ApplicationName:myApp /Server:myServer /Database:BizTalkMgmtDb
        
    5. Then use the same custom command line utility (used in undeploy step above) to start up receive locations, send ports, orchestrations, and host.
    6. Finally, it will be necessary to stop and restart the host so its cache is updated. Here’s an example:net stop “BizTalk Service BizTalk Group : BizTalkServerApplication”
      net start “BizTalk Service BizTalk Group : BizTalkServerApplication”
        
  6. Copy in files
    1. Use the xcopy.exe command to copy in files into BizTalk pickup locations specified on Receive Locations – BizTalk should now process these
  7. Run unit tests
    1. These are standard unit tests that are run as follows:


              

      C:\Program Files\Microsoft Visual Studio 8\Common7\IDE\mstest.exe
              

      C:\BuildArea\mySystem\Source
              

      /testcontainer:myCompany.myProduct.myBTlibrary
      .UnitTests\bin\Release\myCompany.myProduct
      .myBTlibrary.UnitTests.dll
      /runconfig:localtestrun.Testrunconfig
      /resultsfile:C:\BuildArea\mySystem\myCompany
      .myProduct.myBTlibrary.UnitTestResults.xml
              


              

    2. Remember to merge the test results into the overall build report as follows:

         


              

      myCompany.myProduct.myBTlibrary.UnitTestResults.xml

             

               

         

          

            


            

       

         


              

    3. The code for the unit test itself is pretty straightforward:
      1. Use the FileSystemWatcher object to wait a reasonable amount of time for each file (make each file check a separate unit test).
      2. Then do a file compare of the actual output file’s contents against a static one kept in the code base. You can get the file compare C# code from Microsoft here: http://support.microsoft.com/kb/320348

I hope this helps others to perform automated testing with BizTalk solutions. Please give me a shout if there is any more detail I can provide.        

- Krip      

One Response to Automated Testing with BizTalk Server

  1. [...] an example of just how much you can do in a script run continuously, see this great blog post on Automated Testing with BizTalk Server. The point is you can and should do the entire enchilada if not on every build, at least daily.  [...]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: