Web Service Testing with soapUI

For a few years now I have used the open source application soapUI to simplify Web Service testing.  Both to test services that I have written and to test external services that I have had to consume.  In this post I'll cover the basic types of testing you can do with soapUI and provide some practical tips on how to use soapUI when working with Web Services.  If you don't want to go through the detailed examples I suggest jumping straight to the Practical Tips section at the end.

Getting Started

After downloading and installing soapUI the easiest way to get started is to create a new project from an initial WSDL or WADL.  For demonstration purposes I'll use the free Weather Web Service at http://www.webservicex.net/globalweather.asmx.  We will create a test for each operation in this SOAP Web Service.  We will also create a Test Suite, that allows us to run all our tests at the click of a button.  Furthermore, we will create a Load Test for our Test Suite and a Mock Service to simulate the functionality of the Web Service.

To do this go to File and select New soapUI Project.  Populate the New soapUI Project screen in the following manner and click OK:

Basically, paste the URL to the web service WSDL file into the Initial WSDL/WADL field and additionally check the field to generate a MockService.  SoapUI will now load the Web Service definition and ask you questions about how to create the MockService:

For this demonstration change the path to "/WeatherMock" and check the Starts the MockService immediately checkbox, but otherwise accept the defaults.  After clicking OK, specify the name of the Mock Service as "Weather MockService" (or a name of your choice) and click OK again. 

This will complete generating artifacts for the SOAP 1.2 version of the GlobalWeather Web Service.
Next you will be asked the exact same questions, for the SOAP 1.1 version.  Since we don't plan to use SOAP 1.1 you can just hit Cancel.  (I haven't found a way to configure soapUI to only generate artifacts for SOAP 1.2.  If you know how to, please let me know! :)

Running Tests

To execute a given test, drill down to the requests that soapUI auto-generated and insert values that make sense for the given Web Service operation.  In our case, in the Navigator on the left, click on GlobalWeatherSoap12, then GetCitiesByCountry and then double-click on Request 1.
We'll set the Country name as Iceland and then hit the green arrow button to execute the test.

Voila, you should get results back from the Web Service, a list of Icelandic cities.  To check if the Web Service response is valid (conforms to the WSDL), right-click in the results window and click Validate.

Lets give this test the name "City Test".  (Right-click Request 1 and select rename) 
Then do a similar test for the GetWeather operation.  Click on GetWeather and then double-click on Request 1. Put in "Reykjavik" for CityName and "Iceland" for CountryName and run the test.  Also rename the test to "Weather Test".  After running the test, the results should look something like this:

Build a TestSuite

Now that we have created two tests, lets add them to a TestSuite so we can easily re-run them at any time.  In the Navigator on the left, right-click GlobalWeatherSoap12 and select Generate TestSuite.  Select Use existing Requests in interface, and check Generates a default LoadTest for each created TestCase and Single TestCase with one Request for each Operation:  

After clicking OK give the TestSuite the name "Weather TestSuite".   Double-click on GlobalWeatherSoap12 TestSuite under Weather TestSuite to show the TestCase editor.
Then click the green arrow button to run the TestSuite.  If all goes well you should be presented with a results-screen like this, indicating a successful test run:

If the TestSuite run fails you'll see FAILED at the top instead of FINISHED and you should get a message explaining why a given test failed.

Add Assertions

To make the tests more meaningful lets add some assertions to validate the responses from the Web Service we are calling.  You do this by opening up the test Request Editor (double-click on a given test step under TestSuite in the Navigator) and then click the plus-sign next to the green arrow button (second from the left).

For both tests lets add the assertion called SOAP Response, to ensure the Web Service is returning a valid SOAP response.  Then add the assertion called Response SLA and specify the response time as 2000 ms.  That basically means that we are going to consider the test a failure if we don't get a response within two seconds.  Lastly, lets add some content validation by selecting a Contains validation.  For the GetCitiesByCountry operation add the string "Iceland" as the content to expect in the response, and for the GetWeather operation add the string "Success". 

Then run the TestSuite again to make sure you get success-results.

Run a Load Test

When we created the TestSuite in last step, we told soapUI to generate a Load Test as well.  You'll find it under the TestSuite in the Navigator on the left, under the heading Load Test.  The default name is LoadTest 1.  Double click on it in the Navigator to open it up.  Before running the LoadTest you can tweak such parameters as number of concurrent threads to run, length of the test-run, and delay between tests.  Once you have made the desired configurations hit the green arrow button to run the Load Test:

In this sample, 5 concurrent threads are running the TestSuite for 60 seconds with a random wait of up to 1 second between the start of each test.
You can track the progress of the load test with the progress bar in the upper right corner.  If it reaches a 100% without reporting any test errors you are good to go.

Use the Mock Service

Back when we imported the Weather Web Service we told soapUI to generate and start a Mock Service.  That service can now be accessed at http://localhost:8088/WeatherMock.  This is  convenient for example if you are developing against a Web Service that has been designed (WSDL/WADL available) but not yet implemented.  Then you can have the Mock return an actual Web Service response to test your code even though the actual implementation hasn't been completed.  A default response has already been generated (under WeatherMock Service,  GetCitiesByCountry, and Response 1), which you can edit as you like.

You can also have the Mock service return different responses depending on which request it receives.  To demonstrate this, lets create two new MockResponses for the GetWeather operation:

• Click on GetWeather under the Weather MockSerive in the Navigator and select New MockResponse.  
• Give it the name "ReykjavikWeatherResponse".  Accept the automatically generated response, but put the value "Reykjavik" in the GetWeatherResult tag.  (Or even better copy the actual response from calling GetWeather for Reykjavik, which should give you a fully valid response).  
• Create another response called AkureyriWeatherResponse and put the text "Akureyri" in the GetWeatherResult tag.  

Now put logic in the Mock service for when to return each response:

• Double-click on GetWeather in the Navigator to show the MockOperation Editor.  
• Select ReykjavikWeatherResponse under MockResponses.
• Select QUERY_MATCH under Dispatch.
• Click the plus sign to add a new match. 
• Give it the name "Reykjavik". 
• Select Reykjavik and then populate the XPath value with:
  
  declare namespace web='http://www.webserviceX.NET';
  declare namespace soap='http://www.w3.org/2003/05/soap-envelope';
  /soap:Envelope/soap:Body/web:GetWeather/web:CityName
  
  This XPath query will grab the value from the CityName tag.
• Under Expected Value enter "Reykjavik".
• Under Dispatch to, select ReykjavikWeatherResponse.
• Repeat the same steps to create a Match that returns AkureyriWeatherResponse when the city name in the request is Akureyri.

Now you can test your mock Match-logic by opening up your GetWeather test and adding http://localhost:8088/WeatherMock as the endpoint to use (select the current endpoint in the dropdown and pick add new endpoint...).  Then run the test and play around with changing the city name in the request to get different responses from the Mock service.

Additionally, if you want to test your client-side error handling you can have soapUI generate a soap:Fault response and have your Mock return it.  You do that by creating a MockResponse and then clicking the apostrophe icon in the MockResponse Editor.  Then edit the auto generated response as appropriate.

Practical Tips: What to use soapUI for

Now that we have covered the basics of soapUI, here are some practical tips for putting it to use during your software development.

Test Web Services You Have to Consume

When consuming external services, before delving into code, use soapUI to "kick the tires" of the Web Service.  This especially applies when consuming newly written services.  Rather than potentially spending hours pulling your hair over why your client code isn't working, spend a few minutes with soapUI validating and getting familiar with the Web Service you are about to consume. In particular:

• Create and run simple tests for key Web Service operations
• Make sure there are no security/access problems
• Is the web service using some proprietary authentication protocol (NTLMv2 comes to mind) that might give you trouble during implementation?
  
• Have soapUI validate that responses conform to the Web Service contract (WSDL/WADL)
• Visually inspect responses 
• Do they make sense or are they some illegible auto generated garble that should really be cleaned up and restructured by the Web Service developer? 
• Do the responses meet your needs?
• Test a few boundary cases
• Does the Web Service implement proper error handling?  Or does it blow up with an HTTP 500 error or some non SOAP compliant text message?  Does the level of error reporting meet your needs?
• Add all your tests to a TestSuite.  That way you can quickly "ping" the Web Service to make sure everything is working on the other end.  When a problem arises, taking your code out of the loop is a good way to make sure the issue is on the remote end and not with your own code.

When developing against a newly written external Web Service, it is very rare that the Web Service works 100% out of the box as expected.  There is usually a fair amount of communication needed between the Web Service developer and the client-side developer to tweak things until the Web Service works as needed.  By using soapUI, you can inspect the Web Service right at the time it is delivered to you, and quickly spot things that may need to be fixed.  I usually take 5 or 10 minutes to do so and almost always have a list of things that need to be modified.  The response time for getting those changes implemented is usually very short, as the Web Service developer is still engaged in the project and things fresh in his mind.  If I don't get back to him/her until some weeks later when I finally get around to implementing my client-side code, the other developer has probably moved onto other things and/or forgotten why he implemented the service in a certain way.   

Lastly, if you have received a WSDL/WADL file, but the service hasn't actually been implemented, and you NEED to start implementation against the service (not ideal), then consider using soapUI to create a Mock service from the WSDL/WADL file.  That way you can have your code hit the Mock service and at least get some preliminary feedback on whether your client code is working.  Since creating a Mock service is really a breeze with soapUI it can sometimes be more practical than implementing Mock objects in your code.

Test Your Own Web Services

When writing a Web Service for others to consume it can be handy to have a soapUI TestSuite to sanity test your service.  Of course you should still write unit and integration tests for your code, but having a good soapUI TestSuite can be quick and easy way to find out if all your services are running as expected.  When you get that 4 AM phone call saying that something is broke, fire up soapUI and at the click of a button sanity test all of your Web Service operations.  If you are smart, you'll hand the TestSuite over to a support team so that you only get woken when the issue truly is on your end ;-)  Just make sure to add proper assertions for Web Service responses and include SLA assertions to test that things are not running dead slow.

If you are concerned about the performance of your Web Service or whether it can handle a given load, then a soapUI Load Test can be a convenient way to test that.  Set the number of threads to imitate the expected number of concurrent users for your service and add SLA assertions to make sure all requests are handled in a timely fashion.  Of course generating all the load from a single machine does not quite imitate real traffic, so for truer numbers consider having coworkers assist you in running simultaneous Load Tests from multiple machines.

Kanban with David Anderson

This week I enjoyed attending a two day Kanban training class by David Anderson, the Father of Kanban-style software development.  The class provided an overview of Kanban and offered many tips on how to get the most out of Kanban.  David rolled through a deck of some 150 slides and answered various questions.  On the practical side, the class was divided into four teams and each team had to design a Kanban board for an actual company and then play an entertaining Kanban game, which turned out to be quite competitive.  The team I was on, Grand Kanban :) managed to squeeze out a slim victory with around $62,000 in revenue, which according to David was a pretty good score ;-)

Few points from the class:

The Kanban Method is based on three core principles:

•    Start with what you do now
•    Agree to pursue incremental evolutionary change
•    Initially, respect current processes, roles, responsibilities & job titles

The five keys to a successful Kanban implementation are:

•    Visualize Workflow
•    Limit Work-in-Progress
•    Manage Flow
•    Make Process Policies Explicit
•    Improve Collaboratively (using models & scientific method)

Key class takeaways according to Dave were:

• Kanban is like water 
• It goes around obstacles and slowly changes them, rather than removing them like a bulldozer
• Change has to come from within
• How many experts does it take to change an organization?  Answer: One.  But people are going to have to want to change

Surprising fact from the class:

According to a research done by David, getting more done only ranks number 4 on most manager's lists!  The list of their preference being:

1. Predictability
2. Business Agility (ability to respond to changes in market)
3. Good Governments (managing budgets, money spent the way intended, etc.)
4. Getting more done

Playing the Kanban game

All in all a good class and I look forward to continuing the improvement of Kanban at Hugsmiðjan where we develop the Eplica CMS.

Jfokus Conference

This week I had the pleasure of attending the JFokus conference in Stockholm, Sweden.  This was a gathering of some 1250 Java geeks and some excellent speakers.  I thoroughly enjoyed sessions by Arun Gupta, Guillaume Laforge, Mario Fusco, Henrik Kniberg, Neal Ford, and many others.  Here are some of the technologies that caught my attention and I will be considering putting to use at my current work on the Eplica CMS system.

• Lambdaj - This library is just brilliant and really simplifies working with Collections.  Write more readable code in fewer lines.  Stop writing loops!  Mario Fusco's slides.
• CKJM - Use this library to intelligently pick areas of your code base to refactor.  To get bang for your buck you want to pick classes with high Cyclomatic Complexity and high Afferent Coupling.  That is, complex code in frequent use.  Comes as part of a Sonar plugin.  And BTW, if you are not already using Sonar, use it!
• Groovy - I have been intrigued by this language for some time now but never actually put it to use in my work.  I think it is time to change that.  Groovy to me is a natural progression of the Java language.  Making the boring/tedious parts of Java optional and adding syntax improvements and useful features like closures.  The kind of changes you might envision seeing in Java 10 or somewhere way down the road.  Why wait so long?
  As a starting point I am thinking about using these Groovy frameworks for our web based testing:
• Geb - Browser automation framework based on Selenium.
• HTTP Builder - A convenient API for complex HTTP requests.
• WADL & Jersey - When consuming/publishing RESTful web services from/to our customers I prefer to have a contract to work by.  WADL is just that and Jersey an API (JAX-RS 1.1 implementation) for creating/consuming REST services.
• JDK 7 - Obviously eagerly awaiting that one.  Planned GA date July 28, 2011, according to Tomas Nilsson at Oracle.

Other technologies I found interesting but don't envision using in the near future (just not a good fit for Eplica CMS): Scala, Clojure, deploying to the cloud,  NoSQL, Vaadin and DDD.

All in all a great conference which I highly recommend to Java developers.

Photos from the trip

Tweets

Easily Size and Arrange Windows on Mac OS X

One idiosyncrasy of Mac OS X is that application windows can only be sized from their lower right corner, and Mac OS does not have any kind of "snap windows into place" feature similar to Windows 7.  At work I have the pleasure of working on a large 27" iMac screen, so I frequently arrange two apps side by side while working, or might even have more than 2 apps visible on the screen at any given time.   Arranging that kind of setup is tedious when done manually with the mouse, but with a nifty little tool called Divvy the task becomes quite easy.   As can be seen in the Divvy in Action video on their site, when activated, Divvy overlays a grid on your screen and can be used to visually size an application to a specific portion of the screen.  That is great, but what I found faster and easier was to assign global OS X shortcuts to window arranging operations I frequently use and let Divvy do its magic without bringing up the grid panel at all.  Creating shortcuts is covered towards the end of the video, but to make them global you need to check the "Global Shortcut" check box.  Here is a screen shot of some of the configurations I set up:

For example if I need to have an application fill the left 50% of my screen I just hit ctrl-shift-1 (with the app in focus) and it will jump right into place.  While developing, I often arrange things so that my IDE uses the entire right-side of the screen, the website I am testing covers the upper left-side of the screen and the application sever console (logging output) covers the lower left-side.  To put the windows in place I simply put them in focus one at a time and hit ctrl-shift-2, ctrl-shift-3 and ctrl-shift-4, in that order, and the apps snap into place, filling the screen like this:

No space is wasted on the screen and no annoying mouse-dragging operations are needed. Of course you can assign whatever keyboard shortcuts make sense to you.
Divvy is not freeware, but low cost, and can be used in demo mode (periodically pops up a registration message) if you don't want to cough up the $14.  It supports multiple monitors and is available for Windows too.