Integration tests using PowerShell – Sitecore PowerShell Extensions pt. 1

Reason

Restoring your Sitecore test environment to a predefined state can quickly become a complex task, especially when required to do so on on a more granular level than simply restoring entire databases from backups.
Sitecore PowerShell Extensions has proven to be a great tool to create and delete test related content, making sure the test environment is in an acceptable state before each test is run.

Although it’s possible to fake some parts of Sitecore, this article focuses on integration tests which exercise the solution as a whole. There is little point in running PowerShell scripts against a test server when you’ve mocked the entire Sitecore context in memory.

By using the ScriptBuilder provided in PowerShell commandlets – Sitecore PowerShell Extensions pt. 2, the PowerShell script required to execute commandlets on a remote machine can be generated effortlessly.
PowerShell script and commandlets can be executed remotely using the RemoteAutomation web service as discussed in Remote Automation web service – Sitecore PowerShell Extensions pt. 3.

Once our commandlets are deploy and registered on the test server the following steps can be used to invoke them remotely:

  1. Create an instance of your commandlet.

  2. Set it’s properties using strongly typed objects.

  3. Use the ScriptBuilder to generate the PowerShell script required to invoke the commandlet.

  4. Execute the commandlet on the remote server using the RemoteAutomation web service’s ExecuteScript method.

  5. Use the web service output in whatever way makes sense.

Flow diagram

The key point to understand is that there’s no real object serialization going on. We’re simply generating and sending the same text that we’d have to type into the Sitecore PowerShell console on the server to make our commandlet run.
This means that creating an instance of the commandlet is only means to an end: we’re using it as a fool proof way of generating the script required to execute the commandlet on the remote server, with the arguments of our choice.
Although we don’t need to install PowerShell on our local development machine, we need the code/binaries containing the commandlet to be able to instantiate it locally.

Once the script required to invoke the commandlet has been generated and sent to the RemoteAutomation web service, Sitecore PowerShell Extensions will execute it in a fully initialized Sitecore context. This makes it a great option for test setup and teardown methods which require interaction with the Sitecore API – creating, modifying or deleting content, users and roles, invoking pipelines, publishing content, rebuilding indexes etc.

The benefits that I’ve found using commandlets in combination with the ScriptBuilder, RemoteAutomation web service and simple extension methods are:

  • allows for a uniform approach to setup and teardown logic that needs to interact with Sitecore
  • avoids littering the solution with e.g. form pages used to reset server state between tests

  • offers a concise syntax that doesn’t draw unneeded attention to plumbing code rather than the actual tests

  • requires little to no knowledge of actual PowerShell syntax and hence comes with little to no costs when introduced into a project

<

p>Examples are based on .NET 4.5, Sitecore 7.1 rev. 130926 and Sitecore PowerShell Extensions 2.5.1.

Note: This is part 1 of 3 regarding the concept of using the RemoteAutomation web service in the creation and deletion of test content as part of a integration test suite, as outlined in my recent talk at the Danish Sitecore User Group.

Code

The following extension methods simply add syntactic sugar and hide code used repetitively during the process of generating script based off of a commandlet, sending it to the RemoteAutomation web service and “unpacking” the web service output:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Management.Automation;
using System.ServiceModel;
using System.ServiceModel.Channels;

public static class CommandExtensions
{
  public static IEnumerable<NameValue> ExecuteOnTestServer(this PSCmdlet command)
  {
    // Environment specific settings are retrieved from app config (i.e. via a settings class).
    return ExecuteRemotely(command,
      TestSettings.Default.RemoteAutomationServiceUrl,
      TestSettings.Default.RemoteAutomationServiceUserName,
      TestSettings.Default.RemoteAutomationServicePassword);
  }

  public static IEnumerable<NameValue> ExecuteRemotely(this PSCmdlet command, Uri serviceUrl, string userName, string password)
  {
    string script = command.ToScript();
    Binding binding = new BasicHttpBinding();
    EndpointAddress address = new EndpointAddress(serviceUrl);
    // The RemoteAutomation web service is discussed in pt. 3 of this article series.
    RemoteAutomationSoapClient remoteAutomationService = new RemoteAutomationSoapClient(binding, address);
    List<NameValue> response = remoteAutomationService.ExecuteScript(userName, password, script, string.Empty);
    return response ?? Enumerable.Empty<NameValue>();
  }

  public static string ToScript(this PSCmdlet command)
  {
    // The ScriptBuilder is discussed in pt. 2 of this article series.
    return new ScriptBuilder(command).CreateScript();
  }

  public static string GetOutput(this IEnumerable<NameValue> responseContents)
  {
    return (from content in responseContents
            where string.Equals(content.Name, "output", StringComparison.InvariantCultureIgnoreCase)
            select content.Value).FirstOrDefault();
  }
}

Example

Shown below is a Cucumber feature specification expressed in the Gherkin language. The feature specification on the left maps to a set of matching steps created using SpecFlow on the right.
Each step performs a set of actions in an actual browser, using a browser automation framework, thus performing an integration test of the feature – in this case a user registration flow.
Note how none of the test code uses PowerShell syntax, and that nearly all plumbing code has been removed from the test class itself.

User registration feature and steps

Once the browser automation has entered the user name and clicked the registration button, the UserExistsCommand shown below is used to verify that a user has indeed been created in Sitecore.

UserExistsCommand source code

Rerunning the test would fail without proper cleanup, due to the fact that the test user name would be in use on the second and subsequent runs. By deleting any remnants of prior test runs using the DeleteUserCommand shown below, we ensure that the test environment is in an acceptable state for our test to yield consistent results.

DeleteUserCommand source code

Accessing the Sitecore API directly to reset the state of the test environment is a great way to avoid “flickering”/”flaky” tests.
Often the alternative to a “proper” test environment reset is running tests in a specific order, creating an interdependency between them. In the example above this would be akin to relying on the “Unregistration” scenario to run before the “Registration” scenario. Although this would work if both scenarios succeed each and every time, it falls flat once one scenario starts to fail or is otherwise interrupted, as the other (dependent) scenario will start failing too.

2 thoughts on “Integration tests using PowerShell – Sitecore PowerShell Extensions pt. 1

  1. That’s a fantastic series. Funny thing is this is exactly why I added this remoting API – for setting up and tearing down tests in one of our projects! We didn’t do the strongly typed tests though – that’s really creative!

  2. Pingback: Update to State of knowledge for PowerShell Extensions for Sitecore – November 2014 | Codality

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s