Synchronous Deployments With Sitecore Package Deployer

When Are Synchronous Deployments Important?

Sitecore Package Deployer (SPD) provides a mechanism to install Sitecore Update Packages generated via TDS Classic on remote servers. Fundamentally, update packages are moved to the Data/SitecorePackageDeployer folder of a Sitecore instance, then the package is installed. This can be accomplished via a scheduled task or by manually executing a URL. In either case, the process of installing the package is asynchronous. This becomes a problem in some release configurations. Namely, in a distributed environment with separate Content Management (CM) and Content Delivery (CD) servers.

My release tool of choice is Visual Studio Team Services (VSTS), although the same issue may occur on any other tool (read: Octopus Deploy). Imagine a scenario where a successful release to the CM server automatically triggers a release to a CD server. If the package installation occurs asynchronously, there is a possibility that the CD deployment will execute before a package is fully installed and Post Deploy Steps have fully executed. This creates a non-ideal racing condition. While it is great that the deploy appears to finish lightning fast, it would be far more ideal if it finished slower, albeit more accurately.

How To Address

Pull Synchronous Fork

I created a fork, with a very simple change to allow for synchronous package installs. The fork is available here. To install, follow the standard SPD installation process, but replace Hhogdev.SitecorePackageDeployer.dll and Hhogdev.SitecorePackageDeployer.Web.dll with the compiled versions from the fork.

Also, it is important to modify Hhogdev.SitecorePackageDeployer.config by commenting out:

<processor type="Hhogdev.SitecorePackageDeployer.Pipelines.Initialize.RunPostInstallSteps, Hhogdev.SitecorePackageDeployer"/>

This ensures the installation triggers only when we call the URL and nothing executes upon initialization.

With this fork, a call to http://mysite.com/StartSitecorePackageDeployer.aspx?synchronous=1 (throwing “&force=1” doesn’t hurt either) will execute the installation synchronously, i.e. wait until all steps have finished.

As an optional step, in my implementations, I have moved the manual trigger file StartSitecorePackageDeployer.aspx to the webroot to make it easier to call via a PowerShell script. Note: This file is removed from the CD release.

Strip Files From Package

In order for the synchronous install to execute properly, we need to make sure that it does not deploy any DLL or config file updates. Either of these file types, once deployed, will cause a recycle of the app pool. This isn’t a problem for installing a single update package, however with multiple Update Packages in the data folder, the request will finish before all packages have finished installing.

Even if no files are included in the update package, TDS Classic includes HedgehogDevelopment.SitecoreProject.PackageInstallPostProcessor.dll in all update packages. This is included to ensure this DLL exists to execute Post Deploy Steps.  Removing this file from the package requires two steps:

1. Be sure this file exists in your \bin before the installation occurs
   1. This can be done by adding it as a reference to your web project. Once it’s deployed, it will then always be there.
2. Open the update package and remove this DLL

Charlie Turano and Joe Bissol were kind enough to provide me with some code to perform step 2. I have included the 3 files required for this operation to work in this repo. The three files can be broken down like this:

1. DotNetZip.dll
   1. This is simply a reference for the exe that needs to run.
2. ExtractDllsFromPackage.exe
   1. This is the exe that extracts the DLL from the update package
3. RemoveFiles.ps1
   1. The PowerShell script that calls the exe with each individual update package

The RemoveFiles.ps1 PowerShell script needs to be executed in either a build step or release step (up to you how you have your DevOps process configured). Ultimately, the files need to be extracted from the Update Package prior to running the synchronous installation.

Running The Synchronous Install

When it comes to any custom task in our release process, our goto method is always PowerShell. It makes it easy to pass parameters and access system resources. With that said, the last step to run a synchronous install is to call the StartSitecorePackageDeployer.aspx?synchronous=1&force=1 page. After many hours struggling to find out why PowerShell fails in this context, I ultimately went with a curl implementation. The curl files I have used are located here.

The following PowerShell script requests the proper URL via curl, instead of the out-of-the-box Invoke-WebRequest method provided by PowerShell.

Param(
    [string]$SiteUrl,
    [string]$Username,
    [string]$Password,
    [int]$Timeout = 900 # 15 minutes
)

$currentPath = (Resolve-Path .\).Path
Write-Host "Current directory: $($currentPath)"
$executable = "$($currentPath)\curl.exe"
Write-Host "Exe location: $($executable)"

$curlCommand
if($Username -and $Password){
	$curlCommand = "'$($executable)' -sS -fail -u $($Username):$($Password) $($SiteUrl) --max-time $Timeout"
}
else{
	$curlCommand = "'$($executable)' -sS -fail $($SiteUrl) --max-time $Timeout"
}
Invoke-Expression "& $curlCommand"

Notice that this script requires that the curl.exe exist in the same directory of execution as this script. It also supports basic authentication, if desired. The timeout is set to 15 minutes by default but can be modified when the PowerShell script is called. It is recommended to adjust this based on the worst case scenario, i.e. a very item heavy deployment.

Wrapping Up

It may be obvious, but the code and instructions in this post grew out of necessity. I never stopped to question if 20-second item deployments were too fast. It wasn’t until a deployment went out and items were missing that I discovered the issue. In that particular case, I had additional actions being triggered that required that the installation was entirely finished. Feel free to reach out to me on Sitecore Slack (jraps) or Twitter with any comments or questions.