XL Deploy and TFS 2015 – Building, Importing and Deploying Packages

Recently XebiaLabs released a build task for Microsoft Visual Studio Team Services (VSTS) and Team Foundation Server (TFS) 2015 that facilitates the integration of the vNext build pipeline with their product, XL Deploy. The new build task is capable of creating a deployment archive (DAR package), importing it to an instance of XL Deploy, and eventually, if requested, triggering the deployment of the newly added package. It also contains some other interesting options that will ease the process of versioning the deployment package.

In the following post I will show you how to build an ASP.NET MVC application, create the deployment package, and deploy it via XL Deploy. Let’s start.

Prerequisites

Before we start, make sure that you have all of the necessary items in place. First of all, your build agent needs to be capable of building your application. In the case of on-premises TFS 2015, this means that you need to have installed Visual Studio on your build server.

You also need to install the XL Deploy build task; if you haven’t done so already, refer to Install the XL Deploy build task. As a prerequisite for the XL Deploy build task, your XL Deploy instance needs to be of version 5.0.0 or later; earlier versions are not supported.

Once you ensure that these criteria are met, you can continue reading.

Building your web application

First things first. Create a new empty build definition in your project. If you are not familiar with how to do this, refer to the MSDN article Create a build definition.

Set the repository mappings accordingly before proceeding any further; again, refer to the MSDN article Specify the repository for more information about doing so. For this example I am going to use TFVC; however, all of the techniques I am going to describe do work also with Git repository.

Now you can specify the necessary build steps to compile and publish the web application.

Add a Visual Studio Build build task.

add-task-vsb

Once added, you need to set the necessary parameters. The first mandatory setting is the path to the solution file in source control. Click the button with three dots and locate your solution file in the version control tree. Once found and confirmed, the Solution box will be populated with the correct path. In my case, that is $/XL Deploy Showcase/AwesomeWebApp/AwesomeWebApp.sln. This is the solution that will be built in the build definition.

Next you need to use the MSBuild Arguments setting to indicate that all of the built items should be delivered in the build staging directory. This is an important step because when the XL Deploy build task starts creating the deployment archive (DAR package), it will search for items to include in it, starting from this directory (considering the build staging directory as root). That’s why I am going to set MSBuild Arguments to /p:OutDir=$(build.stagingDirectory). This is a common way for MSBuild to pass a parameter; I’m using a variable that will resolve in a specific path when the build is run. A list of available variables and their usage is explained in the MSDN article Use variables. For now, I will not change any other setting. Make sure that Restore NuGet Packages is selected and that the correct version of Visual Studio is listed.

vsbuild-task

Save the build definition. Enter a relevant name and optionally set the comment.

save-definition

Now, you can test the build. Queue a new build and check the results. If everything went well, you will get a ‘green’ build.

first-build

Now you are sure that the project builds correctly and we can continue with our next task, which is deploying the application via XL Deploy.

You can find more information about build steps on MSDN at Specify your build steps.

XL Deploy build task

In order to continue, you need to have a valid XL Deploy manifest file checked in to our source control. You can read about creating a manifest file at XL Deploy manifest format. I checked in the following manifest:

<?xml version="1.0" encoding="utf-8"?>
<udm.DeploymentPackage version="1.0.0.0" application="AwesomeWebApp">
  <deployables>
	<iis.WebContent name="website-files" file="_PublishedWebsites\AwesomeWebApp">
	  <targetPath>C:\inetpub\AwesomeWebApp</targetPath>
	</iis.WebContent>
	<iis.ApplicationPoolSpec name="AwesomeWebApp">
	  <managedRuntimeVersion>v4.0</managedRuntimeVersion>
	</iis.ApplicationPoolSpec>
	<iis.WebsiteSpec name="AwesomeWebApp-website">
	  <websiteName>AwesomeWebApp</websiteName>
	  <physicalPath>C:\inetpub\AwesomeWebApp</physicalPath>
	  <applicationPoolName>AwesomeWebApp</applicationPoolName>
	  <bindings>
	<iis.WebsiteBindingSpec name="AwesomeWebApp/8088">
	  <port>8088</port>
	</iis.WebsiteBindingSpec>
	  </bindings>
	</iis.WebsiteSpec>
  </deployables>
</udm.DeploymentPackage>

As you can see, I specified the required IIS application pool, a web site, and, most importantly, WebContent. The WebContent element indicates that the files that do represent the web site.

An attribute called file is present on the deployable element iis.WebContent. This attribute is very important, as it will guide the build task in creating the deployment archive. A relative path is set as the value and indicates the folder that needs to be included in the DAR package. This relative path assumes that the root folder is the build staging directory; it will search this directory for the folders specified here. If they are not present, packaging will fail. Now you understand why I delivered the build output in the staging folder. Although this behavior can be overridden in the advanced settings of the XL Deploy build task, I would advise you not to change it if it is not necessary and until you became confident and sufficiently experienced with TFS 2015 builds.

The manifest file also indicates that I expect MSBuild to package the web application under a default folder called _PublishedWebsites. This is standard behavior for MSBuild.

After the manifest file is correctly checked in, you can continue and edit the recently created build definition by adding another build step. This time, go to the Deploy group and select XL Deploy.

add-task-xld

The first mandatory parameter that you need to set is the path to the manifest file. As you did previously for the solution file, click the button with three dots and select the manifest file in source control. In my case, that will be $/XL Deploy Showcase/AwesomeWebApp/deployit-manifest.xml.

Secondly, you need to set the endpoint, which is a definition of the URL and credentials for the XL Deploy server that you intend to use in this deployment. If no XL Deploy endpoints are defined, click Manage and define one. For information about doing so, refer to Add an endpoint in Team Foundation Server 2015. After you are done, click Refresh and your newly added endpoint will appear.

Versioning the package

As you can see, there is a Version option available on the XL Deploy build task. If selected, the deployment package version will be set to the build number format during the process. The build number format is a value that you can set in the General tab of the build definition.

When composing the build number, you can choose between constants and special tokens that are available for that purpose. In Specify general build definition settings, you can read about the available tokens and ways of setting the build number format.

If you are using a different pattern for versioning your deployment package and your build number format has a different use, you can override this behavior by setting a custom value in Version value override parameter under Advanced options.

For this example, I will set this option and then set the build number format to a simple ‘1.1.1$(Rev:.r)’.

Automated deployment

You have the option to automatically deploy the newly imported package to the environment of your choice. If you select the Deploy check-box on the XL Deploy build task, a new field called Target Environment will be available. Here, you need to specify the fully qualified name of the environment defined in XL Deploy. In my case, it is called Test. You can also allow the deployment to be rolled back in case of failure by selecting Rollback on deployment failure.

Note that this requires that you have correctly defined the environment in XL Deploy, with the relevant infrastructure and components. For more information, refer to Create an environment in XL Deploy.

Ready to roll

After you set all of the parameters and save the build definition, you are ready to trigger the new build. Make sure that you set all of the necessary parameters as shown in the picture.

xld-task

You can now trigger the new build. In the build console you should now see the information relevant to the process of creating and versioning the package.

final-build

In case of problems, you can add a new build definition variable named system.debug and set the value to true. Then, in the build log, you will see more detailed messages about all of the actions preformed by the tasks (note that this is not shown in the console). It should be sufficient to diagnose the issue.

Conclusions

This was a small practical example about taking advantage of the new XL Deploy build task. I haven’t covered all of the available options; you can read more about them at Introduction to the Team Foundation Server 2015 plugin. However, this example is sufficient to get started. You can combine other build tasks in your build definition with the XL Deploy build task; but it is important that you invoke the XL Deploy build task after all of the elements that will compose your deployment package are available.

Passing values between TFS 2015 build steps

Introduction

It may happen that you need to pass or make available a value from one build step in your build definition, to all other build steps that are executed after that one. It is not well documented, but there are two ways of passing values in between build steps.
Both of the approaches I am going to illustrate here are based on setting a variable on the task context. The first task can set a variable, and following tasks are able to use the variable. The variable is exposed to the following tasks as an environment variable. When ‘issecret’ option is set to true, the value of the variable will be saved as secret and masked out from log.

In order to test my approaches, I made 4 build tasks, two of them for the first approach and two of them for the second one. They look something like this:

task-1-set

You can download all of the examples here.

Task Logging Command

Build task context supports several types of logging commands. Logging command are a facility available in the run context of the build engine and do allow us to perform some particular actions. A full list of available logging commands is available here. Be aware that some of them are only available since a certain version of the build agent, and before using them check the minimum agent version for clarity, eventually limit your build task to the indicated version.
In our example, I will use a variable provided as a parameter to my task with a value set from the UI, print the value out and store it in the task context. This will be my build task called task1. Once I set the task manifest correctly, I will upload it to my on-premise TFS (it will also work on VSTS/VSO). Let’s check the code.

Write-Output "Message is: $msg"
Write-Output ("##vso[task.setvariable variable=task1Msg;]$msg")

Now, I will create another task and call it task2. This task will try to retrieve the value from the task context, then print the value out. Following is the code I’m using.

$task1Msg = $env:task1Msg

if ($task1Msg)
{
	Write-Output "Value of the message is set and equals to $task1Msg"
}
else
{
	Write-Output "Value of the message is not set."
}

You can find all of the examples I’m using available for download here.

If you now set your build definition with tasks task1 and task2, you will see that based on what you set as a Message in task1, will be visualized in the console by the task2.

task-1-2-run

Set-TaskVariable and Get-TaskVariable cmdlets

The same result can be achieved by using some cmdlets that are made available by the build execution context. We are going to create a task called task3 which will have the same functionality as task1, but it will use a different approach to achieve the same. Let’s check the code:

Import-Module "Microsoft.TeamFoundation.DistributedTask.Task.Common"
    
Set-TaskVariable "task1Msg" $msg 

As you can see we are importing modules from a specific library called Microsoft.TeamFoundation.DistributedTask.Task.Common. You can read more about the available modules and exposed cmdlets in my post called Available modules for TFS 2015 build tasks.
After that we are able to invoke the Set-TaskVariable cmdlet and pass the variable name and value as parameters. This will do the job.

In order to retrieve task we need to import another module which is Microsoft.TeamFoundation.DistributedTask.Task.Internal. Although those are quite connected actions, for some reason Microsoft decided to package them in two different libraries. We are going to retrieve our variable value in the following way:

Import-Module "Microsoft.TeamFoundation.DistributedTask.Task.Internal"

$task1Msg = Get-TaskVariable $distributedTaskContext "task1Msg"

if ($task1Msg)
{
	Write-Output "Value of the message is set and equals to $task1Msg"
}
else
{
	Write-Output "Value of the message is not set."
}

As you can see we are invoking the Get-TaskVariable cmdlet with two parameters. First one is the task context. As for other things, build execution context will be populated with a variable called distributedTaskContext which is of type Microsoft.TeamFoundation.DistributedTask.Agent.Worker.Common.TaskContext and beside other exposes information like ScopeId, RootFolder, ScopeType, etc. Second parameter is the name of the variable we would like to retrieve.

The result still stays the same as you can see from the following screenshot:

task-3-4-run

Conclusion

Both approaches will get the job done. It is on you to choose the preferred way of storing and retrieving values. A mix of both also works. You need to experiment a bit with the build task context and see what works for you the best. Be aware that this is valid at the moment I’m writing this, and it is working with TFS 2015, TFS 2015.1 and with VSTS as of 2016-02-19. Seems however that in the near future all of this will be heavily refactored by Microsoft, then my guide will not be relevant anymore.

Stay tuned!

Available modules for TFS 2015 build tasks

It is not yet well documented but if you are writing a custom build task for your TFS 2015 build system, you get at your disposition some of the modules that are made available by your VsoWorker.
As I couldn’t find a list of available modules and cmdlets they expose, I decided to dig into them and see what is there. I also wanted to check if with the Update 1 (and the new version of the build agent) there will be more of them.
I wrote a handy task that will list all of the available cmdlets for all of the modules shipped by the agent worker. Following are my results.

With TFS 2015 the agent version delivered is 1.83.2. It offers the following modules:

  • Microsoft.TeamFoundation.DistributedTask.Task.Common.dll
  • Microsoft.TeamFoundation.DistributedTask.Task.Deployment.Azure.psm1
  • Microsoft.TeamFoundation.DistributedTask.Task.Deployment.Chef.psm1
  • Microsoft.TeamFoundation.DistributedTask.Task.DevTestLabs.dll
  • Microsoft.TeamFoundation.DistributedTask.Task.DTA.dll
  • Microsoft.TeamFoundation.DistributedTask.Task.Internal.dll
  • Microsoft.TeamFoundation.DistributedTask.Task.TestResults.dll

I will now list of all importable functions and cmdlets in those modules.

  • Microsoft.TeamFoundation.DistributedTask.Task.Common.dll
    • Add-TaskIssue
    • Complete-Task
    • Find-Files
    • Get-LocalizedString
    • Set-TaskProgress
    • Set-TaskVariable
    • Write-TaskDetail
  • Microsoft.TeamFoundation.DistributedTask.Task.Deployment.Azure.psm1
    • Get-AzureCmdletsVersion
    • Get-AzureModuleLocation
    • Get-AzureVersionComparison
    • Get-RequiresEnvironmentParameter
    • Get-SelectNotRequiringDefault
    • Import-AzurePowerShellModule
    • Initialize-AzurePowerShellSupport
    • Initialize-AzureSubscription
  • Microsoft.TeamFoundation.DistributedTask.Task.Deployment.Chef.psm1
    • Get-DetailedRunHistory
    • Get-PathToNewtonsoftBinary
    • Get-ShouldWaitForNodeRuns
    • Get-TemporaryDirectoryForChef
    • Initialize-ChefRepo
    • Invoke-GenericMethod
    • Invoke-Knife
    • Invoke-WithRetry
    • Wait-ForChefNodeRunsToComplete
  • Microsoft.TeamFoundation.DistributedTask.Task.DevTestLabs.dll
    • Complete-EnvironmentOperation
    • Complete-EnvironmentResourceOperation
    • Complete-ResourceOperation
    • Copy-FilesToAzureBlob
    • Copy-FilesToRemote
    • Copy-FilesToTargetMachine
    • Copy-ToAzureMachines
    • Get-Environment
    • Get-EnvironmentProperty
    • Get-EnvironmentResources
    • Get-ProviderData
    • Invoke-BlockEnvironment
    • Invoke-EnvironmentOperation
    • Invoke-PsOnRemote
    • Invoke-ResourceOperation
    • Invoke-UnblockEnvironment
    • New-OperationLog
    • Register-Environment
    • Register-EnvironmentDefinition
    • Register-Provider
    • Register-ProviderData
    • Remove-Environment
    • Remove-EnvironmentResources
  • Microsoft.TeamFoundation.DistributedTask.Task.DTA.dll
    • Invoke-DeployTestAgent
    • Invoke-RunDistributedTests
  • Microsoft.TeamFoundation.DistributedTask.Task.Internal.dll
    • Add-BuildArtifactLink
    • Add-BuildAttachment
    • Convert-String
    • Copy-BuildArtifact
    • Get-JavaDevelopmentKitPath
    • Get-MSBuildLocation
    • Get-ServiceEndpoint
    • Get-TaskVariable
    • Get-ToolPath
    • Get-VisualStudioPath
    • Get-VssConnection
    • Get-X509Certificate
    • Invoke-Ant
    • Invoke-BatchScript
    • Invoke-IndexSources
    • Invoke-Maven
    • Invoke-MSBuild
    • Invoke-PublishSymbols
    • Invoke-Tool
    • Invoke-VSTest
    • Publish-BuildArtifact
    • Register-XamarinLicense
    • Unregister-XamarinLicense
  • Microsoft.TeamFoundation.DistributedTask.Task.TestResults.dll
    • Invoke-ResultPublisher
    • Publish-TestResults

The Update 1 RC1 for TFS increased the agent to the version 1.89.0 and the RC2 incremented it to version 1.89.1. As of the time I’m writing this blog post, the RC2 is the latest version available, I’ll list and show the differences with a plain TFS 2015 build agent.

Three new modules are added:

  • Microsoft.TeamFoundation.DistributedTask.Task.CodeCoverage
  • Microsoft.TeamFoundation.DistributedTask.Task.Deployment.Internal
  • Microsoft.TeamFoundation.DistributedTask.Task.Deployment.RemoteDeployment

Following the list of all importable functions and cmdlets in the those modules.

  • Microsoft.TeamFoundation.DistributedTask.Task.CodeCoverage
    • Enable-CodeCoverage
    • Publish-CodeCoverage
  • Microsoft.TeamFoundation.DistributedTask.Task.Deployment.Internal
    • Get-OperationLogs
    • Get-ResourceCredentials
    • Get-ResourceFQDNTagKey
    • Get-ResourceHttpsTagKey
    • Get-ResourceHttpTagKey
    • Get-ResourceOperationLogs
    • Get-SkipCACheckTagKey
    • Get-SqlPackageCommandArguments
    • Import-DevtestLabsCommomDll
    • Write-ResponseLogs
  • Microsoft.TeamFoundation.DistributedTask.Task.Deployment.RemoteDeployment
    • Invoke-RemoteDeployment

Furthermore some cmdlets are added to the existing modules:

  • Microsoft.TeamFoundation.DistributedTask.Task.Deployment.Azure.psm1
    • Set-CurrentAzureRMSubscription
    • Set-CurrentAzureSubscription
  • Microsoft.TeamFoundation.DistributedTask.Task.DevTestLabs.dll
    • Get-ExternalIpAddress
    • Get-ParsedSessionVariables
  • Microsoft.TeamFoundation.DistributedTask.Task.Internal.dll
    • Get-IndexedSourceFilePaths
    • Get-TfsClientCredentials

It also seems that in the Microsoft.TeamFoundation.DistributedTask.Task.Internal.dll, Invoke-IndexSources cmdlet is not available any more.

You can reference these modules from your custom Build tasks by calling a standard import module cmdlet.

import-module "Microsoft.TeamFoundation.DistributedTask.Task.Common"

If you are interested about the agent version, you can check it by executing the vsoagent on your build server, with the following parameter:

VsoAgent.exe /version

As soon as an RTM version of the Update 1 becomes available, I’ll update this post.

In case you are interested in a short description of these functions, you can their source code for further info