Starting a TFS release via REST API

Mario Majcica May 10, 2019 1 Comments

Intro

It is a common need to inject variables in your release. When it comes to a build, it was always an easy task, however, the release didn’t support such a thing out of the box. In case you are using Azure DevOps (service and server), you are good to go, as this is now possible after you do mark a variable as ‘Settable at the release time’. But what about those who are stuck to previous versions of TFS? Well, there are some tricks that I’ll illustrate in this post.

Leveraging Draft release

There are two ways of achieving our goal. The first one is to create a new release and do not trigger the deployment in the environments that are set to deploy on the release creation, set the variables then trigger the deployments. This is a more laborious and complicated way. Second method is to create a draft release, then populate the draft with the necessary variables and then start the release. I’ll show you the necessary steps to achieve this via the REST API which you can try for free. 
In the upcoming cmdlets I’ll focus on achieving a goal, which is to create a draft release, add a variable and start the release. So if you find code not really reusable or not framework like, please consider that was out of my goal and it would take way more effort to write.
First of all I need a couple of helper functions that I will use in order to authenticate and get the right release id based on the release name.

$pat = '6yhymn3foxuqmsobktekvukeffhqifjt4yeldfj33v6wk4kr4idq'
$url = "https://myTest.tfs.local/tfs/DefaultCollection"
$project = "MarioTest"

function Get-PatHeader {
    [CmdletBinding()]
    param
    (
        [Parameter(Mandatory = $true)] [string]$Pat
    )
    BEGIN { }
    PROCESS {
        $encodedCredentials = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$pat"))
        $header = @{ }
        $header.Authorization = "Basic $encodedCredentials"

        return $header
    }
    END { }
}

function Get-ReleaseDefinitionId {
    [CmdletBinding()]
    param
    (
        [Parameter(Mandatory = $true)] [string]$CollectionUrl,
        [Parameter(Mandatory = $true)] [string]$Project,
        [Parameter(Mandatory = $true)] [string]$Name,
        [Parameter(Mandatory = $true)] [string]$Pat
    )
    BEGIN { }
    PROCESS {
        $headers = Get-PatHeader $Pat
        $reponse = Invoke-RestMethod "$CollectionUrl/$Project/_apis/release/definitions?searchText=$Name" -Headers $headers

        return $reponse.value.id
    }
    END { }
}

$releaseDefinitionId = Get-ReleaseDefinitionId $url $project "Test1" $pat

The above is to get the necessary authentication header and resolve the Release Definition name into the id that we are going to use across all of our other calls. As you can see, my release definition is called “Test1”.

Before we start a draft release, we need to collect some relevant information and that is information about the artifacts that we would like to use with the new release. Often this is a pain point for those with less experience with TFS. However, the following cmdlets should do the trick:

function Get-ReleaseArtifacts {
    [CmdletBinding()]
    param
    (
        [Parameter(Mandatory = $true)] [string]$CollectionUrl,
        [Parameter(Mandatory = $true)] [string]$Project,
        [Parameter(Mandatory = $true)] [int]$ReleaseDefinitionId,
        [Parameter(Mandatory = $true)] [string]$Pat
    )
    BEGIN { }
    PROCESS {
        $headers = Get-PatHeader $Pat
        $reponse = Invoke-RestMethod "$CollectionUrl/$Project/_apis/Release/artifacts/versions?releaseDefinitionId=$ReleaseDefinitionId" -Headers $headers

        return $reponse
    }
    END { }
}

function Get-DefaultReleaseArtifacts {
    [CmdletBinding()]
    param
    (
        [Parameter(Mandatory = $true)] $ReleaseArtifacts
    )
    BEGIN { }
    PROCESS {
        $artifacts = @()

        foreach ($artifactVersion in $ReleaseArtifacts.artifactVersions) {
            $artifact = [PSCustomObject]@{ }

            $artifact | Add-Member -MemberType NoteProperty -Name "alias" -Value $artifactVersion.alias
            $artifact | Add-Member -MemberType NoteProperty -Name "instanceReference" -Value $artifactVersion.defaultVersion
        
            $artifacts += $artifact
        }

        return $artifacts
    }
    END { }
}

$releaseArtifacts = Get-ReleaseArtifacts $url $project $releaseDefinitionId $pat
$defaultArtifacts = Get-DefaultReleaseArtifacts $releaseArtifacts

As in the Web UI, you are asked to specify the version of artifacts to use for the release that you are creating. The same is asked by the REST API. The above cmdlets will allow you to retrieve the list of all the available artifacts and pick the last (default) one. You can see further examples here https://docs.microsoft.com/en-us/azure/devops/integrate/previous-apis/rm/releases?view=azure-devops-2019#create-a-release

In case you are interested in using non the latest artifacts, you can explore further the response and implement the necessary logic do provide the ones to use in the next stage.

Now it’s time to create our draft release. This is easily achieved with the following cmdlet.

function New-Release {
    [CmdletBinding()]
    param
    (
        [Parameter(Mandatory = $true)] [string]$CollectionUrl,
        [Parameter(Mandatory = $true)] [string]$Project,
        [Parameter(Mandatory = $true)] [string]$Pat,
        [Parameter(Mandatory = $true)] $ReleaseArtifacts,
        [Parameter(Mandatory = $true)] [int]$ReleaseDefinitionId,
        [Parameter()] [string]$ReleaseDescription,
        [Parameter()] [bool]$IsDraft = $false,
        [Parameter()] [string[]]$ManualEnvironments
    )
    BEGIN { }
    PROCESS {
        $requestBody = @{ }
        $requestBody.definitionId = $ReleaseDefinitionId
        $requestBody.isDraft = $IsDraft
        $requestBody.description = $ReleaseDescription
        $requestBody.reason = "manual"
        $requestBody.manualEnvironments = $ManualEnvironments
        $requestBody.artifacts = $ReleaseArtifacts
        
        $headers = Get-PatHeader $Pat
        $body = $requestBody | ConvertTo-Json -Depth 10
        $body = [System.Text.Encoding]::UTF8.GetBytes($body);

        return Invoke-RestMethod "$CollectionUrl/$Project/_apis/release/releases?api-version=4.1-preview.6" -Method Post -Headers $headers -Body $body -ContentType "application/json"
    }
    END { }
}

$release = New-Release $url $project $pat $defaultArtifacts $releaseDefinitionId "desc" $true

As you can notice, with the above script, you can start not only a draft release, but also an ‘ordinary’ one.

At this point we are ready to set our variables. I’ll set both, one on the release level, another one on the environment scope, then update the release.

function Add-ReleaseVariable {
    [CmdletBinding()]
    param
    (
        [Parameter(Mandatory = $true)] [object]$ReleaseVariables,
        [Parameter(Mandatory = $true)] [string]$VariableName,
        [Parameter(Mandatory = $true)] [string]$VariableValue
    )
    BEGIN { }
    PROCESS {
        $value = [PSCustomObject]@{ value = $VariableValue }
        $ReleaseVariables | Add-Member -Name $VariableName -MemberType NoteProperty -Value $value -Force

        return $ReleaseVariables
    }
    END { }
}

$release.variables = Add-ReleaseVariable $release.variables "var1" "value"
$release.environments[0].variables = Add-ReleaseVariable $release.environments[0].variables "MarioInDraftEnv" "value1"

The above cmdlets will make things easier. In case the variable is already declared, the value will be overwritten with the one that you set at this stage. You can also see that I’m setting a variable on a release level and on an environment level. Environments are set to be an array, so to find out the desired one by name, you’ll need to search for it first. In this example, I’m just setting it on the first (and in my demo case, only) environment in the list. As mentioned earlier, last but not least, we will update the release.

function Update-Release {
    [CmdletBinding()]
    param
    (
        [Parameter(Mandatory = $true)] [string]$CollectionUrl,
        [Parameter(Mandatory = $true)] [string]$Project,
        [Parameter(Mandatory = $true)] [int]$ReleaseId,
        [Parameter(Mandatory = $true)] [object]$Release,
        [Parameter(Mandatory = $true)] [string]$Pat
    )
    BEGIN { }
    PROCESS {
        $headers = Get-PatHeader $Pat

        $body = $Release | ConvertTo-Json -Depth 10
        $body = [System.Text.Encoding]::UTF8.GetBytes($body);

        $reponse = Invoke-RestMethod "$CollectionUrl/$Project/_apis/release/releases/$($ReleaseId)?api-version=4.1-preview.6" -Method Put -Body $body -ContentType 'application/json' -Headers $headers

        return $reponse
    }
    END { }
}

$release = Update-Release $url $project $release.id $release $pat

Unfortunately, we can’t update the variables and start the release in the same call. The above call will update the variables in that specific release (not in the release definition) and the following will start the release.

function Start-Release {
    [CmdletBinding()]
    param
    (
        [Parameter(Mandatory = $true)] [string]$CollectionUrl,
        [Parameter(Mandatory = $true)] [string]$Project,
        [Parameter(Mandatory = $true)] [int]$ReleaseId,
        [Parameter(Mandatory = $true)] [string]$Pat
    )
    BEGIN { }
    PROCESS {
        $patch = '{ "status": "active" }'

        $headers = Get-PatHeader $Pat
        $reponse = Invoke-RestMethod "$CollectionUrl/$Project/_apis/release/releases/$($ReleaseId)?api-version=4.1-preview.6" -Method Patch -Body $patch -ContentType 'application/json' -Headers $headers

        return $reponse
    }
    END { }
}

$release = Start-Release $url $project $release.id $pat

That’s it. We now started our release with variables that are injected into it.

Azure DevOps

As mentioned before, this is not necessary anymore in Azure DevOps, or to be more precise, since version 5.0 of the REST API. In case you are looking for an example on how to achieve the same with the Azure DevOps, the following script will do.

function New-Release {
    [CmdletBinding()]
    param
    (
        [Parameter(Mandatory = $true)] [string]$CollectionUrl,
        [Parameter(Mandatory = $true)] [string]$Project,
        [Parameter(Mandatory = $true)] [string]$Pat,
        [Parameter(Mandatory = $true)] $ReleaseArtifacts,
        [Parameter(Mandatory = $true)] [int]$ReleaseDefinitionId,
        [Parameter()] $Variables,
        [Parameter()] [string]$ReleaseDescription,
        [Parameter()] [bool]$IsDraft = $false,
        [Parameter()] [string[]]$ManualEnvironments
    )
    BEGIN { }
    PROCESS {
        $requestBody = @{ }
        $requestBody.definitionId = $ReleaseDefinitionId
        $requestBody.isDraft = $IsDraft
        $requestBody.description = $ReleaseDescription
        $requestBody.reason = "manual"
        $requestBody.manualEnvironments = $ManualEnvironments
        $requestBody.artifacts = $ReleaseArtifacts
        $requestBody.variables = $Variables

        $headers = Get-PatHeader $Pat
        $body = $requestBody | ConvertTo-Json -Depth 10
        $body = [System.Text.Encoding]::UTF8.GetBytes($body);

        return Invoke-RestMethod "$CollectionUrl/$Project/_apis/release/releases?api-version=5.0" -Method Post -Headers $headers -Body $body -ContentType "application/json"
    }
    END { }
}

$variables = [PSCustomObject]@{ }
$variables = Add-ReleaseVariable $variables "var2" "value"

$release = New-Release $url $project $pat $defaultArtifacts $releaseDefinitionId $variables "desc" $false

Important changes compared to the previous version of the cmdlet are to be found in the extra parameter called Variables and the URL api-version parameter now set to 5.0.

Be however aware that you need create the variables in your release definition and mark them as ‘Settable at the release time’.

In case your variable is not defined, your API call will fail with the following error:

“Variable(s) another do not exist in the release pipeline at scope: Release. New variables cannot be added while creating a release. Check the scope of the variable(s) or remove them and try again.”

Previously described technique will still work, even with the Azure DevOps if adding variables dynamically in the release is what you want. However, I would always advise you to define them, so that they are present and do not “materialize” from nowhere.

I hope I covered the necessary. Let me know in comments if any.

Monitoring TFS 2015 availability from F5 LTM

Mario Majcica December 1, 2016 0 Comments

Introduction

If redundancy and performance are the thing you are looking for your TFS application tier setup, for sure you stumbled upon the term Network Load Balancing (NLB). Microsoft describes the benefits of such a setup and prerequisites in the document named How to: Create a Team Foundation server farm (high availability), thus I will not go in the details about these topics if you continue reading. However, in the documentation, Microsoft encourages you to setup the NLB feature that is integrated in the Windows Server operating system. In many situations that is not an option due to the network restrictions or company policies and the only choice is to use preexisting networking appliances. Reasons for using a hardware based NLB can also be a performance as it offloads the AppTier machines from this task that, for how minor it can be on today’s machines, it adds some load.

Monitoring

In case of using the Windows NLB feature, nodes participating in the pool of the machines used for the load distribution are monitored directly by the system itself, meanwhile for the hardware based solutions we need to setup a health monitor. This is essential as the load balancer needs to know if the node is available and in healthy state, otherwise it is excluded from the pool and the traffic is not sent towards that node.

Now, what is the best practice when it comes to the health status of TFS? Googling around you can’t find much, there are some pointers towards a SOAP method called GetServerStatus exposed, however it doesn’t bring the necessary information.
Luckily there is a non documented rest resource that is exposed on TFS 2015 and beyond and you can reach it at the URL

http(s)://your.tfs.address:port/tfs/_apis/health

It will return just a simple current time stamp by default using the JSON notation. Accessing this resource still requires the user to be authenticated.

When it comes to the F5 in particular, you need to create HTTPS Health Monitor (Local Traffic > Monitors > Create…)

The most important fields to set are Send and Receive string. Here we will send a request towards TFS at the above mentioned address and expect a status code 200 in the response. We can ignore the time stamp in the response body.
The send string will be:

GET /tfs/_apis/health HTTP1.1\r\nHost: your.tfs.address:port\r\n

meanwhile the receive string should be set to:

HTTP/1.1 200 OK

A simple check that the request succeeded (we are not interested in the timestamp in this case).

Do not also forget to provide a username and password of the account that has sufficient rights to access this resource on your TFS server. Username needs to be provided in the form of DOMAIN\UserName. A bare minimum of access rights are necessary for accessing this resource and a View instance-level information permission on the server level is more than sufficient. You can set server-level permissions from the Team Foundation Administration Console or using the TFSSecurity command line tool. Now assign the newly created health monitor to your NLB pool and you are ready to go.

In case you are trying to do so from a script for some of your custom dashboards, I wrote a CmdLet that will return true or false based on the response received from the call to the above mentioned REST resource.

function Get-ServerStatus()
{
	[CmdletBinding()]
    [OutputType([bool])]
	param
	(
		[parameter(Mandatory = $true)]
		[Uri]$url,
        [System.Management.Automation.PSCredential]$credential
	)
	BEGIN
    {
        if ($url.AbsoluteUri)
        {
            $url = $url.AbsoluteUri.TrimEnd('/')
        }

        Add-Type -AssemblyName System.Net.Http
	}
	PROCESS
	{
		$httpClientHandler = New-Object System.Net.Http.HttpClientHandler
 
        if ($Credential)
        {
		    $networkCredential = New-Object System.Net.NetworkCredential @($Credential.UserName, $Credential.Password)
		    $httpClientHandler.Credentials = $networkCredential
        }
        else
        {
            $httpClientHandler.UseDefaultCredentials = $true
        }

        $httpClient = New-Object System.Net.Http.Httpclient $httpClientHandler

        try
        {
			$response = $httpClient.GetAsync("$url/_apis/health").Result
 
			if ($response.IsSuccessStatusCode)
			{
				return $true
			}
 
			return $false
        }
        catch [Exception]
        {
			$PSCmdlet.ThrowTerminatingError($_)
        }
        finally
        {
            if($null -ne $httpClient)
            {
                $httpClient.Dispose()
            }
 
            if($null -ne $response)
            {
                $response.Dispose()
            }
        }

		return $false
	}
	END { }
}

It is sufficient to invoke this cmdlet by passing in the URL of your TFS instance and eventually the credentials. If no credentials are provided, current process credentials will be used.

$uri = "http(s)://your.tfs.address:port/tfs"
$credential = Get-Credential

$state = Get-ServerStatus $uri $credential

A simple solution is now in place that will keep other tools informed about the availability of our TFS instance.

Good luck!

Querying XL TestView via PowerShell

Mario Majcica March 18, 2016 0 Comments

Since the version 1.4.0 XL TestView started exposing several functionality via REST API. This is inline with other XebiaLabs products and it is a welcome characteristic. If you tried invoking a web request towards your XL TestView server, you may be surprised that the authentication fails (no matter the usual technique of passing the credentials). This is due to the fact that XL TestView doesn’t support the challenge-response authentication mechanism.

An example:

$credential = Get-Credential
$server = "http://xld.westeurope.cloudapp.azure.com:6516/api/v1"
    
Invoke-WebRequest $server/projects -Credential $credential

After executing this code you will receive a 401 Unauthorized response with Jetty (XL TestView web server) stating “Full authentication is required to access this resource”.

Invoke-WebRequest cmdlet doesn’t send the authentication headers with the first call and it expects a
401 response with the correct WWW-Authenticate header as described in RFC2617. Then, based on the authentication schema token received, prepares a call with a proper authentication method if supported.
Unfortunately this behavior is not supported by XL TestView. Still, do not desperate, there is a way to interact with XL TestView via your PowerShell scripts.

Authentication header

In order to authenticate on the first call, we need to provide the authentication header manually and include it in our web request. Knowing that XL TestView uses the Basic authentication we need to prepare the necessary for this operation. First of all we need to create the value that we are going to provide for the header called Authorization. It is following the well know standard described in RFC1945 which states that the username and password are combined into a string separated by a colon, e.g.: username:password, that the resulting string is encoded using the RFC2045-MIME variant of Base64, except not limited to 76 char/line and that the authorization method and a space i.e. “Basic ” is then put before the encoded string.

In order to create such a header I created a cmdlet that sums those steps.

function Get-AuthorizationHeader
{
	[CmdletBinding()]
	param
	(
	[string][parameter(Mandatory = $true)]$Username,
	[string][parameter(Mandatory = $true)]$Password
	)
	BEGIN { }
	PROCESS
	{
		$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $UserName, $Password)))
		
		return @{Authorization=("Basic {0}" -f $base64AuthInfo)}
	}
	END { }
}

Invoking this cmdlet and providing the requested username and password, will return the requested header object that we can include in our call towards the XL TestView REST API.

Invoking the web request

Once our cmdlet for the necessary authentication header is set, we can invoke our call simply as follows:

$Username = "username"
$Password = "password"
Invoke-WebRequest $url -Headers (Get-AuthorizationHeader $Username $Password)

You can see that we are not telling the Invoke-WebRequest to used credentials to authenticate, however we are specifying the necessary header for the authentication. This will pass all of the necessary on the first request towards XL TestView and our call should succeed.

Be aware that with the Basic authentication the credentials are passed in clear (encoded as base64 string) and an encrypted connection is advised (https).

This technique is valid for all of the services that do not use challenge-response authentication mechanism, not only XL TestView.