Querying XL TestView via PowerShell

Since the version 1.4.0 XL TestView started exposing several functionality via REST API. This is inline with other XebiaLabs products and it is a welcome characteristic. If you tried invoking a web request towards your XL TestView server, you may be surprised that the authentication fails (no matter the usual technique of passing the credentials). This is due to the fact that XL TestView doesn’t support the challenge-response authentication mechanism.

An example:

$credential = Get-Credential
$server = "http://xld.westeurope.cloudapp.azure.com:6516/api/v1"
    
Invoke-WebRequest $server/projects -Credential $credential

After executing this code you will receive a 401 Unauthorized response with Jetty (XL TestView web server) stating “Full authentication is required to access this resource”.

ps401

Invoke-WebRequest cmdlet doesn’t send the authentication headers with the first call and it expects a
401 response with the correct WWW-Authenticate header as described in RFC2617. Then, based on the authentication schema token received, prepares a call with a proper authentication method if supported.
Unfortunately this behavior is not supported by XL TestView. Still, do not desperate, there is a way to interact with XL TestView via your PowerShell scripts.

Authentication header

In order to authenticate on the first call, we need to provide the authentication header manually and include it in our web request. Knowing that XL TestView uses the Basic authentication we need to prepare the necessary for this operation. First of all we need to create the value that we are going to provide for the header called Authorization. It is following the well know standard described in RFC1945 which states that the username and password are combined into a string separated by a colon, e.g.: username:password, that the resulting string is encoded using the RFC2045-MIME variant of Base64, except not limited to 76 char/line and that the authorization method and a space i.e. “Basic ” is then put before the encoded string.

In order to create such a header I created a cmdlet that sums those steps.

function Get-AuthorizationHeader
{
	[CmdletBinding()]
	param
	(
	[string][parameter(Mandatory = $true)]$Username,
	[string][parameter(Mandatory = $true)]$Password
	)
	BEGIN { }
	PROCESS
	{
		$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $UserName, $Password)))
		
		return @{Authorization=("Basic {0}" -f $base64AuthInfo)}
	}
	END { }
}

Invoking this cmdlet and providing the requested username and password, will return the requested header object that we can include in our call towards the XL TestView REST API.

Invoking the web request

Once our cmdlet for the necessary authentication header is set, we can invoke our call simply as follows:

$Username = "username"
$Password = "password"
Invoke-WebRequest $url -Headers (Get-AuthorizationHeader $Username $Password)

You can see that we are not telling the Invoke-WebRequest to used credentials to authenticate, however we are specifying the necessary header for the authentication. This will pass all of the necessary on the first request towards XL TestView and our call should succeed.

Be aware that with the Basic authentication the credentials are passed in clear (encoded as base64 string) and an encrypted connection is advised (https).

This technique is valid for all of the services that do not use challenge-response authentication mechanism, not only XL TestView.

TFS 2015 behind a proxy

In many enterprise realities it is quite common that all of the internet access is made via a proxy server. Recently I wrote a post about a Tough life behind a proxy, you may check that for some of my rants and tips. What if your servers do also require internet access? Although it may seem it doesn’t, TFS 2015 has a need to access the web. Not having the access to web will not compromise any of it’s core functionality, however it will continue logging an error in the event log. It has to do with the News panel that is shown (or not) on your main portal page. As you can see on the following screenshot next to the Recent team rooms panel, on the right side, the usual News panel is not shown.

before

If you check your event log, you will probably find the following error:

System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 23.198.69.66:80
       at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
       at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Exception& exception)
       --- End of inner exception stack trace ---
       at System.Net.HttpWebRequest.GetResponse()
       at System.Xml.XmlDownloadManager.GetNonFileStream(Uri uri, ICredentials credentials, IWebProxy proxy, RequestCachePolicy cachePolicy)
       at System.Xml.XmlUrlResolver.GetEntity(Uri absoluteUri, String role, Type ofObjectToReturn)
       at System.Xml.XmlTextReaderImpl.FinishInitUriString()
       at System.Xml.XmlTextReaderImpl..ctor(String uriStr, XmlReaderSettings settings, XmlParserContext context, XmlResolver uriResolver)
       at System.Xml.XmlReaderSettings.CreateReader(String inputUri, XmlParserContext inputContext)
       at Microsoft.TeamFoundation.Server.WebAccess.Controllers.ApiCommonController.GetNews(Int32 maxCount)

This is how it looks like in the log:

Event

It is pretty clear that the action called GetNews on the ApiCommonControler was unable to fulfill the Http request it made. This is because my TFS is behind a proxy server for what concerns the web access.
In order to set the access for the TFS application via a proxy, you need to locate the correct web.config file, which in may case is present in C:\Program Files\Microsoft Team Foundation Server 14.0\Application Tier\Web Services.
What you will need to do is to edit that file and under the system.net section add the following:

<system.net>
    <defaultProxy>
    	<proxy usesystemdefault="True" proxyaddress="http://swg.eu.myproxy.com:8080" bypassonlocal="True"/>
    </defaultProxy>
</system.net>

Make sure the correct proxyaddress is set and give your TFS main portal page a go. If all went as expected and your settings are valid, you should see the following:

after

That’s it. It may not be an essential setting, however it is nice to see that there are no errors in the event log and that your users do get a full experience.

Cheers

Detailing TFS configuration – IIS

One of the most annoying things when it comes to accessing the TFS portal is that you need to specify the context path /tfs. In other words, if you just type http://mytfs.com:8080 you will not get redirected to your application, which has the full path of http://mytfs.com:8080/tfs. Same if SSL is used, often and by default you are obliged to indicate the https or even worst in case SSL is made mandatory, you will get a nice 403.

Well this is a bit of shame. Many people standing behind and supporting TFS are often not keen to set this details up. This can be because of the leak of knowledge, “fear” of the unknown, negligence, character. It is such a simple operation that shows that you do care. So let’s check a couple of things you can do in order to make this happen.

The Beauty Is In the Details

There are several improvements that we can make on IIS that is running our TFS instance. I will try to make you a couple of suggestion, if some of them can’t apply on your case by any reason, it is not mandatory to set them. Following are just suggestions on how to tide up your default TFS installation.

The cleanup

Often I do see on the IIS of TFS server a Default Web Site. In 99,9% of the cases it is not used. If that is also your case (running only TFS on that machine) you are safe to remove it.

iis-initial

As you can see, aside of the Team Foundation Server site, there is the Default Web Site in my case. I will just right click it and choose Remove.

remove-default

After you removed the Default Web Site, you can do the same for all unused Application pools.

app-pool-remove

As from the image, get to the Application Pools and remove all of the pools which name doesn’t start on per Microsoft Team Foundation.

The redirect

It will be handy that in the browser you do not need to type over and over the context path of /tfs. In order to set this up we can leverage the Http Redirect feature of IIS. By default it is not installed thus we will need to add it. Open Server Manager and choose Manage -> Add Roles and Features.

server-manager-add-role

Now get to the Server Roles and under Web Server (IIS) – Web Server – Common HTTP Features select the HTTP Redirects.

add-roles-http-redirection

Conclude the installation procedure and restart the IIS Manager.
Now after selecting your Team Foundation Server site, you will see the HTTP Redirect feature.

http-redirect

Select this option by double clicking it and enable the Redirect Requests as set on the following image.

set-redirect

In the text box you will need to enter the full URL of your TFS comprehensive of the context path. Now, once the IIS recieves a request towards the root of your application it will redirect it towards your TFS application, called tfs. Make sure before Applying these settings that the Only redirect requests to content in this directory (not subdirectories) is selected, otherwise all of your calls will result in an recursive redirect. Apply these settings and try calling your server without the context path. If you are monitoring your web calls with tools like Fiddler, you will see that your first call is redirected by the server towards the URL we specified under the Redirect request option.

This technique is only working with your portal and browser. You will always need to specify the full URL in your Visual Studio or any other tooling that requires the TFS path. This is because they are not able to understand the redirect and act in the way your browser does. Keep this in mind.

Connecting people

By default TFS will set it’s default port to 8080. Again if it is the only application on your server it is a shame being in need to specify the port for the each call. What about letting it replay also to a port 80, which is the default http port and doesn’t need to be specified?
Welcome bindings. Select your Team Foundation Server and chose in between the available actions the one called Bindings. You will be presented with the following screen.

site-bindings

Make sure that aside the http binding to port 8080 there is the one that binds the requests to the port 80. If it is not there, first edit the current binding of port 8080 and change it to port 80. Then add a new one and make it replay to port 8080. Click close and try calling your TFS without specifying the port 8080. Your IIS should replay correctly.

The result of this change will work with all of the tooling accessing TFS, like Visual Studio. You are now not anymore obliged to specify the port 8080.

Talking under four eyes

It may be a good idea or a necessity to use transport layer security. Enabling HTTPS on you web site is fairly simple. How to create and import a certificate is out of the scope of this post. Given that you have correctly imported a certificate into the IIS certificate store, open the site binding and add a new one.

add-site-binding-ssl

As a type choose https and select the certificate that you intend to use for TFS.
From now on, you can point to https://yourTFS/tfs and you will be using a secure connection. In case you omit the context path, you will end up on a non protected connection. To sort that out, change your redirect and make it point to the secure version of link.

Also you may desire to make the HTTPS mandatory. If that is the case, you could simply enable Require SSL option under SSL Settings for your TFS Web Site.

require-ssl

If you do so, pointing to a non SSL version of your site will result in a 403 Forbidden response. Although it is self explanatory as a message it is not nicely handled. It would be nicer if you would be redirected to the https version of your request. For that, we can use a trick. Open Error pages pane and edit the 403 page.

custom-error-403

Set the Respond with a 302 redirect and set your desired https URL. Now, instead of showing a Forbidden message, your browser will be automatically redirected to a correct link.

Note that in case you set the SSL mandatory, you need to do it for both the WebSite and the Virtual folder usually called tfs. After you set both of them to be required, you will need to open the TFS Administration console and on Application Tier screen, choose the Change URL’s action:

change-urls2

Make sure that Notification and server URL are correctly set to point to the new link, otherwise you may experience some issues with TFS Administration console and some other local tooling.

Conclusion

I showed you a couple of tips on how to set you TFS IIS in order to be more friendly in responses and to remove the unnecessary things. There are other tips I may have on this argument, however they do fall in a maintenance domain, such as managing log files, etc. Soon I will publish a separate, more detailed blog post about this argument.

Cheers