Working with TFS in IntelliJ IDEA via VSTS Plugin

Mario Majcica January 2, 2017 0 Comments

Recently Microsoft released a plugin for JetBrains IntelliJ IDEA in order to integrate this IDE with VSTS and TFS. Although Microsoft did a good job describing on how this plugin should be installed and described some basic settings, there are many non covered questions by the documentation provided. This is especially true when it comes to TFS and HTTPS. I will show you what is necessary in order to setup a connection for both Git and TFVC repositories on TFS.

Installing the plugin

In order to install the plugin, in the main screen of IDEA, choose Plugins from Configure menu:

Once the plugin window opens, choose Browse repositories

In browse repositories search for Visual Studio

If the Visual Studio Team Services plugin is not found, your connection may not be setup correctly. In case you are, as I am, behind a proxy, you need to click on HTTP Proxy settings button in the same screen (bottom left) and you will be presented with the following dialog

Here you need to setup the necessary parameters (Auto detect proxy settings worked for me) and test your connection by clicking on Check connection button. Once done you will be prompted to enter an address that is behind the proxy so that internet connection can be verified. I used http://www.google.com/ for my test.

After confirming, you should see the following message

Now, back to Browse repositories screen, the Visual Studio Team Services plugin now should be found. Click on install and after the procedure finishes you will be asked to restart IntelliJ IDEA. Do so, restart the IDE.

The plugin is now installed. In case this is not sufficiently clear, you can also follow the Microsoft guide Visual Studio Team Services Plugin for IntelliJ IDEA and Android Studio.

Be also sure that you have at least version v1.111.0 installed as I encountered issues with TFVC and previous versions of the plugin.

Importing the certificate

In case you are not accessing TFS through SSL connection you can skip this part. For Visual Studio Team Services plugin to connect to the TFS via https, that is using a self-signed certificate, no matter if you do intend using Git or TFVC, you need to import the certificate in the IntelliJ IDEA certificate store. To do so, export your certificate in the Base-64 encoded X.509 format. You can read about this in one of mine previous posts, Installing self-signed certificates into Git cert store.
Once you exported your certificate and let’s say saved it in C:\temp folder under the name tfs.cer, you need to open the command prompt and position yourself under the folder containing keytool.exe application. You can find it in the IntelliJ IDEA install folder, which in my case is C:\Program Files (x86)\JetBrains\IntelliJ IDEA Community Edition 2016.3.1\jre\jre\bin.

Once there you will then execute the following command

keytool -keystore "C:\Program Files (x86)\JetBrains\IntelliJ IDEA Community Edition 2016.3.1\jre\jre\lib\security\cacerts" -importcert -alias TFS -file "C:\temp\tfs.cer"

You will be then prompted for the keystore password. If you haven’t changed it, the default password is

changeit

After typing the password you will see the details about the certificate you are trying to import, and again you will be prompted to confirm that you trust this certificate. Type yes and you are done. Your screen should look like this

Visual Studio Team Services plugin and Git

Before we even start, we need to be sure that a Git client is installed on our machine. From the Welcome screen choose Configure then Settings.

In the settings window move to Version control -> Git pane and test the path to your Git client

In case all is good you should get back the version of your Git client

If this is not the case, or the path is wrong (change it and try again) or the Git client is not present on your machine. You can install Git for windows and you will find the necessary here.

In case you are using the https connection with TFS and it is based on a self-signed certificate do not forget to add that certificate into the Git cert store. This is something different then adding it to the Java cert store that I described in the previous chapter. You will need to do both of these steps. As I already described this procedure in detail, you can follow my other blog post Installing self-signed certificates into Git cert store.

Once the Git client is installed and certificate is imported, we can continue setting up the Visual Studio Team Services plugin. Get back to the welcome scree of IntelliJ IDEA and in the version control drop down choose Team Services Git

At this point a new dialog will be presented to you.

Move to the Team Foundation Server tab and specify the address of your TFS server, then click connect. You will now be prompted for the credentials and if everything is ok, you will be show the list of available repositories

You are now able to clone the repository of your choice and start working with it directly from IntelliJ IDEA.

Visual Studio Team Services plugin and TFVC

Before starting with TFVC, as for Git, we need an external tool. The tool in question is TF command line tool. It ships with the Microsoft Team Explorer Everywhere 2015 and you can download it here.
The file we are interested in is TEE-CLC-14.0.3.zip. Download it and unzip it in a folder of your choice. You should end up with something similar to this.

Now, open the command prompt, move into the folder where you have extracted the TF command line tool and run the following:

tf eula /accept

If command succeeded and you haven’t received any error, you are good to go.

Now back to IDEA. Open the settings panel.

and move to Version Control -> TFVC pane. In the select path to executable field, enter the exact path to the tf.cmd command file located in TF command line tool folder.

Once done, press the test button and you should see the following message

Confirm all of the open windows and get back to the IDEA welcome page. Now you are ready to choose Team Services TFVC (Preview) version control.

At this point, same as for Git, you will be prompted about the connection towards your TFS. The following dialog will be shown.

Move to the Team Foundation Server tab and specify the address of your TFS server, then click connect. You will now be prompted for the credentials and if everything is ok, you will be shown the list of available TFVC repositories.

You can now create a new workspace directly from IDEA and start working with your TFVC repositories.

Troubleshooting

There are a couple of common issues you may encounter in following what I just described. I will tell you about the most common ones and how to overcome those.

In case you see the message in the following screenshot

You are probably facing some issues with the certificate. Make sure that you exported/imported the certificate correctly.

During the test of the TF command line utility, you may encounter the following exception

In may case it turned out to be a problem with allocating the heap memory from TF process. I could clearly see in the log file the following:

2016-12-22 13:14:08,949 [ 17642] INFO - ugin.external.commands.Command - 167680800(ns) - elapsed time for add -noprompt -?
2016-12-22 13:15:46,592 [ 115285] INFO - lugin.external.ToolRunnerCache - getRunningToolRunner: toolLocation={0}
2016-12-22 13:15:46,592 [ 115285] INFO - lugin.external.ToolRunnerCache - getRunningToolRunner: slow version - null
2016-12-22 13:15:46,592 [ 115285] INFO - alm.plugin.external.ToolRunner - ToolRunner.start: toolLocation = C:\Utils\TEE-CLC-14.0.3\tf.cmd
2016-12-22 13:15:46,592 [ 115285] INFO - alm.plugin.external.ToolRunner - ToolRunner.start: workingDirectory = null
2016-12-22 13:15:46,592 [ 115285] INFO - alm.plugin.external.ToolRunner - arguments: add -noprompt -?
2016-12-22 13:15:46,716 [ 115409] INFO - ugin.external.commands.Command - CMD: Error occurred during initialization of VM
2016-12-22 13:15:46,717 [ 115410] INFO - ugin.external.commands.Command - CMD: Could not reserve enough space for 2097152KB object heap
2016-12-22 13:15:46,731 [ 115424] WARN - ugin.external.commands.Command - CMD: parsing output failed
com.microsoft.alm.plugin.external.exceptions.ToolBadExitCodeException: KEY_TF_BAD_EXIT_CODE
at com.microsoft.alm.plugin.external.tools.TfTool.throwBadExitCode(TfTool.java:109)
at com.microsoft.alm.plugin.external.commands.Command$1.completed(Command.java:155)
at com.microsoft.alm.plugin.external.ToolRunner$ListenerProxy.completed(ToolRunner.java:289)
at com.microsoft.alm.plugin.external.ToolRunner$ProcessWaiter.run(ToolRunner.java:327)

A workaround for this issue is to modify the tf.cmd file by specifying a lower -Xmx parameter. By default it is set to 2014MB however a 1024MB also worked well for me.

Another issue with this version of Visual Studio Team Services plugin is in changing in between the Team Services TFVC (Preview) and Team Services Git, or viceversa. You can read more about it here, TfsAuthenticator hangs the IDE.

No matter the issue, you can always find more information about it in the log file. The log file is located in my case in C:\Users\majcicam\.IdeaIC2016.3\system\log. Adapt this path to your case. The log file name is idea.log.
Visual Studio Team Services Plugin settings can be found in vsts_settings.xml file under C:\Users\majcicam\.IdeaIC2016.3\config\options folder.

Useful links

On visualstudio.com you can read more about this topic and see some useful how-to video.

1. Installing Visual Studio Team Services Plugin for IntelliJ IDEA and Android Studio
2. Using Visual Studio Team Services Plugin for IntelliJ

With all of these information I do hope you can get on going with the plugin and boost your productivity.

Happy coding.

TFS 2015 Build System overview

Mario Majcica December 1, 2015 1 Comments

Introduction

With TFS 2015 a new build system was introduced. If you were using Visual Studio Online you may already have seen it. With TFS 2015 the build system has been completely revamped.

Team Foundation Server has included a build system since its first release. Historically, we started running our builds with XML based MSBuild scripts using a windows service as host. With TFS 2010 a new build system was introduced. It included a more robust infrastructure based on build controllers and agents and the build process was driven by a Workflow Foundation XAML script. Although it had its own limitations and very steep learning curve it was maintained through two versions of TFS, 2012 and 2013. Luckily, with TFS 2015, Microsoft decided to solve major issues of the build system and came up with a new solution. The new build system is finally cross platform and provides more flexibility for infrastructure.

What’s new

A great news for all of you who do not fancy the WWF and XAML (and I’m part of that group), the process is no longer based on it! Infrastructure got simplified and as already mentioned, there is now support for Mac OSX and Linux. On Microsoft website you can find an overview of the new features at https://msdn.microsoft.com/Library/vs/alm/Build/feature-overview

Infrastructure

Some new concepts are introduced into the build infrastructure landscape. There is no longer the necessity for having build controllers and all of the limitations and complexity they brought. Instead, two new concepts are now there to help us with grouping and organizing our build resources. Pool and queues are only “organizational” units and do not require any specific software to be installed or configured.

Pools

A pool is essentially an association of a collection of one or more build agents to one or more queues. Pools are defined at the TFS application tier level. You can create as many pools as you like and you can assign pool administrators to each of them. Different people/groups can be assigned to different pools allowing different groups of people to manage your various build assets.

In synthesis, agent pools are used to organize and define permission boundaries around your build agents.

Queues

Queues are defined at the project collection level and are tied directly to a pool. A queue can be tied to at most one pool; however, a pool can be tied to more than one queue assuming the queues are configured on multiple project collections. As build definitions are created they are associated with a queue. The queue you select for a build definition ultimately limits the set of build agents that can be utilized for the builds generated by that definition.

When you create a new queue you must immediately select the pool that is associated with it. You can also choose to have the queue created and associated with the new pool automatically at the time the pool is created (it’s an option on the new pool dialog).

Agents

The build agent is a small application residing on a build server, a machine that is intended to be used for executing automated builds. The build agents are responsible for building your application (based on the build definitions). Although a pool can be associated with many build agents, a build agent is associated with at most a single pool. With the new build system a Microsoft Cross Platform Build Agent is available and Mac OSX and Linux platforms are supported. We will see later on how to proceed with the installation of a build agent.

The following diagram is a schematic overview of these concepts:

Build process

Build process has drastically changed. We do not have any more the build process template, which is substituted by a collection of build tasks (steps). In order to define your build process, you will add one or more build tasks in your build definition. By default several build tasks are available as Visual Studio Build, Visual Studio Test and many more. As you can guess, each of them perform an operation in order to accomplish the process you had in mind for your build.

More tasks are available online as also all the source code of all the tasks that are available on TFS. You can find a list of the ones supplied by Microsoft at the following address https://msdn.microsoft.com/Library/vs/alm/Build/steps/index and the relevant source code at https://github.com/Microsoft/vso-agent-tasks.

Aside of this important change, there are several things we can benefit of, such as, versioning and real-time output.

Versioning

Build definition versioning and auditing is introduced with the new system so now you can easily see who made the change and exactly what change was made on your build definition. Any change to build definition is logged and you can add a note corresponding to your changes. You can see what changed and revert to the desired version directly from the web interface.

Real-time output

Some of the most noticeable things about the way that build runs under this new system are the ability to get a real-time visibility while the build runs, which means that we will spend less time trying to dig through logs to see what really happened with your build. You can expect that you will get the same output as your build run locally.

Interface

Unlike the previous version, where you can only make changes to your build definition from Visual Studio, new build definition management is all web based. You do not need any other tool than your web browser in order to create, execute and manage your builds.

Build task anatomy

Every build task is composed from at least a file describing the build task. Describing a task and defining the elements in the interface is done via a predefined structure expressed via a JSON format. At example:

{
   "id": "61ed2e1d-efb7-406e-a31b-81f5d22e6d54",
   "name": "TestTask",
   "friendlyName": "Name that is displayed in the list",
   "description": "Testing new TFS 2015 build system.",
   "category": "Package",
   …

At the bottom of the task file you need to indicate the name of the associated PowerShell script which actually implements the logic of the task itself, or its node .js equivalent in case the task is meant to run cross-platform. If the task should run on both windows and xplat agents, you can indicate both:

"execution": {
   "Node": {
      "target": "ant2.js",
      "argumentFormat": ""
   },
   "PowerShell": {
      "target": "$(currentDirectory)\ant.ps1",
      "argumentFormat": "",
      "workingDirectory": "$(currentDirectory)"
   }
}

Other custom files such as executable can also be shipped with the task. In case you need to localize the task, it can be done by adding the necessary resources. Also you can specify the icon that will represent the task.

Once all of the necessary items are in the folder, you need to zip it and upload to the server. Uploading is done by calling a specific resource on TFS REST API or by utilities provided by Microsoft. Currently only ‘Agent Pool Administrators’ are able to add/update or remove tasks. Tasks are server-wide, this means that you will upload to the server, not to a specific collection or project.

Once you start creating your custom task, the best thing is to check how certain functionalities are implemented by peeking at the source code of the tasks available on GitHub.

TFS 2015 Update 1

As of today, Microsoft started shipping Update 1 for TFS 2015. It brings a couple of things in regard to a build system. With this update, build administrators can now add permissions to agent queue, which will restrict who can use that queue in a build definition. Several new build tasks are added and several minor improvements are present.
For a full list of the improvements you can consult the release notes.

I hope this post gave you an idea about what the new build system brings and made some of the new concepts clearer. If you are not enthusiastic about it maybe the following article could help you getting the right reasons to onboard on TFS 2015 build system.
In the following months I plan to extensively write about TFS 2015 and it’s new build system, bringing several examples from a real life implementation and answers to the questions I got during a training course I held on this argument.

I bet you will enjoy it.

Happy building!