Uploading artifacts into Nexus via PowerShell

It may not be the most common thing, however it may happen that you would need to upload an artifact to a maven repository in Nexus via PowerShell. In order to achieve that, we will use Nexus REST API which for this task requires a multipart/form-data POST to /service/local/artifact/maven/content resource. This is however not a trivial task. It is notoriously difficult to manage a Multipart/form-data standard in PowerShell, as I already described in one of my previous post PowerShell tips and tricks – Multipart/form-data requests. As you can see this type of call is necessary in order to upload an artifact to your Nexus server. I will not get in a details about Multipart/form-data requests and if interested about the details, you can check just mentioned post.

For this purpose I created two cmdlets that will allow me to upload an artifact by supplying GAV parameters or by passing the GAV definition in a POM file. A couple of simple functions do support both of these cmdlets.

GAV definition in a POM file

As the heading suggests, this cmdlet will let you upload your artifact and specify the GAV parameters via a POM file.

function Import-ArtifactPOM()
{
	[CmdletBinding()]
	param
	(
		[string][parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]$EndpointUrl,
		[string][parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]$Repository,
		[string][parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]$PomFilePath,
		[string][parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]$PackagePath,
		[System.Management.Automation.PSCredential][parameter(Mandatory = $true)]$Credential
	)
	BEGIN
	{
		if (-not (Test-Path $PackagePath))
		{
			$errorMessage = ("Package file {0} missing or unable to read." -f $PackagePath)
			$exception =  New-Object System.Exception $errorMessage
			$errorRecord = New-Object System.Management.Automation.ErrorRecord $exception, 'ArtifactUpload', ([System.Management.Automation.ErrorCategory]::InvalidArgument), $PackagePath
			$PSCmdlet.ThrowTerminatingError($errorRecord)
		}
		
		if (-not (Test-Path $PomFilePath))
		{
			$errorMessage = ("POM file {0} missing or unable to read." -f $PomFilePath)
			$exception =  New-Object System.Exception $errorMessage
			$errorRecord = New-Object System.Management.Automation.ErrorRecord $exception, 'ArtifactUpload', ([System.Management.Automation.ErrorCategory]::InvalidArgument), $PomFilePath
			$PSCmdlet.ThrowTerminatingError($errorRecord)
		}

		Add-Type -AssemblyName System.Net.Http
	}
	PROCESS
	{
		$repoContent = CreateStringContent "r" $Repository
		$groupContent = CreateStringContent "hasPom" "true"
		$pomContent = CreateStringContent "file" "$(Get-Content $PomFilePath)" ([system.IO.Path]::GetFileName($PomFilePath)) "text/xml"
		$streamContent = CreateStreamContent $PackagePath

		$content = New-Object -TypeName System.Net.Http.MultipartFormDataContent
		$content.Add($repoContent)
		$content.Add($groupContent)
		$content.Add($pomContent)
		$content.Add($streamContent)

		$httpClientHandler = GetHttpClientHandler $Credential

		return PostArtifact $EndpointUrl $httpClientHandler $content
	}
	END { }
}

In order to invoke this cmdlet you will need to supply the following parameters:

  • EndpointUrl – Address of your Nexus server
  • Repository – Name of your repository in Nexus
  • PomFilePath – Full, literal path pointing to your POM file
  • PackagePath – Full, literal path pointing to your Artifact
  • Credential – Credentials in the form of PSCredential object

I will create a POM file with the following content:

<project>
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.mycompany.app</groupId>
  <artifactId>my-app</artifactId>
  <version>3</version>
</project>

And invoke my cmdlet in the following way:

$server = "http://nexus.maio.com"
$repoName = "maven"
$pomFile = "C:\Users\majcicam\Desktop\pom.xml"
$package = "C:\Users\majcicam\Desktop\junit-4.12.jar"
$credential = Get-Credential

Import-ArtifactPOM $server $repoName $pomFile $package $credential

If everything is correctly setup, you will be first asked to provide the credentials, then the upload will start. If it succeeds, you will receive back a string like this:

{"groupId":"com.mycompany.app","artifactId":"my-app","version":"3","packaging":"jar"}

It is a json representation of the information about imported package.

Manually supplying GAV parameters

The other cmdlet is based on a similar principle, however it doesn’t require a POM file to be passed in, instead it let you provide GAV values as parameter to the call.

function Import-ArtifactGAV()
{
	[CmdletBinding()]
	param
	(
		[string][parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]$EndpointUrl,
		[string][parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]$Repository,
		[string][parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]$Group,
		[string][parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]$Artifact,
		[string][parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]$Version,
		[string][parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]$Packaging,
		[string][parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]$PackagePath,
		[System.Management.Automation.PSCredential][parameter(Mandatory = $true)]$Credential
	)
	BEGIN
	{
		if (-not (Test-Path $PackagePath))
		{
			$errorMessage = ("Package file {0} missing or unable to read." -f $packagePath)
			$exception =  New-Object System.Exception $errorMessage
			$errorRecord = New-Object System.Management.Automation.ErrorRecord $exception, 'XLDPkgUpload', ([System.Management.Automation.ErrorCategory]::InvalidArgument), $packagePath
			$PSCmdlet.ThrowTerminatingError($errorRecord)
		}

		Add-Type -AssemblyName System.Net.Http
	}
	PROCESS
	{
		$repoContent = CreateStringContent "r" $Repository
		$groupContent = CreateStringContent "g" $Group
		$artifactContent = CreateStringContent "a" $Artifact
		$versionContent = CreateStringContent "v" $Version
		$packagingContent = CreateStringContent "p" $Packaging
		$streamContent = CreateStreamContent $PackagePath

		$content = New-Object -TypeName System.Net.Http.MultipartFormDataContent
		$content.Add($repoContent)
		$content.Add($groupContent)
		$content.Add($artifactContent)
		$content.Add($versionContent)
		$content.Add($packagingContent)
		$content.Add($streamContent)

		$httpClientHandler = GetHttpClientHandler $Credential

		return PostArtifact $EndpointUrl $httpClientHandler $content
	}
	END { }
}

In order to invoke this cmdlet you will need to supply the following parameters:

  • EndpointUrl – Address of your Nexus server
  • Repository – Name of your repository in Nexus
  • Group – Group Id
  • Artifact – Maven artifact Id
  • Version – Artifact version
  • Packaging – Packaging type (at ex. jar, war, ear, rar, etc.)
  • PackagePath – Full, literal path pointing to your Artifact
  • Credential – Credentials in the form of PSCredential object

An example of invocation:

$server = "http://nexus.maio.com"
$repoName = "maven"
$group = "com.test"
$artifact = "project"
$version = "2.4"
$packaging = "jar"
$package = "C:\Users\majcicam\Desktop\junit-4.12.jar"
$credential = Get-Credential

Import-ArtifactGAV $server $repoName $group $artifact $version $packaging $package $credential

If all goes as expected you will again receive a response confirming the imported values.

{"groupId":"com.test","artifactId":"project","version":"2.4","packaging":"jar"}

Supporting functions

As you could see, my cmdlets relay on a couple of functions. They are essential in this process so I will analyse them one by one.

function CreateStringContent()
{
	param
	(
		[string][parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]$Name,
		[string][parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]$Value,
		[string]$FileName,
		[string]$MediaTypeHeaderValue
	)
		$contentDispositionHeaderValue = New-Object -TypeName  System.Net.Http.Headers.ContentDispositionHeaderValue -ArgumentList @("form-data")
		$contentDispositionHeaderValue.Name = $Name

		if ($FileName)
		{
			$contentDispositionHeaderValue.FileName = $FileName
		}
		
		$content = New-Object -TypeName System.Net.Http.StringContent -ArgumentList @($Value)
		$content.Headers.ContentDisposition = $contentDispositionHeaderValue

		if ($MediaTypeHeaderValue)
		{
			$content.Headers.ContentType = New-Object -TypeName System.Net.Http.Headers.MediaTypeHeaderValue $MediaTypeHeaderValue
		}

		return $content
}

function CreateStreamContent()
{
	param
	(
		[string][parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]$PackagePath
	)
		$packageFileStream = New-Object -TypeName System.IO.FileStream -ArgumentList @($PackagePath, [System.IO.FileMode]::Open)
		
		$contentDispositionHeaderValue = New-Object -TypeName  System.Net.Http.Headers.ContentDispositionHeaderValue "form-data"
		$contentDispositionHeaderValue.Name = "file"
		$contentDispositionHeaderValue.FileName = Split-Path $packagePath -leaf

		$streamContent = New-Object -TypeName System.Net.Http.StreamContent $packageFileStream
		$streamContent.Headers.ContentDisposition = $contentDispositionHeaderValue
		$streamContent.Headers.ContentType = New-Object -TypeName System.Net.Http.Headers.MediaTypeHeaderValue "application/octet-stream"

		return $streamContent
}

First two functions are there to create a correct HTTP content. Each content object we create will correspond to a content-disposition header. The first function will return a string type values for the just mentioned header, meanwhile the Stream content will return a stream that will be consumed later on, having as a value octet-stream representation of our artifact.

GetHttpClientHandler function is a helper that will create the right http client handler that contains the credentials to be used for our call.

function GetHttpClientHandler()
{
	param
	(
		[System.Management.Automation.PSCredential][parameter(Mandatory = $true)]$Credential
	)

	$networkCredential = New-Object -TypeName System.Net.NetworkCredential -ArgumentList @($Credential.UserName, $Credential.Password)
	$httpClientHandler = New-Object -TypeName System.Net.Http.HttpClientHandler
	$httpClientHandler.Credentials = $networkCredential

	return $httpClientHandler
}

Last but not least, a function that actually invokes the post call to our server.

function PostArtifact()
{
	param
	(
		[string][parameter(Mandatory = $true)]$EndpointUrl,
		[System.Net.Http.HttpClientHandler][parameter(Mandatory = $true)]$Handler,
		[System.Net.Http.HttpContent][parameter(Mandatory = $true)]$Content
	)

	$httpClient = New-Object -TypeName System.Net.Http.Httpclient $Handler

	try
	{
		$response = $httpClient.PostAsync("$EndpointUrl/service/local/artifact/maven/content", $Content).Result

		if (!$response.IsSuccessStatusCode)
		{
			$responseBody = $response.Content.ReadAsStringAsync().Result
			$errorMessage = "Status code {0}. Reason {1}. Server reported the following message: {2}." -f $response.StatusCode, $response.ReasonPhrase, $responseBody

			throw [System.Net.Http.HttpRequestException] $errorMessage
		}

		return $response.Content.ReadAsStringAsync().Result
	}
	catch [Exception]
	{
		$PSCmdlet.ThrowTerminatingError($_)
	}
	finally
	{
		if($null -ne $httpClient)
		{
			$httpClient.Dispose()
		}

		if($null -ne $response)
		{
			$response.Dispose()
		}
	}
}

This is all the necessary to upload an artifact to our Nexus server. You can find the just show scripts also on GitHub in the following repository tfs-build-tasks. Further information about programtically uploading artifacts into Nexus can be found in the following post, How can I programatically upload an artifact into Nexus?.

Securing Nexus Repository Manager OSS

Recently one of my articles got published on SonaType web site. It is based on my blog post Nexus Repository Manager OSS as Nuget server, which, after reviewing it for SonaType blog, somehow didn’t seemed complete. Thus, I decided to add an extra chapter about the security which I would like to share with you, also on my personal blog, here.

Security

I suppose that your computers are part of an Active Directory domain. I will propose here a setup to make your new Nexus instance dialogue with your domain controller when it comes to authenticating users. Why? Well, uploading packages to the hosted repository requires an API-key and it is assigned on per user basis. If you wish that your users (services) to be distinguished and limit some of them, you would need to setup the authentication and roles correctly. You could do this by simply using Nexus internal database, however, as we all know, adding users manually and creating another identity, from convenience and maintenance prospective is not recommendable. Not to speak about the security implications.

Setting up LDAP connection

In the left-hand main menu panel expand the security group and select the LDAP Configuration option. A new tab will open and you will be presented with a long list of parameters:

ldap-configuration

We are going to analyze them one by one and explain the values you do need to enter in order to make this connection work.

Protocol
Here you have two choices Lightweight Directory Access Protocol and the Lightweight Directory Access Protocol over SSL. Probably you are going to use just a simple LDAP protocol and not the

Hostname
The hostname or IP address of the LDAP server. Simply add the FQDN towards your domain controller.

Port
The port on which the LDAP server is listening. Port 389 is the default port for the ldap protocol, and port 636 is the default port for the ldaps.
If you have a multi domain, distributed Active Directory forest, you should connect to the Active Directory through port 3268. Connecting directly to port 389 might lead to errors. Port 3268 exposes Global Catalog Server that exposes the distributed data. The SSL equivalent connection port is 3269.

Search Base
The search base is the Distinguished Name (DN) to be appended to the LDAP query. The search base usually corresponds to the domain name of an organization. For example, the search base on my test domain server could be dc=maio,dc=local.

Authentication Method
Simple authentication is what we are going to use in first instance for our Active Directory authentication. It is not recommended for production deployments not using the secure ldaps protocol as it sends a clear-text password over the network. You can change

Username
For the LDAP service to establish the communication with the AD server, a valid account is necessary. Usually you will create a service account in your AD for this purpose only.

Password
Password for the above mentioned account.

This is the first part of configuration. Now we need to check if what we entered is valid and if the connection can be established.
Click on Check Authentication button and if the configuration is valid, you should get the following message:

authentication-test

Base DN
You should indicate in which organizational unit Nexus should search for user accounts. By default it can equal to ‘cn=users’ however if you store your user accounts in a different OU, please specify the full path. An example, if you created an OU at the root level called MyBusiness then an OU called Users, you will need to specify the following value as base DN, ‘OU=Users,OU=MyBusiness’.

User Subtree
This needs to be check only if you are having other OU’s, under the one specified in Base DN, in which users are organized. If that is not the case, you can leave it unchecked.

Object Class
For AD it needs to be set yo ‘user’.

User ID Attribute
In the MS AD implementation this value equals to sAMAccountName

Real Name Attribute
You can set this to ‘givenName’ attribute, or simply to cn if givenName is not used during the user creation.

E-Mail Attribute
This attribute should map to ‘mail’

Group Type
Group Type needs to be set to Dynamic Groups.

Member Of Attribute
In case of MS AD this equals to ‘memberOf’.

Now that we have set all of the relevant values, we can ask Nexus OSS to give us a preview of retrieved accounts and mappings to make sure everything is set up correctly. Click on the Check User Mapping button and you should see a preview of the retrieved values and how they are mapped into Nexus OSS. If everything looks fine , save your configuration.

Once the configuration is persisted, the last thing to do is to let Nexus know that you would like to use this new setup for the authentication. This is done by enabling the LDAP Authentication Realm.

In the left-hand main menu panel, from the administration group, choose Server. In the nexus tab that opens, go to the security settings

selected-realms

You will find the OSS LDAP Authentication Realm in the group shown on the right side, called Available Realms. Move it to Selected realms (as shown in the picture above) and save this settings.

Configuring roles

Now that we are able to authenticate through LDAP (AD) we need to set who can do what. This is when the roles do get handy. I usually create security groups and then associate them to roles as this moves the operations of granting the access rights to merely assigning a user to a group.
You can still associate roles to the user in more or less the same way, however I’m going to show you my favorite approach.

From the left-hand main menu panel, in the Security group, select Roles. A new tab showing a list of roles will appear.

external-role-mapping

Choose Add and then External Role Mapping option. You will then be presented with the following dialog:

map-external-role

As you can see from this screenshot, choose LDAP as Realm and the role (an AD security group to be precise) you wish to map. Be aware that in some cases, not all of the groups will be listed. It is a well know bug. If this is your case, there is a workaround. Add a Nexus Role instead of the External Role Mapping and name the role precisely as your missing group is named. This should do the trick until this issue gets solved.

Once the mapping is added, you will need to assign roles (actual rights) to selected security group.

external-role-mapping

On the Role/Privilege Management bar, choose Add button and select the following roles:

  • Nexus API-Key Access
  • Nexus Deployment Role
  • Repo: All NuGet Repositories (Full Control)
  • Artifact Upload

These set of rights will enable whoever part of that group to login in Nexus OSS, manually upload packages, get the API-Key, and generally to control all NuGet repositories. In other words, a sort of developers role when it comes to NuGet.

When it comes to administrator role, it is sufficient to assign only the Nexus Administrator Role.
At the end of selecting roles, do not forget to click on Save button and persist this configuration.

This two basic profiles should be sufficient to start using Nexus. If further profiles are necessary, check the managing roles guide.

You can now test all of our security setup. Make sure you AD user account is part of the administrative security group that you just mapped to Nexus Administrator Role. Log out and log back in with your domain account. If you succeed to login, congratulations LDAP is set correctly.

Querying XL TestView via PowerShell

Since the version 1.4.0 XL TestView started exposing several functionality via REST API. This is inline with other XebiaLabs products and it is a welcome characteristic. If you tried invoking a web request towards your XL TestView server, you may be surprised that the authentication fails (no matter the usual technique of passing the credentials). This is due to the fact that XL TestView doesn’t support the challenge-response authentication mechanism.

An example:

$credential = Get-Credential
$server = "http://xld.westeurope.cloudapp.azure.com:6516/api/v1"
    
Invoke-WebRequest $server/projects -Credential $credential

After executing this code you will receive a 401 Unauthorized response with Jetty (XL TestView web server) stating “Full authentication is required to access this resource”.

ps401

Invoke-WebRequest cmdlet doesn’t send the authentication headers with the first call and it expects a
401 response with the correct WWW-Authenticate header as described in RFC2617. Then, based on the authentication schema token received, prepares a call with a proper authentication method if supported.
Unfortunately this behavior is not supported by XL TestView. Still, do not desperate, there is a way to interact with XL TestView via your PowerShell scripts.

Authentication header

In order to authenticate on the first call, we need to provide the authentication header manually and include it in our web request. Knowing that XL TestView uses the Basic authentication we need to prepare the necessary for this operation. First of all we need to create the value that we are going to provide for the header called Authorization. It is following the well know standard described in RFC1945 which states that the username and password are combined into a string separated by a colon, e.g.: username:password, that the resulting string is encoded using the RFC2045-MIME variant of Base64, except not limited to 76 char/line and that the authorization method and a space i.e. “Basic ” is then put before the encoded string.

In order to create such a header I created a cmdlet that sums those steps.

function Get-AuthorizationHeader
{
	[CmdletBinding()]
	param
	(
	[string][parameter(Mandatory = $true)]$Username,
	[string][parameter(Mandatory = $true)]$Password
	)
	BEGIN { }
	PROCESS
	{
		$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $UserName, $Password)))
		
		return @{Authorization=("Basic {0}" -f $base64AuthInfo)}
	}
	END { }
}

Invoking this cmdlet and providing the requested username and password, will return the requested header object that we can include in our call towards the XL TestView REST API.

Invoking the web request

Once our cmdlet for the necessary authentication header is set, we can invoke our call simply as follows:

$Username = "username"
$Password = "password"
Invoke-WebRequest $url -Headers (Get-AuthorizationHeader $Username $Password)

You can see that we are not telling the Invoke-WebRequest to used credentials to authenticate, however we are specifying the necessary header for the authentication. This will pass all of the necessary on the first request towards XL TestView and our call should succeed.

Be aware that with the Basic authentication the credentials are passed in clear (encoded as base64 string) and an encrypted connection is advised (https).

This technique is valid for all of the services that do not use challenge-response authentication mechanism, not only XL TestView.