SonarQube LDAP plugin: a love story

It is been a while, since I struggled more then with the recent changes with the SonarQube LDAP plugin.
As I have already mentioned in my previous post, how to configure LDAP with your SonarQube instance, I’ll share with you what I experienced in the recent changes that affected the LDAP plugin.

Introduction

It all started with my upgrade of SonarQube from version 5.5 to version 5.6. With the version 5.6 the support for the LDAP plugin version 1.5.1 (which I was using) was dropped and based on compatibility matrix, we are forced to used the new 2.0 version of LDAP plugin. But wait, if we are using the Microsoft Active Directory as our LDAP directory (which I do), the documentation is pointing us towards a new, spin off, plugin called Active Directory Plugin. You can get more information about this at the official documentation page of LDAP plugin here http://docs.sonarqube.org/display/PLUG/LDAP+Plugin.
Considering this an advised practice, I decided to abandon the LDAP plugin in favor of the newcomer.
In the next paragraph I’ll show you how did my configuration changed in case you got here with the intention to achieve the same.

Just to conclude my story in regard to the LDAP integration and SonarQube upgrades. My plan went well until I decided to upgrade SonarQube again to the next new version, 6.0 in my case. I was surprised to understand that the new Active Directory Plugin was not compatible with the version 6.0 of SonarQube and that if I was keen to keep LDAP integration in place I would need to turn back to LDAP 2.0 plugin.
Not only, I also came to understand that SSO is not supported anymore. This means that my users from now on will need to type in the username and password. Hopefully the following feature will become available soon, which would solve all of these issues, https://jira.sonarsource.com/browse/SONAR-5430. As soon ready, I will write a blog post about it.

From 5.5 to 5.6

If you were using the LDAP plugin version 1.5.1 on SonarQube version 5.2-5.5 and you followed my guide in this post, your current configuration should look like this:


#----------------------------------------------------------------------
# LDAP

sonar.security.realm=LDAP
sonar.authenticator.downcase=true

Now moving towards the Active Directory plugin you need to change it to look like this:


#----------------------------------------------------------------------
# LDAP

sonar.security.realm=ACTIVE_DIRECTORY
sonar.authenticator.downcase=true

This change is sufficient to keep your LDAP integration working.

Before making this change in your configuration make sure that Active Directory plugin is installed.

You will not find the Active Directory Plugin in the Update Center. You need to download it here and install it manually (by placing the sonar-activedirectory-plugin-1.0.jar file into the extensions\plugins folder.

From 5.6 to 6.0 (and beyond)

Now the hard part. If you upgrade from your 5.6.x version to 6.0, the plugin that we just started using, will not be supported anymore. Also the SSO will not work no matter the plugin you are using. If you are planning to achieve the LDAP integration, you will have no other choice than get using the LDAP plugin version 2.0 again. In that case you LDAP configuration needs to change quite drastically. I’ll show you an example and then I’ll explain the various settings and how you should adapt it for your case.


#----------------------------------------------------------------------
# LDAP

sonar.security.realm=LDAP
sonar.authenticator.downcase=true
ldap.url=ldap://ldap.mydomain.com
ldap.bindDn=myServiceAccount@mydomain.com
ldap.bindPassword=myPwd

ldap.user.baseDn=dc=my,dc=domain,dc=com
ldap.user.request=(&(objectClass=user)(sAMAccountName={login}))

ldap.group.baseDn=ou=Groups,dc=my,dc=domain,dc=com
ldap.group.request=(&(objectClass=group)(member={dn}))
ldap.group.idAttribute=sAMAccountName

Let’s check the different settings and try to explain them.

Setting Description
ldap.url URI that should point towards your domain server.
ldap.bindDn A valid domain user account that will be used to query the LDAP.
ldap.bindPassword Password of the above mentioned account
ldap.user.baseDn Distinguished Name of the root node in AD from which to search for users.
ldap.user.request LDAP user search pattern. It should stay as is for AD.
ldap.group.baseDn Root node in AD from which to search for groups
ldap.group.request Same as for the user request but for group search.
ldap.group.idAttribute Property used to specifiy the attribute to be used for returning the list of user groups in the compatibility mode.

If you now have set the necessary for your domain, you should be able to log in SonarQube. Be aware however that your user id is going to be changed same as for your groups. Probably you will need to use the local admin account to setup the new group names and assign the correct Global Permissions. Check my previous blog post on how to achieve that here, SonarQube on Windows and MS SQL.
This also means that if you have issues assigned to users and other user related information, they would be lost. I tried changing some of these values manually in database, however I was not successful. As soon as I do get more information about migrating user profiles, I will create a separate blog post on that.

After you set up everything as I suggested, you still may encounter some issues. If you can’t log in check your log file and see if you have any similar lines in it:
You can't sign up because email 'Mario.Majcica@mydomain.com' is already used by an existing user.
If this is the case you need to deactivate the previous user from Administration > Security > Users panel (log in as admin) or modify a relevant records in you DB in table users.

Update

Meanwhile writing this blog post, a newer version of SonarQube and LDAP plugin became available (SonarQube 6.1 and LDAP plugin 2.1). There are no braking changes and all of the above should work the same.

SonarQube on Windows and MS SQL

Introduction

In the following post we will see what is necessary to install and configure SonarQube 5.4. We will also see how to setup some basic security concerns by making our SonarQube part of our LDAP infrastructure and map security groups to roles.
I’m sure that there are plenty of guides out there, but what I found most annoying meanwhile reading some of them, is that all of them do give several things for granted. Also the information is segmented and not easy to find. I will try in this post to cover even the basic steps that can save you hours of struggling. I’m going to install SonarQube on Windows platform using MS SQL as my database of choice, you can also try Couchbase. Both of these services in my case are going to reside on the same machine, but nothing limits you to use multiple machines for your setup.

Prerequisites

Java runtime is the main prerequisite. Although it works with Java 7, my advice is to install and use JDK 8. At the moment of writing the latest version for my platform is jdk-8u77-windows-x64.exe.
For what concerns MS SQL versions, 2008, 2012 and 2014 are supported. Also the SQL Express is supported. Your SQL server needs to support case-sensitive (CS) and accent-sensitive (AS) collation.

Installing the database

After you installed your MS SQL version of choice, you need to create a database. Add a new database and name it SonarQube.

new-database

Now the important step. In the Options page you need to specify the right collation. It needs to be one of the case-sensitive (CS) and accent-sensitive (AS) collations. In my case I will go for SQL_Latin1_General_CP1_CS_AS.

new-database-options

Once that is set, click OK and create the new database.

After the database is created, we need to make sure that the TCP/IP protocol is enabled for our SQL instance. Open the Sql Server Configuration Manager and, in the console pane, expand SQL Server Network Configuration. Choose the Protocols for your instance. In the details pane, right-click TCP/IP, and then click Enable. Once done, restart the service. A detailed guide is available on Technet at Enable TCP/IP Network Protocol for SQL Server.

conf-manager

Last but not least, make sure that SQL Server Browser service is running. Often it is disabled by default, however for the JDBC driver to work, it needs to be enabled and running. Open the Services management console and find the Service called SQL Server Browser. If disabled, enable it and start the service.

services

That’s all for now for what database concerns.

Installing SonarQube

Before we start, make sure that the latest JDK is installed, then download the SonarQube installation file from SonarQube website. For this demo I will be using the latest available version of SonarQube at the moment of writing and that is 5.4. After I downloaded sonarqube-5.4.zip I will extract it’s content in a folder of my choice and that is D:\SonarQube.

There is another important file we need to get and set before we can continue configuring SonarQube and that is Microsoft JDBC driver. Go to Download the Microsoft JDBC Driver 6.0 (Preview), 4.2, 4.1, or 4.0 for SQL Server and download sqljdbc_4.2.6420.100_enu.tar.gz file. Once done, open the just downloaded file with compression tool of your choice and extract all of it’s content in a temporary folder. Get into sqljdbc_4.2\enu\auth\x64 folder and copy the only file present in that path, sqljdbc_auth.dll and paste it into your System32 directory, usually C:\Windows\System32.

Now we are ready to start the configuration. Open the main configuration file of SonarQube called sonar.properties. You can find it in the conf folder in your SonarQube installation path. Open it with the editor of your choice and search for the line reporting ‘Microsoft SQLServer 2008/2012/2014 and SQL Azure’. Under that line you should see a the following configuration item that is commented out:

#sonar.jdbc.url=jdbc:sqlserver://localhost;databaseName=sonar;integratedSecurity=true

We need to uncomment this line by removing the hash sign in front of it and change the connection string to point towards our SQL database instance (the one we create earlier).
Following, an example of the connection string using a name instance of SQL:

sonar.jdbc.url=jdbc:sqlserver://localhost;databaseName=SonarQube;instanceName=DEV_01;integratedSecurity=true

If you are using the default instance, you can simply omit the instanceName=DEV_01 from your connection string.

Also you can see I’ve set to use the integrated security. If you want to use SQL Authentication, remove the integratedSecurity=true part and specify the credentials as separate configuration items under your connection string (also create users in SQL accordingly and map the newly create user to dbo schema).

sonar.jdbc.url=jdbc:sqlserver://localhost;databaseName=SonarQube
sonar.jdbc.username=sonarqube
sonar.jdbc.password=mypassword

Once the connection string is set, save the configuration file and try starting SonarQube. Open the command prompt and move to ...\bin\windows-x86-64 folder and execute StartSonar.bat

start-sonar

If everything is set right, you should see a message in the console INFO app[o.s.p.m.Monitor] Process[web] is up.

first-run

Now you can open the web browser of your choice and head to http://localhost:9000. A welcome page on SonarQube should be shown.

first-run-browser

If the page loaded, congratulations, SonarQube is running correctly on your machine.

What is left to do is to create a service that will run SonarQube. Stop the current execution with a CTRL+C and terminate the batch job. In the same bin folder where StartSonar.bat is located, you will find InstallNTService.bat. Execute the just mentioned batch file and you should receive the wrapper | SonarQube installed. message. This means that a new service is created. Check your services management console and you should find a service called SonarQube:

services-sonar

As you can see from the picture, service is created but not started.
By default, the “Local System” account is used to execute the SonarQube service. If this account doesn’t have the required permission to create some directories/files in the SonarQube installation directory (which is the case by default on recent Windows versions), the execution of the SonarQube service will fail. In such case, the SonarQube service must be configured to run in the context of a suitable account.
Right click on the SonarQube service and choose properties then move to Log On tab choose “This account”, and select an account that can read/write the folder in which SonarQube is installed. Hopefully you will have a specific service account created for this purpose.

service-logon

Now, you can start the service manually or by launching StartNTService.bat.

Services configuration

SonarQube is the only web application running on my server, so I will move it from the port 9000 to the default 80. To do so, edit the sonar.properties configuration file and find the #sonar.web.port=9000 comment line. Uncomment it and change port value to 80, sonar.web.port=80.

After this change you need to restart your SonarQube service and try to reach your localhost in the browser. If all went fine you will not need to specify the port at the end of the address.

SonarQube behind a proxy

I wrote in the past time numerous post about running services and applications behind a proxy. SonarQube will not be an exception to that practice. You may wonder why SonarQube should have access to internet and my answer is, plug-ins. Plug-ins are essential to SonarQube and installing and updating them is easiest done via Update Center, a functionality integrated in the administrative portal. In order for it to work, SonarQube needs to be able to access the internet. In case you are behind a proxy, you need to modify again sonar.properties configuration file.

Search for #sonar.web.javaAdditionalOpts= configuration line and modify it by specifying http, https proxy host and port:

sonar.web.javaAdditionalOpts=-Dhttp.proxyHost=swg.myProxy.com -Dhttp.proxyPort=8080 -Dhttps.proxyHost=swg.myProxy.com -Dhttps.proxyPort=8080

Restart the service and try the Update Center. Open SonarQube web page and log in with the default admin user (password is also admin). Click on the administration menu item and then, in the sub-menu, choose System -> Update Center. Check if the updates are retrieved and try to update one of the plug-ins installed by default, like C#. If all goes well you will see the following screen

update-center

Once the plug-in is installed you will see a button in the notification message that offers to restart the server for you. In my case it never worked and after choosing this option my server stopped replaying. In order to get it back online, you need to manually restart the service.

This problem is addressed in SONAR-7422 and it is a recognized bug. It should be fixed in the SonarQube version 5.6.

If you where able to install or update plug-in correctly, then your proxy settings where picked up fine.

Securing SonarQube connection

You can setup SonarQube to run on a https secure connection. It natively supports the SSL certificates however it is not advised to configure it. Using a reverse proxy infrastructure is the recommended way to set up your SonarQube installation on production environments which need to be highly secured. This allows to fully master all the security parameters that you want. I will not dig into details on how to set up IIS to leverage the reverse proxy setup. If interested in this, you can read the following blog post on Jesse Houwing’s blog, Configure SSL for SonarQube on Windows. It will guide you in setting up IIS that will act as a proxy for the secure calls towards your SonarQube server.

Security configuration

My desire is to integrate the authentication of the SonarQube server with my LDAP (Active Directory domain services). In order to do that, we need to install LDAP plug-in. Locate the LDAP plug-in in update center under available plug-ins and install it.

plugin-ldap

Before you restart your SonarQube service, open the sonar.properties configuration file and add the following section:

#----------------------------------------------------------------------
# LDAP

sonar.security.realm=LDAP
sonar.forceAuthentication=true
sonar.authenticator.downcase=true

This are the only necessary settings if you are part of the Active Directory domain. Restart the SonarQube service and open the portal. If all went well, SSO kicked in, and you should be logged in with your domain account. Now comes the fun part. Log out, then, log in again as administrator and go to Administration -> Security -> Users screen. You should see in the list the domain account you logged in with. Update groups for this account and assign it to sonar-administrators group.

user-to-admin

Now close the browser and reopen it. Surprise, surprise, you are logged in again via your user profile but you do not see Administration option in your menu, as you would expect. Once the LDAP is configured, on each login, the membership information will be retrieved and local settings will be overwritten. Thus no group membership we assigned will be persisted. In this case, LDAP/AD becomes the one and only place to manage group membership. In order to do so, we need to create a security group in AD and map it in the SonarQube Security Groups.

Before we create a new group in the SonarQube Security Groups we need to get the groups precise name. Group names are case sensitive and do require the domain to be specified. This is not something we can guess but we can extract it from our log file.

Add your user to the AD security group of choice. Edit sonar.properties configuration file again and set the logging level to a higher setting. In order to do so, find the #sonar.log.level=INFO line, uncomment it and change the level from INFO to DEBUG. You line should now look like sonar.log.level=DEBUG. Restart the service and open the portal.

If you are successfully logged in, open the log file. In the SonarQube directory there is a folder called logs, in my case it is, sonarqube-5.4\logs. Inside you will find a file called sonar.log. Open it with your editor of choice and search for your domain username. Next to your username (probably at the bottom of the log file) you will find a couple of log lines made by web[o.s.p.l.w.WindowsAuthenticationHelper] and in one of those lines you will find written Groups for the user YOURDOMAIN\YOU and a list of security groups you are part of. Find the correct one and copy it, in my case this is sonar@maiolocal. Now log in as admin and open the Groups screen. Create new group by clicking to the Create Group button in top right corner and set the name to your group of choice, in my case sonar@maiolocal.

create-group

Once the group is created, move to Global Permissions screen (always in the Security menu), and assign the desired permissions to just created group. Let’s suppose that this group will list all of the administrators, under Administer System permission, click on groups and select the newly created group.

Now if you close your browser and reopen it pointing to your SonarQube portal, you will get logged in via SSO and you should be able to see the Administration button in the menu. Same can be done for the users.

Conclusion

This is roughly it. There are some details you would probably like to set as SMTP/Email settings and Source Control Manager settings, however all of this is quite trivial as you can find all of the necessary settings in the UI under General Settings. For more details check Notifications – Administration page in SonarQube documentation site, as SCM support page.

Your SonarQube server should now be correctly installed and configured to access LDAP. Ahhh, I almost forgot it, get the logging level back to INFO, otherwise you are risking quite a large log files on your disk.

UPDATE

After I published my post I realized that there is a better way of forcing the authentication. As Nicolas Bontoux pointed out this setting should be set in Administration – General Settings – Security pane. Otherwise you do risk encountering a problem during the upgrade of your SonarQube instance to a newer version.

Set the Force user authentication in previously mentioned pane.

security_force

Once done, comment or remove the sonar.forceAuthentication=true line from the sonar.properties configuration file and restart your service.
In this way you will not be bothered during your SonarQube updates.

UPDATE 2

In regard to the proxy settings, since SonarQube 5.5 specifying your proxy address via javaAdditionalOpts is not necessary anymore (and it is not advisable). Now it is sufficient to search in the sonar.properties configuration file the following line #http.proxyHost= and set the proxy parameters as shown here:

http.proxyHost=swg.myproxy.com
http.proxyPort=8080
https.proxyHost=swg.myproxy.com
https.proxyPort=8080

Save your settings and restart the service. Your Update Center should still be working correctly.

Securing Nexus Repository Manager OSS

Recently one of my articles got published on SonaType web site. It is based on my blog post Nexus Repository Manager OSS as Nuget server, which, after reviewing it for SonaType blog, somehow didn’t seemed complete. Thus, I decided to add an extra chapter about the security which I would like to share with you, also on my personal blog, here.

Security

I suppose that your computers are part of an Active Directory domain. I will propose here a setup to make your new Nexus instance dialogue with your domain controller when it comes to authenticating users. Why? Well, uploading packages to the hosted repository requires an API-key and it is assigned on per user basis. If you wish that your users (services) to be distinguished and limit some of them, you would need to setup the authentication and roles correctly. You could do this by simply using Nexus internal database, however, as we all know, adding users manually and creating another identity, from convenience and maintenance prospective is not recommendable. Not to speak about the security implications.

Setting up LDAP connection

In the left-hand main menu panel expand the security group and select the LDAP Configuration option. A new tab will open and you will be presented with a long list of parameters:

ldap-configuration

We are going to analyze them one by one and explain the values you do need to enter in order to make this connection work.

Protocol
Here you have two choices Lightweight Directory Access Protocol and the Lightweight Directory Access Protocol over SSL. Probably you are going to use just a simple LDAP protocol and not the

Hostname
The hostname or IP address of the LDAP server. Simply add the FQDN towards your domain controller.

Port
The port on which the LDAP server is listening. Port 389 is the default port for the ldap protocol, and port 636 is the default port for the ldaps.
If you have a multi domain, distributed Active Directory forest, you should connect to the Active Directory through port 3268. Connecting directly to port 389 might lead to errors. Port 3268 exposes Global Catalog Server that exposes the distributed data. The SSL equivalent connection port is 3269.

Search Base
The search base is the Distinguished Name (DN) to be appended to the LDAP query. The search base usually corresponds to the domain name of an organization. For example, the search base on my test domain server could be dc=maio,dc=local.

Authentication Method
Simple authentication is what we are going to use in first instance for our Active Directory authentication. It is not recommended for production deployments not using the secure ldaps protocol as it sends a clear-text password over the network. You can change

Username
For the LDAP service to establish the communication with the AD server, a valid account is necessary. Usually you will create a service account in your AD for this purpose only.

Password
Password for the above mentioned account.

This is the first part of configuration. Now we need to check if what we entered is valid and if the connection can be established.
Click on Check Authentication button and if the configuration is valid, you should get the following message:

authentication-test

Base DN
You should indicate in which organizational unit Nexus should search for user accounts. By default it can equal to ‘cn=users’ however if you store your user accounts in a different OU, please specify the full path. An example, if you created an OU at the root level called MyBusiness then an OU called Users, you will need to specify the following value as base DN, ‘OU=Users,OU=MyBusiness’.

User Subtree
This needs to be check only if you are having other OU’s, under the one specified in Base DN, in which users are organized. If that is not the case, you can leave it unchecked.

Object Class
For AD it needs to be set yo ‘user’.

User ID Attribute
In the MS AD implementation this value equals to sAMAccountName

Real Name Attribute
You can set this to ‘givenName’ attribute, or simply to cn if givenName is not used during the user creation.

E-Mail Attribute
This attribute should map to ‘mail’

Group Type
Group Type needs to be set to Dynamic Groups.

Member Of Attribute
In case of MS AD this equals to ‘memberOf’.

Now that we have set all of the relevant values, we can ask Nexus OSS to give us a preview of retrieved accounts and mappings to make sure everything is set up correctly. Click on the Check User Mapping button and you should see a preview of the retrieved values and how they are mapped into Nexus OSS. If everything looks fine , save your configuration.

Once the configuration is persisted, the last thing to do is to let Nexus know that you would like to use this new setup for the authentication. This is done by enabling the LDAP Authentication Realm.

In the left-hand main menu panel, from the administration group, choose Server. In the nexus tab that opens, go to the security settings

selected-realms

You will find the OSS LDAP Authentication Realm in the group shown on the right side, called Available Realms. Move it to Selected realms (as shown in the picture above) and save this settings.

Configuring roles

Now that we are able to authenticate through LDAP (AD) we need to set who can do what. This is when the roles do get handy. I usually create security groups and then associate them to roles as this moves the operations of granting the access rights to merely assigning a user to a group.
You can still associate roles to the user in more or less the same way, however I’m going to show you my favorite approach.

From the left-hand main menu panel, in the Security group, select Roles. A new tab showing a list of roles will appear.

external-role-mapping

Choose Add and then External Role Mapping option. You will then be presented with the following dialog:

map-external-role

As you can see from this screenshot, choose LDAP as Realm and the role (an AD security group to be precise) you wish to map. Be aware that in some cases, not all of the groups will be listed. It is a well know bug. If this is your case, there is a workaround. Add a Nexus Role instead of the External Role Mapping and name the role precisely as your missing group is named. This should do the trick until this issue gets solved.

Once the mapping is added, you will need to assign roles (actual rights) to selected security group.

external-role-mapping

On the Role/Privilege Management bar, choose Add button and select the following roles:

  • Nexus API-Key Access
  • Nexus Deployment Role
  • Repo: All NuGet Repositories (Full Control)
  • Artifact Upload

These set of rights will enable whoever part of that group to login in Nexus OSS, manually upload packages, get the API-Key, and generally to control all NuGet repositories. In other words, a sort of developers role when it comes to NuGet.

When it comes to administrator role, it is sufficient to assign only the Nexus Administrator Role.
At the end of selecting roles, do not forget to click on Save button and persist this configuration.

This two basic profiles should be sufficient to start using Nexus. If further profiles are necessary, check the managing roles guide.

You can now test all of our security setup. Make sure you AD user account is part of the administrative security group that you just mapped to Nexus Administrator Role. Log out and log back in with your domain account. If you succeed to login, congratulations LDAP is set correctly.