Sharing Our Passion for Technology

& Continuous Learning

Code Quality Metrics with Sonar, Part III: Sonar in a Ant-based Java Project

By Akrem Saed | June 01, 2013 | Development

Now we will cover the fun stuff for which we've been waiting. In this post, I'll go over how to setup Sonar for a Java project that utilizes Ant for its build. I'll go through the basic steps for installing and running a Sonar instance, and how to use a MySQL database for collecting metrics. Then I'll go into some details around analyzing a Java project using Ant and Sonar. This involves writing Ant script, pointing to the source codes, analyzing the binaries, analyzing JUnit test cases, analyzing Ecl Emma coverage, etc.

What's my Environment ?

One of the challenges that I faced, was figuring out which tool versions were compatible. Matching Ecl Emma, Ant and Sonar Ant Task was the trickiest part. To insure we are all on the same page here is the setup of my environment:

Windows OS (XP/Vista). All other Windows version should work fine.
Java 1.6
Apache Ant 1.7.0
MySQL 5.5.19
Sonar 2.12
Sonar Ant Task 1.2
JUnit 4
Ant JUnit Library 1.7.0 (part of Apache Ant)
Ecl Emma 2.1.5320
Apache Ivy 2.2.0

Installing and Running Sonar

Installing and running Sonar is fairly easy because Sonar comes packaged with a Derby database engine and a JEE wrapper. Hence, you can run it right out of the box. As the Sonar documentation states there are three components for Sonar: a database, a web server and a client.

To install and configure the web server part :

Download the zip file for Sonar 2.12 from downloads page.
After downloading it, unzip to "F:\MyServers\sonar-2.12" so that the following path is valid "F:\MyServers\sonar-2.12\bin" .
Make sure that Sonar configurations found in "F:\MyServers\sonar-2.12\conf\sonar.properties" have the following lines uncommented:

sonar.web.host: localhost
sonar.web.port: 9000
sonar.web.context: /sonar

sonar.jdbc.username: sonar
sonar.jdbc.password: sonar

sonar.jdbc.url:  jdbc:derby://localhost:1527/sonar;create=true
sonar.jdbc.driverClassName: org.apache.derby.jdbc.ClientDriver

I personally prefer to run Sonar as a Service on my Windows machine. You can manage Sonar as a Windows Service using the following commands:
Install the NT service for 32-bit machine (might require administrator credentials): "F:\;MyServers\sonar-2.12\bin\windows-x86-32\InstallNTService.bat"
Uninstall the NT service for 32-bit machine : "F:\MyServers\sonar-2.12\bin\windows-x86-32\UninstallNTService.bat"
Start the NT service: "F:\MyServers\sonar-2.12\bin\windows-x86-32\StartNTService.bat"
Stop the NT service: "F:\MyServers\sonar-2.12\bin\windows-x86-32\StopNTService.bat"
Please note: there is a 64-bit version under "F:\MyServers\sonar-2.12\bin\windows-x86-64\"
Install and Start Sonar as a service using the appropriate command from the list above (i.e. Install and Start commands).
After running the Sonar instance using the commands above, use your browser to navigate to http://localhost:9000/sonar and you should be able to see the sonar home page. When you see the page, it means you have successfully installed and ran Sonar.
Please note: You can log into the web page as an administrator using user name "admin" and password "admin"

Switching to MySQL

Now let's configure the database component of Sonar.

Sonar documentation recommends using Derby in-memory database just for testing purposes and I agree that you need to switch to MySQL or any other database engine at this step. Sonar not only stores the metrics in the database, but it also stores web server configuration like user IDs, passwords, customization to the UI, etc. So to avoid losing your customization, you should switch to MySQL before you customize and analyze projects.

Switching to MySQL is simple by following these steps :

Install MySQL 5.5.19.
If you like using a GUI for managing the MySQL server, then try installing MySQL Workbench
Create "sonar" schema in your MySQL database by executing the script found in "F:\MyServers\sonar-2.12\extras\database\mysql\create_database.sql". The script will create the schema and an account called "sonar"' with a password "sonar". You will notice that the schema is empty because it will be built and populated the next time Sonar is started and pointed to the MySQL database.
If Sonar is running, then Shut it down by executing the command "F:\MyServers\sonar-2.12\bin\windows-x86-32\StopNTService.bat".
Verify that Sonar has been stopped by navigating to http://localhost:9000/sonar . You should see an error page saying the browser cannot find the web page.
Change the Sonar configurations in "F:\MyServers\sonar-2.12\conf\sonar.properties" by commenting the following lines :

#sonar.jdbc.url:                            jdbc:derby://localhost:1527/sonar;create=true
#sonar.jdbc.driverClassName:                org.apache.derby.jdbc.ClientDriver

Change the Sonar config in "F:\MyServers\sonar-2.12\conf\sonar.properties" by setting the following properties as follow (assuming that your MySQL database is accessible through localhost:3306 ). Everything else should stay the same :

sonar.jdbc.validationQuery:                select 1
sonar.jdbc.url:                            jdbc:mysql://localhost:3306/sonar?useUnicode=true&amp;characterEncoding=utf8
sonar.jdbc.driverClassName:                com.mysql.jdbc.Driver

Start Sonar now by executing "F:\MyServers\sonar-2.12\bin\windows-x86-32\StartNTService.bat" and you will see that Sonar will populate the MySQL schema with tables and data.
Navigate to http://localhost:9000/sonar using your browser to see that Sonar has started successfully.
You can login as an adminsitrator using user name "admin" and password "admin"

Analyzing a Project with Ant

Now comes the client component of Sonar.

Assuming that you are familiar with Ant and how its build process works, we will jump into the specifics on how Ant works with Sonar without going into details of how Ant itself works for the sake of brevity.

General Structure

Let assume you have an Ant-based Java project called SonarBlogBeanoh in your workspace, i.e. in "F:\workspace\SonarBlogBeanoh" and you have all your Ant-based build scripts in "F:\workspace\SonarBlogAntBasedBuildTools\" .

Please note: you will find the links to the source code and build scripts at the end of this post.

Moreover, let assume that your build scripts in SonarBlogAntBasedBuildTools already has the following Ant targets:

clean: this Ant target cleans up the temporary folder on which we keep the build's temporary files.
compile: this Ant target compiles the main source code written in our java project.
compile-test: this Ant target compiles the JUnit test source codes written in our java project.
test: this Ant target runs the compiled JUnit tests with Emma coverage analysis

Let assume that SonarBlogAntBasedBuildTools has the following structure and files:

SonarBlogAntBasedBuildTools\apache-ant-1.7.0\ : a folder that contains the full Apache Ant setup.
SonarBlogAntBasedBuildTools\emma-2.1.5320\ : a folder that contains the Ecl Emma Ant task.
SonarBlogAntBasedBuildTools\build.xml : the main Ant build script that setup all properties, create temporary folders, compiles the source code and test code, etc.
SonarBlogAntBasedBuildTools\build-test.xml : the main build script that runs the JUnit test code with Emma coverage analysis.
SonarBlogAntBasedBuildTools\build-properties.xml : this script files loads some Ant properties.
SonarBlogAntBasedBuildTools\settings.properties : a properties file that contains folder locations.

Let assume that our SonarBlogBeanoh project has the following structure and files:

SonarBlogBeanoh\build.xml : the main Ant build script that we will invoke.
SonarBlogBeanoh\src\main\java\ : a folder that contains the non-test source code of our project.
SonarBlogBeanoh\src\main\resources\ : a folder that contains configuration files for our project.
SonarBlogBeanoh\src\test\java\ : a folder that contains test source code for our project.
SonarBlogBeanoh\src\test\resources\ : a folder that contains configuration files for the test code.
SonarBlogBeanoh\lib\ : a folder that contains all the Jars our project depends on.

Installing the Sonar Ant Task

First we need to download the Sonar Ant task 1.2 from Sonar repository. We only need the "sonar-ant-task-1.2.jar" file, and let's save it in "SonarBlogAntBasedBuildTools\sonar\".

Sonar Ant target

Let's create our Ant script for doing the sonar analysis by creating "build-sonar.xml" under "SonarBlogAntBasedBuildTools" folder. Initially we will just define a "sonar" target, import the sonar ant task, and add dependencies to relevant Ant targets. We want to run our sonar target after we have cleaned the temporary folder, compiled the source code, compiled the test code, and ran the tests. That is we will depend on : clean, compile, compile-test, and test targets. We will also import this new script into the main build.xml file.

<project name="SonarAntProject" default="sonar">

	<import file="build-properties.xml"/>

	<path id="sonar.classpath">
		<fileset dir="${buildtools.dir}/sonar" includes="**/*.jar" />
 	</path>

	<taskdef uri="antlib:org.sonar.ant" resource="org/sonar/ant/antlib.xml">
    		<classpath refid="sonar.classpath" />
  	</taskdef>

  	<target name="sonar" depends="clean, compile, compile-test, test">

 	</target>

</project>

Please note that properties whose name starts with "sonar." are official sonar properties that the Sonar Ant task expects to see. Any other properties whose names don't start with "sonar." are my own and you are free to rename them. Just make sure that your properties are clearly differentiated from sonar's.

Point to Sonar instance

The above build-sonar.xml script does nothing at this stage. Actually, all it does is to execute the ant targets it depends on. We've not configured it to do anything sonar-specific yet.

The first step of analyzing a project using the Sonar Ant task is to tell it where the Sonar instance is located. It's worth noting that the client (Sonar Ant task) component is the one that does the actual analysis of projects and store the metrics to the database. That is, the client does not delegate the analysis or storing metrics to the web server component. That's why it's necessary to point the client to the Sonar instance. Later, we will also need to tell the client in which database it needs to store the metrics.

We point to the sonar instance by adding the following lines to the build-sonar.xml file :

<project name="SonarAntProject" default="sonar">
	......

  	<target name="sonar" depends="clean, compile, compile-test, test">

		<property name="sonar.host.url" value="http://localhost:9000/sonar" />

 	</target>

</project>

Please note we are using localhost, but when you run Ant from a CI server running on a different machine, you need to provide the IP address or DNS name for your sonar instance and MySQL database.

Point to MySQL database

Now we need to tell the sonar target to point to the appropriate MySQL database by adding the following lines to build-sonar.xml file :

<project name="SonarAntProject" default="sonar">
	......

  	<target name="sonar" depends="clean, compile, compile-test, test">

		<property name="sonar.jdbc.url"
		     value="jdbc:mysql://localhost:3306/sonar?useUnicode=true&amp;amp;characterEncoding=utf8" />
		<property name="sonar.jdbc.driverClassName" value="com.mysql.jdbc.Driver" />
		<property name="sonar.jdbc.username" value="sonar" />
		<property name="sonar.jdbc.password" value="sonar" />

		<property name="sonar.host.url" value="http://localhost:9000/sonar" />			

 	</target>

</project>

Analyze Source Code

Now comes the fun part of actually analyzing the source code using Sonar. This is done by using the "sonar:sonar" task defined by the Sonar Ant task library we downloaded. So we further modify build-sonar.xml as follows:

<project name="SonarAntProject" default="sonar">
	......
  	<property name="build.version" value="0.0.0.1-Sonar"/>
  	<property name="mysonar.organizationName" value="com.asaed"/>
   	<property name="sonar.projectName" value="${project.name}" />

  	<target name="sonar" depends="clean, compile, compile-test, test">
		.......
		<property name="sonar.host.url" value="http://localhost:9000/sonar" />
	        <sonar:sonar key="${mysonar.organizationName}:${sonar.projectName}" version="${build.version}" xmlns:sonar="antlib:org.sonar.ant">

	     	    <sources>
	    		  <path location="${project.dir}/src/main/java"/>
	    	    	  <path location="${project.dir}/src/main/resources"/>
	    	    </sources>

	    	</sonar:sonar>

  	</target>

</project>

Let me go through each property of the previous snippet :

build.version : this property sets a version number for this analysis. Ideally, you want to increment this version number every time you run the sonar ant task which can be done easily when you integrate this Ant script with a CI server. But for now, make sure you change this property value everytime you run a sonar analysis from Ant so that you don't override the metrics calculated.
mysonar.organizationName : This property will be used in conjunction with "sonar.projectName" property to create a Sonar key for the project being analyzed.
sonar.projectName : This property takes the value of "project.name" which is the name of the project we are analyzing.
project.name : This property is set manually for each project being analyzed by defining it in each project's build.xml file.
project.dir : This property is set manually for each project being analyzed by defining it in each project's build.xml file.

Second, we are defining the "sources" tag and point it to the appropriate folder locations of our source code and the resource files.

If you run the "sonar" target at this time, sonar will analyze the project. When the analysis has completed, you will be able to see the result in the web server http://localhost:9000/sonar where SonarBlogBeanoh will be listed. If you click on the project, you will see the metrics that were derived from static analysis of source code. However, some metrics won't display any values yet. For example, Unit Tests, Code Coverage and Package Design sections are empty because they need the tests results and binary analysis. We will add those in the next steps.

Analyze JUnit tests

When it comes to adding JUnit tests to my sonar analysis, I prefer to reuse the JUnit reports that have been generated by the "test" ant target. This approach minimizes the amount of configuration I have to write and reduce the amount of duplication. When running JUnit tests using the JUnit Ant task, I can configure it to generate an XML report file. This is done by using inside the JUnit Ant task. I can find the details by checking the Ant documentation around JUnits tests.

From Sonar's point of view, here is what I need to add to my build-sonar.xml file :

<project name="SonarAntProject" default="sonar">
	......
  	<property name='sonar.surefire.reportsPath' value='${junit.report.xml}'/>

  	<target name="sonar" depends="clean, compile, compile-test, test">
		.......
	        <sonar:sonar key="${mysonar.organizationName}:${sonar.projectName}" version="${build.version}" xmlns:sonar="antlib:org.sonar.ant">

	     	    ......
	    	<tests>
	    		<path location="${project.dir}/src/test/java"/>
	    		<path location="${project.dir}/src/test/resources"/>
	    	</tests>

	    	</sonar:sonar>

  	</target>

</project>

Note that:

I'm setting the sonar property "sonar.surefire.reportsPath" to the value of "junit.report.xml". This way, Sonar is going to reuse the JUnit xml report generated by my "test" ant target.
"junit.report.xml" property is the folder location where I have saved our JUnit test report in XML format.

Analyze Emma coverage

If I run the "sonar" target at this point, I'll find that the Unit Test section in Sonar web server has been filled for SonarBlogBeanoh. However, Code Coverage is still empty. Let me solve this now.

I'll reuse the Emma report generated while running the JUnit tests. When running the JUnit tests as part of my "test" ant target, I have set it up to use Emma for coverage analysis by defining several JVM args (namely, -Demma.coverage.out.file, -Demma.coverage.out.merge, and -Demma.rt.control=false) which generates a coverage file in *.ec format. To get more details on how to accomplish this, please refer to the appropriate documentation of Emma and Ant.

I will further modify my build-sonar.xml file by adding the following lines :

<project name="SonarAntProject" default="sonar">
	......
  	<property name="sonar.core.codeCoveragePlugin" value="emma"/>
  	<property name='sonar.emma.reportPath' value='${emma.outdir}'/>

  	<target name="sonar" depends="clean, compile, compile-test, test">
		.......
	        <sonar:sonar key="${mysonar.organizationName}:${sonar.projectName}" version="${build.version}" xmlns:sonar="antlib:org.sonar.ant">

	     	    ......

	    	</sonar:sonar>

  	</target>

</project>

I'm instructing Sonar to use Emma as our coverage tool by setting "sonar.core.codeCoveragePlugin" property. Moreover, I'm letting Sonar know where to find the emma report by setting "sonar.emma.reportPath" property to the value of "emma.outdir" property. The "emma.outdir" property points to the folder location of the Emma coverage reports.

Before I run the "sonar" target, I need to install Emma plugin for in sonar's web server :

Go to http://localhost:9000/sonar
Log as "admin" with password "admin"
Click on "Configuration", then "Update Center"
Click on "Available Plugins" tab and find the Emma plugin.
Click on "Emma" and it will show an install button. Click on install.
Before restarting Sonar, go to "General Settings" -> "Code Coverage".
In the text box type "emma" and click "Save Code Coverage Setting" button.
Now to restart Sonar, just shut it down using the command mentioned earlier in this post.
Start sonar
Run "sonar" target to run the analysis for the project, and when it completes you should be able to see the Code Coverage section populated in the web server.

Please note, if you're using incompatible versions of Emma, Sonar Ant task, or Ant, you will receive errors that say something like "could not read emma coverage report" or "emma report is not formatted properly". If you do get similar errors, make sure you have the right versions, that emma is writing a coverage report to the folder you specified, and Sonar is trying to read this report from the correct folder location.

Analyze Byte Code

Some of Sonar's plugins perform byte code analysis to find issues or violations. One of those plugins is findbugs. For such plugins, we need to tell Sonar where to find the compiled source code and the libraries used. Hence, we modify build-sonar.xml by adding the following lines :

<project name="SonarAntProject" default="sonar">
	......

  	<target name="sonar" depends="clean, compile, compile-test, test">
		.......
	        <sonar:sonar key="${mysonar.organizationName}:${sonar.projectName}" version="${build.version}" xmlns:sonar="antlib:org.sonar.ant">

	     	    ......
	    	<binaries>
	    		<path location="${compile.outdir}"/>
	    	</binaries>
	    	<libraries>
	    		<path>
			    <fileset dir="${project.dir}/lib" includes="&#42;.jar" />
			</path>
	    	</libraries>

	    	</sonar:sonar>

  	</target>

</project>

In the previous snippet we are setting the "binaries" tag to point to the appropriate folder of our compiled source code generated by the "compile" ant target. Also, we are setting the "libraries" tag to point to the Jar libraries on which our code depends.

Now, if I go to the Quality Profiles section under Configuration in the web server and choose "Sonar way with findbugs" as the default profile and then run "sonar" target, then byte code analysis will be performed on my source code. You will notice that the number of violations will increase because I added Findbugs analysis that finds more issues. Moreover, you will see Chidamber and Kemerer and Package Design sections being populated which means they rely on byte code analysis.

To give you an idea of what to expect when viewing the metrics for SonarBlogBeanoh in the web server, below is a screenshot of it :

What's Next?

Now as we reach the end of this post, we have been able to successfully analyze a Java project using Ant and Sonar Ant task. However, before you roll out this new build and analysis process, you might want to address the following points :

Create different accounts on MySQL database because the MySQL database accounts and passwords used by the build script is going to be available to anyone who wants to run Sonar analysis. To avoid any mishaps, make sure you create a more restricted account that only inserts data to the MySQL database and use that account in the build script.
Create different accounts on Sonar to restrict access to Sonar's admin tasks. Sonar is flexible in that you can hook it up to LDAP or Active Directory to manage accounts and its privileges.
Run Sonar from CI Servers so you can take advantage of CI servers' scheduling, automatically increasing version numbers, setup dependencies, etc.
Hook up Eclipse with Sonar so that you can have an integrated user experience as I've covered in my previous post.

Limitations of Ant with Sonar

To be clear, if I had a choice of choosing my build tool to support Sonar analysis, I won't use Ant at all. The only reason I had to use it is because my partner's development shop was based on Ant. Ant is becoming very outdated when it comes to build processes and it's being surpassed by its counterparts like Gradle and Maven.

One main disadvantage of using Ant with Sonar is that there is no support for libraries and dependencies. One of Sonar's main features is managing libraries that I've talked about in my previous post. Unfortunately, this feature is not supported when you run Sonar using Ant because they depend on Maven dependency management.

Download the Examples

You can download the examples for this post from GitHub. The examples are divided into two GitHub projects:

SonarBlogBeanoh: this is the Ant-based SonarBlogBeanoh java project which you should checked out to "F:\workspace\SonarBlogBeanoh". The url for this project is https://github.com/asaed/SonarBlogBeanoh.git
SonarBlogAntBasedBuildTools: this is the Ant scripts that we used and should be checked out to "F:\workspace\SonarBlogAntBasedBuildTools\". The url for this project is https://github.com/asaed/SonarBlogAntBasedBuildTools.git

Credits

I want to thank David Kessler for allowing me to use Beanoh as a sample project for analyzing with Sonar. Please go and checkout Beanoh by visiting http://beanoh.org/overview.html

You can also check the first two parts of this series; Part I and Part II.