Semmle 1.22
Skip to end of metadata
Go to start of metadata

 This topic describes how to define the build commands for a project.

Terminology note

In the context of Semmle analysis, a "project" comprises all of the files in a single programming language stored within a single source code repository. For example, all Java files within a single repository are treated as one project. Python files within the same repository are treated as a separate project, even though both types of files may be required to build a single application.

Task overview

Each project configuration file must define how to analyze the code to generate a snapshot database. For projects written in compiled languages this includes the commands required to build the source code. The build method is defined by adding one or more build elements to the autoupdate element of the project file. When you create a project file using the bootstrap command, you are prompted to define one build command. For example, for a simple Java code base built using Maven, this single command may be adequate to define how to build the source files. However, many build processes and developer environments require you to define more complex build commands. For example:

  • The build process may have environment configuration requirements.
  • The process may be substantially customized.
  • The process may require the source files to be stored in a specific location

You can edit your project file to define multiple build elements, each containing a separate command, in order to build the project correctly. By default, the build commands are run in order from the ${src} location. If any of the commands need to be run from a specific location then you can define an alternative location using the dir attribute. 

See project file reference page for full details of the attributes and elements available for use in the project configuration file.

HIDDEN

HIDDEN

Editing the build commands

You can edit the project file using your preferred text editor.

To edit the build commands
  1. Open the project file for editing.
    This file is stored at a location such as: SEMMLE_HOME/projects/<project-name>/project. The contents of your project file depends on the options you have selected during the bootstrap process. For example, a project file for a Java code base will contain three build elements by default, of which the second element is the build process you specified while running bootstrap:

    project file
    <project language="java">
    ...
       <autoupdate>
          <checkout>git clone -n ${repository} ${src}</checkout>
          <checkout>git checkout ${revision}</checkout>
          <build index="true">mvn compile</build>
          <build>odasa duplicateCode --ram 2048 --minimum-tokens 100</build>
       </autoupdate>
    ...
    <project>

    The index = "true" attribute in the second build element is used to define the build step that you want to trace – that is, to monitor and and extract data from. This means that, in the above example, the Maven build process will be traced when you generate a snapshot for analysis. When you edit the build commands, you must add the index = "true" attribute to any build elements that you want to trace.

  2. In this case, you can change your build process by editing the second build element in the file, or you can add further build steps by inserting additional elements on the following lines. Optionally, if the source files require a single fixed destination, you can also define a detached directory with the source-location element:

    project file
    <project language="java">
    ...
      <autoupdate>
         <checkout>git clone -n ${repository} ${src}</checkout>
         <checkout>git checkout ${revision}</checkout>
         <build dir="relative/path">BUILD COMMAND 1</build>
         <build>BUILD COMMAND 2</build>
         <build index="true">BUILD COMMAND 3</build>... 
         <build>odasa duplicateCode --ram 2048 --minimum-tokens 100</build>
         <source-location>/absolute/path/source_directory</source-location>
      </autoupdate>
    ...
    <project>
  3. Edit the existing build commands as necessary. For further information see the examples below, which illustrate different build scenarios for several compiled languages.

  4. Save the project file.

  • BUILD COMMAND 1 is run in the directory defined by the dir attribute, that is, ${src}/relative/path.

  • BUILD COMMANDS 2 and 3 are run in the directory defined by the ${src} variable.

${src} is set to SEMMLE_DATA/projects/Project_Name/Snapshot_Name/src unless you have used the source-location element to define an alternative location.

Language-specific extractor tools are included in the Semmle Core installation. For further information on using these tools, see Customizing calls to extractors.

Examples

The following examples are designed to give you an idea of the flexibility of build commands for compiled languages. 

C or C++ project built using Visual Studio 12's msbuild, from a fixed source location

project file - build and source-location elements only
<build index="true">"C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\vcvarsall.bat" &amp;&amp; msbuild /t:rebuild "My Project.sln"</build>
<build>odasa duplicateCode --ram 2048 --minimum-tokens 100</build>
<source-location>C:\Documents and Settings\ExampleUser\My Documents\Visual Studio 2012\Projects\My Project</source-location>

where "C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\vcvarsall.bat" && msbuild /t:rebuild "My Project.sln" is the normal build command for the source code.

Notice the use of the source-location element to define the location of the source code: the solution file named in the first build element should be in this directory, and the associated source code should either be in this directory or a sub-directory of this directory.

C or C++ project built on Linux using autoconf and make

project file - build elements only
<build>autoconf</build>
<build>./configure</build>
<build>make clean</build>
<build index="true">make</build>
<build>odasa duplicateCode --ram 2048 --minimum-tokens 100</build>

C# project built using msbuild with additional indexing of XML files

project file - build elements only
<build index="true">"C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\vcvarsall.bat" ^&amp;^&amp; msbuild /t:rebuild "path\to\project.sln"</build>
<build>odasa index --xml</build>
<build>odasa duplicateCode --ram 2048 --minimum-tokens 100</build>

where "C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\vcvarsall.bat" && msbuild /t:rebuild "path\project.sln" is the normal build command for the source code.

Java project built using Eclipse

project file - build elements only
<build>odasa index --eclipse ${src}</build>
<build>odasa duplicateCode --ram 2048 --minimum-tokens 100</build>

Java project built using Gradle

project file - build elements only
<build index="true">gradle --no-daemon clean testClasses</build>
<build>odasa duplicateCode --ram 2048 --minimum-tokens 100</build>

Notice the use of the --no-daemon flag to ensure that the code is built in the same thread. This enables Semmle Core to monitor the build.

Java project built using Maven

project file - build elements only
<build index="true">mvn clean install</build>
<build>odasa duplicateCode --ram 2048 --minimum-tokens 100</build>

Project built using a custom build script

project file - build elements only
<build index="true">${project}/scripts/build.sh</build>
<build>odasa duplicateCode --ram 2048 --minimum-tokens 100</build>

Note that this runs a custom build script stored in the scripts subdirectory of the project directory. The build.sh script contains all the commands required to build the project.

build.sh
#!/usr/bin/sh
 
# Your commands here

Java project built using ant in a subdirectory called build

project file - build elements only
<build dir="${src}/build">ant -f build.xml clean</build>
<build dir="${src}/build" index="true">ant -f build.xml</build>
<build>odasa duplicateCode --ram 2048 --minimum-tokens 100</build>

Note that ant is invoked twice: once to clean the project, and then once to actually build the project.