Semmle 1.19
Skip to end of metadata
Go to start of metadata

During the Semmle analysis process the code being analyzed is checked out from the source code repository. This advanced topic describes how to define the checkout commands for a project manually after you have created a project file using either bootstrap or createProject.

Terminology note

In the context of Semmle analysis, a "project" comprises all of the files in a single programming language stored within a single source code repository. For example, all Java files within a single repository are treated as one project. Python files within the same repository are treated as a separate project, even though both types of files may be required to build a single application.


-

Task overview

The configuration file for a specific project is a file called project in the SEMMLE_HOME/projects/<project-name> directory. Amongst other things, the project file defines how to access the source code files for the project. This "checkout" process is defined using one or more checkout elements within the autoupdate element.

If you used bootstrap, the project file that is created will already contain a checkout element. For a simple code base stored in Git, Subversion, or Mercurial, the checkout element created by bootstrap will usually define how to check out all the source files for the project adequately. However, some source code management systems, build processes and developer environments require you to define more complex checkout commands. For example, the source code for a project might be:

  • Stored in more than one source code management system.
  • Stored as multiple applications within a source code management system.
  • Built using a process that requires the source files to be stored in a specific location.

You can define multiple checkout elements, each containing a separate command, to define how to access all the files required by the project. By default, the checkout commands are run in order from the ${src} location (for details of the ${src} variable, see Variables). If any of the commands need to be run from a specific location, you can define an alternative location using the dir attribute of the checkout element.

-

See project file for full details of the attributes and elements available for use in the project configuration file. 

-

Editing the checkout commands

Edit the project file using your preferred text editor.

To edit the checkout commands
  1. Open the project file for editing.
    This file is stored at a location such as: SEMMLE_HOME/projects/<project-name>/project.
    If the file has been generated by bootstrap, it will include the default elements for your defined source repository. For example, if you are using Git: 

    Example of a project file for a Java project in Git
    <project language="java">
       ...
       <autoupdate>
          <checkout>git clone -n ${repository} ${src}</checkout>
          <checkout>git checkout ${revision}</checkout>
          <build>...
       </autoupdate>
       ...
    </project> 
  2. Add the required checkout commands, each in an individual checkout element, before the first build element within the autoupdate element:

    <autoupdate>
       <checkout>COMMAND 1</checkout>
       <checkout>COMMAND 2</checkout>
       <checkout dir="relative/path">COMMAND 3</checkout>
       <build>... 
  3. Optionally, if you require a single fixed destination for the source files, you can define a detached directory with the source-location element. By convention, the source-location element appears after the final checkout element, and it applies to all checkout commands:

    <autoupdate>
       <checkout>COMMAND 1</checkout>
       <checkout>COMMAND 2</checkout>
       <checkout dir="relative/path">COMMAND 3</checkout>
       <source-location>/absolute/path/source_directory</source-location>
       <build>...  
  4. Save the project file.

When you create a new snapshot using the addSnapshot or addLatestSnapshot command, the checkout commands are run in the order defined in the project file. Commands are run in the directory defined by the ${src} variable which, by default, expands to SEMMLE_DATA/projects/<project-name>/<snapshot-name>/src. However, if you define a source-location element, ${src} expands to the value of this element instead. Therefore, in the example above:

  • COMMAND 1 and COMMAND 2 are both run from the path /absolute/path/source_directory, as defined by source-location.
  • COMMAND 3 is run from the ${src}/relative/path directory, where ${src} expands to the source-location.

If the source code repository is password protected then you can use a credentials store to keep the username and password for the account used to access the repository secure. See Creating and using a credentials store  for details.

Examples

The following examples are designed to give you an idea of flexibility of checkout commands.

  • Subversion checkout command run from a specific directory:
    <checkout dir="/my-directory">svn co myproject ${src}</checkout>
  • Clone two Git workspaces—where code from two different repositories is required in one project:
    <checkout>git clone git://repo1 ${src}/product1</checkout>
    <checkout>git clone git://repo2 ${src}/product2</checkout>
  • Custom checkout script—for a custom SCM or a system that requires multiple steps (for example, login followed by checkout):
    <checkout>${project}/scripts/checkout.sh ${src}</checkout>

For more examples of checkout commands in the project file, see Project file checkout examples.