Semmle 1.22
Skip to end of metadata
Go to start of metadata

This topic describes how to generate a snapshot for a specific version of your code by checking out your source files and building a snapshot database.

Overview

You can check out your source files using two different commands, addLatestSnapshot and addSnapshot. The addLatestSnapshot command runs your checkout calls without requesting a specific version number of the code base, so can be used in scripts written to automatically check out the latest version of your code for analysis. The addSnapshot command allows you to check out specific revisions of your code. This enables you to compare the quality of your code between releases. Before you can check out the files for a specific version of the code base, you must know:

  • the version to check out. For example, the revision number (Subversion), or revision hash (Git).
  • whether or not the build method currently used is correct for the historic version of the code base.

After checking out the code, you must then build and index a snapshot database using the buildSnapshot command before you can generate any query results or export the snapshot for analysis in a QL plugin or extension.

Prerequisites

Before generating a snapshot you must create a project using either the bootstrap command or the createProject tool. When you run either of these commands a project configuration file is created in a new subdirectory of odasa/projects. This file provides information for other commands on how to checkout your source files and how to build your project. For further information see Preparing your code for analysis.

Checking out the latest version of your source code

The addLatestSnapshot command checks out the latest version of your source code using the default configuration defined in the project file. If necessary, you can use one or more flags to override any configuration element, however, in general, it is simplest to use the configuration defined in the project file wherever possible.

To add a snapshot using the project configuration

odasa addLatestSnapshot --project <project-dir>

where <project-dir> is the directory that contains the project configuration file of the project that you want to add a snapshot to.

Checking out older versions of your code

You can define a specific revision of the source code to check out using the addSnapshot command using the default configuration defined in the project file by specifying your chosen revision. As with addLatestSnapshot, you can use one or more flags to override any configuration element but, in general, it is simplest to use the configuration defined in the project file wherever possible. When using addSnapshot, you must always check to see if the version of the code you wish to check out uses the same build method as defined in the project file. You can override any configuration elements with appropriate flags if this is not the case.

Historic code that uses the current build method

If the historic code base can be built using the build commands defined in the project configuration file, then you can add a historic snapshot by overriding the normal checkout command using the --checkout flag.

To checkout a historic snapshot – no change to build method
  1. Optional, review the checkout commands in the project configuration file.
  2. To checkout a specific revision of the code base, use the addSnapshot command as follows:
    odasa addSnapshot --project <project-config-dir> --checkout <checkout_command> --default-build --date <revision_date> --name <label>

where <project-config-dir> is the directory where the project configuration file is stored and <checkout_command> is the checkout command for the required revision. The <revision date> is the date and the <label> is the name of this snapshot.

If a project has more than one checkout command, then you can repeat the --checkout flag with additional commands. The commands are executed in the order in which they are defined.

Example

The following example checks out the code base for Hadoop that has been tagged as release-2.0.0.-alpha. Like the current version of Hadoop, this code base is built using Maven, so the default project build commands can be used. The text in the example below is a single call to the addSnapshot command, line-breaks have been added to separate the different flags for clarity only.

Linux example: checkout the release-2.0.0.-alpha revision
odasa addSnapshot 
      --project projects/Hadoop
      --checkout 'git clone "https://github.com/apache/hadoop-common/" ${src}'
      --checkout 'git checkout release-2.0.0-alpha'
      --default-build
      --date 2012-05-23
      --name revision-Release-2.0.0-alpha


At a Windows command-line prompt the checkout commands must be enclosed in double-quotes instead of single-quotes, that is:

Historic code that uses a different build method

If the build method has changed since the code version of interest, then you must also define the build method for the code base. In this case you can checkout the historic snapshot as follows:

To checkout a historic snapshot – defining an old build method
  1. Optional, review the checkout and build commands in the project configuration file.
  2. To checkout a specific revision of the code base, use the addSnapshot command as follows:
    odasa addSnapshot --project <project-config-dir> --checkout <checkout_command> --build <build_command> --date <revision_date> --name <label>

where <project-config-dir> is the directory of the project to add a snapshot to, <checkout_command> is the checkout command for the required revision and <build_command> is the command required to build this version. The <revision date> is the date and the <label> is the name of the snapshot directory.

If a project has more than one checkout or build command, then you can repeat the --checkout and --build flags with additional commands. The commands are executed in the order in which they are defined.

Example

The following example checks out the code base for Hadoop that has been tagged as release-1.0.1. This earlier version of Hadoop is built using Ant and so the build commands defined in the project file must be overridden. The text in the example below is a single call to the addSnapshot command, line-breaks have been added to separate the different flags for clarity only. For longer commands like this you may prefer to define the build commands in a file and define a build command that calls this file.

odasa addSnapshot 
      --project projects/Hadoop
      --checkout 'git clone "https://github.com/apache/hadoop-common/" ${src}'
      --checkout 'git checkout release-1.0.1'
      --build 'ant clean'
      --build 'odasa index --auto ant -f build.xml'
      --build 'odasa duplicateCode --ram 2048 --minimum-tokens 100'
      --date 2012-05-23
      --name Release-1.0.1

Adding a snapshot for code stored in a detached directory

If you have already checked out the source code that you want to analyze into a detached directory, then call the addLatestSnapshot or addSnapshot command and use the source-location flag to define the location of the source code. For further information on using this check out configuration, see 64751788.

To analyze a historic snapshot stored in a detached directory
  1. Optional, review the build commands in the project configuration file.

  2. To generate a snapshot of the most recent revision of the code base which has already been checked out of the version control system, using the default build commands, use the addLatestSnapshot command as follows:
    odasa addLatestSnapshot --project <project-config-dir> --source-location <path to source>

where <project-config-dir> is the directory of the project to add a snapshot to, <path to source> is the directory of the source for the code base that you want to analyze.


  • Or, if you want to generate a snapshot of a different version of your code, use addSnapshot:
  • odasa addSnapshot --project <project-config-dir> --default-build --source-location <path to source> --date <revision_date> --name <label>


where <project-config-dir> is the directory of the project to add a snapshot to, <path to source> is the directory of the source for the code base that you want to analyze. The <revision_date> is the date and the <label> is the name for the snapshot directory. Snapshot directory names are generally of the form revision-year-month-day-time or revision-version-number.

Example

The following example adds an historic version of the source code that has been checked out and stored in the detached directory: /hadoop/revisions/version/2.0.0 . Like the current version of Hadoop, you build this code base using Maven, so you can use the default project build commands. The text in the example below is a single call to the addSnapshot command, line-breaks have been added to separate the different flags for clarity only.

odasa addSnapshot 
      --project projects/Hadoop
      --default-build
	  --source-location /hadoop/revisions/version/2.0.0
	  --date 2012-05-23
      --name revision-Release-2.0.0-alpha

Building a snapshot database

The addSnapshot and addLatestSnapshot commands are used to obtain a copy of your project's source code but you cannot analyze this copy of the code directly, or export it for analysis in the QL for IDE plugin. First, you must build a snapshot database using the buildSnapshot command. This tool uses build commands which are defined in the snapshot configuration file. In the example below, the buildSnapshot command is used to build a snapshot database for the most recently added snapshot for the project named myProject. If the snapshot has already been built, the --overwrite flag tells the command that the existing build should be overwritten. See buildSnapshot for further information.
odasa buildSnapshot --latest --overwrite --project $SEMMLE_HOME/projects/myProject

You can also append the directory name of a specific snapshot to build the database of an older revision of your code. These directory names are generally of the form revision-year-month-day-time or revision-version-number:

odasa buildSnapshot --overwrite --project $SEMMLE_HOME/projects/myProject revision-2018-August-13--14-44-39

After you have successfully built the snapshot using the buildSnapshot command, there are a number of further options you may consider: