In this tutorial you will create a snapshot to allow you to analyze LuaJIT, which is an open source Just-In-Time compiler for the Lua programming language.
The tutorial covers basic use of the
odasa bootstrap tool to retrieve source files, start code analysis and export a snapshot of the project.
Time needed: 10–15 minutes
This tutorial assumes:
- You have installed Semmle's analysis software—see Installing Semmle Core.
- You have set up the environment for Semmle Core—see Setting up the environment.
- You have installed Git and added it to your path – see http://git-scm.com/book/en/Getting-Started-Installing-Git.
- You have installed:
Create a new project with odasa bootstrap
The quickest way to create a new Semmle analysis project is to use the
odasa bootstrap command. The bootstrap tool steps you through the process and generates configuration files that you can reuse later. The process described here gives the steps required to analyze a particular C project built using gcc (Linux) or Visual Studio (Windows). Other projects will require different responses to some of the bootstrap prompts.
The bootstrap tool's on-screen instructions indicate what you should enter at each step. However, the first time you run it, you may find it useful to refer to the following procedure for some additional information.
Open a command console.
Important - Windows command prompt
On Windows, the LuaJIT build process requires that Visual Studio's environment variables are set. You must therefore either run
odasa bootstrapin a Visual Studio Command Prompt window, or run the following batch script in a standard Windows command console before running
If you are using an old version of Visual Studio (2014 or earlier), you may need to replace `vsdevcmd.bat` with `vsvars32.bat`.
NoteThe requirement to use the Visual Studio Command Prompt (or to run
vsdevcmd.bat) is specific to LuaJIT build process, and not a general requirement for building C/C++ projects on Windows.
- Change to the directory where Semmle Core is installed – for example,
- Make sure the ODASA environment is set:
Linux and OS X:
See Setting up the environment.
Enter the following command to start the bootstrap tool:
The tool guides you through the rest of the process. The remainder of the steps provided below give some additional guidance that you may find useful the first time you run the bootstrap tool.
For additional on-screen information, enter
?at any of the prompts.
The project LuaJIT is written in C.
- Make sure to enter a lowercase c for C/C++ and not an uppercase C for C#.
- The languages listed here depend on your Semmle license and the version of the Semmle software you have installed. You may, therefore, see a different selection of languages than the list shown here.
The LuaJIT project uses Git as the version control system for its source code.
The LuaJIT project stores its source code in Git.
bootstraptool with the URL of the repository makes it possible to automatically download the latest version of the software each time an analysis is triggered.
Press Enter without specifying a branch name.
The bootstrap tool will default to cloning the code from the "master" branch for the project.
The Semmle software determines which files to analyze (and which libraries are needed) by monitoring the build process. This means you need to provide the
bootstraptool with information about how the software is built.
Most C/C++ projects are built on OS X using an Xcode project file. LuaJIT does not provide an Xcode project file. Instead, it is built using gcc and make.
LuaJIT project is built on Linux using gcc and make. The bootstrap process automatically selects the "other" option and proceeds to the next step.
Most C/C++ projects are built on Windows using a Visual Studio project or solution file. LuaJIT does not provide a Visual Studio project/solution file. Instead, it provides a batch script to build the project.
- yif building on Windows
Windows only: Name of the working directory
When building projects using the
makecommand, usually an incremental build is performed to avoid recompilation of previously compiled source files. Hence, projects built using
maketypically provide a
cleancommand (such as
make clean) to remove the existing build artifacts so that a full non-incremental build can be performed. Some features of Semmle Core (such as building multiple snapshots from the same source code) require a full build to ensure correct behavior. A clean step is essential when using such features.
Since this tutorial performs a full non-incremental build on newly checked-out source code, a clean step is not essential. On Windows, the LuaJIT project always performs a non-incremental build.
- makeif building on Linux or OS X
- msvcbuildif building on Windows
Creating a snapshot
You have now supplied all the information needed for the project file that is used each time a snapshot is generated. You can now go ahead and create a snapshot.
This tells the bootstrap tool that you want to clone the current LuaJIT source files from GitHub and generate files needed to build a Semmle snapshot. This command will be run at the end of the bootstrap process.
Running a set of analyses
This tells the bootstrap tool that you want to run a set of queries to analyze the snapshot. This command will be run at the end of the bootstrap process.
Specifying an analysis suite
If you press Enter without specifying a suite, the bootstrap tool will run the standard set of C/C++ queries.
Alternatively, you can tell the bootstrap tool to run the same queries that are run on LGTM by specifying the LGTM query suite for C/C++:
Exporting the database archive
This tells the bootstrap tool to create a zip archive containing a snapshot in your project directory. This command will be run at the end of the bootstrap process.
The bootstrap tool now:
- Clones the current LuaJIT source files from GitHub, and generates files needed to build the snapshot.
- Indexes the code base, builds a snapshot, performs C/C++ analysis, and saves the results of the analysis in a SARIF results file in the project directory.
- Exports the snapshot as a zip archive. You can use this snapshot to run queries in your IDE, for example using the QL for Eclipse plugin.
Completing these operations will take a few minutes.
When the bootstrap tool finishes, it displays a message like this:
Now you can view the analysis results in a SARIF viewer, or import the snapshot database into your IDE and run additional analyses.
If error messages are reported, you can investigate the problem by reviewing the log files:
Before rerunning the bootstrap process, enable prototyping mode. This will help with any further troubleshooting you need to do.
Spend some time investigating the analysis of the LuaJIT code base. This will help you familiarize yourself with the standard rules available for C analysis.
Query the snapshot in your IDE
If you install a QL plugin or extension, you can easily write custom queries to analyze snapshots and view the results directly in your IDE.
Create more snapshots
Use the bootstrap tool to create a project for your own code base.
Work through the tutorial on advanced project creation to see some examples of the available configuration options.