Note: This documentation is for the legacy command-line tool odasa.
The final version was released in May 2020. Support for this tool expires in May 2021.

For documentation on the new generation CodeQL CLI, see CodeQL CLI .
In particular, you may find the Notes for legacy QL CLI users useful in planning your migration.

Skip to end of metadata
Go to start of metadata

Purpose

This tool may be used to compile new or updated query files to speed up future analyses. The analyzeSnapshot command automatically calls this before analyzing snapshots. All queries in the specified locations are reviewed and all new or updated queries are compiled (creating a .qlo file for each query). The primary use of this tool is to compile custom queries so that they are ready for use for analysis.

Usage

This tool is run from the command line as follows:

odasa prepareQueries [--verbose] [--verbosity <level>]
 [--display-strings] [--no-urls] [--include-results-without-url]
 [--icon-paths] [--fast-compilation] [--local-checking]
 [--fail-on-warnings] [--compilation-cache <path>]
 [--threads <num_threads>]  [--fallback-language <lang>]
 [--query-root <path>] [-suite <path>]
 [--debug-log <logfile>] [--slice <m/n>]  [--ram <memory>]
 [--check-only] [--save-qlos] <queries>

Preparing queries for use in dashboards

When you use the tool to pre-compile queries that you plan to run using the analyzeSnapshot command, it is important to use the default values for the results columns to output. If you override the default results columns by setting one or more of --display-strings--no-urls--include-results-without-url--icon-paths, queries will be re-compiled before they are run. Specifying the --fast-compilation or --local-checking flags will also result in re-compilation when the query is run by analyzeSnapshot.

Flags

The prepareQueries tool supports the following flags:

FlagsValueExampleNotes
--verbose--

Optional. Output more detailed information about actions. This increases the verbosity to level 4.

Default: level verbosity

--verbosity<level>2

Optional. Define the precise level of reporting required where 0 suppresses all output and 6 reports all levels of detail available. You can use the --verbose flag as shorthand for --verbosity 4 .

Default: 3

--display-strings--

Optional. Enable toString() computation. When the query is run each result will include a column of display strings. This provides readable output and is also used by QL plugins and extensions.

Default: the internal identifier for each result is reported.

--no-urls--

Optional. Disable the inclusion of URLs in the results. When the query is run results will not include a column of URLs. This information is required for results to be interpreted correctly.

Default: URLs are included for any result columns that have them.

--include-results-without-url--

Optional. Enable this option to include all results, whether or not they have a valid URL. Create a fake URL for any results that without a valid URL.

Default: any results without a valid URL are suppressed.

--fast-compilation--

Optional. If query compilation is required, perform a quick compilation, omitting some optimizations.

Default: any compilation of queries includes all possible optimizations.

--local-checking--

Optional. If query compilation is required, restrict error checking to the parts of the QL program that are used. Any areas of the imported QL libraries that are not used by the query are ignored.

Default: any error checking covers the full QL program, including all areas of the QL libraries that are imported.

--fail-on-warnings--

Optional. Any warnings reported during compilation are treated as an error and the compilation fails.

Default: any warnings are ignored. The process only fails if an error is detected.

--compilation-cache<path>custom-location/cache

Optional. Use to define an alternative directory to use for the compilation cache.

Default: the directory defined by the SEMMLE_COMPILATION_CACHE environment variable is used.

--threads<num_threads>2

Optional. Defines the number of threads to use.

Default: 1

--fallback-language<lang>java

Optional. The auto-detected query language (based on the QL file's associated queries.xml file) is used in preference to this. If a query file does not have an associated queries.xml file, the value of this option is used, instead of printing an error.

The language should be a valid language from the following list:

 Click to show valid languages...



  • C or C++ — language="cpp"
  • C# — language="csharp"
  • Go — language="go"
  • Java — language="java"
  • JavaScript — language="javascript"
  • Python — language="python"

The other supported option is language="odasa-dashboard" which is used to analyze dashboard databases

Default: an error is reported if the query language cannot be detected and this option is undefined.

--query-root<path>custom-location/queries

Optional. Defines the location of the default query libraries.

Default: SEMMLE_DIST/queries

--suite<path>custom-suites/extra-java

Optional. Defines the location of a query suite. All queries defined in the suite are compiled, in addition to any defined by specifying a location for the command.

Default: if this is undefined and no location is defined, then all queries under the SEMMLE_DIST/queries directory are compiled.

--skip-metadata-validation--

Optional. Don't validate query metadata and results patterns.

Default: query compilation will fail if any queries have select clause patterns that are not consistent with the @kind defined in their metadata. For further information about choosing the correct select clause for your query, see Select clause in the QL help.

--debug-log<logfile_name>query.log

Optional. Defines the name of a file to output the log messages to.

Default: none; debug logs are normally not produced.

--slice<m/n>1/3

Optional. Defines a subset of the queries to compile. The queries are divided into n slices and all queries in the m th slice are compiled.

Default: all queries in the specified directories or files are compiled (equivalent to 1/1).

--ram<memory> 2mb

Optional. Defines the amount of memory to use.

The value is assumed to be defined in 1024-based megabytes unless a unit is specified. Units are case insensitive.
 Show valid units...
  • k or kb—kilobytes, the value will be rounded to the nearest megabyte
  • m or mb—megabytes, the default unit if none is specified
  • g or gb—gigabytes
  • t or tb—terabytes

Default: 2048Mb.

--check-only--

Optional. Validate the queries and exit without performing any compilation.

Default: any queries that have not been compiled or that have been edited since their last compilation, are compiled.

--save-qlos--

Optional. Use to save the QLO files generated by the tool alongside the queries.

Default: The compiled queries are not saved.

<queries>-queries/custom

Optional. Defines the query files or directories of query files that you want to search for queries that need to be compiled. Specify . to search the current working directory and subdirectories For example:

odasa prepareQueries .

Default: if this is undefined and the --suite option is not defined, then all queries under the SEMMLE_DIST/queries directory are compiled.

Results

When prepareQueries is complete, all queries that were successfully compiled have a valid, associated *.qlo file. Running the prepared queries via analyzeSnapshot will not result in recompilation unless you have specified one or more of the following flags: --display-strings--no-urls--include-results-without-url--icon-paths,  --fast-compilation, --local-checking--check-only.

analyzeSnapshot normally compiles *.ql files to *.qlo files automatically, so this command is rarely used directly. A common use case is to pre-compile a set of custom queries before distributing them to several installations, so that the compilation only needs to happen once.

Exit code

The exit code of prepareQueries is 0 on success. Otherwise, the exit code is the number of queries that failed to compile (capped at a maximum of 255).