LGTM Enterprise 1.22.1

Checking progress for added projects

The Add tab of the Projects administration page has a History section. You can use this to check the status of the projects that you added to LGTM.

The History section shows every batch addition that an LGTM administrator adds using that tab. The most recent batch is listed first.

Viewing progress for a batch

To view the progress information for a batch addition, click the appropriate arrow button in the History section.

The Add projects page shows the following status information:

  • Batch creation—date and time the batch of projects was added to LGTM.
  • Last updated—date and time the information was most recently updated.

Initially this page is set to auto-reload the progress of the jobs that are running for the listed project(s). The details on the page change as jobs complete. If you leave the page and then return to it later, auto-reloading is turned off. Click Auto-reload if you want the page to update automatically.

The page also shows one or both of the following sections:

  • Attempted—one row for every repository URL that matched a repository provider, and for which attempt-build jobs have been queued.
  • Could not be attempted—where present, one row for every repository URL where it wasn't possible to start the attempt-build jobs. For example, because the host failed to respond.

Monitoring attempt-build jobs

In the Attempted section, each potential project name is listed with details of every attempt-build job that is queued, is running, or has completed. Click to open the repository URL in a new browser tab.

The status is indicated as follows:

  • job in progress
  • job completed successfully
  • job failed

If at least one language was analyzed successfully, the repository is registered as an LGTM project. You can see the full log information for completed jobs by clicking the red cross or green check mark.

In the example above, you can see that LGTM has successfully analyzed JavaScript code for the first project. The repository was registered as the project yarnpkg/yarn. Click the project name to display the newly created project administration page.

Failed attempt-build jobs

The main reasons for attempt-build jobs to fail are:

  1. There was a problem checking out the code (all jobs for that repository will fail).
  2. The project doesn't have any code written using that language.
  3. The environment isn't set up correctly for that project and some dependencies are missing.
  4. For compiled languages, LGTM is unable to build the project using the global configuration.

If many projects use a single programming language, you'll see many failed jobs displayed in the Attempted section. There will be one failed job for each unused language, in addition to any jobs that failed because of genuine problems.

For example, the table below shows four projects that were written predominately in one language. The primary languages for most of the projects have been analyzed successfully. The only exception is apache/ant, which is primarily written in Java.

Retrying failed jobs

Click the red cross for a failed job to display the full log information. The Log details page is displayed (for general information about this page, see Exploring work pool logs).

The stdout and stderr generated by the job are displayed (truncated if the output is long). Click the Show link to see the full output as raw text. You can retry any project using the Retry button on the History page (see image above).

  1. On the History page, click the Retry button for the attempted project to display a Retry project page.

  2. In the Manual configuration section, add a YAML extraction block to customize building and analyzing the code. For details of what to enter here, see the topics on customizing extraction code and the lgtm.yml configuration file in the user help.

  3. When you've finished defining the configuration, click Retry.

  4. A new set of attempt-build jobs is queued for the project, using the customized project configuration, and the History page is redisplayed.

Example: Java build failure

In the example above, the Java code for the Apache Ant project failed to build. The stdout includes the following error:

[2019-06-28 09:31:37] [build] [2019-06-28 09:31:37] [autobuild] > ant -noinput -f "build.xml" clean "main"
[2019-06-28 09:31:38] [build] [2019-06-28 09:31:38] [autobuild] Picked up JAVA_TOOL_OPTIONS: -Djava.io.tmpdir="/var/cache/lgtm/workers/1/private"
[2019-06-28 09:31:38] [build] [2019-06-28 09:31:38] [autobuild] Running '/var/cache/lgtm/workers/1/downloads/setup_dist/fdfd7276708505ad41e0c268bed67a37.unzipped/odasa/tools/preload_tracer ant -noinput -f build.xml clean main' with tracer /var/cache/lgtm/workers/1/downloads/setup_dist/fdfd7276708505ad41e0c268bed67a37.unzipped/odasa/tools/preload_tracer
[2019-06-28 09:31:38] [build] [2019-06-28 09:31:38] [autobuild] Runner failed to start 'ant': No such file or directory
[2019-06-28 09:31:38] [build] [2019-06-28 09:31:38] [autobuild] [2019-06-28 09:31:38] [ERROR] Spawned process exited abnormally (code 1; tried to run: [/var/cache/lgtm/workers/1/downloads/setup_dist/fdfd7276708505ad41e0c268bed67a37.unzipped/odasa/tools/preload_tracer, ant, -noinput, -f, build.xml, clean, main])

The Java autobuild process failed to find and start ant. The Ant project can only be built using the latest version of the Ant build system. Even if Ant is already installed on the worker machines, this job will fail. You need to use a YAML extraction block in the project configuration to specify a build command that will bootstrap the Ant code.

For example:

extraction:
  java:
    before_index:
      - export ANT_HOME="${LGTM_SRC}/bootstrap"
      - export PATH="${PATH}:${ANT_HOME}/bin"
      - ./bootstrap.sh

This block adds an extra step before the standard LGTM index step. The before_index step runs the following commands:

  1. Define the ANT_HOME environment variable as the bootstrap folder in the directory where LGTM has downloaded the source code. This tells LGTM where to find Ant.
  2. Add the Ant executable to the path.
  3. Run the bootstrap.sh script. This generates a folder called bootstrap that contains a basic version of Ant .

Now when the index step runs, it will use the bootstrapped version of Ant to build the full codebase.

In this example, the worker daemons are Linux-based so the commands are written to run under bash. For Windows-based workers, the commands run in the Windows command shell. They also need to set Windows environment variables and to use the bootstrap.bat script instead of bootstrap.sh. That is:

extraction:
  java:
    before_index:
      - set ANT_HOME="%LGTM_SRC%/bootstrap"
      - set PATH="%PATH%;%ANT_HOME%/bin"
      - ./bootstrap.bat

Related topicsRelated topics