LGTM uses daemon processes, known as "workers," to build and analyze each revision of a project. This produces a variety of data, such as code quality alerts (see Code analysis in the user help). Each worker needs to run in an environment in which it can successfully build the project revision it is being asked to analyze.
All projects have dependencies, and these vary from project to project. Dependencies can include: a specific build tool (or a specific version of that build tool), a specific set of libraries, a particular operating system, or a minimum amount of available memory. If these requirements are not met then the code will not build. If you have projects that have very different, perhaps conflicting, dependencies then you can set up specific environments on individual worker host machines. You can then use labels to specify that code from a particular project—or all code written in a particular language—is processed by the workers running on a specific machine.
What are labels?
Labels are simply arbitrary text strings. They have no inherent meaning in themselves; they're just a way of matching up projects to workers. When you're choosing the text to use as a label, you should try to come up with something that will clearly indicate a feature that determines where a project should be processed—for example,
maven3-3 for projects that will only build successfully using Apache Maven 3.3.
Labels can contain the following characters:
a–z 0–9 - (hyphen) _ (underline)
Labels cannot contain uppercase letters.
Where are labels applied?
Labels can be applied to:
- Workers running on a particular host machine
- Individual programming languages
- Code in a specific language from an individual project
You define labels for particular worker host machines in the cluster configuration file.
You apply language-wide labels and project-specific labels in the LGTM administration interface.
Before defining language-wide and project-level labels you must know what labels are available on the worker side. Applying a label that has not been assigned to any worker, will prevent code from being analyzed.
The code of a specific language in a project gains all of the labels defined at the project level plus any labels defined globally for that language.
How labels work
Workers build and analyze revisions of the code in a particular language from an individual project—one revision at a time. Each time a worker is free to process another such job it requests one from the LGTM coordinator server. LGTM then compares the labels possessed by the worker with the labels of the jobs that are waiting to be processed, and passes an appropriate job to the worker.
For a worker to be assigned a job to process, the worker must have all of the labels associated with the code in that job (that is, all of the labels directly assigned at the project level, plus all of the labels assigned at the global language level). The worker can have additional labels that are not associated with the code in that job, but if it does not have all of the project/language labels it will not be used to process the code of that language from that project.
Each worker has an automatically assigned label to indicate its operating system and, optionally, labels assigned as part of the configuration of worker host machines.
Automatically assigned worker labels
Each worker is automatically assigned one of the following labels:
windows—if the host machine is running Windows
unix—if the operating system is not Windows
A typical use-case for labels is to specify that certain projects can only be built—or cannot be built—on Windows. Automatic worker labeling means that all you need to do to achieve this is to assign the
unix labels at the language-wide level and/or project level in the administration interface.
Assigning worker labels manually
The cluster configuration file (
lgtm-cluster-config.yml) defines machines that are used to run worker daemons. For each machine specified in this file, you can, optionally, define one or more labels. When you do this, all of the worker daemons that are started on this machine when you start LGTM Enterprise are assigned these labels.
For full details of the cluster configuration file, see lgtm-cluster-config.yml file.
The code sample below shows an extract from a cluster configuration file. It shows the
workers section of the configuration, in which four host machines are specified. The first of these, with the host name
lgtmcontrol, runs a single worker daemon to process queries submitted in LGTM's query console. Each of the other three machines runs two workers.
- The workers on workerhostA are not assigned labels manually.
- The workers on workerhostB get the labels
- The workers on workerhostC get a single label:
workers: hosts: - hostname: "lgtmcontrol" query: 1 - hostname: "workerhostA" general: 2 - hostname: "workerhostB" general: 2 labels: - highmemory - maven3-3 environment: LGTM_RAM: "8400" - hostname: "workerhostC" general: 2 labels: - pythonenv
After editing the cluster configuration file and adding labels for worker hosts, you must generate and deploy service configurations to the machines in the work pool. For details, see Editing and deploying the cluster configuration.
Checking which labels have been assigned to workers
Applying a label to a language
To ensure that all analysis of a particular language is carried out by workers that have a particular label, add a label to the language:
In the administration interface, click Code analysis.
In the relevant language section, type the name of a label into Worker labels field.
The labels you add must match labels assigned to one or more workers.
Add more labels, if required.
Click Update <language> settings.
A confirmation message is displayed at the top of the page: "Labels updated."
Code written in this language will only be processed by workers that have the labels you specified.
Applying a label directly to an individual project
To ensure that the code from a specific project is processed on a machine where particular build requirements are met, you can assign one or more labels directly to the project.
You can specify labels for an existing project in LGTM Enterprise, or when you add projects using the Add projects page in the administrator interface.
Applying labels when adding projects
The Worker labels box in the Advanced options section of the Add projects page, allows you to apply labels to projects. The projects you add will only be processed by workers that have all of the labels you specify (plus any relevant language labels that you have configured).
For example, if you specify the labels
maven3-3 when you bulk add some projects, then those projects will only be processed by workers that have both of these labels.
For more information about bulk adding projects, see Adding projects.
Applying labels to existing projects
In the administration interface, click Projects.
Click the project to which you want to assign a label.
If your system analyzes a lot of projects, you can search for the project by its name.
Click the Analysis settings tab.
This tab contains a section for adding labels. There is a label field for each language in the project that is analyzed by LGTM.
If you used the Add projects page to add the project, and you specified worker labels, then those labels will be displayed in the field for any languages that have been successfully analyzed.
In the language field that corresponds to the code in this project that you want to label, type the name of a label.
Add more labels, if required.
A confirmation message is displayed at the top of the page: "Updated worker labels for: <languages>."
Code from this project, in the relevant language, will only be processed by workers that have been assigned all of the labels you specified (plus any labels applied globally across this language).
When labels take effect
Changes to the labels defined in the cluster configuration file are applied to worker hosts when you generate and deploy updated service configurations. For details, see Editing and deploying the cluster configuration.
Changes to the language-wide and project-specific labels defined in the administration interface are applied to all new jobs that get queued for processing after you save the label changes. Jobs that are already queued for processing when you add or remove labels are not affected by the changes.
The diagram below represents a situation where three virtual machines have been set up to run workers. The job of the workers is to build and analyze revisions of seven projects, each of which contains code written in a single programming language. Labels have been assigned to determine which workers can be used to process revisions for each project.
In this example, the three virtual machines each run two worker daemons. Worker host B has a lot of RAM and, consequently, its worker daemons are assigned the label
highmemory in the cluster configuration file. Worker host C has been set up with the dependencies necessary for Python projects, so its worker daemons are labeled
There are seven projects, named P1 to P7. Projects P4 and P7 have been labeled
highmemory, in the LGTM administration interface, to ensure they are only processed on worker host B.
Also in the administration interface, the label
windows has been assigned to C# to ensure that all C# projects are processed on worker host A. This occurs because worker host A is running Windows, so all workers on this machine are automatically given the label
windows. Python has been given the label
pythonenv to ensure that all Python projects are processed on worker host C.
Project P4 has the label
highmemory and, being a Python project, also gets the label
pythonenv. However, none of the worker daemons has both of these labels, so build/analysis jobs for this project will fail. The Logs page will list these jobs with the status "RETURNED_FROM_QUEUE (NO_SUITABLE_WORKER)." To fix this problem, either remove the
highmemory label from this project, or reconfigure the workers so that at least one of them has both of these labels. For more details, see Fixing NO_SUITABLE_WORKER errors.
The following table provides a quick view of the machines on which projects, from the example above, are built and analyzed.
|Project||Language||Project + language labels||Processed by workers on host(s)|
|P1||C#||windows||Worker host A|
|P2||C||unix||Worker host B
Worker host C
|P3||Python||pythonenv||Worker host C|
|*(Jobs for this project will not be processed and will be logged as having failed.)|
Worker host B
Worker host C
Worker host B
Worker host C
|P7||Java||highmemory||Worker host C|