LGTM Enterprise 1.22.1

Editing and deploying the cluster configuration

The cluster configuration file lists the machines used by LGTM and specifies which services are hosted on each. This file is used by the lgtm-config-gen.jar tool to generate files that are deployed on the machines specified in the cluster configuration.

You will need to edit this file when you want to update the cluster of machines used by LGTM. For example:

Prerequisities

Password to decrypt the manifest file

If the LGTM manifest file was encrypted using a custom password, you must know this password as you will be prompted to enter it when you generate the files for a new, or updated, cluster configuration. If you need to reset the password, see Changing or resetting a manifest password.

File copying

You must be able to copy files from the coordinator server (that is, the server in the control pool that hosts the LGTM scheduler service), to the servers in the LGTM cluster.

Root or administrator access

The process involves running an installation script that requires raised privileges, root access on Linux machines and administrator access on Windows machines. You will, therefore, need to be able to run commands using sudo or the "Run as administrator" option.

Access to third-party packages

If you add further machines to the LGTM cluster, or change their use, all Linux machines require access to the packages required to set up the relevant LGTM services during deployment. This access is usually provided either by an internet connection, or by access to an internal mirror with the relevant packages. For more information, see Third-party software installed by LGTM.

Process overview

Whenever you change the configuration of your LGTM cluster, there are three distinct steps:

  1. Edit the cluster configuration file
  2. Generate the files for the updated configuration
  3. Deploy the updated configuration on:

Editing the cluster configuration file

On the coordinator server (that is, the server in the control pool that hosts the LGTM scheduler service):

  1. Open the LGTM cluster configuration file in a text editor.

    • There is no set name or path for the cluster configuration file, but by convention it is state/lgtm-cluster-config.yml. The state directory is typically held under version control and checked out into the LGTM releases directory. A complete path might therefore be: /home/lgtmadminuser/lgtm-releases/state/lgtm-cluster-config.yml.

    • The cluster configuration file is not used directly by LGTM—it is a description of a configuration. The current configuration is stored internally by LGTM. You can generate a cluster configuration file that describes the current configuration by using the lgtm-config-gen.jar tool's init action.

  2. Edit the file as required.

  3. Save your changes to the cluster configuration file.

  4. Check that the file is still valid YAML—for example, by using a validation tool such as yamllint. For more information, see yamllint.readthedocs.io.

Generating files for the configuration

At a command line on the coordinator server:

  1. Change to the lgtm-<version> directory.

    This is the directory that contains generated and lgtm subdirectories.

  2. Enter:

    java -jar lgtm/lgtm-config-gen.jar generate --overwrite

  3. If the manifest file is encrypted using a custom password, you're prompted for this password.

    Defining a custom password for this file is an option during installation.

    This command generates files for your LGTM configuration based on the details in the updated cluster configuration file—overwriting the old version of those files stored in the generated directory. A set of files is generated for each machine in the cluster. Each set of files is written to a machine-specific subdirectory of the generated directory, named with the hostname specified in the cluster configuration file.

Deploying an updated configuration on Linux hosts

You have generated the files for an LGTM cluster configuration. These are stored within separate directories: one for each host machine named in the cluster configuration file. You need to copy the appropriate files to each machine, and deploy the changes.

Preparation

  • Existing cluster machines—you may want to take a copy of the existing lgtm-<version> directory on the host, before overwriting this directory with the new version.
  • New cluster machines—create a directory as your LGTM releases directory. This machine will need access to the internet or to an internal mirror with the third-party software required by LGTM Enterprise during deployment.

It is up to you where the LGTM releases directory is located and what it is called. However, it is referred to in this documentation as the LGTM releases directory, so you may want to call it lgtm-releases.

Copying the new files to Linux hosts

For each Linux machine in the cluster, copy the lgtm-<version>/generated/<remote machine hostname> directory from the coordinator server to an <LGTM releases directory>/lgtm-<version>/generated/<remote machine hostname> directory on the cluster host machine.

The generated directory and its contents are readable only by the user who installed, or upgraded, LGTM Enterprise, so you will need to copy the files across as this user, or change the ownership of the directory and its contents.

Copying using rsync

rsync provides a fast, effective way to copy files from the coordinator server to the remote machine. Use a command such as:

rsync --recursive --archive --compress --progress \
   --include "generated/<remote-machine-hostname>/" \
   --exclude "generated/*/" \
   <path-to>/lgtm-releases/lgtm-<version>/ \
   <user>@<remote-machine-hostname>:<path-to>/lgtm-releases/lgtm-<version>

For example:

rsync --recursive --archive --compress --progress \
   --include "generated/lgtmworkerhost1/" \
   --exclude "generated/*/" \
   $HOME/lgtm-releases/lgtm-1.21.0/ \
   lgtmadminuser@lgtmworkerhost1:lgtm-releases/lgtm-1.21.0
  • In the above example, the contents of the lgtm-1.21.0 directory are copied to the lgtm-releases/lgtm-1.21.0 directory in the home directory of the "lgtmadminuser" user, with the exception of all the host-specific contents of the generated subdirectory other than the lgtmworkerhost1 directory.

  • rsync needs to be able to connect to the remote server. If you have not set up public/private key file authentication between the local and remote machines you'll be prompted to enter a password for the specified user.

Deploying the new configuration on Linux hosts

  1. On each Linux machine in the cluster, change to the lgtm-<version>/generated/<remote-machine-hostname> directory.

  2. Run the install machine script in this directory:

    sudo ./install-machine.sh

    The script installs and starts the specified service or services. The output from the script ends:

    [success] All LGTM services on this machine have been started.

    [success] LGTM started.

Any changes to the configuration take effect immediately.

Deploying an updated configuration on Windows hosts

You have generated the files for an LGTM cluster configuration. These are stored within separate directories: one for each host machine named in the cluster configuration file. You need to copy the appropriate files to each machine, and deploy the changes.

Preparation

On new hosts, create a directory to store the generated files, the build scripts, and the installer for the LGTM worker. Unlike Linux hosts, this machine does not need access to the internet or to an internal mirror during deployment.

Copying the new files to Windows hosts

For each Windows worker host machine in the cluster:

  1. Copy the lgtm-<version>/generated/<hostname> directory from the coordinator server to the cluster host machine (where <hostname> is the host name specified in the cluster configuration file).
  2. Copy the lgtm-<version>/lgtm/lgtm-worker_<version>.msi into the newly copied <hostname> directory.

The <hostname> directory on the Windows machine should now contain the lgtm-worker_<version>.msi installer, an install-machine script, and an lgtm-worker subdirectory that defines the configuration for that machine.

The <hostname> directory and its contents are readable only by the user who installed, or upgraded, LGTM Enterprise, so you will need to copy the files across as this user, or change the ownership of the directory and its contents.

Deploying the new configuration on Windows hosts

On each Windows worker host machine in the cluster, run the install machine script stored in the newly copied <hostname> directory. Depending on which account you want the LGTM worker daemons to run under, run one of the following commands:

  • User account: install-machine.bat <domain>\<username> <password>
  • LocalSystem account: install-machine.bat

When you use a user account, it must have full access to the working directory specified for this machine in the cluster configuration (temp_path property). The account details are passed directly to sc.exe and used to set up services for LGTM. This account will run the LGTM worker daemons.

The script installs and starts the specified service or services. The output from the script ends:

[success] All LGTM services on this machine have been started.

[success] LGTM started.

Any changes to the configuration take effect immediately.

Related topicsRelated topics