Quick links
Semmle 1.19
On this page:
Related topics:
This topic describes the infrastructure required to set up an installation of Enterprise Insight.
Overview
Enterprise Insight consists of three main components:
- Semmle Insight server—hosts one or more Insight server services to receive data and import it into one or more databases.
- Data providers—publish data to the Insight server service - either from Project Insight dashboards or from third-party applications.
- Visualization server—runs the data visualization software.
An overview of the standard server configuration:
Servers
The servers shown in the example configuration above perform the following actions:
| Server | Actions |
|---|---|
| Data providers |
|
| Insight server |
|
| Visualization server |
|
Requirements
Enterprise Insight is based on Semmle Core technology (see System requirements for general requirements). For each project that you want to include in Enterprise Insight, you need to configure a working Project Insight dashboard. The additional requirements for Enterprise Insight are described in more detail below.
Data requirements
The data providers require passwordless public/private key connections to the Insight server to enable data to be passed as shown in the diagram above. This can be configured using either SSH (OpenSSH or Tectia SSH preferred) or HTTPS. For third-party applications, the main requirement is that any data that you want to include in Enterprise Insight can be exported into well-defined CSV files. You can then push a zip archive of the exported CSV files to the Insight server service ready to be imported into the Enterprise Insight database.
| Server | Software required to be installed | Other requirements |
|---|---|---|
| Data providers |
|
|
| Insight server |
|
|
| Visualization server |
|
|
If Enterprise Insight is configured for a weekly update, then all builds and analysis must be able to complete in a timely fashion, allowing time for the aggregation and integration of data and update of the visualization server. For companies with many projects, this requires substantial resources and, in practice, Enterprise Insight is often configured for monthly updates in line with the availability of new data from financial reporting systems.
Data storage requirements
The data storage requirements for the visualization server and Semmle Insight server are fairly small.
The data providers are set up to run Project Insight and need enough space to obtain a full copy of the source code, build it and save the results of analysis. In practice, they typically need a minimum of disk space equivalent to about 10 times the size of the largest project configured for analysis, multiplied by the number of threads that you intend to run on that server.
Visualization requirements
The Insight server converts the data received from the data providers into a form suitable for the nominated business intelligence reporting tool and stores it in the database. This SQL database is normally hosted by either the Semmle Insight server or the visualization server.
The two business intelligence visualization tools currently supported are: Tableau and Qlikview. Please refer to the product documentation for details of the server requirements for your chosen application.
Server maintenance and backup
The standard server maintenance and file system backup responsibilities are as follows:
| Server | Maintenance | Backup |
|---|---|---|
| Semmle Insight server | Semmle:
| Customer:
|
| Data providers | Customer:
| Customer:
|
| Visualization server | Customer or Semmle, as appropriate2:
| Customer:
|
1 Semmle Core software is usually updated on a regular schedule agreed with the customer.
2 Where an existing visualization application is used to serve Enterprise Insight data, the customer normally takes responsibility for the maintenance of the software. If Semmle installs a new visualization application then they are normally responsible for its maintenance. Visualization software typically needs to be updated less frequently, unless there are specific reasons to upgrade to a new revision of the software.
Minimum backup requirements
The data stored on the Semmle Insight server and visualization server is derived and so it is not essential to back up all analysis and results—providing that, in the event of a server failure, you are prepared to take the time/resources to regenerate all the data again. The minimum backup requirements are as follows:
| Backup | Server | Restore to enable |
|---|---|---|
| Visualization data/workbooks | Visualization | Users to view existing data and workbooks |
Semmle SQL database | Insight | Users to view existing data and workbooks Insight server to process new data received |
| Semmle Core, configuration and any associated scripts | Insight | Insight server to process new data received |
| Configuration and build environment | Data providers | Analysis of new versions of code |
Planning for a deployment
The number and specification of servers required for an installation of Enterprise Insight varies widely according to the size and number of projects selected for analysis. Semmle recommends that you identify a potential data provider and then select a representative sample of projects for testing and configure them for analysis by Project Insight on this server. This will enable you to verify the:
- Current build prerequisites for each project
- Time taken to build/analyze the code
- Size of a Project Insight snapshot and associated results for a single revision of the source code
This information can be used to confirm the build prerequisites for Project Insight, and help you determine a more accurate estimate of the number and specification of servers required to provide Project Insight data.
Overview
Content Tools