3.3. Worker Configuration

Note

This configuration is new for versions 1.18+. Best practice is to backup any pre-1.18 worker configuration prior to conversion to the 1.18+ format.

The Ngenea Worker configuration should be added to /etc/ngenea/ngenea-worker.conf. The configuration is in ini format. For example:

[settings]
site = site1
api_host = localhost
api_port = 8000
api_secure = true
redis_port = 6379
gpfs_nodes = ["localhost"]

Note

api_secure=true requires a valid SSL certificate and is not compatible with self-signed SSL certificates.

ngenea-worker defaults to api_secure=true if the api_secure setting is not specified.

Refer to: External SSL for provisioning SSL certificates

Alternately, specify api_secure=false to disable use of SSL.

The following is a list of available settings:

Option

Type

Default

Required

Description

site

string

Yes

The name of the queue to listen to

api_host

string

Yes

The base url for Ngenea Hub, this will be the url without the https protocol or port.

api_port

string

Yes

The port that the Ngenea Hub is being hosted on, by default within Ngenea Hub it is 8000.

api_secure

string

No

If the API is behind a secure HTTPS connection, by default this is true. Refer to: External SSL for provisioning SSL certificates if api_secure=true.

api_secure_verify

string

No

If the API requests should verify certificates, by default this is true.

threads

int

10

No

The number of concurrent tasks that can be run.

heartbeat

bool

true

No

Key for Disabling/Enabling celery heartbeats, by default Enabled.

gossip

bool

false

No

Key for Disabling/Enabling celery gossip, by default Disabled.

mingle

bool

false

No

Key for Disabling/Enabling celery mingle, by default Disabled.

redis_health_check_interval

int

60

No

The Redis backend supports health checks. This value must be set as an integer whose value is the number of seconds between health checks.

redis_socket_timeout

int

60

No

The Redis results backend supports a socket connection timeout, this value must be set as an integer whose value is the number of seconds.

celery_socket_timeout

int

60

No

The Redis celery broker supports a socket timeout, this value must be set as an integer whose value is the number of seconds.

celery_connection_timeout

int

60

No

The Redis celery broker supports a socket connection timeout, this value must be set as an integer whose value is the number of seconds.

gpfs_nodes

list

No

A list of nodes to run the snapdiff policy scan on as a list of hostnames

enable_plugins

bool

false

No

Key for Disabling/Enabling worker plugin behaviour (currently in alpha), by default Disabled.

loglevel

str

INFO

No

Key for setting the worker loglevel (more specific to worker main process). valid choices are INFO, DEBUG, WARNING, CRITICAL, ERROR (by default INFO).

Note

The initial access credentials can be found in the /etc/sysconfig/ngeneahub configuration file on the Ngenea Hub server. By default, the BROKER_PASSWORD is used for both the broker_url and the result_backend passwords, and the BROKER_USER is used for the broker_url username.

3.3.1. Controlling Functions

Ngenea Worker contains multiple internal function queues, each of these are configurable and can be disabled. For details on what these functions are see the queue function.

The following are the configuration values to control them on both a custom or default queue:

For a detailed list of workflow steps and the functions that run them see Workflow Steps

3.3.2. Custom Queues

Custom job queues can be added to the worker by adding Queue sections to the worker config.

[Queue highpriority]
threads = 20

For more information on queues, see Queues

3.3.3. Passing Custom Config File to Ngenea Worker

The custom config file will be present in /etc/ngenea/ngenea_config_arg.conf file to run the systemd services.The format of the file is given below.

CONFIG=/etc/ngenea/ngenea-worker-custom.conf

Note

By default, this file will have the CONFIG commented to allow the default config to pass through if custom config file is not specified.

Uncomment the CONFIG in /etc/ngenea/ngenea_config_arg.conf file. Create a new worker config file (as ngenea-worker.conf) with different site name. Also site should be added to hub and worker using the commands listed below.

ngeneahubctl addsite <sitename>
ngenea-worker join --user <username> --site <sitename>

3.3.3.1. To Run ngenea-worker with multiple sites

To run services with two sites,

  1. Create new service file same as /usr/lib/systemd/system/ngenea-worker.service with different filename (Eg: ngenea-worker-site1.service).

  2. Create new custom config file same as /etc/ngenea/ngenea_config_arg.conf with different filename (Eg: ngenea_config_arg1.conf).

  3. Create new worker config file same as /etc/ngenea/ngenea-worker.conf with different filename and give the filename in ngenea_config_arg1.conf.

  4. In ngenea-worker-site1.service, change the value of EnvironmentFile=... to new custom config file created (Eg: EnvironmentFile=/etc/ngenea/ngenea_config_arg1.conf).

  5. Now run systemctl start ngenea-worker.service and systemctl start ngenea-worker-site1.service.

3.3.4. Setting Ngenea Worker debug level in systemd service script

In /usr/lib/systemd/system/ngenea-worker.service file, set the value of Environment=DYNAMO_DEBUG=.. to true as below. It will be false by default. This debug level is more specific to tasks logging.

Environment=DYNAMO_DEBUG=true