3.3. Worker Configuration¶

Note

In the installation guide we have explained how to configure the Ngenea Worker, create or edit the /etc/ngenea/ngenea-worker.conf file with the required settings for your site, API server, Redis, and GPFS nodes. You can also configure SSL for secure communication with the API.

For more details, please refer to the link.

3.3.1. Settings¶

The following is a list of available settings:

Option	Type	Default	Required	Description
site	string		Yes	The name of the queue to listen to
api_host	string		Yes	The base url for Ngenea Hub, this will be the url without the https protocol or port.
api_port	string		Yes	The port that the Ngenea Hub is being hosted on, by default within Ngenea Hub it is 8000.
api_secure	string		No	If the API is behind a secure HTTPS connection, by default this is true. Refer to: External SSL for provisioning SSL certificates if `api_secure=true`.
api_secure_verify	string		No	If the API requests should verify certificates, by default this is true.
threads	int	10	No	The number of concurrent tasks that can be run.
heartbeat	bool	true	No	Key for Disabling/Enabling celery heartbeats, by default Enabled.
gossip	bool	false	No	Key for Disabling/Enabling celery gossip, by default Disabled.
mingle	bool	false	No	Key for Disabling/Enabling celery mingle, by default Disabled.
redis_health_check_interval	int	60	No	The Redis backend supports health checks. This value must be set as an integer whose value is the number of seconds between health checks.
redis_socket_timeout	int	60	No	The Redis results backend supports a socket connection timeout, this value must be set as an integer whose value is the number of seconds.
celery_socket_timeout	int	60	No	The Redis celery broker supports a socket timeout, this value must be set as an integer whose value is the number of seconds.
celery_connection_timeout	int	60	No	The Redis celery broker supports a socket connection timeout, this value must be set as an integer whose value is the number of seconds.
gpfs_nodes	list		No	A list of nodes to run the snapdiff policy scan on as a list of hostnames
enable_plugins	bool	false	No	Key for Disabling/Enabling worker plugin behaviour (currently in alpha), by default Disabled.
loglevel	str	INFO	No	Key for setting the worker loglevel (more specific to worker main process). valid choices are INFO, DEBUG, WARNING, CRITICAL, ERROR (by default INFO).

3.3.2. Controlling Functions¶

Ngenea Worker uses internal function queues to process tasks. These queues can be adjusted based on your needs.

What Are Function Queues ?

Function queues are like task managers that organize and run specific tasks. You can turn these on or off and adjust their settings.

How to Configure Function Queues ?

Open the configuration file (/etc/ngenea/ngenea-worker.conf.) for Ngenea Worker.
Modify settings to enable or disable queues.
Refer to the Workflow Steps documentation to see what each queue does.

3.3.3. Creating Custom Queues¶

If the default queues don’t meet your needs, you can create custom job queues. These allow you to prioritize specific tasks or allocate more resources.

Example Configuration for a Custom Queue

Steps to Add a Custom Queue

Open the worker’s configuration file (/etc/ngenea/ngenea-worker.conf.).
Add a new section with the name of the queue in brackets, like [Queue highpriority].
Specify the number of threads for that queue. (Optional)
Save and reload the worker service to apply changes.

For detailed information, check Queues Documentation.

3.3.4. Using a Custom Configuration File¶

Configuration files are text files used to set options and preferences for a system or application. Sometimes, you may need to override the default configuration with a custom one. Here’s how to set it up:

By default, the system uses the file located at:

/etc/ngenea/ngenea_config_arg.conf

How to Use a Custom Config File ?

Uncomment the CONFIG Line: Open the file /etc/ngenea/ngenea_config_arg.conf and remove the # symbol in front of this line:

CONFIG=/etc/ngenea/ngenea-worker-custom.conf

Create a Custom Worker Config File: Save a new file with a name like ngenea-worker-custom.conf. Use this file to define your settings.
Register Your Site: Run these commands to register the custom site:

ngeneahubctl addsite <sitename>
ngenea-worker join --user <username> --site <sitename>

3.3.5. Running Ngenea Worker with Multiple Sites¶

Note: Running multiple sites on a node has several drawbacks, so using custom queues is preferred where possible.

To run services with two sites:

Create new service file same as /usr/lib/systemd/system/ngenea-worker.service with different filename (Eg: ngenea-worker-site1.service).
Create new custom config file same as /etc/ngenea/ngenea_config_arg.conf with different filename (Eg: ngenea_config_arg1.conf).
Create new worker config file same as /etc/ngenea/ngenea-worker.conf with different filename and give the filename in ngenea_config_arg1.conf.
In ngenea-worker-site1.service, change the value of EnvironmentFile=... to new custom config file created (Eg: EnvironmentFile=/etc/ngenea/ngenea_config_arg1.conf).
Now run systemctl start ngenea-worker.service and systemctl start ngenea-worker-site1.service.

3.3.6. Setting Ngenea Worker Debug Level in systemd Service Script¶

Environment Directive: In a system, there are background tasks that run automatically (called services). The `Environment` directive in the service’s setup tells it about certain settings it should use while running.

What Does `DYNAMO_DEBUG=true` Do?

Environment=DYNAMO_DEBUG=true sets a setting called `DYNAMO_DEBUG` to `true` for the service called ngenea-worker.

What Happens with This Setting?

Normally, the ngenea-worker service only logs basic information about what it’s doing (like regular updates).
When you set `DYNAMO_DEBUG=true`, the service will log extra, detailed information about its activities.