3.5. Worker Configuration¶

Note

In the installation guide we have explained how to configure the Ngenea Worker, create or edit the /etc/ngenea/ngenea-worker.conf file with the required settings for your site, API server, Redis, and GPFS nodes. You can also configure SSL for secure communication with the API.

For more details, please refer to the link.

3.5.1. Settings¶

The following is a list of available settings:

Option	Type	Default	Required	Description
site	string		Yes	The name of the queue to listen to
api_host	string		Yes	The base url for Ngenea Hub, this will be the url without the https protocol or port.
api_port	string		Yes	The port that the Ngenea Hub is being hosted on, by default within Ngenea Hub it is 8000.
api_secure	string		No	If the API is behind a secure HTTPS connection, by default this is true. Refer to: Installing SSL Certificates for provisioning SSL certificates if `api_secure=true`.
api_secure_verify	string		No	If the API requests should verify certificates, by default this is true.
broker_provider	string	redis	No	The service that will provide worker connectivity with the hub, must be one of the following: redis\|rabbitmq\|external_rabbitmq\|external_redis
result_provider	string	redis	No	The result storage that will store completed worker tasks for the hub to access, must be one of the following: redis\|external_redis\|external_redis_cluster
threads	int	10	No	The number of concurrent tasks that can be run.
heartbeat	bool	true	No	Key for Disabling/Enabling celery heartbeats, by default Enabled.
gossip	bool	false	No	Key for Disabling/Enabling celery gossip, by default Disabled.
mingle	bool	false	No	Key for Disabling/Enabling celery mingle, by default Disabled.
redis_health_check_interval	int	60	No	The Redis backend supports health checks. This value must be set as an integer whose value is the number of seconds between health checks.
redis_socket_timeout	int	60	No	The Redis results backend supports a socket connection timeout, this value must be set as an integer whose value is the number of seconds.
redis_host	string		No	The host for the Redis service. - For Internal Redis: if provided, this host will be used instead of api_host. - For External Redis: required; must be a valid IP or hostname.
redis_port	string	6379	No	The host port for a Redis service that is not within the hub’s services, this value must be a string of a valid port.
redis_password	string		No	The password for a Redis service that is not within the hub’s services, this value must be a string.
redis_tls	bool	false	No	Enables TLS for external Redis service configurations. This does not disable TLS when not configured as external, disabled by default.
redis_tls_ca	string		No	The absolute path to the root certificate for an external Redis service, this value must be a string.
redis_tls_cert	string		No	The absolute path to the client certificate for an external Redis service, this value must be a string.
redis_tls_key	string		No	The absolute path to the certificate key for an external Redis service, this value must be a string.
redis_cluster_hosts_json	string		No	The absolute path to a JSON file which maps Redis Cluster nodes’ internal addresses to external addresses. For use when Redis Cluster is configured as the result provider.
rabbitmq_host	string		No	The host for a RabbitMQ service that is not within the hub’s services, this value must be a valid IP or hostname.
rabbitmq_port	string	5671	No	The port for a RabbitMQ service both external and within the hub’s services, this value must be string of a valid port.
rabbitmq_vhost	string	nghub	No	The virtual host for a RabbitMQ service that both external and within the hub’s services, this value must a string.
rabbitmq_user	string	ngeneahub	No	The username for a RabbitMQ service that both external and within the hub’s services, this value must a string.
rabbitmq_password	string	ngeneahub	No	The password for a RabbitMQ service that both external and within the hub’s services, this value must a string.
rabbitmq_tls	bool	false	No	Enables TLS for external RabbitMQ service configurations. This does not disable TLS when not configured as external, disabled by default.
rabbitmq_tls_ca	string		No	The absolute path to the root certificate for an external RabbitMQ service, this value must be a string.
rabbitmq_tls_cert	string		No	The absolute path to the client certificate for an external RabbitMQ service, this value must be a string.
rabbitmq_tls_key	string		No	The absolute path to the certificate key for an external RabbitMQ service, this value must be a string.
rabbitmq_queue_type	string	classic	No	Queue type to use when creating queues in RabbitMQ, when there is only one rabbitmq_host. Either classic or quorum. Configure as quorum if you plan to scale out RabbitMQ.
celery_socket_timeout	int	60	No	The Redis celery broker supports a socket timeout, this value must be set as an integer whose value is the number of seconds.
celery_connection_timeout	int	60	No	The Redis celery broker supports a socket connection timeout, this value must be set as an integer whose value is the number of seconds.
gpfs_nodes	list		No	A list of nodes to run the snapdiff policy scan on as a list of hostnames
enable_plugins	bool	false	No	Key for Disabling/Enabling support for custom worker plugins, by default Disabled.
loglevel	str	INFO	No	Key for setting the worker loglevel (more specific to worker main process). valid choices are INFO, DEBUG, WARNING, CRITICAL, ERROR (by default INFO).

3.5.2. Controlling Functions¶

Ngenea Worker uses internal function queues to process tasks. These queues can be adjusted based on your needs.

What Are Function Queues ?

Function queues are like task managers that organize and run specific tasks. You can turn these on or off and adjust their settings.

How to Configure Function Queues ?

Open the configuration file (/etc/ngenea/ngenea-worker.conf.) for Ngenea Worker.
Modify settings to enable or disable queues.
Refer to the Workflow Steps documentation to see what each queue does.

3.5.3. Creating Custom Queues¶

If the default queues don’t meet your needs, you can create custom job queues. These allow you to prioritize specific tasks or allocate more resources.

Example Configuration for a Custom Queue

Steps to Add a Custom Queue

Open the worker’s configuration file (/etc/ngenea/ngenea-worker.conf.).
Add a new section with the name of the queue in brackets, like [Queue highpriority].
Specify the number of threads for that queue. (Optional)
Save and reload the worker service to apply changes.

For detailed information, check Queues Documentation.

3.5.4. Using a Custom Configuration File¶

Configuration files are text files used to set options and preferences for a system or application. Sometimes, you may need to override the default configuration with a custom one. Here’s how to set it up:

By default, the system uses the file located at:

/etc/ngenea/ngenea_config_arg.conf

How to Use a Custom Config File ?

Uncomment the CONFIG Line: Open the file /etc/ngenea/ngenea_config_arg.conf and remove the # symbol in front of this line:

CONFIG=/etc/ngenea/ngenea-worker-custom.conf

Create a Custom Worker Config File: Save a new file with a name like ngenea-worker-custom.conf. Use this file to define your settings.
Register Your Site: Run these commands to register the custom site:

ngeneahubctl addsite <sitename>
ngenea-worker join --user <username> --site <sitename>

3.5.5. RabbitMQ HA¶

Ngenea Hub supports the use of High Availability (HA) RabbitMQ, using Quorum Queues, as the broker provider.

RabbitMQ HA replicates queues to additional hosts to support failover in the event that one or more hosts go down.

To use RabbitMQ HA as the broker provider, configure

broker_provider=external_rabbitmq
rabbitmq_host=my-pixstor-mn01,my-pixstor-ng01,my-pixstor-ng02
rabbitmq_port=5671
# other rabbitmq_* settings

Configure all the hosts in the RabbitMQ cluster as a comma-separated list in rabbitmq_host. The same rabbitmq_port will used for all hosts by default. If each host uses a different port, the ports can be included in rabbitmq_host, for example host1:1234,host2:5678,...

If multiple hosts are configured, Hub will assume that queues should be quorum queues. If only one host is configured, Hub will assume the queues are non-quorum classic queues by default.

Note

Queues cannot change type once created. If you wish to move from a single-host to HA, existing ‘classic’ queues will need to be deleted and re-created as quorum queues.

If you have a new system with only one RabbitMQ host, but plan to switch to HA in future, configure rabbitmq_queue_type=quorum. This ensures that you will not need to delete and re-create your queues when switching to HA.

3.5.6. Redis Cluster¶

Ngenea Hub supports the use of Redis Cluster as a result provider.

Redis Cluster provides high availability (HA) and improved performance and throughput compared to single instance Redis.

When Redis Cluster is used as the result provider, configure

result_provider = external_redis_cluster
redis_host = my-pixstor-mn01
redis_port = 6001
# other redis settings

Where redis_host and redis_port point to any node within the Redis Cluster. The Redis client will automatically discover the other nodes in the cluster.

3.5.6.1. Redis Cluster Host Mapping¶

In a Pixstor environment, the nodes within a Redis Cluster will communicate with each other on the ‘backplane’ (internal) network. However, sites which aren’t within the same cluster network will not be able to access Redis Cluster via its internal addresses.

Pixstor provides external access to Redis Cluster via proxying. However, the Redis client needs to be told how to map its nodes’ internal addresses to the corresponding external proxy addresses.

The host mapping is a JSON formatted file, which defines the mapping of each node’s internal address to its external address

{
    "10.100.0.1:6000": "my-pixstor-mn01:6010",
    "10.100.0.2:6001": "my-pixstor-mn02:6011",
    "10.100.0.3:6002": "my-pixstor-sn01:6012"
}

The worker is then configured using redis_cluster_hosts_json, to tell the worker where the JSON file is located.

redis_cluster_hosts_json = /etc/ngenea/redis_cluster_hosts.json

When Redis Cluster can be reached on its internal addresses, the redis_cluster_hosts_json configuration can be omitted. Alternatively, the addresses can be mapped to their selves, or to null to indicate that no mapping is necessary.

The host mapping is also used to define ‘startup nodes’. Start up nodes are used as fallbacks, in the event that the configured redis_host is offline or not contactable when the worker starts up.

3.5.7. Running Ngenea Worker with Multiple Sites¶

Note: Running multiple sites on a node has several drawbacks, so using custom queues is preferred where possible.

To run services with two sites:

Create new service file same as /usr/lib/systemd/system/ngenea-worker.service with different filename (Eg: ngenea-worker-site1.service).
Create new custom config file same as /etc/ngenea/ngenea_config_arg.conf with different filename (Eg: ngenea_config_arg1.conf).
Create new worker config file same as /etc/ngenea/ngenea-worker.conf with different filename and give the filename in ngenea_config_arg1.conf.
In ngenea-worker-site1.service, change the value of EnvironmentFile=... to new custom config file created (Eg: EnvironmentFile=/etc/ngenea/ngenea_config_arg1.conf).
Now run systemctl start ngenea-worker.service and systemctl start ngenea-worker-site1.service.

3.5.8. Setting Ngenea Worker Debug Level in systemd Service Script¶

Environment Directive: In a system, there are background tasks that run automatically (called services). The `Environment` directive in the service’s setup tells it about certain settings it should use while running.

What Does `DYNAMO_DEBUG=true` Do?

Environment=DYNAMO_DEBUG=true sets a setting called `DYNAMO_DEBUG` to `true` for the service called ngenea-worker.

What Happens with This Setting?

Normally, the ngenea-worker service only logs basic information about what it’s doing (like regular updates).
When you set `DYNAMO_DEBUG=true`, the service will log extra, detailed information about its activities.