3.2. Hub Configuration¶
3.2.1. Settings¶
The main configuration file for Ngenea Hub is at /etc/sysconfig/ngeneahub.
This is an environment file which holds the information required for connecting
to the various backend services.
All of these mandatory settings are defaulted by Ngenea Hub if not present.
3.2.1.1. Mandatory Settings¶
Setting |
Description |
|---|---|
DJANGO_SECRET |
Secret string used secure signed data within django |
POSTGRES_DB |
Internal database name |
POSTGRES_USER |
Internal database username |
POSTGRES_PASSWORD |
Internal database password |
WORKER_THREADS |
The default number of celery concurrency threads to deal with multiple queued tasks for all celery based containers. This can be overwritten by container specific settings. Defaults to 2. |
DAG_THREADS |
The number of celery-dag container concurrency threads to deal with multiple queued DAG based tasks. Defaults to 7. More threads result in faster task resolutions for all job types and more frequent job updates. |
CELERY_BROKER |
Choice of broker must be either RABBITMQ or REDIS, defaults to REDIS |
RABBITMQ_USER |
Username for the rabbitmq broker, must be set regardless of CELERY_BROKER, defaults to ngeneahub |
RABBITMQ_PASSWORD |
Username for the rabbitmq broker, must be set regardless of CELERY_BROKER, defaults to ngeneahub |
RABBITMQ_VHOST |
Virtual hostname to be used for the rabbitmq broker, must be set regardless of CELERY_BROKER, defaults to nghub |
TASK_DAEMON_BATCH_SIZE |
Controls the amount of tasks to update the state of STARTED to once a DAG related job has been started. Defaults to 100 |
JWT_PRIVATE_KEY |
The private key to use for ALL JWTs used within the hub for authenticating requests, generates one on first startup |
JWT_PUBLIC_KEY |
The public key to use for ALL JWTs used within the hub for authenticating requests, generates one on first startup |
JWT_EXPOSED_JSON |
An exposed json containing details for JWT encryption, generates one on first startup |
GF_SECURITY_ADMIN_PASSWORD |
The grafana security password for the internal grafana, defaults on first startup |
HUB_PORT |
User configurable hub port, defaults to 8000 |
WEB_BIND_IP |
User configurable web bind IP, defaults to 0.0.0.0 |
SHARED_SECRET |
Secret key shared internally within the hub services for TLS operations, generated on first startup |
DJANGO_ENCRYPTED_FIELDS_KEY |
Defines the key for encrypting keys within the database, generated on first startup |
DJANGO_EJF_CRYPTER_KEY |
Additional cryptography key for obfuscating django responses, generated on first startup |
3.2.1.2. Optional settings¶
Setting |
Description |
|---|---|
REDIS_HOST |
Address of the Redis queue results store. Defaults to the container service address. |
WORKERS |
The number of gunicorn workers to spawn for serving API requests. Default to 8. |
API_TIMEOUT |
The timeout for requests against the API server, in seconds. Default: 600 |
GATEWAY_TIMEOUT |
The timeout for requests going through nginx, in seconds. This should be set to greater or equal than API_TIMEOUT. Default: 600 |
CONSUMER_TIMEOUT |
The timeout for rabbitmq consumer delivery acknowledgement in seconds. Default: 10800000 (3 hours) |
PUBLIC_URL |
User configurable base url for the hub stack to be served from, must not end in a trailing slash. |
CELERY_THREADS |
The number of main celery container concurrency threads to deal with multiple queued internal tasks. More threads means that more internal tasks and system tasks can run in parallel. |
EVENT_THREADS |
The number of celery-events container concurrency threads to deal with multiple queued event reporting tasks. More threads ensure that snapdiff events can be reported to the hub faster. |
RESULT_THREADS |
The number of celery-results container concurrency threads to deal with multiple queued result reporting tasks. More threads mean streamed task results can report the the hub faster. |
HEARTBEAT |
(bool) Key for Disabling/Enabling celery heartbeats, default: true (enabled) |
GOSSIP |
(bool) Key for Disabling/Enabling celery gossip, default: false (disabled) |
MINGLE |
(bool) Key for Disabling/Enabling celery mingle, default: false (disabled) |
REDIS_HEALTH_CHECK_INTERVAL |
(int) The Redis backend supports health checks. This value must be set as an integer whose value is the number of seconds between health checks. default: 60 |
REDIS_TCP_BACKLOG |
(int) In high requests-per-second environments you need a high backlog in order to avoid slow client connections issues to redis. Default: 511 |
REDIS_SOCKET_TIMEOUT |
(int) When there are network issues redis backend connection sockets can become stale, this timeout setting will reset the socket connection after this value in seconds after becoming idle and resume operation. Default: 60 |
CELERY_SOCKET_TIMEOUT |
(int) When there are network issues redis broker sockets can become stale, this timeout setting will re-acquire the socket after becoming idle for this value in seconds and resume operation. Default: 60 |
CELERY_CONNECTION_TIMEOUT |
(int) When there are network issues redis broker connection via the acquired sockets can become stale, this timeout setting will reset the connection after becoming idle for this value in seconds and resume operation. Default: 60 |
EXPIRE_OLD_JOBS_INTERVAL |
(cron) schedule for when old job expiration will be run. When the task runs, jobs older than the configured jobs_ttl will be expired. Default: 0 0 * * * (minutes can be random from 0-59) |
REMOVE_OLD_SEARCH_RESULTS_INTERVAL |
(cron) schedule for when search result removal will run. When the task runs, search results older than the configured search_result_ttl will be expired. Default: 0 0 * * * (minutes can be random from 0-59) |
INVALIDATE_CANCELLED_JOB_TASKS_INTERVAL |
(cron) schedule for when cancelled jobs are revoked. When the task runs, any tasks still active in a cancelled job will be automatically cancelled. Default: 0 * * * * (minutes can be random from 0-59) |
CLEANUP_OLD_EVENTS_INTERVAL |
(cron) schedule for when old snapdiff events will be cleaned up. When the task runs, events for all but the 2 most recent completed snapdiff jobs per workflow will be deleted. Default: 0 * * * * (minutes can be random from 0-59) |
INACTIVE_TASKS_INTERVAL |
(cron) schedule for when inactive tasks will be invalidated. When the task runs, any STARTED task which is not actually running in a worker will be marked as FAILED. Default: 0 * * * * (minutes can be random from 0-59) |
SYNC_SITE_SETTINGS_INTERVAL |
(cron) schedule for when sync of the site settings will be run. When the task runs, the site settings will be created or updated in the DB. Default: 0 * * * * (minutes can be random from 0-59) |
SYNC_GLOBAL_SETTINGS_INTERVAL |
(cron) schedule for when sync of the global settings will be run. When the task runs, the global settings will be sent to sites to be in sync. Default: 0 * * * * (minutes can be random from 0-59) |
REFRESH_SITE_ANALYTICS_INTERVAL |
(cron) schedule for when refresh of the site analytics will be run. When the task runs, refresh of site analytics will be triggered. Default: 37 */12 * * * |
SYNC_STORAGE_POOLS_INTERVAL |
(cron) schedule for when sync of the storage pools will be run. When the task runs, storage pools will be synced. Default: */30 * * * * |
SYNC_REMOTE_SERVERS_INTERVAL |
(cron) schedule for when sync of the remote servers will be run. When the task runs, remote servers will be synced. Default: 0 0 * * * (minutes can be random from 0-59) |
SYNC_SPACES_QUOTA_INTERVAL |
(cron) schedule for when sync of the spaces' quotas will be run. When the task runs, spaces' quotas will be synced. Default: */30 * * * * |
SYNC_SPACES_INTERVAL |
(cron) schedule for when sync of spaces will be run. When the task runs, spaces will be synced. Default: 0 * * * * (minutes can be random from 0-59) |
SYNC_ALERTS_INTERVAL |
(cron) schedule for when sync of alerts will be run. When the task runs, alerts will be fetched from all sites. Default: * * * * * |
EXPIRE_OLD_FSOBJECTS_INTERVAL |
(cron) schedule for when old fsobjects expiration will be run. When the task runs, fsobjects older than the configured fsobjects_ttl will be expired. Default: 0 0 * * * (minutes can be random from 0-59) |
3.2.2. Enabling LDAP/Active Directory Authentication¶
To enable LDAP/Active Directory Authentication, provide the following settings in the /etc/sysconfig/ngeneahub configuration file.
Setting |
Description |
|---|---|
LDAP_ENABLED |
(bool) Key for Disabling/Enabling LDAP/Active Directory Authentication. default: false (disabled) |
LDAP_HOSTNAME |
The hostname of the LDAP/AD server |
LDAP_USER_SEARCH |
An LDAPSearch object that identifies the set of relevant user objects. E.G. cn=Users,dc=MYDOMAIN,dc=MYCOMPANY,dc=COM |
LDAP_GROUP_SEARCH |
An LDAPSearch object that identifies the set of relevant group object. E.G. ou=Security,ou=OurGroups,dc=MYCOMAIN,dc=MYCOMPANY,dc=COM |
LDAP_MIRROR_GROUPS |
(bool) If AUTH_LDAP_MIRROR_GROUPS is True, then every time a user logs in, LDAPBackend will update the database with the user’s LDAP groups. default: false (disabled) |
LDAP_ALWAYS_UPDATE_USER |
(bool) If AUTH_LDAP_ALWAYS_UPDATE_USER is True, the fields of the user object will be updated with the latest values from the LDAP directory everytime the user logs in. otherwise, it will be populated only when it is automatically created. default: true (enabled) |
LDAP_AUTHORIZE_ALL_USERS |
(bool) If AUTH_LDAP_AUTHORIZE_ALL_USERS is True, LDAPBackend will be able to furnish permissions for any Django user, regardless of which backend authenticated it. default: false (disabled) |
LDAP_BIND_AS_AUTHENTICATING_USER |
(bool) If AUTH_LDAP_BIND_AS_AUTHENTICATING_USER is True, authentication will leave the LDAP connection bound as the authenticating user, rather than forcing it to re-bind with the default credentials after authentication succeeds. Remember to set AUTH_LDAP_USER_DN_TEMPLATE to avoid initial connection to LDAP with default bind credentials. default: false (disabled) |
LDAP_REFRESH_DN_ON_BIND |
(bool) If AUTH_LDAP_REFRESH_DN_ON_BIND is True and AUTH_LDAP_BIND_AS_AUTHENTICATING_USER is True and AUTH_LDAP_USER_DN_TEMPLATE is set, after performing bind login it refresh the DN attribute of the user. default: false (disabled) |
LDAP_CACHE_TIMEOUT |
The value determines the amount of time, in seconds, a user’s group memberships and distinguished name are cached. default: 0 |
LDAP_CONNECTION_OPTIONS |
A dictionary of options to pass to each connection to the LDAP server via LDAPObject.set_option().default: {} |
LDAP_DENY_GROUP |
The distinguished name of a group; authentication will fail for any user that belongs to this group. default: None |
LDAP_FIND_GROUP_PERMS |
(bool) If True, LDAPBackend looks up Django Groups matching LDAP group names, and assigns user permissions based on the Django Group permissions. AUTH_LDAP_GROUP_SEARCH and AUTH_LDAP_GROUP_TYPE must also be set. default: false (disabled) |
LDAP_GLOBAL_OPTIONS |
A dictionary of options to pass to ldap.set_option(). Keys are ldap.OPT_* constants. default: {} |
LDAP_MIRROR_GROUPS_EXCEPT |
It must be a list or other collection of group names. This will enable group mirroring, except that it will never change the membership of the indicated groups. default: None |
LDAP_PERMIT_EMPTY_PASSWORD |
(bool) If True, authentication will be allowed with empty password. default: false (disabled) |
LDAP_REQUIRE_GROUP |
The distinguished name of a group, authentication will fail for any user that does not belong to this group. default: None |
LDAP_NO_NEW_USERS |
(bool) Prevent the creation of new users during authentication. Any users not already in the Django user database will not be able to login. default: false (disabled) |
LDAP_START_TLS |
(bool) If True, each connection to the LDAP server will enable TLS encryption over the standard LDAP port. default: false (disabled) |
LDAP_USER_QUERY_FIELD |
when set, it is used to query the authenticating user in the database. If unset, it uses the username. default: None |
LDAP_USER_ATTRLIST |
A list of operational attributes to load for the authenticated user. example- AUTH_LDAP_USER_ATTRLIST = ["*", "+"]. default: None |
LDAP_USER_DN_TEMPLATE |
A string template that describes any user’s distinguished name based on the username. This must contain the placeholder %(user)s. default: None |
LDAP_USER_FLAGS_BY_GROUP |
A mapping from boolean User field names to distinguished names of LDAP groups. The corresponding field is set to True or False according to whether the user is a member of the group. default: {} |
LDAP_DOMAIN |
This is the domain name of the LDAP server. default: None. When provided and LDAP_USER_DN_TEMPLATE is not provided, we attempt to interpolate like so %(user)s@%(domain)s. Also when LDAP_USER_SEARCH and/or LDAP_GROUP_SEARCH are provided, we attempt to use the domain to work it out in the form dc=domain,dc=com. |
3.2.3. Broker Settings¶
See Hub Messaging Queue Configuration for more broker details.
3.2.4. Server Configurations¶
Some settings are stored in the Ngenea Hub DB.
They can be viewed and changed via the REST API /api/configurations/ endpoint.
See Configuration for more details.
3.2.5. Docker Compose configuration¶
The docker-compose file is stored in /usr/share/ngeneahub/docker/docker-compose.yml.
This can be extended by creating an override file at /usr/share/ngeneahub/docker/docker-compose.override.yml.