7.5. Automated Tasks¶
All detailed tasks below are run on automated schedules defined by the hub itself, some of these can have their schedules adjusted and thresholds adjusted.
7.5.1. Every minute¶
7.5.1.1. alerts.tasks.fetch_alerts¶
Submits the task dynamo.tasks.get_alerts to retrieve and sync the alerts on all sites with the Ngenea Hub's internal
database. This is run on the discovery function of the . The returning task that stores the
alerts in the database runs within the internal celery queue within Ngenea Hub.
The schedule of this task can be controlled by using the SYNC_ALERTS_INTERVAL config setting, any provided value must comply with cron syntax.
7.5.2. Every 30 minutes¶
7.5.2.1. dynamocore.tasks.admin.load_storagepools¶
Fetches the GPFS storage pools from all active sites registered within Ngenea Hub and stores them within the database. Additionally, it will also create the default filesystem root spaces if they don't exist for a site.
The schedule of this task can be controlled by using the REFRESH_SITE_ANALYTICS_INTERVAL config setting, any provided value must comply with cron syntax.
7.5.2.2. dynamocore.tasks.space_site.send_check_space_site_quota¶
Sends a task to the Ngenea Hub's internal celery queue to collect all spaces within every site named
dynamocore.tasks.space_site.send_check_space_site_quota which will then queue the worker task
dynamo.tasks.check_space_quotas on every site's worker function queue. It will then send a task back to the hub
celery queue to store the quotas in the database.
The schedule of this task can be controlled by using the SYNC_SPACES_QUOTA_INTERVAL config setting, any provided value must comply with cron syntax.
7.5.3. Hourly¶
7.5.3.1. dynamocore.tasks.jobs.update_state_as_failed_for_inactive_tasks¶
This task will assess all tasks currently that have been started in Ngenea Hub and will ensure that they are not lost due to a communication error.
Any tasks found to not be actively running within any of the defined queues will be marked as invalid after the
configurable time threshold task_invalidation_timeout in the configuration API.
The schedule of this task can be controlled by using the INACTIVE_TASKS_INTERVAL config setting, any provided value must comply with cron syntax.
7.5.3.2. dynamocore.tasks.jobs.invalidate_cancelled_job_tasks¶
After jobs are cancelled within the Ngenea Hub some tasks can be left in an un-cancelled state if the service un-expectedly exits leaving tasks in a non-cancelled state indefinitely. This task will automatically clean up these tasks that are in incorrect states.
The schedule of this task can be controlled by using the INVALIDATE_CANCELLED_JOB_TASKS_INTERVAL config setting, any provided value must comply with cron syntax.
7.5.3.3. dynamocore.tasks.network.ensure_consistent_tc_rules¶
With bandwidth controls enabled for any sites this task ensures that these rules are enacted on each site. Sites that are already compliant with their networking rules will not see any effect.
7.5.3.4. dynamocore.tasks.events.cleanup_old_events¶
This task automatically removes bidirectional sync events within the database. It will clear all objects for all but the latest 2 jobs along with any jobs that have been cancelled and failed
The schedule of this task can be controlled by using the CLEANUP_OLD_EVENTS_INTERVAL config setting, any provided value must comply with cron syntax.
7.5.3.5. dynamocore.tasks.browsing.browsing_task_clear¶
Removes all cached task results for API requests for file details for the file browser in the UI. It will remove all entries that are older than an hour along with any that received errors when retrieving results.
7.5.3.6. dynamocore.tasks.system_settings.create_or_update_site_settings_auto¶
This task starts up a task in the main celery queue within Ngenea Hub for updating or creating outstanding
settings across all active sites within the management of Ngenea Hub. This starts by executing
dynamocore.tasks.site.active_sites followed immediately by
dynamocore.tasks.system_settings.create_or_update_site_settings_for_sites. This will cause Ngenea Hub to
retrieve the settings for every site using dynamo.tasks.get_config_for_keys and update the hub database accordingly
using an additional task on the main celery queue within Ngenea Hub.
This call uses the main worker function queue with the , more details on queue functions are
available in the function configuration.
The schedule of this task can be controlled by using the SYNC_SITE_SETTINGS_INTERVAL config setting, any provided value must comply with cron syntax.
7.5.3.7. dynamocore.tasks.system_settings.sync_global_settings_auto¶
Starts up a task in the main celery within Ngenea Hub that creates a collection of tasks to retrieve the global
settings and external Ngenea targets within the hub database and apply them to every .
It achieves this with the Ngenea Worker task dynamo.tasks.set_config_and_apply this is run on the main worker
function.
The schedule of this task can be controlled by using the SYNC_GLOBAL_SETTINGS_INTERVAL config setting, any provided value must comply with cron syntax.
7.5.3.8. dynamocore.tasks.spaces.sync_spaces¶
When a space is created or updated and the job fails, this automated task schedules a retry attempt on making the space. If this is already being processed, it will not update the space within this interval.
The schedule of this task can be controlled by using the SYNC_SPACES_INTERVAL config setting, any provided value must comply with cron syntax.
7.5.4. Every 12 Hours¶
7.5.4.1. dynamocore.tasks.site.fetch_site_analytics¶
.. note:: This has an offset of 37 minutes to prevent overlapping system tasks.
Retrieves the site-wide analytics from all sites and then stores them within the Ngenea Hub database. The timeout
for this operation is controlled via the analytics_timeout in the configuration API. This
analytics task is run on the discovery function within the Ngenea Hub.
The schedule of this task can be controlled by using the REFRESH_SITE_ANALYTICS_INTERVAL config setting, any provided value must comply with cron syntax.
7.5.5. Daily¶
7.5.5.1. dynamocore.tasks.search.remove_old_search_results¶
This automated task removes all the old search result caches for searches submitted by the UI. The time period for this
lifetime is controlled by the search_result_ttl in the configuration API, the default for this
threshold is 7 days.
The schedule of this task can be controlled by using the REMOVE_OLD_SEARCH_RESULTS_INTERVAL config setting, any provided value must comply with cron syntax.
7.5.5.2. dynamocore.tasks.jobs.expire_old_jobs¶
This automated task removes all the old jobs (settings, workflows, scheduled) that have been submitted and processed within Ngenea Hub.
The threshold for this can be controlled by the jobs_ttl in the configuration API, the default
for this is 30 days.
The schedule of this task can be controlled by using the EXPIRE_OLD_JOBS_INTERVAL config setting, any provided value must comply with cron syntax.
7.5.5.3. dynamocore.tasks.network.update_cloud_ips_for_s3¶
Once per day, Ngenea Hub collects the IP addresses published by Amazon for their S3 service for uses in controlling the bandwidth of any S3 related operations going through Ngenea Worker.
If there are any changes in the address listings, this will be reported to the Ngenea Worker which will adjust any network interface it may be managing.
If bandwidth controls are not enabled, the changes will have no effect on any existing Ngenea Worker instances.
7.5.5.4. dynamocore.tasks.network.update_cloud_ips_for_azure¶
Once per day, Ngenea Hub collects the IP addresses published by Microsoft for their Azure Blob service for uses in controlling the bandwidth of any Azure related operations going through Ngenea Worker.
If there are any changes in the address listings, this will be reported to the Ngenea Worker which will adjust any network interface it may be managing.
If bandwidth controls are not enabled, the changes will have no effect on any existing Ngenea Worker instances.
7.5.5.5. dynamocore.tasks.network.update_cloud_ips_for_google¶
Once per day, Ngenea Hub collects the IP addresses published by Google for their GCS service for uses in controlling the bandwidth of any GCS related operations going through Ngenea Worker.
If there are any changes in the address listings, this will be reported to the Ngenea Worker which will adjust any network interface it may be managing.
If bandwidth controls are not enabled, the changes will have no effect on any existing Ngenea Worker instances.
7.5.5.6. dynamocore.tasks.server.get_remote_servers¶
Once per day Ngenea Hub has to retrieve the currently configured CTDB servers on each Ngenea Worker. It
achieves this through the task dynamo.tasks.get_servers on the worker function of each site.
This is then stored through the main Ngenea Hub celery queue through the dynamocore.tasks.server.store_server_callback
task.
The schedule of this task can be controlled by using the SYNC_REMOTE_SERVERS_INTERVAL config setting, any provided value must comply with cron syntax.
7.5.5.7. dynamocore.tasks.jobs.expire_old_fsobjects¶
Removes the stored long term file browser caching entries in the database and will expire any entries that have not been accessed within 7 days.
The schedule of this task can be controlled by using the EXPIRE_OLD_FSOBJECTS_INTERVAL config setting, any provided value must comply with cron syntax.
7.5.5.8. dynamocore.tasks.refresh.delete_removed_queues¶
Once queues are marked for deletion after being removed from a sites config file and the queue removal tool has been run, it will collect these queues and remove them along with and alerts linked to them.
The schedule of this task can be controlled by using the REMOVED_QUEUE_CLEANUP_INTERVAL config setting, any provided value must comply with cron syntax.