Release notes - Ngenea Hub - 2.1.0: 2023-12-22
==============================================
Feature
-------
Ability to configure Ngenea Backup
Ability to manage NAS users and groups
Ability to view disk metrics
New storage info page
New Alerts UI
Release notes - Ngenea Hub - 2.0.2: 2023-11-17
==============================================
Feature
-------
New Search page, integrating PixStor Search into Hub
New Policies page, to configure and run data migration policies from the Hub UI
HUB2-27 - Prevent access to UI components according to user permissions
HUB2-660 - Display all GPFS filesystems in Site Dashboard
HUB2-691 - Ability to give space shares custom names
HUB2-692 - Ability to create multiple shares per space at different paths
HUB2-754 - API to fetch settings tasks list for a job
HUB2-791 - Global view of Jobs in the UI
HUB2-792 - Support for submitting custom workflows in the UI
HUB2-823 - Support for setting analytics and metrics URL from UI
HUB2-828 - Ability to specify target path for Space creation
HUB2-831 - Filesystems API endpoint
HUB2-1003 - System folders are hidden in the UI
HUB2-1014 - Ability to bookmark and link locations in the file browser
HUB2-1043 - Ability to view job file lists in job details page rightbar
HUB2-1071 - Added DNS search domains to Site settings
HUB2-1075 - Support for editing job Schedules in the UI
Improvement
-----------
HUB2-370 - Improved progress bars proportion in job details page
HUB2-527 - Sensitive settings such as ngenea secret keys are hidden from the UI and API
HUB2-785 - Include share path in file list for share jobs
HUB2-795 - Jobs page view persists when refreshing the page
HUB2-825 - All members of the group can be viewed via the "view all members" button
HUB2-833 - Improved UI for settings Job details
HUB2-834 - Adds user-friendly job descriptions
HUB2-835 - Snapshot time can only be set when the frequency is greater than 1 day
HUB2-861 - Ability to search for a job by ID on jobs page
HUB2-905 - Updated storage target type naming in ngenea wizard
HUB2-915 - "view task" modal is now full width in the jobs page
HUB2-927 - Users list is sorted alphabetically in the groups tab
HUB2-951 - Server address is now optional when creating S3 ngenea targets
HUB2-1025 - Tooltip for long values in the details sidebar
HUB2-1030 - Validation notice for Samba shares in the Space wizard
HUB2-1070 - Don't allow setting server names longer than NETBIOS allowed limit
HUB2-1124 - Updated labels and icons for default workflows
Bug
---
HUB2-595 - Create space parent directory if it doesn't already exist
HUB2-783 - Set owner for automated settings tasks to 'system'
HUB2-796 - Exclude certain interface types from site NAS settings
HUB2-805 - Filters not working on Users or Groups pages
HUB2-816 - Workflow file list selected whole space instead of individual files
HUB2-832 - Validation for space placement pools
HUB2-854 - Folder metadata not loading in the file browser
HUB2-856 - Fix settings page link for when a custom base URL is configured
HUB2-931 - None values being written to samba shares
HUB2-932 - Multiple delete jobs being triggered for the same share
HUB2-933 - Don't force changing site shortcode on create site wizard
HUB2-935 - Broken spaces page when only one site is configured
HUB2-949 - Go back button for "advanced configuration" breaks UI on ngenea target wizard summary
HUB2-953 - Add api_secure_verify support to ngclient
HUB2-1028 - Site settings weren't updated in the UI after applying a change
Release notes - Ngenea Hub - 2.0.1: 2023-09-19
==============================================
Bug
---
HUB2-931: Don't write None values to samba shares
Documentation
-------------
HUB2-673: Updated user docs for Ngenea Hub 2.0
Release notes - Ngenea Hub - 2.0.0: 2023-08-11
==============================================
Feature
-------
Initial Release:
- Hub 2.0 UI
- Provide the ability to view pixstor sites and storage
- Provide the ability to configure pixstor sites from the Hub UI
- Provide the ability to create Spaces
- Provide the ability to view files globally across all Space associated sites
- Provide the ability to perform data workflows
- Provide the ability to monitor jobs and tasks
- Provide the ability to manage Hub users and groups
- Support for Active Directory/LDAP logon
- Worker support of Hub 2.0 functions
- Analytics data
- Get and Set pixstor settings for a site
- Autojoin to Hub
Release notes - Ngenea Hub - 1.28.1: 2023-11-23
===============================================
Improvement
-----------
DYNAMOENG-1692: Increase Postgres working memory to reduce disk IO
Bug
---
DYNAMOENG-1691: Database transaction error during stuck task invalidation
Release notes - Ngenea Hub - 1.28.0: 2023-11-17
===============================================
Improvement
-----------
DYNAMOENG-1610 - Workflow names and labels can now be much larger
DYNAMOENG-1650 - Added a flag to move_paths_on_gpfs to allow the task to abort missing files
DYNAMOENG-1666 - Skipped jobs are now automatically cleared with other inactive jobs
Bug
---
DYNAMOENG-1582 - Empty directory ACL syncing is now working correctly in bidirectional sync
DYNAMOENG-1617 - Unexpected task results from transparent recalls now correctly report in the hub
DYNAMOENG-1644 - ngclient now correctly updates the discovery_options in workflow definition
DYNAMOENG-1679 - An error message when accessing files outside gpfs on the browser in the worker has been removed to reduce noise
DYNAMOENG-1681 - Inactive tasks are now correctly invalidated after the defined threshold
Release notes - Ngenea Hub - 1.27.1: 2023-08-24
===============================================
Bug
---
DYNAMOENG-1561 - Reverse stub error when syncing metadata for stub files
Release notes - Ngenea Hub - 1.27.0: 2023-08-16
===============================================
Improvement
-----------
DYNAMOENG-1511 - An additional option for bypassing SSL validation has been added to ngclient
DYNAMOENG-1491 - Snapdiff workflows now include the discovery_option condense_moves
DYNAMOENG-1481 - Plugin tasks can now add to the total stats for a job
DYNAMOENG-1500 - Running plugin installs now always installs the latest version of the plugin code
DYNAMOENG-1521 - Job filters now time out after 30 seconds instead of 10
DYNAMOENG-1535 - path_prefix in the job filters is now input_path_prefix and is significantly faster
DYNAMOENG-1445 - Plugins can now be deleted on uninstall
DYNAMOENG-1495 - Task log messages when a job fails due to a system link error have been improved
DYNAMOENG-1514 - Snapdiff task manager now waits 10 minutes by default for a task to start instead of 1 minute and can be configured
Bug
---
DYNAMOENG-749 - File endpoint now correctly returns 404 if a path does not exist
DYNAMOENG-1051 - Recursive delete workflows now only take one run to remove all data
DYNAMOENG-1473 - Snapdiff task manager task now correctly waits for the correct amount of events
DYNAMOENG-1479 - skip_old_ctimes in a snapdiff workflow now apply to moved files
DYNAMOENG-1489 - PATCH operations on a workflow now correctly allow you to change discovery_options
DYNAMOENG-1522 - The systemd service for ngenea-worker now correctly restarts after all the threads exit completely
DYNAMOENG-1536 - The API no longer returns 500 when an empty set is returned
DYNAMOENG-1531 - Offline files now have all of their ngenea metadata updated during reverse_stub
Release notes - Ngenea Hub - 1.26.2: 2023-07-05
===============================================
Bug
---
DYNAMOENG-1509 - Fixed containers not being able to initiate new threads preventing websocket connections
Release notes - Ngenea Hub - 1.26.1: 2023-06-30
===============================================
Bug
---
DYNAMOENG-1499 - Fixed issue with bidirectional_sync where it would not run the second site on a task error
Release notes - Ngenea Hub - 1.26.0: 2023-06-22
===============================================
Feature
-------
DYNAMOENG-897 - The worker can now be passed a path to a different configuration file via the CLI
DYNAMOENG-1440 - Users can now control all of the automated cleanup tasks that run in the background of the hub
DYNAMOENG-1451 - A new field has been added to workflow definition named "discovery_options" to control discovery behaviour
DYNAMOENG-1468 - Alpha plugin support can now be enabled within the worker
Improvement
-----------
DYNAMOENG-857 - Snapdiff snapshot tracking files now includes the fileset ID
DYNAMOENG-1352 - recursive_action now adds all found directories to its total file count
DYNAMOENG-1474 - Ngenea based worker tasks now support "moved" snapdiff rules
DYNAMOENG-1483 - Added a flag to abort a file when the file is missing for the ctime check
Bug
---
DYNAMOENG-1224 - There is no longer a "could not read properties of undefined" error after clicking search results
DYNAMOENG-1242 - Jobs now do not require to be cancelled twice in snapdiff workflows
DYNAMOENG-1457 - Unexpected files are no longer created during a bidirectional sync
DYNAMOENG-1469 - Files are correctly filtered out from a bidirectional sync when there are old jobs without a started time
DYNAMOENG-1459 - Files created and renamed during a sync are not longer absent on the target site
DYNAMOENG-1462 - Files with an old ctime are synced by default with a toggle to enable filtering them
DYNAMOENG-1480 - Migrate tasks will no longer fail when a message was already within the path details
Release notes - Ngenea Hub - 1.25.0: 2023-06-07
===============================================
Feature
-------
DYNAMOENG-203 - The Ngenea logging level can now be increased for Ngenea based tasks
DYNAMOENG-467 - Job details can now be retrieved by clicking the pie graph
DYNAMOENG-1395 - Recalls can now be called on files that do not exist on the filesystem
DYNAMOENG-1452 - The hub ngenea tasks now support --sync-metadata
DYNAMOENG-1456 - All ngenea tasks now support --fail-on-mismatch
DYNAMOENG-1454 - Cloud SDK based tasks can now specify one ngenea target to operate on
Improvement
-----------
DYNAMOENG-1055 - Skipped tasks now clearly state the paths skipped when there is no work to be performed
DYNAMOENG-1376 - Check sync state now checks the xattrs, acls and other metadata when comparing changes on directories
DYNAMOENG-1488 - Workflow field lists can now be edited within the UI at runtime
DYNAMOENG-1042 - There are no longer un-needed warnings during snapdiff operations
Bug
---
DYNAMOENG-1201 - Duplicate results are no longer created under certain sync workflows
DYNAMOENG-1351 - Task resulting in the error "reversely stubbed object does not exist" no longer occur with xattrs containing "max"
DYNAMOENG-1453 - Cloud SDK based worker tasks now work with all discovery types
DYNAMOENG-1467 - Cloud SDK based tasks now correctly map matching paths to all ngenea targets
Release notes - Ngenea Hub - 1.24.0: 2023-05-09
===============================================
Feature
-------
DYNAMOHUB-1341 - The Ngenea worker can now make use of multiple GPFS nodes for the snapdiff policy and configurable threads
DYNAMOHUB-1372 - check_sync_state now has a flag to allow files to result in the aborted state instead of skipped
Improvement
-----------
DYNAMOHUB-944 - Reverse stub now respects the sync_preference during bi-directional syncs
DYNAMOHUB-1391 - The lambda function documentation has been updated and improved
DYNAMOHUB-1400 - Task specific API keys are now more robust
DYNAMOHUB-1413 - ngenahubctl wipe now correctly removes volumes
DYNAMOHUB-1416 - The hydrate argument in tasks within bi_directionalsync is now a run time field
Bug
---
DYNAMOHUB-1061 - The Ngenea worker no longer goes into a failed state when its systemd service is stopped
DYNAMOHUB-1119 - remove_location_xattrs_for_moved now errors correctly when paths do not match the regex provided for any Ngenea policy are provided
DYNAMOHUB-1167 - Revoked is no longer a state that a task can enter when cancelled
DYNAMOHUB-1176 - Hub containers no longer start on boot after the service is disabled
DYNAMOHUB-1397 - The celery monitor container now correctly exits
Release notes - Ngenea Hub - 1.23.0: 2023-04-12
===============================================
Improvement
-----------
DYNAMOHUB-1347 - Files now have their change time checked before a delete to ensure the file is not deleted after being edited
Bug
---
DYNAMOHUB-934 - Transparent recall jobs can now be correctly filtered
DYNAMOHUB-961 - Tasks can no longer have a negative run time
DYNAMOHUB-1056 - Certain characters are no longer parsed incorrectly that resulted in incorrect paths in jobs
DYNAMOHUB-1023 - The snapdiff task now correctly parses backslash escaped paths
DYNAMOHUB-1124 - Surrogate characters in paths are no longer causing exceptions and are instead no longer supported
DYNAMOHUB-1168 - Character special files no longer cause the websocket logic to error
DYNAMOHUB-1360 - UI schedule field now correctly supports offset mode
DYNAMOHUB-1363 - Internal hub processing directories are now ignored by the recursive action discovery
DYNAMOHUB-1369 - Broken symlinks can now be deleted and moved in a bidirectional sync
DYNAMOHUB-1384 - The UI no longer crashes when selecting a path for a recursive schedule
DYNAMOHUB-1385 - Empty directories are no long sent back when both sites in a bidirectional sync alter the same directory
Release notes - Ngenea Hub - 1.22.0: 2023-03-28
===============================================
Feature
-------
DYNAMOHUB-1342 - New single step move task: dynamo.tasks.one_step_move_paths_on_gpfs
Improvement
-----------
DYNAMOHUB-1361 - Optimised the migration process for pre-1.21 releases to stop migrations taking multiple hours while drastically lowering memory usage
DYNAMOHUB-1359 - Authentication errors on the task files endpoint now have more verbose logging
Bug
---
DYNAMOHUB-1202 - Bidirectional sync tasks that run on a different target site now report their site correctly
DYNAMOHUB-1309 - Non bidirectional snapdiff scheduled tasks no longer attempt to run while one is on-going
DYNAMOHUB-1349 - Excluded or included files not longer cause "Received 0 of the expected 1 results" in move or delete tasks
DYNAMOHUB-1371 - Fixed issue with file move and delete task keys in non bidirectional sync runs
Release notes - Ngenea Hub - 1.21.0: 2023-03-09
===============================================
Feature
-------
DYNAMOHUB-527 - Symlinks are now supported by the recursive discovery
Improvement
-----------
DYNAMOHUB-1343 - Relative symlink moves are now supported
Bug
---
DYNAMOHUB-1336 - Directory symlinks are now correctly deleted
DYNAMOHUB-1186 - The deletion of a subscribed workflow no longer causes it to run anyway
DYNAMOHUB-1356 - Deletes are no longer run on both sites during certain conditions in a bidirectional_sync
DYNAMOHUB-1215 - Sites can now correctly be renamed
Release notes - Ngenea Hub - 1.20.0: 2023-02-24
===============================================
Feature
-------
DYNAMOHUB-746 - The processed files list can now be downloaded in full in JSON
Bug
---
DYNAMOHUB-1320 - Tasks that log "critical: watchdog timeout expired" now log the error the correct amount of times
DYNAMOHUB-1286 - Tasks that encounter files that fail with "unexpected end of file" now correctly abort the file operation
DYNAMOHUB-1327 - Tasks stuck in pending due to their parent task not having results now resolve themselves after 30 minutes
Release notes - Ngenea Hub - 1.19.0: 2023-02-23
===============================================
Features
--------
DYNAMOHUB-1267 - There are now more connection timeout settings for robust redis broker and results connections
Improvements
------------
DYNAMOHUB-1314 - Each move and delete operation is now fully logged within the worker
DYNAMOHUB-1316 - Celery workers have had an improvement to how they handle latent connections
DYNAMOHUB-1160 - Synced file and directory deletes now do not replay deletes in given circumstances
DYNAMOHUB-1312 - Snapshot operations are now retried if they fail during all snapdiff tasks
Bug
---
DYNAMOHUB-1319 - Fixed "FileNotFoundError" from passing to further tasks
DYNAMOHUB-1315 - Files that produce "no such file or directory" now correctly abort during a migrate
DYNAMOHUB-1300 - Fixed GCS reverse_stub issue due to "Too many open files"
DYNAMOHUB-1328 - Fixed issue caused by REDIS_HEALTH_CHECK_INTERVAL in the hub
Release notes - Ngenea Hub - 1.18.0: 2023-02-15
===============================================
Features
--------
- DYNAMOHUB-617: Multiple jobs can now be restarted on the jobs page
- DYNAMOHUB-848: All redis communication now uses TLS
- DYNAMOHUB-932: Clicking outside of a model now closes that modal
- DYNAMOHUB-964: There is now an automatic cleanup of old jobs
- DYNAMOHUB-1161: Multiple state filters can now be chosen on the job details page
- DYNAMOHUB-1173: dynamo.tasks.recall now supports the delete_remote argument
- DYNAMOHUB-1197: Users can now use the API to retrieve the files that have been processed without the full payload
- DYNAMOHUB-1205: The hub now uses redis for its broker instead of rabbitmq
- DYNAMOHUB-1213: Sync now supports directories for all its operations
- DYNAMOHUB-1241: Grafana now reports on the redis metrics
Improvements
------------
- DYNAMOHUB-643: Job statistics now take cancelled tasks in their totals
- DYNAMOHUB-1153: The docker shared memory now has a different default for the database
- DYNAMOHUB-1191: Moved file/folder conflict resolution has been made more robust
- DYNAMOHUB-1195: Syncs will now perform actions delete, move and then other operations in that order
- DYNAMOHUB-1245: Celery workers can now toggle the heartbeat, mingle and gossip behaviours
- DYNAMOHUB-1255: The gossip, mingle and heartbeat are now enabled by default instead of disabled
- DYNAMOHUB-1258: redis_backend_health_check_interval can now be configured for the hub's redis service
- DYNAMOHUB-1260: Ngeneahub now makes use of ngenea 1.20
- DYNAMOHUB-1291: REDIS_TCP_BACKLOG can now be configured for the hub's redis service
- DYNAMOHUB-1305: Optimise the storing of task results for faster sync
- DYNAMOHUB-1306: Removed redundant 'Number of directories' from job details page
- DYNAMOHUB-1315: Mark "no such file or directory" as 'aborted' in migrate and reverse stub tasks
Bug
---
- DYNAMOHUB-707: Snapdiff workflows without any additional work no longer show as having a pending file
- DYNAMOHUB-1099: The jobs page now loads signifcantly faster when there are a large volume of jobs
- DYNAMOHUB-1142: Migration tasks no longer fail due to "no such file or directory"
- DYNAMOHUB-1148: Syncs not correctly filter out previously completed directory moves
- DYNAMOHUB-1151: Directory moves are now performed in order
- DYNAMOHUB-1175: The jobs details page no longer attempts to load the failed files list unprompted
- DYNAMOHUB-1216: Syncs can now be cancelled and it will roll back the snapshot correctly
- DYNAMOHUB-1220: Create and complete time filters now work as expected on the job list
- DYNAMOHUB-1225: Syncs now correctly rotate snapshots when all tasks are completely successful
- DYNAMOHUB-1227: Schedule deletion no longer times out
- DYNAMOHUB-1230: Recall resulting in "deletion of remote objects failed" no longer causes job failures
- DYNAMOHUB-1248: Complex sync operations no longer cause errors in dynamo.tasks.filter_snapshot_results
- DYNAMOHUB-1249: Fixed race condition of task completing too quickly to report the results from redis
- DYNAMOHUB-1259: Syncs now correctly tracks the amount of input files
Release notes - Ngenea Hub - 1.17.3: 2023-02-09
===============================================
Improvement
-----------
- Ability to configure RabbitMQ max_message_size (DYNAMOHUB-1221)
Bug
---
- Migration failure due to "no such file or directory" from ngenea (DYNAMOHUB-1285)
- Reverse stub failure due to TypeError (DYNAMOHUB-1294)
- Migration failure due to "no such file or directory" when checking ctime (DYNAMOHUB-1302)
Release notes - Ngenea Hub - 1.17.0: 2022-12-07
===============================================
Improvement
-----------
- Shareable URLs that navigate to a specific position in the filebrowser tree (DYNAMOHUB-650)
- Search filters that require a timestamp input now present a date-picker UI (DYNAMOHUB-652)
- Improve search filters interface to display only relevant operators (DYNAMOHUB-653)
- New configuration UI for global settings (DYNAMOHUB-696)
- Support for placing the web interface behind a reverse proxy \(e.g. SSL termination\) and customising the base URL (DYNAMOHUB-813)
- Remember filter setting for "Hide successful jobs with no processed files" between pages with job lists (DYNAMOHUB-906)
- Subscribed workflows in the schedule UI now display in alphabetical order (DYNAMOHUB-909)
- Allow the resubmit button to be used on cancelled jobs (DYNAMOHUB-938)
- Improve websocket connection robustness by reusing the same connection between pages (DYNAMOHUB-982)
- Improve advice on transparent recalls regarding space and quote limitiations (DYNAMOHUB-998)
- Improve messaging in the file browser view when only hidden files are in a directory. (DYNAMOHUB-1000)
- UI Interface for linking sites to datastores (DYNAMOHUB-1008)
- Support symlinks in default workflows and tasks (DYNAMOHUB-1018)
- Prevent administrative task \(cleanup\_old\_events\) from running multiple instances at the same time and improve performance (DYNAMOHUB-1038)
- Add support to synchronise empty directories via site-sync (DYNAMOHUB-1090)
- Refreshing the job details page will keep any currently applied filters (DYNAMOHUB-1096)
- Provide support for ngenea --sync-metadata option for syncing metadata without re-transmitting data (DYNAMOHUB-1101)
- Improve job filtering by setting sensible defaults for job states (DYNAMOHUB-1104)
- Improve memory usage when refreshing large jobs (DYNAMOHUB-1130)
- Improved logging when encountering large celery payloads: "message size XXX is larger than configured max size YYY" (DYNAMOHUB-1132)
Bug
---
- "Cannot read properties of null" errors when opening jobs (DYNAMOHUB-700)
- Console constantly outputting web socket connection errors (DYNAMOHUB-701)
- Fixed an issue where ngenea workers would unnecessarily set bandwidth controls when no changes have been made. (DYNAMOHUB-855)
- Fixes an issue where duplicate items appear in the file browser when they are created on the file system (DYNAMOHUB-1002)
- Fixes an issue where some unicode code points in a path could cause the snapdiff discovery task to fail (DYNAMOHUB-1028)
- Could not filter tasks in a job with a large number of tasks (DYNAMOHUB-1040)
- Gracefully handle errors if a file is deleted during a migration (DYNAMOHUB-1059)
- Clean-up stale internal snapdiff related data \(ObjectEvents\) from failed snapdiff jobs (DYNAMOHUB-1094)
- Discrepancy between hwdev and hwdev03 if a mv is run one one site while a rm is run on the other (DYNAMOHUB-1100)
- Fixes an issue in the Job summary filters where the job type filter dropdown could render incorrectly (DYNAMOHUB-1108)
- Fixes an issue where passing a symlink to a migrate step would cause an error (DYNAMOHUB-1110)
- Fixes an issue where filtering search results on sites would reset unexpectedly (DYNAMOHUB-1112)
- Fixes an issue in remove\_location\_xattrs\_for\_moved workflow step, which causes an error if the file is deleted during processing (DYNAMOHUB-1117)
- Fixes an issue where tasks in a large job could be marked as failed due to a race condition with the lost message detection logic (DYNAMOHUB-1120)
- Fixes an issue in the migrate step where an erroneous message "duplicate file skipped" could be logged as a file error is the file changed multiple times in quick succession (DYNAMOHUB-1131)
- Upgrade of worker threw error likely in trying to modify worker config (DYNAMOHUB-1149)
- Fixes an issue that prevented users re-logging in after logging out in the same browser session (DYNAMOHUB-1159)
- Fixes an issue where files that should be excluded by the check\_sync\_state step were still being processed by subsequent steps in a workflow (DYNAMOHUB-1169)
Release notes - Ngenea Hub - 1.16.0: 2022-11-08
===============================================
Improvement
-----------
- Display summarized item counts when deleteing, enabling and disabling. (DYNAMOHUB-1003)
- Remove site health details from the site details page. (DYNAMOHUB-1011)
- First and last page buttons are added to table pagination. (DYNAMOHUB-619)
- Job filter displays site name instead of site id in the filter. (DYNAMOHUB-654)
- Job filter owner list is calculated faster on the backend. (DYNAMOHUB-713)
- Fixed the issue with changing url when redirects to 404 not-found page (DYNAMOHUB-805)
- Make job stats endpoint return statistics about last 7 days (DYNAMOHUB-950)
- Gray out resubmit button for schedules jobs. (DYNAMOHUB-983)
- Improve site health details (DYNAMOHUB-985)
- Fixed font so that 0 and 'o' , 1 and l won't be mixed up. (DYNAMOHUB-994)
Bugfixes
--------
- Fixed displaying gray screen when there is no site health data. (DYNAMOHUB-1006)
- Error when migrating tasks from before version 1.15 on demand (DYNAMOHUB-1113)
- A nicer API response is sent to the users when filesets can't be retrieved due to celery timeout error. (DYNAMOHUB-929)
- Fixed the issue related to updating the file browser when the filename contains emojis. (DYNAMOHUB-988)
Features
--------
- Confirm workflow modal summarizes items if there are too many items. (DYNAMOHUB-826)
- Support cold failover on deployments where ngenea hub is deployed on a pixstor cluster. (DYNAMOHUB-966)
Documentation
-------------
- Improved example policy in the transparent recall docs, which resolves issues with deferred deletes (DYNAMOHUB-997)
- Updated advice on transparent recall policy
Due to limitations of GPFS, users must choose between a policy which supports paths with whitespace or paths with single quotes. (DYNAMOHUB-998)
Release notes - Ngenea Hub - 1.15.1: 2022-09-29
===============================================
Bugfixes
--------
- Fixed a bug where Jobs created before 1.15.0 could not be viewed or cancelled. (DYNAMOHUB-1113)
- Fixed a bug where a bi-directional site-sync could fail if directory moves are present (DYNAMOHUB-1116)
Release notes - Ngenea Hub - 1.15.0: 2022-09-27
===============================================
Improvement
-----------
- Improve DB locking to make the UI more responsive while large jobs are refreshing (DYNAMOHUB-1024)
- Optimise event comparison for Bidirectional site sync (DYNAMOHUB-1035)
- Remove unneeded list of spawned tasks from result payload of filter_snapshot_results (DYNAMOHUB-1037)
- Add a runtime field to jobs that use snapdiff: "snapdiff_rotate_on_error". If set to True, this will rotate a snapdiff forwards even if there are failed files. (DYNAMOHUB-1045)
- Subscribed workflows are editable on the UI. (DYNAMOHUB-1049)
- Don't overwrite remote objects for default sync workflows. (DYNAMOHUB-1063)
- Change snapdiff based workflows, such as site sync, to always rotate snapshots even when errors occur. (DYNAMOHUB-1077)
- Job files API endpoint is paginated. (DYNAMOHUB-916)
- Job files modal on the job details page uses paginated file endpoint now. (DYNAMOHUB-917)
- Apply cron expression to add and update schedule ui. (DYNAMOHUB-941)
- Ensure that the file list for snapdiff based jobs matches the reported number of files (DYNAMOHUB-976)
- Users can not delete running jobs by using the API (DYNAMOHUB-977)
- Prevent users from deleting pending and started jobs. (DYNAMOHUB-978)
- Display dash(-) for empty node fields. (DYNAMOHUB-986)
Bugfixes
--------
- Ensure snapshots are rolled-back when a job using the snapdiff discovery is cancelled (DYNAMOHUB-1027)
- Don't cleanup events for active, unscheduled syncs (DYNAMOHUB-1029)
- Ensure that cancelled bidirectional site sync jobs don't have tasks stuck as PENDING (DYNAMOHUB-1039)
- Job cancel hangs when the job has a lot of tasks (DYNAMOHUB-1047)
- Dispaly jobtypes in alphabetical order (DYNAMOHUB-933)
- Stat paths task results recorded as an error (DYNAMOHUB-942)
- Ensure that files which are skipped due to checking sync state are reported as skipped in the UI Job details page (DYNAMOHUB-943)
- No discovery selection for schedules is fixed. (DYNAMOHUB-967)
- Make show_noop parameter work only for successful jobs & make the default setting to True (DYNAMOHUB-999)
Features
--------
- Adds 'aborted' state which could not be processed but may be retried later. (DYNAMOHUB-1067)
- Allow user to delete jobs (DYNAMOHUB-618)
- New default workflow for deleting individual files (DYNAMOHUB-893)
- Allow users to enable and disable multiple schedules from the administrative ui. (DYNAMOHUB-940)
Release notes - Ngenea Hub - 1.14.2: 2022-08-12
===============================================
Bugfixes
--------
- Optimised snapdiff event processing for initial sync runs (DYNAMOHUB-1014)
Release notes - Ngenea Hub - 1.14.1: 2022-07-27
===============================================
Bugfixes
--------
- Ensure snapdiff soft-lock is removed in the event of an error (DYNAMOHUB-975)
Release notes - Ngenea Hub - 1.14.0: 2022-07-15
===============================================
Highlights
----------
- It's now possible to cancel running or stuck jobs via the Jobs Details page.
- Introducing a new and improved workflow: bi-directional site-sync. A schedule-only workflow that allows 2 sites to stay in sync.
- New heath status page - showing the status of ngenea hub and it's workers.
- It's now possible to have site-wide include and exclude lists.
Improvement
-----------
- Autocomplete selection for task types is added to filter tasks modal. (DYNAMOHUB-522)
- Search filters are displayed as an autocomplete input box (DYNAMOHUB-546)
- Site selection for search UI is enabled. (DYNAMOHUB-859)
- Sidebar toggler icon is animated (DYNAMOHUB-874)
- Jobs can be filtered by completion time. (DYNAMOHUB-887)
- Path picker component is added for schedule management. (DYNAMOHUB-889)
- Limit bidirectional_snapdiff discovery schedules to one bidirectional_sync subscribed workflow (DYNAMOHUB-894)
- Tasks are sorted by their start time by default. (DYNAMOHUB-895)
- bidirectional_snapdiff discovery schedules can be created in the UI (DYNAMOHUB-899)
- Only allow one bi-directional sync run at a time in a schedule (DYNAMOHUB-901)
- Jobs can be cancelled on the UI (DYNAMOHUB-911)
- New tasks are introduced for retrieving search metadata fields (DYNAMOHUB-546)
- Support for different fileset and pool name format in Analytics 2.X (DYNAMOHUB-797)
- Move snapdiff lock file location to the .rotate directory (DYNAMOHUB-931)
Bugfixes
--------
- Fixed users deactivating themselves. (DYNAMOHUB-578)
- Halting the search issue is fixed when the search filters are changed (DYNAMOHUB-647)
- Fixed jobs filter not syncing with recent jobs issue. (DYNAMOHUB-821)
- Possible web socket connection flaws are resolved. (DYNAMOHUB-885)
- Fix intermittent transparent recall hangs (DYNAMOHUB-902)
- RPM creates a blank config file on new install (DYNAMOHUB-947)
- Skip moves which have already been applied (DYNAMOHUB-926)
Features
--------
- Added user configurable port to access hub (DYNAMOHUB-198)
- Support for site-global includes and excludes (DYNAMOHUB-613)
- Removed HubSchedule model from DB (DYNAMOHUB-769)
- Added validation to prevent call to workflow/schedule when no work is done (DYNAMOHUB-798)
- Added endpoint for both site specific and hub wide health status reporting (DYNAMOHUB-838)
- Added Site Health Feature (DYNAMOHUB-840)
- Display cron schedule in human readable format (DYNAMOHUB-868)
- Snapdiff task results are now tracked within the hub database (DYNAMOHUB-870)
- Background task to clean up old sync events (DYNAMOHUB-871)
- Workflows can now be set for use only within a schedule (DYNAMOHUB-873)
- New workflow added for two way syncing between sites within schedules (DYNAMOHUB-877)
Documentation
-------------
- Documentation on how to set up site sync (DYNAMOHUB-574)
- Search set-up documentation (DYNAMOHUB-781)
- Document using ngclient to interact with feature flags (DYNAMOHUB-843)
- Fixed worker service name in upgrade guide (DYNAMOHUB-882)
- Documentation on how to set up bidirectional site sync (DYNAMOHUB-908)
- Troubleshooting guide (DYNAMOHUB-948)
Release notes - Ngenea Hub - 1.13.0: 2022-06-13
===============================================
Bugfixes
--------
- Timeout is handled on login. (DYNAMOHUB-348)
- If the token is expired, redirect to original target page after login. (DYNAMOHUB-611)
- Removed non-deterministic behaviour while displaying the search bar. (DYNAMOHUB-646)
- Long paths are wrapped to a new line in job details page. (DYNAMOHUB-659)
- Missing ngclient sub-command man pages (DYNAMOHUB-692)
- Fixed Api-Key issues in Resubmit API (DYNAMOHUB-703)
- Added new celery beat that runs every one hour to update the state from STARTED to FAILURE for inactive tasks (DYNAMOHUB-704)
- Single search web socket connection is managed against working intermittently. (DYNAMOHUB-729)
- Show jobs with no processed files filter is configurable for all job tables. (DYNAMOHUB-744)
- Ensures that a snapshot rotation task is created when a job is resolved regardless of the task chain (DYNAMOHUB-747)
- Ensured that a clearer message is provided when a GPFS error occurs on worker for a stat (DYNAMOHUB-761)
- Recent job counts are corrected for both API and web socket consumer. (DYNAMOHUB-783)
- Fixed schedule job resulting in pending state for snapdiff task (DYNAMOHUB-790)
- Fixed the issue which chases when sumitting bandwidth form with no changes. (DYNAMOHUB-800)
- Bug is fixed when managed_paths is null for a schedule. (DYNAMOHUB-817)
- Removed extra file batch field (DYNAMOHUB-852)
- Sort actions in alphabetical order (DYNAMOHUB-752)
- Fixed issue on UUID mismatch using ngrecall overwrite-recall by comparing local and remote UUID (DYNAMOHUB-464)
- Added interface type dummy to virtual drivers list to exclude it from physical interface (DYNAMOHUB-757)
- Correctly caught GPFS errors to prevent task from returning stack trace. (DYNAMOHUB-761)
- Directory moves in move_paths_on_gpfs now move all files along with the directory themselves (DYNAMOHUB-782)
- Removed the strict format for snapshots within the snapdiff tracking file (DYANMOHUB-699)
Features
--------
- Feature added for limiting fields in metadata returned (DYNAMOHUB-376)
- feature added for search REST API to provide more_results key if elasticsearch backend has more results than max_results (DYNAMOHUB-550)
- Added support for overriding includes and excludes at runtime (DYNAMOHUB-640)
- Added reverse_stub in default workflows (DYNAMOHUB-681)
- Added recursive flag for move_paths_on_gpfs task in docs workflow_steps.md (DYNAMOHUB-685)
- Feature added for chunk_size to chunk paths in snapdiff tasks (DYNAMOHUB-711)
- Feature added for Id based endpoint for IpAddress Apis GET, DELETE and PATCH (DYNAMOHUB-712)
- Added support for pixstor_search backend in hub (DYNAMOHUB-738)
- Add Datastore managemant and IP address management feature. (DYNAMOHUB-774)
- Updated sync_preference type to choices for site_sync workflow (DYNAMOHUB-787)
- NgeneaHub UI supports having new themes now. (DYNAMOHUB-842)
- Add list of available metadata fields to results of Hub searches performed against Pixstor Search (DYNAMOHUB-734)
- Add mechanism to return specified metadata fields only, in Hub searches against Pixstor Search (DYNAMOHUB-736)
Documentation
-------------
- Upgrade guide documentation (DYNAMOHUB-687)
- Add site-specific configurations to the docs (DYNAMOHUB-739)
- Documentation for default workflows (DYNAMOHUB-741)
- Document how to use search metadata field limiting (DYNAMOHUB-756)
Removals
--------
- Removed the site field for IP address objects as they were once depreciated (DYNAMOHUB-657)
Improvement
-----------
- Job runtime field is introduced and becomes sortable on the UI. (DYNAMOHUB-714)
- feature improvement done in stats to add extended attributes as a dict (DYNAMOHUB-201)
- feature improvement added for allowing directory deletes with kwargs (DYNAMOHUB-345)
- Add support for `--endpoint` flag on recall and reverse stub tasks (DYNAMOHUB-361)
- Workaround for stale file handles when performing reverse stub (DYNAMOHUB-439)
- Add support for worker auto-scaling (DYNAMOHUB-461)
- Record exact ngenea command in task results (DYNAMOHUB-462)
- Adjusted recursive task to allow for multiple operations based of generic rules (DYNAMOHUB-476)
- feature improvement added for failure scenario for remove_location_xattrs_moved (DYNAMOHUB-509)
- feature improvement added for snapdiff tasks to allow only single path else raise an exception with error message (DYNAMOHUB-515)
- Add Ngenea server and client as rpm package dependencies for installing the ngenea worker, and require at least version 1.15. (DYNAMOHUB-694)
- Ensured that ngenea binary stderr capture is not blocked by reporting the status of a job (DYNAMOHUB-765)
Release notes - Ngenea Hub - 1.12.0: 2022-05-20
===============================================
Improvement
-----------
- Hash is used instead of content hash for cache busting. (DYNAMOHUB-743)
- Fixed not displaying error or failure state jobs with no operations (DYNAMOHUB-745)
- File browser performance improvement for file status updates. (DYNAMOHUB-766)
- Log PID instead of process name within the celery logs. (DYNAMOHUB-784)
- List actions in alphabetical order. (DYNAMOHUB-752)
- Allow deleting of /ipaddresses/{ipaddr}/ if an ip address is associated with multiple datastores. (DYNAMOHUB-712)
- Add new reverse stub workflow as part of deployment. (DYNAMOHUB-681)
- Provide mechanism to limit fields of search metadata returned. (DYNAMOHUB-376)
Features
--------
- Feature added for limiting metadata fields in search_analytics (DYNAMOHUB-376)
Bugfixes
--------
- Fixed migration bug with mis-matching schedule IDs during migration. (DYNAMOHUB-824)
- UI now handles managed_paths being null within a schedule. (DYNAMOHUB-817)
- Ensured rotate task do not remain stuck in a PENDING state. (DYNAMOHUB-747)
- Users can now show no-op jobs across all job tables. (DYNAMOHUB-744)
- Search no longer works intermittently when previous search is stopped. (DYNAMOHUB-729)
- Searching by gpfs.filesetname now works correctly. (DYNAMOHUB-727)
- When task gets killed, it is no longer marked as "STARTED" in celery. (DYNAMOHUB-704)
- Jobs can now be resubmited using an API Key. (DYNAMOHUB-703)
- Search bar now always shows when you first log into UI. (DYNAMOHUB-646)
- Fixed migrate task issue when migrating same filepath to multi targets (DYNAMOHUB-726)
Release notes - Ngenea Hub - 1.11.0: 2022-05-09
===============================================
Improvement
-----------
- Descriptive messages for all tables (DYNAMOHUB-541)
- Default search filter is changed to core.pathname (DYNAMOHUB-545)
- Workflow details can be inspected on job details page. (DYNAMOHUB-648)
- Table row highlighting is improved visually. (DYNAMOHUB-656)
- Make number of files diplay dialog nicer (DYNAMOHUB-661)
- Elasticsearch url management for sites (DYNAMOHUB-693)
- Added a message when there are more searched results to be displayed (DYNAMOHUB-697)
- Task details preview is disabled when task details JSON is too large. (DYNAMOHUB-719)
- file_size_gb field of sites became managable from UI. (DYNAMOHUB-740)
- Update UI for new discovery schedules, to which multiple workflows can be subscribed. (DYNAMOHUB-772)
- Feature added in search_analytics to hint users if there are more_results than max_results in elasticsearch backend (DYNAMOHUB-550)
- Added flag to recall and reverse_stub tasks for gid and uid (DYNAMOHUB-614)
- Added elasticsearch credentials for analytics search backend (DYNAMOHUB-689)
- Support for AP-Analytics 2.X search backend (DYNAMOHUB-760)
Bugfixes
--------
- Fix upgrade generating spurious systemd services (DYNAMOHUB-359)
- Added support for CredentialsJSON in GCS ngenea configs (DYNAMOHUB-655)
- Adjusted the start date of all tasks to return UTC times for consistency (DYNAMOHUB-663)
Release notes - Ngenea Hub - 1.10.0: 2022-04-01
===============================================
Improvement
-----------
- Added user friendly log viewer (DYNAMOHUB-436)
- feature improvement added for filter_results to return clear results for the user (DYNAMOHUB-491)
- Workflow configuration is visible in job details and job list pages (DYNAMOHUB-626)
- Table optional fields are managed by switches in Show/Hide column dialog. (DYNAMOHUB-631)
- Converted runtime value into human readable value (DYNAMOHUB-644)
- Resolved UI Typescript compile issues & unit test issues (DYNAMOHUB-665)
- Tab view for Administration, SiteDetails, ScheduleDetails pages (DYNAMOHUB-666)
- Refactoring on the file browser (DYNAMOHUB-667)
- Refactoring forms on the UI (DYNAMOHUB-668)
- Refactoring tables & sticky table controls (DYNAMOHUB-669)
- UI test coverage is increased. (DYNAMOHUB-670)
- Refactoring on websocket serialization (DYNAMOHUB-680)
- Bandwidth controls are hidden behind a feature flag. (DYNAMOHUB-750)
Features
--------
- Support for setting search-related configurations via the REST API (DYNAMOHUB-474)
- feature added for supporting default filter rules for snapdiff and recursive discovery tasks (DYNAMOHUB-533)
- API endpoint for setting speed for site (DYNAMOHUB-568)
- Support for setting the IP address for a DataStore (DYNAMOHUB-569)
- Feature added for auto update mechanism of cloud hosted storage target ips for s3,azure,gcs (DYNAMOHUB-570)
- UI for setting bandwidth controls (DYNAMOHUB-571)
- Updated docs with endpoint flag for migrate tasks (DYNAMOHUB-580)
- Snapdiff discovery tasks now only rotate their snapshot on no errors in other tasks. (DYNAMOHUB-630)
- Ability to scan for worker nodes via the REST API (DYNAMOHUB-638)
- Feature added to allow directory move in move_paths_on_gpfs (DYNAMOHUB-542)
- Feature improvement added endpoint flag for migrate task (DYNAMOHUB-580)
- Adjusted the parsing method to new json output for ngenea 1.15 (DYNAMOHUB-606)
- Added worker task for rotating snapshots outside of the snapdiff task (DYNAMOHUB-630)
Bugfixes
--------
- Display workflow name as the tooltip text (DYNAMOHUB-554)
- fixed issue job fails while other tasks are running by refreshing the job when there are running/pending remaining tasks (DYNAMOHUB-577)
- Fixed clicking to search result path for hidden items (DYNAMOHUB-683)
- Fix inconsistencies between PixStor Search and Analyitcs search backends (DYNAMOHUB-556)
- Fixed ngenea error reporting by adding proper status logs (DYNAMOHUB-624)
- fixed issue for moving files between different filesets (DYNAMOHUB542)
Deprecations and Removals
-------------------------
- Setting IP address for a site is deprecated as of release 1.9 and will be removed in 1.10 (DYNAMOHUB-569)
Release notes - Ngenea Hub - 1.9.0: 2022-02-23
==============================================
Features
--------
- Model for object stores (S3, GCS, etc.) (DYNAMOHUB-207)
- SiteLink model for connecting sites to datastores (DYNAMOHUB-224)
- Worker node model and monitoring (DYNAMOHUB-563)
- Feature added for configuring file batch size in site model (DYNAMOHUB-572)
- Job started time, completed time, runtime fields are accessible on the UI. (DYNAMOHUB-573)
- Modifications done for file count for all tasks to report job statistics (DYNAMOHUB-587)
- Added endpoint to job API to cancel all on-going tasks (DYNAMOHUB-71)
- changes added for chunk size in recursive action (DYNAMOHUB-472)
Improvement
-----------
- Allow user to search job id from the job table (DYNAMOHUB-411)
- Complex queries can be made with the search bar (DYNAMOHUB-547)
- changed tooltip to display workflow name (DYNAMOHUB-554)
- Update frontend depedancy: follow-redirects to address CVE-2022-0536 (DYNAMOHUB-637)
Bugfixes
--------
- Job filter preferences are remembered on the jobs page (DYNAMOHUB-427)
- Display correct page size after applying the filters. (DYNAMOHUB-531)
- Jobs page table glitch is fixed. (DYNAMOHUB-632)
Release notes - Ngenea Hub - 1.8.0: 2022-02-08
==============================================
Improvement
-----------
- Usability improvements on task detail modal (DYNAMOHUB-420)
- Automatically create hub environment auth file if it does not exist (DYNAMOHUB-576)
Features
--------
- Added UI element for "choices" field type (DYNAMOGUB-551)
- Added ngenea locking mode support for default workflow operations (DYNAMOHUB-537)
- feature added for migrating empty directories (DYNAMOHUB-490)
- Added flag to recall, reverse_stub and migrate tasks for controlling ngenea locking levels (DYNAMOHUB-537)
- changes added for chunk size in recursive action (DYNAMOHUB-472)
Bugfixes
--------
- fixed misleading error messages on login page (DYNAMOHUB-348)
- changed return code of /api/file/workflow endpoint from 200 to 201 (DYNAMOHUB-488)
- Informative messages when the worker is shutdown (DYNAMOHUB-514)
- Usability improvements on tables and schedule pages (DYNAMOHUB-517)
- Ensure debug mode isn't enabled in production (DYNAMOHUB-518)
- fixed the warning issue raised for timezone in task started (DYNAMOHUB-519)
- Removed default option as recursive set for discovery for /api/file/workflow (DYNAMOHUB-534)
- Added jobid in signatures for recursive discovery task (DYNAMOHUB-543)
- Ensured that the detault value of a field is respected in the UI (DYNAMOHUB-558)
- Ensure secure file permissions on hub auth configuration file (DYNAMOHUB-561)
- Ensured that the workflow API route has full validation on JSON fields (DYNAMOHUB-562)
- More detailed messages for file browser when the worker is not available (DYNAMOHUB-566)
- No results returned for date range search (DYNAMOHUB-489)
- Number of search results differs for path with trailing slash (DYNAMOHUB-500)
- added try/except block for get_xattrs for gpfs.FileHeat key error (DYNAMOHUB-525)
- Fixed an issue where if a file was skipped by a step in a workflow, they were not being processed by a migrate or recall step later in the workflow. (DYNAMOHUB-540)
Release notes - Ngenea Hub - 1.7.0: 2022-01-24
==============================================
Improvement
-----------
- feature improvement added choices as field type (DYNAMOHUB-154)
- Record and report errors when performing searches (DYNAMOHUB-365)
- Search UI has now filtering option. Before submitting a search, users can add more filters to narrow down the search. (DYNAMOHUB-374)
- Time range selection for job filtering (DYNAMOHUB-403)
- Multiselect functionality on the job state field for job filtering is added. (DYNAMOHUB-449)
- Owner selection while filtering jobs (DYNAMOHUB-457)
- All alerts are manually closed now. (DYNAMOHUB-458)
- Job state and job type selection is swapped visually on the job filtering dialog (DYNAMOHUB-468)
- Replaced "In progress" label with "Started" for jobs (DYNAMOHUB-470)
- Performance improvement on job table and auto refresh toggle for web socket live updates (DYNAMOHUB-480)
- feature improvement added for snapdiff tasks numfiles (DYNAMOHUB-510)
Features
--------
- feature improvement added for runtime fields to have default values for choices type (DYNAMOHUB-153)
- feature added for choices runtime field to support list of objects (DYNAMOHUB-154)
- Added support for providing file states for non-discovery workflows (DYNAMOHUB-344)
- Allowed the use of multiple generic rules within the recursive discovery task (DYNAMOHUB-476)
- API endpoint for feature flags (DYNAMOHUB-482)
- Search UI is a configurable feature now. (DYNAMOHUB-483)
- ngclient sub-commands for listing and setting feature flags (DYNAMOHUB-484)
- Feature API is used for enabling and disabling the usage of UI elements. (DYNAMOHUB-485)
- Tasks for moving files and directories within cloud storage (DYNAMOHUB-355)
Bugfixes
--------
- Number of search results per site is 200 results by default (DYNAMOHUB-374)
- fixed issue by adding fields in job model instance for resubmit tasks (DYNAMOHUB-465)
- Performance improvements on the Jobs page (DYNAMOHUB-466)
- A timeout is added for failed search operations. If no results are found in 100 seconds, the error will show up and the search will halt. (DYNAMOHUB-498)
- Search results can be sorted by metadata fields. (DYNAMOHUB-557)
- added exception for permission errors (DYNAMOHUB-345)
- Policy error in snapdiff (DYNAMOHUB-450)
- fixed issue raised when submitting a workflow with multiple filters in recursive task (DYNAMOHUB-476)
Release notes - Ngenea Hub - 1.6.0: 2021-12-21
==============================================
Improvement
-----------
- return extended attributes in file API (DYNAMOHUB-201)
- Add support for `--endpoint` flag on recall and reverse stub tasks (DYNAMOHUB-361)
- Support for workflows without a discovery task, through the API (DYNAMOHUB-402)
- limit the number of hub celery worker threads to 2 (DYNAMOHUB-405)
- Include error type when reporting task errors (DYNAMOHUB-418)
- Apply dynamo.tasks.remove_location_xattrs_for_moved to send and snapdiff workflows to better handle moving files (DYNAMOHUB-422)
- Improve handling of file system xattr "stale nfs handle" message scenarios (DYNAMOHUB-439)
- Disable validation of non-included fields when patching workflows via api (DYNAMOHUB-456)
Features
--------
- Set graceful timeout on stat requests, default to 10 seconds. (DYNAMOHUB-338)
- JWTs are created by using RS256 algorithm (DYNAMOHUB-353)
- Use search result to jump to directory in file browser (DYNAMOHUB-380)
- Expose JWK set used in token verification via API endpoint (DYNAMOHUB-430)
- Use Hub authentication for Grafana access (DYNAMOHUB-431)
- Initial framework and metrics for Grafana dashboads (DYNAMOHUB-432)
- ngclient command to import and export custom workflows (DYNAMOHUB-434)
- NgeneaHub UI management of scheduled workflows (DYNAMOHUB-435)
- Added task for checking the existence of files in remote storage (DYNAMOHUB-421)
- Task to remove remote location xattrs if a premigrated file was moved (DYNAMOHUB-422)
Bugfixes
--------
- Migration path not correct when upgrading from 1.3.0 (DYNAMOHUB-397)
- Fix incorrect PDF Documention download link (DYNAMOHUB-440)
- Fixed issue with task start time formatting (DYNAMOHUB-441)
- Custom workflow steps can get wiped on upgrade (DYNAMOHUB-442)
- Fix incorrect download link for ngenea client (DYNAMOHUB-444)
- Mark tasks as completed when skipped (DYNAMOHUB-477)
- Fix date string parsing error in update_task celery task (DYNAMOHUB-508)
- Don't require ArcaPix policy driver to be executable (DYNAMOHUB-419)
- Check for no paths passed to recall and reverse_stub tasks (DYNAMOHUB-438)
- fixed the status of snapdiff tasks processed files to created instead of created and deleted (DYNAMOHUB-492)
- Fix issue with parsing ngenea config file when the [General] section is present (DYNAMOHUB-506)
- Ensured that all chained tasks parse results correctly (DYNAMOHUB-511)
Release notes - Ngenea Hub - 1.5.0: 2021-11-22
==============================================
Features
--------
- Workflows can be enabled/disabled. Also, workflows can be available to use via the API only. (DYNAMOHUB-167)
- File browser can be filtered by filetype, size, change date, accessed date. (DYNAMOHUB-196)
- Job list filtering options are extended. Jobs can be filtered by site and path prefix information too. (DYNAMOHUB-299)
- Endpoint to submit search requests (DYNAMOHUB-319)
- Endpoint for retrieving search results (DYNAMOHUB-320)
- Submit async search tasks and store results (DYNAMOHUB-321)
- Periodically remove old search results (DYNAMOHUB-328)
- Support for search backend configurations (DYNAMOHUB-329)
- Added the option to provide the django secret key via the environment (DYNAMOHUB-342)
- Include available metadata fields with search results (DYNAMOHUB-382)
- Add job completion and run time to metadata (DYNAMOHUB-63)
- Added support for ngenea-hub to delete GPFS files (DYNAMOHUB-164)
- Added support for ngrecall --skip-check-hash in workflows (DYNAMOHUB-176)
- Snapdiff discovery task (DYNAMOHUB-245)
- Task to enumerate independent filesets on a site (DYNAMOHUB-251)
- AP-Analytics based search backend task (DYNAMOHUB-321)
- Accept configuration settings for search backend tasks (DYNAMOHUB-329)
- Compatibility with ngenea 1.12 (DYNAMOHUB-333)
- Added argument "delete_remote_xattrs" to the move_paths task to remove remote location metadata (DYNAMOHUB-346)
- Provide available metadata fields with search results (DYNAMOHUB-382)
- Size-aware file list chunking in recursive_action (DYNAMOHUB-53)
- Track size of files being processed for reporting (DYNAMOHUB-63)
Bugfixes
--------
- Set task status to ERROR if any file failed (DYNAMOHUB-161)
- Fixed job processed and failed files incorrectly returning empty lists (DYNAMOHUB-264)
- Fix transparent recall policy to handle paths with whitespace (DYNAMOHUB-324)
- Retry on error when waiting for transparent recall (DYNAMOHUB-336)
- Don't treat individual file failures as a task failure (DYNAMOHUB-161)
- Don't treat ngenea warnings as errors (DYNAMOHUB-186)
- Disable task late ack to try to mitigate consumer timeouts (DYNAMOHUB-291)
- Friendlier error response when Elasticsearch is unavailable (DYNAMOHUB-364)
Improved Documentation
----------------------
- Clarify transparent recall documentation (DYNAMOHUB-324)
Release notes - Ngenea Hub - 1.4.0: 2021-10-15
==============================================
Features
--------
- Consolidate the 3 ngenea-worker systemd services to a single service for easier management. (DYNAMOHUB-149)
Release notes - Ngenea Hub - 1.3.0: 2021-09-22
==============================================
Features
--------
- “live” changes to file status (DYNAMOHUB-105)
- Transparent Recall logging (DYNAMOHUB-100)
- Expose Groups model via REST API (DYNAMOHUB-277)
- Option to show/hide hidden objects in the filebrowser (DYNAMOHUB-262)
- UI for setting IP addresses against Site (DYNAMOHUB-232)
- Extend site model to store it's IP addresses (DYNAMOHUB-229)
- ClientKey UI (DYNAMOHUB-143)
- PATCH support for /workflow/ endpoint (DYNAMOHUB-285)
- Spinners on UI components waiting for API response (DYNAMOHUB-281)
- Having visible alerts or redirects according to the websocket warnings (DYNAMOHUB-276)
- Websocket request should wait for response before sending another request (DYNAMOHUB-272)
- Reconsider websockets broadcast approach (DYNAMOHUB-271)
- Revamp 404 page (DYNAMOHUB-252)
- Allow the client-key to access all routes excluding user and client-key (DYNAMOHUB-243)
- Control number of worker threads (DYNAMOHUB-237)
- Change colours for paginated table (DYNAMOHUB-218)
- Make action dropdown wider (DYNAMOHUB-212)
- Make task table sortable (DYNAMOHUB-202)
- Execute the production code via gunicorn (DYNAMOHUB-129)
- Task for checking the current filesets residing on a site (DYNAMOHUB-251)
- Site worker snapdiff celery discovery task (DYNAMOHUB-245)
- Ability to submit tasks to an existing job (DYNAMOHUB-215)
- Ngenea Hub remote client (DYNAMOHUB-213)
Bugfixes
--------
- Workers can timeout on waiting for response from rabbitmq (DYNAMOHUB-291)
- Fails to open directories with strange file names (DYNAMOHUB-288)
- Swagger logs exception (DYNAMOHUB-283)
- Prevent resubmitting a job multiple times in quick succession (DYNAMOHUB-274)
- "Failure" job filter only shows failures for one site (DYNAMOHUB-273)
- Actions button visible on the user profile page (DYNAMOHUB-270)
- Unable to deactivate users (DYNAMOHUB-269)
- Update user profile "save" button is always enabled (DYNAMOHUB-268)
- Inaccurate "last login" info (DYNAMOHUB-267)
- Sort order of jobs (DYNAMOHUB-266)
- Browser tries to show contents of a non-directory (DYNAMOHUB-260)
- Fix frontend issue when submitting large number of files to a workflow (DYNAMOHUB-259)
- Clicking on "Owner" for a clientkey gives a 404 (DYNAMOHUB-253)
- Include ngeneahub-frontend in ngeneahub-images RPM (DYNAMOHUB-242)
- Resubmitting a job doesn't use the same discovery method (DYNAMOHUB-239)
- Some overall job stats don't update until the dir walk is complete (DYNAMOHUB-210)
- Files without a file extension are listed as both directory and folder (DYNAMOHUB-200)
Release notes - Ngenea Hub - 1.2.0: 2021-08-11
==============================================
Features
--------
- Update RabbitMQ to 3.9 (DYNAMOHUB-223)
- paginate tasks in /api/jobs/<id> (DYNAMOHUB-204)
- Group transparent recall tasks (DYNAMOHUB-216)
- Ability to submit non-recursive tasks (DYNAMOHUB-214)
Bugfixes
--------
- Rest API can not be authenticated with an access token (DYNAMOHUB-227)
- Fix client side pagination for task list in job details (DYNAMOHUB-219)
- List of files is job details is empty (DYNAMOHUB-211)
- Using a boolean as a workflow step parameter fails (DYNAMOHUB-208)
Release notes - Ngenea Hub - 1.1.0: 2021-06-19
==============================================
Features
--------
- Send data between different sites via the UI (DYNAMOHUB-124)
- Rework ngenea output parsing (DYNAMOHUB-115)
- Rebuild UI using React (DYNAMOHUB-103)
- Improve stat task to make directory handling explicit (DYNAMOHUB-188)
- step argument: skip-check-hash (DYNAMOHUB-176)
- step argument: overwrite-remote (DYNAMOHUB-175)
- step argument: overwrite-local (DYNAMOHUB-174)
- step argument: stub-size (DYNAMOHUB-173)
- React 404 should not change URL (DYNAMOHUB-166)
- Support file delete workflow (DYNAMOHUB-164)
- Recent jobs list should show the most recent jobs, nevermind what state they are in (DYNAMOHUB-147)
- Job reporting scaling improvements (DYNAMOHUB-84)
Bugfixes
--------
- `api/users/` endpoint throwing errror when a username containing "." exists (DYNAMOHUB-187)
- "failed to acquire a DMAPI lock EXCL immediately; keeping trying..." is treated as an error (DYNAMOHUB-186)
- Job reporting not providing correct counts (DYNAMOHUB-185)
- Resubmit button should not be enabled when not usable (DYNAMOHUB-184)
- Using UI to trigger workflow fails (DYNAMOHUB-177)
- Attempting to post to /api/file/workflow with only a token causes 500 (DYNAMOHUB-171)
- Bug when a directory is in intermediate selection state and closed (DYNAMOHUB-170)
Release notes - Ngenea Hub - 1.0.4: 2021-06-10
==============================================
Features
--------
- Allow overriding the site on a per-step basis (DYNAMOHUB-151)
- Submission time arguments to workflow support (DYNAMOHUB-150)
- Validate runtime fields (DYNAMOHUB-136)
- Show verbose error information on failure (DYNAMOHUB-156)
- New task: reverse stub (DYNAMOHUB-155)
- Report task types with more meaningful names (DYNAMOHUB-152)
Bugfixes
--------
- Fix bug where sub-directories can fail to be processed (DYNAMOHUB-148)
- recursive_action failures report as success (DYNAMOHUB-142)
Release notes - Ngenea Hub - 1.0.3: 2021-05-19
==============================================
Features
--------
- Refactor existing ngenea hub tasks to message passing format (DYNAMOHUB-140)
- Support static arguments to a step (DYNAMOHUB-137)
- Provide existing actions as default workflows (DYNAMOHUB-131)
- enable easy access to dbshell (DYNAMOHUB-134)
- Workflows are now defined dynamically (DYNAMOHUB-132)
- Allow users to create (API) client keys (DYNAMOHUB-141)
Bugfixes
--------
- Fixed an issue with submitting workflows with Client Key (DYNAMOHUB-144)
- Fixed warnings generated from auto field (DYNAMOHUB-133)
Release notes - Ngenea Hub - 1.0.2: 2021-04-26
==============================================
Features
--------
- Expose JWT Login API endpoints (DYNAMOHUB-90)
- Add Swagger API Documentation (DYNAMOHUB-94)
- Validate that site names do not end with the suffix we use to identify queue types (DYNAMOHUB-85)
- Allow middle-click/right-click to open navbar links in new tabs (DYNAMOHUB-83)
- Ship ngeneahub cli tool as a venv with docker-compose included (DYNAMOHUB-81)
- Improve performance of job page by doing pagination server-side (DYNAMOHUB-56)
Release notes - Ngenea Hub - 1.0.1: 2021-04-01
==============================================
Bugfixes
--------
- Symlinks cause file browser to fail (DYNAMOHUB-86)
- "Creation time" should be "ctime" - "Last changed time" (DYNAMOHUB-78)
- Files and folders with newlines in their names fails, with console error (DYNAMOHUB-55)
Features
--------
- Standalone UI providing interfaces for core Ngenea/Dynamo workflows (DYNAMOHUB-1)
- Positioning and content of UI buttons (DYNAMOHUB-38)
- Show percentage completion of job based on # of files transferred vs remaining (DYNAMOHUB-34)
- Remove redundant environment settings (DYNAMOHUB-82)
- Show a user view page when clicking on a user link (DYNAMOHUB-79)
- Pop-up confirmation of job actions (DYNAMOHUB-57)
- Support for premigrate task (DYNAMOHUB-50)
- Allow viewing directories from different sites (DYNAMOHUB-12)