Tasks

Tasks are functions which are called when an object is created (POST) updated (PATCH) or deleted (DELETE).

These tasks are called via database event hooks, and they perform changes to the underlying filesystem, corresponding to the database change being performed.

For example, when a template is deleted, a TemplateDeleter task will remove the corresponding template files from the filesystem.

Tasks are executed via the configured Job Engine - see PixStor Schedulers.

API Documentation

ORM Task

ORMTask is the base class for all tasks which affect objects on the filesystem.

class arcapix.management.tasks.ormtask.DBTask(*args: Any, **kwargs: Any)

Specialisation of the Task base class for altering the database.

Contains a reference to the database driver from which sessions can be created. This must be set before attempting to use ORM Tasks.

Parameters

type – Class object representing the type of object the task applies to
id (int) – numerical id of object the task applies to
etag (int) – etag of the object at the time the job was submitted If this changes by the time the job runs, we have an issue

Params are used to look-up the relevant items from the SQL database

get_db(): Gets a database session object for the database associated with the task

class arcapix.management.tasks.ormtask.ORMTask(*args: Any, **kwargs: Any)

Specialisation of the Task base class for dealing with our ORMed stuff.

This is further sub-classed by the individual tasks that are defined in other parts of this python package

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform()

Main part of task execution.

Returns: the state of the resource after perform completes. default: ACTIVE

post(resultstatus)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre()

Pre-flight.

Sets the resource’s status to INPROGRESS

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

class arcapix.management.tasks.ormtask.RollbackMixin: Convenience mixin for tasks with rollback.

Exposer Tasks

class arcapix.management.tasks.exposertasks.CIFSExposerBuilder(*args: Any, **kwargs: Any)

Task to create a CIFS exposer.

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform(): Add a new share to the SMB config.

post(resultstatus)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre(): Load samba config.

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

class arcapix.management.tasks.exposertasks.CIFSExposerDeleter(*args: Any, **kwargs: Any)

Task to delete a CIFS exposer.

cancel_delete(): Set the instance to not (hard) delete.

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform(): Remove the relevant share from SMB config file.

post(resultstatus)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre(): Load samba config.

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

should_delete()

Set the instance to soft deleted.

This will cause the instance to be hard-deleted iff it is in an ACTIVE state at the end of the task.

class arcapix.management.tasks.exposertasks.CIFSExposerUpdater(*args: Any, **kwargs: Any)

Task to update a CIFS exposer.

An extras field with a value of None will cause that field to be ‘unset’

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform(): Change the name or path of a share in SMB config.

post(resultstatus)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre(): Load samba config.

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

class arcapix.management.tasks.exposertasks.CIFSTask(*args: Any, **kwargs: Any)

Base class for CIFS tasks.

Handles reading and writing the samba config file.

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform()

Main part of task execution.

Returns: the state of the resource after perform completes. default: ACTIVE

post(resultstatus)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre(): Load samba config.

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

class arcapix.management.tasks.exposertasks.MigrationMode

Enum specifying when non-pixmanaged shares should be converted to pixmanaged.

AUTO = 'auto': convert to pixmanaged when modified

NEVER = 'never': never convert to pixmanaged

class arcapix.management.tasks.exposertasks.NFSExposerBuilder(*args: Any, **kwargs: Any)

Task to create a NFS exposer.

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform(): Add a new share to the NFS config file.

post(resultstatus)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre(): Load NFS config.

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

class arcapix.management.tasks.exposertasks.NFSExposerDeleter(*args: Any, **kwargs: Any)

Task to delete a NFS exposer.

cancel_delete(): Set the instance to not (hard) delete.

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform(): Remove the relevant share from NFS config file.

post(resultstatus)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre(): Load NFS config.

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

should_delete()

Set the instance to soft deleted.

This will cause the instance to be hard-deleted iff it is in an ACTIVE state at the end of the task.

class arcapix.management.tasks.exposertasks.NFSExposerUpdater(*args: Any, **kwargs: Any)

Task to update a NFS exposer.

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform(): Change the name or path of a share in NFS config.

post(resultstatus)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre(): Load NFS config.

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

class arcapix.management.tasks.exposertasks.NFSTask(*args: Any, **kwargs: Any)

Base class for NFS tasks.

Handles reading and writing the nfs config file.

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform()

Main part of task execution.

Returns: the state of the resource after perform completes. default: ACTIVE

post(resultstatus)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre(): Load NFS config.

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

Profile Tasks

class arcapix.management.tasks.profiletasks.ProfileBuilder(*args: Any, **kwargs: Any)

Task which makes the changes required for a new Profile.

Key things needed are setting up of the relevant placement rule for the first and subsequent ILM steps.

Each step’s source pool is the target pool from the prior step

i.e. (Placement) -> pool_1 -> (Management_1) -> pool_2 -> (Management_2) …

Filesets will use names to match for pool positioning

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform(): Create the relevant rules in the filesystem placement policy.

post(resultstatus): Save changes to policy.

pre(): Load relevant placement policy.

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

class arcapix.management.tasks.profiletasks.ProfileDeleter(*args: Any, **kwargs: Any)

Task to delete a profile.

cancel_delete(): Set the instance to not (hard) delete.

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform(): Remove the policy rules corresponding to the profile.

post(resultstatus): Save changes to policy.

pre(): Load relevant placement policy.

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

should_delete()

Set the instance to soft deleted.

This will cause the instance to be hard-deleted iff it is in an ACTIVE state at the end of the task.

class arcapix.management.tasks.profiletasks.ProfileTask(*args: Any, **kwargs: Any)

Generic Profile task to handle load/save of placement policy.

Doing this in pre/post rather than in perform makes unittesting easier.

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform()

Main part of task execution.

Returns: the state of the resource after perform completes. default: ACTIVE

post(resultstatus): Save changes to policy.

pre(): Load relevant placement policy.

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

class arcapix.management.tasks.profiletasks.ProfileUpdater(*args: Any, **kwargs: Any)

Task to update a profile.

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform()

Update the policy rules corresponding to the profile.

Essentially just finds rules in the placement policy corresponding to Profile ILM Steps and modifies their associated inline comments to include the new name

post(resultstatus): Save changes to policy.

pre(): Load relevant placement policy.

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

Provider Tasks

class arcapix.management.tasks.providertasks.BoxProviderBuilder(*args: Any, **kwargs: Any)

Create a Box provider.

This only sets config values in PixStor salt. Replicas are responsible for initialising data sync.

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform()

Main part of task execution.

Returns: the state of the resource after perform completes. default: ACTIVE

post(resultstatus)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre()

Pre-flight.

Sets the resource’s status to INPROGRESS

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

class arcapix.management.tasks.providertasks.BoxProviderDeleter(*args: Any, **kwargs: Any)

Delete a Box Provider.

If the provider is being used to replicate spaces, it can’t be deleted.

cancel_delete(): Set the instance to not (hard) delete.

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform()

Main part of task execution.

Returns: the state of the resource after perform completes. default: ACTIVE

post(resultstatus)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre()

Pre-flight.

Sets the resource’s status to INPROGRESS

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

should_delete()

Set the instance to soft deleted.

This will cause the instance to be hard-deleted iff it is in an ACTIVE state at the end of the task.

class arcapix.management.tasks.providertasks.BoxProviderTask(*args: Any, **kwargs: Any)

Baseclass with common logic for Box provider tasks.

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform()

Main part of task execution.

Returns: the state of the resource after perform completes. default: ACTIVE

post(resultstatus)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre()

Pre-flight.

Sets the resource’s status to INPROGRESS

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

class arcapix.management.tasks.providertasks.BoxProviderUpdater(*args: Any, **kwargs: Any)

Update a Box Provider.

This might be used to re-authenticate expired tokens.

Theoretically, this could be used to change sync to use a different account. If that new account doesn’t have access to the same (Box) folders, the sync results could get messy.

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform()

Main part of task execution.

Returns: the state of the resource after perform completes. default: ACTIVE

post(resultstatus)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre()

Pre-flight.

Sets the resource’s status to INPROGRESS

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

arcapix.management.tasks.providertasks.authenticate(auth_code): Get access tokens and username from an auth_code.

Replica Tasks

class arcapix.management.tasks.replicatasks.BoxSyncReplicaBuilder(*args: Any, **kwargs: Any)

Configure and initialise downstream replica from Box to PixStor.

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform()

Main part of task execution.

Returns: the state of the resource after perform completes. default: ACTIVE

post(status)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre()

Pre-flight.

Sets the resource’s status to INPROGRESS

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

class arcapix.management.tasks.replicatasks.BoxSyncReplicaDeleter(*args: Any, **kwargs: Any)

Stop and disable downstream BoxSync replica.

cancel_delete(): Set the instance to not (hard) delete.

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform()

Main part of task execution.

Returns: the state of the resource after perform completes. default: ACTIVE

post(status)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre()

Pre-flight.

Sets the resource’s status to INPROGRESS

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

should_delete()

Set the instance to soft deleted.

This will cause the instance to be hard-deleted iff it is in an ACTIVE state at the end of the task.

class arcapix.management.tasks.replicatasks.BoxSyncReplicaTask(*args: Any, **kwargs: Any)

Baseclass with common logic for BoxSync replica tasks.

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform()

Main part of task execution.

Returns: the state of the resource after perform completes. default: ACTIVE

post(resultstatus)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre()

Pre-flight.

Sets the resource’s status to INPROGRESS

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

class arcapix.management.tasks.replicatasks.RcloneError(*args: Any, **kwargs: Any): Exception raised when rclone exits with an error.

class arcapix.management.tasks.replicatasks.RcloneReplicaBuilder(*args: Any, **kwargs: Any)

Configure and initialise upstream replica from PixStor to Box.

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform()

Main part of task execution.

Returns: the state of the resource after perform completes. default: ACTIVE

post(status)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre()

Pre-flight.

Sets the resource’s status to INPROGRESS

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

class arcapix.management.tasks.replicatasks.RcloneReplicaDeleter(*args: Any, **kwargs: Any)

Stop and disable upstream Rclone replica.

cancel_delete(): Set the instance to not (hard) delete.

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform()

Main part of task execution.

Returns: the state of the resource after perform completes. default: ACTIVE

post(status)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre()

Pre-flight.

Sets the resource’s status to INPROGRESS

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

should_delete()

Set the instance to soft deleted.

This will cause the instance to be hard-deleted iff it is in an ACTIVE state at the end of the task.

class arcapix.management.tasks.replicatasks.RcloneReplicaTask(*args: Any, **kwargs: Any)

Baseclass with common logic for Rclone replica tasks.

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform()

Main part of task execution.

Returns: the state of the resource after perform completes. default: ACTIVE

post(resultstatus)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre()

Pre-flight.

Sets the resource’s status to INPROGRESS

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

arcapix.management.tasks.replicatasks.check_conflicting_replicas(task, direction)

Check if there are any existing replicas which conflict with the task.

A replica is conflicting if it is a BoxSync replica of the same space syncing either the same exact direction or bidirectional.

Oposite direction replicas of the same space, or replicas of different types are fine.

ref: RNDNXSREST-521

Raises: OperationNotAllowed if any conflicting replicas exist

arcapix.management.tasks.replicatasks.get_space_fileset(space): Get the fileset corresponding to a space.

arcapix.management.tasks.replicatasks.rclone(source, target, *, args=None)

Perform an rclone, using the BoxSync UI rclone config.

Params args: list of additional rclone cli flags

arcapix.management.tasks.replicatasks.systemctl_disable(service_name): Stop and disable a systemd service.

arcapix.management.tasks.replicatasks.systemctl_enable(service_name): Start and enable a systemd service.

arcapix.management.tasks.replicatasks.try_remove_boxsync_db(root): Try to remove the BoxSync DB and log an error if it fails.

Space Tasks

class arcapix.management.tasks.spacetasks.SpaceBuilder(*args: Any, **kwargs: Any)

Task to create a new space.

classmethod default_requirements(): List of resources required by this space builder.

get_db(): Gets a database session object for the database associated with the task

get_exposer(): Helper method - find a GPFSNativeExposer in the space’s exposers list.

get_id(): Get the resource id associated with this task.

get_link_path()

Get the link path for the space’s native exposer.

Returns None if the space doesn’t have a native exposer (unlinked)

get_placement_filesystem(): Get the space’s placement filesystem.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform()

Creates an Independent or Dependent Fileset.

Also sets up any profile, template, etc.

post(resultstatus)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre()

Pre-flight.

Sets the resource’s status to INPROGRESS

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

class arcapix.management.tasks.spacetasks.SpaceDeleter(*args: Any, **kwargs: Any)

Task to delete a space.

This can only work with EVE versions which support SOFT_DELETE (>=0.6)

cancel_delete(): Set the instance to not (hard) delete.

get_db(): Gets a database session object for the database associated with the task

get_exposer(): Helper method - find a GPFSNativeExposer in the space’s exposers list.

get_id(): Get the resource id associated with this task.

get_link_path()

Get the link path for the space’s native exposer.

Returns None if the space doesn’t have a native exposer (unlinked)

get_placement_filesystem(): Get the space’s placement filesystem.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform(): Unlink and delete the relevant fileset.

post(resultstatus)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre()

Pre-flight.

Sets the resource’s status to INPROGRESS

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

should_delete()

Set the instance to soft deleted.

This will cause the instance to be hard-deleted iff it is in an ACTIVE state at the end of the task.

class arcapix.management.tasks.spacetasks.SpaceUpdater(*args: Any, **kwargs: Any)

Tasks to update Space properties.

get_db(): Gets a database session object for the database associated with the task

get_exposer(): Helper method - find a GPFSNativeExposer in the space’s exposers list.

get_id(): Get the resource id associated with this task.

get_link_path()

Get the link path for the space’s native exposer.

Returns None if the space doesn’t have a native exposer (unlinked)

get_placement_filesystem(): Get the space’s placement filesystem.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform(): Change fileset properties as requested.

post(resultstatus)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre()

Pre-flight.

Sets the resource’s status to INPROGRESS

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

arcapix.management.tasks.spacetasks.get_path_inode_space(filesystem: str, path: str): Get the inode space that an arbitrary path belongs to.

arcapix.management.tasks.spacetasks.rollback_unlink(fset): Unlink fileset, with rollback if there’s an error in context.

Snapshots Tasks

class arcapix.management.tasks.snapshottasks.SnapshotBuilder(*args: Any, **kwargs: Any)

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform()

Main part of task execution.

Returns: the state of the resource after perform completes. default: ACTIVE

post(resultstatus)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre()

Pre-flight.

Sets the resource’s status to INPROGRESS

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

class arcapix.management.tasks.snapshottasks.SnapshotDeleter(*args: Any, **kwargs: Any)

cancel_delete(): Set the instance to not (hard) delete.

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform()

Main part of task execution.

Returns: the state of the resource after perform completes. default: ACTIVE

post(resultstatus)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre()

Pre-flight.

Sets the resource’s status to INPROGRESS

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

should_delete()

Set the instance to soft deleted.

This will cause the instance to be hard-deleted iff it is in an ACTIVE state at the end of the task.

Template Tasks

class arcapix.management.tasks.templatetasks.TemplateBuilder(*args: Any, **kwargs: Any)

Task to create a template.

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform()

Create the template from a provided path.

Copies the template from a user specified location to {template_store}/{template_name}

post(resultstatus)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre()

Pre-flight.

Sets the resource’s status to INPROGRESS

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

class arcapix.management.tasks.templatetasks.TemplateDeleter(*args: Any, **kwargs: Any)

Task to delete a template.

Warning

You can’t delete a template that is installed in any spaces.

If you attempt to delete an installed template, this task will raise an exception and the template will enter an ERRORED state.

The ERRORED template will still be usuable, however.

cancel_delete(): Set the instance to not (hard) delete.

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform(): Delete the template from the template store directory.

post(resultstatus)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre()

Pre-flight.

Sets the resource’s status to INPROGRESS

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

should_delete()

Set the instance to soft deleted.

This will cause the instance to be hard-deleted iff it is in an ACTIVE state at the end of the task.

class arcapix.management.tasks.templatetasks.TemplateTooBig(*args: Any, **kwargs: Any): Exception raised when trying to create a template from a directory tree bigger than the configured size limit.

class arcapix.management.tasks.templatetasks.TemplateUpdater(*args: Any, **kwargs: Any)

Task to update a template.

get_db(): Gets a database session object for the database associated with the task

get_id(): Get the resource id associated with this task.

get_resource(): Get the resource associated with this task.

get_target_item()

Get the relative url of the items associated with this task.

Used by schedulers to provide back-reference links

perform()

Update the template name or replace it from a different target.

This is called before an item is updated

Future values are held in self.todo

post(resultstatus)

Post-flight.

Parameters: resultstatus – the status of the resource, as returned from perform.

pre()

Pre-flight.

Sets the resource’s status to INPROGRESS

run()

Execute the task.

Executes: pre -> perform -> post

Also takes care of boilerplate, including database interactions and error handling.

arcapix.management.tasks.templatetasks.copy_template(source, dest)

Calls the copy_template script.

The copy_template script should be added to sudoers so copying templates can be done by a non-root user and still preserve file permissions

arcapix.management.tasks.templatetasks.delete_template(location)

Calls the delete_template script.

The delete_template script should be added to sudoers so deleting templates can be done by a non-root user and still preserve file permissions

arcapix.management.tasks.templatetasks.move_template(source, dest)

Calls the move_template script.

The move_template script should be added to sudoers so moving templates can be done by a non-root user and still preserve file permissions

Template Tools

The following are command line tools which are used by tasks. They need to be run as root to be able to manipulate templates, regardless of the permissions of the objects in the templates.

Important

Ideally, these scripts should be added to the sudoers file for the user which runs the management REST server (i.e. the user that tasks are run as).

Copy Template

class arcapix.management.tasks.copy_template.CopyNotAllowed(*args: Any, **kwargs: Any): Exception raised when attempting an invalid copy operation.

arcapix.management.tasks.copy_template.copy_gpfs_acl(source, dest): Copy gpfs acls using mm-commands.

arcapix.management.tasks.copy_template.copy_perms(source, dest): Copy acls and ownership from one file to another.

arcapix.management.tasks.copy_template.get_inode_space(fs, path)

Get the inode space a path belongs to.

Path must be a fileset link path.

arcapix.management.tasks.copy_template.get_pool(fset)

Polls the filesystem PlacecmentPolicy to try to figure out what pool files in ‘fset’ are assigned to.

Fileset placement can be specified either as “FOR FILESET …” or “WHERE FILESET_NAME …”

If no specific placement rule exists, the default placement rule is used.

Any pool macros are resolved.

Parameters: fset (Filesets object) – fileset to find placement pool for
Returns: pool name

arcapix.management.tasks.copy_template.is_parent_dir(parent, child): Check if a path is a parent directory of another path.

Delete Template

delete_template – tool for removing a template from the template store.

This is essentially just a way of calling shutil.rmtree with sudo.

However, that is potentially very dangerous, so we have various checks to make sure the script can ONLY be used to uninstall templates

arcapix.management.tasks.delete_template.delete_template(location): Remove a template from the template store.

Move Template

move_template – tool for moving (renaming) templates in the template store.

This is essentially just a way of calling os.rename with sudo.

However, that is potentially very dangerous, so we have various checks to make sure the script can ONLY be used to move templates WITHIN the configure template store

class arcapix.management.tasks.move_template.MoveNotAllowed(*args: Any, **kwargs: Any): Exception raised when attempting an invalid move operation.

arcapix.management.tasks.move_template.move_template(source, target): Move (rename) a template within the template store.

Uninstall Template

class arcapix.management.tasks.uninstall_template.UninstallNotAllowed(*args: Any, **kwargs: Any)

arcapix.management.tasks.uninstall_template.check_installed_template(template_path, space_path)

Check if a template installed in a space is different from the original template.

If the template is the same, it contains no user data and we can safely remove it.

arcapix.management.tasks.uninstall_template.files_match(source, dest): Compare two files based on stat attributes.

arcapix.management.tasks.uninstall_template.uninstall_template(template_path, space_path)

Remove a template from a space.

Assumes the space has been checked and the installed template is ‘empty’

Populate Tasks

These are methods for populating or reconciling the database based on the current state of the filesystem.

Hint

Use the adminctl populate tool to perform populate operations

class arcapix.management.tasks.populatetasks.ReconcileDBTask(*args: Any, **kwargs: Any)

Task to perform database reconcile

To be run via the job engine

get_db(): Gets a database session object for the database associated with the task

perform(session): Bring the database up to date with the cluster

pre_submit(context)

Check the requesting user has the required rights/roles.

This check is only applied when called as a task in Schedulers. It is not applied if reconcile is performed via adminctl since adminctl is not (apcore-auth) authenticated.

arcapix.management.tasks.populatetasks.check_placement_rules_are_installed(cluster): Check whether any placement pools are missing default placement rules.

arcapix.management.tasks.populatetasks.create_default_placement_rules(cluster): Create a default placement rule for each data pool in each filesystem.

arcapix.management.tasks.populatetasks.create_empty_tables(db): Initialise the tables (schema) in an empty db.

arcapix.management.tasks.populatetasks.make_clean_db(path=None)

Create an empty database file.

Note - this doesn’t initialise tables. Use py:func:create_empty_tables

If the target path already exists, it will be replaced.