4.6. Iris Configuration¶

This page describes how to configure Ngenea Hub to integrate with the Iris product.

4.6.1. Iris Tasks and RabbitMQ¶

Ngenea Hub provides a set of tasks — also known as ‘publishers’ — for generating and extracting file metadata. These tasks integrate with Iris by publishing messages to the Iris RabbitMQ.

The configurations for these Iris tasks to connect with RabbitMQ are defined in /etc/ngenea/iris-rabbitmq.yaml.

Important

In a Pixstor environment, when Iris is enabled, this file is pre-configured and should not be edited.

The config file is divided into several sections, described below.

4.6.1.1. RabbitMQ¶

The rabbitmq section defines general connection settings for RabbitMQ. These settings apply to all publishers.

host — Hostname or IP address of the RabbitMQ instance to connect to.
port — Port of the RabbitMQ instance, typically 5671.
username — Username to authenticate the connection to RabbitMQ (default: guest).
password — Password for the username used to connect to RabbitMQ (default: guest).
use_ssl — Whether to connect to RabbitMQ using SSL (default: false).
ca_cert_path — Path to the certificate authority file for SSL connections.
client_cert_path — Path to the client certificate file, signed by the configured certificate authority.
client_key_path — Path to the client key file matching the configured client certificate.
virtual_host — Virtual host to use when connecting to RabbitMQ (default: /).
max_retries — Number of connection attempts if a connection cannot be achieved (default: 3).
retry_interval — Time to wait between connection attempts, in seconds (default: 5).
log_level — Controls the level of RabbitMQ connection logging in Ngenea Worker logs. Values range from 0 (least verbose; default) to 2 (most verbose).

Warning

password is stored in plaintext. Ensure permissions on iris-rabbitmq.yaml prevent unwanted access.

Example rabbitmq section:

rabbitmq:
    host: localhost
    port: 5671

    username: guest
    password: guest

    use_ssl: true
    ca_cert_path: /path/to/rabbitmq/tls/ca.crt
    client_cert_path: /path/to/rabbitmq/tls/tls.crt
    client_key_path: /path/to/rabbitmq/tls/tls.key

    virtual_host: /

    max_retries: 3
    retry_interval: 5

    log_level: 0

4.6.1.2. Publishers¶

The publishers section defines which RabbitMQ exchange the ‘publishers’ — Iris tasks — should publish their messages to.

There is one subsection per publisher, and each subsection defines the following:

exchange_name — Name of the RabbitMQ exchange to publish messages to.

Currently, there are three publishers:

filenotify — Generates file system change events, such as when a file is created or a directory is moved.
mediainfo — Extracts media file metadata using the mediainfo tool.
exiftool — Extracts file metadata using the exiftool tool.

Example publishers section:

publishers:
    filenotify:
        exchange_name: fs-events-xchg
    mediainfo:
        exchange_name: metadata-xchg
    exiftool:
        exchange_name: metadata-xchg

4.6.1.3. Messages¶

The messages section defines per-publisher metadata.

There is one subsection per publisher, and each subsection defines the following key:

endpoint_url — URL of the Versity endpoint which exposes files processed by this publisher.

Typically, the endpoint_url has the form https://VERSITY_GATEWAY:7070. It will normally be the same for all publishers.

The existing publishers are listed in the Publishers section above.

Example messages section:

messages:
    filenotify:
        endpoint_url: https://10.100.0.100:7070
    mediainfo:
        endpoint_url: https://10.100.0.100:7070
    exiftool:
        endpoint_url: https://10.100.0.100:7070

4.6.1.4. Error Handling¶

The error_handling section defines how tasks should handle errors when processing files.

There is one subsection per task (publisher), and each subsection defines the following key:

on_error_set_aborted — Whether to mark a file as aborted (true) or failure (false) when the file cannot be processed.

When files are marked as failed, the whole job is also marked as failed. When files are marked as aborted, the job will be marked as successful (unless other, unrelated failures occur in the job).

The existing publishers are listed in the Publishers section above.

Additionally, on_error_set_aborted can be set at the top level of the error_handling section to apply the same setting to all tasks.

Example error_handling section:

error_handling:
    on_error_set_aborted: true

    exiftool:
        on_error_set_aborted: true
    mediainfo:
        on_error_set_aborted: false

In the above example, the exiftool task (and any others not explicitly included) will inherit the top-level on_error_set_aborted: true.

4.6.2. Iris Workflow Queue¶

When a space is enabled for Iris in Ngenea Hub, a scheduled workflow is created which runs the Iris tasks mentioned in the previous section.

This scheduled workflow is configured to run on the dedicated iris_notify queue. This means that the notify workflow will not block or be blocked by any other tasks running at the same time in Ngenea Hub, such as file sync.

The Iris queue must be defined in the Ngenea Worker configuration file, as described in Queues.

For example:

[Queue iris_notify]
threads = 10

If the queue is not defined in the worker config or is removed, the Iris tasks will not execute.

Note

In a Pixstor deployment, this queue is defined automatically. Configuration changes, such as altering the number of threads, must be done via Pixstor