2.8. Troubleshooting

2.8.1. Common Issues

Symptom	Likely Cause	Fix
Task stuck in Pending	Task does not return status.dump() or an equivalent dict.	Ensure the task returns a valid payload. Check Worker logs for tracebacks.
Task marked as Failed	Unhandled exception in the task function.	Add try/except around processing logic. Mark failed paths with status.add("failures", ...).
Job progress not updating	Missing "jobid" in the return payload.	Ensure jobid is passed to FileActionStatus and included in the return dict.
Inaccurate statistics in Job UI	"summary" counts do not match "details" lists in the return payload.	Use FileActionStatus to avoid manual counting errors.
“Missing” files in job report	Not all input paths are accounted for in the return payload.	Ensure every path in the input list is categorised into a state (processed, skipped, failures, or aborted).
Plugin not listed after install	Version number not bumped in setup.py, or service not restarted.	Increment the version in setup.py, reinstall, and run systemctl restart ngenea-worker.
Dependency conflict on install	Plugin requests a version of a constrained package (e.g., celery) that conflicts with Ngenea Worker requirements.	Adjust the dependency version in setup.py to be compatible. Check constraints.txt for required versions.
Task not found in workflow	Task name in the workflow definition does not match the name parameter in the @app.task decorator.	Ensure the names match exactly (e.g., dynamo.custom.my_task).
Job reports “ONGOING” after completion	Dynamic children spawned without proper will_spawn_children tracking.	Use DynamicSubmitter which handles this automatically. See Dynamic Children.

2.8.2. Logging

Use Python’s standard logging module in your tasks. Log output appears in the Ngenea Worker service logs:

import logging

logger = logging.getLogger(__name__)

@app.task(bind=True, name="dynamo.custom.my_task")
def my_task(self, *args, paths=None, jobid=None, **kwargs):
    logger.info("Starting task %s for job %s with %d paths", self.name, jobid, len(paths))

    for item in paths:
        logger.debug("Processing: %s", item["path"])
        # ...

View logs with:

journalctl -u ngenea-worker -f

You can also add messages to the Ngenea Hub job detail view using status.add_log():

status.add_log("Connected to external API successfully")
status.add_log("Processed 150 of 200 files")

These messages are visible in the UI alongside the job progress.

2.8.3. Inspecting Return Payloads

During development, it can be helpful to log the return payload to verify its structure:

result = status.dump()
logger.debug("Return payload: %s", result)
return result

The expected structure is documented in The Task Contract. Key things to verify:

• "jobid" is present and correct.

• "status" > "summary" counts match "status" > "details" list lengths.

• "paths" contains only the paths to forward to the next task (typically processed and skipped).

• Every input path appears in exactly one state category.