apiserver: completed `resolve-package-metadata` task seems to get rescheduled indefinitely

[migus-rs]

Current Behavior

Checking logs I noticed a specific warning message occurring repeatedly (every ~50-90s).

A successfully completed activity task with ID ActivityTaskId[queueName=default, workflowRunId=019edf6f-b706-766e-8743-90e97c654731, createdEventId=0] could not be deleted, likely because its lock expired, causing another worker to pick it up. Expect duplicate task executions. To prevent this from happening, consider increasing the activity's lock timeout, and sending heartbeats more frequently.

This seems to have been ongoing for a month now, as can also be seen in the UI when checking the UUID:

I was unable to find any notes in documentation (here and here) about how to tweak the lock timeout as the message states (or if it is even possible).

There also seems no way to manually cancel this (seemingly stuck) operation.

Seeing #6740 reminded me of this behavior. That issue only mentions policy evaluation, though.
Feel free to mark this a duplicate if this refers to the same problem.

Steps to Reproduce

1. Run DTv5 with some projects.
2. Observe some tasks getting rescheduled repeatedly.

Expected Behavior

All workflow tasks should be able to progress or fail within a reasonable time frame.

Dependency-Track Version

5.x

Browser

N/A

Checklist

• I have read and understand the contributing guidelines
• I have checked the existing issues for whether this defect was already reported

[nscuro]

It looks like the candidate query (i.e. "which components need metadata resolution?") takes too long in your case. Working on a fix that should also cover #6654. #6740 may sound related but fundamentally the candidate query here should never take even remotely as long as the default lock timeout.