[kv] Support asynchronous manual compaction for primary-key tables #4769

Search before asking

• I searched in the issues and found nothing similar.

Motivation

Primary-key tables need an operator-controlled way to compact their RocksDB state and rewrite existing SSTs when storage settings change. For example, enabling partitioned indexes changes how new SSTs are written, while existing SSTs retain their original layout until rewritten. Operators should be able to migrate those files explicitly, skip rewriting files that already match, and observe a durable result.

The same interface should support LSM optimization and reclamation of eligible obsolete versions, tombstones, and expired data. These operations can take time and need asynchronous submission, bounded execution, progress queries, cancellation, and restart handling.

Solution

Provide asynchronous manual compaction for ordinary primary-key tables at partition, table, database, and cluster scope, through Java Admin methods and Flink SQL procedures.

• OPTIMIZE: compact the LSM using the existing compaction-filter policy.
• RECLAIM: explicitly process bottommost data to reclaim eligible state; do not promise a particular number of reclaimed bytes.
• MIGRATE: inspect live SST properties and rewrite only when the supported layout differs from the effective cluster configuration. Do not accept caller-supplied index settings.

Return an operation ID and expose list/get/cancel interfaces, raw bucket counts, current execution phases, and the first error. Allow one active operation per cluster and one bucket at a time per TabletServer, without a waiting task queue. Fix the target bucket set at submission and deduplicate identical active requests.

A bucket succeeds only after its post-operation KV snapshot is uploaded, its commit is confirmed, and the completion record is durably published. Even a no-op MIGRATE must create a new snapshot. Persist dispatch before starting work, and never replay a dispatched bucket after leader movement or Coordinator recovery. Cancellation is best-effort; the TS execution slot and local resources remain owned until actual execution and cleanup exit.

Store bounded summaries and remote metadata handles in ZooKeeper, and keep large plans/progress in remote files with retention cleanup. Reuse existing disk and write-pressure admission guards and coalesce normal progress publication on the control cycle.

Anything else?

Related: #4607 and #4612 introduce partitioned metadata configuration and mixed-layout compatibility. Manual MIGRATE provides the explicit maintenance step for existing SSTs.

Willingness to contribute

• I'm willing to submit a PR!