Comet Native Executors: cancel a query on the cluster when Spark cancels it #7372

What is the problem the feature request solves?

When a Spark job running on a Comet Native Executor cluster (#7289) is cancelled, for example from the Spark UI, by SparkContext.cancelJob or by a Spark Connect interrupt, the query keeps running on the cluster. The driver waits in NativeExecutorsClient.submit until the whole query ends, and nothing tells the scheduler to stop it.

Describe the potential solution

Submit the query without waiting for it, keep its job id, and cancel the Ballista job when Spark cancels the task or job that's waiting on it, through a Flight action on the scheduler that calls Ballista's job cancellation.

Additional context

The scheduler's Flight service is native/native-executors/src/frontend.rs, which waits for each job in run.

Part of #7288.

Willingness to contribute

Yes. I can contribute this feature independently.