partitioning: store start and end bounds in Partitioning::Range #26509

Is your feature request related to a problem or challenge?

See datafusion-contrib/datafusion-distributed#628 (comment)

Original Data

Distributed Query

There's no way to split up / distribute Partitioning::Range without losing information (losing a split point).

This makes it so that coalese operations must use Partitionoing::Unknown because we cannot prove the partitions are dijoint ranges. Similar issues for shuffles.

Describe the solution you'd like

I want to be able to store the start and end bounds in Partitioning::Range. This is a pretty easy migration, we can default to -inf and inf.

That will unlock well defined ranges.

Describe alternatives you've considered

For coalesces, we use Partitioning::Unknown for now, but I would like to use range partitioning if the ranges can be joined together.

Say I'm distributing 13 partitions, A–M, with boundaries 10, 20, …, 120 across 4 workers with 4 partitions each.

  Task 0: [ A |10| B |20| C |30| D ]     splits: [10, 20, 30]
  Task 1: [ E |50| F |60| G |70| - ]     splits: [50, 60, 70]
  Task 2: [ H |80| I |90| J |100| - ]    splits: [80, 90, 100]
  Task 3: [ - |100| K |110| L |120| M ]  splits: [100, 110, 120]

I have to duplicate a split point, 100, which is a smell.

Additional context

No response

Willingness to contribute

None