Repository navigation
AggregateFnProperties #7843
Description
pub struct AggregateFnProperties {
pub order_sensitivity: AggregateOrderSensitivity,
pub partial_aggregation: PartialAggregation,
pub distinct_sensitivity: DistinctSensitivity,
}
pub enum AggregateOrderSensitivity {
Insensitive, // planner may reorder input rows
Sensitive, // input order is part of the result
}
pub enum PartialAggregation {
Unsupported, // do not introduce partial/final aggregation
OrderedMerge, // may split contiguous input, must merge partials in input order
UnorderedMerge, // may split/repartition and merge partials in any order/tree
}
pub enum DistinctSensitivity {
Sensitive,
Insensitive, // agg(DISTINCT x) == agg(x)
}
trait AggregateFnVTable {
...
fn properties(&self, options: &Self::Options, input_dtype: &DType) -> AggregateFnProperties;
}| Aggregate | Properties |
|---|---|
| count, nan_count | order-insensitive, unordered-merge |
| min_max | order-insensitive, unordered-merge, distinct-insensitive |
| is_constant | likely order-insensitive, unordered-merge |
| first, last, is_sorted | order-sensitive, ordered-merge |
| sum(bool), sum(unsigned) | order-insensitive, unordered-merge |
| sum(signed), sum(decimal), sum(float) | order-sensitive, partial aggregation unsupported under strict Vortex semantics |
| mean | inherits sum’s dtype caveats |
Activity
joseph-isaacs commentedon May 8, 2026
We are really asking the question AggregateOrderSensitivity is that operation commutative?
For
PartialAggregationwe are asking is the merge over the partialOrderedMerge-> associative andUnorderedMerge-> associative + commutativeFor
DistinctSensitivityis the partialidempotentjoseph-isaacs commentedon May 8, 2026
we could get away with
joseph-isaacs commentedon May 8, 2026
Or maybe we have a single operator struct
Then this can be used for aggregations and overwritten for partial merge since aggregation op properties ==> partial merge properties.
gatesn commentedon May 8, 2026
I think you want to stick to the domain-specific meaning, rather than infer it from low level properties.
e.g. is
last(...)idempotent? I find I have to think much harder about it...joseph-isaacs commentedon May 8, 2026
its not, but its not clear in both cases?
joseph-isaacs commentedon May 8, 2026
Okay but can we have the same meaning for both operators. agg and lifted agg (over partial).
If we use associative and commutative then we can use the same definition for aggregations and scalar_fns. I guess we could infer these but right now these are a little muddled.