You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
LayoutWriter doesn't respect validity of the top level Struct array #710
This is likely not a good first issue. The solution here is likely to either forbid writing top level structs with validity other than nonnullable and provide a function to push validity of the struct to the children, i.e. run a filter. Alternatively we need to teach column layout about one extra validity array which shouldn't be bad but likely unnecessary
In the current format, isn't it true that this is actually any struct array? I'd like to at least known the fix here to make sure it's not a breaking change when we do.
Activity
doki23 commentedon Oct 7, 2024
Could you please provide more details? I'm interested in this issue and would like to attempt to resolve it.
robert3005 commentedon Oct 15, 2024
This is likely not a good first issue. The solution here is likely to either forbid writing top level structs with validity other than nonnullable and provide a function to push validity of the struct to the children, i.e. run a filter. Alternatively we need to teach column layout about one extra validity array which shouldn't be bad but likely unnecessary
gatesn commentedon Apr 14, 2025
In the current format, isn't it true that this is actually any struct array? I'd like to at least known the fix here to make sure it's not a breaking change when we do.
gatesn commentedon May 29, 2025
We should assert at write-time that struct arrays have
Validity::NonNullableorValidity::AllValidStructLayout requires chunks to be all valid (#3407)