Support Map as a ScalarValue #11128

Activity

1. Blizzara

   added

   enhancementNew feature or requestNew feature or request

   on Jun 26, 2024
2. Blizzara

   added a commit that references this issue on Jun 26, 2024

   feat: Support Map type in Substrait conversions

   ...

   1fa6596
3. goldmedal commented on Jun 27, 2024

   

   goldmedal

   on Jun 27, 2024

   Contributor

   More actions

   take

   🎉React with 🎉1Reacted by Arttu
4. github-actions

   assigned

   goldmedal

   on Jun 27, 2024
5. goldmedal commented on Jun 27, 2024

   

   goldmedal

   on Jun 27, 2024

   Contributor

   More actions

   I'm interested in this issue.
   As @alamb said in #8262 (comment) :

   l I think ScalarValue::Map can follow the model of ScalarValue::List (as Maps are just lists of structs, as I undertand):

   I think I can used a list of structs with the same type struct to implemented a basic version like:

   List<Struct<K, V>>
   

   Then, provide the HashMap (or others) implementation for the physical plan.
   Does it make sense?
6. alamb commented on Jun 27, 2024

   

   alamb

   on Jun 27, 2024

   Contributor

   More actions

   I think ScalarValue::Map should follow the same pattern as ScalarValue::List ScalarValue::Struct

   datafusion/datafusion/common/src/scalar/mod.rs

   Line 244 in 8216e32

   List(Arc<ListArray>),

   So that would mean a single element MapArray: https://docs.rs/arrow/latest/arrow/array/struct.MapArray.html

   enum ScalarValue {
     ...
     Map(Arc<MapArray>)
     ...
   }

   Does that make sense?

   ❤️React with ❤️2Reacted by Jax Liu and Dharan Aditya
7. goldmedal commented on Jun 27, 2024

   

   goldmedal

   on Jun 27, 2024

   Contributor

   More actions

   enum ScalarValue {
     ...
     Map(Arc<MapArray>)
     ...
   }

   Does that make sense?

   Many thanks! It looks good to me. I didn't notice this struct. I will try to follow this pattern to implement this.

   👍React with 👍1Reacted by Andrew Lamb
8. alamb

   added a commit that references this issue on Jun 27, 2024

   feat: Support Map type in Substrait conversions (#11129)

   ...

   f58df32
9. alamb

   closed this as completedin #11224on Jul 3, 2024
10. findepi

    added a commit that references this issue on Jul 16, 2024

    feat: Support Map type in Substrait conversions (apache#11129)

    ...

    878926a