Skip to content

Implement nested identifier access ( "Nested identifiers not yet supported" ) #11445

Closed

Description

@alamb

Is your feature request related to a problem or challenge?

This came up on discord: https://discord.com/channels/885562378132000778/885562378132000781/1261359404197089443

> create or replace view foo as select  {'field1': { 'subfield1': 'foo', subfield2: 'bar', subfield3: [10,20,30,40]}} as "the_column";
0 row(s) fetched.
Elapsed 0.001 seconds.

> select * from foo;
+-------------------------------------------------------------------------+
| the_column                                                              |
+-------------------------------------------------------------------------+
| {field1: {subfield1: foo, subfield2: bar, subfield3: [10, 20, 30, 40]}} |
+-------------------------------------------------------------------------+
1 row(s) fetched.
Elapsed 0.002 seconds.

> select the_column.field1.subfield3[2] from foo;
Internal error: Nested identifiers not yet supported for column foo.the_column.
This was likely caused by a bug in DataFusion's code and we would welcome that you file an bug report in our issue tracker

Describe the solution you'd like

The query should work and return 20

Describe alternatives you've considered

Single level access works fine:

> create or replace view foo as select { 'subfield1': 'foo', subfield2: 'bar', subfield3: [10,20,30,40]} as "field1" ;
0 row(s) fetched.
Elapsed 0.008 seconds.

> select * from foo;
+---------------------------------------------------------------+
| field1                                                        |
+---------------------------------------------------------------+
| {subfield1: foo, subfield2: bar, subfield3: [10, 20, 30, 40]} |
+---------------------------------------------------------------+
1 row(s) fetched.
Elapsed 0.003 seconds.

> select field1.subfield3[2] from foo;
+----------------------------------------------------+
| get_field(foo.field1, Utf8("subfield3"))[Int64(2)] |
+----------------------------------------------------+
| 20                                                 |
+----------------------------------------------------+
1 row(s) fetched.
Elapsed 0.002 seconds.

Additional context

No response

Activity

  1. changed the title [-]"Nested identifiers not yet supported" error[/-] [+]Implement nested identifier access ( "Nested identifiers not yet supported" )[/+] on Jul 12, 2024
  2. kavirajk commented on Jul 17, 2024

    @kavirajk
    Contributor

    take

  3. kavirajk commented on Jul 22, 2024

    @kavirajk
    Contributor

    @alamb kinda stuck on this. I track down where the error is coming from.

    But don't know which datatype (I assume Datatype::Struct?) methods needs to be modified to allow nested access here in getfield.rs. Appreciate any leads/tips? thanks.

  4. alamb commented on Jul 22, 2024

    @alamb
    ContributorAuthor

    Sorry @kavirajk I am not super familar with the struct access so I can't offer any code specific suggestions

    What I would probably do is remove the check, then write some tests, and then work to get all the tests passing.

  5. kavirajk commented on Jul 22, 2024

    @kavirajk
    Contributor

    Thanks @alamb good idea.

  6. Lordworms commented on Sep 23, 2024

    @Lordworms
    Contributor

    based on the duckdb implementation, I think implement struct_extract function first might be a good start, I'll add that
    image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions