Skip to content

To to_dict, or to not aim to support Altair for now? #133

Closed

Description

@MarcoGorelli

The way altair is able to support different DataFrame libraries is via the interchange protocol:

https://github.com/altair-viz/altair/blob/d7c0b228ba005851a0aaa83c3ed86621d0e41a37/altair/utils/data.py#L202-L213

    elif isinstance(data, pd.DataFrame):
        data = sanitize_dataframe(data)
        return data.to_json(orient="records", double_precision=15)
    elif isinstance(data, dict):
        if "values" not in data:
            raise KeyError("values expected in data dict, but not present.")
        return json.dumps(data["values"], sort_keys=True)
    elif hasattr(data, "__dataframe__"):
        # experimental interchange dataframe support
        pi = import_pyarrow_interchange()
        pa_table = pi.from_dataframe(data)
        return json.dumps(pa_table.to_pylist())

Example of how this looks:

from vega_datasets import data
import pyarrow.interchange as pi
import polars as pl

cars = data.cars().head(2)
cars_pl = pl.from_pandas(cars)
print(cars.to_dict(orient='records'))

pa_table = pi.from_dataframe(cars_pl)
print(pa_table.to_pylist())

shows

[{'Name': 'chevrolet chevelle malibu', 'Miles_per_Gallon': 18.0, 'Cylinders': 8, 'Displacement': 307.0, 'Horsepower': 130.0, 'Weight_in_lbs': 3504, 'Acceleration': 12.0, 'Year': Timestamp('1970-01-01 00:00:00'), 'Origin': 'USA'}, {'Name': 'buick skylark 320', 'Miles_per_Gallon': 15.0, 'Cylinders': 8, 'Displacement': 350.0, 'Horsepower': 165.0, 'Weight_in_lbs': 3693, 'Acceleration': 11.5, 'Year': Timestamp('1970-01-01 00:00:00'), 'Origin': 'USA'}]

[{'Name': 'chevrolet chevelle malibu', 'Miles_per_Gallon': 18.0, 'Cylinders': 8, 'Displacement': 307.0, 'Horsepower': 130.0, 'Weight_in_lbs': 3504, 'Acceleration': 12.0, 'Year': Timestamp('1970-01-01 00:00:00'), 'Origin': 'USA'}, {'Name': 'buick skylark 320', 'Miles_per_Gallon': 15.0, 'Cylinders': 8, 'Displacement': 350.0, 'Horsepower': 165.0, 'Weight_in_lbs': 3693, 'Acceleration': 11.5, 'Year': Timestamp('1970-01-01 00:00:00'), 'Origin': 'USA'}]

Do we want something like this in the Standard, which returns a list of each row represented as a dict? @kkraus14

If not, we may need to drop altair as a target for now

Activity

  1. shwina commented on Apr 11, 2023

    @shwina
    Contributor

    What is done downstream of the to_dict call that necessitates this representation of the data?

  2. MarcoGorelli commented on Apr 11, 2023

    @MarcoGorelli
    ContributorAuthor

    Haven't check yet

    But they'd need to refactor to be able to be able to use the Standard, and given that the interchange protocol already works for them, there probably wouldn't be much incremental gain to doing this

    They also do all kinds of Python-world operations

    https://github.com/altair-viz/altair/blob/d7c0b228ba005851a0aaa83c3ed86621d0e41a37/altair/utils/core.py#L327-L397

    So they might not be a good target

  3. jorisvandenbossche commented on Apr 13, 2023

    @jorisvandenbossche
    Member

    What is done downstream of the to_dict call that necessitates this representation of the data?

    Didn't look at their code, but my assumption is that they need some form of JSON representation of the data (given that it's javascript based)

  4. MarcoGorelli commented on Jul 7, 2023

    @MarcoGorelli
    ContributorAuthor

    I'd suggest that we add this, actually. It would be nice to support altair

    If some library doesn't want this, then they can just raise NotImplementedError, but I don't think we should be cutting off use cases

  5. MarcoGorelli commented on Oct 24, 2023

    @MarcoGorelli
    ContributorAuthor

    will take care of this in implementations, ok to leave out of the standard, if libraries want altair support please comment and we can reopen

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions