Repository navigation
Release the spec as PyPI package #73
Description
Hello everyone!
I've been mulling over introducing the Dataframe Exchange protocol in Pandas and Modin, and I think it would be beneficial for every end library implementing the protocol to have the exact same base.
Right now the protocol interface is defined by code, but said code is not "published" as a ready to use Python source.
I would like to make it a real PyPI package to use it in type hinting and (ideally) mypy type checking and to enable other libraries to do the same.
I propose to publish the package as dataframe-protocol or df-protocol and rename protocol/ directory to df_protocol turning it in a real Python package.
That way any library which would be implementing the protocol would just from df_protocol import exchange and use it for type hints (and for enum values - as now they're embedded in docstrings which just look really weird to me).
Am I missing something here? Are there any objections?
I can make the PR with necessary changes if there is the agreement, and can keep it published both in PyPI and conda-forge (or can turn the publishing to someone else in the consortium).
P.S. Keeping the top-level df_protocol would allow us to add another subpackage for cross-operation API if/when we feel ready for that (so keeping this future-proof).
Activity
rgommers commentedon Feb 7, 2022
Thanks @vnlitvinov
I think the main thing is that a runtime dependency is expensive, and I would not imagine Pandas or others would want a new dependency just for some improved type hinting. Nor would I recommend anyone doing that - vendoring seems preferable here. In which case, does it make sense to release a PyPI package - just tagging versions may be better?
vnlitvinov commentedon Feb 8, 2022
That's a good point, though I personally don't think it is so expensive, given how many packages usually a library already depends on, and taking into account that this particular package would be updated very rarely.
But anyway... I think I will make a PR preparing for the package (probably without
setup.cfg), so we can publish that or vendor later on if we decide against publishing.vnlitvinov commentedon Feb 10, 2022
We've also started working on a simple, "smoke"-like test suite which should be pluggable anywhere (@Rubtsowa is working on that now).
rgommers commentedon Jul 14, 2022
Related, the
array-api-testsis used in test suites of array libraries; it's not on PyPI, but does make releases by tagging with a date-based version scheme: https://github.com/data-apis/array-api-tests/tagsrgommers commentedon Sep 13, 2022
@vnlitvinov given that all that is left in this repo is a single file,
dataframe_protocol.py, can we close this? I doubt it's helpful to add packaging files now, and if that changes in the future we can still reconsider.astrojuanlu commentedon Jun 16, 2023
Maybe implementors (like pandas) don't need such an additional dependency, but downstream users would benefit from it I think. Both Altair (vega/altair#2888) and Plotly Express (plotly/plotly.py#3387) seem to make use of
if hasattr(data, "__dataframe__"), but one could envision dedicated methods that could dowhich would definitely help during development.
MarcoGorelli commentedon Aug 31, 2023
Big +1 to publishing
My experience using both the interchange protocol and the dataframe API is that it's not a great user experience to type such unergonomic and unusual names (e.g.
select_columns_by_nameand friends)Being able to tab-complete would be a big plus
Regarding dependencies - this definitely wouldn't be a runtime dependency of pandas', but consumers of the api / interchange protocol could use it as a dev dependency
MarcoGorelli commentedon Oct 23, 2023
You can now install the dataframe api directly from github:
Then, for the dataframe api at least, you'll get type hints / autocomplete:
astrojuanlu commentedon Oct 23, 2023
xref #278