Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: writing an empty dataframe with an object column fails with use_arrow #513

Open
theroggy opened this issue Dec 21, 2024 · 0 comments
Open

Comments

@theroggy
Copy link
Member

theroggy commented Dec 21, 2024

When an empty dataframe (no rows) with an "object" dtype column is written using use_arrow=True, this leads to this type of error: pyogrio.errors.FieldError: Error while creating field from Arrow for field 3 with name 'HFDTLT' and type n (Type 'n' for field HFDTLT is not supported.).

The same problem occurs when an object type column contains only None values.

Looking deeper into this, the issue is that in the table returned by pyarrow.Table.from_pandas, the object column is returned as having the "null" type... and this "null" type is not supported by GDAL.

table = pa.Table.from_pandas(df, preserve_index=False)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant