Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error instead of panic on timestamp overflow #91

Merged
merged 1 commit into from
May 24, 2024

Conversation

progval
Copy link
Contributor

@progval progval commented May 24, 2024

I would have prefered either returning a null value or trying to encode as a non-nanosecond timestamp (sometimes possible if the ORC data does not use nanosecond-level precision), but this would require more significant changes to the code.

Copy link
Collaborator

@waynexia waynexia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks @progval

I'm curious why pyarrow_timestamps.orc is present in the diff file list.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this file is not changed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I re-ran tests/basic/data/generate_orc_timestamps.py, which writes both files. It probably changed some metadata, I'll undo it.

I would have prefered either returning a null value or trying to encode
as a non-nanosecond timestamp (sometimes possible if the ORC data does
not use nanosecond-level precision), but this would require more significant
changes to the code.
@waynexia
Copy link
Collaborator

Great 👍 I'm going to merge this

@waynexia waynexia merged commit f223788 into datafusion-contrib:main May 24, 2024
9 checks passed
@progval progval deleted the timestamp-overflow branch May 24, 2024 09:04
@Jefffrey
Copy link
Collaborator

Thanks for this 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
3 participants