You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
reading /srv/softwareheritage/ssd/data/vlorentz/datasets/2024-05-16/orc/revision/revision-0c45576a-59f7-48d1-a9a8-2e5c64098905.orc
patch_bit_width= 40 value_bit_width= 30 N::BYTE_SIZE= 8
Error: Could not read chunk 29525 of /srv/softwareheritage/ssd/data/vlorentz/datasets/2024-05-16/orc/revision/revision-0c45576a-59f7-48d1-a9a8-2e5c64098905.orc
Caused by:
0: External error: Out of spec, message: combined patch width and value width cannot exceed the size of the integer type being decoded
1: Out of spec, message: combined patch width and value width cannot exceed the size of the integer type being decoded
Stack backtrace:
0: anyhow::context::<impl anyhow::Context<T,E> for core::result::Result<T,E>>::with_context
1: repro::main
2: std::sys_common::backtrace::__rust_begin_short_backtrace
3: std::rt::lang_start::{{closure}}
4: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:284:13
5: std::panicking::try::do_call
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40
6: std::panicking::try
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19
7: std::panic::catch_unwind
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14
8: std::rt::lang_start_internal::{{closure}}
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:148:48
9: std::panicking::try::do_call
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40
10: std::panicking::try
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19
11: std::panic::catch_unwind
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14
12: std::rt::lang_start_internal
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:148:20
13: main
14: __libc_start_main
at ./csu/../csu/libc-start.c:308:16
15: _start
The text was updated successfully, but these errors were encountered:
this check:
datafusion-orc/src/reader/decode/rle_v2/patched_base.rs
Lines 55 to 60 in 16b5704
can be falsified while reading files created with
pyorc
. To reproduce:(sorry, it's 4GB. I don't have a smaller example on hand)
then checkout 28e911b (a commit from #96 because it's the only way not to hit an overflow crash before this bug), apply this patch:
then run this code with
./revision-0c45576a-59f7-48d1-a9a8-2e5c64098905.orc
as parameter:which prints:
The text was updated successfully, but these errors were encountered: