updated async reader to read local files directly, no async

[maximedion2]

No description provided.

[maximedion2]

@tshauck is that what you had in mind? looking at your comment on the issue, you mentioned the enum that ObjectStore::get returns, so I looked into using that, the changes here basically make it so that the async reader behaves like a sync reader when the file is local, using the std lib (blocking) methods to read files.

I have this large-ish toy dataset on my computer, I don't commit it to the test data repo, but I use it for very rough benchmarking occasionally. reading it takes about 7-8 secs, there's definitely some variability, but with the changes I made in this PR, the async reader was consistently faster, I'd say by about 10% or so.

[tshauck]

This looks like it'd work to me. I was thinking more at the FileOpener level w/i datafusion, then delegate to either the async or non-async reader zarr reader. To your point this this mades the async reader work in non-async contexts... which may obviate the need for non-async reader?

[maximedion2]

Right, so initially I thought I was gonna create a non-async reader in the FileOpener like you said, but I looked into the ObjectStore::get method in more details, and I thought I could do something really simple and clean within the async reader. In the case of a local file, we now have an async method that has a relatively heavy, blocking operation in it, which I think in general is something to avoid, but in this specific case, I think it's acceptable, since the operation, reading a local file, doesn't seem to be very efficient when it runs async anyway.

Regarding the non-async reader, currently the async reader essentially creates a non-async reader internally and uses that (through a little trick with the ZarrInMemoryChunkContainer), so that the "core" logic is all in the non-async reader. We could definitely combine all that into a single async reader, it would just be a little bit of refactoring. The main reason I could think of not doing that would be that ultimately, even for calls with local files, if we only have an async reader you always need to set up the async runtime (e.g. with tokio) and all that. If someone wanted to read zarr chunks without the async runtime, the sync reader allows them to do that. Overall that's probably a pretty minor concern though.

All that to say that I'll leave things as is for now and merge, but I'm open to combining everything into a single reader at some point if we think it makes the code cleaner.