You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Paul Taylor / @trxcllnt:
There are a few JS Parquet implementations, with parquetjs being the more mature one from what I recall.
A while back I put together this demo converting Arrow -> Parquet in pure JS. The major drawback is the ParquetJS writer is row-oriented, so performance will be an issue.
I opened this issue to get some clarification, but haven't heard back yet.
Kyle Barron:
I have a working prototype here of the Rust parquet and arrow crates compiled to WASM. As described on the Rust repo, I've gotten Snappy, Gzip, Brotli, and ZSTD codecs working in WASM, and it looks like LZ4 should be attainable by switching out the dependency.
My idea is that the bindings can be pretty small by just parsing the Parquet input and creating Arrow IPC data to send back to the JS side. Then the Arrow JS bindings would parse the IPC data and Arrow JS could be used from then on.
Unfortunately, there seem to be issues currently with Arrow JS' IPC implementation (the reason why I was looking through these issues 😄), so Arrow JS often can't read the IPC data returned from Rust (though Pyarrow can read the files, so they seem correctly-formatted).
See question in apache/arrow#2209
Reporter: Wes McKinney / @wesm
Note: This issue was originally created as ARROW-2786. Please see the migration documentation for further details.
The text was updated successfully, but these errors were encountered: