Skip to content

[JS] Read Parquet files in JavaScript #123

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
asfimport opened this issue Jul 2, 2018 · 2 comments
Open

[JS] Read Parquet files in JavaScript #123

asfimport opened this issue Jul 2, 2018 · 2 comments
Labels
Type: enhancement New feature or request

Comments

@asfimport
Copy link

See question in apache/arrow#2209

Reporter: Wes McKinney / @wesm

Note: This issue was originally created as ARROW-2786. Please see the migration documentation for further details.

@asfimport
Copy link
Author

Paul Taylor / @trxcllnt:
There are a few JS Parquet implementations, with parquetjs being the more mature one from what I recall.

A while back I put together this demo converting Arrow -> Parquet in pure JS. The major drawback is the ParquetJS writer is row-oriented, so performance will be an issue.

I opened this issue to get some clarification, but haven't heard back yet.

@asfimport
Copy link
Author

Kyle Barron:
I have a working prototype here of the Rust parquet and arrow crates compiled to WASM. As described on the Rust repo, I've gotten Snappy, Gzip, Brotli, and ZSTD codecs working in WASM, and it looks like LZ4 should be attainable by switching out the dependency.

My idea is that the bindings can be pretty small by just parsing the Parquet input and creating Arrow IPC data to send back to the JS side. Then the Arrow JS bindings would parse the IPC data and Arrow JS could be used from then on.

Unfortunately, there seem to be issues currently with Arrow JS' IPC implementation (the reason why I was looking through these issues 😄), so Arrow JS often can't read the IPC data returned from Rust (though Pyarrow can read the files, so they seem correctly-formatted).

@assignUser assignUser added the Type: enhancement New feature or request label May 26, 2025
@assignUser assignUser transferred this issue from apache/arrow May 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants