Skip to content

Make the runtime parser output real Fluent AST #220

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
stasm opened this issue Jun 5, 2018 · 2 comments
Closed

Make the runtime parser output real Fluent AST #220

stasm opened this issue Jun 5, 2018 · 2 comments

Comments

@stasm
Copy link
Contributor

stasm commented Jun 5, 2018

The runtime parser currently outputs an abbreviated JSON structure which the runtime resolver then interprets. This design goes back to the Firefox OS days where we didn't have much disk space nor memory to store the translations pre-parsed on buildtime.

I wonder if we still need this optimization today. If not, I'd like to suggest we drop the custom optimized JSON structure and replace it with an optimized real Fluent AST. Here's what I suggest we do:

  • Change the resolver to work on the the real Fluent AST.
  • Change the runtime parser to be an optimized version of the tooling parser, outputting the real Fluent AST. Some nodes, like comments, spans, and annotations, would not be produced at all. The runtime parser could still feature performance optimizations in the parsing logic and only produce the well-formed AST, leaving the validation step to the resolver*.
  • Allow using the new runtime parser and the tooling parser interchangeably. This would allow running Fluent in a development mode, where it uses the tooling parser and produces a lot more errors, warnings and annotations during parsing.

* Before the resolver has these checks, we can rely on the checks implemented in Pontoon and in compare-locales.

@stasm
Copy link
Contributor Author

stasm commented Jun 26, 2018

With changes planned in #217, we could also make the parser skip the AST and generate the runtime representation of the Fluent file ready to be used by the resolver. Let's call it FluentDocument. The main difference is that FluentDocument could be a subclass of Map to make lookups fast and it wouldn't store comments nor junk entries. Instead, it would have a separate field to store errors encountered during parsing. Spans and annotations would be ignored as well.

@stasm
Copy link
Contributor Author

stasm commented Jul 30, 2019

With the recent changes to the runtime parser and resolver in #380 which shipped in @fluent/bundle 0.14, the need for a more readable runtime AST is less pronounced. While working on 0.14 I tested a few alternatives for storing simple strings; anything other than storing them as primitive strings was significantly slower, unfortunately.

@stasm stasm closed this as completed Jul 30, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant