Issue with pluralisation in the NumberField

[jbovenschen]

Provide a general summary of the issue here

Using the NumberField in slavic languages causes issues when adding the formatOptions { type: 'unit', unit: 'year', style, unitDisplay: 'long' }.

🤔 Expected Behavior?

The user should still be able to cycle through the options. For example for polish it should start with:

• 0 lat
• 1 rok
• 2-4 lata
• 5 lat

But in case the value uses decimals it should be 3.1 roku.

😯 Current Behavior

When the user starts interacting with the NumberField the value would always be 0 lat.

💁 Possible Solution

The issue seems to be caused in the getSymbols method of the NumberFieldParser where the characters are determined which can be in front or at the end of the entered value. This only checks the used symbols for the a large decimal number 10000.111 -10000.111 and 1 causing the year option not to work in case pluralisation rules needs to be applied.

react-spectrum/packages/@internationalized/number/src/NumberParser.ts

Lines 196 to 198 in a399db1

	let allParts = formatter.formatToParts(-10000.111);
	let posAllParts = formatter.formatToParts(10000.111);
	let singularParts = formatter.formatToParts(1);

🔦 Context

There is an article I found showing the different types of pluralisation rules in different languages. https://lingohub.com/blog/2019/02/pluralization

🖥️ Steps to Reproduce

https://codesandbox.io/s/cool-fast-mhls6q?file=/src/App.js

Version

1.0.0-alpha-5

What browsers are you seeing the problem on?

Other

If other, please specify.

The issue is in all browsers

What operating system are you using?

MacOS

🧢 Your Company/Team

No response

🕷 Tracking Issue

No response

[snowystinger]

I wish we'd found that article when we initially authored the component! Thank you so much for sharing, it looks like exactly what I'd been trying to find when I was helping out with this #2075 (comment)

I'll try to find some time to look at that PR again, but feel free to contribute as well. Probably good to use that as a starting place. I think I'd get the actual structured data for all the locales we support and then in the spot that you found, just format the relevant numbers based on that data. Across ~30 locales x 5 numbers, that shouldn't be too bad.

I'm not quite sure how they built up that table from cldr-common-43.0/common/main/en.xml, that's the first thing to determine
After that, we might also need all the possible "group" symbols in a more reliable way too. I can find those in the same file I mentioned under numbers -> <symbols numberSystem="latn">. I recall I had some difficulties determining when they would appear. Such as group symbol in some locales not appearing until 100000 or something

[jbovenschen]

Happy to be of help, I did start to look into contributing a solution to this myself.

I did not find the specific rules for the pluralisation under the cldr-common-43.0/common/main/en.xml but in cldr-common-43.0/common/supplemental/plurals.xml you can find the specific rukes which should be applied to apply a certain rule.

So based on what you're suggesting we could I think indeed just try to find specific values which are only true for the specific rule.

Also found this document which displays it in a more readable form:
https://www.unicode.org/cldr/charts/43/supplemental/language_plural_rules.html#pl

[jbovenschen]

Ok, I looked a bit further into it, I think the way you've described above would mean that for every locale in the project we could determine what the lowest possible value is for every rule by following these tables https://www.unicode.org/cldr/charts/43/supplemental/language_plural_rules.html#comparison.

And then just generate all the possible symbols based on those numbers. This would mean that we just only need to include a small list of numbers which would cover all the cases for every languages. I think the overall overhead would be not that big since it only runs once when initialising the NumberFormatter.

The only concern I have is that it is a lot of manual work, which could be a problem when the used version of Common Locale Data Repository is updated too for example the 44.0 version.

[devongovett]

Maybe Intl.PluralRules could help?

[jbovenschen]

I looked into that as an option as well, but it would only provide us with the following information;

1. Which value is applied for a certain value through the Intl.PluralRules('en-GB').select(0)
2. Which rules can be applied for a specific language by calling Intl.PluralRules('en-GB').resolveOptions().

The thing we can't get from this is for example in case the user want's to format a year that it knows:

For dutch we have the words jaar and jaren based on the pluralization rules.
For english we have year and years
For polish we have rok, roku, lat and lata.

[snowystinger]

The only concern I have is that it is a lot of manual work, which could be a problem when the used version of Common Locale Data Repository is updated too for example the 44.0 version.

If we can script the creation of that table from the blog you shared, then we could just watch cldr and manually update when they release a new version. That's the general gist of what that PR I shared was doing, though inelegantly, in that there is no table, it's just a mishmash of tested values that get us close to the coverage this would provide.

[jbovenschen]

I opened a pull request for it, including how I extracted a list of numbers from the table. Tried to keep things as much the same as you already did in #2075 but that seemed to include some other issues as wel related to percentages for example, so decided to just open a new PR for those specific errors.

[snowystinger]

Perfectly fine, thank you for opening that