Skip to content

Segmentation of combined emojis #42

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weโ€™ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
RazrFalcon opened this issue May 20, 2018 · 1 comment
Closed

Segmentation of combined emojis #42

RazrFalcon opened this issue May 20, 2018 · 1 comment

Comments

@RazrFalcon
Copy link

for c in UnicodeSegmentation::graphemes("๐Ÿณ๏ธโ€๐ŸŒˆ", true) {
    println!("{}", c);
}

Outputs:

๐Ÿณ๏ธโ€
๐ŸŒˆ

๐Ÿณ๏ธโ€
๐ŸŒˆ

But should output:

๐Ÿณ๏ธโ€๐ŸŒˆ

๐Ÿณ๏ธโ€๐ŸŒˆ

Another example: ๐Ÿ‘ฎโ€โ™€.

Is it UnicodeSegmentation bug or am I doing this wrong? For my current task this should be a single "character".

@Manishearth
Copy link
Member

We're operating off an old unicode version (9) where that's not in the tables.

https://www.unicode.org/Public/9.0.0/ucd/auxiliary/GraphemeBreakProperty.txt

Filed #43

That may take a while to fix, but it may be worth updating to Unicode 10 in the interim (which is an easier update than 10 to 11), and will also fix your issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants