Word-final sigma in str::to_lowercase

[SimonSapin]

By design, str::to_lowercase and str::to_uppercase do not depend on the language of the text (which shouldn’t be assumed to be the same as the locale of the machine running the program).

Mostly, this means ignoring the conditional mappings in Unicode’s SpecialCasing.txt, with one exception: the greek letter Sigma is Σ in upper-case and σ in lower-case except in word-final position, where it is ς. The corresponding mapping in SpecialCasing.txt is:

# <code>; <lower>; <title>; <upper>; (<condition_list>;)? # <comment>
03A3; 03C2; 03A3; 03A3; Final_Sigma; # GREEK CAPITAL LETTER SIGMA

With Final_Sigma defined in the Unicode standard:

C is preceded by a sequence consisting of a cased letter and then zero or more case-ignorable characters, and C is not followed by a sequence consisting of zero or more case-ignorable characters and then a cased letter.

(cased letter and other terms have a precise definition given beforehand.)

Since char::to_lowercase doesn’t know context, I think it should just return σ for Σ. But str::to_lowercase does have context and could implement this conditional mapping.

[SimonSapin]

CC @aturon, @alexcrichton

[alexcrichton]

Our current trend of conventions and such leads me to think that we should do the special behavior in str::to_lowercase.

[SimonSapin]

Untested first attempt:

impl str {
    pub fn to_lowercase(&self) -> String {
        let mut s = String::with_capacity(self.len());
        for (i, c) in self[..].char_indices() {
            if c == 'Σ' && is_final_sigma(self, i) {
                s.push_str("ς")
            } else {
                s.extend(c.to_lowercase());
            }
        }
        return s;

        fn is_final_sigma(s: &str, i: usize) -> bool {
            debug_assert!('Σ'.len_utf8() == 2);
            s[..i].chars().rev().skip_while(is_case_ignorable).any(is_cased_letter) &&
            !s[i + 2..].chars() .skip_while(is_case_ignorable).any(is_cased_letter)
        }
    }
}

… where is_cased_ignorable and is_cased_letter would be private functions based on tables generated by src/etc/unicode.py, according to the definitions in the linked PDF.

[SimonSapin]

In other words: since there is only one such mapping, I think it makes more sense to hard-code it than to try and generalize rustc_unicode::tables::conversions::to_lowercase_table to include arbitrary conditions.

[SimonSapin]

According to a Greek speaker, not doing this is "Very, very bad. Basically, bad enough that people won’t use it.".