x/text/unicode/bidi: `ReverseString` doesn't preserve modifiers

[hajimehoshi]

What version of Go are you using (go version)?

$ go version
go version go1.17.4 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/hajimehoshi/Library/Caches/go-build"
GOENV="/Users/hajimehoshi/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/hajimehoshi/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/hajimehoshi/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.17.4"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/hajimehoshi/ebiten/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/60/khbk2xqn1c5bml1byjn89dwc0000gn/T/go-build4087252806=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Run this with x/text 0.3.7

package main

import (
	"fmt"

	"golang.org/x/text/unicode/bidi"
)

func main() {
	str := "a\u0308u"
	fmt.Println(str)
	fmt.Println(bidi.ReverseString(str))
}

What did you expect to see?

äu
uä

The document of ReverseString says:

Modifiers will still follow the runes they modify.

So the modifiers should be reserved. Probably this is the same for AppendReverse.

By the way, in the ideal world, should all the graphemes be reserved (e.g., emoji flags)?

What did you see instead?

äu
üa

CC /@pgundlach

[hajimehoshi]

Modifiers will still follow the runes they modify.

Oops, I was misunderstanding this sentence, but is this expected that the position of modifiers are swapped with the modifying targets?

[pgundlach]

I admit that this looks unexpected and strange. When I implemented the function I just followed the explanation in the doc string and thought that the implementation follows this.

I have no idea what the ReverseString was good for in the first place.

[hajimehoshi]

I see. It was not clear to me what 'still' in the doc meant.

I have no idea what the ReverseString was good for in the first place.

This can be useful to reorder Arabic presentation forms characters in my case.