proposal: strings: add a Join method to Builder

[muhlemmer]

Sometimes, I find myself wanting to join a slice of strings and append the result to a strings.Builder. This can be done by:

1. Use something like Builder.WriteString(strings.Join([]string{"foo", "bar"}, ", ")), less coding, but results in 1 extra allocation.
2. Write my own joining loop against the builder, basically duplication of code of the current strings.Join function.

Currently, strings.Join() already uses a strings.Builder. This issue proposes to move the code from this function into a method of strings.Builder. The current strings.Join() function will become a wrapper which allocates a new builder and calls this new method for further processing.

Example (adjusted copy of strings.Join()):

type Builder struct {
	strings.Builder
}

func (b *Builder) Join(elems []string, sep string) {
	switch len(elems) {
	case 0:
		return
	case 1:
		b.Grow(len(elems[0]))
		b.WriteString(elems[0])
	}

	n := len(sep) * (len(elems) - 1)
	for i := 0; i < len(elems); i++ {
		n += len(elems[i])
	}

	b.Grow(n)
	b.WriteString(elems[0])
	for _, s := range elems[1:] {
		b.WriteString(sep)
		b.WriteString(s)
	}
}

Benchmarks code:

func Benchmark_Builder_Join(b *testing.B) {
	for i := 0; i < b.N; i++ {
		var b Builder
		b.Join([]string{"Hello", "World!"}, ", ")
	}
}

func Benchmark_Join_and_WriteString(b *testing.B) {
	for i := 0; i < b.N; i++ {
		var b Builder
		b.WriteString(
			strings.Join([]string{"Hello", "World!"}, ", "),
		)
	}
}

Benchmark run:

go test -benchmem -bench . 
goos: linux
goarch: amd64
pkg: github.com/muhlemmer/pbpgx
cpu: AMD Ryzen 9 5900HX with Radeon Graphics        
Benchmark_Builder_Join-16               30865467                72.17 ns/op           16 B/op          1 allocs/op
Benchmark_Join_and_WriteString-16       19650562               138.8 ns/op            32 B/op          2 allocs/op
PASS
ok      github.com/muhlemmer/pbpgx      5.132s

Items open for discussion:

1. Name of the method (Join, AppendJoin or WriteJoin?).
2. Should the method return total bytes written, as returned by WriteString()?
3. Should the method return error, as returned by WriteString()? (Is always nil)

Related: PR #42850, which got put on hold for not going trough the proposal process (and duplicates code)

[andig]

Bonus question: should bytes mirror this api?

[ianlancetaylor]

It would help to see some examples of real code that would use this new feature. As you say, it's pretty easy to write your own loop, and that loop could be in an imported third party package. Does this happen often enough to make it useful to add to the standard library? https://golang.org/doc/faq#x_in_std

[ericlagergren]

It seems like this proposal is primarily a workaround for a “deficiency” in the compiler—that it cannot see that the outer strings.Builder is redundant and eliminate it (probably because strings.Join can’t be inlined?).

Looking at my own code, I think the pattern of creating a temporary buffer ([]byte, strings.Builder, etc.) then appending/writing it to a larger buffer is actually fairly common. Perhaps it would be better to update the compiler (somehow) instead of adding a new, very specific API.

[muhlemmer]

It would help to see some examples of real code that would use this new feature. As you say, it's pretty easy to write your own loop, and that loop could be in an imported third party package. Does this happen often enough to make it useful to add to the standard library? https://golang.org/doc/faq#x_in_std

First, I have nothing against keeping this additional API in a separate package. I just came across this and felt it can help a bigger audience.

One example would be SQL query building. I've condensed something simplified in a single function, for demonstration purpose:

To build a query like:

SELECT id, title, price FROM public.products WHERE color = 'red';

Using the existing API:

func selectQuery(schema, table string, columns []string, condCol, value string) string {
	var buf strings.Builder

	buf.WriteString("SELECT ")
	buf.WriteString(
		strings.Join(columns, ", "),
	)
	buf.WriteString(" FROM ")
	buf.WriteString(
		strings.Join([]string{schema, table}, "."),
	)
	buf.WriteString(" WHERE ")
	buf.WriteString(
		strings.Join([]string{condCol, "'" + value + "'"}, " = "),
	)
	buf.WriteByte(';')

	return buf.String()
}

With the proposed API:

func selectQuery2(schema, table string, columns []string, condCol, value string) string {
	var buf Builder

	buf.WriteString("SELECT ")
	buf.Join(columns, ", ")
	buf.WriteString(" FROM ")
	buf.Join([]string{schema, table}, ".")
	buf.WriteString(" WHERE ")
	buf.Join([]string{condCol, "'" + value + "'"}, " = ")
	buf.WriteByte(';')

	return buf.String()
}

Benchmarks:

func Benchmark_selectQuery(b *testing.B) {
	for i := 0; i < b.N; i++ {
		selectQuery("public", "products", []string{"id", "title", "price"}, "color", "red")
	}
}

func Benchmark_selectQuery2(b *testing.B) {
	for i := 0; i < b.N; i++ {
		selectQuery2("public", "products", []string{"id", "title", "price"}, "color", "red")
	}
}

go test -benchmem -bench . 
goos: linux
goarch: amd64
pkg: builder
cpu: AMD Ryzen 9 5900HX with Radeon Graphics        
Benchmark_selectQuery-16         3683500               487.4 ns/op           224 B/op          7 allocs/op
Benchmark_selectQuery2-16        6028978               312.7 ns/op           120 B/op          3 allocs/op
PASS
ok      builder 4.218s

[jimmyfrasche]

A more general solution would be a strings/bytes.Join variant that took an io.Writer:

package strings
func WriteJoin(w io.Writer, elems []string, sep string) (n int64, err error)
// and similarly for bytes

That wouldn't have to live in the stdlib either but if it did I would certainly use it a lot.

[DeedleFake]

I was going to propose something very similar, @jimmyfrasche (Bikeshedding: JoinTo()), but it wouldn't have an easy way to call Grow() to prevent repeated allocations. You could have it check for a Grow() method, but it feels kind of strange to me to support that in just that one place.

Edit: Fix the bikeshed's name.

[muhlemmer]

After doing some more investigations, I have concluded that it is indeed better to create and maintain my own package with specific use cases, which sometimes is more than just Join operations.

Thanks everyone for taking the time to respond. Leaving this issue open for now, in case people will still want to vote to include the Join method to strings.Builder. If that's the case, I'd still be willing to make the CL.

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

[rsc]

Based on the discussion above, this proposal seems like a likely decline.
— rsc for the proposal review group

[jimmyfrasche]

I wrote https://pkg.go.dev/github.com/jimmyfrasche/jointo so there's something to look at and because I'm sure I'll want to use it if this proposal is declined.

[rsc]

No change in consensus, so declined.
— rsc for the proposal review group