goroutines and tcp errors #723
Description
Full disclosure I may have something misconfigured with postgres.
max_connections on db = 1000
My understanding is that once you get the db object from the db.Open() that it is a connection pool, and it can be safely used inside goroutines for concurrent access.
However the below minimal example does one of 3 things on my machine.
- Very rarely it runs to completion as expected.
- Most often it fails with : "read tcp [::1]:port->[::1]:5432: read: connection reset by peer"
- Rarely it fails with "write tcp [::1]:port->[::1]:5432: write: broken pipe"
Lowering the goroutines int variable increases the reliability, raising it makes failure a guarantee.
The number of records that prints to the terminal before an error changes each time I run a given binary.
package main
import (
"database/sql"
"fmt"
_ "github.com/lib/pq"
"os"
)
func main() {
// Connect to db
connStr := "postgres://@localhost/test?sslmode=disable"
db, err := sql.Open("postgres", connStr)
if err != nil {
panic(err.Error())
}
defer db.Close()
err = db.Ping()
if err != nil {
panic(err.Error())
}
goroutines := 300
// Channel to syncronize goroutines
channel := make(chan int, goroutines)
// Generate goroutines to select records and interate them
for i := 0; i < goroutines; i++ {
go func(j int) {
rows, err := db.Query("select name, age from contacts where age = $1", j)
if err != nil {
fmt.Println(err)
os.Exit(-1)
}
var name string
var age int
for rows.Next() {
err := rows.Scan(&name, &age)
if err != nil {
fmt.Println(err)
os.Exit(-1)
}
}
fmt.Println(name, age)
err = rows.Err()
if err != nil {
fmt.Println(err)
os.Exit(-1)
}
rows.Close()
channel <- 0
}(i)
}
// Wait for goroutines to finish
for i := 0; i < goroutines; i++ {
<-channel
}
}