Segmentation Fault when trying to upgrade from Gogs

[0mhu]

Gitea Version

1.1.4 and 1.1.0

Git Version

2.34.1

Operating System

Linux

How are you running Gitea?

Downloaded gitea as binary.

Database

PostgreSQL

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Description

Hi,
I try to switch from gogs to gitea. I followed the following steps

• https://docs.gitea.io/en-us/upgrade-from-gogs/
• Migration from latest gogs (0.11.46.0418) is possible #4286

I started with v1.0.2 and everything seemed to be running. All repositories/issues etc. were found.

I then switched to v1.1.4 by simply swapping the binary to the new version

This version isn't starting at all and throws a segmentation fault (see below).
How can I upgrade from gogs to gitea?

2021/12/25 14:23:45 [T] Custom path: /opt/gitea/custom
2021/12/25 14:23:45 [T] Log path: /home/git/gitea/log
fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0xe5 pc=0x7f88c77546e4]

runtime stack:
runtime.throw(0x13557aa, 0x2a)
	/usr/local/go/src/runtime/panic.go:596 +0x95
runtime.sigpanic()
	/usr/local/go/src/runtime/signal_unix.go:274 +0x2db

goroutine 1 [syscall, locked to thread]:
runtime.cgocall(0x105b1b0, 0xc420486860, 0x410001)
	/usr/local/go/src/runtime/cgocall.go:131 +0xe2 fp=0xc420486818 sp=0xc4204867d8
os/user._Cfunc_mygetpwuid_r(0x3cc, 0xc420470c00, 0x7f88c00008c0, 0x400, 0xc4203fa098, 0x0)
	os/user/_obj/_cgo_gotypes.go:178 +0x4d fp=0xc420486860 sp=0xc420486818
os/user.lookupUnixUid.func1.1(0x3cc, 0xc420470c00, 0x7f88c00008c0, 0x400, 0xc4203fa098, 0xc420486908)
	/usr/local/go/src/os/user/lookup_unix.go:100 +0x175 fp=0xc4204868b8 sp=0xc420486860
os/user.lookupUnixUid.func1(0x10)
	/usr/local/go/src/os/user/lookup_unix.go:104 +0x50 fp=0xc4204868f8 sp=0xc4204868b8
os/user.retryWithBuffer(0xc42045eb60, 0xc4204869f0, 0xc42045eb60, 0xc420018000)
	/usr/local/go/src/os/user/lookup_unix.go:253 +0x2b fp=0xc420486960 sp=0xc4204868f8
os/user.lookupUnixUid(0x3cc, 0x0, 0x0, 0x0)
	/usr/local/go/src/os/user/lookup_unix.go:105 +0x161 fp=0xc420486a28 sp=0xc420486960
os/user.current(0x0, 0x0, 0xc420091800)
	/usr/local/go/src/os/user/lookup_unix.go:49 +0x2f fp=0xc420486a58 sp=0xc420486a28
os/user.Current(0x131ed1f, 0xa, 0x0)
	/usr/local/go/src/os/user/lookup.go:9 +0x22 fp=0xc420486a80 sp=0xc420486a58
code.gitea.io/gitea/vendor/github.com/lib/pq.(*conn).handlePgpass(0xc420193180, 0xc420470b40)
	/srv/app/src/code.gitea.io/gitea/vendor/github.com/lib/pq/conn.go:163 +0x6d6 fp=0xc420486be0 sp=0xc420486a80
code.gitea.io/gitea/vendor/github.com/lib/pq.DialOpen(0x1c5ed00, 0x237b2d8, 0xc4201ec900, 0x3d, 0x0, 0x0, 0x0, 0x0)
	/srv/app/src/code.gitea.io/gitea/vendor/github.com/lib/pq/conn.go:318 +0x5e6 fp=0xc420486e20 sp=0xc420486be0
code.gitea.io/gitea/vendor/github.com/lib/pq.Open(0xc4201ec900, 0x3d, 0x0, 0x0, 0xc420487000, 0x20130000)
	/srv/app/src/code.gitea.io/gitea/vendor/github.com/lib/pq/conn.go:236 +0x67 fp=0xc420486e70 sp=0xc420486e20
code.gitea.io/gitea/vendor/github.com/lib/pq.(*drv).Open(0x237b2d8, 0xc4201ec900, 0x3d, 0xc420486f00, 0xc42016ea40, 0x20, 0x0)
	/srv/app/src/code.gitea.io/gitea/vendor/github.com/lib/pq/conn.go:44 +0x35 fp=0xc420486eb0 sp=0xc420486e70
database/sql.(*DB).conn(0xc420153ae0, 0x1c632c0, 0xc4200123d0, 0x1, 0x12ca2e0, 0x1, 0xc420470b10)
	/usr/local/go/src/database/sql/sql.go:965 +0x146 fp=0xc420487088 sp=0xc420486eb0
database/sql.(*DB).PingContext(0xc420153ae0, 0x1c632c0, 0xc4200123d0, 0xc420487128, 0xc4203ffb00)
	/usr/local/go/src/database/sql/sql.go:593 +0x60 fp=0xc4204870f0 sp=0xc420487088
database/sql.(*DB).Ping(0xc420153ae0, 0x0, 0xc420470b10)
	/usr/local/go/src/database/sql/sql.go:615 +0x43 fp=0xc420487128 sp=0xc4204870f0
code.gitea.io/gitea/vendor/github.com/go-xorm/xorm.(*Session).Ping(0xc420192f00, 0x0, 0x0)
	/srv/app/src/code.gitea.io/gitea/vendor/github.com/go-xorm/xorm/session_schema.go:24 +0x88 fp=0xc420487160 sp=0xc420487128
code.gitea.io/gitea/vendor/github.com/go-xorm/xorm.(*Engine).Ping(0xc420153c20, 0x0, 0x0)
	/srv/app/src/code.gitea.io/gitea/vendor/github.com/go-xorm/xorm/engine.go:256 +0x13b fp=0xc4204871c0 sp=0xc420487160
code.gitea.io/gitea/models.NewEngine(0x1322929, 0xc)
	/srv/app/src/code.gitea.io/gitea/models/models.go:268 +0x48 fp=0xc420487228 sp=0xc4204871c0
code.gitea.io/gitea/routers.GlobalInit()
	/srv/app/src/code.gitea.io/gitea/routers/init.go:54 +0x318 fp=0xc4204872d8 sp=0xc420487228
code.gitea.io/gitea/cmd.runWeb(0xc420411040, 0x0, 0xc420411000)
	/srv/app/src/code.gitea.io/gitea/cmd/web.go:170 +0x9d fp=0xc420487900 sp=0xc4204872d8
code.gitea.io/gitea/vendor/github.com/urfave/cli.HandleAction(0x11c62c0, 0x136d970, 0xc420411040, 0xc420408600, 0x0)
	/srv/app/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/app.go:471 +0xb9 fp=0xc420487928 sp=0xc420487900
code.gitea.io/gitea/vendor/github.com/urfave/cli.Command.Run(0x131332f, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1337772, 0x16, 0x0, ...)
	/srv/app/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/command.go:191 +0xb4b fp=0xc420487d08 sp=0xc420487928
code.gitea.io/gitea/vendor/github.com/urfave/cli.(*App).Run(0xc4203f29c0, 0xc42000c160, 0x2, 0x2, 0x0, 0x0)
	/srv/app/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/app.go:241 +0x65f fp=0xc420487ee8 sp=0xc420487d08
main.main()
	/srv/app/src/code.gitea.io/gitea/main.go:44 +0x3cd fp=0xc420487f88 sp=0xc420487ee8
runtime.main()
	/usr/local/go/src/runtime/proc.go:185 +0x20a fp=0xc420487fe0 sp=0xc420487f88
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2197 +0x1 fp=0xc420487fe8 sp=0xc420487fe0

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2197 +0x1

goroutine 18 [chan receive]:
code.gitea.io/gitea/modules/notification.(*notificationService).Run(0x2353238)
	/srv/app/src/code.gitea.io/gitea/modules/notification/notification.go:37 +0x63
created by code.gitea.io/gitea/modules/notification.init.1
	/srv/app/src/code.gitea.io/gitea/modules/notification/notification.go:31 +0x41

goroutine 9 [select]:
code.gitea.io/gitea/modules/log.(*Logger).StartLogger(0xc42016a080)
	/srv/app/src/code.gitea.io/gitea/modules/log/log.go:265 +0x2a7
created by code.gitea.io/gitea/modules/log.newLogger
	/srv/app/src/code.gitea.io/gitea/modules/log/log.go:191 +0x11f

goroutine 10 [select]:
code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index.AnalysisWorker(0xc42006b500, 0xc42006b560)
	/srv/app/src/code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index/analysis.go:75 +0x13a
created by code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index.NewAnalysisQueue
	/srv/app/src/code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index/analysis.go:67 +0xdf

goroutine 11 [select]:
code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index.AnalysisWorker(0xc42006b500, 0xc42006b560)
	/srv/app/src/code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index/analysis.go:75 +0x13a
created by code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index.NewAnalysisQueue
	/srv/app/src/code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index/analysis.go:67 +0xdf

goroutine 12 [select]:
code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index.AnalysisWorker(0xc42006b500, 0xc42006b560)
	/srv/app/src/code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index/analysis.go:75 +0x13a
created by code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index.NewAnalysisQueue
	/srv/app/src/code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index/analysis.go:67 +0xdf

goroutine 13 [select]:
code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index.AnalysisWorker(0xc42006b500, 0xc42006b560)
	/srv/app/src/code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index/analysis.go:75 +0x13a
created by code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index.NewAnalysisQueue
	/srv/app/src/code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index/analysis.go:67 +0xdf

goroutine 35 [syscall]:
os/signal.signal_recv(0x0)
	/usr/local/go/src/runtime/sigqueue.go:116 +0x104
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:22 +0x22
created by os/signal.init.1
	/usr/local/go/src/os/signal/signal_unix.go:28 +0x41

goroutine 36 [select]:
code.gitea.io/gitea/modules/log.(*Logger).StartLogger(0xc420050100)
	/srv/app/src/code.gitea.io/gitea/modules/log/log.go:265 +0x2a7
created by code.gitea.io/gitea/modules/log.newLogger
	/srv/app/src/code.gitea.io/gitea/modules/log/log.go:191 +0x11f

goroutine 37 [select]:
code.gitea.io/gitea/modules/log.(*Logger).StartLogger(0xc420050140)
	/srv/app/src/code.gitea.io/gitea/modules/log/log.go:265 +0x2a7
created by code.gitea.io/gitea/modules/log.newLogger
	/srv/app/src/code.gitea.io/gitea/modules/log/log.go:191 +0x11f

goroutine 38 [chan receive]:
database/sql.(*DB).connectionOpener(0xc420153ae0)
	/usr/local/go/src/database/sql/sql.go:837 +0x4a
created by database/sql.Open
	/usr/local/go/src/database/sql/sql.go:582 +0x212

Screenshots

No response

[0mhu]

Okay. I figured out, that for some reason gitea was crashing becasue the database password was missing although it is not needed for the postgresql unix socket connection.

I am now able to upgrade to version 1.6.x Versions 1.7.x throw a segmentation fault again:

fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0xe5 pc=0x7f20902226e4]

runtime stack:
runtime.throw(0x19c3d99, 0x2a)
	/usr/local/go/src/runtime/panic.go:617 +0x72
runtime.sigpanic()
	/usr/local/go/src/runtime/signal_unix.go:374 +0x4a9

goroutine 1 [syscall]:
runtime.cgocall(0x1578c20, 0xc0005e2ef0, 0xc0000107f8)
	/usr/local/go/src/runtime/cgocall.go:128 +0x5b fp=0xc0005e2ec0 sp=0xc0005e2e88 pc=0x407f8b
os/user._Cfunc_mygetpwuid_r(0x3cc, 0xc00054b650, 0x4e862e0, 0x400, 0xc0000107f8, 0x0)
	_cgo_gotypes.go:173 +0x4d fp=0xc0005e2ef0 sp=0xc0005e2ec0 pc=0x86ffdd
os/user.lookupUnixUid.func1.1(0x3cc, 0xc00054b650, 0xc0005661b0, 0xc0000107f8, 0x4e862e001578c90)
	/usr/local/go/src/os/user/cgo_lookup_unix.go:103 +0x14c fp=0xc0005e2f40 sp=0xc0005e2ef0 pc=0x8716ac
os/user.lookupUnixUid.func1(0x8)
	/usr/local/go/src/os/user/cgo_lookup_unix.go:103 +0x45 fp=0xc0005e2f78 sp=0xc0005e2f40 pc=0x871745
os/user.retryWithBuffer(0xc0005661b0, 0xc0005e3070, 0xc0005661b0, 0x110)
	/usr/local/go/src/os/user/cgo_lookup_unix.go:251 +0x3b fp=0xc0005e2fe0 sp=0xc0005e2f78 pc=0x870abb
os/user.lookupUnixUid(0x3cc, 0x0, 0x0, 0x0)
	/usr/local/go/src/os/user/cgo_lookup_unix.go:96 +0x130 fp=0xc0005e30a8 sp=0xc0005e2fe0 pc=0x870400
os/user.current(0xc0005e3120, 0x10, 0xc0005e3118)
	/usr/local/go/src/os/user/cgo_lookup_unix.go:49 +0x4a fp=0xc0005e30e8 sp=0xc0005e30a8 pc=0x87029a
os/user.Current.func1()
	/usr/local/go/src/os/user/lookup.go:15 +0x22 fp=0xc0005e3110 sp=0xc0005e30e8 pc=0x8714f2
sync.(*Once).Do(0x3cf05e0, 0x19fad80)
	/usr/local/go/src/sync/once.go:44 +0xb3 fp=0xc0005e3140 sp=0xc0005e3110 pc=0x46b4f3
os/user.Current(0x0, 0x0, 0x0)
	/usr/local/go/src/os/user/lookup.go:15 +0x3d fp=0xc0005e3170 sp=0xc0005e3140 pc=0x86fc6d
code.gitea.io/gitea/modules/user.CurrentUsername(0xc00023c380, 0x197439d)
	/go/src/code.gitea.io/gitea/modules/user/user.go:16 +0x22 fp=0xc0005e3198 sp=0xc0005e3170 pc=0xb3db92
code.gitea.io/gitea/modules/setting.NewContext()
	/go/src/code.gitea.io/gitea/modules/setting/setting.go:790 +0x30ab fp=0xc0005e3660 sp=0xc0005e3198 pc=0xb50cbb
code.gitea.io/gitea/routers.GlobalInit()
	/go/src/code.gitea.io/gitea/routers/init.go:67 +0x37 fp=0xc0005e3770 sp=0xc0005e3660 pc=0x12b2b67
code.gitea.io/gitea/cmd.runWeb(0xc000257180, 0x0, 0x0)
	/go/src/code.gitea.io/gitea/cmd/web.go:121 +0xa4 fp=0xc0005e3958 sp=0xc0005e3770 pc=0x14c4c24
code.gitea.io/gitea/vendor/github.com/urfave/cli.HandleAction(0x1758940, 0x19f1fe8, 0xc000257180, 0xc0001c2900, 0x0)
	/go/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/app.go:471 +0xad fp=0xc0005e3980 sp=0xc0005e3958 pc=0x10ec45d
code.gitea.io/gitea/vendor/github.com/urfave/cli.Command.Run(0x196bd9a, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0, 0x19965cc, 0x16, 0x0, ...)
	/go/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/command.go:191 +0x928 fp=0xc0005e3c10 sp=0xc0005e3980 pc=0x10ed558
code.gitea.io/gitea/vendor/github.com/urfave/cli.(*App).Run(0xc0005b09c0, 0xc00000e080, 0x2, 0x2, 0x0, 0x0)
	/go/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/app.go:241 +0x658 fp=0xc0005e3ed8 sp=0xc0005e3c10 pc=0x10ea5a8
main.main()
	/go/src/code.gitea.io/gitea/main.go:57 +0x426 fp=0xc0005e3f98 sp=0xc0005e3ed8 pc=0x14d25c6
runtime.main()
	/usr/local/go/src/runtime/proc.go:200 +0x20c fp=0xc0005e3fe0 sp=0xc0005e3f98 pc=0x4342ec
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1337 +0x1 fp=0xc0005e3fe8 sp=0xc0005e3fe0 pc=0x462121

goroutine 36 [select]:
code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index.AnalysisWorker(0xc0005a4e40, 0xc0005a4ea0)
	/go/src/code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index/analysis.go:102 +0x102
created by code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index.NewAnalysisQueue
	/go/src/code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index/analysis.go:94 +0xc8

goroutine 35 [select]:
code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index.AnalysisWorker(0xc0005a4e40, 0xc0005a4ea0)
	/go/src/code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index/analysis.go:102 +0x102
created by code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index.NewAnalysisQueue
	/go/src/code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index/analysis.go:94 +0xc8

goroutine 9 [select]:
code.gitea.io/gitea/modules/log.(*Logger).StartLogger(0xc0000aab40)
	/go/src/code.gitea.io/gitea/modules/log/log.go:303 +0xf5
created by code.gitea.io/gitea/modules/log.newLogger
	/go/src/code.gitea.io/gitea/modules/log/log.go:233 +0xba

goroutine 37 [select]:
code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index.AnalysisWorker(0xc0005a4e40, 0xc0005a4ea0)
	/go/src/code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index/analysis.go:102 +0x102
created by code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index.NewAnalysisQueue
	/go/src/code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index/analysis.go:94 +0xc8

goroutine 38 [select]:
code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index.AnalysisWorker(0xc0005a4e40, 0xc0005a4ea0)
	/go/src/code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index/analysis.go:102 +0x102
created by code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index.NewAnalysisQueue
	/go/src/code.gitea.io/gitea/vendor/github.com/blevesearch/bleve/index/analysis.go:94 +0xc8

goroutine 19 [chan receive]:
code.gitea.io/gitea/modules/notification/ui.(*notificationService).Run(0xc0005ea030)
	/go/src/code.gitea.io/gitea/modules/notification/ui/ui.go:40 +0x4d
created by code.gitea.io/gitea/modules/notification.RegisterNotifier
	/go/src/code.gitea.io/gitea/modules/notification/notification.go:22 +0x4a

goroutine 40 [syscall]:
os/signal.signal_recv(0x462126)
	/usr/local/go/src/runtime/sigqueue.go:139 +0x9c
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:23 +0x22
created by os/signal.init.0
	/usr/local/go/src/os/signal/signal_unix.go:29 +0x41

[lunny]

What's your Linux kernel version? And the architecture?

[Gusted]

What's your Linux kernel version? And the architecture?

Adding to this to add the go version you're using. The stacktrace reminds me of this bug on go golang/go#49250 but it should only exist on MacOS(even though the fix for that bug is now applied on Linux OS as well), but you could try to suggested workaround of using MallocNanoZone=0.

But the stack trace is indicating a go bug, not sure if we can much do about that.

[0mhu]

• Kernel Version: 5.15.11 (Arch)
• go version go1.17.5 linux/amd64
• Architecture: x86_64

I solved the problem by moving everything to a Debian VM and performing the migrations there.
Additionally to the above problems, I encountered the following:

• Some wikis are broken (cannot delete pages etc.). Furthermore, deleting pages in Wikis redirects to a 404 page.
• Migrations do not work from version 1.8 to 1.9 due to a missing index in the database. The migration tries to drop it but it is not present. I created it, so it could be dropped.
• File attachments aren't working, because the 'size' of each attachment is zero in the database. I wrote a script which goes through the attachment folder and patches the database entries with the correct size.

Now everything seems to work (except for some minor oddities in the Wikis).