Gitea 1.17-dev crashing Out of memory

[gboddin]

Description

Gitea starts what seems like an infinite loop and runs out of memory,

Issues happend 3 times, randomly browsing the instance, no pattern noticed.

Will update if a pattern is found

Gitea Version

7671a33

Can you reproduce the bug on the Gitea demo site?

Yes

Log Gist

https://gist.github.com/gboddin/7956cd755b034c1b399672c1abc6ab2f

Screenshots

Nothing to see, gitea hangs until OOM kills it

Git Version

git version 2.30.2

Operating System

debian-bullseye

How are you running Gitea?

• From binary local build 7671a33 (main while reporting )
• Built with TAGS="bindata sqlite sqlite_unlock_notify" make build
• Handled by systemd service

Database

SQLite

[gboddin]

Suspecting /user/events ( which looks like a stream ) after checking logs at https://gist.github.com/gboddin/7956cd755b034c1b399672c1abc6ab2f

[gboddin]

Happened 3 more times when checking an issue and hitting the browser Back button

[gboddin]

Ok I crashed the test instance.
Was playing around https://try.gitea.io/gbo/testreoi/issues and linked it to milestone

[gboddin]

Commented
https://github.com/go-gitea/gitea/blob/4266bd924bb8fff20271c17f8989b39b04ef6673/routers/web/web.go#L217..218
as a work-around for now.

We needed packages more than notifications and stopwatches :)

Let us know if we can provide anything else to help with 1.17 !

[silverwind]

Can reproduce locally as well by just starting my dev instance gitea. A lot of GetNotificationCount: context canceled spam in the log and finally a crash:

runtime: goroutine stack exceeds 1000000000-byte limit
runtime: sp=0xc37e9806c8 stack=[0xc37e980000, 0xc39e9800002022/05/24 20:26:56 ...user/notification.go:39:func1() [E] [628d234b] GetNotificationCount: context canceled
]
fatal error: stack overflow

runtime stack:
runtime.throw({0x645c942?, 0x81c7420?})
  /usr/local/Cellar/go/1.18.2/libexec/src/runtime/panic.go:992 +0x71
runtime.newstack()
  /usr/local/Cellar/go/1.18.2/libexec/src/runtime/stack.go:1101 +0x5cc
runtime.morestack()
  /usr/local/Cellar/go/1.18.2/libexec/src/runtime/asm_amd64.s:547 +0x8b

goroutines of crash

goroutine 1320 [running]:
reflect.funcLayout(0x5f245a0?, 0x0?)
  /usr/local/Cellar/go/1.18.2/libexec/src/reflect/type.go:3066 +0x6af fp=0xc37e9806d8 sp=0xc37e9806d0 pc=0x40989ef
reflect.Value.call({0x5f245a0?, 0x66293f8?, 0x198?}, {0x6364751, 0x4}, {0xc2dfffd470, 0x1, 0x6b1c700?})
  /usr/local/Cellar/go/1.18.2/libexec/src/reflect/value.go:442 +0x450 fp=0xc37e980cc8 sp=0xc37e9806d8 pc=0x4099f30
reflect.Value.Call({0x5f245a0?, 0x66293f8?, 0xc2e2e38e60?}, {0xc2dfffd470, 0x1, 0x1})
  /usr/local/Cellar/go/1.18.2/libexec/src/reflect/value.go:339 +0xbf fp=0xc37e980d40 sp=0xc37e980cc8 pc=0x409989f
text/template.safeCall({0x5f245a0?, 0x66293f8?, 0xc0067d8a80?}, {0xc2dfffd470?, 0x6b1c700?, 0x5f7b880?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/funcs.go:368 +0x9d fp=0xc37e980de0 sp=0xc37e980d40 pc=0x455739d
text/template.(*state).evalCall(0xc37e981398, {0x5fc09e0?, 0xc0067e34a0?, 0x2?}, {0x5f245a0?, 0x66293f8?, 0x1?}, 0x0, {0x6ae5338, 0xc003e7f350}, ...)
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:781 +0x87c fp=0xc37e980f40 sp=0xc37e980de0 pc=0x455229c
text/template.(*state).evalFunction(0xc37e981398, {0x5fc09e0?, 0xc0067e34a0?, 0x40e807f?}, 0x40ec620?, {0x6ae5338, 0xc003e7f350}, {0xc0050cce50, 0x1, 0x1}, ...)
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:601 +0x209 fp=0xc37e981038 sp=0xc37e980f40 pc=0x4550c69
text/template.(*state).evalCommand(0xc37e981398, {0x5fc09e0?, 0xc0067e34a0?, 0xc003ece000?}, 0xc37e981160?, {0x5eff460?, 0xc0067d8a80?, 0x6ac5900?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:488 +0xea fp=0xc37e981108 sp=0xc37e981038 pc=0x454f6ca
text/template.(*state).evalPipeline(0xc37e981398, {0x5fc09e0?, 0xc0067e34a0?, 0x3?}, 0xc00382dda0)
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:457 +0x145 fp=0xc37e9811f8 sp=0xc37e981108 pc=0x454f265
text/template.(*state).walk(0xc37e981398, {0x5fc09e0?, 0xc0067e34a0?, 0x5fc09e0?}, {0x6ae51d0?, 0xc0038491a0})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:261 +0x225 fp=0xc37e981298 sp=0xc37e9811f8 pc=0x454d985
text/template.(*state).walk(0xc37e981398, {0x5fc09e0?, 0xc0067e34a0?, 0x1?}, {0x6ae5530?, 0xc0038490b0?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:274 +0x415 fp=0xc37e981338 sp=0xc37e981298 pc=0x454db75
text/template.(*state).walkTemplate(0xc37e9815c0, {0x5fc09e0?, 0xc0067e34a0?, 0xc2dffa9930?}, 0xc005d5b540)
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:439 +0x285 fp=0xc37e9813f0 sp=0xc37e981338 pc=0x454f0c5
text/template.(*state).walk(0xc37e9815c0, {0x5fc09e0?, 0xc0067e34a0?, 0x10000?}, {0x6ae56e0?, 0xc005d5b540?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:279 +0x119 fp=0xc37e981490 sp=0xc37e9813f0 pc=0x454d879
text/template.(*state).walk(0xc37e9815c0, {0x5fc09e0?, 0xc0067e34a0?, 0x5fbca20?}, {0x6ae5530?, 0xc005d6c7b0?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:274 +0x415 fp=0xc37e981530 sp=0xc37e981490 pc=0x454db75
text/template.(*Template).execute(0xc005d5b500, {0x6acd840?, 0xc4529c9890?}, {0x5fc09e0?, 0xc0067e34a0?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:218 +0x2c5 fp=0xc37e981620 sp=0xc37e981530 pc=0x454d1a5
text/template.(*Template).Execute(...)
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:201
html/template.(*Template).ExecuteTemplate(0xc00572c220?, {0x6acd840, 0xc4529c9890}, {0x63f3d2a?, 0x0?}, {0x5fc09e0, 0xc0067e34a0})
  /usr/local/Cellar/go/1.18.2/libexec/src/html/template/template.go:139 +0x6a fp=0xc37e981658 sp=0xc37e981620 pc=0x48774ea
github.com/unrolled/render.HTML.Render({{{0xc465df43c0, 0x18}, 0x1f4}, {0x63f3d2a, 0xa}, 0xc00572ea80, {0x6ad9338, 0xc00572c220}}, {0x11309b80, 0xc00679cdc0}, ...)
  /Users/silverwind/go/pkg/mod/github.com/unrolled/render@v1.4.1/engine.go:94 +0x145 fp=0xc37e9816f0 sp=0xc37e981658 pc=0x511f405
github.com/unrolled/render.(*HTML).Render(0x0?, {0x11309b80?, 0xc00679cdc0?}, {0x5fc09e0?, 0xc0067e34a0?})
  <autogenerated>:1 +0xdd fp=0xc37e9817a0 sp=0xc37e9816f0 pc=0x5123f3d
github.com/unrolled/render.(*Render).Render(0xc004bc8180, {0x11309b80, 0xc00679cdc0}, {0x6ad5160?, 0xc2df4255c0?}, {0x5fc09e0?, 0xc0067e34a0?})
  /Users/silverwind/go/pkg/mod/github.com/unrolled/render@v1.4.1/render.go:413 +0x50 fp=0xc37e9817f8 sp=0xc37e9817a0 pc=0x5122630
github.com/unrolled/render.(*Render).HTML(0xc004bc8180, {0x11309b80, 0xc00679cdc0}, 0x1f4, {0x63f3d2a, 0xa}, {0x5fc09e0, 0xc0067e34a0}, {0x0, 0x0, ...})
  /Users/silverwind/go/pkg/mod/github.com/unrolled/render@v1.4.1/render.go:470 +0x49c fp=0xc37e981990 sp=0xc37e9817f8 pc=0x5122cdc
code.gitea.io/gitea/modules/context.(*Context).HTML(0xc0067d6dd0, 0x1f4, {0x63f3d2a, 0xa})
  /Users/silverwind/git/gitea/modules/context/context.go:226 +0x25c fp=0xc37e981a30 sp=0xc37e981990 pc=0x55e389c
code.gitea.io/gitea/modules/context.(*Context).serverErrorInternal(0xc0067d6dd0, {0x64c01bb?, 0x14?}, {0x6acf040?, 0xc00003c230?})
  /Users/silverwind/git/gitea/modules/context/context.go:304 +0x214 fp=0xc37e981a88 sp=0xc37e981a30 pc=0x55e4334
code.gitea.io/gitea/modules/context.(*Context).ServerError(...)
  /Users/silverwind/git/gitea/modules/context/context.go:286
code.gitea.io/gitea/routers/web/user.GetNotificationCount.func1()
  /Users/silverwind/git/gitea/routers/web/user/notification.go:39 +0x57 fp=0xc37e981ac8 sp=0xc37e981a88 pc=0x5bd9c77
runtime.call16(0xc003e555c0, 0xc00612e850, 0x0, 0x0, 0x0, 0x0, 0xc37e981ff8)
  /usr/local/Cellar/go/1.18.2/libexec/src/runtime/asm_amd64.s:701 +0x49 fp=0xc37e981ae8 sp=0xc37e981ac8 pc=0x406ca29
runtime.reflectcall(0x0?, 0x0?, 0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
  <autogenerated>:1 +0x3c fp=0xc37e981b28 sp=0xc37e981ae8 pc=0x4070e7c
reflect.Value.call({0x5ef4d60?, 0xc00612e850?, 0xc003c143d8?}, {0x6364751, 0x4}, {0x84402c8, 0x0, 0x487f8ed?})
  /usr/local/Cellar/go/1.18.2/libexec/src/reflect/value.go:556 +0x845 fp=0xc37e982118 sp=0xc37e981b28 pc=0x409a325
reflect.Value.Call({0x5ef4d60?, 0xc00612e850?, 0xc002c2d2cb?}, {0x84402c8, 0x0, 0x0})
  /usr/local/Cellar/go/1.18.2/libexec/src/reflect/value.go:339 +0xbf fp=0xc37e982190 sp=0xc37e982118 pc=0x409989f
text/template.safeCall({0x5ef4d60?, 0xc00612e850?, 0xc00406cd60?}, {0x84402c8?, 0x2?, 0x0?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/funcs.go:368 +0x9d fp=0xc37e982230 sp=0xc37e982190 pc=0x455739d
text/template.call({0x5f7b880?, 0xc2e2e38d00?, 0xc002a1d500?}, {0x84402c8, 0x0, 0xc00406cd70?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/funcs.go:353 +0x3d5 fp=0xc37e982328 sp=0xc37e982230 pc=0x45570d5
runtime.call64(0xc003e55530, 0x662a350, 0x0, 0x0, 0x0, 0x30, 0xc37e982888)
  /usr/local/Cellar/go/1.18.2/libexec/src/runtime/asm_amd64.s:703 +0x49 fp=0xc37e982378 sp=0xc37e982328 pc=0x406cb69
runtime.reflectcall(0x5ed4ae0?, 0xc2dfffd1b8?, 0x8?, 0x64a96de?, 0x0?, 0x12?, 0x5ed4ae0?)
  <autogenerated>:1 +0x3c fp=0xc37e9823b8 sp=0xc37e982378 pc=0x4070e7c
reflect.Value.call({0x5facd00?, 0x662a350?, 0x6330240?}, {0x6364751, 0x4}, {0xc2dfffd188, 0x1, 0x5fc09e0?})
  /usr/local/Cellar/go/1.18.2/libexec/src/reflect/value.go:556 +0x845 fp=0xc37e9829a8 sp=0xc37e9823b8 pc=0x409a325
reflect.Value.Call({0x5facd00?, 0x662a350?, 0xc0067e34a0?}, {0xc2dfffd188, 0x1, 0x1})
  /usr/local/Cellar/go/1.18.2/libexec/src/reflect/value.go:339 +0xbf fp=0xc37e982a20 sp=0xc37e9829a8 pc=0x409989f
text/template.safeCall({0x5facd00?, 0x662a350?, 0xc0067e34a0?}, {0xc2dfffd188?, 0x6b1c700?, 0x6330240?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/funcs.go:368 +0x9d fp=0xc37e982ac0 sp=0xc37e982a20 pc=0x455739d
text/template.(*state).evalCall(0xc37e983690, {0x5fc09e0?, 0xc0067e34a0?, 0xc2e2e38cc0?}, {0x5facd00?, 0x662a350?, 0x98?}, 0x1, {0x6ae5338, 0xc00380d6e0}, ...)
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:781 +0x87c fp=0xc37e982c20 sp=0xc37e982ac0 pc=0x455229c
text/template.(*state).evalFunction(0xc37e983690, {0x5fc09e0?, 0xc0067e34a0?, 0x17?}, 0x6ae5458?, {0x6ae5338, 0xc00380d6e0}, {0xc0054bb600, 0x2, 0x2}, ...)
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:601 +0x209 fp=0xc37e982d18 sp=0xc37e982c20 pc=0x4550c69
text/template.(*state).evalCommand(0xc37e983690, {0x5fc09e0?, 0xc0067e34a0?, 0xffffffffffffffff?}, 0x5fc09e0?, {0x5fdd4e0?, 0x84402c8?, 0x5fc09e0?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:488 +0xea fp=0xc37e982de8 sp=0xc37e982d18 pc=0x454f6ca
text/template.(*state).evalPipeline(0xc37e983690, {0x5fc09e0?, 0xc0067e34a0?, 0xc2e2e38ce0?}, 0xc00382c4e0)
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:457 +0x145 fp=0xc37e982ed8 sp=0xc37e982de8 pc=0x454f265
text/template.(*state).walk(0xc37e983690, {0x5fc09e0?, 0xc0067e34a0?, 0x5ef4d60?}, {0x6ae51d0?, 0xc00380d770})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:261 +0x225 fp=0xc37e982f78 sp=0xc37e982ed8 pc=0x454d985
text/template.(*state).walk(0xc37e983690, {0x5fc09e0?, 0xc0067e34a0?, 0x82?}, {0x6ae5530?, 0xc00380d680?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:274 +0x415 fp=0xc37e983018 sp=0xc37e982f78 pc=0x454db75
text/template.(*state).walkIfOrWith(0xc37e983690, 0xa, {0x5fc09e0?, 0xc0067e34a0?, 0xc00382c3c0?}, 0x5fc10a0?, 0xc003e54a50?, 0x0)
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:304 +0x1f8 fp=0xc37e9830e0 sp=0xc37e983018 pc=0x454ddd8
text/template.(*state).walk(0xc37e983690, {0x5fc09e0?, 0xc0067e34a0?, 0x5ee6720?}, {0x6ae54e8?, 0xc005beaac0?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:271 +0x1d6 fp=0xc37e983180 sp=0xc37e9830e0 pc=0x454d936
text/template.(*state).walk(0xc37e983690, {0x5fc09e0?, 0xc0067e34a0?, 0xc0067e34a0?}, {0x6ae5530?, 0xc0037d3b60?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:274 +0x415 fp=0xc37e983220 sp=0xc37e983180 pc=0x454db75
text/template.(*state).walkIfOrWith(0xc37e983690, 0xa, {0x5fc09e0?, 0xc0067e34a0?, 0x5fc09e0?}, 0x6364751?, 0xc4529c91a0?, 0xc0038fe120)
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:304 +0x1f8 fp=0xc37e9832e8 sp=0xc37e983220 pc=0x454ddd8
text/template.(*state).walk(0xc37e983690, {0x5fc09e0?, 0xc0067e34a0?, 0x0?}, {0x6ae54e8?, 0xc005afac40?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:271 +0x1d6 fp=0xc37e983388 sp=0xc37e9832e8 pc=0x454d936
text/template.(*state).walk(0xc37e983690, {0x5fc09e0?, 0xc0067e34a0?, 0x5fc09e0?}, {0x6ae5530?, 0xc0037d3ad0?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:274 +0x415 fp=0xc37e983428 sp=0xc37e983388 pc=0x454db75
text/template.(*state).walkIfOrWith(0xc37e983690, 0xa, {0x5fc09e0?, 0xc0067e34a0?, 0xc005ae2bc0?}, 0xc00561ee65?, 0x3?, 0xc0037d3ad0)
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:307 +0x22a fp=0xc37e9834f0 sp=0xc37e983428 pc=0x454de0a
text/template.(*state).walk(0xc37e983690, {0x5fc09e0?, 0xc0067e34a0?, 0x5fc09e0?}, {0x6ae54e8?, 0xc005afac80?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:271 +0x1d6 fp=0xc37e983590 sp=0xc37e9834f0 pc=0x454d936
text/template.(*state).walk(0xc37e983690, {0x5fc09e0?, 0xc0067e34a0?, 0xc001514301?}, {0x6ae5530?, 0xc0037d25d0?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:274 +0x415 fp=0xc37e983630 sp=0xc37e983590 pc=0x454db75
text/template.(*state).walkTemplate(0xc37e983a90, {0x5fc09e0?, 0xc0067e34a0?, 0x0?}, 0xc005c0ed40)
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:439 +0x285 fp=0xc37e9836e8 sp=0xc37e983630 pc=0x454f0c5
text/template.(*state).walk(0xc37e983a90, {0x5fc09e0?, 0xc0067e34a0?, 0x5ee6720?}, {0x6ae56e0?, 0xc005c0ed40?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:279 +0x119 fp=0xc37e983788 sp=0xc37e9836e8 pc=0x454d879
text/template.(*state).walk(0xc37e983a90, {0x5fc09e0?, 0xc0067e34a0?, 0x5fc09e0?}, {0x6ae5530?, 0xc003a0a5a0?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:274 +0x415 fp=0xc37e983828 sp=0xc37e983788 pc=0x454db75
text/template.(*state).walkIfOrWith(0xc37e983a90, 0xa, {0x5fc09e0?, 0xc0067e34a0?, 0xc005c0ecc0?}, 0xc37e983990?, 0x445433e?, 0x0)
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:304 +0x1f8 fp=0xc37e9838f0 sp=0xc37e983828 pc=0x454ddd8
text/template.(*state).walk(0xc37e983a90, {0x5fc09e0?, 0xc0067e34a0?, 0x5fc09e0?}, {0x6ae54e8?, 0xc005c0ef00?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:271 +0x1d6 fp=0xc37e983990 sp=0xc37e9838f0 pc=0x454d936
text/template.(*state).walk(0xc37e983a90, {0x5fc09e0?, 0xc0067e34a0?, 0x1?}, {0x6ae5530?, 0xc0038490b0?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:274 +0x415 fp=0xc37e983a30 sp=0xc37e983990 pc=0x454db75
text/template.(*state).walkTemplate(0xc37e983cb8, {0x5fc09e0?, 0xc0067e34a0?, 0xc2dffa9600?}, 0xc005d5b540)
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:439 +0x285 fp=0xc37e983ae8 sp=0xc37e983a30 pc=0x454f0c5
text/template.(*state).walk(0xc37e983cb8, {0x5fc09e0?, 0xc0067e34a0?, 0x10000?}, {0x6ae56e0?, 0xc005d5b540?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:279 +0x119 fp=0xc37e983b88 sp=0xc37e983ae8 pc=0x454d879
text/template.(*state).walk(0xc37e983cb8, {0x5fc09e0?, 0xc0067e34a0?, 0x5fbca20?}, {0x6ae5530?, 0xc005d6c7b0?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:274 +0x415 fp=0xc37e983c28 sp=0xc37e983b88 pc=0x454db75
text/template.(*Template).execute(0xc005d5b500, {0x6acd840?, 0xc4529c8870?}, {0x5fc09e0?, 0xc0067e34a0?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:218 +0x2c5 fp=0xc37e983d18 sp=0xc37e983c28 pc=0x454d1a5
text/template.(*Template).Execute(...)
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:201
html/template.(*Template).ExecuteTemplate(0xc00572c220?, {0x6acd840, 0xc4529c8870}, {0x63f3d2a?, 0x0?}, {0x5fc09e0, 0xc0067e34a0})
  /usr/local/Cellar/go/1.18.2/libexec/src/html/template/template.go:139 +0x6a fp=0xc37e983d50 sp=0xc37e983d18 pc=0x48774ea
github.com/unrolled/render.HTML.Render({{{0xc465df42b8, 0x18}, 0x1f4}, {0x63f3d2a, 0xa}, 0xc00572ea80, {0x6ad9338, 0xc00572c220}}, {0x11309b80, 0xc00679cdc0}, ...)
  /Users/silverwind/go/pkg/mod/github.com/unrolled/render@v1.4.1/engine.go:94 +0x145 fp=0xc37e983de8 sp=0xc37e983d50 pc=0x511f405
github.com/unrolled/render.(*HTML).Render(0x0?, {0x11309b80?, 0xc00679cdc0?}, {0x5fc09e0?, 0xc0067e34a0?})
  <autogenerated>:1 +0xdd fp=0xc37e983e98 sp=0xc37e983de8 pc=0x5123f3d
github.com/unrolled/render.(*Render).Render(0xc004bc8180, {0x11309b80, 0xc00679cdc0}, {0x6ad5160?, 0xc2df425500?}, {0x5fc09e0?, 0xc0067e34a0?})
  /Users/silverwind/go/pkg/mod/github.com/unrolled/render@v1.4.1/render.go:413 +0x50 fp=0xc37e983ef0 sp=0xc37e983e98 pc=0x5122630
github.com/unrolled/render.(*Render).HTML(0xc004bc8180, {0x11309b80, 0xc00679cdc0}, 0x1f4, {0x63f3d2a, 0xa}, {0x5fc09e0, 0xc0067e34a0}, {0x0, 0x0, ...})
  /Users/silverwind/go/pkg/mod/github.com/unrolled/render@v1.4.1/render.go:470 +0x49c fp=0xc37e984088 sp=0xc37e983ef0 pc=0x5122cdc
code.gitea.io/gitea/modules/context.(*Context).HTML(0xc0067d6dd0, 0x1f4, {0x63f3d2a, 0xa})
  /Users/silverwind/git/gitea/modules/context/context.go:226 +0x25c fp=0xc37e984128 sp=0xc37e984088 pc=0x55e389c
code.gitea.io/gitea/modules/context.(*Context).serverErrorInternal(0xc0067d6dd0, {0x64c01bb?, 0x14?}, {0x6acf040?, 0xc00003c230?})
  /Users/silverwind/git/gitea/modules/context/context.go:304 +0x214 fp=0xc37e984180 sp=0xc37e984128 pc=0x55e4334
code.gitea.io/gitea/modules/context.(*Context).ServerError(...)
  /Users/silverwind/git/gitea/modules/context/context.go:286
code.gitea.io/gitea/routers/web/user.GetNotificationCount.func1()
  /Users/silverwind/git/gitea/routers/web/user/notification.go:39 +0x57 fp=0xc37e9841c0 sp=0xc37e984180 pc=0x5bd9c77
runtime.call16(0xc003e555c0, 0xc00612e850, 0x0, 0x0, 0x0, 0x0, 0xc37e9846f0)
  /usr/local/Cellar/go/1.18.2/libexec/src/runtime/asm_amd64.s:701 +0x49 fp=0xc37e9841e0 sp=0xc37e9841c0 pc=0x406ca29
runtime.reflectcall(0x0?, 0x0?, 0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
  <autogenerated>:1 +0x3c fp=0xc37e984220 sp=0xc37e9841e0 pc=0x4070e7c
reflect.Value.call({0x5ef4d60?, 0xc00612e850?, 0xc003c143d8?}, {0x6364751, 0x4}, {0x84402c8, 0x0, 0x487f8ed?})
  /usr/local/Cellar/go/1.18.2/libexec/src/reflect/value.go:556 +0x845 fp=0xc37e984810 sp=0xc37e984220 pc=0x409a325
reflect.Value.Call({0x5ef4d60?, 0xc00612e850?, 0xc002c2d2cb?}, {0x84402c8, 0x0, 0x0})
  /usr/local/Cellar/go/1.18.2/libexec/src/reflect/value.go:339 +0xbf fp=0xc37e984888 sp=0xc37e984810 pc=0x409989f
text/template.safeCall({0x5ef4d60?, 0xc00612e850?, 0xc00406cd60?}, {0x84402c8?, 0x2?, 0x0?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/funcs.go:368 +0x9d fp=0xc37e984928 sp=0xc37e984888 pc=0x455739d
text/template.call({0x5f7b880?, 0xc2e2e28de0?, 0xc002a1d500?}, {0x84402c8, 0x0, 0xc00406cd70?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/funcs.go:353 +0x3d5 fp=0xc37e984a20 sp=0xc37e984928 pc=0x45570d5
runtime.call64(0xc003e55530, 0x662a350, 0x0, 0x0, 0x0, 0x30, 0xc37e984f80)
  /usr/local/Cellar/go/1.18.2/libexec/src/runtime/asm_amd64.s:703 +0x49 fp=0xc37e984a70 sp=0xc37e984a20 pc=0x406cb69
runtime.reflectcall(0x5ed4ae0?, 0xc2dfff7938?, 0x8?, 0x64a96de?, 0x0?, 0x12?, 0x5ed4ae0?)
  <autogenerated>:1 +0x3c fp=0xc37e984ab0 sp=0xc37e984a70 pc=0x4070e7c
reflect.Value.call({0x5facd00?, 0x662a350?, 0x6330240?}, {0x6364751, 0x4}, {0xc2dfff7908, 0x1, 0x5fc09e0?})
  /usr/local/Cellar/go/1.18.2/libexec/src/reflect/value.go:556 +0x845 fp=0xc37e9850a0 sp=0xc37e984ab0 pc=0x409a325
reflect.Value.Call({0x5facd00?, 0x662a350?, 0xc0067e34a0?}, {0xc2dfff7908, 0x1, 0x1})
  /usr/local/Cellar/go/1.18.2/libexec/src/reflect/value.go:339 +0xbf fp=0xc37e985118 sp=0xc37e9850a0 pc=0x409989f
text/template.safeCall({0x5facd00?, 0x662a350?, 0xc0067e34a0?}, {0xc2dfff7908?, 0x6b1c700?, 0x6330240?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/funcs.go:368 +0x9d fp=0xc37e9851b8 sp=0xc37e985118 pc=0x455739d
text/template.(*state).evalCall(0xc37e985d88, {0x5fc09e0?, 0xc0067e34a0?, 0xc2e2e28da0?}, {0x5facd00?, 0x662a350?, 0x98?}, 0x1, {0x6ae5338, 0xc00380d6e0}, ...)
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:781 +0x87c fp=0xc37e985318 sp=0xc37e9851b8 pc=0x455229c
text/template.(*state).evalFunction(0xc37e985d88, {0x5fc09e0?, 0xc0067e34a0?, 0x17?}, 0x6ae5458?, {0x6ae5338, 0xc00380d6e0}, {0xc0054bb600, 0x2, 0x2}, ...)
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:601 +0x209 fp=0xc37e985410 sp=0xc37e985318 pc=0x4550c69
text/template.(*state).evalCommand(0xc37e985d88, {0x5fc09e0?, 0xc0067e34a0?, 0xffffffffffffffff?}, 0x5fc09e0?, {0x5fdd4e0?, 0x84402c8?, 0x5fc09e0?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:488 +0xea fp=0xc37e9854e0 sp=0xc37e985410 pc=0x454f6ca
text/template.(*state).evalPipeline(0xc37e985d88, {0x5fc09e0?, 0xc0067e34a0?, 0xc2e2e28dc0?}, 0xc00382c4e0)
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:457 +0x145 fp=0xc37e9855d0 sp=0xc37e9854e0 pc=0x454f265
text/template.(*state).walk(0xc37e985d88, {0x5fc09e0?, 0xc0067e34a0?, 0x5ef4d60?}, {0x6ae51d0?, 0xc00380d770})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:261 +0x225 fp=0xc37e985670 sp=0xc37e9855d0 pc=0x454d985
text/template.(*state).walk(0xc37e985d88, {0x5fc09e0?, 0xc0067e34a0?, 0x82?}, {0x6ae5530?, 0xc00380d680?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:274 +0x415 fp=0xc37e985710 sp=0xc37e985670 pc=0x454db75
text/template.(*state).walkIfOrWith(0xc37e985d88, 0xa, {0x5fc09e0?, 0xc0067e34a0?, 0xc00382c3c0?}, 0x5fc10a0?, 0xc003e54a50?, 0x0)
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:304 +0x1f8 fp=0xc37e9857d8 sp=0xc37e985710 pc=0x454ddd8
text/template.(*state).walk(0xc37e985d88, {0x5fc09e0?, 0xc0067e34a0?, 0x5ee6720?}, {0x6ae54e8?, 0xc005beaac0?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:271 +0x1d6 fp=0xc37e985878 sp=0xc37e9857d8 pc=0x454d936
text/template.(*state).walk(0xc37e985d88, {0x5fc09e0?, 0xc0067e34a0?, 0xc0067e34a0?}, {0x6ae5530?, 0xc0037d3b60?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:274 +0x415 fp=0xc37e985918 sp=0xc37e985878 pc=0x454db75
text/template.(*state).walkIfOrWith(0xc37e985d88, 0xa, {0x5fc09e0?, 0xc0067e34a0?, 0x5fc09e0?}, 0x6364751?, 0xc4529c8180?, 0xc0038fe120)
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:304 +0x1f8 fp=0xc37e9859e0 sp=0xc37e985918 pc=0x454ddd8
text/template.(*state).walk(0xc37e985d88, {0x5fc09e0?, 0xc0067e34a0?, 0x0?}, {0x6ae54e8?, 0xc005afac40?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:271 +0x1d6 fp=0xc37e985a80 sp=0xc37e9859e0 pc=0x454d936
text/template.(*state).walk(0xc37e985d88, {0x5fc09e0?, 0xc0067e34a0?, 0x5fc09e0?}, {0x6ae5530?, 0xc0037d3ad0?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:274 +0x415 fp=0xc37e985b20 sp=0xc37e985a80 pc=0x454db75
text/template.(*state).walkIfOrWith(0xc37e985d88, 0xa, {0x5fc09e0?, 0xc0067e34a0?, 0xc005ae2bc0?}, 0xc00561ee65?, 0x3?, 0xc0037d3ad0)
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:307 +0x22a fp=0xc37e985be8 sp=0xc37e985b20 pc=0x454de0a
text/template.(*state).walk(0xc37e985d88, {0x5fc09e0?, 0xc0067e34a0?, 0x5fc09e0?}, {0x6ae54e8?, 0xc005afac80?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:271 +0x1d6 fp=0xc37e985c88 sp=0xc37e985be8 pc=0x454d936
text/template.(*state).walk(0xc37e985d88, {0x5fc09e0?, 0xc0067e34a0?, 0xc001514301?}, {0x6ae5530?, 0xc0037d25d0?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:274 +0x415 fp=0xc37e985d28 sp=0xc37e985c88 pc=0x454db75
text/template.(*state).walkTemplate(0xc37e986188, {0x5fc09e0?, 0xc0067e34a0?, 0x0?}, 0xc005c0ed40)
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:439 +0x285 fp=0xc37e985de0 sp=0xc37e985d28 pc=0x454f0c5
text/template.(*state).walk(0xc37e986188, {0x5fc09e0?, 0xc0067e34a0?, 0x5ee6720?}, {0x6ae56e0?, 0xc005c0ed40?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:279 +0x119 fp=0xc37e985e80 sp=0xc37e985de0 pc=0x454d879
text/template.(*state).walk(0xc37e986188, {0x5fc09e0?, 0xc0067e34a0?, 0x5fc09e0?}, {0x6ae5530?, 0xc003a0a5a0?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:274 +0x415 fp=0xc37e985f20 sp=0xc37e985e80 pc=0x454db75
text/template.(*state).walkIfOrWith(0xc37e986188, 0xa, {0x5fc09e0?, 0xc0067e34a0?, 0xc005c0ecc0?}, 0xc37e986088?, 0x445433e?, 0x0)
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:304 +0x1f8 fp=0xc37e985fe8 sp=0xc37e985f20 pc=0x454ddd8
text/template.(*state).walk(0xc37e986188, {0x5fc09e0?, 0xc0067e34a0?, 0x5fc09e0?}, {0x6ae54e8?, 0xc005c0ef00?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:271 +0x1d6 fp=0xc37e986088 sp=0xc37e985fe8 pc=0x454d936
text/template.(*state).walk(0xc37e986188, {0x5fc09e0?, 0xc0067e34a0?, 0x1?}, {0x6ae5530?, 0xc0038490b0?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:274 +0x415 fp=0xc37e986128 sp=0xc37e986088 pc=0x454db75
text/template.(*state).walkTemplate(0xc37e9863b0, {0x5fc09e0?, 0xc0067e34a0?, 0x20?}, 0xc005d5b540)
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:439 +0x285 fp=0xc37e9861e0 sp=0xc37e986128 pc=0x454f0c5
text/template.(*state).walk(0xc37e9863b0, {0x5fc09e0?, 0xc0067e34a0?, 0x454f265?}, {0x6ae56e0?, 0xc005d5b540?})
  /usr/local/Cellar/go/1.18.2/libexec/src/text/template/exec.go:279 +0x119 fp=0xc37e986280 sp=0xc37e9861e0 pc=0x454d879
created by net/http.(*Server).Serve
  /usr/local/Cellar/go/1.18.2/libexec/src/net/http/server.go:3071 +0x4db

goroutine 1 [IO wait]:
internal/poll.runtime_pollWait(0x1047ece8, 0x72)
  /usr/local/Cellar/go/1.18.2/libexec/src/runtime/netpoll.go:302 +0x89
internal/poll.(*pollDesc).wait(0xc004995c80?, 0xc007b15150?, 0x0)
  /usr/local/Cellar/go/1.18.2/libexec/src/internal/poll/fd_poll_runtime.go:83 +0x32
internal/poll.(*pollDesc).waitRead(...)
  /usr/local/Cellar/go/1.18.2/libexec/src/internal/poll/fd_poll_runtime.go:88
internal/poll.(*FD).Accept(0xc004995c80)
  /usr/local/Cellar/go/1.18.2/libexec/src/internal/poll/fd_unix.go:614 +0x22c
net.(*netFD).accept(0xc004995c80)
  /usr/local/Cellar/go/1.18.2/libexec/src/net/fd_unix.go:172 +0x35
net.(*TCPListener).accept(0xc001277170)
  /usr/local/Cellar/go/1.18.2/libexec/src/net/tcpsock_posix.go:139 +0x28
net.(*TCPListener).AcceptTCP(0xc001277170)
  /usr/local/Cellar/go/1.18.2/libexec/src/net/tcpsock.go:275 +0x3d
code.gitea.io/gitea/modules/graceful.(*wrappedListener).Accept(0xc0055cadc0)
  /Users/silverwind/git/gitea/modules/graceful/server.go:180 +0x47
net/http.(*Server).Serve(0xc00081a380, {0x6adb898, 0xc0055cadc0})
  /usr/local/Cellar/go/1.18.2/libexec/src/net/http/server.go:3039 +0x385
code.gitea.io/gitea/modules/graceful.(*Server).Serve(0xc003ca2070, 0xc005002ab0)
  /Users/silverwind/git/gitea/modules/graceful/server.go:133 +0xf7
code.gitea.io/gitea/modules/graceful.(*Server).ListenAndServe(0xc003ca2070, 0x4997a40?)
  /Users/silverwind/git/gitea/modules/graceful/server.go:95 +0x1bf
code.gitea.io/gitea/modules/graceful.HTTPListenAndServe({0x6360691?, 0x16?}, {0xc0053ec2c0?, 0x12?}, {0x635d9df?, 0x0?}, {0x6acdf20?, 0xc001dd0340?})
  /Users/silverwind/git/gitea/modules/graceful/server_http.go:33 +0x3d
code.gitea.io/gitea/cmd.runHTTP(...)
  /Users/silverwind/git/gitea/cmd/web_graceful.go:19
code.gitea.io/gitea/cmd.listen({0x6acdf20, 0xc001dd0340}, 0x1)
  /Users/silverwind/git/gitea/cmd/web.go:231 +0x4e5
code.gitea.io/gitea/cmd.runWeb(0xc001e25e40)
  /Users/silverwind/git/gitea/cmd/web.go:176 +0x831
github.com/urfave/cli.HandleAction({0x5f219c0?, 0x6622630?}, 0xc001e3b500?)
  /Users/silverwind/go/pkg/mod/github.com/urfave/cli@v1.22.9/app.go:524 +0xa8
github.com/urfave/cli.(*App).Run(0xc001e3b500, {0xc00003c200, 0x1, 0x1})
  /Users/silverwind/go/pkg/mod/github.com/urfave/cli@v1.22.9/app.go:286 +0x7c5
main.main()
  /Users/silverwind/git/gitea/main.go:115 +0xcf9

goroutine 375 [select]:
code.gitea.io/gitea/modules/eventsource.(*Manager).Run(0xc000968048, {0x6adf8c0, 0xc000327470})
  /Users/silverwind/git/gitea/modules/eventsource/manager_run.go:38 +0x147
code.gitea.io/gitea/modules/graceful.(*Manager).RunWithShutdownContext(0xc001e7e0f0, 0xc004722930)
  /Users/silverwind/git/gitea/modules/graceful/manager.go:114 +0xc7
created by code.gitea.io/gitea/modules/eventsource.(*Manager).Init
  /Users/silverwind/git/gitea/modules/eventsource/manager_run.go:26 +0xdb

goroutine 5 [select]:
code.gitea.io/gitea/modules/log.(*MultiChannelledLog).Start(0xc000120240)
  /Users/silverwind/git/gitea/modules/log/event.go:325 +0x2af
created by code.gitea.io/gitea/modules/log.(*MultiChannelledLog).AddLogger
  /Users/silverwind/git/gitea/modules/log/event.go:206 +0x176

goroutine 7 [select]:
github.com/blevesearch/bleve_index_api.AnalysisWorker({0xc001c1a1e0?, 0xc001c1a240?})
  /Users/silverwind/go/pkg/mod/github.com/blevesearch/bleve_index_api@v1.0.1/analysis.go:46 +0x8f
created by github.com/blevesearch/bleve_index_api.NewAnalysisQueue
  /Users/silverwind/go/pkg/mod/github.com/blevesearch/bleve_index_api@v1.0.1/analysis.go:38 +0x92

goroutine 8 [select]:
github.com/blevesearch/bleve_index_api.AnalysisWorker({0xc001c1a1e0?, 0xc001c1a240?})
  /Users/silverwind/go/pkg/mod/github.com/blevesearch/bleve_index_api@v1.0.1/analysis.go:46 +0x8f
created by github.com/blevesearch/bleve_index_api.NewAnalysisQueue
  /Users/silverwind/go/pkg/mod/github.com/blevesearch/bleve_index_api@v1.0.1/analysis.go:38 +0x92

goroutine 9 [select]:
github.com/blevesearch/bleve_index_api.AnalysisWorker({0xc001c1a1e0?, 0xc001c1a240?})
  /Users/silverwind/go/pkg/mod/github.com/blevesearch/bleve_index_api@v1.0.1/analysis.go:46 +0x8f
created by github.com/blevesearch/bleve_index_api.NewAnalysisQueue
  /Users/silverwind/go/pkg/mod/github.com/blevesearch/bleve_index_api@v1.0.1/analysis.go:38 +0x92

goroutine 10 [select]:
github.com/blevesearch/bleve_index_api.AnalysisWorker({0xc001c1a1e0?, 0xc001c1a240?})
  /Users/silverwind/go/pkg/mod/github.com/blevesearch/bleve_index_api@v1.0.1/analysis.go:46 +0x8f
created by github.com/blevesearch/bleve_index_api.NewAnalysisQueue
  /Users/silverwind/go/pkg/mod/github.com/blevesearch/bleve_index_api@v1.0.1/analysis.go:38 +0x92

goroutine 11 [select]:
code.gitea.io/gitea/modules/graceful.(*Manager).handleSignals(0x0?, {0x6adf8c0, 0xc000327530})
  /Users/silverwind/git/gitea/modules/graceful/manager_unix.go:132 +0x205
created by code.gitea.io/gitea/modules/graceful.(*Manager).start
  /Users/silverwind/git/gitea/modules/graceful/manager_unix.go:76 +0x9a5

goroutine 21 [syscall]:
os/signal.signal_recv()
  /usr/local/Cellar/go/1.18.2/libexec/src/runtime/sigqueue.go:148 +0x28
os/signal.loop()
  /usr/local/Cellar/go/1.18.2/libexec/src/os/signal/signal_unix.go:23 +0x19
created by os/signal.Notify.func1.1
  /usr/local/Cellar/go/1.18.2/libexec/src/os/signal/signal.go:151 +0x2a

goroutine 1414 [IO wait]:
internal/poll.runtime_pollWait(0x1047eb08, 0x72)
  /usr/local/Cellar/go/1.18.2/libexec/src/runtime/netpoll.go:302 +0x89
internal/poll.(*pollDesc).wait(0xc05d5f8d80?, 0xc06592f391?, 0x0)
  /usr/local/Cellar/go/1.18.2/libexec/src/internal/poll/fd_poll_runtime.go:83 +0x32
internal/poll.(*pollDesc).waitRead(...)
  /usr/local/Cellar/go/1.18.2/libexec/src/internal/poll/fd_poll_runtime.go:88
internal/poll.(*FD).Read(0xc05d5f8d80, {0xc06592f391, 0x1, 0x1})
  /usr/local/Cellar/go/1.18.2/libexec/src/internal/poll/fd_unix.go:167 +0x25a
net.(*netFD).Read(0xc05d5f8d80, {0xc06592f391?, 0xc00030cea0?, 0xc001eaaf48?})
  /usr/local/Cellar/go/1.18.2/libexec/src/net/fd_posix.go:55 +0x29
net.(*conn).Read(0xc05cffb2f0, {0xc06592f391?, 0xc005013d50?, 0xc03975c9b0?})
  /usr/local/Cellar/go/1.18.2/libexec/src/net/net.go:183 +0x45
net/http.(*connReader).backgroundRead(0xc06592f380)
  /usr/local/Cellar/go/1.18.2/libexec/src/net/http/server.go:672 +0x3f
created by net/http.(*connReader).startBackgroundRead
  /usr/local/Cellar/go/1.18.2/libexec/src/net/http/server.go:668 +0xca

goroutine 114 [select]:
code.gitea.io/gitea/modules/log.(*ChannelledLog).Start(0xc00097c690)
  /Users/silverwind/git/gitea/modules/log/event.go:80 +0xcc
created by code.gitea.io/gitea/modules/log.NewChannelledLog
  /Users/silverwind/git/gitea/modules/log/event.go:69 +0x365

goroutine 54 [select]:
github.com/syndtr/goleveldb/leveldb/util.(*BufferPool).drain(0xc00081a1c0)
  /Users/silverwind/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/util/buffer_pool.go:206 +0xc7
created by github.com/syndtr/goleveldb/leveldb/util.NewBufferPool
  /Users/silverwind/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/util/buffer_pool.go:237 +0x18c

goroutine 55 [select]:
github.com/syndtr/goleveldb/leveldb.(*DB).compactionError(0xc0002a3860)
  /Users/silverwind/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db_compaction.go:90 +0x158
created by github.com/syndtr/goleveldb/leveldb.openDB
  /Users/silverwind/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db.go:142 +0x4ea

goroutine 56 [select]:
github.com/syndtr/goleveldb/leveldb.(*DB).mpoolDrain(0xc0002a3860)
  /Users/silverwind/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db_state.go:101 +0xa8
created by github.com/syndtr/goleveldb/leveldb.openDB
  /Users/silverwind/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db.go:143 +0x52a

goroutine 57 [select]:
github.com/syndtr/goleveldb/leveldb.(*DB).tCompaction(0xc0002a3860)
  /Users/silverwind/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db_compaction.go:825 +0x657
created by github.com/syndtr/goleveldb/leveldb.openDB
  /Users/silverwind/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db.go:149 +0x598

goroutine 58 [select]:
github.com/syndtr/goleveldb/leveldb.(*DB).mCompaction(0xc0002a3860)
  /Users/silverwind/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db_compaction.go:762 +0x113
created by github.com/syndtr/goleveldb/leveldb.openDB
  /Users/silverwind/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db.go:150 +0x5d6

goroutine 65 [select]:
database/sql.(*DB).connectionOpener(0xc001af9ee0, {0x6adf818, 0xc000021bc0})
  /usr/local/Cellar/go/1.18.2/libexec/src/database/sql/sql.go:1226 +0x8d
created by database/sql.OpenDB
  /usr/local/Cellar/go/1.18.2/libexec/src/database/sql/sql.go:794 +0x18d

goroutine 1020 [select]:
code.gitea.io/gitea/modules/graceful.(*Server).awaitShutdown(0xc000759290?)
  /Users/silverwind/git/gitea/modules/graceful/server_hooks.go:16 +0xab
created by code.gitea.io/gitea/modules/graceful.(*Server).ListenAndServe
  /Users/silverwind/git/gitea/modules/graceful/server.go:83 +0x65

goroutine 287 [select]:
github.com/gogs/cron.(*Cron).run(0xc000952af0)
  /Users/silverwind/go/pkg/mod/github.com/gogs/cron@v0.0.0-20171120032916-9f6c956d3e14/cron.go:197 +0x596
created by github.com/gogs/cron.(*Cron).Start
  /Users/silverwind/go/pkg/mod/github.com/gogs/cron@v0.0.0-20171120032916-9f6c956d3e14/cron.go:150 +0x65

goroutine 360 [select]:
code.gitea.io/gitea/modules/queue.(*ByteFIFOQueue).readToChan(0xc0047826e0)
  /Users/silverwind/git/gitea/modules/queue/queue_bytefifo.go:254 +0x5aa
code.gitea.io/gitea/modules/queue.(*ByteFIFOQueue).Run(0xc0047826e0, 0xc004791210, 0xc004791220)
  /Users/silverwind/git/gitea/modules/queue/queue_bytefifo.go:164 +0x23c
code.gitea.io/gitea/modules/graceful.(*Manager).RunWithShutdownFns(0xc001e7e0f0, 0xc004890198)
  /Users/silverwind/git/gitea/modules/graceful/manager.go:82 +0x123
created by code.gitea.io/gitea/modules/indexer/issues.InitIssueIndexer.func2
  /Users/silverwind/git/gitea/modules/indexer/issues/indexer.go:242 +0x6fb

goroutine 1351 [se]:
code.gitea.io/gitea/routers/web/events.Events(0xc0b48a6000)
  /Users/silverwind/git/gitea/routers/web/events/events.go:71 +0x698
code.gitea.io/gitea/modules/web.convertHandler.func3({0x6adb808?, 0xc0055e1400?}, 0xc0b3c8fb00, {0x0?, 0x4014e16?, 0x0?})
  /Users/silverwind/git/gitea/modules/web/wrap_convert.go:44 +0xb7
code.gitea.io/gitea/modules/web.wrapInternal.func1({0x6adb808, 0xc0055e1400}, 0xc003220990?)
  /Users/silverwind/git/gitea/modules/web/wrap.go:41 +0x12a
net/http.HandlerFunc.ServeHTTP(0x5fbf780?, {0x6adb808?, 0xc0055e1400?}, 0xc0b4892830?)
  /usr/local/Cellar/go/1.18.2/libexec/src/net/http/server.go:2084 +0x2f
github.com/go-chi/chi/v5.(*Mux).routeHTTP(0xc003d01440, {0x6adb808, 0xc0055e1400}, 0xc0b3c8fb00)
  /Users/silverwind/go/pkg/mod/github.com/go-chi/chi/v5@v5.0.7/mux.go:442 +0x216
net/http.HandlerFunc.ServeHTTP(0xc0b489dce0?, {0x6adb808?, 0xc0055e1400?}, 0xc0b489dbf0?)
  /usr/local/Cellar/go/1.18.2/libexec/src/net/http/server.go:2084 +0x2f
code.gitea.io/gitea/modules/web.Middle.func1.1({0x606cde0?, 0x616bf00?}, 0xc0b3c8fb00)
  /Users/silverwind/git/gitea/modules/web/wrap.go:63 +0x130
net/http.HandlerFunc.ServeHTTP(0xc0b48a6000?, {0x6adb808?, 0xc0055e1400?}, 0x4010e47?)
  /usr/local/Cellar/go/1.18.2/libexec/src/net/http/server.go:2084 +0x2f
code.gitea.io/gitea/modules/web.Middle.func1.1({0x606cde0?, 0x616bf00?}, 0xc0b3c8fb00)
  /Users/silverwind/git/gitea/modules/web/wrap.go:63 +0x130
net/http.HandlerFunc.ServeHTTP(0xc0b48a6000?, {0x6adb808?, 0xc0055e1400?}, 0xc000738000?)
  /usr/local/Cellar/go/1.18.2/libexec/src/net/http/server.go:2084 +0x2f
code.gitea.io/gitea/modules/web.Middle.func1.1({0xc0b488fd20?, 0xc059fd4e80?}, 0xc0b3c8fb00)
  /Users/silverwind/git/gitea/modules/web/wrap.go:63 +0x130
net/http.HandlerFunc.ServeHTTP(0x6395c90?, {0x6adb808?, 0xc0055e1400?}, 0x400d7cd?)
  /usr/local/Cellar/go/1.18.2/libexec/src/net/http/server.go:2084 +0x2f
github.com/go-chi/chi/v5/middleware.GetHead.func1({0x6adb808?, 0xc0055e1400?}, 0xc0b3c8fb00?)
  /Users/silverwind/go/pkg/mod/github.com/go-chi/chi/v5@v5.0.7/middleware/get_head.go:37 +0x1e5
net/http.HandlerFunc.ServeHTTP(0xc0b48a6000?, {0x6adb808?, 0xc0055e1400?}, 0x8a96108?)
  /usr/local/Cellar/go/1.18.2/libexec/src/net/http/server.go:2084 +0x2f
code.gitea.io/gitea/modules/web.Middle.func1.1({0x606cde0?, 0x616bf00?}, 0xc0b3c8fb00)
  /Users/silverwind/git/gitea/modules/web/wrap.go:63 +0x130
net/http.HandlerFunc.ServeHTTP(0x5fc09e0?, {0x6adb808?, 0xc0055e1400?}, 0x8?)
  /usr/local/Cellar/go/1.18.2/libexec/src/net/http/server.go:2084 +0x2f
code.gitea.io/gitea/modules/context.Contexter.func1.1({0x6adb808?, 0xc0055e1400}, 0xc0b3c8fa00)
  /Users/silverwind/git/gitea/modules/context/context.go:800 +0x19db
net/http.HandlerFunc.ServeHTTP(0x5f91520?, {0x6adb808?, 0xc0055e1400?}, 0x40561f0?)
  /usr/local/Cellar/go/1.18.2/libexec/src/net/http/server.go:2084 +0x2f
github.com/go-chi/chi/v5.(*Mux).ServeHTTP(0xc003d01440, {0x6adb808, 0xc0055e1400}, 0xc0b3c8fa00)
  /Users/silverwind/go/pkg/mod/github.com/go-chi/chi/v5@v5.0.7/mux.go:71 +0x48d
github.com/go-chi/chi/v5.(*Mux).Mount.func1({0x6adb808, 0xc0055e1400}, 0xc0b3c8fa00)
  /Users/silverwind/go/pkg/mod/github.com/go-chi/chi/v5@v5.0.7/mux.go:314 +0x19c
net/http.HandlerFunc.ServeHTTP(0x5fbf780?, {0x6adb808?, 0xc0055e1400?}, 0xc0b4892720?)
  /usr/local/Cellar/go/1.18.2/libexec/src/net/http/server.go:2084 +0x2f
github.com/go-chi/chi/v5.(*Mux).routeHTTP(0xc0049610e0, {0x6adb808, 0xc0055e1400}, 0xc0b3c8fa00)
  /Users/silverwind/go/pkg/mod/github.com/go-chi/chi/v5@v5.0.7/mux.go:442 +0x216
net/http.HandlerFunc.ServeHTTP(0xc0b489dbf0?, {0x6adb808?, 0xc0055e1400?}, 0x113137c0?)
  /usr/local/Cellar/go/1.18.2/libexec/src/net/http/server.go:2084 +0x2f
code.gitea.io/gitea/routers/web.Recovery.func1.1({0x6adb808?, 0xc0055e1400?}, 0x6ac9201?)
  /Users/silverwind/git/gitea/routers/web/base.go:173 +0x92
net/http.HandlerFunc.ServeHTTP(0x6adf738?, {0x6adb808?, 0xc0055e1400?}, 0x6ac9250?)
  /usr/local/Cellar/go/1.18.2/libexec/src/net/http/server.go:2084 +0x2f
gitea.com/go-chi/session.Sessioner.func1.1({0x6adb808, 0xc0055e1400}, 0xc0b3c8f900)
  /Users/silverwind/go/pkg/mod/gitea.com/go-chi/session@v0.0.0-20211218221615-e3605d8b28b8/session.go:257 +0x2fe
net/http.HandlerFunc.ServeHTTP(0xc059fd5580?, {0x6adb808?, 0xc0055e1400?}, 0xc0055e1300?)
  /usr/local/Cellar/go/1.18.2/libexec/src/net/http/server.go:2084 +0x2f
code.gitea.io/gitea/modules/web.WrapWithPrefix.func1.1({0x6adb808?, 0xc0055e1400?}, 0xc0b3c8f900?)
  /Users/silverwind/git/gitea/modules/web/wrap.go:110 +0x109
net/http.HandlerFunc.ServeHTTP(0x4100ab4?, {0x6adb808?, 0xc0055e1400?}, 0x40561f0?)
  /usr/local/Cellar/go/1.18.2/libexec/src/net/http/server.go:2084 +0x2f
github.com/go-chi/chi/v5.(*Mux).ServeHTTP(0xc0049610e0, {0x6adb808, 0xc0055e1400}, 0xc0b3c8f900)
  /Users/silverwind/go/pkg/mod/github.com/go-chi/chi/v5@v5.0.7/mux.go:71 +0x48d
github.com/go-chi/chi/v5.(*Mux).Mount.func1({0x6adb808, 0xc0055e1400}, 0xc0b3c8f900)
  /Users/silverwind/go/pkg/mod/github.com/go-chi/chi/v5@v5.0.7/mux.go:314 +0x19c
net/http.HandlerFunc.ServeHTTP(0x5fbf780?, {0x6adb808?, 0xc0055e1400?}, 0xc0b3cdf1a4?)
  /usr/local/Cellar/go/1.18.2/libexec/src/net/http/server.go:2084 +0x2f
github.com/go-chi/chi/v5.(*Mux).routeHTTP(0xc004961080, {0x6adb808, 0xc0055e1400}, 0xc0b3c8f900)
  /Users/silverwind/go/pkg/mod/github.com/go-chi/chi/v5@v5.0.7/mux.go:442 +0x216
net/http.HandlerFunc.ServeHTTP(0x835e3d0?, {0x6adb808?, 0xc0055e1400?}, 0x4100ab4?)
  /usr/local/Cellar/go/1.18.2/libexec/src/net/http/server.go:2084 +0x2f
code.gitea.io/gitea/routers/common.Middlewares.func2.1({0x6adb808?, 0xc0055e1400?}, 0x444d365?)
  /Users/silverwind/git/gitea/routers/common/middleware.go:79 +0x83
net/http.HandlerFunc.ServeHTTP(0x6466ac4?, {0x6adb808?, 0xc0055e1400?}, 0x5a6a966?)
  /usr/local/Cellar/go/1.18.2/libexec/src/net/http/server.go:2084 +0x2f
github.com/go-chi/chi/v5/middleware.StripSlashes.func1({0x6adb808, 0xc0055e1400}, 0xc0b3c8f900)
  /Users/silverwind/go/pkg/mod/github.com/go-chi/chi/v5@v5.0.7/middleware/strip.go:30 +0x139
net/http.HandlerFunc.ServeHTTP(0xc0b3c8f900?, {0x6adb808?, 0xc0055e1400?}, 0x63a2b4f?)
  /usr/local/Cellar/go/1.18.2/libexec/src/net/http/server.go:2084 +0x2f
github.com/chi-middleware/proxy.ForwardedHeaders.func1.1({0x6adb808, 0xc0055e1400}, 0xc0b3c8f900)
  /Users/silverwind/go/pkg/mod/github.com/chi-middleware/proxy@v1.1.1/middleware.go:37 +0x197
net/http.HandlerFunc.ServeHTTP(0x637c6ba?, {0x6adb808?, 0xc0055e1400?}, 0xc0b3ca3338?)
  /usr/local/Cellar/go/1.18.2/libexec/src/net/http/server.go:2084 +0x2f
code.gitea.io/gitea/routers/common.Middlewares.func1.1({0x6adddb8?, 0xc0047f01c0}, 0xc0b3c8f800)
  /Users/silverwind/git/gitea/routers/common/middleware.go:32 +0x473
net/http.HandlerFunc.ServeHTTP(0x6adf818?, {0x6adddb8?, 0xc0047f01c0?}, 0x835e3d0?)
  /usr/local/Cellar/go/1.18.2/libexec/src/net/http/server.go:2084 +0x2f
github.com/go-chi/chi/v5.(*Mux).ServeHTTP(0xc004961080, {0x6adddb8, 0xc0047f01c0}, 0xc038b3f500)
  /Users/silverwind/go/pkg/mod/github.com/go-chi/chi/v5@v5.0.7/mux.go:88 +0x442
code.gitea.io/gitea/modules/web.(*Route).ServeHTTP(0x0?, {0x6adddb8?, 0xc0047f01c0?}, 0xc000738000?)
  /Users/silverwind/git/gitea/modules/web/route.go:200 +0x2e
net/http.serverHandler.ServeHTTP({0xc06592f380?}, {0x6adddb8, 0xc0047f01c0}, 0xc038b3f500)
  /usr/local/Cellar/go/1.18.2/libexec/src/net/http/server.go:2916 +0x43b
net/http.(*conn).serve(0xc003dc0a00, {0x6adf8c0, 0xc003ca01e0})
  /usr/local/Cellar/go/1.18.2/libexec/src/net/http/server.go:1966 +0x5d7
created by net/http.(*Server).Serve
  /usr/local/Cellar/go/1.18.2/libexec/src/net/http/server.go:3071 +0x4db

Interestingly, crash did not occur again on second startup.

[zeripath]

Agh I hit this a few days ago whilst I was in debug mode but thought it was due to too many checkpoints.

This is a regression from #19748 but it is looping because of a mistake in #10961

The problem is in here:

	c.Data["NotificationUnreadCount"] = func() int64 {
		count, err := models.GetNotificationCount(c, c.Doer, models.NotificationStatusUnread)
		if err != nil {
			c.ServerError("GetNotificationCount", err)
			return -1
		}

		return count
	}

The function here is called during rendering.

If there is an err it's too late really to use ServerError as we're already rendering - and we should simply return -1 here and log the problem.

I'll pop up a PR.

[zeripath]

As the bug is a regression due to unreleased code the PR to fix it has been marked skip-changelog.