Debian 9.6 AMD64: Segfault on start at os/user.Current (1.7.0+)

[hendrikbl]

• Gitea version (or commit ref): 1.6.4
• Git version: 2.11.0
• Operating system: Debian 9.6
• Database (use [x]):
  • PostgreSQL
  • MySQL (MariaDB)
  • MSSQL
  • SQLite
• Can you reproduce the bug at https://try.gitea.io:
  • Yes (provide example URL)
  • No
  • Not relevant
• Log gist:

Description

After upgrading from gitea version 1.6.4 to 1.7.0 (replacing gitea binary with new one) I'm unable to start gitea. The gitea service fails with the following message (service gitea status):

● gitea.service - Gitea (Git with a cup of tea)
   Loaded: loaded (/etc/systemd/system/gitea.service; enabled; vendor preset: enabled)
   Active: activating (auto-restart) (Result: exit-code) since Fri 2019-05-03 12:27:24 CEST; 1s ago
  Process: 31656 ExecStart=/usr/local/bin/gitea web -c /etc/gitea/app.ini (code=exited, status=2)
 Main PID: 31656 (code=exited, status=2)

If I try to run it directly via gitea web I get the following:
Gist

More logs:

Mai 03 12:18:43 gitea systemd[1]: Started Gitea (Git with a cup of tea).
Mai 03 12:18:43 gitea gitea[29491]: fatal error: unexpected signal during runtime execution
Mai 03 12:18:43 gitea gitea[29491]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x0]
Mai 03 12:18:43 gitea gitea[29491]: runtime stack:
Mai 03 12:18:43 gitea gitea[29491]: runtime.throw(0x19c3d99, 0x2a)
Mai 03 12:18:43 gitea systemd[1]: gitea.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Mai 03 12:18:43 gitea systemd[1]: gitea.service: Unit entered failed state.
Mai 03 12:18:43 gitea systemd[1]: gitea.service: Failed with result 'exit-code'.

I get this error with every version greater than 1.6.4. All permissions were set the same with every version I tried. I haven't tried building gitea myself since this is something I only want to do if there really is no other way.

[zeripath]

Are you running Gitea on a raspberry pi? If so there is a known problem with the arm7 binary and you need to use arm6.

If not what kernel version do you have? If it's very old you're likely to need to build Gitea yourself as we build on Ubuntu 18.04.

Finally, please don't run 1.7.0 there are a number of security issues in that version and we recommend 1.8 if not at least 1.7.6 (there are fixes in 1.8 that cannot be ported back to 1.7.6)

[hendrikbl]

I'm on a virtual machine running Debian 9.6 x86_64 with kernel version 4.9.0-4-amd64.

I tried version 1.7.0, 1.7.6 and 1.8. They all give me the same error.

[zeripath]

Ok.

Under what user are you trying to run Gitea?
And where? Things don't make sense.

The segfault looks like it's happening when Gitea tries to get the current username. That implies that you're running Gitea as a weird user.

Although you state:

/usr/local/bin/gitea web -c /etc/gitea/app.ini

Gitea seems to be trying get its config:

/usr/local/bin/custom/conf/app.ini

Not in /etc...

What does your service file look like?

Does Gitea run when you just run it as a normal user from a normal directory?

[hendrikbl]

I followed the documentation when first installing gitea. So gitea service is running as git user:

root@gitea:~# cat /etc/passwd | grep git
git:x:109:116:Git Version Control,,,:/home/git:/bin/bash

The binary is located at /usr/local/bin. The working directory is /var/lib/gitea/. As seen in the documentation.

Here's my service file: gitea.service

I tried running gitea as root and git from the corresponding home directory but i still get a segfault.

Although you state:

/usr/local/bin/gitea web -c /etc/gitea/app.ini

Gitea seems to be trying get its config:

/usr/local/bin/custom/conf/app.ini

Not in /etc...

Thats my fault. The log I gave you was from running gitea without the -c argument. Here are the three things I tried:

• User: git, Directory: /usr/local/bin - Gist
• User: git, Directory: /home/git - Gist
• User: root, Directory: home/root - Gist

Does Gitea run when you just run it as a normal user from a normal directory?

So the answer to this question is: No, I can't run gitea anywhere. The strange thing is that version 1.6.4 runs everytime on every circumstance. Everything thats newer than that fails.

Edit: Everything here was tested with version 1.8 of gitea.

[sapk]

This thread on discourse seem related: https://discourse.gitea.io/t/solved-segfault-due-to-misconfigured-unix-user/1104

[zeripath]

Hmm. It looks like this is:

golang/go#24787

Which if I'm reading correctly causes panic on os/user.Current or Lookup for uid <1000.

Now it seems like it's suggested that we need to add more flags to our static build:

golang/go#24787 (comment)

But I bet a simple workaround is to have a uid>=1000. Now you may actually need that in any case because of the SSH requirements - in that if you want to clone over SSH you need to be able to login as that user and run authorized_keys or an authorized keys command which I'm not certain you can do as a uid <1000.

[sapk]

@zeripath It is a pity that we need an other tag (osusergo) like netgo but if it ensure consistency across env we should add it.

[hendrikbl]

I tried running gitea with a new user with a uid>1000:

root@gitea:/usr/local/bin# grep gitea /etc/shadow
gitea:!:18022:0:99999:7:::
root@gitea:/usr/local/bin# grep gitea /etc/passwd
gitea:x:1003:1002:Gitea,,,:/home/gitea:/bin/bash

Still no luck. I get exactly the same. Current version 1.6.4 runs perfectly, everything above that fails.

Can I use the gitea dump command on my current version, create a fresh install of version 1.8 somewhere else and restore the dump? If that's the case I will try that.

[lunny]

@hendrikbl That's dump and restore are not guaranteed.

[zeripath]

@hendrikbl I assume that by a uid > 1000 you haven't marked the user as a system user?

OK, so I wonder how we test this. I've just downloaded the docker debian:latest and that doesn't replicate the issue with 1.8.0

[zeripath]

and I've just tried docker pull debian:9.6 and haven't got this issue either and 1.8 and 1.7.6

[hendrikbl]

@zeripath correct, I just used adduser and followed the setup without setting a password.

Hmm, have you done everything according to the installation guide? Because that's what I did. I'll see if I can try this on a fresh debian install this week. I'm not sure when I have time for that though.

[zeripath]

OK, I mean I can start from the command line - I think getting systemd started on a debian docker is fairly difficult.