Consider democratizing access to production logs

[davidalber]

I am interested in investigating instances of unexpected Highfive behavior. For example, sometimes, but not consistently, Highfive sends the welcome message to existing contributors. Another example: in rust-lang/rust#49849, Highfive did not take any action at all. It's difficult to know what is going on there without access to the logs. Looking at Highfive's code I can speculate that maybe the diff request failed, but there's several other possibilities.

The first question is: are the organization members comfortable granting access to the logs to a non-member?

If the answer is yes, we should find a reasonable, low-cost-to-@nrc way to accomplish it. A lot of options come to mind, but they generally branch on whether we want to provide access to the host Highfive runs on or whether we want to ship the logs to somewhere else.

1. Provide access to host running Highfive.
   1. Continue running Highfive where it is now?
   2. Move Highfive to another location?
2. Ship the logs elsewhere.
   1. Add application logic to handle this?
   2. Use external process to ship logs to somewhere?

I'm happy to talk more about the possibilities, if the answer to the first question is yes.

[nrc]

Making logs available is not a problem, they shouldn't ever contain anything that can't be found publicly on GH, and I don't see any danger from aggregation here.

We plan to move Highfive to some Rust project infra rather than my personal server. Though this has been stalled for quite a while now.

I expect we won't want to hand out creds to that server, however, shipping the logs elsewhere seems fine. I guess from a user perspective it would be better to pull logs, rather than push them on a regular basis. I have no idea how to engineer that.

When debugging Highfive, the most useful logs I have looked at have not come from Highfive, but from the GitHub web-hook. I'm not sure how or if we can make those available, but I expect that is what we really want.

[davidalber]

I expect we won't want to hand out creds to that server

I completely understand that. Is the concern about giving non-Mozilla access to the machine, the @rust-highfive OAuth token, or both? I could pretty quickly put Highfive on an EC2 instance that is only for Highfive, although that would give me "leverage" that might be a little strange. If I did that, though, there wouldn't be an immediate incentive to engineer a way to give access to logs, but for that to work, the @rust-highfive token would need to be on the machine. 😬

Incidentally, I've been harboring a secret desire to turn Highfive into a GitHub app. I was waiting for the right time to propose this, but it's relevant here because it would eliminate the need for the @rust-highfive token. I could go into more details (I actually started writing a doc for you at one point), but the short version:

• It would allow non-Rust projects to use Highfive (Highfive would need changes to support this).
• It would also create some other work to generalize, but still allow customization (like having an outgoing webhook from Highfive to enable things like pinging on IRC).

This idea is way outside the scope of Rust's needs, of course, so I won't be shocked if your response is not enthusiastic. 😀 I just had to mention the possibility.