Code Intel - LSIF Indexing CNCF effort tracking

[macraig]

Our goal is to make our best effort with the tools we have today to index as many repos as possible to give CNCF a great code intel experience. This issue will serve as a central point to track our efforts for this project.

• Index all Go repos by 2020-10-05

• Index all C++ repos by 2020-10-05

• Index all Typescript repos by 2020-10-05

Related issues:
#14318
#14343

List of repos and their status: https://docs.google.com/spreadsheets/d/125x9ayF7irnQGnWSwRg5nFoU23fsIt37bIeBmpjYNBY/edit#gid=0

[aidaeology]

I am going to attempt to manually index all the remaining GO projects today, will update the spreadsheet with results.

[aidaeology]

Adding more details for consideration here:

lsif-go (@eric)

Eric is point on lsif-go and is also working on auto-indexing, which was planned as part of 3.21 milestone anyways but its unlikely auto-indexing will be completed by 10/5 - that said I will work on manually indexing the remaining Go projects today.

lsif-clang (@gbrik)

Garo is on point on lsif-clang is working on manually indexing as many of the C-language repos they can by end of day today. They have planned PTO tomorrow (Friday) and all of next week, therefore unless were able to index withwe can get someone to help index remaining C projects they may not get completed by 10/5.

lsif-java / lsif-node (@Strum355)

Noah is point on lsif-java and I think he volunteered for JS/TS ;) Noah is out on planned PTO tomorrow (Friday), which also may means there's a good chance that this may not be completed by 10/5.

The team consensus is that the best UX will be with auto-indexing support so that users are always getting the latest and most accurate responses. @ code-intel team please update this issue with your progress so we can see if we are on track for 10/5th and any details on issues you encounter (severity, time needed to fix, etc).

[christinaforney]

Hi all - catching up on the CNCF efforts. It looks like this team is making a big push to help this go live! 🎉 I'm wondering about the decision to spend time manually uploading LSIF data. Just to sanity check - the iteration plan included having auto-indexing complete by next Friday 10/9, and we appear to be at ~50-60% coverage already.

@sqs and @poojaj-tech - it sounds like the goal is to go live with the CNCF page on Monday. Is it acceptable to have 60% LSIF coverage, with all remaining Go and TS repositories auto-indexing by the end of next week? It feels like the right use of the team's time is to do this properly (so that our manual uploads don't quickly become stale), and it's awesome that this is already in-progress and planned to be completed next week.

[garobrik]

update: I indexed all of the primarily C++ repos manually. documented the steps here sourcegraph/lsif-clang#12 if anyone wants to add them to the auto-indexer while I'm gone, but I can do it when I get back.

There is a primarily Go repo with some C code that I quickly gave up on indexing because it uses some arcane Docker based build system to target WASM and i

[aidaeology]

Update: I manually indexed the remaining Go and TS/JS projects, these are marked with the comment "Completed manually".

[aidaeology]

For the case where we have to go live on 10/5 or 10/6th - we can setup a cron job to re-index the projects until auto-indexing is completed as a stop gap to ensure code freshness.

[macraig]

Update: 40% of the repos (Go and TypeScript) were manually indexed and another 20% were automatically indexed. The team will focus on getting the autoindexer working by the end of next week.

[efritz]

Follow up with https://github.com/sourcegraph/sourcegraph/issues/14343.