Experimental API search exposed through query.

[isoos]

#1179

[isoos]

Deploying to staging...

[isoos]

Looks like the indexing takes a huge toll on the service, and increases the number of 502 responses.

[isoos]

I've profiled it a bit, and during indexing most of the time is spent with the maintenance of the inverted index, more specifically in the TokenIndex.remove. The number of indexed ID grew 10x, while the number of indexed tokens grew ~3x compared to the sum of the rest we have now.

As the (mostly-updating) indexing is on the CPU-blocking path of the search, I'll seek two path to improve the situation:

• slowing down the updating path (most of the index is built from the snapshot in a fast way)
• detecting whether an update is really needed

[isoos]

@jakobr-google: PTAL. I've deployed this version in staging, you can check it via "PackageAnalyzer" vs "PackageAnalyzer !!api". HTTP 502 errors are now gone, memory use is still under 1G.

[jakobr-google]

LGTM