Revisit internal caching arrangements

[mp911de]

We create quite some objects and use heavy concurrent code in code paths that are primarily for read-only operations. We should explore whether we can optimize for less GC pressure.