Reconsider current BERT models

[lefnire]

I'm using the following (code). See SOTA model leaderboards for ideas on what to try next.

• Embeddings: sentence-transformers/roberta-base-nli-stsb-mean-tokens. This is actually really good, and the recommended model for cosine-similarity (since that's how it's trained). But we'll likely wanna tone it down computationally, see distillation.
• Question-answering: allenai/longformer-large-4096-finetuned-triviaqa. Really solid results (better than any I've played with), but a computational non-starter; I need to get off this ASAP, and find something that performs as well but less compute.
• Summarization: facebook/bart-large-cnn. Works great actually, but maybe someone knows of something better?
• Emotions: mrm8488/t5-base-finetuned-emotion. Absolutely horrendous; none of my emotions are correct to my entries. But the only model I've found which has emotions; the others are just positive/negative.

Considerations:

• Should we train/fine-tune models on entries? What would be the labels? Is that why I'm getting such poor results? Are there models which perform better off-the-shelf (no fine-tuning) than others? We'd need to consider Privacy Poly re: model-training on entries.
• For assessment, I've found model performance in their whitepapers to be irrelevant compared to subjective performance on mine / my wife's entries. The models I chose above are after trying out tons of models, so we'd want some way to compare these subjectively?
• I stopped trying new models after transformers<3.1.0, so their release notes since then would be useful to see what's new. 3.1.0 (pegasus, keep eye for f16 version); 3.2.0; 3.3.0; 3.3.1 (facebook/pag for QA?)
• We prefer fast over accurate, but not by too much. We want models that can crunch as much as possible (eg Longformer's 4096 tokens is great), since it captures more together vs separate chunks. So (1) compute (2) accuracy (3) batch efficiency. But course, something that balances all 3 well (in that order)