[float8] add _auto_filter_for_recipe for float8 training

[danielvegamyhre]

Fixes #1207

Problem

• float8 rowwise + vanilla TP in torchtitan had flat perf with respect to bfloat.
• RCA In float8 rowwise vanilla TP low throughput #1207 found attention.wk and attention.wv layers were so small that float8 rowwise conversion resulted in approx ~40% slowdown for those layers, thus the perf benefits from fp8 rowwise conversion on larger linears were nullified.
• This is because the default filter_fqns for float8 model conversion are fine for the fp8 tensorwise recipe, but bad for the float8 rowwise recipe.

Solution

This has been a footgun for various users as well (including Poolside), so I created an "auto filter" (pytorch/ao#2410) which automatically filters Linears for a given float8 recipe, by checking for the following criteria:

1. dims not divisible by 16 (hardware requirement for float8)
2. dim sizes below thresholds that will result in worse perf for that given recipe, using simple heuristics based on the linked recipe perf tables above.
3. fqn matches one of the user defined filter_fqns

It prevents users from hitting this common footgun, while also preserving the flexibility to define their model-specific fqns.

Results

Benchmarks show a ~10% TPS improvement for TP and ~15% TPS improvement for async TP (over bf16 TP baseline).

Llama3 70b on 256 H100s with FSDP=32, TP=8, torch.compile, full AC, local batch size 16:

• bfloat16 baseline = ~597TPS
• fp8 rowwise WITH attention.wk, attention.wv converted = ~600 TPS
• fp8 rowwise WITHOUT attention.wk, attention.wv converted = ~660 TPS
• fp8 rowwise + async TP WITH attention.wk, attention.wv converted = ~625 TPS
• fp8 rowwise + async TP WITHOUT attention.wk, attention.wv converted = ~695 TPS