Add short cut for recomputed tv

[zasdfgbnm]

Fixes #2094

[zasdfgbnm]

Unrelated but I think it is a good idea to have it

[csarofeen]

LGTM

[naoyam]

So, is this an alternative fix for the problem you previously proposed the "mirror" approach? How does having dummy outputs solve the problem?

[zasdfgbnm]

So, is this an alternative fix for the problem you previously proposed the "mirror" approach? How does having dummy outputs solve the problem?

Yes, it is. Let's say we had

T0[I]
T1[b b I] = broadcast(T0)
T2[b b r] = reduction(T1)
T3[b b b] = broadcast(T2)
T4[b, b, I] = T1 + T3
T5[b, b, r] = reduction(T4)

After projection, it becomes

T0[I]
T1[b b I] = broadcast(T0)
T2[b b r] = reduction(T1)
T3[b b b] = broadcast(T2)
T6[b b I] = broadcast(T0)
T4[b, b, I] = T6 + T3
T5[b, b, r] = reduction(T4)

Then neither the propagation path T2->T3->T4->T5 nor T2->T1->T0->T6->T4->T5 works because they both have missing root domain. But adding T7 = T1 + T6 creates a new propagation path T2->T1->T7->T6->T4->T5 which has all root domain information.

(edited)

[naoyam]

Oh, I see. Interesting approach. Please have your explanation as a code comment.

I wonder how robust and generic this approach would be, but seems like good enough.

[csarofeen]

If it was directly on iter domains instead of tensor views it would be very robust. The fact that it's on tensor views means that we can only build relationships across tensorviews that can go through a pointwise operation. Since this is replicated tensors exactly, it should be sufficient for all our use cases. Though same day in the future all replays should probably be IterDomain based, not TensorView based.