IterDomain-centric graph analysis

[jacobhinkle]

This is an attempt at creating an IterDomain graph using manual rules for various Expr types. This provides another way to visualize or analyze a Fusion. For example, a simple Fusion like this

  auto tv0 = makeConcreteTensor({2, 3});
  fusion.addInput(tv0);
  auto tv1 = makeConcreteTensor({2});
  fusion.addInput(tv1);

  auto tv2 = broadcast(tv1, {false, true});
  auto tv3 = mul(tv0, tv2);

  fusion.addOutput(tv3);

can be represented as a TV-centric graph like so:

or as an ID-centric graph like so:

where the ID classes above represent the following sets of IDs:

Inputs:
  T0_g[ iS0{2}, iS1{3} ], float
  T1_g[ iS2{2} ], float
Outputs:
  T3_g[ iS5{2}, iS6{3} ], float

%kernel_math {
T2_l[ iS3{2}, bS4{1} ]
   = broadcast( T1_g[ iS2{2} ] )
T3_g[ iS5{2}, iS6{3} ]
   = T0_g[ iS0{2}, iS1{3} ]
   * T2_l[ iS3{2}, bS4{1} ];
}

Broadcast op: T2_l[ iS3{2}, bS4{1} ]
   = broadcast( T1_g[ iS2{2} ] )

Equivalence classes of IterDomains:
  c2: bS4{1}, 
  c4: iS5{2}, iS3{2}, iS0{2}, iS2{2}, 
  c5: iS6{3}, iS1{3}, 
Equivalence classes of extents:
  e0: 1, 
  e1: 2, 2, 
  e3: 3, 

Clearly this lets us derive some equality constraints on extents, which we also track. So far we do not perform any kind of term rewriting on Vals, but we could do so. Also, so far I do not have support for ViewOp, or many of the other op types like scatter and gather; unsupported ops are skipped with a warning so in their presence there will be more apparent ID classes than there should be. A challenge in this PR's approach is that for example ViewOp does not carry direct information about which input domains are transformed, or even what the original int vector arguments were, which we could use to reconstruct.

What can we do with ID graphs

ID graphs give us way to pattern match certain cases that we'll need to handle. For example, a Gram matrix computation looks like the following:

  // [n, d]
  auto tv0 = makeConcreteTensor({5, 7});
  fusion.addInput(tv0);

  // [1, n, d]
  auto tv1 = broadcast(tv0, {true, false, false});
  // [n, 1, d]
  auto tv2 = broadcast(tv0, {false, true, false});

  // [n, n, d]
  auto tv3 = mul(tv1, tv2);

  // [n, n]
  auto tv4 = sum(tv3, {2});

  fusion.addOutput(tv4);

We see that two separate output IDs use the same ID class c7. This is a problem and indicates we need to recompute class c7 so that it appears as two classes that can be separately parallelized (note that recomputing is an operation we don't yet support).

We can also infer the ordering of ID classes and persist those back using reorder(). We can split and merge domains at the ID class level then persist those as well. Generally, this approach might allow us to transform nodes in our Fusion based on groups of ID classes, instead of the current reference tensor approach.

[jacobhinkle]

ComputeAt with ID-centric graphs

One thing to notice about the above graphs are that they represent only EXACT mappings (in the language of csarofeen#2316). That is to say in an ID graph as constructed above, each equivalence class (and any other classes that are related to it by an edge, not counting the edges to and from input/output TVs) could be parallelized in the same manner and the computeAt for each tensor could be above any ID in the class. However, this is not always possible or practical. NEED EXAMPLE HERE