Investigate elimination of the -raise pass

[lattner]

Bugzilla Link	1072
Resolution	FIXED
Resolved on	Feb 22, 2010 12:47
Version	trunk
OS	MacOS X

Extended Description

The -raise pass was mainly around to eliminate casts from int -> uint etc through some tricky
transformations. With the recent signless work, this pass should be mostly useless. Eliminating it is good
as it reduces compile time (one fewer pass), eliminates a bunch of ugly code (the raise pass is very nasty),
and potentially eliminating the LLVMTransforms library (ExprTypeConvert.cpp, LevelRaise.cpp, and
TransformInternals.cpp).

We should do performance analysis (e.g. spec2k and 2k6) to see whether this is completely dead, and if
not, reimplement the useful pieces in other passes.

-Chris

[lattner]

When this happens, we can eliminate Type::canLosslesslyBitCastTo and probably some other stuff only
used by -raise.

-Chris

[llvmbot]

Is there anything preventing this from being eliminated now?

Can we use the stats to determine how much it fires (if at all)?

What pieces does it implement that might still be useful to retain?

[lattner]

Nothing prevents it. We just need someone to do an llvm-test run with an without it to see if there are
any perf differences.

-Chris

[lattner]

or even just run SPEC2K + SPEC2K6.

-Chris

[llvmbot]

I will set up overnight performance test run.

[llvmbot]

I want this gone. :)

[llvmbot]

External Nightly Test Results With -raise

[llvmbot]

External Nightly Test Results Without -raise

[llvmbot]

Performance Data Spreadsheet
This attachment contains a spreadsheet in Excel format that contains the raw
data and a comparison of the External tests with and without the -raise pass.

[llvmbot]

Summary Of Performance Results (See the Spreadsheet)

Four SPEC test suites were compared: CINT2000, CFP2000, CINT2006, and CFP2006.

In general it looks positive to remove the -raise pass. The overall results
(sum of all tests) are as follows:

                        +--Compile-+   +----------Run Times--------+

ITEM GCCAS Bytecode LLC JIT GCC CBE LLC JIT
Delta 151.05 818298 -9.07 16.2 14.66 20.81 46.40 57.15
Percent 6.49% 1.95% -1.09% 3.09% 2.18% 3.14% 7.12% 4.04%

The "Delta" values shown are the difference in aggregate time between "with
-raise" and "without -raise" runs across all test cases. Negative values mean
the "without -raise" value was larger. Positive values mean the "without -raise"
value was smaller. Because these values represent the subtraction of the
"without -raise" from the "with -raise" data sets, positive values favor removal
of -raise while negative values favor keeping -raise.

The "Percent" values are computed by dividing the "Delta" value by the "with
-raise" sum. This gives an approximation of the magnitude of change introduced
by removing the -raise pass.

To interpret this correctly, one would say: "On average, the -raise pass
increases bytecode volumes by nearly 2% and LLC runtimes by over 7%".

The only negative impact of removing -raise seems to be a 1.09% increase in the
LLC compilation time. This is rather surprising given that there is less work
done by llc without -raise.

Note that these results account for only one run of the nightly tests.
Additional samples would be welcome.

[llvmbot]

Performance Data Spreadsheet
This updates some formatting and gets rid of columns that weren't run in both
tests and rows that indicate test failures.

[llvmbot]

Chris,

Please let me know if this is sufficient evidence to support removal of the
-raise pass.

Reid

[lattner]

Yes, it looks like we are close to it. Things worth investigating:

• What does -raise do on 401.bzip2 that shrinks the bc file 18K? Once understood, we can reimplement
  this in some other pass.

• I have little confidence in the stabilities of these numbers, they seem very noisy. Can you see if the
  254.gap/LLC, 254.gap/CBE, 255.vortex/CBE, and 256.bzip2/cbe slowdowns reproduce? If so, we should
  investigate what is happening there too.

-Chris

[llvmbot]

• What does -raise do on 401.bzip2 that shrinks the bc file 18K? Once
  understood, we can reimplement this in some other pass.

I ran 401.bzip2 using "head" from today and compared it against the "without
-raise" run I did last night. The bytecode sizes are identical. Consequently, I
don't think I was comparing apples with apples. I used the results of my nightly
run from a couple days ago for the "with -raise" data but my "without -raise"
also included the shift patch, which may have affected optimization (positively).

So, guess I get to do this again.

Devang, did you run this comparison on Darwin?

• I have little confidence in the stabilities of these numbers, they seem very
  noisy.

I eliminated as much interference workload as I could by shutting down email,
xchat, etc. and just let it run. I think the issue here is the difference in the
builds. I'll run the comparison again and report new numbers. Unfortunately,
it'll have to wait until this evening for "quiet machine".

Can you see if the 254.gap/LLC, 254.gap/CBE, 255.vortex/CBE, and
256.bzip2/cbe slowdowns reproduce? If so, we should investigate what is
happening there too.

I'll defer this until after I get new numbers. If they still look problematic,
I'll investigate then.

[lattner]

Makes sense. A lot has changed in a few days. I suspect the 'noise' is really measuring these changes :)

-Chris