Skip to content

incr.comp.: Create Test Case for Incr. Comp. Hash for enums #36674

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
michaelwoerister opened this issue Sep 23, 2016 · 4 comments
Closed

incr.comp.: Create Test Case for Incr. Comp. Hash for enums #36674

michaelwoerister opened this issue Sep 23, 2016 · 4 comments
Labels
A-incr-comp Area: Incremental compilation A-testsuite Area: The testsuite used to check the correctness of rustc E-mentor Call for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion.

Comments

@michaelwoerister
Copy link
Member

michaelwoerister commented Sep 23, 2016

This issue is part of a broader effort to implement comprehensive regression testing for incremental compilation. For the tracking issue go to #36350.

Background

For incremental compilation we need to determine if a given HIR node has changed in between two versions of a program. This is implemented in the calculate_svh module. We compute a hash value for each HIR node that corresponds to a node in the dependency graph and then compare those hash values. We call this hash value the Incremental Compilation Hash (ICH) of the HIR node. It is supposed to be sensitive to any change that might make a semantic difference to the thing being hashed.

Testing Methodology

The auto-tests in the src/test/incremental directory all follow the same pattern:

  • Each source file defines one test case
  • The source file is compiled multiple times with incremental compilation turned on, each time with a different --cfg flag, allowing each version to differ via conditional compilation. Each of these versions we call a revision.
  • During each revision, the test runner will make sure that some assertions about the dependency graph of the program hold.
  • These assertions are specified in the source code of the test file as attributes attached to the items being tested (e.g. #[rustc_clean]/#[rustc_dirty]).

The ICH-specific tests use this framework by adhering to the following pattern:

  • There are two versions of the definition under test, one marked with #[cfg(cfail1)] and the second marked with #[cfg(not(cfail1))]. As a consequence, the first revision will be compiled with the first version of the definition, and all other revisions will be compiled with the second version. The two versions are supposed to differ in exactly one aspect (e.g. the visibility of a field is different, or the return of a function has changed).
  • The second definition has a #[rustc_dirty(label="Hir", cfg="cfail2")] attribute attached. This attribute makes the test runner check that a change of the Hir dependency node of the definition has been detected between revisions cfail1 and cfail2. This will effectively test that a different ICH value has been computed for the two versions of the definition.
  • The second definition also has a #[rustc_clean(label="Hir", cfg="cfail3")] attribute. This makes sure that the Hir dependency node (and thus the ICH) of the definition has not changed between revisions cfail2 and cfail3. That's what we expect, because both revisions use the same version of the definition.
  • For definitions that are exported from the crate, we also want to check the ICH of the corresponding metadata. This is tested using the #[rustc_metadata_clean]/#[rustc_metadata_dirty] attributes and works analogous to the Hir case: We add #[rustc_metadata_dirty(cfg="cfail2")] to the second definition to make sure that the ICH of the exported metadata is not the same for the different versions of the definition, and we add #[rustc_metadata_dirty(cfg="cfail3")] to make sure that the ICH is the same for the two identical versions of the definition.

Why are the revisions called "cfail"? That's because of reasons internal to how
the test-runner works. Prefixing a revision with "cfail" tells the test runner to treat the test like a "compile-file" test, that is: compile the test case but don't actually run it (which would be the case for an "rpass" revision). For the ICH tests we need to compile "rlibs", so that we can test metadata ICHs. As a consequence we cannot declare them "rpass". In an additional directive (// must-compile-successfully), we tell the test runner that we actually expect the file to not produce any errors.

Each test case must contain the following test-runner directives and crate attributes at the top:

// must-compile-successfully
// revisions: cfail1 cfail2 cfail3
// compile-flags: -Z query-dep-graph

#![feature(rustc_attrs)]  // <-- let's us use #[rustc_dirty], etc.
#![crate_type="rlib"]     // <-- makes sure that we export definitions

See src/test/incremental/hashes/struct_defs.rs for a full example of such a ICH regression test.

Enum Specific ICH Testing

Each of the following things should be tested with its own definition pair:

  • Change enum visibility
  • Change name of a c-style variant
  • Change name of a tuple-style variant
  • Change name of a struct-style variant
  • Change the value of a c-style variant
  • Add a c-style variant
  • Remove a c-style variant
  • Add a tuple-style variant
  • Remove a tuple-style variant
  • Add a struct-style variant
  • Remove a struct-style variant
  • Change the type of a field in a tuple-style variant
  • Change the type of a field in a struct-style variant
  • Change the name of a field in a struct-style variant
  • Change the visibility of a field in a tuple-style variant N/A for enums
  • Change the visibility of a field in a struct-style variant N/A for enums
  • Change order of fields in a tuple-style variant
  • Change order of fields in a struct-style variant
  • Add a field to a tuple-style variant
  • Add a field to a struct-style variant
  • Add #[must_use] to the enum
  • Add #[repr(C)] to the enum
  • Change the name of a type parameter
  • Add a type parameter
  • Change the name of a lifetime parameter
  • Add a lifetime parameter
  • Add a lifetime bound to a lifetime parameter
  • Add a lifetime bound to a type parameter
  • Add a trait bound to a type parameter
  • Add a lifetime bound to a lifetime parameter in where clause
  • Add a lifetime bound to a type parameter in where clause
  • Add a trait bound to a type parameter in where clause
  • In an enum with two variants, swap usage of type parameters
  • In an enum with two variants, swap usage of lifetime parameters

EDIT: Some more cases

  • Change field type in tuple-style variant indirectly by modifying a use statement
  • Change field type in record-style variant indirectly by modifying a use statement
  • Change trait bound of type parameter indirectly by modifying a use statement
  • Change trait bound of type parameter in where clause indirectly by modifying a use statement
@michaelwoerister michaelwoerister added E-mentor Call for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion. A-incr-comp Area: Incremental compilation labels Sep 23, 2016
@michaelwoerister
Copy link
Member Author

cc @nikomatsakis

@michaelwoerister michaelwoerister added the A-testsuite Area: The testsuite used to check the correctness of rustc label Sep 23, 2016
@eulerdisk
Copy link
Contributor

eulerdisk commented Sep 29, 2016

I can work on this (and maybe even on #36681, #36812.)
For "Change the visibility of a field in a [tuple|struct]-style variant", do you mean using the "pub(restricted)" RFC ?

@michaelwoerister
Copy link
Member Author

I can work on this

Cool! I'd start out with taking test/incremental/hashes/struct_defs.rs as a template. Feel free to ask here if there are any questions.

For "Change the visibility of a field in a [tuple|struct]-style variant", do you mean using the "pub(restricted)" RFC ?

No, that's actually an oversight on my part. Good catch! I'll remove it from the list.

@michaelwoerister
Copy link
Member Author

FYI: A good way to run the incremental tests is: make check-stage1-incremental -j4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-incr-comp Area: Incremental compilation A-testsuite Area: The testsuite used to check the correctness of rustc E-mentor Call for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion.
Projects
None yet
Development

No branches or pull requests

2 participants