-
Notifications
You must be signed in to change notification settings - Fork 280
Description
This is part of #1579, we want to test the Updater in various key/signature change scenarios in a setup where:
- tests are readable / understandable
- a new test scenario adds as little code as possible
- adding new scenarios should be easy and the added test runtime should be negligible
I have been looking at existing tests (test_key_revocation_integration.py, test_root_versioning_integration.py and test_updater_root_rotation_integration.py) and I believe they all revolve around a few variables that we should be able to turn into a table testing setup where no additional code is needed for most new tests. The most interesting and complex case is root rotations (because root delegates not just to other roles but to "future root role") so I'll start there.
All root key/signature change cases can be defined by listing N sequential root states, each with
- key list
- threshold
- signature list
- expected result
In practice a table like this should do it:
@dataclass
class RootVersion:
keys: list[int]
threshold: int
signatures: list[int]
result: Optional[Exception] = None
root_cases = {
"key rotation: 1-of-1 scenario" : [
RootVersion([1], 1, [1]),
RootVersion([2], 1, [2,1]),
RootVersion([2], 1, [2]),
],
"failing threshold decrease: 2-of-2 becomes 1-of-2" : [
RootVersion([1,2], 2, [1,2]),
RootVersion([1,2], 1, [2], UnsignedMetadataError),
]
}
The "key rotation" test case written out would be
- root v1 contains root key1 with threshold of 1, signed by key1
- root v2 contains root key2 with threshold of 1, signed by key1 and key2
- root v2 contains root key2 with threshold of 1, signed by key2
There is no exception listed so Updater is expected to load all the versions correctly
The "failing" test case written out would be
- root v1 contains root key1 and key2 with threshold of 2, signed by key1 and key2
- root v2 contains root key1 and key2 with threshold of 1, signed by key2
Updater should fail to update v2 as it's not signed by both keys as v1 requires
This is not super readable (using numbers to represent keys is admittedly a bit confusing) but similar 3-5 lines can describe quite tricky situations like threshold increases while rotating keys etc