Skip to content

Clarify generate_targets_metadata() #1033

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 24 additions & 5 deletions tuf/repository_lib.py
Original file line number Diff line number Diff line change
Expand Up @@ -1173,8 +1173,14 @@ def generate_targets_metadata(targets_directory, target_files, version,

target_files:
The target files tracked by 'targets.json'. 'target_files' is a
dictionary of target paths that are relative to the targets directory and
a fileinfo dict matching tuf.formats.LOOSE_FILEINFO_SCHEMA
dictionary mapping target paths (relative to the targets directory) to
a dict matching tuf.formats.LOOSE_FILEINFO_SCHEMA. LOOSE_FILEINFO_SCHEMA
can support multiple different value patterns:
1) an empty dictionary - for when fileinfo should be generated
2) a dictionary matching tuf.formats.CUSTOM_SCHEMA - for when fileinfo
should be generated, with the supplied custom metadata attached
3) a dictionary matching tuf.formats.FILEINFO_SCHEMA - for when full
fileinfo is provided in conjunction with use_existing_fileinfo

version:
The metadata version number. Clients use the version number to
Expand All @@ -1192,11 +1198,16 @@ def generate_targets_metadata(targets_directory, target_files, version,
write_consistent_targets:
Boolean that indicates whether file digests should be prepended to the
target files.
NOTE: when use_existing_fileinfo is True, this parameter will not cause
consistent targets to be written. It is assumed that the consistent
targets already exist in the targets directory of the repository.

use_existing_fileinfo:
Boolean that indicates whether to use the complete fileinfo, including
hashes, as already exists in the roledb (True) or whether to generate
hashes (False).
NOTE: if write_consistent_targets is True, the caller is responsible for
ensuring the consistent files exist in the targets directory.

storage_backend:
An object which implements
Expand Down Expand Up @@ -1240,9 +1251,12 @@ def generate_targets_metadata(targets_directory, target_files, version,
securesystemslib.formats.BOOLEAN_SCHEMA.check_match(write_consistent_targets)
securesystemslib.formats.BOOLEAN_SCHEMA.check_match(use_existing_fileinfo)

# If the caller requested consistent targets *and* supplied their own fileinfo
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# If the caller requested consistent targets *and* supplied their own fileinfo
# If the caller requested consistent targets *and* supplied their own fileinfo,

# consistent versions of the targets can not be created, because target
# files and their storage are not accessed when using existing fileinfo.
if write_consistent_targets and use_existing_fileinfo:
raise securesystemslib.exceptions.Error('Cannot support writing consistent'
' targets and using existing fileinfo.')
logger.info('Consistent targets will not be created because targets '
'metadata is being generated from existing fileinfo.')

if delegations is not None:
tuf.formats.DELEGATIONS_SCHEMA.check_match(delegations)
Expand All @@ -1253,20 +1267,25 @@ def generate_targets_metadata(targets_directory, target_files, version,
filedict = {}

if use_existing_fileinfo:
# Use the provided fileinfo dicts, conforming to FILEINFO_SCHEMA, rather than
# generating fileinfo
for target, fileinfo in six.iteritems(target_files):

# Ensure all fileinfo entries in target_files have a non-empty hashes dict
if not fileinfo.get('hashes', None):
raise securesystemslib.exceptions.Error('use_existing_hashes option set'
' but no hashes exist in roledb for ' + repr(target))

if fileinfo.get('length', -1) < 0:
# and a non-empty, non-zero, length
if fileinfo.get('length', -1) < 1:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it doesn't make sense to have zero length targets files, but maybe this should be restored to the prior check of: if len < 0 ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it doesn't make sense to have zero length targets files, but maybe this should be restored to the prior check of: if len < 0 ?

@joshuagl, the classical __init__.py file is a good example for a target file with zero length.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, thanks for pointing out that case @lukpueh. In which case I should restore this to the prior if len < 0. Will force push a fix later today.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I already restored the behaviour for checking len < 0 in #1034, which contains an updated version of this change. Apologies for the confusion.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to apologize. I have no business commenting on closed PRs. :D

raise securesystemslib.exceptions.Error('use_existing_hashes option set'
' but fileinfo\'s length is not set')

filedict[target] = fileinfo

else:
# Generate the fileinfo dicts by accessing the target files on storage.
# Default to accessing files on local storage.
if storage_backend is None:
storage_backend = securesystemslib.storage.FilesystemBackend()

Expand Down