Skip to content

Add retentionTime information #23

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
milsi96 opened this issue Jan 25, 2023 · 1 comment
Open

Add retentionTime information #23

milsi96 opened this issue Jan 25, 2023 · 1 comment

Comments

@milsi96
Copy link

milsi96 commented Jan 25, 2023

At the moment, in the DP descriptor only the startDate of the dataset is available but not all dataset will have an infinite retention time: it's necessary to include a field that represents the time window data will be kept available in the dataset. The proposed solution is to add a RetentionTime field at StartDate level.

milsi96 pushed a commit to milsi96/Data-Product-Specification that referenced this issue Jan 25, 2023
milsi96 pushed a commit to milsi96/Data-Product-Specification that referenced this issue Jan 25, 2023
@tritemio
Copy link

tritemio commented Feb 2, 2023

I agree with the proposal. Very often, data exposed by an output port has a fixed retention time that ranges from days (in case on output ports of type "events") to years (for output port of type "Files").

Currently the specifications has only the field startDate to hold this information. This could be a date or a time interval.

It would be beneficial to leave startData only for the timestamp of the oldest data initially published on the output port, and add retentionTime for specifying the time interval of the retention policy (if any).

With these two fields is possible to express also transient states. For example, an outputport with retention time of 1 year can be first published with 1 month of data. It will have a startDate fixed to 1 month earlier than the date of publication, and retentionTime = 1Y. During the first year the output port is "accumulating data" and the startDate indicates the oldest data published. After the first year the retention kicks in and all without changing the descriptor.

CC: @agile-lab @erond

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants