Skip to content
This repository was archived by the owner on Sep 11, 2023. It is now read-only.
This repository was archived by the owner on Sep 11, 2023. It is now read-only.

Turn off temporal interpolation of numerical weather predictions (NWPs) #135

Closed
@JackKelly

Description

@JackKelly

Detailed Description

The raw numerical weather predictions (NWPs) are hourly. But nowcasting_dataset interpolates the hourly data to produce 5-minutely NWPs in the pre-prepared batches.

Interpolating the NWPs massively blows up the filesize of the batches and - in theory (!) - this interpolation shouldn't be necessary for 'fully attentional' ML models like Perceiver, which don't need the inputs to all be aligned in time.

Context

"fully attentional" models view their inputs as a set. To tell the model about the position of each row of input data, we manually encode the position in time and space. As such, we should be able to feed in hourly NWPs and 5-minutely satellite images and PV data, and the model should learn how these things relate to each other in time and space.

Implementation

The implementation should be quite simple, because the code already handles inputs at different sample periods after the GSP pull request. We might need to make sure the NetCDFDataset correctly interprets the new batches.

Metadata

Metadata

Assignees

Labels

dataNew data source or feature; or modification of existing data sourceenhancementNew feature or request

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions