specially handle as_of==$TODAY

[melange396]

We have seen many entries in our logs of people requesting covidcast data "as_of" the current date. Presumably they think that this ensures that they get the most recent information available -- which it does, but it comes at a cost because as_of queries are more complicated (and thus are more computationally expensive and take longer to return). Such users could get the exact same results by omitting the as_of altogether.

To save the user some waiting time and to save us some CPU cycles, we can catch the case when the as_of argument is set to "today", and treat it as though the argument was not included at all.

[krivard]

We told people to do this as a data quality workaround when latest was out of date with full. Before we add special handling for it, we need to make sure that

• definitely: the data quality workaround is no longer needed
• ideally: we have a data validation system checking for recurrences of data quality problems that require the workaround

[brookslogan]

Some alternative user reasoning: When I have used as_of today, it is probably from trying to use the same code for pseudoprospective forecasting and true prospective forecasting; I think that's sort of valid. Though now I'll probably try to get everything in an epi_archive, especially when we eventually implement functionality to cache+update epi_archives with new issue data.

Were there also some data sources that can actually have issue data beyond the current date somehow? @krivard I though you mentioned this was possible sometime, not sure if it's what you described above.

[krivard]

It is technically possible to insert future-dated data through the patching mechanism (a batch issue upload with issue>today) but we'd expect that to fail earlier automated and manual checks first. (cc @neul3)

The problem from above is when new data is added to full, but latest remains on an older issue. When that happens, latest queries return stale data, and the only way to get up-to-date data is to use as-of-today.