`signal:*` queries return a 504 Gateway Error

[dshemetov]

These queries should return the data for all the signals in each data source, but instead they spin until a 504 Gateway Error

https://api.covidcast.cmu.edu/epidata/covidcast/?signal=usa-facts:*&time=day:20220104&geo_type=state&geo_values=ca
https://api.covidcast.cmu.edu/epidata/covidcast/?signal=jhu-csse:*&time=day:20220104&geo_type=state&geo_values=ca
https://api.covidcast.cmu.edu/epidata/covidcast/?signal=safegraph:*&time=day:20220104&geo_type=state&geo_values=ca

You can confirm that at least one signal has data for each of these with

https://api.covidcast.cmu.edu/epidata/covidcast/?signal=usa-facts:confirmed_cumulative_num&time=day:20220104&geo_type=state&geo_values=ca
https://api.covidcast.cmu.edu/epidata/covidcast/?signal=jhu-csse:confirmed_cumulative_num&time=day:20220104&geo_type=state&geo_values=ca
https://api.covidcast.cmu.edu/epidata/covidcast/?signal=safegraph:bars_visit_num&time=day:20220104&geo_type=state&geo_values=ca

Possibly a bug with the signal:* handling?

[dshemetov]

It doesn't seem like we integration tested this. The SourceSignalPair parsing are tested in test_params.py and correctly return SourceSignalPair(signal='usa-facts', signal=True). It may be an issue in forming the database query.

[krivard]

@LeonLu2 is picking this up. Leon -- I just did some checks and it seems the recent v4 improvements have resolved the 504 problem, but we indeed do not have any integration tests for signal wildcard.

The task then is to add a test_signal_wildcard integration test similar to the one we have for test_location_wildcard.

You'll probably need to define a new placeholder set -- it will need one source and multiple signals, but only one geo. Here's the example used for test_location_wildcard; it has multiple geos, but only one source & one signal. Source for _make_placeholder_row is here.

[krivard]

Okay weird, it works for source:signal notation but not for source=&signal= notation:

$ curl -sL "https://api.covidcast.cmu.edu/epidata/api.php?source=covidcast&data_source=safegraph&signals=*&time=day:20220104&geo_type=state&geo_values=ca"
{ "result": -2, "message": "no results" }krivard@rinkitink:~/projects/covid/driver/repos/delphi/delphi-epidata$ 
$ curl -sL "https://api.covidcast.cmu.edu/epidata/api.php?source=covidcast&signal=safegraph:*&time=day:20220104&geo_type=state&geo_values=ca"
{ "epidata": [{"geo_value":"ca","signal":"bars_visit_num","time_value":20220104,"direction":null,"issue":20220224,"lag":51,"missing_value":0,"missing_stderr":5,"missing_sample_size":5,"value":181.0,"stderr":null,"sample_size":null},{"geo_value":"ca","signal":"bars_visit_prop","time_value":20220104,"direction":null,"issue":20220224,"lag":51,"missing_value":0,"missing_stderr":5,"missing_sample_size":5,"value":36.8597152,"stderr":null,"sample_size":null},{"geo_value":"ca","signal":"restaurants_visit_num","time_value":20220104,"direction":null,"issue":20220224,"lag":51,"missing_value":0,"missing_stderr":5,"missing_sample_size":5,"value":44746.1129146,"stderr":null,"sample_size":null},{"geo_value":"ca","signal":"restaurants_visit_prop","time_value":20220104,"direction":null,"issue":20220224,"lag":51,"missing_value":0,"missing_stderr":5,"missing_sample_size":5,"value":130.5966887,"stderr":null,"sample_size":null}], "result": 1, "message": "success" }

A place to start:

delphi-epidata/src/server/endpoints/covidcast.py

Line 48 in ccad94a

def parse_source_signal_pairs() -> List[SourceSignalPair]: