Getting 500 response for valid url

[idod]

[benoitc]

can you provide more informations: logs, trace or anything that could help to reproduce the issue?

[idod]

Sorry,
I got an error on hackney and not on curl.
No I get it also on curl :)
No issue.

[idod]

So there is an issue:
when trying to get the url of "https://assets.superbalistcdn.co.za/500x720/filters:quality(75):format(jpg)/spree/product/s/p/spree2018september12picslgzh0650.jpg" with hacnkey I get an error, although it is working fine on curl / browser:

when getting with hackney:
hackney:get(<<"https://assets.superbalistcdn.co.za/500x720/filters:quality(75):format(jpg)/spree/product/s/p/spree2018september12picslgzh0650.jpg">>).
{ok,500,
[{<<"Date">>,<<"Tue, 08 Jan 2019 14:11:53 GMT">>},
{<<"Content-Type">>,<<"text/html; charset=UTF-8">>},
{<<"Transfer-Encoding">>,<<"chunked">>},
{<<"Connection">>,<<"keep-alive">>},
{<<"Set-Cookie">>,
<<"__cfduid=d8081ddfb7dd7ce93d45546477584bcc11546956713; expires=Wed, 08-Jan-20"...>>},
{<<"X-Varnish">>,<<"15930532">>},
{<<"Via">>,<<"1.1 varnish-v4, 1.1 google">>},
{<<"Alt-Svc">>,<<"clear">>},
{<<"CF-Cache-Status">>,<<"MISS">>},
{<<"X-Content-Type-Options">>,<<"nosniff">>},
{<<"Expect-CT">>,
<<"max-age=604800, report-uri="https://report-uri.cloud"...>>},
{<<"Server">>,<<"cloudflare">>},
{<<"CF-RAY">>,<<"495f41806f2029dd-SEA">>}],
#Ref<0.2804061591.1561329665.145241>}

When curl:
curl -v "https://assets.superbalistcdn.co.za/500x720/filters:quality(75):format(jpg)/spree/product/s/p/spree2018september12picslgzh0650.jpg" -O /dev/null
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 104.20.2.217...

• TCP_NODELAY set
• Connected to assets.superbalistcdn.co.za (104.20.2.217) port 443 (#0)
• ALPN, offering h2
• ALPN, offering http/1.1
• Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@strength
• successfully set certificate verify locations:
• CAfile: /etc/ssl/cert.pem
  CApath: none
• TLSv1.2 (OUT), TLS handshake, Client hello (1):
  } [233 bytes data]
• TLSv1.2 (IN), TLS handshake, Server hello (2):
  { [96 bytes data]
• TLSv1.2 (IN), TLS handshake, Certificate (11):
  { [3229 bytes data]
• TLSv1.2 (IN), TLS handshake, Server key exchange (12):
  { [115 bytes data]
• TLSv1.2 (IN), TLS handshake, Server finished (14):
  { [4 bytes data]
• TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
  } [37 bytes data]
• TLSv1.2 (OUT), TLS change cipher, Client hello (1):
  } [1 bytes data]
• TLSv1.2 (OUT), TLS handshake, Finished (20):
  } [16 bytes data]
• TLSv1.2 (IN), TLS change cipher, Client hello (1):
  { [1 bytes data]
• TLSv1.2 (IN), TLS handshake, Finished (20):
  { [16 bytes data]
• SSL connection using TLSv1.2 / ECDHE-ECDSA-CHACHA20-POLY1305
• ALPN, server accepted to use h2
• Server certificate:
• subject: OU=Domain Control Validated; OU=PositiveSSL Multi-Domain; CN=ssl876453.cloudflaressl.com
• start date: May 18 00:00:00 2018 GMT
• expire date: Mar 9 23:59:59 2019 GMT
• subjectAltName: host "assets.superbalistcdn.co.za" matched cert's "*.superbalistcdn.co.za"
• issuer: C=GB; ST=Greater Manchester; L=Salford; O=COMODO CA Limited; CN=COMODO ECC Domain Validation Secure Server CA 2
• SSL certificate verify ok.
• Using HTTP2, server supports multi-use
• Connection state changed (HTTP/2 confirmed)
• Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
• Using Stream ID: 1 (easy handle 0x7fa71f006000)

GET /500x720/filters:quality(75):format(jpg)/spree/product/s/p/spree2018september12picslgzh0650.jpg HTTP/2
Host: assets.superbalistcdn.co.za
User-Agent: curl/7.54.0
Accept: /

• Connection state changed (MAX_CONCURRENT_STREAMS updated)!
  < HTTP/2 200
  < date: Tue, 08 Jan 2019 14:12:52 GMT
  < content-type: image/jpeg
  < content-length: 29939
  < set-cookie: __cfduid=d0c7d00daab667fc4417daa3ad7e908a01546956772; expires=Wed, 08-Jan-20 14:12:52 GMT; path=/; domain=.superbalistcdn.co.za; HttpOnly
  < alt-svc: clear
  < cache-control: public, max-age=7776000
  < cf-bgj: imgq:100
  < cf-polished: origSize=33101
  < etag: "022e2364f299e640a24e5995a280ad35b9e6591b"
  < expires: Mon, 08 Apr 2019 14:12:52 GMT
  < via: 1.1 varnish-v4, 1.1 google
  < x-varnish: 9712551
  < cf-cache-status: HIT
  < accept-ranges: bytes
  < x-content-type-options: nosniff
  < expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
  < server: cloudflare
  < cf-ray: 495f42f4ae57ad6f-TLV
  <
  { [898 bytes data]
  100 29939 100 29939 0 0 308k 0 --:--:-- --:--:-- --:--:-- 311k
• Connection #0 to host assets.superbalistcdn.co.za left intact
• malformed
• Closing connection -1

[aditrades]

Happens to me too :-\

[joeyfeldberg]

If you remove the "filters:quality(75):format(jpg)" part of the URL it works, so might be a parser issue?

[benoitc]

@joeyfeldberg good catch

Hackney will normalize the URL and encode ( and ):

5> hackney_url:normalize(Url).
{hackney_url,hackney_ssl,https,
             <<"assets.superbalistcdn.co.za">>,
             <<"/500x720/filters:quality(75):format(jpg)/spree/product/s/p/spree2018september12picslgzh0650.jpg">>,
             <<"/500x720/filters:quality%2875%29:format%28jpg%29/spree/product/s/p/spree2018september12picsl"...>>,
             <<>>,<<>>,"assets.superbalistcdn.co.za",443,<<>>,<<>>}

Normally the server must decode it. Anyway as a temporary solution @idod if you don't want to en,code the Url you can pass your own function:

9> hackney:request(get, Url, [], [], [{path_encode_fun, fun(U) -> U end}]).
{ok,200,
    [{<<"Date">>,<<"Wed, 09 Jan 2019 09:13:52 GMT">>},
     {<<"Content-Type">>,<<"image/jpeg">>},
     {<<"Content-Length">>,<<"33101">>},
     {<<"Connection">>,<<"keep-alive">>}, 
     {<<"Set-Cookie">>,
      <<"__cfduid=dc4abf8a53694e4c9c40839736259fa0c1547025231; expires=Thu, 09-Jan-20"...>>},
     {<<"Expires">>,<<"Tue, 09 Apr 2019 09:13:52 GMT">>},
     {<<"Etag">>,
      <<"\"022e2364f299e640a24e5995a280ad35b9e6591b\"">>},
     {<<"Cache-Control">>,<<"public, max-age=7776000">>},
     {<<"X-Varnish">>,<<"16710304">>},
     {<<"Via">>,<<"1.1 varnish-v4, 1.1 google">>},
     {<<"Alt-Svc">>,<<"clear">>},
     {<<"CF-Cache-Status">>,<<"MISS">>},
     {<<"Accept-Ranges">>,<<"bytes">>},
     {<<"X-Content-Type-Options">>,<<"nosniff">>},
     {<<"Expect-CT">>,
      <<"max-age=604800, report-uri=\"https://"...>>},
     {<<"Server">>,<<"cloudflare">>},
     {<<"CF-RAY">>,<<"4965ca53c879a899-CDG">>}],
    #Ref<0.2882158659.3792961537.99124>}

I will figure about a better way to handle it.