Resume vs. Multiple URIs on Command Line?

Chris Jepeway

2017-08-15 01:06:22 UTC

So, I’m a tad confused by curl’s behavior wrt command line URIs.

When I let curl handle the brace expansion of URIs like this:

$ curl -O 'https://ftp.fr.openbsd.org/pub/OpenBSD/6.1/amd64/{SHA256,BUILDINFO}’

I get the files SHA256 and BUILDINFO:

$ ls -l
total 16
-rw-r--r-- 1 jepeway staff 54 Aug 14 20:50 BUILDINFO
-rw-r--r-- 1 jepeway staff 1989 Aug 14 20:50 SHA256

Now, let’s say I don’t actually believe I’ve d/l’ed the entire contents of these files. So, I try -C, while letting my shell handle brace expansion:

$ curl -C - -O https://ftp.fr.openbsd.org/pub/OpenBSD/6.1/amd64/{SHA256,BUILDINFO}

SHA256 works, but I get a:

416 Range Not Satisfiable

for BUILDINFO.

Why is this? Well, with -v, you’ll see:

$ curl -v -C - -O https://ftp.fr.openbsd.org/pub/OpenBSD/6.1/amd64/{SHA256,BUILDINFO}
** Resuming transfer from byte position 2440
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 145.238.209.46...
* TCP_NODELAY set
* Connected to ftp.fr.openbsd.org (145.238.209.46) port 443 (#0)
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
* Server certificate: pond.obspm.bsdfrog.org
* Server certificate: Let's Encrypt Authority X3
* Server certificate: DST Root CA X3

GET /pub/OpenBSD/6.1/amd64/SHA256 HTTP/1.1
Host: ftp.fr.openbsd.org

===> > Range: bytes=2440-

User-Agent: curl/7.51.0
Accept: */*

* HTTP 1.0, assume close after body
< HTTP/1.0 416 Range Not Satisfiable
< Date: Tue, 15 Aug 2017 00:57:13 GMT
< Server: OpenBSD httpd
< Connection: close
< Content-Type: text/html
< Content-Length: 451
< Content-Range: bytes */1989
<
{ [451 bytes data]
* Curl_http_done: called premature == 0
100 451 100 451 0 0 816 0 --:--:-- --:--:-- --:--:-- 815
* Closing connection 0
* Hostname ftp.fr.openbsd.org was found in DNS cache
* Trying 145.238.209.46...
* TCP_NODELAY set
* Connected to ftp.fr.openbsd.org (145.238.209.46) port 443 (#1)
* SSL re-using session ID
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
* Server certificate: pond.obspm.bsdfrog.org
* Server certificate: Let's Encrypt Authority X3
* Server certificate: DST Root CA X3

GET /pub/OpenBSD/6.1/amd64/BUILDINFO HTTP/1.1
Host: ftp.fr.openbsd.org

===> > Range: bytes=2440-

User-Agent: curl/7.51.0
Accept: */*

* HTTP 1.0, assume close after body
< HTTP/1.0 416 Range Not Satisfiable
< Date: Tue, 15 Aug 2017 00:57:14 GMT
< Server: OpenBSD httpd
< Connection: close
< Content-Type: text/html
< Content-Length: 451
< Content-Range: bytes */54
<
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
<title>416 Range Not Satisfiable</title>
<style type="text/css"></style>
</head>
<body>
<h1>416 Range Not Satisfiable</h1>
<hr>
<address>OpenBSD httpd</address>
</body>
</html>
* Curl_http_done: called premature == 0
* Closing connection 1

Note how the Range: header for the first file sets the Range: header for the second (bytes=2440-). Since the second file is only 54 bytes long, the http server says “uh, no, 416 for you.”

Is this expected behavior based on how curl parses the command line? Am I supposed to invoke as follows

$ curl -O -C - https://ftp.fr.openbsd.org/pub/OpenBSD/6.1/amd64/SHA256 -O -C - https://ftp.fr.openbsd.org/pub/OpenBSD/6.1/amd64/BUILDINFO

instead?

Chris.

-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiquett