Discussion:
Understanding the process in which curl access HTTPS server
Peng Yu via curl-users
2021-04-03 13:46:23 UTC
Permalink
Hi,

I see here is the detail on how curl access https. I also showed how
to access it via openssl. But it is not clear how the steps of two
processes match each other. Could anybody help understand the detail?
Thanks.

$ curl -vv https://httpbin.org/get
* Trying 34.199.75.4:443...
* Connected to httpbin.org (34.199.75.4) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /usr/local/etc/***@1.1/cert.pem
* CApath: /usr/local/etc/***@1.1/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=httpbin.org
* start date: Dec 21 00:00:00 2020 GMT
* expire date: Jan 19 23:59:59 2022 GMT
* subjectAltName: host "httpbin.org" matched cert's "httpbin.org"
* issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fbcda813c00)
GET /get HTTP/2
Host: httpbin.org
user-agent: curl/7.74.0
accept: */*
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 200
< date: Sat, 03 Apr 2021 13:32:12 GMT
< content-type: application/json
< content-length: 257
< server: gunicorn/19.9.0
< access-control-allow-origin: *
< access-control-allow-credentials: true
<
{
"args": {},
"headers": {
"Accept": "*/*",
"Host": "httpbin.org",
"User-Agent": "curl/7.74.0",
"X-Amzn-Trace-Id": "Root=1-60686e5c-6187459a5f0f467614027f6f"
},
"origin": "195.181.162.175",
"url": "https://httpbin.org/get"
}
* Connection #0 to host httpbin.org left intact



$ openssl s_client -connect httpbin.org:443
CONNECTED(00000006)
depth=2 C = US, O = Amazon, CN = Amazon Root CA 1
verify return:1
depth=3 C = US, ST = Arizona, L = Scottsdale, O = "Starfield
Technologies, Inc.", CN = Starfield Services Root Certificate
Authority - G2
verify return:1
depth=4 C = US, O = "Starfield Technologies, Inc.", OU = Starfield
Class 2 Certification Authority
verify return:1
depth=3 C = US, ST = Arizona, L = Scottsdale, O = "Starfield
Technologies, Inc.", CN = Starfield Services Root Certificate
Authority - G2
verify return:1
depth=2 C = US, O = Amazon, CN = Amazon Root CA 1
verify return:1
depth=1 C = US, O = Amazon, OU = Server CA 1B, CN = Amazon
verify return:1
depth=0 CN = httpbin.org
verify return:1
depth=0 CN = httpbin.org
verify return:3
write W BLOCK
---
Certificate chain
0 s:/CN=httpbin.org
i:/C=US/O=Amazon/OU=Server CA 1B/CN=Amazon
1 s:/C=US/O=Amazon/OU=Server CA 1B/CN=Amazon
i:/C=US/O=Amazon/CN=Amazon Root CA 1
2 s:/C=US/O=Amazon/CN=Amazon Root CA 1
i:/C=US/ST=Arizona/L=Scottsdale/O=Starfield Technologies,
Inc./CN=Starfield Services Root Certificate Authority - G2
3 s:/C=US/ST=Arizona/L=Scottsdale/O=Starfield Technologies,
Inc./CN=Starfield Services Root Certificate Authority - G2
i:/C=US/O=Starfield Technologies, Inc./OU=Starfield Class 2
Certification Authority
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFbzCCBFegAwIBAgIQAzSkeUH+rIUZVH0Oqpvw5jANBgkqhkiG9w0BAQsFADBG
MQswCQYDVQQGEwJVUzEPMA0GA1UEChMGQW1hem9uMRUwEwYDVQQLEwxTZXJ2ZXIg
Q0EgMUIxDzANBgNVBAMTBkFtYXpvbjAeFw0yMDEyMjEwMDAwMDBaFw0yMjAxMTky
MzU5NTlaMBYxFDASBgNVBAMTC2h0dHBiaW4ub3JnMIIBIjANBgkqhkiG9w0BAQEF
AAOCAQ8AMIIBCgKCAQEAsR1fF+JAA+s/O8YGXONxK5JRc+4z/NtiFAww39lC/2qW
D2b/30ojrJVNbE500QzOTOO0YoJ3eWJupjIqFm3ImK+x8gSCWNz+pneWVY7RaoEb
mSgZRQ9YizYpVUnS4wb5UxhkzvCwsaEhqWja6yP0inaUIuW3gzrTuKTG2VBAnMtc
xYn7ttes9/BZebh0giSO0vtj7pg8Ai0n3I2moNHUumiJ1ye3pYEjys32sSb0HUGJ
f+T0k5ELs+dBM/Z7SuCq9toLNX/Uj196ZPQiv+BhCmM4VQyID5wR5riIR11/0z0E
r4FZjLAPNBFlE5bOMC87UXSf7kylvgZUuLHOKOFc2wIDAQABo4IChzCCAoMwHwYD
VR0jBBgwFoAUWaRmBlKge5WSPKOUByeWdFv5PdAwHQYDVR0OBBYEFHZOjrUHdkbF
RaBO20nVbqaNuinoMCUGA1UdEQQeMByCC2h0dHBiaW4ub3Jngg0qLmh0dHBiaW4u
b3JnMA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUH
AwIwOwYDVR0fBDQwMjAwoC6gLIYqaHR0cDovL2NybC5zY2ExYi5hbWF6b250cnVz
dC5jb20vc2NhMWIuY3JsMCAGA1UdIAQZMBcwCwYJYIZIAYb9bAECMAgGBmeBDAEC
ATB1BggrBgEFBQcBAQRpMGcwLQYIKwYBBQUHMAGGIWh0dHA6Ly9vY3NwLnNjYTFi
LmFtYXpvbnRydXN0LmNvbTA2BggrBgEFBQcwAoYqaHR0cDovL2NydC5zY2ExYi5h
bWF6b250cnVzdC5jb20vc2NhMWIuY3J0MAwGA1UdEwEB/wQCMAAwggEFBgorBgEE
AdZ5AgQCBIH2BIHzAPEAdwApeb7wnjk5IfBWc59jpXflvld9nGAK+PlNXSZcJV3H
hAAAAXaC8EAVAAAEAwBIMEYCIQDNxxAPWT/gy6wFsupHR/4vfmLTtpxkCZt8FhqE
+quq9gIhAPXuElDhqg376uQXhF9W6q2TluHS/Xs4f1hB5sNwA9XNAHYAQcjKsd8i
RkoQxqE6CUKHXk4xixsD6+tLx2jwkGKWBvYAAAF2gvBAHgAABAMARzBFAiB1FlMU
CTCW/tGD8Pskp1jxeelWhd4/uJMhsuBjXs81dQIhANcn0p9bbEkyVjPopr3xBh1H
VxY7nZNecoqRzeE2e0elMA0GCSqGSIb3DQEBCwUAA4IBAQAnnWtk2DykzfJ+Xs2q
tYFutai46RkGb0rwXgeWNXzYl7MFu44gXCR5n3ctrrz83YlPJM7fBHsI1NR39XeD
yn1XiZeVuF3JGse2/gDuYf7aUKJoEXMpZclF8MLUAMKZS4zj6WwJdhYuIDHr8quP
7AkunuP8fZ8qyApDiDNAklENqsz40C26Nest8oSYSAbcXBD3RLtBLQot5O6XI88f
qKx1DjJaaDsFNdXT8O5NRX85Sy8XrgIt6fgi25Sw4HXepZmsmXDIAUyQnnCCLkiy
UAHjRHcbR6pWDmxjoMCPkVpPEPxPXfXjsZXELPbKewwW57x3xsmUidnR/WowI0ru
s+Zn
-----END CERTIFICATE-----
subject=/CN=httpbin.org
issuer=/C=US/O=Amazon/OU=Server CA 1B/CN=Amazon
---
No client certificate CA names sent
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 5494 bytes and written 413 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES128-GCM-SHA256
Session-ID: B0284753FBF1744B849AF801099872C3E55BC04A4A6198C5469453306D1FA284
Session-ID-ctx:
Master-Key:
357939820901A7351AFDF315F328A535D4C383F0927CDE2EF436AAA3A2AF4CD7D3E697A9559B396D351953A9B726C535
TLS session ticket lifetime hint: 43200 (seconds)
TLS session ticket:
0000 - e0 ab c5 a4 af b5 e0 eb-1c 87 cc 3c ce d6 0f 0a ...........<....
0010 - 74 d2 c3 8c 82 fa 5b c5-1e 1f 74 57 ab c6 e7 2e t.....[...tW....
0020 - d3 fa 7e 63 95 33 9e 05-58 c3 aa 33 98 68 5a e1 ..~c.3..X..3.hZ.
0030 - e0 c1 fa db a1 da 7e e3-6b 8b 4c 67 02 a5 41 5c ......~.k.Lg..A\
0040 - 1d b7 ac 97 84 39 a9 87-c7 67 74 67 fb 3d 13 91 .....9...gtg.=..
0050 - f0 a8 a1 b9 6b 36 f8 3e-73 67 81 4a be c0 fb 56 ....k6.>sg.J...V
0060 - c8 14 29 fd 93 e7 6f b6-81 ff 11 d2 73 60 91 7a ..)...o.....s`.z
0070 - 1a 89 a7 2b 0d c6 0a ea-8a be b6 39 73 df a3 b6 ...+.......9s...
0080 - e9 39 36 87 09 8a 11 9a-37 f3 d2 54 21 0b d7 2c .96.....7..T!..,
0090 - a5 aa 52 7d 7c cf c0 b8-2c 0e 68 c6 2d b3 98 1d ..R}|...,.h.-...
00a0 - 1a 40 46 75 b3 39 fd 02-31 c2 75 b6 4a ea 24 77 ***@Fu.9..1.u.J.$w

Start Time: 1617457155
Timeout : 7200 (sec)
Verify return code: 0 (ok)
---
GET /get HTTP/1.1
Host: myhost

HTTP/1.1 200 OK
Date: Sat, 03 Apr 2021 13:39:45 GMT
Content-Type: application/json
Content-Length: 191
Connection: keep-alive
Server: gunicorn/19.9.0
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true

{
"args": {},
"headers": {
"Host": "myhost",
"X-Amzn-Trace-Id": "Root=1-60687021-3e701e8c57eb2dee0543c8ab"
},
"origin": "195.181.162.175",
"url": "https://myhost/get"
}


--
Regards,
Peng
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/e
Ray Satiro via curl-users
2021-04-03 17:42:01 UTC
Permalink
Post by Peng Yu via curl-users
I see here is the detail on how curl access https. I also showed how
to access it via openssl. But it is not clear how the steps of two
processes match each other. Could anybody help understand the detail?
Thanks.
I don't understand the question, can you be more specific?

-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Et
Peter Sylvester via curl-users
2021-04-03 19:01:52 UTC
Permalink
probably one has to read the ALPN rfc ;-)   The http 2 nego is piggybacked on the tls handshake.
Post by Ray Satiro via curl-users
Post by Peng Yu via curl-users
I see here is the detail on how curl access https. I also showed how
to access it via openssl. But it is not clear how the steps of two
processes match each other. Could anybody help understand the detail?
Thanks.
I don't understand the question, can you be more specific?
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette:   https://curl.haxx.se/mail/etiquette.html
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiquette.h
Peng Yu via curl-users
2021-04-04 01:53:10 UTC
Permalink
On Sat, Apr 3, 2021 at 12:51 PM Ray Satiro via curl-users
Post by Ray Satiro via curl-users
Post by Peng Yu via curl-users
I see here is the detail on how curl access https. I also showed how
to access it via openssl. But it is not clear how the steps of two
processes match each other. Could anybody help understand the detail?
Thanks.
I don't understand the question, can you be more specific?
For example, curl lists those. What openssl output does each row match?

* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /usr/local/etc/***@1.1/cert.pem
* CApath: /usr/local/etc/***@1.1/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
...
--
Regards,
Peng
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
E
Ray Satiro via curl-users
2021-04-04 03:33:38 UTC
Permalink
Post by Peng Yu via curl-users
On Sat, Apr 3, 2021 at 12:51 PM Ray Satiro via curl-users
Post by Ray Satiro via curl-users
Post by Peng Yu via curl-users
I see here is the detail on how curl access https. I also showed how
to access it via openssl. But it is not clear how the steps of two
processes match each other. Could anybody help understand the detail?
Thanks.
I don't understand the question, can you be more specific?
For example, curl lists those. What openssl output does each row match?
* ALPN, offering h2
* ALPN, offering http/1.1
That is verbose output that comes from curl. Most of it is TLS protocol
messages that come from the ossl_trace callback [1] which is called by
OpenSSL. Basically curl is responsible for outputting the messages from
OpenSSL. So the output is not from OpenSSL directly.

[1]:
https://github.com/curl/curl/blob/curl-7_76_0/lib/vtls/openssl.c#L2076-L2181
Loading...