$ curl -o whitelist.txt https://example.com/BLUECOAT/whitelist.txt
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0c
url: (60) SSL certificate problem, verify that the CA cert is OK. Details:
error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify fail
ed
More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.

When I added the -k option, I was able to download the file successfully.

$ curl -o whitelist.txt -k https://example.com/BLUECOAT/whitelist.txt

But I wanted to know what the issue was with the public key certificate and I wanted to get that information from a Bash shell prompt. You can get the certificate from a website using the command openssl s_client -showcerts -connect fqdn:443, where fqdn is the fully qualified domain name for the website, e.g. example.com. Port 443 is the standard port used for HTTPS. The certificate should be stored as a .pem file. When I used openssl s_client -showcerts -connect example.com:443 >example.pem, I saw the message "verify error:num=19:self signed certificate in certificate chain" displayed, which revealed the source of the problem.

A self-signed certificate is one that has been signed by the same entity whose identity it certifies. For a site using a self-signed certificate, your traffic to and from that site is protected from eavesdroppers along the path of the traffic, but the certificate doesn't offer validation that the site belongs to the entity claiming to own it. But, if you have other reasons to trust the site or are only concerned about third parties eavesdropping on your communications with the site, then a self-signed certificate may be adequate. E.g., the site could be your own site or belong to someone or an entity you know is in control of the website. Some organizations use self-signed certificates for internal sites with the expectation that members/employees will ignore browser warnings for the internal websites, though if people become accustomed to ignoring such errors there is the danger that they will also be more prone to ignore such warnings for external sites where a site's true controlling entity isn't the one they expect.

$ openssl s_client -showcerts -connect example.com:443 >example.pem
depth=1 /C=US/ST=Maryland/L=Greenbelt/O=ACME/OU=EXAMPLE/CN=EXAMPLE CA
verify error:num=19:self signed certificate in certificate chain
verify return:0
read:errno=0

The s_client parameter uses a generic SSL/TLS client to establish the connection to the server.

       s_client  This implements a generic SSL/TLS client which can establish
                 a transparent connection to a remote server speaking SSL/TLS.
                 It's intended for testing purposes only and provides only
                 rudimentary interface functionality but internally uses
                 mostly all functionality of the OpenSSL ssl library.

The certificate is stored in example.pem in this case. You would need to edit the file to remove everything but the "BEGIN CERTIFICATE" and "END CERTIFICATE" lines below and the lines that lie between those two lines.

-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----

Or you can use a Bash script retrieve_certifcate to obtain the certificate; it will stip off the extraneous lines. The code for the script is shown below:

#!/bin/sh
#
# usage: retrieve-cert.sh remote.host.name [port]
#
REMHOST=$1
REMPORT=${2:-443}

echo |\
openssl s_client -connect ${REMHOST}:${REMPORT} 2>&1 |\
sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p'

You can obtain information for the certificate from the PEM file using the command openssl x509 -text -in example.pem. If -issuer is appended, then only the issuer information will be displayed, so I could see that the cerificate was self-signed with the following command:

$ openssl x509 -noout -in example.pem -issuer
issuer= /C=US/ST=Maryland/L=Greenbelt/O=ACME/OU=EXAMPLE/CN=EXAMPLE CA

If you just want to verify the status of a certificate from the command line without storing the certificate locally, you can add the -verify 0 option.

       -verify depth - turn on peer certificate verification

E.g.:

$ openssl s_client -showcerts -verify 0 -connect example.com:443 
verify depth is 0
CONNECTED(00000003)
depth=1 /C=US/ST=Maryland/L=Greenbelt/O=ACME/OU=EXAMPLE/CN=EXAMPLE CA
verify error:num=19:self signed certificate in certificate chain
verify return:0
88361:error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed:/SourceCache/OpenSSL098/OpenSSL098-52.6.1/src/ssl/s3_clnt.c:998:

You can ignore all output from the command but the "verify error" line with commands like the following:

$ openssl s_client -showcerts -verify 0 -connect example.com:443 2>&1 | grep "verify error"
verify error:num=19:self signed certificate in certificate chain

For another internal website, when I accessed the site in Firefox with https://cmportal, Firefox reported the following:

This Connection Is Untrusted

You have asked Firefox to connect securely to cmportal, but we can't confirm that your connection is secure.

Normally, when you try to connect securely, sites will present trusted identification to prove that you are going to the right place. However, this site's identity can't be verified.

What Should I Do?

If you usually connect to this site without problems, this error could mean that someone is trying to impersonate the site, and you shouldn't continue.

When I viewed the technical details for the certificate, Firefox informed me that:

code760cmportal uses an invalid security certificate. The certificate is only valid for the following names: 192.168.160.242, servera.example.com (Error code: ssl_error_bad_cert_domain)

When I tried downloading the home page for the site with curl, I saw the message below:

$ curl https://cmportal
curl: (51) SSL peer certificate or SSH remote key was not OK

I was able to get past that error with the -k or --insecure parameter to curl, though then the page returned reported I was being denied access to the requested web page due to invalid credentials.

I downloaded the certificate for that site with openssl; since openssl would wait for input after verify return:0, I used an echo "" | to get it to complete.

$ echo "" | openssl s_client -showcerts -connect cmportal:443 >example.pem
depth=2 /C=US/O=Acme/OU=Anvils/OU=Certification Authorities/OU=Anvils Root CA
verify error:num=20:unable to get local issuer certificate
verify return:0
DONE

I removed all the lines before "BEGIN CERTIFICATE" and all those after "END CERTIFICATE" and then checked the certificate for that .pem file with the openssl command. That showed me a reference to servera whereas I had accessed the site using cmportal..

$ openssl x509 -noout -in example.pem -subject
subject= /C=US/O=Acme/OU=Anvils/OU=Services/CN=servera.example.com

If you've accepted a self-signed certificate, or a certificate with other issues, in Firefox, you can view the certificate following the steps noted in Forgetting a certificate in Firefox.

References:

1. Retrieving Password Protected Webpages Using HTTPS With Curl
   Date: September 8, 2011
   MoonPoint Support
2. How To Verify SSL Certificate From A Shell Prompt
   Date: May 23, 2009
   nixCraft
3. Example sites with broken security certs [closed]
   Asked: November 9, 2009
   Stack Overflow
   
4. Command line tool for fetching and analyzing SSL certificate
   Asked: April 17, 2014
   Server Fault
5. OpenSSL Command-Line HOWTO"
   Published: June 13, 2004
   Most recent revision: June 25, 2014
   By: Paul Heinlein
   madboa.com
6. x509 - Certificate display and signing utility
   OpenSSL: The Open Source toolkit for SSL/TLS

[/network/web/tools/curl] permanent link