Happening Now: Over 2 Percent of Sites Using a Let’s Encrypt TLS Certificate May Throw Security Warnings

On Wednesday, March 4, 2020, 3 million Transport Layer Security (TLS) certificates issued by Let’s Encrypt will be revoked because of a Certificate Authority Authorization (CAA) bug. This is 2.6% of the over 116 million active certificates issued by Let’s Encrypt.

Let’s Encrypt has contacted all certificate holders affected by this bug, and they’ve created a tool and a list of serial numbers to determine if your TLS certificate is affected by the bug.

Let’s Encrypt have not set an exact time for revocation of the certificates, however, they say that the earliest timeframe will be UTC 00:00.

Some certificate holders have received emails that they’re affected, but they may have received that alert erroneously, either because the certificate was issued in the last few days after the bug was fixed, or by not meeting certain timing criteria necessary for the bug to trigger, adding to confusion.

How to tell if you’re affected

Let’s Encrypt created a tool where you can check your site’s host name and determine if your Let’s Encrypt-issued certificate is affected by this bug.

Let’s Encrypt can also see the list of all affected serial numbers.

On a Linux/BSD-like system, you can also run the following command to show your domain’s current certificate serial number. Replace example.com below with your own domain name:
openssl s_client -connect example.com:443 -servername example.com -showcerts </dev/null 2>/dev/null | openssl x509 -text -noout | grep -A 1 Serial\ Number | tr -d :

If your hosting provider provided a certificate for your website, they were likely the ones contacted by Let’s Encrypt. Numerous site owners have received notifications from hosting providers that they would be handling the reissuance of those certificates.

If you have created your own Let’s Encrypt certificate, you will need to update yourself if you are affected.

What will happen if I don’t fix this?

A secure TLS certificate ensures that your site visitors have encrypted traffic between their browsers and your website. Site visitors might see a certificate revoked error, a “not secure” warning, or other security warnings in their browser that may erode trust in your site.

What happened in technical terms?

Boulder, the software builder used by Let’s Encrypt’s CA, checks CAA records for a domain name at the same time that it verifies that a certificate requester controls that domain. Most subscribers to the service issue a certificate immediately after they validate domain control, however Let’s Encrypt trusts that validation for 30 days. Due to that trust, they sometimes have to recheck CAA records a second time, just prior to issuing the certificate. The timeframe for rechecking is 8 hours, meaning that any domain name validated more than 8 hours ago requires a recheck.

According to Let’s Encrypt:

The bug: when a certificate request contained N domain names that needed CAA rechecking, Boulder would pick one domain name and check it N times. What this means in practice is that if a subscriber validated a domain name at time X, and the CAA records for that domain at time X allowed Let’s Encrypt issuance, that subscriber would be able to issue a certificate containing that domain name until X+30 days, even if someone later installed CAA records on that domain name that prohibit issuance by Let’s Encrypt.

Let’s Encrypt confirmed the bug at 2020-02-29 03:08 UTC, and halted issuance two minutes later. They deployed a fix at 05:22 UTC and re-enabled certificate issuance at that time.

According to security researcher Scott Helme, who posted his investigation on Twitter:

Possibly worth noting that while some of these numbers are really large, it represents only ~2.6% of currently active certs that are impacted. Within that ~2.6% there are *significant* numbers of duplicate certs with the exact same CN/SAN list but a different serial number.

— Scott Helme (@Scott_Helme) March 3, 2020

Does this mean we should use something other than Let’s Encrypt for SSL certificates?

Let’s Encrypt have been very transparent about this bug, both in identifying the problem themselves and reporting the CA incident. They are acting exactly how a certificate authority should act. As such, we are confident that Let’s Encrypt is still a good source for TLS certificates.

You can find details of the bug on the Let’s Encrypt bug tracker.