The Opera Rootstore

Earlier this week news reached us that a small subordinate CA in Malaysia, Digicert Sdn. Bhd. (which we hasten to add, has no association with, and must not be confused with, the US-based Root CA Digicert, Inc., which is a member of Opera's Rootstore), had been discovered to have issued certificates that did not meet several technical and contractual requirements it was supposed to follow. We have therefore blacklisted several of their certificates.

The problems were:

• Their certificates do not contain a field called "Extended Key Usage", which is used to limit what a certificate can be used for. Without this field the certificate can be used as more than a SSL/TLS server certificate, e.g. to sign executables (Object signing).
• Their certificates did not contain any pointers to revocation information, that is, there were no OCSP URL in their certificate, and neither was there a URL to a CRL, meaning that the validity of the certificates could not be checked.
• They had issued several certificates for RSA keys that were only 512-bits long. If you have been following our security related articles, you probably have a good idea of what we here at Opera think about such keys. Summary: Don't even think of using them. These keys can be broken in days(!), and there are clear indications that at least some of these specific weak keys have been compromised!
• At least one of these weak certificates is actively being exploited in a phishing attack. Presumably it has been possible to crack the certificate due to the weak security.

Digicert Sdn. Bhd. is not a Root CA, but is a subordinate CA that operates using an Intermediate CA certificate issued by another CA, in this case they have certificates for their key issued by Cybertrust/Verizon and Entrust, both of which are members of Opera's Rootstore.

There are no indications that any certificates have been issued fraudulently, but the above points break not just the CA's own rules, they break the contracts that they signed with the two Root CAs that issued the intermediate CA certificates they have been using. For this reason Cybertrust have already revoked the certificate they issued to Digicert Sdn. Bhd., and Entrust will revoke theirs in a few days. Entrust is delaying the revocation a few days so that the affected web sites can have a few days grace time to obtain new certificates from a different issuer.

Since such certificates do not display as secure in Opera anyhow (see below), we have decided to allow this grace period. Should there be any signs of problems in the meanwhile, we can and will revoke the intermediate certificate ourselves.

In the meantime we have been provided a list of the 22 certificates that had weak keys, and due to the missing revocation information and signs of abuse we have decided to blacklist these particular certificates in today's update of the Rootstore.

For Opera users there are a few things to note:

• Digitally signed Java applets will not be affected by this update, as the Java plug-in handles all verification itself. Until Java's certificate store has been updated, or Entrust has revoked the intermediate CA, it suffices to not trust any Java popups.
• The missing CRL caused Opera to remove the "Secure" indication for all web sites using certificates issued by Digicert Sdn. Bhd. The reason for this is that there was information about the CRL in their intermediate CA certificate, and that causes Opera to require revocation information in all certificates in the chain, or the security level is lowered.
• The web sites using the certificates with 512-bits keys would also trigger a certificate warning dialog about the weak key, as Opera is currently warning about any RSA key shorter than 900 bits long.
• The update will be automatically downloaded and installed within the next week in supported versions, no user action is needed. However, if you would like to get the update immediately, you may do so using the menu option Help > Check for Updates, which will trigger the update.

Given that the CA certificate will be revoked shortly, and that there is no indication of additional significant threats, this blacklisting is only a temporary measure, and we plan to remove these certificates from the list within a few weeks, once the revocation information has been distributed.

Related to this, we have also learned that a few other CAs have also issued about 25 certificates with 512-bit keys. At present we do not have details about these certificates, but we have been informed that the certificates should be revoked within a week.

We stress that such certificates could not have been used to signal any secure connections in Opera, and that Opera users who pay attention to the security badge have nothing to worry about.

We have contacted other browsers suggesting ways to address various issues around weak certificates and certificates that are not issued according to the recognized best practices, and we are also working with CAs to improve procedures and security in general.

In order to stay safe online we encourage users to configure their web browser to ensure that it does not allow weak security, if it does not already do so by default, such as Opera. Users also need to pay attention to the security indications.

There is a first time for many things. Sometimes this is positive, but other times it is unfortunate. This time, we're dealing with one of the latter variety.

Today, we have updated the Opera Rootstore and disabled the DigiNotar Root (Note: DigiNotar is not in any way associated with DigiCert, which is another CA).

The reason for this action is due to last week's news about the large scale attack on DigiNotar's CA systems and the subsequent discoveries about what has happened after the attack.

This is the first step in our handling of this incident. We are currently testing further updates that will add the DigiNotar Root to our "Untrusted" repository.

That a CA gets attacked and even tricked into issuing certificates that turn out to be fraudulent is not, by itself, a reason to revoke trust in a Root CA. CAs are high profile targets for criminals given the role CAs have in the online security framework; it is how the CA responds to such events that determine how much we can trust it.

In this case, DigiNotar's response has unfortunately fallen short in many respects, including the following:

• They did not inform the affected websites, the browsers or the CABForum (CA/Browser Forum) about what had happened for more than 5 weeks, and they only did so after one of the certificates turned up being used in a MITM attack against Google websites. This can be compared against the fact that the browsers were informed about the (successful) attack on Comodo, as well as the (unsuccessful) attack on Startcom, within a few days of the events.
• While they did revoke a number of certificates that were fraudulently issued, they did not inform the affected websites, and they did not discover that a certificate for Google's domain had been issued until after Google had learned of the certificate from users being actively attacked with it (a discovery made possible by a recent Chrome security measure implemented for Google's sites) and informed DigiNotar and the browsers about it.
• The reason for the delay in discovering this certificate was that the attackers had tampered with the log information in the certificate issuance system for at least one of the subCAs. It is presently unknown whether the attackers also tampered with the logs for the other subCAs.
• Due to the log problems, it is in fact still unknown how many certificates that were really issued by this system or for what websites! We have seen reports that more than 500 certificates were issued during the attack, with less than 300 being positively identified, according to our information.

For these reasons, we are today discontinuing distribution of DigiNotar's Root.

A couple of points to note:

• No user action is needed to get this update. If your installation does not have the DigiNotar Root installed already, visiting a site with a DigiNotar issued certificate will trigger a "Unknown issuer" dialog; if this happens, click "Reject" on the dialog.
• This first step of our planned revocation does not remove, or untrust, the DigiNotar Root from Opera installations that already have the Root installed. For these installations, sites using DigiNotar certificates will continue to work as before, indicating a secure connection, as will the EV-enabling of the Root until the next weekly download from the repository server (using the menu choice Help->Check for Updates will trigger an immediate update). For advanced users: if your installation has this certificate, and you want to untrust it now, you can do so by following these steps:

1. Go to the Root Certificate Management dialog in the Menu->Setting->Preferences->Advanced
   Preferences->Advanced->Security->Manage Certificate->Authorities.
2. Locate the "DigiNotar Root CA" entry and click "View".
3. Uncheck the "Allow connections to sites using this certificate".
4. Click "OK" on the viewer dialog, the Certificate Manager and the preference dialog.
5. The DigiNotar certificate is now untrusted, and visiting sites issued from the Root will cause Opera to refuse to connect to the site, without displaying a certificate warning.

Please note that this action does not affect the Dutch Government CA "PKI Overheid" Root CA, which is in the process of revoking the subCA certificate they issued to DigiNotar.

In relation to these events, the browsers and the Certificate Authorities in the CABForum are monitoring the situation and are discussing further improvements to how certificates for secure websites work, both in browsers and on the CA side.


Update Sep 8: Second stage deployed.

Today, we have updated the Rootstore with the following:

• CNNIC, the Chinese Registrar and CA have been EV enabled. As
  part of this, a new EV-enabled Root was added (the old is not EV-enabled). Testsite
• An update of the Public Suffix list, adding the suffixes gob.ec
  and bv.nl. Version 1.5 is available for download from our source repository.

Additionally, the "major" update is due to the recent attack on the CA Comodo^1,2, where an attacker was able to hack into an account and trigger the issuance of 9 certificates for major communication sites (a few other accounts have apparently also been compromised, though no certificates have been issued from these accounts according to our information). Rumors are flying about who was behind the attack, and why, but I will let those rest.

We learned of the attack late evening March 16th, and, by then, the issued certificates had already been revoked by Comodo for more than 24 hours (Comodo detected the attack within a few hours).

I evaluated whether to update the Rootstore Untrusted list of certificates with those certificate already that week, but decided not to do so at the time, in particular since Opera's blacklist could not blunt any attack within the 2 days remaining of the pre-revocation OCSP response validity, and there had been no accesses for the revocation information in the open window.

Also, after the (possible) pre-revocation OCSP responses had expired, any attempt to block revocation checking would remove the secure indication of the site when loaded by Opera, so any users that are keeping an eye on the padlock (we know that a lot you do) would be protected, so after that time the certificates could not in any way have been used to flag pages as secure in Opera.

Another consideration is that the Untrusted list is not intended to be an extra revocation list for normal certificates; it is only intended to include problematic certificates that cannot be revoked in any other fashion, such as the MD5 collision cert a few years ago or a renegade Root CA.

Adding fraudulently issued certificates is not among the list's intended uses, except in special situations. The reason is that revoking those certificates is what the revocation lists (CRLs and OCSP) are for.

At present, however, we have a situation where it is not practical to refuse access to a secure site because the browser cannot access up-to-date revocation information, particularly since the responders have occasional momentary down periods. Opera removes the padlock if we can't get the revocation information, as far as I know Chrome puts a (Golden? Shouldn't that have been red?) warning triangle over the padlock, others give no indication at all that there was a problem.

In the aftermath of the Comodo incident, several participants in the discussion, including me, have suggested that we should move towards stricter requirements regarding having revocation information available before allowing a secure connection to complete.

There are several downsides to this, particularly that it requires exceedingly high stability for the revocation servers, and there is also an implied requirement that all browsers need to do this policy change at the same time, to provide the same user experience.

The stability requirement can be worked around by requiring secure servers to implement TLS OCSP stapling, an extension to the TLS protocol where the TLS server caches its own OCSP response and sends it to the client when it connects. This saves the extra roundtrip for the client when it retrieves it directly, and provides an up-to-date status. There are two problems at present: 1) only 3% of servers support the extension and 2) to be truly useful, all the revocation info for all the certificates in the server's chain needs to be included, and that is currently not possible; but I have submitted a proposal to the TLS WG that would allow that (although there are some concerns that the extra data will cause performance issues for big sites).

In the meantime, using a blacklist may be the best option for limiting impact, but it is also an option that does not scale in the long run, and it also has a relatively long deployment time, compared with revocation. Excluding time for users to update, it took the browsers that shipped blacklist updates last week a week to ship their updates, and that was probably achieved by pushing their release cycles to the limit. Even this Rootstore update took a week to prepare.

In this case we have decided to follow the other browsers, and update our Untrusted list with these certificates, and we might continue to do so in the short term, should there be more incidents, but in the long term other solutions will have to be found.

The are some things to note, even though I hope none of you will ever encounter any of these certificates in the wild:

• Only 7 of the 9 certificates are included, due to the ID of two certificates colliding with a third certificate; the ID is based on the certificate's subject name and public key. This problem is due to a design error in the repository handling; plans are under way to fix the problem.
• The blocked certificates are only pushed to Opera 9.5x and 9.6x versions, since they are not able to use the newer dynamic system. The above point means that two certificates are not installed in those version and are not blocked. In all other versions, there is no visible indication of the list having been updated.
• In newer versions the blocked certificates are only downloaded if they are ever needed. If the download fails, the certificate will still be blocked, which also means that the two certificates not in the list will actually be blocked anyway.
• In Opera 10.x before 10.50, the certificates will be blocked in the way the functionality was designed, by displaying a message that the certificate is untrusted.
• In Opera 10.50 and later, including 11.10, there is a bug. While the blocking works, it works by either showing a blank page or a "Could not connect" message. We are looking into this.
• If the blocked certificates are ever encountered, by Opera 10.0 or later, revocation is checked first, and only if the certificate is not revoked (or the information could not be retrieved) will the untrusted list be consulted.

This update will propagate to all Opera versions that support the online Rootstore during the next week; if you are impatient, just use Menu->Help->"Check for updates" to fetch it (The CNNIC EV-site mentioned above can be used to test that you got the update).

Summary: Opera users who consult the security indication could not have been tricked into believing a site with the fraudulent certificates was secure once the Comodo revocation lists became authoritative, the addition to the blacklist is an extra security measure that strictly speaking is not necessary. No action is needed by our users to download this update.

New EV-enabled CAs

The Rootstore repository has been updated with two CAs being EV-enabled:

• Startcom/StartSSL, an Israeli CA added in July. General test site. Test site for Opera 11.00.1133+
• TrustCenter, a German CA. Test site

The reason there is a specific 11.00 test site for Startcom is that, up until now, Opera has implemented a restriction on the relative key sizes for the certificates in an EV certificate chain. The rule was that an intermediate CA certificate had to use a key that had at least as many bits as the key being signed.

The primary reason for this rule was to make the CA keys a harder target than the web site's key, but unfortunately this policy have caused some issues in the past, including the Startcom site, and we are now removing the policy as of Opera 11.00.1133. Earlier versions of Opera will, for example, not display Startcom's page as having an EV certificate, while Opera 11 now will do so.


Public Suffix v1.3

We recently discovered that we had lagged behind the current state of Mozilla's Public Suffix List. The reason turned out to be that the URL for the changelog RSS feed I had originally subscribed to had changed.

This new update contain adjustments to many TLDs, as well as adding initial entires for a number of new internationalized (IDN) TLDs such as for China, Egypt and Russia.

As usual, an unsigned version is available from our source repository.

The Basque CA Izenpe (EV-enabled in September) is preparing a new line of EV certificates to be used by Spanish government web sites. These new certificates are mandatory for all the public administrations according to the 11/2007 law. Currently, only Izenpe is able to issue these certificates as EV because Izenpe is the only Spanish CA certified to issue EV certs. Izenpe have designated an extra EV-OID for this line of certificates. This new OID has now been added to the list of OIDs recognized for the Izenpe EV Root. This is the first time a CA has had two EV-OIDs enabled for the same Root in Opera. A testsite is available

We have also added a few certificates to the list of untrusted certificates.

* Two of these certificates leveraged differences (related to handling of NUL bytes) in the processing of hostnames between a CA's domain name checking systems and some browsers to trick the CA into thinking it was validating a certificate for www.mybank.com<nul>.www.example.com, while the browser would think the certificate was for another site, www.mybank.com, which could facilitate a Man-In-The-Middle attack on the user. While the issuing CA has revoked these certificates we are taking the extra precaution of adding the certificates to the list of untrusted certificates. Therefore, they will not accidentally be accepted by users if the revocation system fails. Both CAs and browsers have been fixing the related issues, and Opera included fixes for this issue in Opera 10.00.

* Additionally, in preparation for future code changes in Opera Presto 2.4, and just to be on the safe side, we have added two object signing certificates that were issued in 2001 to someone pretending to act on behalf of Microsoft. While these certificates have long since expired, the possibility exists that they could still be used maliciously.

These certificates are only downloaded and installed in the untrusted repository when they are actually encountered.

In version 1.1 of the Opera Public Suffix list we have added the domain operaunite.com as a public suffix domain. We have also submitted a patch request to the Public Suffix project and to Microsoft for inclusion of the domain in their lists. The updated version is available from our repository.