The Death of SSL
Secure Sockets Layer (SSL) is one of the foundational technologies that enabled and allowed commerce to exist in an electronic, decentralized medium. Without it (or something similar), many of us would not have the types of jobs we do today. There are a number of implementations of this historical protocol—one of the most common being the opensourced implementation published under the OpenSSL name.We’ve been dealing with SSL issues in PCI DSS since version 2.0 where SSL version 2
implementations were no longer allowed due to vulnerabilities in the protocol. I can remember counseling a number of customers through the migration process. The ones that were the hardest to resolve were implementations in embedded systems or those third-party “black box” solutions that end up playing a critical role in a firm’s IT environment.
Then in 2014, we had two specific vulnerabilities related to SSL that caused a bit of a ruckus through the world. The first was Heartbleed—a gruesomely named vulnerability reported by Neel Mehta from Google’s Security Team. Unfortunately, this vulnerability was present in OpenSSL for a couple of years before its discovery. The vulnerability in the OpenSSL codebase was so severe and omnipresent that it forced the operators of millions of websites to revoke and reissue their SSL certificates after they patched. The bug allowed an attacker to “bleed” arbitrary amounts of data from memory revealing sensitive things like private keys that allow anyone to decrypt the SSL stream. It’s like the magic decoder ring that has the power to reveal the contents of any encrypted stream. The lock in the browser was just a false sense of security.
The second was a series of vulnerabilities that started with the first POODLE attack published in October of 2014. This initial release (plus future iterations that targeted early versions of TLS) is what kicked off the process at the Council to kill SSL and early versions of Transport Layer Security (TLS) in favor of TLS 1.2 (or greater,
PCI DSS 3.1. DOI: http://dx.doi.org/10.1016/B978-0-12-804627-2.00002-3 © 2016 Elsevier Inc. All rights reserved.
depending on when you are reading this book). While the debate over the severity of this particular vulnerability rages, it was scored at 4.3 on the CVSS list which means you have to remediate it to remain compliant with PCI DSS.
Following this, the Council removed the three instances of SSL from PCI DSS 3.0 and the genesis for PCI DSS 3.1 was born. Given the CVSS score, it almost seems unnecessary for the Council to rush out PCI DSS 3.1 with sunset dates on SSL. Let’s consider this for a moment.
PCI DSS Requirement 11.2 mandates quarterly scans of your networks. As of last October, any machine that is vulnerable to the POODLE attack is flagged as a vulnerability that needs to be fixed. Remember, any vulnerability with a CVSS score over 4.0 must be fixed in order to obtain a passing or clean scan. Therefore, if you have not solved this issue yet, then you are unable to comply with PCI DSS Requirement 11.2. So while the Council rushed to remove those words from PCI DSS 3.1, they did so (among other reasons) to make sure that these two items were in alignment. Otherwise, you would have someone trying to argue an ASV scan is wrong because the standard clearly supports the use of SSL. The reverse argument applies as well.
The tricky part, is the lack of attention on internal vulnerability scans. Companies tend to pay more attention to their external scans than internal ones because the external ones are performed by a qualified third party and may be reported directly to your bank. There tends to be more pressure to obtain clean external scans than internal ones. I’m not saying that people outright lie on their internal scans, but I sat through a number of meetings where three quarters of failing scans were explained away by the lack of a breach.
Regardless, we are here because the SSL protocol has a number of problems with it, and we now rely on TLS version 1.2 or greater for our security.
Let’s also discuss some of the pressures the Council is under right now. I’m writing this from the perspective of a participant in the PCI DSS ecosystem, not someone who has any inside knowledge of what is going on at the Council. When all of this was going down, I was at a QSA/ASV company and not on the Board of Advisors. Any similarities to secret, closed-door discussions are purely coincidental.
Payment card breaches continue, with 2014 being a particularly bad year for them. The term “cyber” is gaining in popularity again and we have all kinds of government attention on information security, breaches, and the havoc they cause to our economy. It’s becoming clearer that PCI DSS is not nearly as effective as it should be. There are two reasons for this. The one you will hear from the Council is that the ecosystem is not fully adhering to the Standard, thus, arguing that the Standard is not at fault for these breaches. I don’t know if you can refute that argument, so let’s just accept it at face value. The other reason you may hear is that the Standard is still too complex with too many shades of gray, therefore it is impossible to really comply in the first place.
The economic drain is real. Preparing for and performing assessments isn’t actually making companies any safer than they would be without these assessments. They, more accurately, tie up capital and have real opportunity cost. For more information on that, go search “the broken window fallacy” and learn how.
The real reason is somewhere in the middle. In defense of the Standard, most companies don’t either take it seriously or make rational decisions about their security posture and how they process cardholder data. When we have a documented, public breach that happens with payment card data where they are found to be fully compliant with the Standard at the time of the breach, we can revisit this. More likely, we will need a string of these to really revisit how effective the Standard is at keeping cardholder data safe.
However, in defense of those trying to comply with the Standard, it needs a major overhaul. It is too complex, there is too much documentation, and every time I go about writing another version of this book I find conflicting or outdated information on the website. I could easily argue that its complex nature is the reason why it is ineffective. It’s trying to be the big round hole that everyone’s custom-shaped peg has to fit through. Solutions to this problem are beyond the scope of this book, but it’s irresponsible for each group to point the finger at the other. Solutions come from collaboration, compromise, and cooperation. If anything, we can look to our own government and the polarizing nature politics has in our society to validate this.
Regardless of your leaning on this issue, SSL is dead and must be replaced. Companies using SSL should have been ahead of this and either not been using it at the time of the announcement, or fully upgraded shortly thereafter. Hopefully this describes you!
Migrating from SSLv3 to TLSv1.2 or later is trivial in modern systems. The Council published an overview document that may be useful for you to review.1 It discusses many of the implications, but it doesn’t go through the actual migration steps or point you to resources that could be helpful. For the most part, you only need to use a search engine to help you. Since this is quite a hot topic, most major web servers, Virtual Private Network (VPN) software providers, and other transport layer security products have well-published steps on how to do this.
Just as a quick guide for you, here are a few configuration changes you can make by product to disable SSL.
- Apache. Look for any SSL configurations and be sure to set the following: “SSLProtocol all -SSLv2 -SSLv3”
- Tomcat. Remove any sslProtocols config lines and add:
- Java. Should be auto disabled if you keep your JDK/JRE up to date.
- IIS. Check with Microsoft on this as you will probably need to get into the registry.
Don’t forget, it’s not just web servers that use SSL. You may be using some version of SSL in your email, VPN, database, TN3270, and authentication servers as well. Be sure to do a thorough job of looking through your environment to find and remediate any instances of SSL v3.
As you can see, this is largely just a configuration change for most of your systems provided they are currently maintained and you have the necessary access to alter those configurations. If you have good configuration management, you may be able to push this change through fairly rapidly by changing just a few master configuration files and pushing it out to your organization. If not, this may end up being one of those nightmare scenarios that forces you to take a hard look at how you currently do configuration.
Pay particular attention to any systems that come supplied, or are fully managed, by a vendor. For current and future installations, demand a “Bill of Goods” from all of your vendors that clearly states what open-source technology is present in the device. This will help your security teams make accurate risk assessments for how new vulnerabilities truly affect your organization. As an example, the large number of standalone devices still vulnerable to Shellshock is, well, shocking.2 The real kicker is that many that are still vulnerable cannot be fixed because they do not have the capacity to be upgraded.
OuchTown: population YOU, bro!
When you are going through your scanning results, make a list of every device that still responds to SSLv3 and then make your game plan. For the devices you control and can make updates to, get those changes scheduled and planned to go in your next update. Your big challenges will come with systems that are either no longer maintained—which probably put you out of compliance anyway—and outdated clients that do not support newer TLS protocols. Those of you who are still on IE 8 (or worse, IE 6) are in trouble. Any applications that use SSL will break with outdated browsers. See the note for a compatibility matrix.
The TLS Wikipedia article has a fantastic compatibility table that may serve as a great reference for you. See that here: http://brando.ws/pcitlsmatrix
For those systems that you do not maintain, or those that you cannot access, it’s time to get with the vendors to find a solution. Many of them will tell you that the versions are no longer supported and you must purchase new products. Before you unleash an expletive-laden tirade on them, ask yourself how long those systems have been in place, if they have maintenance contracts on them, and how critical they are to your business. Sometimes we blame software vendors when they cannot fix our problems, but then run systems well past their expected life. I’m looking at all the Windows XP and Windows Server 2003 users out there. In the words of a great mentor of mine, you have to participate in your own rescue. Technology is a two-way street. If you have been neglecting to pay a vendor because you didn’t want to upgrade, don’t get mad at them when they tell you that the only fix is to upgrade.
For these systems, you also have a rare opportunity to widen the scope of the problem to include the business. For those of you whose arm went numb when I suggested making the problem bigger, focus less on the tactical nature of removing SSL from your environment and more on using technology as a business enabler. In non-buzzword speak, go back and ask why this legacy application is in use and see if there may be a better technology alternative that will accomplish both of your goals as well as add features for the future.
Troy Leach, the current CTO of the PCI Security Standards Council, often jokes that this whole thing started when they tried to remove three words from PCI DSS 3.0. While this is a grandiose oversimplification of what is actually happening here, it’s not terribly inaccurate. A quick search of SSL in PCI DSS 3.0 will find a handful of instances where it is present as part of an acceptable example to meet a requirement. The three main requirements that were updated to remove SSL as an acceptable technology are 2.2.3, 2.3, and 4.1.
Requirement 2.2.3 now prohibits the use of SSL as a way to secure services, protocols, or dæmons that are inherently insecure such as NetBIOS, Telnet, FTP, and others. If you were using SSL, you simply need to look at the configuration for whatever mechanism you were leveraging to secure the transport layer and modify it to only use TLSv1.2 and greater. In cases of email, it may just be a change in the dæmon configuration. As an example, both Dovecot and Postfix (popular open source email server software) allow for this configuration directly. You don’t need to leverage an outside transport layer security mechanism. If you are using sTunnel to secure something like NetBIOS, you need to modify the sTunnel configuration to accomplish this.
Next, Requirement 2.3 deals with nonconsole, administrative access. If you are in the Unix world, you are probably already using SSH so those systems are OK as is. Where you will get into trouble is if you are using any SSL VPNs to gain access into secured enclaves and then using a cleartext technology like telnet to access those systems. First off, if you are doing that, shame on you. I’m writing this in 2015, shortly after PCI DSS 3.1 became public. There is absolutely no excuse for using technologies like telnet or rlogin today. Jump on the SSH bandwagon and hopefully you will solve your issue. Keep in mind, if you are using SSL VPNs for any sort of segmentation, you will still need to adjust the configuration to make it comply with Requirement 4.1, which we will discuss later.
For those of you in the Windows world using RDP, make sure you have secured it appropriately. Early versions of their SSL security used TLS 1.0 which falls under the “you need to fix it” clause.
And finally, you mainframe (or large system) users. If you are using TN3270 with an SSL wrapper, you have to adjust the configuration to prohibit SSL and early versions of TLS. Depending on the version of software you are running, this may be an easy task or it could be something more painful. Many large systems also allow for SSH communication as an alternative to the SSL-wrapped TN3270. Large systems often sneak past the security department when it comes to security patches or software upgrades. I once did work for a company that only did updates to their large systems every six months (well, until PCI DSS came around anyway). If your large system is out of date (e.g., you are running a really old version of zOS), this is going to be terribly painful for you. Arguably, this falls in the category of technical debt, and it’s time to pay the man.
Any other systems such as networking equipment, virtual infrastructure, or purpose-built appliances will need to be checked to see if these upgrades are required. Be sure to contact your vendor for more information and get those scheduled.
Finally, we need to visit Requirement 4.1 to close out all of this SSL business. If you are up to date on your ASV scans, this should not be an issue at all for you. You probably had to deal with this before any holiday freeze last year, or you updated your configuration at the beginning of this year. If you haven’t, your QSA is going to give you a hard time when you go to show four quarters of clean scans.
There are a number of areas where Requirement 4.1 can come into play, but for SSL it is primarily going to come in five main flavors. The first would be in any SSL VPN technology you are using to secure the connections from clients or other sites back to your offices. Not all VPNs use SSL, but it is a fairly common technology to use. If you are using an SSL VPN, you need to make the configuration change to disallow SSL and early TLS. You also may have to upgrade your software—especially on the client side—if you are not current.
Next, your websites that are in scope for PCI DSS. In the previous section, I shared a few configuration changes for Apache and IIS—two very popular web servers. This should not be an issue for most of you because it should pop up on your ASV scans. If you find web servers that have not been fixed to address this and they did not show up on your ASV scans, you might want to check to make sure they are included in your normal quarterly scan process. Chances are, they were left out.
After web servers, check any email configuration you have set up. I know it’s almost ridiculous to think that this still happens, but I’ve added it in here for two reasons. SSL and TLS are a part of the email fabric, and many servers will automatically add this layer of protection for either passing credentials or for transmitting email content. There are still some data flows out there that may leverage email. If you are using one of those flows, be sure you check your email server configuration. The other reason why I bring it up here is that many companies will try to use single authentication sources for their employees either through LDAP or Active Directory. This means that your username and password may be the same for both your email account and some system that contains cardholder data. You need to be securing the transmission of those credentials, so make sure you make the configuration change to disallow SSL and early TLS.
For those of you who use IP-enabled payment terminals, check to make sure that you don’t see either SSL or early versions of TLS as the primary protection mechanism for keeping data secure. Terminals can be a bit harder to deal with, but there are a few opportunities to resolve this. You may be able to update the configuration directly, you may take this opportunity to upgrade to a new terminal to include NFC Payments and EMV, or you may qualify for the exception listed at the end of the Requirement 4.1 notes. The note says that any terminals and their termination points “that can be verified as not being susceptible to any known exploits for SSL and early TLS may continue using these as a security control after June 30, 2016.” It’s a nice exemption, but I will be interested to see how QSAs and penetration testers approach this exemption. Be wary of this exemption and only use it temporarily.
Finally, any other transmissions over public networks for insecure protocols—similar to Requirement 2.2.3. Sometimes old habits die hard, and workflows may still be using protocols like FTP to batch transfer cardholder data, and those may be protected by a transport layer SSL stream. STunnel is a common tool used to accomplish this goal. This is another one that should have turned up in your ASV scans, so it shouldn’t be that big of a deal.
There’s also an interpretation issue that I predict will pop up unless the Council specifically addresses it. With PCI DSS 3.1, we have a situation where something is permitted until the middle of 2016, while ASV scans should have been flagging these as needing to be fixed since October(ish). Which one takes precedence? Do you take your scans that are passing except for SSLv3 vulnerabilities to your QSA so he can bless you as compliant until next year? Or does your QSA point out that you have to have four quarters of clean scans, thus causing you to fail your validation assessment until you address the items?
If you remember, one of the goals for PCI DSS 3.0 was to remove ambiguity from the standard to make it easier for different QSAs to come to the same judgment after reviewing the facts. Interpretation variance is a significant issue with more complex environments. What should happen is that the QSA should accept any scans for existing systems only that are failing due to SSLv3 and reject any new implementations of SSLv3 that were put into production after April 15, 2015. And if you are an entity going through an assessment, don’t try to fudge your dates. As a QSA, I would be highly skeptical of any system that went live in April 2015, regardless of the day. Plus, even if you pull a fast one on your QSA, you won’t be able to pull a fast one on the forensic examiner that is going through your breached systems.
PCI DSS 3.1 allows for some leeway if your 3.1 assessment still covers some early TLS and SSL versions that have officially sunset. Testing procedures 2.2.3.c, 2.3.f, and 4.1.i are identical representations of something called a Risk Mitigation and Migration Plan. Any existing installations must have a plan in place to reduce the risks associated with using these older versions and show how you will migrate before June 30, 2016. Here is what your plan must contain (pulled from the aforementioned testing procedures in PCI DSS 3.1):
- Description of usage (including what data is being transmitted), types, and number of systems that use/support these prohibited technologies;
- Risk-assessment results and risk-reduction controls in place, which should use an accepted methodology performed by qualified individuals;
- Description of processes to monitor for new vulnerabilities associated with SSL/early TLS, which should be as simple as making sure your vulnerability scanners receive updates;
- Description of change control processes you alter so only accepted technology is implemented into new environments; and
- A project plan with a migration completion date no later than June 30, 2016.
Your QSA should review this plan for completeness as they must document it during your normal review.
SUMMARY OF SSL CHANGES
That’s it. One acronym removed from three little requirements. If it were only this easy. The good news is that these requirements are not effective until June 30 of 2016—roughly four months before PCI DSS v4.0 will come out. Can you believe we’re past the halfway mark in the three-year cycle?
Don’t forget, there is some painful news with this update with respect to interpretation. Does a failing ASV scan trump PCI DSS 3.1, or the other way around? Ultimately, this will be up to your QSA and/ or acquiring bank. As I mentioned earlier, PCI DSS 3.1 should take precedent over a failing ASV scan for existing implementations, with one caveat. You should work to fix any externally exposed services using SSL and early TLS as soon as you can. Once those are done, work to migrate your internal systems away from SSLv3.
Joanna just received her latest vulnerability report. As expected, their legacy order management system popped up with SSL vulnerabilities. Joanna is not surprised as this system routinely shows up on vulnerability reports. Her company inherited this system through an acquisition, and some of the business users have been hanging on to this system as it was the primary system of record for the group before the acquisition. Some business processes were never converted away from this system because of a lack of an upgrade path and the cost of adding new per-seat licensing for all of the business users. Those processes are historical in nature—no new orders are placed through this system. As a security professional, Joanna knows that this system has a number of vulnerabilities, but management has never had an appetite to migrate away from the system as it was not a priority at the time.
Joanna’s industry has had a number of public security breaches. While her firm has escaped thus far, management is starting to pay attention to security issues and their risks to the firm. Given the recent attention, Joanna knows that this is a great opportunity to approach management with a solution to migrating from this legacy ordering system and removing a major headache from her job responsibilities. Her plan has four phases.
Phase 1: Remove all PANs from legacy order management system. After explaining the risks to the company from this system, Joanna talks to the business users of the system to understand how they use credit card data. As it turns out, they no longer do any major research related to the PAN. The managers agree to alter their business process to use other data instead of the PAN for analysis, and Joanna is able to have Oliver, a DBA with some expertise in this legacy product, write a script to remove the PAN data from the system.
Phase 2: Move the system out of the Cardholder Data Environment (CDE). The system currently lives in the same network as the typical card processing. This makes it in scope for PCI DSS. Since no new orders are taken through this system, she works with Dan in her IT group to find a new logical home for this system in the network that is outside the CDE in another secured area of the network. As far as PCI DSS 3.1, she has removed the compliance risk after completing this task.
Phase 3: Migrate data to the currently maintained order system. Joanna works with Oliver to get the requirements for conversion as well as data replication for current business users to keep licensing costs low. She also works with procurement to ensure the firm has enough seats in their license to cover all of the business users of the product.
Phase 4: Decommission. After the data migration project, Joanna works with Dan to safely shut down the system and archive its data for archival purposes. Finally, Dan removes the server from production and safely destroys all of the data on the drive before sending the hardware off for recycling.
Phases 1 and 2 address PCI DSS 3.1, while phases 3 and 4 address the technical debt associated with neglecting this IT system. After completion, Joanna knows that her security posture and business efficiency have improved.
For these kinds of requirements, your future proofing will come in the form of staying on top of TLS technology and ensuring you are preferring the most recent version of the TLS protocol as well as being mindful of the methods you choose to enable for encryption. You will need to make sure that you maintain both the client and server ends of these connections as you make the appropriate configuration changes.
As you procure technology for use in your environment, be sure you are aware of the expected lifespan of that particular piece of equipment. Do they expect you to upgrade or replace it within three to four years? Be sure you include that in your modeling. In addition, ask for a Bill of Goods with your purchase so that you know any thirdparty code—especially open source—that is included as part of the products’ inner workings. You need to make sure that the vendor has a method to upgrade any vulnerable code inside these systems for the next Heartbleed or Shellshock vulnerability.
Of course, for the “over the wire” data protection component, you could always change to a different type of transport layer encryption. Both IPSec and SSH make for decent alternatives to SSL or TLS—but there is no guarantee that either is free from security vulnerabilities. In addition, any open source tools that may be deployed to enable these transports must be maintained as above (think OpenSSH or Openswan as examples). As with most security problems, stay on top of your systems and make sure you update them in a timely manner. NOTES