PCI From the Other Side: An Ex-QSA’s Worst Nightmare

Once again I have chosen a dramatic title to sucker you in. But seeing as it’s PCI related it’s never going to be even remotely exciting.

First; per the title, I’m not actually a QSA any more, but I have been in the trenches of PCI since before there were QSAs. Anyone remember QDSPs? I am therefore reasonably well qualified to write about it.

Second; by “PCI From the Other Side”, I mean that I found myself in a scenario where I was the one being assessed. The remainder of this blog is about that experience. It was truly eye-opening, and I hereby apologise to every client I’VE assessed over the last decade. I only now feel your pain.

But it’s not until you find yourself on the other side of the assessment fence that you can truly appreciate the challenges. I am both a PCI and cybersecurity expert, and even I had a hell of a time putting my organisation through the process. And I designed the infrastructure with compliance in mind!

My first challenge was finding a PCI compliant service provider (SP) who covered the vast majority of the infrastructure related processes. From configuration standards, to AV, to logging and monitoring I didn’t want to do anything in-house. I spoke to several service providers, and found myself guiding THEM in the design of their PCI services! Regardless, they were universally unhelpful, and if I had this much difficulty, what chance does anyone else have?

Even Amazon Web Services does a better job than every SP to whom I spoke. While AWS basically devolve almost every aspect of compliance back on the client, they at least break down the EXACT responsibilities for all parties. Yes, PCI DSS v3.X does a better job of making this a requirement, but it can still be very difficult to get the right information based on a vendor documentation. If you don’t ask the right questions, no SP seems anxious to provide them for you.

The second major challenge was the sheer volume of ‘paperwork’. Policies, Procedures and Standards make up roughly 35% of the PCI DSS requirements, and at least 47% of validation against all requirements involves review of some form of documentation. Even if it’s just a screenshot.

As an assessor, I would give my clients a spreadsheet that tells them what kind of document I need against any given requirement. They would then complete this with the document THEY believe meets the intent of the Testing Procedure. For my QSA I went one stage further and mapped my policies and procedures (including Section numbers!) against the Report on Compliance v3.X template itself.

Now these are Policies that I have mapped against the PCI DSS / ISO2700X/ CoBIT etc. I have even sold them to several clients to help with their compliance efforts, yet it took me several weeks get them where they needed to be. As for the Procedures and Standards, I had to create 24 separate documents to cover everything from Change Control to Vulnerability Management. This was NOT fun, or easy!

Like finding a Service Provider, I had a huge advantage over most people in charge of putting together their organisation’s documentation. If this was the pain I went through, I do not want to begin to imagine the pain for anyone not an expert. [Note: The ‘paperwork’ is critical not just to PCI, but to security in general, and should NEVER be done with just compliance in mind!]

I have written too many blogs about the problems with the PCI DSS to harp on about them here, and there were far more issues I faced than you want to hear about. Needless to say, if the SSC really want to train new QSAs, they should throw out their entire curriculum and put them in the client’s shoes for a day.

95% of them would fail.

Who am I kidding, 95% of CURRENT QSAs would fail, I almost did!

[If you liked this article, please share! Want more like it, subscribe!]

If you think I'm wrong, please tell me why!

This site uses Akismet to reduce spam. Learn how your comment data is processed.