Once again I have chosen a dramatic title to sucker you in, but seeing as it’s PCI related it’s never going to be even remotely exciting, but it may be of interest…
First; per the title, I’m not actually a QSA any more, but I have been in the trenches of PCI since before there were QSAs (anyone remember QDSPs?), so I am reasonably well qualified to write about it.
Second; by “PCI From the Other Side”, I mean that I recently found myself in a scenario where I was the one being assessed. The remainder of this blog is about that experience. It was truly eye-opening, and I hereby apologise to every client I’VE assessed over the last decade. I only now feel your pain.
The above apology aside, it’s not until you find yourself on the other side of the assessment fence that you can truly appreciate the challenges faced by those organisations with PCI obligations. I am both a PCI and Information Security expert, and even I had a hell of a time putting my current organisation thorough the process and I DESIGNED the infrastructure with compliance in mind!
My first challenge was finding a PCI compliant service provider (SP) who could handle the vast majority of the infrastructure and the security related processes. From configuration standards, to AV, to logging and monitoring I didn’t want to do anything in-house. I spoke to several service providers, for some of whom I was actually the one who had provided guidance in the design of their PCI services. Regardless, they were all SPECTACULARLY unhelpful, and if I, an expert, had this much difficultly, what chance does anyone else have?
Even Amazon Web Services does a better job than every SP to whom I spoke, and while AWS basically devolve almost every aspect of compliance back on the client, they at least break down the EXACT responsibilities for all parties against every single requirement in the DSS. Yes, PCI DSS v3.0 does a better job of making this a requirement, but it can still be very difficult to get the right information based on a vendor documentation and Attestation of Compliance (AoC). If you don’t ask the right questions, no SP seems anxious to provide them for you.
The second major challenge was the sheer volume of ‘paperwork’. Policies, Procedures and Standards make up roughly 35% of the PCI DSS requirements, and at least 47% of validation against all requirements involves review of some form of documentation, even if it’s just a screenshot.
As an assessor I would give my clients a spreadsheet that tells them what kind of document I need against any given requirement, for them to complete with the document THEY believe meets the intent of the Testing Procedure. For my QSA I went one stage further and mapped my policies and procedures (including Section numbers!) against the Report on Compliance v3.0 template itself.
Now these are Policies that I have mapped against the PCI DSS / ISO2700X/ CoBIT etc. and even sold them to several clients to help with their compliance efforts, yet it took ME several weeks to fine tune these policies to get them exactly where they needed to be. As for the Procedures and Standards, I had to create 24 separate documents to cover everything from Change Control to Vulnerability Management and I can tell you, this was NOT fun!
Like finding a Service Provider, I had a huge advantage over most people in charge of putting together their organisation’s documentation, and if this was the pain I went through, I do not want to begin to imagine the pain for anyone not an expert. [Note: The ‘paperwork’ is critical not just to PCI, but to security in general, and should NEVER be done with just compliance in mind!]
I have written too many blogs about the problems with the PCI DSS to harp on about them here, and there were far more issues I faced than you want to hear about. Needless to say, if the SSC really want to train new QSAs, they should throw out their entire curriculum and put them in the client’s shoes for a day.
95% of them would fail.
Who am I kidding, 95% of CURRENT QSAs would fail, I almost did!