Change Your QSA

Too Scared to Change Your QSA?

Or perhaps the question should be; “Can’t be bothered to change your QSA?

Or an even worse scenario; you know you can’t change your QSA because the new one might discover things you’ve been hiding from the last one!

I can almost empathise with the first two, but if it’s the third scenario you deserve the bad things that will happen when you get breached.

The fact is, if you have been working with a good QSA, not one of the challenges I list below will apply to you, Changing QSAs, or even QSA companies will not be an issue. You will have been doing security properly, and not just faking compliance.


A significant number of organisations are faced with at least 1 of these 5 main challenges;

  1. Lack of Continuity – Employee attrition is inevitable. QSAs have historically bounced from one QSA company to the next following the money. Which has been abundant for almost a decade now. This has left many clients in the unfortunate position of having to start all over again with another QSA. Often one who has received little to no hand-over.
  2. Lack of Guidance – This is the QSA’s only real job. Other than writing the second half of the Report on Compliance, ALL of the remediation work belongs to the client. The role of the QSA is to ensure that the client NEVER hits a roadblock. QSAs are supposed to have ‘been-there-done-that’, so “What’s next?” should never be a question the client has to ask.
  3. Inconsistent Opinions – Every security consultant has a different skill-set. Some are network wizards, others know encryption, most should be very familiar with policies and procedures. What happens when your last QSA agreed something that your current QSA won’t accept? Who is accountable for the loss of resource and/or capital cost?
  4. Starting Over Again Every Year – Too many  QSAs are ‘just QSAs‘, with little experience or capability in fitting PCI into sustainable security processes. If your PCI compliance looks like an annual project, it is. Validation of compliance should be a simple process that should fit neatly into your BAU program. If it doesn’t, there’s a very good chance your QSA is at least partially to blame.
  5. Black-Hole Communications – If you expect your QSA to be a project manager, you have completely misunderstood the dynamic. But if you expect them to respond to emails requesting guidance in a timely fashion, you should. A QSA is there to tell you everything you have to do to achieve compliance, that’s their job, they must be readily available.


OK, so those are all the bad things, how do you fix them? Easy, choose the right QSA in the first place!

Facetious yes, and likely a moot point, but it’s never too late to change:

  1. For Lack of Continuity – All good QSAs have a methodology; Have you seen it? Does you QSA even have one? If the answer is no, you don’t have a good QSA. Continuity is simple, it just requires discipline, and a plan.
  2. For Lack of Guidance – As stated above, this is the QSA’s only purpose, if they can’t provide it, find someone who can. Interview your QSA(s) before letting them onsite, but have your questions prepared. Insist on having access to QSAs suitably qualified in ALL 12 DSS Requirements. You’ll never find this in a single QSA, unless it’s one of the 3 that I know that come close (I’m not one of them).
  3. For Inconsistent Opinions – Agree a process whereby the QSA company accepts mitigation plans or compensating controls, not individual QSAs. Agree, in writing, that ALL QSAs they send will accept a company approved option.
  4. For Starting All Over Again – This is as much your fault as the theirs. If you had a security program in place that appropriately covered your business, PCI would fit into it, not the other way around.
  5. Black-Hole Communications – Vendor Due Diligence + Service Level Agreements + Vendor Management. Period / Full Stop.

Changing QSAs every few years is a best practice, you should ALWAYS want fresh eyes on such a critical process. If changing your QSA is too difficult or inconvenient, it says a lot about both your current QSA, and your organisation’s attitude toward security;

They both leave a lot to be desired.

Here’s some old guidance I threw together a while back; Selecting the Right QSA for Your Business

Like this Article? Don’t forget to subscribe!

It's Not MY Report on Compliance!

I’m Just the QSA, It’s not MY Report on Compliance!

If you have ever been on the receiving end of a PCI assessment, you had one of two reactions to this blog’s title. You said;

  1. “Yes it is, that’s what I hired you for!”, or;
  2. “Damned right it’s not yours, the QSA is only here to validate it.”

95% of you are likely in the first group, unless you had someone like me as your assessor. It is not the QSA’s report, it is yours! The QSA is only there to:

  1. confirm that you have completed your parts of the Report on Compliance’s (RoC) Executive Summary (Sections 1 – 5) correctly;
  2. edit the QSA relevant sections;
  3. document the validation results in Section 6 – Findings & Observations; and
  4. validate the evidence you provide, and for which you are entirely responsible.

A QSA will likely never know your environment as well as you. So if you don’t take FULL responsibility for the contents of your RoC it will be your organisation that it liable for any mistakes, not the QSA. You will also then have absolutely no remedy if you are breached, as your forensic investigation will expose significant differences between the RoC and reality. This is also why you should never, EVER, hide anything from your QSA.

PCI is too often seen as an audit (it’s an assessment), and the QSA an auditor (s/he’s an assessor) and volunteering information is considered a no-no. I have actually had a client say; “But you didn’t ask me about that!”. I always try to explain that I’m a consultant first and there to help. I can’t help if I don’t have all the information. But if I do find out that they’re hiding something from me, any sampling privileges are now out the window.

That’s one of the differences between clients who use their PCI budgets to spend on securing the business, and those who only care about tick-in-the-box compliance. The first type will spend far less in the long run, even if the process does take longer. Not only that, they will likely not only STAY compliant, they will have actually protected their business …their ENTIRE business.

Setting PCI compliance as the end goal is like telling your kids to aim for a C average in school. Even the Card Brands and the SSC themselves have only ever said the DSS is a “minimum set of security controls”. So why would a QSA, whom you have hopefully chosen well (see Selecting the Right QSA for Your Business), take any ownership in a process where the goal is almost never fit for purpose?

Anyone who thinks that the PCI assessment process is structured, formal, and conducted using well established parameters has never been through one. Every good QSA does their own internal Risk Assessment from day 1, and based on their gut instinct, will determine whether or not validation sampling is even an option. If I don’t trust you, you stay at 100%.

Want to get some benefit from a PCI assessment?:

  1. Choose the right QSA;
  2. Tell them EVERYTHING; and
  3. Take FULL ownership of both the process and the output.

It’s your RoC, accept it.

[If you liked this article, please share! Want more like it, subscribe!]

PCI From the Other Side: An Ex-QSA’s Worst Nightmare

Once again I have chosen a dramatic title to sucker you in. But seeing as it’s PCI related it’s never going to be even remotely exciting.

First; per the title, I’m not actually a QSA any more, but I have been in the trenches of PCI since before there were QSAs. Anyone remember QDSPs? I am therefore reasonably well qualified to write about it.

Second; by “PCI From the Other Side”, I mean that I found myself in a scenario where I was the one being assessed. The remainder of this blog is about that experience. It was truly eye-opening, and I hereby apologise to every client I’VE assessed over the last decade. I only now feel your pain.

But it’s not until you find yourself on the other side of the assessment fence that you can truly appreciate the challenges. I am both a PCI and cybersecurity expert, and even I had a hell of a time putting my organisation through the process. And I designed the infrastructure with compliance in mind!

My first challenge was finding a PCI compliant service provider (SP) who covered the vast majority of the infrastructure related processes. From configuration standards, to AV, to logging and monitoring I didn’t want to do anything in-house. I spoke to several service providers, and found myself guiding THEM in the design of their PCI services! Regardless, they were universally unhelpful, and if I had this much difficulty, what chance does anyone else have?

Even Amazon Web Services does a better job than every SP to whom I spoke. While AWS basically devolve almost every aspect of compliance back on the client, they at least break down the EXACT responsibilities for all parties. Yes, PCI DSS v3.X does a better job of making this a requirement, but it can still be very difficult to get the right information based on a vendor documentation. If you don’t ask the right questions, no SP seems anxious to provide them for you.

The second major challenge was the sheer volume of ‘paperwork’. Policies, Procedures and Standards make up roughly 35% of the PCI DSS requirements, and at least 47% of validation against all requirements involves review of some form of documentation. Even if it’s just a screenshot.

As an assessor, I would give my clients a spreadsheet that tells them what kind of document I need against any given requirement. They would then complete this with the document THEY believe meets the intent of the Testing Procedure. For my QSA I went one stage further and mapped my policies and procedures (including Section numbers!) against the Report on Compliance v3.X template itself.

Now these are Policies that I have mapped against the PCI DSS / ISO2700X/ CoBIT etc. I have even sold them to several clients to help with their compliance efforts, yet it took me several weeks get them where they needed to be. As for the Procedures and Standards, I had to create 24 separate documents to cover everything from Change Control to Vulnerability Management. This was NOT fun, or easy!

Like finding a Service Provider, I had a huge advantage over most people in charge of putting together their organisation’s documentation. If this was the pain I went through, I do not want to begin to imagine the pain for anyone not an expert. [Note: The ‘paperwork’ is critical not just to PCI, but to security in general, and should NEVER be done with just compliance in mind!]

I have written too many blogs about the problems with the PCI DSS to harp on about them here, and there were far more issues I faced than you want to hear about. Needless to say, if the SSC really want to train new QSAs, they should throw out their entire curriculum and put them in the client’s shoes for a day.

95% of them would fail.

Who am I kidding, 95% of CURRENT QSAs would fail, I almost did!

[If you liked this article, please share! Want more like it, subscribe!]

Want to Save Money On PCI Compliance? Don’t Cheap Out On Your QSA.

Analogy: A family member needs surgery, and you have two doctors in a side-by-side bake-off. One is respected, enormously experienced, and expensive. The other is fresh out of residency, inexperienced, and cheap.

Whom do you go for?

Unless you’re a sociopath, you pay for the one with the greatest expectation for success. So by a similar (though far less life threatening) extension, why would you cheap out on your choice of QSA? Or any consultant that matter?

Not only that, you probably expect the same results from every QSA, right? They all went through the standard training, so they should all be the same, right?

Are all doctors the same?

Like any profession, you have a MINIMUM standard to achieve before you start. For QSAs it’s 5 years in security (no-one lies on CV’s/resumes, right?), OR a CISA/CISM/CISSP (anyone can read a book and pass a multiple choice test), AND pass the QSA test. I can, quite literally, take ANY person and get them to a point they can pass that test in one week.

Instead of focusing the QSA test on their domain knowledge (networking, encryption, policy formation etc.) it focuses on merchant / service provider levels and a bunch of other stuff that does not test the consultant’s security or auditing skills in any fashion that makes sense to me. Can they read a firewall ruleset to determine if they have met the intent of requirements 1.X? Can they look at a netstat and see if their OS configuration standards are being followed per requirements 2.x?

The answer to those questions is; not necessarily, and while I cannot think of one security consultant who is an expert on all 12 DSS sections (I suck at encryption and anything to do with coding for example), you need someone with real-world experience to measure your compliance against not only the standard, but its intent. And if that intent does not align with the goals of the business in question, the process falls apart.

When it comes to PCI, you’re paying for experience / guidance / been-there-done-that, otherwise you’re better served doing it yourself. At least you know the business better than the QSA ever will.

I wrote something resembling a white paper on Selecting The Right QSA For Your Business a few months ago, and will be building on this process over the next few months. Anything is simple if you know how to do it, but that’s the point; YOU probably don’t know how to do PCI, nor would you then know the right questions to ask to find someone who does.

This may sound like I’m trying to push you into hiring only the expensive guys, but that’s not it, it’s never just about the money, it’s about VALUE for, and appropriate USE of, money. The issue most often is that businesses choose their QSA based on price. They didn’t want to do PCI compliance in the first place (believe me, no-one WANTS to do PCI), and therefore settled for the lowest bidder.

In my fairly significant experience, the cheapest QSA up front rarely ends up being the cheapest in the end. These are the top 5 things to watch out for, and reflect the SOPs of some of the less scrupulous vendors;

  1. Scope Creep – A proposal written in such a way that you THINK you’re buying what you need, but you end up having to buy additional services from them to finish the job;
  2. Cheap Labour – You get what you pay for, and if you pay pennies, you’ll get the least experienced QSA at their disposal (this one serves you right by the way);
  3. Pushing Other Services or Products – Some of the larger QSAs have entire suites of products and services they try and push your way. They will sell the QSA for cheap hoping to massively up-sell/cross-sell the more profitable managed services / products etc. This is permissible under the SSC regs., but hardly best practice, and in some cases even ethical, especially when the products don’t even support your compliance;
  4. Lack of Appropriate Guidance – Achieving PCI compliance the first time is a project, staying complaint is a process. At no time during the assessment should there be roadblocks that are a direct results of the QSA’s inexperience. Projects that should take months often take years, and the additional costs can be significant;
  5. The True Cost of Compliance – Usually the most significant cost of a PCI project is the labour cost of internal resources. Performed correctly, PCI can have significant benefits in terms of improved security posture, but unless the resources are used efficiently, the cost to the business can be very significant, especially in terms of availability for initiatives related to transformation or innovation.

In the end, you will get what you pay for, and if you have not chosen your QSA based on best-fit, you deserve what you get. Choosing a QSA / consultant is relatively simple, and I believe that It Takes A Consultant, To Hire A Consultant.

If you need help, do your homework, then ask the opinion of someone with zero vested interest.

[If you liked this article, please share! Want more like it, subscribe!]

Heads-Up to the Just-QSAs, It’s Time to Diversify

There are 2 types of Qualified Security Assessor (QSA):  The ‘also-QSA’, who was a security consultant long before PCI, and had performed much of the work as detailed in the Security Core Concept blogs.

Then there are the ‘just-QSAs’ who managed to read a book, pass the CISA/CISM/CISSP exam, and qualify for the ever-so-difficult QSA training. They have delivered nothing but PCI ever since.

In case it’s unclear; first one good, second one bad.

Well, bad for you if you’re one of the ‘justs’, and bad for your clients if you’re all they have to rely on.  You won’t learn how to do security properly, or be able to provide consulting services regardless of the data type, compliance regime, or industry sector. Your clients will never get anything other than tick-in-the-box compliance.

PCI has a shelf life, and I imagine that at the current rate of payments innovation, you have only a few years to diversify. After that there is no way you will be able to maintain your current compensation package. Without some significant experience in non-PCI security areas, your usefulness is limited.

Progress will be difficult if you work for a ‘just-QSA-Company’, because you HAVE nothing else to do. You may want to seriously consider working for a security company that’s in the ‘also’ category.  There are many.

You probably have a training budget too, so spend it.  ISO Lead Auditor, CIPP/X, CLAS (UK), ITIL, Prince II, while not all security specific, are most certainly relevant. Relevant to providing the kind of  guidance that is in depressingly short supply.  If you can use this training to your current company’s benefit, great. If you can help them design non-PCI services, you are way ahead of the game.

However, there is a better than average chance that your career preferences will fall more on one side or the other of ‘business-focused’, or ‘technical-focused’. It’s important therefore that you NOT try to embrace all 6 core concepts at once.  Even security experts need to specialise.

What we are all working towards is an understanding that IT and IT security are business enablers, not a roadblock. PCI is ‘just an expense, with limited to no return on investment’, or at least thats how it is mostly seen.  Our job is to put security into a business context so that the benefits are clear to every level of the organisation.

The CEO cares about the bottom line, s/he does not care about the detail until that detail gets in the WAY of business.  This is why there is so little management buy-in when it comes to security and compliance.  If we can show that a well run IT infrastructure enables business transformation, innovation, enhanced efficiency and so on, we’ll have demonstrated our worth.

THAT’S our job, not just protecting credit card data with a minimal set of security controls to which you had no input.

The fundamentals of security have never changed, and won’t any time soon. So if you take the time to get back to basics, you’ll future-proof your career.

Security is simple, it’s not easy, but it is simple.