Category: Continuous Compliance

From everything I have seen in my many years performing PCI assessments, logging is not only one of the least understood of the requirements, it is the most under-utilised, and the one that gives/gave my clients the most pain.

Logging is the most important detective security control you have, bar none, and done correctly, logging is the foundation of your incident response program. Notice I didn’t say ‘disaster recovery’ as well, because if your incident response was where it should be you should not HAVE to recover from a disaster.

The confusion stems mostly from a lack of understanding of logging mechanisms themselves, even for Windows (for which PCI was clearly written). For example, do you think that Windows logs to the PCI requirements out of the box? I did too, but have been assured that it does not. Do you know HOW to get it to log appropriately? No, me either.

I have been further assured this if you WERE to turn logging on to cover the 10.2.X requirements, the logging would be so verbose as to render the device that’s doing the logging useless. Is that really the INTENT of logging? Of course not.

Also, can syslog EVER record the events required in 10.2.x, or even the event content as required in 10.3.x? Once again, I have been told no, but I am no expert.

Yes, you SHOULD have people who DO know this stuff, but how many organisations out there can truly afford that kind of deep expertise in-house? Yes you can outsource, but where is the guidance on EXACTLY how to configure operating systems to log to the PCI requirements? It probably exists, but in 10 years of doing PCI I have not found it, and I’ve even asked ‘experts’ in the field; Security Incident and Event Monitoring (SIEM) vendors. On that note, I have yet to see a SIEM vendor also be an expert in PCI, which to me is an absolute joke if that’s why they are selling it to their clients.

We have free hardening guides for Windows (CIS Security Benchmarks for example), but where is the guide that breaks down the Windows operating system into a mapping between the registry settings for logging and the PCI DSS? Or *nix flavours, or Cisco, or AS400, or Power Series? If you have them, please share?!

So let’s, for the sake of argument, assume that you cannot reasonably log to the letter of PCI, or more to the point, you do not WANT to for usability issues. Are you non-compliant? Let me answer that with another question; What’s more important, configuring your logging to record events for a forensics investigation, or configure your logging to help prevent a breach in the first place?

If you chose the second one, you are correct, and if you also choose to maximise your logging mechanisms, you will not only be PCI compliant to its intent, you will also be doing security properly.

First, logging is not about crunching masses of data through a correlation engine, its about the RIGHT data put into a base-lined context. There is no such thing as log event correlation without a deep understanding of what the end systems SHOULD look like, AND what your normal business processes are from start to finish. In other words, tell any vendor trying to sell you compliance though their SIEM, to put it where the sun don’t shine.

While we’re on the subject of SIEMs, how many of them do you think can accept Windows logs natively, or have to convert the logs to syslog via an agent (e.g. Snare)? Very few. What’s the point of buying a log mechanism that cannot even read WINDOWS events without butchering them DOWN to syslog?! You MUST ask the right questions before buying ANYTHING, especially a SIEM.

OK, so how DO you go above and beyond? Simple, in one way;

Do NOT perform your log reviews daily (10.6), because that’s just plain stupid, not to mention impossible to do adequately. Perform log reviews in real-time via some form of automation.

The automation you need is threefold;

  1. Events you should NEVER see: Each system admin, from OS, to network device to application SHOULD know which they events should never be seen under normal operating conditions. Look for these ‘strings’ and alert immediately.
    o
  2. Events you should not see in a certain quantity and velocity (i.e. thresholds): I don’t care if I see an admin fail to log in once, I do care if s/he fails 10 times in 2 seconds (for example).
    o
  3. Quantity of events over the course of time (i.e. trending): You have to save logs for 1 year (DSS 10.7), so why not put them to good use by trending events over time? Even if it’s just quantity of event (as opposed to quantity of type of event per device), the information you get can be extremely useful.

Perform all three of these things, and you have not only covered the ridiculous ‘daily reviews’ automatically, you now have input into your incident response mechanism that gives you real security.

Choosing the right centralised logging mechanism for your business is one of the most important decisions you can make, and it cannot be done ONLY for PCI. You must buy a system that can cover you enterprise-wide, and unless you have significant in-house expertise, you must build into the RFP the requirement for consulting support, and potentially some form of on-going managed service. Nothing stays the same, so your future state / needs will also need to be taken into account.

Do NOT penny-pinch here, but don’t buy anything that’s not appropriate. Your risk assessment process should tell you exactly what you need, and if you’ve not done one, start there.

Everyone loves toys, and IT/IS administrators are no different. However, it’s a very different thing to buy a new PlayStation for yourself, than it is to spend a considerable amount of your company’s money chasing after yet another buzz-phrase or a vendor-induced panic.

Worse than this is to mis-interpret a regulatory compliance standard (like PCI) and spend all your IT budget on technology, when the vast majority of these standards revolve around policy, procedure and standards. Like security should. Behind every security control, in every regulatory standard, is an intent, and until you have examined what that intent entails for your business, you simply have no justification buying anything.

Both IT and IT Security departments have only one purpose; to enable the business’s goals. That’s it. However, it the business’s responsibility to make those goals VERY clear, and fully support IT/IS when required. This does not happen without robust Risk Management process(es) maintained by a Governance Committee of some sort.

In every organisation for whom I have provided security guidance, they had the exact same dynamic; IT/IS massively overestimated their budgetary needs knowing full well the business side will reduce it as much as possible. Usually to point of making a lot of departments ineffective in any way that matters. Then things like PCI come along and suddenly IT/IS departments have ammunition to up their budgets to meet a supposed business requirement.

The smartest managers used this money to do things properly knowing that PCI compliance will fall out the back end of a security program done well. The majority however line up behind the security vendors like sheep buying exactly what PCI says. Every vendor of firewalls, anti-virus, DLP, FIM, IDS and all the other acronyms have made fortunes while the actual security posture in most organisations has barely improved, if at all.

Any consultant worth his/her salt has stopped a client from buying technology until they are satisfied that the client has the necessary processes in place to determine the ACTUAL need, perform a gap analysis, and exhausted all other options. That consultant also had in mind that buying ANY technology comes with a whole series of post-purchase events that must be complete in order to obtain any benefit from that purchase;

  1. Can existing staff actually USE the technology, or does some / all aspects of it’s running require outsourcing?;
  2. Does the product integrate seamlessly with the existing infrastructure management systems?;
  3. Is the increased security posture in-line with its on-going cost of ownership (management metrics)?;
  4. Has any thought been given to the products life cycle and future-proofing?;
  5. Does the product scale with the business?

…and so on.

The ages old (but still completely relevant) concept of Confidentiality, Integrity, and Availability (C.I.A.) ensures that every security program follows the law of negative returns; the more you have of one, the less you have of the others. This is equally true of the complexity of your program; the more complex your program, the less secure you are.

Technology has its place, no arguing that, but only technology in a business context is sustainable.

[If you liked this article, please share! Want more like it, subscribe!]

First, let me be clear; I hate anti-virus. I guess more accurately, I hate anti-virus companies who are still making squillions peddling their no-longer-relevant wares (in my opinion).

Blacklisting (i.e. signature based) end-point protection is meaningless and almost completely ineffective against zero-day attacks. It’s a game of constant catch-up that can (and will) never be won. Yet here we are, still buying anti-virus software because we don’t know better, and standards like the PCI DSS still call for it by name instead of dealing with the actual underlying issue.

What is the INTENT of anti-virus?  According to the DSS, you should;

5.1 Deploy anti-virus software on all systems commonly affected by malicious software (particularly personal computers and servers).

…and;

5.1.1 Ensure that anti-virus programs are capable of detecting, removing, and protecting against all known types of malicious software.

Commonly affected? As defined by whom? Clearly they mean Windows but can’t just come out and say it. Yes, other OSs are becoming increasingly affected by viruses, but would you call them common? More to the point; if you had to install and maintain anti-virus on all of your *nix and Apple products would YOU classify them as ‘commonly affected’?

No, neither would I.

The intent of anti-virus is sound; do not let bad stuff run on your systems. However, if you were doing security properly, would this not be basically redundant? Even PCI includes the means by which anti-virus becomes [in my view] excessive;

  1. Security Awareness Training (Req. 12.6) – If users were properly educated, a huge chunck of malware outbreaks would not happen in the first place. If your organisation does not have a very robust program for ongoing security training, they have missed the cheapest, and most effective security control that has, and will, ever exist. Ignorance is a choice, never an excuse.
    o
  2. Configuration Standards (Req. 2.x) – In my continuing theme of never backing up bold statements with actual facts, I will pronounce that the majority of malware out there is ONLY effective because the systems on which the malware is loaded are not configured correctly. Either the hardening guides are absent or inadequate, or the ongoing maintenance of the configurations was neglected.
    o
  3. Vulnerability Management (Req. 6.1) – If all you are relying on is patch releases from your OS vendors, then you deserve what you get. Vulnerability Management is everything from Patching, to Vulnerability Scanning, to Penetration Testing, to Change Control and Incident Response. Done well, vulnerability management is the only way you stand even half a chance of keeping up with the bad guys, but something I have personally never seen done well.
    o
  4. File Integrity Monitoring (Req 11.5) – Don’t buy Tripwire (I hate them too), but figure out a way to detect if a known-good file changes in some way. I have seen a client write an MD5 recursive hash on system32 and write the results to event logs for monitoring. It was free, and effective, but required significant expertise. All you’re trying to do here is make sure things stay the same, and it almost begs the question; Why have AV at all if you have FIM? This question becomes far more relevant the more of these points you master, but I will never negate the concept / cliché of defence-in-depth.
    o
  5. Logging & Monitoring (Req. 10.x) – In my opinion, nothing in your detective security portfolio is as important as this control, and can be used to create the most effective and ‘blanket’ compensating control for PCI there is. If you know what every system SHOULD be doing, anything NOT that is something to investigate. Daily review of log files is a farce, only real-time alerts triggered by base-line deviations makes sense, and should be the top of any organisation priorities to get right. Few do, and the majority of Managed / Cloud Security Services don’t do this either.
    o
  6. Incident Response (Req. 12.9) – Why bother being in business if you don’t intend staying in business? Incident Response can prevent an event from becoming a business crippling disaster, yet, like Vulnerability Management, is almost universally neglected. Do this one badly and I for one have no sympathy.

You should notice one unifying theme across all 6 of these controls; they have a significant process component, not technology. Most security is process, and yet PCI has driven more technology spend than all other compliance / regulatory standards in history combined (yes, that’s another fact-less statement, but I would be amazed if it wasn’t true). Anti-virus vendors, FIM vendors, logging vendors (and QSAs of course) have all made multi-millions from PCI, and not one of these vendors (including the QSAs) has ever made the effort to put their products into the proper context; A business focused solution that provides true benefit. Staying is business  IS an ROI!

All 6 factors will be addressed in their own Beyond The Standard posts, that should give some indication to their importance.

OK [deep breath], end of rant (and my longest blog of the series yet)! I’m not saying don’t use anti-virus if you believe it provides true benefit, and is not a massive capital / resource drain. But do NOT do it just because PCI says you should, do NOT rely on it, and focus your efforts on the above 6 factors as they are the things that actually meet the intent.

If you are only doing PCI minimums your QSA probably has no choice but to insist on AV (especially for Windows), your job is to give them an alternative.

Behind every PCI requirement is an intent. Unfortunately the intent is almost always obscured by the level of detail and specificity within each requirements’ description and testing procedures. But it’s important to understand this concept or you’ll end up chasing rainbows.

The networking section (PCI DSS Section 1.X) is no exception, so before I can provide an above-and-beyond I have to provide a baseline.

The intent of Section 1 is basically four-fold (I’m ignoring personal firewalls and network diagrams for now):

  1. Keep the bad guys out, especially from the Internet
  2. Limit communication between trusted and un-trusted systems to what’s necessary for their business function
  3. Limit ANY connectivity to/from in-scope systems to/from the Internet
  4. Limit insecure protocols, or ensure that their use is both justified and the risk mitigated

The ONLY thing in PCI that you cannot compensate for is the retention of sensitive authentication data (SAD) post-authorisation, EVERYTHING else can replaced with other controls as long as the risk mitigated by the original control is not greater with the replacement control(s).

For example; you do not have a personal firewall running on administrator laptops. All you have to do here is restrict all connectivity to in-scope devices to a jump server / bastion host through which ALL administrative connections must pass (BTW, this works just as well for lack of 2-factor authentication too). Note: You’ll hear QSAs use the phrase; “above and beyond” here, but it’s the intent of the requirements that’s the overriding factor.

As long as you have met the intent of the above 4 bullet points, you have your baseline, all of the following therefore are above-and-beyond (AaB.):

  1. A rule-set review more frequently than twice a year – Depending on your environment, having each rule owner (which should correspond to a data and/or business process owner) confirm that all rules are still required more than semi-annually is considered AaB. This would be especially effective if the network administrators could confirm the frequency of each rules’ use during that quarter.
    o
  2. A business justification for every rule – PCI only says that you must limit inbound and outbound traffic; “to that which is necessary for the card holder data environment.”, and between ‘trusted’ and ‘un-trusted’ networks. Which means that not only can these justifications be done at the protocol level (and not at the individual IP level), but does NOT have to be in effect between trusted networks. Therefore, if you have the following in place, you have gone wayyyyy AaB:
    o
    i.    Business justification of ALL ingress and egress filtering, including that between trusted networks
    ii.   Filtering down to the IP level (not subnet blocks / services), but this can be EXTREMELY complex so a judgement call is required
    o
  3. Automated confirmation of rule set accuracy – If both firewall rule sets AND the end systems were perfectly configured, there would be no ‘denies’ in the firewall logs except that which corresponds to changes in the environment (which should have a change control request next to it), or to things that should be investigated. This can work in both directions; system configuration validation and rule set validation, but a significant base-lining effort would need to be performed and maintained by Asset Management (see PCI – Going Beyond the Standard: Part 6, Asset Management).
    o
  4. Review and baseline firewall / router traffic logs – The logging requirements in PCI refer to administrative connections to the network devices themselves, nothing in PCI says you must collect and retain traffic logs, therefore including these in your ‘daily review’ can be considered AaB. It will also be impossible to do 3. above if you don’t.
    o
  5. Network devices capable of more than stateful pack inspection (SPI) – PCI only requires that the network devices in use are capable of SPI, which can be performed at layers lower down the OSI stack than todays’ devices are capable of performing. Therefore, a network device which also performs application layer filtering can be considered as AaB in all instances except where it would be required anyway (i.e. Requirement 6.6)

A recurring message throughout the next 12 blogs (which corresponds the DSSs’ 12 sections), is the need to read the requirements VERY carefully. Good security consultants will naturally make assumptions on a requirements’ intent based upon their perception of good practice, but PCI is a bare minimum standard, and quite often allows things that make little sense (daily review of log files for example).

Most of the above AaB points are easy to achieve, and by doing so you are building a portfolio of compensating controls which can be used in places where you cannot meet the DSS requirements language.

There will be many.

For first timers to PCI; Regardless of the security posture you THINK you have, the specific controls in the PCI have always – in my experience – resulted in several major efforts in a number of areas. These should be defined up-front and worked in parallel to minimise your efforts and bring your compliance date forward.

For re-certifiers; If you didn’t put management controls around the PCI validation requirements in previous years, you will be doing much of this all over again, at least in terms of providing validation evidence.

PCI compliance is impossible without remediating the relevant action items to do so, and these will not be done until such times as a named person is assigned the task. Not a role, not a department, a person, with a due date and transparent accountability up his/her management chain.

The first step is to define what those major project are in your organisation, then assign a Project Manager to each and every one. I’m not saying you need to hire additional staff, but unless there is a named person in charge of a SPECIFIC goal, things will simply not get done. The trick, however, is to ensure that these people are fully aware of their responsibilities and action items at the very beginning, or the nasty surprises – for which PCI is infamous – will cause roadblock after roadblock.

That’s really all PCI is, a series of action items assigned to an individual, and a QSA in the wings to provide all necessary guidance to keep things moving forward. The better QSA companies understand this, and will have a well defined and proven methodology to take ANY organisation where they need to go. If that’s PCI compliance only, so be it, but a real QSA will give you the option to take the exact same programme all the way to real business-enabling security.

Generally, these are the major projects required for PCI, along with the generic role names usually best placed to own them;

  • DSS Requirements 1.x: Review of firewall / router configuration standards and rule sets / ACLs.
    Role(s): Network Administrators
  • DSS Requirements 2.x: Configuration / Hardening standards for all system types, plus business justification for all functionality (i.e. Running services and listening ports)
    Role(s): System Administrators, Network Administrators, Application Developers, DBAs
  • DSS Requirements 3.x: Encryption of data at rest (hopefully not applicable)
    Role(s): Application Developers, DBAs
  • DSS Requirements 4.x: Encryption of data at in motion – SSL / HTTPS / email etc.
    Role(s): Application Developers, Web Developers
  • DSS Requirements 5.x: Anti virus
    Role(s): System Administrators
  • DSS Requirements 6.1.x – 6.2.x: Vulnerability management (including patching)
    Role(s): [All roles, managed centrally as overarching project]
  • DSS Requirements 6.3.x – 6.5.x: Secure coding / SDLC
    Role(s): Application Developers, Web Developers
  • DSS Requirements 6.4.x: Change control
    Role(s): [All roles, managed centrally as overarching project]
  • DSS Requirements 7.x: Access control
    Role(s): [All roles, managed centrally as overarching project]
  • DSS Requirements 8.x: User ID management / Password complexity
    Role(s): [All roles, managed centrally as overarching project]
  • DSS Requirements 9.1.x – 9.4.x: Physical security at data centres
    Role(s): Facilities / Data Centre Manager 
  • DSS Requirements 9.5.x – 9.10.x: Media management
    Role(s): System Administrators
  • DSS Requirements 10.1 – 10.3.x: Logging attributes
    Role(s): [All roles, managed centrally as overarching project]
  • DSS Requirements 10.4.x: Time synch processes
    Role(s): System Administrators, Network Administrators
  • DSS Requirements 10.5.x – 10.7.x: Log collection, protection, monitoring and retention
    Role(s): [All roles, managed centrally as overarching project]
  • DSS Requirements 11.1.x – 11.3.x: Wireless access point detection, internal/external vulnerability scanning and penetration testing
    Role(s): [Network Administrators, managed centrally as overarching project]
  • DSS Requirements 11.4.x: Intrusion Detection/Protection Systems
    Role(s): [Network Administrators, managed centrally as overarching project]
  • DSS Requirements 11.5.x: File Integrity Monitoring
    Role(s): [System Administrators, managed centrally as overarching project]
  • DSS Requirements 12.1 – 12.5.x: Policy attributes
    Role(s): [Management Designate, managed centrally as overarching project]
  • DSS Requirements 12.6.x: Security Awareness Training
    Role(s): [Management Designate, managed centrally as overarching project]
  • DSS Requirements 12.7.x: Background Investigations
    Role(s): [Management Designate, managed centrally as overarching project]
  • DSS Requirements 12.8.x: Vendor management and due diligence
    Role(s): [Management Designate, Procurement, managed centrally as overarching project]
  • DSS Requirements 12.9.x: Incident response
    Role(s): [All roles, managed centrally as overarching project]

As you can see, a lot of these projects will either be multi-PM’d (you’ll have a separate configuration standard for each system for example), or SHOULD be handled centrally (no point in doing logging in any way but centrally, and as a corollary, Incident Response). Unless some of these things are centralised and maintained centrally, sampling becomes very difficult.

In PCI sampling starts out at 100%, you must earn a reduction in that. The only way to do that is to show how you have: 1) installed all systems identically, maintained them identically, managed them centrally, monitor them centrally and show this from a centralised console of some sort. More on this in later ‘Beyond the Standard’ blogs.

There may be more or less projects in your environment, but unless you engage a QSA at the beginning of these exercises there is no guarantee your hard work will get you where you need to be. PCI is simple when you know what to do, so if you DON’T know, ask.