Quality & Reporting

One of the things I’ve learned in 40 years of consulting is that appearances are everything and nothing, particularly in substantive reporting. A book can be judged by its cover, in at least some of the cases. Which means you need to know the difference, or you might be paying for junk, and worse, relying on junk to make risk decisions.

Reporting is what you pay for in consulting. We’re not providing product, and we’re not configuring systems. We’re listening, analyzing, researching, and documenting. We might do it verbally, but in almost every case, eventually there’s a report produced.

If you’re smart, when you’re contracting with consultants, you examine sample reports and ask questions and make sure you have your requirements defined and that their report will cover what you need. Before you sign the contract. You have been advised.

The funny thing about reporting is that there’s a spectrum of possibilities. It could be garbage or gold, and it could look like a piece of art or something you’d expect in a business writing course with a D- and lots of red ink on it.

It’s the stuff that looks good but has no substance that is the real problem. Because companies think they got good advice. I’m not only referring to the report looking respectable, I’m referring to the content. Because if it looks like junk, you’re going to take a closer look at the content and probably discover if it is junk too. But if it’s slick, with fancy graphics and charts giving you the wrong advice, can you detect it?

It is the sneaky ones, the ones that look good and have zero useful content, or worse, BAD ADVICE, are the dangerous ones.

In an organization with limited ability to ascertain the quality of the contents of the report, which is most businesses looking at a cybersecurity report, the risk is that decisions get made based on poor advice from this report.

How do you protect yourself from this situation?

The easiest way is to get a second opinion. Have it proofed by someone you trust that you know understands the material. Everyone should have that level of contact in their circle of acquaintances. Even if it costs you $2000, have someone that counts as an expert validate the report.

Another alternative is to have your people validate it, but this may be impossible, or cost prohibitive.

It’s not the sort of thing you need to do all the time, especially given the level of trust most organizations have for the people they work with, but it is something to consider when dealing with new organizations, or if you’re just that serious about quality control.

One final note, a firm’s reputation has little or no bearing on the output it provides in any given situation, because it’s isolated and the repercussions to the business are very light. BigTwelveCyber doesn’t notice if one of it’s consultants churns out a junk report unless the client complains, and it happens. So do your diligence, and if you can’t validate reports internally, have someone else help out and validate it for you. Nobody wants to be the next headline, and the cost is trivial.