With mountains of data at the average organization’s disposal these days, getting the most out of that information plays a big part in meeting both short and long-term cyber security objectives. However, for many training program administrators, the only way to gauge success has historically been through basic, built-in stats, such as whether or not a course has been completed, global pass/fail rates, and so on.
While it is important to look at whether a course was started, completed, passed, or failed, it only tells you the most basic information. There are dozens of other data points that organizations can utilize to better determine knowledge gaps, pinpoint areas for optimization, and fuel improved overall decision-making.
This, in a nutshell, is the powerful effect of personalized reporting.
This blog post will explain the difference between generic cyber security data and the granular details you need to succeed. It’ll also walk you through the key components your first personalized reporting dashboard should contain and propose a few ways you can use them to their fullest potential.
The Difference Between Generic and Personalized Reporting
Let’s be clear. “Has a course been completed?” is a crucial metric to track, but it’s not the best tool to use when assessing whether or not end-users have absorbed the content.
By the same logic, this type of high-level data point won’t tell you if content taught in the course shrunk cyber security knowledge gaps and targeted the right user behaviors. As a result, it can be a challenge to answer a simple question like, “Should I hold these courses more often?”.
Whether a user passed or failed a course doesn’t really tell you anything, and that’s pretty much the depth of information generic reporting can provide.
You want to know how many failed in a specific region or department of your company. Or if they exhibit behaviors that could lead to future data breaches. Or even something as simple as knowing how far along specific users are in their courses so you can personalize the reminder email you send them.
These are questions you can only answer with targeted, personalized reporting like the features we’ve added to our platform.
Building Your Personalized Reporting Dashboard
While there are some general themes every organization faces, your cyber security challenges and goals are probably pretty unique compared to your peers.
With that in mind, the following questions are an excellent place to get started to build your dashboard. It would help if you took the time to dig deep into your organization’s cyber security reality and tailor them to your needs. This process will help you make the best dashboard possible.
Which user behaviors is your organization targeting with training courses and phishing simulations?
Personalized reporting provides you more structured data, but it primarily allows you to identify the most problematic behaviors in your organization and minimize the data breaches that could be related to these behaviors. Not only will it make it easier for you to organize your future initiatives, but it will also help you focus more attention on the issues that matter.
Of those behaviors, which ones are tied to your biggest security awareness training priorities?
The most important user behavior to target, whether it’s because it’s the most prominent or one that was exploited in a recent cyber attack, should be the first one you include in your dashboard. Your reporting setup should also have as many data points as possible, with improvements monitored and displayed in real-time.
Which initiatives will you/have you launched that directly support those priorities?
Mapping all the different courses and tests currently in progress is a good first step when initially building your dashboard. Each initiative can then be linked to a relevant set of statistics you want to monitor to achieve a draft version of the dashboard you can start monitoring to see if it fits your needs.
What are the goals for each initiative (e.g., training course, simulation, ongoing campaign)?
You don’t monitor a new course the same way you monitor a targeted simulation or a campaign you hold all year long. Ensure every initiative has set data monitoring parameters for collecting and refresh the information displayed. This will help to save you time overall.
Concerning those goals, what metrics are most critical in determining each initiative?
After you’ve done each of the steps we mentioned, then you’re ready to start digging down into which specific metrics would be the most relevant to track. While it may seem like a lot of high-level steps to take into account before determining metrics, this process will make your life much easier.
The main reason for putting these guidelines in place is that, while this might be your first dashboard, it certainly won’t be your last. You’ll probably need to come up with a new dashboard soon, and having this plan will make building future iterations way faster.
The Key Components of a Personalized Reporting Dashboard
As we mentioned earlier, there are at least three aspects we think should be a part of every cyber security training dashboard:
Security awareness training courses
Make sure you don’t just follow fail/pass, look into specific modules or quizzes. If a particular question is really important to you, you can monitor it directly. A bonus is that this data can be fun to share with your users in a newsletter (i.e., “Congrats everyone, you all passed the latest security awareness training course!”).
Filtered data breakdowns
This is more useful for larger companies, but it can also be insightful to delve deeper into a specific department or region’s performance. Maybe your developers use specific software that is more prone to cyber security risks. Or perhaps one of the countries you have users in has a new hack only present in that location. This type of data filter will allow you to remain one step ahead of these issues and keep your users better informed about potential risks.
Personalized Reporting: Next Steps
At this point, the first iteration of your dashboard may be done, but the data it delivers isn’t set in stone. These metrics will likely change over time, and there’s a possibility those evolutions will occur frequently.
Once you have your dashboard set up and everything seems to be running smoothly, build a new checklist to answer the following questions:
- Are any new cyber security goals or essential metrics not represented by a
widget or dashboard?
- Of your existing reporting assets, is any data being collected no longer relevant
or directly supporting a training initiative or goal?
- Is any of the data being collected too general, inaccurate, or incomplete?
Again, these three questions are the essential housekeeping items you can consider regarding your dashboard but don’t be afraid to add more questions to the list based on your organization’s reality.
The main takeaway is that you should be assessing the efficacy of your dashboard often and continually improve it.
A Dashboard Is Just The Beginning
You run dozens of courses, tests, and campaigns every year for many issues and potential dangers, why are you relying on the same metrics for all of them? There’s just no way analyzing them with one size fits all metrics is a good use of your time.
However, there is such a thing as too many metrics. Even if the type of plan outlined in this blog post works perfectly for you, make one before building your first dashboard. It’ll make your life easier, and it’ll bring you one step closer to your ultimate goal: protecting your users better.
Download the Whitepaper
For more information on how personalized reporting data can be an amazing, data-driven difference-maker for your organization’s security awareness training program, download the complete whitepaper!