1. No Leadership Buy-In
The most important component of a champions program is leadership buy-in. The sneaky blunder here is when program owners think they’ve received buy-in because they presented a fabulous proposal to their leader and then walked away with some good vibes. Reliable leadership buy-in must come in more tangible forms such as OKRs, budget approvals, time investment, and evangelization. This locks in your ability to make long term investments, and will boost your message across the company. Not securing this commitment from leaders to be executive sponsors could leave you wondering what went wrong in the aftermath of major changes. Depending on your organizational structure, your executive sponsors might be one or all of the VPs, the CISO, the CTO, or even the CEO. In your proposal meeting, try to secure an invitation to present at their next staff meeting as well, and then repeat this process until you find leaders at a level that can absorb a performance goal around the program. Also, don’t forget, if there’s a change in leadership, you’ll need to lock in new sponsors!
So, what should be in a proposal to leadership? Like any company-wide initiative, there should be a strong business case for this effort. Leaders will want to see the specific investments, and the exciting impact they can achieve with this program. The good news is that security champion programs have many exciting proven impacts on an organization! The benefits we see most often presented fall into these categories:
-
Developer Productivity
-
Vulnerability Management
-
Adoption of Security Practices
-
Customer Confidence
Don’t forget to round out your proposal with a strong story about metrics. Include both baseline research and projected metrics. To establish a baseline, take a look at existing measurements such as “# of Vulns out of SLA” and “# of Security Help Tickets”. Then translate these metrics into projections with a successful security champion program in place. Also consider metrics that track the engagement of the participants over time. Leaders need to know how much time is being invested in the program, and engagement metrics can help illustrate this. Champions are a force multiplier for change in the organization, so while they might be tracking 20% of their time on security champions’ work, the influence they have on others around them makes an impact as well that you can capture with anecdotal evidence along with your metrics.
2. No Vision
Perhaps you can relate to this story. Your CISO comes back from the RSA Conference with a list of hot new ideas including something called “security champions” that is going to solve your scaling challenges. You’ve been assigned an action item in a staff meeting after the team excitedly tosses out a flurry of ideas, and expectations are piling up. You read a few blog posts and make a plan to start a monthly brown bag zoom meeting… if anyone answers your call for presenters. Fast forward 6 months, and attendance in the meeting is dwindling and it feels like you’re spending all of your time chasing down speakers at the last minute. You’re in the weeds because you didn’t start with a clear vision.
A champion program owner will never be lonely, because they have stakeholders to communicate with in all directions! A vision statement and a charter will help keep everyone on the same page. Another related blunder here is a common one for people that like to move fast; it’s not soliciting stakeholders input during the early draft of the vision. If you send out a polished statement for input, the people you need to adopt this program will be in the mindset of agreeing or disagreeing with you, and instead you want them to help you fill in the gaps you don’t have visibility into with their own thoughts.
So, what should be in a charter document? It’s whatever would go in any charter document for a company-wide initiative! :D The point here is to make this documentation have the same look and feel as the most important initiatives in your organization. These documents serve a wide audience and give essential information to every type of stakeholder. If you ask ChatGPT what to include in a charter, she’ll say, “A charter typically includes sections on the purpose and objectives, scope, stakeholders, roles and responsibilities, governance structure, timeline, resources and budget, risks and assumptions, success criteria, communication plan, and approval to clearly define and guide the project or program.” We’d suggest that you spend the most time on roles and responsibilities. In our experience, most of your stakeholders will need this information, and otherwise won’t have any idea what you’re asking them to agree to.
If all of this seems much too official for the stage you’re in, just remember to start with the end in mind. Your vision can be as simple as describing the impact that champions could have on your organization if all goes well.
3. Promoting Negative Social Proof
Sometimes a blunder can happen with a slip of the tongue. You might be in your champions’ monthly meeting, and you say out loud, “I can’t believe only 4 people showed up for this webinar! Last month we had 20!” While transparency can be a powerful tool, what you’re communicating here is that the norm is to not show up for this meeting. Anyone on the fence about it just received social proof that they should trust their instincts next time and not show up either. Sometimes we can unconsciously sabotage participation by sending negative signals about how either we or the audience feels about it. Consider human psychology, and put yourself in the shoes of your audience.
Instead, promote good behaviors and find those silver linings. Celebrate those 4 people that show up, especially if they have perfect attendance. Capture the attendee names in the minutes of the meeting to be shared with the group later. Or change your own mindset, and embrace the idea that participation can take many forms. Track participation for the year as a whole and don’t get too fussed about numbers for a single meeting. (Easier said than done if you’ve invited a guest speaker, I know!)
4. Passing Off Security Team Responsibilities
A security champion has a unique role with unique responsibilities. Another common blunder is assuming that developers must use the same techniques as security engineers to address security issues. Sometimes program owners even assume that the champions can replace the security team in some ways. They may fall into this trap when the Product Security team becomes overwhelmed with their work and looks for creative ways to achieve their goals. We’re all about thinking outside the box here, but this arrangement often goes wrong because teams underestimate the enormous ongoing cost of training security champions to take on security practices, or simply don’t provide enough education or experience for champions to have the skills to do this well.
You’ll get better results faster by creating a custom set of practices that complement the developers’ skill set and advanced knowledge of the product to address issues in a way that security engineers cannot. For example, threat modeling templates can be customized for a developer, and reveal firsthand architecture information that would take weeks of research for a security engineer to derive from scratch. The end impact is the same: revealing weaknesses in a design before it goes to production. This solution acknowledges that developers have different goals; they want high quality code and security is a part of that with a different priority.
But admittedly sometimes you don’t get to pick your origin story, and leadership is really expecting the security champion program to make a visible impact on taking vulnerabilities off the security team’s plate. In this case, here are a few pieces of advice:
-
Set expectations early and often that this process will take years to build and mature.
-
Show tiny incremental improvements every quarter. Pick one scenario at a time.
-
Look for a small subset of volunteers to take on advanced security practices and build up a few people at a time. Reward this group heavily (in the style that they most appreciate!)
5. Nothing In It For Champs
We should take a moment here to acknowledge the wonderful people that volunteer to make their product, and in some ways the world, more trustworthy. These people deserve recognition, but sometimes we commit the unfortunate blunder of overlooking this. Volunteers can be motivated by a variety of intrinsic and extrinsic drivers, but loss is not one of them. You can’t force a volunteer to participate (even when they’re voluntold) with punitive rules. In most cases, there’s an assumption that there needs to be an expensive prize to motivate participation, but also in most cases, that’s not possible because budgets are stretched thin as it is. Here is where we should look for creative solutions to work around a tight budget.
It is also a blunder to assume that everyone likes the same things. There are several ways to offer a variety in rewards and recognition that motivate your champions to participate. You could do a persona motivation analysis and find characteristics of your champions that they have in common. You could offer various reward types and see what the response rate is for each one. You could outsource the reward program to a third party and award points to a system that lets participants choose their preferred reward from a store. It’s better to have too many choices than too few, both with ways to participate and with recognition.
The reward and recognition structure should be clear and visible, with a perception of choice in both the work and the reward. One thing to be wary of is that when you create a list of rewards that have monetary value, you create a transactional mindset where people will ask themselves, “Is this gift card worth my valuable time?” If the time involved is minimal, or the reward is unexpected, then perhaps it very well may be worth it. Every person’s preferences could be different. Consider using the SAPS rewards model to tailor the reward to the person you are appealing to.
6. Not Evolving Over Time
Imagine you’ve discovered a new show with exciting mysteries and new characters to follow on their journey. Now imagine your disappointment next season when the channel just repeats broadcasting season one. You might leave the show on in the background while you work, but the rapt attention is gone. Or imagine you’re working on an exciting new project. You spend months getting to the finish line with your team and you’re proud of how you helped your customers. (You see where I’m going here…) Now imagine that instead of moving on to a new high-profile challenge, you are assigned to the exact same project again as if the win never happened. Total disappointment. Similarly, a common blunder that program owners make is not evolving the program over time to maintain their champions’ attention and grow with them.
One way to structure a champion’s experience to evolve over time is through a “leveling system.” You may have seen a level system with colorful belts if you’re trained in martial arts. Students are given a white belt to wear in their beginner classes, and then when they accomplish a list of skills they pass on to the next level. The color of the belt represents the accumulation of specific accomplishments and this is well understood by everyone in the same art. For a security champion program, be sure to learn from this popular system and make your levels enriched with accomplishments that define the champion journey.
You may incorporate a points system to hook the participants, for example by assigning a few points in the beginning for making comments on Slack. Over time the motivation to participate in conversations on Slack will evolve beyond caring about points and becomes a higher calling to help their fellow champions succeed.
Even without a leveling system, you can still evolve your champion program by shifting the activities over time from seeking participation through extrinsic motivations (e.g. curiosity, prizes) to intrinsic motivations (e.g. mentoring, access to challenging work). While program owners often start with “stuff” as rewards, this extrinsic motivation is short-lived as it becomes less special, while intrinsic rewards are special much longer because of the value they give it themselves.
Quality content is essential to a thriving community, and not putting effort into producing content that's relevant and interesting to the audience can be a big blunder. Content can be as simple as sharing a meme on Slack or as complex as a month long awareness event. Security related content educates but also builds trust and transparency. Host a “lunch and learn” series about various aspects of how the security program works. As much as you are able, be transparent about security incidents and discuss root causes in a blameless manner. If your organization has a Red Team, be sure to coordinate with them on communications and partner on shared goals. Don’t forget to get feedback after the meeting on how to improve the content for next time.
You know great content when you see it, and vice-versa, if a story is boring to you, it will be boring for your audience! Occasionally, we’ve been guilty of tossing a link to a video in our champions community Slack channel that drops like a stone. It can sometimes feel like you’re feeding coal into the content engine, and there’s no time to craft clever posts that go viral. We recommend creating a structure for sharing content that has a reliable variety of sources, such as industry news and newsletters. Don’t forget some of the less glamorous content such as program announcements in the monthly meetings. Crowdsourcing is a handy tool too. Recognize your vocal champions who are always known to have insightful posts or are generous with knowledge sharing during brown bag sessions.
7. Poor Content
8. Not Allocating Resources
You shouldn’t start traveling on the Oregon Trail without strategically loading up your wagon with supplies, and similarly with champion programs you must gather resources and teammates before you begin your journey or face dire consequences.
Security champion program budget should include program management, activity resources, rewards, and tools. As with any program, you can build it quickly or cheaply, but probably not both. If budget constraints are a concern, there are ways to use free content, manual processes, and intrinsic rewards but they take substantial time to develop. A blunder we’ve seen in practice, and have made ourselves in our enthusiasm to launch quickly, is not matching the size of the initial program to the size of the resources allocated. There’s nothing wrong with starting small by recognizing just a few people and growing organically over time.
Perhaps the most important resource of all is time. These initiatives should be monitored over time and not simply “thrown over the fence” at the engineering team. For example, a popular component of a champion community is a Slack channel. But it will have an overall negative impact if that channel is left over time to be a home for crickets. Deploy a variety of techniques to keep the conversation lively in the channel, and commit to keep doing those things (or evolving new things) on an ongoing basis. In a thriving community, delegates will emerge who can take on some of the responsibilities of keeping the conversation going. (A good opportunity for their growth as well!)
9. No Metrics
Not having metrics to back up your program’s success story isn’t always a blunder. Perhaps you’re not at this phase of maturity yet as an organization. But for those program owners that are feeling the pressure to come up with perfect performance metrics, the biggest blunder you can commit is overthinking it! This leads to a phenomenon we’ll call “metrics paralysis” where you haven’t been measuring anything because you’ve been afraid to report numbers that don’t look like what you think your leaders want. Metrics are simply a signal that you’re achieving your goals. Since you’ve gathered executive buy-in on your vision and goals, this should be no problem.
Stick to only a few effective metrics that best demonstrate the health of your program. Having a metric for every little thing is only confusing to senior leaders who want to hear about the big picture. Choose measurements that align with company goals and OKRs.
When it comes to metrics and performance indicators, they’re as different as snowflakes from one organization to another, and depend on the ultimate purpose and goals of their programs, but here are some themes that we’ve seen:
-
Application Security Practices - e.g. # of Threat Models Performed, # of Vulnerabilities Closed Within SLA, % of Security Posture Report Card Complete
-
Education - e.g. % of Champions Completed Training, Avg quiz score >80%, # of Workshops Per Year
-
Engagement - e.g. % of Active Champions, Increase # of Recognition Points Earned per Month, # Issues Reported to Security, % Survey Satisfaction Score (NPS Score)
Whether or not you have metrics, you should be able to tell an impactful story to your leadership about the value being returned on their investment. To do this, you need to identify what’s changed (for the better) since you’ve started. Consider the difference between causation and correlation with these changes. Causation is often impossible to directly show from a champion program, but there will be many examples of correlation between teams with a champion and a positive metric. Tell this story often so that it’s top of mind for your leaders that security champions are a force multiplier for their goals and the more they invest in this program, the more impact they will achieve in return.
10. Not Having Fun! 🤪
Champion program management is a ✨creative✨ process! Your champions can tell if you’re not enjoying it and that feeling is contagious. We see this unfortunate blunder time after time. On a practical level, why should the champions care if you don’t? This concept is so important because it greatly affects your chances of success. Joy is an effective behavior change technique, but only if it’s being prioritized.
Some program improvements you can make to kick the fun up a notch include:
-
Look for delegates who are enthusiastic about the program to join the committee
-
Crowdsource ideas and incorporate surprise and delight into them
-
Embrace gamification by applying creative motivational techniques from the games industry. This doesn’t necessarily mean you have to turn the experience into a game, but this may also be effective - you could make the program operation tasks into some sort of game, for instance.
-
Form a few large teams, have long-running participation opportunities, and post a leaderboard with a prize at the end of the year
The point is, if you’re not excited about it, your champions won’t be either, so design something you actually ARE excited about!
Conclusion
Ten different blunders may seem like a lot to prepare for. You may be curious to ask if there’s a priority order to this list and if so, which ones can be safely ignored until later. The answer is none; these 10 missteps are all equally important to prevent. We’ve seen every one of them individually be the root cause of the eventual failure of a program.
The good news is that a blunder is also simple to avoid with preparation and knowledge. These are functions within your control, and planning with a strong foundation will lead to a successful program. If you’d like some assistance with the planning, design and program management for any type of champion program, reach out to us to learn more about Katilyst!
Through a combination of conversations and firsthand experience, we have developed a sixth sense about the mistakes that are most commonly made over the lifetime of a champion program which lead to failures. In some cases, the circumstances surrounding a program are beyond anyone's control, but many times we see a programmatic failure that is fully preventable with the right knowledge. These are the blunders that we’ve also made ourselves over the years, and we share the most common ones with you now in the hopes that you’ll be able to avoid these mistakes to maximize your chances of success in the future.
Top 10 Security Champion Program Blunders
Written by Marisa Fagan
Marisa is the Head of Product at Katilyst, and has built and scaled multiple Security Champion programs in her career, some with hundreds of active and engaged champions.