Protection Poker: An agile game for mitigating risk

Unfortunately for the security of our applications, we usually don't "build security in" with development; instead we wait until the last possible moment and slap in security measures, typically forming a police-type security department and creating adversarial feelings between the security folks and the developers.

There are lots of great security resources, ranging from tools for DevOps teams to thoughtful analysis of how DevSecOps is changing security. It all boils down to where in your development cycle you do security. If your team is implementing a shift-left approach, you already know security needs to be top of mind before any code is written.

Why we need to build in security

Security threats are very real—Verizon's latest data breach report cites 2,216 confirmed breaches and over 53,000 cybersecurity incidents occurring in 65 countries in the 12 months ending March 2018.

Consider these two major breach reports from September 2018:

• Facebook data breach: According to Facebook, attackers exploited a vulnerability in the website's code to access 50 million accounts in a monumental security breach that enabled hackers to see users' personal info, photos, and even private messages. The hack specifically impacted the "View As" feature, which lets users see what their profile looks like to other people. Hackers used this feature to steal Facebook's access tokens, the digital keys that keep you logged into Facebook so that you don't have to re-enter your password every time you use the app.

• Marriott/Starwood reservation data breach: A half-billion Marriott guest records were reported stolen in a Starwoods reservation data breach, with unauthorized data access dating to 2014. The breach caused Marriott and Starwood customers to join the 34% of U.S. consumers who experienced a compromise of their personal information in the year ending July 2018.

No company wants to be on this list. To help avoid that, Dr. Laurie Williams, a professor from North Carolina State University, introduced a game called Protection Poker to help teams build security into the workflow just before iteration planning.

What is Protection Poker?

Protection Poker is a collaborative and informal way to do threat modeling that takes advantage of team members' diversity of knowledge and perspectives. It's similar to Planning Poker but focuses on security risk assessment rather than effort estimation. In Protection Poker, security knowledge is passed around the team, building risk mitigation (i.e., dealing with the security threat before the requirement has been met) into iteration planning.

The game provides a structured means for team members to educate one another, uncover assumptions, raise awareness of potential threats and complications, and uncover alternatives towards achieving desired goals.

Ideally, the game is played during iteration planning by everyone involved in the product's development—including product managers, project managers, software developers, testers, usability engineers, security engineers, and other stakeholders. This yields the overall relative security risk for each potential requirement for the iteration.

Afterward, participants can use the relative risk to add stories to the iteration to mitigate risk and change the acceptance criteria for stories to include steps for secure development. This way, the work to implement the requirement securely is factored into the iteration.

Defining risk

While "risk" is traditionally calculated as (probability of loss) x (impact of loss), Protection Poker defines a security risk as:

Security risk = (ease of attack) x (the value of the asset that could be exploited with a successful attack)

This game surfaces the security risks that have a potential impact on the highest-value asset that is the easiest to exploit. This allows the team to discuss and potentially address these risks during iteration planning.

How to play Protection Poker

This exercise uses a small sample of customer data assets relating to when a hotel room is booked: 

Step 1: Value and rank your software assets

A Protection Poker game begins by valuing the data assets. An asset's value is based on historical values for the asset or, if a new asset must be created to implement a requirement, the team's discussion and votes.

Value points

What is the value of your software asset? The asset could be data in a database, a system process, or elsewhere. In this step, you're assessing and taking ownership of your DNA—the data assets and other intellectual property that make your company unique—as these are potential targets. This step involves accurately identifying, categorizing, protecting, and optimizing data from inception to final disposition.

In this step, you'll list your assets and rank their value. Before you assign value points, think about the following questions to determine what value each asset holds for each party:

To the company running the software:

• How critical is the process controlled by the functionality to critical operations?
• How critical is the data in the affected database to company operations?
• Can the data be recovered/recreated if it's maliciously modified?
• How harmful is a data leak to the company's or user's reputation?

To the attacker:

• Who would benefit from attacking the applications? Remember insider threats!
• What can be done with the data once obtained?
• How much damage can be caused by obtaining or modifying the data?
• What is the impact of a denial of service (DoS) on the company's business? On the attacker's business? On a competitor's business? (e.g., would a DoS on Marriott improve another hotel chain's sales?)

List your assets and rank their value points. Save this Asset Value list for the next step.

Imagine your team played a round of Protection Poker on hotel reservation data and got the following results:

Step 2: Calibrate the ease of attack for new requirements

In this example, three feature requests are coming up in the next iteration:

Feature 1: Add "known allergy" feature. A new mobile application has been developed that enables customers to order food to be delivered to their room from the hotel's restaurant. The restaurants would like (optional) information of known food allergies to be tagged to hotel guests to prevent guests with severe known allergies from being exposed to foods containing the risk.

Feature 2: Add "emergency responder" role. Currently, there are only three roles in the hotel reservation system: hotel management professional, reservation administrator, and customer. There's a need for another role: emergency responder (ER)—police, fire, emergency medical technicians (EMTs), and other medically trained emergency responders who provide care while at or in transport from the site of an emergency; in this case, the hotel. The new role would give ERs access to a subset of information on hotel guests involved in an emergency: name, IDs, arrival date, and departure date. The feature would also send a notification email to the hotel guest when an ER views their records.

Feature 3: Add "customer group" role. To create a group reservation for customers booking from the same company or organization, the hotel needs the following information:

• Group name
• Group number
• Customer ID numbers included in that customer group

For each of these new features, try to come up with a consensus on the requirement that will make an attack easiest and the requirement that will make the attack hardest for an attacker to exploit each of the new functionalities.

Ease points

Using your poker cards, assign ease points; these will be your endpoints for the rest of the Protection Poker exercise. You can update these values if you change your opinion or learn new information.

For the sake of this exercise, let's say your team has decided that Feature 1 rates a 1 (hardest to attack) and Feature 3 has 20 ease points (easiest to attack). 

Step 3: Compute the security risk

For each requirement, perform the following steps to compute the security risk:

1. To the Value Point table in Step 1, add any new data assets required for the new features and assign values to them using your poker cards and considering the relative values assigned before. Then identify which assets are used in the new features and add that information to the table as a new column.

2. Add up the total value points mapped to each feature and record that sum.

3. Next, your team will assign ease points to each feature. FIRST, however, everyone must get into a hacker mindset by pretending they have malicious intent. Once you have the right mindset, discuss how easily someone with malicious intent could exploit the new features, including whether the new capability makes it easy to attack. Then use your poker cards to assign ease points for each requirement.

Remember: Protection Poker works when members feel free to express their opinions, diversity of opinions is encouraged and celebrated, and everyone is open to learning from the others. The point of the game isn't the resulting number; it's all about the conversations and sharing knowledge.

If you need help thinking like a hacker, go back to the questions in step 1, especially:

• Who would benefit from attacking the applications?
• What insider threats do you have?
• What can be done with the data once obtained?
• How much damage can be caused by obtaining or modifying the data?
• What is the impact of a denial of service (DoS) on the company's business and on the attacker's business?

Don't shy away from disagreements and misunderstandings—they can signal potential security risks. By having these debates during iteration planning, your team can surface and resolve risks before a requirement is designed or implemented. We often find that the discussions that begin during Protection Poker sessions continue afterward, as team members discover many different ways of exploiting their systems.

The facilitator should jot down any mitigations for potential threats that come up during the game. Comments such as, "Perhaps only the admin should execute this functionality" or "I don't think both features need credit card information" can become additions that mitigate risk.

After completing a round of Protection Poker—thinking like hackers and coming up with scenarios to potentially exploit the new features—imagine your team assigned the following ease points:

• Feature 1 remains unchanged at 1 (harder to attack)
• Feature 2 is 5
• Feature 3 is now 13 (down from 20, meaning it's harder to attack than previously thought)

4. Compute the security risk by multiplying the value points (step 3B) by the ease points (step 3C) using the security risk formula:

Security risk = (ease of attack) x (the value of asset that could be exploited with a successful attack)

Step 4: Add mitigation to the iteration

In this step, your team decides what goes into the next iteration to mitigate the risk. It helps if the facilitator uses a whiteboard or other visual representation to record the comments as the game is played.

Consider:

• Do you really need to collect this customer data? (Think GDPR: If you are a privacy-concerned, ethical data management company, you ask this question often.)
• Do you need to add logging for email access?
• Should you remove a user, providing only admin permission?
• Do you need additional threat assessments?
• Do you need to implement data encryption? Pseudonymization? Masking solutions?

Give it a whirl!

Protection Poker provides a structured means to:

• Obtain a shared understanding of software security knowledge within the team
• Allow engineers to prioritize the verification and fortification of assets to produce more secure software by considering risk.
• Identify areas where valuable and limited time has been over-invested in low-risk areas of the code that are already adequately fortified, uninteresting to an attacker, or contain hard-to-exploit vulnerabilities.
• Surface and resolving ambiguities by leveraging the team's diversity of experience and knowledge about software security.
• Regularly "build security in" by performing a lightweight software risk assessment of new requirements during iteration planning.
• Reduce vulnerabilities in the product through an overall increase of software security knowledge within the team.

Have you used Protection Poker in your company? Tell us how it went in the comments.