This is the blog post version of my other BSides SF 2023 talk. Yes, I’m tired. Video to come. Slides are here.
Threat modeling is all about approaching a new product, system, or situation and evaluating it with fresh eyes. Dev and product teams get immersed enough in what they’re building that excitement and familiarity can both make it hard to identify potential risk. Enter the security team!
Thing, security folks can get used to things too. If, for instance, you find yourself reviewing similar features over and over again, incremental changes to the same feature, or otherwise getting too accustomed to the state of things, it can become harder to stay sharp and see what’s in front of you.
We’re going to walk through a situation and explore how to push against that feeling, pull back beginner’s mind, and continue to offer useful advice for the teams that depend on us.
What’s threat modeling?
I like to describe threat modeling as the application of structured thinking to my own natural paranoia. That sounds flip, but I mean it: something I have in common with a lot of people who do this kind of work is that I naturally imagine worst-case scenarios, things that can break, and other potential problems. It makes me great to travel with and has contributed to my long, loyal streak as a stellar therapy client. It also means that I love threat modeling.
There are lots of formal threat modeling frameworks. If you haven’t explored this before, I suggest looking up STRIDE, PASTA, and MITRE ATT&CK. (Cybersecurity loves an acronym. If you didn’t know, now you know.) There is no one perfect framework, and cybersecurity also loves to argue, so everyone has their own opinion on what’s good and bad. My feeling on it is that if it works for you, it’s great. If the people you need to evaluate risk for understand what you create by applying the framework to their situation, it’s great. If anyone acts rude to you about your choice of framework, send them to me, and I will deal with them.
For my work as a security partner on a product security team, I don’t use anything as structured as that, instead just moving from part to part and looking at design choices and possibilities. If I worked somewhere more formal or had to talk to engineers and product folks less happy about working with me, I might reach for a more structured tool. Instead, I work less formally, and the fictional exercise we’re about to embark on reflects that.
Also useful to know: my security practice (and that of my company, which is why I work there) is more about suggesting and guiding rather than issuing stern dictates. I point out risk, offer mitigations, and record unmitigated risk for our own records. It’s a consultative role, which you’ll see reflected in how I talk about the threat modeling we’re about to walk through.
This is reflected in a structure I use sometimes for my advice: do, try, and consider. Recommendations labeled do are the closest we get to saying you must do this. We want the teams to take them very seriously. Try is a little less serious: give this a shot, please, and see if it works. Consider just tells them something that we think could be useful to them. Keep this idea or principle in mind; it may satisfy something you need in the future. I won’t use this for all the recommendations to come, but it pops up sometimes, and maybe you’ll find it useful too.
Welcome to SnackTracker product security!
It’s a lovely morning in the snack loyalty startup, and you are a horrible wonderful product security engineer. We work for SnackTracker, which has had a popular app for several years—but they’re ready to go bigger.
They want to go from their mobile and web apps to partner with fine snack shops across a carefully selected test region to offer in-person check-ins and bonuses to snack fiends who already enjoy using the SnackTracker app to track snacks they’ve eaten, wish list snacks they long to try, and find new snacks and places to buy them. Our product team envisions a tablet where users can log snacks in person for those wish lists and get big rewards by checking into businesses, including during special events.
We’ve previously expanded some of our existing features, but it’s been a while since we’ve had something that’s this much of a departure, so we want to do this right.
Up until now, there have been two kinds of users: shop owners and snack fiends. Shop owners need to see customer details, snack trends and reviews, and receive and cash out loyalty points. Snack fiends need to see their account details and point balances, learn about events and new snacks, enter and update their snack preferences, and adjust communication details.
Both user types logged into the same mobile and web apps, and any differences in authorization were connected to the user type, which affected what was displayed after login.
Up until now, all threat modeling and security guidance were done in this previous context. We could make certain assumptions:
- Users would enter their full username and password to log in, and only one to two users would ever use a single device.
- Authentication with MFA wasn’t a hard sell, because those snack points are popular and because we
ruthlessly bribedincentivized adoption by offering bonus points for MFA setup. - Sessions could be valid for a month for snack fiends if they checked that they were on a private device. For shop owners, because there’s a transactional element that translates to real money and because their devices were in the middle of their shops, sessions would end after a minute of inactivity, much like a credit card or bank app.
- Snack fiends did not need to log out after using the app, and we didn’t need to log them out often.
This will all change with the dawn of SnackAdder. (Product likes Rowan Atkinson, and frankly we can’t blame them.)
The in-person team is psyched about the tie-in opportunities here. They know they want a simpler way for users to check in that doesn’t involve using the long, complex, unique passwords that obviously our superior users use without fault. They also think having people use MFA to log into a device they don’t own for a simple interaction will be a hard sell, particularly as the users will likely only interact with the SnackAdder device for a minute at a time.
The user types and authorized operations associated will change too. The snack fiends want to log snacks and get rewards. The shop owners want a way to say “this device is connected to this business and location” without having more work to do or a new thing to worry about.
Our goal: offer actionable security advice to a product team that’s excited to move fast, protects our users and their data—and keep this team happy to work with security in the future.
Our guidance until now
It’s useful to know the history when dealing with a team. This team is new to SnackTracker, but it includes engineers and product folks who are on teams we’ve worked with before. Past context matters when you’re working with the same engineers across time, because you get the chance to either give them deeper knowledge when bringing up something familiar again or to point out that something is new and may contrast with things they already know.
What have we told these engineers previously when consulting on mobile and web app features of the past?
- The usual guidance around encryption: in transit and at rest, please, and use an encryption cipher that’s still considered a good industry standard. When we say that, we provide a link to part of our security guidance library, which includes a page about what’s considered good enough now. We update these pages as needed or every six months, whichever comes first.
- Authentication works similarly: our advice has been to use our company’s go-to recommended authentication methods, which is overseen by another team. Typically, we advise against too much improvisation around both encryption ciphers and authentication.
- For authorization, we have established user types and permissions. We also strongly recommend that authorization is enforced in code, so that certain operations are dependent on a permissions check rather than “we just don’t show the admin panel link in the navigation for snack fiends.”
- For user IDs, and all other kinds of IDs, we recommend using UUIDs rather than anything sequential that can be enumerated or otherwise brute forced.
- We have a list of suggested and preferred software libraries and mandate use of Dependabot in our repos.
We also offer guidance around privacy. While this is often more of a legal concern, we have a good relationship with the SnackTracker legal team, which means product security can offer some guidance around good data practices with respect to the laws of various states and countries. We like to do better than that, though, and also promote a philosophy of not asking for any data that isn’t strictly necessary for users to use the app. Do we care about someone’s gender if they just want a snack? I hope not! If we don’t need it, we don’t ask, and we don’t store it. We also advise our teams to have a data retention policy. Also, no one in marketing has ever argued with us about this, because this is my story, and I can pretend it’s the better world I believe we’re capable of having.
Which brings us to the future: let’s offer some guidance for SnackAdder.
Threat modeling and new horizons
Let’s look over this new system, using some of the perspectives we’ve used in the past.
Data use and collection
We always want to ask what data is being collected in a product or feature, where it’s going, where it’s being stored, how it’s being stored, and what else might happen to it. Is it staying entirely within servers we control? Does it need to be send to a third party? Is it being appropriately treated considering its sensitivity?
In the case of SnackAdder, the product team is proposing collecting both less data than our existing product does and more. Because the tablet will only log check-ins and snack preferences, a lot of personal data isn’t even on the table. However, location-based check-isn are something we’ve discussed before. Being vehement defenders of our snack fiends, we’ve previously decided as a company not to collect location data from personal devices—slippery slopes and all. However, this device will be connected to a location, and use of it means users consent specifically to connecting this location to their account. Because of this, we offer a warning to treat this data carefully (including, ideally, a timeline for expiring it off a snack fiend’s account) but say that going forward with check-ins is fine to do, particularly as it isn’t mandatory for logging a snack to a wish list.
Authentication in the wild
We know the team wants to use a different kind of authentication than we find usual at SnackTracker, and so possibly the biggest challenge of this threat model is figuring out what we want to recommend. Being inventive is great; being inventive in bad or unhelpful ways, reinventing mistakes others have already learned from, is not.
The team wants to do something other than requiring the user to enter their username, password, and potentially MFA OTP too, all while a line might be forming behind them to interact with our incredibly popular app’s new interface. Fortunately, the team’s already done some research, and we get to walk through their proposed options.
A PIN has familiarity on its side from its frequent use in ATMs and phones, but it has the downsides of being forgotten and having much lower entropy than a proper password, even if we require six or eight numbers instead of just four. Discussing it within security, we also feel a little queasy having a second credential for the same account, which might confuse users in a way that has security implications. This could work, but we have reservations, so let’s look at what else the team has come up with.
The next option is what I call second-screen authentication. Despite its common use in signing into streaming services, I haven’t found a single consistent term for it. (If you have one, let me know.) It’s a different version of the increasingly common method of web app login where you enter your phone number or email and receive an OTP to enter to log in. In our case, one device provides a code, and we enter it into another device with an authenticated session in order to link our activity in the first device to our account. What the team envisions is entering your SnackTracker user ID into the tablet, which will generate a code that you enter on your phone. This is certainly less fiddly than the first method, but it’s still a little involved and has the additional risk of there not being strong phone signal or wifi in the snack shop. Still, this has potential.
Finally, the team proposes logging in via a QR code generated on the user’s device via the SnackTracker app. If you wanted, they add hopefully, it could be dynamic and regenerate periodically, or every time even! (They really want this project to get moving.) This lets the user log in fast without typing anything in.
We like this option, particularly since we plan on providing additional guidance on sanitizing input, with a special dive into what can be concealed in a QR code.
Authorization in a new context
We realize quickly that there are several ways to achieve what the team is requesting. The admin needs to be able to connect the tablet to the store’s account in order to claim the ability for snack fiends to check in. The snack fiends need to be able to scan snacks and check in. Allowing just partial permissions for a user on a new device, though, gets interesting.
New use of a role can be simpler than creating or changing a role and its permissions, though. It’s always a good idea to get allies early if you’re making changes like this, because a fairly settled product can make it tough to alter this stuff. It can be too rigid, which means having to do difficult persuading and possibly equally difficult technical work, particularly if your codebase is older and has gotten a little brittle. Possibly worse still is when it’s too easy to change these features, because that can result in bloated, unfocused sets of permissions where the difference between two user types might not matter anymore. Think of it as the chaos katamari. Either way, we need to approach this carefully.
The admin needs to log into the tablet, but we don’t want any other admin features accessible. We think about this and realize a useful mental model is to think of it less like “a known user type does a new thing in a new way” and more like a third-party integration, like when you scope a personal access token on GitHub. Your account has a bunch of permissions, but the token may just have one.
The snack fiend role needs to be approached differently too. We suggest limiting the actions a snack fiend can take on the tablet to updating: adding a snack to the wish list, doing a check-in. No deletions, no account details. And on personal devices, we’ve allowed long sessions for snack fiends; for this, we recommend a session lasting either 30 seconds or for one operation, whichever comes first.
The admin sessions need to change too. We know from product research that we can’t rely on a shop owner being physically in the store every day, which means we can’t expect them to create a new check-in tablet session daily. We compromise and give a try-level recommendation of two-week sessions, offering to discuss it further if they find this is too restrictive for our pilot group. We add a do-level recommendation too: admins need to be able to invalidate their SnackAdder sessions in case someone does a runner with the tablet.
Do snack shops have IT?
From product research, we have a sense of how snack shop owners are used to using SnackTracker in their stores: a counter-mounted tablet next to their point of sale, where they can log user purchases for loyalty points when snack fiends request it. What we don’t know is who maintains that device.
Many small businesses don’t have a dedicated tech or IT person. Instead, in a way you may be very familiar with, it might be delegated to “tech-savvy” friend or relative, or anyone who’s willing to read instructions and troubleshoot. All this is to say that we can’t expect a ton of available effort to be put into setting this tablet up, and some additional guidance may engender a lot of good will.
“Just spitballing: what if we manufactured the tablets ourselves???”
Product enthusiasm is a gift, and occasionally we need to rein that gift in a little. The team saved this one for last: they’ve been thinking about tie-in potential for creating the check-in SnackAdder devices and selling or leasing them to snack shop owners. On the plus side, they say, this means we’ve have total control over the device and what’s on it.
It’s true, and this can be really helpful in a security context. However, it would require hiring for skills we haven’t had to seek before in order to do it safely and effectively, which is pretty concerning. Our initial thought is no, followed by oh wow no, and the emerging do-level recommendation is the kind that I’d escalate in security management to make sure there aren’t any diverging feelings, because a single product security engineer or team saying, “Hello, we’d really prefer you didn’t go into a completely new area of business” is a large statement and one that’s best delivered with backing and solidarity. Fortunately, we find that the powers that be also don’t want to get into proprietary touchscreen interfaces, despite the potential for immersion and branding.
The team takes this well, thank goodness, and instead we suggest that the team consider offering security guidance to shop owners who will be using and maintaining a new device in very new circumstances. Suggesting updates and sending along security advisories could be worth our while, particularly as we expect some secondhand device use in the mix.
Awesome! Job well done, all. Let’s look at how to relay those findings.
Don’t kill the messenger
We want to preserve our existing good relationship with these product and engineering allies. Along with delivering a thorough, well-structured report, here are some ways to make that work better.
Clear recommendations
I mean the kind of clarity that it takes a few drafts to get to. We want it concise, easy to understand, and ruthlessly clear. Spend some time on it, and if you feel uncertain about whether you managed it, ask someone else (especially someone like who you’re trying to reach) to look it over and give an opinion.
Ground recommendations in security principles
This is useful because it means our recommendations are more solid, but it also ensures greater consistency across time and gives teams the chance to learn what backs our suggestions. After a few reviews, some teams may begin to guess what you’ll consider an issue and will fix things before you get a chance to see them.
Share the knowledge
We don’t want exhaustive further reading to be required to understand or implement most of our recommendations, but sharing sources can be really powerful, especially for people who are interested in security. Having useful further reading is one thing that led me into security engineering, and I like leaving that path for others.
Show up
I know, I’m suggesting you go to more meetings, but not all of them. This is a powerful tool if you think your recommendations may not be taken as you mean them, if you’re not confident that your approach is right for the team, or if you’ve gotten any sense of uncertainty from the team. If things feel shaky, don’t rely on third-party interpretations. Your recs may be clear, but sometimes there’s nothing like the clarity of you presenting them yourself and being present to answer questions.
Position yourself as a security ally
It’s useful to remind people periodically that security’s advice is made with the intention of making the product team’s life better and protecting our users too. I like to mention that they’re the ones who will receive the 3 AM phone call that something’s gone terribly wrong. I will be happy to help too, of course, but I may not come online until west coast business hours. Better still is to avoid the crisis altogether.
Seek feedback and mean it
We can’t grow, and we can’t know that our recommendations are doing what we intend, without asking for feedback. If you ask more than once and in different ways—survey, formal feedback request, verbally—people will eventually believe that you really really want to know and will be more likely to tell you. This is a gift; express thanks accordingly.
Takeaways
- Push against complacency and cultivate beginner’s mind, even if you keep seeing similar features over and over.
- Think creatively to explain new things well; I like using metaphors until I can see the person really gets it.
- Reduce complex technical ideas to key guidelines, with no noise.
- Encourage ambitious technical work rather than penalizing it with heavier overhead (and make it clear that you share priorities with product and engineering—we all want to make something cool!).
- Ask how you can do better next time, and mean it!