BSides SF 2023: New Apps, Good Snacks: Effective Threat Modeling for New Territory

This is the blog post version of my other BSides SF 2023 talk. Yes, I’m tired. Video to come. Slides are here.

Threat modeling is all about approaching a new product, system, or situation and evaluating it with fresh eyes. Dev and product teams get immersed enough in what they’re building that excitement and familiarity can both make it hard to identify potential risk. Enter the security team!

Thing, security folks can get used to things too. If, for instance, you find yourself reviewing similar features over and over again, incremental changes to the same feature, or otherwise getting too accustomed to the state of things, it can become harder to stay sharp and see what’s in front of you.

We’re going to walk through a situation and explore how to push against that feeling, pull back beginner’s mind, and continue to offer useful advice for the teams that depend on us.

What’s threat modeling?

Graphic of a bowl of noodles with chopsticks sticking out of the top and a cute smiling face

I like to describe threat modeling as the application of structured thinking to my own natural paranoia. That sounds flip, but I mean it: something I have in common with a lot of people who do this kind of work is that I naturally imagine worst-case scenarios, things that can break, and other potential problems. It makes me great to travel with and has contributed to my long, loyal streak as a stellar therapy client. It also means that I love threat modeling.

There are lots of formal threat modeling frameworks. If you haven’t explored this before, I suggest looking up STRIDE, PASTA, and MITRE ATT&CK. (Cybersecurity loves an acronym. If you didn’t know, now you know.) There is no one perfect framework, and cybersecurity also loves to argue, so everyone has their own opinion on what’s good and bad. My feeling on it is that if it works for you, it’s great. If the people you need to evaluate risk for understand what you create by applying the framework to their situation, it’s great. If anyone acts rude to you about your choice of framework, send them to me, and I will deal with them.

For my work as a security partner on a product security team, I don’t use anything as structured as that, instead just moving from part to part and looking at design choices and possibilities. If I worked somewhere more formal or had to talk to engineers and product folks less happy about working with me, I might reach for a more structured tool. Instead, I work less formally, and the fictional exercise we’re about to embark on reflects that.

Also useful to know: my security practice (and that of my company, which is why I work there) is more about suggesting and guiding rather than issuing stern dictates. I point out risk, offer mitigations, and record unmitigated risk for our own records. It’s a consultative role, which you’ll see reflected in how I talk about the threat modeling we’re about to walk through.

This is reflected in a structure I use sometimes for my advice: do, try, and consider. Recommendations labeled do are the closest we get to saying you must do this. We want the teams to take them very seriously. Try is a little less serious: give this a shot, please, and see if it works. Consider just tells them something that we think could be useful to them. Keep this idea or principle in mind; it may satisfy something you need in the future. I won’t use this for all the recommendations to come, but it pops up sometimes, and maybe you’ll find it useful too.

Welcome to SnackTracker product security!

Graphic of pepperoni pizza with a cute smiling face

It’s a lovely morning in the snack loyalty startup, and you are a horrible wonderful product security engineer. We work for SnackTracker, which has had a popular app for several years—but they’re ready to go bigger.

They want to go from their mobile and web apps to partner with fine snack shops across a carefully selected test region to offer in-person check-ins and bonuses to snack fiends who already enjoy using the SnackTracker app to track snacks they’ve eaten, wish list snacks they long to try, and find new snacks and places to buy them. Our product team envisions a tablet where users can log snacks in person for those wish lists and get big rewards by checking into businesses, including during special events.

We’ve previously expanded some of our existing features, but it’s been a while since we’ve had something that’s this much of a departure, so we want to do this right.

Up until now, there have been two kinds of users: shop owners and snack fiends. Shop owners need to see customer details, snack trends and reviews, and receive and cash out loyalty points. Snack fiends need to see their account details and point balances, learn about events and new snacks, enter and update their snack preferences, and adjust communication details.

Both user types logged into the same mobile and web apps, and any differences in authorization were connected to the user type, which affected what was displayed after login.

Up until now, all threat modeling and security guidance were done in this previous context. We could make certain assumptions:

  • Users would enter their full username and password to log in, and only one to two users would ever use a single device.
  • Authentication with MFA wasn’t a hard sell, because those snack points are popular and because we ruthlessly bribed incentivized adoption by offering bonus points for MFA setup.
  • Sessions could be valid for a month for snack fiends if they checked that they were on a private device. For shop owners, because there’s a transactional element that translates to real money and because their devices were in the middle of their shops, sessions would end after a minute of inactivity, much like a credit card or bank app.
  • Snack fiends did not need to log out after using the app, and we didn’t need to log them out often.

This will all change with the dawn of SnackAdder. (Product likes Rowan Atkinson, and frankly we can’t blame them.)

The in-person team is psyched about the tie-in opportunities here. They know they want a simpler way for users to check in that doesn’t involve using the long, complex, unique passwords that obviously our superior users use without fault. They also think having people use MFA to log into a device they don’t own for a simple interaction will be a hard sell, particularly as the users will likely only interact with the SnackAdder device for a minute at a time.

The user types and authorized operations associated will change too. The snack fiends want to log snacks and get rewards. The shop owners want a way to say “this device is connected to this business and location” without having more work to do or a new thing to worry about.

Our goal: offer actionable security advice to a product team that’s excited to move fast, protects our users and their data—and keep this team happy to work with security in the future.

Our guidance until now

Graphic of a pink cupcake with a dark pink cup, a cherry on top, and a cute smiling face

It’s useful to know the history when dealing with a team. This team is new to SnackTracker, but it includes engineers and product folks who are on teams we’ve worked with before. Past context matters when you’re working with the same engineers across time, because you get the chance to either give them deeper knowledge when bringing up something familiar again or to point out that something is new and may contrast with things they already know.

What have we told these engineers previously when consulting on mobile and web app features of the past?

  • The usual guidance around encryption: in transit and at rest, please, and use an encryption cipher that’s still considered a good industry standard. When we say that, we provide a link to part of our security guidance library, which includes a page about what’s considered good enough now. We update these pages as needed or every six months, whichever comes first.
  • Authentication works similarly: our advice has been to use our company’s go-to recommended authentication methods, which is overseen by another team. Typically, we advise against too much improvisation around both encryption ciphers and authentication.
  • For authorization, we have established user types and permissions. We also strongly recommend that authorization is enforced in code, so that certain operations are dependent on a permissions check rather than “we just don’t show the admin panel link in the navigation for snack fiends.”
  • For user IDs, and all other kinds of IDs, we recommend using UUIDs rather than anything sequential that can be enumerated or otherwise brute forced.
  • We have a list of suggested and preferred software libraries and mandate use of Dependabot in our repos.

We also offer guidance around privacy. While this is often more of a legal concern, we have a good relationship with the SnackTracker legal team, which means product security can offer some guidance around good data practices with respect to the laws of various states and countries. We like to do better than that, though, and also promote a philosophy of not asking for any data that isn’t strictly necessary for users to use the app. Do we care about someone’s gender if they just want a snack? I hope not! If we don’t need it, we don’t ask, and we don’t store it. We also advise our teams to have a data retention policy. Also, no one in marketing has ever argued with us about this, because this is my story, and I can pretend it’s the better world I believe we’re capable of having.

Which brings us to the future: let’s offer some guidance for SnackAdder.

Threat modeling and new horizons

Graphic of a dark pink heart-shaped treat on a stick with sprinkles and a cute smiling face

Let’s look over this new system, using some of the perspectives we’ve used in the past.

Data use and collection

We always want to ask what data is being collected in a product or feature, where it’s going, where it’s being stored, how it’s being stored, and what else might happen to it. Is it staying entirely within servers we control? Does it need to be send to a third party? Is it being appropriately treated considering its sensitivity?

In the case of SnackAdder, the product team is proposing collecting both less data than our existing product does and more. Because the tablet will only log check-ins and snack preferences, a lot of personal data isn’t even on the table. However, location-based check-isn are something we’ve discussed before. Being vehement defenders of our snack fiends, we’ve previously decided as a company not to collect location data from personal devices—slippery slopes and all. However, this device will be connected to a location, and use of it means users consent specifically to connecting this location to their account. Because of this, we offer a warning to treat this data carefully (including, ideally, a timeline for expiring it off a snack fiend’s account) but say that going forward with check-ins is fine to do, particularly as it isn’t mandatory for logging a snack to a wish list.

Authentication in the wild

We know the team wants to use a different kind of authentication than we find usual at SnackTracker, and so possibly the biggest challenge of this threat model is figuring out what we want to recommend. Being inventive is great; being inventive in bad or unhelpful ways, reinventing mistakes others have already learned from, is not.

The team wants to do something other than requiring the user to enter their username, password, and potentially MFA OTP too, all while a line might be forming behind them to interact with our incredibly popular app’s new interface. Fortunately, the team’s already done some research, and we get to walk through their proposed options.

A PIN has familiarity on its side from its frequent use in ATMs and phones, but it has the downsides of being forgotten and having much lower entropy than a proper password, even if we require six or eight numbers instead of just four. Discussing it within security, we also feel a little queasy having a second credential for the same account, which might confuse users in a way that has security implications. This could work, but we have reservations, so let’s look at what else the team has come up with.

The next option is what I call second-screen authentication. Despite its common use in signing into streaming services, I haven’t found a single consistent term for it. (If you have one, let me know.) It’s a different version of the increasingly common method of web app login where you enter your phone number or email and receive an OTP to enter to log in. In our case, one device provides a code, and we enter it into another device with an authenticated session in order to link our activity in the first device to our account. What the team envisions is entering your SnackTracker user ID into the tablet, which will generate a code that you enter on your phone. This is certainly less fiddly than the first method, but it’s still a little involved and has the additional risk of there not being strong phone signal or wifi in the snack shop. Still, this has potential.

Finally, the team proposes logging in via a QR code generated on the user’s device via the SnackTracker app. If you wanted, they add hopefully, it could be dynamic and regenerate periodically, or every time even! (They really want this project to get moving.) This lets the user log in fast without typing anything in.

We like this option, particularly since we plan on providing additional guidance on sanitizing input, with a special dive into what can be concealed in a QR code.

Authorization in a new context

We realize quickly that there are several ways to achieve what the team is requesting. The admin needs to be able to connect the tablet to the store’s account in order to claim the ability for snack fiends to check in. The snack fiends need to be able to scan snacks and check in. Allowing just partial permissions for a user on a new device, though, gets interesting.

New use of a role can be simpler than creating or changing a role and its permissions, though. It’s always a good idea to get allies early if you’re making changes like this, because a fairly settled product can make it tough to alter this stuff. It can be too rigid, which means having to do difficult persuading and possibly equally difficult technical work, particularly if your codebase is older and has gotten a little brittle. Possibly worse still is when it’s too easy to change these features, because that can result in bloated, unfocused sets of permissions where the difference between two user types might not matter anymore. Think of it as the chaos katamari. Either way, we need to approach this carefully.

The admin needs to log into the tablet, but we don’t want any other admin features accessible. We think about this and realize a useful mental model is to think of it less like “a known user type does a new thing in a new way” and more like a third-party integration, like when you scope a personal access token on GitHub. Your account has a bunch of permissions, but the token may just have one.

The snack fiend role needs to be approached differently too. We suggest limiting the actions a snack fiend can take on the tablet to updating: adding a snack to the wish list, doing a check-in. No deletions, no account details. And on personal devices, we’ve allowed long sessions for snack fiends; for this, we recommend a session lasting either 30 seconds or for one operation, whichever comes first.

The admin sessions need to change too. We know from product research that we can’t rely on a shop owner being physically in the store every day, which means we can’t expect them to create a new check-in tablet session daily. We compromise and give a try-level recommendation of two-week sessions, offering to discuss it further if they find this is too restrictive for our pilot group. We add a do-level recommendation too: admins need to be able to invalidate their SnackAdder sessions in case someone does a runner with the tablet.

Do snack shops have IT?

From product research, we have a sense of how snack shop owners are used to using SnackTracker in their stores: a counter-mounted tablet next to their point of sale, where they can log user purchases for loyalty points when snack fiends request it. What we don’t know is who maintains that device.

Many small businesses don’t have a dedicated tech or IT person. Instead, in a way you may be very familiar with, it might be delegated to “tech-savvy” friend or relative, or anyone who’s willing to read instructions and troubleshoot. All this is to say that we can’t expect a ton of available effort to be put into setting this tablet up, and some additional guidance may engender a lot of good will.

“Just spitballing: what if we manufactured the tablets ourselves???”

Product enthusiasm is a gift, and occasionally we need to rein that gift in a little. The team saved this one for last: they’ve been thinking about tie-in potential for creating the check-in SnackAdder devices and selling or leasing them to snack shop owners. On the plus side, they say, this means we’ve have total control over the device and what’s on it.

It’s true, and this can be really helpful in a security context. However, it would require hiring for skills we haven’t had to seek before in order to do it safely and effectively, which is pretty concerning. Our initial thought is no, followed by oh wow no, and the emerging do-level recommendation is the kind that I’d escalate in security management to make sure there aren’t any diverging feelings, because a single product security engineer or team saying, “Hello, we’d really prefer you didn’t go into a completely new area of business” is a large statement and one that’s best delivered with backing and solidarity. Fortunately, we find that the powers that be also don’t want to get into proprietary touchscreen interfaces, despite the potential for immersion and branding.

The team takes this well, thank goodness, and instead we suggest that the team consider offering security guidance to shop owners who will be using and maintaining a new device in very new circumstances. Suggesting updates and sending along security advisories could be worth our while, particularly as we expect some secondhand device use in the mix.

Awesome! Job well done, all. Let’s look at how to relay those findings.

Don’t kill the messenger

Graphic of a taco with a cute smiling face

We want to preserve our existing good relationship with these product and engineering allies. Along with delivering a thorough, well-structured report, here are some ways to make that work better.

Clear recommendations

I mean the kind of clarity that it takes a few drafts to get to. We want it concise, easy to understand, and ruthlessly clear. Spend some time on it, and if you feel uncertain about whether you managed it, ask someone else (especially someone like who you’re trying to reach) to look it over and give an opinion.

Ground recommendations in security principles

This is useful because it means our recommendations are more solid, but it also ensures greater consistency across time and gives teams the chance to learn what backs our suggestions. After a few reviews, some teams may begin to guess what you’ll consider an issue and will fix things before you get a chance to see them.

Share the knowledge

We don’t want exhaustive further reading to be required to understand or implement most of our recommendations, but sharing sources can be really powerful, especially for people who are interested in security. Having useful further reading is one thing that led me into security engineering, and I like leaving that path for others.

Show up

I know, I’m suggesting you go to more meetings, but not all of them. This is a powerful tool if you think your recommendations may not be taken as you mean them, if you’re not confident that your approach is right for the team, or if you’ve gotten any sense of uncertainty from the team. If things feel shaky, don’t rely on third-party interpretations. Your recs may be clear, but sometimes there’s nothing like the clarity of you presenting them yourself and being present to answer questions.

Position yourself as a security ally

It’s useful to remind people periodically that security’s advice is made with the intention of making the product team’s life better and protecting our users too. I like to mention that they’re the ones who will receive the 3 AM phone call that something’s gone terribly wrong. I will be happy to help too, of course, but I may not come online until west coast business hours. Better still is to avoid the crisis altogether.

Seek feedback and mean it

We can’t grow, and we can’t know that our recommendations are doing what we intend, without asking for feedback. If you ask more than once and in different ways—survey, formal feedback request, verbally—people will eventually believe that you really really want to know and will be more likely to tell you. This is a gift; express thanks accordingly.

Takeaways

Graphic of a red and yellow bucket with a cute smiling face with three chicken legs sticking out of the top
  • Push against complacency and cultivate beginner’s mind, even if you keep seeing similar features over and over.
  • Think creatively to explain new things well; I like using metaphors until I can see the person really gets it.
  • Reduce complex technical ideas to key guidelines, with no noise.
  • Encourage ambitious technical work rather than penalizing it with heavier overhead (and make it clear that you share priorities with product and engineering—we all want to make something cool!).
  • Ask how you can do better next time, and mean it!

BSides SF 2023: No Adversaries: Getting Users on Your Side for Tough Transformations

This is the blog post version of my talk with Amy Martin, Project Manager, SF Digital Services, on April 22, 2023, at BSides San Francisco. Video to come. Slides are here.

Every security initiative has two sides. One is purely technical: we need to prevent abusive emails from getting into company inboxes with a product, or we need a code scanner built into CI/CD to look for common insecure patterns in new pull requests. All of these, even those that seem purely like a matter of automation once they’re set up, also have a second side: the people involved.

We’ve both dealt with the human side of implementing security-related processes. Breanne has grappled with it as a site reliability engineer working to refine IAM roles and as a security engineer creating and refining processes around evaluating and approving Chrome extensions and GitHub Actions. Amy has encountered it while pushing technical advances needed to get ahead of software EOL dates and standardize CMS usage among teams who depended on stability to do their jobs. Both of us have learned the value of creating change through policy and process without making everyone hate you.

To that end, we’re using the term users a little differently than usual. Often, this refers to end users, people using your company’s product, the public, or anyone on the outside who needs what we create. What we’re discussing here certainly affects them too, but our focus here is on internal users: partners within our organization, people who depend on our work to get their work done, and people who need to buy into what we’re doing in order to make a better, more secure experience for them and everyone else. Without them, no security transformation will ever truly work.

Two failures, and hope

In January 2022, Amy left her public library job and joined the Delivery team at San Francisco Digital Services. They needed to move about 80 websites off their Drupal 7 platform and onto SF.gov, the City’s unified and accessible website, in advance of the planned end of life of Drupal 7. That was meant to be in November 2022 and did not happen, but that’s another story.

When Amy started at Digital Services in January, about a third of these departments didn’t know yet that they had to move their website by November, and many of them liked their current, government-speak oriented websites just fine. SF.gov is built for usability and accessibility, and plain language is key—not government speak. Every site they moved required content redesign, and Department staff had to be trained to do the work

How can government Digital Service staff convince civil servants and other agencies to embrace digital transformation in the face of a big unwanted project?

The short answer is that they only did part of that. They moved the websites before their deadline, and they got people started on building user-centered websites and writing in plain language. However, most did not continue that work. Some of them even got rid of the content that we made for them and pasted their old content back in.

Although they met our goal of moving the websites by the deadline, they found that they had fallen short of setting government agencies up for future success with user-centered websites

Breanne’s story is from a few jobs ago, before security engineering, when she was an SRE with an interest in security. The company saw its first rash of phishing attempts: texts and emails posing as executive staff, asking for sadly normal things: Google Play gift cards and other easily sold gift cards, urging the recipients to please move quickly, because the gift cards were needed now now now.

Breanne and a colleague teamed up to write both a presentation and written training to let people know what these attempts looked like and to establish company norms around communication so that it would be clearer to everyone what legitimate company communication looked like.

The company had fewer than 100 people then, so it made more sense to write a single version of the training that would serve both the engineering department and other staff, including sales and marketing. The distinction here is less to do with who is or isn’t “technical” and whose jobs depend on answering unknown numbers and replying to texts and emails from strangers.

How could two engineers write a workable, concise anti-phishing guide for everyone? How can it go wrong? In our case: not everyone is experienced at writing precise, well-grounded documentation about a complex subject for a varied audience. In our case, they got the job done, but they bumped into some of the limitations of trying to write something about a very specific topic that’s meant to be for everyone.

How can a project’s start doom it?

It’s good to start as you mean to end, which means that thinking of what makes a project like this successful from the start goes a long way toward ensuring its success. So what gets baked into a project plan that can doom it to failure?

Maybe your planning doesn’t entirely account for reality. Maybe your engagement doesn’t account for the real needs limitations and motivations of your users
Or maybe it’s meteor from space or other things outside our control. (This probably felt way less possible before March 2020. Now it’s a pretty good idea to write “large scale crisis” into every risk plan.)

We can’t control everything, but we can control some things. What can we do?

Remember that your partners are users too. Keeping this in mind prevents a lot of issues. Like with end users, it’s worth your time to get to know your internal users’ motivations, limitations, and wants, as well as their hard NOs: things they absolutely will not tolerate.

Seek the similarities. See what behaviors, preferences, and common experiences make groups of users start to cohere in your interactions with them. If your team is having trouble wrapping their heads around this, consider using a lightweight version of personas. Rather than the more involved versions used in marketing and UX, we don’t need to know anyone’s educational level, income, or other demographic information, though it’s not unlikely that some of these divisions will fall neatly between job titles or departments. Instead, look at what’s going to affect your plans:

  • What tools are they familiar with? What tools that you’re asking them to use might they not have encountered before?
  • Do they work with security a little or often? If they hear from security via a direct message or an email, will their first thought be that they’re in trouble?
  • Are they amenable to change or resistant? If they’re resistant, why? (This is a really important question to answer.)
  • What motivates them? What’s their definition of professional success?

Avoid stereotypes or unhelpful guesses. Making them too artificial or overly specific is also more likely to harm than help. If it’s just you and a colleague working on this project, it can be plenty to discuss the groups you’re seeing form and plan based on that. However, if you’re doing a longer project or perhaps a larger body of work, you may wish to write them down to share, shape, and reference.

Encountering resistance? Slow down. When you’re starting to roll out your initiative, you may encounter resistance. If the resistance is starting to threaten the velocity or the success of what you’re trying to do, tap the brake and regroup. Form a new hypothesis or think of another strategy—and then seek feedback from the group that wasn’t feeling served before. This is extremely important to do, because we can’t afford to guess in these situations. Fortunately, when you deputize people, they often want to step up and help. If I think someone will be reluctant, I tell them that they hold a key to my success that I can’t under any circumstances produce myself. It’s important to mean this when you say it, but fortunately, it’s completely true, so that shouldn’t be hard.

Make it easy for people to do what you want. Breanne talked about this in her BSides SF talk on documentation last year. Some of the same tactics apply here too: bribery is a legitimate option to overcome some objects. Everyone likes stickers and snacks. However, we can do a little better than that too.

Don’t make users wade through extra reading, video, or other hurdles to do the important thing. Make what you need them to do succinct, clear, and as easy to do as possible. Clear instructions help a lot, as do easy contacts to ask for help. Have the critical stuff up front if you’re asking people to take in information. Executive summaries, a block of text or page or two at the front of a longer document, make sure that busy people have an easier time learning what they need. In longer trainings, the occasional tl;dr block can let people with different attention spans hopscotch through without missing the critical stuff.

This is also a great time to think again of accessibility. (Although it’s always a great time to think about accessibility.) It’s the right thing to do and often legally mandated, but it has one more useful effect, which is that it makes it easier to reach different kinds of learners too. Breanne can hear a video but vastly prefers reading, so a video with transcription reaches those with hearing difficulties as well as those who just like reading, those who can’t turn the sound on, and those who just need to CMD-F or CTRL-F to find the one detail they need to do their job.

This is also a good time for another look back, because history can teach us something. Previous efforts may not be perfect, but they may contain practices and approaches that people are already used to, which means they won’t have to do extra work to learn what you’re doing. It may serve your project well to add additional steps and methods of outreach, but if part of the old approach is getting you a good amount of the way there, it’s likely worth keeping. Security doesn’t have the luxury of perfection a lot of the time, but we can do an 80 percent job in several different ways and achieve something closer to it than we would otherwise.

When you’re asking about the history of projects at your organization, make it clear that you actually want to hear real opinions and that there won’t be retaliation or sore feelings if someone confesses a negative opinion about something that came before.

If all else fails, come back to bribery: security skills look good on resumes, even if the person doesn’t work in security. Try getting people on your side to help them master something important or lucrative. This creates allies.

Okay, but why do we need to think about this at all?

Why do we need to think about this? Aren’t people used to security initiatives and communication around them being subpar or even extremely disruptive? Security is important, and shouldn’t that be enough?

Yes, but—people are part of the system too, and if you’re looking to change the system in an effective and lasting way, you need to work the people part of it too. Both people and the non-people part of systems work better if you give them what they need.

People will not adopt the changes you recommend if you don’t make it easy, explain it well, and make the preferred behavior the easy path. Technical motivations have human implications. It’s just true, and until we reach the robot-controlled singularity, this is just how it’s going to be. We have to adapt.

No matter how brilliant your new security solution, clever process, or other big idea is, it will not work as well as you want it to unless people are on your side. Worse still, you might get a false sense of success as people initially go through the motions of doing what you wanted… only to revert to old habits the next time work gets stressful. We’re looking for something more robust than superficial change.

Once more: how do projects fail?

Projects fail if you don’t get buy-in. This doesn’t need to be unanimous support and a company pep rally celebrating how great your project is. It can be as simple as sending a series of emails ahead of time letting people know what’s coming, when changes will happen, and what they need to do. A little consideration goes a long way.

Projects fail when large, disruptive projects, even ones for the benefit of security, are pushed into being without considering how they affect the people who depend on efficiency or otherwise prefer the previous system for good reason. If people are reluctant, we need to know why and to address that.

Possibly worst of all, projects fail when they’re rotted from the inside by contempt for users and other affected parties by those in power. All of your future efforts will be diminished if the people you’re supposed to be serving can tell you think you’re smarter than them, better than them, or a greater authority on their own lives. People hate this and remember it, and you will face increased friction forever if you get up to this kind of thing. So don’t.

How can we fix this?

Unless you’re Ben Affleck and company, we can’t stop a meteor on a collision course with earth, but there are other levers we can pull to make our projects go smoothly.

Going back to story time: Breanne’s anti-phishing presentation had some things to learn. It couldn’t be too long: people are busy. It needed a short, memorable set of guidelines. It needed to relate causes to effects: “If we don’t present a unified front in not answering phishing texts and looking more carefully at emails, we’ll keep getting hit, and that will have an effect on the business.” It needed to be informed by technical needs but without including too much jargon, because most people who needed anti-phishing guidelines didn’t need to know about how spam emails sneak by Gmail’s spam-targeting AI. Technical curiosity is worth indulging, but it was better as a leave-behind piece for those who want it, leaving the rest of the audience free to go back to their regular work.

Amy’s SF.gov migration is still a work in progress. A lot of their work in 2023 has been around evolving practices and platforms to help their users experience digital transformation. Here are some of their guidelines for that work:

Consider accessibility and cultural differences.
Amy is working on the accessibility of the artifacts she creates and becoming more flexible with the formats and platforms she puts them into. City government is a Microsoft house, and it’s not her team’s preferred set of tools, but many of their users are wary of anything else or simply aren’t allowed to use other products.

Emphasize common ground.
Amy’s background is outside of tech. When she talks with users, she leads with her government background, which allows her to show users their similarities. She makes it a point to visibly not understand things and ask questions, to normalize those behaviors to make others more comfortable.

Say things the easy way.
On SF.gov, they write in plain language, but it can be hard to remember to do that with their users too. If what you need to say is written down, put it into hemingwayapp.com and try to lower the reading level using their suggestions.

Avoid tech jargon and business jargon.
When you need to introduce a technical term, level-set by saying “do you already know the word ____?” Give them a chance to say yes or no, then explain it if they don’t.

Use nonjudgmental language.
Rather than calling something “bad” or another derogatory word, explain the pros and cons. Try to be precise about what you’re pointing out so that you are not expressing judgment. That goes tenfold for anything that your user currently uses or wants.

Speak at a comfortable pace for your audience.
Watch for facial expressions that look worried, like wrinkled foreheads: that could mean people are concentrating hard to keep up with you.

Maintain neutral facial expressions.
Amy does not laugh and does not smile when people ask tech questions. Every question is valid, and every question, especially a very basic or exceptionally strange question, gives you information about what that person needs to know from you.

We sometimes think a smile will be reassuring, but consider that even a reassuring smile can convey “I know the answer and you do not”. As we mentioned earlier, you do not want to give even the appearance of thinking of yourself as smarter than the other person, because that is extremely sensitive and it will be clocked.

Listen actively.
Repeat back what you’re hearing. Paraphrase, but not with tech jargon, because that will sound like you’re correcting them.

Celebrate mastery.
The first time Amy ever felt confident, good, and proud when using a tech tool was in an intermediate Excel class. Amy is not an Excel master, but “confident, good, and proud” is the feeling of mastery. When Amy sees a user experiencing a feeling of mastery, she parties with them, no matter how small a thing it’s for. Amy saw a user feeling mastery because she learned the keyboard shortcut for paste. They celebrated her success as a team, and they’re still a team.

Question your assumptions about your users.
A little humility truly does go a long way. When you don’t understand what your users are doing or why they want what they want, you could try the five whys, because asking why five times can lead you to the real problem.
Example: a tech product that library bought, failed to adopt, and canceled within a year.

  • Why did we fail to adopt the product? Staff never made it part of their workflow.
  • Why was that? They had trouble training and onboarding everyone.
  • Why was that? Rolling out new software to busy front-line staff is very hard, and they did not set aside enough of everyone’s time to make the training stick.
  • Why was that? The software wasn’t seen as enough of a priority to take people away from public service.
  • And why was that? Well, one person in leadership signed a contract for the software despite every other person saying it wasn’t a good fit for the library.

Answer: lack of buy-in.

Try it out and find a version of it that works for you. Sometimes it’s a good tool to use when you find yourself feeling annoyed by the users you’re meant to be serving.

Ask like a librarian: use reference interviews.
Amy was a librarian for 17 years and wants to teach you about reference interviews. When you go to a library and say can you help me find a book, that is the beginning of your reference interview. In this interaction, a trained librarian is assessing:

  • The words you chose
  • Whether there was hesitation in your voice
  • Your body language
  • Tone
  • The way you approached them
  • What else you are holding in your hands 
  • And more

A secret of reference interviews: the cover isn’t blue. Amy has been surprised, in the tech world, by how literally people take requests from users. There’s a librarian adage about “the book with the blue cover.” Asking for things inaccurately is expected behavior. Not because people are lying or not smart, but because this is the way human brains work: imprecisely. 

A good reference interview helps a person form their question. Try tangential lines of questioning. When you’re looking for the book with the blue cover, it’s tempting to keep asking about the cover. Is there a picture? Light blue or dark blue? 

Don’t bother. The cover is irrelevant. 

Ask questions about something else. When did you read the book? In the 70s? Last year? Was it a book you were assigned in school? Did you get it at a Scholastic book fair? Was the main character an animal or a person? Amy uses a mix of broad and specific questions to try to snag on some detail that triggers a new, meaningful memory. The same technique can work for zeroing in on product requirements. 

Learn user vocabulary and motivations.
Every question is valid. A question might sound silly to you, but it is not silly to the person asking. If you are a knowledge authority figure, chances are they had to screw up their courage to ask you.


Keep a neutral facial expression.
We won’t tell you not to smile, but remember that a smile can read as condescending.

Get to know their vocabulary, and use it.
People often adapt their language in ways they assume make sense to us when they ask questions. They use vocabulary they’ve heard people in our position use. They focus on a detail that’s related to what they want because they associate that detail with techy people.

A common example of this in tech is a user who insists on a particular feature, but has an inaccurate understanding of what it does. They know what outcome they want, but ask for a thing that won’t get them that outcome. Why? Because they are trying to communicate the outcome to you in what they assume is your language. Repeating what users said and paraphrasing, in addition to being good active listening, gives them a chance to edit and refine their ask. Listen to the vocabulary your partners use and use it too.

Be wary of contempt.
If you work with anyone who expresses contempt for your user, be very wary of them. All users are beautiful, but some are harder to reach. If you don’t like working with users and find yourself disdainful of their needs, consider that in your career path. You have more than one kind of user and they are all important.

Tie directives to outcomes your users value.
They may not be the same outcomes you value! Find common fears and desires among the users you’re meant to serve and appeal to them, ideally leaning more on the desires than the fears. It’s better to only reference the scarier things (“If we don’t complete this initiative, we’ll have our government certification pulled, and we will all be unemployed”) only with people who know you well. Instead, try invoking more immediate stakes. Breanne likes “I don’t want you to get a phone call at 3 AM if something goes wrong, because everyone understands that.

Takeaways

  • Work to understand actual motivations behind resistance.
  • Learn to speak the user’s language: vocabulary, accessibility, and technological knowledge.
  • No dictates without relating them to your user’s actual needs and priorities.
  • Try reference interview secrets to understand the bigger picture.
  • Helping people to master something important/lucrative will get them on your side.

Further reading

You Can Put WHAT in DNS TXT records?! A blog post for !!con West 2020

Why use a phone book metaphor to explain DNS when you can get even more dated and weird? (This will make more sense when I link the video, but in the meantime, enjoy.)

It Is Known that DNS contains multitudes. Beyond its ten official record types are myriad RFC-described and wildly off-brand uses of them, and DNS TXT records contain the most possibility of weird and creative use. I quote Corey Quinn:

“I mean for God’s sake, my favorite database is Route 53, so I have other problems that I have to work through.”

(From this interview.)

He’s also described it as the only database that meets its SLAs 100 percent of the time. (Route 53 is AWS’s DNS product, encompassing TXT records and many other things, if you have not had the pleasure.)

What is this mystery that is woven through the internet? Let me introduce you to (or reacquaint you with, if you’ve met) the DNS TXT record.

DNS and its ten beautiful children

There are ten kinds of DNS records, each of which will include a reference to a specific domain or subdomain, which usually exists to enable access to that domain’s server or otherwise help with business connected to that domain (email settings, for instance).

The one you might’ve seen or made the most is an A record, or an address mapping record. This is the one that matches URL to IP address – IPv4, in this case. AAAA does the same for IPv6. There are CNAMES, or canonical name records, which alias a hostname to another hostname, often used for things like $marketingproject.business.com, when the marketing project site is being hosted on Heroku or somewhere other than your company’s primary business. You can read about them all here. This post, however, and its accompanying talk (link to come) is about my favorite of them all: TXT records.

TXT records are used for important, fairly official things, but it’s only by agreed-upon practice. While you’ll see very consistent formatting in them for things like SPF, DKIM, DMARC, or domain ownership verification (often in the form of a long random string value for a key that likely starts with _g), the truth is that you can put almost anything in there. My favorite off-brand but still computer-related one I heard about was a large university that put lat/long information in each server’s TXT records, for the sake of finding it more efficiently on a sprawling campus.

For the records’ contents, there are a few restrictions:

  • You cannot exceed 255 characters per string
  • You can include multiple strings in a single TXT record, but they must be enclosed in straight quotes and separated by commas. These can be concatenated into necessarily longer records, like DKIM with longer keys or very elaborate SPF records
  • All characters must be from the 128-character printable ASCII set (no emoji allowed, and no curly quotes or apostrophes either)
  • At least on AWS, you can’t exceed 512 bytes per record, whether it’s a single string or several
  • They are not returned in the order they were added (which made the Emily Dickinson poem I added as three records come out a little funny in my terminal; it still kind of worked, though)

I cribbed that together from a mix of (provider-accented) experimentation and anecdotal information from others who have pondered this stuff. The official docs are often a little hand-wavy on this level of detail (appropriately, I’d say). RFC 1035, for instance, states: “The semantics of the text depends on the domain where it is found.” For its RDATA packet, it offers this:

3.3.14. TXT RDATA format

    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
    /                   TXT-DATA                    /
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+

where:

TXT-DATA        One or more <character-string>s.

I mean, fair. (Data goes in data place; I cannot argue with this.) Future compatibility and planning for unexpected needs to come are a part of every RFC I’ve dug into. Meanwhile, RFC 1464 more directly portends some of the weirdness that’s possible, while also explaining the most common format of TXT files I’ve seen:

        host.widgets.com   IN   TXT   "printer=lpr5"
        sam.widgets.com    IN   TXT   "favorite drink=orange juice"

   The general syntax is:

        <owner> <class> <ttl> TXT "<attribute name>=<attribute value>"

I am accustomed, when dealing with infrastructure tags, to having the key-value format be required, either through a web UI that has two associated fields to complete or through a CLI tool that is very prepared to tell you when you’re doing it wrong.

I have not found this to be the case with TXT records. Whether you’re in a web UI, a CLI, or Terraform, you can just put anything – no keys or values required. Like many standards, it’s actually an optional format that’s just become normal. But you can do what you want, really.

And there are peculiarities. When doing my own DNS TXT poking for this presentation and post, I worked with Dreamhost and AWS, and they acted very differently. AWS wanted only one TXT record per domain and subdomain (so you could have one on example.com and another on wow.example.com), while Dreamhost let me make dozens – but it made DNS propagation really sluggish, sometimes getting records I’d deleted an hour ago, even after clearing the cache. (Dreamhost, meanwhile, has a hardcoded four-minute TTL for its DNS records, which you have to talk to an administrator to change, specially, on your account. It’s always interesting in DNS.) In short, the system is not prepared for that much creativity.

Too bad for the system, though. :)

DNS, ARPANET, hostnames, and the internet as we know it™️

DNS did not always exist in its current scalable, decentralized state, of course. Prior to around 1984, the connection of hostname to IP was done in a file called hosts.txt, which was maintained by the Stanford Research Institute for the ARPANET membership. The oldest example I found online is from 1974, and you can see other examples here and chart the growth of the protointernet. It went from a physical directory to more machine-readable formats, telling you and/or your computer how to reach the host identified as, say, EGLIN or HAWAII-ALOHA. These were static, updated and replaced as needed, and distributed weeklyish.

hosts.txt began its saunter out of our lives when DNS was described in 1983 and implemented in 1984, allowing the internet to scale more gracefully and its users to avoid the risk of stale files. Instead, independent queries began flying around a decentralized infrastructure, with local caches, recursive resolvers, root servers that pointed to top-level domain servers, and nameservers that kept the up-to-date IP addresses and other details for the domains in their care. (You can find a less breezy, more detailed description of this technology you used today here.)

The joys and sorrows of successful adoption

The great and terrible thing about DNS is that so many things rely on it. So if DNS is having a bad day (a much-used DNS server is down, for instance), it can interrupt a lot of processes.

That means, though, that it can also be used to do all sorts of interesting stuff. For instance, a DNS amplification attack involves sending out a ton of DNS queries from lots of sources and spoofing the source address in the packets so they all appear to come from one place, so the responses all go to one targeted server, possibly taking it down. 

TXT records figure into some of this weirdness. Let’s get into some of the interesting backflips folks have done with DNS and its most flexible of record types.

Shenanigans, various

DNS tunneling

This is, so far as I’ve been able to tell, the OG of DNS weirdness. It’s been going on for about 20 years and was first officially described at Black Hat in 2004 by Dan Kaminsky (who stays busy finding weird shit in DNS; if you like this stuff, you’ll enjoy digging into his work).

There are a few different ways to do this, but the central part is always smuggling some sort of information in a DNS query packet that isn’t supposed to be there. 

DNS packets are often not monitored in the same way as regular web traffic (but my Google ads, in the wake of researching this piece, will tell you that there are plenty of companies out there who’d love to help you with that). The permissiveness of DNS query packet movement makes a great vector for exfiltrating data or getting malware into places it would otherwise be hard to reach.

Data is sometimes smuggled via nonexistent subdomains in the URLs the packet seems to be querying for (c2VyaW91cyBldmlsIGJ1c2luZXNzIGlzIGFmb290IGhlcmUgb2ggbm8.evil.com, for instance), but if your packet is designed to return, say, a nice chunk of TXT records? You can really stuff some information or code in there. DNS queries: they smuggle stuff AND evade lots of firewalls. Awesome!

Thwarting internet censorship

The more common DNS association with censorship is avoiding government DNS poisoning by manually setting a DNS server to 8.8.8.8. This isn’t a perfect solution and is getting less useful as more sophisticated technology is put to monitoring and controlling tech we all rely on. However, there’s another way, like David Leadbeater’s 2008 project, which put truncated Wikipedia articles in TXT records. They aren’t live anymore, but there are so many possible uses for this! Mayhem, genuine helpfulness… why not both?

DNSFS OMG OMG

Ben Cox, a British programmer, found DNS resolvers that were open to the internet and used TXT records to cache a blog post on servers all around the world. He used 250-character base64 strings of about 187 bytes each to accomplish this, and he worked out that the caches would be viable for at least a day.

I love all of this stuff, but this is probably my favorite off-brand TXT use. I honestly screamed in my apartment when I saw his animated capture of the sending and reassembly of the blog post cache proof of concept. 

Contributing to the shenanigans corpus

So naturally, I wanted in on this. One does not spend weeks reading about DNS TXT record peculiarities without wanting to play too. And naturally, as a creator of occasionally inspirational zines, I wanted to leave a little trove of glitter and encouragement in an unlikely part of the internet that was designed for no such thing.

Pick a number between 1 and 50. (It was going to be 0 and 49, but Dreamhost will allow 00 as a subdomain but not 0. Go figure!) Use that number as the subdomain of maybethiscould.work. And look up the TXT record. For example:

dig txt 3.maybethiscould.work

will yield

3.maybethiscould.work. 14400 IN TXT "Tilt your head (or the thing) and look at it 90 or 180 degrees off true."

Do a dig txt on only maybethiscould.work, and you’ll see them all, if DNS doesn’t choke on this. My own results have varied. If you like spoilers, or not having to run dig 50 times to see the whole of one thing you’re curious about, you can also see the entire list here.

In the meantime, now you know a little bit more about a thread of the internet that you’ve been close to and benefited from for some time. And next time: always dig the TXT record too. Start with dns.google if you want a strong beginning.

/etc/services Is Made of Ports (and People!)

Hospital switchboard, 1970s
I’ve been using a switchboard/phone extension metaphor to explain ports to my non-engineer friends who have been asking about my talk progress. Feels fitting to start the blog version of it with this switchboard professional.

You can view the video version of this talk here.

I learned about /etc/services in my first year of engineering. I was working with another engineer to figure out why a Jenkins master wasn’t connecting to a worker. Peering through the logs and netstat output, my coworker spied that a service was already listening on port 8080. “That’s Jenkins,” he said.

“But how did you know that?”

“Oh, /etc/services,” he replied. “It has all the service-port pairings for stuff like this.”

Jenkins is not, in fact, in /etc/services, but http-alt is listed at port 8080. The more immediately relevant answer was probably “through experience, because I’ve seen this before, o junior engineer,” but his broader answer got me curious. I spent some time that afternoon scrolling through the 13,000-plus-line file, interested in the ports but especially curious about the signatures attached to so many of them: commented lines with names, email addresses, and sometimes dates, attribution for a need as yet unknown to me.

I got on with the business of learning my job, but /etc/services stayed in my head as a mystery of one of my favorite kinds: everyone was using it, but many well-qualified engineers of my acquaintance had only partial information about it. They knew what used it, or they at least knew that it existed, but not where it came from. The names in particular seemed to surprise folks, when I asked colleagues for their knowledge about this as I was doing this research.

This post, a longer counterpart to my !!con West talk on the same subject, digs into a process and a file that was once commonplace knowledge for a certain kind of back-end and network engineer and has fallen out of more regular use and interaction.  I’ll take you through some familiar services, faces old and new, correspondence with contributors, and how you – yes, you – can make your mark in the /etc/services file.

What is it, where does it live

In *nix systems, including Mac OS, /etc/services lives exactly where you think it does. Windows also has a version of this file, which lives at C:\Windows\System32\drivers\etc\services. Even if you’ve never opened it, it’s been there, providing port name and number information as you go about your life.

The file is set up like this: name, port/protocol, aliases, and then usually a separate line for any comments, which is where you’ll often find names, email addresses, and sometimes dates. Like so:

ssh      22/udp  # SSH Remote Login Protocol

ssh      22/tcp  # SSH Remote Login Protocol

#                   Tatu Ylonen <ylo@cs.hut.fi>

The most common protocols are UDP and TCP, as those were the only ones you could reserve until a few years ago. However, as of an August 2011 update to RFC 6335 (more on that later), you can now snag a port to use with SCTP and/or DCCP as well. This RFC update added more protocols, and it also initiated a change from the old practice of assigning a port for both UDP and TCP for a service to only allocating the port for the protocol requested, and just reserving it for the others, though they’ll only be used if other port availability dwindles significantly.

Incidentally, the presence of a service in /etc/services does not mean the service is running on your computer. The file is a list of possibilities, not attendance on your machine (which is why your computer is probably not currently on fire).

Going through the first 500-odd lines of the file will show you some familiar friends. ssh is assigned port 22. However, ssh also has an author: Tatu Ylonen. His bio includes a lot of pretty typical information for someone who’s listed this far up in this file: he designed the protocol, but he has also authored several RFCs, plus the IETF standards on ssh.

Jon Postel is another common author here, with 23 entries. His representation in this file just hints at the depth of his contributions – he was the editor of the Request for Comment document series, he created SMTP (Simple Mail Transfer Protocol), and he ran IANA until he died in 1998.  A high /etc/services count is more a side effect of the enormity of his work rather than an accomplishment unto itself.

It’s cool to see this grand, ongoing repository of common service information, with bonus attribution. However, that first time I scrolled (and scrolled, and scrolled) through the entirety of /etc/services, what stayed with me were how many services and names I wasn’t familiar with – all this other work, separate of my path in tech thus far, with contact information and a little indicator of what that person was up to in, say, August 2006.

For instance: what’s chipper on port 17219? (It’s a research rabbit hole that took me about 25 minutes and across Google translate, the Wayback Machine, LinkedIn, Wikipedia, a 2004 paper from The European Money and Finance Forum, AMONG OTHER RESOURCES. Cough.) chipper, by Ronald Jimmink, is one of two competing e-purse schemes that once existed in the Netherlands; the longer-lasting competitor, Chipknip, concluded business in 2015. The allure of these cards, over a more traditional debit card, was that the value was stored in the chip, so merchants could conduct transactions without connectivity for their card readers. This was a common race across Europe, in the time before the standardization of the euro and banking protocols, and chipper is an artifact of the Netherlands’s own efforts to find an easier way to pay for things in a time before wifi was largely assumed.

Then there’s octopus on port 10008 (a port which apparently also earned some notoriety for being used for a worm once upon a time). Octopus is a a professional Multi-Program Transport Stream (MPTS) software multiplexer, and you can learn more about it, including diagrams, here.

There are, of course more than 49,000 others; if you have some time to kill, I recommend scrolling through and researching one whose service name, author, or clever port number sparks your imagination. Better still, run some of the service names by the longer-tenured engineers in your life for a time capsule opening they won’t expect.

Port numbers and types

Ports are divided into three ranges, splitting up the full range of 0-65535 (the range created by 16-bit numbers).

  • 0-1023 are system ports (also called well-known ports or privileged ports)
  • 1024-49151 are user ports (or registered ports)
  • And 49152-65535 are private ports (or dynamic ports)

Any services run on the system ports must be run by the root user, not a less-privileged user. The idea behind this (per W3) is that you’re less likely to get a spoofed server process on a typically trusted port with this restriction in place.

Ok, but what actually uses this file? Why is it still there?

Most commonly, five C library routines use this file. They are, per (YES) the services man page:

  • getservent(), which reads the next entry from the services database (see services(5)) and returns a servent structure containing the broken-out fields from the entry.
  • getservbyname(), which returns a servent structure for the entry from the database that matches the service name using protocol proto
  • getservbyport(), which returns a servent structure for the entry from the database that matches the port port (given in network byte order) using protocol proto
  • setservent(), which opens a connection to the database, and sets the next entry to the first entry
  • endservent()

The overlapping use of these routines makes service name available by port number and vice versa. Thus, these two commands are equivalent:

  • telnet localhost 25
  • telnet localhost smtp

And it’s because of information pulled from /etc/services.

The use  you’ve most likely encountered is netstat, if you give it flags to show service names. The names it shows are taken directly from /etc/services (meaning that you can futz with netstat’s output, if you have a little time.)

In short: /etc/services used to match service to port to give some order and identity to things, and it’s used to tell developers when a certain port is off limits so that confusion isn’t introduced. Human readable, machine usable.

Enough about ports; let’s talk about the people

First, let’s talk about the method. Shortly after getting the acceptance for my !!con talk, I went through the entire /etc/services file, looking for people to write to. I scanned for a few things:

  • Email addresses with domains that looked personal and thus might still exist
  • Interesting service names
  • Email addresses from employers whose employees tended to have long tenures
  • Anything that sparked my interest

I have a lot of interest, and so I ended up with a list of 288 people to try to contact. The latest date in my local copy of /etc/services is in 2006, so I figured I’d be lucky to get responses from three people. And while more than half of the emails certainly bounced (and the varieties of bounce messages and protocols had enough detail to support their own fairly involved blog post), I got a number of replies to my questions about how their work came to involve requesting a port assignment, how it was that they knew what to do, and how the port assignment experience went for them.

I will say that the process revealed an interesting difference between how I’d write to folks as a writer and researcher (my old career) vs. how one writes questions as an engineer. As a writer working on a story that would eventually involve talking to people about a subject, I would research independently and only later approach the people involved; my questions would start a few steps back from what I already knew to be true from my research. This allows room for people to feel like experts, to provide color and detail, and to offer nuance that doesn’t come out if the person asking questions charges in declaring what they know and asking smaller, more closed questions.

This is… not the case in computer science, when questions are typically prefaced with a roundup of all 47 things you’ve googled, attempted, and wondered about, in the interest of expediency. This meant that my very second-person questions, in the vein of “how did you do this” and “what was the nature of your process,” sometimes were taken as some email rando not being able to, how you say, navigate the internet in a most basic way.

The more you know.

Happily, I got more than three responses, and people were incredibly generous in sharing their experiences, details of their work, and sometimes relevant messages from their astonishingly deep email archives.

bb, port 1984: Sean MacGuire

The first response I got, delightfully, was for the service named after my initials: bb, on port 1984. More delightfully, this turned out to be the port reserved for software called Big Brother, “the first Web-based Systems and Network Monitor.” Its author, Sean MacGuire, figured out the process for reserving a port after researching IANA’s role in it. At the time (January 1999), it was approved in 12 days. He described it as “totally painless.” Fun fact: Sean also registered the 65th registered domain in Canada, which required faxing proof of the company and making a phone call to the registrar.

The thing I started to learn with Sean’s response was how this was, at one point, pretty ordinary. Most web-based services restrict themselves to ports 80 and 443 now, in large part because a lot of enterprise security products clamp down on access by closing less-commonly used ports, so reserving a port for your new service isn’t always a necessary step now.

In which I am written to by one of the chief architects of the internet as we know it

The next response I got was from a little further back in computer science history. For context, I’ll tell you how I went about contacting people: back in December, after my talk was accepted for !!con West, I went through the /etc/services file on my computer and selected people to contact. I picked people whose email address domains looked like they might still be around, who worked for companies where people tend to have long tenures, or who were contacts tied to interesting-sounding services.

I did this across about ten days, which meant that, by the time I got to the end, I couldn’t have recounted to you the first folks I selected, particularly as I’d chosen 288 people to try to reach in all. Incidentally, about half of those bounced – not nearly as many as I expected.

This is all to say that I was a little startled to read this response:

> How did you come to be the person in charge of reserving the port

I designed the HTTP protocol

Which reminded me that I had indeed selected this entry as one worth diving into, when I was first getting a handle on this research:

http          80/udp www www-http # World Wide Web HTTP

http          80/tcp www www-http # World Wide Web HTTP

#                       Tim Berners-Lee <timbl@W3.org>

He was, I am pleased to say, impeccably polite in his brief response, and he recommended his book, Weaving the Web, which is such a wonderful look at finding compromise between competing standards and design decisions across dozens of institutions, countries, and strong-willed people. As he said, more information on his place within this work and that file can be found there, and if you’re at all curious, I so recommend it.

More contributors

I also liked that some people had fun with this or considered it, as Christian Treczoks  of Digivote, port 3223, put it, a “real “YEAH!” moment.” Barney Wolff, of LUPA, worked a few layers of meaning into his assignment of port 1212: “I picked 1212 because the telephone info number was <area>-555-1212. And LUPA (an acronym for Look Up Phone Access) was a pun on my last name. I don’t know if my bosses at ATT or anyone at IANA ever noticed that.”

Christian Catchpole claimed port 1185, appropriately named catchpole. He requested a low port number in the interest of claiming something memorable. He explained: “The original project back in 2002 involved a compact wire protocol for streaming objects, data structures and commands etc. While the original software written is no longer in the picture, the current purpose of the port number involves the same object streaming principal.  I am currently using the port for async communications for my autonomous marine robotics project.” The original uses of many ports have shifted into computer science history, but Christian’s projects live on.

Alan Clifford (mice, port 5022) claimed his space for a personal project; approval took 21 days. (He, like several people I contacted, keeps a deep and immaculate email archive.) Mark Valence (keysrvr at 19283 and keyshadow at 19315) recounted his involvement thusly: “I was writing the code, and part of that process is choosing a port to use.” He ended up in /etc/services around 1990 or 1991, when his team was adding TCP/IP as an option for their network service a year or so prior, enabling Macs, PCs, and various Unix systems to communicate with each other.

Ulrich Kortenkamp (port 3770, cindycollab) was one of two developers of Cinderella, and he claimed a port in /etc/services to codify their use of a private port for data exchange. He added: “And I am still proud to be in that file :)”

Greg Hudson’s contributions date to his time as a staff engineer at MIT, when he became a contributor to and then a maintainer of the school’s Zephyr IM protocol (zephyr-hm in the file) and then similarly with Subversion, the open-source version control system now distributed by Apache. His name is connected to ports 2102-2104 for Zephyr and port 3690 for Subversion.

Jeroen Massar has his name connected to four ports:

He noted that AURORA also has an SCTP allocation too, which is still fairly rare, despite that protocol being available since 2011. He remarked, “[This] is actually likely the ‘cool’ thing about having ‘your own port number’: there is only 65536 of them, and my name is on 4 of them ;)”

I asked people how they knew what to do; some were basically like :shrug: “The RFC?” But others explained their context at the time. Mostly, folks seemed to have general industry awareness of this process and file just because of the work they did. (“I was the ‘Network Guy’ in the company,” said Christian Treczoks.)  Some knew the RFC or knew to look for it; others had been involved with the IETF and were around for the formation of these standards. My anecdotal impression was that it was, at that point, just part of the work. If you were on a project that was likely to need a port, it was known how you’d go about getting it.

Who controls this file? Where does the official version come from?

Like so many things in the big world of the internet, /etc/services and its contents are controlled by IANA. The official version varies; what’s on IANA’s official and most up-to-date version deviates some from what you might find locally. The version of /etc/services on my Mac, as I’ve mentioned, is about 13 years out of date. However, people are still claiming ports, and you can see the most current port assignments at IANA’s online version.

On most Unixes, the version of /etc/services you see is a snapshot of the file taken from IANA’s official version at the time that version of the OS was released. When installing new services, often you’ll want to tweak your local copy of /etc/services to reflect the new service, if it’s not already there, even if only as a reminder.

However, updates vary between OSes; the version included with the Mac OS is not the most current, and how updates are added and communicated can vary widely. Debian, for instance, furnishes /etc/services as part of the netbase package, which includes “the necessary infrastructure for basic TCP/IP based networking.” If the included version of /etc/services got out of date, one could file a bug to get it updated.

To learn how /etc/services managed and how to contribute, the golden standard is:

RFC 6335

No, seriously. The Procedures for the Management of the Service Name and Transport Protocol Port Number Registry has most of what you would need to figure out how to request your own port. While the process is less common now, it’s still regimented and robustly maintained. There’s a whole page just about statistics for port request and assignment operations. While this isn’t as commonly used as it once was, it’s still carefully governed.

RFC 7605, Recommendations on Using Assigned Transport Port Numbers, includes guidance on when to request a port). 7605 and 6335 are concatenated together as BCP 165, though 6335 is still referred to frequently and is the most commonly sought and cited resource.

How can you get your piece of /etc/services glory?

There’s real estate left to claim; as of this writing, more than 400 ports are still available. Others have notes that the service claims are due to be removed, with timestamps from a few years ago, and just haven’t yet.

If you have a service in need of a port, I am pleased to tell you that there is a handy form to submit your port assignment request. You’ll need a service name, transport protocol, and your contact information, as you might guess. You can also provide comments and some other information to bolster your claim. The folks managing this are pretty efficient, so if your request is valid, you could have your own port in a couple of weeks, and then you could be hiding inside of your computer many years from now, waiting to startle nosy nerds like me.

Despite the shift to leaning more heavily on ports 80 and 443, people are absolutely still claiming ports for their services. While the last date in my computer’s /etc/services file is from about a decade ago, the master IANA list already has a number of dates from this year.

So: make your service, complete the form, and wait approximately two weeks. Then cat /etc/services | grep $YourName, and your immortality is assured (until IANA does an audit, anyway).  

And if you do, please let me know. Enabling people to do great, weird, and interesting things is one of my favorite uses of time, and it’d make my day. Because there are no computers without people (or not yet, anyway), and a piece of that 16-bit range is waiting to give your work one more piece of legitimacy and identity.

Thanks to everyone who wrote back to me for being so generous with their experiences, to Christine Spang for the details about Debian /etc/services updates, to the patient overseers of RFC 6335, and the authors of all the RFCs and other standards that have managed to keep this weird world running.

New Talk: Of Tracked Changes and Diffs at StarCon 2018

I write to you from beautiful and astonishingly cold Waterloo, Ontario, Canada, where the people are kind and the excitement is palpable. (Really – everyone’s excited about what they’re doing and about sharing it. It’s great.) I did a new talk during the morning session about what I learned from my life in editorial that applied to dealing with code reviews as an engineer. Slides to come; for now, I have a written-out version for you over at Truss.works.

In the meantime, enjoy a picture of me in a tuque provided to me by a kind-hearted organizer so I’d be slightly less likely to die on the trek between my Airbnb and the university. I like it here.