things

s21e26 · 27 August 2026

Defaults and threat-model.md; Making a toaster, but societal institutions

This issue grapples with how to bake good practices into the defaults of AI-generated code, drawing parallels between software security principles and broader value alignment. The writer argues that rather than relying on humans to make the right choices, we should encode best practices—like security checklists and threat modeling—directly into coding agents as defaults, using existing frameworks from organizations like OWASP and the EFF. Underneath is a recurring theme about designing systems where doing the right thing is the path of least resistance, whether that's in software architecture, institutional design, or preventing misaligned AI behavior.

1,400 words · 7 min read · 16 links · canonical link · JSON

0.0 Context setting

Thursday, 27 August 2026 in Portland, Oregon. I started this on Tuesday and then now suddenly it’s Thursday? That’s no fun.

Separately, I’ve also started noticing a bit of burnout or a bit of pressure? I enjoy writing, but I also don’t want this to become some sort of horrific pressure. Which is why I’m trying very hard for it to not be a big deal that I had a gap of a couple days.

Also! This is episode 26 of season 21, which is a lot of episodes. The previous record, if you discount season one, looks like it was season 12 with 59 episodes. I’m trying to figure out (not that I need to really) what will feel like a boundary for the end of this season and the start of season 22. One obvious one would be the beginning of the new school year, which for us starts in about a week. But that also doesn’t feel quite right? Not sure there.

A lot of writing about AI stuff in here recently, I think. Probably reflects the walkabout conversations that I’ve been having. Meantime, it’s heads down on what feels like a pretty consequential proposal, so wish me luck on that?

0.1 Events

Next up, it’s last call to register for...

Margo Stern, the author of Good Work: Managing Yourself and Managing Managing, a two-sided book that shares all the tips, tricks, hacks and cheat codes to working in tech. Drawing not only on her experience managing and being an IC at places like Google, Twitter, and Lyft, she shares the wisdom from interviews with PMs, Designers and Engineers who have been there too. Margo is also a Staff Content Designer at Lyft, and she lives in San Francisco, CA.

... and continuing the weekly clinic, you can register for How People Work... Does Happy Hour! this Tuesday 1 September at 1pm Pacific.

You can subscribe to How People Work, Live! as a podcast now, too:

  • Apple Podcasts: https://podcasts.apple.com/us/podcast/how-people-work-live/id6804736221
  • Spotify: https://open.spotify.com/show/2oScCZu5ownmrUI0Die9LJ
  • And the Internet’s own RSS: http://feeds.libsyn.com/628045/rss and https://howpeoplework.libsyn.com

In the meantime if you’re also a background video person, there’s always the YouTube channel. (Sigh: smash like, smash subscribe, all those things. I have 67 subscribers now, I think! I will throw myself a small celebration when I hit 100.)

If you’re enjoying the show, remember that it’s actually based on my workshop! You can read more about it there, and get in touch with me by email, or book some time to have a short meet-and-greet about it. Mention the show and you’ll get a discount, too.


1.0 Some Things That Caught My Attention

1.1 Defaults and threat-model.md

One principle that exists in software engineering, ish, and one that stuck in my head around the mid 2010s for making software was to make it easy to do the right thing. A sort of making defaults good.

Feels like this is increasingly important in an era where, like it or not, people are making software or stuff-that-computers-do by generating code. It’s also a bit like the alignment problem in micro; the alignment problem being the weirdo thing existential AI people worry about where “do stuff that helps humanity” accidentally becomes “wipe out humanity because that’s the way to ensure world peace”.

The example here for the alignment problem is “do stuff that helps humanity” includes “don’t make crap software that has security hole and/or puts people at risk and so on”.

So, some thoughts:

There’s a set of security principles called OWASP Top Ten, which is, like, a good list of ten things to check if you want your software to be secure. A thing you can do, if you’re going to get an LLM to write software, is to simply say: check what you’re doing against the OWASP top ten and then... fix it. Obviously this relies on the check being accurate and the fix being accurate, but my understanding is that the check and fix is more likely than it ever was to be correct, or at least useful than not doing it. Yes, and the caveat about this built on a tool from original sin.

So easy version: make sure your coding agent has an OWASP Top Ten skill if it doesn’t already. Use stuff that already exists.

Here’s another one: the Electronic Frontier Foundation or some other organization should have a threat-model.md skill. Again, the short version is that if you want to Do An Alignment With Values, then ideally you get to do it as a default without anyone thinking about it.

Bad defaults are easy: just think of Venmo’s default to sharing your Venmo activity on a public feed unless you’re the kind of person who saw that during signup/onboarding and thought: fuck no.

In other words, if you want people to follow a default, then make that default the default, which means baking it into coding agents. Make it easy to do the right thing.

For a threat-model.md skill that means documenting common high-severity (all the way up to life-threatening!) threats that people may encounter through using software and then making it more likely that coding agents are going to check against them. This is kind of building on the last episode where I talked about building codes. This isn’t a top-down regulation that’s ultimately enforced by government, but more of a bottom-up descriptive approach. It would be super interesting for an advocacy organization (whether the Electronic Frontier Foundation or otherwise) to encode their values in code that informs and guides autonomously generated code.

For those reading along who aren’t into or following the AI stuff that deeply (and honestly I don’t blame you, and am simultaneously impressed that you’re still here, so great: the whole deal with these “.md” files is that they’re a silly way of telling an agent to follow rules that are a historical accident of how a specific kind of technologists writes documentation. They are, in essence, something like operating guidelines to be followed. Sorry about the jargon.

1.2 Making a toaster, but societal institutions

The joke these days in the U.S. is that “we used to be a society”, which covers the both slowly and all-at-once destruction and dismantling of societal-scale institutions that let you delegate trust to professionals.

This is the sort of thing where it means you can buy food with the reasonable knowledge that it doesn’t have glass in it, or that you can get on a plane (but not a helicopter) and be reasonably sure that you’re not going to crash and die.

The analogy I have here is about the guy who tried to figure out how to build a toaster on his own1 to illustrate exactly how much of a web of interdependency exists in order to make use of a consumer commodity, i.e. making toast.

Do people know how much work goes into checking whether food is safe to eat? And you know, that’s just the level of “is food safe to eat” that we have now, which includes recalls, and even the level of recalls before Trump 2 decided that they could dismantle processes and requirements in the name of both individual liberty and “the market will decide” or whatever.

You simply can’t do it on your own. You have to rely on other people, “making your own decisions” and “doing your own research” simply doesn’t work, so in that way I’d love to see someone try to viscerally point that out. I don’t know, these days it’d be a year-long vertical video series or something, when I grew up it would either be an hourlong BBC Horizon show, or later on a 6 part popular science BBC One show narrated by that guy from D:Ream who is a physics professor2. Maybe it’s a New York Times series, but that’s persuading or getting in front of the wrong people, isn’t it.


OK, that’s it for today. How have you been? There was a whole other thing I was going to write about restaurant review apps that is going to have to wait for the next episode.

Best,

Dan