0.0 Context setting

It’s Wednesday 2 September in Portland, Oregon, where it was the first day back at school which also meant it was time to start raining again. We made it on time! That was a big achievement, back pats and handshakes all around everyone.

I started writing this episode a good four days ago on 29 August, and jeez has it been weighing on my mind in a very you should have finished this and sent it already sense. For anyone with the belief that writing this newsletter is easy, let me tell you: quite often it is not, and the last few days have been those days when it is not. It would be helpful if I would remind myself that I’ve had a bunch of other stuff going on that would at least make it understandable as to why I haven’t gotten an episode out in a while, but it would be hard for me to listen to that voice. And yet.

0.1 How People Work

Last week’s episode with Margo Stern was super fun. You can catch up on anywhere RSS works with your podcast app, Apple Podcasts, Spotify, and YouTube. (78 subscribers on YouTube now! My kids will be super impressed)

Next up:

Friday 4 September at 9am PT: How People Work, Live! ... with Sarah Vieira

Sarah Vieira is a product leader and founder of Mother Blackbird, where she helps organizations build better products, websites, and systems. She loves crafts, creative problem solving, and the kind of projects that start with “I wonder if I could make that.”

Friday 11 September at 9am PT: How People Work, Live! ... with Dana Chisnell

Dana Chisnell, a pioneering civic designer dedicated to improving government services through human-centered design. She was the first head of customer experience at the Department of Homeland Security. Dana co-founded the Center for Civic Design, where she led initiatives like the Field Guides to Ensuring Voter Intent, and designed the prototype that is the basis of nearly every digital ballot in the U.S. As a founding member of the U.S. Digital Service, Dana contributed to modernizing federal digital services. She has taught design in government at Harvard Kennedy School and co-authored the seminal Handbook of Usability Testing.

Friday 18 September at 9am PT: How People Work, Live! ... with Anna Pickard

Anna Pickard, a writer who does more talking about writing than writing. She mainly works as a consultant in Voice and Tone, helping companies sound human, and helping writing teams get their work done. After training in theatre, she worked in journalism, advertising and games, before stumbling into tech. From 2014, she was responsible for moulding, evolving and shaping the voice of Slack, and since leaving in 2020, has worked with Apple, Airbnb, Atlassian and a bunch of other companies that don’t begin with A.

Meantime, there’s the open How People Work... Does Happy Hour open clinic on Tuesdays at 1pm PT, where you get to bring what you’ve been thinking about and we chat about it as a small group.

1.0 Some Things That Caught My Attention

1.1 Neal Stephenson is leaking again

Two things that caught my attention. First, this headline from Progressive Grocer dot com¹: Gas Prices and GLP-1s Help Power Costco’s Strongest Quarter in Years.

Costco attributes its sizable Q3 gains to its robust, still-growing retail media strategy; the impact of GLP-1 medications on its pharmacy business' bottom line and rising gas prices driving more members to its fuel pumps.²

This is a full-on Neal Stephenson Snow Crash-esque headline if you’re asking me. You’ve got your big box store (albeit not one that’s wholly hollowed out by the kind of parasitic capitalism that looks like it’s going to stupidly kill its host), you’ve got your “still using gas” part out of that stupid U.S. future where of course electrification isn’t a thing that really happens, and then you’ve got the full-on GLP-1 drugs going on. Fantastic. Love it. (not entirely complimentary)

Second! There’s this dancer/choreographer I’ve followed for ages on TikTok (smac³) who first came across my attention when the algorithm decided to show me someone doing dance interpretations of the stuff-being-crushed-by-a-hydraulic-press meme. You may have seen them! They are fun. Then they posted other dance and choreography that was also fun.

Anyway. I noticed the other day that they’d announced (last year, it turns out) they’d become a resident choreographer for Epic’s Fortnite. Which is nuts? I mean, it’s sort of amazing nuts? This one is all Neal Stephenson’s Diamond Age⁴ which some people maintain if you squint is kind of set after Snow Crash⁵. In this book, there’s ractors who are kidn of mocap performers for interactive entertainment. This kind of mocap doesn’t involve a crap-ton of flops being used to track and splat, but it’s more advanced than the ping pong balls and tracking marks we stick on people, instead it involves a bunch of injections to essentially tattoo thousands of tracking dots into your skin. Which isn’t what’s happening here. And I realize that mocap has been going on for games for, like, decades now, but there was something in this particular combination of “someone who’s a dancer/choreographer” who now is “resident choreographer for one of the biggest entertainment properties in the entire world”. I told the big kid about this and they were very impressed and they remarked they imagined smac was being paid well for it. That was a whole other conversation about the Neal Stephenson-ness of exploitation and how much money Epic are making off Fortnite (and emotes!) through digital content and, you know, what an equitable level of compensation would be for smac’s labour, but then the big kid brought up that they’d heard Epic had laid a whole bunch of people off, so maybe it was complicated. Good kid, I thought, but I’m also biased.

1.2 When computer isn’t computer

So there’s this thing going around where an LLM user will try to be helpful and say “hey, remember when you prompt an LLM tell it not to do {these stupid things}” and then a bunch of other people reply and point out that of course, you can’t make the hallucinator not hallucinate and you’ll never stop it from doing the stupid things because duh, it is fundamentally just making shit up.

Sorry, but this annoys me. I think there’s an important thing going on here. You can prompt and LLM to effectively hallucinate less and the thing is, it will effectively hallucinate less. It won’t not hallucinate, but there’s a good chance that will hallucinate less, and in general, that’s probably what you want.

Part of what I think is the shitstorm going on here is the collapse of what a computer is supposed to be, what a computer is, and what people have been trained to think of as a computer. The marketing -- and the marketing is totally problematic -- of these LLMs and generative AI is allowing people to think and believe and behave as if these systems have the accuracy of “computers” when they don’t operate like “computers” at all.

I’ll explain: LLMs don’t “do maths” the way computers and programs do. They don’t “count letters” like the number of “r”s in the word strawberry. They are the fancy statistics (don’t even get me started on how diffusion text models work, that shit will completely blow your mind if you’re only just comfortable with token predictors -- the diffusion stuff works like image generation where you start off with noise and then you figure out/learn what operations you need to do to go from noise to signal. Really. And it works!), there is literally no “working the way you think a computer program that counts the number of “r”s in the word strawberry” in there at all.

And yet! For all intents and purposes, a computer program that works through that algorithmic process, of having data and then operating over it using those types of steps looks identical to one that doesn’t work that way. They present the same way, the user experience is the same, the iconography can be exactly the same. And yet they don’t work that way. It is a little bit like how people feel a bit bamboozled when the computer program they thought was transcribing their voicemail turns out not to be a computer program but a bunch of people far away who look different than you.

In this way the marketing is irresponsible. You are being shown something that looks like a calculator, you are told to treat it like a calculator (albeit with some fine print that says: this is totally not a calculator, you should not treat it like a calculator, it’s totally on you if you believe anything it says, in fact, go check a real calculator after you take a look at anything this not-a-calculator produces), and... it is not. It does not work that way.

Where it gets silly and weird is that the tokens produced by the statistical process -- the text that comes out, whether it’s English or Mandarin or Python -- has meaning, and then in the particular case of code, that the meaning is executable. It can then do something else. In this way, the thing that looks like it is thinking and computing but isn’t computing can actually end up computing. Let me try to explain another way.

There is totally computing going on, and that computing is outputting text. That text itself looks like it’s the result of a different kind of computing, but it’s not!

The tool part is interesting because now you’ve got something that’s capable of generating text that has meaning (code) and then giving it to something that then operates upon that meaning (running the code that was produced). And then you tell it to go around in circles. It’s the part that’s the executing the untrusted or unknown code that’s problematic, and it gets even more problematic when you find the lever that says “make more and make it faster” and then throw it to one of the ends until it’s broken.

I think... this one was difficult. I think it landed, maybe? Some of you may have been able to follow along. Perhaps next episode will be better.

How are you doing?

Best,

Dan