{"id":"1988966f-1408-4047-83a8-ee6da2ca6a5a","slug":"s14e03-success-criteria-chat","subject":"s14e03: Success Criteria; Chat","title":"Success Criteria; Chat","publish_date":"2023-01-04T18:03:09.315700+00:00","year":2023,"month":1,"season":14,"episode":3,"episode_label":"s14e03","canonical_url":"https://newsletter.danhon.com/archive/s14e03-success-criteria-chat/","is_premium":0,"word_count":998,"body_html":"<h1>0.0 Context Setting</h1>\n<p>It's another cold, cloudy morning in Portland, Oregon on Wednesday, 4 January, 2022.</p>\n<hr />\n<h1>1.0 Some Things That Caught My Attention</h1>\n<p>A few small things again today as I get back into the practice of hitting 15 minutes.</p>\n<h1>1.1 Success Criteria</h1>\n<p>Another thing that stuck in my head from Tim Bray's essay about <a href=\"https://www.tbray.org/ongoing/When/202x/2022/12/30/Mastodon-Privacy-and-Search#p-10\">search on Mastodon</a><sup class=\"footnote-ref\"><a href=\"#fn1\" id=\"fnref1\">[1]</a></sup> was his section on success criteria. Here's the first para, but I encourage you to read the rest (there's only three more paras of roughly the same length):</p>\n<blockquote>\n<p>I’d like it if nobody were ever deterred from conversing with people they know for fear that people they don’t know will use their words to attack them. I’d like it to be legally difficult to put everyone’s everyday conversations to work in service to the advertising industry. I’d like to reduce the discomfort people in marginalized groups feel venturing forth into public conversation.</p>\n</blockquote>\n<p>Caught my attention because: It clearly outlines relatable examples without going into too much detail. Which isn't to say detail isn't important, but to me it's a good example of sketching out a direction as well as different contexts:</p>\n<ul>\n<li>From a user/poster's point of view, of understanding the particular context they're conversing in <em>as an individual with specific needs</em>, and understanding usage as more like conversation than written communication, I think</li>\n<li>The context of business usage (&quot;legally difficult&quot;), and reinforcing a point in the piece about for example carving out exclusions for usage in ML training.</li>\n<li>The fuzzy goal of reducing discomfort, which presupposes going out and actively understanding what that discomfort may be, while at the same time <em>not seeking to completely eliminate that discomfort</em></li>\n</ul>\n<p>As an aside, I especially appreciate how Bray's essay doesn't talk about how the usage of hashtags might solve search. Hashtags are useful and interesting, but from my perspective they impose too much work on the part of the writer -- especially if you <em>want</em> your conversation to be indexed and searchable -- as a sort of per-post, opt-in high cognitive cost. Much easier, I think, to either set indexing and licensing globally for the entire account, or if you <em>really</em> want, to tie it to per-post posting contexts. But maybe not! It would be interesting to see!</p>\n<h1>1.2 Chat</h1>\n<p>I went along to the Near Future Laboratory's <a href=\"https://www.generalseminar.com/season-03-episode-26\">General Seminar 26 on ChatGPT futures</a> last week and it was... interesting?</p>\n<p>There's something about ChatGPT that feels limiting and I'm pretty sure it's in the chat/REPL<sup class=\"footnote-ref\"><a href=\"#fn2\" id=\"fnref2\">[2]</a></sup> interface to the language model.</p>\n<p>It's somewhat astonishing to me that nobody has yet hacked up a Young Lady's Shitty Illustrated Primer. Given infinite AI arrays<sup class=\"footnote-ref\"><a href=\"#fn3\" id=\"fnref3\">[3]</a></sup> appeared this week, it doesn't feel like it would be too far off.</p>\n<p>Bear in mind this list is in the realm of &quot;just&quot; as in &quot;just do these things&quot; where &quot;just&quot; is &quot;a giant yawning chasm of implementation&quot; and the actual job is neatly obscured using one word.</p>\n<ul>\n<li>Pick a few topics. Maybe just one.</li>\n<li>Ask ChatGPT to write an introduction to that topic.</li>\n<li>Ask it to write some problems in the form of a children's book.</li>\n<li>Use all the above as static text.</li>\n<li>Bound figuring out the problems in a ChatGPT REPL, i.e. as a text adventure.</li>\n<li>For bonus points:\n<ul>\n<li>generate dynamic illustrations based on, I don't know, identifying salient noun phrases (see, I can word salad just as well as GPT) or whatever by piping them through whatever image generation model is your favorite</li>\n<li>text-to-speech the entire lot</li>\n<li>speech-to-text the other side of your REPL</li>\n<li>(Actually, has someone tts-stt a ChatGPT implementation and thrown it online for people to play with yet?)</li>\n</ul>\n</li>\n</ul>\n<p>See? Easy.</p>\n<p>The one thing that I'm not sure has stuck around here has been continuous training. ChatGPT is good (I think?) at preserving state through each session (e.g. you can tell it to &quot;do that again, but x&quot; and it'll mostly work), but in terms of learning preferences and updating a model continuously, I don't think I've seen that? I mean, I think there's the whole federated on-device learning thing going on, but I don't yet see that in terms of integrating ongoing training with conversational large language models? Or maybe I'm not looking in the right places.</p>\n<p>As an aside, Siri continues to be dumb. Here is an example of what I would like Siri to be able to do, because from the point of view of a stupid user (i.e. me) all the data is there:</p>\n<blockquote>\n<p>Me:  Hey Siri, play some music for me</p>\n</blockquote>\n<blockquote>\n<p>Siri: Sure Dan, here's some music. [plays music]</p>\n</blockquote>\n<blockquote>\n<p>Me: No no no I get that I have this weird thing of listening to college a capella covers of songs, but I really don't want that right now. Play music for me <em>but don't play any college a capella</em></p>\n</blockquote>\n<p>See, this would in theory rely on sufficient metadata like all the college a capella being tagged with the a capella genre, but this whole dealing with sets is something that a) Siri clearly can't do, and most frustratingly, b) is something that Apple Music can't do, and c) is something that iTunes could kind of but not really do with Smart Playlists.</p>\n<p>It's frustrating because you look at it and you think <em>ugh this is right there</em> and you can't really do it? (Note: I know, there are a lot of <em>justs</em> here)</p>\n<h1>1.3 Updates</h1>\n<p>Thank you to the reader who let me know that you can sign in to your <a href=\"https://profile.squareup.com\">Square profile</a> and turn off marketing emails for everyone you've ever bought anything from. (Scroll down to Notifications, hit Email, you should be able to see it there)</p>\n<hr />\n<p>This was longer than 15 minutes. In fact the Tim Bray thing itself was about 12 minutes of wall time. Not great for my goal.</p>\n<p>How are you doing?</p>\n<p>Best,</p>\n<p>Dan</p>\n<hr class=\"footnotes-sep\" />\n<section class=\"footnotes\">\n<ol class=\"footnotes-list\">\n<li id=\"fn1\" class=\"footnote-item\"><p><a href=\"https://www.tbray.org/ongoing/When/202x/2022/12/30/Mastodon-Privacy-and-Search#p-10\">Private and Public Mastodon</a>, Tim Bray, 30 December, 2022 <a href=\"#fnref1\" class=\"footnote-backref\">↩︎</a></p>\n</li>\n<li id=\"fn2\" class=\"footnote-item\"><p><a href=\"https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop\">Read–eval–print loop</a>, Wikipedia <a href=\"#fnref2\" class=\"footnote-backref\">↩︎</a></p>\n</li>\n<li id=\"fn3\" class=\"footnote-item\"><p><a href=\"https://ianbicking.org/blog/2023/01/infinite-ai-array.html\">Infinite AI Array</a>, Ian Bicking, 2 January, 2023 <a href=\"#fnref3\" class=\"footnote-backref\">↩︎</a></p>\n</li>\n</ol>\n</section>\n","body_text":"0.0 Context Setting\n\nIt's another cold, cloudy morning in Portland, Oregon on Wednesday, 4 January, 2022.\n\n1.0 Some Things That Caught My Attention\n\nA few small things again today as I get back into the practice of hitting 15 minutes.\n\n1.1 Success Criteria\n\nAnother thing that stuck in my head from Tim Bray's essay about\nsearch on Mastodon\n[1]\nwas his section on success criteria. Here's the first para, but I encourage you to read the rest (there's only three more paras of roughly the same length):\n\nI’d like it if nobody were ever deterred from conversing with people they know for fear that people they don’t know will use their words to attack them. I’d like it to be legally difficult to put everyone’s everyday conversations to work in service to the advertising industry. I’d like to reduce the discomfort people in marginalized groups feel venturing forth into public conversation.\n\nCaught my attention because: It clearly outlines relatable examples without going into too much detail. Which isn't to say detail isn't important, but to me it's a good example of sketching out a direction as well as different contexts:\n\nFrom a user/poster's point of view, of understanding the particular context they're conversing in\nas an individual with specific needs\n, and understanding usage as more like conversation than written communication, I think\n\nThe context of business usage (\"legally difficult\"), and reinforcing a point in the piece about for example carving out exclusions for usage in ML training.\n\nThe fuzzy goal of reducing discomfort, which presupposes going out and actively understanding what that discomfort may be, while at the same time\nnot seeking to completely eliminate that discomfort\n\nAs an aside, I especially appreciate how Bray's essay doesn't talk about how the usage of hashtags might solve search. Hashtags are useful and interesting, but from my perspective they impose too much work on the part of the writer -- especially if you\nwant\nyour conversation to be indexed and searchable -- as a sort of per-post, opt-in high cognitive cost. Much easier, I think, to either set indexing and licensing globally for the entire account, or if you\nreally\nwant, to tie it to per-post posting contexts. But maybe not! It would be interesting to see!\n\n1.2 Chat\n\nI went along to the Near Future Laboratory's\nGeneral Seminar 26 on ChatGPT futures\nlast week and it was... interesting?\n\nThere's something about ChatGPT that feels limiting and I'm pretty sure it's in the chat/REPL\n[2]\ninterface to the language model.\n\nIt's somewhat astonishing to me that nobody has yet hacked up a Young Lady's Shitty Illustrated Primer. Given infinite AI arrays\n[3]\nappeared this week, it doesn't feel like it would be too far off.\n\nBear in mind this list is in the realm of \"just\" as in \"just do these things\" where \"just\" is \"a giant yawning chasm of implementation\" and the actual job is neatly obscured using one word.\n\nPick a few topics. Maybe just one.\n\nAsk ChatGPT to write an introduction to that topic.\n\nAsk it to write some problems in the form of a children's book.\n\nUse all the above as static text.\n\nBound figuring out the problems in a ChatGPT REPL, i.e. as a text adventure.\n\nFor bonus points:\n\ngenerate dynamic illustrations based on, I don't know, identifying salient noun phrases (see, I can word salad just as well as GPT) or whatever by piping them through whatever image generation model is your favorite\n\ntext-to-speech the entire lot\n\nspeech-to-text the other side of your REPL\n\n(Actually, has someone tts-stt a ChatGPT implementation and thrown it online for people to play with yet?)\n\nSee? Easy.\n\nThe one thing that I'm not sure has stuck around here has been continuous training. ChatGPT is good (I think?) at preserving state through each session (e.g. you can tell it to \"do that again, but x\" and it'll mostly work), but in terms of learning preferences and updating a model continuously, I don't think I've seen that? I mean, I think there's the whole federated on-device learning thing going on, but I don't yet see that in terms of integrating ongoing training with conversational large language models? Or maybe I'm not looking in the right places.\n\nAs an aside, Siri continues to be dumb. Here is an example of what I would like Siri to be able to do, because from the point of view of a stupid user (i.e. me) all the data is there:\n\nMe: Hey Siri, play some music for me\n\nSiri: Sure Dan, here's some music. [plays music]\n\nMe: No no no I get that I have this weird thing of listening to college a capella covers of songs, but I really don't want that right now. Play music for me\nbut don't play any college a capella\n\nSee, this would in theory rely on sufficient metadata like all the college a capella being tagged with the a capella genre, but this whole dealing with sets is something that a) Siri clearly can't do, and most frustratingly, b) is something that Apple Music can't do, and c) is something that iTunes could kind of but not really do with Smart Playlists.\n\nIt's frustrating because you look at it and you think\nugh this is right there\nand you can't really do it? (Note: I know, there are a lot of\njusts\nhere)\n\n1.3 Updates\n\nThank you to the reader who let me know that you can sign in to your\nSquare profile\nand turn off marketing emails for everyone you've ever bought anything from. (Scroll down to Notifications, hit Email, you should be able to see it there)\n\nThis was longer than 15 minutes. In fact the Tim Bray thing itself was about 12 minutes of wall time. Not great for my goal.\n\nHow are you doing?\n\nBest,\n\nDan\n\nPrivate and Public Mastodon\n, Tim Bray, 30 December, 2022\n↩︎\n\nRead–eval–print loop\n, Wikipedia\n↩︎\n\nInfinite AI Array\n, Ian Bicking, 2 January, 2023\n↩︎","raw_format":"markdown","source":"app","summary":"This issue explores how we design systems around human needs and constraints. The first piece examines Tim Bray's framework for evaluating Mastodon search—not through abstract metrics but through concrete user experiences like protecting marginalized voices and preventing corporate exploitation of conversations. The second piece riffs on ChatGPT's limitations, sketching out what an actually inventive educational AI interface might look like (interactive problem-solving paired with dynamic illustrations and speech integration) and lamenting how even basic preference learning and filtering remain stubbornly out of reach despite the data being right there—a frustration that extends from language models to Apple Music's persistent inability to let you simply exclude a genre from playback.","reading_minutes":5,"links":[{"url":"https://www.tbray.org/ongoing/When/202x/2022/12/30/Mastodon-Privacy-and-Search#p-10","normalized_url":"https://tbray.org/ongoing/When/202x/2022/12/30/Mastodon-Privacy-and-Search","domain":"tbray.org","anchor_text":"search on Mastodon","is_archive":0,"archive_url":null,"archive_service":null,"category":"external"},{"url":"https://www.generalseminar.com/season-03-episode-26","normalized_url":"https://generalseminar.com/season-03-episode-26","domain":"generalseminar.com","anchor_text":"General Seminar 26 on ChatGPT futures","is_archive":0,"archive_url":null,"archive_service":null,"category":"external"},{"url":"https://profile.squareup.com","normalized_url":"https://profile.squareup.com/","domain":"squareup.com","anchor_text":"Square profile","is_archive":0,"archive_url":null,"archive_service":null,"category":"external"},{"url":"https://www.tbray.org/ongoing/When/202x/2022/12/30/Mastodon-Privacy-and-Search#p-10","normalized_url":"https://tbray.org/ongoing/When/202x/2022/12/30/Mastodon-Privacy-and-Search","domain":"tbray.org","anchor_text":"Private and Public Mastodon","is_archive":0,"archive_url":null,"archive_service":null,"category":"external"},{"url":"https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop","normalized_url":"https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop","domain":"wikipedia.org","anchor_text":"Read–eval–print loop","is_archive":0,"archive_url":null,"archive_service":null,"category":"external"},{"url":"https://ianbicking.org/blog/2023/01/infinite-ai-array.html","normalized_url":"https://ianbicking.org/blog/2023/01/infinite-ai-array.html","domain":"ianbicking.org","anchor_text":"Infinite AI Array","is_archive":0,"archive_url":null,"archive_service":null,"category":"external"}],"sections":[{"id":3974,"ord":0,"number":"0.0","heading":"Context Setting","level":1,"word_count":15},{"id":3975,"ord":1,"number":"1.0","heading":"Some Things That Caught My Attention","level":1,"word_count":23},{"id":3976,"ord":2,"number":"1.1","heading":"Success Criteria","level":1,"word_count":335},{"id":3977,"ord":3,"number":"1.2","heading":"Chat","level":1,"word_count":518},{"id":3978,"ord":4,"number":"1.3","heading":"Updates","level":1,"word_count":102}]}