• 1 Post
  • 895 Comments
Joined 1 year ago
cake
Cake day: March 22nd, 2024

help-circle

  • Can confirm that about Zitron’s writing. He even leaves you with a sense of righteous fury instead of smug self-satisfaction.

    And I think that the whole bullshit “foom” argument is part of the problem. For the most prominent “thinkers” in related or overlapping spaces with where these LLM products are coming from the narrative was never about whether or not these models were actually capable of what they were being advertised for. Even the stochastic parrot arguments, arguably the strongest and most well-formulated anti-AI argument when the actual data was arguably still coming in, was dismissed basically out of hand. “Something something emergent something.” Meanwhile they just keep throwing more money and energy into this goddamn pit and the real material harms keep stacking up.




  • That’s probably true, but it also speaks to Ed Zitron’s latest piece about the rise of the Business Idiot. You can explain why Wikipedia disrupted previous encyclopedia providers in very specific terms: crowdsourced production to volunteer editors cuts costs massively and allows the product to be delivered free (which also increases the pool of possible editors and improves quality), and the strict* adherence to community standards and sourcing guidelines prevents the worse loss of truth and credibility that you may expect.

    But there is no such story that I can find for how Wikipedia gets disrupted by Gen AI. At worst it becomes a tool in the editor’s belt, but the fundamental economics and structure just aren’t impacted. But if you’re a business idiot then you can’t actually explain it either way and so of course it seems plausible



  • As the bioware nerd I am it makes my heart glad to see the Towers of Hanoi doing their part in this fight. And it seems like the published paper undersells how significant this problem is for the promptfondlers’ preferred narratives. Given how simple it is to scale the problem complexity for these scenarios, it seems likely that there isn’t a viable scaling-based solution here. No matter how big you make the context windows and how many steps the system is able to process it’s going to get out scaled by simply increasing some Ns in the puzzle itself.

    Diz and others with a better understanding of what’s actually under the hood have frequently referenced how bad Transformer models are at recursion and this seems like a pretty straightforward way to demonstrate that and one that I would expect to be pretty consistent.








  • A) “Why pay for ChatGPT when you could get a math grad student (or hell an undergrad for some of the basics) to do it for a couple of craft beers? If you find an applied math student they’d probably help out just for the joy of being acknowledged.” -My wife

    B) I had not known about the cluster fuck of 2016, but I can’t believe it was easier for the entire scientific establishment to rename a gene than to get Microsoft to introduce an option to disable automatic date detection, a feature that has never been actually useful enough to justify the amount it messes things up. I mean, I can believe it, butI it’s definitely on the list of proofs that we are not in God’s chosen timeline.





  • I guess that’s fair. I was focusing in on his attitude towards craft, which seems incompatible with actually taking pride in doing a good job as opposed to simply skating by. But while I still take issue with his attitude there and want to give him a clockwork orange-style refresher about tech debt I think a bigger problem is that he’s taking predictable problems of the median programmer trying to use these systems and saying, effectively, “get gud”. This is especially galling given that the tech here is going to replace or supplant the kind of junior developer roles that allowed fresh graduates to actually get that experience that allows you to shepherd the next generation of junior devs (or I guess LLM assistants now).