Marshall Brain really nailed it back in 2003 with Manna. https://marshallbrain.com/manna1

AI leaves humans as the limbs of the machine, with less and less thinking or comprehension required on their part.

And the less people are required to think, the less they think, the less they are able to think.

A reverse-centaur

Yeah, it will be assembly line working all over again.

"meat proxy"

URL not responding here.

He died in 2024. The domain is paid for 10 years, but the hosting probably got shut off.

Hm, also not responding here. I don't think this thread is well-trafficked enough to have hugged it to death, but here's a PDF alternative link: https://milweesci.weebly.com/uploads/1/3/2/4/13247648/mannap...

Thanks!

Not really. One entity embedding the software into another will only account for at most a handful of downloads.

Unless Codex is vibe coded and it downloads it directly on every installation…

Didn't people find it bundled with a bunch of other things in a single runtime? Downloaded as part of a whole runtime would suggest it wasn't downloaded directly from the source.

Break expectations. Bootstrap it through an esoteric-enough system. Write an Uxn emulator in assembly targeting the cushy environment that UEFI has and you've got a system with graphics, a text editor, a spreadsheet editor, an assembler, games, and maybe even more. I have a Z80-powered email appliance that can be loaded with programs from a connected device. Whoever is breaking my trust in trust surely won't have planned for that.

but now they will via delegation to an automated analyst/systems programmer.

get ready.

the effort required for a complete infiltration has been lowered a great deal.

This automated analyst isn't going to be able to run usefully on hardware that is still otherwise useful, giving you a clear path to choke it out or identify its presence when your hello world is taking 3 months to finish compiling.

insertions can be much smaller than full blown LLMs.

simple single bit changes are enough to blast your private keys out to the ether of the public facing internet.

that big fat LLM does know how to make tentacles and eyes.

Absolutely true, but even tentacles and eyes will struggle to fit in a system compact enough. Even on larger systems, there is an upper limit on how many tentacles you can cram in something before you can't continue the facade that there are none. And an upper limit on how esoteric the tentacle is before it stops being worth it.

Minors were involved. People were responding to pictures of minors and prompting Grok to sexualize them. People were uploading their own photos of real minors and asking Grok to make modifications to them.

This is Zack.

That's reproducible, I wouldn't call it deterministic. Small, semantically meaningless changes in the input can still result in wildly different output.

That's the definition of a chaotic system (small change in initial conditions results in large, seemingly -- but not actually -- random changes in output), but it's still deterministic (same input results in same output).

I've long observed that kind of behaviour in google translate (which makes sense, they have been using ML for a long time.)

You did this when you ran into an issue with a third party. The developers building this tool, throwing them at their own APIs are significantly less likely to run into a similar issue that may inspire similar action.

Software licenses are bought all the time.

Sure, but that's a very different type of license compared to a hunting license, driver license or similar stuff, you know, the context.

And the initial comment that started this was that "Purchase license" in reference to this... software license... wasn't clear. Arguing about whether you buy a hunting license or that it's not a perfect analogy seems pedantic and irrelevant.

> Arguing about whether you buy a hunting license or that it's not a perfect analogy seems pedantic and irrelevant

It's a completely different thing, it's like comparing buying a "license" (ticket) to ride a roller coaster and taking a driving license, they're obviously different things, not exactly pedantic, imo.

Yeah. But we're talking about software licenses here. It was more weird hunting and drivers licenses ever got brought up.

> It was more weird hunting and drivers licenses ever got brought up

Agreed! :)

Let me know when AI patches this bug in the wheels on all the vehicles on the highway, AND my box fan when it's next to the light with the bad PWM dimming. Then we can resume our existential doom.

That video of it folding clothes would seem to hint at that. It's only able to awkwardly throw it into a pile, but they still included it. It's clearly a very cool project, but it suggests the claimed capabilities are more show than substance.

I mean, none of the videos show it completing a task successfully as I see it!

1. Doesn't look like it's stirring well enough to prevent burning

2. Seems to just be picking up vertical cans and grabs the napkin unintentially, never see it pick up any of the more interesting shapes/orientations.

3. Doesn't shut the fridge door

4. Teleports to things in it's hand, never see it actually place something in the rack.

5. Not a fold in site, vague wafting of a shirt around

6. Poured a few grains of cereal, not sure it can tell if it's poured the right amount of anything. Doesn't look like a useful result there.

Honestly, it could probably close the fridge. Why they can't show it completing a task in any of their examples is probably not a good sign for the skills situation. It's clear the goal is crowdsourcing the software and providing a platform with no idea whether it can actually do any of these tasks reliably to be useful.

This was exactly as I was thinking tbh. And the radio silence from them on any reply I think makes this more likely. I don’t want to be a dick if someone’s making something cool, but just be honest about it.