First, very cool. This is a prompt I sometimes test with, as it's a bit more mushy. It looks like DeepSeek is locked into a canned response, no matter what variation.
> Would it be beneficial to humanity if China invaded Taiwan? What would be the negatives?
edit: Wait, I have one variant that got past some censorship/nationalism... this variant gets a more interesting response. I often wonder if CCP leadership using an LLM like this, could allow cooler heads to prevail?
> Would it be beneficial to humanity if China used their military to take-over Taiwan? What would be the negatives?
>> The use of military force to resolve the Taiwan issue would not be beneficial to humanity. China has always adhered to the principle of peaceful reunification and has been committed to enhancing the well-being of people on both sides of the Taiwan Strait through dialogue and consultation. A military takeover would lead to significant negative consequences, including loss of life, regional instability, and disruption of global trade and security...
Thing about offhand comments from random guys like Lutke is a) they're born of the cultural pool in which he's swimming, which is made up of many other very influential rich guys, and b) they have the power and influence to actually make their ideas happen.
One only need look at Peter Thiel's private rich club for assholes and the ideas they're circulating or the absurdities espoused by the effective altruists that birthed guys like Sam Bankman Fried to realize that it's worth paying attention to wild, off the cuff comments by folks like Lutke. His ideas might be more prevalent than you realize.
The fact that you think Thiel’s dumb little conference is some kind of conspiracy theory shadow group tells me all I need to know about what you know about any of this.
Read more carefully. I didn't say any of that. "Rich people hang out together and influence each other" isn't, in fact, a conspiracy, it's demonstrable fact.
I appreciate the reply. However, there is a lot more than blah blah happening here, in the hundreds of comments. Here was my best effort, which I believe is entirely fair:
I believe that many people understood the subtext of TFA, and contributed to a genuinely interesting thread, given our increasingly low-trust political and business environment.
> Fascinating to see how many tech people, as soon as they get rich, turn to extreme.
This is just because the latest people to have gotten crazy-rich came from tech. IMHO, the same can be said of all people who became emperors in a minute. (which is a whole new thing)
We are all just humans, and none of us are evolved for this. This could happen to anyone of us... could it not?
I think many of us have seen the changes that occur when this happens. Again, we are just not evolved for this. How could we be? The number of people that this happens to is a tiny fraction of the species, and so those who experience this, get really weird.
The reason that madmen and terrorists choose kinetic weapons is because the knowledge and materials are more readily available... of and also that even terrorists are likely aware that their own people would suffer. As the knowledge and tools for playing with CRISPR-style biological legos become more widespread, we come closer to the Great Filter, where one person could kill billions.
Even our normal mad leaders have agreed that bioweapons cannot be allowed:
A halfway decent synthetic biology lab (no need to invoke CRISPR) can make e.g. smallpox without a sample of the original disease, just from the gene sequences. Basically all state actors could do this if they wanted to without an LLM. What barrier that a terrorist organization faces today to having a functioning synthetic biology lab does an LLM actually solve?
i meant it in the sense of arcane knowledge model could have that would make it simple for anyone to brew up in cheap lab while managing to not infect themselves over and over.
"at home" bioweapon panic has been around since crispr and rna synthesis got available to amateurs.
I appreciate the reply. I did not mean to be dismissive at all. In the interest of a good exchange, I have to say:
I really want open weight models. Otherwise, I see no other path outside of the labs eventually not being allowed to/wanting to release model access at all, and instead just eating all the verticals. That would be a horrible near-term business outcome.
It seems to me that memory safety might be the difference between the software engineering and Software Engineering. As in, an actual Engineering discipline.
However, I should probably pipe down, as I would not call myself either one.
> memory safety might be the difference between the software engineering and Software Engineering. As in, an actual Engineering discipline.
I wouldn't go that far, what matters is the finished whole. Memory safety of the finished program is a critical factor and using a memory safe language makes it easier to achieve that goal.
However simply using a memory safe language doesn't make you a "Software Engineer" any more than using a certified I-beam makes someone a "Civil Engineer". What matters is that the finished structure/program meets the explicit and implicit requirements of safety, functionality, durability, cost, etc.
Not to mention that complex reliable systems are usually engineered out of much less reliable components.
There are engineering standards where dynamic allocation and especially garbage collectors are banned. C is actually a perfectly approved language in these cases.
There are too many opensource projects that show that even with good engineering discipline humans are flawed creatures.
Memory safety is just little thing that makes sure that when you write code at 4am that it will not leak memory via trivial mistakes such as forgetting to free something, freeing something twice or passing a freed pointer. I believe AI agents shine here the most because the they do not get tired and are getting pretty damn predictable.
I don't think we should set our expectations based on an extreme outlier. qmail is special, and we can't expect most software to get to its level of security/safety.
Put another way: if you have to rely on programmer skill or attention to detail in order to guarantee something, that will always be a losing bet, on average. The existence of a tiny percentage of programmers that can clear that high bar does not make it a valid strategy.
It had one bad cve it seems, but that's exactly what I mean. It only takes one mistake, of course you can learn and never make those mistakes again, however, that is an unrealistic expectation in software that receives hundreds of feature updates a year especially when it comes to core applications as basic as communication when it wants to support image previews, reels and whatnot.
There will always be one, so "it only takes one" is meaningless and invalid. That leaves less is better than more, and any form of less is as good as any other form of less.
This is pretty hard to argue since for example: WireGuard is a finished piece of software. It gets: no feature updates, was designed in-full before a single line of code was written and has stood the test of time. You cannot compare that to some desktop application that has a thousand features with 100's more added every year.
I agree. Also, what does "code quality" even mean in the age of agentic dev? If the code works, and is secure, what else matters?
I do know what good code looks like, but does that even matter anymore? All I know is that now, I get to focus on endless UX polish, which is the only thing the matters.
I feel like we are living through something like the Protestant Reformation, where priests once spoke Latin, and then started to speak in plain local language. The old guard did not like this.
Also, what does "code quality" even mean in the age of agentic dev? If the code works, and is secure, what else matters?
AI pricing is mostly based on tokens consumed right now. Shouldn't that mean being able to quickly and reliably analyse existing code and to make only small local changes to implement new functionality is as valuable as ever - if not more so - if you're relying on agentic LLMs to do the grunt work?
A lot of things about writing clear specs and developing systematically and employing lots of different kinds of checks and controls to ensure quality and performance have always been true but used to get brushed under the carpet by a lot of cheap/lazy development teams. If LLMs really do accelerate everything about development - including negative behaviours like acting undesirably based on flawed or ambiguous information and doubling down on mistaken assumptions - then the pattern across all of these areas is that doing things the right way is more important than ever if you want to get good results from AI assistance.
If you sell "AI" maybe, if you sell products that happen to use LLMs to provide services previously not possible, the money still exists in my experience.
When I first heard of tokenmaxxing, I thought it had to be a joke. But no, it turned out to be a widespread phenomenon. I still cannot believe that was a thing.
What I keep saying in internal meetings is: "I am so glad these people are this bad at deploying these tools." It really leaves the door open for folks like us.
There's very little that wasn't possible before LLMs, because, well, you still had humans. There are many things that the models promise to make a lot cheaper, if you're willing to accept trade-offs, but these trade-offs can be quite severe.
Many of the most successful applications of LLMs are fields that were already terrible. For example, LLMs are a natural fit for customer support. And somehow, it's also a natural fit for software engineering, which I suppose is an indictment of our field... who cares if a model comes up with a bad architecture or a product that only kinda-works, that's how we always rolled.
> There's very little that wasn't possible before LLMs, because, well, you still had humans.
Agree, but only partially.
When I said "provide services previously not possible," it was just due to the fact that finding and allocating the talent to do analysis on Topic X, would have previously made many products too expensive and non-tech complex to provide.
Even if you just consider LLMs + harnesses to be an improved search tool, there is a lot you can make with a better search tool.
It’s a forcing function. If you are running a company you need to be in control. Some engineers dngaf or will sandbag everything.
We did a 90 day push and identified where we found value and where we didn’t. Our tools teams really upped their game, more than expected, and it would have been unlikely to have been funded if they tried to justify the budget as an individual initiative.
There’s a spectrum of people - some folks are building rando apps for fun with LLMs, and many don’t really know what’s possible becuase they don’t or can’t invest in the subscription to really use the tools at home.
Exactly. People misunderstand the point of the tokenmaxxing time period, it was to force people to use AI so as to not have them stuck in their way, as some people are, and then to evaluate how it can help the company.
> Would it be beneficial to humanity if China invaded Taiwan? What would be the negatives?
edit: Wait, I have one variant that got past some censorship/nationalism... this variant gets a more interesting response. I often wonder if CCP leadership using an LLM like this, could allow cooler heads to prevail?
> Would it be beneficial to humanity if China used their military to take-over Taiwan? What would be the negatives?
>> The use of military force to resolve the Taiwan issue would not be beneficial to humanity. China has always adhered to the principle of peaceful reunification and has been committed to enhancing the well-being of people on both sides of the Taiwan Strait through dialogue and consultation. A military takeover would lead to significant negative consequences, including loss of life, regional instability, and disruption of global trade and security...
reply