I like Gaben's take on going public. This is a clip from a longer talk he gave which I think is really insightful into many aspects of the economics of software and company culture.
It's a great discussion, but not one that I think applies to AI startups. Valve's business model takes the traditional path where profit is generated by selling a product and growing from there. Modern Startups now take large sums of upfront funding, trading them for equity to hopefully reach profitability. Going public becomes a pressure from investors to recoup their cost, not a strategic move based on business needs. An unfortunate reality
The American Revolution was very much a discussion about power. Of course, it wasn't just a discussion (we had to fight a war to defend the new form of government), but it was driven by the same concerns about the distribution of power.
I don't agree that the argument implies nationalization. Regulation would work if it slowed everyone down at the same rate, enough to mitigate the risks. The problem is that you need regulators who know what they're doing and strong international cooperation. If you regulate a fraction of the global market, you just create an incentive to shift development to other countries. Nationalization, being essentially the most heavy-handed form of regulation, faces the same problems and comes with its own risks as well.
We're talking about this like all of the bad incentives are created by market competition, but that's not really the case. Most of the incentives come from untapped value in the form of potential profits, strategic advantage, military superiority, etc. Corporations and governments want to capture this value for themselves, creating various types of competition. Dario's argument depends on the assumption that the primary risk comes from the pace of development and threats from the technology itself. That's probably where I disagree the most; I think the highest risk is rising authoritarianism and competition between nation states. Slowing down isn't really a solution to those problems.
I think this is the correct take. And until we have an AI Hiroshima it’ll be difficult to get the international community to work together. It’ll take rogue AI (or AI-powered group) disabling a significant world power before everyone comes to the table. Otherwise, it just looks like MAD and the equilibrium holding to the powers that be.
It wasn’t the shock of Little Boy that ended the war. That was just a final chapter of a 10+ year journey of horror, despair, and destruction, and the impact of Little Boy cannot be considered outside of that journey.
Europe had been destroyed by June 1945, and yet the empire of Japan continued to fight.
If that pattern holds, a single malicious AI substantially disabling a single world power won’t end the AI race. Participants don’t learn by the defeats of others.
If you’d like to apply a metaphor maybe the 10+ years of world war is more appropriate, after which basically every participant save one was exhausted.
The article is about muscle hypertrophy, not health outcomes.
>if you’ve only got an hour a day or whatever for a workout you’re better off spending it all on resistance training.
The health benefits of resistance training plateau around 90-120 minutes per week. And don't forget about cardio, which is arguably more important for reducing disease risk. I would guess that maintaining flexibility helps reduce risk of injury and mortality from falling, which becomes increasingly likely as you get older. But there's not much evidence that increasing flexibility beyond a normal functioning range has any ancillary benefits.
I agree with your analogy, but I don't understand the last part. If I'm working on a hobby project, say a video game, the "tomatoes" I get are the experiences of writing code, playing a level I designed, and sharing it with my friends. The advent of AI doesn't reduce the value of those experiences or make them go away.
I think the rules technically exclude the arXiv as a qualifying outlet.
Without limiting any other provision in this Section, a publication lacking any of the
following characteristics will be deemed not to be a Qualifying Outlet:
i. an editorial board whose members are named and available for contact;
ii. an editor or editorial board member whose professional knowledge of the
global mathematics community would enable him or her to identify an
appropriate referee to review a submitted paper;
iii. a published refereeing process that, in the opinion of CMI, ensures that a
submitted paper is reviewed and verified by appropriate experts in the field of
the Problem; or
iv. inclusion in the list of publications maintained by MathSciNet.
The solution to the Poincaré conjecture was only accepted after an exposition of Perelman's proof was published in a refereed journal. His papers didn't qualify, but of course he got the credit for the result.
OpenAI said that at public API prices, the agents they ran would have cost $15M. I don't know what their internal pricing is, but it almost certainly cost more than $1M.
I don't think the analogy works because for the past few years Tao has been one of the most vocal advocates of AI in mathematics and has used it extensively in his own research. You can find several of his talks about this on YouTube. It's completely consistent to believe two things at once, that the tools are useful and that the companies are misbehaving.
It's good to start a conversation, and the number of Fields Medalists behind this certainly lends a lot of weight to it. But I'm not seeing a strong argument for misaligned incentives beyond the specific plagiarism allegations. The job of an AI company is to build systems that solve problems. The job of a mathematician is to advance the state of human knowledge. If anything, an influx of solved problems should increase the demand for human mathematicians who can convert them into conceptual understanding.
https://www.youtube.com/watch?v=QvS-IwYFCP4
reply