Not another language because the performance issues are limited to the code being sync. Anyway, a gradual migration was a must as well.
This is the first I hear of Litestar and I think it does seem very values aligned. I fastapi was picked simply because it was already known by the team and had a very thriving ecosystem. That being said, the hard part is getting off the old sync web.py stuff. Once everything is in async land moving to another framework if needed is relatively trivial. Thanks for sharing that though I'll keep an eye on it!
I’ve been using litestar for loads of projects recently and have been loving it. The ability to have full request/response typing (and openapi spec autogeneration) with your favorite dataclass framework is quite nice (I use attrs). I didn’t want to be forced to use pydantic like some other frameworks do.
I was one of the initial developers of Open Library and wrote most of it's backend code. I doubt any of the performance issues could be because of code being sync. Open Library is IO bound and it is most likely spending time waiting for db to respond.
Was there any performance analysis done that indicated that the "code being sync" the main bottleneck?
The issue wasn't the latency of a single DB query (though many were improved/cached), but sync worker saturation:
When traffic spikes, slow network calls to archive.org (for availability) and Solr tie up all available sync workers. The queue backs up, and suddenly every page becomes slow because there are no free workers left to handle requests.
This was handled from both sides: handling high-concurrency waiting better, while simultaneously optimizing Solr and removing archive.org from the critical path where possible.
I see the problem. But I feel switching to async is too big a change and there could be simpler ways to address this.
1. Solr was used (I think still continue to be used) for many things it is not quite the right fit. It was done then because, it was simpler to do. For example, it is used to get list of editions when a work is loaded. Limit Solr only to search and figure out a better way to manage the other pieces, would be a good start. Search could be made a different service and the frontend could call it via API. Both of these together will remove Solr from the critical path.
2. The availability status of archive.org could come from a database and a backend task can continue to update it very frequently.
Wonderful to see @raybb and @anandology in this thread.
Lots of operational challenges come up when running a service for 14M patrons.
And Open Library in particular has a handful of challenges.
1. It's database has grown significantly (800+ GB) and Anand is right that IO (even on SSDs) is a challenge. The `thing` (infobase/infogami) triple-store design is well thought out and gets us a lot, and any system has to be tuned as it scales to hundreds of millions of rows. One strategy here is being smarter about cache and also shifting some of the load from psql to solr. Rishabh and others volunteers have been amazing assets as we've moved in this direction. Jim Champ on staff has been helping me tune psql, pgbouncer, and some of our high IO crons to improve raw db performance.
2. Limited hardware resources. We're trying to move some of our services within the Internet Archive's kubernetes cluster and we've done a great job migrating towards a world where everything is dockerized. It used to be a very painful process for our team of 3 to handle server ops, upgrades, and networking for nearly 15 manually orchestrated servers. One of the bare-metal racks running much of Open Library is significanly oversubscribed on vCPUs and so moving services off to free space and eliminate steal is critical for us right now. Our main web server (ol-www0) suffers from up to 20% steal and we're seeing a lot of congestion before requests even get to our web nodes (app servers). We have a plan and it takes time.
3. Open Library is still dependent on Archive.org for many lookups -- like book availability (which Ben Deitch has been helping me and Drini move into solr). When there are network issues and a network requests takes 5+ seconds, every web.py worker on that thread grinds to a halt and Ray's work moving us to FastAPI has made a significant impact
4. Solr. Drini has been heroic at restructuring our setup to use replicated solr in a way that has increased performance and relieved some of the pressure on our main cluster. This was a huge bottleneck for us this time last year and we've taken a lot of steps to ameliorate our situation. See: https://blog.openlibrary.org/2025/09/12/open-library-search-...
5. Raw spikes in traffic. We are seeing massive amounts of traffic that slams our book pages, increasing the pain of all the above. It saturates our limited resources, puts more strain on our database, ties us web workers... It makes modsecurity even more expensive. Part of the solutions is being more clever about provisioning, part of the solution is using fail2ban to prevent bad traffic from subtracting from the experience of the patrons who depend on us. Part of the solution is caching and optimizing our database to scale with load.
There isn't just one solution and the same 3 engineers on staff (and the support of a completely stellar community of dedicated volunteers fellows and leads) are doing our best to balance ops improvements with the necessary "product" and design improvements necessary that ensure we're useful to people to begin with.
I hope this gives the world a bit more of a glimpse how we operate and what some of our challenges are. We're an open source project and our goal is to share as many learnings as we can and to build something useful, sustainable, and beneficial for the community at large.
Thank you Ray, Anand, Drini, Jim, Lokesh, Lisa, Charles, and so many dozens more for your tremendous work (present and past) and thank you for being in our corner.
It’s not whataboutism when you’re comparing the exact same actions and the two things can be substituted for each other.
Whataboutism is “how dare you complain about our gulags when you treat your black people like slaves.” It is not “our system kills people but it kills fewer people than what we currently have.” Especially when killing people is a result of being on the bleeding edge of what’s possible rather than being a deliberate choice to hurt people.
I think, pragmatically, we have to accept that death is the outcome for us all, and trending towards fewer avoidable deaths and longer fuller lives is approximately the aim. Nonetheless deaths from a technology is inevitable.
I believe what you're trying to talk about are excess deaths when a technology is less capable than the incumbent? I'm not sure the stats, but I'd suspect FSD is already superior to the often distracted, sometimes drunk, phone in hand driver that is the norm today.
If chatgpt is to be trusted, Waymo and Tesla FSD are already massively safer than human drivers.
Yeah? We need millions of personal, individual trucks? For work, of course, ignore all those white collar job parking lots full of extended cab pickups that get 10mpg.
Have you considered that sometimes people buy automobiles that can accommodate multiple use cases? Just because you don't need to haul hundreds of pounds of supplies every single day doesn't mean that you don't need to do it a couple times a month, or sometimes every day for short bursts of time.
With vehicles being as expensive as they are, some people prioritize the one vehicle that can suit their regular _as well as_ occasional needs.
To put it into terms this crowd might understand better, to a lot of regular people the wildly expensive and overpowered computing hardware we own seems like a ridiculous waste. And certainly, we don't need all that horsepower every minute of the day. But we need it from time to time.
I can see what you're about to say though: those people who buy trucks should just drive some crappy little three cylinder for day-to-day use and rent a truck when they need it. To which I'd ask, how would you react if someone who knew absolutely nothing about what you do on a daily basis told you that you should just buy a dinky little Chromebook and spin up a beefy EC2 instance every time you need some horsepower?
Maybe you don't understand people's needs and motivations as much as you think you do. And this is frankly where a lot of resentment to the climate action stuff comes from. A lot of know-it-alls lecturing people on how their lifestyle is completely idiotic based on very little understanding of how those people might actually be living their lives.
"I see a pickup truck with nothing in the back, therefore in my expert opinion you don't need it."
Hauling "hundreds of pounds of supplies every single day" is a fringe use case, essentialy nobody does this, it's not relevant.
"Maybe you don't understand people's needs and motivations as much as you think you do."
It's the opposite. You're confusing "needs" with "wants" and extrapolating the needs of the very, very few people who genuinely need a truck onto the entire population.
The vast majority of people use their cars to transport only themselves to work each day, then maybe pick/drop off one more person, then maybe go buy groceries.
I don't think you quite understand just how ridiculously niche valid use cases for trucks are. It's literally just towing, or some very specific off-road hauling. Everything else is going to be handled way better by something else, starting with a regular car, ending with a van.
Hell, for most people even a car is way overkill, a bike works perfectly fine to go to work each day, I assure you. So does public transport. You don't need 2 tons of steel just to to move your 100 kg body around.
To your analogy, I can use my dinky little chromebook 99% of the time and only boot up my PC when I actually need the power.
The current potus was directly elected because “know-it-alls” kept trying to jam ideologies down peoples throats. I have this awful feeling we’re going to swing back that way, and the escalation on “both sides” will be an absolute disaster.
I used my general physician. It may not be as high quality as a psych, but he determined the symptoms correlated with the diagnosis and the medicine has helped me immensely in my life.
There are also all the companies building iOS apps that need mac mini compute clusters for CI/CD because of dumb Apple requirements (ask me how I know..).
reply