We Tracked a Shipment of Rare Books. It Ended at an Amazon AI Training Facility

andsoitis

> We’re not revealing the titles of the books included in the shipment we tracked, but they are rare, meaning there are not many copies of them in circulation. Sometimes that’s because not many copies of them were ever printed, and sometimes because they are in a foreign language not many people speak. Not even the title of one of those rare books?

koliber

I watched Short Circuit yesterday. All I can think now is "need input".

Havoc

Bad enough to steal IP but destroying books…that’s just evil

jcarrano

According to the article, the scanning facility has been there for a while, so it may well be that they have been destructively scanning books for a while, even before AI. It makes sense for the major book retailer to have digital copies of every book so that it can begin selling those as soon as it can negotiate the rights (if there are any). In any case, all the indignation about destructive digitization misses the point that rare books takings space in a warehouse for years without being bought will eventually be destroyed anyway.

fg137

I don't understand why Amazon still bothers to blow money on their in-house AI effort (outside AWS Bedrock). What have all the scraping, scanning and training led to? Nobody is using their models.

fancyfredbot

I look forward to the day that AI companies pay large advances for new books. It'll take a while before publishing collapses due to the availability of the same information via an LLM, but once this happens new books will get a lot more expensive for AI companies and a lot cheaper for everyone else.

infecto

Does anyone else get pretty tired with 404s style of writing? I hate to make this comparison but it feels like a Fox News for the left. I always struggle with their articles because it feels like they built it for rage bait on a topic and leave the other interesting topics out of it. It is an absolute shame for books to be destroyed but what does it really mean to be rare here? I know they kind of tried to differentiate but it sounds like this could be John Doe’s self help book that never sold well. If you ever are connected to a library you will start to realize how many books simply get thrown out or sold for nothing because nobody wants them. To me the problem is partly the copyright law. I think it’s. Hard problem but I always lean more towards books having short copyright shelf lives and making it legal for digital copies to be shared after which I think would eliminate I good part of this problem. Not to mention 99% of the books published are probably garbage but that is highly subjective. So it is bit of a meta rant but I think there a couple holes to go down that could be extremely interesting but they always write these informationally light articles. Like scrolling through a NYT visualization for just some shipping datapoints. Don’t really dig deep on anything and then end with a trust me bro these are rare books that Amazon is destroying for AI when I cannot be that upset with the amazons of the world. There are a lot of reasons for a business to digitize books, most books are worthless and it makes sense to cut the bindings for scanning. I would rather talk about how could you fix copyright to make this less an issue but is it even an issue with how many books get thrown out?

tdeck

Why is the textual data bound up in these paper books worth the trouble? These LLM training runs have already ingested essentially the whole public internet. What marginal value is to be gained from scanning and destroying obscure books? For me the biggest functional issues with LLMs don't seem to have any connection with "I wish they had read this obscure community cookbook from 1946". Is that going to get Claude to stop saying "honestly" to me? Is it going to get LLMs to stop making up sources that don't exist? What is in it for Amazon or any LLM company to chase more obscure data like this.

Aurornis

The length of this article and the unnecessary visualizations felt like a waste after getting to the end and discovering they won’t reveal anything about the books that were scanned. The closest they got was admitting that the rare books weren’t anything that someone might care about in the sense that people assume when we hear “rare books” > As the bookseller who sold them told me, there are not many people in the world who would care about them in the same way people might care about the first edition of Oliver Twist, but that doesn’t mean they’re not valuable. Okay? But then why exactly where they considered valuable enough to warrant an entire article about them going to a book scanning facility? Without revealing anything about these books I have no idea if they were classic literary works that were underappreciated, or if this was some old guide about How to Use Microsoft Office 97. There was a more balanced take on Twitter (which I’m unable to find again, because Twitter) from a book seller who said it was more of the latter type: Books that were rare because they were no longer in demand and most everyone had thrown their copies away. Some parts of the media are doing backflips to try to imply that these are cherished literary classics being fed into the shredder to deprive humanity of something valuable, but the book seller seemed happy to be making sales for useless old books that no human was interested in buying.

faidit

How many AI companies are doing this? Say there are only 5 copies of a book left out there but 100 different companies want to shred it. Rare items could effectively vanish from the market. "Rare" meaning inclusive of high-quality items, disregard the bot opinion that mass book destruction is fine because most books are worthless anyway. And are any safeguards in place to make sure scanned books get saved in their original form for posterity before being recycled into trainingslop? Aside from Google's quasi-legal/ethical mass book scanning operation of course.

ChrisArchitect

Discussions: 2 days ago https://news.ycombinator.com/item?id=49310725 21 days ago https://news.ycombinator.com/item?id=49068738

im3w1l

Something I've been thinking about is that a physical book is a physical artifact, and it's age and authenticity can be verified by treating it as such. In contrast a digital book has no such marks. It's impossible to tell if it was redacted to hide inconvenient passages, or even completely rewritten or fabricated (which can now easily be done at scale with AI). If the physical book is a primary source then the digital version must be considered a secondary source: A potentially biased retelling. For this reason, a digital book can never be a perfect substitute for the physical book it was created from.

ButlerianJihad

It seems like most bookshop owners today are hoarders-turned-entrepreneur. They rent a storefront, adopt 2-5 cats, and they set up shop with their hoard of books so that they can hoard on a commercial level that would never be possible in a private home. It also seems that American society has been swept away by the worship of books, rather than literary appreciation or promotion of education. "Banned Books Week" as the primary exhibit here. A tug-of-war over books that are supposedly "banned" when the verb itself has been twisted beyond recognition. I see posters and television shows and public service announcements that promote "Books!" and "Reading!" for no other purpose. Many books are trash and they will fill your head with garbage as sure as social media can, so why the indiscriminate worship of books? It is conspicuous that, aside from established booksellers, and perhaps the LFL owners, many of the people crying out and moaning over book-destruction seem unwilling or unable to actually take those books in and store them. That is the point, that these books are unworthy of taking up storage space, which is an ongoing cost, and maintenance, and therefore, it is fiscally responsible to sell them, scan them, and destroy them. If you want to dedicate a room full of shelves to books that nobody will ever read, then go ahead and buy out your local bookseller! They will not stop you or cancel your order! Tell them you are saving those innocent books from the Big AI Bugaboo! They'll grovel at your feet! Also now, I'm seeing small booksellers who feel "suspicious" or "skeptical" about large book orders. There was one Down Under who said "oh we got an order for over 70 and we can't physically handle that!" I feel like it's becoming a "shut up and take their money" situation. These booksellers, as professional business owners, should know that they may never liquidate stock, and this is their golden opportunity to simply get rid of some of that excess hoarded inventory. But if you're merely cosplaying as a bookseller, and deep down you're a hoarder and a worshipper of books, you may really be reluctant to do transactions or earn money that sells away your books.

bhouston

Great article and work to reverse engineer who was mass buying used books.

GuB-42

> As the bookseller who sold them told me, there are not many people in the world who would care about them in the same way people might care about the first edition of Oliver Twist, but that doesn’t mean they’re not valuable. So books that would probably have ended up as trash. These AI training facilities are actually doing these book a service. Not only they are probably going to keep the scans safe (for future training), but having the book end up in an AI model may be the only way it is going to have any use at all. Let's say for instance that the book in question is about woodworking, and it is not great, lots of mistakes and inaccuracies, unoriginal content, etc... except for a single thing, maybe a trick for making a certain measurement or something like that. Who would read such a crappy book for this single good trick he doesn't know is there, well, an computer will, computers process terabytes of crap without tiring and complaining, that's what they are for, and with a well designed LLM, that one trick may resurface, waiting for someone to ask about that specific measurement. The problem here is not that rare books end up in AI training facilities, it is that these AI facilities are owned by for-profit companies keeping the data to themselves. These books should go to public libraries instead, for everyone to access, it would be better if these books weren't destroyed in to process too. But the question becomes: why didn't public libraries didn't do that in the fist place? And maybe in a more respectful way. The AI companies would just have had to license the database to libraries, probably simpler and cheaper than having their own scanning facilities. To me, this mess is a failure of the copyright system. One one hand, large scale book digitization projects intended to preserve and make the original text accessible get lawsuits by publishers, while AI training is "fair use". It means we have built a system that encourages destroying rather than preserving these books!

AceyMan

While we can quibble (and are) over the details of this particular occurrence, I'm beginning to take the opinion that used/rare booksellers need to implement some form of "KYC" to help guard against the epistemic threats posed by this practice.

wheatone

Nothing but a drawing of an alligator eating a book. Where is the article?

wheatone

Link to URL? There is no article at the above link.

whatever1

This is what the copyright laws dictate no ?

VariousPrograms

I support physical media and doing whatever you want with said physical media. If you want to overpay for a copy of Sharepoint 2007 For Dummies and destroy it, knock yourself out. Just because media has been printed and is "rare" doesn't mean it has any practical value. You could easily write an article about how Goodwill and the Salvation Army dump millions of "rare" books that no one would ever buy for $0.50. And thrift stores will dump multiple copies of said books. AI companies would only ever need one! The only thing I disagree with is AI companies feeling entitled to freely use every piece of copywritten work without restriction, but that applies just as much to web scraping, pirated books, etc.