Tagged: Akkoma

Only a few pages into this paper that I just learned about via @timnitGebru, but am hugely intrigued by the findings and what this must eventually mean for regulation and policy around LLMs.

Essentially these models will remain insecure as they increase in size and lead to all sorts of serious problems. On the one hand they will inevitably contain significant amounts of privacy-sensitive information. On the other hand they will continue to be vulnerable to poisoning attacks. There is not enough trustworthy content in the world and to remain safe they must be reduced in size - and yes, performance must drop.

These paragraphs provide a small glimpse of the dilemma(s) outlined in the paper:

« Foundation models achieve their best performances by leveraging ever larger amounts of data, without major diminishing returns so far. Unfortunately, verified texts seem insufficient to reach state-of-the-art performance. Indeed, the English Wikipedia only contains around 4 billion words. Meanwhile, a book has around 10^5 words. While there are 10^8 books, only a fraction of them are arguably trustworthy. Many books are instead full of biases and dangerous misinformation, such as ethnic-based hate speech, historical propaganda, or outdated (possibly harmful) medical advice. As a striking illustration, up to the 1980s, the American Psychiatric Association listed homosexuality as a mental illness in its flagship manual. In fact, most books should be regarded as unverified user-generated data.

Most importantly, even if they are not problematic, the combination of these books represents a small amount of data, compared to what Internet users produce on a daily basis. Indeed, assuming that a user writes 300 words per day on an electronic device (the equivalent of one page), a billion of such users produce 10^(to the power of)15 words per decade. This adds up to a hundred times more data than the set of books, and a million times more than the English Wikipedia. This makes it very tempting to either scrape the web, exploit private messaging (e.g., emails, shared documents), or leverage other written texts (e.g., phones’ smart keyboards). In fact, Wikipedia represented only 4% of Google’s PaLM foundation model training dataset [33], while books represented 13% of it. Meanwhile, 27% of the dataset was made of webpages, and 50% were social media conversations. Crucially, these data are generated by a myriad of users, who may be malicious and/or unaware that their activities are being leveraged to train foundation models. This raises serious security and privacy risks, even when we (wrongly) restrict our attention to publicly released models (most of the actual foundation model training is likely performed secretly by private groups). »

https://dair-community.social/@timnitGebru/110147589284016869

#AIHype #AIEthics #DigitalEthics

🔗 Originally posted on Akkoma