18 ms·

Would this be reasonable material on which to fine tune the new Gemma 3 270M model?

Disposal8433 1y ago
Half of the occult books are talking about magic and irrelevant stuff. The other half is philosophy and spirituality hidden behind materialistic concepts (think Freemasons for example). All those books would most likely be useless or detrimental for LLMs I guess.
- literalAardvark 1y ago
  More than useful for running a d&d campaign
  - gtsnexp 1y ago
    Thinking of a RAG with the entire Ritman library collection as a GM.
- dr_dshiv 1y ago
  Most of the books are the outcomes of the Renaissance. The relationship between “science” and spirituality was much closer then than now. Further, most books published in Europe between 1300-1700 were written in Neo-Latin. Most of these books, therefore, have not been digitized and translated. Now, to me, it seems like a real shame if this humanist core of European thought is deemed too dangerous for consumption. But it wouldn’t be the first time. The library behind these works, the Biblioteca Philosophica Hermetica, specializes in books banned by various church authorities. I personally believe that these materials should definitely be part of large model training. The renaissance, esoteric though it may be, deserves to be part of the diversity of thought used to train LLMs. We can easily imagine an AI apocalypse - maybe these books might even help us imagine an AI renaissance…
  - zozbot234 1y ago
    > I personally believe that these materials should definitely be part of large model training. Already done: https://news.ycombinator.com/item?id=37752272 https://news.ycombinator.com/item?id=37752272 It turns out that the real safety risk with AI is not Mecha-Hitler, it's just that it might end up reading the wrong sorts of books and accidentally conjure a horde of demons.
    - dr_dshiv 1y ago
      > accidentally conjure a horde of demons. If we have a certain perspective on demons as self-sustaining information processing loops—we’ll, yeah, demons abound. And, yes, this hn post from BenBreen — is amazing. We’ve been in touch! And if Ben is reading, we’ve made progress on our version controlled community—LLM book translation prototype. Of course, it’s much to early to share on hn. Or whatever, it’s great, take a peek: https://www.philosopherslibrary.com/ https://www.philosopherslibrary.com/ Upload books, get translations. We have a separate system where neolatin scholars can evaluate randomly selected paragraphs — so we can measure and report on the base rate of different qualities for each book as a whole. I don’t know if it is technically important for advancing AI, but it might be. 80% of the Neo-Latin books in the library have not been translated. Most NeoLatin hasn’t been translated. And it is even worse with Sanskrit. So much material has not been digitized or translated. Is it meaningful to try to get this humanist core into our language models? Maybe including only modern writing is a good bias, but I doubt it. It will take at least 2 years to get all this stuff scanned, digitized, translated and published. So it’s kind of urgent, given all the AI training taking place over the next two years. I’m writing this in a rush, but I’m at the Houghton Library at Harvard and they just brought up 5 books from Marsilio Ficino, published 1497 and 1516. (If you or anyone is into this kind of thing, my email is in my profile — it’s my favorite hobby)
      - carlosjobim 1y ago
        > If we have a certain perspective on demons as self-sustaining information processing loops—we’ll, yeah, demons abound. Such as paranoid schizophrenia, drug addiction, self harm, harming others, etc.
UlisesAC4 1y ago
Probably absolutely no. I was studying about the corresponding names between tarot card and Shem Hamphorash and gave me incorrect names, it gave me a correct angel name but not the correct one of several cards. So for studying? Nope, for practicing neither.

Half of the occult books are talking about magic and irrelevant stuff. The other half is philosophy and spirituality hidden behind materialistic concepts (think Freemasons for example). All those books would most likely be useless or detrimental for LLMs I guess.

literalAardvark 1y ago
More than useful for running a d&d campaign
- gtsnexp 1y ago
  Thinking of a RAG with the entire Ritman library collection as a GM.
dr_dshiv 1y ago
Most of the books are the outcomes of the Renaissance. The relationship between “science” and spirituality was much closer then than now. Further, most books published in Europe between 1300-1700 were written in Neo-Latin. Most of these books, therefore, have not been digitized and translated. Now, to me, it seems like a real shame if this humanist core of European thought is deemed too dangerous for consumption. But it wouldn’t be the first time. The library behind these works, the Biblioteca Philosophica Hermetica, specializes in books banned by various church authorities. I personally believe that these materials should definitely be part of large model training. The renaissance, esoteric though it may be, deserves to be part of the diversity of thought used to train LLMs. We can easily imagine an AI apocalypse - maybe these books might even help us imagine an AI renaissance…
- zozbot234 1y ago
  > I personally believe that these materials should definitely be part of large model training. Already done: https://news.ycombinator.com/item?id=37752272 https://news.ycombinator.com/item?id=37752272 It turns out that the real safety risk with AI is not Mecha-Hitler, it's just that it might end up reading the wrong sorts of books and accidentally conjure a horde of demons.
  - dr_dshiv 1y ago
    > accidentally conjure a horde of demons. If we have a certain perspective on demons as self-sustaining information processing loops—we’ll, yeah, demons abound. And, yes, this hn post from BenBreen — is amazing. We’ve been in touch! And if Ben is reading, we’ve made progress on our version controlled community—LLM book translation prototype. Of course, it’s much to early to share on hn. Or whatever, it’s great, take a peek: https://www.philosopherslibrary.com/ https://www.philosopherslibrary.com/ Upload books, get translations. We have a separate system where neolatin scholars can evaluate randomly selected paragraphs — so we can measure and report on the base rate of different qualities for each book as a whole. I don’t know if it is technically important for advancing AI, but it might be. 80% of the Neo-Latin books in the library have not been translated. Most NeoLatin hasn’t been translated. And it is even worse with Sanskrit. So much material has not been digitized or translated. Is it meaningful to try to get this humanist core into our language models? Maybe including only modern writing is a good bias, but I doubt it. It will take at least 2 years to get all this stuff scanned, digitized, translated and published. So it’s kind of urgent, given all the AI training taking place over the next two years. I’m writing this in a rush, but I’m at the Houghton Library at Harvard and they just brought up 5 books from Marsilio Ficino, published 1497 and 1516. (If you or anyone is into this kind of thing, my email is in my profile — it’s my favorite hobby)
    - carlosjobim 1y ago
      > If we have a certain perspective on demons as self-sustaining information processing loops—we’ll, yeah, demons abound. Such as paranoid schizophrenia, drug addiction, self harm, harming others, etc.

UlisesAC4 1y ago

Probably absolutely no. I was studying about the corresponding names between tarot card and Shem Hamphorash and gave me incorrect names, it gave me a correct angel name but not the correct one of several cards. So for studying? Nope, for practicing neither.