5 ms·
I think that in principle one could tag the training set with "source" tags and express weights as a sum of subweights for each source tag; during backpropagati
by devit 2y ago
I think that in principle one could tag the training set with "source" tags and express weights as a sum of subweights for each source tag; during backpropagation, the overall weight and subweight for the training sample would be updated, and during inference linear operations would happen on the subweights as well, while nonlinear operations would scale all subweights by the ratio.
This should in principle allow to determine how much each source influenced each output token of the LLM.
The problem is that this multiplies storage and compute time for tagged inference by the number of source tags, so it may be impractical to actually tag single documents or authors, but might be useful for very broad categories like "copyrighted" vs "non-copyrighted", "synthetic" vs "human generated", "photo" vs "drawing" vs "rendering", year range of publication, etc.
- jart 2y agoThat sounds like a cool research project. However I think it would be enough to simply (1) not erase authorship data, and (2) not fine-tune and monitor LLMs to suppress outputs that mention people. It's an emergent property that they can trace the provenance of text. For example, if I give it an often repeated quote from a book written in the 1800's, then it can tell me where it came from. That's hundreds of years of noise it's weeding through. Imagine what language models could do for recently created knowledge.
- squigz 2y ago> For example, if I give it an often repeated quote from a book written in the 1800's, then it can tell me where it came from. That's hundreds of years of noise it's weeding through. Imagine what language models could do for recently created knowledge. Alternatively: That's 100s of years of mentions of that quote it can pull from.
- carolinahoward 2y ago[dead]
- lawrencooo 2y ago[dead]