5 ms·

I worked in the tracing and tracking all the thousands of data sets that got tweaked and permuted and changed hands between thousands of researchers and data en

I worked in the tracing and tracking all the thousands of data sets that got tweaked and permuted and changed hands between thousands of researchers and data engineers at a major lab. The data that goes into training runs is permuted so much from the OG data that tracing the lineage is not trivial (dramatic understatement). And the difficulty is harder than just the extreme scale of text searching. but also explodes with organizational difficulty since there are so many people tweaking/shifting data independently upstream of the actual training run, and no they will not all add the telemetry you wish they did. In the ideal, should it be this hard? Well, no, but that's org wrangling for you.

ashkankiani 8d ago
It feels convenient to not spend time on engineering around tooling that could be used to answer a question like “did you violate copyright by training on X?”
- benmathes 8d ago
  Don't attribute to malice what is better explained by coordination headwinds in extremely large companies. The engineering around tooling wasn't remotely the issue. It's getting all the (thousands?) data researchers mostly iterating on fine tuning datasets that would get bristly if they couldn't work outside version control in a python notebook iteratively tweaking their dataset that processed and reprocessed a few datasets until a threshold was reached. The only _guaranteed_ chains of custody are down at the compute job and file read level. Which in a massively distributed computing job is... [redacted] nodes reading [redacted] fanouts of "datasets" that is just an abstraction over [redacted] individual files. There's no malice here. Just way way way more complex than you'd first think.
  - ashkankiani 6d ago
    The malice would be in not prioritizing the provenance tool at the start as a requirement of the rest of the product. Ethics would tell you that if you can't make the product in an ethical way, then you probably shouldn't make it.
- deleted 8d ago
  [deleted]
Phemist 8d ago
It definitely is solvable though. Data versioning is a thing and it can work quite transparently to the mutations done on the data. To not know who made and who approved a set of mutations on data can easily become equally as mind-blowingly stupid as not knowing who made mutations to code. Code is a subset of data after all and search over (provenance of) data can be implemented as DAG traversal. Not tracking data changesets like code changesets is certainly a choice, not really a constraint anymore. A similar choice I feel is implied by "extreme scale of text searching". > no they will not all add the telemetry you wish they did ...is just a failure of the corporate policy surrounding data handling. Is git-for-data already considered telemetry? Of course the truth is provenance of data is something best institutionally forgotten as quickly as possible. The only thing that matters is it's there, that the data has no history, and that's why it can be used in whatever way deemed necessary.
- benmathes 8d ago
  All of your points are valid, and believe me I was trying to make them. The problem is one of culture. Most of the people doing this kind of work didn't like version control, and their work was really just running notebooks (like iPython or Google Colab) until a number was good enough and they'd submit the file for inclusion into training runs. You can call it a policy failure, but these people were in very high talent demand and so top down dictates would risk "X people leaving lab Y for lab Z" headlines and morale hits. I am not saying this is good. I am telling you that on the ground it is so much messier than it should be.
SantalBlush 8d ago
This sounds like data laundering in effect.
- benmathes 8d ago
  The appearance of heroic efforts to get authoritative lists of what datasets went into which major model versions prevented actual data laundering (up to intent and mistakes). But don't attribute malice to that which is far far easier to explain with coordination headwinds: https://komoroske.com/slime-mold/ https://komoroske.com/slime-mold/
  - SantalBlush 8d ago
    Malice is not required, which is precisely why I added, "in effect". If the effect is the same as data laundering, that is reason enough to encourage the practice, regardless of the original motives (and I'm sure there are plenty of legitimate ones).

ashkankiani 8d ago

It feels convenient to not spend time on engineering around tooling that could be used to answer a question like “did you violate copyright by training on X?”

benmathes 8d ago
Don't attribute to malice what is better explained by coordination headwinds in extremely large companies. The engineering around tooling wasn't remotely the issue. It's getting all the (thousands?) data researchers mostly iterating on fine tuning datasets that would get bristly if they couldn't work outside version control in a python notebook iteratively tweaking their dataset that processed and reprocessed a few datasets until a threshold was reached. The only _guaranteed_ chains of custody are down at the compute job and file read level. Which in a massively distributed computing job is... [redacted] nodes reading [redacted] fanouts of "datasets" that is just an abstraction over [redacted] individual files. There's no malice here. Just way way way more complex than you'd first think.
- ashkankiani 6d ago
  The malice would be in not prioritizing the provenance tool at the start as a requirement of the rest of the product. Ethics would tell you that if you can't make the product in an ethical way, then you probably shouldn't make it.
deleted 8d ago
[deleted]

Phemist 8d ago

It definitely is solvable though. Data versioning is a thing and it can work quite transparently to the mutations done on the data. To not know who made and who approved a set of mutations on data can easily become equally as mind-blowingly stupid as not knowing who made mutations to code. Code is a subset of data after all and search over (provenance of) data can be implemented as DAG traversal. Not tracking data changesets like code changesets is certainly a choice, not really a constraint anymore. A similar choice I feel is implied by "extreme scale of text searching". > no they will not all add the telemetry you wish they did ...is just a failure of the corporate policy surrounding data handling. Is git-for-data already considered telemetry? Of course the truth is provenance of data is something best institutionally forgotten as quickly as possible. The only thing that matters is it's there, that the data has no history, and that's why it can be used in whatever way deemed necessary.

benmathes 8d ago
All of your points are valid, and believe me I was trying to make them. The problem is one of culture. Most of the people doing this kind of work didn't like version control, and their work was really just running notebooks (like iPython or Google Colab) until a number was good enough and they'd submit the file for inclusion into training runs. You can call it a policy failure, but these people were in very high talent demand and so top down dictates would risk "X people leaving lab Y for lab Z" headlines and morale hits. I am not saying this is good. I am telling you that on the ground it is so much messier than it should be.

SantalBlush 8d ago

This sounds like data laundering in effect.

benmathes 8d ago
The appearance of heroic efforts to get authoritative lists of what datasets went into which major model versions prevented actual data laundering (up to intent and mistakes). But don't attribute malice to that which is far far easier to explain with coordination headwinds: https://komoroske.com/slime-mold/ https://komoroske.com/slime-mold/
- SantalBlush 8d ago
  Malice is not required, which is precisely why I added, "in effect". If the effect is the same as data laundering, that is reason enough to encourage the practice, regardless of the original motives (and I'm sure there are plenty of legitimate ones).