6 ms·
If the model were just repeating the distribution baked into the training data, could you help me understand why the R^2 values in Table 3 are mostly below 0.1?
by striking 1mo ago
If the model were just repeating the distribution baked into the training data, could you help me understand why the R^2 values in Table 3 are mostly below 0.1?
Could you explain what you meant when you wrote "the women who published this"?
- nullbio 1mo ago> If the model were just repeating the distribution baked into the training data, could you help me understand why the R^2 values in Table 3 are mostly below 0.1? Because it's only part of the picture. Firstly, the complexity metrics are not a silver-bullet, you'd have to actually evaluate the inputs against the outputs individually, against the metrics individually, to get a realistic guage. You also can't just lump 6 different metrics together under a single score and expect interpretable results, they behave differently. Secondly, the models are heavily fine-tuned. Statistical distribution repetition is the default mode of a base model and transformer, but obviously the models we use are heavily finetuned, so any stray from the distribution is due to post-training. That's not to suggest there's a wild stray from the distribution here though - the way the metrics are measured in this paper is not convincing and muddies the waters. > Could you explain what you meant when you wrote "the women who published this"? I meant what I said, I'm not sure what you're asking.
- striking 1mo agoIs the table not a per-metric comparison, though? And if it is a result of finetuning, isn't that by definition not something inherent to how LLMs are? As to the latter bit, it just seemed a weird framing to me.