6 ms·
The hard part is that it locks you in to whatever people believe at the moment, which might be proven wrong in the future
by joegibbs 21d ago
The hard part is that it locks you in to whatever people believe at the moment, which might be proven wrong in the future
- toomuchtodo 21d agoHow would you design the pipeline and user interface to reflect the temporal nature and potential theta decay of evaluations? Reprocess comments on a cadence? For example, however you represent a comment, it could then support expansion of a tree to each evaluation snapshot point in time, like expanding a directory listing its files. My thought on this is somewhat similar to Wayback Machine’s interface per URL; the URL remains static over time, with a calendar and indicators for date of snapshot, and drill down into each snapshot. In this use case, the comment is the URL, and each LLM evaluation of the comment is a snapshot. Append only.