rishramanathan

6 ms·

1.
Constitutional AI for All(openlayer.com)
9 points by rishramanathan 2y ago|0 comments
2.
by rishramanathan 3y ago
Not sure I follow — could you elaborate on what you mean by “directly interacting”?
3.
by rishramanathan 3y ago
Thanks! We’ve broken our evals down into three primary categories — integrity, consistency and performance. Integrity tests tackle data quality issues (e.g. no PII in input data, no duplicate rows, schema checks on specific fields). Consist
4.
by rishramanathan 3y ago
Thanks! Glad Openlayer is working well for you :)
5.
by rishramanathan 3y ago
We’ve actually been building a testing and evaluation platform from the start, but started with discriminative ML tasks like classification and regression. We waited to do a Launch HN because we were mostly focused on enterprise / mid-
6.
by rishramanathan 3y ago
We realize the lack of information about pricing isn’t ideal, and that people will be turned away by this. In the meantime, we do have a free plan with generous limits that allows you to get started self-serve. This plan isn’t time bounded,
7.
by rishramanathan 3y ago
Broadly, on the monitoring side, we’re more focused on evaluating the quality of the model’s outputs (is it violating your rules, handling specific subpopulations / edge cases correctly etc.). OpenLLMetry is more focussed on telemetry
8.
by rishramanathan 3y ago
Oops, thanks for the heads up! Fixed.
9.
Launch HN: Openlayer (YC S21) – Testing and Evaluation for AI
94 points by rishramanathan 3y ago|31 comments
10.
by rishramanathan 3y ago
Good question! Security is our highest priority - here's a few key things we've done to help ensure the privacy of customer data: - We offer on-prem deployments on all cloud platforms, so your data never needs to leave your infras
11.
by rishramanathan 4y ago
BentoML is by far the best open source tool I've seen for model deployment. Congrats, team!