11 ms·

Show HN: Roundtable – Survey fraud and bot detection API

21 points by timshell 3y ago

Hi HN, we're Mayank and Matt of Roundtable (https://www.roundtable.ai https://www.roundtable.ai). We launched our Survey Simulator as part of YC S23 (see Launch HN: https://news.ycombinator.com/item?id=36865625 https://news.ycombinator.com/item?id=36865625), and ultimately decided to pivot because we couldn't convince users to trust our results and our value proposition wasn't clear. We conducted tons of user discovery interviews, and a lot of big players talked about the pain of manual QC and bot detection. We launched a survey fraud and bot detection API (Alias; docs.roundtable.ai) that determines whether a bot is answering your survey or not. Our API embeds Javascript into survey programs (Forsta, Qualtrics, jsPsych, etc.) to track user behavior and make a judgment on whether the data is from a high-quality participant. You can use our demo, and try to break it, here: https://www.roundtable.ai/hn-demo https://www.roundtable.ai/hn-demo. See if you can generate LLM-based text without any flags going off! Feedback welcome!

camdoody 3y ago
Mayank and Matt at Roundtable are two of the most special people I’ve met. No telling what RT looks like in 3 years.
- colesantiago 3y ago
  I'm assuming you invested in roundtable right?
warkdarrior 3y ago
What are the privacy guarantees you give to users subjected to your tracking? I don't see anything in your terms, and the javascript tracker seems to collect everything the user does on the page without any privacy-preserving techniques applied.
- timshell 3y ago
  We only track typing behavior in pre-specified text boxes. We are thinking of having version 2 track more data, but we need to determine the privacy implications of that
LaundroMat 3y ago
From the docs it appears this is only for open-ended survey questions. Do any tools exist to detect suspicious behaviour for e.g. multiple choice surveys? I'm thinking of tab switching to Google for an answer for instance.
- timshell 3y ago
  Correct, it's only for open-ended survey questions! We are building functionality for multiple choice surveys, but it seems a bit harder to build
BottingRocks 3y ago
Here is some of the feedback. In terms of bot protection you would get a 1/10. Perhaps, a 2/10 on a good day. First, your payload is being sent using base64 encoded using the default alphabet. You only need a simple atob() to decrypt your payload. Second of all, your bot detection script is very readable making the job of the attacker relatively easy to reverse. Third, but not least, you do not have enough signals/fingerprints which means that your false positives are going to sky-rocket. Bots are not dumb, they are programmed by real humans, your site is extremely easy to reverse. You need to add more obfuscation, more signals, better client-side protection in order to quality for real "bot detection". Source: I reverse antibots for fun and profit, is literally all I've been doing for the past 2 years straight.
- timshell 3y ago
  Thank you for this feedback! We'd love to contract you in the future to try and break our system
  - compootr 3y ago
    hey, I'd be happy to be contracted to break your solution as well the arms race between bot and anti-bot is fascinating and I think I could reasonably overcome barriers like it, so HMU!

21 points by timshell 3y ago

camdoody 3y ago

Mayank and Matt at Roundtable are two of the most special people I’ve met. No telling what RT looks like in 3 years.

colesantiago 3y ago
I'm assuming you invested in roundtable right?

warkdarrior 3y ago

What are the privacy guarantees you give to users subjected to your tracking? I don't see anything in your terms, and the javascript tracker seems to collect everything the user does on the page without any privacy-preserving techniques applied.

timshell 3y ago
We only track typing behavior in pre-specified text boxes. We are thinking of having version 2 track more data, but we need to determine the privacy implications of that

LaundroMat 3y ago

From the docs it appears this is only for open-ended survey questions. Do any tools exist to detect suspicious behaviour for e.g. multiple choice surveys? I'm thinking of tab switching to Google for an answer for instance.

timshell 3y ago
Correct, it's only for open-ended survey questions! We are building functionality for multiple choice surveys, but it seems a bit harder to build

BottingRocks 3y ago

Here is some of the feedback. In terms of bot protection you would get a 1/10. Perhaps, a 2/10 on a good day. First, your payload is being sent using base64 encoded using the default alphabet. You only need a simple atob() to decrypt your payload. Second of all, your bot detection script is very readable making the job of the attacker relatively easy to reverse. Third, but not least, you do not have enough signals/fingerprints which means that your false positives are going to sky-rocket. Bots are not dumb, they are programmed by real humans, your site is extremely easy to reverse. You need to add more obfuscation, more signals, better client-side protection in order to quality for real "bot detection". Source: I reverse antibots for fun and profit, is literally all I've been doing for the past 2 years straight.

timshell 3y ago
Thank you for this feedback! We'd love to contract you in the future to try and break our system
- compootr 3y ago
  hey, I'd be happy to be contracted to break your solution as well the arms race between bot and anti-bot is fascinating and I think I could reasonably overcome barriers like it, so HMU!