8 ms·

This is why we built Trix (trix.co). It's a consumer-facing photo editing app that uses adversarial AI to manipulate your photos in such a way that companies l

This is why we built Trix (trix.co). It's a consumer-facing photo editing app that uses adversarial AI to manipulate your photos in such a way that companies like Clearview AI can't train facial recognition algorithms off your data. You can download the Android or iOS version at trix.co - we're in beta right now. Would love to get anyone's feedback!

chance_state 6y ago
Do you have any data on how effective this technique is?
- geraldkleber 6y ago
  Not yet - much of this research came out of U Chicago this summer and we just launched our beta. You can download at trix.co (iOS and Android).
  - srtjstjsj 6y ago
    So you're Theranosing it?
    - geraldkleber 6y ago
      Nope - you should check out the research! https://sandlab.cs.uchicago.edu/fawkes/ https://sandlab.cs.uchicago.edu/fawkes/
polishdude20 6y ago
Hmm trying to download it on Android, I click the link and it starts to take me to google play but it just loads forever. Never gets to the page to download it
- geraldkleber 6y ago
  Hmm, lemme do some bug squashing. Will reply when it's dealt with!
  - geraldkleber 6y ago
    You should be good to go - let me know if this still gives you trouble!
    - anadem 6y ago
      not OP but trix.co's Android link goes to Play's "Welcome to the testing program" and that has link "download it on Google Play" which gets "We're sorry, the requested URL was not found on this server."
    - polishdude20 6y ago
      Hey it works! Although I don't want to have to give you my phone number to signup. Why can't it just take a photo locally and apply the filter to it?
samename 6y ago
>“We require that you sign up with a valid US phone number to verify your identity as a human for security purposes.” How does collecting personal data improve security? Why is identity verification needed when the point of using the service is to avoid automated identification in the first place?
- bigbubba 6y ago
  Such a database would be a tasty treat for any unsavory company looking to purchase them.
  - geraldkleber 6y ago
    This is a measure designed to help us prevent facial recognition companies from gaining programmatic access to our api to test against it. In addition, because we are a new startup with limited computer resources such testing could also harm our throughput capabilities for actual users. Unfortunately, there are far better sources of simple name and phone number data such as whitepages.com, or any other CNAM service, so we doubt we would be a target of an attack for this data or that this data would be sell-able even if we were bad actors. Our perspective is that requiring phone verification allows us to provide a better level of service to customers.
_tulpa 6y ago
I really don't think you can solve technology with more technology...
- geraldkleber 6y ago
  In this instance, we're well convinced we can.
  - amelius 6y ago
    How do you know you can in the future? And do you tell your customers that their photos, after you have edited them, might not be safe from future versions of AI?
godelski 6y ago
This is cool, do you have a technical blog post?
- geraldkleber 6y ago
  No technical blog post yet, but have some faqs up that could help out! https://www.trix.co/faq https://www.trix.co/faq
vaccinator 6y ago
Sounds like a fight that can never be won
- geraldkleber 6y ago
  We don't think that's the case!
searchableguy 6y ago
What if this works to increase bias against certain group of people because the facial recognition software isn't trained on them? I am not sure technology can solve this completely.
- geraldkleber 6y ago
  I very much appreciate that question, and having been in this space for a few years now, it's certainly one that's relevant to the tech as whole, but less so our app. Our technology is simply indexed to a public data-set of 30k individuals and when our deep learning model scrambles the key-points on your photos to confuse the clearviews of the world it does in a random manner. The model truly is a black box in that way.
srtjstjsj 6y ago
Do you have any evidence that your system works? "Adversarial AI" doesn't really work against systems like Clearview that aren't using "AI" in the first place.
- geraldkleber 6y ago
  You should check out the research that was done on this. The technology works - we're simply the first to really productize it in this manner. https://sandlab.cs.uchicago.edu/fawkes/ https://sandlab.cs.uchicago.edu/fawkes/
  - Ruthalas 6y ago
    In what manner does your tool protect against future improvements to image recognition tools running against images obscured today?
    - toydog534 6y ago
      Good question, and one we have thought about quite a bit. We very much know that we’ll be involved in a cat and mouse game with facial recognition companies over time. Our objective is to protect users from facial recognition today and going forward. Even if next year, for instance, a facial recognition company can develop tech that works on protected images posted this year, if we are able to advance our protection to match advances in facial recognition next year the user will still be protected in real time - which is what really matters. So in short - it likely won’t matter if/when there are advances in facial recognition if we can keep up!
      - Ruthalas 6y ago
        > Even if next year, for instance, a facial recognition company can develop tech that works on protected images posted this year, if we are able to advance our protection to match advances in facial recognition next year the user will still be protected in real time... This isn't entirely clear to me. Are you saying something to the effect of, "While historical images would indeed be compromised, there is value in at least the current images not being compromised"?
Jon_Lowtek 6y ago
Split your Privacy Policy into parts along technological borders like "site, crm, app" instead of one size fits all. Disclose what you are actually doing and restrict yourselve to that instead of trying to have as much wiggle room as possible. This sounds counter-intuitive because web-business standard is to be as consumer unfriendly as legally possible, but the privacy tool market likes higher standards. After reading your policy and tos i think someone should check the app for facebook scraping, because the legal texts sure imply you are doing that: > "By granting Trix access to any Third-Party Accounts, you understand that Trix may access, .. any information .. that you have provided to and stored in such Third-Party Account (“SNS Content”) ... all SNS Content shall be considered to be your User Content" Some great advice from the privacy policy: > "You should always review, and if necessary, adjust your privacy settings on third-party websites and services before linking or connecting them to Trix’s websites or Service." Statements like this are red flags: > "We may collect metadata associated with User Content. Metadata typically consists of how, when, where and by whom a piece of User Content was collected and how that content has been formatted." > "Trix may transfer information that we collect about you, including personal information, to affiliated entities, or to other third parties" ... "for the purpose of providing the Service" This is a typical blank. Basically they reserve the right to do as they please with your data and all data they can access through services you link with their service.
- toydog534 6y ago
  technical cofounder of trix here. we do not currently have social logins for the app to this point doesnt make much sense. but we probably can further modify the templates we used for our tos/ privacy policy to asuage your concerns :) we absolutely do not (and would not) scrape data from your social media accounts. unfortunately, terms like this are standard for many tech companies today.
  - Jon_Lowtek 6y ago
    I too find it unfortunate, that is why i am a bit picky about it. Nothing changes if people don't ask for change. I would find it in much better taste if your policy tells what you do instead of focusing on creating legal room for what you say you don't. What i find most interesting is the metadata piece. Photos often have time and location information attached (exif), especially if your service sits close to the camera. You reserve the right to harvest those. Why? (It's a rhetoric question: the answer is of course: the lawyer said such wording protects you from getting sued)

chance_state 6y ago

Do you have any data on how effective this technique is?

geraldkleber 6y ago
Not yet - much of this research came out of U Chicago this summer and we just launched our beta. You can download at trix.co (iOS and Android).
- srtjstjsj 6y ago
  So you're Theranosing it?
  - geraldkleber 6y ago
    Nope - you should check out the research! https://sandlab.cs.uchicago.edu/fawkes/ https://sandlab.cs.uchicago.edu/fawkes/

polishdude20 6y ago

Hmm trying to download it on Android, I click the link and it starts to take me to google play but it just loads forever. Never gets to the page to download it

geraldkleber 6y ago
Hmm, lemme do some bug squashing. Will reply when it's dealt with!
- geraldkleber 6y ago
  You should be good to go - let me know if this still gives you trouble!
  - anadem 6y ago
    not OP but trix.co's Android link goes to Play's "Welcome to the testing program" and that has link "download it on Google Play" which gets "We're sorry, the requested URL was not found on this server."
  - polishdude20 6y ago
    Hey it works! Although I don't want to have to give you my phone number to signup. Why can't it just take a photo locally and apply the filter to it?

samename 6y ago

>“We require that you sign up with a valid US phone number to verify your identity as a human for security purposes.” How does collecting personal data improve security? Why is identity verification needed when the point of using the service is to avoid automated identification in the first place?

bigbubba 6y ago
Such a database would be a tasty treat for any unsavory company looking to purchase them.
- geraldkleber 6y ago
  This is a measure designed to help us prevent facial recognition companies from gaining programmatic access to our api to test against it. In addition, because we are a new startup with limited computer resources such testing could also harm our throughput capabilities for actual users. Unfortunately, there are far better sources of simple name and phone number data such as whitepages.com, or any other CNAM service, so we doubt we would be a target of an attack for this data or that this data would be sell-able even if we were bad actors. Our perspective is that requiring phone verification allows us to provide a better level of service to customers.

_tulpa 6y ago

I really don't think you can solve technology with more technology...

geraldkleber 6y ago
In this instance, we're well convinced we can.
- amelius 6y ago
  How do you know you can in the future? And do you tell your customers that their photos, after you have edited them, might not be safe from future versions of AI?

godelski 6y ago

This is cool, do you have a technical blog post?

geraldkleber 6y ago
No technical blog post yet, but have some faqs up that could help out! https://www.trix.co/faq https://www.trix.co/faq

vaccinator 6y ago

Sounds like a fight that can never be won

geraldkleber 6y ago
We don't think that's the case!

searchableguy 6y ago

What if this works to increase bias against certain group of people because the facial recognition software isn't trained on them? I am not sure technology can solve this completely.

geraldkleber 6y ago
I very much appreciate that question, and having been in this space for a few years now, it's certainly one that's relevant to the tech as whole, but less so our app. Our technology is simply indexed to a public data-set of 30k individuals and when our deep learning model scrambles the key-points on your photos to confuse the clearviews of the world it does in a random manner. The model truly is a black box in that way.

srtjstjsj 6y ago

Do you have any evidence that your system works? "Adversarial AI" doesn't really work against systems like Clearview that aren't using "AI" in the first place.

geraldkleber 6y ago
You should check out the research that was done on this. The technology works - we're simply the first to really productize it in this manner. https://sandlab.cs.uchicago.edu/fawkes/ https://sandlab.cs.uchicago.edu/fawkes/
- Ruthalas 6y ago
  In what manner does your tool protect against future improvements to image recognition tools running against images obscured today?
  - toydog534 6y ago
    Good question, and one we have thought about quite a bit. We very much know that we’ll be involved in a cat and mouse game with facial recognition companies over time. Our objective is to protect users from facial recognition today and going forward. Even if next year, for instance, a facial recognition company can develop tech that works on protected images posted this year, if we are able to advance our protection to match advances in facial recognition next year the user will still be protected in real time - which is what really matters. So in short - it likely won’t matter if/when there are advances in facial recognition if we can keep up!
    - Ruthalas 6y ago
      > Even if next year, for instance, a facial recognition company can develop tech that works on protected images posted this year, if we are able to advance our protection to match advances in facial recognition next year the user will still be protected in real time... This isn't entirely clear to me. Are you saying something to the effect of, "While historical images would indeed be compromised, there is value in at least the current images not being compromised"?

Jon_Lowtek 6y ago

Split your Privacy Policy into parts along technological borders like "site, crm, app" instead of one size fits all. Disclose what you are actually doing and restrict yourselve to that instead of trying to have as much wiggle room as possible. This sounds counter-intuitive because web-business standard is to be as consumer unfriendly as legally possible, but the privacy tool market likes higher standards. After reading your policy and tos i think someone should check the app for facebook scraping, because the legal texts sure imply you are doing that: > "By granting Trix access to any Third-Party Accounts, you understand that Trix may access, .. any information .. that you have provided to and stored in such Third-Party Account (“SNS Content”) ... all SNS Content shall be considered to be your User Content" Some great advice from the privacy policy: > "You should always review, and if necessary, adjust your privacy settings on third-party websites and services before linking or connecting them to Trix’s websites or Service." Statements like this are red flags: > "We may collect metadata associated with User Content. Metadata typically consists of how, when, where and by whom a piece of User Content was collected and how that content has been formatted." > "Trix may transfer information that we collect about you, including personal information, to affiliated entities, or to other third parties" ... "for the purpose of providing the Service" This is a typical blank. Basically they reserve the right to do as they please with your data and all data they can access through services you link with their service.

toydog534 6y ago
technical cofounder of trix here. we do not currently have social logins for the app to this point doesnt make much sense. but we probably can further modify the templates we used for our tos/ privacy policy to asuage your concerns :) we absolutely do not (and would not) scrape data from your social media accounts. unfortunately, terms like this are standard for many tech companies today.
- Jon_Lowtek 6y ago
  I too find it unfortunate, that is why i am a bit picky about it. Nothing changes if people don't ask for change. I would find it in much better taste if your policy tells what you do instead of focusing on creating legal room for what you say you don't. What i find most interesting is the metadata piece. Photos often have time and location information attached (exif), especially if your service sits close to the camera. You reserve the right to harvest those. Why? (It's a rhetoric question: the answer is of course: the lawyer said such wording protects you from getting sued)