5 ms·

Can you blame them? To me, an AI grading platform for children seems like it should be regulated.

BenJacob1 1y ago
My gripe isn’t “no regulation,” it’s the shape of this one for teams in a sector that’s already stretched. Some of these requirements are so high burden that it will add 10% to our OpEx e.g. data governance/provenance reqs: document sources, bias reviews, copyright posture, retention + redoing this after every model change; quality management system across the company (policies, roles, training, audits); conformity assessment + CE marking, with a third-party review all of which will be costly and slow. I’m all for protecting students. But in practice this shifts months of team time and serious cash into paperwork before schools even pilot the tool. Big incumbents can eat that but small teams either raise prices, slow down, or skip the EU. That’s the innovation tax I’m worried about especially when schools need better tools now.
- binarymax 1y ago
  Sure. This seems a cultural difference between US and EU. I've lived extensively in both. EU is known to be burdensome in terms of tech regulations. Normally I find this egregious. However when children are involved I am far more tolerant of their requirements. 10% OpEx overhead to make sure children are protected seems entirely reasonable to me. If you're not prepared to meet their requirements then skip the EU.
  - quantified 1y ago
    Seconding this. If you're just a team of 10, you are way too small to take on the responsibility of the thing you're trying to provide. I shudder to think of the educational quality a team of ten will provide to hundreds of thousands of schoolchildren in thousands of different schools. You're too small for the less-regulated US. Add people to deal with things.
    - BenJacob1 1y ago
      You're presenting a false dichotomy between building good products and being capable of fulfiling extremely burdensome (not to mention, expensive) compliance requirements from the EU. We're perfectly qualified to build the core tech. We're just not paperwork experts. Are you suggesting if we can't do paperwork, we shouldn't build innovative products that improve educational outcomes?
      - quantified 1y ago
        Absolutely. You can build an innovative product, sure, but you can't refine it or support it the way it needs to be supported. Perhaps you have other teams working on product management, customer success (odd term for education, but applicable), etc.? Are the grading rubrics tuned for each student, what they need, what's in their IEP or 509? What is your plan for handling the legal challenges when the system massively screws up? This isn't a random B2B. This is a subtly very important influence on a great many actual humans.
- magicalhippo 1y ago
  > That’s the innovation tax I’m worried about especially when schools need better tools now. But how do you, and more importantly the school, know that the tool is better? If you skip the bias review, how can you, and more importantly the school, be sure your AI tool isn't grading every kid with a certain geographical slang less than others for example? To me this isn't unlike the security stuff, like ISO27001. We make B2B software and had about 15 employees when our customers started pushing ISO27001 or similar hard. We had no choice but to step up. We have hired multiple people just to deal with that aspect, as well as implementing large changes in our IT infra. And frankly I think it was a very good thing and has made our org much better and robust.
- lordkrandel 1y ago
  Suppose for a moment that instead of AI, it's guns. You say: that's ridiculous, I have this safety testing bureacracy and all this reporting to do in the EU, it might not worth opening a firearm factory or shop in Europe. Yes! That's the whole point! We don't want shootings in schools. We don't want unsafe AI. If innovation is dangerous for people, we slow it down until you can demonstrate it's safe. Safety and privacy first, innovation is less important than the resulting quality of life of the consumer.

My gripe isn’t “no regulation,” it’s the shape of this one for teams in a sector that’s already stretched. Some of these requirements are so high burden that it will add 10% to our OpEx e.g. data governance/provenance reqs: document sources, bias reviews, copyright posture, retention + redoing this after every model change; quality management system across the company (policies, roles, training, audits); conformity assessment + CE marking, with a third-party review all of which will be costly and slow. I’m all for protecting students. But in practice this shifts months of team time and serious cash into paperwork before schools even pilot the tool. Big incumbents can eat that but small teams either raise prices, slow down, or skip the EU. That’s the innovation tax I’m worried about especially when schools need better tools now.

binarymax 1y ago
Sure. This seems a cultural difference between US and EU. I've lived extensively in both. EU is known to be burdensome in terms of tech regulations. Normally I find this egregious. However when children are involved I am far more tolerant of their requirements. 10% OpEx overhead to make sure children are protected seems entirely reasonable to me. If you're not prepared to meet their requirements then skip the EU.
- quantified 1y ago
  Seconding this. If you're just a team of 10, you are way too small to take on the responsibility of the thing you're trying to provide. I shudder to think of the educational quality a team of ten will provide to hundreds of thousands of schoolchildren in thousands of different schools. You're too small for the less-regulated US. Add people to deal with things.
  - BenJacob1 1y ago
    You're presenting a false dichotomy between building good products and being capable of fulfiling extremely burdensome (not to mention, expensive) compliance requirements from the EU. We're perfectly qualified to build the core tech. We're just not paperwork experts. Are you suggesting if we can't do paperwork, we shouldn't build innovative products that improve educational outcomes?
    - quantified 1y ago
      Absolutely. You can build an innovative product, sure, but you can't refine it or support it the way it needs to be supported. Perhaps you have other teams working on product management, customer success (odd term for education, but applicable), etc.? Are the grading rubrics tuned for each student, what they need, what's in their IEP or 509? What is your plan for handling the legal challenges when the system massively screws up? This isn't a random B2B. This is a subtly very important influence on a great many actual humans.
magicalhippo 1y ago
> That’s the innovation tax I’m worried about especially when schools need better tools now. But how do you, and more importantly the school, know that the tool is better? If you skip the bias review, how can you, and more importantly the school, be sure your AI tool isn't grading every kid with a certain geographical slang less than others for example? To me this isn't unlike the security stuff, like ISO27001. We make B2B software and had about 15 employees when our customers started pushing ISO27001 or similar hard. We had no choice but to step up. We have hired multiple people just to deal with that aspect, as well as implementing large changes in our IT infra. And frankly I think it was a very good thing and has made our org much better and robust.
lordkrandel 1y ago
Suppose for a moment that instead of AI, it's guns. You say: that's ridiculous, I have this safety testing bureacracy and all this reporting to do in the EU, it might not worth opening a firearm factory or shop in Europe. Yes! That's the whole point! We don't want shootings in schools. We don't want unsafe AI. If innovation is dangerous for people, we slow it down until you can demonstrate it's safe. Safety and privacy first, innovation is less important than the resulting quality of life of the consumer.