6 ms·

Project Aria 'Digital Twin' Dataset by Meta

180 points by socratic1 3y ago

dustypotato 3y ago
First thing I notice while browsing from EU is that there's only one option regarding cookies. Accept all! Even if I click "Learn more" there's no "accept necessary cookies" or "Reject cookies". First time I encounter something like this.
- otikik 3y ago
  Yep, closed it down immediately. IANAL but that might not be legal in Europe
- littlestymaar 3y ago
  > First time I encounter something like this. Facebook itself use to have this exact banner with no alternative until they were strong-armed to properly comply with GDPR.
- WhackyIdeas 3y ago
  Noticed the same thing as I always select ‘necessary only’ or something like that. Instead, I just closed the page and clicked on HN comments to see what it was about.
- anthonyskipper 3y ago
  That's a common dark pattern.
  - constantly 3y ago
    It’s not really a dark pattern because they’re not tricking you into accepting or coercing more than is acceptable into doing so, they’re just not allowing you to decline. Which means if not accepting cookies is a dealbreaker for you, no problem, just: move along.
    - deleted 3y ago
      [deleted]
  - master-lincoln 3y ago
    Dark pattern is a term normally used for legal, but immoral user experience. In this case legality is questionable as they redirect you to other external sites to manage your "consent" https://optout.aboutads.info https://optout.aboutads.info (at least from a European GDPR perspective)
- coldpie 3y ago
  I have to admit, for technical folks like ourselves, I don't understand why you care what options the dialog presents. Just use a "Kill Sticky" plugin to nuke the stupid dialog so you can read the page, or even Accept them, and then instruct your browser to do whatever you like with the cookies the site creates (i.e. delete them). It's all in your hands, the popup dialog doesn't do anything you can't do yourself.
  - master-lincoln 3y ago
    you should care because often this agreement is not about the technical detail of cookies, but allowing the company you are interacting with to share data about you with 3rd parties.
    - coldpie 3y ago
      If you think which HTML div element you click on to dismiss a sticky banner has any bearing whatsoever on how your data is handled server-side, I've got a trip to the Titanic in an Oceangate submersible to sell you.
      - Etheryte 3y ago
        Legally it does, at least if you're in the EU. The only reason websites are slow on the uptake around this is that the legal gears are slow, however we have seen many considerable fines in this space in the last few years and things are improving.
    - bad_username 3y ago
      It's also in direct violation of GDPR.
  - explaininjs 3y ago
    Learned helplessness. Teach the citizens that the only thing protecting them from the Big Scary Internet is their benevolent government. Meanwhile the politicians collect millions from the mass media companies that lobbied for the god-awful implementation of the law we all ended up with, and as everyone gets worn down into Accepting All always, they simultaneously forget that their User Agent holds all the cards (or in this case, cookies), and it can be instructed to do anything the User wants with them.
- marcinzm 3y ago
  Isn't that a GDPR violation since they're not allowed to prevent access if you refuse to share data not necessary for the service to function? Since it's from Meta I suspect regulators would enjoy another thing to add to the list for future fine calculations.
  - dspillett 3y ago
    It depends. If everything stored & tracked is genuinely necessary (as defined by the regulations) then consent is not required. If they are just telling you that they are storing & tracking necessary things like session information, then all is good (unless the things they store & track are not, in fact, strictly necessary and non-privacy-affecting). Of course if they don't need consent, then why are they making a song & dance about it having us click an accept button?
    - IanCal 3y ago
      They explicitly say a use is for ads on and off this site.
  - mike_d 3y ago
    Not at all. You can have a single prompt with a single accept button if you are only using functional cookies on a site.
    - capableweb 3y ago
      In that case, you don't need anything, no notice nor consent required at all, get rid of the entire modal/popup/banner.
- nightpool 3y ago
  This is common on many, many sites like this because they do not have any tracking cookies or anything else that they would need consent for, but they're still required to display a cookie banner "notifying" you that cookies are "in use" as per the terms of the old 2009 ePrivacy Directive. In this case, it appears that projectaria.com sets 1) one cookie for the user's DPR (1 or 2) so that the backend can serve optimized images, 2) one cookie for the user's locale, and 3) one cookie for a CSRF token for form submission.
  - capableweb 3y ago
    > but they're still required to display a cookie banner "notifying" you that cookies are "in use" Common misconception but this is not true. If you use cookies only for functional purposes (not for tracking for example), you do not need to show any cookie banners. Like if you have a shopping cart and you have a cookie for keeping track of what's in it, it's for functional purposes for the user and hence needs no notice to be used. The UK's ICO made a handy summary for people who are curious about what the directive actually says: https://ico.org.uk/media/for-organisations/documents/1545/cookies_guidance.pdf https://ico.org.uk/media/for-organisations/documents/1545/co... Specifically: > Exceptions from the requirement to provide information and obtain consent > Activities likely to fall within the exception: [...] Some cookies help ensure that the content of your page loads quickly [...] Certain cookies providing security that is essential to comply with the security requirements [...]
    - sanderjd 3y ago
      But it's easier and less risky to just always put in the standard language that everyone ignores and mindlessly clicks through anyway. Which is why this was very silly legislation. People helping develop future legislation (in the EU and elsewhere) should be aware of this as a cautionary tale of incentivizing theater with only cost and no benefit.
    - nightpool 3y ago
      > Common misconception but this is not true. If you use cookies only for functional purposes (not for tracking for example), you do not need to show any cookie banners. Like if you have a shopping cart and you have a cookie for keeping track of what's in it, it's for functional purposes for the user and hence needs no notice to be used. Personally, I would not put a cookie banner of any kind on my website. However, given this text: The term 'strictly necessary' means that such storage of or access to information should be essential, rather than reasonably necessary, for this exemption to apply. However, it will also be restricted to what is essential to provide the service requested by the user, rather than what might be essential for any other uses the service provider might wish to make of that data. It will also include what is required to comply with any other legislation the person using the cookie might be subject to, for example, the security requirements of the seventh data protection principle. Where the setting of a cookie is deemed 'important' rather than 'strictly necessary', those collecting the information are still obliged to provide information about the device to the potential service recipient and obtain consent. I think it's clear why a more risk-conscious organization like Meta might take a more conservative reading of "Strictly necessary" that does not apply to e.g. bandwidth optimizations related to a device's DPI
- cutler 3y ago
  This one is tame compared with a whole industry of "data privacy" popups which hide "Legitimate Interest" opt-outs within Vendor lists containing hundreds of entries. Someone needs to slap some serious lawsuits on these gangsters.
- mensetmanusman 3y ago
  It’s amazing how much worse browsing the web has gotten after all these pop-ups everywhere.
  - kaffeeringe 3y ago
    You mean how much worse surfing the web got after it became visible, how much companies are tracking us? The problem is the tracking, not the banner. If you don't sell your readers data, you don't need a banner.
- Dirak 3y ago
  This is a perfect example of how GDPR can present challenges to innovation. The fact the top comment in this announcement revolves around GDPR compliance and associated fines raises questions about whether companies will be motivated to share research and open source datasets in the future.
machinekob 3y ago
Non-commercial licence :[
- optymizer 3y ago
  "I can't use this to make money". Sad face. If I had a company, which is a for-profit entity, I wouldn't want competitors to use my company's tech for free either.
- Roark66 3y ago
  I wonder, let's say one uses it on an open source project that accepts donations. Is that non-commercial enough? Or is it a no-no because money is involved? I'm just curious. I'm glad such free datasets exist even if they are restricted to educational/hobby use.
  - sp332 3y ago
    You can't use it in an open-source project. Open source licenses don't restrict commercial use.
quietthrow 3y ago
Genuine question: What is an egocentric data set? Eli5 please? What can this be used for?
- ikhatri 3y ago
  Egocentric means that the sensors frame of reference is the same as the ego (self). The person walking around is wearing the sensors/glasses themselves & their pose is given as “ego pose”. This dataset is clearly targeted for research on AR/XR/VR applications.
  - AndrewKemendo 3y ago
    +1 I’ll “yes and” here…beyond AR/VR a more powerful use case is multi-modal learning (with RL) which is what Meta is probably the leader in IMO. Example paper here: “ Towards Continual Egocentric Activity Recognition: A Multi-modal Egocentric Activity Dataset for Continual Learning” https://arxiv.org/abs/2301.10931 https://arxiv.org/abs/2301.10931 This IMO is the pathway to AGI, as it combines all sense-plan-do data into a time coordinated stream and mimics how humans transfer learning to children via demonstration recording and behavior authoring. If we can create robotics with locomotion and dexterous manipulation, egocentric exploration, and a behavior authoring loop that uses human behavior demonstration and trajectory reinforcement - well, we’ll have the AI we’ve been all talking about. Probably the most exciting area of research that most people don’t know or care about. That’s why head mounted all day ego centric AR is so important - it gives eyes ears and sense perception to our learning systems with human directed egocentric behaviors, guiding the whole thing. Just like pushing your kid down the street in the stroller.
    - socratic1 3y ago
      +1 Applications of embodied AI very interesting. Additionally a lot of hard problems are increasingly being solved in simulation like this. See Wayve's GAIA world model
    - dleeftink 3y ago
      Just to make sure I understand your excitement: we need guinea-pigs ahem people to wear 'head mounted all day ego centric AR' with who knows how many integrated sensors for long stretches on end, so we can finally get to our fabled A.G.i? That is some B.F. Skinner level future we're aiming for--only this time around, humans become the fully surveilled 'teaching machine'.
germinalphrase 3y ago
While acknowledging the utility of this kind of object/environment mapping in AR applications - the privacy obliteration is stark.
- KaiserPro 3y ago
  Yes, and it scares the shit out of me. Now. Facebook, the company that nobody trusts to do this sort of thing, is going to have to really work hard to demonstrate that they are to be trusted with this data. Apple, and to a lesser extent google, don't. That cool startup could get away with lots of things, so long as people like the product. Fortunately for us, AR glasses are limited by power consumption, this means that they can't really do always on realtime streaming of data to the backend for mining. Sure you could have always on mm accurate location, but you can't have video recording at the same time. If you want facial recognition, you'll have to stop the music playing. Now, what would help is a decent set of privacy laws, ie: Any cameras smaller than x, must only allow recording of data from persons that expressly allow it, unless in the public domain. People attempting to re-create personally identifiable data from such sensors will be liable to 5 years in jail and or an unlimited fine. (insert carveouts for legitimate research and persons working towards providing evidence for court cases) This isnt perfect, but its a lot better than what we have now.
  - jayd16 3y ago
    I think you underestimate the power consumption of streaming video compared to what a VR headset already does. I mean, they already Chromecast the feed if you so choose. I will say that Meta is fairly aware of their reputation. The TOS is clear that they do not upload or share any video capture and they seem committed to it. As it is now, all the scene understanding is done on device.
    - KaiserPro 3y ago
      > I think you underestimate the power consumption of streaming video compared to what a VR headset already does Rayban stories have a 167mah battery[1] the quest2 has a 3640mah battery[2] even then, it only last 2 hours, more or less, rather than an entire day. [1]https://beckystern.com/2023/04/30/ray-bans-stories-teardown/ https://beckystern.com/2023/04/30/ray-bans-stories-teardown/ [2]https://www.meta.com/gb/legal/quest/product-information-sheet-oculus-quest-2/ https://www.meta.com/gb/legal/quest/product-information-shee...
achr2 3y ago
> All sequences within the Aria Digital Twin Dataset have been captured using fully consented researchers in controlled environments in Meta offices. The requirement and boastful nature of this heading is a frightening tell against the company/industry's perceived practices.
zombiwoof 3y ago
love Meta. they fell face first into the "Metaverse" and now are just releasing all their AI related work to juice their stock and be "innovative".
deleted 3y ago
[deleted]
phyzome 3y ago
This will be complementary to the SecondSelf Skinset Pro: https://samkriss.substack.com/p/how-to-enjoy-your-secondself-skinset https://samkriss.substack.com/p/how-to-enjoy-your-secondself...

180 points by socratic1 3y ago

First thing I notice while browsing from EU is that there's only one option regarding cookies. Accept all! Even if I click "Learn more" there's no "accept necessary cookies" or "Reject cookies". First time I encounter something like this.

otikik 3y ago
Yep, closed it down immediately. IANAL but that might not be legal in Europe
littlestymaar 3y ago
> First time I encounter something like this. Facebook itself use to have this exact banner with no alternative until they were strong-armed to properly comply with GDPR.
WhackyIdeas 3y ago
Noticed the same thing as I always select ‘necessary only’ or something like that. Instead, I just closed the page and clicked on HN comments to see what it was about.
anthonyskipper 3y ago
That's a common dark pattern.
- constantly 3y ago
  It’s not really a dark pattern because they’re not tricking you into accepting or coercing more than is acceptable into doing so, they’re just not allowing you to decline. Which means if not accepting cookies is a dealbreaker for you, no problem, just: move along.
  - deleted 3y ago
    [deleted]
- master-lincoln 3y ago
  Dark pattern is a term normally used for legal, but immoral user experience. In this case legality is questionable as they redirect you to other external sites to manage your "consent" https://optout.aboutads.info https://optout.aboutads.info (at least from a European GDPR perspective)
coldpie 3y ago
I have to admit, for technical folks like ourselves, I don't understand why you care what options the dialog presents. Just use a "Kill Sticky" plugin to nuke the stupid dialog so you can read the page, or even Accept them, and then instruct your browser to do whatever you like with the cookies the site creates (i.e. delete them). It's all in your hands, the popup dialog doesn't do anything you can't do yourself.
- master-lincoln 3y ago
  you should care because often this agreement is not about the technical detail of cookies, but allowing the company you are interacting with to share data about you with 3rd parties.
  - coldpie 3y ago
    If you think which HTML div element you click on to dismiss a sticky banner has any bearing whatsoever on how your data is handled server-side, I've got a trip to the Titanic in an Oceangate submersible to sell you.
    - Etheryte 3y ago
      Legally it does, at least if you're in the EU. The only reason websites are slow on the uptake around this is that the legal gears are slow, however we have seen many considerable fines in this space in the last few years and things are improving.
  - bad_username 3y ago
    It's also in direct violation of GDPR.
- explaininjs 3y ago
  Learned helplessness. Teach the citizens that the only thing protecting them from the Big Scary Internet is their benevolent government. Meanwhile the politicians collect millions from the mass media companies that lobbied for the god-awful implementation of the law we all ended up with, and as everyone gets worn down into Accepting All always, they simultaneously forget that their User Agent holds all the cards (or in this case, cookies), and it can be instructed to do anything the User wants with them.
marcinzm 3y ago
Isn't that a GDPR violation since they're not allowed to prevent access if you refuse to share data not necessary for the service to function? Since it's from Meta I suspect regulators would enjoy another thing to add to the list for future fine calculations.
- dspillett 3y ago
  It depends. If everything stored & tracked is genuinely necessary (as defined by the regulations) then consent is not required. If they are just telling you that they are storing & tracking necessary things like session information, then all is good (unless the things they store & track are not, in fact, strictly necessary and non-privacy-affecting). Of course if they don't need consent, then why are they making a song & dance about it having us click an accept button?
  - IanCal 3y ago
    They explicitly say a use is for ads on and off this site.
- mike_d 3y ago
  Not at all. You can have a single prompt with a single accept button if you are only using functional cookies on a site.
  - capableweb 3y ago
    In that case, you don't need anything, no notice nor consent required at all, get rid of the entire modal/popup/banner.
nightpool 3y ago
This is common on many, many sites like this because they do not have any tracking cookies or anything else that they would need consent for, but they're still required to display a cookie banner "notifying" you that cookies are "in use" as per the terms of the old 2009 ePrivacy Directive. In this case, it appears that projectaria.com sets 1) one cookie for the user's DPR (1 or 2) so that the backend can serve optimized images, 2) one cookie for the user's locale, and 3) one cookie for a CSRF token for form submission.
- capableweb 3y ago
  > but they're still required to display a cookie banner "notifying" you that cookies are "in use" Common misconception but this is not true. If you use cookies only for functional purposes (not for tracking for example), you do not need to show any cookie banners. Like if you have a shopping cart and you have a cookie for keeping track of what's in it, it's for functional purposes for the user and hence needs no notice to be used. The UK's ICO made a handy summary for people who are curious about what the directive actually says: https://ico.org.uk/media/for-organisations/documents/1545/cookies_guidance.pdf https://ico.org.uk/media/for-organisations/documents/1545/co... Specifically: > Exceptions from the requirement to provide information and obtain consent > Activities likely to fall within the exception: [...] Some cookies help ensure that the content of your page loads quickly [...] Certain cookies providing security that is essential to comply with the security requirements [...]
  - sanderjd 3y ago
    But it's easier and less risky to just always put in the standard language that everyone ignores and mindlessly clicks through anyway. Which is why this was very silly legislation. People helping develop future legislation (in the EU and elsewhere) should be aware of this as a cautionary tale of incentivizing theater with only cost and no benefit.
  - nightpool 3y ago
    > Common misconception but this is not true. If you use cookies only for functional purposes (not for tracking for example), you do not need to show any cookie banners. Like if you have a shopping cart and you have a cookie for keeping track of what's in it, it's for functional purposes for the user and hence needs no notice to be used. Personally, I would not put a cookie banner of any kind on my website. However, given this text: The term 'strictly necessary' means that such storage of or access to information should be essential, rather than reasonably necessary, for this exemption to apply. However, it will also be restricted to what is essential to provide the service requested by the user, rather than what might be essential for any other uses the service provider might wish to make of that data. It will also include what is required to comply with any other legislation the person using the cookie might be subject to, for example, the security requirements of the seventh data protection principle. Where the setting of a cookie is deemed 'important' rather than 'strictly necessary', those collecting the information are still obliged to provide information about the device to the potential service recipient and obtain consent. I think it's clear why a more risk-conscious organization like Meta might take a more conservative reading of "Strictly necessary" that does not apply to e.g. bandwidth optimizations related to a device's DPI
cutler 3y ago
This one is tame compared with a whole industry of "data privacy" popups which hide "Legitimate Interest" opt-outs within Vendor lists containing hundreds of entries. Someone needs to slap some serious lawsuits on these gangsters.
mensetmanusman 3y ago
It’s amazing how much worse browsing the web has gotten after all these pop-ups everywhere.
- kaffeeringe 3y ago
  You mean how much worse surfing the web got after it became visible, how much companies are tracking us? The problem is the tracking, not the banner. If you don't sell your readers data, you don't need a banner.
Dirak 3y ago
This is a perfect example of how GDPR can present challenges to innovation. The fact the top comment in this announcement revolves around GDPR compliance and associated fines raises questions about whether companies will be motivated to share research and open source datasets in the future.

machinekob 3y ago

Non-commercial licence :[

optymizer 3y ago
"I can't use this to make money". Sad face. If I had a company, which is a for-profit entity, I wouldn't want competitors to use my company's tech for free either.
Roark66 3y ago
I wonder, let's say one uses it on an open source project that accepts donations. Is that non-commercial enough? Or is it a no-no because money is involved? I'm just curious. I'm glad such free datasets exist even if they are restricted to educational/hobby use.
- sp332 3y ago
  You can't use it in an open-source project. Open source licenses don't restrict commercial use.

quietthrow 3y ago

Genuine question: What is an egocentric data set? Eli5 please? What can this be used for?

ikhatri 3y ago
Egocentric means that the sensors frame of reference is the same as the ego (self). The person walking around is wearing the sensors/glasses themselves & their pose is given as “ego pose”. This dataset is clearly targeted for research on AR/XR/VR applications.
- AndrewKemendo 3y ago
  +1 I’ll “yes and” here…beyond AR/VR a more powerful use case is multi-modal learning (with RL) which is what Meta is probably the leader in IMO. Example paper here: “ Towards Continual Egocentric Activity Recognition: A Multi-modal Egocentric Activity Dataset for Continual Learning” https://arxiv.org/abs/2301.10931 https://arxiv.org/abs/2301.10931 This IMO is the pathway to AGI, as it combines all sense-plan-do data into a time coordinated stream and mimics how humans transfer learning to children via demonstration recording and behavior authoring. If we can create robotics with locomotion and dexterous manipulation, egocentric exploration, and a behavior authoring loop that uses human behavior demonstration and trajectory reinforcement - well, we’ll have the AI we’ve been all talking about. Probably the most exciting area of research that most people don’t know or care about. That’s why head mounted all day ego centric AR is so important - it gives eyes ears and sense perception to our learning systems with human directed egocentric behaviors, guiding the whole thing. Just like pushing your kid down the street in the stroller.
  - socratic1 3y ago
    +1 Applications of embodied AI very interesting. Additionally a lot of hard problems are increasingly being solved in simulation like this. See Wayve's GAIA world model
  - dleeftink 3y ago
    Just to make sure I understand your excitement: we need guinea-pigs ahem people to wear 'head mounted all day ego centric AR' with who knows how many integrated sensors for long stretches on end, so we can finally get to our fabled A.G.i? That is some B.F. Skinner level future we're aiming for--only this time around, humans become the fully surveilled 'teaching machine'.

germinalphrase 3y ago

While acknowledging the utility of this kind of object/environment mapping in AR applications - the privacy obliteration is stark.

KaiserPro 3y ago
Yes, and it scares the shit out of me. Now. Facebook, the company that nobody trusts to do this sort of thing, is going to have to really work hard to demonstrate that they are to be trusted with this data. Apple, and to a lesser extent google, don't. That cool startup could get away with lots of things, so long as people like the product. Fortunately for us, AR glasses are limited by power consumption, this means that they can't really do always on realtime streaming of data to the backend for mining. Sure you could have always on mm accurate location, but you can't have video recording at the same time. If you want facial recognition, you'll have to stop the music playing. Now, what would help is a decent set of privacy laws, ie: Any cameras smaller than x, must only allow recording of data from persons that expressly allow it, unless in the public domain. People attempting to re-create personally identifiable data from such sensors will be liable to 5 years in jail and or an unlimited fine. (insert carveouts for legitimate research and persons working towards providing evidence for court cases) This isnt perfect, but its a lot better than what we have now.
- jayd16 3y ago
  I think you underestimate the power consumption of streaming video compared to what a VR headset already does. I mean, they already Chromecast the feed if you so choose. I will say that Meta is fairly aware of their reputation. The TOS is clear that they do not upload or share any video capture and they seem committed to it. As it is now, all the scene understanding is done on device.
  - KaiserPro 3y ago
    > I think you underestimate the power consumption of streaming video compared to what a VR headset already does Rayban stories have a 167mah battery[1] the quest2 has a 3640mah battery[2] even then, it only last 2 hours, more or less, rather than an entire day. [1]https://beckystern.com/2023/04/30/ray-bans-stories-teardown/ https://beckystern.com/2023/04/30/ray-bans-stories-teardown/ [2]https://www.meta.com/gb/legal/quest/product-information-sheet-oculus-quest-2/ https://www.meta.com/gb/legal/quest/product-information-shee...

achr2 3y ago

> All sequences within the Aria Digital Twin Dataset have been captured using fully consented researchers in controlled environments in Meta offices. The requirement and boastful nature of this heading is a frightening tell against the company/industry's perceived practices.

zombiwoof 3y ago

love Meta. they fell face first into the "Metaverse" and now are just releasing all their AI related work to juice their stock and be "innovative".

deleted 3y ago

[deleted]

phyzome 3y ago

This will be complementary to the SecondSelf Skinset Pro: https://samkriss.substack.com/p/how-to-enjoy-your-secondself-skinset https://samkriss.substack.com/p/how-to-enjoy-your-secondself...