6 ms·

What we need is sensors that can scan polarization on a per-pixel basis (like 256 orientations per pixel per image. Then it would be much easier to detect and r

by kloch 2y ago

What we need is sensors that can scan polarization on a per-pixel basis (like 256 orientations per pixel per image. Then it would be much easier to detect and remove consistently polarized components of the image (as specular reflections from glass are). This would just be a fully electronic/computational version of a mechanical polarizing filter.

planede 2y ago
> like 256 orientations per pixel per image You only need 4 parameters to describe the polarization at a single wavelength[1]. Naively this could be 4 parameters per color channel, so 12 channels overall. I think you could potentially need more color channels though to capture the full spectrum. But 12 channels at least looks feasible for a camera. [1] https://en.wikipedia.org/wiki/Stokes_parameters https://en.wikipedia.org/wiki/Stokes_parameters edit: On second thought for dealing with reflections you might get away not capturing the "V" Stokes parameter, as you might not care about circular polarization. edit2: The I,Q and U parameters can be captured fully by a single polarization filter at three different rotations. This could be feasible with existing cameras with a tripod and a static subject. I wonder if this has been done before.
- cornellwright 2y ago
  You can buy [1] polarization cameras, both mono and with a Bayer filter. They're expensive right now, but I agree it would be really cool to see what could be done with a consumer grade version in a smart phone. [1] https://thinklucid.com/product/phoenix-5-0-mp-polarized-model/ https://thinklucid.com/product/phoenix-5-0-mp-polarized-mode... (among many others)
  - c0nfused 2y ago
    What is the advantage here vs a polarization filter and a standard camera?
    - londons_explore 2y ago
      You can capture the whole scene in a single exposure. Handy for moving images.
  - planede 2y ago
    Interesting. From what I can find the pixel format is 4 polarization directions per pixel, 45 degrees apart. Even though there are 4 channels this doesn't allow to deduce the V Stokes parameter (this camera can't capture circular polarization). Technically one channel is redundant here, but I guess it can be useful for reducing error. I wonder if an alternative pixel format, with 3 polarization directions 60 degrees apart and a circular polarization channel would be desirable for some applications.
- im3w1l 2y ago
  I'm pretty sure he means a single byte-valued parameter. As you mention a single parameter is not enough to fully describe the polarization but maybe it's good enough - I guess you would average across colors, and say circular polarization would lead to a basically random value.
  - kloch 2y ago
    I did indeed mean a singe, byte-valued parameter indicating angle (similar to the single angle parameter of a mechanical polarizing filter) Full polarization and phase info would be great to have also but probably not necessary for reflection suppression. And yes purely circular polarization would be undefined in this scenario but again not common (possible?) with reflections.
HALtheWise 2y ago
Due to quantum physics, there's actually only two degrees of freedom in the ways light can be polarized, referred to as the "Jones Vector". In other words, it's impossible even in theory to distinguish between light that has exactly two perpendicular polarizations mixed together and light that is fully unpolarized and has thousands located all around the circle. That makes it surprisingly possible to build a camera that captures _everything_ there is to know about light at some particular frequency. https://en.wikipedia.org/wiki/Jones_calculus https://en.wikipedia.org/wiki/Jones_calculus
- amluto 2y ago
  Not quite — that’s for polarized light. For general light that may be unpolarized, you need four parameters. You can use the Stokes parameters, or, if you’re feeling very quantum, you can describe the full polarization state of a photon by a 2x2 density matrix. (I have never personally calculated this, but I’m pretty sure you can straightforwardly translate one formulation to the other — the density matrix captures the polarization distribution of a photon sampled, by whatever means, from any source of incoming light.)
buildbot 2y ago
These exist, I have one, and it lets you do exactly what this paper does! https://thinklucid.com/product/triton-5-mp-polarization-camera/ https://thinklucid.com/product/triton-5-mp-polarization-came... https://thinklucid.com/polarized-camera-resource-center/ https://thinklucid.com/polarized-camera-resource-center/
- dylan604 2y ago
  Yes, because my mom* is not going to carry that around to take pics of the grandkids. Just because something exists does not mean it is practical. I can totally see how having a software solution that Apple can include in its fakeypics app, then my mom would be able to take advantage of this. *Avoiding the use of the phrase "your mom"!
  - buildbot 2y ago
    …Nobody said they were going to… Apple could request a sensor with the polarsens mask. It’s just not worth it, from a resolution & light gathering perspective. Big tradeoffs improvements in specific scenarios is not a path Apple has taken typically for their cameras.
    - dylan604 2y ago
      > …Nobody said they were going to… talking about missing the point... Apple is not going to make a hardware change like your suggestion, but they would be much more likely use the software concept from TFA. I'm assuming that Googs, Samsung, CCPhardware would be similar. They need to do something compelling with all of the specific compute they are including in their devices.
radlad 2y ago
> [...] remove consistently polarized components of the image (as specular reflections from glass are). It was my understanding that reflections in glass can be either polarized or non-polarized, or a mix of both. If you use a polarizing filter on a camera (e.g. when taking photos of artwork through glass, or shooting over water that you want to see into), you will often find that it does not remove all reflections. https://en.wikipedia.org/wiki/Brewster%27s_angle https://en.wikipedia.org/wiki/Brewster%27s_angle
ikari_pl 2y ago
i used to have a Lytro camera very interesting device, also took about 8 angles of every photo and built a spatial interpretation, not too advanced

by kloch 2y ago

planede 2y ago

> like 256 orientations per pixel per image You only need 4 parameters to describe the polarization at a single wavelength[1]. Naively this could be 4 parameters per color channel, so 12 channels overall. I think you could potentially need more color channels though to capture the full spectrum. But 12 channels at least looks feasible for a camera. [1] https://en.wikipedia.org/wiki/Stokes_parameters https://en.wikipedia.org/wiki/Stokes_parameters edit: On second thought for dealing with reflections you might get away not capturing the "V" Stokes parameter, as you might not care about circular polarization. edit2: The I,Q and U parameters can be captured fully by a single polarization filter at three different rotations. This could be feasible with existing cameras with a tripod and a static subject. I wonder if this has been done before.

cornellwright 2y ago
You can buy [1] polarization cameras, both mono and with a Bayer filter. They're expensive right now, but I agree it would be really cool to see what could be done with a consumer grade version in a smart phone. [1] https://thinklucid.com/product/phoenix-5-0-mp-polarized-model/ https://thinklucid.com/product/phoenix-5-0-mp-polarized-mode... (among many others)
- c0nfused 2y ago
  What is the advantage here vs a polarization filter and a standard camera?
  - londons_explore 2y ago
    You can capture the whole scene in a single exposure. Handy for moving images.
- planede 2y ago
  Interesting. From what I can find the pixel format is 4 polarization directions per pixel, 45 degrees apart. Even though there are 4 channels this doesn't allow to deduce the V Stokes parameter (this camera can't capture circular polarization). Technically one channel is redundant here, but I guess it can be useful for reducing error. I wonder if an alternative pixel format, with 3 polarization directions 60 degrees apart and a circular polarization channel would be desirable for some applications.
im3w1l 2y ago
I'm pretty sure he means a single byte-valued parameter. As you mention a single parameter is not enough to fully describe the polarization but maybe it's good enough - I guess you would average across colors, and say circular polarization would lead to a basically random value.
- kloch 2y ago
  I did indeed mean a singe, byte-valued parameter indicating angle (similar to the single angle parameter of a mechanical polarizing filter) Full polarization and phase info would be great to have also but probably not necessary for reflection suppression. And yes purely circular polarization would be undefined in this scenario but again not common (possible?) with reflections.

HALtheWise 2y ago

Due to quantum physics, there's actually only two degrees of freedom in the ways light can be polarized, referred to as the "Jones Vector". In other words, it's impossible even in theory to distinguish between light that has exactly two perpendicular polarizations mixed together and light that is fully unpolarized and has thousands located all around the circle. That makes it surprisingly possible to build a camera that captures _everything_ there is to know about light at some particular frequency. https://en.wikipedia.org/wiki/Jones_calculus https://en.wikipedia.org/wiki/Jones_calculus

amluto 2y ago
Not quite — that’s for polarized light. For general light that may be unpolarized, you need four parameters. You can use the Stokes parameters, or, if you’re feeling very quantum, you can describe the full polarization state of a photon by a 2x2 density matrix. (I have never personally calculated this, but I’m pretty sure you can straightforwardly translate one formulation to the other — the density matrix captures the polarization distribution of a photon sampled, by whatever means, from any source of incoming light.)

buildbot 2y ago

These exist, I have one, and it lets you do exactly what this paper does! https://thinklucid.com/product/triton-5-mp-polarization-camera/ https://thinklucid.com/product/triton-5-mp-polarization-came... https://thinklucid.com/polarized-camera-resource-center/ https://thinklucid.com/polarized-camera-resource-center/

dylan604 2y ago
Yes, because my mom* is not going to carry that around to take pics of the grandkids. Just because something exists does not mean it is practical. I can totally see how having a software solution that Apple can include in its fakeypics app, then my mom would be able to take advantage of this. *Avoiding the use of the phrase "your mom"!
- buildbot 2y ago
  …Nobody said they were going to… Apple could request a sensor with the polarsens mask. It’s just not worth it, from a resolution & light gathering perspective. Big tradeoffs improvements in specific scenarios is not a path Apple has taken typically for their cameras.
  - dylan604 2y ago
    > …Nobody said they were going to… talking about missing the point... Apple is not going to make a hardware change like your suggestion, but they would be much more likely use the software concept from TFA. I'm assuming that Googs, Samsung, CCPhardware would be similar. They need to do something compelling with all of the specific compute they are including in their devices.

radlad 2y ago

> [...] remove consistently polarized components of the image (as specular reflections from glass are). It was my understanding that reflections in glass can be either polarized or non-polarized, or a mix of both. If you use a polarizing filter on a camera (e.g. when taking photos of artwork through glass, or shooting over water that you want to see into), you will often find that it does not remove all reflections. https://en.wikipedia.org/wiki/Brewster%27s_angle https://en.wikipedia.org/wiki/Brewster%27s_angle

ikari_pl 2y ago

i used to have a Lytro camera very interesting device, also took about 8 angles of every photo and built a spatial interpretation, not too advanced