5 ms·

Hey all, wanted to share a new ARKit app I recently published. You can choose a photo from your gallery, specify its dimensions, and see how it'd look directly

Hey all, wanted to share a new ARKit app I recently published. You can choose a photo from your gallery, specify its dimensions, and see how it'd look directly on your wall! Inspiration for this project came when I was looking to buy artwork over the summer, but wasn't sure what size would look best on my wall. I made this so ideally you could preview how a piece of art would look on your wall before you buy it. Would love any feedback on it

mensetmanusman 5y ago
AR camera shots often have an uncanny valley appearance. It would be cool if there were an option to have the initial simulated image ‘annealed’ with a generative adversarial network. Maybe you could also partner with local frame builders (ask for 2% of sales or something) to send clients your way, since, so often, the frame has a large effect on the art’s aesthetic impact.
- 10000truths 5y ago
  I think the uncanny valley thing is because the superimposition does not take into account the lighting conditions of the room.
  - hitmyapi 5y ago
    Yup the superimposition is currently setup to ignore light in the room. I experimented with having it respond to light in the room, but ran into issues with the image rendering way too dark. I'll spend some more time fine tuning it and seeing if I can get more realistic light rendering into a future release
    - jetbooster 5y ago
      Isn't determining lighting without some reference card a "hard" problem? How do you tell the difference between a yellowish room illuminated by white (sunlight) versus a whitish room illuminated by warmer yellower bulbs? I am a not a photography person so this a genuine question asked from ignorance
      - hitmyapi 5y ago
        Yup that's a great question, I think generally speaking that is a really difficult problem in the AR space, which I think is part of the reason why most AR models still look fake or have the uncanny valley. Using the ARKit libraries, it gets handled automatically for the most part. I think there's a lot of computer vision/image processing algorithms running in the background that potentially pick up on shadows/reflections in real world objects, which help it determine the white balance/color temperature of the room. I'm no expert on it myself though and not 100% sure on the science behind it
- TAForObvReasons 5y ago
  Go all the way and make a "buy now" option. That would entail partnering with printers and framers to offer standalone posters and framed versions of standard / custom artwork.
huskyr 5y ago
Thanks, worked very simple and fast. Great that you published this for free on the app store. One small feature request: it would be useful to change between metric/imperial measurements. I don't live in the US and don't now how long an inch is.
- hitmyapi 5y ago
  Thank you, happy it worked for you :) yup that's a great call, adding support for different units is in the works!
DonHopkins 5y ago
Great job! My ultimate (and totally unrealistic) wish list: Multitouch dragging and resizing and rotating and pushing and pulling images. Really cool if it works seamlessly with camera motion and rotation too. So you can correct the size and depth of misplaced pictures easily. (The dragging gesture currently does, hurrya! Now "just" make the resizing and twisting and pulling and pushing gestures work that way by following camera motion too! Yes it's tricky code but it'll be worth it and satisfying.) Bump the edges of dragging and resizing pictures up against the corners of the room. And when picture edges bump together they push each other around, with 2d physics (or use 3d physics with gravity oriented so the wall is a flat or graded floor). So you can't drag a picture into an orthogonal wall, or make pictures on the same wall overlap, and the wall corners and picture frames have more physicality, so you can easily align multiple images along a corner. Scroll all the pictures on the wall at once by dragging the background like a virtual desktop that clips to the corners of the room. Full depth 3d frames, that you can look around and see the back of. Hinged frames that you can open this way or that to reveal a hidden safe (that you can open by twisting the dial to see what's inside) or window (that you can see through in parallax to another outdoor photo (or 3d scene with dancing characters) you've placed outside in the distance, like the a-ha "Take On Me" music video door AR demo). https://www.youtube.com/watch?v=djV11Xbc914 https://www.youtube.com/watch?v=djV11Xbc914 https://techcrunch.com/2017/07/27/someone-made-the-take-on-me-music-video-come-to-life-with-ar-and-its-glorious/ https://techcrunch.com/2017/07/27/someone-made-the-take-on-m... https://www.youtube.com/watch?v=gaEeRoBY8jY https://www.youtube.com/watch?v=gaEeRoBY8jY Flat screen TVs for walls, big deep retro monitors for desktops, and big wooden Magnivox Color Television consoles with rounded screens for floors. Sometimes the reception cuts out and you have to fiddle with the antenna to get the picture back. In-app purchase of a cable tv subscription to fix all your bad reception problems once and for all. Simulating a backlighted screen solves the "uncanny gap" lighting problem by being emissive like a TV, instead of reflective like a painting. Well you still have to light the frame itself, but ARKit will tell you the light direction estimate, and it's ok if your frames have dark and shadowy edges and bright specular highlights, since it's not the actual picture you're trying to look at, just a frame. Save and exchange scenes online! Map (load and save over the net) your sets of pictures on one individual flat wall surface to other people's flat walls in different rooms, instead of trying to map between the entire 3d rooms, which probably won't match in size and shape. So I could have one wall mapped back-and-forth with one person's wall, and another wall mapped back-and-forth with another person's wall. Then stream live video in the pictures!
jw1224 5y ago
Cool project! I have a pretty extensive niche art collection at home, and this sort of thing would be really useful. If I could suggest one important addition... It would be great if you could set a surrounding frame and matting. You could maybe offer a few different frame styles/colours, and let users access a colour picker to select the shade for the matting. Then you could project a "framed" piece of art on the wall with AR. This would make it really easy to see how a new piece would look in context, alongside other artwork which has been professionally framed. It would save a lot of uncertainty before a visit to the framing shop :)
- hitmyapi 5y ago
  Agreed I think that would be a great addition to be able to select and customize different frames, I could see how it'd be valuable to see it framed next to other framed artwork already on the wall. It was a feature I decided to leave off for the initial launch for simplicity, but will prioritize adding it into a future release. Appreciate the suggestion! :)
  - jw1224 5y ago
    Thanks! Also, just curious… How does developing for ARKit work? Does the Xcode Simulator have “virtual rooms” to test your code in, or do you have to compile and run from a real device each time?
    - hitmyapi 5y ago
      Unfortunately the app won't even compile on the simulator, it seems like some of the AR libraries aren't actually available when building for the simulator. I have to test on real device each time for the time being. That would be super cool if there were virtual rooms to test AR code in, I feel it could expedite some of the development time haha
  - ramses0 5y ago
    Pro mode to allow anything other than 2-3 basic frame styles. “See your photo on the wall for free… match it to a frame size, color, mat size, color for $9.99” Art costs at least $50 for something framed, so $10 for an app that can be re-used is great. For true pros, I can’t think of what would be something that was subscription-valuablenorher than maybe $10/yr, which would be mostly fine for homeowners who have just moved, and would help have a sustaining income for “true pros” who would want to use it continuously.
2muchcoffeeman 5y ago
Why are we trying to “-fy” everything?
- deleted 5y ago
  [deleted]
- CharlesW 5y ago
  The suffix "-ify" means "to become, or to make someone or something become, something". So if you make an app that allows a photo to become a virtual canvas, "Canvify" is short, unique name that's at least semi-self-descriptive.
  - 2muchcoffeeman 5y ago
    I understand the convention. And I don’t want to be too negative. But it’s a really awkward sounding name. And in tech I see naming conventions in products that go in and out of fashion. ‘-ly’, ‘-fy’, ‘i-‘, ‘-(2 digit year)’.
    - hitmyapi 5y ago
      Yeah I agree "Canvify" doesn't exactly roll off the tongue nicely. I probably could've spent more time brainstorming a better name, but mainly I wanted something unique and somewhat related to artwork/canvases. For better or for worse, it's such a unique name that this is the only app that shows up in the App Store when searching "canvify"!
      - DonHopkins 5y ago
        You done good. NEVER let brainstorming and bikeshedding for the perfect name delay you from writing code for one nanosecond. (Just do that in your free time while you're taking a walk or riding a bike or sleeping, and rename it later.) If your idea isn't any good without the perfect name, then you idea isn't any good. That being said, how about Canv-O-Matic? No?
mortenjorck 5y ago
This is off to a great start! A few initial UX suggestions: - Most users are probably just looking to see how a single copy of something looks on their wall, so instead of subsequent taps adding subsequent copies, they should probably just move the existing one. - My iPhone doesn’t have LiDAR, so it’s somewhat hit or miss whether ARKit hits the actual wall, or ends up a foot in front of it (or gets the angle wrong). Consider offering an adjustment mode to manually slide the artwork into place. And now a product suggestion: - Consider pitching an integration to Society6, one of the largest marketplaces for wall art. It would be a huge value add for customers to be able to view prints directly from creators’ catalogs with pre-filled sizes, and I’d imagine you could work something out that would support this project well into the future.
- hitmyapi 5y ago
  Love those suggestions, thank you for your comment! Definitely agree on adding better support for the non-LiDAR devices, I've noticed it's really hit or miss when placing and should be easier to correct
biztos 5y ago
I like this and I bet eventually it will be standard practice for galleries to have something like this. There's a lot of money in the art world, and especially since the pandemic not everyone can travel to every show even if they could afford it! So here's my business advice: make this a white-label app and try to license it to Gagosian et al. Now for a massive feature suggestion, probably not possible with ARKit: I want to record a room, then arrange pictures in it "AR-style" when I am not in the room anymore. Because it's much more common for me to be where the picture is, and wonder how it would look in my room, than to be in the room and have the picture not there. (I make and also collect art.)
sci_prog 5y ago
Is there a reason your minimum targeted iOS version is iOS 15? Are you using any APIs/features that are only available in the latest iOS?
scrooched_moose 5y ago
As someone who custom builds a lot of picture frames, the ability to add frame + mat ratio to the image would be great. Those can more than double the linear dimensions (4X+ on the area) and are a pain to properly size for the wall. For example, something like: +3" dark brown frame (for walnut, wood texture would be a plus) +2" left/right; +2.5" top, +3" bottom white mat board (reveals are not usually uniform in custom work) I currently spend a lot of time mocking up these ratios in Gimp before starting work, something that automates it and shows it on the wall would be amazing. Adding depth would be a nice touch too if that's possible, stretched canvas can stick out 1.5" or more. Great work though, I like it so far and might be able to use it to help my mockups a bit already.

mensetmanusman 5y ago

AR camera shots often have an uncanny valley appearance. It would be cool if there were an option to have the initial simulated image ‘annealed’ with a generative adversarial network. Maybe you could also partner with local frame builders (ask for 2% of sales or something) to send clients your way, since, so often, the frame has a large effect on the art’s aesthetic impact.

10000truths 5y ago
I think the uncanny valley thing is because the superimposition does not take into account the lighting conditions of the room.
- hitmyapi 5y ago
  Yup the superimposition is currently setup to ignore light in the room. I experimented with having it respond to light in the room, but ran into issues with the image rendering way too dark. I'll spend some more time fine tuning it and seeing if I can get more realistic light rendering into a future release
  - jetbooster 5y ago
    Isn't determining lighting without some reference card a "hard" problem? How do you tell the difference between a yellowish room illuminated by white (sunlight) versus a whitish room illuminated by warmer yellower bulbs? I am a not a photography person so this a genuine question asked from ignorance
    - hitmyapi 5y ago
      Yup that's a great question, I think generally speaking that is a really difficult problem in the AR space, which I think is part of the reason why most AR models still look fake or have the uncanny valley. Using the ARKit libraries, it gets handled automatically for the most part. I think there's a lot of computer vision/image processing algorithms running in the background that potentially pick up on shadows/reflections in real world objects, which help it determine the white balance/color temperature of the room. I'm no expert on it myself though and not 100% sure on the science behind it
TAForObvReasons 5y ago
Go all the way and make a "buy now" option. That would entail partnering with printers and framers to offer standalone posters and framed versions of standard / custom artwork.

huskyr 5y ago

Thanks, worked very simple and fast. Great that you published this for free on the app store. One small feature request: it would be useful to change between metric/imperial measurements. I don't live in the US and don't now how long an inch is.

hitmyapi 5y ago
Thank you, happy it worked for you :) yup that's a great call, adding support for different units is in the works!

DonHopkins 5y ago

Great job! My ultimate (and totally unrealistic) wish list: Multitouch dragging and resizing and rotating and pushing and pulling images. Really cool if it works seamlessly with camera motion and rotation too. So you can correct the size and depth of misplaced pictures easily. (The dragging gesture currently does, hurrya! Now "just" make the resizing and twisting and pulling and pushing gestures work that way by following camera motion too! Yes it's tricky code but it'll be worth it and satisfying.) Bump the edges of dragging and resizing pictures up against the corners of the room. And when picture edges bump together they push each other around, with 2d physics (or use 3d physics with gravity oriented so the wall is a flat or graded floor). So you can't drag a picture into an orthogonal wall, or make pictures on the same wall overlap, and the wall corners and picture frames have more physicality, so you can easily align multiple images along a corner. Scroll all the pictures on the wall at once by dragging the background like a virtual desktop that clips to the corners of the room. Full depth 3d frames, that you can look around and see the back of. Hinged frames that you can open this way or that to reveal a hidden safe (that you can open by twisting the dial to see what's inside) or window (that you can see through in parallax to another outdoor photo (or 3d scene with dancing characters) you've placed outside in the distance, like the a-ha "Take On Me" music video door AR demo). https://www.youtube.com/watch?v=djV11Xbc914 https://www.youtube.com/watch?v=djV11Xbc914 https://techcrunch.com/2017/07/27/someone-made-the-take-on-me-music-video-come-to-life-with-ar-and-its-glorious/ https://techcrunch.com/2017/07/27/someone-made-the-take-on-m... https://www.youtube.com/watch?v=gaEeRoBY8jY https://www.youtube.com/watch?v=gaEeRoBY8jY Flat screen TVs for walls, big deep retro monitors for desktops, and big wooden Magnivox Color Television consoles with rounded screens for floors. Sometimes the reception cuts out and you have to fiddle with the antenna to get the picture back. In-app purchase of a cable tv subscription to fix all your bad reception problems once and for all. Simulating a backlighted screen solves the "uncanny gap" lighting problem by being emissive like a TV, instead of reflective like a painting. Well you still have to light the frame itself, but ARKit will tell you the light direction estimate, and it's ok if your frames have dark and shadowy edges and bright specular highlights, since it's not the actual picture you're trying to look at, just a frame. Save and exchange scenes online! Map (load and save over the net) your sets of pictures on one individual flat wall surface to other people's flat walls in different rooms, instead of trying to map between the entire 3d rooms, which probably won't match in size and shape. So I could have one wall mapped back-and-forth with one person's wall, and another wall mapped back-and-forth with another person's wall. Then stream live video in the pictures!

jw1224 5y ago

Cool project! I have a pretty extensive niche art collection at home, and this sort of thing would be really useful. If I could suggest one important addition... It would be great if you could set a surrounding frame and matting. You could maybe offer a few different frame styles/colours, and let users access a colour picker to select the shade for the matting. Then you could project a "framed" piece of art on the wall with AR. This would make it really easy to see how a new piece would look in context, alongside other artwork which has been professionally framed. It would save a lot of uncertainty before a visit to the framing shop :)

hitmyapi 5y ago
Agreed I think that would be a great addition to be able to select and customize different frames, I could see how it'd be valuable to see it framed next to other framed artwork already on the wall. It was a feature I decided to leave off for the initial launch for simplicity, but will prioritize adding it into a future release. Appreciate the suggestion! :)
- jw1224 5y ago
  Thanks! Also, just curious… How does developing for ARKit work? Does the Xcode Simulator have “virtual rooms” to test your code in, or do you have to compile and run from a real device each time?
  - hitmyapi 5y ago
    Unfortunately the app won't even compile on the simulator, it seems like some of the AR libraries aren't actually available when building for the simulator. I have to test on real device each time for the time being. That would be super cool if there were virtual rooms to test AR code in, I feel it could expedite some of the development time haha
- ramses0 5y ago
  Pro mode to allow anything other than 2-3 basic frame styles. “See your photo on the wall for free… match it to a frame size, color, mat size, color for $9.99” Art costs at least $50 for something framed, so $10 for an app that can be re-used is great. For true pros, I can’t think of what would be something that was subscription-valuablenorher than maybe $10/yr, which would be mostly fine for homeowners who have just moved, and would help have a sustaining income for “true pros” who would want to use it continuously.

2muchcoffeeman 5y ago

Why are we trying to “-fy” everything?

deleted 5y ago
[deleted]
CharlesW 5y ago
The suffix "-ify" means "to become, or to make someone or something become, something". So if you make an app that allows a photo to become a virtual canvas, "Canvify" is short, unique name that's at least semi-self-descriptive.
- 2muchcoffeeman 5y ago
  I understand the convention. And I don’t want to be too negative. But it’s a really awkward sounding name. And in tech I see naming conventions in products that go in and out of fashion. ‘-ly’, ‘-fy’, ‘i-‘, ‘-(2 digit year)’.
  - hitmyapi 5y ago
    Yeah I agree "Canvify" doesn't exactly roll off the tongue nicely. I probably could've spent more time brainstorming a better name, but mainly I wanted something unique and somewhat related to artwork/canvases. For better or for worse, it's such a unique name that this is the only app that shows up in the App Store when searching "canvify"!
    - DonHopkins 5y ago
      You done good. NEVER let brainstorming and bikeshedding for the perfect name delay you from writing code for one nanosecond. (Just do that in your free time while you're taking a walk or riding a bike or sleeping, and rename it later.) If your idea isn't any good without the perfect name, then you idea isn't any good. That being said, how about Canv-O-Matic? No?

mortenjorck 5y ago

This is off to a great start! A few initial UX suggestions: - Most users are probably just looking to see how a single copy of something looks on their wall, so instead of subsequent taps adding subsequent copies, they should probably just move the existing one. - My iPhone doesn’t have LiDAR, so it’s somewhat hit or miss whether ARKit hits the actual wall, or ends up a foot in front of it (or gets the angle wrong). Consider offering an adjustment mode to manually slide the artwork into place. And now a product suggestion: - Consider pitching an integration to Society6, one of the largest marketplaces for wall art. It would be a huge value add for customers to be able to view prints directly from creators’ catalogs with pre-filled sizes, and I’d imagine you could work something out that would support this project well into the future.

hitmyapi 5y ago
Love those suggestions, thank you for your comment! Definitely agree on adding better support for the non-LiDAR devices, I've noticed it's really hit or miss when placing and should be easier to correct

biztos 5y ago

I like this and I bet eventually it will be standard practice for galleries to have something like this. There's a lot of money in the art world, and especially since the pandemic not everyone can travel to every show even if they could afford it! So here's my business advice: make this a white-label app and try to license it to Gagosian et al. Now for a massive feature suggestion, probably not possible with ARKit: I want to record a room, then arrange pictures in it "AR-style" when I am not in the room anymore. Because it's much more common for me to be where the picture is, and wonder how it would look in my room, than to be in the room and have the picture not there. (I make and also collect art.)

sci_prog 5y ago

Is there a reason your minimum targeted iOS version is iOS 15? Are you using any APIs/features that are only available in the latest iOS?

scrooched_moose 5y ago

As someone who custom builds a lot of picture frames, the ability to add frame + mat ratio to the image would be great. Those can more than double the linear dimensions (4X+ on the area) and are a pain to properly size for the wall. For example, something like: +3" dark brown frame (for walnut, wood texture would be a plus) +2" left/right; +2.5" top, +3" bottom white mat board (reveals are not usually uniform in custom work) I currently spend a lot of time mocking up these ratios in Gimp before starting work, something that automates it and shows it on the wall would be amazing. Adding depth would be a nice touch too if that's possible, stretched canvas can stick out 1.5" or more. Great work though, I like it so far and might be able to use it to help my mockups a bit already.