7 ms·

Barcode Detection API

331 points by addcninblue 5y ago

miohtama 5y ago
As the page does not discuss the motivation behind the API, does anyone know why this is ”native” API and not just a generic image processing script implemented with JavaScript / WebGL / etc.?
- bena 5y ago
  Because while barcode scanning is a subset of image processing, it's a use case that's pretty common.
- rahimnathwani 5y ago
  To allow the implementation to take advantage of libraries provided by the operating system[0] (where available), without the web site developer needing to even know that it's happening. [0] or things like Google Play Services.
- PaulHoule 5y ago
  At this point in time it's a pain point for me that Android devices don't usually have built-in QR reader software. I can tell iOS users to just "scan the code with the Camera" app but I have to tell Android users to find a QR scanner app on the app store. My experience with an actual bottom-of-the-line prepaid phone was that the first QR scanner app I downloaded from Google Play worked right the first time but I had to try six different apps before I found one that worked with my Samsung Tablet. A web-based scanner would be a possible answer, though really Google and the phone makers and the carriers should have prioritized a "just works" solution years ago.
  - eertami 5y ago
    Honest question: for what reasons do you ever need to scan a random QR code? I think I've used them to sign in to some apps (eg, WhatsApp) while logged in on a PC, and other than that exactly 0 times. The Google camera app does support QR codes FWIW, but I don't know if any other vendors except Google use it.
    - simonbw 5y ago
      In general, assuming you have an easy-to-use QR scanner, it's easier to scan a QR code than type in a URL on a phone, especially a complicated one. A couple examples I've run into recently: - Lately it's been pretty common for restaurants to replace their physical menus with a QR code on the table that links to their online menu. - When I've taken COVID tests recently, the site gives me a sticker with a QR code that links to my results page. - I encoded my wifi network/password into a QR code and printed it out, so guests can just scan it to join, rather than have to scroll and find the right network and type in a password.
      - PaulHoule 5y ago
        Displaying a QR code on a screen is a quick way to beam a URL to a phone or tablet either from a PC or another mobile. You can put up a QR code to point people to the web app which controls the IoT functions in a space, music system, etc.
    - deleted 5y ago
      [deleted]
    - PaulHoule 5y ago
      I print ‘three-sided cards’ that have a photo or art reproduction on one side, some documentation on the back, and more information if you scan the QR code. One series of the cards are ‘music cards’ that will play a song, another application is ‘constellations’ that let me stick anywhere from 3 to 40 cards on the wall (so far) and you can see the back sides and other narration by scanning a code. So my use cases are (1) I am demoing the cards for people, and (2) I give the cards away for free (If it wasn’t for the supply chain crisis I’d be creating a new card design every day) or put up an installation somewhere and people interact with them. Either way I want it to be very easy for people.
    - LouisSayers 5y ago
      I literally just scanned a couple while at the doctors - one to check in, and another provided a url of a form to fill in. They're also in some restaurants where you can get to a menu to order from your phone. You also scan QR codes for use in two factor authentication apps. Covid has normalised QR code scanning in Australia - we have to do it at certain venues in order to show our vaccine passes.
  - Bilal_io 5y ago
    I've been scanning QR codes with my Android phone for a long time. I know at least two native cameras work, Pixel's and Samsung's since 2019. But I am sure legacy phones with older android versions don't support it natively.
    - fy20 5y ago
      I have a flagship Xiaomi phone and the built in QR code scanner is garbage. I went to a restaurant with friends which only had a QR code for the menu (which turned out just to be a link to the root of their website - why not write the URL too?). In the end I had to take a picture of the QR code, then crop it, then open that in the scanner app.
    - calmoo 5y ago
      My Huawei P20 doesn't have QR scanning native to the camera app.
    - Lev1a 5y ago
      > I know at least two native cameras work, Pixel's and Samsung's since 2019 My Motorola One can do that as well and IIRC the "moto g4" I had before that too.
      - WesternWind 5y ago
        I've had intermittent issues with my motorola's camera app. Sometimes it picks up qr codes at restaurants, sometimes it refuses, maybe because of lighting issues, though turning on the flash doesn't help. I ended up downloading BinaryEye from F-Droid and I just default to that now.
  - RenThraysk 5y ago
    It's built into Google Lens. Use it occasionally to send files over wifi to phone, with webwormhole.
  - SkyPuncher 5y ago
    > At this point in time it's a pain point for me that Android devices don't usually have built-in QR reader software It's hidden, but I believe it's built in the default camera app now. I've learned that my last 3 phones have had it built in.
  - easrng 5y ago
    My Motorola phone from 2017 has QR support in the Camera app.
  - OJFord 5y ago
    As always, some fragment of Android actually got there before Apple (to put it in the Camera app), but no, it's not standard. My current phone (Nokia 3.4) doesn't have it, but previous one (Motorala One) did. Ironic, since I chose them for being about as close to stock as you can get, both in the 'Android One' programme (so also comparatively LTS).
    - olvrng 5y ago
      It's always included in Google Lens (and Nokia 3.4 uses Android One). You can open Google Lens and point to the QR code.
      - OJFord 5y ago
        Yes I said I chose them for being Android One, not that it wasn't. I didn't know Lens had it, I've never used it. (I have all Google apps disabled.)
- idealmedtech 5y ago
  Native code allows for performance gains that a script (even a performant one) may not be able to match. After all, this is fundamentally an image processing problem, one of the more computationally expensive things a browser has to do.
  - Gigachad 5y ago
    Isn't that the point of webassembly? Just seems like we are still moving in the wrong direction on trying to move the whole of NPM in to the browser web apis, rather than building a solid base you can build anything on.
    - idealmedtech 5y ago
      I'm not a browser developer, but to wager a guess, it probably has something to do with the inertia of existing solutions being native, rather than in WebAssembly, which is still quite new as standards go.
    - michaelt 5y ago
      High performance barcode decoding (e.g. scandit) uses the GPU. After all, a modern cell phone could be producing 4k 30fps video, and high performance barcode decoders will detect linear barcodes with lines a single pixel wide, even in the presence of moderate noise/blur/overexposure (e.g. the barcode on a shiny can of coke) Of course, I've got no idea if browsers will provide high performance barcode reading, or something simpler like zxing. And I've got to say, it's not obvious to me why browsers would implement a barcode scanning API in preference to a more general accelerated image processing API. Plenty of other applications want to do real time video processing in the browser - such as video call background removal.
      - nwsm 5y ago
        Do you have any examples of what a more general image processing API would feature? Everything between reading pixels and reading barcodes is a mystery to me.
- agucova 5y ago
  It's a heavily standardized, very recurrent problem that can take a lot of advantage of native code and platform-specific optimization, so it's really an ideal target for a Web API.
- afavour 5y ago
  Barcode detectors can be pretty big dependencies. Given that this will often just be forwarding directly so some kind of OS capability it makes sense to ship in browser.
- michaelt 5y ago
  > does anyone know why this is ”native” API and not just a generic image processing script Consider the following facts: 1. This was designed by Google 2. Google Play Services provides an API for detecting barcodes, faces and text https://developers.google.com/android/reference/com/google/android/gms/vision/barcode/package-summary https://developers.google.com/android/reference/com/google/a... 3. Chrome gets a shape detection API for detecting barcodes, faces and text https://web.dev/shape-detection/ https://web.dev/shape-detection/ Of course, it's a matter of perspective whether you want to see this as lovers of the free web helping webapps to feature parity with native apps; or Google churning out yet another low-effort standard that's easy for them and difficult for all their competitors.
Eyght 5y ago
I would love to use this for pairing a product with the correct image by decoding barcodes in the image.
jacob019 5y ago
I recently made a webapp for scanning barcodes using the phone camera for warehouse use. I used QuaggaJS. I disabled the barcode locator because it was too heavy, instead I just draw a box on the screen that the barcode is to be positioned in. The barcode detection and decoding happens entirely in the browser and it works very well.
- egfx 5y ago
  I also built a web app for barcode scanning. For testing quaggaJS is fine but it’s highly inconsistent and requires an extreme degree of customization to get adequate results. I recommend scandit. It’s a good production ready library with a very nice web sdk.
  - silviogutierrez 5y ago
    Not OP, but I looked into scandit. While excellent, it's quite expensive right?
    - egfx 5y ago
      You’re right. I talked with the sales team and worked out a deal. They’re interested in user adoption.
    - bonestamp2 5y ago
      Same here. Scandit was the best thing we tested but they quoted us $50k/yr. We don’t have many users that actually need that feature so the next best option was compiling zxing to wasm. That performed a little better than using the js version.
  - WesleyJohnson 5y ago
    My team was trying to get a JS solution working as well, using zxing-js. The barcode basically had to be perfect size, on a flat surface and no glare. we ended up swapping it out for ScandIT's Web SDK and have been very please with performance. As someone else said, it is quite expensive and being a public-facing web application, we had to get a per-scan license vs device license. We're going under contract for 1 year with a "shot in the dark" scan estimate and will renegotiate next year once we have usage statistics. Would be nice to see if this new spec gets full adoption so we can avoid high licensing fees.
djbusby 5y ago
There is a full copy of the zxing library in JS. It's been working great for my purposes - scanning QR and C128 Barcodes. E: https://github.com/zxing-js/library https://github.com/zxing-js/library
- asyncscrum 5y ago
  Used this in a b2b saas for a few years. Worked well and helped remove complexity in a previous way we did it that relied on a native app called pic2shop. We lost a bit of functionality specifically a clear target zone, green scanning line and better low light support. It made up for it in ease of support and reduction in perceived jank.
- bob1029 5y ago
  This is what we use as a fallback if the user doesnt have a handheld 2D scanner. Very fast & accurate in most of my testing.
- crossroadsguy 5y ago
  ZXing is/was solid. I used it for Android, then moved to (suggested) Google’s ml-kit offering around a year ago. ZXing is pretty much abandoned now. It was anyway “kind of” from Google.
gruez 5y ago
>Secure context: This feature is available only in secure contexts (HTTPS), in some or all supporting browsers. What's the point of this? I understand why you might want this for APIs that expose private user information, but this just performs some computation. You could do the same thing with a js library or webassembly. Is this just to punish http sites?
- dblohm7 5y ago
  Basically, yes. Browser vendors have had an agreement for some time now to only expose new APIs in secure contexts in order to encourage HTTPS adoption.
  - wyager 5y ago
    I'm not sure I feel great about gating feature A behind completely unrelated feature B just because manufacturers like feature B. Even if feature B is concretely pretty good (which is debatable for https as it exists now), this seems bad on principle.
    - bhawks 5y ago
      The browser security model is trivially defeated without the usage of https. IT is not an orthogonal or unrelated feature. Making all new API development available only when content is delivered via a mechanism that provides the foundation for the rest of web security is good engineering practice.
    - judge2020 5y ago
      If it helps, it really doesn’t care what certificate the page it signed with, whether that be localhost, Mozilla CA participants, or maybe even a future CA fork for Russia if that happens. Plaintext HTTP is just not advisable even on a private network when some inline network device like LAN turtle could be passively exfiltrating traffic.
    - jorl17 5y ago
      I'm interested: why do you think it is debatable if HTTPS is "concretely pretty good" as it exists now?
matthewmacleod 5y ago
Huh, TIL. Recognition performance is pretty modest (on Chrome/Mac at least) but it's fast and probably more than adequate for most close-up mobile device scans. It basically just wraps MacOS's Vision framework on the Mac, and ML Kit on Android.
sleepy_keita 5y ago
I bought my first Android device to try this feature out. It's so much faster than zxing on iOS.
adamrezich 5y ago
all of these small features seem good in aggregate but each one makes it that much harder to ever build a competing browser, especially from scratch.
veysiertekin 5y ago
A different context but related: a website plugin that I have built years ago for barcode scanning machines: JBarcode - https://github.com/veysiertekin/jbarcode https://github.com/veysiertekin/jbarcode I didn't made any changes since then. But it can be easily ported into a more modern framework (React.js etc)
bricss 5y ago
In the latest Chrome: > Uncaught ReferenceError: BarcodeDetector is not defined
xg15 5y ago
It's a nice feature, but, honest question, why is this a web API? I don't see anything this has to do with browsers. This could be implemented perfectly well in a library.
ape4 5y ago
Even thought its on mozilla.org is not supported by Firefox according to the matrix at the bottom.
- happytoexplain 5y ago
  These docs are a general web developer resource that Mozilla maintains - they are not specific to Mozilla products.
- hunterb123 5y ago
  You'll find many features in MDN docs that applies to. That's mainly because of the expansiveness of MDN docs, but somewhat due to the funding distribution of Mozilla going to officers, not Firefox devs.
- agucova 5y ago
  MDN is meant to be a somewhat universal resource for web development. I've even seen official Google web dev resources pointing to MDN.
- dblohm7 5y ago
  And if you look at the badges in the matrix, the API is not a standard yet, either.
PaulHoule 5y ago
... it decodes the barcode too?
- soylentgraham 5y ago
  It does
- happytoexplain 5y ago
  According to this documentation, yes, it returns the raw value encoded in the barcode. What it does not appear to do is further decode the raw value into logical data. E.g. data matrices commonly contain GS1 element strings that are not trivial to decode. There are libraries for that, though.
  - joshua-at-banxa 5y ago
    Arguably, as a Barcode Detection API that's all it should do.
    - happytoexplain 5y ago
      I think I agree, I'm just being explicit.
amelius 5y ago
I'm still looking for an open source python library that can decode Data-Matrix barcodes as seen on Digikey bags. Should I give up and switch to JavaScript+browser?
- michaelt 5y ago
  Is pylibdmtx not up to it?
  - marcjuul 5y ago
    It's not great but libdtmx is the best open source datamatrix decoding as far as I know.
    - alex_suzuki 5y ago
      libdtmx is great, but it's really slow. Can't do real-time detection (as in scanning from a continuous camera feed) with it.
- pabs3 5y ago
  zxing-cpp has a Python wrapper that should work for that.
  - marcjuul 5y ago
    zxing does not have a usable datamatrix decoder last I checked. it's there but it's no good.
    - pabs3 5y ago
      I just tried the example zxing-cpp Python module wrapper script and it worked with datamatrix.png from the pylibdmtx test suite. Hmm, but read_datamatrix from pylibdmtx detects two values while zxing-cpp detects one.
- schappim 5y ago
  For Digikey bags (we make eletronics) we use SwipeTrack app + JS.
timando 5y ago
I wanted this to be a barcode scanning API where a web app can use the camera to scan a barcode without being able to see anything else in the image.
bob1029 5y ago
This stuff has been a nightmare for us for a long time. We have to scan pdf417 barcodes from physical identification as part of our offering. Recently, we stumbled upon some cheap USB keyboard emulator 2d scanners that can pick these up very reliably. For our product/customers, this is a viable path. We have tested probably 30 different webcams by this point. The only 2 that are as reliable as the handheld CCD scanner are the iPad and my Canon DSLR tethered to my PC in automatic mode with a macro lens. Resolution is a factor, but so is frame rate and environment. You have to consider the full UX stack. Someone is required to physically position the barcode in a particular way with certain lighting constraints, etc. With a handheld CCD scanner, you get a laser guide for precisely what distance to go for, and an ergonomic experience that is much more conducive to successful scans. Most offer on board illumination, so even in a pitch black room you will get a successful scan.
- unqueued 5y ago
  I found the most reliable barcode scanner to be an old Symbol usb scanner. I also found that I could scan barcodes off of a screen with a laser scanner if I put a translucent white plastic bag over the screen. Maybe someone with more knowledge about optics could give some insight into why that trick works. My understanding is that laser scanners should only be able to scan by reflection, and shouldn't work off of a screen at all.
  - masklinn 5y ago
    Note that they’re scanning 2D “pdf417” barcodes. Reading a linear barcode is quite easy, the shitty webcam of my 2010 macbook pro was enough for Delicious Library to pick most every article instantly, usually faster than I could even present the article.
- danachow 5y ago
  Purpose built barcode scanners typically don’t fumble with autofocus have faster 0-latency autoexposure (adjusted during frame readout) and even the good 2D CMOS sensor ones will start decoding linear or stacked codes such as PDF417 while the frame is being transferred. That can’t be beaten for latency.
pabs3 5y ago
Anyone know what the backend for this is? Is it zxing-cpp or similar? Edit: guess it depends on the browser and OS.
cyode 5y ago
Bit off-topic: can anyone recommend resources (videos/papers/tutorials) that could help me learn how QR barcode detection and scanning itself is implemented? I've been thinking that building a basic scanner from scratch (i.e. only using vanilla cpp/python/whatever + image processing libraries, but nothing barcode-specific) could teach me a lot.
- lgvld 5y ago
  I guess it helps to understand how to generate a QR code: https://www.nayuki.io/page/qr-code-generator-library https://www.nayuki.io/page/qr-code-generator-library (see also the references at the end) Good luck. ;-)
marcjuul 5y ago
Does anyone happen to know where in the Chromium code base the implementation is hiding? The android version seems to use a library that's part of Google Play Services: /** * Implementation of mojo BarcodeDetection, using Google Play Services vision package. */ and // The vision library will be downloaded the first time the API is used // on the device; this happens "fast", but it might have not completed, // bail in this case. Also, the API was disabled between and v.9.0 and // v.9.2, see https://developers.google.com/android/guides/releases. From: https://github.com/chromium/chromium/blob/c4d3c31083a2e1481253ff2d24298a1dfe19c754/services/shape_detection/android/java/src/org/chromium/shape_detection/BarcodeDetectionImpl.java https://github.com/chromium/chromium/blob/c4d3c31083a2e14812... The desktop version references a third party library called "barhopper" here: https://github.com/chromium/chromium/blob/e1e495b29e1178a451f65980a6c4ae017c34dc94/services/shape_detection/barcode_detection_impl_barhopper.cc https://github.com/chromium/chromium/blob/e1e495b29e1178a451... and barhopper seems to come from a non-public Google repo: 'src/third_party/barhopper': { 'url': 'https://chrome-internal.googlesource.com/chrome/deps/barhopper.git' + '@' + 'ad3c4382875afdd0340f1549f8b9c93cbbc16e37', 'condition': 'checkout_src_internal and checkout_chromeos', }, from https://source.chromium.org/chromium/chromium/src/+/master:DEPS https://source.chromium.org/chromium/chromium/src/+/master:D... I didn't realize that Chromium had missing APIs compared to Chrome but maybe that's always been the case? This appears not to be a W3C standard: https://wicg.github.io/shape-detection-api/ https://wicg.github.io/shape-detection-api/ I was interested in this because there are no really good open source DataMatrix decoders (to my knowledge) with the one in ZXing being basically unusable (last I checked) and libdtmx being ok but not comparable to commercial offerings.
- sevenf0ur 5y ago
  Yeah, I haven't been able to find any decent open source DataMatrix libraries as well. If you find one I'd love to know. This feature uses native libraries and only works on Chrome OS, Android, and MacOS currently according to this page: https://chromestatus.com/feature/4757990523535360 https://chromestatus.com/feature/4757990523535360
noodlesUK 5y ago
I’d love to see this in safari on iOS as well. The native barcode detection capabilities in iOS are substantially better than libraries like zximg, and I like barcodes as an interface between the real world and tech. I also wish text recognition were in safari.
schappim 5y ago
Here is an example of using the Barcode Detection API w/ WebRTC on Android / Chrome: https://files.littlebird.com.au/barcode20.html https://files.littlebird.com.au/barcode20.html
derac 5y ago
I forked another person's code and added bounding boxes, if you want to check it out. Seems like it works pretty well, though the bounding boxes are erratic. https://gist.github.com/derac/9dfa8884bb04df59ac498d3bac4a93da https://gist.github.com/derac/9dfa8884bb04df59ac498d3bac4a93... https://htmlpreview.github.io/?https://gist.githubusercontent.com/derac/9dfa8884bb04df59ac498d3bac4a93da/raw/25f660ba8486774a8ac281f8442e8b58b1afbaaf/index.html https://htmlpreview.github.io/?https://gist.githubuserconten...
paxys 5y ago
The dream of the early 2000s is finally being realized.
duck-valentine 5y ago
There's a couple good datasets with trained models on Roboflow Universe that would pair nicely for a Computer Vision project/app: Object Detection for QR Code and Bar-Codes - https://universe.roboflow.com/new-workspace-mrc2b/optiscan-lwrhq/overview https://universe.roboflow.com/new-workspace-mrc2b/optiscan-l... another one for just QR Codes - https://universe.roboflow.com/lihang-xu/qr-code-oerhe https://universe.roboflow.com/lihang-xu/qr-code-oerhe
numlock86 5y ago
Cool. Another feature that Firefox won't support. Same with the WebSerial API and others. I sort of understand their reasoning that they want to make a web browser and not an application platform, but that's where the web is shifting to ... or rather has already for a long time. And people wonder why every new tech/browser is based on Chromium all the time. Well, this is one of the many reasons.
- schwartzworld 5y ago
  Why would Firefox implement non-standard APIs?
  - numlock86 5y ago
    I get what you are trying to bring up as an argument, but even Firefox doesn't argue this way. See their answers on issues related to these kind of APIs.
Scoundreller 5y ago
I’m happy to see things like this. My ISP used to ask for a picture of my docsis modem sticker because people kept making mistakes entering in the MAC address. Unfortunately, the csr was just keying them in and keyed mine in wrong so I was without internet for a few days. Something to automatically pick it out would be a lot better.

331 points by addcninblue 5y ago

As the page does not discuss the motivation behind the API, does anyone know why this is ”native” API and not just a generic image processing script implemented with JavaScript / WebGL / etc.?

bena 5y ago
Because while barcode scanning is a subset of image processing, it's a use case that's pretty common.
rahimnathwani 5y ago
To allow the implementation to take advantage of libraries provided by the operating system[0] (where available), without the web site developer needing to even know that it's happening. [0] or things like Google Play Services.
PaulHoule 5y ago
At this point in time it's a pain point for me that Android devices don't usually have built-in QR reader software. I can tell iOS users to just "scan the code with the Camera" app but I have to tell Android users to find a QR scanner app on the app store. My experience with an actual bottom-of-the-line prepaid phone was that the first QR scanner app I downloaded from Google Play worked right the first time but I had to try six different apps before I found one that worked with my Samsung Tablet. A web-based scanner would be a possible answer, though really Google and the phone makers and the carriers should have prioritized a "just works" solution years ago.
- eertami 5y ago
  Honest question: for what reasons do you ever need to scan a random QR code? I think I've used them to sign in to some apps (eg, WhatsApp) while logged in on a PC, and other than that exactly 0 times. The Google camera app does support QR codes FWIW, but I don't know if any other vendors except Google use it.
  - simonbw 5y ago
    In general, assuming you have an easy-to-use QR scanner, it's easier to scan a QR code than type in a URL on a phone, especially a complicated one. A couple examples I've run into recently: - Lately it's been pretty common for restaurants to replace their physical menus with a QR code on the table that links to their online menu. - When I've taken COVID tests recently, the site gives me a sticker with a QR code that links to my results page. - I encoded my wifi network/password into a QR code and printed it out, so guests can just scan it to join, rather than have to scroll and find the right network and type in a password.
    - PaulHoule 5y ago
      Displaying a QR code on a screen is a quick way to beam a URL to a phone or tablet either from a PC or another mobile. You can put up a QR code to point people to the web app which controls the IoT functions in a space, music system, etc.
  - deleted 5y ago
    [deleted]
  - PaulHoule 5y ago
    I print ‘three-sided cards’ that have a photo or art reproduction on one side, some documentation on the back, and more information if you scan the QR code. One series of the cards are ‘music cards’ that will play a song, another application is ‘constellations’ that let me stick anywhere from 3 to 40 cards on the wall (so far) and you can see the back sides and other narration by scanning a code. So my use cases are (1) I am demoing the cards for people, and (2) I give the cards away for free (If it wasn’t for the supply chain crisis I’d be creating a new card design every day) or put up an installation somewhere and people interact with them. Either way I want it to be very easy for people.
  - LouisSayers 5y ago
    I literally just scanned a couple while at the doctors - one to check in, and another provided a url of a form to fill in. They're also in some restaurants where you can get to a menu to order from your phone. You also scan QR codes for use in two factor authentication apps. Covid has normalised QR code scanning in Australia - we have to do it at certain venues in order to show our vaccine passes.
- Bilal_io 5y ago
  I've been scanning QR codes with my Android phone for a long time. I know at least two native cameras work, Pixel's and Samsung's since 2019. But I am sure legacy phones with older android versions don't support it natively.
  - fy20 5y ago
    I have a flagship Xiaomi phone and the built in QR code scanner is garbage. I went to a restaurant with friends which only had a QR code for the menu (which turned out just to be a link to the root of their website - why not write the URL too?). In the end I had to take a picture of the QR code, then crop it, then open that in the scanner app.
  - calmoo 5y ago
    My Huawei P20 doesn't have QR scanning native to the camera app.
  - Lev1a 5y ago
    > I know at least two native cameras work, Pixel's and Samsung's since 2019 My Motorola One can do that as well and IIRC the "moto g4" I had before that too.
    - WesternWind 5y ago
      I've had intermittent issues with my motorola's camera app. Sometimes it picks up qr codes at restaurants, sometimes it refuses, maybe because of lighting issues, though turning on the flash doesn't help. I ended up downloading BinaryEye from F-Droid and I just default to that now.
- RenThraysk 5y ago
  It's built into Google Lens. Use it occasionally to send files over wifi to phone, with webwormhole.
- SkyPuncher 5y ago
  > At this point in time it's a pain point for me that Android devices don't usually have built-in QR reader software It's hidden, but I believe it's built in the default camera app now. I've learned that my last 3 phones have had it built in.
- easrng 5y ago
  My Motorola phone from 2017 has QR support in the Camera app.
- OJFord 5y ago
  As always, some fragment of Android actually got there before Apple (to put it in the Camera app), but no, it's not standard. My current phone (Nokia 3.4) doesn't have it, but previous one (Motorala One) did. Ironic, since I chose them for being about as close to stock as you can get, both in the 'Android One' programme (so also comparatively LTS).
  - olvrng 5y ago
    It's always included in Google Lens (and Nokia 3.4 uses Android One). You can open Google Lens and point to the QR code.
    - OJFord 5y ago
      Yes I said I chose them for being Android One, not that it wasn't. I didn't know Lens had it, I've never used it. (I have all Google apps disabled.)
idealmedtech 5y ago
Native code allows for performance gains that a script (even a performant one) may not be able to match. After all, this is fundamentally an image processing problem, one of the more computationally expensive things a browser has to do.
- Gigachad 5y ago
  Isn't that the point of webassembly? Just seems like we are still moving in the wrong direction on trying to move the whole of NPM in to the browser web apis, rather than building a solid base you can build anything on.
  - idealmedtech 5y ago
    I'm not a browser developer, but to wager a guess, it probably has something to do with the inertia of existing solutions being native, rather than in WebAssembly, which is still quite new as standards go.
  - michaelt 5y ago
    High performance barcode decoding (e.g. scandit) uses the GPU. After all, a modern cell phone could be producing 4k 30fps video, and high performance barcode decoders will detect linear barcodes with lines a single pixel wide, even in the presence of moderate noise/blur/overexposure (e.g. the barcode on a shiny can of coke) Of course, I've got no idea if browsers will provide high performance barcode reading, or something simpler like zxing. And I've got to say, it's not obvious to me why browsers would implement a barcode scanning API in preference to a more general accelerated image processing API. Plenty of other applications want to do real time video processing in the browser - such as video call background removal.
    - nwsm 5y ago
      Do you have any examples of what a more general image processing API would feature? Everything between reading pixels and reading barcodes is a mystery to me.
agucova 5y ago
It's a heavily standardized, very recurrent problem that can take a lot of advantage of native code and platform-specific optimization, so it's really an ideal target for a Web API.
afavour 5y ago
Barcode detectors can be pretty big dependencies. Given that this will often just be forwarding directly so some kind of OS capability it makes sense to ship in browser.
michaelt 5y ago
> does anyone know why this is ”native” API and not just a generic image processing script Consider the following facts: 1. This was designed by Google 2. Google Play Services provides an API for detecting barcodes, faces and text https://developers.google.com/android/reference/com/google/android/gms/vision/barcode/package-summary https://developers.google.com/android/reference/com/google/a... 3. Chrome gets a shape detection API for detecting barcodes, faces and text https://web.dev/shape-detection/ https://web.dev/shape-detection/ Of course, it's a matter of perspective whether you want to see this as lovers of the free web helping webapps to feature parity with native apps; or Google churning out yet another low-effort standard that's easy for them and difficult for all their competitors.

Eyght 5y ago

I would love to use this for pairing a product with the correct image by decoding barcodes in the image.

jacob019 5y ago

I recently made a webapp for scanning barcodes using the phone camera for warehouse use. I used QuaggaJS. I disabled the barcode locator because it was too heavy, instead I just draw a box on the screen that the barcode is to be positioned in. The barcode detection and decoding happens entirely in the browser and it works very well.

egfx 5y ago
I also built a web app for barcode scanning. For testing quaggaJS is fine but it’s highly inconsistent and requires an extreme degree of customization to get adequate results. I recommend scandit. It’s a good production ready library with a very nice web sdk.
- silviogutierrez 5y ago
  Not OP, but I looked into scandit. While excellent, it's quite expensive right?
  - egfx 5y ago
    You’re right. I talked with the sales team and worked out a deal. They’re interested in user adoption.
  - bonestamp2 5y ago
    Same here. Scandit was the best thing we tested but they quoted us $50k/yr. We don’t have many users that actually need that feature so the next best option was compiling zxing to wasm. That performed a little better than using the js version.
- WesleyJohnson 5y ago
  My team was trying to get a JS solution working as well, using zxing-js. The barcode basically had to be perfect size, on a flat surface and no glare. we ended up swapping it out for ScandIT's Web SDK and have been very please with performance. As someone else said, it is quite expensive and being a public-facing web application, we had to get a per-scan license vs device license. We're going under contract for 1 year with a "shot in the dark" scan estimate and will renegotiate next year once we have usage statistics. Would be nice to see if this new spec gets full adoption so we can avoid high licensing fees.

djbusby 5y ago

There is a full copy of the zxing library in JS. It's been working great for my purposes - scanning QR and C128 Barcodes. E: https://github.com/zxing-js/library https://github.com/zxing-js/library

asyncscrum 5y ago
Used this in a b2b saas for a few years. Worked well and helped remove complexity in a previous way we did it that relied on a native app called pic2shop. We lost a bit of functionality specifically a clear target zone, green scanning line and better low light support. It made up for it in ease of support and reduction in perceived jank.
bob1029 5y ago
This is what we use as a fallback if the user doesnt have a handheld 2D scanner. Very fast & accurate in most of my testing.
crossroadsguy 5y ago
ZXing is/was solid. I used it for Android, then moved to (suggested) Google’s ml-kit offering around a year ago. ZXing is pretty much abandoned now. It was anyway “kind of” from Google.

gruez 5y ago

>Secure context: This feature is available only in secure contexts (HTTPS), in some or all supporting browsers. What's the point of this? I understand why you might want this for APIs that expose private user information, but this just performs some computation. You could do the same thing with a js library or webassembly. Is this just to punish http sites?

dblohm7 5y ago
Basically, yes. Browser vendors have had an agreement for some time now to only expose new APIs in secure contexts in order to encourage HTTPS adoption.
- wyager 5y ago
  I'm not sure I feel great about gating feature A behind completely unrelated feature B just because manufacturers like feature B. Even if feature B is concretely pretty good (which is debatable for https as it exists now), this seems bad on principle.
  - bhawks 5y ago
    The browser security model is trivially defeated without the usage of https. IT is not an orthogonal or unrelated feature. Making all new API development available only when content is delivered via a mechanism that provides the foundation for the rest of web security is good engineering practice.
  - judge2020 5y ago
    If it helps, it really doesn’t care what certificate the page it signed with, whether that be localhost, Mozilla CA participants, or maybe even a future CA fork for Russia if that happens. Plaintext HTTP is just not advisable even on a private network when some inline network device like LAN turtle could be passively exfiltrating traffic.
  - jorl17 5y ago
    I'm interested: why do you think it is debatable if HTTPS is "concretely pretty good" as it exists now?

matthewmacleod 5y ago

Huh, TIL. Recognition performance is pretty modest (on Chrome/Mac at least) but it's fast and probably more than adequate for most close-up mobile device scans. It basically just wraps MacOS's Vision framework on the Mac, and ML Kit on Android.

sleepy_keita 5y ago

I bought my first Android device to try this feature out. It's so much faster than zxing on iOS.

adamrezich 5y ago

all of these small features seem good in aggregate but each one makes it that much harder to ever build a competing browser, especially from scratch.

veysiertekin 5y ago

A different context but related: a website plugin that I have built years ago for barcode scanning machines: JBarcode - https://github.com/veysiertekin/jbarcode https://github.com/veysiertekin/jbarcode I didn't made any changes since then. But it can be easily ported into a more modern framework (React.js etc)

bricss 5y ago

In the latest Chrome: > Uncaught ReferenceError: BarcodeDetector is not defined

xg15 5y ago

It's a nice feature, but, honest question, why is this a web API? I don't see anything this has to do with browsers. This could be implemented perfectly well in a library.

ape4 5y ago

Even thought its on mozilla.org is not supported by Firefox according to the matrix at the bottom.

happytoexplain 5y ago
These docs are a general web developer resource that Mozilla maintains - they are not specific to Mozilla products.
hunterb123 5y ago
You'll find many features in MDN docs that applies to. That's mainly because of the expansiveness of MDN docs, but somewhat due to the funding distribution of Mozilla going to officers, not Firefox devs.
agucova 5y ago
MDN is meant to be a somewhat universal resource for web development. I've even seen official Google web dev resources pointing to MDN.
dblohm7 5y ago
And if you look at the badges in the matrix, the API is not a standard yet, either.

PaulHoule 5y ago

... it decodes the barcode too?

soylentgraham 5y ago
It does
happytoexplain 5y ago
According to this documentation, yes, it returns the raw value encoded in the barcode. What it does not appear to do is further decode the raw value into logical data. E.g. data matrices commonly contain GS1 element strings that are not trivial to decode. There are libraries for that, though.
- joshua-at-banxa 5y ago
  Arguably, as a Barcode Detection API that's all it should do.
  - happytoexplain 5y ago
    I think I agree, I'm just being explicit.

amelius 5y ago

I'm still looking for an open source python library that can decode Data-Matrix barcodes as seen on Digikey bags. Should I give up and switch to JavaScript+browser?

michaelt 5y ago
Is pylibdmtx not up to it?
- marcjuul 5y ago
  It's not great but libdtmx is the best open source datamatrix decoding as far as I know.
  - alex_suzuki 5y ago
    libdtmx is great, but it's really slow. Can't do real-time detection (as in scanning from a continuous camera feed) with it.
pabs3 5y ago
zxing-cpp has a Python wrapper that should work for that.
- marcjuul 5y ago
  zxing does not have a usable datamatrix decoder last I checked. it's there but it's no good.
  - pabs3 5y ago
    I just tried the example zxing-cpp Python module wrapper script and it worked with datamatrix.png from the pylibdmtx test suite. Hmm, but read_datamatrix from pylibdmtx detects two values while zxing-cpp detects one.
schappim 5y ago
For Digikey bags (we make eletronics) we use SwipeTrack app + JS.

timando 5y ago

I wanted this to be a barcode scanning API where a web app can use the camera to scan a barcode without being able to see anything else in the image.

bob1029 5y ago

This stuff has been a nightmare for us for a long time. We have to scan pdf417 barcodes from physical identification as part of our offering. Recently, we stumbled upon some cheap USB keyboard emulator 2d scanners that can pick these up very reliably. For our product/customers, this is a viable path. We have tested probably 30 different webcams by this point. The only 2 that are as reliable as the handheld CCD scanner are the iPad and my Canon DSLR tethered to my PC in automatic mode with a macro lens. Resolution is a factor, but so is frame rate and environment. You have to consider the full UX stack. Someone is required to physically position the barcode in a particular way with certain lighting constraints, etc. With a handheld CCD scanner, you get a laser guide for precisely what distance to go for, and an ergonomic experience that is much more conducive to successful scans. Most offer on board illumination, so even in a pitch black room you will get a successful scan.

unqueued 5y ago
I found the most reliable barcode scanner to be an old Symbol usb scanner. I also found that I could scan barcodes off of a screen with a laser scanner if I put a translucent white plastic bag over the screen. Maybe someone with more knowledge about optics could give some insight into why that trick works. My understanding is that laser scanners should only be able to scan by reflection, and shouldn't work off of a screen at all.
- masklinn 5y ago
  Note that they’re scanning 2D “pdf417” barcodes. Reading a linear barcode is quite easy, the shitty webcam of my 2010 macbook pro was enough for Delicious Library to pick most every article instantly, usually faster than I could even present the article.
danachow 5y ago
Purpose built barcode scanners typically don’t fumble with autofocus have faster 0-latency autoexposure (adjusted during frame readout) and even the good 2D CMOS sensor ones will start decoding linear or stacked codes such as PDF417 while the frame is being transferred. That can’t be beaten for latency.

pabs3 5y ago

Anyone know what the backend for this is? Is it zxing-cpp or similar? Edit: guess it depends on the browser and OS.

cyode 5y ago

Bit off-topic: can anyone recommend resources (videos/papers/tutorials) that could help me learn how QR barcode detection and scanning itself is implemented? I've been thinking that building a basic scanner from scratch (i.e. only using vanilla cpp/python/whatever + image processing libraries, but nothing barcode-specific) could teach me a lot.

lgvld 5y ago
I guess it helps to understand how to generate a QR code: https://www.nayuki.io/page/qr-code-generator-library https://www.nayuki.io/page/qr-code-generator-library (see also the references at the end) Good luck. ;-)

marcjuul 5y ago

Does anyone happen to know where in the Chromium code base the implementation is hiding? The android version seems to use a library that's part of Google Play Services: /** * Implementation of mojo BarcodeDetection, using Google Play Services vision package. */ and // The vision library will be downloaded the first time the API is used // on the device; this happens "fast", but it might have not completed, // bail in this case. Also, the API was disabled between and v.9.0 and // v.9.2, see https://developers.google.com/android/guides/releases. From: https://github.com/chromium/chromium/blob/c4d3c31083a2e1481253ff2d24298a1dfe19c754/services/shape_detection/android/java/src/org/chromium/shape_detection/BarcodeDetectionImpl.java https://github.com/chromium/chromium/blob/c4d3c31083a2e14812... The desktop version references a third party library called "barhopper" here: https://github.com/chromium/chromium/blob/e1e495b29e1178a451f65980a6c4ae017c34dc94/services/shape_detection/barcode_detection_impl_barhopper.cc https://github.com/chromium/chromium/blob/e1e495b29e1178a451... and barhopper seems to come from a non-public Google repo: 'src/third_party/barhopper': { 'url': 'https://chrome-internal.googlesource.com/chrome/deps/barhopper.git' + '@' + 'ad3c4382875afdd0340f1549f8b9c93cbbc16e37', 'condition': 'checkout_src_internal and checkout_chromeos', }, from https://source.chromium.org/chromium/chromium/src/+/master:DEPS https://source.chromium.org/chromium/chromium/src/+/master:D... I didn't realize that Chromium had missing APIs compared to Chrome but maybe that's always been the case? This appears not to be a W3C standard: https://wicg.github.io/shape-detection-api/ https://wicg.github.io/shape-detection-api/ I was interested in this because there are no really good open source DataMatrix decoders (to my knowledge) with the one in ZXing being basically unusable (last I checked) and libdtmx being ok but not comparable to commercial offerings.

sevenf0ur 5y ago
Yeah, I haven't been able to find any decent open source DataMatrix libraries as well. If you find one I'd love to know. This feature uses native libraries and only works on Chrome OS, Android, and MacOS currently according to this page: https://chromestatus.com/feature/4757990523535360 https://chromestatus.com/feature/4757990523535360

noodlesUK 5y ago

I’d love to see this in safari on iOS as well. The native barcode detection capabilities in iOS are substantially better than libraries like zximg, and I like barcodes as an interface between the real world and tech. I also wish text recognition were in safari.

schappim 5y ago

Here is an example of using the Barcode Detection API w/ WebRTC on Android / Chrome: https://files.littlebird.com.au/barcode20.html https://files.littlebird.com.au/barcode20.html

derac 5y ago

I forked another person's code and added bounding boxes, if you want to check it out. Seems like it works pretty well, though the bounding boxes are erratic. https://gist.github.com/derac/9dfa8884bb04df59ac498d3bac4a93da https://gist.github.com/derac/9dfa8884bb04df59ac498d3bac4a93... https://htmlpreview.github.io/?https://gist.githubusercontent.com/derac/9dfa8884bb04df59ac498d3bac4a93da/raw/25f660ba8486774a8ac281f8442e8b58b1afbaaf/index.html https://htmlpreview.github.io/?https://gist.githubuserconten...

paxys 5y ago

The dream of the early 2000s is finally being realized.

duck-valentine 5y ago

There's a couple good datasets with trained models on Roboflow Universe that would pair nicely for a Computer Vision project/app: Object Detection for QR Code and Bar-Codes - https://universe.roboflow.com/new-workspace-mrc2b/optiscan-lwrhq/overview https://universe.roboflow.com/new-workspace-mrc2b/optiscan-l... another one for just QR Codes - https://universe.roboflow.com/lihang-xu/qr-code-oerhe https://universe.roboflow.com/lihang-xu/qr-code-oerhe

numlock86 5y ago

Cool. Another feature that Firefox won't support. Same with the WebSerial API and others. I sort of understand their reasoning that they want to make a web browser and not an application platform, but that's where the web is shifting to ... or rather has already for a long time. And people wonder why every new tech/browser is based on Chromium all the time. Well, this is one of the many reasons.

schwartzworld 5y ago
Why would Firefox implement non-standard APIs?
- numlock86 5y ago
  I get what you are trying to bring up as an argument, but even Firefox doesn't argue this way. See their answers on issues related to these kind of APIs.

Scoundreller 5y ago

I’m happy to see things like this. My ISP used to ask for a picture of my docsis modem sticker because people kept making mistakes entering in the MAC address. Unfortunately, the csr was just keying them in and keyed mine in wrong so I was without internet for a few days. Something to automatically pick it out would be a lot better.