Computer Vision

Live scanning QR codes in iOS using Vision framework

4 min readMay 3, 2021

Are you looking to make your physical business contactless? COVID-19 has transformed how business with physical presence need to interact with their customers. Many customers now expect the ability to perform transaction on their mobile devices. Restaurants in the UK only allow to view their menu through your mobile phone by scanning a QR code. Some even allow you to pay the bill be scanning the QR code on your table through their app. QR codes were already rising in popularity however now they have become the standard for businesses to interact with customers and augment their experience.

In this post I will show you how to detect and extract information from a QR code using Apple’s already included Vision framework. For such we will make use of the iOS device back camera. We’ll receive each camera frame and analyse it using Vision framework for QR codes.

In this post I won’t delve into how to connect to the camera, stream the frames from it and then process it. We’ll start with a starter project which already does this for us. This post focuses on searching and extracting barcodes.

I have used Swift 5.3.2 and Xcode 12.4 for this article.

Getting started

In this section we’ll get started by downloading a starter pack. The starter pack includes an Xcode project with an iOS app which connects to the back camera and streams the frames to the view controller.

Then we’ll analyse each frame and search for QR codes and log its content to console.

Here is a summary of the steps we are going to take:

Download the starter project
Extracting QR codes using Vision framework

Let’s do it!

1. Download the starter project

Let’s start by retrieving the starter project. For such we’ll make use if the terminal. Open the terminal app and execute the following commands:

cd $HOME
curl https://github.com/anuragajwani/vision_qr_codes_scanning/archive/starter.zip -o qr_codes_scanning.zip -L -s
unzip -q qr_codes_scanning.zip
cd vision_qr_codes_scanning-starter/

Let’s open the project. Execute the following command:

open -a Xcode qr_codes_scanning.xcodeproj

2. Extracting QR codes using Vision framework

Next we’ll use Vision framework to analyse the frame. Let’s add a function to ViewController.swift to do so:

private func extractQRCode(fromFrame frame: CVImageBuffer) -> String? {
}

In this function we’ll take each frame and then output the contents of the QR code within the frame if present.

Note for this post I am assuming there will be only one QR code within the frame.

Next let’s fill in the function body. In order to extract QR codes in Vision framework we need to make use of VNDetectBarcodesRequest. However before we can use Vision we must import it in our view controller. Just below import AVFoundation add the following line:

import Vision

Next let’s create a new instance of barcode request within the extractQRCode function body:

let barcodeRequest = VNDetectBarcodesRequest()
barcodeRequest.symbologies = [.QR]

Next let’s execute to detect (and extract) the QR barcode request. For such we’ll make use of VNSequenceRequestHandler. First let’s create a property to hold an instance of the sequence handler within the ViewController class:

private let sequenceHandler = VNSequenceRequestHandler()

Then let’s execute the detect barcode request at the end of our extractQRCode function body:

try? self.sequenceHandler.perform([barcodeRequest], on: frame)

There are 2 ways to execute a Vision request:

What’s the difference? From the docs:

Unlike the VNImageRequestHandler, you don’t specify the image on creation. Instead, you supply each image frame one by one as you continue to call one of the perform methods.

However digging through the docs you can find an example on how to track objects using Vision framework which states:

Whereas the VNImageRequestHandler handles object detection requests on a still image, the VNSequenceRequestHandler handles tracking requests.

For this post we’ll stick VNSequenceRequestHandler.

Next let’s get the results from barcode request after performing image analysis. Add the following lines at the end of extractQRCode function:

guard let results = barcodeRequest.results as? [VNBarcodeObservation], let firstBarcode = results.first?.payloadStringValue else {
    return nil
}
return firstBarcode

Finally let’s call the function from captureOutput replace print(“did receive image frame”) with the following:

if let barcode = self.extractQRCode(fromFrame: frame) {
    print("did extract barcode \(barcode)")
}

That’s it! Run the app and scan a QR code. Here is one for convenience:

Summary

In this post we have learnt:

how to scan QR codes using Apple’s Vision framework in iOS

Final thoughts

QR codes are becoming more and more popular over the years. It was already quite popular in many places. However Apple made QR code scanning default behaviour within the stock Camera app since iOS 11 which made it even more popular than it previously was. Thus user and consumers are becoming habitual to these interactions to connect software and the physical world which makes this knowledge a good tool for your iOS problem solving belt.

Vision framework is not limited to QR codes. You can also scan PDF417, EAN13 and many more. For the full list check out the documentation. Furthermore you can scan faces, rectangles and more.

Stay tuned for more posts on iOS development! Follow me on Twitter or Medium!