How to avoid a distorted Android camera preview with ZXing.Net.Mobile [Updated]

QR code scanning is a common task a lot of apps are using lately. With ZXing.Net.Mobile, we have a library at hand that makes this pretty easy in our Xamarin applications. This post will show you how to avoid a distorted camera preview when using the library on Android and explains why it might happen.

Update: The application I used to determine this blog post is portrait only, that’s why I totally missed the landscape part. I have to thank Timo Partl, who pointed me to that fact on Twitter. I updated the solution/workaround part to reflect the orientation as well.


If you need to implement QR-scanning into your Xamarin (Forms) app, chances are high you will be using the ZXing.Net.Mobile library (as it is the most complete solution out there). In one of my recent projects, I wanted to exactly that. I have used the library already before and thought it might be an easy play.

Distorted reality…

Reality hit me hard when I realized that the preview is totally distorted:

distorted camera preview

Even with that distortion, the library detects the QR code without problems. However, the distortion is not something you want your user to experience. So I started to investigate why this distortion happens. As I am not the only one experiencing this problem, I am showing to easily fix that issue and the way that led to the solution (knowing also some beginners follow my blog).

Searching the web …

.. often brings you closer to the solution. Most of the time, you are not the only one that runs into such a problem. Doing so let me find the Github issue above, showing my theory is correct. Sometimes, those Github issues provide a solution  – this time, not. After not being able to find anything helpful, I decided to fork the Github repo and downloaded it into Visual Studio.

Debugging

Once the solution was loaded in Visual Studio, I found that there are some samples in the repo that made debugging easy for me. I found the case that matches my implementation best (using the ZXingScannerView). After hitting the F10 (to move step by step through the code) and F11 (to jump into methods) quite often, I understood how the code works under the hood.

The cause

If you are not telling the library the resolution you want to have, it will select one for you based on this rudimentary rule (view source on Github):

// If the user did not specify a resolution, let's try and find a suitable one
if (resolution == null)
{
    foreach (var sps in supportedPreviewSizes)
    {
        if (sps.Width >= 640 && sps.Width <= 1000 && sps.Height >= 360 && sps.Height <= 1000)
        {
            resolution = new CameraResolution
            {
                Width = sps.Width,
                Height = sps.Height
            };
            break;
        }
    }
}

Let’s break it down. This code takes the available resolutions from the (old and deprecated) Android.Hardware.Camera API and selects one that matches the ranges defined in the code without respect to the aspect ratio of the device display. On my Nexus 5X, it always selects the resolution of 400×800. This does not match my device’s aspect ratio, but the Android OS does some stretching and squeezing to make it visible within the SurfaceView used for the preview. The result is the distortion seen above.

Note: The code above is doing exactly what it is supposed to do. However, it was updated last 3 years ago (according to Github). Display resolutions and aspect ratios changed a lot during that time, so we had to arrive at this point sooner or later.

Solution/Workaround

The solution to this is pretty easy. Just provide a CameraResolutionSelectorDelegate with your code setting up the ZXingScannerView. First, we need a method that returns a CameraResolution and takes a List of CameraResolution, let’s have a look at that one first:

public CameraResolution SelectLowestResolutionMatchingDisplayAspectRatio(List<CameraResolution> availableResolutions)
{            
    CameraResolution result = null;

    //a tolerance of 0.1 should not be visible to the user
    double aspectTolerance = 0.1;
    var displayOrientationHeight = DeviceDisplay.MainDisplayInfo.Orientation == DisplayOrientation.Portrait ? DeviceDisplay.MainDisplayInfo.Height : DeviceDisplay.MainDisplayInfo.Width;
    var displayOrientationWidth = DeviceDisplay.MainDisplayInfo.Orientation == DisplayOrientation.Portrait ? DeviceDisplay.MainDisplayInfo.Width : DeviceDisplay.MainDisplayInfo.Height;

    //calculatiing our targetRatio
    var targetRatio = displayOrientationHeight / displayOrientationWidth;
    var targetHeight = displayOrientationHeight;
    var minDiff = double.MaxValue;

    //camera API lists all available resolutions from highest to lowest, perfect for us
    //making use of this sorting, following code runs some comparisons to select the lowest resolution that matches the screen aspect ratio and lies within tolerance
    //selecting the lowest makes Qr detection actual faster most of the time
    foreach (var r in availableResolutions.Where(r => Math.Abs(((double)r.Width / r.Height) - targetRatio) < aspectTolerance))
    {
            //slowly going down the list to the lowest matching solution with the correct aspect ratio
            if (Math.Abs(r.Height - targetHeight) < minDiff)
            minDiff = Math.Abs(r.Height - targetHeight);
            result = r;                
    }

    return result;
}

First, we are setting up a fixed tolerance for the aspect ratio. A value of 0.1 should not be recognizable for users. The next step is calculating the target ratio. I am using the Xamarin.Essentials API here, because it saves me some code and works both in Xamarin.Android only projects as well as Xamarin.Forms ones.

Before we are able to select the best matching resolution, we need to notice a few points:

  • lower resolutions result in faster QR detection (even with big ones)
  • preview resolutions are always presented landscape
  • the list of available resolutions is sorted from biggest to smallest

Considering these points, we are able to loop over the list of available resolutions. If the current ratio is out of our tolerance range, we ignore it and move on. By setting the minDiff double down with every iteration, we are moving down the list to arrive at the lowest possible resolution that matches our display’s aspect ratio best. In the case of my Nexus 5X 480×800 with an aspect ratio of 1.66666~, which matches the display aspect ratio of 1,66111~ pretty close.

Delegating the selection call

Now that we have our calculating method in place, we need to pass the method via the CameraResolutionSelectorDelegate to our MobileBarcodeScanningOptions.

If you are on Xamarin.Android, your code will look similar to this:

var options = new ZXing.Mobile.MobileBarcodeScanningOptions()
{
   PossibleFormats = new List<ZXing.BarcodeFormat>() { ZXing.BarcodeFormat.QR_CODE },
CameraResolutionSelector = new CameraResolutionSelectorDelegate(SelectLowestResolutionMatchingDisplayAspectRatio)
}

If you are on Xamarin.Forms, you will have to use the DependencyService to get to the same result (as the method above has to be written within the Android project):

var options = new ZXing.Mobile.MobileBarcodeScanningOptions()
{
   PossibleFormats = new List<ZXing.BarcodeFormat>() { ZXing.BarcodeFormat.QR_CODE },
CameraResolutionSelector = DependencyService.Get<IZXingHelper>().CameraResolutionSelectorDelegateImplementation
}

The result

Now that we have an updated resolution selection mechanism in place, the result is exactly what we expected, without any distortion:

matching camera preview

Remarks

In case none of the camera resolutions gets selected, the camera preview automatically uses the default resolution. In my tests with three devices, this is always the highest one. The default resolution normally matches the devices aspect ratio. As it is the highest, it will slow down the QR detection, however.

The ZXing.Net.Mobile library uses the deprecated Android.Hardware.Camera API and Android.FastCamera library on Android. The next step would be to migrate over to the Android.Hardware.Camera2 API, which makes the FastCamera library obsolete and is future proof. I had already a look into that step, as I need to advance in two projects (one personal and one at work) with QR scanning, however, I postponed this change.

Conclusion

For the time being, we are still able to use the deprecated mechanism of getting our camera preview right. After identifying the reason for the distortion, I got pretty fast to a workaround/solution that should fit most use cases. As devices and their specs are evolving, we are at least not left behind. I will do another writeup once I found the time to replace the deprecated API in my fork.

As always, I hope this post will be helpful for some of you.

Until the next post, happy coding, everyone!

Title Image Credit

Comments 14
  1. Thanks for the article! About moving to Camera2 API, are you planning on creating a separate branch of zxing library for Xamarin? The project has tons of open issues and a lot of pull requests but Redth doesn’t seem to be doing much with it

    1. Glad you like my post. As I recognized that the original library is not actively developed atm, I will push the changes into my fork. You will be able to fork it from there once I uploaded my changes.

  2. Hi, i followed your instructions but IZXingHelper interface cannot be found. I searched for it in the latest ZXing master and it’s not there.
    Could you please detail where this interface resides or publish its code ?

    Thanks.

    1. Hey – sorry for the late reply. You need to create the interface yourself in your Xamarin.Forms project and connect it to the Android implementation. This post focuses on the platform implementation. If you don’t know how to use the DependencyService, check the docs. Hope this helps.

  3. Hi. Thanks for the interesting article, but I find it doesn’t work for me (it doesn’t find any suitable resolution and returns null) and I have some feedback…
    1. I think your solution seems to expect the camera preview to be full screen? How would you account for a smaller preview in only part of the screen, one which is more letterbox style like the shape of a barcode? Should I select the most widescreen resolution offered? We have found if we return a resolution that is not one of the ones offered (such as 400 x 60), then it works on some devices, crashes one device, and totally screws up the preview on another device.
    2. Your code states that the offered resolutions are highest to lowest. In fact on my Samsung S6 (Android 7) and Xamarin Forms 4.2 the are actually offered lowest to highest.
    3. When you calculate targetRatio you use Height/Width. But when finding the optimal resolution, you are using Width/Height. Is this a mistake or am I missing something?
    4. Your calculation of minDiff is having no effect, and you end up using the last non-landscape resolution offered. Combined with point 2 this means you do end up using the highest resolution that fits the bill. I think in your code you probably meant to wrap the assignment of minDiff and the following assignment of result in braces. This means you would end up returning the resolution with the lowest difference as I think you intended.
    5. For the interest of other readers, if using Xamarin Forms, Xamarin Essentials offers DeviceDisplay information meaning the delegate code does not need to be Android specific and therefore use of a dependancy service is not necessary, and the same code could be applied to non-Android platforms such it be needed (thought iOS doesn’t appear to suffer the same issue).

    Interested to read your thoughts on these points.

    Paul

    1. Hi Paul, thanks for your feedback.
      I am writing this answer out my head, as I moved on and did not dive too deep into that code again.
      1. you’re right, the code works best with full-screen previews.
      2. I tested this code on my Nexus and two Nokia (Android One) devices. All three (running Oreo) had the same sort order. It would not hurt to update the code to manually sort it that way, though. I took it on my list for updating this post.
      3. There is a reason for it, to name it exactly, I will need to go back into testing. (If I remember right, it was to match the available resolutions list – not sure, though).
      4. to be verified. I took it on my list as well.
      5. The IZxingHelper interface had more platform-specific code to run (besides the resolution), so I needed to call it, anyways.
      Additional note: The project I was using this code has died, so I will have to come up with a new sample for it. I will follow my own advice and then test your points. I have no concrete time-frame for you, though.

  4. Hi, Thanks for your awesome logic , but i have a device of Xiomi Redmi Note 7 Pro with Android OS version 9.0, you can fine device screen resolution here https://www.91mobiles.com/xiaomi-redmi-note-7-pro-price-in-india

    in your code , my result is return always zero and finally view does not get impact, you set aspectTolerance is “0.1” while all the possible resolution i tried which mention as below

    Camera Resolution for Width/Height where Width 1920 & Height 1440 Ration is 0.833333333333333
    Camera Resolution for Height/Width where Width 1920 & Height 1440 Ration is 1.41666666666667

    Width 1920 & Height 1440
    Math.Abs(((double)r.Width / r.Height) – targetRatio) = 0.833333333333333
    Math.Abs(((double)r.Height / r.Width) – targetRatio) = 1.41666666666667

    Width 1920 & Height 1080
    Math.Abs(((double)r.Width / r.Height) – targetRatio) = 0.388888888888889
    Math.Abs(((double)r.Height / r.Width) – targetRatio) = 1.60416666666667

    Width 1600 & Height 1200
    Math.Abs(((double)r.Width / r.Height) – targetRatio) = 0.833333333333333
    Math.Abs(((double)r.Height / r.Width) – targetRatio) = 1.41666666666667

    Width 1440 & Height 1080
    Math.Abs(((double)r.Width / r.Height) – targetRatio) = 0.833333333333333
    Math.Abs(((double)r.Height / r.Width) – targetRatio) = 1.41666666666667

    Width 1440 & Height 720
    Math.Abs(((double)r.Width / r.Height) – targetRatio) = 0.166666666666667
    Math.Abs(((double)r.Height / r.Width) – targetRatio) = 1.66666666666667

    Width 1280 & Height 960
    Math.Abs(((double)r.Width / r.Height) – targetRatio) = 0.833333333333333
    Math.Abs(((double)r.Height / r.Width) – targetRatio) = 1.41666666666667

    Width 1280 & Height 720
    Math.Abs(((double)r.Width / r.Height) – targetRatio) = 0.388888888888889
    Math.Abs(((double)r.Height / r.Width) – targetRatio) = 1.60416666666667

    Width 800 & Height 600
    Math.Abs(((double)r.Width / r.Height) – targetRatio) = 0.833333333333333
    Math.Abs(((double)r.Height / r.Width) – targetRatio) = 1.41666666666667

    Width 720 & Height 480
    Math.Abs(((double)r.Width / r.Height) – targetRatio) = 0.666666666666667
    Math.Abs(((double)r.Height / r.Width) – targetRatio) = 1.5

    Width 640 & Height 480
    Math.Abs(((double)r.Width / r.Height) – targetRatio) = 0.833333333333333
    Math.Abs(((double)r.Height / r.Width) – targetRatio) = 1.41666666666667

    Width 640 & Height 360
    Math.Abs(((double)r.Width / r.Height) – targetRatio) = 0.388888888888889
    Math.Abs(((double)r.Height / r.Width) – targetRatio) = 1.60416666666667

    Width 352 & Height 288
    Math.Abs(((double)r.Width / r.Height) – targetRatio) = 0.944444444444444
    Math.Abs(((double)r.Height / r.Width) – targetRatio) = 1.34848484848485

    Width 320 & Height 240
    Math.Abs(((double)r.Width / r.Height) – targetRatio) = 0.833333333333333
    Math.Abs(((double)r.Height / r.Width) – targetRatio) = 1.41666666666667

    Width 176 & Height 144
    Math.Abs(((double)r.Width / r.Height) – targetRatio) = 0.944444444444444
    Math.Abs(((double)r.Height / r.Width) – targetRatio) = 1.34848484848485

    so, in all the cases Math.Abs(((double)r.Width / r.Height) – targetRatio) < aspectTolerance) is always false and final result is remain null

    Please suggest, am i missing something or my way is not correct?waiting for your reply

    1. Hi,
      to be honest, I did not touch that code again as it was working for all my touch devices. I even turned away from ZXing for XF and instead created my own control using the native (iOS)/Firebase and Camera2 APIs (maybe I’ll blog about that one in the new year).

      I would try to play around with the aspectTolerance and find a value that does not distort the preview again.

      Best,
      Marco

Join the discussion right now!

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Prev
Crypto and Blockchain projects & tools (April 2019)
blockchain-hand-title

Crypto and Blockchain projects & tools (April 2019)

Next
How to host a code file on Github as Gist to use in your application
gist_title_featured_image

How to host a code file on Github as Gist to use in your application

You May Also Like

This website uses cookies. By continuing to use this site, you accept the use of cookies.  Learn more