How to Create Android OpenCV Camera App and Image Capture

My last tutorial was on how to install and use OpenCV library in android project.

This tutorial will partly depend on it since I will not explain the steps involved in setting up OpenCV in android project again.

If you have not read the previous tutorial, I will suggest you do so before you continue with this tutorial.

On the other hand, if you have install OpenCV library in your android project before you can as well skip the previous tutorial and continue with this tutorial.

Before we start coding, it is important for us to have an idea of what we are going to create.

1. Open the previous android project with OpenCV already setup

Open the project in Android Studio. Once the project is loaded, go to AndroidManifest.xml file and add the following permissions and uses features.

<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.INTERNET" />
<uses-feature
    android:name="android.hardware.camera"
    android:required="false" />
<uses-feature
    android:name="android.hardware.camera.autofocus"
    android:required="false" />
<uses-feature
    android:name="android.hardware.camera.front"
    android:required="false" />
<uses-feature
    android:name="android.hardware.camera.front.autofocus"
    android:required="false" />

Since we are going to take picture and stored the captured image in our device internal storage, we are going to ask for the Write and Read external storage permissions from our app users.

2. Open activity_main_layout file

We are going to add two Button widgets. The first button is used to trigger OpenCV camera preview while the second button will be used in our next OpenCV tutorial.

We are going to cover how to create Android Document Scanner Application using OpenCV library.

Add the code below to the layout file.

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:orientation="vertical"
    android:gravity="center"
    tools:context=".MainActivity">

    <Button
        android:id="@+id/start_button"
        android:layout_width="200dp"
        android:layout_height="wrap_content"
        android:text="OpenCV Camera"
        android:paddingLeft="20dp"
        android:paddingStart="20dp"
        android:paddingEnd="20dp"
        android:paddingRight="20dp" />

    <Button
        android:id="@+id/document_button"
        android:layout_width="200dp"
        android:layout_height="wrap_content"
        android:text="Document Scanner"
        android:layout_marginTop="10dp"
        android:paddingLeft="20dp"
        android:paddingStart="20dp"
        android:paddingEnd="20dp"
        android:paddingRight="20dp" />
</LinearLayout>

3. Open MainActivity.java file

In MainActivity.java file, we are going to get the reference of the OpenCv camera preview button we added in the layout file. 

In addition, we need to request for user permission using a third party android library name Dexter

Add the below line of code to your module level build.gradle file.

implementation 'com.karumi:dexter:5.0.0'

Now, add the code below to your MainActivity class.

public class MainActivity extends AppCompatActivity {
    
    private static final String TAG = MainActivity.class.getSimpleName();

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        Dexter.withActivity(this)
                .withPermissions(Manifest.permission.WRITE_EXTERNAL_STORAGE, Manifest.permission.READ_EXTERNAL_STORAGE, Manifest.permission.CAMERA)
                .withListener(new MultiplePermissionsListener() {
                    @Override
                    public void onPermissionsChecked(MultiplePermissionsReport report) {
                        if(!report.areAllPermissionsGranted()){
                            Toast.makeText(MainActivity.this, "You need to grant all permission to use this app features", Toast.LENGTH_SHORT).show();
                        }
                    }

                    @Override
                    public void onPermissionRationaleShouldBeShown(List<PermissionRequest> permissions, PermissionToken token) {

                    }
                })
                .check();

        Button startButton = (Button)findViewById(R.id.start_button);
        startButton.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
                Intent cvIntent = new Intent(MainActivity.this, OpenCVCamera.class);
                startActivity(cvIntent);
            }
        });

        Button scannerButton = (Button)findViewById(R.id.document_button);
        scannerButton.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
                //todo in the next tutorial
            }
        });
    }
}

4. Create a new Activity class

Create a new Activity class which we will navigate to when the camera preview button is clicked.

Go to File > New > Activity > Empty Activity, you can name your activity any name of your choice. I have named mine OpenCvCamera.

Open the layout file (activity_open_cvcamera.xml) and add the code below.

<?xml version="1.0" encoding="utf-8"?>
<FrameLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools"
    xmlns:opencv="http://schemas.android.com/apk/res-auto"
    android:id="@+id/activity_opencv_camera"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:paddingBottom="0dp"
    android:paddingLeft="0dp"
    android:paddingRight="0dp"
    android:paddingTop="0dp"
    tools:context=".OpenCVCamera">

    <inducesmile.com.opencvexample.OpenCameraView
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:visibility="gone"
        android:id="@+id/camera_view"
        opencv:show_fps="true"
        opencv:camera_id="any"/>

    <ImageView
        android:id="@+id/take_picture"
        android:layout_width="64dp"
        android:layout_height="64dp"
        android:layout_marginBottom="24dp"
        android:contentDescription="@string/app_name"
        android:src="@drawable/ic_camera_red_600_48dp"
        android:layout_gravity="center|bottom"/>

</FrameLayout>

In the layout file above, we have add a custom view name OpenCameraView.

We will come back to it later. We have also add an ImageView which work as the image capture button.

Open the OpenCVCamera class and add the code below to it. I will explain the code thereafter.

import android.os.Bundle;
import android.support.v7.app.AppCompatActivity;
import android.util.Log;
import android.view.SurfaceView;
import android.view.View;
import android.view.WindowManager;
import android.widget.ImageView;
import android.widget.Toast;

import org.opencv.android.BaseLoaderCallback;
import org.opencv.android.CameraBridgeViewBase;
import org.opencv.android.LoaderCallbackInterface;
import org.opencv.android.OpenCVLoader;
import org.opencv.core.CvType;
import org.opencv.core.Mat;

import java.io.File;

import inducesmile.com.opencvexample.preprocess.ImagePreprocessor;
import inducesmile.com.opencvexample.utils.Constants;
import inducesmile.com.opencvexample.utils.FolderUtil;
import inducesmile.com.opencvexample.utils.Utilities;

public class OpenCVCamera extends AppCompatActivity implements CameraBridgeViewBase.CvCameraViewListener2{

    private static final String TAG = OpenCVCamera.class.getSimpleName();

    private OpenCameraView cameraBridgeViewBase;
    private Mat colorRgba;
    private Mat colorGray;
    private Mat des, forward;
    private ImagePreprocessor preprocessor;


    private BaseLoaderCallback baseLoaderCallback = new BaseLoaderCallback(this) {
        @Override
        public void onManagerConnected(int status) {
            switch (status) {
                case LoaderCallbackInterface.SUCCESS:
                    cameraBridgeViewBase.enableView();
                    break;
                default:
                    super.onManagerConnected(status);
                    break;
            }
        }
    };

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);
        setContentView(R.layout.activity_open_cvcamera);

        preprocessor = new ImagePreprocessor();

        cameraBridgeViewBase = (OpenCameraView) findViewById(R.id.camera_view);
        cameraBridgeViewBase.setVisibility(SurfaceView.VISIBLE);
        cameraBridgeViewBase.setCvCameraViewListener(this);
        cameraBridgeViewBase.disableFpsMeter();

        ImageView takePictureBtn = (ImageView)findViewById(R.id.take_picture);
        takePictureBtn.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
                String outPicture = Constants.SCAN_IMAGE_LOCATION + File.separator + Utilities.generateFilename();
                FolderUtil.createDefaultFolder(Constants.SCAN_IMAGE_LOCATION);

                cameraBridgeViewBase.takePicture(outPicture);
                Toast.makeText(OpenCVCamera.this, "Picture has been taken ", Toast.LENGTH_LONG).show();
                Log.d(TAG, "Path " + outPicture);
            }
        });
    }

    @Override
    public void onPause() {
        super.onPause();
        if (cameraBridgeViewBase != null)
            cameraBridgeViewBase.disableView();
    }

    @Override
    public void onResume(){
        super.onResume();
        if (!OpenCVLoader.initDebug()) {
            Log.d(TAG, "Internal OpenCV library not found. Using OpenCV Manager for initialization");
            OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_2_4_11, this, baseLoaderCallback);
        } else {
            Log.d(TAG, "OpenCV library found inside package. Using it!");
            baseLoaderCallback.onManagerConnected(LoaderCallbackInterface.SUCCESS);
        }
    }

    public void onDestroy() {
        super.onDestroy();
        if (cameraBridgeViewBase != null)
            cameraBridgeViewBase.disableView();
    }

    @Override
    public void onCameraViewStarted(int width, int height) {
        colorRgba = new Mat(height, width, CvType.CV_8UC4);
        colorGray = new Mat(height, width, CvType.CV_8UC1);

        des = new Mat(height, width, CvType.CV_8UC4);
        forward = new Mat(height, width, CvType.CV_8UC4);
    }

    @Override
    public void onCameraViewStopped() {
        colorRgba.release();
    }

    @Override
    public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
        colorRgba = inputFrame.rgba();
        preprocessor.changeImagePreviewOrientation(colorRgba, des, forward);
        return colorRgba;
    }
}

In the code above, we instantiate the BaseLoaderCallback interface which is used to manager the state / status of OpenCV library.

We check if OpenCV library is loader through an asyn method call (initAsync).

The callback interface is passed as a parameter to this method.

The onManagerConnected(int status) method is used to enable our custom view if the OpenCV library is loaded and connected with the activity page.

The OpenCameraView class also implement the CvCameraViewListener2 interface.

By implementing this interface, we are going to override the following three methods – onCameraViewStarted(int width, int height)onCameraViewStopped() and onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame)

5. Create a new Java Class

Create a new java class. I will name the class OpenCameraView. Free feel to name your class any name of your choice.

This class will extends the JavaCameraView and implement the Camera.PictureCallback interface.

We will override the the onPictureTaken(byte[] data, Camera camera)

Open the created java file and add the code below to it.

import android.content.Context;
import android.graphics.Bitmap;
import android.graphics.BitmapFactory;
import android.graphics.Matrix;
import android.hardware.Camera;
import android.net.Uri;
import android.util.AttributeSet;
import android.util.Log;

import org.opencv.android.JavaCameraView;

import java.io.FileOutputStream;
import java.util.List;

public class OpenCameraView extends JavaCameraView implements Camera.PictureCallback {

    private static final String TAG = OpenCameraView.class.getSimpleName();

    private String mPictureFileName;

    public static int minWidthQuality = 400;

    private Context context;


    public OpenCameraView(Context context, AttributeSet attrs) {
        super(context, attrs);
        this.context = context;
    }

    public List<String> getEffectList() {
        return mCamera.getParameters().getSupportedColorEffects();
    }

    public boolean isEffectSupported() {
        return (mCamera.getParameters().getColorEffect() != null);
    }

    public String getEffect() {
        return mCamera.getParameters().getColorEffect();
    }

    public void setEffect(String effect) {
        Camera.Parameters params = mCamera.getParameters();
        params.setColorEffect(effect);
        mCamera.setParameters(params);
    }

    public List<Camera.Size> getResolutionList() {
        return mCamera.getParameters().getSupportedPreviewSizes();
    }

    public void setResolution(Camera.Size resolution) {
        disconnectCamera();
        mMaxHeight = resolution.height;
        mMaxWidth = resolution.width;
        connectCamera(getWidth(), getHeight());
    }

    public Camera.Size getResolution() {
        return mCamera.getParameters().getPreviewSize();
    }

    public void takePicture(final String fileName) {
        Log.i(TAG, "Taking picture");
        this.mPictureFileName = fileName;
        mCamera.setPreviewCallback(null);

        mCamera.takePicture(null, null, this);
    }

    @Override
    public void onPictureTaken(byte[] data, Camera camera) {
        Log.i(TAG, "Saving a bitmap to file");
        // The camera preview was automatically stopped. Start it again.
        mCamera.startPreview();
        mCamera.setPreviewCallback(this);

        Bitmap bitmap = BitmapFactory.decodeByteArray(data, 0, data.length);
        Uri uri = Uri.parse(mPictureFileName);

        Log.d(TAG, "selectedImage: " + uri);
        Bitmap bm = null;
        bm = rotate(bitmap, 90);

        // Write the image in a file (in jpeg format)
        try {
            FileOutputStream fos = new FileOutputStream(mPictureFileName);
            bm.compress(Bitmap.CompressFormat.JPEG, 80, fos);
            fos.close();

        } catch (java.io.IOException e) {
            Log.e("PictureDemo", "Exception in photoCallback", e);
        }
    }

    private static Bitmap rotate(Bitmap bm, int rotation) {
        if (rotation != 0) {
            Matrix matrix = new Matrix();
            matrix.postRotate(rotation);
            Bitmap bmOut = Bitmap.createBitmap(bm, 0, 0, bm.getWidth(), bm.getHeight(), matrix, true);
            return bmOut;
        }
        return bm;
    }
}

We have added a method to rotate the captured image to normal orientation since OpenCV camera has a default image orientation of 270 degrees.

If you are interested in using default Android Camera 2 API, you can read my tutorial on Android Camera2 API example.

In my next tutorial on Android OpenCV, we are going to learn how to create an Android Document Scanner App.

Free feel to download the source code from our Github account.

If you have questions or suggestions kindly use the comment box below.

If you have done a project with OpenCV before you can as well share it with us.

                                                                                                                                            

Add a Comment