How to create your Optical Character Recognition (OCR) Android App and convert the text into speech?

This video shows how one can use google vision API (available from Play services) to implement the Optical Character Recognition in your Android App.

It uses TextRecognizer API from the google vision. To use this API the required libraries are implemented in the gradle (App) file. Once implemented in the gradle file, it makes the required APIs available in the Java code.

In layout, it creates a separate constraint layout for surface view. This surface view is used to project the camera focus area.

In the end, in this App, it also converts the read text into Speech using the TextToSpeech API.

In this video the App testing is shown in emulator. In the emulator, already there were 2 images created which read the text – “Hi, how are you?”

This App is also hosted on the Google Play Store and can be found using the below link: https://play.google.com/store/apps/details?id=com.programmerworld.ocr_opticalcharacterrecognition

For any suggestions or queries, one can contact us at: https://programmerworld.co/contact/ or email at: programmerworld1990@gmail.com

Source Code:

package com.example.ocr_opticalcharacterrecognition;

import android.content.pm.PackageManager;
import android.os.Bundle;
import android.os.Handler;
import android.os.Looper;
import android.speech.tts.TextToSpeech;
import android.util.SparseArray;
import android.view.SurfaceHolder;
import android.view.SurfaceView;
import android.view.View;
import android.widget.TextView;

import androidx.appcompat.app.AppCompatActivity;
import androidx.core.app.ActivityCompat;

import com.google.android.gms.vision.CameraSource;
import com.google.android.gms.vision.Detector;
import com.google.android.gms.vision.text.TextBlock;
import com.google.android.gms.vision.text.TextRecognizer;

import java.io.IOException;

import static android.Manifest.permission.CAMERA;

public class MainActivity extends AppCompatActivity {

    private TextView textView;
    private SurfaceView surfaceView;

    private CameraSource cameraSource;
    private TextRecognizer textRecognizer;

    private TextToSpeech textToSpeech;
    private String stringResult = null;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        ActivityCompat.requestPermissions(this, new String[]{CAMERA}, PackageManager.PERMISSION_GRANTED);
        textToSpeech = new TextToSpeech(this, new TextToSpeech.OnInitListener() {
            @Override
            public void onInit(int status) {
            }
        });
    }

    @Override
    protected void onDestroy() {
        super.onDestroy();
        cameraSource.release();
    }

    private void textRecognizer(){
        textRecognizer = new TextRecognizer.Builder(getApplicationContext()).build();
        cameraSource = new CameraSource.Builder(getApplicationContext(), textRecognizer)
                .setRequestedPreviewSize(1280, 1024)
                .build();

        surfaceView = findViewById(R.id.surfaceView);
        surfaceView.getHolder().addCallback(new SurfaceHolder.Callback() {
            @Override
            public void surfaceCreated(SurfaceHolder holder) {
                try {
                    cameraSource.start(surfaceView.getHolder());
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }

            @Override
            public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
            }

            @Override
            public void surfaceDestroyed(SurfaceHolder holder) {
                cameraSource.stop();
            }
        });


        textRecognizer.setProcessor(new Detector.Processor<TextBlock>() {
            @Override
            public void release() {
            }

            @Override
            public void receiveDetections(Detector.Detections<TextBlock> detections) {

                SparseArray<TextBlock> sparseArray = detections.getDetectedItems();
                StringBuilder stringBuilder = new StringBuilder();

                for (int i = 0; i<sparseArray.size(); ++i){
                    TextBlock textBlock = sparseArray.valueAt(i);
                    if (textBlock != null && textBlock.getValue() !=null){
                        stringBuilder.append(textBlock.getValue() + " ");
                    }
                }

                final String stringText = stringBuilder.toString();

                Handler handler = new Handler(Looper.getMainLooper());
                handler.post(new Runnable() {
                    @Override
                    public void run() {
                   stringResult = stringText;
                   resultObtained();
                    }
                });
            }
        });
    }

    private void resultObtained(){
        setContentView(R.layout.activity_main);
        textView = findViewById(R.id.textView);
        textView.setText(stringResult);
        textToSpeech.speak(stringResult, TextToSpeech.QUEUE_FLUSH, null, null);
    }

    public void buttonStart(View view){
        setContentView(R.layout.surfaceview);
        textRecognizer();
    }
}

apply plugin: 'com.android.application'

android {
    compileSdkVersion 29
    buildToolsVersion "29.0.3"

    defaultConfig {
        applicationId "com.example.ocr_opticalcharacterrecognition"
        minSdkVersion 23
        targetSdkVersion 29
        versionCode 1
        versionName "1.0"

        testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
    }

    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
        }
    }

}

dependencies {
    implementation fileTree(dir: 'libs', include: ['*.jar'])

    implementation 'androidx.appcompat:appcompat:1.1.0'
    implementation 'androidx.constraintlayout:constraintlayout:1.1.3'
    testImplementation 'junit:junit:4.12'
    androidTestImplementation 'androidx.test.ext:junit:1.1.1'
    androidTestImplementation 'androidx.test.espresso:espresso-core:3.2.0'

    implementation 'com.google.android.gms:play-services-vision:19.0.0'
}

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.example.ocr_opticalcharacterrecognition">

    <uses-permission android:name="android.permission.CAMERA"/>

    <application
        android:allowBackup="true"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:roundIcon="@mipmap/ic_launcher_round"
        android:supportsRtl="true"
        android:theme="@style/AppTheme">
        <activity android:name=".MainActivity">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />

                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
    </application>

</manifest>

<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".MainActivity">

    <TextView
        android:id="@+id/textView"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_marginStart="164dp"
        android:layout_marginTop="68dp"
        android:text="Hello World!"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toBottomOf="@+id/buttonStart" />

    <Button
        android:id="@+id/buttonStart"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_marginStart="140dp"
        android:layout_marginTop="134dp"
        android:onClick="buttonStart"
        android:text="@string/start_reading"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toTopOf="parent" />

</androidx.constraintlayout.widget.ConstraintLayout>

<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout
    xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent">

    <SurfaceView
        android:id="@+id/surfaceView"
        android:layout_width="308dp"
        android:layout_height="503dp"
        android:layout_marginStart="64dp"
        android:layout_marginTop="32dp"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toTopOf="parent" />
</androidx.constraintlayout.widget.ConstraintLayout>

Comments (22)

sticktoy says:
December 28, 2020 at 5:15 pm
How do I add a capture button to capture image since the timeout for camera is very less, about 1 second. Please help me with that.
Reply
1. ProgrammerWorld says:
  December 29, 2020 at 10:29 am
  There can be multiple ways to do this, including the attributes of CameraSource API.
  However, the easiest approach which can be quickly tried (and will give better control over the camera preview) is to implement your own camera preview option in the App. The details are shown in below page:
  https://programmerworld.co/android/how-to-create-a-simple-android-camera-app-or-mirror-app-to-preview-the-images-using-android-studio/
  Once the position of camera is as per requirement then have the capture button to call the OCR code as explained in this page.
  I hope above information helps.
  Cheers
  Programmer World
  –
  Reply
sri says:
February 1, 2021 at 7:27 pm
In ocr code I am getting error at surfacecreated block for camerasource.start() even after adding the try and catch blocks
how can i resolve this
Reply
1. ProgrammerWorld says:
  February 1, 2021 at 8:03 pm
  Could you please copy the error text the App is throwing? Otherwise please let us know the exception being thrown in the line of code.
  Cheers
  Programmer World
  –
  Reply
  1. Anonymous says:
    February 1, 2021 at 8:46 pm
    in the line 66
    cameraSource.start(surfaceView.getHolder());
  2. sri says:
    February 1, 2021 at 8:48 pm
    in the line 66
    cameraSource.start(surfaceView.getHolder());
  3. ProgrammerWorld says:
    February 1, 2021 at 9:12 pm
    Thanks, but I think the complete error message is not posted here. It only says the line number but not the actual reason for the error. Can you put the complete text please.
    Cheers
    Programmer World
    –
sri says:
February 1, 2021 at 9:23 pm
Call requires permission which may be rejected by user: code should explicitly check to see if permission is available (with checkPermission) or explicitly handle a potential SecurityException
Reply
1. ProgrammerWorld says:
  February 1, 2021 at 9:29 pm
  Perfect, then please check if you have gotten the CAMERA permission from user of the App.
  Please check the permission by using below line in your Java code:
  ActivityCompat.requestPermissions(this, new String[]{CAMERA}, PackageManager.PERMISSION_GRANTED);
  and respective entry in manifest file.
  Cheers
  Programmer World
  –
  Reply
  1. Sri says:
    February 1, 2021 at 9:35 pm
    Now it is showing error in “this” statement on the above line and we have added camera permission in manifest file also
  2. Sri says:
    February 1, 2021 at 9:59 pm
    Now it is showing error in “this” statement on the above line and we have added camera permission in manifest file also
jym126 says:
February 8, 2021 at 12:10 am
Hi
the app suddenly stop and im having this error:
E/Vision: Error loading optional module com.google.android.gms.vision.ocr: com.google.android.gms.dynamite.DynamiteModule$LoadingException: No acceptable module found. Local version is 0 and remote version is 0.
W/DynamiteModule: Local module descriptor class for com.google.android.gms.vision.dynamite.ocr not found.
I/DynamiteModule: Considering local module com.google.android.gms.vision.dynamite.ocr:0 and remote module com.google.android.gms.vision.dynamite.ocr:0
W/DynamiteModule: Local module descriptor class for com.google.android.gms.vision.ocr not found.
I/DynamiteModule: Considering local module com.google.android.gms.vision.ocr:0 and remote module com.google.android.gms.vision.ocr:0
E/Vision: Error loading optional module com.google.android.gms.vision.ocr: com.google.android.gms.dynamite.DynamiteModule$LoadingException: No acceptable module found. Local version is 0 and remote version is 0.
D/Camera: app passed NULL surface
D/Camera: app passed NULL surface
D/AndroidRuntime: Shutting down VM
E/AndroidRuntime: FATAL EXCEPTION: main
Process: com.example.ocr_opticalcharacterrecognition, PID: 23018
java.lang.NullPointerException: Attempt to invoke virtual method ‘void android.widget.TextView.setText(java.lang.CharSequence)’ on a null object reference
at com.example.ocr_opticalcharacterrecognition.MainActivity.resultObtained(MainActivity.java:118)
at com.example.ocr_opticalcharacterrecognition.MainActivity.access$300(MainActivity.java:26)
at com.example.ocr_opticalcharacterrecognition.MainActivity$3$1.run(MainActivity.java:108)
at android.os.Handler.handleCallback(Handler.java:938)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:223)
at android.app.ActivityThread.main(ActivityThread.java:7656)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:592)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:947)
I/Process: Sending signal. PID: 23018 SIG: 9
Reply
jym126 says:
February 9, 2021 at 7:44 pm
Im having this issue in the code
could you please help me?
W/DynamiteModule: Local module descriptor class for com.google.android.gms.vision.dynamite.ocr not found.
I/DynamiteModule: Considering local module com.google.android.gms.vision.dynamite.ocr:0 and remote module com.google.android.gms.vision.dynamite.ocr:0
D/TextNativeHandle: Cannot load feature, fall back to load dynamite module.
W/DynamiteModule: Local module descriptor class for com.google.android.gms.vision.ocr not found.
I/DynamiteModule: Considering local module com.google.android.gms.vision.ocr:0 and remote module com.google.android.gms.vision.ocr:0
E/Vision: Error loading module com.google.android.gms.vision.ocr optional module true: com.google.android.gms.dynamite.DynamiteModule$LoadingException: No acceptable module found. Local version is 0 and remote version is 0.
W/DynamiteModule: Local module descriptor class for com.google.android.gms.vision.dynamite.ocr not found.
I/DynamiteModule: Considering local module com.google.android.gms.vision.dynamite.ocr:0 and remote module com.google.android.gms.vision.dynamite.ocr:0
D/TextNativeHandle: Cannot load feature, fall back to load dynamite module.
W/DynamiteModule: Local module descriptor class for com.google.android.gms.vision.ocr not found.
I/DynamiteModule: Considering local module com.google.android.gms.vision.ocr:0 and remote module com.google.android.gms.vision.ocr:0
E/Vision: Error loading module com.google.android.gms.vision.ocr optional module true: com.google.android.gms.dynamite.DynamiteModule$LoadingException: No acceptable module found. Local version is 0 and remote version is 0.
D/Camera: app passed NULL surface
D/AndroidRuntime: Shutting down VM
E/AndroidRuntime: FATAL EXCEPTION: main
Process: com.example.ocr_opticalcharacterrecognition, PID: 25251
java.lang.NullPointerException: Attempt to invoke virtual method ‘void android.widget.TextView.setText(java.lang.CharSequence)’ on a null object reference
at com.example.ocr_opticalcharacterrecognition.MainActivity.resultObtained(MainActivity.java:118)
at com.example.ocr_opticalcharacterrecognition.MainActivity.access$300(MainActivity.java:26)
at com.example.ocr_opticalcharacterrecognition.MainActivity$3$1.run(MainActivity.java:108)
at android.os.Handler.handleCallback(Handler.java:938)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:223)
at android.app.ActivityThread.main(ActivityThread.java:7656)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:592)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:947)
I/Process: Sending signal. PID: 25251 SIG: 9
Reply
1. ProgrammerWorld says:
  February 20, 2021 at 11:56 am
  Sorry for late response, but after going through the error message it seems that the issue is mostly because of the missing library class.
  I hope the below library has been successfully implemented and compiled in the gradle file as also mentioned in the source code above:
  implementation ‘com.google.android.gms:play-services-vision:19.0.0’
  I hope the above helps in resolving your issue.
  Cheers
  Programmer World
  –
  Reply
jym126 says:
February 9, 2021 at 7:46 pm
Im having the issue bellow.
could you please help me to solve it?
W/DynamiteModule: Local module descriptor class for com.google.android.gms.vision.dynamite.ocr not found.
I/DynamiteModule: Considering local module com.google.android.gms.vision.dynamite.ocr:0 and remote module com.google.android.gms.vision.dynamite.ocr:0
D/TextNativeHandle: Cannot load feature, fall back to load dynamite module.
W/DynamiteModule: Local module descriptor class for com.google.android.gms.vision.ocr not found.
I/DynamiteModule: Considering local module com.google.android.gms.vision.ocr:0 and remote module com.google.android.gms.vision.ocr:0
E/Vision: Error loading module com.google.android.gms.vision.ocr optional module true: com.google.android.gms.dynamite.DynamiteModule$LoadingException: No acceptable module found. Local version is 0 and remote version is 0.
W/DynamiteModule: Local module descriptor class for com.google.android.gms.vision.dynamite.ocr not found.
I/DynamiteModule: Considering local module com.google.android.gms.vision.dynamite.ocr:0 and remote module com.google.android.gms.vision.dynamite.ocr:0
D/TextNativeHandle: Cannot load feature, fall back to load dynamite module.
W/DynamiteModule: Local module descriptor class for com.google.android.gms.vision.ocr not found.
I/DynamiteModule: Considering local module com.google.android.gms.vision.ocr:0 and remote module com.google.android.gms.vision.ocr:0
E/Vision: Error loading module com.google.android.gms.vision.ocr optional module true: com.google.android.gms.dynamite.DynamiteModule$LoadingException: No acceptable module found. Local version is 0 and remote version is 0.
D/Camera: app passed NULL surface
D/AndroidRuntime: Shutting down VM
E/AndroidRuntime: FATAL EXCEPTION: main
Process: com.example.ocr_opticalcharacterrecognition, PID: 25251
java.lang.NullPointerException: Attempt to invoke virtual method ‘void android.widget.TextView.setText(java.lang.CharSequence)’ on a null object reference
at com.example.ocr_opticalcharacterrecognition.MainActivity.resultObtained(MainActivity.java:118)
at com.example.ocr_opticalcharacterrecognition.MainActivity.access$300(MainActivity.java:26)
at com.example.ocr_opticalcharacterrecognition.MainActivity$3$1.run(MainActivity.java:108)
at android.os.Handler.handleCallback(Handler.java:938)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:223)
at android.app.ActivityThread.main(ActivityThread.java:7656)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:592)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:947)
I/Process: Sending signal. PID: 25251 SIG: 9
Reply
1. ProgrammerWorld says:
  February 20, 2021 at 11:56 am
  Sorry for late response, but after going through the error message it seems that the issue is mostly because of the missing library class.
  I hope the below library has been successfully implemented and compiled in the gradle file as also mentioned in the source code above:
  implementation ‘com.google.android.gms:play-services-vision:19.0.0’
  I hope the above helps in resolving your issue.
  Cheers
  Programmer World
  –
  Reply
jym126 says:
February 9, 2021 at 7:48 pm
im having the issue bellow.
could you please help me?
Reply
1. ProgrammerWorld says:
  February 20, 2021 at 11:57 am
  Sorry for late response, but after going through the error message it seems that the issue is mostly because of the missing library class.
  I hope the below library has been successfully implemented and compiled in the gradle file as also mentioned in the source code above:
  implementation ‘com.google.android.gms:play-services-vision:19.0.0’
  I hope the above helps in resolving your issue.
  Cheers
  Programmer World
  –
  Reply
Sara says:
April 8, 2021 at 1:15 am
Hello, I am trying to build this app but I would like to make it be able to highlight certain inputted text. I understand the idea of how to do this but not necessarily the implementation. Do you have any tips? Also I love your content thank you.
Reply
1. ProgrammerWorld says:
  April 10, 2021 at 10:59 pm
  I am not sure what you mean by “highlight certain inputted text”. If you mean you want to take a pic and highlight it in an image form then you may have to do in a separate camera activity as shown in below:
  https://programmerworld.co/android/how-to-create-a-simple-android-camera-app-or-mirror-app-to-preview-the-images-using-android-studio/
  https://programmerworld.co/android/how-to-create-custom-camera-app-to-take-pictures-in-android-phone/
  If you want to just highlight the scanned texts, then that is shown in the code in this page. Once you get the text in the below line of code then it can be used to highlight in any form:
  stringResult = stringText;
  Say for example this text can be printed in the textview as shown in below line of codes:
  textView.setText(stringResult);
  Cheers
  Programmer World
  –
  Reply
Jesper N says:
October 21, 2021 at 5:19 am
Hi. Would it be possible to make the source code downloadable as a complete project? Seems silly that we have to copy paste and create each file.
If not here, stick it on github or something please.
Reply
1. ProgrammerWorld says:
  October 25, 2021 at 6:25 pm
  Yes, we have plans to create Github repository for the project files. Over here, on our portal, some of the file types are not supported and gives issues even after zipping the whole folder. However, thanks for raising the concerns.
  Cheers
  Programmer World
  –
  Reply

Comments (22)

Leave a Reply Cancel reply