This video shows how one can use google vision API (available from Play services) to implement the Optical Character Recognition in your Android App.
It uses TextRecognizer API from the google vision. To use this API the required libraries are implemented in the gradle (App) file. Once implemented in the gradle file, it makes the required APIs available in the Java code.
In layout, it creates a separate constraint layout for surface view. This surface view is used to project the camera focus area.
In the end, in this App, it also converts the read text into Speech using the TextToSpeech API.
In this video the App testing is shown in emulator. In the emulator, already there were 2 images created which read the text – “Hi, how are you?”
This App is also hosted on the Google Play Store and can be found using the below link: https://play.google.com/store/apps/details?id=com.programmerworld.ocr_opticalcharacterrecognition
For any suggestions or queries, one can contact us at: https://programmerworld.co/contact/ or email at: programmerworld1990@gmail.com
Source Code:
package com.example.ocr_opticalcharacterrecognition;
import android.content.pm.PackageManager;
import android.os.Bundle;
import android.os.Handler;
import android.os.Looper;
import android.speech.tts.TextToSpeech;
import android.util.SparseArray;
import android.view.SurfaceHolder;
import android.view.SurfaceView;
import android.view.View;
import android.widget.TextView;
import androidx.appcompat.app.AppCompatActivity;
import androidx.core.app.ActivityCompat;
import com.google.android.gms.vision.CameraSource;
import com.google.android.gms.vision.Detector;
import com.google.android.gms.vision.text.TextBlock;
import com.google.android.gms.vision.text.TextRecognizer;
import java.io.IOException;
import static android.Manifest.permission.CAMERA;
public class MainActivity extends AppCompatActivity {
private TextView textView;
private SurfaceView surfaceView;
private CameraSource cameraSource;
private TextRecognizer textRecognizer;
private TextToSpeech textToSpeech;
private String stringResult = null;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
ActivityCompat.requestPermissions(this, new String[]{CAMERA}, PackageManager.PERMISSION_GRANTED);
textToSpeech = new TextToSpeech(this, new TextToSpeech.OnInitListener() {
@Override
public void onInit(int status) {
}
});
}
@Override
protected void onDestroy() {
super.onDestroy();
cameraSource.release();
}
private void textRecognizer(){
textRecognizer = new TextRecognizer.Builder(getApplicationContext()).build();
cameraSource = new CameraSource.Builder(getApplicationContext(), textRecognizer)
.setRequestedPreviewSize(1280, 1024)
.build();
surfaceView = findViewById(R.id.surfaceView);
surfaceView.getHolder().addCallback(new SurfaceHolder.Callback() {
@Override
public void surfaceCreated(SurfaceHolder holder) {
try {
cameraSource.start(surfaceView.getHolder());
} catch (IOException e) {
e.printStackTrace();
}
}
@Override
public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
}
@Override
public void surfaceDestroyed(SurfaceHolder holder) {
cameraSource.stop();
}
});
textRecognizer.setProcessor(new Detector.Processor<TextBlock>() {
@Override
public void release() {
}
@Override
public void receiveDetections(Detector.Detections<TextBlock> detections) {
SparseArray<TextBlock> sparseArray = detections.getDetectedItems();
StringBuilder stringBuilder = new StringBuilder();
for (int i = 0; i<sparseArray.size(); ++i){
TextBlock textBlock = sparseArray.valueAt(i);
if (textBlock != null && textBlock.getValue() !=null){
stringBuilder.append(textBlock.getValue() + " ");
}
}
final String stringText = stringBuilder.toString();
Handler handler = new Handler(Looper.getMainLooper());
handler.post(new Runnable() {
@Override
public void run() {
stringResult = stringText;
resultObtained();
}
});
}
});
}
private void resultObtained(){
setContentView(R.layout.activity_main);
textView = findViewById(R.id.textView);
textView.setText(stringResult);
textToSpeech.speak(stringResult, TextToSpeech.QUEUE_FLUSH, null, null);
}
public void buttonStart(View view){
setContentView(R.layout.surfaceview);
textRecognizer();
}
}
apply plugin: 'com.android.application'
android {
compileSdkVersion 29
buildToolsVersion "29.0.3"
defaultConfig {
applicationId "com.example.ocr_opticalcharacterrecognition"
minSdkVersion 23
targetSdkVersion 29
versionCode 1
versionName "1.0"
testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
}
buildTypes {
release {
minifyEnabled false
proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
}
}
}
dependencies {
implementation fileTree(dir: 'libs', include: ['*.jar'])
implementation 'androidx.appcompat:appcompat:1.1.0'
implementation 'androidx.constraintlayout:constraintlayout:1.1.3'
testImplementation 'junit:junit:4.12'
androidTestImplementation 'androidx.test.ext:junit:1.1.1'
androidTestImplementation 'androidx.test.espresso:espresso-core:3.2.0'
implementation 'com.google.android.gms:play-services-vision:19.0.0'
}
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="com.example.ocr_opticalcharacterrecognition">
<uses-permission android:name="android.permission.CAMERA"/>
<application
android:allowBackup="true"
android:icon="@mipmap/ic_launcher"
android:label="@string/app_name"
android:roundIcon="@mipmap/ic_launcher_round"
android:supportsRtl="true"
android:theme="@style/AppTheme">
<activity android:name=".MainActivity">
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
</activity>
</application>
</manifest>
<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
tools:context=".MainActivity">
<TextView
android:id="@+id/textView"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_marginStart="164dp"
android:layout_marginTop="68dp"
android:text="Hello World!"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toBottomOf="@+id/buttonStart" />
<Button
android:id="@+id/buttonStart"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_marginStart="140dp"
android:layout_marginTop="134dp"
android:onClick="buttonStart"
android:text="@string/start_reading"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toTopOf="parent" />
</androidx.constraintlayout.widget.ConstraintLayout>
<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout
xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent">
<SurfaceView
android:id="@+id/surfaceView"
android:layout_width="308dp"
android:layout_height="503dp"
android:layout_marginStart="64dp"
android:layout_marginTop="32dp"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toTopOf="parent" />
</androidx.constraintlayout.widget.ConstraintLayout>
How do I add a capture button to capture image since the timeout for camera is very less, about 1 second. Please help me with that.
There can be multiple ways to do this, including the attributes of CameraSource API.
However, the easiest approach which can be quickly tried (and will give better control over the camera preview) is to implement your own camera preview option in the App. The details are shown in below page:
https://programmerworld.co/android/how-to-create-a-simple-android-camera-app-or-mirror-app-to-preview-the-images-using-android-studio/
Once the position of camera is as per requirement then have the capture button to call the OCR code as explained in this page.
I hope above information helps.
Cheers
Programmer World
–
In ocr code I am getting error at surfacecreated block for camerasource.start() even after adding the try and catch blocks
how can i resolve this
Could you please copy the error text the App is throwing? Otherwise please let us know the exception being thrown in the line of code.
Cheers
Programmer World
–
in the line 66
cameraSource.start(surfaceView.getHolder());
in the line 66
cameraSource.start(surfaceView.getHolder());
Thanks, but I think the complete error message is not posted here. It only says the line number but not the actual reason for the error. Can you put the complete text please.
Cheers
Programmer World
–
Call requires permission which may be rejected by user: code should explicitly check to see if permission is available (with checkPermission) or explicitly handle a potential SecurityException
Perfect, then please check if you have gotten the CAMERA permission from user of the App.
Please check the permission by using below line in your Java code:
ActivityCompat.requestPermissions(this, new String[]{CAMERA}, PackageManager.PERMISSION_GRANTED);
and respective entry in manifest file.
Cheers
Programmer World
–
Now it is showing error in “this” statement on the above line and we have added camera permission in manifest file also
Now it is showing error in “this” statement on the above line and we have added camera permission in manifest file also
Hi
the app suddenly stop and im having this error:
E/Vision: Error loading optional module com.google.android.gms.vision.ocr: com.google.android.gms.dynamite.DynamiteModule$LoadingException: No acceptable module found. Local version is 0 and remote version is 0.
W/DynamiteModule: Local module descriptor class for com.google.android.gms.vision.dynamite.ocr not found.
I/DynamiteModule: Considering local module com.google.android.gms.vision.dynamite.ocr:0 and remote module com.google.android.gms.vision.dynamite.ocr:0
W/DynamiteModule: Local module descriptor class for com.google.android.gms.vision.ocr not found.
I/DynamiteModule: Considering local module com.google.android.gms.vision.ocr:0 and remote module com.google.android.gms.vision.ocr:0
E/Vision: Error loading optional module com.google.android.gms.vision.ocr: com.google.android.gms.dynamite.DynamiteModule$LoadingException: No acceptable module found. Local version is 0 and remote version is 0.
D/Camera: app passed NULL surface
D/Camera: app passed NULL surface
D/AndroidRuntime: Shutting down VM
E/AndroidRuntime: FATAL EXCEPTION: main
Process: com.example.ocr_opticalcharacterrecognition, PID: 23018
java.lang.NullPointerException: Attempt to invoke virtual method ‘void android.widget.TextView.setText(java.lang.CharSequence)’ on a null object reference
at com.example.ocr_opticalcharacterrecognition.MainActivity.resultObtained(MainActivity.java:118)
at com.example.ocr_opticalcharacterrecognition.MainActivity.access$300(MainActivity.java:26)
at com.example.ocr_opticalcharacterrecognition.MainActivity$3$1.run(MainActivity.java:108)
at android.os.Handler.handleCallback(Handler.java:938)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:223)
at android.app.ActivityThread.main(ActivityThread.java:7656)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:592)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:947)
I/Process: Sending signal. PID: 23018 SIG: 9
Im having this issue in the code
could you please help me?
W/DynamiteModule: Local module descriptor class for com.google.android.gms.vision.dynamite.ocr not found.
I/DynamiteModule: Considering local module com.google.android.gms.vision.dynamite.ocr:0 and remote module com.google.android.gms.vision.dynamite.ocr:0
D/TextNativeHandle: Cannot load feature, fall back to load dynamite module.
W/DynamiteModule: Local module descriptor class for com.google.android.gms.vision.ocr not found.
I/DynamiteModule: Considering local module com.google.android.gms.vision.ocr:0 and remote module com.google.android.gms.vision.ocr:0
E/Vision: Error loading module com.google.android.gms.vision.ocr optional module true: com.google.android.gms.dynamite.DynamiteModule$LoadingException: No acceptable module found. Local version is 0 and remote version is 0.
W/DynamiteModule: Local module descriptor class for com.google.android.gms.vision.dynamite.ocr not found.
I/DynamiteModule: Considering local module com.google.android.gms.vision.dynamite.ocr:0 and remote module com.google.android.gms.vision.dynamite.ocr:0
D/TextNativeHandle: Cannot load feature, fall back to load dynamite module.
W/DynamiteModule: Local module descriptor class for com.google.android.gms.vision.ocr not found.
I/DynamiteModule: Considering local module com.google.android.gms.vision.ocr:0 and remote module com.google.android.gms.vision.ocr:0
E/Vision: Error loading module com.google.android.gms.vision.ocr optional module true: com.google.android.gms.dynamite.DynamiteModule$LoadingException: No acceptable module found. Local version is 0 and remote version is 0.
D/Camera: app passed NULL surface
D/AndroidRuntime: Shutting down VM
E/AndroidRuntime: FATAL EXCEPTION: main
Process: com.example.ocr_opticalcharacterrecognition, PID: 25251
java.lang.NullPointerException: Attempt to invoke virtual method ‘void android.widget.TextView.setText(java.lang.CharSequence)’ on a null object reference
at com.example.ocr_opticalcharacterrecognition.MainActivity.resultObtained(MainActivity.java:118)
at com.example.ocr_opticalcharacterrecognition.MainActivity.access$300(MainActivity.java:26)
at com.example.ocr_opticalcharacterrecognition.MainActivity$3$1.run(MainActivity.java:108)
at android.os.Handler.handleCallback(Handler.java:938)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:223)
at android.app.ActivityThread.main(ActivityThread.java:7656)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:592)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:947)
I/Process: Sending signal. PID: 25251 SIG: 9
Sorry for late response, but after going through the error message it seems that the issue is mostly because of the missing library class.
I hope the below library has been successfully implemented and compiled in the gradle file as also mentioned in the source code above:
implementation ‘com.google.android.gms:play-services-vision:19.0.0’
I hope the above helps in resolving your issue.
Cheers
Programmer World
–
Im having the issue bellow.
could you please help me to solve it?
W/DynamiteModule: Local module descriptor class for com.google.android.gms.vision.dynamite.ocr not found.
I/DynamiteModule: Considering local module com.google.android.gms.vision.dynamite.ocr:0 and remote module com.google.android.gms.vision.dynamite.ocr:0
D/TextNativeHandle: Cannot load feature, fall back to load dynamite module.
W/DynamiteModule: Local module descriptor class for com.google.android.gms.vision.ocr not found.
I/DynamiteModule: Considering local module com.google.android.gms.vision.ocr:0 and remote module com.google.android.gms.vision.ocr:0
E/Vision: Error loading module com.google.android.gms.vision.ocr optional module true: com.google.android.gms.dynamite.DynamiteModule$LoadingException: No acceptable module found. Local version is 0 and remote version is 0.
W/DynamiteModule: Local module descriptor class for com.google.android.gms.vision.dynamite.ocr not found.
I/DynamiteModule: Considering local module com.google.android.gms.vision.dynamite.ocr:0 and remote module com.google.android.gms.vision.dynamite.ocr:0
D/TextNativeHandle: Cannot load feature, fall back to load dynamite module.
W/DynamiteModule: Local module descriptor class for com.google.android.gms.vision.ocr not found.
I/DynamiteModule: Considering local module com.google.android.gms.vision.ocr:0 and remote module com.google.android.gms.vision.ocr:0
E/Vision: Error loading module com.google.android.gms.vision.ocr optional module true: com.google.android.gms.dynamite.DynamiteModule$LoadingException: No acceptable module found. Local version is 0 and remote version is 0.
D/Camera: app passed NULL surface
D/AndroidRuntime: Shutting down VM
E/AndroidRuntime: FATAL EXCEPTION: main
Process: com.example.ocr_opticalcharacterrecognition, PID: 25251
java.lang.NullPointerException: Attempt to invoke virtual method ‘void android.widget.TextView.setText(java.lang.CharSequence)’ on a null object reference
at com.example.ocr_opticalcharacterrecognition.MainActivity.resultObtained(MainActivity.java:118)
at com.example.ocr_opticalcharacterrecognition.MainActivity.access$300(MainActivity.java:26)
at com.example.ocr_opticalcharacterrecognition.MainActivity$3$1.run(MainActivity.java:108)
at android.os.Handler.handleCallback(Handler.java:938)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:223)
at android.app.ActivityThread.main(ActivityThread.java:7656)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:592)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:947)
I/Process: Sending signal. PID: 25251 SIG: 9
Sorry for late response, but after going through the error message it seems that the issue is mostly because of the missing library class.
I hope the below library has been successfully implemented and compiled in the gradle file as also mentioned in the source code above:
implementation ‘com.google.android.gms:play-services-vision:19.0.0’
I hope the above helps in resolving your issue.
Cheers
Programmer World
–
im having the issue bellow.
could you please help me?
Sorry for late response, but after going through the error message it seems that the issue is mostly because of the missing library class.
I hope the below library has been successfully implemented and compiled in the gradle file as also mentioned in the source code above:
implementation ‘com.google.android.gms:play-services-vision:19.0.0’
I hope the above helps in resolving your issue.
Cheers
Programmer World
–
Hello, I am trying to build this app but I would like to make it be able to highlight certain inputted text. I understand the idea of how to do this but not necessarily the implementation. Do you have any tips? Also I love your content thank you.
I am not sure what you mean by “highlight certain inputted text”. If you mean you want to take a pic and highlight it in an image form then you may have to do in a separate camera activity as shown in below:
https://programmerworld.co/android/how-to-create-a-simple-android-camera-app-or-mirror-app-to-preview-the-images-using-android-studio/
https://programmerworld.co/android/how-to-create-custom-camera-app-to-take-pictures-in-android-phone/
If you want to just highlight the scanned texts, then that is shown in the code in this page. Once you get the text in the below line of code then it can be used to highlight in any form:
stringResult = stringText;
Say for example this text can be printed in the textview as shown in below line of codes:
textView.setText(stringResult);
Cheers
Programmer World
–
Hi. Would it be possible to make the source code downloadable as a complete project? Seems silly that we have to copy paste and create each file.
If not here, stick it on github or something please.
Yes, we have plans to create Github repository for the project files. Over here, on our portal, some of the file types are not supported and gives issues even after zipping the whole folder. However, thanks for raising the concerns.
Cheers
Programmer World
–