How to build end to end Agentic AI Chatbot on Android with RAG Using Gemini and Pinecone?

Agentic AI RAG chatbot Android App

Steps:

  1. Get the subject PDFs. For example Financial Law document, Cancer Diagnosis pdf, etc.
  2. Vectorize the content using Gemini embedding-001 model and store it in Pinecone db
  3. Query the content from Android App UI

Sample Prompts:
Financial Law PDF:

  1. What are the responsibilities of the Registrar in financial administration?
  2. How is the programme budget prepared?
  3. What happens to unspent appropriations after a financial period?
  4. Who can authorize cash advances at the Court?

Cancer Diagnosis PDF:

  1. What is the third step, “access to treatment”?

For Pinecone API Key: https://app.pinecone.io
For Gemini API Key: https://aistudio.google.com/apikey

I hope you like this video. For any questions, suggestions or appreciation please contact us at: https://programmerworld.co/contact/ or email at: programmerworld1990@gmail.com

Details:

Code:

package com.programmerworld.agenticaichatbotapp;

import android.content.Intent;
import android.net.Uri;
import android.os.Bundle;
import android.util.Log;
import android.widget.Button;
import android.widget.EditText;
import android.widget.TextView;
import android.widget.Toast;
import androidx.appcompat.app.AppCompatActivity;

import com.programmerworld.agenticaichatbotapp.api.GeminiApi;
import com.programmerworld.agenticaichatbotapp.api.PineconeApi;
import com.programmerworld.agenticaichatbotapp.models.GeminiRequest;
import com.programmerworld.agenticaichatbotapp.models.GeminiResponse;
import com.programmerworld.agenticaichatbotapp.models.GeminiChatRequest;
import com.programmerworld.agenticaichatbotapp.models.GeminiChatResponse;
import com.programmerworld.agenticaichatbotapp.models.PineconeUpsertRequest;
import com.programmerworld.agenticaichatbotapp.models.PineconeQueryRequest;
import com.programmerworld.agenticaichatbotapp.models.PineconeQueryResponse;

import okhttp3.OkHttpClient;
import retrofit2.Call;
import retrofit2.Callback;
import retrofit2.Response;
import retrofit2.Retrofit;
import retrofit2.converter.gson.GsonConverterFactory;

import com.itextpdf.kernel.pdf.PdfDocument;
import com.itextpdf.kernel.pdf.PdfReader;
import com.itextpdf.kernel.pdf.canvas.parser.PdfTextExtractor;

import java.io.InputStream;
import java.io.IOException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

public class MainActivity extends AppCompatActivity {

    private static final int PICK_PDF_REQUEST = 101;
    private static final String TAG = "AgenticAIChatbot";
    private static final String GEMINI_API_KEY = "AIzaSyDekRf7eL4rVfZXmL9R6GTU9sF7HUpVDSg"; // Replace
    private static final String PINECONE_API_KEY = "pcsk_7BUqFA_TMZdd9dsdcMCRcvdPHQK73cNm6sViYuMjuNQAcjFfiYV9yimkpCH5kmScNvjzmh"; // Replace

    private GeminiApi geminiApi;
    private PineconeApi pineconeApi;
    private EditText queryInput;
    private TextView responseText;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        // Initialize Retrofit clients
        Retrofit retrofitGemini = new Retrofit.Builder()
                .baseUrl("https://generativelanguage.googleapis.com/")
                .client(new OkHttpClient())
                .addConverterFactory(GsonConverterFactory.create())
                .build();
        geminiApi = retrofitGemini.create(GeminiApi.class);

        Retrofit retrofitPinecone = new Retrofit.Builder()
                .baseUrl("https://my-pdf-index-je5df9k.svc.aped-4627-b74a.pinecone.io")
                .client(new OkHttpClient())
                .addConverterFactory(GsonConverterFactory.create())
                .build();
        pineconeApi = retrofitPinecone.create(PineconeApi.class);

        // UI elements
        queryInput = findViewById(R.id.queryInput);
        responseText = findViewById(R.id.responseText);

        // Upload PDF button
        Button uploadButton = findViewById(R.id.uploadButton);
        uploadButton.setOnClickListener(v -> openFilePicker());

        // Send query button
        Button sendButton = findViewById(R.id.sendButton);
        sendButton.setOnClickListener(v -> handleQuery());
    }

    private void openFilePicker() {
        Intent intent = new Intent(Intent.ACTION_OPEN_DOCUMENT);
        intent.addCategory(Intent.CATEGORY_OPENABLE);
        intent.setType("application/pdf");
        startActivityForResult(intent, PICK_PDF_REQUEST);
    }

    @Override
    protected void onActivityResult(int requestCode, int resultCode, Intent data) {
        super.onActivityResult(requestCode, resultCode, data);
        if (requestCode == PICK_PDF_REQUEST && resultCode == RESULT_OK && data != null) {
            Uri uri = data.getData();
            if (uri != null) {
                processPdf(uri);
            }
        }
    }

    private void processPdf(Uri uri) {
        new Thread(() -> {
            try {
                String fileName = uri.getLastPathSegment();
                String text = extractTextFromPdf(uri);
                List<String> chunks = splitTextIntoChunks(text);
                upsertToPinecone(fileName != null ? fileName : "unknown.pdf", chunks);
                runOnUiThread(() -> Toast.makeText(this, "PDF processed: " + fileName, Toast.LENGTH_SHORT).show());
            } catch (Exception e) {
                Log.e(TAG, "Error processing PDF: " + e.getMessage());
                runOnUiThread(() -> Toast.makeText(this, "Error processing PDF", Toast.LENGTH_SHORT).show());
            }
        }).start();
    }

    private String extractTextFromPdf(Uri uri) {
        StringBuilder textBuilder = new StringBuilder();
        try {
            InputStream inputStream = getContentResolver().openInputStream(uri);
            if (inputStream != null) {
                PdfReader reader = new PdfReader(inputStream);
                PdfDocument document = new PdfDocument(reader);
                for (int i = 1; i <= document.getNumberOfPages(); i++) {
                    textBuilder.append(PdfTextExtractor.getTextFromPage(document.getPage(i)));
                }
                document.close();
                inputStream.close();
                Log.d(TAG, "Extracted text: " + textBuilder.substring(0, Math.min(textBuilder.length(), 50)) + "...");
            }
        } catch (IOException e) {
            Log.e(TAG, "Error reading PDF: " + e.getMessage());
        }
        return textBuilder.toString();
    }

    private List<String> splitTextIntoChunks(String text) {
        List<String> chunks = new ArrayList<>();
        int chunkSize = 500;
        int start = 0;
        while (start < text.length()) {
            int end = Math.min(start + chunkSize, text.length());
            chunks.add(text.substring(start, end));
            start = end;
        }
        Log.d(TAG, "Split into " + chunks.size() + " chunks");
        return chunks;
    }

    private void upsertToPinecone(String fileName, List<String> chunks) {
        List<PineconeUpsertRequest.Vector> vectors = new ArrayList<>();
        for (int i = 0; i < chunks.size(); i++) {
            String chunk = chunks.get(i);
            Call<GeminiResponse> call = geminiApi.getEmbedding(GEMINI_API_KEY, new GeminiRequest(chunk));
            try {
                Response<GeminiResponse> response = call.execute();
                if (response.isSuccessful() && response.body() != null) {
                    List<Float> embedding = response.body().getEmbedding().getValues();
                    Map<String, String> metadata = new HashMap<>();
                    metadata.put("text", chunk);
                    vectors.add(new PineconeUpsertRequest.Vector(fileName + "-chunk-" + i, embedding, metadata));
                } else {
                    Log.e(TAG, "Failed to embed chunk " + i + ": " + response.message());
                }
            } catch (Exception e) {
                Log.e(TAG, "Embedding error: " + e.getMessage());
            }
        }

        if (!vectors.isEmpty()) {
            Call<Object> upsertCall = pineconeApi.upsertVectors(PINECONE_API_KEY, new PineconeUpsertRequest(vectors));
            upsertCall.enqueue(new Callback<Object>() {
                @Override
                public void onResponse(Call<Object> call, Response<Object> response) {
                    if (response.isSuccessful()) {
                        Log.d(TAG, "Upserted " + vectors.size() + " vectors");
                    } else {
                        Log.e(TAG, "Upsert failed: " + response.message());
                    }
                }

                @Override
                public void onFailure(Call<Object> call, Throwable t) {
                    Log.e(TAG, "Upsert error: " + t.getMessage());
                }
            });
        }
    }

    private void handleQuery() {
        responseText.setText("Querying the data ... ");
        String query = queryInput.getText().toString().trim();
        if (query.isEmpty()) {
            Toast.makeText(this, "Please enter a query", Toast.LENGTH_SHORT).show();
            return;
        }

        new Thread(() -> {
            try {
                // Step 1: Embed the query
                Call<GeminiResponse> embedCall = geminiApi.getEmbedding(GEMINI_API_KEY, new GeminiRequest(query));
                Response<GeminiResponse> embedResponse = embedCall.execute();
                if (!embedResponse.isSuccessful() || embedResponse.body() == null) {
                    throw new Exception("Embedding failed: " + embedResponse.message());
                }
                List<Float> queryEmbedding = embedResponse.body().getEmbedding().getValues();
                Log.d(TAG, "Query embedding size: " + queryEmbedding.size());

                // Step 2: Query Pinecone for relevant chunks
                Call<PineconeQueryResponse> queryCall = pineconeApi.queryVectors(PINECONE_API_KEY, new PineconeQueryRequest(queryEmbedding, 3));
                Response<PineconeQueryResponse> queryResponse = queryCall.execute();
                if (!queryResponse.isSuccessful()) {
                    throw new Exception("Pinecone query failed: " + queryResponse.message() + " - " + queryResponse.code());
                }
                PineconeQueryResponse responseBody = queryResponse.body();
                Log.d(TAG, "Pinecone raw response: " + (queryResponse.raw() != null ? queryResponse.raw().toString() : "null"));
                Log.d(TAG, "Pinecone parsed response: " + (responseBody != null ? responseBody.toString() : "null"));

                List<PineconeQueryResponse.Match> matches = responseBody != null ? responseBody.getMatches() : null;
                if (matches == null || matches.isEmpty()) {
                    Log.w(TAG, "No matches found in Pinecone response");
                    runOnUiThread(() -> responseText.setText("No relevant information found in the vector database."));
                    return;
                }

                StringBuilder context = new StringBuilder();
                for (PineconeQueryResponse.Match match : matches) {
                    Log.d(TAG, "Match: " + match.toString());
                    Map<String, String> metadata = match.getMetadata();
                    if (metadata != null && metadata.containsKey("text")) {
                        context.append(metadata.get("text")).append("\n");
                    } else {
                        Log.w(TAG, "Match " + match.getId() + " has no valid metadata");
                    }
                }
                if (context.length() == 0) {
                    Log.w(TAG, "No valid context extracted from matches");
                    runOnUiThread(() -> responseText.setText("No relevant text found in the vector database."));
                    return;
                }
                Log.d(TAG, "Context extracted: " + context.toString());

                // Step 3: Generate response with Gemini using RAG
                Call<GeminiChatResponse> chatCall = geminiApi.generateContent(GEMINI_API_KEY, new GeminiChatRequest(query, context.toString()));
                Response<GeminiChatResponse> chatResponse = chatCall.execute();
                Log.d(TAG, "Gemini chat raw response: " + (chatResponse.raw() != null ? chatResponse.raw().toString() : "null"));
                if (!chatResponse.isSuccessful()) {
                    String errorBody = chatResponse.errorBody() != null ? chatResponse.errorBody().string() : "No error body";
                    throw new Exception("Chat generation failed: " + chatResponse.code() + " - " + chatResponse.message() + " - " + errorBody);
                }
                if (chatResponse.body() == null) {
                    throw new Exception("Chat generation failed: Response body is null");
                }
                String answer = chatResponse.body().getResponseText();
                Log.d(TAG, "Gemini response: " + answer);
                runOnUiThread(() -> responseText.setText(answer));
            } catch (Exception e) {
                Log.e(TAG, "Query error: " + e.getMessage(), e);
                runOnUiThread(() -> Toast.makeText(this, "Error processing query: " + e.getMessage(), Toast.LENGTH_LONG).show());
            }
        }).start();
    }

}
package com.programmerworld.agenticaichatbotapp.api;

import com.programmerworld.agenticaichatbotapp.models.GeminiChatRequest;
import com.programmerworld.agenticaichatbotapp.models.GeminiChatResponse;
import com.programmerworld.agenticaichatbotapp.models.GeminiRequest;
import com.programmerworld.agenticaichatbotapp.models.GeminiResponse;

import retrofit2.Call;
import retrofit2.http.Body;
import retrofit2.http.POST;
import retrofit2.http.Query;

public interface GeminiApi {
    @POST("v1/models/embedding-001:embedContent")
    Call<GeminiResponse> getEmbedding(
            @Query("key") String apiKey,
            @Body GeminiRequest request
    );

    @POST("v1/models/gemini-1.5-flash:generateContent")
    Call<GeminiChatResponse> generateContent(
            @Query("key") String apiKey,
            @Body GeminiChatRequest request
    );
}
package com.programmerworld.agenticaichatbotapp.api;

import com.programmerworld.agenticaichatbotapp.models.PineconeQueryRequest;
import com.programmerworld.agenticaichatbotapp.models.PineconeQueryResponse;
import com.programmerworld.agenticaichatbotapp.models.PineconeUpsertRequest;

import retrofit2.Call;
import retrofit2.http.Body;
import retrofit2.http.Header;
import retrofit2.http.POST;

public interface PineconeApi {
    @POST("vectors/upsert")
    Call<Object> upsertVectors(
            @Header("Api-Key") String apiKey,
            @Body PineconeUpsertRequest request
    );

    @POST("query")
    Call<PineconeQueryResponse> queryVectors(
            @Header("Api-Key") String apiKey,
            @Body PineconeQueryRequest request
    );
}
package com.programmerworld.agenticaichatbotapp.models;

import java.util.List;

public class GeminiChatRequest {
    private List<Content> contents;

    public GeminiChatRequest(String query, String context) {
        this.contents = List.of(
                new Content("user", List.of(new Part("Context: " + context))),
                new Content("user", List.of(new Part("Query: " + query)))
        );
    }

    public List<Content> getContents() {
        return contents;
    }

    public static class Content {
        private String role;
        private List<Part> parts;

        public Content(String role, List<Part> parts) {
            this.role = role;
            this.parts = parts;
        }

        public String getRole() {
            return role;
        }

        public List<Part> getParts() {
            return parts;
        }
    }

    public static class Part {
        private String text;

        public Part(String text) {
            this.text = text;
        }

        public String getText() {
            return text;
        }
    }
}
package com.programmerworld.agenticaichatbotapp.models;

import java.util.List;

public class GeminiChatResponse {
    private List<Candidate> candidates;

    public String getResponseText() {
        if (candidates != null && !candidates.isEmpty()) {
            return candidates.get(0).getContent().getParts().get(0).getText();
        }
        return "No response";
    }

    public static class Candidate {
        private Content content;

        public Content getContent() {
            return content;
        }
    }

    public static class Content {
        private List<Part> parts;

        public List<Part> getParts() {
            return parts;
        }
    }

    public static class Part {
        private String text;

        public String getText() {
            return text;
        }
    }
}
package com.programmerworld.agenticaichatbotapp.models;

import java.util.Collections;
import java.util.List;

public class GeminiRequest {
    private String model = "embedding-001";
    private Content content;
    private String task_type;

    public GeminiRequest(String text) {
        this.content = new Content(Collections.singletonList(new Part(text)));
        this.task_type = "retrieval_document";
    }

    public String getModel() {
        return model;
    }

    public Content getContent() {
        return content;
    }

    public String getTaskType() {
        return task_type;
    }

    public static class Content {
        private List<Part> parts;

        public Content(List<Part> parts) {
            this.parts = parts;
        }

        public List<Part> getParts() {
            return parts;
        }
    }

    public static class Part {
        private String text;

        public Part(String text) {
            this.text = text;
        }

        public String getText() {
            return text;
        }
    }
}
package com.programmerworld.agenticaichatbotapp.models;

import java.util.List;

public class GeminiResponse {
    private Embedding embedding;

    public Embedding getEmbedding() {
        return embedding;
    }

    public static class Embedding {
        private List<Float> values;

        public List<Float> getValues() {
            return values;
        }
    }
}
package com.programmerworld.agenticaichatbotapp.models;

import java.util.List;

public class PineconeQueryRequest {
    private List<Float> vector;
    private int topK;
    private boolean includeMetadata; // Added to request metadata

    public PineconeQueryRequest(List<Float> vector, int topK) {
        this.vector = vector;
        this.topK = topK;
        this.includeMetadata = true; // Set to true by default
    }

    public List<Float> getVector() {
        return vector;
    }

    public int getTopK() {
        return topK;
    }

    public boolean isIncludeMetadata() {
        return includeMetadata;
    }
}
package com.programmerworld.agenticaichatbotapp.models;

import java.util.List;
import java.util.Map;

public class PineconeQueryResponse {
    private List<Match> matches;

    public List<Match> getMatches() {
        return matches;
    }

    @Override
    public String toString() {
        return "PineconeQueryResponse{matches=" + (matches != null ? matches.size() : "null") + "}";
    }

    public static class Match {
        private String id;
        private float score; // Added to capture score for debugging
        private Map<String, String> metadata;

        public String getId() {
            return id;
        }

        public float getScore() {
            return score;
        }

        public Map<String, String> getMetadata() {
            return metadata;
        }

        @Override
        public String toString() {
            return "Match{id='" + id + "', score=" + score + ", metadata=" + metadata + "}";
        }
    }
}
package com.programmerworld.agenticaichatbotapp.models;

import java.util.List;
import java.util.Map;

public class PineconeUpsertRequest {
    private List<Vector> vectors;

    public PineconeUpsertRequest(List<Vector> vectors) {
        this.vectors = vectors;
    }

    public List<Vector> getVectors() {
        return vectors;
    }

    public static class Vector {
        private String id;
        private List<Float> values;
        private Map<String, String> metadata;

        public Vector(String id, List<Float> values, Map<String, String> metadata) {
            this.id = id;
            this.values = values;
            this.metadata = metadata;
        }

        public String getId() {
            return id;
        }

        public List<Float> getValues() {
            return values;
        }

        public Map<String, String> getMetadata() {
            return metadata;
        }
    }
}
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools">

    <uses-permission android:name="android.permission.INTERNET"/>

    <application
        android:allowBackup="true"
        android:dataExtractionRules="@xml/data_extraction_rules"
        android:fullBackupContent="@xml/backup_rules"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:roundIcon="@mipmap/ic_launcher_round"
        android:supportsRtl="true"
        android:theme="@style/Theme.AgenticAIChatbotApp"
        tools:targetApi="31">
        <activity
            android:name=".MainActivity"
            android:exported="true">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />

                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
    </application>

</manifest>
plugins {
    alias(libs.plugins.android.application)
}

android {
    namespace = "com.programmerworld.agenticaichatbotapp"
    compileSdk = 35

    defaultConfig {
        applicationId = "com.programmerworld.agenticaichatbotapp"
        minSdk = 33
        targetSdk = 35
        versionCode = 1
        versionName = "1.0"

        testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
    }

    buildTypes {
        release {
            isMinifyEnabled = false
            proguardFiles(
                getDefaultProguardFile("proguard-android-optimize.txt"),
                "proguard-rules.pro"
            )
        }
    }
    compileOptions {
        sourceCompatibility = JavaVersion.VERSION_11
        targetCompatibility = JavaVersion.VERSION_11
    }
}

dependencies {

    implementation(libs.appcompat)
    implementation(libs.material)
    implementation(libs.activity)
    implementation(libs.constraintlayout)
    testImplementation(libs.junit)
    androidTestImplementation(libs.ext.junit)
    androidTestImplementation(libs.espresso.core)

    implementation("com.squareup.retrofit2:retrofit:2.9.0")
    implementation("com.squareup.retrofit2:converter-gson:2.9.0")
    implementation("com.squareup.okhttp3:okhttp:4.11.0")
    implementation("com.itextpdf:itext7-core:7.2.5")

    
}
<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:padding="16dp">

    <Button
        android:id="@+id/uploadButton"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_marginTop="40dp"
        android:text="Upload PDF"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toTopOf="parent" />

    <EditText
        android:id="@+id/queryInput"
        android:layout_width="0dp"
        android:layout_height="wrap_content"
        android:layout_marginTop="16dp"
        android:hint="Enter your question"
        android:inputType="text"
        android:textSize="20sp"
        app:layout_constraintEnd_toStartOf="@id/sendButton"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toBottomOf="@id/uploadButton" />

    <Button
        android:id="@+id/sendButton"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="Send"
        app:layout_constraintTop_toBottomOf="@id/uploadButton"
        app:layout_constraintEnd_toEndOf="parent"
        android:layout_marginTop="16dp" />

    <TextView
        android:id="@+id/responseText"
        android:layout_width="0dp"
        android:layout_height="0dp"
        android:layout_marginTop="16dp"
        android:gravity="top"
        android:text="Response will appear here"
        android:textSize="20sp"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toBottomOf="@id/queryInput" />

</androidx.constraintlayout.widget.ConstraintLayout>

Screenshots:

What is the third step, “access to treatment”?

How is the programme budget prepared?

1st Response:

2nd Response:

For API Keys and Vector DB:

Gemini:

Pinecone:

In the database there are 367 records of data

Sample PDF documents used in this demo are attached below:

finance_lawDownload
cancer_diagnosisDownload

Leave a Reply

Your email address will not be published. Required fields are marked *

wpChatIcon
    wpChatIcon