Introducing the Local Copilot Chatbot Application: Your Ultimate Document-Based Query Assistant

                                        

actual screenshot taken of the knowledge bot

Introducing the Local Copilot Chatbot Application: Your Ultimate Document-Based Query Assistant

In today's fast-paced world, finding precise information quickly can make a significant difference. Our Local Copilot Chatbot Application offers a cutting-edge solution for accessing and querying document-based knowledge with remarkable efficiency. This Flask-based application utilizes the powerful Ollama and Phi3 models to deliver an interactive, intuitive chatbot experience. Here's a deep dive into what our application offers and how it leverages modern technologies to enhance your productivity.

What is the Local Copilot Chatbot Application?

The Local Copilot Chatbot Application is designed to serve as your personal assistant for document-based queries. Imagine having a copilot that understands your documents, provides precise answers, and adapts to your needs. That's exactly what our application does. It transforms your document uploads into a dynamic knowledge base that you can query using natural language.

Key Features

- Interactive Chatbot Interface: Engage with a responsive chatbot that provides accurate answers based on your document content.

- Document Upload and Processing: Upload your documents, and our system processes them into a searchable knowledge base.

- Vector Knowledge Base with RAG System: Utilize a sophisticated Retrieval-Augmented Generation (RAG) system that combines vector embeddings and document retrieval to deliver precise responses.

- Microservices Architecture: Our application uses a microservices approach, keeping the front-end and back-end isolated for greater flexibility and scalability.

- Session Management: Each user's interaction is managed through unique sessions, allowing for individualized queries and responses.

- Redis Cache with KNN: Used KNN algorithm with Redis cache to find similar questions already asked in session to get a faster response back.

Technologies Used

1. Flask: The back-end of our application is powered by Flask, a lightweight web framework that facilitates smooth interaction between the front-end and the chatbot service.

2. Ollama and Phi3 Models: These models form the core of our chatbot’s capabilities, enabling sophisticated language understanding and generation.

3. Chroma and Sentence Transformers: Chroma handles the vector database for document retrieval, while Sentence Transformers provide embeddings to compare and find relevant documents.

4. Redis: Used for caching responses to improve performance and reduce query times.

5. Docker: The entire application, including all its components, runs within Docker containers. This approach ensures consistent development and deployment environments, making it easy to manage dependencies and run the application locally.

6. Asynchronous Processing: Handles multiple user requests simultaneously, ensuring a smooth and efficient user experience.

How It Works

1. Document Upload: Start by uploading your documents through the front-end application. These documents are processed and stored in a vector knowledge base.

2. Knowledge Base Creation: Our system converts the document content into vector embeddings, making it searchable through the Chroma database.

3. Query Handling: When you pose a question, the chatbot uses the RAG system to retrieve relevant documents and generate a precise response.

4. Caching and Performance Optimization: Responses are cached in Redis to speed up future queries and enhance the overall performance of the system.

5. Session Management: Each session is tracked independently, ensuring personalized interactions and allowing multiple users to operate concurrently without interference.

What Can You Expect?

- Accurate Responses: The combination of advanced models and efficient retrieval systems ensures that you receive relevant and accurate answers.

- Flexible Integration: The microservices architecture allows for easy integration with various front-end frameworks and other back-end services.

- Enhanced Productivity: Quickly find and retrieve information from large volumes of documents, saving time and improving decision-making.

- Local Development: With all components running in Docker containers, you can easily set up and run the application on your local system.

Get Started

To explore the Local Copilot Chatbot Application, follow the setup instructions provided in our GitHub repository. Experience the power of a well-integrated chatbot system that understands your documents and delivers insightful answers at your fingertips.

System Used:

Medium power low RAM. However, if you can use 32GB RAM with Nvidia GPU and i7 CPU would be great and run after the first compilation.

GitHub Repo

https://github.com/dhirajpatra/ollama-langchain-streamlit

GCP Python Microservices Application

Here is a guide for building an end-to-end application using Django, Celery, DRF, Google Cloud Platform (GCP), RabbitMQ, PostgreSQL, and an API Gateway:

Project Setup:

1. Create a GCP Project:

   • Head over to the Google Cloud Console (https://console.cloud.google.com/) and create a new project or select an existing one.
   • Enable billing if you intend to use paid GCP services.

2. Install Prerequisites:

   • Ensure you have Python (version 3.6 or later recommended) and pip (package manager) installed on your development machine.
   • Install required libraries:
     Bash

     pip install django celery django-rest-framework djcelery psycopg2-binary pika google-cloud-pubsub
     

3. Set Up Virtual Environment (Optional):

   • Consider creating a virtual environment using venv or virtualenv to isolate project dependencies.
   • Activate the virtual environment using platform-specific commands.

Django Project and App Structure:

1. Create Django Project:
   • Initialize a new Django project using:
     Bash

     django-admin startproject myproject
     

2. Create Django App:
   • Within your project directory, generate a new Django app for your main functionality:
     Bash

     cd myproject
     python manage.py startapp myapp
     

Database (PostgreSQL):

1. Cloud SQL Instance:
   • In the GCP Console, navigate to Cloud SQL (https://cloud.google.com/sql) and create a Cloud SQL instance:
     • Choose PostgreSQL as the database engine.
     • Select a region and zone for deployment based on your needs.
     • Define instance configuration (machine type, storage) considering expected workload.
   • Take note of the instance connection details (host, port, username, password).

Celery Configuration:

1. Celery Settings:

   • In your project's settings.py file, add the following configuration for Celery:
     Python

     CELERY_BROKER_URL = 'amqp://<username>:<password>@<host>:<port>/<vhost>'  # Replace placeholders with RabbitMQ details (optional)
     CELERY_RESULT_BACKEND = 'djcelery.backends.database:DatabaseBackend'
     CELERY_ACCEPT_CONTENT = ['application/json']
     CELERY_TASK_SERIALIZER = 'json'
     CELERY_RESULT_SERIALIZER = 'json'
     

   • Replace <username>, <password>, <host>, <port>, and <vhost> with the connection details for your RabbitMQ broker if you plan to use it for task queuing. Otherwise, Celery can use Django's database for results.

2. Celery Tasks:

   • Within myapp/tasks.py, define your asynchronous tasks using the @celery.shared_task decorator:
     Python

     from .models import MyModel
     
     @celery.shared_task
     def process_data(data):
         # Perform long-running or CPU-intensive task with the data here
         my_model = MyModel.objects.create(data=data)
         # ... other processing logic
         return data  # You can return a result if needed
     

Django REST Framework (DRF) API:

1. DRF Installation:

   • Ensure you have Django REST Framework (DRF) installed: pip install django-rest-framework

2. Models:

   • Define data models in myapp/models.py to represent your application's data:
     Python

     from django.db import models
     
     class MyModel(models.Model):
         data = models.CharField(max_length=255)
         created_at = models.DateTimeField(auto_now_add=True)
     

3. Serializers:

   • Create serializers in myapp/serializers.py using DRF to represent models as JSON:
     Python

     from rest_framework import serializers
     
     from .models import MyModel
     
     class MyModelSerializer(serializers.ModelSerializer):
         class Meta:
             model = MyModel
             fields = '__all__'
     

4. Views:

   • Implement views in myapp/views.py using Django and DRF to handle API requests and responses:
     Python

     from rest_framework