Showing posts with label docker. Show all posts
Showing posts with label docker. Show all posts

Thursday

Simple FastAPI App with Docker and Minikube

 Let's start with the simplest one. Which we can develop and test in our local system or laptop, or Mac.

✅ Simple FastAPI App with Docker and Minikube (Kubernetes)


πŸ“ Folder Structure

fastapi-k8s-demo/
├── app/
│   └── main.py
├── Dockerfile
├── requirements.txt
├── k8s/
│   ├── deployment.yaml
│   └── service.yaml

πŸ“„ app/main.py

from fastapi import FastAPI

app = FastAPI()

@app.get("/")
def read_root():
    return {"message": "Hello from FastAPI on Kubernetes!"}

πŸ“„ requirements.txt

fastapi
uvicorn

πŸ“„ Dockerfile

FROM python:3.11-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY app/ .

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

πŸ“„ k8s/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: fastapi-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: fastapi
  template:
    metadata:
      labels:
        app: fastapi
    spec:
      containers:
      - name: fastapi
        image: fastapi-demo:latest
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 8000

πŸ“„ k8s/service.yaml

apiVersion: v1
kind: Service
metadata:
  name: fastapi-service
spec:
  type: NodePort
  selector:
    app: fastapi
  ports:
    - port: 8000
      targetPort: 8000
      nodePort: 30036

πŸ§ͺ Run Locally with Minikube

# Start minikube
minikube start

# Build docker image inside minikube
eval $(minikube docker-env)
docker build -t fastapi-demo .

# Apply Kubernetes configs
kubectl apply -f k8s/deployment.yaml
kubectl apply -f k8s/service.yaml

# Access app in browser
minikube service fastapi-service

✅ FastAPI on Minikube with Ingress, Live Reload, and Persistent Volume


πŸ“ Updated Structure

fastapi-k8s-demo/
├── app/
│   └── main.py
├── Dockerfile
├── requirements.txt
├── k8s/
│   ├── deployment.yaml
│   ├── service.yaml
│   ├── ingress.yaml
│   └── pv-claim.yaml

πŸ“„ Dockerfile (with live reload)

FROM python:3.11-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY app/ .

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000", "--reload"]

πŸ“„ requirements.txt

fastapi
uvicorn[standard]

πŸ“„ k8s/deployment.yaml (with volume)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: fastapi-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: fastapi
  template:
    metadata:
      labels:
        app: fastapi
    spec:
      containers:
        - name: fastapi
          image: fastapi-demo:latest
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 8000
          volumeMounts:
            - mountPath: /app
              name: app-volume
      volumes:
        - name: app-volume
          persistentVolumeClaim:
            claimName: fastapi-pvc

πŸ“„ k8s/pv-claim.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: fastapi-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

πŸ“„ k8s/service.yaml

apiVersion: v1
kind: Service
metadata:
  name: fastapi-service
spec:
  selector:
    app: fastapi
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8000

πŸ“„ k8s/ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: fastapi-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
    - host: fastapi.local
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: fastapi-service
                port:
                  number: 80

πŸ”§ Enable Ingress and Mount App

minikube start --addons=ingress

# Use Minikube's Docker
eval $(minikube docker-env)
docker build -t fastapi-demo .

# Apply all configs
kubectl apply -f k8s/pv-claim.yaml
kubectl apply -f k8s/deployment.yaml
kubectl apply -f k8s/service.yaml
kubectl apply -f k8s/ingress.yaml

# Add /etc/hosts entry (Linux/macOS)
echo "$(minikube ip) fastapi.local" | sudo tee -a /etc/hosts

# Open in browser
http://fastapi.local/

✅ FastAPI + PostgreSQL on Minikube with Secrets & ConfigMaps


πŸ“ Project Structure

fastapi-k8s-demo/
├── app/
│   └── main.py
├── Dockerfile
├── requirements.txt
├── k8s/
│   ├── deployment.yaml
│   ├── service.yaml
│   ├── ingress.yaml
│   ├── postgres-deployment.yaml
│   ├── postgres-service.yaml
│   ├── secret.yaml
│   └── configmap.yaml

πŸ“„ requirements.txt

fastapi
uvicorn[standard]
psycopg2-binary

πŸ“„ app/main.py

import os
import psycopg2
from fastapi import FastAPI

app = FastAPI()

@app.get("/")
def root():
    return {"message": "Hello from FastAPI + Postgres!"}

@app.get("/db-check")
def db_check():
    conn = psycopg2.connect(
        host=os.getenv("DB_HOST"),
        dbname=os.getenv("DB_NAME"),
        user=os.getenv("DB_USER"),
        password=os.getenv("DB_PASSWORD"),
    )
    cur = conn.cursor()
    cur.execute("SELECT version();")
    version = cur.fetchone()
    conn.close()
    return {"postgres_version": version}

πŸ“„ Dockerfile

FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app/ .
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

πŸ“„ k8s/postgres-deployment.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgres-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres
spec:
  selector:
    matchLabels:
      app: postgres
  replicas: 1
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
        - name: postgres
          image: postgres:13
          env:
            - name: POSTGRES_DB
              valueFrom:
                configMapKeyRef:
                  name: pg-config
                  key: POSTGRES_DB
            - name: POSTGRES_USER
              valueFrom:
                secretKeyRef:
                  name: pg-secret
                  key: POSTGRES_USER
            - name: POSTGRES_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: pg-secret
                  key: POSTGRES_PASSWORD
          ports:
            - containerPort: 5432
          volumeMounts:
            - mountPath: /var/lib/postgresql/data
              name: pgdata
      volumes:
        - name: pgdata
          persistentVolumeClaim:
            claimName: postgres-pvc

πŸ“„ k8s/postgres-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: postgres
spec:
  ports:
    - port: 5432
  selector:
    app: postgres

πŸ“„ k8s/secret.yaml

apiVersion: v1
kind: Secret
metadata:
  name: pg-secret
type: Opaque
stringData:
  POSTGRES_USER: myuser
  POSTGRES_PASSWORD: mypassword

πŸ“„ k8s/configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: pg-config
data:
  POSTGRES_DB: mydb

πŸ“„ k8s/deployment.yaml (FastAPI updated)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: fastapi-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: fastapi
  template:
    metadata:
      labels:
        app: fastapi
    spec:
      containers:
        - name: fastapi
          image: fastapi-demo:latest
          ports:
            - containerPort: 8000
          env:
            - name: DB_HOST
              value: postgres
            - name: DB_NAME
              valueFrom:
                configMapKeyRef:
                  name: pg-config
                  key: POSTGRES_DB
            - name: DB_USER
              valueFrom:
                secretKeyRef:
                  name: pg-secret
                  key: POSTGRES_USER
            - name: DB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: pg-secret
                  key: POSTGRES_PASSWORD

πŸ“„ k8s/service.yaml (FastAPI)

apiVersion: v1
kind: Service
metadata:
  name: fastapi-service
spec:
  selector:
    app: fastapi
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8000

πŸ“„ k8s/ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: fastapi-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
    - host: fastapi.local
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: fastapi-service
                port:
                  number: 80

πŸ”§ Deployment Commands

minikube start --addons ingress
eval $(minikube docker-env)

# Build image
docker build -t fastapi-demo .

# Apply K8s resources
kubectl apply -f k8s/configmap.yaml
kubectl apply -f k8s/secret.yaml
kubectl apply -f k8s/postgres-deployment.yaml
kubectl apply -f k8s/postgres-service.yaml
kubectl apply -f k8s/deployment.yaml
kubectl apply -f k8s/service.yaml
kubectl apply -f k8s/ingress.yaml

# Add to /etc/hosts
echo "$(minikube ip) fastapi.local" | sudo tee -a /etc/hosts

# Open
http://fastapi.local/db-check

Hope now you have a solid understanding of how to develop a Python FastAPI application in Docker and deploy it in Kubernetes clusters.

The whole application code is here with different branches https://github.com/dhirajpatra/fastapi-k8s-demo


Friday

Deploy ML Application to Azure

Azure DevOps + Azure ML based ML application deployment tutorial:


πŸ”· Introduction: Context of Azure, Azure ML & Azure DevOps

1. Azure Overview Azure is Microsoft’s cloud computing platform offering a vast ecosystem of services for compute, storage, networking, databases, machine learning, DevOps, and more. It enables organizations to build, deploy, and manage applications and services through Microsoft-managed data centers.

Key Benefits:

  • Global scalability and reliability

  • Pay-as-you-go pricing

  • Integrated security and compliance

  • Strong ecosystem for enterprise DevOps and AI/ML workflows


2. What is Azure Machine Learning (Azure ML)? Azure Machine Learning is a cloud-based platform for training, deploying, automating, and managing machine learning models. It supports both code-first (Python SDK, CLI) and no-code (Designer, Studio) approaches.

Key Components:

  • Workspaces: Central place to manage assets and operations

  • Compute Targets: For training & inference (e.g., Azure ML Compute, AKS, ACI)

  • Datasets & Datastores: For accessing and versioning data

  • Pipelines: For orchestrating workflows (training, evaluation, deployment)

  • Endpoints: Real-time and batch deployment options


3. What is Azure DevOps? Azure DevOps is a SaaS platform providing a complete DevOps toolchain for software development and deployment. It is widely used for continuous integration (CI) and continuous deployment (CD) of applications, including ML workflows.

Core Services:

  • Azure Repos: Git repositories for version control

  • Azure Pipelines: CI/CD for building, testing, and deploying code

  • Azure Boards: Agile project planning

  • Azure Artifacts: Package management

  • Test Plans: Manual and automated testing


🎯 Purpose of This Tutorial This tutorial aims to build an end-to-end CI/CD pipeline for a machine learning application using:

  • Azure ML for model training & deployment

  • Azure DevOps for version control, build, and release automation

You'll learn how to:

  • Train ML models in Azure ML

  • Store and manage code using Azure Repos

  • Trigger training via Azure Pipelines

  • Deploy models as endpoints

  • Automate the entire ML lifecycle


✅ Step 1: Setup & Prerequisites for starting the Azure ML + Azure DevOps deployment tutorial.


πŸ”Ή Step 1: Prerequisites & Initial Setup

πŸ“Œ Objective:
Set up the Azure and DevOps environment required for ML application CI/CD deployment.


🧰 Prerequisites:

  1. Azure Subscription

  2. Azure DevOps Account

    • Sign up or log in: https://dev.azure.com

    • Create an organization and a project (e.g., ml-deployment-demo)

  3. Azure Machine Learning Workspace

    • Create a workspace from the Azure portal or CLI

    • It should include:

      • A resource group

      • A region (e.g., East US)

      • Storage Account

      • Key Vault

      • Application Insights

      • Container Registry (optional)

  4. Git & GitHub (Optional but Recommended)

    • For managing code in local repo and syncing with Azure Repos

  5. Local Setup (for development)

    • Python ≥ 3.8

    • Install Azure CLI:

      curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
      
    • Install Azure ML CLI extension:

      az extension add -n azure-cli-ml
      
    • Install Azure ML SDK:

      pip install azureml-sdk
      
  6. Azure DevOps Agent (Optional for self-hosted runs)

    • Can be added later for running jobs on your own compute


πŸ“ Output of This Step:

  • Azure DevOps project is ready

  • Azure ML workspace is created

  • Local dev environment is ready

  • CLI and SDK tools installed



πŸ–Ό️ Azure ML CI/CD Flow

This diagram shows the end-to-end ML model lifecycle in an Azure DevOps pipeline using Azure ML.

🧩 Components Explained:

  1. Create/Get Workspace

    • Use Azure CLI or Azure ML SDK to create or fetch an existing Azure ML workspace.

    • All assets (models, experiments, compute, images) are tied to this workspace.

  2. Model Training (Experiment Run)

    • Train your ML model (e.g., using scikit-learn, TensorFlow, etc.)

    • Log metrics, outputs, and register as an experiment run in Azure ML.

  3. Model Evaluation

    • Compare current trained model against a baseline/production model using evaluation metrics.

    • Decision logic: Promote model only if metrics improve.

  4. Model Registration

    • Register the trained model in Azure ML Model Registry with versioning.

    • Enables reproducibility and controlled model promotion.

  5. Scoring Image Creation (Docker)

    • Create a Docker scoring image using Azure ML for deployment.

    • This image will include the model + score.py + dependencies.


Step 2: Create Azure ML Workspace

πŸ”§ Tools:

  • Azure CLI

  • Azure ML Python SDK

πŸ“Steps:

(A) Using Azure CLI

az login
az group create --name ml-demo-rg --location eastus
az ml workspace create --name ml-demo-ws --resource-group ml-demo-rg

(B) Using Python SDK

from azureml.core import Workspace

workspace = Workspace.create(
    name='ml-demo-ws',
    resource_group='ml-demo-rg',
    create_resource_group=True,
    location='eastus'
)
workspace.write_config()  # saves config.json locally

πŸ“ Output of This Step:

  • Azure ML workspace created

  • Config file config.json saved to local project (used by SDK/CLI to auto-connect)



Step 3: Prepare Training Script & Register Experiment in Azure ML

🎯 Objective:
Train your ML model and log it as an experiment run in Azure ML.


🧱 Project Folder Structure (Suggested)

ml_project/
├── train.py
├── environment.yml  (or conda_dependencies.yml)
├── config.json      (created in Step 2)
└── outputs/         (for model artifacts)

πŸ§ͺ Training Script: train.py (Example)

from azureml.core import Run
import joblib
from sklearn.datasets import load_diabetes
from sklearn.linear_model import Ridge
from sklearn.model_selection import train_test_split

# Get run context
run = Run.get_context()

# Load data
X, y = load_diabetes(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y)

# Train model
model = Ridge(alpha=0.5)
model.fit(X_train, y_train)

# Log metrics
score = model.score(X_test, y_test)
run.log("r2_score", score)

# Save model
joblib.dump(model, "outputs/model.pkl")

πŸ“¦ Define Environment: environment.yml

name: sklearn-env
dependencies:
  - python=3.8
  - scikit-learn
  - pip
  - pip:
      - azureml-defaults

πŸš€ Submit Training Script to Azure ML

from azureml.core import Workspace, Experiment, ScriptRunConfig, Environment

# Load workspace from config.json
ws = Workspace.from_config()

# Create experiment
exp = Experiment(workspace=ws, name='train-diabetes-model')

# Set up environment
env = Environment.from_conda_specification(name='sklearn-env', file_path='environment.yml')

# Submit job
src = ScriptRunConfig(source_directory='.', script='train.py', environment=env)
run = exp.submit(config=src)
run.wait_for_completion(show_output=True)

πŸ“ Output of This Step:

  • train.py runs in Azure ML

  • Metrics (r2_score) are logged

  • Model saved to outputs/model.pkl

  • Experiment is visible in Azure ML portal



πŸ–Ό️ Azure ML Model Lifecycle (Training to Deployment)

This diagram shows the end-to-end ML lifecycle on a Linux VM agent using Azure Machine Learning:

πŸ” Overview of Steps:

  1. Get or Create Workspace
    Set up the Azure ML workspace using CLI, SDK, or UI.

  2. Train the Model
    Run your training script on the Linux VM and log results as an experiment.

  3. Evaluate Model
    Compare the new model’s performance with existing/production models.

  4. Register the Trained Model
    Store the trained model in Azure ML Model Registry for versioning and reuse.

  5. Create a Scoring Image (Docker-based)
    Build a container image using the trained model + scoring script + environment. This is used for deployment to endpoints (AKS, ACI, etc.).

The bottom section (Source Code, Configs, Output) represents all your assets used throughout this workflow.


Step 4: Register Trained Model in Azure ML

Once the training script has completed and model.pkl is saved in the outputs/ folder, we can register it.


πŸ“Œ Code to Register Model

from azureml.core import Workspace, Model

# Load workspace
ws = Workspace.from_config()

# Register model from outputs folder
model = Model.register(
    workspace=ws,
    model_path="outputs/model.pkl",  # Path to model file
    model_name="diabetes-ridge-model",  # Name for model in registry
    tags={"area": "diabetes", "type": "regression"},
    description="Ridge regression model to predict diabetes progression"
)

print("Model registered:", model.name, "version:", model.version)

πŸ“ Output of This Step:

  • Model appears in Azure ML Model Registry with versioning.

  • You can now deploy this model via ACI/AKS.




Step 5: Create Scoring Image (Inference Setup)

This step prepares the model for deployment by packaging it with a scoring script and environment in a Docker image using Azure ML.


πŸ“ What You Need:

  • Registered model

  • Scoring script (score.py)

  • Environment file (myenv.yml or CondaDependencies)


🧠 1. Create score.py

This script defines how input data is processed and predictions are returned.

import joblib
import json
from azureml.core.model import Model
from sklearn.linear_model import Ridge

def init():
    global model
    model_path = Model.get_model_path('diabetes-ridge-model')
    model = joblib.load(model_path)

def run(raw_data):
    data = json.loads(raw_data)
    prediction = model.predict([data['data']])
    return prediction.tolist()

πŸ“¦ 2. Create Environment (e.g., myenv.yml)

name: diabetes-env
dependencies:
  - python=3.8
  - scikit-learn
  - pandas
  - pip:
      - azureml-defaults

πŸ› ️ 3. Build the Inference Image

from azureml.core import Environment
from azureml.core.model import InferenceConfig
from azureml.core.webservice import AciWebservice, Webservice

# Load workspace
ws = Workspace.from_config()

# Define environment
env = Environment.from_conda_specification(name='diabetes-env', file_path='myenv.yml')

# Define inference config
inference_config = InferenceConfig(entry_script='score.py', environment=env)

# Define deployment config
deployment_config = AciWebservice.deploy_configuration(cpu_cores=1, memory_gb=1)

# Deploy as web service (next step)













Step 6: Deploy Model on ACI (Azure Container Instance)


πŸ–Ό️ 50-deployOnAci.py

This script deploys your scoring image on Azure Container Instances (ACI) using:

  • CPU: 1 core

  • Memory: 1 GB

  • Tags: area=diabetes, type=regression

  • Dynamically generated ACI name using timestamp


🧾 Deployment Code Snippet:

from azureml.core.webservice import AciWebservice, Webservice
import datetime

aci_config = AciWebservice.deploy_configuration(
    cpu_cores=1,
    memory_gb=1,
    tags={"area": "diabetes", "type": "regression"},
    description="A sample description",
)

aci_service_name = "aciwebservice" + datetime.datetime.now().strftime("%m%d%H")

service = Webservice.deploy_from_image(
    deployment_config=aci_config,
    image=image,
    name=aci_service_name,
    workspace=ws,
)

service.wait_for_deployment(show_output=True)
print("Deployed ACI Webservice: {}\nWebservice URI: {}".format(
    service.name, service.scoring_uri
))

No, I haven’t described how to create, save (.pkl), and load the model yet. Here's a quick complete snippet for that:


🧠 Train & Save ML Model (.pkl)

train_model.py

import pickle
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split

# Load dataset
X, y = load_iris(return_X_y=True)

# Train-test split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Train model
clf = RandomForestClassifier()
clf.fit(X_train, y_train)

# Save to .pkl
with open("model.pkl", "wb") as f:
    pickle.dump(clf, f)

Load .pkl in FastAPI

Already shared in the main.py:

with open("model.pkl", "rb") as f:
    model = pickle.load(f)

Let me know if you also want to:

  • Save as .joblib or .onnx

  • Add test script for the FastAPI endpoint

  • Package this into CI/CD YAML pipeline automatically

πŸ“Œ Next Step: Test ACI Webservice

Once deployed, we'll write a script to send a test request to the model endpoint (60-AciWebserviceTest.py).



πŸ’Ό Project Context

  • Project: ML-Devops

  • Pipeline: Deploy Webservice

  • Release: Release-17

  • Artifact Source: devops-for-ai-CI from branch master


πŸ“¦ Continuous Deployment Trigger

  • User: 99.user.name

  • Date/Time: 12/04/2023, 18:53

  • Trigger Type: Automatic – the release is automatically triggered when new artifacts are available.


πŸš€ Deployment Stages

1. QA - Deploy on ACI (Azure Container Instances)

  • Status: In Progress

  • Tasks: 5 out of 6 completed

  • Current Task: 50. Deploy Webservice on ACI

  • Start Time: 12/04/2023, 18:54

  • Triggered At: 18:53

  • Details:

    • Deployment is automatically triggered.

    • It’s deploying on ACI, ideal for lightweight, quick QA/staging deployments.


2. Prod - Deploy on AKS (Azure Kubernetes Service)

  • Status: Not Deployed

  • Triggered After: QA stage completes successfully.

  • Target: Full-scale production deployment using AKS for scalability and orchestration.


πŸ“‘ Logs and Tracking

  • Tabs: Summary, Commits, Work Items, View Logs

  • Associated Changes:

    • Commit ID: 32

    • Branch: master


Here’s a structured response covering both parts:


1. YAML Configuration & Pipeline Task Explanation

πŸ”§ Pipeline Tasks in Deploy Webservice

Typical tasks in a deployment stage for an ML model using Azure DevOps:

  1. Download Artifacts
    Downloads model files from previous CI pipeline (e.g., .pkl, .onnx, or .joblib model).

  2. Install Dependencies
    Sets up Python environment and required libraries using pip.

  3. Build Docker Image
    Uses a Dockerfile to containerize the FastAPI service wrapping the model.

  4. Push Docker Image to ACR
    Pushes image to Azure Container Registry.

  5. Deploy to ACI/AKS
    Uses Azure CLI or Kubernetes manifests to deploy on ACI (QA) or AKS (Prod).

  6. Post-deployment Tests (Optional)
    Runs integration tests or probes service health.


πŸ“ YAML Snippet Example

trigger:
- master

stages:
- stage: DeployWebservice
  jobs:
  - job: DeployModel
    pool:
      vmImage: 'ubuntu-latest'
    steps:
    - task: UsePythonVersion@0
      inputs:
        versionSpec: '3.9'

    - script: pip install -r requirements.txt
      displayName: 'Install dependencies'

    - task: Docker@2
      inputs:
        containerRegistry: '$(dockerRegistryServiceConnection)'
        repository: 'fastapi-ml-model'
        command: 'buildAndPush'
        Dockerfile: '**/Dockerfile'
        tags: |
          $(Build.BuildId)

    - task: AzureCLI@2
      inputs:
        azureSubscription: 'YourAzureSub'
        scriptType: 'bash'
        scriptLocation: 'inlineScript'
        inlineScript: |
          az container create \
            --resource-group your-rg \
            --name fastapi-model \
            --image youracr.azurecr.io/fastapi-ml-model:$(Build.BuildId) \
            --dns-name-label your-model-api \
            --ports 80

πŸš€ 2. FastAPI-Based Webservice to Serve ML Model

πŸ“ Folder Structure

fastapi-ml-service/
├── model.pkl
├── Dockerfile
├── main.py
├── requirements.txt

main.py Example

from fastapi import FastAPI, Request
import pickle
from pydantic import BaseModel

app = FastAPI()

# Load model
with open("model.pkl", "rb") as f:
    model = pickle.load(f)

# Request schema
class PredictRequest(BaseModel):
    feature1: float
    feature2: float
    # Add more features as per model input

@app.post("/predict")
def predict(request: PredictRequest):
    input_data = [[request.feature1, request.feature2]]
    prediction = model.predict(input_data)
    return {"prediction": prediction[0]}

requirements.txt

fastapi
uvicorn
scikit-learn
pydantic

Dockerfile

FROM python:3.9

WORKDIR /app

COPY . .

RUN pip install -r requirements.txt

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "80"]

Let me know if you need:

  • Sample model training + pickle code

  • Kubernetes manifest for AKS

  • ACI-specific deployment CLI

  • Health check integration


All images used in this tutorial are credited to Azure. Also, you can find informative tutorials on Azure DevOps here and here.

You can find many tutorials related to Fastapi and other Python frameworks on how to develop web services which can call the ML model. 

Saturday

TensorRT-Specific LLM Optimizations for Jetson (NVIDIA Edge AI)

 

πŸš€ TensorRT-Specific LLM Optimizations for Jetson (NVIDIA Edge AI)

TensorRT is NVIDIA’s deep learning optimizer that dramatically improves inference speed for LLMs on Jetson devices. It enables:
Faster inference (2-4x speedup) with lower latency.
Lower power consumption on edge devices.
Optimized memory usage for LLMs.


1️⃣ Install TensorRT & Dependencies

First, install TensorRT on your Jetson Orin/Nano:

sudo apt update
sudo apt install -y nvidia-cuda-toolkit tensorrt python3-libnvinfer

Confirm installation:

dpkg -l | grep TensorRT

2️⃣ Convert LLM to TensorRT Engine

TensorRT requires models in ONNX format before optimization.

Convert GGUF/Quantized Model → ONNX

First, convert your LLaMA/Mistral model to ONNX format:

python convert_to_onnx.py --model model.gguf --output model.onnx

(Use onnx_exporter.py from Hugging Face if needed.)


3️⃣ Optimize ONNX with TensorRT

Use trtexec to compile the ONNX model into a TensorRT engine:

trtexec --onnx=model.onnx --saveEngine=model.trt --fp16

πŸ”Ή --fp16: Uses 16-bit floating point for speed boost.
πŸ”Ή --saveEngine: Saves the optimized model as model.trt.


4️⃣ Run Inference Using TensorRT-Optimized LLM

Now, run the optimized .trt model with TensorRT:

import tensorrt as trt
import numpy as np

# Load TensorRT model
TRT_LOGGER = trt.Logger(trt.Logger.WARNING)
runtime = trt.Runtime(TRT_LOGGER)
with open("model.trt", "rb") as f:
    engine = runtime.deserialize_cuda_engine(f.read())

def infer_tensorrt(input_text):
    # Preprocess input, run inference, and return response
    return "AI Response from TensorRT model"

print(infer_tensorrt("What is Edge AI?"))

5️⃣ Deploy as a FastAPI Edge AI Agent

Run a FastAPI-based chatbot on Jetson:

from fastapi import FastAPI
import subprocess

app = FastAPI()

@app.get("/ask")
def ask(question: str):
    cmd = f'./main --engine model.trt -p "{question}" -n 100'
    response = subprocess.check_output(cmd, shell=True).decode()
    return {"response": response}

# Run API: uvicorn app:app --host 0.0.0.0 --port 8000

πŸ”₯ Benchmark TensorRT vs. CPU/GPU Performance

Compare TensorRT vs. CPU vs. GPU inference speed:

trtexec --loadEngine=model.trt --benchmark

πŸ’‘ Expected Speedup:
πŸš€ TensorRT (2-4x faster) > CUDA (cuBLAS) > CPU (Slowest)


πŸ“Œ Conclusion

TensorRT accelerates LLM inference on Jetson Edge AI.
Use ONNX + TensorRT Engine to optimize LLaMA/Mistral models.
Deploy as a FastAPI agent for real-time inference.


πŸš€ Docker Setup for TensorRT-Optimized LLM on Jetson

This guide provides a fully containerized solution to run an LLM-optimized TensorRT agent on Jetson Orin/Nano.


πŸ“¦ 1️⃣ Create Dockerfile for TensorRT LLM

Create a Dockerfile to set up TensorRT, FastAPI, and LLM inference:

# Base image with CUDA and TensorRT (JetPack version should match your Jetson)
FROM nvcr.io/nvidia/l4t-tensorrt:r8.5.2-runtime

# Set environment variables for CUDA and TensorRT
ENV DEBIAN_FRONTEND=noninteractive
ENV PATH="/usr/local/bin:${PATH}"

# Install necessary dependencies
RUN apt update && apt install -y \
    python3 python3-pip wget git \
    && rm -rf /var/lib/apt/lists/*

# Install Python dependencies
RUN pip3 install --upgrade pip
RUN pip3 install fastapi uvicorn numpy onnxruntime-gpu tensorrt

# Copy LLM model and scripts
WORKDIR /app
COPY model.trt /app/
COPY server.py /app/

# Expose API port
EXPOSE 8000

# Start FastAPI server
CMD ["uvicorn", "server:app", "--host", "0.0.0.0", "--port", "8000"]

πŸ“ 2️⃣ Create FastAPI Server (server.py)

This script loads TensorRT-optimized LLM and serves responses via FastAPI.

from fastapi import FastAPI
import tensorrt as trt
import numpy as np

app = FastAPI()

# Load TensorRT engine
TRT_LOGGER = trt.Logger(trt.Logger.WARNING)
runtime = trt.Runtime(TRT_LOGGER)

with open("model.trt", "rb") as f:
    engine = runtime.deserialize_cuda_engine(f.read())

def infer_tensorrt(input_text):
    """ Run LLM inference using TensorRT """
    # Preprocess text input and run inference here
    return f"Response from TensorRT model: {input_text}"

@app.get("/ask")
def ask(question: str):
    return {"response": infer_tensorrt(question)}


🐳 3️⃣ Build & Run Docker Container

Build the Docker Image

docker build -t jetson-trt-llm .

Run the Container

docker run --runtime nvidia --network host --rm -it jetson-trt-llm

πŸ”₯ 4️⃣ Test the Edge AI LLM API

Once the container is running, test the API:

curl "http://localhost:8000/ask?question=What is Edge AI?"

πŸ”Ή Expected Output:

{"response": "Response from TensorRT model: What is Edge AI?"}

πŸ“Œ Conclusion

Dockerized FastAPI agent running a TensorRT-optimized LLM on Jetson.
Real-time, low-latency inference with NVIDIA TensorRT acceleration.
Scalable Edge AI solution for private, offline GenAI models.


Convert Docker Compose to Kubernetes Orchestration

If you already have a Docker Compose based application. And you may want to orchestrate the containers with Kubernetes. If you are new to Kubernetes then you can search various articles in this blog or Kubernetes website.

Here's a step-by-step plan to migrate your Docker Compose application to Kubernetes:


Step 1: Create Kubernetes Configuration Files

Create a directory for your Kubernetes configuration files (e.g., k8s-config).

Create separate YAML files for each service (e.g., api.yaml, pgsql.yaml, mongodb.yaml, rabbitmq.yaml).

Define Kubernetes resources (Deployments, Services, Persistent Volumes) for each service.


Step 2: Define Kubernetes Resources

Deployment YAML Example (api.yaml)

YAML

apiVersion: apps/v1

kind: Deployment

metadata:

  name: api-deployment

spec:

  replicas: 1

  selector:

    matchLabels:

      app: api

  template:

    metadata:

      labels:

        app: api

    spec:

      containers:

      - name: api

        image: <your-docker-image-name>

        ports:

        - containerPort: 8000

Service YAML Example (api.yaml)

YAML

apiVersion: v1

kind: Service

metadata:

  name: api-service

spec:

  selector:

    app: api

  ports:

  - name: http

    port: 8000

    targetPort: 8000

  type: ClusterIP

Repeat this process for other services (pgsql, mongodb, rabbitmq).


Step 3: Configure Persistent Storage

Create Persistent Volume Claims (PVCs) for databases.

Define StorageClass for dynamic provisioning.

Persistent Volume Claim YAML Example (pgsql-pvc.yaml)

YAML

apiVersion: v1

kind: PersistentVolumeClaim

metadata:

  name: pgsql-pvc

spec:

  accessModes:

    - ReadWriteOnce

  resources:

    requests:

      storage: 5Gi

StorageClass YAML Example (storage-class.yaml)

YAML

apiVersion: storage.k8s.io/v1

kind: StorageClass

metadata:

  name: local-storage

provisioner: kubernetes.io/no-provisioner

volumeBindingMode: WaitForFirstConsumer


Step 4: Deploy to Kubernetes

Create namespace: kubectl create namespace <namespace-name>

Apply configuration files: kubectl apply -f <config-file>.yaml

Verify deployments: kubectl get deployments -n <namespace-name>

Verify pods: kubectl get pods -n <namespace-name>

Verify services: kubectl get svc -n <namespace-name>

Step 5: Expose Services

Create Ingress resources for external access.

Configure Ingress Controller (e.g., NGINX).

Ingress YAML Example (ingress.yaml)

YAML

apiVersion: networking.k8s.io/v1

kind: Ingress

metadata:

  name: api-ingress

spec:

  rules:

  - host: <your-domain>

    http:

      paths:

      - path: /

        backend:

          serviceName: api-service

          servicePort: 8000


Step 6: Monitor and Maintain

Set up monitoring tools (e.g., Prometheus, Grafana).

Implement logging solutions (e.g., Fluentd, Elasticsearch).

Regularly update images and configurations.

Implementation Tools

kubectl: Kubernetes command-line tool.

kustomize: Configuration management tool.

Helm: Package manager for Kubernetes.

Additional Tips

Test locally with Minikube or Kind before deploying to production.

Use environment variables for configuration.

Implement rolling updates for zero-downtime deployments.

This migration plan provides a solid foundation. Adjust configurations according to your specific requirements.


Say I have a Docker Compose yaml file eg.

# version: '3.8'


services:

  pgsql:

    container_name: pgsql

    image: postgres:15-alpine

    volumes:

      - postgres-data-volume:/var/lib/postgresql/data/

      - ./app/init.sql:/docker-entrypoint-initdb.d/init.sql

    ports:

      - 5433:5432

    environment:

      - POSTGRES_USER=postgres

      - POSTGRES_PASSWORD=password123

      - POSTGRES_DB=lcnc_db_dev

    networks:

      - app-network

    healthcheck:

      test: ["CMD-SHELL", "pg_isready -U postgres"]

      interval: 5s

      retries: 5


  # adminer:

  #   image: adminer

  #   container_name: pgsql-adminer

  #   ports:

  #     - 8080:8080

  #   depends_on:

  #     - pgsql

  #   networks:

  #     - app-network


  app:

    build:

      context: ./app  # Change context to the app folder

      dockerfile: Dockerfile  # Dockerfile name remains the same

    container_name: app

    env_file:

      - ./.env.dev

    environment:

      - PYTHONPATH=/app

    volumes:

      - ./app:/app

      - ./app/generated_files:/app/generated_files  # Mount for generated backend files 

      - ./app/dart_files:/app/dart_files

    ports:

      - "8001:8001"

    depends_on:

      pgsql:

        condition: service_healthy

      redis:

        condition: service_healthy

    networks:

      - app-network

    deploy:

      resources:

        limits:

          cpus: '2.00'

          memory: 4G


  redis:

    image: redis:alpine

    container_name: redis

    ports:

      - "6379:6379"

    networks:

      - app-network

    healthcheck:

      test: ["CMD", "redis-cli", "ping"]

      interval: 5s

      timeout: 3s

      retries: 3

    deploy:

      resources:

        limits:

          cpus: '1.00'

          memory: 2G

    restart: always


  redis_insight:

    image: redis/redisinsight:latest

    container_name: redis_insight

    ports:

      - "5540:5540"

    networks:

      - app-network

    depends_on:

      redis:

        condition: service_healthy


  celery_worker:

    build:

      context: ./app

      dockerfile: Dockerfile.celery-worker

    container_name: celery_worker

    environment:

      - PYTHONPATH=/app

      - BROKER_URL=redis://redis:6379/0

      - RESULT_BACKEND=redis://redis:6379/0

      - PREFETCH_MULTIPLIER=1

      - MAX_MEMORY_PER_CHILD=512

    depends_on:

      - app

      - redis

      - pgsql

    volumes:

      - ./app:/app

      - ./app/generated_files:/app/generated_files

      - ./app/dart_files:/app/dart_files

    networks:

      - app-network

    restart: unless-stopped


volumes:

  postgres-data-volume:  # Named volume for PostgreSQL data


networks:

  app-network:

    driver: bridge


Here's the equivalent Kubernetes configuration for the provided Docker Compose file:

Step 1: Create Namespace

Bash
kubectl create namespace dev-namespace

Step 2: PostgreSQL Deployment and Service

YAML
# pgsql-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: pgsql-deployment
  namespace: dev-namespace
spec:
  replicas: 1
  selector:
    matchLabels:
      app: pgsql
  template:
    metadata:
      labels:
        app: pgsql
    spec:
      containers:
      - name: pgsql
        image: postgres:15-alpine
        ports:
        - containerPort: 5432
        volumeMounts:
        - name: postgres-data-volume
          mountPath: /var/lib/postgresql/data/
        - name: init-sql-volume
          mountPath: /docker-entrypoint-initdb.d/
        env:
        - name: POSTGRES_USER
          value: "postgres"
        - name: POSTGRES_PASSWORD
          value: "password123"
        - name: POSTGRES_DB
          value: "lcnc_db_dev"
      volumes:
      - name: postgres-data-volume
        persistentVolumeClaim:
          claimName: postgres-pvc
      - name: init-sql-volume
        configMap:
          name: init-sql-cm
  strategy:
    type: Recreate

# pgsql-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: pgsql-service
  namespace: dev-namespace
spec:
  selector:
    app: pgsql
  ports:
  - name: pgsql
    port: 5432
    targetPort: 5432
  type: ClusterIP

# Create ConfigMap for init.sql
kubectl create configmap init-sql-cm --from-file=init.sql -n dev-namespace

# Create Persistent Volume Claim
# postgres-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgres-pvc
  namespace: dev-namespace
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi

Step 3: Redis Deployment and Service

YAML
# redis-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis-deployment
  namespace: dev-namespace
spec:
  replicas: 1
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
      - name: redis
        image: redis:alpine
        ports:
        - containerPort: 6379
  strategy:
    type: Recreate

# redis-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: redis-service
  namespace: dev-namespace
spec:
  selector:
    app: redis
  ports:
  - name: redis
    port: 6379
    targetPort: 6379
  type: ClusterIP

Step 4: App Deployment and Service

YAML
# app-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app-deployment
  namespace: dev-namespace
spec:
  replicas: 1
  selector:
    matchLabels:
      app: app
  template:
    metadata:
      labels:
        app: app
    spec:
      containers:
      - name: app
        image: <your-docker-image-name>
        ports:
        - containerPort: 8001
        volumeMounts:
        - name: app-volume
          mountPath: /app
        - name: generated-files-volume
          mountPath: /app/generated_files
        - name: dart-files-volume
          mountPath: /app/dart_files
        env:
        - name: PYTHONPATH
          value: "/app"
  strategy:
    type: Recreate
  volumes:
  - name: app-volume
    persistentVolumeClaim:
      claimName: app-pvc
  - name: generated-files-volume
    emptyDir: {}
  - name: dart-files-volume
    emptyDir: {}

# app-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: app-service
  namespace: dev-namespace
spec:
  selector:
    app: app
  ports:
  - name: app
    port: 8001
    targetPort: 8001
  type: ClusterIP

# Create Persistent Volume Claim
# app-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: app-pvc
  namespace: dev-namespace
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi

Step 5: Celery Worker Deployment

YAML
# celery-worker-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: celery-worker-deployment
  namespace: dev-namespace
spec:
  replicas: 1
  selector:
    matchLabels:
      app: celery-worker
  template:
    metadata:
      labels:
        app: celery-worker
    spec:
      containers:
      - name: celery-worker
        image: <your-docker-image-name>
        volumeMounts:
        - name: app-volume
          mountPath: /app
        - name: generated-files-volume
          mountPath: /app/generated_files
        - name: dart-files-volume
          mountPath: /app/dart_files
        env:
        - name: PYTHONPATH
          value: "/app"
        - name: BROKER_URL
          value: "redis://redis-service:6379/0"
        - name: RESULT_BACKEND
          value: "redis://redis-service:6379/0"
  strategy:
    type: Recreate
  volumes:
  - name: app-volume
    persistentVolumeClaim:
      claimName: app-pvc
  - name: generated-files-volume
    emptyDir: {}
  - name: dart-files-volume
    emptyDir: {}

Step 6: Apply Kubernetes Configuration

Bash
kubectl apply -f pgsql-deployment.yaml
kubectl apply -f pgsql-service.yaml
kubectl apply -f redis-deployment.yaml
kubectl apply -f redis-service.yaml
kubectl apply -f app-deployment.yaml
kubectl apply -f app-service.yaml
kubectl apply -f celery-worker-deployment.yaml

Step 7: Verify Deployments

Bash
kubectl get deployments -n dev-namespace
kubectl get pods -n dev-namespace
kubectl get svc -n dev-namespace
This Kubernetes configuration mirrors the Docker Compose file. Adjust image names, resource requests and other parameters as necessary.

Additional Tips

Use Secret resources for sensitive data (e.g., passwords).
Implement Horizontal Pod Autoscaling (HPA) for dynamic scaling.
Monitor cluster performance with Prometheus and Grafana.

Here are examples of applying Kubernetes in Google Cloud and Azure:

Google Cloud (GKE)

Step 1: Create a GKE Cluster

Create a new project: gcloud projects create <project-name>
Enable Kubernetes Engine API: gcloud services enable container.googleapis.com
Create a cluster: gcloud container clusters create <cluster-name> --zone <zone> --num-nodes 3

Step 2: Deploy Application

Create Deployment YAML file (e.g., deployment.yaml)
Apply Deployment: kubectl apply -f deployment.yaml
Expose Service: kubectl expose deployment <deployment-name> --type LoadBalancer --port 80

Step 3: Verify Deployment

Get Cluster credentials: gcloud container clusters get-credentials <cluster-name> --zone <zone>
Verify pods: kubectl get pods
Verify services: kubectl get svc

GKE Example Commands
Bash
# Create project and enable API
gcloud projects create my-project
gcloud services enable container.googleapis.com

# Create GKE cluster
gcloud container clusters create my-cluster --zone us-central1-a --num-nodes 3

# Deploy application
kubectl apply -f deployment.yaml

# Expose service
kubectl expose deployment my-app --type LoadBalancer --port 80

# Verify deployment
gcloud container clusters get-credentials my-cluster --zone us-central1-a
kubectl get pods
kubectl get svc


Azure (AKS)

Step 1: Create AKS Cluster

Create resource group: az group create --name <resource-group> --location <location>
Create AKS cluster: az aks create --resource-group <resource-group> --name <cluster-name> --node-count 3

Step 2: Deploy Application

Create Deployment YAML file (e.g., deployment.yaml)
Apply Deployment: kubectl apply -f deployment.yaml
Expose Service: kubectl expose deployment <deployment-name> --type LoadBalancer --port 80

Step 3: Verify Deployment

Get Cluster credentials: az aks get-credentials --resource-group <resource-group> --name <cluster-name>
Verify pods: kubectl get pods
Verify services: kubectl get svc
AKS Example Commands
Bash
# Create resource group and AKS cluster
az group create --name my-resource-group --location eastus
az aks create --resource-group my-resource-group --name my-aks-cluster --node-count 3

# Deploy application
kubectl apply -f deployment.yaml

# Expose service
kubectl expose deployment my-app --type LoadBalancer --port 80

# Verify deployment
az aks get-credentials --resource-group my-resource-group --name my-aks-cluster
kubectl get pods
kubectl get svc

Additional Tips
Use managed identities for authentication.
Implement network policies for security.
Monitor cluster performance with Azure Monitor or Google Cloud Monitoring.

Kubernetes Deployment YAML Example
YAML
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: my-app
        image: <your-docker-image-name>
        ports:
        - containerPort: 80

Friday

Microservices Application with Flutter Flask MongoDB RabbitMQ

A complete microservice application setup with a Flutter app, MongoDB, and RabbitMQ, along with all the necessary files and folder structure. The setup uses Docker Compose to orchestrate the services.


Folder Structure

```

microservice-app/

├── backend/

│   ├── Dockerfile

│   ├── requirements.txt

│   ├── main.py

│   └── config.py

├── frontend/

│   ├── Dockerfile

│   ├── pubspec.yaml

│   └── lib/

│       └── main.dart

├── docker-compose.yml

└── README.md

```


1. `docker-compose.yml`

```yaml

version: '3.8'


services:

  backend:

    build: ./backend

    container_name: backend

    ports:

      - "8000:8000"

    depends_on:

      - mongodb

      - rabbitmq

    environment:

      - MONGO_URI=mongodb://mongodb:27017/flutterdb

      - RABBITMQ_URI=amqp://guest:guest@rabbitmq:5672/

    networks:

      - microservice-network


  mongodb:

    image: mongo:latest

    container_name: mongodb

    ports:

      - "27017:27017"

    networks:

      - microservice-network


  rabbitmq:

    image: rabbitmq:3-management

    container_name: rabbitmq

    ports:

      - "5672:5672"

      - "15672:15672"

    networks:

      - microservice-network


  frontend:

    build: ./frontend

    container_name: frontend

    ports:

      - "8080:8080"

    depends_on:

      - backend

    networks:

      - microservice-network


networks:

  microservice-network:

    driver: bridge

```


2. Backend Service


2.1 `backend/Dockerfile`

```dockerfile

FROM python:3.9-slim


WORKDIR /app


COPY requirements.txt requirements.txt

RUN pip install -r requirements.txt


COPY . .


CMD ["python", "main.py"]

```


2.2 `backend/requirements.txt`

```txt

fastapi

pymongo

pika

uvicorn

```


2.3 `backend/config.py`

```python

import os


MONGO_URI = os.getenv('MONGO_URI')

RABBITMQ_URI = os.getenv('RABBITMQ_URI')

```


2.4 `backend/main.py`

```python

from fastapi import FastAPI

from pymongo import MongoClient

import pika

import config


app = FastAPI()


client = MongoClient(config.MONGO_URI)

db = client.flutterdb


# RabbitMQ Connection

params = pika.URLParameters(config.RABBITMQ_URI)

connection = pika.BlockingConnection(params)

channel = connection.channel()


@app.get("/")

async def read_root():

    return {"message": "Backend service running"}


@app.post("/data")

async def create_data(data: dict):

    db.collection.insert_one(data)

    channel.basic_publish(exchange='', routing_key='flutter_queue', body=str(data))

    return {"message": "Data inserted and sent to RabbitMQ"}

```


3. Frontend Service


3.1 `frontend/Dockerfile`

```dockerfile

FROM cirrusci/flutter:stable


WORKDIR /app


COPY . .


RUN flutter build web


CMD ["flutter", "run", "-d", "chrome"]

```


3.2 `frontend/pubspec.yaml`

```yaml

name: flutter_app

description: A new Flutter project.


version: 1.0.0+1


environment:

  sdk: ">=2.7.0 <3.0.0"


dependencies:

  flutter:

    sdk: flutter

  http: ^0.13.3


dev_dependencies:

  flutter_test:

    sdk: flutter

```


#### 3.3 `frontend/lib/main.dart`

```dart

import 'package:flutter/material.dart';

import 'package:http/http.dart' as http;


void main() {

  runApp(MyApp());

}


class MyApp extends StatelessWidget {

  @override

  Widget build(BuildContext context) {

    return MaterialApp(

      title: 'Flutter Demo',

      theme: ThemeData(

        primarySwatch: Colors.blue,

      ),

      home: MyHomePage(),

    );

  }

}


class MyHomePage extends StatefulWidget {

  @override

  _MyHomePageState createState() => _MyHomePageState();

}


class _MyHomePageState extends State<MyHomePage> {

  Future<void> sendData() async {

    final response = await http.post(

      Uri.parse('http://backend:8000/data'),

      body: {'key': 'value'},

    );

    print('Response status: ${response.statusCode}');

    print('Response body: ${response.body}');

  }


  @override

  Widget build(BuildContext context) {

    return Scaffold(

      appBar: AppBar(

        title: Text('Flutter Microservice App'),

      ),

      body: Center(

        child: ElevatedButton(

          onPressed: sendData,

          child: Text('Send Data to Backend'),

        ),

      ),

    );

  }

}

```


4. `README.md`

```markdown

# Microservice Application


## Overview


This is a microservice application setup consisting of a Flutter app (frontend), a FastAPI service (backend), MongoDB, and RabbitMQ. All services are orchestrated using Docker Compose.


## How to Run


1. Clone the repository:

   ```bash

   git clone https://github.com/your-repo/microservice-app.git

   cd microservice-app

   ```


2. Build and run the containers:

   ```bash

   docker-compose up --build

   ```


3. Access the services:

   - Frontend: `http://localhost:8080`

   - Backend: `http://localhost:8000`

   - RabbitMQ Management: `http://localhost:15672`

   - MongoDB: `mongodb://localhost:27017`

```


### Instructions to Run the Application

1. Ensure Docker and Docker Compose are installed on your machine.

2. Place the folder structure and files as described above.

3. Navigate to the root of the `microservice-app` folder.

4. Run `docker-compose up --build` to build and start the application.

5. Access the frontend on `http://localhost:8080`, backend on `http://localhost:8000`, and RabbitMQ Management UI on `http://localhost:15672`.


This setup provides a working microservice application with a Flutter frontend, FastAPI backend, MongoDB for storage, and RabbitMQ for messaging.

Thursday

Code Generation Engine Concept

Architecture Details for Code Generation Engine (Low-code)


1. Backend Framework:


- Python Framework:


  - FastAPI: A modern, fast (high-performance) web framework for building APIs with Python 3.6+ based on standard Python type hints.


  - SQLAlchemy: SQL toolkit and Object-Relational Mapping (ORM) library for database management.


  - Jinja2: A templating engine for rendering dynamic content.


  - Pydantic: Data validation and settings management using Python type annotations.




2. Application Structure:


- Project Root:


  - `app/`


    - `main.py` (Entry point of the application)


    - `models/`


      - `models.py` (Database models)


    - `schemas/`


      - `schemas.py` (Data validation schemas)


    - `api/`


      - `endpoints/`


        - `code_generation.py` (Endpoints related to code generation)


    - `core/`


      - `config.py` (Configuration settings)


      - `dependencies.py` (Common dependencies)


    - `services/`


      - `code_generator.py` (Logic for code generation)


    - `templates/` (Directory for Jinja2 templates)


  - `Dockerfile`


  - `docker-compose.yml`


  - `requirements.txt`




3. Docker-based Application:




#Dockerfile:


```dockerfile


# Use an official Python runtime as a parent image


FROM python:3.9-slim




# Set the working directory in the container


WORKDIR /app




# Copy the current directory contents into the container at /app


COPY . /app




# Install any needed packages specified in requirements.txt


RUN pip install --no-cache-dir -r requirements.txt




# Make port 80 available to the world outside this container


EXPOSE 80




# Define environment variable


ENV NAME CodeGenEngine




# Run app.py when the container launches


CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "80"]


```




#docker-compose.yml:


```yaml


version: '3.8'




services:


  web:


    build: .


    ports:


      - "80:80"


    environment:


      - DATABASE_URL=postgresql://user:password@db/codegen


    depends_on:


      - db




  db:


    image: postgres:12


    environment:


      POSTGRES_USER: user


      POSTGRES_PASSWORD: password


      POSTGRES_DB: codegen


    volumes:


      - postgres_data:/var/lib/postgresql/data




volumes:


  postgres_data:


```




4. Code Generation Engine:




- Template Engine:


  - Jinja2: Use templates to define the structure of the generated code.


  


- Model-Driven Development:


  - Pydantic Models: Define the models for data validation and generation logic.


  


- Code Generation Logic:


  - Implement logic in `services/code_generator.py` to translate user configurations into functional code using templates.




5. API Endpoints:


- Define API endpoints in `api/endpoints/code_generation.py` to handle user requests and trigger the code generation process.




6. Sample Endpoint for Code Generation:




```python


from fastapi import APIRouter, Depends


from app.schemas import CodeGenRequest, CodeGenResponse


from app.services.code_generator import generate_code




router = APIRouter()




@router.post("/generate", response_model=CodeGenResponse)


def generate_code_endpoint(request: CodeGenRequest):


    code = generate_code(request)


    return {"code": code}


```




7. Sample Code Generation Logic:




```python


from jinja2 import Environment, FileSystemLoader


from app.schemas import CodeGenRequest




def generate_code(request: CodeGenRequest) -> str:


    env = Environment(loader=FileSystemLoader('app/templates'))


    template = env.get_template('template.py.j2')


    code = template.render(model=request.model)


    return code


```




8. Sample Template (`template.py.j2`):




```jinja


class {{ model.name }}:


    def __init__(self{% for field in model.fields %}, {{ field.name }}: {{ field.type }}{% endfor %}):


        {% for field in model.fields %}self.{{ field.name }} = {{ field.name }}


        {% endfor %}


```


Saturday

Compare Ububtu and MacOS

 Features #Ubuntu Desktop #macOS Overall developer experience:


Ubuntu Offers a seamless, powerful platform that mirrors production environments on cloud, server, and IoT deployments. A top choice for AI and machine learning developers.


macOS Provides a user-friendly and intuitive interface with seamless integration across other Apple devices. Its well-documented resources and developer tools make it attractive for developers within the Apple ecosystem.


#Cloud development:


Ubuntu Aligns with Ubuntu Server, the most popular OS on public clouds, for simplified cloud-native development. Supports cloud-based developer tools like #Docker, LXD, MicroK8s, and #Kubernetes. Ensures portability and cost optimisation since it can run on any private or public cloud platform.


macOSRelies on Docker and other #virtualisation technologies for cloud development. Has seamless integration with iCloud services and native support for cloudbased application development.


#Server operations:


Ubuntu Offers wide support for server-side #development, including a range of supported applications and services, automation and debugging tools, and scripting languages. Offers robust security features.


macOS Provides robust support for server-side development with strong security features and a user-friendly approach, but the range of application and service support may not be as extensive as Ubuntu.


#IoT innovation:


Ubuntu Core is designed specifically for IoT and embedded devices, offering a smooth development process. The snap packaging system simplifies the creation of highly confined, self-contained applications.


macOS Does not offer a comparable IoT-focused operating system.


#AI and #machinelearning:


Ubuntu With native support for #Python, $R, and other popular AI/ML languages, developers can easily create their preferred environment. Ubuntu is the reference platform for #NVIDIA’s #CUDA, optimal for #GPU accelerated ML tasks. Popular ML libraries run efficiently on Ubuntu.


macOS Provides native support for popular AI/ ML languages such as Python and R, but doesn’t have the same level of integration with GPU-accelerated tasks. Offers robust support for ML frameworks and tools.


collected from Ubuntu.

House Based Manufacturing Micro Clustering

                                 image generated by meta ai House-based manufacturing micro-clustering in China refers to the hyper-local, v...