Posts

Showing posts with the label python

Causal Modeling

Image
                                                               generated by meta ai Yes. In fact, causal  modelling  is one of the most advanced topics in quantitative finance and is becoming increasingly important because traditional ML models (LSTM, XGBoost, Transformers) often learn  correlations , whereas causal models aim  to discover why prices move. Since I am  pursuing an  MSc in Financial Engineering  at  WQU , I'll teach it at that level, covering  theory, mathematics, and Python implementations. Learning Roadmap We'll build from scratch. Correlation vs Causation Structural Causal Models (SCM) Directed Acyclic Graphs (DAGs) Causal Discovery Algorithms Do-Calculus (Pearl) Counterfactual Prediction Causal Forecasting Applying causal models to stock prediction Building an end-t...

Scaling a Python application to serve millions of users

Scaling a Python application to serve millions of users requires moving past single-server setups and bypassing Python’s Global Interpreter Lock (GIL) through modern architecture . True scale is achieved by making the application stateless, caching aggressively, offloading heavy lifting, and ensuring the database never becomes a bottleneck. 1. Master Concurrency and Framework Selection Python handles traffic differently depending on the chosen framework and runtime strategy: Use Async Frameworks : Transition from synchronous frameworks (like standard Flask or Django) to asynchronous frameworks like FastAPI or Sanic. Async frameworks handle thousands of concurrent I/O-bound connections on a single process using an event loop. Optimize WSGI/ASGI Servers : Run applications behind multi-process workers. For Django/Flask, use Gunicorn with or workers. For FastAPI, utilize Uvicorn with a defined number of worker processes to fully utilize multi-core CPU architectures. 2. Design for Horiz...

Python code examples for implementing an ML ensemble using Redis

  Important Setup Note: RedisAI and RedisML are no longer actively maintained as of 2025/2026. Therefore, Pattern B has evolved into standard deployment patterns using lightweight modern tools (like ONNX Runtime or FastAPI workers coupled directly with Redis), ensuring your pipeline remains production-grade. Pattern A: Asynchronous Parallel Processing (Redis Streams) Best for: Heavy models (Deep Learning, large Random Forests) that need separate workers or hardware (GPUs) to compute in parallel without locking your main API. Architecture Flow [Client Request] │ ▼ ┌───────────┐ XADD ┌───────────────┐ │ FastAPI │ ────────────────>│ Redis Stream │──┐ (Model 1 Worker) │ Gateway │ │(ml:requests) │──┼─> [Model 2 Worker] └───────────┘ └───────────────┘──┘ (Model 3 Worker) │ │ │ (Polls responses via │ XADD │ ...

Maersk’s digital twin ecosystem

Maersk’s digital twin ecosystem integrates advanced algorithms, machine learning, AI, IoT sensor networks, and satellite connectivity for operational optimization, predictive analytics, and real-time decision-making; each component plays a specific technical role for their vessels and logistics platforms. Algorithms and ML Techniques Voyage Simulation Algorithms : Maersk’s digital twins simulate “ghost ship” voyages using data-driven algorithms that include time series analysis, regression models, and vessel hydrodynamics optimization; these help forecast fuel consumption, emissions, and routing efficiency before a voyage is booked. Predictive Modeling : ML models (e.g. XGBoost, Random Forest, Neural Networks) are used to estimate future cargo demand, predict maintenance needs (predictive maintenance), detect anomalies (such as abnormal sensor readings), and optimize speed and course under varying weather and market conditions.metalab. Prescriptive Analytics : Reinforcement learning an...