Skip to main content

Posts

Telegram Bot for Monitoring Summarizing and Sending Periodic Qverviews of Channel Posts

  pexel To develop a Telegram bot for monitoring, summarizing, and sending periodic overviews of channel posts, follow these steps: Step 1: Set Up Your Environment 1. Install Python : Ensure you have Python installed on your system. 2. Install Required Libraries :     ```python     pip install python-telegram-bot requests beautifulsoup4     ``` Step 2: Create the Telegram Bot 1. Create a Bot on Telegram : Talk to [@BotFather](https://telegram.me/BotFather) to create a new bot. Note the API token provided. Step 3: Develop the Bot 1. Monitor Telegram Channels :     ```python     from telegram import Bot, Update     from telegram.ext import Updater, CommandHandler, MessageHandler, Filters, CallbackContext     import requests     from bs4 import BeautifulSoup     TOKEN = 'YOUR_TELEGRAM_BOT_TOKEN'     CHANNELS = ['@example_channel_1', '@example_channel_2']     SUMMARY_PERIOD = 6...

The new feature in Python 3.13 allowing CPython to run without the Global Interpreter Lock

Understanding Free-threaded CPython and Parallel Execution The new feature in Python 3.13, allowing CPython to run without the Global Interpreter Lock (GIL), is significant for improving parallelism in Python programs. Here’s a detailed explanation along with a code example to illustrate how it works and the benefits it brings: Key Points 1. Disabling the GIL : CPython can be built with the `--disable-gil` option, allowing threads to run in parallel across multiple CPU cores. 2. Parallel Execution : This enables full utilization of multi-core processors, leading to potential performance improvements for multi-threaded programs. 3. Experimental Feature : This is still experimental and may have bugs and performance trade-offs in single-threaded contexts. 4. Optional GIL : The GIL can still be enabled or disabled at runtime using the `PYTHON_GIL` environment variable or the `-X gil` command-line option. 5. C-API Extensions : Extensions need to be adapted to work without the GIL. Demo Code...

Preparing a Dataset for Fine-Tuning Foundation Model

  I am trying to preparing a Dataset for Fine-Tuning on Pathology Lab Data. 1. Dataset Collection    - Sources:  Gather data from pathology lab reports, medical journals, and any other relevant medical documents.    - Format:  Ensure that the data is in a readable format like CSV, JSON, or text files. 2. Data Preprocessing    - Cleaning:  Remove any irrelevant data, correct typos, and handle missing values.    - Formatting:  Convert the data into a format suitable for fine-tuning, usually pairs of input and output texts.    - Example Format:      - Input:  "Patient exhibits symptoms of hyperglycemia."      - Output:  "Hyperglycemia" 3. Tokenization    - Tokenize the text using the tokenizer that corresponds to the model you intend to fine-tune. Example Code for Dataset Preparation Using Pandas and Transformers for Preprocessing 1. Install Required Libraries: ...

Develop a Customize LLM Agent

  Photo by MART PRODUCTION at pexel If you’re interested in customizing an agent for a specific task, one way to do this is to fine-tune the models on your dataset.  For preparing dataset you can see this article . 1. Curate the Dataset - Using NeMo Curator:   - Install NVIDIA NeMo: `pip install nemo_toolkit`   - Use NeMo Curator to prepare your dataset according to your specific requirements. 2. Fine-Tune the Model - Using NeMo Framework:   1. Setup NeMo:      ```python      import nemo      import nemo.collections.nlp as nemo_nlp      ```   2. Prepare the Data:      ```python      # Example to prepare dataset      from nemo.collections.nlp.data.text_to_text import TextToTextDataset      dataset = TextToTextDataset(file_path="path_to_your_dataset")      ```   3. Fine-Tune the Model:      ```python ...

Code Auto Completion with Hugging Face LangChain and Phi3 SLM

  Photo by energepic.com at pexel You can create your own coding auto-completion co-pilot using Hugging Face LangChain and Phi3 SLM ! Here's a breakdown of the steps involved: 1. Setting Up the Environment: Install the required libraries: Bash pip install langchain transformers datasets phi3 Download the Phi3 SLM model: Bash from transformers import AutoModelForSeq2SeqLM model_name = "princeton-ml/ph3_base" model = AutoModelForSeq2SeqLM.from_pretrained(model_name) 2. Preprocessing Code for LangChain: LangChain provides a AutoTokenizer class to preprocess code. Identify the programming language you want to support and install the corresponding tokenizer from Hugging Face. For example, for Python: Bash from langchain.llms import AutoTokenizer tokenizer = AutoTokenizer.from_pretrained( "openai/gpt-code-code" ) Define a function to preprocess code into LangChain format. This might involve splitting the code into tokens, adding special tokens (e.g., start/e...