Latest Open Source Projects
RAG_Techniques
RAG_Techniques
TLDR: This repository is a comprehensive collection of advanced Retrieval-Augmented Generation (RAG) techniques. It includes various methods for enhancing RAG systems such as query enhancement, context enrichment, advanced retrieval methods, iterative and adaptive techniques, evaluation, explainability, and advanced architectures.
Building-llama3-from-scratch
Building-llama3-from-scratch
TLDR: This repository contains code to build the LLaMA 3 language model from scratch using Python. It explains the components of LLaMA 3 such as pre-normalization using RMSNorm, SwiGLU activation function, Rotary Embeddings (RoPE), and Byte Pair Encoding (BPE) Algorithm. The code shows how to tokenize input data, create embeddings for each token, implement attention heads, self-attention, multi-head attention, SwiGLU activation function, and generate the output.
LLMs-from-scratch
LLMs-from-scratch
TLDR: This repository contains code for developing, pretraining, and finetuning a GPT-like LLM. It is the official code repository for the book 'Build a Large Language Model (From Scratch)'. The code is designed to run on conventional laptops and automatically utilizes GPUs if available. It also includes bonus materials and has specific hardware requirements.
llm-course
llm-course
TLDR: This repository contains a comprehensive course on large language models, divided into three parts: LLM Fundamentals, The LLM Scientist, and The LLM Engineer. It includes notebooks and articles related to various aspects of large language models such as fine-tuning, quantization, and building applications.