RAG_Techniques

RAG_Techniques

TLDR: This repository is a comprehensive collection of advanced Retrieval-Augmented Generation (RAG) techniques. It includes various methods for enhancing RAG systems such as query enhancement, context enrichment, advanced retrieval methods, iterative and adaptive techniques, evaluation, explainability, and advanced architectures.

2024-07-13 Github

Building-llama3-from-scratch

Building-llama3-from-scratch

TLDR: This repository contains code to build the LLaMA 3 language model from scratch using Python. It explains the components of LLaMA 3 such as pre-normalization using RMSNorm, SwiGLU activation function, Rotary Embeddings (RoPE), and Byte Pair Encoding (BPE) Algorithm. The code shows how to tokenize input data, create embeddings for each token, implement attention heads, self-attention, multi-head attention, SwiGLU activation function, and generate the output.

2024-05-27 Github

LLMs-from-scratch

38,600
@rasbt

LLMs-from-scratch

TLDR: This repository contains code for developing, pretraining, and finetuning a GPT-like LLM. It is the official code repository for the book 'Build a Large Language Model (From Scratch)'. The code is designed to run on conventional laptops and automatically utilizes GPUs if available. It also includes bonus materials and has specific hardware requirements.

2023-07-23 Github

llm-course

llm-course

TLDR: This repository contains a comprehensive course on large language models, divided into three parts: LLM Fundamentals, The LLM Scientist, and The LLM Engineer. It includes notebooks and articles related to various aspects of large language models such as fine-tuning, quantization, and building applications.

2023-06-17 Github