A complete roadmap to master LLMs for absolute beginners to advanced
📘 Support this project
This repository is free and open source. The same content is also available as a published book: LLM Roadmap: From Beginner to Advanced.
If you want to support the work behind this roadmap, you can buy the book version.
Large Language Models (LLMs) are now an important part of modern AI systems. They are used in chatbots, search systems, coding assistants, data analysis tools, agents, and many other applications. Because of this, LLM-related skills are becoming increasingly important for data scientists, machine learning engineers, AI engineers, and software developers working in the AI field.
This repository provides a structured roadmap for learning LLMs from beginner to advanced level. The goal is to help you understand the core concepts, learn how LLMs are built and adapted, and practice building real applications with them.
The roadmap is divided into four main sections.
The first section covers the foundations of Large Language Models, including LLM architecture, transformers, attention mechanisms, tokenization, embeddings, and other core concepts needed to understand how these models work.
The second section focuses on building and training LLMs. It covers dataset preparation, fine-tuning, evaluation, quantization, alignment techniques such as RLHF, and the importance of staying updated with new model releases and research.
The third section moves from model understanding to application development. It covers prompt engineering, vector databases, Retrieval-Augmented Generation (RAG), running LLMs locally, deployment, inference optimization, LLMOps, and security considerations for production systems.
The final section is focused on portfolio building. It includes project ideas and guided projects that can help you apply what you learned and demonstrate your skills through practical work.
Each section builds on the previous one, moving from fundamentals to training, then to production applications, and finally to portfolio projects. By the end of the roadmap, you should have a clear learning path and a practical understanding of how to work with LLMs across different stages of the development lifecycle.
- Part I: LLM Basics & Architecture
- Part II: Building & Training LLM From Scratch
- 1. Best Resources on Building Datasets to Train LLMs
- 2. Practical Guide to LLM Fine-Tuning
- 3. Best Resources to Learn & Understand Evaluating LLMs
- 4. Overview of LLM Quantization Techniques
- 5. Top Resources to Learn & Understand RLHF & LLM Alignment
- 6. Best Resources to Build & Understand Vision Language Models
- 7. How to Stay Updated with LLM Research & Industry News?
- Part III: Building LLMs Production Applications
- 8. Best Resources to Learn Prompt Engineering
- 9. Top Resources to Master Vector Databases & Building a Vector Storage
- 10. Master RAG: From Basic Level to Advanced
- 11. Learning Resources to Master LLM Agents
- 12. Getting Started with LLM Inference Optimization
- 13. What is LLMOps and How to Get Started With It
- 14. Securing LLMs: Best Learning & Educational Resources
- 15. Deploying LLMs: Top Learning & Educational Resources
- 16. Five Free Tools to Run LLMs Locally on Your Laptop
- 17. Master MCP: The Best Free Learning Resources
- Part IV: Building Your LLM Portfolio Projects
In the first section, you'll grasp the foundational concepts of Large Language Models (LLMs) and their core architectures, focusing on transformers, attention mechanisms, and tokenization. Key resources include Andrej Karpathy's "Let's Build the GPT Tokenizer," Jay Alammar's "The Illustrated Transformer" and "The Illustrated GPT-2," and 3Blue1Brown's "Visual Intro to Transformers." You'll also explore "nanoGPT" by Karpathy, "Attention? Attention!" by Lilian Weng, various decoding strategies, Karpathy's "Intro to Large Language Models," and top practical and theoretical courses on LLMs. This section provides a blend of theoretical and useful insights, preparing you for the next sections.
- The Illustrated Transformer by Jay Alammar
- The Illustrated GPT-2 by Jay Alammar
- Attention? Attention! by Lilian Weng
- Decoding Strategies in LLMs by Maxime Lebonne
- Let's build the GPT Tokenizer by Andrej Karpathy
- nanoGPT by Andrej Karpathy
- Intro to Large Language Models by Andrej Karpathy
- Visual Intro to Transformers by 3Blue1Brown
- Generative AI with Large Language Models
- Full Stack LLM Bootcamp
- Training & Fine-Tuning LLMs for Production
- H2O.ai LLM Learning Path
The second section guides you through the complete process of training and fine-tuning a Large Language Model (LLM) from scratch, covering every crucial step from data preparation to ensuring optimal model performance. This section begins with the best resources for building datasets to train LLMs, providing comprehensive guidance on collecting, cleaning, and organizing data for effective model training.
Next, you will delve into mastering the fine-tuning process with top learning resources, exploring techniques and strategies to adapt pre-trained models to specific tasks or domains. The section also includes 14 free LLM fine-tuning notebooks, offering practical, hands-on experience with fine-tuning processes. Evaluating LLMs is a critical aspect covered in this section, with the best resources to learn and understand various evaluation metrics and methodologies, ensuring your model's performance meets the desired standards. Additionally, you will gain an overview of LLM quantization techniques, which help in optimizing models for efficiency and speed, along with resources for mastering each technique.
Understanding Reinforcement Learning from Human Feedback (RLHF) and LLM alignment is another key component, with top resources provided to deepen your knowledge in aligning models with human values and preferences. Finally, this section offers insights on how to stay updated with the latest LLM research and industry news, ensuring you remain at the forefront of advancements in the field.
By the end of this section, you will have a comprehensive understanding of how to build, train, fine-tune, evaluate, and optimize LLMs, equipped with the knowledge and practical skills to develop high-performing models tailored to specific needs.
Large language models (LLMs), such as OpenAI’s GPT series and Google’s Bard, are driving profound technological changes. Recently, with the emergence of open-source large model frameworks like LlaMa and ChatGPT, training an LLM is no longer the exclusive domain of resource-rich companies. Training LLMs by small organizations or individuals has become an important interest in the open-source community, with some notable works including Alpaca, Vicuna, and Luotuo.
In addition to large model frameworks, large-scale and high-quality training corpora are also essential for training large language models. Currently, relevant open-source corpora in the community are still scattered. Therefore, this section aims to introduce and collect high-quality resources to learn how to build training datasets for LLM applications.
LLM datasets are extensive sets of text used to train large language models. These datasets typically contain texts in multiple languages, topics, and styles, used to train models to predict and generate text related to given input text. They are commonly employed for various natural language processing tasks like machine translation, summarization, question-answering systems, and more. The dataset hubs contain open-source datasets that are pivotal in training or fine-tuning many LLMs that ML engineers use today.
Resources:
- LLMDataHub: Awesome Datasets for LLM Training
- How to Use HuggingFace’s Datasets
- Open-Sourced Training Datasets for Large Language Models (LLMs)
Training a chatbot LLM that can follow human instructions effectively requires access to high-quality datasets that cover a range of conversation domains and styles. In this section, we provide a curated collection of resource datasets specifically designed for building an instruction dataset for instruction-tuning LLM.
Resources:
- How to Fine-Tune an LLM Part 1: Preparing a Dataset for Instruction Tuning
- How I created an instruction dataset using GPT 3.5 to fine-tune Llama 2 for news classification
- Dataset creation for fine-tuning LLM
- How to Generate Instruction Datasets from Any Documents for LLM Fine-Tuning
Enhancing the performance of the LLM and RAG systems depends on efficiently processing diverse unstructured data sources. In this section, you’ll learn techniques for representing all sorts of unstructured data, like text, images, and tables, from many different sources and implement them to extend your LLM RAG pipeline to include Excel, Word, PowerPoint, PDF, and EPUB files.
Resources:
- How to Build an Effective Data Collection and Processing Strategy for LLM Training
- Preprocessing Unstructured Data for LLM Applications
Large language models (LLMs) have transformed the field of natural language processing with their advanced capabilities and highly sophisticated solutions. These models, trained on massive datasets of text, perform a wide range of tasks, including text generation, translation, summarization, and question-answering. But while LLMs are powerful tools, they’re often incompatible with specific tasks or domains. Fine-tuning allows users to adapt pre-trained LLMs to more specialized tasks. By fine-tuning a model on a small dataset of task-specific data, you can improve its performance on that task while preserving its general language knowledge. In this section, we will provide the best learning resource to learn what fine-tuning is, how it works, and how fine-tuning LLMs can significantly improve model performance, reduce training costs, and enable more accurate and context-specific results. Also, these resources will cover different fine-tuning techniques and applications to show how fine-tuning has become a critical component of LLM-powered solutions.
LLMs are trained on massive datasets of text and can perform a wide range of tasks, including text generation, translation, summarization, and question-answering. But while LLMs are powerful tools, they’re often incompatible with specific tasks or domains. Fine-tuning allows users to adapt pre-trained LLMs to more specialized tasks. By fine-tuning a model on a small dataset of task-specific data, you can improve its performance on that task while preserving its general language knowledge. For example, a Google study found that fine-tuning a pre-trained LLM for sentiment analysis improved its accuracy by 10 percent.
In this section, you will explore how fine-tuning LLMs can significantly improve model performance, reduce training costs, and enable more accurate and context-specific results. You will also learn the different fine-tuning techniques and applications to show how fine-tuning has become a critical component of LLM-powered solutions.
Learning Resources:
- The Novice’s LLM Training Guide
- Fine-Tuning LLMs: Overview, Methods, and Best Practices
- Finetuning Large Language Models
Transfer learning plays a crucial role in the development of large language models such as GPT-3 and BERT. It is an ML technique in which a model trained on a certain task is used as a starting point for a distinct but similar task. The idea behind transfer learning is that the knowledge gained by a model from solving one problem can be leveraged to help solve another problem. However, with the parameter count of large language models reaching trillions, fine-tuning the entire model has become computationally expensive and often impractical.
In response, the focus has shifted towards in-context learning, where the model is provided with prompts for a given task and returns in-context updates. However, inefficiencies like processing the prompt each time the model makes a prediction and its poor performance at times make it a less favorable choice.
This is where Parameter-efficient Fine-tuning (PEFT) comes in as an alternative paradigm to prompting. PEFT aims to fine-tune only a small subset of the model’s parameters, achieving comparable performance to full fine-tuning while significantly reducing computational requirements. These learning resources will introduce you to the PEFT method in detail, exploring its benefits and how it has become an efficient way to fine-tune LLMs on downstream tasks. Also, it will discuss different methods of the PEFT with practical examples of each.
Learning Resources:
- Optimizing Pre-trained Models: A Guide to Parameter-Efficient Fine-Tuning (PEFT)
- QLoRA paper explained (Efficient Finetuning of Quantized LLMs)
- QLoRA — How to Fine-tune an LLM on a Single GPU (w/ Python Code)
- LoRA Fine-tuning & Hyperparameters Explained (in Plain English)
- Fine-Tuning Mistral-7B with LoRA (Low-Rank Adaptation)
- Finetuning LLMs with LoRA and QLoRA: Insights from Hundreds of Experiments
Fine-tuning large language models (LLMs) has become a crucial skill for NLP practitioners, enabling customization and improved performance across various tasks. This article introduces 14 free Colab notebooks that provide hands-on experience in fine-tuning LLMs. From efficient training methodologies like LoRA and Hugging Face to specialized models such as Llama, Guanaco, and Falcon, each notebook explores unique aspects of the fine-tuning process. Advanced techniques like PEFT Finetune, Bloom-560m-tagger, and Meta_OPT-6–1b_Model offer insights into state-of-the-art approaches.
Whether you’re interested in GPT-Neo-X, MPT-Instruct-30B, or Microsoft Phi 15B, these notebooks cover a diverse range of LLMs, making them suitable for both beginners and experienced practitioners. Delve into custom dataset training, self-supervised methods, and RLHF techniques, gaining a comprehensive understanding of fine-tuning.
This section provides a roadmap to navigate these notebooks, making it an essential read for anyone keen on mastering the art of fine-tuning large language models.
- Fine-Tuning Large Language Models with LoRA and Hugging Face
- Fine-Tuning Llama 2 Model in a Colab Notebook
- Guanaco Chatbot Demo with LLaMA-7B Model
- PEFT Finetune-Bloom-560m-tagger
- Fine-Tuning Meta_OPT-6–1b Model with bnb_peft
- Fine-Tuning Falcon-7b with BNB Self-Supervised Training
- Fine-Tuning LLaMa2 with QLoRa
- Stable Vicuna 1 3B-8bit in Google Colab
- GPT-Neo-X 20B bnb2bit Training
- MPT-Instruct-30B Model Training
- RLHF Training for Custom Dataset for Any Model
- Fine-Tuning Microsoft Phi 1.5 On Custom Dataset
- Fine-Tuning OpenAI GPT3.5 Turbo
- Finetuning Mistral-7b using Autotrain Advanced
As LLMs continue to play a vital role in both research and daily use, their evaluation becomes increasingly critical, not only at the task level but also at the societal level, for a better understanding of their potential risks. Over the past years, significant efforts have been made to examine LLMs from various perspectives. This section presents a comprehensive set of resources that will help you understand LLM evaluation, starting from what to evaluate, where to evaluate, and how to evaluate.
With the rapid advancement and integration of large language models (LLMs) in business workflows, ensuring these models are reliable and efficient has become critical. This need underscores the significance of understanding and deploying robust evaluation and benchmarking techniques for successful model implementation.
LLMs are evaluated and benchmarked on various tasks such as language generation, translation, reasoning, summarization, question-answering, and relevance. A representative set of evaluations helps build well-rounded, robust, and secure models across different dimensions and detects any regressions over a period of time.
In this section, we explore the nuances of evaluation metrics, the significance of LLM benchmarks in quantifying model performance, and the challenges associated with building standardized metrics. We also touch upon the latest trends in benchmarking and provide a comprehensive guide on building effective evaluation protocols.
Resources:
- Understanding LLM Evaluation and Benchmarks: A Complete Guide
- Decoding LLM Performance: A Guide to Evaluating LLM Applications
- A Survey on Evaluation of LLMs
- Evaluating and Debugging Generative AI
It is evident that merely training LLMs is not sufficient. Thus, the question arises: How can we confidently assert that LLM ‘A’ (with ’n’ number of parameters) is superior to LLM ‘B’ (with ‘m’ parameters)? Or is LLM ‘A’ more reliable than LLM ‘B’ based on quantifiable, reasonable observations? There needs to be a standard to benchmark LLMs, ensuring they are ethically reliable and factually performant.
In this section, you will learn about the current evaluation paradigm and understand the terminology of LLM benchmarking/evaluation. You will also learn about some prominent research on evaluating benchmarking and comparing LLMs on various tasks or scenarios.
Resources:
In the era of artificial intelligence and machine learning, evaluating the performance of models is crucial for their development and improvement. Large Language Models (LLMs) have shown incredible capabilities in generating human-like text, and their application has been extended to code generation. In this section, you will explore different LLM evaluation methods for different use cases. Starting with traditional ones, such as BLEU, to code generation evaluation metrics, such as HumanEval.
Resources:
- BLEU at your own risk by Rachael Tatman
- Perplexity of fixed-length models
- HumanEval: Decoding the LLM Benchmark for Code Generation
Following the great success of ChatGPT, there has been a proliferation of open-source large language models that are finetuned to follow instructions. These models are capable of providing valuable assistance in response to users’ questions/prompts. Notable examples include Alpaca and Vicuna, based on LLaMA, and OpenAssistant and Dolly, based on Pythia.
Despite the constant release of new models every week, the community faces a challenge in benchmarking these models effectively. Benchmarking LLM assistants is extremely challenging because the problems can be open-ended, and it is very difficult to write a program to automatically evaluate the response quality. In this case, we typically have to resort to human evaluation based on pairwise comparison. In this section, you will learn about the Elo rating system, which is a widely used rating system in chess and other competitive games. The Elo rating system is promising to provide the desired property mentioned above.
Resources:
Retrieval Augmented Generation (RAG) stands out as one of the most popular use cases of large language models (LLMs). This method facilitates the integration of an LLM with an organization’s proprietary data. Therefore it is important to evaluate and track experimentation to improve your RAG pipeline’s performance. Also, to understand the RAG triad: Context Relevance, Groundedness, and Answer Relevance, which are methods to evaluate the relevance and truthfulness of your LLM’s response.
Resources:
When building applications with generative AI, model behavior is less predictable than traditional software. That’s why systematic testing can make an even bigger difference in saving you development time and cost. Continuous integration is a key part of LLMOps, which is the practice of making small changes to software in development and thoroughly testing them to catch issues early when they are easier to fix. With a robust automated testing pipeline, you’ll be able to isolate bugs before they accumulate — when they’re easier and less costly to fix. Automated testing lets your team focus on building new features so that you can iterate and ship products faster.
Resources:
Model Quantization enhances the efficiency of large language models (LLMs) by representing their parameters in low-precision data types. This article presents an overview of LLM quantization techniques and resources for learning each of them. This section covers different quantization methods, including GGUF, AWQ, PTQ, GPTQ, and QAT, elucidating their mechanisms and applications in LLM optimization. Each sub-section provides learning resources, including tutorials, specifications, and practical guides, facilitating a deeper understanding of the quantization techniques. This section serves as a comprehensive guide for individuals interested in exploring LLM quantization, offering insights into various techniques and resources for continued learning and professional development.
Model Quantization is a topic that has been gaining popularity recently. The concept of quantization in AI or specifically neural networks, is a technique to represent the weights, biases, and activations in low-precision data types like 8-bit integer (int8) instead of the usual 32-bit floating point (float32). The two most common quantization cases are float32 -> float16 and float32 -> int8. In this section, you will be introduced to model quantization and what are the main techniques for it:
Learning Resources:
- A Guide to Quantization in LLMs
- Introduction to Weight Quantization
- LLM Quantization | GPTQ | QAT | AWQ | GGUF | GGML | PTQ
- Which Quantization Method is Right for You? (GPTQ vs. GGUF vs. AWQ)
- Model Quantization Methods In TensorFlow Lite
- ExLlamaV2: The Fastest Library to Run LLMs
- Democratizing LLMs: 4-bit Quantization for Optimal LLM Inference
GGML is a C library focused on machine learning. It was created by Georgi Gerganov, which is what the initials “GG” stand for. This library not only provides foundational elements for machine learning, such as tensors but also a unique binary format to distribute LLMs. This format recently changed to GGUF. This new format is designed to be extensible so that new features don’t break compatibility with existing models. It also centralizes all the metadata in one file, such as special tokens, RoPE scaling parameters, etc. In short, it answers a few historical pain points and should be future-proof. For more information, you can read the specification at this address. GGML was designed to be used in conjunction with the llama.cpp library, also created by Georgi Gerganov. The library is written in C/C++ for efficient inference of Llama models. It can load GGML models and run them on a CPU. Originally, this was the main difference with GPTQ models, which are loaded and run on a GPU. However, you can now offload some layers of your LLM to the GPU with llama.cpp. To give you an example, there are 35 layers for a 7b parameter model. This drastically speeds up inference and allows you to run LLMs that don’t fit in your VRAM. Learning Resources:
Learning Resources:
AWQ takes the concept of weight quantization to the next level by considering the activations of the model during the quantization process. In traditional weight quantization, the weights are quantized independently of the data they process. In AWQ, the quantization process takes into account the actual data distribution in the activations produced by the model during inference. Here’s how AWQ works:
- Collect Activation Statistics: During a calibration phase, a subset of the data is used to collect statistics on the activations produced by the model. This involves running the model on this data and recording the range of values and the distribution of activations.
- Searching Weight Quantization Parameters: Weights are quantized by taking the activation statistics into account. Concretely, we perform a space search for quantization parameters (e.g., scales and zero points), to minimize the distortions incurred by quantization on output activations. As a result, the quantized weights can be accurately represented with fewer bits.
- Quantizing: With the quantization parameters in place, the model weights are quantized using a reduced number of bits.
Learning Resources:
- AWQ for LLM Quantization
- Understanding Activation-Aware Weight Quantization (AWQ): Boosting Inference Serving Efficiency in LLMs
- How to Quantize an LLM with GGUF or AWQ
Post-training quantization computes the scale after the network has been trained. A representative dataset is used to capture the distribution of activations for each activation tensor, then this distribution data is used to compute the scale value for each tensor. Each weight distribution is used to compute the weight scale.
Learning Resources:
- Post-training Quantization
- Diving deeper into Quantization Realm: Post-Training Magic (PTQ)
- Post Training Quantization with OpenVINO Toolkit
GPTQ is a post-training quantization ( PTQ) method to make the model smaller with a calibration dataset. The idea behind GPTQ is very simple: it quantizes each weight by finding a compressed version of that weight, that will yield a minimum mean squared error. The GPTQ algorithm requires calibrating the quantized weights of the model by making inferences on the quantized model.
The effectiveness of quantization greatly depends on the samples for evaluating and refining their quality. These samples serve as a basis for comparing the outputs of the original and quantized models. By using a higher number of samples, the potential for precise and impactful comparisons increases, subsequently enhancing the quality of quantization.
Learning Resources:
- WTH is LLM quantization? 4-bit GPTQ?
- LLM Quantization w/ QLoRA, GPTQ and Llamacpp, LLama 2
- Optimize open LLMs using GPTQ and Hugging Face Optimum
- 4-bit Quantization with GPTQ
Quantization Aware Training (QAT) aims at computing scale factors during training. Once the network is fully trained, Quantize (Q) and Dequantize (DQ) nodes are inserted into the graph following a specific set of rules. The network is then further trained for a few epochs in a process called Fine-Tuning. Q/DQ nodes simulate quantization loss and add it to the training loss during fine-tuning, making the network more resilient to quantization. In other words, QAT can better preserve accuracy when compared to PTQ.
Learning Resources:
Reinforcement Learning from Human Feedback (RLHF) has become one of the main building blocks of building chatbots and large language applications. This section aims to provide you with a curated list of top resources that will help you learn and understand RLHF in depth. Whether you are a researcher, developer, or simply curious about this exciting field, this compilation of blogs, videos, talks, and research papers will serve as a comprehensive guide on RLHF.
Reinforcement Learning from Human Feedback (RLHF) is a branch of machine learning that combines reinforcement learning (RL) algorithms with human guidance or feedback to improve the learning process. In RLHF, instead of relying solely on an environmental reward signal, the learning agent interacts with human experts who provide feedback or demonstrations to guide the learning process. The primary motivation behind RLHF is to enable machines to learn complex tasks more efficiently and effectively by leveraging human expertise. While RL algorithms can learn from trial and error, they can require a large number of interactions with the environment to achieve desirable performance. By incorporating human feedback, RLHF aims to reduce the number of interactions needed and accelerate the learning process. RLHF is used in the development of chatbots to enhance their performance and improve their ability to interact with users. Here are some reasons why RLHF is used in developing chatbots:
- User Satisfaction: Chatbots aim to provide helpful and engaging conversations with users. By incorporating RLHF, chatbots can learn from human feedback and adapt their responses based on user preferences, leading to more satisfying interactions. Human feedback helps the chatbot understand what kind of responses are desirable and how to improve over time.
- Rapid Learning: Training chatbots solely through traditional methods, such as rule-based systems or supervised learning, can be time-consuming and limited in their ability to handle diverse user inputs. RLHF enables chatbots to learn directly from interactions with human experts, reducing the reliance on large amounts of pre-existing data. This accelerates the learning process and allows chatbots to quickly adapt to new situations.
- Handling Uncertainty: Chatbot conversations can often involve ambiguity and uncertainty. RLHF provides a mechanism for chatbots to seek clarification and guidance from human experts when faced with challenging or unfamiliar user inputs. This helps the chatbot make more informed decisions and provide accurate responses, even in uncertain situations.
- Personalization: Chatbots that can understand and adapt to individual user preferences can deliver a more personalized experience. RLHF allows chatbots to learn from user feedback and tailor their responses based on individual preferences, improving user satisfaction and engagement. This personalization enhances the chatbot’s ability to understand and fulfill user needs.
- Continuous Improvement: Chatbots can benefit from continuous learning and improvement based on ongoing user interactions. RLHF enables chatbots to receive feedback from users in real-time, helping them refine their responses and behavior. This iterative learning process allows chatbots to continually enhance their performance and adapt to evolving user requirements.
- Ethical Considerations: Chatbots that rely solely on pre-existing data may inherit biases or produce inappropriate responses. RLHF provides an opportunity to incorporate human guidance and ensure that the chatbot’s behavior aligns with ethical standards. Human experts can help shape the chatbot’s responses, ensuring they are fair, unbiased, and respectful.
1. Important Blogs
- How RLHF actually works by Nathan Lambert: This article will build the intuition behind RLHF and how and why it works simply.
- RLHF: Reinforcement Learning from Human Feedback by Chip Huyen: How exactly does RLHF work? Why does it work? This post will discuss the answers to those questions with a focus on theoretical details.
- StackLLaMA: A hands-on guide to train LLaMA with RLHF by HuggingFace: In this blog post, you will be shown all the steps involved in training a LlaMa model to answer
2. Important Videos & Talks
- Reinforcement Learning from Human Feedback: From Zero to ChatGPT: In this talk, speakers will cover the basics of Reinforcement Learning from Human Feedback (RLHF) and how this technology is being used to enable state-of-the-art ML tools like ChatGPT. Most of the talk will be an overview of the interconnected ML models and cover the basics of Natural Language Processing and RL that one needs to understand how RLHF is used on large language models. It will conclude with an open question in RLHF.
- State of GPT by Andrej Karpathy: In this talk, you will learn about the training pipeline of GPT assistants like ChatGPT, from tokenization to pretraining, supervised finetuning, and Reinforcement Learning from Human Feedback (RLHF). In addition to that, you will dive deeper into practical techniques and mental models for the effective use of these models, including prompting strategies, finetuning, the rapidly growing ecosystem of tools, and their future extensions.
3. Important Research Papers
- Deep reinforcement learning from human preferences by OpenAI: In this work, the authors explore goals defined in terms of (non-expert) human preferences between pairs of trajectory segments. They show that this approach can effectively solve complex RL tasks without access to the reward function, including Atari games and simulated robot locomotion, while providing feedback on less than one percent of our agent’s interactions with the environment. This reduces the cost of human oversight far enough that it can be practically applied to state-of-the-art RL systems. To demonstrate the flexibility of this approach, they show that we can successfully train complex novel behaviors with about an hour of human time. These behaviors and environments are considerably more complex than any that have been previously learned from human feedback.
- Learning to summarize from human feedback by OpenAI: In this work, the authors show that it is possible to significantly improve summary quality by training a model to optimize for human preferences. They collect a large, high-quality dataset of human comparisons between summaries, train a model to predict the human-preferred summary, and use that model as a reward function to fine-tune a summarization policy using reinforcement learning. They apply the method to a version of the TL;DR dataset of Reddit posts and find that our models significantly outperform both human reference summaries and much larger models fine-tuned with supervised learning alone. The models also transfer to CNN/DM news articles, producing summaries nearly as good as those of a human
- Training language models to follow instructions with human feedback by OpenAI: In this paper, the authors show an avenue for aligning language models with user intent on a wide range of tasks by fine-tuning them with human feedback. Starting with labeler-written prompts and prompts submitted through the OpenAI API, they collect a dataset of labeler demonstrations of the desired model behavior, which we use to fine-tune GPT-3 using supervised learning. They then collect a dataset of rankings of model outputs, which they use to fine-tune this supervised model further using reinforcement learning from human feedback. We call the resulting models InstructGPT. In human evaluations on the prompt distribution, outputs from the 1.3B parameter InstructGPT model are preferred to outputs from the 175B GPT-3, despite having 100x fewer parameters. Moreover, InstructGPT models show improvements in truthfulness and reductions in toxic output generation while having minimal performance regressions on public NLP datasets. Even though InstructGPT still makes simple mistakes, the results show that fine-tuning with human feedback is a promising direction for aligning language models with human intent.
Vision-Language Models (VLMs) lie at the intersection of computer vision and natural language processing, enabling systems to understand and generate language grounded in visual context. These models power a wide range of applications — from image captioning and visual question answering to multimodal search and AI assistants. This article offers a curated guide to learning and building VLMs, exploring key concepts in multimodality, foundational architectures, hands-on coding resources, and advanced topics like retrieval-augmented generation for multimodal inputs. Whether you’re a beginner trying to grasp the basics or a practitioner looking to deepen your technical understanding, this guide brings together practical and conceptual resources to support your journey into the world of vision-language modeling.
Chip Huyen, in this article, explores the rising field of multimodal AI, where models handle and integrate different types of data like text, images, and audio. She explains why multimodality matters — real-world applications often involve multiple input types, from medical imaging paired with patient records to robotic systems interpreting both vision and commands. The piece walks through how models like CLIP and Flamingo work, focusing on how they align modalities (like vision and language) using contrastive learning or combination architectures. It also outlines key research directions, such as expanding to more data types, building instruction-following systems, efficiently training with adapters, and generating outputs across modalities. In short, the article gives a clear, technical yet accessible roadmap of how multimodal models are evolving and why they’re essential for building more capable AI systems.
The article covers the following topics:
- Part 1. Understanding Multimodal
- Part 2. Fundamentals of Multimodal Training
- Part 3. Research Directions for LMMs
The smol vision course is maintained by Merve Noyan and others from Hugging Face. It contains tutorials for shrinking, optimizing, and customizing cutting-edge vision models.
In this 5-hour video, Umar Jamil will be coding the PaliGemma Vision Language Model from scratch while explaining all the concepts behind it:
- Transformer model (Embeddings, Positional Encoding, Multi-Head Attention, Feed Forward Layer, Logits, Softmax)
- Vision Transformer model
- Contrastive learning (CLIP, SigLip)
- Numerical stability of the Softmax and the Cross-Entropy Loss
- Rotary Positional Embedding
- Multi-Head Attention
- Grouped Query Attention
- Normalization layers (Batch, Layer, and RMS)
- KV-Cache (prefilling and token generation)
- Attention masks (causal and non-causal)
- Weight tying
- Top-P Sampling and Temperature
Awesome Vision-Language Models is a repository of Vision Language Models for Vision Tasks: a Survey, a systematic survey of VLM papers in various visual recognition tasks, including image classification, object detection, semantic segmentation, etc. LLM
Finally, one of the important applications of vision language models is multimodal RAGs. In this hands-on course, you will learn how to build a multimodal RAG system by breaking down each step and building it up. This course covers:
- Introduction to Multimodal RAG Applications
- Multimodal RAG Application Architecture
- Multimodal Embeddings
- Processing Videos for Multimodal RAG
- Multimodal Retrieval from Vector Stores
- Large Vision Language Models (LVLMs)
- Multimodal RAG with Multimodal LangChain
- Putting it All Together! Building Multimodal RAG Application
In the rapidly evolving landscape of Large Language Models (LLMs), staying abreast of the latest research breakthroughs and industry developments is paramount for professionals and enthusiasts alike. This blog is a comprehensive guide with a curated list of resources to meet this need. This section begins by spotlighting the pioneers and thought leaders in LLM research, providing insights into their work and contributions to the field. It then examines the key players driving innovation and applications within the LLM industry. Additionally, it explores prominent organizations dedicated to advancing LLM research and fostering collaboration within the community. Furthermore, the section identifies influential individuals and content creators shaping discourse and disseminating valuable insights across various platforms. It delves into the realm of newsletters and blogs, highlighting essential sources for staying updated on the latest trends and developments in LLM research and industry. Whether one is an aspiring researcher, industry practitioner, or simply intrigued by the capabilities of LLMs, this blog equips readers with the essential resources to remain informed and engaged in this dynamic domain.
- Yann LeCun: Yann is also one of the deep learning pioneers, and he is currently a Chief AI Scientist at Meta AI. He publishes the research output at Meta AI in addition to his opinions on AI research.
- Andrej Karpathy: Andrej is one of the leading AI researchers, and currently, his latest position was at OpenAI; before that, he was the director of AI at Tesla. By following Andrej, you will stay updated with recent AI research and also important opinions about the AI advances and research.
- Andrew Ng: Andrew Ng, a prominent figure in the AI community, shares research, publications, projects, courses, and industry news. He engages in discussions, offers guidance, and provides valuable perspectives on AI ethics and applications. Following Andrew Ng on Twitter keeps you informed about the latest developments and connects you with the AI community.
- Demis Hassabis: Demis is the co-founder and CEO of DeepMind. Following him will keep you updated on the new research output and new projects from DeepMind.
- Thomas Wolf: Thomas is the Co-founder of HuggingFace. He shares the new projects and important models of HuggingFace in addition to important AI news.
- Oriol Vinyals: Oriol is the VP of research and deep learning lead at Google DeepMind. He shares updates about research projects and research output at Google DeepMind
- Sam Altman: Sam is the co-founder of OpenAI. His tweets are mainly about AI ethics, regulations, and recent outputs from OpenAI.
- Abubakar Abid: Abubakar is the founder of Gradio, which was recently acquired by HuggingFace. He shares important updates on Gardio in addition to demos on Gardio. Also, he is the founder of the Fatima fellowship, and he shares important announcements related to it.
- Jay Alammar: Jay is the director of engineering at Cohere. Jay is well known for his illustrated NLP blogs in which he simplifies complex concepts using illustrations.
- Sebastian Raschka: Sebastian is a machine learning researcher and author of Machine Learning with Pytorch and Scikit-Learn. Sebastian shares very insightful tweets about AI, deep learning, and Pytorch.
- NeurIPS Conference: NeurIPS:, a prestigious conference in the ML and AI community, offers valuable updates, keynote announcements, registration details, submission deadlines, and highlights from past conferences. The account also shares research papers, workshop announcements, and tutorials. By following NeurIPS, you join a global network of leading researchers, practitioners, and industry professionals, gaining knowledge and networking opportunities. Stay tuned to NeurIPS on Twitter for the latest trends and breakthroughs in ML research. Prepare for an exciting journey through cutting-edge ML research.
- Google DeepMind: DeepMind, a subsidiary of Google, is known for its cutting-edge advancements in AI. By following their account, you gain access to updates on research findings, publications, and breakthroughs. DeepMind’s work in reinforcement learning, neural networks, and deep learning is highlighted, along with applications in various domains. Their Twitter account also features events, talks, and panel discussions with their researchers.
- Google AI:: Google AI shares updates on AI projects, research papers, technology advancements, and practical applications across domains. Google AI’s Twitter account also features highlights from conferences, events, and talks by its researchers. By following Google AI, you stay informed about cutting-edge developments and engage with diverse AI applications. Stay tuned to their account for valuable insights and inspiration in the field of AI.
- Meta AI: This is the official account for Meta AI. By following it, you will stay updated with the new research output by Meta AI.
- Hugging Face: Hugging Face is a company known for its state-of-the-art models and NLP tools. By following their account, you gain access to model releases, library improvements, NLP research advancements, tips, tutorials, and community engagement. Stay connected to cutting-edge developments and leverage their tools for your own NLP projects and research.
- OpenAI: OpenAI is a leading research organization focused on safe and beneficial AI development. By following their account, you gain access to updates on research projects, new AI models/tools, insights from team members, and announcements. The account engages with the AI community, discusses ethical implications, and promotes responsible AI development. Stay informed about AI trends and discussions by following OpenAI on Twitter.
- Omar Sanseviero: Omar Sanseviero is a prominent figure in the machine learning community, currently serving as a machine learning advocate at Google DeepMind. He plays a significant role at the intersection of open source, product development, research, and community engagement. He shares valuable insights on LLM development and recent news.
- Rohan Paul: Rohan is known for his educational content and resources on machine learning. He has a YouTube channel, "Rohan-Paul-AI," where he shares tutorials and insights on topics like LLM, GANs (Generative Adversarial Networks), and other deep learning models.
- Rowan Cheung: Rowan Cheung is the founder of “The Rundown AI,” a popular newsletter that provides updates and insights on the latest developments in artificial intelligence. He covers a wide range of AI topics, including new technologies, industry trends, and significant advancements from leading tech companies like Microsoft and Google.
- AK: AK is a machine learning engineer at Gardio. He is well known for publishing summaries of recent machine learning and AI papers on a daily basis.
- Yannic Kilcher: Yannic is the co-founder of DeepJudge, and he runs a YouTube channel under his name. He is very active in making videos for in-depth explanations of recent important research papers and important AI news.
- Jeremy Howard: Jeremy Howard is a renowned data scientist, entrepreneur, and educator. He is known for his contributions to the field of AI, particularly in deep learning, and for founding Fast ai. Jeremy shares important opinions about AI and deep learning, in addition to news and updates related to Fast ai.
- Santiago: Santiago was previously a director of computer vision at Levatas and is currently managing his own machine-learning school. Santiago shares valuable technical and career tips that will help you build both better careers and better projects.
- Elvis: Elvis is a former machine learning engineer at Meta working on the Paper with Code project, and currently, he is focusing on DIAR, an open-source educational platform. He shares very important summaries of recent machine learning papers in addition to important AI news, and also DIAR's recent projects.
- Sanyam Bhutani: Sanyam is a senior data scientist at Weights and Biases,v and he is also one of the world's Kaggle grandmasters. He is very active on Twitter and shares summaries of important research papers on a daily basis in addition to important AI news.
- Harrison Kinsley: Harrison runs one of the best AI & Programming YouTube channels under the name of Sentdex. He covers different topics on his YouTube channel, including explaining recent papers, news, and also in-depth explanations for different concepts.
- Lilian Weng: Lilian is working on AI safety at OpenAI. Her posts are mainly about LLM politics and safety at openAI and also new announcements related to OpenAI work.
- To Data & Beyond Newsletter: The “To Data & Beyond” newsletter by Youssef Hosni is an excellent resource for staying updated with the latest research and developments in large language models (LLMs). It offers in-depth analysis, summaries of recent research papers, and discussions on trends in data science and machine learning. The newsletter aims to provide valuable insights for both professionals and enthusiasts in the field, making complex topics more accessible.
- The AiEdge Newsletter: The AiEdge Newsletter, curated by Damien Benveniste, is an excellent source for staying updated with the latest research and developments in large language models (LLMs). The newsletter covers a wide array of topics, including machine learning applications, system design, MLOps, and the latest techniques and news in the field. It aims to make complex machine-learning concepts accessible to a broader audience, often diving into subjects not typically covered in mainstream sources.
- Ahead of AI: Ahead of AI newsletter, authored by Sebastian Raschka, is a highly regarded newsletter that provides in-depth coverage of the latest research and developments in AI, particularly focusing on machine learning and large language models (LLMs). With over a decade of experience in AI and a passion for education, Raschka curates content that is valuable for both researchers and practitioners aiming to stay ahead in the rapidly evolving AI field.
- Daily Papers by Hugging Face: Hugging Face’s daily paper initiative focuses on providing the community with access to significant papers, facilitating discussions around them, and offering a centralized place to explore these papers and related artifacts like models, datasets, and demos.
- The Rundown: The Rundown newsletter gives you access to the latest AI news and learn how to apply it in 5 minutes. By simply following this newsletter, you’ll be able to: Keep up with the rapid pace of AI, Learn how to use AI to automate your work, Discover the best AI tools and job opportunities, and Gain expert insights on how AI will reshape the future of work.
- KDNuggets: KDnuggets is a highly reputable source for staying updated with industry news related to large language models (LLMs) and other advancements in data science, machine learning, and AI. It offers a wealth of resources, including articles, tutorials, opinions, and industry news, making it a comprehensive platform for professionals and enthusiasts in these fields.
- AI Weekly: AI Weekly tracks what influential AI experts and organizations are reading and sharing, then ranks and explains developments in models, agents, funding, policy, and research. It is free and published three times a week for more than 53,000 professionals.
Prompt engineering, an emerging field, focuses on the development and refinement of prompts to enhance the utilization of language models (LMs) across diverse applications and research domains. Proficiency in prompt engineering contributes to a deeper comprehension of the strengths and weaknesses inherent in large language models (LLMs). Researchers leverage prompt engineering to enhance the performance of LLMs in tasks ranging from commonplace to intricate, including question-answering and arithmetic reasoning. Meanwhile, developers employ prompt engineering to craft resilient and efficient prompting techniques that seamlessly interface with LLMs and other tools. Motivated by the high interest in developing with LLMs, in this section, I will share five resources that will help you understand prompt engineering and learn how to write better prompts.
- Awesome GPT Prompt Engineering GitHub Repository: This is a curated list of awesome resources, tools, and other shiny things for GPT Prompt Engineering.
- Prompt Engineering Guide: Elvis Saravia has created this Prompt Engineering Guide that contains all the latest papers, learning guides, lectures, references, and tools related to prompt engineering for LLMs.
- Prompt Engineering for LLMs: Prompt Engineering for LLMs is a 4-day hands-on course that teaches how to efficiently and effectively use LLMs. It covers the best and latest prompting techniques that you can apply to a variety of use cases that range from building long article summarizers to prompt injection detectors to LLM-powered evaluators. Be ready to dive deep and acquire advanced skills for effectively prompting and building with LLMs. By the end of the course, the goal is NOT to teach you the “10 best prompts”. The goal is for you to learn how to apply advanced prompting techniques to help you grow in your career, build personal projects, or build advanced LLM-powered products.
- Maximizing the Potential of LLMs: A Guide to Prompt Engineering: LLMs have rapidly improved in recent years, with LLMs such as GPT-3 and GPT-4 taking center stage. These models have become popular due to their ability to perform a great variety of tasks with incredible skill. Also, as the number of parameters of these models (in the billions!) has increased, these models have unpredictably gained new abilities. In this article, you will explore LLMs, the tasks they can perform, their shortcomings, and various prompt engineering strategies.
- ChatGPT Prompt Engineering for Developers: In ChatGPT Prompt Engineering for Developers, you will learn how to use a large language model (LLM) to quickly build new and powerful applications. Using the OpenAI API, you’ll be able to quickly build capabilities that teach you to innovate and create value in ways that were cost-prohibitive, highly technical, or simply impossible before now. This short course, taught by Isa Fulford (OpenAI) and Andrew Ng (DeepLearning.AI), will describe how LLMs work, provide best practices for prompt engineering, and show how LLM APIs can be used in applications for a variety of tasks.
1. Vector Databases: from Embeddings to Applications: This course will help you gain the knowledge to make informed decisions about when to apply vector databases to your applications. You’ll explore: how to use vector databases and LLMs to gain deeper insights into your data, build labs that show how to form embeddings and use several search techniques to find similar embeddings, and explore algorithms for fast searches through vast datasets and build applications ranging from RAG to multilingual search.
2. Building Applications with Vector Databases: In this course, you’ll explore the implementation of six applications using vector databases:
- Semantic Search: Create a search tool that goes beyond keyword matching, focusing on the meaning of content for efficient text-based searches on a user Q/A dataset.
- RAG: Enhance your LLM applications by incorporating content from sources the model wasn’t trained on, like answering questions using the Wikipedia dataset.
- Recommender System: Develop a system that combines semantic search and RAG to recommend topics, and demonstrate it with a news article dataset.
- Hybrid Search: Build an application that finds items using both images and descriptive text, using an eCommerce dataset as an example.
- Facial Similarity: Create an app to compare facial features, using a database of public figures to determine the likeness between them.
- Anomaly Detection: Learn how to build an anomaly detection app that identifies unusual patterns in network communication logs.
3. The Top 5 Vector Database Blog: This blog by Moez Ali is a comprehensive guide to the best vector databases. It will help you master high-dimensional data storage, decipher unstructured information, and leverage vector embeddings for AI applications.
4. LangChain — Text splitters: LangChain — Text splitters is a list of different text splitters implemented in LangChain. Once you’ve loaded documents, you’ll often want to transform them to better suit your application.
5. Sentence Transformers Library: Sentence Transformers library is a popular Python framework for state-of-the-art sentence, text, and image embeddings. The initial work is described in our paper Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. You can use this framework to compute sentence/text embeddings for more than 100 languages. These embeddings can then be compared, e.g., with cosine-similarity to find sentences with a similar meaning. This can be useful for semantic textual similarity, semantic search, or paraphrase mining. The framework is based on PyTorch and Transformers and offers a large collection of pre-trained models tuned for various tasks. Further, it is easy to fine-tune your models.
6. MTEB Leaderboard: MTEB Leaderboard is a leaderboard for embedding models, so you can compare different embedding models to use.
Learning Retrieval Augmented Generation (RAG) Basics entails grasping the fundamental concepts of merging retrieval and generation models in natural language processing. RAG involves understanding how retrieval mechanisms can enhance generative models by leveraging pre-existing knowledge bases to inform text generation. By acquiring knowledge of how to integrate retrieval techniques into the generation process, one can harness the power of structured information to produce more contextually relevant and coherent text outputs. Mastering RAG basics is crucial for advancing capabilities in various applications such as question answering, summarization, and content generation, as it represents a significant advancement in the field of language modeling.
Learning Resources:
- What Is Retrieval-Augmented Generation, aka RAG?: A Comprehensive article by Nvidia that introduces RAG and how it works.
- RAG Applications with Llama-Index: This tutorial introduces you to Llama-Index and how to build production-ready RAG applications.
- Building RAG Applications with LangChain: A comprehensive list of articles that walk you step by step through building RAG applications using LangChain.
- LangChain — OpenAI’s RAG: Overview of the RAG strategies employed by OpenAI, including post-processing.
This step involves providing a comprehensive set of articles to deepen the understanding of LangChain concepts. It serves as a foundation for exploring the intricacies of LangChain technology. Key topics covered include LangChain Query Construction, which offers insights into effective query construction methods for information retrieval. Additionally, a tutorial on LangChain SQL guides readers on interfacing with SQL databases using Large Language Models (LLMs), covering text-to-SQL conversion and introducing an optional SQL agent for seamless interaction. These resources aim to impart a holistic understanding of LangChain fundamentals and practical applications, enabling readers to tackle complex language-based tasks more proficiently.
Learning Resources:
- Basics of LangChain: A comprehensive set of articles that introduces you to the most important LangChain concepts.
- LangChain — Query Construction: A blog post about different types of query construction.
- LangChain — SQL: Tutorial on how to interact with SQL databases with LLMs, involving Text-to-SQL and an optional SQL agent.
This learning step offers a range of learning resources aimed at enhancing skills in Retrieval Augmented Generation. Deep Learning.ai provides a short course titled “Advanced Retrieval for AI with Chroma,” focusing on identifying and improving queries through large language models (LLMs) and embedding fine-tuning with user feedback. Additionally, Sam Witteveen’s advanced RAG tutorial series covers various advanced topics, including Self Querying Retrieval, Parent Document Retriever, Hybrid Search with BM25 & Ensembles, Contextual Compressors & Filters, HyDE (Hypothetical Document Embeddings), and RAG Fusion. These resources provide learners with in-depth knowledge and practical techniques to excel in the realm of Retrieval Augmented Generation.
Learning Resources:
- Advanced Retrieval for AI with Chroma: This is a short course provided by Deep Learning.ai in which you will learn how to recognize when queries are producing poor results, how to use a large language model (LLM) to improve your queries, and how to fine-tune your embeddings with user feedback.
- Advanced RAG by Sam Witteveen: Throughout this advanced RAG series of tutorials, you will learn: Self-Querying Retrieval, Parent Document Retriever, Hybrid Search BM25 & Ensembles, Contextual Compressors & Filters, HyDE —Hypothetical Document Embeddings, and Advanced RAG 06 — RAG Fusion
The combined resources of “RAG Pipeline — Metrics” and “Building and Evaluating Advanced RAG Applications” offer learners a comprehensive understanding of evaluating Retrieval Augmented Generation (RAG) systems. The former introduces essential evaluation metrics, while the latter delves into advanced retrieval methods, evaluation techniques, and the RAG triad for assessing relevance and accuracy. This holistic approach equips learners with the skills to effectively evaluate and optimize RAG systems in various applications.
Learning Resources:
- RAG Pipeline — Metrics: This article introduces you to the main metrics used to evaluate RAG pipelines.
- Building and Evaluating Advanced RAG Applications: This course delves into advanced retrieval methods, including sentence-window retrieval and auto-merging retrieval, which outperform the baseline RAG pipeline. It also covers evaluation techniques and experiment tracking to iteratively enhance the RAG pipeline’s performance. Furthermore, the course explores the RAG triad — Context Relevance, Groundedness, and Answer Relevance — as methods for assessing the relevance and accuracy of responses generated by large language models (LLMs).
Agents are transforming industries with their ability to process, understand, and generate human-like language. As the demand for expertise in LLM agents grows, so does the need for accessible learning resources. This section provides a curated list of top free resources for mastering LLM agents, from foundational guides to hands-on tutorials. Designed for beginners and seasoned AI practitioners alike, this guide covers essential courses, documentation, interactive tools, and community resources.
The first set of resources will cover the basics of agents and will help you build a foundation to build upon in the next steps.
Learning Resources:
- Introduction to LLM Agents — Nvidia: The first resource is a comprehensive introduction blog to LLM Agents from Nvidia. The blog covers the following topics: What is an AI agent?, and Agents for enterprise applications
- Large Language Model Agents — UC Berkeley: The course first discusses fundamental concepts that are essential for LLM agents, including the foundation of LLMs, essential LLM abilities required for task automation, as well as infrastructures for agent development. It will also cover representative agent applications, including code generation, robotics, web automation, medical applications, and scientific discovery. Meanwhile, the course will discuss limitations and potential risks of current LLM agents and share insights into directions for further improvement.
- AI Agent Mastery Bootcamp: From Architecture to Optimization — Arize AI: You will dive into the world of AI agent development in this comprehensive bootcamp. You’ll gain hands-on experience with the latest tools and techniques in AI agent development. From understanding core architectures to troubleshooting complex issues, our expert-led sessions will equip you with the knowledge and skills to build sophisticated AI agents that can tackle real-world challenges. You’ll learn how to navigate popular frameworks, optimize performance, and evaluate your agents effectively.
Now that you have built the foundation and understood how agents work and their main components. It is time to gain more practical skills in building a multi-agent system.
Learning Resources:
- Practical Multi AI Agents and Advanced Use Cases with crewAI — CrewAI & Deep Learning.AI: In this course, you will build several practical apps like an automated project planning system, lead-scoring and engagement automation, support data analysis, and content creation at scale.
- Multi AI Agent Systems with crewAI — CrewAI & Deep Learning.AI: You will learn key principles of designing effective AI agents and organizing a team of AI agents to perform complex, multi-step tasks. Apply these concepts to automate 6 common business processes.
- Scaling AI Agents for Real-World Tasks: In this lecture on “High Bit,” AJ Asver, CEO of Parcha, discusses the challenges of building AI agents for complex tasks. Parcha’s AI agents automate operations and compliance, addressing the gap between proof of concept and production.
Now that we have learned the foundations of agents and how to build a multi-agent workflow. In this step, you will learn how to build a specialized agent for certain tasks.
Learning Resources:
- Building Your Own Database Agent — Microsoft & Deep Learning.AI: In this course, you will develop an AI agent that interacts with databases using natural language, simplifying the process for querying and extracting insights. Created in partnership with Microsoft and taught by Adrian Gonzalez Sanchez, Data and AI Specialist at Microsoft, this course is designed for developers, data professionals, as well as business analysts and professionals who want more sophisticated interaction with their databases through natural language instead of advanced SQL queries.
- LLMs as Operating Systems: Agent Memory — Letta & Deep Learning.AI: An LLM can use any information stored in its input context window, but has limited space. Using a longer input context also costs more and causes slower processing. Managing this context window and what to input becomes very important. Based on the innovative approach in the MemGPT research paper “Towards LLMs as Operating Systems,” its authors, two of whom are Charles and Sarah, proposed using an LLM agent to manage this context window, building a management system that provides applications with managed, persistent memory.
- AI Agents in LangGraph — LangChain & Deep Learning.AI: In this course, you will learn to build an agent from scratch using Python and an LLM, and then you will rebuild it using LangGraph, learning about its components and how to combine them to build flow-based applications.
- Building Agentic RAG with LlamaIndex — LlamaIndex & Deep Learning.AI: In this course from Jerry Liu, co-founder and CEO at LlamaIndex, you will learn how to use agentic RAG, a framework designed to build research agents skilled in tool use, reasoning, and decision-making with your data.
Now that you have learned the foundation, master the practicalities and build customized agents. The final step will focus on evaluating agents so you can build a complete agentic workflow.
Learning Resources:
- How to Build, Evaluate, and Iterate on LLM Agents — Deep Learning.AI: LLM Agents are one of the most in-demand uses of large language models. This workshop, led by the expert founders of LlamaIndex and TruEra, will show you how to develop, evaluate, and iterate LLM Agents so that you can build powerful, effective LLM Agents quickly. In this workshop, you will learn: How to use a framework like LlamaIndex to build your LLM Agent. How to evaluate your LLM Agent using open-source LLM Observability tools like TruLens — testing for effectiveness, hallucinations, and bias. How to iterate your way to an effective app that will be approved for production. What to do to keep your app high-performing in production.
- How to evaluate an LLM agent? — TaskWeaver: This is a comprehensive guide from TaskWeaver on how to evaluate LLM agents with hands-on code.
- AgentBench — A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR’24) AgentBench is the first benchmark designed to evaluate LLM-as-Agent across a diverse spectrum of different environments. It encompasses 8 distinct environments to provide a more comprehensive evaluation of the LLMs’ ability to operate as autonomous agents in various scenarios.
Combining layers in transformer models makes them bigger and better at understanding language tasks. But making these big models costs a lot to train and they need a lot of memory and computer power to use afterward. The most popular Large Language Models (LLM) today such as ChatGPT have billions of settings and sometimes they have to handle long pieces of text, which makes them even more expensive to use. For example, RAG pipelines require putting large amounts of information into the input of the model, greatly increasing the amount of processing work the LLM has to do. In this section, you will be provided with a comprehensive list of resources to delve into the foremost challenges encountered in LLM inference and proffer practical solutions.
Stacking transformer layers to create large models results in better accuracies, few-shot learning capabilities, and even near-human emergent abilities on a wide range of language tasks. These foundation models are expensive to train, and they can be memory- and compute-intensive during inference (a recurring cost). The most popular large language models (LLMs) today can reach tens to hundreds of billions of parameters in size and, depending on the use case may require ingesting long inputs (or contexts), which can also add expense.
For example, RAG pipelines require putting large amounts of information into the input of the model, greatly increasing the amount of processing work the LLM has to do. In this section, you will explore the most pressing challenges in LLM inference, along with some practical solutions. It would be best if you had a basic understanding of transformer architecture and the attention mechanism in general. It is essential to grasp the intricacies of LLM inference, which we will address in the next section.
Learning Resources:
Open-source LLMs are great for conversational applications, but they can be difficult to scale in production and delivery latency and throughput that are incompatible with your cost-performance objectives. In this section, you will zoom in on optimizing LLM inference, and study key mechanisms that help reduce latency and increase throughput: the KV cache, continuous batching, and speculative decoding, including the state-of-the-art Medusa approach.
Learning Resources:
Unlike CPUs, GPUs are the standard choice of hardware for machine learning because they are optimized for memory bandwidth and parallelism. To keep up with the larger sizes of modern models or to run these large models on existing and older hardware, there are several optimizations you can use to speed up GPU inference. In this section, you’ll learn how to use FlashAttention-2 (a more memory-efficient attention mechanism), BetterTransformer (a PyTorch native fastpath execution), and bitsandbytes to quantize your model to a lower precision. Finally, learn how to use 🤗 Optimum to accelerate inference with ONNX Runtime on Nvidia and AMD GPUs.
Learning Resources:
- GPU Inference by Hugging Face
- Optimizing LLMs for Speed and Memory by Hugging Face
- Assisted Generation by Hugging Face
In the last section, you will explore and compare the features and capabilities of leading LLM libraries for inference Optimization. In addition to exploring you will explore how to optimize OpenAI in inference, and finally, you will explore the future trends in this field.
Learning Resources:
- Comparing Top Inference Optimization Libraries LLM Roadmap from Absolute Beginner to Advanced 100 | Page
- Large language model inference optimizations on AMD GPUs
- Accelerate Large Language Model (LLM) Inference on Your Local PC
- OpenAI Latency Optimization
- Inference Optimization Strategies
LLMOps is primarily focused on enhancing operational capabilities and establishing the necessary infrastructure for refining existing foundational models and seamlessly integrating these optimized models into products. Although LLMOps may not seem groundbreaking to most observers within the MLOps community, it serves as a specialized subset within the broader MLOps domain. A more specific definition can elucidate the intricate requirements involved in fine-tuning and deploying these models effectively.
Foundational models, such as GPT-3 with its massive 175 billion parameters, demand substantial amounts of data and compute resources for training. While fine-tuning these models may not require the same scale of data or computational power, it remains a significant task that necessitates robust infrastructure capable of parallel processing and handling large datasets.
This section delves into essential resources to help initiate your journey into LLMOps, providing valuable insights and guidance for getting started effectively. LLMOps consists of two parts:
- Large Language Models: LLM-as-a-Service is where a vendor offers the LLM as an API on their infrastructure. This is how primarily closed-source models are delivered. Custom LLM stack is a broader category of tools necessary for fine-tuning and deploying proprietary solutions built on top of open-source models.
- Prompt Engineering tools: enable in-context learning instead of fine-tuning at lower costs and without using sensitive data. Vector Databases retrieve contextually relevant information for certain prompts. Prompt Execution enables optimizing and improving the model output by managing prompt templates to build chain-like sequences of relevant prompts. Prompt Logging, Testing, and Analytics … Let’s just say it’s an emerging space that has no categories yet.
Learning Resources:
- Building LLM Applications for Production: It’s easy to build LLMs, but very hard to make something production-ready with them. This article by Chip Huyen covers how to put LLMs into production.
- Awesome LLMOps: This GitHub repo contains a curated list of the best LLMOps resources and tools for developers.
- LLMOps Course: In this course, you’ll go through the LLMOps pipeline of pre-processing training data for supervised instruction tuning, and adapt a supervised tuning pipeline to train and deploy a custom LLM. This is useful in creating an LLM workflow for your specific application. For example, create a question-answer chatbot tailored to answer Python coding questions, which you’ll do in this course.
- Automated Testing for LLMOps: In this course, you will learn how to create a continuous integration (CI) workflow to evaluate your LLM applications at every change for faster, safer, and more efficient application development. When building applications with generative AI, model behavior is less predictable than traditional software. That’s why systematic testing can make an even bigger difference in saving you development time and cost. Continuous integration, a key part of LLMOps, is the practice of making small changes to software in development and thoroughly testing them to catch issues early when they are easier to fix. With a robust automated testing pipeline, you’ll be able to isolate bugs before they accumulate — when they’re easier and less costly to fix. Automated testing lets your team focus on building new features so that you can iterate and ship products faster.
Large Language Models (LLMs) represent a revolutionary advancement in artificial intelligence, yet their deployment introduces significant security challenges. This section provides a comprehensive resource for developers, engineers, architects, and managers seeking to fortify their understanding and defenses against potential vulnerabilities in LLM applications. These resources are indispensable for anyone involved in the development, deployment, or management of LLM applications.
Learning Resources:
- List of the 10 most critical vulnerabilities seen in LLM applications: This article aims to educate developers, designers, architects, managers, and organizations about the potential security risks when deploying and managing LLMs. The article lists the top 10 most critical vulnerabilities often seen in LLM applications, highlighting their potential impact, ease of exploitation, and prevalence in real-world applications. Examples of vulnerabilities include prompt injections, data leakage, inadequate sandboxing, and unauthorized code execution, among others. The goal is to raise awareness of these vulnerabilities, suggest remediation strategies, and ultimately improve the security posture of LLM applications.
- How to hack Google Bard, ChatGPT, or any other chatbot: Google Bard, ChatGPT, Bing, and all those chatbots have their own security systems, but they are, of course, not invulnerable. This article introduces you to how to hack Google and all these other huge tech companies and get the idea behind LLM Attacks.
- Prompt Injection Primer for Engineers: Prompt injection is the highest profile vulnerability in AI-powered features and applications. It’s also one of the most misunderstood. The impact varies greatly depending on who will use the feature, what data is accessible, and what functionality is exposed to the LLM. This guide aims to assist developers in creating secure AI-powered applications and features by helping them understand the actual risks of prompt injection.
- Quality and Safety for LLM Applications: In this course, you’ll explore new metrics and best practices to monitor your LLM systems and ensure safety and quality.
- Red Teaming LLM Applications: In this course, you’ll attack various chatbot applications using prompt injections to see how the system reacts and understand security failures. LLM failures can lead to legal liability, reputational damage, and costly service disruptions. This course helps you mitigate these risks proactively. Learn industry-proven red teaming techniques to proactively test, attack, and improve the robustness of your LLM applications.
- LLM Security: A comprehensive list of resources and papers: LM security is the investigation of the failure modes of LLMs in use, the conditions that lead to them, and their mitigations. It contains a curated list of links to large language model security content — research, papers, and news — posted by llm_sec
Deploying Large Language Models (LLMs) is pivotal in leveraging their capabilities across various applications, from enhancing user experiences to addressing privacy concerns. There are four distinct deployment techniques: local, demo, server, and edge deployment.
In this section, you will be provided with a selection of learning resources for each deployment technique, equipping readers with the knowledge needed to navigate and implement these techniques effectively. The section begins by emphasizing the significance of local deployment, offering insights into five free tools for running LLMs locally on personal devices. It then delves into demo deployment, where readers can learn to build interactive applications with minimal coding experience. Server deployment is subsequently addressed, guiding readers through the process of deploying LLMs using cloud platforms and containers. Finally, the article elucidates edge deployment, highlighting the benefits of embedding LLMs directly into real-world systems and providing resources for implementing this approach.
This comprehensive resource is tailored for individuals seeking to harness the power of LLMs across diverse deployment scenarios. Whether you are a novice exploring local deployment or an experienced developer venturing into edge computing, this article offers invaluable learning materials to accelerate your journey. Designed to empower AI enthusiasts, developers, and researchers alike, this article serves as a roadmap for deploying LLMs effectively and efficiently.
While accessing LLM-based chatbots online is simple with just an internet connection and a good browser, it comes with potential privacy risks. For example, OpenAI stores your interactions and metadata to improve its models, raising concerns for privacy-conscious users. Opting to use these models locally provides a solution for those seeking greater control over their data. In this section, you will explore five methods to utilize large language models (LLMs) locally. Compatible across major operating systems, these tools can be swiftly downloaded and installed. With locally-run LLMs, you retain control over model selection and can easily access models from the HuggingFace hub. Moreover, granting access to project folders enables context-aware responses.
Learning Resources:
In this section, you will learn how you can, with just a few lines of code, create a user-friendly app (usable for non-coders) to take input text, then apply different tasks with an open-source large language model, and display the output. By the end of the course, you’ll gain the practical knowledge to rapidly build interactive apps and demos to validate your project and ship faster.
Learning Resources:
- Streamlit — Build a basic LLM app
- Building Generative AI Applications with Gradio
- Serving an LLM Application as an API Endpoint using FastAPI in Python
- Deploying ML Models in 60 Minutes using Python, Flask & Render | Step-by-Step Tutorial
Deploying Large Language Models (LLMs) is a step towards enhancing user experience. But knowing where to start and which aspects to consider before LLM deployment is essential. LLMs have been instrumental in powering everything from machine translation to content creation to virtual assistants and chatbots. In this section, you will learn how to deploy LLMs using Amazon Bedrock, Amazon SageMaker, and the Hugging Face inference container.
Learning Resources:
- Serverless LLM apps with Amazon Bedrock
- HF LLM Inference Container
- Philschmid blog
- Build Customize and Deploy LLMs at scale on Azure with NVIDIA NeMo | DISFP08
Running LLMs on the edge is of great importance. By embedding LLMs directly into real-world systems such as the Copilot services (coding, smart reply, and office) on laptops, in-car entertainment systems, vision-language assistants in robots, or spaceship control interfaces, users can access instant responses and services without relying on a stable internet connection. Moreover, this approach alleviates the inconvenience of queuing delays often associated with cloud services. As such, running LLMs on the edge enhances user experience and addresses privacy concerns, as sensitive data remains localized and reduces the risk of potential breaches. In this learning section, you will learn how to deploy LLMs on edge devices using MLC, TinyChat, and the NVIDIA IGX Orin Developer Kit.
Learning Resources:
- Bringing Open Large Language Models to Consumer Devices
- Bringing Hardware Accelerated Language Models to Android Devices
- TinyChat: Large Language Model on the Edge
- Deploy Large Language Models at the Edge with NVIDIA IGX Orin Developer Kit
- Bringing GenAI Offline: running SLMs like Phi-2/Phi-3 and Whisper Models on Mobile Devices
While accessing LLM-based chatbots online is simple with just an internet connection and a good browser, it comes with potential privacy risks. For example, OpenAI stores your interactions and metadata to improve its models, raising concerns for privacy-conscious users. Opting to use these models locally provides a solution for those seeking greater control over their data.
In this section, we’ll explore five methods to utilize large language models (LLMs) locally. Compatible across major operating systems, these tools can be swiftly downloaded and installed.
With locally run LLMs, you retain control over model selection and can easily access models from the Hugging Face hub. Moreover, granting access to project folders enables context-aware responses
GPT4All is open-source software that enables you to use the state-of-the-art open-source LLM on your local machine with ease and in simple steps. To get started, simply download GPT4ALL from the website and install it on your system.
After downloading and installing, you should be able to find the application in the directory you specified in the installer. You will find a desktop icon for GPT4All after installation. Next, choose the model from the panel that suits your needs and start using it. If you have CUDA (Nvidia GPU) installed, GPT4ALL will automatically start using your GPU to generate quick responses of up to 30 tokens per second.
You can provide access to multiple folders containing important documents and code, and GPT4ALL will generate responses using Retrieval-Augmented Generation. GPT4ALL is user-friendly, fast, and popular among the AI community.
LM Studio is an easy-to-use desktop app for experimenting with local and open-source Large Language Models (LLMs). With LM Studio, you can:
- Run LLMs on your laptop, entirely offline.
- Use models through the in-app Chat UI or an OpenAI-compatible local server.
- Download any compatible model files from Hugging Face repositories.
- Discover new & noteworthy LLMs on the app’s home page.
LM Studio offers several advantages over GPT4ALL. The user interface is excellent, and you can install any model from Hugging Face Hub with a few clicks. Additionally, it provides GPU offloading and other options that are not available in GPT4ALL. However, LM Studio is a closed-source program, and it doesn’t have the option to generate context-aware responses by reading project files.
Ollama provides a lightweight and user-friendly way to set up and run various open-source LLMs on your computer. This eliminates the need for complex configurations or relying on external servers, making it ideal for various purposes:
- Development: It allows developers to experiment and iterate quickly on LLM projects without needing to deploy them to the cloud.
- Research: Researchers can use Ollama to study LLM behavior in a controlled environment, facilitating in-depth analysis.
- Privacy: Running LLMs locally ensures that your data never leaves your machine, which is crucial for sensitive information.
Ollama comes with a pre-built library of trained language models, such as:
- Llama 2: A large language model capable of various tasks like text generation, translation, and question answering.
- Mistral: A factual language model trained on a massive dataset of text and code.
- Gemma: A conversational language model designed for engaging dialogue.
- LLaVA: A robust model trained for both chat and instruction use cases.
This library allows you to easily integrate these pre-trained models into your applications, eliminating the need to train them from scratch and saving time and resources. Ollama accelerates running models using NVIDIA GPUs as well as modern CPU instruction sets such as AVX and AVX2 if available. No configuration or virtualization is required!
In the example below, we can download Mistral LLM:
After the model is downloaded, we can start asking it and chatting with it:
LLaMa.cpp was developed by Georgi Gerganov. It implements the Meta’s LLaMa architecture in efficient C/C++, and it is one of the most dynamic open-source communities around the LLM inference with more than 390 contributors, 43000+ stars on the official GitHub repository, and 930+ releases. Llama.cpp’s backbone is the original Llama model, which is also based on the transformer architecture. The authors of Llama leverage various improvements that were subsequently proposed and use different models such as PaLM.
You can install it using the following command:
pip install llama-cpp-python
Once it is installed, you can import it using the following command:
from llama_cpp import Llama
The Llama class imported above is the main constructor leveraged when using Llama.cpp, and it takes several parameters and is not limited to the ones below. The complete list of parameters is provided in the official documentation:
- model_path: The path to the Llama model file being used
- prompt: The input prompt to the model. This text is tokenized and passed to the model.
- device: The device to use for running the Llama model; such a device can be either CPU or GPU.
- max_tokens: The maximum number of tokens to be generated in the model’s response.
- stop: A list of strings that will cause the model generation process to stop.
- temperature: This value ranges between 0 and 1. The lower the value, the more deterministic the result. On the other hand, a higher value leads to more randomness, hence more diverse and creative output.
- top_p: This is used to control the diversity of the predictions, meaning that it selects the most probable tokens whose cumulative probability exceeds a given threshold. Starting from zero, a higher value increases the chance of finding a better output but requires additional computations.
- echo: A boolean used to determine whether the model includes the original prompt at the beginning (True) or does not include it (False)
For instance, let’s consider that we want to use a large language model called <MY_AWESOME_MODEL> stored in the current working directory. The instantiation process will look like this:
# Instantiate the model
my_aweseome_llama_model = Llama(model_path="./MY_AWESOME_MODEL")
prompt = "This is a prompt"
# Define the parameters
max_tokens = 100
temperature = 0.3
top_p = 0.1
echo = True
stop = ["Q", "\n"]
model_output = my_aweseome_llama_model(
prompt,
max_tokens=max_tokens,
temperature=temperature,
top_p=top_p,
echo=echo,
stop=stop,
)
final_result = model_output["choices"][0]["text"].strip()ChatRTX is a demo app that lets you personalize a GPT LLM connected to your own content — docs, notes, or other data. Leveraging RAG, TensorRT-LLM, and RTX acceleration, you can query a custom chatbot to quickly get contextually relevant answers. Because it all runs locally on your Windows RTX PC or workstation, you’ll get fast and secure results.
With Chat with RTX, you can run LLaMA and Mistral models locally on your laptop. It’s a fast and efficient application that can even learn from documents you provide or YouTube videos. However, it’s important to note that Chat with RTX relies on TensorRTX-LLM, which is only supported on 30 series GPUs or newer.
Model Context Protocol (MCP) is becoming more and more one of the most important standards in context-aware AI production. In this article, we recommend resources to break down what MCP is, why it matters, and how you can start working with it, whether you're new to it or want to learn more about it. From novice-friendly tutorials to tool-focused demo projects, community-created courses, and practice guides, this chapter collects the best of the best free tools that can help you become an MCP master and create AI applications seamlessly integrated with external data, tools, and services.
Learning Resources:
- Everything you need to know about MCP: The first learning resource is a beginner-friendly introduction to MCP by Replit. It explains how MCP works in simple, accessible language, making it easy to understand without requiring deep technical knowledge of large language models. It’s a great starting point for quickly grasping the key concepts and components of MCP.
- Model Context Protocol (MCP): A Guide With Demo Project: In this blog, you will be guided through building an MCP-powered PR review server that integrates with Claude Desktop. This server will: Fetch PR details and changed files from GitHub, analyze code changes using the Claude Desktop application, and generate PR review summaries and suggestions Save reviews to Notion. You will use the Model Context Protocol (MCP) to standardize communication between the server and Claude Desktop, making it modular and scalable.
- Model Context Protocol (MCP) Hugging Face Course: This free course will take you on a journey, from beginner to informed, in understanding, using, and building applications with MCP. In this course, you will: Study Model Context Protocol in theory, design, and practice, Learn to use established MCP SDKs and frameworks, Share your projects and explore applications created by the community, Participate in challenges where you will evaluate your MCP implementations against those of other students, Earn a certificate of completion by completing assignments. At the end of this course, you’ll understand how MCP works and how to build your own AI applications that leverage external data and tools using the latest MCP standards.
- MCP: Build Rich-Context AI Apps with Anthropic The MCP: Build Rich-Context AI Apps with Anthropic, a short course created in partnership with Anthropic and taught by Elie Schoppik. In this hands-on course, you’ll learn the core concepts of MCP and how to implement it in your AI Application. You’ll make a chatbot MCP-compatible, build and deploy an MCP server, and connect the chatbot to your MCP server and other open-source servers.
- Official MCP Documents: The official MCP docs are a good resource to learn the fundamentals, a tutorial to create your first MCP server, debugging, and inspection instructions.
- Awesome MCP Servers: A curated list of awesome Model Context Protocol (MCP) servers: A curated list of awesome Model Context Protocol (MCP) servers. It covers the following: What is MCP?, Clients, Tutorials, Community, Legend, Server Implementations, Frameworks, Tips & Tricks
