Generative AI and LLMs: Architecture and Data Preparation

Go to Course: https://www.coursera.org/learn/generative-ai-llm-architecture-data-preparation

Introduction

Differentiate between generative AI architectures and models, such as RNNs, Transformers, VAEs, GANs, and Diffusion Models.

Describe how LLMs, such as GPT, BERT, BART, and T5, are used in language processing.

Implement tokenization to preprocess raw textual data using NLP libraries such as NLTK, spaCy, BertTokenizer, and XLNetTokenizer.

Create an NLP data loader using PyTorch to perform tokenization, numericalization, and padding of text data.

Syllabus

Generative AI Architecture

In this module, you will learn about the significance of generative AI models and how they are used across a wide range of fields for generating various types of content. You will learn about the architectures and models commonly used in generative AI and the differences in the training approaches of these models. You will learn how large language models (LLMs) are used to build NLP-based applications. You will build a simple chatbot using the transformers library from Hugging Face.

In this module, you will learn to prepare data for training large language models (LLMs) by implementing tokenization. You will learn about the tokenization methods and the use of tokenizers. You will also learn about the purpose of data loaders and how you can use the DataLoader class in PyTorch. You will implement tokenization using various libraries such as nltk, spaCy, BertTokenizer, and XLNetTokenizer. You will also create a data loader with a collate function that processes batches of text.