Month: August 2023

Create a Large Language Model from Scratch with Python – Tutorial



Learn how to build your own large language model, from scratch. This course goes into the data handling, math, and transformers behind large language models. You will use Python. ✏️ Course developed by @elliotarledge ? Code and course resources: https://github.com/Infatoshi/fcc-intro-to-llms ⭐️ Contents ⭐️
(0:00:00) Intro
(0:03:25) Install Libraries
(0:06:24) Pylzma build tools
(0:08:58) Jupyter Notebook
(0:12:11) Download wizard of oz
(0:14:51) Experimenting with text file
(0:17:58) Character-level tokenizer
(0:19:44) Types of tokenizers
(0:20:58) Tensors instead of Arrays
(0:22:37) Linear Algebra heads up
(0:23:29) Train and validation splits
(0:25:30) Premise of Bigram Model
(0:26:41) Inputs and Targets
(0:29:29) Inputs and Targets Implementation
(0:30:10) Batch size hyperparameter
(0:32:13) Switching from CPU to CUDA
(0:33:28) PyTorch Overview
(0:42:49) CPU vs GPU performance in PyTorch
(0:47:49) More PyTorch Functions
(1:06:03) Embedding Vectors
(1:11:33) Embedding Implementation
(1:13:06) Dot Product and Matrix Multiplication
(1:25:42) Matmul Implementation
(1:26:56) Int vs Float
(1:29:52) Recap and get_batch
(1:35:07) nnModule subclass
(1:37:05) Gradient Descent
(1:50:53) Logits and Reshaping
(1:59:28) Generate function and giving the model some context
(2:03:58) Logits Dimensionality
(2:05:17) Training loop + Optimizer + Zerograd explanation
(2:13:56) Optimizers Overview
(2:17:04) Applications of Optimizers
(2:18:11) Loss reporting + Train VS Eval mode
(2:32:54) Normalization Overview
(2:35:45) ReLU, Sigmoid, Tanh Activations
(2:45:15) Transformer and Self-Attention
(2:46:55) Transformer Architecture
(3:17:54) Building a GPT, not Transformer model
(3:19:46) Self-Attention Deep Dive
(3:25:05) GPT architecture
(3:27:07) Switching to Macbook
(3:31:42) Implementing Positional Encoding
(3:36:57) GPTLanguageModel initalization
(3:40:52) GPTLanguageModel forward pass
(3:46:56) Standard Deviation for model parameters
(4:00:50) Transformer Blocks
(4:04:54) FeedForward network
(4:07:53) Multi-head Attention
(4:12:49) Dot product attention
(4:19:43) Why we scale by 1/sqrt(dk)
(4:26:45) Sequential VS ModuleList Processing
(4:30:47) Overview Hyperparameters
(4:32:14) Fixing errors, refining
(4:34:01) Begin training
(4:35:46) OpenWebText download and Survey of LLMs paper
(4:37:56) How the dataloader/batch getter will have to change
(4:41:20) Extract corpus with winrar
(4:43:44) Python data extractor
(4:49:23) Adjusting for train and val splits
(4:57:55) Adding dataloader
(4:59:04) Training on OpenWebText
(5:02:22) Training works well, model loading/saving
(5:04:18) Pickling
(5:05:32) Fixing errors + GPU Memory in task manager
(5:14:05) Command line argument parsing
(5:18:11) Porting code to script
(5:22:04) Prompt: Completion feature + more errors
(5:24:23) nnModule inheritance + generation cropping
(5:27:54) Pretraining vs Finetuning
(5:33:07) R&D pointers
(5:44:38) Outro ? Thanks to our Champion and Sponsor supporters:
? davthecoder
? jedi-or-sith
? 南宮千影
? Agustín Kussrow
? Nattira Maneerat
? Heather Wcislo
? Serhiy Kalinets
? Justin Hual
? Otis Morgan — Learn to code for free and get a developer job: https://www.freecodecamp.org Read hundreds of articles on programming: https://freecodecamp.org/news

Make $300/Day on YouTube Using Only AI (Faceless YouTube Automation Channel)



In this video, I'll show you how to make $300 per day on YouTube using only AI with a faceless YouTube automation channel, without showing your face. ✅ FREE Cheatsheet: Top 100 Most Popular Faceless YouTube Niches
DOWNLOAD HERE ? https://track.tubeai.pro/niche ✅ 5 Day Challenge: Learn How I Use Simple A.I. Tools To Create Faceless YouTube Channel & Make $2k To $10k+/Month!
CLICK HERE ? https://track.tubeai.pro/challenge-yt1 —————————- ? Watch Next: How To Make Money On YouTube WITHOUT Showing Your Face in 2023 ($200 Per Day)
https://youtu.be/EmYAgjj61V4 How I Make Faceless YouTube Videos in 10 Minutes Using AI (Step by Step Tutorial)
https://youtu.be/c160m22mT9w —————————- ⚙️ AI Tools & Resources To Grow Your Faceless YouTube Channel Canva (AI Thumbnail Designer) ? https://track.tubeai.pro/canva Fliki (AI Video Creator) ? https://track.tubeai.pro/fliki VidIQ (AI Video Idea/Keyword Research) ? https://track.tubeai.pro/vidiq —————————- ✅ SUBSCRIBE to my Youtube Channel + Turn ON notification
? https://track.tubeai.pro/subscribe-yt1 #BransonTay #TubeAIPro #FacelessYoutube —————————- EARNINGS DISCLAIMER:
This video is for educational purposes only. There is no guarantee that you will earn any money using the techniques and ideas mentioned in this video today. This is in no way intended to be financial advice. Your level of success in attaining the results claimed in this video will require hard-work, experience, and knowledge. We have taken reasonable steps to ensure that the information on this video is accurate, but we cannot represent that the website(s) mentioned in this video are free from errors. You expressly agree not to rely upon any information contained in this video. AFFILIATE DISCLOSURE: This video and description may contain affiliate links, which means that if you click on one of the product links, I’ll receive a small commission. I won't put anything here that I haven't verified and/or personally used myself. —————————- This video is brought to you by Tube AI Pro, an education platform for people who want to start & grow faceless YouTube channel using AI, and make a good living online out of it. Visit our website to learn more: https://www.tubeai.pro/
Join our 5-day Only AI Video Challenge: https://www.tubeai.pro/challenge CAUTION: We do NOT send you PM and ask you to invest in any crypto scheme on Instagram or other social media platform. Be wary of fake profile trying to impersonate Branson Tay. Always do your due diligence and research. Stay safe.

How To Make $300/Day on YouTube Using Only AI (Faceless YouTube Automation Channel)



In this video, I'll show you how to make $300 per day on YouTube using only AI with a faceless YouTube automation channel, without showing your face. ✅ FREE Cheatsheet: Top 100 Most Popular Faceless YouTube Niches
DOWNLOAD HERE ? https://track.tubeai.pro/niche ✅ 5 Day Challenge: Learn How I Use Simple A.I. Tools To Create Faceless YouTube Channel & Make $2k To $10k+/Month!
CLICK HERE ? https://track.tubeai.pro/challenge-yt1 —————————- ? Watch Next: How To Make Money On YouTube WITHOUT Showing Your Face in 2023 ($200 Per Day)
https://youtu.be/EmYAgjj61V4 How I Make Faceless YouTube Videos in 10 Minutes Using AI (Step by Step Tutorial)
https://youtu.be/c160m22mT9w —————————- ⚙️ AI Tools & Resources To Grow Your Faceless YouTube Channel Canva (AI Thumbnail Designer) ? https://track.tubeai.pro/canva Fliki (AI Video Creator) ? https://track.tubeai.pro/fliki VidIQ (AI Video Idea/Keyword Research) ? https://track.tubeai.pro/vidiq —————————- ✅ SUBSCRIBE to my Youtube Channel + Turn ON notification
? https://track.tubeai.pro/subscribe-yt1 #BransonTay #TubeAIPro #FacelessYoutube —————————- EARNINGS DISCLAIMER:
This video is for educational purposes only. There is no guarantee that you will earn any money using the techniques and ideas mentioned in this video today. This is in no way intended to be financial advice. Your level of success in attaining the results claimed in this video will require hard-work, experience, and knowledge. We have taken reasonable steps to ensure that the information on this video is accurate, but we cannot represent that the website(s) mentioned in this video are free from errors. You expressly agree not to rely upon any information contained in this video. AFFILIATE DISCLOSURE: This video and description may contain affiliate links, which means that if you click on one of the product links, I’ll receive a small commission. I won't put anything here that I haven't verified and/or personally used myself. —————————- This video is brought to you by Tube AI Pro, an education platform for people who want to start & grow faceless YouTube channel using AI, and make a good living online out of it. Visit our website to learn more: https://www.tubeai.pro/
Join our 5-day Only AI Video Challenge: https://www.tubeai.pro/challenge CAUTION: We do NOT send you PM and ask you to invest in any crypto scheme on Instagram or other social media platform. Be wary of fake profile trying to impersonate Branson Tay. Always do your due diligence and research. Stay safe.