How To Fine-Tune A Large Language Model (Step-By-Step)

AI is great at a lot of things. Sounding

human is not one of them.

>> Hasta la vista, baby. So, when it comes

to the promise of making our lives

easier, specifically when it comes to

writing things like blog posts and

articles and scripts or social media

posts or whatever, we still need to

spend a ton of time reviewing and

editing the work before publishing. But,

that can end right now. In this video,

I'm going to walk you through a

step-by-step process on how to make LLMs

sound more like you, Ron. Sorry, were

you talking to me? Unless, of course,

you aren't human. Are you human, Ron?

Anyway, this video is a complete guide

to fine-tuning your large language

models. I'll show you some use cases

specifically related to copywriting and

my own needs, but the applications

really are [music] limitless. So,

without further ado, join me on this

journey.

Before we get too deep, let me explain

what fine-tuning is. So, fine-tuning is

basically teaching an AI to act a

certain way, as opposed to know certain

things. So, imagine you hire a writer

for your YouTube channel to help you

write scripts. If you give them a Google

Drive full of research material, well,

they'll know all of the facts that need

to go into the project, but they still

don't know how to write or talk like

you. So, fine-tuning is the process of

giving that writer repeated examples of

exactly how you talk. And you give them

so much of that that eventually they

naturally start sounding like you.

That's not creepy at all. You may have

also heard of RAG or retrieval augmented

generation. It's another way to give

additional context and details to a

model. However, the difference between

RAG and fine-tuning is that RAG is more

like giving your writer a giant

reference manual with details and facts

that they can pull from, but it's not

going to change the way they actually

respond stylistically. They'll just now

have the ability to know and retrieve

that additional information. So, when

you fine-tune a model, you basically

start with one of the big general models

that are out there that's trained on a

ton of information. And then you give

that model specific examples of the sort

of outputs that you want from it. That

process is then repeated over and over

and over again until it learns your

tone, your structure, your formatting,

your jokes, your habits, and your

preferences. So, the simplest comparison

is basically RAG is like giving AI

access to your notebook, so it has all

the information that it needs.

Fine-tuning is more like giving it

acting lessons, so that it actually

becomes more like you and starts to

respond just like you. So, if you need a

model to know and understand a certain

details, you could feed those details to

the model through RAG. That's basically

what you're doing when you upload PDFs

or text files or things like that inside

of ChatGPT. If you want the model to

talk like you, RAG's not going to quite

cut it. You need to actually fine-tune

the model to achieve this.

Here's a model inside of a platform

called Nebius that I've already

fine-tuned. You can see this one's

called MW YouTube, and it's built on top

of Llama 3.3 7B Instruct. With this

particular model, I trained it on all of

my YouTube transcripts, like probably a

hundred hours worth of training data

transcripts from my videos. The idea

being that I can tell it to go write a

script on any topic I want in the style

of me, and it will write a script that

sounds the way I would talk on a video.

If I go into the playground here, this

is my fine-tuned model, and I can give

it a system prompt, and I can chat with

it just like I would ChatGPT. And I am

going to get to a step-by-step breakdown

in just a minute. I just want to finish

showing off this example that I already

did, and then we'll do one from scratch.

In my system prompt, I'm going to go

ahead and say, "You are Matt Wolf. Use

correct punctuation and markdown." And I

can give it a prompt like, "Create a

detailed outline with six to eight H2

sections for a 10-minute YouTube video

on Nvidia's dominance in AI." "Label

with ## headers and three to five

bullets per section." I just wanted to

give it some extra details cuz like with

any large language model, the more

context you can get it, the better the

output you're going to get. Now, if I

submit this, theoretically, it's going

to write a script for me that sounds

similar to how I would write. And that's

exactly what it did here. But, I can

also compare it to what it would have

looked like if I didn't use a fine-tuned

model. So, if I go to compare here, and

then I switch this second model on the

right to, let's go Llama 3.3 7B

Instruct. That is the same model that I

previously fine-tuned here. Let's go

ahead and clear our chat on the left,

and I'll give it a prompt, "Write an

outro and closing to a video that wraps

up why Nvidia is so dominant in AI." So,

let's go ahead and give it that and let

it write an outro for us, and we can see

side-by-side comparisons of how it would

write that outro. Now, the un-fine-tuned

model did a little bit better with

formatting, but that actually makes

sense because I gave it just a ton of

transcripts from YouTube videos. And if

you've ever looked at the transcripts

from YouTube videos, there's usually not

much formatting or punctuation, so it

kind of overfit for that. But, if we

look at the normal Llama response, "As

we conclude our explanation of Nvidia's

dominance in the AI landscape, it's

clear that their success can be

attributed to a combination of strategic

innovation, forward thinking,

investments, and relentless pursuit of

technological advancement." And then the

version that's trained to sound kind of

like me. So, there you have it. "That's

why Nvidia is so dominant in AI right

now. They've been preparing this moment

for over a decade. They're the leader in

the hardware that's required to train AI

models. They're the leader in the

hardware that's required to run AI

inference, and they're even building

their own AI models." Now, I'm not going

to read the whole thing, but it's even

trained on some of my old calls to

action that I used to put in my videos.

If you've watched a lot of my videos,

you know that this reads like me.

Hopefully, you found this video helpful.

Hopefully, you feel more looped in on

the whole Nvidia ecosystem and why

everybody's making such a big deal about

Nvidia We can even see here it went on

to say, "If you like stuff like this and

you want to stay looped in in the AI

world and the latest AI tools, get the

TLDR of everything that's going on in

the world of AI, check out

futuretools.io. Thank you so much for

tuning in. I really, really appreciate

you, and thanks again to Wirestock for

sponsoring this video. I'll see you guys

in the next video. Bye-bye." Now, it's

funny because Wirestock hasn't sponsored

the channel in like a couple years now,

but that's in the training data. So, it

wrote an output that sounds like what I

would have written. Now, granted, the

formatting sucks, but that's my fault on

the way I trained in the data. I didn't

clean up the data. I just gave it sort

of unformatted transcripts, so it gives

me back unformatted transcripts. Garbage

in, garbage out. So, real quickly,

here's how I trained that one, and then

we'll start a new training from scratch.

I found this random website called

downloadyoutubetranscripts.com.

It looks pretty quickly made and slapped

together, but for seven bucks, I was

able to download all of the transcripts

from my YouTube channel in one click. It

exported all of those transcripts as

just one giant text file. You can also