MIT 6.S087: Foundation Models & Generative AI. INTRODUCTION

all right well um sorry we're a little

bit late but let's get started

um so welcome to the first lecture on

the lecture series called future of AI

Foundation models and generative AI this

is actually the second year we we hold

this class and so I started working on

this course before the recent hype and

breakthroughs of CHP um and I really

felt that we're starting to see kind of

a new approach to AI in the community

that was really going to change things

for real H and I think we started to see

that right

now um and really what I want to

accomplish in this lecture series is to

give you an understanding of why this is

happening right now what's the

underlying kind of change in perspective

and also going Beyond just kind of the

tip of the iceberg which is chtp so I'm

going to give you a deep but

non-technical introduction to these

subjects

and and uh last year when I gave this

course we were excited about uh t to

video and text image models right guess

we still are uh we were excited about uh

superum

robotics uh self-driving cars and AI

applied now to other domains as well

like genomics

Etc

and um of course A lot of these

breakthroughs even if they're in very

different domains they come back to this

underlying technal technological

achievement of foundation Ms generative

AI

um we going going to dive into last year

uh we were excited about chat GPT right

so we could ask it to write an engaging

introduction to an a lecture if we ask

the updated

gp4 it does it produces more text but

maybe does a better job as well uh last

year we asked it to produce a engaging

artwork of artificial

intelligence uh and now we asking the

newest version uh it also might be doing

a better better job it looks more

involved at least uh AR is subjective

but uh I think it's

better so if we were excited about these

things uh in you know early 2023 what's

happened right what H what's happened

during this year well uh a lot of things

of course there been a tremendous hype

so there's been a lot of uh you know

money pouring into these uh areas we've

had companies that only you know few

weeks or months old reaching A2 billion

dollar valuation which is a team of five

people um we've had excitement about

autonomous agents we're going to talk

about during this course as well like

GPT engineer that's able to plan and and

even act in a more humanlike way in

terms of

intelligence uh Nvidia that provides all

the different dpus right that these

models need have reached a a huge

valuation like the $1 trillion club with

we've seen uh sweeping uh regulatory uh

kind of Acts and and uh uh initiatives

right uh both from the White House and

the European Union for

example there's been a lot of drama in

the a space right openi for example the

CEO and the company behind CH CHP the

CEO was outed and then came back in so

maybe the transparency problem in AI

doesn't only apply to the models but to

the structures and companies behind them

and also you know we're seeing kind of

some hype and some winners u in terms of

the this new AI Technologies but also

now some companies are actually losing

uh users and usage right stack Overflow

for example people saying that kind of

is being killed by an AI That's training

its own data which is kind of

ironic uh of course one of the big

questions that remain is have you

reached artificial general intelligence

yet H some people say we have I think uh

there's still quite a long way to go but

we're going to try to also explore a

little bit uh you know what what can we

actually mean with a AGI and how could

we potentially reach it given the

technology that we have right now and

can we give some kind of very uh order

of magnitude estimation to when we'll

get

there so uh I'm a Richard I was uh born

in the land of abania which is Sweden

and uh before MIT I was at Stanford uh

for seven years and I did research as

well on AI and and this stuff also

started a company that that does uh

Foundation models inative AI in

commercial settings I hope to bring in a

little bit of that those perspectives as

well H for the last four years almost

now I think yeah almost four years I've

been at MIT uh where I do research on on

South press learning and and financial

models and that good

stuff

okay so uh quickly on this SC course

schedule so today we'll give an

introduction and kind of a history of AI

from a high level perspective as well

what's going on and giving some kind of

intuition and and then in the next

lecture we'll dive much more into

details about how these different

algorithms work and how we arrive at

these models that we use H after that

we'll do an in-depth analysis of CHP

like a case study then we'll do a

similar case study on image generation

and stable diffus

then right and these four first lectures

will be very similar to last year's

offering but then we adding on Jan 23rd

we going to talk about kind of emerging

Foundation models right it's going to be

a combination of them existing out there

probably not one single model to rule

them all so we're going to talk about

that especially how that looks in uh

industry and corporate setting uh so

Professor man Kellis uh will come as

well as you he's an expert on U biology

and genomics and then also artm working

who's uh an MBA from MIT he will talk

about autonomous agents and then we'll

have a fun kind of uh or it's going to

be a fun hopefully fun lecture on AI and

ethics which is of course perhaps a

little bit more fussy but we also bring

in regulations what kind of what's

happening in terms of the institutions

regulating AI H and after that we also

have a panel with manolis and artm uh so

should be fun all right so what will

cover well we'll cover all the busws and

new network supervised learning

representation on supervised learning

reinforcement learning genive AI

Foundation model self superus learning

and we'll try to put together with a lot

of applications a lot of intuition uh

because it really you know should be

non-technical and I think as well I want

to try to explain things in in simple

but true you know deep ways and I think

if you're not able to explain something

in a simple way you're actually not

doing a good job explaining it so that's

what we're going to try to do at

least today we're going to give you a

short succinct answer to what is the

secret Source behind Foundation

malternative Ai and then when we've done

that we're gonna ask how's the world

structured because how we think the

world is structures structured uh really

influences how we learn in the world so

we're going to kind of explore that from

a more philosophical perspective and see

how that actually leads us to uh

Foundation models and generative AI

and then we'll at the end we'll cover

two applications of how we can use this

in in in both research and in

business again right we're going to try

to use intuition examples and example

from both sciences and business and

hopefully you'll you'll understand why

the hype actually is real I said it's

last year but it is real and maybe we

understand what's actually just hype and

what's the the kind of more foundational

aspect of it right what

matters okay so I think in trying

to uh understand and and U you know have

a