MIT 6.S087: Foundation Models & Generative AI. ECOSYSTEM

uh welcome to the fifth lecture on

foundation

multii and today should be especially

fun because we have two guest lectures

so

uh uh Professor M Kellis from MIT will

show up and give a talk about biology in

Ai and the AI Frontiers Frontiers in

computational biology then artam is here

he's flown in from uh Silicon Valley to

talk about autonomous agents so it

should be a lot of fun and and I'll

start off talking

about a framework for foundation models

right so will it be a single Foundation

model like a single brain to rule them

all will open AI have a monopoly on AI

or not like what will be if if not right

what kind of foundation most will exist

what can you leverage and I think it's

going to be useful if you're a

researcher but also especially if you're

in business understand what kind of

foundation models technology

technologies that will exist and how you

can uh use them and leverage them and

this is based on a talk that I've been

giving to uh priority firms and kind of

C Level of bigger companies around uh

how to survive Daya explosion okay like

a lot of things are changing now and and

people feel it's a different type of

technology and it's changing the

landscape and how do you survive survive

and prosper in this new age so that's

what we're going to figure out

today um okay so You' seen this before

but a primary so what was the

Breakthrough that happened that allowed

us to do all of these different ad

advancement that that we've been seeing

well we've been using ourselves as a

reference frame right to to ask the

critical question how do we learn about

the world how do we go from being a

blank slate baby with no knowledge about

the world fairly useless to becoming a

useful knowledgeable adult what what

enables us to learn from the world that

we uh interact with right this is a key

question in

AI uh what we've been saying is

basically

that what's responsible for giving you

most of the knowledge that you have

about the world it's not your parents

it's not your teacher it's not Academia

so it's not supervised learning you not

you don't learn from experts okay so

that's not that's that's a technology we

that we tried for a long time but it's

not the answer it's part I mean it's

helpful but it's not the answer also

it's not a DNA right it's not your

genetics it's not your immediate

environment and you're trying to

optimize your goals in that environment

so it's not reinforcement learning

reinforcement by itself is not the key

answer to how we learn about the world

right it's helpful but is not the the

main uh uh responsible part so what is

it right well it turns out the most

things that we know we learn by

ourselves so this is the key you know

Insight that allows us to do all the

things we're doing right now and this is

possible by defining company like

meaning by the company it keeps so if

you take a dog right you don't know what

a dog is from your parent telling you or

your emotions getting you you learn what

a dog is by observing dogs in different

context correlating conting dog with

other cont Concepts so what you get what

how you understand a dog is a dog is

something that's walked by an owner with

a leash it's something that has an

antagonistic relationship with cats it's

something that chases fris with those

theone this is what allows you to

understand what a dog is and this is how

you define what a dog is and what you

get by this is a very relational

understanding of meaning right and as

you learn about dogs by correlating

contesting dogs with other Concepts like

cats you intern learn what cats are so

it's kind of self-referential and and

very very powerful you can pick up know

across modalities of course because the

word dog will be Ed or named more in

context where dogs appear so it's

extremely

powerful so what does this lead to well

it leads to that the more relations that

you're able to understand the better

understanding you get of meaning you

understanding what a dog what a Love Is

Right you understand what love is helps

you understand what a dog is because an

owner loves his dog so it's like a lot

of synergy networks affected play okay

this is the key Insight here

so why not take a you know huge model

with as many parameters as possible and

train train it on as much data as

possible to learn all of these different

relations to get the most precise and

Powerful understanding of meaning and

then use this model basically everywhere

right and this is what a foundation

model is right is it a key breakthrough

you know throughout generative Ai and

everything that we're seeing right now

and also of course your brain is a prime

example of

this okay so here kind of going divert a

little bit I've thought about this

before so what does this actually mean

right what does it mean now that we have

these Foundation models that learn

relational meaning and the more data

you're training on them on the better

they get somehow and just get bigger and

bigger so one thing that you know that's

happened right is that there been a

complete change and transformation in

the research world so six years ago when

I was at Stanford there used to be you

know one research lab data set and and

AI model for different language task

right so this start this start off

initialing language you would have one

data set one model one research team

working on translation another one

working on question answering a third

one of sentiment analysis and then a

fourth one of predictions right in

isolation isolate effort but they ask

them hey you know this seems to be a

shared perspective here and a shared

Intelligence being language modeling or

language intelligence can we optimize

that instead and know pull our data and

efforts together and optimize single

intelligence that synergetic because we

don't have a you know separate brain for

each language task so real intelligence

should be General and and and synergetic

right so that's what I did we're able

now to build this language models that

are trained and really understand

language in a deep intelligent way so

all the different tasks that we care

about in language are just downam tasks

of this real intelligence this real

Foundation

model okay and of course even though you

know a lot of companies are behind in

this the same applies to companies and

businesses if you come to a company

today right how the think about AI or

data or Technologies is typically very

isolated efforts you know solving

something right so if you if you go to a

retailer for example they may might have

a SE separate service and data data set

and team working on product search

another one for recommendations a third

one for assortment planning right

campaigns marketing but these of course

are not separate intelligences these are

very very synergetic right if you're

able to recommend the right products at

the right time to user recommendations

that should influence your marketing

like what product should you Market who

so someh you want to build right this

intelligence around a company as well

and this what you can do right so you

can build a single brain around an a

company to really get at all the

synergies and get the most

performance okay so what this leads to

is basically right so at Stanford now

the only research team that survived and

prospers are the one focusing on

building a foundation all around

language right like the key core