Essential Skills for AI Coding from Planning to Production — Matt Pocock (@mattpocockuk )

Yeah, we good.

>> Okay, folks, we're at capacity. Let's

kick off. I don't want you waiting here

for 25 more minutes before we some

arbitrary deadline. So, welcome. My name

is Matt. Uh I'm a teacher and I suppose

now I teach AI. Um

we have a link up here if you've not

already been to this which is has the

exercises for the um stuff we're going

to do today. This is going to be around

two hours. So we might just sort of kick

off two hours from now. Is that right

Mike?

>> Yeah. Perfect. Um, and the theory behind

this talk or at least the thesis under

which I've been operating for the last

kind of six months or so is that

we all think that AI is a new paradigm,

right? AI is obviously changing a lot of

things. You guys are obviously

interested in this and that's why you've

come to this talk. And

I feel that

when we talk about AI being a new

paradigm, we forget that actually

software engineering fundamentals, the

stuff that's really crucial to working

with humans, also works super well with

AI. And this is what my keynote is on

tomorrow. Really, I'm going to sort of

be fleshing that out a lot more. And in

this workshop, I'm hopefully going to be

able to direct your attention to those

things and uh hopefully show you that

I'm right, but we'll see. Um, can I get

a quick heads up first? How many of you

guys um are coding have ever coded with

AI? Raise your hand if you've ever coded

with AI. Perfect. Okay. Uh, keep your

hand raised.

Uh, let's all uh share those armpits

with the world. Um,

how many of you code every day with AI?

Cool. Okay. Uh, ra keep your hand raised

if you've ever been frustrated with AI.

Okay. Very good. You can put your hands

down. Thank you for that show of

obedience. I really appreciate that. Um,

we are also being live streamed to the

Gilgood room as well. I've not uh did we

send someone up to the Gilgood room to

just check they're okay? Don't know. But

I see you. Uh, and there is a way that

you can participate which is we have the

um a Q&A. We're going to be doing kind I

have a sort of hatred of Q&As's because

they're not very democratic. The mostly

the sort of um most talkative people get

to um get to participate and share. And

so we're going to be going through this

um QA here. So why do we have to wait

till 3:45? The room is packed. The doors

are closed. 100% agree. And so if you

want to uh ask a question, we're going

to be I would like you to pile into this

async and then we can vote on each

other's questions and hopefully get the

best question surface so the for the

entire room to enjoy.

So I want to talk about first the kind

of weird constraints that LLMs have and

those weird constraints are sort of what

we have to base a lot of our work

around. Now,

there's a guy called Dex Hy who runs a

company called Human Layer, and he came

up with this idea, which is that

when you're working with LLMs, they have

a smart zone and a dumb zone. When

you're first kind of like working with

an LM and it's like you just started a

new conversation, you start from

nothing. That's when the LLM is going to

do its best work because in that

situation, the attention relationships

are the least strained. Every time you

add a token to an LLM, it's kind of like

you're adding a team to a football

league. You think of the number of

matches that get added every time you

add a team to a football league. It just

go scales quadratically. And that's

because you have attention relationships

going from essentially each token to the

other that are positional and the sort

of meaning of the individual token. And

so this means that by around sort of 40%

or around I would say around 100k is

kind of my new marker for this because

it doesn't matter whether you're using 1

million uh context window or 200k. It's

always going to be about this.

It starts to just get dumber. So as you

continually keep adding stuff to the

same context window, it just gets dumber

and dumber until it's making kind of

stupid decisions. Raise your hand if

that feels familiar to you. Yeah. Cool.

So this means that we kind of want to

size our tasks in a way that sticks

within the smart zone, right? We don't

want the AI to bite off more than it can

chew. And this goes back to old advice

like Martin Fowler in refactoring uh

like uh the pragmatic programmer talks

about this. Don't bite off more than you

can chew. Keep your tasks small so that

you as a developer, a human developer

don't freak out and don't start acting

and going into the dumb zone.

But how do you tackle big tasks? How do

you take a large task like I don't know

cloning a company or something or just

doing something crazy? And how do you

break it into small tasks so they all

fit into the dumb zone? One way of

course you could do is I mean kind of

what the AI companies maybe want you to

do or the natural way of doing it is

just keep going and going and going. You

end up in the dumb zone charging you

tons of tokens per request. You then

compact back down. We'll talk about

compacting properly in a minute. And you

keep going, keep going, keep going,

compact back down, keep going, keep

going, keep going. And I think that's

doesn't really work very well because

the more sediment, we'll talk about that

in a minute. So the theory here is then,

and this is what I was doing for a

while, is I would use these kind of

multi-phase plans where I would say,

okay, we have this sort of number four

thing here, this large large task. Let's

break it down into small sections so

that we can then kind of chunk it up and

do each little bit of work in the smart

zone. Raise your hand if you've ever

used a multi-phase plan before. Yeah,

really common practice, right? This is

kind of how we've been doing it.

Certainly, this is how I was doing it up

until December last year really.

And any developer worth their salt will

look at this and go, "This is a loop,

right? This is a loop. We've just got

phase one, phase two, phase three, phase

four. Why don't we just have phase n,

right?

Phase n where we essentially just say,

okay, we have, let's say, a plan

operating in the background and then we

just loop over the top of it and we go

through until it's complete. And this is

where um raise your hand if you've heard

of Ralph Wiggum as a software practice.

Okay, cool. Raise your hand if you've

not heard of Ralph Wigum as a software

practice. Actually, that's more like it.

Okay. So there's this idea called Ralph

Wigum uh which is kind of um sort of

based on this which is essentially

all you need to do is sort of specify

the end of the journey where you just

say okay we create a PRD a product

requirements document to say okay let's

describe where we're going and then we

just say to the AI just make a small

change make a small change that gets us

closer and closer to there and Ralph

works okay but I prefer a little bit

more structure so that's kind where we

got to in terms of thinking about the

smart zone. And that's kind of where I

want you to first start thinking about

here. Another weird constraint of LLM is

LLM are kind of like the guy from

Momento, right? They just continually

forget. They could just keep resetting

back to the base state. Let me pull up

this diagram.

I sort of I I I really should use

slides, but I just prefer just like

randomly scrolling around a infinite uh

TL draw canvas. Thank you, Steve.

Um,

so let's say another concept I want you

to have is that every session with an