How to Get Users and Grow - Alex Schultz Video Lecture | Startup School Online by Y Combinator - Entrepreneurship

Good afternoon everybody.
Today, we have Alex Schultz,
VP of growth at Facebook, and
probably the world expert on growth,
as he can vouch from this weekend,
the person I go to when I have
a really hard growth question.
Many people who work at Facebook, many of
these founders at Facebook in fact, and
very early employees have said hiring
Alex has been one of the secret
successes of Facebook that
is not well understood.
You joined almost ten years ago, and
I think at that point Facebook
had less than 100 million users?
>> Yeah.
>> Way less than 100 million users.
Facebook now has 1.6 billion and
Alex has been after the importance
of having a great product, maybe
the second most important driver of that.
>> Having Naomi,
these very important people.
>> And the team, anyway, thank you
very much for coming to speak to us.
>> Thank you.
I did not pay Sam for that introduction
and I would note that Havi is my boss so,
hi Havi.
But no, Havi and Naomi, Danny,
they are these people who've been in
the growth team for four years and
then people like Blake, and Martin, and
Ray and James Wang who set the team up.
And it's very nice, but I am a VP on
growth, I am not the VP of growth.
So you've had an amazing lecture series.
I just watched all of them yesterday to
make sure I didn't say anything without
actually having seen
what everybody else said.
And it was totally awesome.
I loved what D'Angelo had to say, so
that lecture, if you weren't here, if you
didn't watch it, I strongly recommend you
go back and see what Adam D'Angelo said,
because I could not emphasize his
points on metrics more than enough.
And so what I'm going to try and do for
the first half is go over some of
the points that have been made already and
emphasize what I think is most important
when you're thinking about growth and
what my take is and our take is on
the Facebook growth team on those.
And then in the second half, try and
talk to you a little bit more about
tactics that have been
particularly useful.
The caveat that I think you should take
onboard is what Stewart from Slack and
Flickr said.
Which is to some extent, you do have 100
million monkeys bashing at typewriters,
and I am very, very lucky to be
on the growth team at Facebook
to get to work on a product like Facebook
that has the retention that Facebook does.
That has everyone in the world wanting
to use the product and to say that,
I drove it or any of us drove it beyond
their being great product market fit
is perhaps not true.
I can't tell you what the AB test
is if we didn't even have a growth
team at Facebook.
So what matters most to growth?
The single most important
thing to growth is retention.
Adam commented on this in his talk, and he
showed you cohorts from some real company.
But here is a stylized cohort of
how you should look at retention.
The number of days from
acquisition is on the x axis.
On the y axis is the percent of
people who has still monthly active.
I care a lot about using monthly active,
not daily active, not weekly active.
There are really good arguments for using
daily active that people say all the time.
Because it's about intensity of usage.
But what really, really matters is
that you got skin in the game and
people come back to your
product at least once a month.
And then you build on top
of that engagement revenue,
whatever your key metrics are which
we'll get to in a minute, but firstly,
I believe monthly active is
the most important thing.
The second thing is how do
you rally calculate it?
So I've got a real graph for
revenue in a few slides from Facebook from
the first 250 days of our ads products,
which I'll show you.
But the thing that's really,
really important is to say that,
it's very easy to calculate this curve
even if you have 1,000 or 100 users.
And what you do is you look at all
of your users on the first day
after they signed up.
Were they monthly active?
Yes, clearly, right?
The first 30 days everyone's
going to be monthly active.
I'd prefer if you defined a month as 28
days, by the way, because multiples of
seven is really, really good for
these kind of metrics because of weeks.
But for the first 30 days, everyone
has been active on their first day,
on their second day, on their third
day as a monthly active user,
whether they came back or not.
And then what you do
is you look at day 31.
And obviously, only some of your users
will have been active for 31 days.
Everyone who signed up a day
ago was not active for 31 days.
So what percentage of them
were active on day 31,
what percentage of them
were active on day 32?
And as you go further and further out in
the number of days since registration,
the number of people who have been
on your service that long declines.
Which means that your metric is
going to get noisier, right,
as you get out to the far
right hand side of that curve.
But that doesn't mean you can't get
a handle of where things flatten out very,
very early on with very
small numbers of users.
The number one criticism people give me
when I talk about retention as a metric.
Or even when you look at what Adam
presented in his presentation earlier is
you need lots and lots of users.
So you need millions of users to
calculate that, that is not true.
We did it with thousands of users in the
early days of advertisers for Facebook for
example.
Or you need to have had all of those
users on for a very long time.
Look at Adam's graph it was multiple years
with months, and months, and months.
No, you can do it day by day, and
you can get to a really good place super,
super fast, using this methodology.
So what you're looking for
in a retention curve?
You're looking for it to flat line above
zero, not flat line like a heart monitor.
You want it to flat line at some line
that asymptotes with the x-axis.
There are three types of
these curves that I've seen.
The first type is where
asymptotes to the x-axis.
Like really, really close,
goes down to almost zero.
You hear about a lot of those companies.
You think about most games,
most games are smash hits.
How many people use FarmVille now versus
how many people used FarmVille at its
absolute, please tell me
you know what FarmVille is.
[LAUGH] Yeah, okay, all right.
How many people used farm,
I'm feeling old.
I had hair when I first did one of these.
How many people use FarmVille now
versus how many people used to use it?
FarmVille was a smash hit, then there's
other ones which look like this,
where they stabilize out.
So you think about Minecraft,
right, or World of Warcraft.
These games that somehow have network
effects and people stay in them for
a long time.
The other version for this curve I've seen
is the ones where they go down, they look
like they're going to hit the x-axis and
then they re-accelerate and they go up.
There's a few different things that can
cause that right, so you have this,
this overwhelming thing that is driving
you down to have zero retention and
somehow the numbers go up again.
So things that drive that for
businesses, and
you actually I think
mentioned this in your intro.
One, network effects, so
when all your friends get on Facebook,
you're much more likely to use Facebook.
When all your friends get on Twitter,
you'll more likely use the Twitter,
WhatsApp, Messenger.
Whichever social service
you're talking about,
there are network effects
that really matter.
Network effects also
count in marketplaces.
When you have enough sellers,
you can have enough buyers.
When you have enough traffic,
you get more sellers.
And so as marketplaces go, often what you
find is these old cohorts that signed up
resurrect and come back to the service.
So network effects can drive the curve
up after it's flatlined somewhere or
is heading towards zero.
The second thing that can do this
is opening up a new interface.
So think about it.
A lot of people release
their apps first on iOS.
I actually loved Diane's talk last week
for this, by the point that he made of
starting with Nokia, focusing on Android
because it's what most of the world use,
but in Silicon Valley we typically build
our apps first on iOS more often than not.
And then when someone releases
an Android application, and
this was something I saw with
Instagram years and years ago.
But when you release
an Android application,
people might have tried it on an iPhone
And then traded their iPhone for
an Android and
no longer be able to access your service.
People might've tried your
service on a friend's iPhone and
then when you add an Android they
come back and use your service.
So that's another thing that
can drive you up again.
And then the third thing that I've seen
on this curves that drive this curves up
again is adding categories,
so think about Amazon.
There was a really good discussion,
was that Stuart earlier or?
Someone earlier in this talked about
Amazon, and how Amazon started with books
because there was this large
inventory available of books.
But then they added in another category.
And then they added in
another category over time.
And imagine, there will be people who came
to Amazon to buy that one book, but then
two years down the line Amazon suddenly
had kitchenware, I don't know, or fashion.
And they came back and
then they bought fashion and
they brought them more use cases
to come back more monthly active.
This was true with eBay.
We added motors.
I remember being at eBay and
we had to do all of our reporting on
gross merchandise volume with and
without motors.
And no one had ever believed that you
could buy a car online, and eBay did it.
And it created a whole new use case for
people, many of whom were existing eBay
customers, who now had one more
category they could shop in.
So you want to curve the asymptotes
to a line parallel to the x-axis.
You can do it with far more
limited amounts of data
than most people think you can by using
the technique I outlined earlier, and
I'll show you a real world
example in a few slides.
And there are three types
of curves that I've seen.
Curves that asymptote like this,
awesome, your company will survive.
And how big will it get?
We have another question.
Curves to go straight to zero.
That's okay, you could be in games and
your job is to produce one hit
after another after another.
EA doesn't expect whatever fee for 2012,
they still have lots of users
now if they've released FIFA 27.
So it's okay, there are businesses
where hitting the x axis is okay.
But you need to know you're
in one of those businesses.
And then the third that I've
seen is the curve going up.
And that can be driven by network effects.
That can be driven by being
available on more platforms.
That can be driven by adding categories
to your marketplace or service.
From this falls out a brilliant
way to look at growth, and
this guy Danny Ferrante who was at Yahoo.
Who was one of the first members of
the growth team runs core data science at
Facebook, he's totally awesome.
He's really the brains behind
the operation I'm the mouth.
Danny came up or brought this to Facebook,
and it's a simple way to look at growth.
We were looking at growth as how many
users were we growing by week on week?
We looked at it on an absolute basis.
And looking at it like
that we were like okay so
we're signing up whatever
it was a million a week and
we're growing by whatever it was
800,000 a week I don't know.
Cool, we're churning 200,000 people.
And we never actually
looked at it in this way.
When we change the way we look
at things to say well okay,
the net growth is 800,000 a week,
the new users are 1 million a week.
But what is churn and resurrection?
And what we actually found at the time,
and
I can't remember the absolute numbers so
I'll just make some up.
But imagine you're churning 1.3 million.
So not 200,000, 1.3 million.
You're resurrecting 1.1 million.
So plus 1 minus 1.3 plus 1.1,
you've got net 800,000 in growth.
But that 800,000 in
growth is being driven,
I think I did the math wrong, but
800,000 in growth is being driven by
a different order of magnitude from churn
and resurrection than you realized.
And so that helps you focus on
how do I stop people churning?
How do I get people to come back to my
service who are not coming back to my
service right now?
So once you have your retention curves,
and
once you start to have
a business with retention.
You need to look at
your growth accounting,
not just how many users are we acquiring.
Make sense?
Overall the number one thing
I focus on for growth,
and the one thing that we focus on for
growth is retention.
So what gets you retained
in any given service?
Magic Moment.
What is the moment when you used Airbnb?
What is the moment when used Door Dash,
when you used Slack,
when you used Facebook,
LinkedIn, WhatsApp?
Whatever the product, what is that
moment when you went, yes, I've got it.
Right, on Facebook it's friends.
You see that first friend,
that person who I've got completely
the wrong crowd for this but the person
from high school who you lost contact
with as you grow older and moved away.
But you see that person and you go, wow.
I haven't talked to them in years.
Let's see what's going on in their life.
And they've had a kid and lost their hair
too, and you feel better about yourself.
That is a magic moment on Facebook.
So how quickly can you drive
the user to that magic moment.
You look at Airbnb, right?
They talked about the delight
that they give people.
They talked about these different
screenshots of the stages of the user
journey that they've created
as a map of the user journey.
Getting that first person
reserve your house.
Staying at that first property,
those are our magic moments.
eBay, buying your first item
from a stranger online.
Nowadays that doesn't seem crazy but
back when I was working at eBay in the UK,
that was revolutionary.
And people really, their eyes lit
up when you talk to them about it.
In Germany they had this great ad, tights
by minds that showed the build up of when
you are waiting for the auction to finish
and view to make your first purchase.
That was such an adrenaline
endorphin moment.
But users loved it.
Nowadays, it seems a long time ago.
Those magic moments are what
you need to look for.
So how do you find them?
Well you can find them in
the non scalable way which is
don't they just sound relatively obvious?
You can actually think through what
is the experience on my product?
What do I want people to do?
You can ask your first users, okay,
you're keeping using my product.
Why are you keeping using my,
what is the thing that makes
you happy to use my product?
What was that first moment?
It's non scalable but it generally works.
When you ask people what is
the best thing about Facebook?
The first thing they say is
connecting with friends and family.
In all the research,
why do you like Facebook?
It helps me connect to my friends and
family.
So you can do it the non-scalable way or
you can do it the scalable way.
Look for correlations.
So Marcus talked at
startup school himself and
he said that we focused on getting
people to ten friends in 14 days.
It's a smooth curve, right?
Number of friends on the x axis and
percent retained on the y axis.
It's not like there's a step change at
ten, if you get nine you don't retain.
If you get 11 you do and
it all changes at ten.
No, it's clearly a smooth curve and
we picked a point on the curve.
But there's a strong correlation between
number of friends you've got in the first
14 days and whether you retain or not.
There's a correlation between number
of friends in the first day but
there are correlations for
any product like did you place a bid,
did you do a buy it now or
did you list an item on eBay?
Was the thing that was completely
correlated with retention for eBay.
You can imagine, I don't know, I mean
DoorDash was founded at Stanford, right?
You could imagine for DoorDash.
You've placed your first order and
that order arrives to your door.
And whether you place an order or not and
how many orders you place you place in
the first week, probably they are actually
correlated with whether you retain or not.
So you need to look for the magic moment
of your product by talking to users and
then try and
validate that with correlations.
Makes sense?
Cool, I love this.
I put it in every presentation I could.
So I'm lucky enough to have been trained
in physics at Cambridge in the UK.
And my favorite and my favorite lecture
series, not my favorite lecture but
my favorite lecture series was in this
thing called dimensional reasoning.
And so it's easy for
me to stand up here and say, okay,
you need to get flat retention and
then you have a viable business.
That's not actually true that's
kind of a glib statement.
If you're producing a social network and
you have 5% retention you're
not going to long term have
a tremendous social network.
Or a messaging app like what use
would WhatsApp be if only 5% of
your friends actually used
WhatsApp who signed up for it.
I mean it would be useless, right?
And there's no way we could release 1
billion plus numbers of active users if
that was true for us.
On the other hand, if you have
a marketplace that is selling fashion.
And you have 5 or 10% retention but
those people spend a lot of money
you have a totally viable business.
And so the level of retention
you need varies dramatically
by the category that you're in.
Obviously, the higher the better
is true in everything.
If you can have an Ebay
type business model or
Amazon type business
model with 90% retention.
You probably end up bigger than Amazon and
that would be incredible.
But I think the point I like to make is,
you can figure out and
I love this as an inspirational story.
So Geoffrey Taylor was
this British physicist,
he was on the edge of
the Manhattan Project,
he wasn't in the corp, he ended up winning
a Noble Prize later for a different area.
And the US and
Russian governments were releasing these
images to say this is how big our bomb is.
But they weren't releasing the actual
power because that was a closely guarded
state secret.
And so what Geoffrey Taylor did he
used this technique called dimensional
reasoning where you look at the units in
the metric you were trying to achieve.
So joules, right, for energy, it's joules.
Kilograms times meters
squared over seconds squared,
that's what joules decomposes into.
In this case, kilograms,
what are you going to use?
The volume of the air times the density
of the air, which handily, is one.
So you've now got meters cubed times
meters squared over seconds squared.
So meters [INAUDIBLE] over
seconds times seconds minus 2.
The time is on here, the diameter,
the radius of the sphere is on here.
You just plug the radius in as m,
you plug the diameter of the time in as
seconds, and you end up getting
25 kilotons as the output.
You get 10 to the 14,
I think is what it works out as.
And it turns out the bomb was 21 kilotons
as measured by the US government.
He could take one photo and
figure out the most closely guarded state
secret in the United States at the time.
You can figure out what retention you
need to have if he can figure that out.
>> [LAUGH]
>> So
here is what I promised,
this is an actual curve.
I presented a few different events, so
I taken off the numbers because
I'm worried they're State secrets.
But this is from 2007 to 2008.
I joined the week we started
our ads business at Facebook,
our self-service ad business by us.
The pink speaks to the right-hand axis,
the blue speaks to the left-hand axis.
What the blue is is how many advertisers
had been in the system at least one day,
two days,
three days because the x-axis is number
of days from acquisition make sense.
So the thing has only
been around 250 days, so
0 people have been
advertisers 251 days ago.
And when they turned 51st date and
all the people have been active on day 0.
And then each of those pink data points is
the total revenue achieved from every user
who had registered.
Who was using the service
on their 30th day, or
their 31st day divided by how many
users signed up 30 or 31 days ago.
Not how many were active, right?
How many signed up.
So total in the cohort,
who signed up 30 or 31 days ago.
You divide the revenue they made in
the day by that number of users.
And that gives you that shape
of curve that's up there.
I stumbled over my words a bit there.
I promise you if you look over
it again it makes perfect sense.
I did this with Danny back in 2008.
The really important things I want you to
take away from looking at this curve is,
look how flat the pink line is.
Like I've been talking about
curves flattening out,
this is an actual curve flattening out.
And look how quickly it's flattened out.
By 100 days you can pretty much predict
what the line is going to be at 200 days.
I've seen this over and over.
I've seen this with engagement.
So if you divide time spent
by the same denominator.
I've seen it actually for sentiment.
If you have large enough
sentiment survey and
you divide it by that same denominator.
I have seen it for posts.
I have seen it for
photos uploaded, messages sent.
You can use the same techniques
that I talked about for
determining how much retention you have
and whether your business is viable,
to figure out what is the lifetime
value of one of your customers.
If I was to acquire the total
addressable market for
my business today,
what revenue will I be making in 100 days?
That's what this curve a allows you to do.
You can predict it for posts.
You can predict it for page views.
Time spent, whatever you want.
The same methodology works for
engagement, that works for retention.
And, actually, retention is often
the biggest driver of this metric.
And we did this with very little data.
Later on, there's a stat we are okay
releasing, which was end of 2012,
we had about 700,000 active advertisers.
Beginning we had about
300,000 active advertisers.
And this was five years before that.
So you can imagine there was a very small
data set that was behind calculating this.
The last thing is I said things
get noisy the further up you go.
You can literally see that on this graph,
right?
The variance in the numbers the further
to the right you go on the graph.
So now, how do you operate for growth?
Most important thing is have a clear goal.
Brian, who is awesome on
growth marketing on Facebook,
great guy, he always says, if you are a
growth team your product is a number.
One of the most brilliant things I think
that Mark did in the early days of
Facebook was he aligned the entire company
on monthly active people as our goal.
Now why does this matter?
It matters for a bunch of reasons,
but the biggest and
most important reason is I
think it aligns everyone.
When you have a company of more than
two or three people, or by the way,
a team of more than two or three people,
certainly when it gets to the hundreds.
You can't control what anyone does.
I mean, you could probably
control what one person does, but
then the other 199 would do
whatever the hell they wanted.
So what do you want to do?
You want to make sure everyone is
pointing in the same direction, and
that direction is generally right.
So let's think back to
the early days of any company.
Whether it's Facebook,
whether it's Air BnB, LinkedIn,
whatever you want to look at.
It would be completely reasonable
to have a goal of revenue, right?
You want your company to stay alive,
you want to make enough money that your
company can still be viable, that you can
employ people, that valuation goes up.
So a reasonable person could say,
I think we need to drive towards revenue.
At the same time,
you could have a metric that was like,
if you're Air BnB number of hosts.
If you're Facebook, number of posts.
If your DoorDash number of orders.
I don't even know what this company's
metrics are beyond Facebook's.
You could easily have a production
metric or a consumption metric.
You could have a time spent
in our service metric, and
every one of those metrics would be a
completely reasonable metric to drive for.
So you've got ten reasonable people,
maybe five of them think you
should have monthly active people.
One is really focused on revenue,
two are looking at time spent and
two are looking at number of posts.
Everyone is pushing in a reasonable
direction that is vaguely aligned but
probably will have conflict.
If you want to get ad in front of
users as quickly as you possibly can
you may not necessarily want to put
friends in front of them immediately.
If you want to get them to post
you may dial up the composer and
push down the ads.
I don't know.
These are trade offs that you could
be making.
And if the whole team isn't clear what
the goal is, they will be in conflict.
So having a clear goal that your
company looks to drive is really,
really important, and
having that goal be MAP in my opinion is
the best metric that you can drive for.
Because it is that once
a month visitation,
that base level you can drive off of.
If you're going to do
a consumer internet company.
But fundamentally,
the most important thing is pick a goal,
your team's product is a goal,
focus on that goal.
Align the company around that goal and
then you are going to
operate correctly for growth.
The other thing I always say is
you don't need a growth team.
If you're a small company, if you're
even series A, don't have a growth team.
Don't hire someone like me.
The whole company's job should be growth.
I'm very happy at Facebook,
so I don't want a job offer.
But the whole company's
goal should be growing.
The purpose of a start up is to grow.
So, what is the number you want to grow?
And align the whole company
against growing that number.
Does that make sense?
Now, Naomi is amazing and Naomi is
someone people don't know that well
out of Facebook, but she runs product and
engineering for growth.
She's brilliant,
she's been at Facebook 12 years.
So, when you say I'm original,
I feel not original everyday.
She sits like three feet from me,
she's amazing.
Anyway, she has this approach that
we use to iterate through our growth
planning cycles,
which I think works really well.
Understand, first of all
understand what is going on.
Look at all the data you have,
talk to your users.
Ask your users what they think.
Understand everything that is going
on with your product that you can
possibly understand.
What matters?
Retention.
Are you churning too many?
Do you have the retention?
Is the revenue going up
on a per user basis?
What are the acquisition
channels you're getting?
Which channel are people
resurrecting through?
Are they resurrecting through email?
Is it push notifications?
What's going on?
Understand what's going on for
your system.
Have that all tracked.
Then identify, look through all of those
different metrics, all of that different
research you get in, and
identify where there are opportunities.
Where do you see big drop-offs?
Do you have a lot of people with
push notifications turned on
who are not receiving push notifications?
Do you have a lot of people who
are receiving push notifications who
are not clicking on them?
Do you have loads and loads of social
media traffic, but no search traffic?
Are your email viral invites working but
your SMS ones not, and
what is the difference between those?
Go and look at the data and identify
where there are gaps, and then try and
double down on fixing those
gaps in the execution phase.
Now to this day, this is still
how we run growth at Facebook and
you can even see a a talk Naomi and I
gave online nine years ago I think at FA.
Eight or nine years ago, where we
talked about using this approach and
we have kept it going.
Understand what's going on, identify
the opportunities in that data you've
understood, and then execute against it.
Another really interesting thing I think
was said earlier is data debt sucks and
you're going to regret it later.
And your intuition is going
to run out at some point and
then you're going to want data.
In 2009,
we actually shut down all of the work
the growth team was doing during January.
And we only spent the time logging, and
making sure every critical
flow on Facebook.
Registration, at the time we
didn't log clicks on emails.
So, click some emails, it seems
obvious now but we didn't have it.
Clicks on the emails,
did you then actually log in or
did you fail in the password entry.
All of these different things,
we logged them so
that we could understand
what was going on.
And that was super critical because
actually in March that year,
we had a massive growth slow down.
And we would not have been able to
understand what was going on if we hadn't
done the logging in January.
Because what went on was
actually super new ones and
without that logging in place,
we wouldn't have known.
Get the data logged so
you can run through this process.
So speaking of data, I am very,
very lucky at Facebook.
I do growth marketing, I've run
the awesome internationalization team.
But I also run the analytics team for
the company and
that's been a couple years now.
When I talk about data
to people internally,
I like to say these three points.
There's a meme in the valley that you
operate either for user experience or for
metrics.
And I really want to bust that meme.
Data when used in the right way
gives you empathy about your users.
One example I've talked about in
the past is we launched a product on
Facebook that you may have seen
on your big 42 inch monitors.
No one has a big 42 inch monitor in here,
but I'm sure some of you do at home.
That has on the right-hand side all the
friends you can chat with on Facebook, but
at the top it has this list of stories
that's ticking over called ticker.
When you're on a small screen,
that product used to collapse into
the right-hand column of Facebook and
push things down so that you could
still see these stories ticking over.
That product worked great for
all of us who had
huge numbers of friends who are creating
a lot of content on big screens.
But if you went to the Philippines where
there was huge usage in Internet cafes and
people run narrow screens,
the majority that amended up having
that in the right-hand column.
And the majority of them were
low engagement with Facebook and
couldn't see people you
may know below the fold.
And so, we actually lost friending amongst
users who are very different to ourselves.
And we only figured out what was
happening by looking at the data and
saying, why is this bad on small
screens and awesome on big screens?
Why is it not so great for low engagement
users and super for power users?
It was because we used the data to
understand these users who are different
to us here in Silicon Valley.
Now think about it,
I presume I don't know.
I'm seeing a lot of Apple products here.
I have a Lenovo at the moment.
But most of you probably have a iPhones.
Who has a iPhone?
Who has an Android?
You're the best.
Most people in the world
are using Androids.
But everyone in this room has an iPhone,
first and foremost.
You don't understand.
I'm actually forgetting
how to use iOS now,
which actually is a problem in my job.
But I've got an Android because
the majority, maybe not the majority,
a vast number of our users use Android,
as Jan said in the last talk.
And I need to understand what
the Android operating system offers.
But I can't understand Android and
iOS well as a user at the same time.
So, I used data to understand
what's going on on iOS.
You need to gain empathy for your users.
And research is great, and
talking to your customers is great.
But if you listen to the Twitch guy,
Emmit, talk to your users then go away and
build the product to serve them.
And look at the data, in my opinion,
would be the next thing to validate
if you built the right thing.
If you're doing the right thing,
if your research actually ties up with
what your users have said to you.
So, data gives you empathy.
It's not a trade-off between metrics and
users.
If you build the right metrics,
you are optimizing for the users.
Two, data predicts the future.
Over and over and over again,
I love this quote.
It's a science fiction outright thing.
But like the future is here,
it's just not evenly distributed yet.
So, those curves that
show you the retention.
When I said you can look at your
total addressable market and
say, if I sign them all up,
how much revenue would I get?
Or you just look at who
you signed up today.
Let's say back in those days, we signed
up a thousand advertisers in a day.
I could tell you how much
revenue they would contribute
hundred days out because of that curve.
Now, also you're users
are going to ramp up.
So, a user who shops in one category
if you're eBay or if you're Amazon and
then buys in a second category and
a third category is clearly
going to get more valuable, right?
Shopping the shop, driving up same store
sales is a common technique in e-commerce.
Okay, so
how many of your users are increasing the
number of categories they're shopping in?
You see someone sign up.
What likelihood is it that they're
going to add extra categories, and
what is your revenue
going to be in the future?
You can predict that with data.
AB tests are another great thing.
See all these people walking in
the science march believing in science.
And then when you apply the scientific
method to their product and
run an AB test, they ship the thing
that looked bad in the AB test
because they didn't believe the AB test.
I see that with all kinds
of startups I talk to.
AB tests tell you what's going to
happen when you ship them to 100%,
use them it works.
Finally, and this is something Mark
actually added recently for me.
Data helps you make decisions faster.
If you can not as the CEO, have to have
a conversation over something where you
could just run a test, your company
is going to be able to move quicker,
because every decision
won't have to come to you.
People on the team,
if they're clear on what that goal is,
that north star metric is.
They can run the test,
see if they moved the metric and
then only come to you when they know
if they've moved the metric or not.
Decisions like that help you move faster.
Knowing what your most important priority
is because you've looked at the data.
That helps you move faster.
Make sense?
Cool.
So now you want me to push as much of
the time as possible so I'm pushing for
as much as time as possible.
So let's talk about tactics.
First 100 users.
Sam talked about this brilliantly.
If you actually listen I thought
Tracy in the previous conversation
was just fantastic.
When she talked about getting 30 of her
friends in the construction industry to
try Plangrid, and then asking them
to pay and 29 of them kept paying.
Like ask your friends first for
your first 100 customers.
Get people who will give
you honest feedback, and
I love the point about make them pay,
if it's a paid product.
Second, research and then reach out to
who you'd like to have as your users.
Totally makes sense, right?
Like, you can do how Oscar was
discovered by the Lob Team right.
Just email a bunch of people,
don't get caught in a spam filter.
Don't just auto generate emails or you'll
definitely get caught in a spam filter.
But send personal emails,
get introductions,
see which friend you have in common.
I emphasize by the way, I have never start
up, so I might be entirely the wrong
person to tell you about first 100 users,
but Sam asked me to.
Third, social media and
PR, PR is unscalable, but
it can give you amazing bumps.
Amazing bumps.
And in the early days I think
PR is a good growth leader.
Later on, I think companies don't realize
when PRs stop being a growth lever,
because they have these other metrics.
But you can get these
great one-time bumps.
Just know you can't get them every single
week, and sometimes they are a one off.
And finally, by ads.
I am not as against ads as Sam is,
as a great method for growing users.
I think that buying ads can
be absolutely fantastic.
I built my career buying ads, two of
the most valuable internet companies,
Google and Facebook make all their
revenue from selling ads like ads
are actually a good thing.
If you get the targeting right and we're
going to talk about that in a second.
If you're having a growth team, if you're
focused on growth tactics what people
talk about is growth hacking,
though I hate the term If you pour fuel.
This is a picture from a guy
called Nick on my team, Nick.
I did not tell him to hold his hand into
the flames, which you'll see in a second.
If you pour lighter fluid onto dead
coals with no spark, you get nothing.
And if there's one thing to
take away from this lecture,
you need to have product
market fit to drive growth.
You need retention to drive growth.
Otherwise, every growth tactic,
every acquisition tactic you could
possibly run doesn't matter.
But if you have a spark there.
If you have a glowing ember you can pour
fuel on the fire of your product and
that is what growth tactics does.
So I look at things based on channel.
Targeting creative and then conversion.
So you think about channel,
I'm going to do some greatest hits here.
If ou look at SEO, you think,
I have to think about keyword research.
Which keywords do I want to show up for
on Google on the other search engines?
And make sure you have content
targeted against those keywords.
You need to get links,
links from the internet to your site.
If you're a Silicon Valley startup you
technically get a lot of links off high
value sites so you usually
don't have to worry about that.
But then you need to
link inside your site.
Many people will tell you
links don't matter anymore,
in my experience that's not true.
I don't know why it's not true, but
it's not been true in my experience.
And then finally, there are a bunch
of basics out there on SEO,
you can look them up.
There are a load of websites
that talk about them.
STO book,
MOOs, there are a bunch of websites out
there you can read about SCO tactics.
Four page search.
The biggest thing I see people do wrong
is they dont think about incrementality.
They by page search and
they don't necessarily say, well,
would I have got those customers anyway?
So look really hard.
If you're going to buy page search as
a startup you should see a step change
in your line.
Your marketing team,
if you've reached the stage where you have
a marketing guy should show you a line
that is flat and then they turn
on page search and it goes up.
That's the same of any ads
if you are a small startup.
If your a big company incrementality
analysis there's this great Berkeley
professor Steve Tedelis whose done a deep
study in it with Ebay that I love and
I think he's really smart.
Big companies, you have to be very,
very smart about getting incrementally.
You're a small startup.
Show me a step change.
Do think about marginal not absolute ROI.
So people always look at I spent
this much I got this much.
How valuable is it?
That's not what you want to look at.
You want to look at
the last dollar you spent.
How much did you get for that?
Not overall does that make sense?
Not as many knots.
I'm going to blast through that.
Look at margin ROI, look at diminishing
return curves and economics.
And then again,
you need to do keyword research.
So email and SMS push notifications.
This is something every
startup gets wrong.
We always optimize for ourselves.
Have you seen the Instagram thing on the
internet where someone shows a phone of
an Instagram user with a million
followers when they posted something and
they have push notifications on.
And the string just goes [SOUND] and
all the notifications
are coming in at high speed?
That's what we're like for
our own products.
For our products we are the power
users and by nature
every start up I have gone and talked
to has always optimized for themselves.
Has always optimized for the power user.
And so by optimizing for
the power user, what you do?
Well, we shouldn't put down by default
because it's going to completely expand my
whole screen.
Power users are smart.
If you give them the options,
they know how to turn off notifications.
It's your marginal users you should
care about with notifications.
That person for whom they received
their first Like on Instagram or
the host that gets their
first notification on Airbnb.
Or the seller that gets
their first bid on Ebay or
the buyer who's just been
out bid in their first item.
That's the person that needs
to get the notification.
And needs to respond to it, and
you need to optimize for in your head.
Marginal user is the number
one thing in email, SMS,
push, notifications, and even offline
mail which Rob was talking about and
is still very important for
many companies.
People get wrong,
they optimize for themselves.
Beyond that you need to make sure it
gets delivered, opened, clicked on, etc.
There's so much more here and
if I push through all of it I'm
not going to get to any questions.
Do you want me to get to questions?
>> This is pretty important.
How about this and no questions.
>> This and no questions,
is everybody okay with that?
Everybody for next week.
>> So think about on site merchandising.
You own a lot of media.
There's this idea of earned,
bought, and owned media.
Does that make sense?
So if you post on Facebook and
it gets loads of likes you've
earned that media, right.
If you bought an ad on Facebook or google
or somewhere, you've bought that media.
And then if they're on your own website,
you own that media.
People underestimate
the value of owned media.
If you have a lot of traffic on your site,
ask them to invite their friends.
Ask them to complete that
next action in your product.
Think about your on-site merchandising
as an extension of your product, but
understand that on-site merchandising
is a marketing problem.
So how can you trigger the right thing?
Someone has just bought an item for
you, immediately like Amazon does.
Trigger for them a set of suggestions
like why don't you buy this and this, so
the customers who bought this also
bought this, works incredibly well.
What about creative?
Creative is really important.
It should be personalized, because they're
on your site, you know their name.
Hey Alex, why don't you buy these items
that other similar customers have bought.
Performs way better than buy these other
items that similar customers have bought.
That's one simple personalization.
But instead of just saying this
customer bought this item,
you should do a prediction.
This person, all the items they've
ever bought let me predict for
them in a personalized way
what item they should buy.
Look at Facebook and how often we say
five of your friends have done x.
Social context, that's personalization.
Use people's own profile picture.
We have a great personalization example
where when you're on a Facebook page,
we show you what your ad would
look like if you bought it,
with a Buy This button underneath it.
It could not be more personalized.
On-site merchandising gives you so
many opportunities to deeply,
deeply personalize the creative.
And finally, paid and organic social.
I do work for Facebook,
we do make all of are money from ads.
And I personally think that I am not the
best person to speak about buying ads on
Facebook, because I'm the guy who buys ads
off Facebook to drive traffic to Facebook.
But what I'll say is if I left
Facebook today, and I was a start-up
I would be crazy not to use our custom
audiences and look-alike targeting.
So I highly, highly encourage you if you
think about nothing else on Facebook ads
to look into those.
It's a great way for
you to say these are my best customers.
Upload the list to Facebook.
How do I find more customers like those?
It works incredibly well for
a lot of companies.
And the other thing across all social
media is to a great extent you're
dealing with interesting squared.
Because the click-through rate
on your thing, the like rate or
whatever on your thing will be
related to how interesting it is, and
also the distribution, whether it's being
shared via a messaging app, reshared on
Twitter, or on Facebook, will be driven
by whether people found it interesting.
Make sense?
Cool.
Okay, targeting.
Behavioral is most important.
So what you did Is much more
important than who you are.
There are 50 year old,
60 year old parents and
grandparents who don't need two
friends to be active on Facebook.
And there are some who need 500.
And there are teenagers who need
500 friends and some who need two,
and you're much better to figure out like
what is the behavior that tells you that.
Then say on a demographic basis,
I'm going to do this versus this.
People who buying Nike's, Men's Nike
trainers typically are men, but sometimes
people buy for their boyfriend, or
their husband, or their dad or whatever.
And so if you just target women
with women's trainers and
men with men's trainers
you are making a mistake.
What you should do is look at what
signals you to show the right ad or
the right content to the right
person at the right time.
Targeting matters.
I've talked about this before.
Okay, creative.
Creative is important but
it's nowhere near as important as the
above two unless you have a 10x creative.
If you produce
the Dollar Shave Club video,
probably it's very very worth it but
most people don't produce
a piece of creative that
amazing that changes the world.
Typically I find targeting comes before
creative in every single way, and
channel comes before that.
Does that make sense?
So creative is awesome and really
important and can give you a big uplift,
but think about your channel first and
your targeting second.
If you have creative though,
you should put it in context,
you should personalize it,
you give it a call to action.
On call to action,
this is a great example.
Where on every ad back in the day on
Facebook we had a little link at the top
that said advertise.
We translated it wrong
by mistake in French.
We found this with the data in French,
we translated to clear an ad.
Create an ad.
And we had significantly more,
actually 40% more acquisition
from that channel that
was at the time our single most important
channel for advertiser acquisition.
We took the French translation and
we reverse translated it and
changed the whole world.
Instead of saying the kind of passive
advertise to the active create an ad.
And we achieved a 40% bump in that
most important channel for us for
advertiser acquisition.
That is the call to action,
like what you say matters.
So when I say creative doesn't
matter as much as channel, and
targeting, it still matters a lot.
And the words you use matter a lot.
This is the example I said
about in context with intent.
If you are on your own page,
this used to be ad we would show to you,
that say hey this is what your ad
would look like elsewhere on the site.
And you can see that by hour this
was the hour that we doubled
advertiser acquisition for
Facebook with the weekly push.
If you want to see more about this, Brian
Hale does a great talk about this at F8,
and it's online.
And then the last thing, yes.
The last thing is conversion.
Conversion is really really important, but
it's not important until you've thought
about all the other things first.
So things to think about with conversion.
The landing page people land on
will determine what your conversion
rate is going to be.
If you can, don't have one button
on it that they then have to click
to go to the registration form.
Put the registration
form in the landing page.
It's important to know what your
most important metric is though.
Back at eBay, what we had was this
original metric confirmed registered
users, and a new metric activated
confirmed registered users.
And an activated confirmed registered
user was someone who bought an item,
bid on an item, listed an item.
And a confirmed registered user was
someone who registered and confirmed and
that was it.
To get confirmed registered users we
wanted to pump all of our search traffic,
this was back in 2004 we were
spending huge amounts on search.
We wanted to pump all of our search
traffic at the registration page.
To get activated confirmed registered
users we actually wanted to land it
on the search term that
you were looking for.
So if you were looking for
a trampoline, show you the trampolines
you could buy on eBay.
That change,
and we changed all of our advertising
to change the landing page, dropped
confirmed registered users significantly
that we were acquiring for eBay.
But it increased activated
confirmed registered users.
It increased revenue and
lifetime value per registered user.
And it increased total revenue per day
from the people we were acquiring.
So although I think you can focus on
the registration form really hard.
Any commas, sometimes it's really,
really important to land people on
what they are searching for, so
that they will activate and
they will buy on your service.
So it's not as simple as just put
the registration form online.
But we did do that on Facebook.
So the final thing I like to
say is Facebook moves fast.
And move fast is an absolute
core value for us as a company.
Like core, core, core value and we still
do bold things like taking messenger out
of Facebook,
that was challenging at the time.
But I think now people are seeing
how valuable a decision that was.
And that was a project that I was
deeply involved with and ran.
We're willing to move fast, we're willing
to be bold because fundamentally a good
plan violently executed on today is
better than a perfect plan next week.
Data helps you move fast.
These growth tactics help you move fast.
Best of talks about making decisions with
70% of the data not 90% of the data.
And I deeply believe in that.
So move fast, focus on retention and
have one key metric
that you care about for
your company's growth.
Thank you.
>> [APPLAUSE]
>> Okay, apparently we have time for
one question.
I'm sorry.
Go for it.
>> So you mentioned that [INAUDIBLE].
How do you think that creating the.
>> So the question was for the magic
moments, you can kind of tell how you can
figure out went they happened and
that how you can create them.
I mean honestly if you look at LinkedIn,
Facebook, Twitter every social
network in the known universe.
What we do is, we immediately
ask you to import your contacts.
So once we know that friends is
the thing that we need to do for
you, we focus on what is the way to
remove friction from getting you friends.
If you look at the example I used with
eBay, we hadn't really figured out that
activating you was important to
generating the revenue long term.
So we shifted the landing page and the
registration form to the search results.
So we got you to the item you wanted
to buy as quickly as possible.
And that caused you to activate and
get to that magic moment faster.
So those are the kind of
things once you identify it
I actually think the tactics
fall out very logically.
My boss Javi likes to say that common
sense is the least common of all senses
and I agree with him.
But if you step back and just look okay,
if I want more people to buy an item on
eBay should I land them on
the registration page or
the search results page?
You'd probably land them on
the search result page and
that would get them to
that magic moment faster.
>> Thank you.
>> Thank you, okay.
>> [APPLAUSE]