Machine Learning and Music over Coffee with Christine Robson Video Lecture | Coffee with a Googler - IT & Software

welcome to coffee with a Googler I'm
here today to talk with Christine Robson
who's a research PM for machine learning
we're gonna talk about what's new and
exciting in the field of machine
learning including project magenta which
is all about machine learning for
generating music we'll also take a look
at some of the things that are happening
in YouTube as well as advances in image
recognition so that video that was an
artist who did a cover of a track that
was generated by computers that yes
exactly so the piano that you were
hearing that's us that's the machine
learned music that's the generative
model the tensorflow model coming up
with a piano track and then we put this
out there as part of our open sourcing
efforts to put all this generative music
models that we're doing online and these
artists they they listen to the music
they were interested in it they were
inspired by it and they decided to sort
of jam along with our music track
were you expecting somebody to do that
no we weren't expecting that at all it
was it was really very excited we got
all we got excited in the office we're
all like passing the video around taking
a look it was it was really pretty
exciting but this is what we hope will
happen right that we can use machine
learning for these really creative
purposes and really get a partnership
going where artists and creators want to
work with machine learning tools to to
be more creative and have fun I remember
you said something once it's like not
about replacing news it's like almost
like inspiring musicians yeah exactly
and and something um you know Doug who's
the lead on this project he talks about
this all the time that it's not that
we're trying to make the best music out
there I mean I'm not even a musician I
you know it's that we're trying to make
tools that the best musicians can use
tools that that spur this kind of
creativity and the magenta project
that's the project that was open source
that that we generated this music from
the magenta projects really about making
those tools for creators you know
starting with music but you know we
won't have tools for creators to do our
generation
to do videos you know to really tackle
all of these questions of how can you
how can you be creative and work with a
machine learning model to to make
yourself more creative and now for this
piece of music it was I recognized that
first bars was like twinkle twinkle
little star yeah and it took over from
there exactly so we give it this quick
bar like thum thum thum thum
okay and then the model is trained over
a huge swath of music everything from
Bach to the Beatles and it's just trying
to predict what the next note should be
so but it uses a type of type of machine
learning called recurrent neural network
so it looks backwards in time over the
music that's been played already and
tries to predict sequentially what the
next note should be okay based on all
the music it's seen before so you give
it just a couple of notes to get it
started and then it just takes off
yeah that's pretty cool it is really
cool and this was built with magenta yes
rights and I know you're a product lead
for magenta and could you'd like if
somebody wants to get started with
magenta and maybe start inspiring other
people and some other kind of art form
or maybe with music how would they go
about doing this
so everything from magenta is online we
have a github it's part of the
tensorflow project so if you go to the
tensorflow slash magenta on github you
can go and you can download all the
tools we encourage everyone to download
to contribute to play with what we've
got there and see if they can they can
do something creative to me that's why
we've put it out there nice and and it's
not just music right there are other art
forms too well right now we've been
focusing the project on music but the
tools are very general right the tools
are really about generative machine
learning this idea of generating music
generating art this tools to create
rather than just making predictions with
machine are ok cool
I like machine learning of course it's
not just the incubation it's it's in use
in the real world already like things
like YouTube are using it for
recommendations when you think about a
machine learn model and you think about
all of the possible data that can go
into it and what the places where the
signals are gonna come from if you can
capture more signals than you might be
able to think of as human you can do
better than a rule-based system and
there's also this notion in it's a very
hard concept to explain there's this
notion of embeddings in computer and
computer science and machine learning
research where you you capture you have
you put a way to capture all this
information together you capture
relationships which aren't easy to
describe by simple human labels like
maybe the relationship between these two
things is really easy to explain you can
imagine writing a rule for it like
and I really like cat videos so maybe
it's a like this cat video is similar to
this other cat video they both contain
cats they're both you know both from
Japan something like this but then there
may be other factors that are really
hard for a human to suss out like maybe
what I really like about this is sort of
a abstract feel about the video the way
in which I like watching the cat jump in
and out of the box and that kind of
abstract feel translates to this totally
other also kind of cute video involving
pandas on slides these two things are
related but it's hard to put your tongue
on exactly why but they're obviously
related and they're related for
everybody who looks at these YouTube
videos and that's something that a
machine learn model can capture really
easily especially with these embeddings
that we use and that it's really hard
for a human to write a rule for right
got it so it's like if you like acts you
might like why and you may not realize
that but when you have all these
variables coming out of an ml model
there you know it can actually show you
that you can actually learn that about
yourself yes it's like an emergent type
thing does feel emergent sometimes these
notions of ideas that you wouldn't have
thought really mattered for your
preferences like things that I might not
have realized I wanted to see in a
YouTube video how I want to spend my
time that I really like this particular
niche II kind of cat video and I've
discovered lots of music that I like in
that way where it's like I'm watching
one piece of music and then it suddenly
goes onto something else on the playlist
of somebody I've never even heard of in
a style that I probably don't even like
but hey that's not bad
yeah and we actually do that we do
machine learning for like music music
lista curation
it's actually the the lead on the
magenta project Doug his his role at
Google before we started this project
was in the music recommendation system
he's a musician he's got a real passion
both for machine learning and for music
and so we spent a lot of time working on
how to predict what music you're gonna
really enjoy listening to based on what
music you already like right and looking
not just a very simple thing like which
people like the same kind of music but
actually analyzing the structure of the
music and understanding how different
musical pieces are related to each other
in sort of subtle ways and fundamental
ways and being able to to really do a
very curated playlists for you
that finds things that you're gonna be
interested in so I understand that
YouTube it's done so well with this kind
of stuff it's even won some awards is
that right yeah so um you don't think
when you're a computer scientist that
you're gonna be working in a field that
gets an Emmy but actually YouTube's won
an Emmy for personalized video
recommendations I was a technical Emmy
and Emmys an Emmy I know it's true and
this was awarded because we do such a
good job of personalizing video
recommendations you know when I see that
perfect cat video that's because we're
doing such a good job prioritizing what
you want to watch and prioritizing how
to find the perfect video for you in
this moment so then like so don't be on
like recommendation engines not gonna
thing one of the things that keeps
reading about is how ml is now used for
image recognition and for attributed
images and you know you've been involved
in that a little bit
yeah so if you haven't used the Google
Photos app you should use the Google
Photos app it's my new favorite thing I
love that I can pick up my phone and ask
it to find me photos of my son ask it to
find me photos of my son you know
playing with my cat and that it has this
great model where it knows you know can
do face detection and find my son for me
it can do protection on my cat and find
my cat for me and it does this because
it's seen we've fed this model all of
the images we've seen from the web of
all of the different kinds of cats and
it has this model built up where it sort
of knows what a cat looks like and so it
can label it very effectively so like
one of the things like about image
recognition is I remember you had like
read something from you like even
talking about a cat and it could
recognize a cat's face even though the
cat didn't have ears and the ears are
one of the most distinctive parts of a
cat yeah that's my cat um so a few years
ago my cat who lives in California of
course in suntan's all the time she gets
some skin cancer in her ears she's fine
now but we had we had to have the vet
had to remove her ears to clear the
cancer and so she looks a little bit
funny now she's still really cute but
she's got no ears and she looks less
like a cat than we might imagine but our
image recognition software is fine like
it can still recognize her as a cat and
so if I ask my phone for photos of my
son and my cat like it still it still
recognizes that she's a cat and can find
her and it's this kind of this
flexibility that you get from training a
model with a lot of different data you
know there's not probably a lot of
examples of cats with no ears on YouTube
but we can sort of extrapolate when we
use these really powerful deep neural
network models to sort of we know we
know basically what the cat should look
like and so we can extrapolate past
things like the fact that she doesn't
have ears anymore
I'm glad she's doing well though a
couple of months ago we had a Joshua
Gordon on the show and there's one thing
that he said to me that I thought was
really fascinating and that was he was
talking about like three four five years
from now the skills of machine learning
and the techniques of machine learning
will be as important to developers as
like JavaScript or SQL are today and
it's a and I think it was right and it's
I would love to inspire people to go out
and do more into this they learn more
about machine learning do things such as
creating music or art or who knows what
other kind of things that come up with
and I know you've got a wealth of
resources and they're just what kind of
things would you ask developers to take
a look at well of course I'm very
passionate about this because machine
learning is my passion but I do think
that josh is right you know we've talked
about this a lot but the thinking about
machine learning is so different than
thinking about other traditional kinds
of software engineering right I often
make the analogy that the difference
between like functional programming and
object oriented programming that that
paradigm shift is much smaller than the
shift of thinking about machine learning
versus thinking about traditional
programming methods so this this shift
is very big and very important and
that's why we want everybody to be
together on it from the beginning that's
why we're putting so much effort into
open sourcing our tools we're really
open sourcing the best of everything
we've got to try and get everybody
together around this so developers in
the outside world can already get the
best of Google yes right which is just
mind-blowing when you think about it
yeah and that I mean this is why we've
put so much work into open sourcing
tensorflow and why we really want to put
this all out there it's so important to
think about what you can do with machine
learn model because I really think this
is going to be transformative for the
kinds of things we can do I mean
everything from you know YouTube where
you're predicting the next video or
other other angles you can think of that
affect your sort of day-to-day life to
these more esoteric creative things like
music generation right things where you
really want to have the best tools at
your disposal and you really want to be
able to be as creative as possible and
machine learning is going to enable all
of these things so we want everybody to
be really
yeah I just thoroughly recommend it for
anybody any developers out there just go
take a look at the sites that we've
linked below with some fabulous stuff
out there and you're getting ready for
the future so thank you so much
Christine this has been awesome thank
you it was a real pleasure chatting with
you had a lot of fun