Voice Activated Artificial Intelligence for Makers: #CoffeeWithAGoogler Video Lecture | Coffee with a Googler - IT & Software

[Music]
so Billy great to have you on the show
and it's amazing some of the stuff that
we've been doing we've had a couple of
shows like with aiy folks and now we've
got the person who's leading the whole
thing can you tell us all about it and
what brought it about yeah absolutely
thanks for having me it's it's been a
while since we last chatted so it's it's
good to be here and have the opportunity
to talk about some other work that we're
doing with the maker community aiy
actually started last summer when we
were thinking about how to better engage
the maker community with what Google has
to offer and I think everyone has really
heard sundar at this point talk about
how Google as an AI first company and so
we felt that getting some of the AI that
we have available
you know from Google research and from
our first party product designs
structured in a way that could work with
makers would be a great idea right and
so we came up with AI combined with DIY
do-it-yourself and created the aiy
projects initiative so can you tell us
that about what the AIA why does the or
at least the first release is that
you've done yeah yeah for sure so what
we started with was actually a survey to
the maker community so at the beginning
of the year we we thought before we jump
too far into this maybe we should spend
a little time and actually see you know
what do makers want and how do we engage
the maker community in the right way
rather than just designing something and
throwing it out there and seeing if it
works okay and so at the end of 2016 we
released our first maker survey and we
asked some good questions to learn more
about makers you know what are the
different groups what are the different
segments within that maker community as
well as what is their perception of
artificial intelligence and you know is
it easy to use is hard to use and if you
had your hands are or your the ability
to access AI what would you do with it
okay and we got some really good
information back cool and then this came
out of that right it did yeah looking at
the survey results we saw you know
probably you know maybe two main groups
one is professional and amatuer makers
mostly engineering folks that have a
full-time job but experiment with maker
tools and
make her class Hardware on the side part
of typing a new idea for a problem
they're trying to solve or possibly an
idea for a startup company that they
would like to create in the next in the
next few years as well as you know
students that are learning about
computer science for their first time
and we we heard from them that AI is is
sort of perceived as hard difficult
requires expensive specialized hardware
and specialized software and we thought
right away well gosh we want to
demystify that because it's actually not
that hard to get started with and
there's so much that you can do on this
you know affordable capable maker class
hardware that's available today and at
Google we have like a lot of cloud
services with the AI behind a lot of
cloud services like the voice
recognition and all yeah which naturally
lends itself to something like this it
does and that's that's really why we
started with voice is because you know
in the last I would say year or two I
think generally people have become a lot
more familiar with and and interested in
using voice user interfaces things like
Google home or Amazon Alexa or you know
voice capabilities in your car it just
it feels more natural to engage with
computers using using you know what's
built in the English language in this
case rather than having to learn a
programming language or work with a
button interface and when we didn't
surveyed the makers you know we clearly
heard that voice recognition and voice
user interface was one of the top things
that they were interested in and so for
that reason we decided to start with
voice right and people were familiar
with Google cardboard for like VR yeah
yeah you went with the cardboard
metaphor we did so we stole a page out
of the cardboard VR teams playbook it's
a good page to take and we're talking
quite well and we thought you know being
a part of the maker community is you
know not only making things affordable
and interesting but also just enough
scaffolding for them to learn about the
technology inside because we feel that
makers would probably you know have
their own vision for a thing that
they're building whether it's a toy
Tyrannosaurus Rex or a coffee robot
machine we have one of those coffee
I know right if we had more expensive or
sophisticated form we thought that might
be a barrier to to to people exploring
the the technology which is really the
purpose I work with a lot of kids since
May always the the barrier to entry is
either complexity or cost yeah right so
it's like you know if you can reduce
complexity as this does yes it is like
having a like a metal box and the
components inside it and as you reduce
cost as well it's like then that
suddenly makes it a whole lot easier and
I know you've had overwhelming demand
right we have like you said
affordability is a key factor in doing
something like this and again back to
our maker survey for just a moment sure
one of the questions that we asked was
you know what what computing platform
and what programming language do you
currently use and you know by far
raspberry pie and in Linux and in Python
and JavaScript were very popular with
the maker audience and no surprise there
and so right away we decided to work
with raspberry pi for our kit here and
and possibly what's coming next
okay well share more and that helped
keep the price point down because the
most makers already have a raspberry pi
and so if we could bundle all the
accessories needed you know in a fund to
a symbol format such that they can put
it together and only spend you know just
a few dollars it makes it much more much
easier for them to get started with I
mean for me personally I've been playing
with raspberry pi for a couple of years
but I just ended up like tinkering just
doing some various software so yeah and
whenever it was to do anything in
hardware or any anything exterior I got
scared
but this was much more approachable yeah
right and so apparently they were really
popular very popular so to test the
waters you know we have this information
from our survey and from talking to
makers at events Maker Faires and such
so we knew we wanted to do a low-cost
kit for makers based on on Raspberry Pi
and focus on voice recognition
capabilities and how to build your own
voice user interface we put a few ideas
together and then we reached out to
raspberry pi that the foundation and ask
Evan up to the founder you know what's
the best way to get this in front of
your your most interested or passionate
Raspberry Pi
and he suggested well magpie magazine is
our is our monthly journal that that
showcases projects that you can do with
your raspberry pi okay so I don't know
why don't we send a few of your voice
kids out to our subscribers and see what
the appetite is nice and that's what
they say
it is yeah so this is this is the this
is the unit that we actually ended up
with it started off with we would send
out some of the PCBs that are inside of
this to the subscribers but it quickly
snowballed into why don't we just build
a kid and brand it magpie and Google and
put everything inside and that's really
where we landed
so we structured the kit like this it
made it look like the magpie magazine
and inside you know is the actual
magazine itself which you know it has I
don't know 35 or 40 pages about building
this kit so it's actually the
instruction set for assembly usage in
and how to hack the kid how nice and
then you know in site we also have the
cardboard form that you fold together so
this is really fun to fold and snap in
that place yeah that's before this is a
chapter and then you know the the PCBs
that go along with it
this is the Hat voice hat which has a
microphone and amplifier audio output
and then the actual microphone is is
separate with an extension cable that
goes back to the voice hat as well as
the speaker and the activation button
and just an interesting interesting
thing the the microphone board we
actually made it a separate board which
attaches back to to the main hat because
we felt makers as they're exploring
creative forms with what they're doing
may want to place the microphone and an
interesting place like maybe in the in
the head of that Trento source Rex for
more in the front of that coffee robot
nice rather than have you know the whole
computing platform you know stuck to to
that side of they're gone in today yeah
the more we got started with it the the
more involved who we became and really
wanted to put together a kit that has
you know good value for the price and
and it prompts people to explore you
know their their inner creativity about
what they would make with a voice user
interface and I know you've worked a lot
with people in education well we have
and I guess it should be no surprise but
we we started out really trying to get
Google
artificial intelligence in front of this
you know amateur and professional maker
audience as I described before but as
the kids went out you know more and more
demand of hey I'd like to get one too
and eventually a Raspberry Pi or the
magpie put them up on their website for
sale and they sold out right away but
what we found was so many teachers and
school club organizers saw the kid and
felt it would be a great way to
introduce their students to computer
science connected devices artificial
intelligence and so yeah a lot of demand
from schools in order to get that
started with raspberry pie when you
first told me about this I said I need
to put that to the test yeah you know
just see what it'd be like before we
shoot something like this so to put it
to the test we gave it to some high
school students to see what they would
build with it right they they built some
amazing things should we take a look
yeah let's have a look cool you can also
have to turn left turn left nice
shop pawn to d5 yep and just take then
you go on shop for a New York Mets
Jersey now she's done candies so we get
plural as well nice and red is my
favorite so I'm glad I got a couple of
Reds what struck me about that was like
how some of them were so inventive with
external hardware yeah right so I
thought like with only five days to work
with writing they might hack some Python
scripts together and stuff like that and
some of them did and it was really
impressive sure but there was like that
one that one team that they built that
little hexapod yeah the little spider
that's going around amazing isn't it so
it's like they took three servo motors
and then they add like a little
Bluetooth circuit and then they use this
to do voice remote control the yeah the
motion of the robot yeah and then
because the Raspberry Pi and the image
the Aoi image had like the Bluetooth
drivers on it they were able to just
send Bluetooth commands over and the
effect of that wow I'm glad to hear it
was easier for them to get started I I
didn't believe they would be able to do
it so quickly yeah five days you said
yeah five days amazing it was it was
really cool and and I'm part of the
other feedback that I thought was really
neat was around Python mm-hmm because
some of these kids had never used Python
before and so it was a pretty good
choice for to get them started with with
Python programming yeah and yeah it's an
by open sourcing everything and it's on
your website right your website is yeah
AEY projects dot with google.com and
we've put a link to it in the
description down yeah okay
now in order to work like does all this
need to be connected to the cloud well
the the default use case is is cloud
connected and we demonstrate how to use
the Google assistant SDK for that you
know how to interact with Google Google
home kind of functionality and then also
we show you how to connect to the Google
cloud speech API which is a more
sophisticated voice recognition natural
language processing library 80 different
languages hence on on jargon and that
sort of thing but no we also include or
will soon include the way to do it on
device using a
tensorflow model that can run on the
Raspberry Pi so you can do the
cloud-connected case as well as the
offline case ok so even like a small
inexpensive computer like gladly hi
we've got tensorflow speech recognition
that's right well we'll have it seen
we're just a few weeks away from making
that available as a release on the
website that's exciting so yes we
intended to include it with a launch but
we were a little bit slow gathering the
training data needed to to to craft that
model and make it available right I saw
there was like a link went out yeah yeah
that's right so there's a link on the
website a NY project start with
google.com slash voice ok and if you
come look on the Left nav under the
makers guide and there's a link to to
understand how to do it on device which
right now there's just a simple step to
submit your speech so we can capture you
know people saying the words that we
want to put into the vocabulary and I
think for now we're looking at simple
transport type of controls words like
stop go left right up down things that
you might tell the device to do to
command it to take some action and once
we have enough data coming through that
channel we'll begin training the model
and then release not only the model so
that you can run it inside of the kit
offline yeah but also release the
training data so you know the spirit of
of aiy is very open open source freely
available low-cost when you have to
include the accessories and so we want
to make sure that the data that we're
gathering through the website to train
the models is also freely available for
everyone to take and maybe create their
own models if that's what interests you
so the more people that do that to train
at the better much better a bit yeah the
higher accuracy the the the better
response because I responded to that
when it first came out and I sent my
voice in and I just hoped lots of other
people yeah yeah well this is only gonna
be able to recognize Irish accent I did
as well I I did as well and and I've
just been looking at them coming through
and we're having quite a good response
so thanks for everyone that's done that
and let's keep them coming in keep them
coming and we'll put the link in the
description below this video right down
here perfect great so given that this
has been so popular where do you think
you're going to go next with it well I
think the internal direction that we're
going next is
to bring more more ways for for people
to create interfaces for computers that
are natural so if this is using your
voice to to communicate with a computer
what are the other ways that human
naturally communicate between themselves
like you and I are are interacting
through voice our ears our eyes and in
hand gestures so looking at a way to
bring those technologies into low cost
affordable kits for people to explore
maybe mix and match and build more
interesting things with this kind of
direction that we hope to go this year
and next year and me personally I've
been like tinkering in the makerspace
for a little while but yeah about having
something that easily expands the
boundaries of what I can do in the
makerspace is just mind-blowing so I
can't wait to see what you do next
fantastic that you get the kits I'm
taking this so thank you so much Billy
this has been a real pleasure and
there's so much fun to play with this
thing and I'm hoping many many many many
more people in the future will be
getting hands-on and building stuff and
me too Yang's with it more makers more
kits and you want to see what people do
with it just keep making awesome sure so
if you want to learn how to build one of
these for yourself if you want to order
the kit aiy projects that with google
comm that's packed with resources
there's instructions on how to build it
yourself of course the open source code
so that you can extend the models in
this are there the voice models that
Billy mentioned are also in there we put
the link in the description below please
go check it out and I can't wait to see
what you're making thank you thanks
again Billy thank you
[Music]