PyTorch Lecture 07 Wide and Deep Video Lecture - Pytorch A Complete Guide

in this lecture we are going to explain
how we're gonna make our neural net more
white and more deep in our previous
lectures we introduced our linear model
and logist liberation with one input X
and they will produce Y head based on
our model for example here let's say we
try to build HKUST ph.d program
application predictor applicants will
submit their GPAs and based on this
input x we're gonna predict go ahead
either she or he is gonna be excited or
not in our source code we can express in
our data like this so each X will have
only one value like GPA here and then Y
value is 0 or 1 however if you can see
this model probably is not really fair
just using only one single input maybe
does not have enough predictive power
for our final result so we need to
introduce more input for example here
let's add one more input called
experience so in this case it's not only
we cannot consider on the GPS but also
our model get the second variable which
is experienced as an input and then
we're gonna predict based on two inputs
in our X data can be expressed in this
way so now we have a tool data for each
X so how can i implement this one so in
order to implement is we don't have to
change our model we just need to
introduce matrix multiplication concept
in our previous models and everything
works fine so for example here in our
data set X data can be expressed as
metrics so here we to have n data and
each data has a and B two values so we
can express this one as metrics with n
by 2 matrix and as X and the reduce on W
XW and they will produce our wide in
this cases our Y is also metrics will
end
we have n data points with output each
output is 1 so n by 1 and the question
here is that which metrics were to use
to produce from this to this we only
need to decide this one and then we set
this linear model with the dead
matrix so in our case is very simple so
what we can do is that we can design our
weight matrix with 2 by 1 and they will
produce this matrix so basically we can
see always this value should be the same
and then they will cancel out eventually
they can regenerate and buy one just
like this so we just need a linear model
with this matrix further generalize this
model not only one or two three inputs
we can have many inputs and then we can
just analyze this one as X as a matrix
and then W is also metrics or produce a
matrix like this in our case because we
have only two inputs what we need is the
we need weights which is linear model
with the 2 by 1 matrix so we can just
really define our linear we can say 2 by
1 also we can see this one as our X
input has 2 so this is our input and
then our Y improvements only one output
so this one get 2 as an input and then
one produce one at an Apple you can
consider this one and then you can
design our linear model like this so not
only for one or two moving us to make n
types of input and then we can might
make our model really wide but not only
wide also we can make a relative so in
previous example we just only used one
layer as our model but it's not limited
to make multiple layers
and to implement these multiple layers
basically what we need to do is that we
just create multiple near components in
here and they we just connect them
together one thing we need to be very
careful is that what is the input size
and watch the Apple size so this input
size X is maybe let's say the input is 2
then the input first linear should
should be 2 by number and then this will
generate 4 and then second air must get
4 as an input and produce any arbitrary
number but make sure the next layer
which get 3 is an input and produce the
last value which in our case is y is 1
so always this one should be 1 so in
this example always this value here is
fixed X is 2 y is 1 so this is fixed but
all the odd numbers in the middle this
can be arbitrary number you make your
decision in the depth we have these
components we just connect them all
components together like using sigmoid
that's it in our lecture we use the
sigmoid as an activation function which
is simple and good enough but when we
will apéritif your network sigmoid can
cause a vanishing gradient problem
basically sigmoid you can squash values
to a small number so if you can multiply
this small number which is smaller than
1 then the number is getting smaller so
if you do pet propagation we're gonna
lose our gradients so which is called
the vanishing gradient problem there are
many solutions to address this issue but
one simple ways that you can use other
types of activation functions especially
this lelou works very well for this
situation so all these different types
of activation functions are input
there so we can try a different
activation functions if you want to more
about activation functions you can visit
this webpage and then you can select any
activation functions and then you can
visualize this activation function with
a very nice explanations indications
using this thief and why the neural net
we can apply our model for real data
sets like this tie reading so this data
file is given as a CSV file so the
column is like this so this is Xterra
and this is our prediction why for some
given some test result we can predict he
or she might get a type II D so we just
read everything using numpy utility
function and then get everything to X Y
so this X is can be X Y yeah everything
and from 0 to n minus 1 color which can
be expressed like 0 to minus 1 so like
this and then why you're gonna take it
alone it last column so which can be
expressed something X Y that everything
and then the last color this is how
connects both x and y and then we can
just transform this numpy to torch and
they will wrap with the variable and
then we're gonna have X&Y data and then
if you want you can use to print out
this the shape of the data and then we
can see so we have basically 759 data
with 8 features 8 X values and then 1 Y
value and then we can just plug in
everything using the same idea in class
now what we want to do is that we want
to make it a little bit wide so here the
input is 8 it's wide and also we want to
call us deep three layers so we need to
learn these components the input is
fixed output Y is fixed but in the
middle how many layers do you want to
use it's up to you and the what's the
output of each layer
also up to you for example here we just
approaches for Cuban eight input gonna
produce six outputs but is six output
can be used for the second layers as
input like this and then also this can
produce four and then this can be used
as an input for the last layer and then
we just connect these components in the
right order and then we're gonna use
this sigmoid in our activation function
and that's it and the entire source code
looks something like this so we can
upload our data from CSV file and then
we'll just make Xterra and why data make
sure there are all high touch variable
and then we build our model using this
class and then our loss is same as PCE
because it's 0 or 1 binary
classification and the users GT this is
our loss and the optimizer and they
rerun at this training cycle basically
we will feed entire data and then we
feed entire data for the loss and then
after recomputing the gradients and they
will just update our weights and that's
it in our example we just use three
layers of the network but it's not just
limited to build three layers you can go
six seven in ten layers you can try out
also using exactly same tip neural nets
we can try with other classification
problems in our example we just used
sigmoid activation function but
especially one new ones you design and
deep neural Nets probably you can choose
Auto activation functions we can share
with other activation functions in our
next lecture we're gonna talk about how
can I make our data loading like much
easier using data loader in the PI torch