PyTorch Lecture 09: Softmax Classifier Video Lecture | Pytorch: A Complete Guide - AI & ML

in this lecture we have want to talk
about softmax classifier which is widely
used in the neural nets suppose we get
this image input which is Hindi written
a single-digit number and then we want
to classify each image into numbers
between 0 to 9 so there are 10
categories so in this case because this
is a classification problem probably we
can reuse existing Raja's to regression
model which gets input and then we have
a linear and activations and then
eventually we produce one output which
can be used to predict either 0 or 1
however in our problem we shall have
with 10 labels so it is more natural to
have 10 outputs like that and then each
output can indicate the probability of Y
to be 0 or Y to be 1 and so on how can i
implement this how we gonna make our
neural nets to produce 10 output rather
than one single output we want to use
matrix multiplications to produce the
matrix the outputs that we want
previously we used 2 by 1 matrix to
produce n by one from given x value and
by 2 so the simple idea here is that
because this tool must be the same they
will cancel out and the new battery the
output of this matrix of the publication
will be n by 1 matrix now the question
here is that if you want to produce hand
outputs what will be our the size of our
weight matrix in this case let's say
suppose our input is 2 4 as an X X and
then we want to produce and by 10 matrix
there will be the matrix size for the W
so this is all the step 2 it should be
same as a 2 so it's gonna be two times
you want to produce 10
so that will be ten so simply in our
last linear model we can just use mid
value which metric size is here Lester
de-emphasize is gonna be M then what we
need is M by 10 matrix for this W then
we'll we'll produce 10 outputs the next
location is how we're gonna make this
output as probability so for this we're
gonna use very famous function called
the soft max this is our soft max
occasion so idea of the soft max is that
forgiving g-numbers is coasted all the
numbers between 0 to 1 value and then
actually the output of the soft max it
will be the probability of to be dead
index so I'm gonna show you an example
here and to understand how it works for
given input X we do some linear
operations and it will produce three
numbers which is called scores
overloaded because it's linear so it can
be any numbers 2.0 1.0 and 0.1 if we
apply soft max to this real number or
logits
we get very nice probability for each
index for example here 0.7 is
probability that Y is gonna be sure and
so on also because each value here are
all probabilities if you sum all of he
is gonna be one
let's summarize our softmax one more
time with a very nice slide from this
course so from our input we can run some
linear model and then this linear model
can produce logit some numbers some
values and then we can apply our soft
max here
and then you can produce a probabilities
and the next step is that we had to
compute the loss for given labels so in
our case the label is gonna given as one
hot labels which means that if Y is zero
then the label will be one one zero and
zero is one hot format and then in order
to complete the differences the loss
we're gonna use cross entropy between
this Y head which is our computations
first real label our cross entropy can
be computed using this equation minus y
log Y head cross entropy can basically
complete the differences between two
distributions of Y and Y head and then
in total loss is just the sum of this
entire course Enter key
let's use very simple source code
example to see how this cross entropy is
working so we are going to use very
simple numpy array for this example for
the Y which is labeled in our case we
use the 1 1 hat expression so 1 0 0 is
our Y and then for the prediction 1 we
also use the same value used in the
example here 0.7 0.2 and your point 1 in
this case our prediction is pretty good
because we say that probability of Y to
be 0 is very high to compare with the
other prediction we also have a
prediction which is wrong prediction
which is 0.6 for a y is going to be 2
the probability of Y will be 2 is very
high but this might slow so this
prediction is wrong so if you can
compare the laws we can expect that
draws value for this one is small but
this is gonna be something high that's
what we can expect
so it does supply is cross-entropy
in this equation y minus y times log
with a y prediction and you expected the
number here is something small but if
you compute the cross entropy laws for
the second prediction the Paleo is gonna
be something high so how this works
moving us to look at them something very
intuitively because we're gonna provide
the Y as a one-half value if we do
multiple ideas we just can't even or
only we can focus on these two values we
can ignore this because it's multiplied
with just one hat so it's going to be
minus one times this log look zero point
seven is a small value and in log 0
point one is a large value so that's how
we're gonna get at this at the competing
discourse entropy in pi-thirds we can
simply use their implementation called
cross-entropy laws do we get this loss
and then we call this function by
feeding our predicted value in the real
value we can compete our lost value but
when using this cross-entropy loss in
Python two things we need to consider
first this Y value here is not what has
this is gonna be class which means that
it's gonna be 0 or 1 or 2 in our case
which is more convenient and the second
thing that is different than previous
example is that our prediction value is
not softmax we can just directly feed
our logic value right here the reason is
inside of this implementation it
includes softmax inside so we don't have
to apply softmax when a computer loss
value which is also very convenient so
basically the cross-entropy
makes really come
by pissing his class and also by pissing
on the logit so this example
particularly shows that we provide zero
class and then we provide this logit and
then using this y value in y prediction
we can compute this loss and we can
expect the first one is correct
prediction so our loss is something
small but second this is their
prediction so our prediction is wrong so
the loss is going to be something high
another loop a nice thing about this PI
torch implementation is we can use batch
mode which means that we can provide
multiple labels and also we can provide
multiple predictions so for example here
the first label is 2 and then 0 & 1
once again Y is the class we provide a
class that one had so for this we need 3
different predictions the first
prediction is this so it's correct
prediction and the second one is this
also correct and third one is this so
which is also correct
so Y prediction 1 is a good prediction
this is very good and the word y
prediction tool is not very good so this
is their prediction which is wrong and
this is also wrong and this is also
wrong so when you compete to the loss
probably expect that this one is a
little bit lower number but this one is
a little bit high number so when you
compute as you expected we can get this
loss function which works pretty well
for our softmax classifier in the
patterns there's another way to use
softmax classifier using enolate laws so
as an exercise you can figure out how
you can use any letter loss for soft max
classifier simple hint is that you can
just use slope soft max with this energy
loss which is basically same used to
using just cross-entropy loss
now we can apply our soft Mexican aspire
to a real problem something like this so
for a given image 28 by 28 pixel we can
classify them as one of the ten
categories between 0 to 9 because it's
handwritten one single digit number so
once again the image of input image size
is 28 by 28 so total it gets 784 pixels
so we can delight our neural net which
get 708 pixels and eventually you're
gonna provide 10 outputs so usually we
call the first input as an input layer
and the output as upper layer in the
middle because it's hidden from either
inside and outside is called hidden
layers in hidden layers you can design
as you want you can make as many layers
as you want and each layer how many
nodes you won't have it's up to you so
for example here we can feel something
like a 4 hidden layers with each node
size is like this 520 320 240 120 and in
October it is in the pide watch
basically we can make each linear
elements for a given input size and
Apple size for example here this hidden
node is 520
so this first linear elements gets a 184
and an input and it produces 520 and
then our second layer will get 520 and
for those 320 outputs and so on so this
output will be exactly the input for the
next layer and then finally what we need
to do is that we have to produce 1000
this is fixed so in summary to see
784 and this 10 are fixed all the middle
you can decide your arbitrary numbers
once we have these elements basically we
can connect them in our fourth function
but input X here is that we get this
vector 1 by 28 by 28 28 by 28 with a 1
color
so first we're gonna have to let this
vector to end to 784 using this command
and then we're gonna feed this one to
our first layer and this to the second
layer and so on and then it last layer
we don't have to apply any activation
function because we are going to use two
logits
as an input for the cross entropy
function in here we are using cross
entropy laws as our criterion and then
in our patch we basically get the data
which is X and then target and they will
fit this data to our model and they will
get output which is our prediction using
this prediction and then keep on target
we can ask the computer laws using our
cross-entropy and they with the pet
propagation and did we update our
weights and that's it and this is the
entire source code first part is that we
are using our data loader we define the
directory in the reload our our data and
this is our whole network which has this
components and we connect them together
at the last once again we are not gonna
use any activation function and they
redefined our model using this class and
then we define our criterion which using
cross a trophy and then we're gonna use
the same sgt here for our optimizer in
training cycle basically we using this
trained order we just get data
and the target for each bitch sighs here
the batch size is given 64 so for our
test data we pass this one to model and
do we predict and the ones we predict to
predict value and make even target we
complete loss and to this ones and here
is basically just printing out what's
the epoch what is current loss and so on
some very nice information about the
progress and then how can see our
training our model is good or not good
wilt measure some sort of accuracy the
best way to measure thickness is that we
cannot divide our data set into two one
is called the training another one is
called the testing so the training we
only use this training data set in the
build our model and there was a finish
this we used this test to data set which
never seen before for this model to
evaluate our model if you see our data
roller carefully so we we have a two
different data loggers one is called the
Train another another one is called test
roller in our training cycle we only use
the Train another to train our model now
once we finish this training we used
this test water to test our model so
basically we fit this model and then we
can complete test rows and here we try
to predict using this Torsten max
function and then see how many of them
are correct by comparing with this
predicted value and it is target error
and they will compare to some and then
we divided with the total number and
they we can complete the percentage how
many of predictions are correct let's
look at the max function a little bit
closely supposed to be here with this
prediction values and this Max return
two things first it returned the maximum
value which is 0.9 for this and then it
returned to index this is 0-1 - that's
the index of the tool so this is going
to be true so in here what we want to do
is set from this output predictions we
want to get the index so that there is
the our classification if you were to
get one index we can use this this is
why I use this one to get index so the
accuracy can be competed once again
using how many of them are correct and
the wickedest to print this percentage
so this is our loss in training it has a
very nice way to print out what is
current epoch and what is our iterations
and then this prints out the percentage
of the situation's and then here is the
lost trading loss and that this is our
test using our separate test to set the
average loss is a little bit high but
accuracy is pretty good
97% using simple neural nets eliminate
something multiple label prediction no
problem we can just use cross-entropy
loss so basically the last layer is
gonna be just linear output and then we
can feed this one to our cross at tripi
loss then everything will be okay as an
exercise we can build a softness
classifier for this very interesting
data set which is available in this URL
of course also you can build your own
data loader
for these datasets in our next lecture
we are want to cover very important
internet which is called convolutional
internet