Tensorflow model using Keras with Java Video Lecture - Machine Learning

Hello Explorers and welcome to another video and today
I'm going to talk a little bit about a tool called Keras and
The thing that actually highlighted this video for me was a discussion. I had in the comment section with
birinhos I think that's how you pronounce that handle. I'm sorry if I butchered your name and
he tried to
actually build a model using sklearn
Which is machine learning made a little bit easier
I realized that setting up a tensorflow model could be a little bit daunting so I tried to find
something that could do this easy and
My first model that I built tensorflow was using pure tensorflow and was a lot of different
parts in it so it was a bit hard to follow
and I looked at sklearn and what tensorflow had
Actually develop around this easier way of building models and I found
skflow or tensorflow and that should
Be an easier way to build models, but I think that skflow wasn't really
done yet, so you couldn't do scikit flow and actually get a good model at least not for
Deep Neural Network or an CNN
So with this convolutional neural network that we wanted to build I wanted to find something that we could do this easier
but with still keeping the same functionality
So I actually found Keras, and I think Keras is some
Improvements, which will make things a much easier
So let's look at what the model I built looks like today
so first up we have this a Python File here, Where I
Have this missed Data which I import I start an interactive session. I have some
Placeholders, and then I have to set up functions for weights, biases
convolutions and pooling and I gave my weights and biases the name w and
B which is something that they actually
tell you in the
video explaining tensor board which is something we will look into later to visualize these things and
then I have this reshape of the image to make it in into a 28 times 28 little square and
then we start our convolution, so we have the first convolution the second convolution down here and
then we have a reshape again to
flatten the structure before we actually do our fully connected layer
We do a drop out layer, and then we do the fully connected layer again to actually get the result
so this is the last
part of this network, and then I actually
also, set a name for my output tensor, so I could easily find them in Java and
I also put this keep probability so you could remove the drop out layer later on and
then we have the cross entropy. I wish I have given a name
We have the training and some accuracy so we can predict our model and see how well it's doing at the moment
we have the saving of the model with this new save model builder and
the actual training steps and then I write my
Information to Tensorflow or tensor board so I write the actual graph and all the different
information that we saved earlier, and I also saved my model, so this is the new steps
all these name scopes and
Also this writer to tensor board is something that I have added to visualize my model
so if we look into this
model that
We actually get when we have done all this work
We will get something that looks like this
Where we have the input down here with this little node down here
And then we go up to the reshape which we have a name for we have the first convolution which has two variables
The biases and the weights and the second convolution two variables their weights and biases then
we'll go up to the flatten
construct, we have the first connected layer with weights and biases we have the drop out second connection a
second fully connected layer weights and drop out
And then we have the part of the training up here
So we have the cross-entropy and accuracy and the output tensor, so this is
Not a really complicated graph
but it could be a bit hard to follow and
If we didn't name our output tensor up here we have to go into this
last part of the fully connected layer and then look into the last operation in this so
To get our output we had to look at FC2/add
and I believe that if we hadn't named this FC two we had to look at
matmul2 perhaps
Which is the next to last and then add so tensorflow gives
every node in the tree a name and
if you don't name them yourself those could be a little bit convoluted and hard to follow so
By naming things we have made things much easier for ourselves
and if we go back to the code again, we can also look at the Java Model to
read and work with this and
Pretty much reading the model up here. We prepare some data
we set the input tensor and we set the keep
probability and this is the runner where we have input tensor, probability get output tensor, so
These are pretty easy steps, and I think I've talked about all this in the other video
So what's the difference with Keras then if we go to the Keras Model
We actually have a lot of difference here first of we have a lot of more things that we initialize
perhaps we could actually take everything from layers and everything from initializers and so on but I have
picked apart everything that I need
So we start our session and we set the session to K so Keras knows which session to work on
we have our images and labels
This is the first part pretty much the same as we had earlier now. We come to the actual
building of our
our network and
This is where it differs a lot because this as you see on screen now is
Our network fully
every piece of the information that we need for our network and it's
Performing pretty much the same I think the other
Network performed 99.23
or 36 or something in the new version of tensorflow and this performs as
98.99 or something so they are very similar. I haven't really figured out. What part is
skewed a little bit different, but I think
That's not really important because if you can
provide very easy setup for something that performs really well then the last
Couple of percents isn't that important
So in our model here. We have the first reshaped into the form we want 28 by 28 we have the first
convolution and to keep it as the
Earlier convolution, we need to specify a lot of things it's a 32 convolution
it has a kernel size of five five and
Strides of one this is pretty much the same as we had in the early model
but we also need to tell it which Kernel initializer we need and that was what how you set up the
Actual weights and biases this is our weights. They are truncated normal with the standard deviation of
0.1 and we use biases and the bias initializers are constant of 0.1 as well
So this is all we need to declare for the first convolution, so this is our first
convolutional layer and then with a max pooling layer second convolution layer
max pooling again, and then we have a
flatten operation, so it actually says what it does
So you don't have to figure out that you have a reshape to flatten your structure
And you don't have to figure out how your large structure had need to be either
It's just flatten it right off the bat, and I have my fully connected layer, and they call it a dense
neural Network, so that's just a difference in name and
It also has a truncated normal and the bias initializer
so we could actually I think have these as
variables and just
Supply the same variables for all of this, but I typed it out to make it a little bit more readable
Then we have the drop-out layer
not much to say there and the last fully connected layer and
I put also this identifiers that I had in my other model to tell
the Network which my output tensor is
so and then we have the cross-entropy down here not much to tell there we have the
Prediction accuracy prediction we have the saver which we have in the middle we initialize our variables
We read our input data. We do all the training. We save all the information
For our graph we write out the information for the tensorboard and last we do a final
accuracy check to see how well
We did our training against our test data
So let's go over and look at the model
tensorboard and this model is a little bit different. It's
much
Easier to Follow I think we start down here. We have the input tensor, and we have reshaped
first convolutional Network in Max pooling
second Max pooling flattened
fully connected dropout and second fully connected and the output tensor
And then you have some parts up here
And I think those are actually the things that I did name so those are from the tensorflow model
so if I give names to all the last part with the cross-entropy and so on I think you will get
some nice boxes up here as well, but I think
the part that is really interesting for this
network is to actually follow how your network is structured, so
the part where you train your network is not that important because I wanted to see that my
actual network stack was the same the rest
I had to copy pasted from the other model, so I think this is a really nice
interpretation of the same network so Keras does a really good job of making it both easy to write and easy
to follow tensorboard
So the last part I want to talk about is how do we do the actual
representation or training in our
Java application and in Keras, and if we look at the model again
We have some part of the training where we say that this is the learning phase, so we say K
learning Phase 1 and
This is a little bit different because when you are
Testing accuracy you give the learning phase 0 so that's a boolean telling it if we are in the learning
Phase of the operations or not and that will actually trigger how to handle dropouts and so on
Lastly if we look at the differences
if we're running either the java application for the
tensorflow model or the Keras model and
Keras is running on tensorflow so
but in here we load our Keras model, and then we load in the images
Exactly the same way that we did in the other model, but there's a difference here the key probability in our
Java application for the Standard Tensorflow, then we had a key probability that was
actually a large array of floats
which all the values were set to one keep everything, but in this model. We actually have a
Learning phase variable that we set to Boolean False
So in this model here. We see that we have the input tensor and on the dropout layer
We have Keras learning phase and this boolean false so we can actually call this
learning
we can call it no_learning and
set that to the learning phase here, so we we have set
the value into the layer of
Dropout learning phase and tell it that this is not in the learning phase of drop-out should not happen
when we are running this model and
This is pretty much the only thing that you need to change
instead of having keep probability you have this drop out Keras learning phase instead and
my guess is that this Keras learning phase is something that you can set either for all your dropouts
pieces of your network or you can put it for the whole network as settings
But that's pretty much the only thing that you need to change in your java code and you will be off running
So I hope that you found this information interesting and I hope that you
know a little bit more about how this network works perhaps and
how to use Keras to make your network a bit simpler and to get started a bit
faster with tensorflow and if you have any questions, or comments
please leave them in the comments below and
yeah, hope you learned something and I really hope to see you in the next one