Using Tensorflow models with Java Video Lecture | Machine Learning with Java - AI & ML

hello explorers and welcome to another
video and today I'm going to talk a
little bit about tensorflow and how to
run tensorflow with Java so to start
this off we're going to talk a little
bit about installing tensorflow from
source and why we need to do that we're
going to talk a little bit about how to
train your model in Python because at
the moment the only way you can train
your model is by using some C
dependencies and Python but in the
future you might be able to train your
models in Java as well and then we're
going to verify our model inside of Java
and talk about that code a little bit so
you can read a model in Java and
actually run it to verify and actually
do predictions in Java so that's quite
nice and lastly we're going to talk a
little about this but we're tensorflow
is in relation to windows so I have done
a lot of work trying to get tensorflow
working on windows and I'm going to talk
a little bit about my experience there
so to start off we're going to talk a
little bit about installing part and I
have written down a little bit of a page
documenting how I did my install I used
ubuntu and a lot of installation
that way most linux's and Mac's should
be fine installing a lot of information
of how to install it
the prerequisites for this project I
have is pretty easy you need Python of
course maven tensorflow the tensorflow
jni and I and the tensorflow Lib jar
file and how to create these
in order to get this project to
work at the moment we need the latest
version from source so if you can get a
hold on in the future a version of
tensorflow
which is one point 1.1
1.0.1 perhaps the functionality in
this project might be there already but
at the moment the only way to get it is
from source because it's quite new
functionality but I think that this new
functionality actually makes a huge
difference in how easy it is to port
your models to Java so the current
version that is out now is 1.0.0 preview
1 and that doesn't have this
functionality so to build from source
you need to do a few steps first you
need to build bazil and that's a pretty
easy process it's not in a main in the
main repository of most Linux's so you
have to add repository and update it
install java of course you need Java to
run it and then you have to install and
build bazil and but the in Linux that's
quite easy easily done and then you
build tensorflow as a pip package I
found that that was the easiest way to
do that and that's pretty much take the
source down configure it with some
dependencies and then run a bazil build
and after that you can build the
bindings pretty much the same thing
there some dependencies configure it and
run a bazil build and you will get the
jni which in linux is an so file so
it's the so library that you install
in a jni directory in in this project
so our runner can find it but actually
the java runtime only need to know where
your library is in order to run this
project and then you get the jar
file that you also need to include in
your runnable process and my way to do
that is to actually add it to
my maven repository
then run my java application in maven
which makes dependencies a breeze in
Java and then I have talked a little bit
about how to run this repo so in the
current version they also have this
download jni and this code could be
found at the tensorflow a webpage
and it pretty much downloads the latest
version of the jni but as you see here
it has this 1.0 0 preview so you get
a tar file with the jnI built for the
preview and this is too old so you need
a newer version and you can also
download the jar file separately but
that that will also be this 1 0-0
preview which is not new enough for this
project and this project that I have
built here you can find on github so you
can download it and try it by yourself
if you like documentation and everything
is included so let's look at some code
then so the training part of this is the
mnist training PI and all this is also
available at the tensorflow web page but
I will point out some things that I
added that I think makes the porting and
working in Java much easier for you so
first up we have some imports that you
connect you can actually import the mnist
model into your Python which is so much
easier than reading it in Java and then
you start a session an
interactive session you create
some placeholders the X placeholder is
where you input the image so that is
very important to have a name so
you can find it in your java application
so I call this the input tensor and then
we have this Y variable placeholder and
that we will not use in Java because
that is the actual label that the model
is trained by
and in Java we will not give it a label
because that is what we want to
predict so that is not important to add
and then the website the tutorial I
followed gave us some small functions
here we define weight variable functions
we can create a weight variable with a
specific shape and then we can create
bias variables with a specific shape we
have a two comp which creates a very
easy convolution in two dimensions and
then we have the max pooling 2x so we
will shrink the model after each
convolution and these function makes the
code a little bit more readable down
below
so we create a weight variable and a
bias variable with a specific shape then
we reshape the image so it will fit into
our work we do a convolution with the
pooling we will shrink the image and get
some of the information out here we
create another weight and bias for
the next round we do the pooling
relo and Max pool again and then we do
some weights and variables again and do
a reshape relo and lastly here is a
very important step to here we
create a placeholder for the drop out
layer and this placeholder is actually
the probability that we will keep a
specific value in the matrix and in Java
when we do our predictions we actually
want to give it a model or a matrix that
is full with ones so every value
will be kept which is very important for
predicting when in training you want to
drop some of the values out but in
prediction you want all the information
so you get the best prediction so this
is also very important to give it in
name you can find that in Java and then
we have some more weights and biases we
do matmul and lastly of this matmul
matrix multiplication we want to
identify this matrix manipulations and
that is actually this y_conv is the
actual result of all this computation
and in my case in Java this value will
get a value like add 3 or add 4 because
the last operation here is actually an
add and to find out which add in our model
we want to do to take out and actually
do some work on or see our predictions
is quite hard to find that so this nifty
function that we just give a variable
and that will add a name to that
variable it's very handy so you can say
that this is my output tensor so you
know where your output will actually go
and then we have a reduce with the logits and then Adam optimizer so this is
the actual training worker which will
optimize each of your weights and biases
to minimize the error and then we check
how many correct predictions we
have and fetch an accuracy and this
accuracy will late to be shown
during the training process we know how
good our training actually is and this
is a very nice addition to tensorflow and
this is the function that actually was
added in 1.1 and this will give you the
functionality say that in this folder
model will I want to save the full model
where the actual
of training model and all the
weights and biases so you can load it
back and use it so it's very nifty to
have this function that you can say I
want to builder for this and then we
start our session and we do the actual
training over 20 thousand rounds with
the batch size of 50 and we do some
training and we print out the accuracy
for each training step and down here we
have the Builder again that we talked
about up here where we
actually add some extra graph metadata
and we gave it a constant and in this
case we say that this is a serving model
and then we save it and this true value
is if we want the model we create to be
humanly readable so it will be a text
output it's actually going to be
a json trees we can read the model in
JSON and or if we want it in a binary
form we'd set this to false so if you're
not interested in actually looking
at the model that will be generated you
can set this to false and that will be
fine too I said it the truth I can
actually look into the model and was
quite nifty when I wanted to debug and
find out which values I needed to give a
name to work with them and last but not
least it will print the test accuracy
and I ran this training and I got it
training accuracy with the test data on
99.18 this is a very good
training or at least pretty good, you could with a larger model
you might get a even better
results but I think for such a simple
model that took a few minutes to run on
my computer it's a very good result for
these training set and I have done some
other calculations with the other
frameworks and most of them actually
will not go over 98% so
99.18 is really good and let's
go over and look at the Java then how do
we extract this model and use it in Java
so in Java here we have some imports and
we print out which version of
tensorflow that we are working with it's
quite nice to do that so we know that we
have the right tensorflow version
running and it's also a good pointer to
have because this function will actually
fail if you haven't the right jar
installed or the right libraries
installed this will fail right off
the bat so you know that you actually
have a working system so it's good to
have this in your runner and
then I have my own little class here
mnist reader it's a very simple class
which takes the mnist information and
reads the binary data and gives me each
label and each image as a separate thing
so we can say give me the next image or
give me the next label and it will give
it a give me read from the binary data
so I have built something that handled
that model that these files are saved in
which is quite handy to have and then
here we have the the load of the model
that we talked about earlier so it will
look for this model and here we have
actually my full a full address in my
virtual machine my virtual machine is
called george so I find this model here
and the tag with called
earlier this serve tag it's important to
say that I want the serve model and so
you could actually tag it with something
else but with this you will get the full
model back and then you can start a
session by us starting a session on that
model and you're up and running that's
pretty much everything you need to do in
order to start working with your
model and then I'm going to test and
predict all the values in Java as well
so I start this runner I will go through
my full test set that's 10,000 images
and from the mnist data set I create a
float buffer for the full image the
image is 28 times 28 pixels so that's
784 pixels
I read the bytes in my for the next
image because the images are saved in
bytes so I will get the byte array from
0 to 255 and in order to give that model
back to to the to the tensorflow model I
need to change it over to a float buffer
because tensorflow is working with float
buffers so I change each byte so they
will be unsigned and then I divide them
by 255 so I get a value from 0 to 1 and
that is what I will feed back into
the model and after I have done that I
rewind the float buffer because
tensorflow would read this float buffer
so I want it to be from the start and
then I have my probability array
that I talked about earlier and I want
1024 floats of probabilities into
the model and I will fill that array
with one so we will keep all the data in
the drop out layer we will not lose any
data and then I create my input tensor
784 with the float buffer and then
keep probability tensor which has the
shape of one and a matrix with one of
the 1024 so it has a it's actually a
matrix with multiple layers so and
that's area just wrapped in a float
buffer so we'll get a float buffer of
that array that I created earlier and
here we have the function that does it
all we will want the tensors result we
will get back here we have this runner
that we will feed the input tensor to we
will feed the key probabilities we'll
keep all the values in the drop out
layer and we will fetch our output
tensor and run it and get the first
tensor back because we can actually get
multiple rating multiple answers here
but we only care about the first for
predicting and down here we create this
little float matrix will which will take
the actual value and I will create a
number of loads here's I actually have a
field array I'm not sure that that is
required but I had a lot of work to get
this to work but so it might be that
these two rows are not actually required
but you need to create somewhere to
store your your result and then I copy
this
tensor into this results matrix and the
matrix I get out I will go through each
value and see how much it's predicted
for each of the results so for
instance the first value that will be 0
so how likely do the model think that
this image is zero the next one is the
first one so how likely though the model
think that this is a 1 and so on and in
order to actually find the highest
predicted value I have this small for
loop where we go through each value and
look for the largest value and save that
prediction and down here I will fetch
the next label and check it against the
predicted label and if they
correspond I will increase my correct predictions so I will
get how many predicted values that were
correct of this function and then I
print out we have these many correct
predictions which is these many
predictions in percent and this is a
little verbose but when I ran
through this
it took about 21 seconds to run through
all the 10,000 predictions and Java
actually got us the same value
predicting this model so we've got a 99.18 result or
9918 results correct
which is really nice we know now that we
can port a model over and actually get
the same result as in Python which is
very nice now we know that this
process will work and then I'm going to
talk a little bit about little small
steps in the setup here we have this
problem file and the way which I talked
about that these things is much easy to
do in with maven to get your dependency
working so I have some setup here that I
just create the project and so on this
assembly plugging is really nice you
just tell it that you want this as your
main class and a descriptor reference
that you want jar with dependencies and
then put an execution goal in the
package phase and that's pretty much
everything you need to know or anything
you need to do and add your dependencies
below and when you run make package you
will get a jar file with all your
dependencies included so you can run
this your file and or move it to a
different machine and run it and so on
to have a packaged and working bundle of
your code which is quite nice and I also
include this small run script and that's
just so you can run Java point out where
the Java library path is which in this
case is jni and there you will put
your library for your system in my case
it's libtensorflow.so it will
find this C binding C bound library
for tensorflow and then I run this jar file
which were built in my target directory
so that's pretty much the setup and how
to run your model in Java so you can
train of a model in Python port it
over to Java and predict stuff in Java
so you can use it in your Java
application if you like and there now
we're going to go back a little bit to
my experience running this in Windows
and as you see here
added it tagged experimental to this
little part of my documentation and
actually this was something that I did I
think I spent two or three days on this
and I actually didn't get it working the
really annoying thing about Bazel is
that in order to build basel you need
bazel and in Windows it's quite hard to
get a version of bazel working in your
system and another really annoying thing
was that the bazel version 0.4.4
had a bug in it which did it
made it impossible to actually build
bazel with it because this bug was
screwing things up so I did some tests
where I tried to install it with the
chocolaty and that did not work well
then I tried to build it from scratch by
building protobuf first then cloning
bazel and then trying to build bazel and
this were really hard to do and after a
long while i actually got it to work
somewhat right but when i was done with
that i tried to build tensorflow
and i'm not really sure if it was in the
build process of tensorflow
or if it was in the build process of bazel
but along the way I had to actually do a
lot of changes to some C libraries I had
to look up a lot of different
dependencies that were not mentioned at
any point in any tutorial and I also had
to change some C code add some extra
define in some of the header files and
so on so some this to make a long
journey and in a long statement to some
short brief description if you want to
build things in tensorflow build tensorflow
in Windows at the moment don't wait
until you we get a more stable version
of bazel perhaps version 0.5 or even 1.0
and wait before tensorflow has not a
preview tag on it maybe things will be
easier to build on the windows but as of
this recording we are in the middle of
March 2017 it's not recommended actually
to build tensorflow in Windows yeah
that's pretty much what I'm going to
talk about today I hope that you have a
little bit more understanding about how
to train a tensorflow model and
actually predict things in Java I hope
you learned something if you thought that
this was a something that was
interesting please share it with your
colleagues with your friends and give
the video a like and subscribe if you
want more videos like this one and I
hope to see you in the next one