Object detection Part 6 - Inference in java [Tensorflow] Video Lecture | Machine Learning with Java - AI & ML

hello explorers and welcome to another
video today I'm gonna talk about
inference and inference is when you take
your model and actually do a run through
your model just because you want to
figure out what is the thing that I'm
looking at so in this specific object
detection model we are trying to train a
model to see different things we started
off by trying to finding waldo and in
the last video I actually used a
pre-built example from I think it was
the coco dataset which had a
lot of images in it and I trained that
and it knows about airplanes buses and
cars and so on and persons and I showed
off how to use that tensorboard and
now we're gonna run inference and that
means that we are gonna run it on one
image and see what can you find in this
image and in order to run inference we
need to export our model so if we export
a model we get a few formats that can be
useful in different applications so by
exporting the model we just run Python
object detection export inference graph
from the models research folder when we
have everything set up we supply the
input type image_tensor and in this
case i want to say that i will supply
you with an image with a specific size
into the model other types that you can
do is encoded image string tensor and
that's if you want to give it an encoded
JPEG or a PNG image string instead you
can also give it a TF_example
and that is a TF record that you give it
in order to do
inference but I choose to actually give it
image with all the bytes from an image
then we need the pipeline configuration
path so that's the path to the
configuration that we are used in all
other runs and we need a training
checkpoint prefix and that's everything
until the last part which tells you how
long this model was running so example
you see up here is model checkpoint
through 200,000 steps and our model was
actually run 200,000 steps so it's ends
in 200,000 and then we have an output
directory so that's where you want to
put the output from this function so
what do we get in our export folder then
this output directory first off we get
the saved model and this is the new save
model format that you can use in Java or
other languages in order to do inference
and that's the one we're gonna use today
we are also have the old checkpoint
model if you want to use that and we
also have this frozen inference graph
and this frozen inference graph
could be converted to a TF lite model
and used on a phone for instance it's
good that you get three different
formats by running just one command and
you can use it as you like and if we're
gonna run inference then we need some
code for that and I have written some
code in Java here first off we have a
labels file and this is all the things
that we can find inside of a our graph
so this is a setup that they had all the
labels that is supported by
this model and here we have my little
string telling me if we actually have
tensorflow
running so hello world I'm using tensorflow version
and then we type out version it's good
to actually have one of these when you
setup Java the first time in order to
ensure that you actually have everything
set up correctly so you can run tensorflow from Java I have a video on how to
setup and run tensorflow from Java so if
you are interested look at that video
and then we have the saved model bundle
that will read this saved model and I
will point it to my export folder and
the saved model directory and I will use
the tag serve and serve is very commonly
used as a tag for the models so most
models had these tags you can get this
specific thing you want to run inference on
I will create a session on that and then
I just read my image to a buffered image
I will get it in here and redraw it on a
new image that is of type RGB because I
can get any kind of image but I want an
RGB image in order to run it through
inference and I type out the print the
type here and then I will get the width
and height and times that with 3 because
we have three colors in our image we
have R G B so we would need three we need a buffer size of height times
width times three so three dimensions
and then we will allocate that buffer in
byte buffer I will go through all the
pixels in the image
get the RGB value and then put a byte of
each in inside of the byte buffer so
this are RGB values that I will put into
the byte buffer and then we will
rewind the byte buffer so I start from
the beginning before I actually
and push it into the tensor or else who
will push an empty bytebuffer because it
will not start from the beginning and
then we create a new input tensor here
with in the first dimension I have one
in this matrix and then I say height and
width and then the last is how many
colors so it's three so this is the
matrix telling you how the shape of the
image is and then I supply the byte buffer
and we see that these are unsigned int
eight so it's eight bits byte so this is
a byte to tell you tell the model what
kind of input we have in the tensor and
then I put in the image tensor here into
I will feed that into my model and then
I will get detection boxers
detection scores detections classes and
number of detections you can also get
detection masks but I'm not using this
in my example so these are the things I
want to fetch out from the model so this
is things that will be created inside of
this tensor list here so I will get a
list of tensors that contains these
values these output values and then I
will run my model and the max number of
classes I have and that's defined in the
configuration how many classes you want
out from the second stage in the model
and that was 300 and then I'll go
through here and create boxes detection
boxes scores detection scores so these
are the different training things I want
to copy out so the detection boxes is a
copy from the result two boxes so I will
get the detection boxes here and scores
the same detection
scores detection classes so I will copy
the values out from the result so 1 2 3
0 1 2 so that's the these and then I will
get the number of detections as the lost
because we have that up here so boxes was
the first one so that's 0 1 2 & 3 so we
will get all these values out and the
number of detections I will round that
value up and present it to the outputs
we know how many detection we actually
got then I have a little for loop here
that I will run through the three best
predictions so 0 to less than 3 and I
will print them out start printing a
line so I will get a line in top and
then I will get the result so we can
read it easily and then I will get min
and Max of Y and X in order to see how
the detection boxes is located in the
image and as we see here they are
actually normalized so it's a value between zero
and one I need to multiply them with
the height and width in order to
actually get the correct value and then
I print out the Box values here and I
print out the score how good it actually
was and I also print out the class that
was predicted and then I set up an image
here so let's see we had the i>mage up
here that we created and we also draw
the image here with this graphics 2d and
I will continue to use this graphics 2d
down here and draw the color with the
color red I will draw a rectangle at the
position of min Max
this XY so actually draw the box and
then I will set the font Verdana size 20
so we have a good sizable font as we
can read and I will print the label on
the image as well and that will be above
the actual box in order so we can see
which box is which prediction and then
when I have printed out these three I
will write the image out to disk so we
can actually look at the result and see
where the box is actually located
because when you are doing object
detection on an image and you get a box
coordinates it could be hard to actually
figure out is this a good result or a
bad result if you don't check it against
a known result so this is an unknown
image that we want to try and then I
will just pack it up with the Maven
package and in this case I've already
run this little little Waldo snapshot
jar with dependencies I put in an
airplane image and I got an airplane out
image you see here that I got 20
detections and the first one had a score
of 65 percent and it thought it was
prediction number one if we go up here
and look number one is an airplane and
then the second one was 55
that's number 15 I know that that's a
person and then we got an airplane again
at 47 so let's look at this image then
so this is the image we see here I think
the large box is the best prediction of
the airplane so that's the 65% then we
have this thing it thinks is a person
perhaps could be reasonable and then we
have an airplane again we could strip
this down to say
we want one prediction perhaps but let's
do one more we take an image I have
never seen before let's search for sheep
we know that sheep is one of the things
in the image we can take one with
multiple sheep perhaps let's say we take
this one and we open this image in new
tab we can save this image as a sheep
there we have it and then we were on
this again sheep jpg sheep and I have
the output of PNG so let's call it
output PNG see what it actually predicts
so this is an image I have never seen
before the model has never seen before
how well can it predict the sheep's in
this image and there we have the
predictions done we see here that
actually exceeded some maximum allowed
memory so maybe you can run this with a
little bit more Java memory heap memory
so you can actually run it faster
perhaps we see here we have really bad
predictions we predicted 12 and 13 and
13 these are quite 32% 28% 27% doesn't
look that good so let's see if we can
look at this image here so here we have
the predictions it thinks that this box
here has a dog and it believes that it
found two boxes with horses so that was
not a really good prediction I believe
so let's try one more we do one easy one
we will search for a person and take
just a one image of one person see if we
can figure that one out
see we if we have any good he looks
happy let's take that image we will save
that image person and we will go back
here we will do a new prediction person
person output PNG let's see if that goes
better with simpler image with less
objects and perhaps an easier prediction
yeah yeah now we have run the model here
it found actually one I think is wrong
number eight eight is not a person so it
found something else in the image with a
42% and then it found two rectangles of
person with 33 and 32 percent accuracy
so let's look at these images again and
see what it actually predicted so I
think the person was really good it
found a person here and it found a
person here so these boxes of person are
really good and this box here we predict
that it found a cat that's a bit weird
and it is better at actually finding a
cat in this image than actually finding
the person so that's an interesting
result but as we saw in the earlier
image earlier video this model is not
well trained it has a lot of error in it
and we can improve on this image in
order to improve on this model in order
to find objects better and so this was
what I wanted to talk about today I
wanted to show off how to do run
inference in Java you
using a trained model and I think this
is pretty much the end of this little
series as well you can use the
techniques from this video use the
frozen model and create TF light model
that we run in the example program on
your phone that I showed off in the
earlier video I hope that you found this
video interesting I hope that you
learned something today
if you have any questions at all please
leave them in the comment section below
if you haven't subscribed yet please do
that and I really hope to see you in the
next video
Oh