Object detection Part 2 - Configuration [Tensorflow] Video Lecture - Machine

hello explorers and welcome to another
video today I'm gonna talk about machine
learning and object detection this is
part two in a series and today we're
gonna talk about the configuration how
we can configure our object detection
training so we actually and evaluation
and so on so we actually get a good
model that we can work with and these
kind of parameters can differ on
different networks of course today I'm
going to talk about the resnet RCNN
version but there can be differences in
the other ones and one thing that I
found when I was researching this is
that there isn't actually much
information about this if you search on
any of these parameters in this
configuration many times you come to
github question where they tell the user
to submit this on Stack Overflow because
that is a better place to ask questions
and when you go to that question on
Stack Overflow it's an question
without an answer so I believe that this
is something that we really need to get
together as a community and actually
gather the information so there is a
good way to read about these different
parameters because they can be quite
obtuse and they can be hard to
understand and I'm gonna give you my
collected knowledge about the different
parameters and some of them can be
unclear incomplete and I can also be
wrong about some of them but I believe
that I have a pretty good understanding
about the different things that goes
into this configuration so let's see
here this is a configuration file we
start off by defining a model and in
this case it's a faster RCNN model it
has a number of classes in my case
with Waldo we only have one class and
that's Waldo and then we have this image
resizer and this is because you want to
give the network any image size but the
network can't handle gigantic images
because that would require too much
memory on the GPU and so on so you need
to downscale it but I don't want to
downscale it too much so they have
actually fixed it in a way that you
downscale it to a dimension and they
want to keep the aspect ratios they just
scale the image down to where you can
actually do feature detection but don't
have a too large image so in this case
they supply you supply it with the min
and Max dimensions so they choose a
value between that that is a good down
scale because sometimes you can't
actually just say one specific size and
then you will get some artifacts in the
image so they will find a good down
scale that you don't get too many
artifacts and then we have the feature
extractor and that's pretty much the
network so here you can say that I want
a faster RCNN res network 152
layers so that's the network I choose and
the first feature stride this means
we're in the image should you find
things and how much should you move your
window when you're looking for things
and in this case I say that this feature
stride should be 16 so it's 16 pixels
moving through the image and that's in
the different scaled images as well
because when you go through the
convolution you actually have a lot of
different sizes down scale all the way
down so if you have really small objects
you might want to change this value to
something lower just to be able to find
these
item so I might try to change this to 8
or 4 with Waldo because Waldo can be
quite small in some images so I'm
thinking about maybe tweaking this value
in the future then we come to the first
stage anchor generator and this is also
something that is really interesting
here they create a lot of small images
in your image where you they can find
features so in the different scales
everything from 1/4 to 1/2 and the full
image and also the double scale of the
image and also different aspect ratios
they move a window over the image and
create different anchor points where
they will actually look for your
specific object so this is a generator
to create where in the image they should
look for objects and how fine-grain this
method should be next up we have our
convolution parameters so here we set
the parameters for each convolution in
the layers and the operation should be
collusion and then we have this L2
regularizer and from my work with the
Java CNN this l2 regularzer is
something where you can give it a little
boost in weight to change how
how easily this model can
over fit so this is one value you can
change with a very small increment so
you can give it a weight of 0.01
perhaps if you feel that your network
will over fit a lot so that's one
measure you can use to fix your network
so it don't over fit and then we have
the
initializer which will use a truncated
normal initialize it with a standard
deviation and that's because you want to
create random numbers in your weights
but to get your training up to running
faster you want it to diverge over the
network so you don't want totally random
you just want it to be pretty much
totally random so this this will give
you a normalization of all your
random weights so they don't you don't
get for instance if everything was
totally random and you had a value
between 0 and 1 then all your values
could actually one that's random but you
don't want everything to be 1 because
that will not make it a good training
set so this is an attempt to give you
something that is similarly random but
still have a diverse group of the random
numbers and then we have this set of
parameters here that we have a lot of
different things going on here we have
the first stage score threshold so in
the first stage of the convolution how
which value should I throw away so if I
find an object with a score of if I set
this to 0.1 then everything that is 10%
and less will just be thrown away and
also if you look at this intersection
over Union so that's how much how
closely you found this object in the
image how good was the match if that's
too bad you throw that away as well I
know that's also a value about between 1
and 0 so if you put it at 0 you will
keep everything in my case I throw away
things that don't really fit well
because I don't want to keep all the
really bad matches
you bought this to 0 then maybe you will
have matches everywhere and that's not
good you want your bounding boxes to be
a little bit better especially when you
in the next stage say how many
proposals do I want to save max in the
first stage and in my case I put 100 and
if you keep all the bad ones then you
will have 100 bad ones that's not good
so these values you can trim to say how
much do I want to keep from the first
round and then trim it down to a good
set of values that you can work with and
then we have the localization loss
weight and objectness loss weight and
this is also similar but they are
looking at the actual loss instead so
here you have a boost in localization
and objectness so how much should the
loss be pushed for localization or
objectness so in this case they are more
interested in where things are located
than what it is but if you can
switch these around if you are more
interested in knowing what it is then
where it is
so that's those values and then we have
the initial crop size haven't found
anything good to say about this it's
it's some cropping of the image you
remove some part of the outer edges of
the image I guess no documentation
whatsoever anywhere I couldn't find
anything in the code so I'm not really
sure what this will entail be aware that
if you put something very close to the
margin of the page and you have a crop
size of 14 pixels maybe you will remove
that that would not be good because then
you will only find things in the middle
of the image perhaps max pooling and
kernel size and stride that's when
you're shrinking the image that's how
much you want to shrink the image in
each layer so if you have a kernel size
of two and the stride of two you will
shrink the image by half I believe
almost half so that's what's going on in
each max pooling layers you have a
convolution max pooling and then some
feature activation and then you have a
convolution max pooling feature
activation and so on so you go round in
each layer and having each of these
parts in each layer then we have the
second stage box predictor so this is
the net when you have run through all
the convolutions what you want the last
part of this model to be and in this
case it's a mask RCNN box predictor
and you tell it that it should be a
fully connected layer and you want to
keep everything so you set you'd drop
out keep probability to one so keep
everything and use drop out this set to
false so I'm not using a drop out layer
at this point and if I feel that my
network over fits too much it's too good
to actually find just the examples given
and bad at finding new examples then
it's good to have a drop out layer set
it to 0.9 for instance so you drop 10%
of the values and then you will get a
network that is not over fitted and also
in this fully connected layer you have
this l2 regularizer so you can
tweak that weight value as well to
also decrease the
likelihood that you have a model that
will over fit and then we have the
initializer for variance scaling
initialize and that's to figure out how
to actually take your features and then
upscale them to your image so you get a
good match with the bounding box on the
image and here you say that it should be
uniform and it should be an average
between input image and output image
that's pretty much what I can say about
this I don't know how this variance
scaling actually works but I think that
these parameters seems reasonable I
don't want to I don't want it to look
only on the input values and I don't
want it only to look at the output
values so an average sounds very
reasonable because you want the bounding
box that are close to the object from
the input but also match the actual
output of the graph as close as possible
so I don't want to measure one of them
over the other so this is one that was a
little bit hard for me to wrap my head
around and then we have the second stage
post-processing and here you actually
tell it what you want to keep at the end
so if you what would you want to know
what you want out from your model and
how strict you want to be with your
output so here we have this score
threshold again so if you have objects
that it not isn't sure what it is then
throw them away and if it's bad match of
the bounding box throw those away
that's the IOU intersection over Union
again and then we have the maximum
detection per
so if you let's say have images with
airplanes bicycles and cars and you want
to find ten of each maximum and maximum
a total of them so you could find ten
airplanes ten bikes and ten cars but you
will not find twelve airplanes for
instance and then we have a score
converter we use softmax and I think
that's reasonable as well it's very
commonly used in convolutional neural
network as the lost activator for class
detection so that's a good one as well
and then we have this weight again
depending on classification or
localization in this stage as well so
you can switch these around depending on
what's most important in your network
then we come to the training
configuration and here we tell it what
batch sites we want depend how much data
do we want to read in each iteration and
here we have an optimizer
momentum optimizer learning rate so this
is just a way for them to encapsulate
different things in this in this
configuration pretty much because
learning rate that's how fast you want
to reach your endpoint and how little or
how much you want to change each time
for each iteration when you find
something that is wrong you want it to
push it a little bit in that direction
and then you have a momentum optimizer
and that's a way to speed this process
up with a little bit so if we continue
to look here we have the initial
learning rate so we want it to push it
with a
0.0001 to start off so we
push it a little bit in that direction
but we have a schedule that says that we
have when we have stepped through more
than 900.000 steps we want
to scale this down even more to a
0.00001 and the same
goes for 1.200.000 steps we
make it even more fine-grained but we
have a momentum optimization
optimizer value of 0.9 which
i think is pretty much a little bit too
much even so I would scale this
momentum down a bit because I would
rather have a model that is well trained
than reaching the result fast maybe you
can skip the momentum optimizer at all I
don't know how this configuration is
actually read in the training if that
is possible at all then we have this
moving average and that's also an
interesting function that I haven't
turned on yet and that is actually a way
to cache the result from the last time
and that cached value will slow down
your pushing of values so you will first
calculate how much you want to change
your weights and then you average them
with the last iteration so you actually
slow down the process and perhaps reach
the good medium quicker and so that's an
interesting feature I haven't turned it
on and I don't at the moment don't see
any use for it but if you want to try it
out it's just a flag
and then we have this gradient clipping
by norm and that's a way to make the
gradients not overflow over the specific
values so when it comes up to that value
you clip it at that values it's pretty
much saying that the gradient shouldn't
reach over ten can't say much about that
the fine tune checkpoint and from
detection checkpoint though these values
is if you want to start your model we're
using an already trained model but you
want to fine-tune it so you take for
instance read inception model put it into
your training and you need to have the
same model set up of course and then you
take that check point remove the last
layers of the fully connected layer and
then retrain it to fine tune and create
your own fully connected layer for your
specific problem so that's a very
powerful feature that you can use in
order to get a good network up and
running fast if you are doing feature
detection that are very similar to an
already trained network and then we have
number of steps and this is how far this
training should run you can remove this
and then it will run indefinitely so
forever but it could be good to actually
set it to a specific number especially
if your training online so if you are
running it in a big cluster in the
compute cloud it's good to set a number
of steps so you know when it should shut
down so you don't just run it without
getting that much better result because
this network will give you pretty good
results even after 200.000
steps depending on your input
of course how good are your examples and
so on but many of the models that are
provided as examples are pretty good
after 200.000 or 500.000 steps and then we have
this data augmentation option random
horizontal flip so it's actually when
it's training this network is actually
flipping the image in horizontal way
back and forth in order to make it more
robust against different orientations of
the image
not orientations but you know what I
mean a different side so if you see a
dog standing you're looking at to the
left it should not be only finding leftwards looking dogs it should be also
finding rightwards looking dogs so on
so that's an interesting feature to
actually flip the image from each
training and then we have the Train
input reader it's pretty
self-explanatory you have a TF record
input reader where you set the path of
your TF records file so from where
should it read your examples then we
have a label map data file and this is
where you have all your different labels
so for instance in my case I have Waldo
so that's one label and we will look at
the label file later so it's pretty
simple file and then if you want you can
supply shuffle the input and how many
readers you should use to read the
records so that's how many threads
should actually read the data and that's
because you if you have a very fast GPU
you might want to spend more time
with different CPUs to read your data
from disk so you can actually provide
enough data for your GPUs to be fully
efficient
and then we have the eval configuration
how many examples do you want to
evaluate and how many times do you want
to evaluate them also very
self-explanatory configuration and then
we have a similar reader for the eval so
here you supply the TF records that
you want to evaluate with and also the
label file for those and if you want to
shuffle and how many readers and lastly
the label file and this is my label file
it's one item so it's only one label it
has an idea of one so that's the actual
label value saved in your records
examples and then we have a name for
that label so this is what I wanted to
go through today as you might have the
understood everything in this example
this video is not facts it's my
understanding of the configuration file
I hope that you find these files a
little bit more readable by now I hope
that you understand this process a
little bit more I hope that you learned
something today if you have any
questions suggestions or thoughts leave
them in the comment section below if you
like this video give it a like share it
with your friends and colleagues if you
haven't subscribed yet and want to
follow along in this series please
subscribe and I really hope to see you
in the next video