Visualizing - 3D Convolutional Neural Network w/ Kaggle and 3D medical imaging p.3 Video Lecture | Machine Learning with Python - AI & ML

everybody and welcome to part three of
our tangled data Science Bowl 2017
challenge first pass-through of the data
in the previous tutorial we kind of
started running through the data but we
still haven't seen it we've seen some
things that are scaring us as as data
scientists but we still have not seen it
so today we are going to be looking at
the data so first thing is we're going
to using that plot loop you should
already have it installed if not to
install math long live if you don't know
anything about matplotlib you can come
to python program internet research
format file lib and you can go to the
crash course or much more full holistic
course would be the data visualization
course here with tens of tutorials
otherwise if you're already familiar or
you just want to learn along as we go
that's totally fine let's get started so
first we're going to import make sure
I'm not falling off my screen here let
me make some more space
great so let me import smack plot lib
dot pipe plot as
TLT and for patience in patience and
we'll just be the first patient for now
just to play with the data what are we
going to do well we're going to say and
in fact I probably could just rerun this
and I'm just going to copy and paste it
label good path slices we don't need to
do this anyone we're going to print it
out but instead already guilty kin show
and here you can pass the pixel arrays
so you can say slices 0.6 'l underscore
array and then i'm going to do peel TS
show shouldn't be necessary if you did
the whole matplotlib in line thing but
whatever up named TLT not defined what
oh did I skip i probably over wrote the
import okay import that
hi long as guilty try again
fantastic now I'm not a doctor I don't
know for certain that that's a lung scan
but it looks about right I mean you know
I probably guess that it is so so okay
so now we're going to do is okay we're
relatively content with that scan but
again we know that well there's a few
things one is we've got many of these
scans but also we know they're too big
so we have quite a few issues now for
the first resizing issue ie the
2-dimensional image resize we should
have no problem resizing that using
OpenCV OpenCV can resize this image
there might be some sort of numpy
function that we could run if you know
of one let me know again this is just my
first pass through the data I'm just
going off of what I know exists so so if
you know of a better way by all means I
mean if we don't have to import another
yet another library or install yet
another library that's ideal anyway we
are going to be using on pipes so make
sure you tip install numpy and cb2 I
don't have any numpy specific tutorials
but I do have open CV comes the home
page data analysis and make a video
analysis so it works there we go and you
can learn all about open CV all the cool
things you can do alright so once you
have this news let's get started so I'm
going to I just want to take this I know
I can't I just look maybe I'll just
modify this one instead so import map
what one it okay we're going to import
CV to import numpy at NT we're going to
set a single constant that constant will
be image picks size for now let's go
with 150 that's still going to be way
too big for a continent I mean most
people even on 2d images are doing like
I don't know 30 by 30 or something so a
3d 150 is you know that's a little
absurd but we're not going to keep that
but this is just for example
so okay so now we're going to do is for
the patient we got the slices and
instead of just seeing one slice we want
to see mini slices so let me we'll hit
enter here and let's do well first yeah
let's be big equals P Ltd figure and
what we're going to do is use a little
bit of programming ninja skills to just
show all of these up in like a single
matplotlib
figure using a for loop so what we're
going to do is for none each slice in
and numerate and we're going to open
enumerate is the slices we're going to
do we're not going to do all the slices
we don't need to visualize all of the
slices but we'll do the first twelve
okay what we're going to do : and then
we'll have this over and then we're just
going to just to define a temporary
figure let's say y equals fig dot add
underscore sub plot
it'll be a three by four sub plot grid
and then the number of this sub plot so
the way stopwatch work again you can
learn more on the data visit series but
basically it starts with a grid and then
what's the plot number in that grid that
you know which one out of you know one
through twelve is this one going to be
well it's going to be num plus one so
that's the figure that we're going to
plot that on now we're going to say new
new image is going to be a mesh that's
probably too small now free so maybe
zoom in a little bit just in case that's
hard to read we're going to say the new
image is going to be a CV 2 dot resize
any resize I'm pretty sure it's an
umpire array but I'm just going to be
absolutely certain and T dot array each
slice so again that's just each of the
slices each slice dot pixel array which
we've already seen what that does so
we're going to resize the first thing is
what are we resizing the second thing
will be you know the size
I'm super tempted to try to pass him
well we don't have all the data
convinced anyway we might I don't know
if see v2 dot resize can do a 3d array
CPR unlearn look it up like I said it's
going to be raw let's see see you too
now resize map 1 to map one needs to be
just as matrix doesn't say it has to be
a 2d matrix
I wonder I'm not going to test that one
because we'd have to also build all the
slices into one that shouldn't be too
difficult I don't know if someone wants
to try it go for it that might work
anyway we're going to just say we're
going to go this and then it was image
what was that keeps constant image picks
size by that so the dot pixel array is
already a 2d array and so we're going to
resize this array into these dimensions
so whatever these dimensions are we're
going to use an open CV function
basically to just make it a 150 by 150
so we're just downsizing everything this
this also just for the record does work
if things aren't all the same size
starting out so that's why I was like oh
I wonder if that works 3d I really don't
know
so yeah I'm almost done with that for
now
so there's nothing to manage we are
going to do y dot M show rather than
slices 0 I'm going to say new image and
I'm going to key won't be dot pixel
array it'll just be new image and then
we don't want to PLC dot show until the
very end and let's run that see what we
got open no errors boom beautiful
beautiful so each of these is a slice in
order basically as we're going down the
line ok so now I'm very confident that
we'll obviously we've got the scans the
colors are a little strange so we're
just going to take note
it we might need to normalize these at
some point mmm try think there's
anything else I really want to bring up
at the moment was it just like the
colors is probably suspect we you can do
like see map equal I think you could
just throw it in here see map equals
gray I think we can do that yeah and
that makes them all gray but there's
still lighter and dark like the computer
doesn't care about the color so I'm not
really sure that would be an issue with
enough data I don't really think that
would be an issue I really think already
show at the end of the day is going to
be the date account okay
so we've solved the you know the side we
saw we solved this issue but we still
haven't solved this issue and even more
important I guess it's not there anymore
but we haven't solved let me do this
this issue where these are different
sizes so that's an issue that we're
going to solve in the next video so if
you have any questions or whatever up to
this point leaving below otherwise I
will see you in the next video