How to use OpenCL for GPU work Video Lecture | Machine Learning with Java - AI & ML

hello explorers and welcome to another
video another day another video another
Monday and today we're going to talk a
little bit about OpenCL and the
applications of running workloads on a
GPU and OpenCL
is really fascinating because this is an
open standard it will work with pretty
much any newer graphics cards in Windows
if you are using nvidia cards you just
have to have cuda installed i don't know
about AMD cards how you do it in windows
but in linux you just have to install
your either cuda or your graphics driver
for your specific card and get that
working on the linux then you install
the OpenCL kernel headers and you also
installed the OpenCL package for that
specific brand of graphic card so it's
quite easy to get into work in the
different environments and I won't go
into that in this video but I will look
at how you implement something in OpenCL
and compare it a bit to CUDA and we have
a video of that too if you just follow
the link up top there I will show you
how to use CUDA with workloads on the
GPU so first of you need some
program that you want to run on the GPU
and this is a very simple program you
have a kernel defined here this is a
sample kernel you have to float arrays
and float array that you have as a
result so A and B are inputs and C is an
output you get a global identifier here
and then you will in your result set
that A and B times each other so it's
a very simple program
it's just multiplies a number
values and then if we go into the main
program here you will create up a few
buffers here you will add some values to
those buffers you will create some
pointers to those buffers and now comes
some boilerplate here you have a
platform index device type and device
index so this is pretty much saying that
you want the first platform you can find
in the first device you can find of any
type
so the first GPU you find on this
platform of any type so if you have a
lot of different GPUs these values might
be something that you want to change and
then we set that we will throw
exceptions is a if anything goes wrong
so we get errors and that's a good thing
to do if you have and here we will just
fetch all the platforms in the system
and select select one of those here so
you will get the ID of that platform and
here we will get all the devices and we
will select one device here you have 12
lines of code only to get the specific
GPU that you want to work on then we
create some context for this platform
with these lines here and we will create
a command queue that will run on the device
it will run the specific workload that
we want and here we are starting to do
something with our specific data we
creates some buffers for this data so we
have a bunch of floats here out inputs
and outputs and then we take the program
we had earlier and we create a new
program from source we build the program
and in CUDA we saw that we
could actually build in pre steps we had
a pre-built binary there are program
to do this with OpenCL as well because
it's a standard you can build them
beforehand and the thing there is that
it's different programs for each
platform so I won't go in for that today
that day either
and then we will create a kernel in
memory of this built program we will set
some kernel arguments so these are the
inputs and outputs that we want to use
and then we will set a global work size
and a local work size and the global
work size is the full workload that
you want to work with and the local work
size is how much you want to do in each
unit and in this case we have one
workload in each units we only want to
do one computation in each unit and we
want to have the full global size and
the important thing here is that the
global size and the local work size
has dependency on each other so the
global work size need to be some
multiple of the local work size and you
can actually run work without specifying
the global local work size at all and
then it will give you a reasonable
amount of local work size which it could
be a bit strange because then you don't
know how much work you will do so
setting both of these is good so you
know what dimensions you're working on
and we work in one dimension so we are
not working with something in 3d or 2d
we are only working in one dimension so
if you're working with an image you want
to work in two dimension and if you're
working with some 3d objects you want to
work in three dimension what I will go
into that later and then we have this enqueue read buffer it will read the
result back from the memory object 2
and so it will actually give some data
back to
as destination buffer that we had
earlier and then we will release all
the objects we had here so everything
will be thrown away because now we have
done our work so if you want to do
multiple sets of work then you can use
them over again and don't release them
at memory but as we are working with
something that has memory that we
allocate memory to then we need to
release it afterwards it's very common
in C language to allocate memory and
then deallocate when you're done with it
and lastly here they have some code that
goes through and checks that all the
different calculations were done
correctly so this is the sample code
that you can find as the first example
on the open CUDA jocl site so jocl
is the Java binding for openCL or the
Java implementation of openCL and so
that's the package I'm using here I have
just added it to my pom file we can look
at that fast here so I have opened jocl
2.0.0 so that's the
version I'm running right now so that's
a simple program but as I want to
compare that with this workload that we
did in CUDA I want also to look at that
program and how to do a blur on an image
so in our other example we did a blur we
have a kernel again we have some image
size and there is actually something
called an int2 and that's a tuple that
will have an x and a y coordinates so
you can say that the image size is in
X & Y and so that's pretty nice to be able
to send those value directly into the
program and then we have the image so
that's a pointer to a buffer
has our image the start image we have a
pointer to a buffer with our filters
that's three by three square with
our little filter logic and then we have
a buffer pointer to a result and then we
have our filter size I just set it here
to three because I had it earlier in the
other CUDA example but that was global
constant and here I don't know exactly
how to do global constants I didn't
think that was that important then I get
the global ID I want to work on and this
is a little interesting here because
when you get a global ID and you have
set a local size that you want to work
on you will actually get a global ID
that is origo or in the middle of your
local size so you want to work around
that middle point so that was something
that I had to work around and figure out
how it actually was implemented and then
I get a width and height of my image you
see here i used tuples i get x and y
values so that's my width and height and
then we have my summary function here
and this is not much that has changed
from my earlier example i use a
mul24 which is something built
into the platform and i have saw another
example using that function I guess that
will actually do a better calculation
for me and perhaps speed up things and
it's probably not probably it's not that
important because we compile the program
beforehand so maybe this little micro
optimization wasn't necessary you see
here that I minus one on my y-axis so if
I have my filter three by three square I
want to start by the value one back
because I'm in the middle of the work so
minus 1 on the y-axis and minus 1 on the
x-axis will put me up in the top of the
of the workload
so I will work on the full set of data
and the filter is pretty much the same
here and then I want to come down here I
will save it to the X and Y value
beforehand I had X plus 1 and Y plus 1
and X plus 1 but that's not needed here
because we are starting from the middle
point so this is the little kernel that
I wrote so let's look at the program
here we have the kernel that I will send
him is a filter dilute and we have some
values here that will make a very subtle
blur on an image then I have some small
code here just to read a file so it
would read a file name which is a string
and give me a string back so that's not
important so I use it here I have this
test kernel that we saw and I read it
into my kernel source again set the
exception then we have all this
boilerplate of the looking for devices
and setting up context and so on and
building my program and creating my
kernel here I have my little reading
image code so I get my pixels back and I
also do a grayscale of the image so we
will work on a grayscale image which
makes things much simpler I get my
pixels and I start a counter to see
how long this operation will take then
we will fix a value that is number of
elements so this is width and height so
we get all the pixels it's pretty much
the size of the image and then I will
create a few buffers here I will create
some pointers to my pixels filter dilute
and also a result set and the result
set is exactly the same size as the
pixels so we get the same image back I
will create three buffers here and they
will have the
values of an int with number of elements
an int with a filter delute length so
that's the size of my filter and then we
have again int and the number of
elements so these are the three memory
buffers that we will work on when I work
on my image and then down here we set
some arguments to my kernel the first
argument is this int2 that I talked
about
it's a width and height and then we have
the three memory elements that I defined
earlier the global word size is the
width and height of the image and the
local work size is a buffer of three by
three which is the filter size that the
different kernels inside of my GPU
will work on so that's the actual size
that it will split up the workload in
and as we see here when we set our in
queue range for the kernel we say that
we work in two dimensions now and then
we set a global sized internal work size
and down here we fetch the memory
back so we read the output data so this
is at the memory two position and the
number of elements that we want back and
the pointer to that element set and then
we free all our pointers again and down
here we take a grayscale image get the
raster and sets all the pixels back to
our result from our result and we write
the grayscale image out and if we run
this little program we can look at the
output here we have the sample image
that I will work on and before we have
this image so this is just grayscale and
not any filtering and if we look at the
result here it's very subtle but you can
see a little bit of a blur
on this image so that's the little
function that I wanted to show you today
and I think that the big difference
between CUDA and OpenCL is that OpenCL
has a bit more boilerplate in it and if
you are working with OpenCL I should
recommend you to create some class or
object that you keep all your kernel
program and command queue variables inside
and that you use that and have some
initialization and some deconstructing
in that class and you send workloads to
that class or kernels and so on and so
building some little framework around
this would be quite nice it's very close
to OpenGL and how you build work in
OpenGL and OpenGL has even more boil
plate but I think it's pretty nice
because it's open and it's a standard
and it works on all graphics cards so if
you want to build something that you can
reuse on different systems and you don't
want to tie yourself to a specific brand
or a specific manufacturer then OpenCL
is a quite good solution to go forward
with I hope that you found this little
video interesting I hope that you
learned something today and if you like
this video give it a like and share it
with your friends and colleagues if you
haven't subscribed yet please do that
and I really hope to see you in the next video