Advanced Fortran Programming : 031 : OpenMP Demo in Fortran Video Lecture | Introduction to Fortran Programming (AdvancedLevel) - Database Management

hello guys my name is Aaron and welcome
to my channel this series is a series of
tutorials on advanced photon programming
now in this tutorial I'll be explained
you guys as this explaining guys how
doing in work openmp procedures in
Fortran now don't get alarmed as I
mentioned in one of my previous
tutorials and one of my last tutorial in
the first series the basic Fortran
tutorial series of openmp and Fortran
combination will be a separate tutorial
series and the planning of that in some
time ok this will be just a demo of how
openmp can be used in Fortran ok now
what is you know what is openmp well
openmp is one of the two kinds of
parallel programming parallel
programming paradigms available in
available so far me I will be enlarged
kill ok this is more of a sad memory
procedure sad memory procedure wherein
you create a lot of company specialized
of routines and functions which does the
Jewish kind of tells the compiler to
split the job with the job of your code
ok among different processes and run
them simultaneously ok if as of now just
take it taken into consideration that
openmp is a method through which you can
make your project make your program run
ran in several processes several
processes ok now there's a small
motivation for it I just give you guys
an example yeah ok now if ever two okay
if I were to open my system monitor
available open available ubuntu okay see
I say mentioned to you guys earlier ok
let me just you know keep this together
okay as I mentioned you guys early
enough me tutorial
okay my laptop is I'm using my laptop
for all mid tutorial series or not okay
my laptop this is an intel i7 of
second-generation third generation
processor okay and it's since his i7 it
is it's a quad core processor but the
proper admin testing thing this each of
my quad each of my four course have to
Hardware threads so what does it implies
that i have 8 processing units in my
laptop okay though so each of my core
behaves as if they have their took
separate course so because of that I
have like eight course I advantage and
each of them each of these course has a
has a processing capacity of speed of
clock speed which every call it of two
point three gigahertz that's pretty
interesting now the point and trying to
make here is that you see at any instant
I have eight of my foot course running
and at the same time eight of my course
running at the same time okay the number
of processors you have at the background
in my system are these I like so many
processes running the point is what
happens is that at any instant yeah like
let's say they are like 50 process over
here at any instant any fraction bit of
a time okay only eight processors can be
run in the system at every fraction of a
second okay in my system only eight
process can be done and each of these
processes will be done by one cpu eek
one cpu okay and now what happens is
that what happens is that but since
there are only eight processes but how
how are all these managed it's simple
that's where your ram comes into the
picture what the process of processes do
with that they consider a certain job
for a very small very very small period
of time and then whole keep it on hold
take some other job take some other job
work it work on it keep it on hold take
summer job and so on and once in a while
the processes wrap the jobs are the jobs
among themselves so that nope no job
gets concentrated by a single process
sir all the time okay now whenever you
run a new program and Fortran what
happens is that you will be running only
one or you'll be using only one core to
its fullest speak okay now in parallel
programming what you can do is that you
can make your multiple course run the
same same program simultaneously when
that happens what happens is that your
time taken for your job reduces
substantially and that is a large is
that I paradigm of parallel programming
okay now either go we're going to look
at that over here and I mean if you
don't get this video so far if you don't
understand why is it go okay let me keep
this aside to give you guys a demo this
will be useful trust me okay now I'll
just walk you guys through this tutorial
you see walk you guys through this okay
yeah I guess walk you guys through this
program but touch me officially the
tutorial series is over this is just a
demo to tell let's ride out notice this
is a demo if you understand this is good
for you if you don't understand this
don't worry you don't have to worry
about and break your head or break your
head about this okay this don't worry
this and this okay now what I'm going to
do is that I use this module college om
beyond the score like lib this is
nothing but this is nothing but a module
that contains the open of the sub
proteins in Fortran gfortran okay I went
okay and then I said n to be thousand
which is a parameter okay and then I
said variable call as num threads to be
four and which is actually an integer I
but the funny thing is unlike these I
return this and with the comment the
exclamation sign for a dollar sign so
you might be thinking hey I don't
because they shouldn't this line be you
know excluded since this since you wrote
about since you wrote an exclamation
symbol soon in this line become a
comment and get excluded or ignored
actually not it does not because though
we have a exclamation symbol over here
the presence of the dollar next to it
makes this a special car
in a statement called as a preprocessor
directive kind of a statement
preprocessor directive statement so what
happens is that whenever the pre when
this process is the preprocessor
statements get activated they're never
certain flags are certain flags are used
and what what does a preprocessor I have
to do directive do it's simple it just
tells the compiler directly to do
certain jobs at the you know at the
lower levels FNA when you are creating
and when this program is executed by the
compiler it produces an assembly level
language or you know a machine level
language depending on a system you're
working with okay and what does the
these each statement has its own set of
as a machine course which gets generated
but what does these do is that these
tell the compiler to make modifications
in the assembly assembly code or the
machine code such a manner that you know
southern off certain pieces are tab then
okay that's the simplest way I can put
it okay and I create three matrices a B
and C and three indices I I JK okay t1
t2 an epr just variables for finding ofi
understanding how much time it took for
processes to they actually do the job me
no job and all okay if you guys remember
the timing timing in fortran tutorial i
will made some time ago that the i'm
going to use the same thing over here
and then you see and writing his entire
print statement mean in the front of the
Ming dollar we exclamation dollar symbol
so what happens is them and open a piece
active this statement gets executed when
OpenAPI is inactive this is not in this
one this will just be this statement
will be just ignored similarly this will
be ignored when OpenMP is inactive that
it's been an active this statement gets
printed and when at this and when OpenMP
execute active this statement also gets
done while this funk what this I on a
subroutine call OMP and the score center
and the score normal Scott threats
following beside come through so what it
is is that just tells a compiler to make
three make the number make
the number of processors are available
if num threads nom underscore threats
say define on the top it can be any
variable i just said there's num threats
but it can be num num products or
anything any variable ok this is said to
be for it has a compile to make ready
for core processors to do the job if you
said this to be one it just tells the
compiler to make only one process one
processor available to the job and if
it's 15 will take 15 depending on the
case ok and that's why I made a comment
here you said the number of process to
be used and then a matrix is said to be
a 1 matrix and 0 beat be to be in these
statements i said be to be an identity
matrix ok and see to be 0 and what I do
is I call the CPU time right here they
close the CP time I hear this loops
takes care of the matrix multiplication
the resultant resident multiplication of
a and B are stored and see I want to do
is that I is just invoke the parallel do
loop statement available for available
in OMP open being Fortran over here ok i
start the barrel doe statement and
parallel and end the parallel blocks
parallel block over here now what it
actually does is that when this block
has been invoked this entire do loop
this entire next to do loop is split
such it split accordingly among the
processes you take and the each of the
process said do a certain part of it for
example if if he a Miss a thousand then
you have like thousand iterations for
the variable I so of that being the case
the may be like if there are four
threats let's both rich maybe the first
thread might take first 120 undone to
first 250 iterations second thread my
second processor or the thread here
takes now another 250 iterations third
one test another 250 and a fourth one
that's not another 250 certificate if
you guys look at it all the processes
will take will share the lower floor of
this hundred thousand iterations over
here ok but that odd one they just stuff
over here is all of them will take the
job simultaneously finishes of
conditional job simultaneously so as a
result if the entire
in this entire loops takes like 10
seconds because of the sharing you might
get this entire thing John done in 2.5
seconds if you have like four processes
okay I'll explain you guys how this will
explain you guys I'll just show you guys
visually how it works okay and then I
call the CP time I here to CP time in
here and what does this does is that it
just it just finds a time over here and
finds a time over here the difference
between these two gives the time taken
for the father's entire matrix
multiplication block to finish this job
ok and then EP will say will give you
the time taken for the matrix
multiplication ok in seconds and then it
gets printed over here if you if you
have done this in openmp what you to do
is that you just have to divide this by
the number of threads number of threats
ok or else what happens is that it will
just show it what happen what happens is
that it will just add the time taken for
each and every processor which split the
loop in this program and add to their
time and gives the cumulated time but
which is which may be in a misleading
which is which will be very misleading
but it's not a big deal but it look it
may look a little counterintuitive and
this is part this is you know if you
want the results to be as accurate as
possible this this is not valid because
the in some place is in some cases the
threads may not be evenly distributed
with Lord sometimes may take some of the
cover or processor here might take a
little more job and come back other
something of that sort so there may be
nappin done but assuming in for this
kind of a case give me you can family
says the state level assume that you
know you might take equal Lord Lord
share it will be kind of equal so that's
it ok now with that this program is over
so what I'm doing is that i just opened
is doing a quick review i am setting the
number of matter the diamonds of the
matrix to be thousand the end to be
thousand i'm using i'm importing the
open of the subroutines here defining
the madness is a B and C is one man is
full one unity matrix with all its value
one he is an identity
at identity matrix C is said to be CSE
the zero matrix I said I call a secret
timer here to start the parallel program
to start a timer before the parallel
programming option okay and you use a
timer to end that I not the time at the
end of the parallel programming option
with East call and with the CP
underscore types of proteins okay and I
call the matrix multiplication operation
between these two and the enclosed with
enclosed within OMP parallel do which
kind of spits spits the this entire loop
we doing the processes okay and on the
top I said the number of processes to be
used okay and then I just calculate the
time taken for this entire process and
then do some arithmetic photos I just
for the threads under the first used and
then prints the same okay now to run
this what is what I've done is that what
I done is that I returned a shell script
but it's doesn't matter you can run this
from the terminal so just like in
previous tutorial I this anyway i'm
writing this in a shell script okay the
major things to be concerned i said i
will show you how this works in cereal
as well as parallel how this code can
have this code which ever written here
okay can be you see really as well as in
peril okay what you do is that when you
when you compile the code normally with
the minus c flag and then use the minus
0 flag to link execute link it linked it
I mean build the executable and run
things get up resist normally okay
you're just the you be just doing a
normal regular serial execution see the
compliation when this is done all the
openmp stuff all opera base of the such
as this line this will be taken but it
won't make it will be ignored are not a
big deal forget this line this line this
line and this statement all the
statements in this light blue color will
be ignored it will be taken as a comment
and it'll be ignored by the compiler
when this when these slides are used
okay the reason is because the open of
the flag is not present over here okay
so when you execute this this will be
just a serial executions that's why I
just named yes Ser and the score Matt
mommy
Cyril matter matrix multiplication
execution ok and now how to do how to
run this in paddle it's simple you just
have to repeat the same syntax you use
on the top weather but the vital
procedure is this compiler flag over
here ok minus F openmp if this flag is
present now this flag will tell the
compiler will tell the compiler say
think that ok I'm invoking a parallel
OpenMP modules over here specially you
specified for fortran ok take that into
consideration and run accordingly man
this is taken and this is taken into
consideration all these light blue
statements over here will be taken and
will be taken into consideration they
will not beat know they'll be treated
actually there will be treated and the
result gets run nicely ok now let's have
a compaq let's have a let's build this
up since these are the same folder that
I made us a shell file executable ok yes
execute this ok it is running in serial
the running and cereal but there is a
small issue is it says there's a
segmentation fault a lecture on what
happens ok and just to test it earlier k
just keep this in mind and low if you
guys look at this graph over here the
CPU history usage you just see a spike
over here I expect in the green color
what the spike is actually this that is
the third cpu of my computer started the
job and if you guys notice if you guess
it right ok when the photo and my photon
program can execute started the
execution ok my third cpu took the job
and did the job over here ok now I'll
explain you guys what the segmentation
fault us see sometimes what happens is
that when you run parallel programming
mechanic when apart and parallel
programming stuff or when you run when
you run mattresses we need an programs
with mattresses which whose sizes are
very very large and now lies in nature
especially when your thermos is limited
like I in her my system has a gb ram but
but that's ok
but in certain systems if you are using
supercomputers and dog what happens is
there you have some memory restrictions
like there's a limit on how much memory
you should use and stuff so to to
rectify that to understand what's going
on what I've done is that I just open a
terminal I mean I just open the terminal
directly in Jeanne okay I just clear
this up okay I just have to see I just
have to see what this Q is that you
limit then type- is and to press enter
okay if this value is not unlimited and
it gives us some finite finite value
then it shows that the amount of memory
allocated for you know intermediate
processing before the ram this little
lesson number lesson number ok now this
cause this might cause you segmentation
fault because whenever a program is
whenever a program runs segmentation
fault that will come segmentation fault
error comes when your program is trying
to your compiler is trying to access
memory for which the program for which
the memory is not allocated let's say
how like to run this program you have
like 15 okay 20 mb of data I like it it
in the memory allocated allocated in
your fam okay and then while compiling
it you are you executed you access
little more than 15 mb then as well the
compile initially allocates only 15 15
mb but do you need 20 so what happens is
that is the remaining fight is not
available so what do you get is a tries
to access memory from data from some
location who's who's which whose whose
location is not available in your part
of it in me which we tried have been
used by some other program or are now on
the region outside the region allocated
for your memory then that's what you
call a segmentation for this it is
trying to access a sec with the program
tries to access a segment which does not
have the right because of the memory
allocation okay now this this can be
avoided but what you have to avoid this
what you have to do is that is just at
the type you will limit your terminal
minus s and then type unlimited okay the
presenter and then you try the same
thing it should if it's unlimited what
happens is that just Alec it just just
make sure that the memory gets all again
other as much as it needs okay now now
this terminal has the you limit feature
set in so instead of running this
directly if I using fi it's better to
run this from here okay you suppose you
every time if we're under fire just get
the same error stating that you know
same error setting the in the compiler
setting that you know this you limit
does not set you mean segmentation
errors and stuff okay so what I do is
that is a bash and then star star dot
such as such okay run this now watch
initially you will be running the serial
part of it and then the paddle paddle
witnesses in that when you run the
serial part of it among these eight
course one of the core will take the job
and you notice a spike you'll notice a
spike in full full capacity and then
when you notice winning when the
parallel programming since the gym
starts you'll notice for spikes in the
history record over here and then to
indicate the same you have a small spike
in the ram memory over here okay look
for this pink line over here this is the
ram memory which will also have a small
plateau like spike for a short period
how look at that ok there's a reason why
I kept it and you might you not you will
appreciate this better if you have a
visual demonstration now watch trying
this in cereal ok there my ok ok now
what just happened I will explain you
guys see this red spike is inish is
because of the sealed execution ok this
red speck is because of sealed execution
this Redis cava corresponding to the CPU
eight so when the serial execution is
done cpu eight does the first job and
then as i'm going to close
the remaining remaining four one two
three four these four are because when
the fourth these are these are because
when the parallel execution starts in
our program we used for threads so it's
in working portraits okay okay let me
put this as six threats okay let me put
this as six threats and let's see what
happens so ideally when that battle the
programming this and six thirds are used
you should see six bytes in the data now
let's see how this goes okay handing in
serial we here okay now watch are you
guys done this in serial you guys under
Cynthia this is what happens you can see
almost six spikes over here this initial
blue one light blue one I mean which is
CPU for took the first job and afterward
when it foul down the other process took
in charge of it okay this FSE procedure
you can make all the eight processes run
as well right all the eight threads and
biomass wealth and watch serial running
in cereal okay parallel threads done now
if you guys look at the spikes look at
the spike now this is very much evident
initially the only one processor used
not because of this since we use a
threats you're using all the eight the
Cupra all the eight cores to do the job
all the eight closer to the job so you
see we are having now we are having
control to under two mana plate the
number of process to menu to force them
a process too you'd run your job
accordingly now this is that one that
this is a beauty of parallel programming
okay and if you guys notice the time
taken for a single loop time taken for
the entire man is a matrix
multiplication by a single processor is
like 4.8 seconds whereas many all the
eight say all the eight processors used
it does drought 21
13 for seconds ok now let's make this to
be too okay just two processors okay and
you guys notice you will see only two
spikes over here if I were to run this
again watch running in cereal okay so
what happens is that the orange one
spiked up okay the orange the CPU one
spiked up and when to threats are used
only through you see only two spikes the
red one and the origin indicating that
only two processes were used and if you
see guys notice the time taken for the
two processes when comfort the one
processor is approximately half not
exactly but approximately half so you
see that the number of processes being
used you get the size now this will be
very much evident than the number of the
read data says is when the time span
between these two will be very much
evident when you have a large number of
data sets so instead of two thousand
right in sir the maddox win2000 cross
two thousand let with thousands of
thousand let me set this in two thousand
cus 2000 this will this will them now
the matrix will be you know eight times
larger and memory kind of high enough so
this will this will take a large time
but you will notice a very large drop so
what I do is they let us take four
processes again for my consideration and
grab this run this tutor on this watch
but I run this in cereal now then that
lag will be less and you can see one
processor will shoot one processor will
shoot right now right now the job is
bringing us a swab routine processor 17
and date continuously they just take
some time and split this with the job
but still there at any instant only one
process is active either seven or eight
is active and you can see clearly right
now that processor one take took the job
and it worked out and it is working on
okay and now when four threads are used
and four threads are you
you can clearly see to spur two more
spikes over here and four threads are
being used for threads are being used
and then after the entire mean there's a
kind of a lab over here without really
getting this but ultimately when this
settles down okay then has been some
issues but if you guys noticed you see
the time difference it takes when one
processor you use it takes like thirty
six seconds there's a 24 process we use
the time touch with 13 seconds imagine
you have a very very large amount of
calculations very large amount of
calculations this is just a simple
matrix multiplication but if you just
have a very very large amount of
calculation imagine the amount of time
you can save by using parallel
programming okay here you might be mad i
have an exact divisible by four stuff
because along with this process there
are like a hundred other processes a
hundred other process running at the
background and the operating system also
is running on the stunning and stuff so
this you may not get a proper load shad
over here at the same time the the low
here the load less symmetrical before
the processes but in certain things
certain times it may not be symmetric
and stuff the process cpu may not be
available and stuff and dog so it may
not get a proper this before but you can
approach that approach that when only
certain job is done okay now to show
that there is a change in there is a
rams if you want you have to set this to
be four thousand or something this time
we'll just exponentially increase the
RAM will also load up in lot up in
memory so i didn't want to do that but i
think you get the point over here I
think and get the point over here so if
you guys are already good and Fortran or
if you are able to come all the way to
all the way to understand large amount
of water on the next interesting point
will be to invest your time on open
happy and I I strongly recommend open
and P or MPI procedures I am NOT giving
a MPI pro NPI Tom over here it's that's
a little more tricky and it's not easy
to explain in a very sharp splitter
they're not easy to explain okay it's a
I mean it's you can explain that but
it's a little tricky okay and that will
be little too far-fetched for now okay
if you guys are into computational
domain a computational Marian have to do
a large amount of kind scientific
computation scientific calculation and
high performance computing cackle a
large number of values especially few
guys from atmospheric and oceanic census
departments or quantum mechanics depart
quantum mechanics a purpose or
Astrophysical simulators somebody
somebody who submit star is submit the
flow Astrophysical fluid flows and stuff
how stars and then generated stuff of
this disorder Starly recommend you guys
learn open and PR op openmp RMP I or any
other parallel programming for
methodology available you the you will
in on find it very very useful and if
your guys are running running stuff in
supercomputers supercomputers then he
opened a beer API are the best way to go
about with them that's all i have for
you guys in this tutorial thank you guys
for watching and I'll see you guys next
tutorial and that will be a wind up
tutorial why not to tell as to what to
be done and what what the billion about
to go beyond from here in Fortran thank
you guys for watching and take care