Practical 1.1 – Torch Video Lecture - Torch A Practical Hands On Tutorial

in this practical we will see an
overview on torch tensor library torch
is an easy to use an efficient
scientific computing framework which
leveraged legit just-in-time compiler
and an underlying C and CUDA
implementation which make every routine
very and very fast we provide an N
dimensional array data structure an
amazing interface to see we are legit
and FFI the foreign function interface
it provides also a library for neural
networks and energy base models it has a
fast and efficient GPU support and
finally it can be easily embeddable and
pores exist already for iOS Android an
FPGA begins
the torch goals are maximum flexibility
obtaining the highest speed but with
extreme simplicity let's see now an
overview on the following tutorial we
start with a generic head and then a
specific head for an item of the torch
library the torch type command which
tells us what kind of element we are
dealing with the torch tensor which is
the main fundamental part of the torch
library the pound operator the
dimensionality and the size of a given
dimension the apply function which
allows us to define a closure on the
element of a given tensor the tensor
types by character shorts and knives and
the torch set default tensor type which
is pretty useful when we would like to
change the default type of the tensor
used by George moraga we'll see what the
tensor and a storage are we'll get
familiar with the resize function will
also encounter a tensor type mismatch
error and how to fix it we'll also learn
the difference between the assignment
and the clone operator we will then
start having a look to one-dimensional
tensors or vectors and the star
multiplication operation then we'll go
and see the matrices which are the two
dimensional tensor and also here we see
what this star operator we do robber is
corner cases row and column vectors are
still two-dimensional tensors where
DeRose dimensionality or the column
dimensionality it's equal to one then we
will learn about how to slice these
tensors with the square bracket curly
bracket operator moreover we will learn
a
some of the constructors of the dancers
like torch range linspace logarithmic
space zeros ones the I matrix uniform
distribution with Rand and the normal
distribution with Rand n moreover we
learn about casting different type of
tensors and finally we'll have a look
about how to visualize these tensors
with new blood and new plot plot and
histogram moreover we see how to
multiply element wise two different
answers how to apply a transposition of
a matrix or a generic tensor with the
transpose operator finally we see how to
concatenate multiple two inserts and we
will see the distinction between the +
and star operator over the your
correspondent add and multiply finally
we'll see what resize with shape and
view are although they perform a very
similar operation they are very distinct
and it's very important to keep in mind
their differences in the previous
tutorial we have learned about row
syntax in this tutorial we see some of
the torch syntax specifically you can
find a write-up at the following address
let's run our torch tripper
by typing th we can see that pressing
the question mark can provide us some
help to get start with th is an enhance
interpreter rebel read evaluate print
loop for torch 7 new one many features
it has tab completion on nested
namespaces it can also type complete
disk file names when we open strings if
we don't want the system to print the
result we can stop it by typing
semicolon at the end of the instruction
if we would like to know some specific
help for a given function
for example torch Rand normal
distribution we can type a question mark
followed by the name of the unknown
object we can run shell commands by
typing a dollar symbol and then we can
escape to the UNIX command line as we
saw in the last video we can also use
the function foo to list all the global
variables present in the current scope
we can clear the screen with control L
we can quit torch with control D so
control L let's try with the help
function so we can type a question mark
then torch tab to complete sq our tab to
complete I remove the open parenthesis
and press Enter
and here we have a summary of what the
function s qrt of torch does if you
would like to know the type of a
specific element within the interface we
can type torch tab to complete again dot
type and then for example I can put a
string hi the output will be a string if
I put a number I press up to recall the
previous command I will have number if I
put torch itself it's a table as we saw
in do one everything it's quite a table
object our table dictionaries and lists
are all tables torch is a
multi-dimensional tensor library and
provides mathematical operator over
these additionally it provides many
utilities for accessing files
serializing objects of arbitrary types
and other useful functions let's start
to see how to create a tensor control l
to clear the screen let's define T equal
torch dot
answer of dimension 2 by 3 by 4 this
tensor can be thought as 2 matrices
overlap of 3 rows and 4 columns if you
would like to query the dimensionality
of this tensor we can type hash T we see
we have two channels three rows and four
columns if we press T and enter we can
see that these numbers are not
initialized let's try to initialize with
a sequence of numbers from 1 to 2 times
3 times 4 24 so let's define I is equal
to 0 and then we take our tensor T we
apply a closure which so last time how
to do so so function of variable X which
is going to be the current value passed
from the tensor which actually we are
not going to use so we can even remove
this one we can set now I equal I plus 1
we can return I and we can end our
function as we saw since we didn't put
the semicolon at the end of the
instruction the torch rapport print out
the result and we have now I initialized
tensor with numbers going from 1 to 24
we can check now the type of this tensor
so we can type torch dot type we send
key we can see it's a double tensor
overall there are these these different
tensors containers we can have white
tensor character tensor which are 8-bit
tensors then we have short tensor
integer tensor long float and double if
you don't specify otherwise double
tensor will be used by default we will
usually set the default tensor type to
float the tensor
in this way we can speed up computations
moreover in deep learning double
precision is usually not required to do
so we can pipe torch set default tensor
type I didn't type everything I just
type set D and tab then we can write
torch float tensor now if I write torch
type of porch tensor one two three we
will see the default tensor type has
been changed to float tensor if it would
like to create a specific type of tensor
let's say a byte tensor we can still
type fully torch dot type tensor and
then specify our tensor so back to our
tensor T a tensor it's simply a view on
a specific underlying storage which can
be thought as a an array on the sea side
if you would like to create another
tensor are equal port tensor on T and
then with the put column resize is in
three eight so now we had a problem it
didn't run nicely because our default
tensor type it's a float whereas torch
type T it's actually a double
so let's write the same example by
specifying here here double tensor so
our now it's the tensor with three rows
and eight columns whereas T
was two planes of three row and four
column matrices both R and T are using
the same storage this means that if we
do our zero we haven't only erase the
content of our but is will the content
of T's if we assign s equal T this is
simply an assignment of the reference of
the tensor T so if we check the
dimensionality of T we have two three
four and the same way dimensionality of
s is going to be two three four if we do
s :
the size of four times six if we check
again the dimensionality of T we see
that is four and six so the assignment
between tensors are simply a copy of the
reference of the tensor it's not a deep
copy of the whole time so therefore if
you would like to make a new tensor view
which we would like to have the same
content of T but a complete new storage
we have to issue : clone which will
perform a deep clone of the tensor T now
you it's going to be the four times six
matrix if we fill you with with random
values we can check that T has been
unchanged let's see now vectors which
are specific kind of tensor which means
we are Hmong dimensional tensor control
able to clear the screen so let's have
our G vector torque tensor and one two
three and four if we print V now we can
see it is the vector 1 2 3 4 the
dimensionality of the
is for if you would like to have exactly
the number for is output of this query
we would type the size of first
dimension which is four if we have
another vector W equal torch ones of
four elements we can check we can check
that is a vector of four elements we can
perform a scalar product simply by
typing V times W which is 10 which is
basically 1 plus 2 Plus 3 plus 4 let's
create a now a new vector let's say X
equal torch tensor two five seven one
nine or we can check that X has one
dimension of 6 X dimensionality is one
it's a one dimensional tensor and the
size of that dimension it's six if you
would like to access for example the
third element of X we can do so as it
would be a simple table so we can type X
square bracket 3 equals 7 in a similar
way we can access the second last
element of the array by type in X minus
2 which is the second counting from the
last element which is the number 9 as
you can see from here if you would like
to extract for example the sub array
from 5 to 1 we can do so by typing X
square bracket curly bracket space curly
bracket from the second element to D we
said 2 3 4 fourth element close curly
bracket
space and close square bracket and here
we have the sub vector five seven one of
size three stay the float tensor let's
go back to our vector G which was
containing the numbers from one to four
we could have also created V by simply
typing the equal torch range from one to
four and then we have again V we can
compute the square of all these values
and replace them into V by typing V :
power of two and now V if we type is
going to be 1 4 9 16 if you would like
to preserve the values of P we can write
so let's redefine V as the range from 1
to 4 and we can write W equal Torche
dot power of t + 2 in this way V is
still the vector from 1 to 4 and W is
the squared vector in this way we
created a new tensor in a new storage
and more memory is utilized if we no
longer need be after this operation it's
advice to perform the operation on the
same tensor especially if it especially
if it's associated with a very big
storage we can now create a matrix M
equal or tensor curly bracket space
curly bracket 9 6 3 4 close curly
bracket comma I go on the new line just
because I like it so it's not required
second row 7 2 8 1 close curly bracket
curls could close curly bracket so now
we have M is the torch float tensor of
size two times four so if we check the
dimensionality of em it's two meaning
there are two dimensions the size of the
first dimension its tone there are just
two rows and if I check the size of the
second dimension we have four meaning
there are four columns in our matrix we
can check a summary of all sizes by
typing hash and where we can see that we
have two rows four columns and overall
we have two dimensions we can now access
the element on the second row and third
column by doing so we out to the number
eight we can do the same by typing M
square bracket curly bracket space two
and three so let's type M so we have it
under our eyes we can extract for
example the first column by typing m
square bracket curly bracket here we
will say all rows and just the first
column in this way we have a column
vector which is different from the
vectors we saw before because in this
case it has two dimensions has two rows
in one column if we would have typed M
square bracket curly bracket both rows
and then simply the column number one in
this case we would have struct it the
vector so the mono dimensional tensor
corresponding to the first column of the
matrix in a similar way we can extract
the second row of the matrix M by typing
m square bracket curly bracket we can
say second row and all columns
and we have seven two eight one which is
a row vector again it's a two
dimensional tensor it's not a
one-dimensional vector if you would like
to extract a single dimensional tensor
we can do so by typing m square bracket
curly bracket we say second row without
the curly bracket and then we say all
columns in this way we have extracted
again this singular dimensional tensor
which components are equal to the
components of the second row of our
matrix M let's clear the screen so we
can serve the our matrix M and then a
vector that is equal to porch tensor 1 2
3 4 so 1 2 3 4 so if we do M times D
it's going to be 46 and 39 where 46 is
the result of the first row so M first
row all columns of M multiplied by P 46
and 39 is the result of the second row
all columns of M times V 39 control L
again for clearing the screen we can see
now some basic constructors for tensors
so the first one we can see is going to
be torch range and from 3 to 8 and
simply is the tensor of size 6 which is
going from 3 to 8 if you would like to
set a different step for example from
plus three to minus 4.2 which steps of
minus 1.9 we can do so by typing torch
range 3 2 minus 4 point so with a step
of minus 1.9
and there we go in order to create a
linearly spaced answer we can type torch
don't win space for example from three
to eight like the first whistle but
let's say with fifty different values
and here it is it's too long to be seen
in the Honda full screen but we can see
it's a tensor of size 15 which goes from
three to eight we can visualize these
things so we just created by requiring
new plot and then you plot dot plot and
we just let's copy we can check the
result
and we have our linearly spaced list of
values going from three to eight with
eighty values we can try something nice
now new plot which is pronounced like
new but it was already taken so they
have people to know which is not the GNU
project so it's it's not pronounced Gino
but new new plot plot torch block space
again from three to eight and we'll
still with fifty points and let's see
how this differs and thirties which goes
from ten to the power of 3 to 10 to the
power of eight if you would like to have
the same range that we got with the
linspace we can type new plot plot torch
log space math log
10:03 math log10 of a these actions are
from the math library of flew up and
still 55 and more and then if you check
now is going to be exactly from 3 to 8
with a logarithmic trend another way to
construct a tensor it's by using the
zeros function for example we can do
Torche dot eros and 0 by 5 columns and
there it is all for example we can do
the same with the ones
Torche ones we can have for example now
3 by 2 by 5 and then we have three
planes of two by five matrices we can
also construct the torch I and let's put
three and we create the 3 by 3 identity
matrix let's make a random tensor new
plot hist of torch random normal of 1000
values and here we have the
representation on the chart and if we
check here we have the representation in
the plot if we increase the number of
samples we will have a more smooth graph
let's put 1 million and here we have the
classical Delta functions
in a similar way we can use a uniform
distribution from zero to one
I simply go up instead of having round
round normal can just type round and
here we have the uniform distribution
going from zero to one just for exercise
let's check the definition of these
functions so question mark torch run the
end and here we have that y equal porch
door and normal returns a
one-dimensional tensor of size n feet
with random numbers from a normal
distribution with mean 0 and variance 1
whereas if we type question mark George
and enter we have that y equal towards
rent of n returns a one-dimensional
tensor of size n filled with random
numbers from a uniform distribution on
the interval from zero included and one
excluded let's see now how we can cast
different kind of time source let's
clear the screen so let's start with our
matrix so two dimensional tensor and
then two which is a float tensor if you
would like to have a double so let's
call it DM you can type M :
double so if we do Torche type d n it's
going to be a double tensor torch type
plus M is still the float tensor of
before we can also do a torch type of M
:
bite which is a byte tensor and if we
display
we see exactly the same because those
numbers are represented is byte we can
also do torch I M cast in little int and
so on with all the types we saw before
which where you should remember this one
before we have seen how to multiply a
matrix times another vector or another
matrix for example if we have M
multiplied by the matrix George
Rand four times six we are going to have
the matrix of two rows and six columns
let's say we'd like now to multiply
every value of M times around the number
from zero to one we can do so by typing
M :
C multiply and then we had torch brand
of the same size so two rows and four
columns and I forgot one close
parenthesis now we have that every value
that we saw before is being multiplied
by a number between 0 and 1 and these
are the outputs the values of M have
been replaced with these new values
because we used the colon operator here
if you would like to perform a
transposition on the matrix M we can
type M column T and then we have the
matrix transpose if you would like to
transpose tensors of multiple dimensions
we can use the transpose function for
example let's have T equal Torche don't
range of from 1 to 24
resize as in three times four times
though so T is three planes of four
rolls to column matrices we can
transpose let's say the dimension one
which is three with the dimension of the
columns which is two by doing this
t : transpose and let's transpose the
first one with the third one and now we
have that in the first plane we had one
and two across the columns and now which
was the last dimension and now one in
two would be across the plains so we
have one and two then three was on the
second row so we go here and row has
been unchanged so three and four has
been changed from columns to planes so
we go three and four here in the other
plane again five is going to be on the
third row and here we can see it stayed
on the third row and six is going to be
on the other plane on different planes
we have 1 9 and 17 and here we can see
that 1 9 17 are now across the columns
instead of the planes let's say now we
would have simply wanted to change the
shape of the tensor so let's start again
with our tensor T and we can perform a
resize as in 2 4 3 now we can see that
the numbers are simply reshaped so we go
from 3 planes to 2 planes and the number
of columns go from 2 columns to 3
columns and numbers are simply
redistributed in this new shape control
elegant to clear the screen let's see
now how we can concatenate different
answers let's say we have a tensor a
which is a row vector so tensor 1 2 3 4
and then we have B equals George tensor
5 6 7 and 8 so we have a this one and
these design so if you would like to
stack those two guys one above each
other we can do this with torch can't we
say tensor a and servi and we won't like
to put this in the we would like to
stack these in the first dimension so a
is going to be the first row V is going
to be the second row and here this
instead if you would like to concatenate
these two row vector in a eighth element
row vector we can do simply this by
typing cut and we say a and B
concatenate them along the columns and
here we have the final output control L
again to clear the screen finally we can
see how to perform operations between
tensors and scalars let's start with a
matrix M equal Torche
tensor of three rows and five columns if
we run the numbers from 1 to 10 so M is
going to be this matrix we can do for
example M times 2 and we are going to
have the output the output is going to
be every number multiplied by 2 we can
do M plus 1 and down the outputs can be
as we can see
4 plus 1 5 plus 1 6 plus 1 we can do
also M divided by a number so let's put
divided by minus 1 so all numbers are
turn negative we can subtract 2 for
example if we try to square the matrix
we are going to have an error like we
can see here if you would like to rise
every element of M to the power of 2 so
M was this one we have to do right
Torche power of the tensor M power of 2
and now every number its squared but M
is the same again if M is not necessary
after this computation it's better to
use the operon on itself so M :
how number 2 so this way the same
storage is utilized and we do not waste
space the tensor documentation can be
found at this link here whereas the API
for the mathematical operations can be
found here
moreover this tutorial has been inspired
by this torch for MATLAB user conversion
table finally the torch website is the
following
in the last chapter of our tutorial we
will see the difference between resize
with shape and view let's start with a
tensor a equal torch range 112 and here
we have a let's have P equal to a re
shape as in 3/4 and this is B then we
can have C equal a
whew four and three and if I print now a
a is still the same and the last
function is going to be a three sighs
112 so if we print a we have the decides
has been changed so resize does change
the size of the given tensor if I have
now a first row the first six element
multiplied by two for example if I print
a we are going to have that the first
six element have been multiplied by two
and if I print C which was the view of a
we have that the first two rows in this
case in the first six elements are also
the double of their previous values if I
print B instead be still the same tensor
B is a complete new tensor with a new
storage where as a and C share the same
underlying storage if we feel B with
random values from 1 to 12 for example
and we print again a and C we see that
the values of a and C have been
unchanged this is because agency share
the same storage which is different from
the storage of B a and C are simply two
different views of the same storage