PyTorch in 5 Minutes Video Lecture - Pytorch A Complete Guide - AI & ML

Hello world
It's Siraj and let's learn about a popular new deep learning framework called pytorch
The name is inspired by the popular Torch deep learning framework which was written in the Lua programming
Language learning Lua is a big barrier to entry if you're just starting to learn deep learning and it doesn't offer the modularity
necessary to interface with other libraries like a more accessible language would so a couple of AI
Researchers who were inspired by torch's programming style decided to implement it in python calling it pyTorch
They also added a few other really cool features to the mix and we'll talk about the two main ones
the first key feature of Py - torch is imperative programming an imperative program performs computation as
You typed it most python code is imperative in this numpy example
We write four lines of code [to] ultimately compute the value for d when the program executes C equals [V] times a it runs the
actual computation than in there, just like you told it to in contrast in a symbolic program there is a clear separation between
Defining the computation graph and compiling it if we were to rewrite the same code symbolically then when C equals e times a is
Executed no computation occurs at that line instead these operations generate a computation or symbolic graph
and then we can convert the graph into a function that can be called be at the compile step so
Computation happens as the last step in the code Ode styles have their trade-offs
Symbolic programs are more efficient since you can safely reuse the memory [of] your values for in-place computation
Tensorflow is made to use symbolic program
Imperative programs are more flexible since [python] is most suited for them so you can use native python features like
printing out values in the middle of computation and
Injecting loops into the computation flow itself the second key feature of Pi [fork] is dynamic computation
Graphing as opposed to static computation graphing in other words [Pi] [park] is defined by Run
So at runtime the system generates the graph structure
Tensorflow is Define and run where we define conditions and iterations in the graph structure
It's like writing the whole program before running it so the degree of freedom is limited so in
We defined the computation graph once then we can execute that same graph many times
The great thing about this is that we can optimize the graph at the start let's say in our model
We want to use some kind of strategy for distributing the graph across multiple machines this kind of computationally expensive
Optimization can be reduced by reusing the same graph static graphs work well for Neural networks [that] are fixed size like feed-forward
networks or
Convolutional networks but for a lot of use cases it would be useful if the graph structure could change
Depending on the input Data like when using recurrent neural networks in this snippet
We're using penter flow to unroll a recurrent network unit over [ward] vectors to do this
We'll need to use a special tensorflow function called while loop we have to use special nodes to represent primitives like loops and conditionals
Because any control flow statements will run only once when the graph is built
But a cleaner way to do this is to use dynamic graphs instead where the computation graph is built and rebuilt as necessary at runtime
The code is more
[straightforward] since we can use standard [four] and if statements any time the amount of work that [needs] to [be] done is variable
Dynamic graphs are useful
using Dynamic graphs makes debugging
really easy since a
Specific line in our Written code is what fails as opposed to something deep under section dot run let's build a simple two layer neural
Network in Pi ports to get a feel [for] this impact we start by importing our framework as well as the auto grab package which
Will let our network automatically implement back-propagation
Then we'll define our batch size input dimension
Hidden dimension and output dimension well then use those values to help define tensors to hold inputs and outputs wrapping them in
Variables well set requires gradients to false since we don't need to compute gradients with respect to these variables during back propagation
The next set of variables will define our our weights
We'll initialize them as variables as well storing random tensors with the float data type and since we do want to compute gradients with respect
To these variables we'll set the flag to true
We'll define a learning rate then we can begin our training loop for 500 iterations during the forward pass
we can compute the predicted label using operations on our variables [MM] stands for Matrix multiply and
clamp
clamps all the elements in the input range into a range between min and Max
Once we've Matrix multiplied for both sets of weights to compute our prediction
we can calculate the difference between them and
Square the sum [of] all the squared errors a popular loss function before we perform back propagation
We need to manually zero the gradients for both sets of weights since the [great] buffers have to be manually reset before Fresh grades are
calculated
Then we can run back propagation by simply calling the backward function on our loss it will compute the gradient of our loss with respect
To all variables we set requires gradient to true for and then we can update our ways using gradient [descent] and our outputs look great
Pretty dope to sum up high [park] offers [two] really useful features dynamic computation graphs [an] imperative programming dynamic
computation graphs are built and rebuilt as necessary at runtime and imperative programs perform computation as
you run them there is no distinction between
Defining the computation graph and compiling right now tensorflow has the best documentation on the web for a machine learning library
So it's still the best way for beginners to start learning
And it's best suited for production use since it was built with distributed computing in mind but for researchers
It seems like pi torch has a clear advantage here a lot of cool new ideas will benefit and rely on the use of dynamic
Grasp please subscribe for more programming videos and for now, I've got to torch my hair so thanks for watching