Overview
PyTorch is a deep learning framework developed by Facebook’s AI Research lab(FAIR) about four years ago (in 2016). This PyTorch framework was designed to make our machine learning and deep learning project journey super fast and smooth.
Pytorch is written in Python, C++, and CUDA and is supported across Linux, macOS, and Windows platforms. It also has a C++ interface but the Python interface is the more polished and a primary focus of development.
PyTorch carries tons of functionalities to make our life easy-starting from the research phase of the project and ultimately deploying it in production. With PyTorch, we can write highly complex deep learning models in minutes without even complicating the project code much.
PyTorch provides two high-level features-
- GPU accelerated super fast Tensor calculations
- Flexibility to assemble Deep Neural Networks
This article throws light on the basic fundamental blocks of PyTorch. If you are already familiar with the basics, you can directly skip to my next article (click here) where I create a simple deep learning model with PyTorch from scratch. And if this is the first time you are reading about PyTorch, these basics are going to help you understand the framework in a much better way.
Quick peek into the Content
- PyTorch Installations
- Tensor
- Tensor Operations in PyTorch
- PyTorch vs NumPy
- Tensors on GPU
- Conclusion
1. PyTorch Installations
Before installation, I would recommend you to first create a python virtual environment in your machine and then install PyTorch there. I am asking this because PyTorch is a big framework and it may break a few existing dependencies in your base python environment.
PyTorch is quite easy to install if you already have a conda environment set up in your machine. You can install it using the following commands:-
1. Using Anaconda
conda install pytorch torchvision -c pytorch
2. Using Pip3
pip3 install torch torchvision
If you still face problems with the installations, kindly refer to the PyTorch documentation here.
2. Tensor
A Tensor is the fundamental building block of PyTorch. Not just PyTorch, Almost all Deep Learning toolkits utilize tensors to store data and perform various mathematical operations over them. A Tensor is just a fancy name for multi-dimensional arrays.
If you have worked with the NumPy toolkit before, you will find Tensors to be very similar to NumPy n-dimensional-arrays. These Tensors can live either on the CPU or the GPU. Let’s create a few tensors and do some calculations to understand them better.
3. Tensor Operations in PyTorch
3.1 Creating a PyTorch Tensor
Creating PyTorch tensors is super easy. Following are the three simple ways to create a PyTorch tensor in your python notebook.
1. Creating tensor by just passing the required dimensions into the torch.Tensor() API-
import torch
print (torch.__version__)
#Creating a Tensor with dimensions: 3*4
#Output is a float Tensor with garbage values
torch.FloatTensor(3,4)
The output would be a FloatTensor of given dimensions having garbage values as this function just allocates the memory and doesn’t initialize the values-
Out[1]: 1.3.1
tensor([[ 1.1632e+33, 5.6003e-02, 7.0374e+22, 5.7453e-44],
[ 0.0000e+00, -2.0000e+00, 6.8374e-25, -2.0005e+00],
[ 1.4569e-19, 6.4069e+02, 4.3066e+21, 1.1824e+22]])
2. Numpy nd-arrays can directly be converted into the PyTorch tensors with torch.Tensor() API-
import numpy
my_numpy_array = numpy.array([[1,2,3], [4,5,6], [7,8,9]])
#converts numpy array into a Tensor
torch.Tensor(my_numpy_array)
Out[1]:tensor([[1., 2., 3.],
[4., 5., 6.],
[7., 8., 9.]])3. Similarly, python lists can also be converted into tensors directly-
my_python_list = [[1,2,3], [4,5,6]]
torch.Tensor(my_python_list)
Out[1]:tensor([[1., 2., 3.],
[4., 5., 6.]])3.2 Simple mathematical operations on PyTorch Tensors
PyTorch tensors support a bunch of mathematical operations, almost every operation you can think of. A list of all the supported operations can be found in Pytorch Documentation.
Here are a few examples of simple operations-
tensor.sum()operation sums up all the elements into a scalar tensor
a = torch.FloatTensor([1,2,3,4])
b = a.sum()
b
Out[1]: tensor(10.)2. Element-wise addition of a number
a = torch.FloatTensor([1,2,3,4])
b = a + 5
b
Out[1]: tensor([6., 7., 8., 9.])3. Element-wise addition of two tensors
a = torch.FloatTensor([1,2,3,4])
b = torch.FloatTensor([3,2,1,5])
c = a + b
c
Out[1]: tensor([4., 4., 4., 9.])For a quick overview of basic PyTorch commands, you can refer to the PyTorch Cheatsheet provided by PyTorch developers.
4. PyTorch vs NumPy
You might be wondering, everything I claim a Tensor can do is also possible through NumPy toolkit then why don’t we write deep learning models straight in NumPy?
Well, NumPy is a huge framework that supports lots of generic scientific operations over n-dimensional arrays. You can actually write deep learning models in NumPy. The only thing wrong with NumPy is that-it does not support GPU based computations for faster results. So you won’t get 10~50x GPU speed-ups if you write your deep learning models using PyTorch.
Modern Deep Learning experiments involve complex model creation and thus require a huge amount of computing power. It makes perfect sense to train these models on GPUs. So, there existed a need for a deep learning framework that makes writing deep learning easy and supports GPU based computations.
PyTorch brings all this onto the table. PyTorch tensors support GPUs for fast numerical computations. Most of the common deep learning concepts are already implemented within PyTorch and can directly be imported and used using simple commands.
PyTorch functionalities are not limited to just model development, It also comes up with various API’s to help you take your deep learning projects into the production.
5. Tensors on GPU
PyTorch transparently supports CUDA GPUs. All PyTorch tensor operations have two different versions-CPU version and GPU version. All the tensor operations we wrote above were CPU-based and have their GPU equivalents.
GPU tensors are written within the torch.cuda package while normal CPU tensors are written in torch package. We can convert all those CPU tensors into GPU tensors by just replacing torch with torch.cuda. For example- to create a FloatTensor on GPU simply call torch.cuda.FloatTensor() instead of torch.FloatTensor().
There is also a tensor operation- to(device) that is used to transfer the tensors across the devices. Using to(device) operation, we can copy tensors from one device type to another device type. Device types could be-cpu or cuda(for GPU tensors).
Examples of GPU tensors-
#creates default cpu tensor
a = torch.FloatTensor([1,2,3,4])
print ("Default CPU tensor: ", a)
#transfer tensor to another device
a = a.to("cuda")
print ("Converted to GPU tensor: ", a)
#checking device of a given tensor
a.device
Default CPU tensor: tensor([1., 2., 3., 4.])
Converted to GPU tensor: tensor([1., 2., 3., 4.], device='cuda:0')
Out[1]: device(type='cuda', index=0)
6. Conclusion
In this article, we examined PyTorch-A deep learning framework. We discussed why we needed something like PyTorch for our deep learning projects. We went through the basics of PyTorch tensors and a few numerical operations supported by them.
After a basic understanding of PyTorch, you might be interested in reading my next article where I talk about writing a deep learning model from scratch using the PyTorch framework.
Link to my next article on PyTorch: Click here.
*To get notifications regarding my brand new articles, kindly register with your email address.
Click here to read my introductory article on Reinforcement Learning.
Thanks for reading this article. Kindly share your feedback through comments.
References
- https://www.pytorch.org (PyTorch official website)
- https://github.com/pytorch/pytorch (PyTorch Github page)
- https://en.wikipedia.org/wiki/PyTorch (Wikipedia Link)