Libraries and Frameworks
In this chapter, we will relate deep learning to the different libraries and frameworks.
Deep learning and Theano
If we want to start coding a deep neural network, it is better we have an idea how different frameworks like Theano, TensorFlow, Keras, PyTorch etc work.
Theano is python library which provides a set of functions for building deep nets that train quickly on our machine.
Theano was developed at the University of Montreal, Canada under the leadership of Yoshua Bengio a deep net pioneer.
Theano lets us define and evaluate mathematical expressions with vectors and matrices which are rectangular arrays of numbers.
Technically speaking, both neural nets and input data can be represented as matrices and all standard net operations can be redefined as matrix operations. This is important since computers can carry out matrix operations very quickly.
We can process multiple matrix values in parallel and if we build a neural net with this underlying structure, we can use a single machine with a GPU to train enormous nets in a reasonable time window.
However if we use Theano, we have to build the deep net from ground up. The library does not provide complete functionality for creating a specific type of deep net.
Instead, we have to code every aspect of the deep net like the model, the layers, the activation, the training method and any special methods to stop overfitting.
The good news however is that Theano allows the building our implementation over a top of vectorized functions providing us with a highly optimized solution.
There are many other libraries that extend the functionality of Theano. TensorFlow and Keras can be used with Theano as backend.
Deep Learning with TensorFlow
Googles TensorFlow is a python library. This library is a great choice for building commercial grade deep learning applications.
TensorFlow grew out of another library DistBelief V2 that was a part of Google Brain Project. This library aims to extend the portability of machine learning so that research models could be applied to commercial-grade applications.
Much like the Theano library, TensorFlow is based on computational graphs where a node represents persistent data or math operation and edges represent the flow of data between nodes, which is a multidimensional array or tensor; hence the name TensorFlow
The output from an operation or a set of operations is fed as input into the next.
Even though TensorFlow was designed for neural networks, it works well for other nets where computation can be modelled as data flow graph.
TensorFlow also uses several features from Theano such as common and sub-expression elimination, auto differentiation, shared and symbolic variables.
Different types of deep nets can be built using TensorFlow like convolutional nets, Autoencoders, RNTN, RNN, RBM, DBM/MLP and so on.
However, there is no support for hyper parameter configuration in TensorFlow.For this functionality, we can use Keras.
Deep Learning and Keras
Keras is a powerful easy-to-use Python library for developing and evaluating deep learning models.
It has a minimalist design that allows us to build a net layer by layer; train it, and run it.
It wraps the efficient numerical computation libraries Theano and TensorFlow and allows us to define and train neural network models in a few short lines of code.
It is a high-level neural network API, helping to make wide use of deep learning and artificial intelligence. It runs on top of a number of lower-level libraries including TensorFlow, Theano,and so on. Keras code is portable; we can implement a neural network in Keras using Theano or TensorFlow as a back ended without any changes in code.
