Hands-On Transfer Learning with Python Implement Advanced Deep Learning and Neural Network Models Using TensorFlow and Keras
The purpose of this book is two-fold, we focus on detailed coverage of deep learning and transfer learning, comparing and contrasting the two with easy-to-follow concepts and examples. The second area of focus is on real-world examples and research problems using TensorFlow, Keras and Python ecosyst...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | eBook |
| Language: | English |
| Published: |
Birmingham
Packt Publishing, Limited
2018
Packt Publishing Limited Packt Publishing |
| Edition: | 1 |
| Subjects: | |
| ISBN: | 9781788831307, 1788831306 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Table of Contents:
- Gradient-based optimization -- The Jacobian and Hessian matrices -- Chain rule of derivatives -- Stochastic Gradient Descent -- Non-linear neural units -- Learning a simple non-linear unit - logistic unit -- Loss functions -- Data representations -- Tensor examples -- Tensor operations -- Multilayered neural networks -- Backprop - training deep neural networks -- Challenges in neural network learning -- Ill-conditioning -- Local minima and saddle points -- Cliffs and exploding gradients -- Initialization - bad correspondence between the local and global structure of the objective -- Inexact gradients -- Initialization of model parameters -- Initialization heuristics -- Improvements of SGD -- The momentum method -- Nesterov momentum -- Adaptive learning rate - separate for each connection -- AdaGrad -- RMSprop -- Adam -- Overfitting and underfitting in neural networks -- Model capacity -- How to avoid overfitting - regularization -- Weight-sharing -- Weight-decay -- Early stopping -- Dropout -- Batch normalization -- Do we need more data? -- Hyperparameters of the neural network -- Automatic hyperparameter tuning -- Grid search -- Summary -- Chapter 3: Understanding Deep Learning Architectures -- Neural network architecture -- Why different architectures are needed -- Various architectures -- MLPs and deep neural networks -- Autoencoder neural networks -- Variational autoencoders -- Generative Adversarial Networks -- Text-to-image synthesis using the GAN architecture -- CNNs -- The convolution operator -- Stride and padding mode in convolution -- The convolution layer -- LeNet architecture -- AlexNet -- ZFNet -- GoogLeNet (inception network) -- VGG -- Residual Neural Networks -- Capsule networks -- Recurrent neural networks -- LSTMs -- Stacked LSTMs -- Encoder-decoder - Neural Machine Translation -- Gated Recurrent Units -- Memory Neural Networks
- Chapter 7: Text Document Categorization -- Text categorization -- Traditional text categorization -- Shortcomings of BoW models -- Benchmark datasets -- Word representations -- Word2vec model -- Word2vec using gensim -- GloVe model -- CNN document model -- Building a review sentiment classifier -- What has embedding changed most? -- Transfer learning - application to the IMDB dataset -- Training on the full IMDB dataset with Word2vec embeddings -- Creating document summaries with CNN model -- Multiclass classification with the CNN model -- Visualizing document embeddings -- Summary -- Chapter 8: Audio Event Identification and Classification -- Understanding audio event classification -- Formulating our real-world problem -- Exploratory analysis of audio events -- Feature engineering and representation of audio events -- Audio event classification with transfer learning -- Building datasets from base features -- Transfer learning for feature extraction -- Building the classification model -- Evaluating the classifier performance -- Building a deep learning audio event identifier -- Summary -- Chapter 9: DeepDream -- Introduction -- Algorithmic pareidolia in computer vision -- Visualizing feature maps -- DeepDream -- Examples -- Summary -- Chapter 10: Style Transfer -- Understanding neural style transfer -- Image preprocessing methodology -- Building loss functions -- Content loss -- Style loss -- Total variation loss -- Overall loss function -- Constructing a custom optimizer -- Style transfer in action -- Summary -- Chapter 11: Automated Image Caption Generator -- Understanding image captioning -- Formulating our objective -- Understanding the data -- Approach to automated image captioning -- Conceptual approach -- Practical hands-on approach -- Image feature extractor - DCNN model with transfer learning
- MemN2Ns -- Neural Turing Machine -- Selective attention -- Read operation -- Write operation -- The attention-based neural network model -- Summary -- Chapter 4: Transfer Learning Fundamentals -- Introduction to transfer learning -- Advantages of transfer learning -- Transfer learning strategies -- Transfer learning and deep learning -- Transfer learning methodologies -- Feature-extraction -- Fine-tuning -- Pretrained models -- Applications -- Deep transfer learning types -- Domain adaptation -- Domain confusion -- Multitask learning -- One-shot learning -- Zero-shot learning -- Challenges of transfer learning -- Negative transfer -- Transfer bounds -- Summary -- Chapter 5: Unleashing the Power of Transfer Learning -- The need for transfer learning -- Formulating our real-world problem -- Building our dataset -- Formulating our approach -- Building CNN models from scratch -- Basic CNN model -- CNN model with regularization -- CNN model with image augmentation -- Leveraging transfer learning with pretrained CNN models -- Understanding the VGG-16 model -- Pretrained CNN model as a feature extractor -- Pretrained CNN model as a feature extractor with image augmentation -- Pretrained CNN model with fine-tuning and image augmentation -- Evaluating our deep learning models -- Model predictions on a sample test image -- Visualizing what a CNN model perceives -- Evaluation model performance on test data -- Summary -- Chapter 6: Image Recognition and Classification -- Deep learning-based image classification -- Benchmarking datasets -- State-of-the-art deep image classification models -- Image classification and transfer learning -- CIFAR-10 -- Building an image classifier -- Transferring knowledge -- Dog Breed Identification dataset -- Exploratory analysis -- Data preparation -- Dog classifier using transfer learning -- Summary
- Cover -- Title Page -- Copyright and Credits -- Dedication -- Packt Upsell -- Foreword -- Contributors -- Table of Contents -- Preface -- Chapter 1: Machine Learning Fundamentals -- Why ML? -- Formal definition -- Shallow and deep learning -- ML techniques -- Supervised learning -- Classification -- Regression -- Unsupervised learning -- Clustering -- Dimensionality reduction -- Association rule mining -- Anomaly detection -- CRISP-DM -- Business understanding -- Data understanding -- Data preparation -- Modeling -- Evaluation -- Deployment -- Standard ML workflow -- Data retrieval -- Data preparation -- Exploratory data analysis -- Data processing and wrangling -- Feature engineering and extraction -- Feature scaling and selection -- Modeling -- Model evaluation and tuning -- Model evaluation -- Bias variance trade-off -- Bias -- Variance -- Trade-off -- Underfitting -- Overfitting -- Generalization -- Model tuning -- Deployment and monitoring -- Exploratory data analysis -- Feature extraction and engineering -- Feature engineering strategies -- Working with numerical data -- Working with categorical data -- Working with image data -- Deep learning based automated feature extraction -- Working with text data -- Text preprocessing -- Feature engineering -- Feature selection -- Summary -- Chapter 2: Deep Learning Essentials -- What is deep learning? -- Deep learning frameworks -- Setting up a cloud-based deep learning environment with GPU support -- Choosing a cloud provider -- Setting up your virtual server -- Configuring your virtual server -- Installing and updating deep learning dependencies -- Accessing your deep learning cloud environment -- Validating GPU-enablement on your deep learning environment -- Setting up a robust, on-premise deep learning environment with GPU support -- Neural network basics -- A simple linear neuron
- Text caption generator - sequence-based language model with LSTM -- Encoder-decoder model -- Image feature extraction with transfer learning -- Building a vocabulary for our captions -- Building an image caption dataset generator -- Building our image language encoder-decoder deep learning model -- Training our image captioning deep learning model -- Evaluating our image captioning deep learning model -- Loading up data and models -- Understanding greedy and beam search -- Implementing a beam search-based caption generator -- Understanding and implementing BLEU scoring -- Evaluating model performance on test data -- Automated image captioning in action! -- Captioning sample images from outdoor scenes -- Captioning sample images from popular sports -- Future scope for improvement -- Summary -- Chapter 12: Image Colorization -- Problem statement -- Color images -- Color theory -- Color models and color spaces -- RGB -- YUV -- LAB -- Problem statement revisited -- Building a coloring deep neural network -- Preprocessing -- Standardization -- Loss function -- Encoder -- Transfer learning - feature extraction -- Fusion layer -- Decoder -- Postprocessing -- Training and results -- Challenges -- Further improvements -- Summary -- Other Books You May Enjoy -- Index
- Hands-On Transfer Learning with Python: Implement advanced deep learning and neural network models using TensorFlow and Keras