Create your own wiki
AI-generated instantly
Updates automatically
Solo and team plans

keras

Language
Python
Created
03/28/2015
Last updated
12/28/2023
License
Apache License 2.0
autowiki
Revision
0
Software Version
0.0.4Basic
Generated from
Commit
8e897f
Generated on
12/28/2023

Keras is a high-level deep learning API that provides building blocks for developing and training neural networks. It runs on top of lower level frameworks like TensorFlow, PyTorch, and JAX.

The key components of Keras include:

Keras uses an object-oriented approach, with base classes like Layer and Model that provide common functionality and subclasses that implement specific logic. The layers, models, losses, and other components integrate seamlessly to enable quickly developing neural network models.

Models

References: keras/models

The Keras model classes Sequential, Functional, and Model provide the core building blocks for constructing neural networks. Sequential defines a linear stack of layers, making it simple to build basic models compositionally. Under the hood, Sequential utilizes an underlying Functional model for more complex operations.

Read more

Model Base Class

References: keras/models/model.py

The Model class acts as the base class for all Keras models. It inherits from both the Layer class and the Trainer class. Inheriting from Layer allows models to be treated as layers themselves that can be connected together in Keras' functional API. Inheriting from Trainer adds common training and inference methods like fit(), predict(), etc.

Read more

Sequential Models

References: keras/models/sequential.py

The Sequential class represents a linear stack of layers and provides a simple interface for sequentially adding layers to a model. It handles the ordering of layers and underlying functionality like input/output shape inference. Sequential inherits from the base Model class.

Read more

Functional Models

References: keras/models/functional.py

The Functional class represents a Keras model defined as a directed graph of layers. It inherits from the Function and Model classes. The __init__ method initializes a functional model from input and output tensors, validating the inputs and outputs are Keras tensors.

Read more

Model Utilities

References: keras/models/cloning.py

The …/cloning.py file contains utilities for cloning Keras models. It allows creating a new model instance with the same architecture and layers as an existing model, but with newly initialized weights. This is useful for tasks like model ensembling where you want multiple similar models.

Read more

Model Testing

References: keras/models/sequential_test.py, keras/models/functional_test.py

This section covers the test cases that validate the behavior of Keras models. The main classes for model testing are SequentialTest and FunctionalTest, which inherit from testing.TestCase.

Read more

Layers

References: keras/layers

The …/layers directory contains implementations of commonly used neural network layers that serve as basic building blocks for constructing models. These layers implement fundamental operations like dense connections, convolutions, recurrent connections, and more that are used widely across different model types and domains. The layers define clean Keras interfaces while delegating the actual computations to lower-level frameworks like TensorFlow wherever possible. This separation of concerns allows new layers to be easily added without having to reimplement core logic.

Read more

Core Layers

References: keras/layers/core

The core layers in Keras implement fundamental neural network building blocks like dense connections, embeddings, and input placeholders. These layers form the basic building blocks that can be combined to construct complex models.

Read more

Convolutional Layers

References: keras/layers/convolutional

The …/convolutional directory contains implementations of common convolutional operations as Keras layers. It provides classes for standard convolutions like Conv1D, Conv2D, Conv3D, as well as transposed/fractionally-strided convolutions with Conv1DTranspose, Conv2DTranspose, Conv3DTranspose. Depthwise convolutions are implemented in DepthwiseConv1D and DepthwiseConv2D. Separable convolutions are supported via SeparableConv1D and SeparableConv2D.

Read more

Recurrent Layers

References: keras/layers/rnn

The core RNN layers in Keras are implemented in the …/rnn directory. This includes commonly used RNN cell types like LSTM, GRU, and SimpleRNN as well as variants like ConvLSTM. Each layer type has its own class that inherits from the base RNN class.

Read more

Normalization Layers

References: keras/layers/normalization

The Keras layers in the …/normalization directory implement various normalization techniques that can be applied to inputs. Normalization is useful for stabilizing the learning process and making the optimization problem easier to solve. The key layers are:

Read more

Activation Layers

References: keras/layers/activations

The Keras layers in the …/activations directory implement common activation functions as reusable Keras layers. This allows different activations to be easily added to models during construction.

Read more

Pooling Layers

References: keras/layers/pooling

Pooling layers downsample inputs spatially to reduce the number of parameters and computations in convolutional networks. The MaxPooling2D and AveragePooling2D layers are commonly used for this purpose. MaxPooling2D performs 2D max pooling on input tensors, taking the maximum value in each pooling window. AveragePooling2D similarly pools inputs by taking the average value in each window. Both layers inherit core pooling logic from the BasePooling layer defined in …/base_pooling.py.

Read more

Regularization Layers

References: keras/layers/regularization

The Dropout, GaussianDropout, SpatialDropout1D/2D/3D, and ActivityRegularization layers implement various regularization techniques to help prevent overfitting during neural network training.

Read more

Attention Layers

References: keras/layers/attention

The …/attention directory implements various attention mechanisms for Keras. The core Attention class in …/attention.py handles dot-product attention.

Read more

Preprocessing Layers

References: keras/layers/preprocessing

The Keras preprocessing layers provide a set of tools for common feature engineering and preprocessing tasks in deep learning models. These layers allow features to be normalized, discretized, encoded, and transformed before being fed into models. They handle common tasks like:

Read more

Merging Layers

References: keras/layers/merging

The Keras layers for merging multiple inputs implement common elementwise operations like addition, multiplication, concatenation, and more. These layers take two or more input tensors and combine them through elementwise functions to produce a single output tensor.

Read more

Reshaping Layers

References: keras/layers/reshaping

The Keras layers in the …/reshaping directory provide common reshaping operations that can modify the dimensions or structure of input tensors. This includes layers like Flatten, Reshape, and Cropping which are important for preprocessing data.

Read more

Callbacks

References: keras/callbacks

Callbacks allow customizing model training by hooking into different stages of the process. Key stages include the start and end of epochs, batches, training, validation, and prediction. The Keras callback system handles this through callback classes that inherit from the base Callback class.

Read more

Callback Base Class

References: keras

The Callback base class is defined in …/callback.py. It provides a common interface that all Keras callbacks must implement. The key aspects of the Callback class are:

Read more

Model Checkpointing

References: keras/callbacks/model_checkpoint.py

The ModelCheckpoint callback saves Keras models during training either at the end of each epoch or every N batches. It allows saving either the full model or just the model weights. The callback implements methods like on_train_batch_end(), on_epoch_begin(), and on_epoch_end() to determine when to save.

Read more

Early Stopping

References: keras/callbacks/early_stopping.py

The EarlyStopping callback implements early stopping to prevent overfitting. It monitors a given metric like validation loss during training and stops training if the metric does not improve for a specified number of epochs, called the patience. This helps avoid wasting resources on epochs unlikely to improve performance.

Read more

Learning Rate Scheduling

References: keras/callbacks/learning_rate_scheduler.py

The LearningRateScheduler callback allows dynamically adjusting the learning rate of the optimizer at the beginning of each epoch during training. It inherits from the Callback class and stores the provided schedule function and verbose flag in its __init__ method.

Read more

Logging

References: keras/callbacks/csv_logger.py, keras/callbacks/tensorboard.py

The CSVLogger and TensorBoard callbacks handle logging metrics and model topology during training. CSVLogger writes metric values to a CSV file at the end of each epoch using the on_epoch_end method. It handles different data types when writing by converting values to strings with handle_value().

Read more

Remote Monitoring

References: keras/callbacks/remote_monitor.py

The RemoteMonitor callback integrates Keras training with remote monitoring platforms. The RemoteMonitor class inherits from the base Callback class and is initialized with parameters like the server URL and request path. During training, the RemoteMonitor overrides the on_epoch_end method to collect the epoch number and log metrics. It handles any NumPy arrays in the data and sends a POST request to the server path, serializing the data to JSON. This allows monitoring metrics and logs on each epoch end.

Read more

Application Callbacks

References: keras/applications/vgg16.py, keras/applications/xception.py

The VGG16 and Xception models defined in …/vgg16.py and …/xception.py include callbacks tailored for these applications. When using these models, certain callbacks can be applied to take advantage of features specific to each model.

Read more

Advanced Callbacks

References: keras/callbacks/lambda_callback.py, keras/callbacks/terminate_on_nan.py

This section covers additional specialized callback classes provided in Keras that implement more advanced or niche functionality compared to the core callbacks.

Read more

Optimizers

References: keras/optimizers

The …/optimizers directory contains implementations of various optimization algorithms that can be used to train Keras models. Optimization algorithms are essential for training deep learning models as they iteratively update model weights to minimize a loss function. The key optimization algorithms implemented in Keras include:

Read more

Stochastic Gradient Descent

References: keras/optimizers/sgd.py

The SGD optimizer implements the stochastic gradient descent algorithm for training neural networks. SGD is one of the most commonly used optimization algorithms in deep learning.

Read more

Adaptive Learning Rate Methods

References: keras/optimizers/rmsprop.py, keras/optimizers/adam.py, keras/optimizers/adadelta.py

These algorithms adapt the learning rate during training based on the characteristics of the gradients:

Read more

Additional Optimization Algorithms

References: keras/optimizers/adagrad.py, keras/optimizers/adamax.py, keras/optimizers/nadam.py, keras/optimizers/ftrl.py

These additional optimization algorithms provide alternative approaches to updating model variables during training. Adagrad, Adamax, Nadam, and FTRL each implement distinct optimization algorithms through custom Keras optimizer classes.

Read more

Learning Rate Schedules

References: keras/optimizers/schedules/__init__.py, keras/optimizers/schedules/learning_rate_schedule.py

The Keras optimizers package provides several classes for controlling the learning rate decay over the course of model training. The core class is LearningRateSchedule, which defines the interface for learning rate schedules through its __call__ method. This method takes a step value and returns the decayed learning rate.

Read more

Optimizer Base Class

References: keras/optimizers/optimizer.py

The Optimizer class serves as the base class for all Keras optimizer implementations. It inherits from either TFOptimizer, TorchOptimizer, or JaxOptimizer depending on the Keras backend in use. These backend-specific subclasses contain the implementations of optimizer updates and gradient computations that are compatible with each backend framework. BaseOptimizer defines a more generic base class with minimal functionality that is used if an unsupported backend is detected.

Read more

Optimizer Testing

References: keras/optimizers/sgd_test.py, keras/optimizers/rmsprop_test.py, keras/optimizers/adam_test.py

The unit tests for Keras optimizers validate that the key optimization algorithms are implemented correctly. There are test files for the main optimizers:

Read more

Losses

References: keras/losses

The …/losses directory contains the core loss functions for training neural networks in Keras. The main loss functions are defined in …/losses.py, including MeanSquaredError and CategoricalCrossentropy.

Read more

Loss Function Implementations

References: keras/losses/losses.py

The …/losses.py file implements many common loss functions for training neural networks. The main classes defined are LossFunctionWrapper and individual loss functions like MeanSquaredError, MeanAbsoluteError, CategoricalCrossentropy, and SparseCategoricalCrossentropy.

Read more

Loss Base Class

References: keras/losses/loss.py

The Loss class is the base class that all Keras loss functions must inherit from. It standardizes the implementation of loss functions by defining a common interface and functionality. The Loss class handles calling the subclass' call() method to calculate the raw loss values from the inputs. It then applies masking, weighting by sample weights, and reduction to the losses.

Read more

Loss Utilities

References: keras/losses/__init__.py, keras/losses/loss_test.py

The Loss base class standardizes the implementation of all loss functions. It defines the core interface and methods that subclasses must implement, including call(). The base call() method handles masking and weighting of losses.

Read more

Loss Function Tests

References: keras/losses/loss_test.py

The unit tests in …/loss_test.py validate that Keras loss function implementations behave as expected. An ExampleLoss class is defined that implements mean squared error, acting as a simple test case.

Read more

Metrics

References: keras/metrics

The …/metrics directory contains classes that implement metrics for evaluating model performance during training and testing. Metrics compute statistics like accuracy, precision and recall on model predictions. They are used to monitor and optimize models.

Read more

Metric Base Class

References: keras/metrics/metric.py

The Metric base class defines the core interface that all Keras metrics must implement. It handles tracking metric state and computations through subclasses.

Read more

Reduction Metrics

References: keras/metrics/reduction_metrics.py

The …/reduction_metrics.py file contains utilities for computing reduction metrics like sums and means. It defines the core Sum() and Mean() metric classes, which handle reducing values across samples.

Read more

Confusion Matrix Utilities

References: keras/metrics/metrics_utils.py

The …/metrics_utils.py file provides important utilities for updating confusion matrices. It contains the ConfusionMatrix Enum which defines the possible confusion matrix variables as TRUE_POSITIVES, FALSE_POSITIVES, TRUE_NEGATIVES, FALSE_NEGATIVES.

Read more

Accuracy Metrics

References: keras/metrics/accuracy_metrics.py

The …/accuracy_metrics.py file implements several important accuracy metrics for evaluating classification models. It contains classes that calculate prediction accuracy in different ways depending on the type of classification problem.

Read more

Regression Metrics

References: keras/metrics/regression_metrics.py

The regression metrics defined in …/regression_metrics.py allow evaluating model performance on regression tasks. Key metrics include:

Read more

Probabilistic Metrics

References: keras/metrics/probabilistic_metrics.py

The …/probabilistic_metrics.py file defines several probabilistic metric classes that can be used to evaluate models trained with probabilistic losses. These metrics wrap loss functions to compute a performance metric rather than an error signal.

Read more

F-beta Metrics

References: keras/metrics/f_score_metrics.py

The …/f_score_metrics.py file defines two main classes for computing F-beta scores for classification problems: FBetaScore and F1Score.

Read more

IoU Metrics

References: keras/metrics/iou_metrics.py

The …/iou_metrics.py file defines several classes for computing the Intersection-Over-Union (IoU) metric, which is commonly used to evaluate semantic segmentation models. The IoU metric measures the overlap between predicted regions and ground truth regions.

Read more

Hinge Metrics

References: keras/metrics/hinge_metrics.py

The …/hinge_metrics.py file defines three classes - Hinge, SquaredHinge, and CategoricalHinge - that implement different hinge-based loss metrics. These classes calculate metrics based on the hinge loss like Hinge, SquaredHinge.

Read more

Confusion Matrix Metrics

References: keras/metrics/confusion_metrics.py

The …/confusion_metrics.py file contains metrics that rely on the confusion matrix for evaluation. Metrics like Precision and Recall are implemented, which compute various true and false positive/negative counts from the confusion matrix to calculate precision and recall scores.

Read more

Datasets

References: keras/datasets

The …/datasets package provides easy access to several common datasets that can be conveniently loaded for testing, debugging, and example code in Keras. It contains modules for loading popular image, text, and tabular datasets such as MNIST, Fashion MNIST, CIFAR10, IMDB reviews, and Boston housing prices data.

Read more

MNIST Dataset

References: keras/datasets/mnist.py

The …/mnist.py file provides easy access to the MNIST dataset of handwritten digits for use in examples and testing. It defines a single function, load_data(), which loads the MNIST training and test data from a compressed .npz file stored on Google Cloud Storage.

Read more

Fashion MNIST Dataset

References: keras/datasets/fashion_mnist.py

The …/fashion_mnist.py file provides access to the Fashion MNIST dataset. This dataset contains 60,000 training images and 10,000 test images of clothing items like shirts, shoes and bags. Each 28x28 grayscale image is labeled with one of 10 class names.

Read more

CIFAR10 and CIFAR100 Datasets

References: keras/datasets/cifar10.py, keras/datasets/cifar100.py

The files …/cifar10.py and …/cifar100.py provide access to the CIFAR-10 and CIFAR-100 image classification datasets.

Read more

IMDB Movie Reviews Dataset

References: keras/datasets/imdb.py

The Keras dataset module provides access to the IMDB movie reviews dataset for sentiment classification. The IMDB dataset contains 50,000 movie reviews from IMDB, labeled as either positive or negative.

Read more

Boston Housing Dataset

References: keras/datasets/boston_housing.py

The …/boston_housing.py file provides access to the Boston housing prices regression dataset. This dataset contains information about houses in Boston from the 1970s, including attributes like crime rate and accessibility to ports. The target variable is the median home price.

Read more

California Housing Dataset

References: keras/datasets/california_housing.py

The load_data() function in …/california_housing.py provides access to the California housing prices regression dataset. This function downloads the dataset from an online source if it is not already cached locally. It loads the data from a file containing NumPy arrays for features x and targets y.

Read more

Reuters Dataset

References: keras/datasets/reuters.py

The load_data() function in …/reuters.py provides access to the Reuters newswire categorization dataset. It downloads the dataset from Cloud storage if needed, then shuffles and splits the word sequences and labels into train and test sets. Preprocessing options like truncating long sequences, filtering rare words, and replacing rare words with a special token can be applied.

Read more

Applications

References: keras/applications

The …/applications directory contains implementations of popular pretrained models that can be used for transfer learning tasks. These include computer vision models like VGG16, ResNet50, InceptionV3, as well as NLP models like BERT.

Read more

Pretrained Models

References: keras/applications/vgg16.py, keras/applications/inception_v3.py

The files …/vgg16.py, …/inception_v3.py, and related files contain implementations of popular convolutional neural network architectures that can be used for transfer learning. These pretrained models were trained on the ImageNet dataset and can serve as base models for feature extraction or fine-tuning on new tasks and datasets.

Read more

Model Utilities

References: keras/applications/imagenet_utils.py

The utilities in …/imagenet_utils.py provide common functionality for loading pretrained weights, preprocessing inputs, and decoding predictions from models trained on ImageNet.

Read more

Model Testing

References: keras/applications/applications_test.py, keras/applications/imagenet_utils_test.py

The Keras applications module contains comprehensive unit tests for the pretrained model implementations in applications.py. These tests are contained in two main files:

Read more

Examples

References: examples/keras_io

The …/keras_io directory contains examples demonstrating end-to-end workflows for training neural network models on different types of data using Keras. It covers domains like computer vision, natural language processing, audio processing, time series forecasting, and structured data modeling.

Read more

Computer Vision

References: examples/keras_io/vision

This section covers workflows for training computer vision models on image data using the Keras examples. The key functionality demonstrated includes:

Read more

Natural Language Processing

References: examples/keras_io/nlp

This section covers several examples that demonstrate common natural language processing workflows using Keras for tasks like text classification, sequence modeling, and language modeling. The …/nlp directory contains examples applying recurrent neural networks, convolutional neural networks, and Transformer architectures to problems involving text data.

Read more

Audio Processing

References: examples/keras_io/tensorflow/audio

The examples in the …/audio directory demonstrate workflows for training audio models on tasks like speaker recognition, speech recognition, and accent classification. The code provides end-to-end examples of processing raw audio data into features, building deep learning models, training them, and evaluating performance.

Read more

Generative Modeling

References: examples/keras_io/generative, examples/keras_io/tensorflow/generative

The code in …/generative and …/generative demonstrates workflows for training popular generative models like GANs, VAEs, and diffusion models on image datasets. It implements models such as CycleGAN, DCGAN, VAE, PixelCNN, DDIM, DDPM and more to generate photos, faces, digits and other types of images.

Read more

Reinforcement Learning

References: examples/keras_io/tensorflow/rl

This section demonstrates workflows for training reinforcement learning models to optimize agent behaviors. The code implements several important RL algorithms.

Read more

Structured Data

References: examples/keras_io/structured_data, examples/keras_io/tensorflow/structured_data

The examples in …/structured_data and …/structured_data demonstrate workflows for training models on tabular and structured data. This includes loading and preprocessing CSV datasets, building neural network models to perform tasks like classification and recommendation, and training the models on the data.

Read more

Time Series

References: examples/keras_io/timeseries

This section demonstrates workflows for training various types of time series models on benchmark datasets. The examples cover common time series tasks like anomaly detection, classification, and forecasting.

Read more

Transfer Learning

References: examples/keras_io/pytorch, examples/keras_io/tensorflow/vision

This section demonstrates workflows for leveraging pretrained models via transfer learning. Transfer learning is a technique where a model pre-trained on a large dataset is reused as the starting point for a new task, rather than training a model completely from scratch. The weights of the pretrained model are used as an initialization for the new model, and then a subset of the layers are fine-tuned on the new task while keeping other layers frozen. This helps the model learn meaningful representations from a smaller dataset more efficiently.

Read more