Jump to documentation:

dl/models.hpp

Overview

Types and Functions

• struct ProfilingData
• template <GenericLayer... T> struct SequentialModel
  • template <typename K, unsigned int n> Tensor<LayerHelper::FlintTypeToCpp< get_output_type<to_flint_type<K>(), T...>()>, get_output_dim<n, T...>()> forward_batch(Tensor &in)
  • template <typename K, unsigned int n> Tensor<LayerHelper::FlintTypeToCpp< get_output_type<to_flint_type<K>(), T...>()>, get_output_dim<n, T...>()> forward(Tensor &in)
  • template <typename K, unsigned int n> void optimize(const Tensor &error)
  • void load(const std::string path)
  • void save(const std::string path)
  • template <typename T1, unsigned int n1> void backward(Tensor &error)
  • void enable_training()
  • void disable_training()
  • void enable_profiling()
  • void disable_profiling()
  • ProfilingData last_profiling_data() const
  • std::string summary()
  • std::array<std::string, sizeof...(T)> layer_names()
  • std::array<std::string, sizeof...(T)> layer_descriptions()
  • std::string optimizer()
  • std::string optimizer_description()
  • std::array<size_t, sizeof...(T)> num_layer_parameters()
  • std::vector<std::vector<FGraphNode *>> collect_weights()

struct ProfilingData

Informations about time consumption per layer and memory consumption

template <GenericLayer... T> struct SequentialModel

Model where each layer outputs the input of the next layer. Best used with C++ auto typing:

 auto model = SequentialModel(
  Connected(10, 20),
  Relu(),
  Dropout(0.1),
  Connected(20, 10),
  SoftMax()
 ); // has type SequentialModel<Connected, Relu, Dropout, Connected, SoftMax>

template <typename K, unsigned int n>
		Tensor<LayerHelper::FlintTypeToCpp<
				   get_output_type<to_flint_type<K>(), T...>()>,
			   get_output_dim<n, T...>()>
		forward_batch(Tensor &in)

Passes a batch of input tensors through all layers and returns the output of the last layer.

template <typename K, unsigned int n>
		Tensor<LayerHelper::FlintTypeToCpp<
				   get_output_type<to_flint_type<K>(), T...>()>,
			   get_output_dim<n, T...>()>
		forward(Tensor &in)

Passes an input tensor through all layers and returns the output of the last layer.

template <typename K, unsigned int n>
		void optimize(const Tensor &error)

Optimizes the weights (calculates the gradients + calls the optimizers) of all layer to an error.

void load(const std::string path)

Loads the weights of a model from a file. The file has to be in a concatenated representation of the deserialization of each weight in the correct order.

void save(const std::string path)

Saves the deserialization of all weights of the model to a file.

template <typename T1, unsigned int n1>
		void backward(Tensor &error)

Calculated gradient to the given error tensor for each weight and optimized the weights with their corres

void enable_training()

Enables the training mode, i.e. layers like dropout will be enabled (internally used by the trainer)

void disable_training()

Disables the training mode, i.e. layers like dropout will be disabled (internally used by the trainer)

void enable_profiling()

Starts collection profiling data. This hurts performance in uneager execution.

void disable_profiling()

Stops collection profiling data

ProfilingData last_profiling_data() const

Returns collected profiling data of the last call to

forward

std::string summary()

Returns a small summary of the model.

std::array<std::string, sizeof...(T)> layer_names()

Returns the name of each layer in an array

std::array<std::string, sizeof...(T)> layer_descriptions()

Returns the description of each layer in an array

std::string optimizer()

Returns the name of the generated optimizer

std::string optimizer_description()

Returns the description of the generated optimizer

std::array<size_t, sizeof...(T)> num_layer_parameters()

Returns the number of parameters of each layer in an array

std::vector<std::vector<FGraphNode *>> collect_weights()

Returns a per-layer vector of all weight-tensors of that layer

dl/trainer.hpp

Overview

Types and Functions

• Trainer(SequentialModel &model, TrainingData &data, L loss) : model(model), data(data), loss(loss)
• void max_epochs(int epochs)
• void stopping_error(double error)
• void set_metric_reporter(MetricReporter *reporter)
• void train(int batch_size = 32)
• class NetworkMetricReporter : public MetricReporter

Trainer(SequentialModel &model,
				TrainingData &data, L loss)
			: model(model), data(data), loss(loss)

Trains the model with input data and the desired output.

```
data
```
contains the input (
```
X
```
) and desired data (
```
Y
```
) and optionally validation data, if it does after each epoch a validation error is calculated.
```
loss
```
The loss function to calculate the error between the actual output and the desired one from the training data. Can be an arbitrary class that implements the
```
GenericLoss
```
concept, some implementations can be found in "losses.hpp".
```
epochs
```
.

void max_epochs(int epochs)

Sets the maximum number of epochs after which the training should be stopped. The complete dataset is passed through the model per epoch (It is split into

batch_size

- configured in the

train

method - slices in the first dimension of the input data and each batch has to be passed through the model once per epoch)

void stopping_error(double error)

Sets the minimum epoch error after which the training should be stopped

void set_metric_reporter(MetricReporter *reporter)

Sets the metric reporter (to print or display informations about the training process)

void train(int batch_size = 32)

Trains the model for the given batch size. A batch is a slice of the first imension of the input data. The input is shuffeled every epoch, which is important if your batch size is smaller then your input size. The weights of the model are optimized per batch that was passed through the model. Meaning small batch sizes lead to faster convergence (since more optimizations are executed) and lower memory consumption, but to more noise and variance, since each batch is only an approximation of the complete dataset. If training times and memory consumption don't matter we suggest full gradient descent (meaning

batch_size = input_size

), else finetune this value to your usecase.

class NetworkMetricReporter : public MetricReporter

Sends the trainings data over a REST API for HTTP connections on the port 5111. For a documentation of the API see

dl/visualization/README.md

Documentation dl/models.hpp, dl/trainer.hpp

Flint's C++ Deep Learning Framework

dl/models.hpp

dl/trainer.hpp