report_generator

class aitoolbox.experiment.result_reporting.report_generator.TrainingHistoryPlotter(experiment_results_local_path)

Bases: object

Plot the calculated performance metrics in the training history

Parameters:

experiment_results_local_path (str) – path to the main experiment results folder on the local drive

generate_report(training_history, plots_folder_name='plots', file_format='png')

Plot all the currently present performance result in the training history

Every plot shows the progression of a single performance metric over the epochs.

Parameters:

• training_history (aitoolbox.experiment.training_history.TrainingHistory) – TrainLoop training history

• plots_folder_name (str) – local dir name where the plots should be saved

• file_format (str) – output file format. Can be either ‘png’ for saving separate images or ‘pdf’ for combining all the plots into a single pdf file.

Returns:

list of saved plot paths

Return type:

list

plot_png(training_history, plots_local_folder_path, plots_folder_name)

plot_pdf(training_history, plots_local_folder_path, plots_file_name)

static generate_plots(training_history)

static plot_performance_curve(metric_name, result_history)

Plot the performance of a selected calculated metric over the epochs

Parameters:

• metric_name (str or int) – name of plotted metric

• result_history (list or np.array) – results history for the selected metric

Returns:

plot figure

Return type:

plt.figure

class aitoolbox.experiment.result_reporting.report_generator.TrainingHistoryWriter(experiment_results_local_path)

Bases: object

Write the calculated performance metrics in the training history into human-readable text file

Parameters:

experiment_results_local_path (str or None) – path to the main experiment results folder on the local drive

generate_report(training_history, epoch, file_name, results_folder_name='', file_format='txt')

Write all the currently present performance result in the training history into the text file

Parameters:

• training_history (aitoolbox.experiment.training_history.TrainingHistory) –

• epoch (int) – current epoch

• file_name (str) – output text file name

• results_folder_name (str) – results folder path where the report file will be located

• file_format (str) – output file format. Can be either ‘txt’ human-readable output or ‘tsv’ for a tabular format or ‘csv’ for comma separated format.

Returns:

file name/path inside the experiment folder, local file_path

Return type:

str, str

static write_txt(training_history, epoch, file_path)

write_csv_tsv(training_history, epoch, file_path, delimiter)

class aitoolbox.experiment.result_reporting.report_generator.GradientPlotter(experiment_grad_results_local_path)

Bases: object

Plot the gradient distributions for model’s layers

Parameters:

experiment_grad_results_local_path (str) – path to the main experiment results folder on the local drive

generate_report(model_layer_gradients, grad_plots_folder_name='grad_plots', file_format='png')

Plot all the gradient distributions for the layers in the model

Parameters:

• model_layer_gradients (list) – list of model’s gradients

• grad_plots_folder_name (str) – name of the folder where gradient distribution plots will be saved

• file_format (str) – output file format. Can be either ‘png’ for saving separate images or ‘pdf’ for combining all the plots into a single pdf file.

Returns:

list of saved plot paths: [file_path_in_cloud_grad_results_dir, local_file_path]

Return type:

list

plot_png(model_layer_gradients, grad_plots_local_folder_path, plots_folder_name)

plot_pdf(model_layer_gradients, plots_local_folder_path, plots_file_name)

static generate_dist_plots(model_layer_gradients, layer_names=None)

static plot_gradient_distribution(gradients, layer_name)

Plot and save to file the distribution of the single layer’s gradients

Parameters:

• gradients (list or np.array) – a flattened list of gradients from a single layer

• layer_name (str or int) – name or index of the layer

Returns:

plot figure

Return type:

plt.figure