experiment

aitoolbox.experiment defines the experiment tracking and performance evaluation components. Because all implemented components are completely independent from the TrainLoop engine they can be used either on their own in a more manual mode or as part of the TrainLoop functionality available in aitoolbox.torchtrain. Due to the independence of the components, certain elements, for performance evaluation can even be utilized for evaluation of non-PyTorch models.

In general, aitoolbox.experiment helps the user with the following:

• Structured and reusable performance evaluation logic definition

  • aitoolbox.experiment.result_package

  • aitoolbox.experiment.core_metrics

• Tracked training performance history primitive

  • aitoolbox.experiment.training_history

• High level experiment tracking API

  • aitoolbox.experiment.experiment_saver

  • aitoolbox.experiment.local_experiment_saver

• Low level experiment tracking primitives for model saving and performance results saving

  • aitoolbox.experiment.local_save

• Saved model re-loading low level primitives

  • aitoolbox.experiment.local_load.local_model_load

Guides:

• Result Package
• Result Metric
• Experiment Saving
• Training History