model_load

class aitoolbox.torchtrain.callbacks.model_load.ModelLoadContinueTraining(saved_experiment_timestamp, saved_model_dir='checkpoint_model', epoch_num=None, ignore_saved_schedulers=False, ignore_missing_saved_schedulers=False, used_data_parallel=False, custom_local_loader_class=None, project_name=None, experiment_name=None, local_model_result_folder_path=None, cloud_save_mode=None, bucket_name=None, cloud_dir_prefix=None, **kwargs)[source]

Bases: AbstractExperimentCallback

(Down)load previously trained and saved model and continue training from this snapshot instead from beginning

Parameters:

  • saved_experiment_timestamp (str) – timestamp of the saved model experiment

  • saved_model_dir (str) – folder where saved model file is inside main experiment folder

  • epoch_num (int or None) – if loading checkpoint model instead of final model this parameter indicates from which epoch of training the model will be loaded

  • ignore_saved_schedulers (bool) – if exception should be raised in the case there are found scheduler snapshots in the checkpoint, but not schedulers are provided to this method

  • ignore_missing_saved_schedulers (bool) – if exception should be raised in the case schedulers are provided to this method but no saved scheduler snapshots can be found in the checkpoint

  • used_data_parallel (bool) – if the saved model was nn.DataParallel or normal model

  • custom_local_loader_class (AbstractLocalModelLoader class or None) – provide a custom local PyTorch model loader definition in case the default one is not suitable for particular use case. For example, in the case of complex custom optimizer initialization.

  • project_name (str or None) – root name of the project

  • experiment_name (str or None) – name of the particular experiment

  • local_model_result_folder_path (str or None) – root local path where project folder will be created

  • cloud_save_mode (str or None) – Storage destination selector. For AWS S3: ‘s3’ / ‘aws_s3’ / ‘aws’ For Google Cloud Storage: ‘gcs’ / ‘google_storage’ / ‘google storage’ Everything else results just in local storage to disk

  • bucket_name (str) – name of the bucket in the cloud storage

  • cloud_dir_prefix (str) – path to the folder inside the bucket where the experiments are going to be saved

  • **kwargs – additional parameters for the local model loader load_model() function

on_train_loop_registration()[source]

Execute callback initialization / preparation after the train_loop_object becomes available

Returns:

None

on_train_begin()[source]

Logic executed at the beginning of the overall training

Returns:

None

init_model_loader()[source]

Initialize model loader object based on provided arguments to the callback object

Returns:

None