mdlearn.nn.utils

Functions

conv_output_dim(input_dim, kernel_size, ...)

Parameters

  • input_dim (int) -- input size. may include padding

conv_output_shape(input_dim, kernel_size, ...)

Parameters

  • input_dim (tuple) -- (height, width) dimensions for convolution input

get_activation(activation, *args, **kwargs)

Parameters

activation (str) -- type of activation e.g. 'ReLU', etc

reset(nn)

same_padding(input_dim, kernel_size, stride)

Returns Keras-like same padding.

Classes

Trainer([seed, in_gpu_memory, ...])

Trainer base class which implements training utility functions.
class mdlearn.nn.utils.Trainer(seed: int = 42, in_gpu_memory: bool = False, num_data_workers: int = 0, prefetch_factor: int = 2, split_pct: float = 0.8, split_method: str = 'random', batch_size: int = 128, shuffle: bool = True, device: str = 'cpu', epochs: int = 100, verbose: bool = False, clip_grad_max_norm: float = 10.0, checkpoint_log_every: int = 10, plot_log_every: int = 10, plot_n_samples: int = 10000, plot_method: Optional[str] = 'TSNE', train_subsample_pct: float = 1.0, valid_subsample_pct: float = 1.0, use_wandb: bool = False)

Trainer base class which implements training utility functions.

__init__(seed: int = 42, in_gpu_memory: bool = False, num_data_workers: int = 0, prefetch_factor: int = 2, split_pct: float = 0.8, split_method: str = 'random', batch_size: int = 128, shuffle: bool = True, device: str = 'cpu', epochs: int = 100, verbose: bool = False, clip_grad_max_norm: float = 10.0, checkpoint_log_every: int = 10, plot_log_every: int = 10, plot_n_samples: int = 10000, plot_method: Optional[str] = 'TSNE', train_subsample_pct: float = 1.0, valid_subsample_pct: float = 1.0, use_wandb: bool = False)
Parameters

  • seed (int, default=42) – Random seed for torch, numpy, and random module.

  • in_gpu_memory (bool, default=False) – If True, will pre-load the entire data array to GPU memory.

  • num_data_workers (int, default=0) – How many subprocesses to use for data loading. 0 means that the data will be loaded in the main process.

  • prefetch_factor (int, by default=2) – Number of samples loaded in advance by each worker. 2 means there will be a total of 2 * num_workers samples prefetched across all workers.

  • split_pct (float, default=0.8) – Proportion of data set to use for training. The rest goes to validation.

  • split_method (str, default=”random”) – Method to split the data. For random split use “random”, for a simple partition, use “partition”.

  • batch_size (int, default=128) – Mini-batch size for training.

  • shuffle (bool, default=True) – Whether to shuffle training data or not.

  • device (str, default=”cpu”) – Specify training hardware either cpu or cuda for GPU devices.

  • epochs (int, default=100) – Number of epochs to train for.

  • verbose (bool, default False) – If True, will print training and validation loss at each epoch.

  • clip_grad_max_norm (float, default=10.0) – Max norm of the gradients for gradient clipping for more information see: torch.nn.utils.clip_grad_norm_ documentation.

  • checkpoint_log_every (int, default=10) – Epoch interval to log a checkpoint file containing the model weights, optimizer, and scheduler parameters.

  • plot_log_every (int, default=10) – Epoch interval to log a visualization plot of the latent space.

  • plot_n_samples (int, default=10000) – Number of validation samples to use for plotting.

  • plot_method (Optional[str], default=”TSNE”) – The method for visualizing the latent space or if visualization should not be run, set plot_method=None. If using "TSNE", it will attempt to use the RAPIDS.ai GPU implementation and will fallback to the sklearn CPU implementation if RAPIDS.ai is unavailable.

  • train_subsample_pct (float, default=1.0) – Percentage of training data to use during hyperparameter sweeps.

  • valid_subsample_pct (float, default=1.0) – Percentage of validation data to use during hyperparameter sweeps.

  • use_wandb (bool, default=False) – If True, will log results to wandb.

Raises

  • ValueError – split_pct should be between 0 and 1.

  • ValueError – train_subsample_pct should be between 0 and 1.

  • ValueError – valid_subsample_pct should be between 0 and 1.

  • ValueError – Specified device as cuda, but it is unavailable.

Note

This base class does not receive optimizer or scheduler settings because in general there could be multiple optimizers.

fit()

Trains the model on the input dataset.

Raises

NotImplementedError – Child class must implement this method.

predict()

Predicts using the trained model.

Raises

NotImplementedError – Child class must implement this method.

step_scheduler(epoch: int, avg_train_loss: float, avg_valid_loss: float)

Implements the logic to step the learning rate scheduler. Different schedulers may have different update logic. Please subclass LinearAETrainer and re-implement this function for support of additional logic.

Parameters

  • epoch (int) – The current training epoch.

  • avg_train_loss (float) – The current epochs average training loss.

  • avg_valid_loss (float) – The current epochs average valiation loss.

Raises

NotImplementedError – If using a learning rate scheduler other than ReduceLROnPlateau, a step function will need to be added.

mdlearn.nn.utils.conv_output_dim(input_dim, kernel_size, stride, padding, transpose=False)
Parameters

  • input_dim (int) – input size. may include padding

  • kernel_size (int) – filter size

  • stride (int) – stride length

  • padding (int) – length of 0 pad

mdlearn.nn.utils.conv_output_shape(input_dim, kernel_size, stride, padding, num_filters, transpose=False, dim=2)
Parameters

  • input_dim (tuple) – (height, width) dimensions for convolution input

  • kernel_size (int) – filter size

  • stride (int) – stride length

  • padding (tuple) – (height, width) length of 0 pad

  • num_filters (int) – number of filters

  • transpose (bool) – signifies whether Conv or ConvTranspose

  • dim (int) – 1 or 2, signifies Conv1d or Conv2d

Returns

(channels, height, width) tuple

mdlearn.nn.utils.get_activation(activation, *args, **kwargs)
Parameters

activation (str) – type of activation e.g. ‘ReLU’, etc

mdlearn.nn.utils.reset(nn)
mdlearn.nn.utils.same_padding(input_dim: Union[int, Tuple[int, int]], kernel_size: int, stride: int) → Union[int, Tuple[int, int]]

Returns Keras-like same padding. Works for rectangular input_dim.

Parameters

  • input_dim (tuple, int) – (height, width) dimensions for Conv2d input int for Conv1d input

  • kernel_size (int) – filter size

  • stride (int) – stride length

Returns

  • int – height of padding

  • int – width of padding