mighty.monitor.monitor.Monitor

class mighty.monitor.monitor.Monitor(mutual_info=None, normalize_inverse=None)

Generic Monitor that provides meaningful statistics in interactive Visdom plots throughout training.

Parameters:

mutual_infoMutualInfo or None, optional

The Mutual Information estimator (the same as in a trainer). Default: None

normalize_inverseNormalizeInverse or None, optional

The inverse normalization transform, taken from a data loader. Default: None

Methods

__init__([mutual_info, normalize_inverse])
advanced_monitoring([level])	Sets the extent of monitoring.
batch_finished(model)	Batch finished monitor callback.
batch_finished_first_epoch(model)	First batch finished monitor callback.
clear()	Clear out all Visdom plots.
epoch_finished()	Epoch finished callback.
log(text)	Logs the text.
log_model(model[, space])	Logs the model.
log_self()	Logs the monitor itself.
open(env_name[, offline])	Opens a Visdom server.
plot_explain_input_mask(model, mask_trainer, ...)	Plot the mask where the model is "looking at" to make decisions about the class label.
plot_psnr(psnr[, mode])	If given, plot the Peak Signal to Noise Ratio.
register_func(func)	Register a plotting function to call on the end of each epoch.
register_layer(layer, prefix)	Register a layer to monitor.
reset()	Reset the parameter records statistics.
update_accuracy(accuracy[, mode])	Update the accuracy plot with a new value.
update_accuracy_epoch(labels_pred, ...)	The callback to calculate and update the epoch accuracy from a batch of predicted and true class labels.
update_grad_norm()	Update the parameters gradient norm.
update_gradient_signal_to_noise_ratio()	Update the SNR, mean divided by std, of the model weight gradients.
update_initial_difference()	Update the L1 normalized difference between the current and starting weights (before training).
update_loss(loss[, mode])	Update the loss plot with a new value.
update_mutual_info()	Update the mutual info.
update_weight_histogram()	Update the model weights histogram.
update_weight_trace_signal_to_noise_ratio()	Update the SNR, mean divided by std, of the model weights.

Attributes

is_active	Returns:
n_classes_format_ytickstep_1

advanced_monitoring(level=7)

Sets the extent of monitoring.

Parameters:

levelMonitorLevel, optional

New monitoring level to apply. * DISABLED - only basic metrics are computed (memory tolerable) * SIGNAL_TO_NOISE - track SNR of the gradients * FULL - SNR, sign flips, weight hist, weight diff Default: MonitorLevel.NORMAL

Notes

Advanced monitoring is memory consuming.

batch_finished(model)

Batch finished monitor callback.

Parameters:

modelnn.Module

A model that has been trained the last epoch.

batch_finished_first_epoch(model)

First batch finished monitor callback.

Parameters:

modelnn.Module

A model that has been trained the last epoch.

clear()

Clear out all Visdom plots.

epoch_finished()

Epoch finished callback.

property is_active

Returns:

bool

Indicator whether a Visdom server is initialized or not.

log(text)

Logs the text.

Parameters:

textstr

Log text.

log_model(model, space='-')

Logs the model.

Parameters:

modelnn.Module

A PyTorch model.

spacestr, optional

A space substitution to correctly parse HTML later on. Default: ‘-’

log_self()

Logs the monitor itself.

open(env_name: str, offline=False)

Opens a Visdom server.

Parameters:

env_namestr

Environment name.

offlinebool

Offline mode (True) or online (False).

plot_explain_input_mask(model, mask_trainer, image, label, win_suffix='')

Plot the mask where the model is “looking at” to make decisions about the class label. Based on [1].

Parameters:

modelnn.Module

The model.

mask_trainerMaskTrainer

The instance of MaskTrainer.

imagetorch.Tensor

The input image to investigate and plot the mask on.

labelint

The class label to investigate.

win_suffixstr, optional

The unique window suffix to distinguish different scenarios. Default: ‘’

References

[1]

Fong, R. C., & Vedaldi, A. (2017). Interpretable explanations of black boxes by meaningful perturbation. In Proceedings of the IEEE International Conference on Computer Vision (pp. 3429-3437).

plot_psnr(psnr, mode='train')

If given, plot the Peak Signal to Noise Ratio.

Used in training autoencoders.

Parameters:

psnrtorch.Tensor or float

The Peak Signal to Noise Ratio scalar.

mode{‘train’, ‘test’}, optional

The update mode. Default: ‘train’

register_func(func)

Register a plotting function to call on the end of each epoch.

The func must have only one argument viz, a Visdom instance.

Parameters:

funccallable

User-provided plot function with one argument viz.

register_layer(layer: Module, prefix: str)

Register a layer to monitor.

Parameters:

layernn.Module

A model layer.

prefixstr

The layer name.

reset()

Reset the parameter records statistics.

update_accuracy(accuracy, mode='batch')

Update the accuracy plot with a new value.

Parameters:

accuracytorch.Tensor or float

Accuracy scalar.

mode{‘batch’, ‘epoch’}, optional

The update mode. Default: ‘batch’

update_accuracy_epoch(labels_pred, labels_true, mode)

The callback to calculate and update the epoch accuracy from a batch of predicted and true class labels.

Parameters:

labels_pred, labels_true(N,) torch.Tensor

Predicted and true class labels.

modestr

Update mode: ‘batch’ or ‘epoch’.

Returns:

accuracytorch.Tensor

A scalar tensor with one value - accuracy.

update_grad_norm()

Update the parameters gradient norm.

update_gradient_signal_to_noise_ratio()

Update the SNR, mean divided by std, of the model weight gradients.

Similar to Monitor.update_weight_trace_signal_to_noise_ratio() but on a smaller time scale.

update_initial_difference()

Update the L1 normalized difference between the current and starting weights (before training).

update_loss(loss, mode='batch')

Update the loss plot with a new value.

Parameters:

losstorch.Tensor

Loss tensor. If None, do noting.

mode{‘batch’, ‘epoch’}, optional

The update mode. Default: ‘batch’

update_mutual_info()

Update the mutual info.

update_weight_histogram()

Update the model weights histogram.

update_weight_trace_signal_to_noise_ratio()

Update the SNR, mean divided by std, of the model weights.

If mean / std is large, the network is confident in which direction to “move”. If mean / std is small, the network is making random walk.