mighty.monitor.monitor.MonitorAutoencoder

class mighty.monitor.monitor.MonitorAutoencoder(mutual_info=None, normalize_inverse=None)[source]

A monitor for mighty.trainer.TrainerAutoencoder.

Methods

__init__([mutual_info, normalize_inverse])

advanced_monitoring([level])

Sets the extent of monitoring.

batch_finished(model)

Batch finished monitor callback.

batch_finished_first_epoch(model)

First batch finished monitor callback.

clear()

Clear out all Visdom plots.

clusters_heatmap(mean[, title, save])

Cluster centers distribution heatmap.

embedding_hist(activations)

Plots embedding activations histogram.

epoch_finished()

Epoch finished callback.

log(text)

Logs the text.

log_model(model[, space])

Logs the model.

log_self()

Logs the monitor itself.

open(env_name[, offline])

Opens a Visdom server.

plot_autoencoder(images, reconstructed, *tensors)

Plot autoencoder reconstructed samples.

plot_explain_input_mask(model, mask_trainer, ...)

Plot the mask where the model is "looking at" to make decisions about the class label.

plot_psnr(psnr[, mode])

If given, plot the Peak Signal to Noise Ratio.

plot_reconstruction_error(pixel_missed, ...)

Plot the reconstruction pixel-wise error, depending on the threshold.

register_func(func)

Register a plotting function to call on the end of each epoch.

register_layer(layer, prefix)

Register a layer to monitor.

reset()

Reset the parameter records statistics.

update_accuracy(accuracy[, mode])

Update the accuracy plot with a new value.

update_accuracy_epoch(labels_pred, ...)

The callback to calculate and update the epoch accuracy from a batch of predicted and true class labels.

update_grad_norm()

Update the parameters gradient norm.

update_gradient_signal_to_noise_ratio()

Update the SNR, mean divided by std, of the model weight gradients.

update_initial_difference()

Update the L1 normalized difference between the current and starting weights (before training).

update_l1_neuron_norm(l1_norm)

Update the L1 neuron norm heatmap, normalized by the batch size.

update_loss(loss[, mode])

Update the loss plot with a new value.

update_mutual_info()

Update the mutual info.

update_pairwise_dist(mean, std)

Updates L1 mean pairwise distance and intra-STD of embedding centroids.

update_sparsity(sparsity, mode)

Update the L1 sparsity of the hidden layer activations.

update_weight_histogram()

Update the model weights histogram.

update_weight_trace_signal_to_noise_ratio()

Update the SNR, mean divided by std, of the model weights.

Attributes

is_active

Returns:

n_classes_format_ytickstep_1

advanced_monitoring(level=7)

Sets the extent of monitoring.

Parameters:
levelMonitorLevel, optional

New monitoring level to apply. * DISABLED - only basic metrics are computed (memory tolerable) * SIGNAL_TO_NOISE - track SNR of the gradients * FULL - SNR, sign flips, weight hist, weight diff Default: MonitorLevel.NORMAL

Notes

Advanced monitoring is memory consuming.

batch_finished(model)

Batch finished monitor callback.

Parameters:
modelnn.Module

A model that has been trained the last epoch.

batch_finished_first_epoch(model)

First batch finished monitor callback.

Parameters:
modelnn.Module

A model that has been trained the last epoch.

clear()

Clear out all Visdom plots.

clusters_heatmap(mean, title=None, save=False)

Cluster centers distribution heatmap.

Parameters:
mean(C, V) torch.Tensor

The mean of C clusters (vectors of size V).

titlestr or None, optional

An optional title to the plot. If None, set to “Embedding activations heatmap”. Default: None

savebool, optional

Save the heatmap plot in a separate fixed window. Default: False

embedding_hist(activations)

Plots embedding activations histogram.

Parameters:
activations(N,) torch.Tensor

The averaged embedding vector.

epoch_finished()

Epoch finished callback.

property is_active
Returns:
bool

Indicator whether a Visdom server is initialized or not.

log(text)

Logs the text.

Parameters:
textstr

Log text.

log_model(model, space='-')

Logs the model.

Parameters:
modelnn.Module

A PyTorch model.

spacestr, optional

A space substitution to correctly parse HTML later on. Default: ‘-’

log_self()

Logs the monitor itself.

open(env_name: str, offline=False)

Opens a Visdom server.

Parameters:
env_namestr

Environment name.

offlinebool

Offline mode (True) or online (False).

plot_autoencoder(images, reconstructed, *tensors, labels=(), normalize_inverse=True, n_show=10, mode='train')[source]

Plot autoencoder reconstructed samples.

Parameters:
images, reconstructed(B, C, H, W) torch.Tensor

A batch of input and reconstructed images.

*tensors(B, C, H, W) torch.Tensor

Other tensors of the same size, showing intermediate steps.

labelstuple of str, optional

The labels of additional tensors.

normalize_inversebool, optional

Perform inverse normalization to the input samples to show images in the original input domain rather than zero-mean unit-variance normalized representation. Default: True

n_showint, optional

The number of samples to show. Default: 10

mode{‘train’, ‘test’}

The update mode. Default: ‘train’

plot_explain_input_mask(model, mask_trainer, image, label, win_suffix='')

Plot the mask where the model is “looking at” to make decisions about the class label. Based on [1].

Parameters:
modelnn.Module

The model.

mask_trainerMaskTrainer

The instance of MaskTrainer.

imagetorch.Tensor

The input image to investigate and plot the mask on.

labelint

The class label to investigate.

win_suffixstr, optional

The unique window suffix to distinguish different scenarios. Default: ‘’

References

[1]

Fong, R. C., & Vedaldi, A. (2017). Interpretable explanations of black boxes by meaningful perturbation. In Proceedings of the IEEE International Conference on Computer Vision (pp. 3429-3437).

plot_psnr(psnr, mode='train')

If given, plot the Peak Signal to Noise Ratio.

Used in training autoencoders.

Parameters:
psnrtorch.Tensor or float

The Peak Signal to Noise Ratio scalar.

mode{‘train’, ‘test’}, optional

The update mode. Default: ‘train’

plot_reconstruction_error(pixel_missed, thresholds, optimal_id=None)[source]

Plot the reconstruction pixel-wise error, depending on the threshold.

When the input images can be considered as binary, like MNIST, this plot helps to choose the correct threshold that minimizes incorrectly reconstructed binary pixels count.

Parameters:
pixel_missed(N,) torch.Tensor

Incorrectly reconstructed pixels count.

thresholds(N,) torch.Tensor

Used thresholds.

optimal_idint or None, optional

The optimal threshold ID used as the “best” threshold. If None, set to pixel_missed.argmin().

register_func(func)

Register a plotting function to call on the end of each epoch.

The func must have only one argument viz, a Visdom instance.

Parameters:
funccallable

User-provided plot function with one argument viz.

register_layer(layer: Module, prefix: str)

Register a layer to monitor.

Parameters:
layernn.Module

A model layer.

prefixstr

The layer name.

reset()

Reset the parameter records statistics.

update_accuracy(accuracy, mode='batch')

Update the accuracy plot with a new value.

Parameters:
accuracytorch.Tensor or float

Accuracy scalar.

mode{‘batch’, ‘epoch’}, optional

The update mode. Default: ‘batch’

update_accuracy_epoch(labels_pred, labels_true, mode)

The callback to calculate and update the epoch accuracy from a batch of predicted and true class labels.

Parameters:
labels_pred, labels_true(N,) torch.Tensor

Predicted and true class labels.

modestr

Update mode: ‘batch’ or ‘epoch’.

Returns:
accuracytorch.Tensor

A scalar tensor with one value - accuracy.

update_grad_norm()

Update the parameters gradient norm.

update_gradient_signal_to_noise_ratio()

Update the SNR, mean divided by std, of the model weight gradients.

Similar to Monitor.update_weight_trace_signal_to_noise_ratio() but on a smaller time scale.

update_initial_difference()

Update the L1 normalized difference between the current and starting weights (before training).

update_l1_neuron_norm(l1_norm: Tensor)

Update the L1 neuron norm heatmap, normalized by the batch size.

Useful to explore which neurons are “dead” and which - “super active”.

Parameters:
l1_norm(V,) torch.Tensor

L1 norm per neuron in a hidden layer.

update_loss(loss, mode='batch')

Update the loss plot with a new value.

Parameters:
losstorch.Tensor

Loss tensor. If None, do noting.

mode{‘batch’, ‘epoch’}, optional

The update mode. Default: ‘batch’

update_mutual_info()

Update the mutual info.

update_pairwise_dist(mean, std)

Updates L1 mean pairwise distance and intra-STD of embedding centroids.

\text{inter-distance} = \frac{pdist(\text{mean}, p=1)} {||\text{mean}||_1} \text{intra-STD} = \frac{||\text{std}||_1}{||\text{mean}||_1},

where ||x||_1 = \frac{\sum_{ij}{|x_{ij}|}}{C}.

When these two metrics equalize, the feature vectors, embeddings, become indistinguishable. We want \text{inter-distance} to be high and \text{intra-STD} to keep low.

Parameters:
mean, stdtorch.Tensor

Tensors of shape (C, V). The mean and standard deviation of C clusters (vectors of size V).

Returns:
torch.Tensor

A scalar, a proxy to signal-to-noise ratio defined as

\frac{\text{inter-distance}}{\text{intra-STD}}

update_sparsity(sparsity, mode)

Update the L1 sparsity of the hidden layer activations.

Parameters:
sparsitytorch.Tensor or float

Sparsity scalar.

mode{‘train’, ‘test’}

The update mode.

update_weight_histogram()

Update the model weights histogram.

update_weight_trace_signal_to_noise_ratio()

Update the SNR, mean divided by std, of the model weights.

If mean / std is large, the network is confident in which direction to “move”. If mean / std is small, the network is making random walk.