Saliency Maps: PyTorch and Captum Guide

Q: Why does Captum return absolute values of gradients by default?

Captum’s captum.attr.Saliency sets abs=True by default because any input feature with a large gradient—whether positive or negative—substantially alters the target class output. In visual saliency, researchers are typically interested in the magnitude of influence regardless of sign. Setting abs=False returns signed directional gradients.

What it is used for in research

In machine learning and computer vision research, saliency maps are first-order gradient attribution visualizations that identify which input features most strongly influence a neural network model’s output prediction. Formally introduced by Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman in their 2013 paper Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps (arXiv:1312.6034), vanilla gradient saliency computes the derivative of an unnormalized class score with respect to the input values.

In research environments, investigators rely on saliency maps for three primary analytical workflows:

Model Debugging and Shortcut Detection: Inspecting whether deep networks make predictions based on valid biological, medical, or physical signals rather than spurious dataset artifacts (such as watermarks, scanner borders, or background noise).
Feature Attribution in Multi-Modal Architectures: Evaluating input sensitivity across continuous multidimensional data, ranging from computer vision to physics-informed neural networks and biomedical imaging pipelines.
Weakly Supervised Localization: Extracting approximate object boundaries or region-of-interest heatmaps from classification networks without requiring pixel-level segmentation masks during training.

When conducting a reproducible AI experiment, publishing the precise attribution algorithms, target class indices, and normalization thresholds ensures rigorous computational reproducibility across research groups.

Install or access (official source only)

Saliency maps can be implemented from first principles in deep learning frameworks or computed using validated model interpretability packages. The official PyTorch library for model interpretability is Captum, developed as open-source software by PyTorch and Meta Platforms, Inc. Captum provides optimized, production-tested attribution algorithms.

According to the official Captum installation specification, system requirements include:

Python: Version 3.6 or higher
PyTorch: Version 1.2 or higher
NumPy: Standard installation
Matplotlib: Required for image attribution rendering

Install PyTorch and Captum using either Conda or Pip directly from official channels:

Anaconda installation (official PyTorch channel):

conda install captum -c pytorch

Pip installation (Python Package Index):

pip install captum

For GPU acceleration of large tensor operations during attribution passes, ensure CUDA drivers match the installed PyTorch build, or utilize CuPy for independent array manipulation on NVIDIA GPUs.

Step-by-step task

The following five-step walkthrough demonstrates how to compute and visualize a saliency map for an image classification model using PyTorch and Captum’s captum.attr.Saliency class.

Import modules and prepare the model:
Load PyTorch, Torchvision, and Captum attribution classes. Set the network to evaluation mode (model.eval()) to disable stochastic operations such as dropout.

import torch
import torchvision.models as models
from captum.attr import Saliency
from captum.attr import visualization as viz
import numpy as np

# Load a pre-trained model and set to evaluation mode
model = models.resnet50(pretrained=True)
model.eval()

Format input tensor and enable gradient tracking:
Ensure the input tensor has batch dimensions (N, C, H, W), is normalized according to model specifications, and explicitly requires gradients:
```
# Shape: (1, 3, 224, 224)
input_tensor = torch.randn(1, 3, 224, 224, requires_grad=True)
```
Initialize the Saliency attribution object:
Instantiate captum.attr.Saliency by passing the forward function of the model:
```
saliency = Saliency(model)
```
Compute the attribution tensor:
Execute the attribute() method, specifying the target class index. By default, abs=True returns the absolute value of the computed gradients:
```
# Specify the target class (e.g., class index 281 for 'cat')
target_class = 281
attribution = saliency.attribute(input_tensor, target=target_class, abs=True)
```

Transpose and visualize the heatmap:
Convert the attribution tensor to a NumPy array, permute dimensions to channels-last format (H, W, C), and render using Captum’s official visualization utility:

# Squeeze batch dimension and transpose to (H, W, C)
attribution_np = np.transpose(attribution.squeeze(0).cpu().detach().numpy(), (1, 2, 0))

# Normalize and render attribution heatmap
fig, ax = viz.visualize_image_attr(
    attr=attribution_np,
    method='heat_map',
    sign='absolute_value',
    show_colorbar=True,
    title='Input Saliency Map'
)

Output and how to read it

The output of saliency.attribute() is a PyTorch tensor with the exact same shape and dimensionality as the input tensor (N, C, H, W). Each element represents the first-order partial derivative of the specified target class output score with respect to that input feature.

When analyzing saliency map output, researchers evaluate three specific structural properties:

Attribution Magnitude: Higher values indicate that small perturbations in those pixels cause large changes in the model’s output score. In Captum, because abs=True is default, positive and negative gradient influences are unified into an unsigned sensitivity magnitude.
Color Channel Reduction: As established in Simonyan et al. (2013), for multi-channel RGB inputs, the overall saliency of each spatial pixel (i, j) is calculated as the maximum absolute gradient across color channels: M(i,j) = max_c |w(i,j,c)|. This prevents individual color channel shifts from canceling out spatial relevance.
Spatial Localization: Heatmaps displayed via visualize_image_attr apply percentile clipping (default outlier_perc=2) to prevent extreme outlier gradients from flattening the visual dynamic range of salient structures.

Troubleshooting

Common technical challenges when computing saliency maps in PyTorch and Captum include:

Zero or completely uniform attributions:
Cause: The input tensor was wrapped in torch.no_grad() or lacked requires_grad=True before the forward call, preventing gradient backpropagation. Alternatively, the model’s forward path terminated at an activation function in a saturated regime (such as negative inputs into a standard ReLU yielding zero gradient).
Resolution: Verify that input_tensor.requires_grad_(True) is called and that computation occurs in gradient-enabled context. For saturated ReLUs, use integrated gradients or replace ReLUs with LeakyReLU during exploratory architecture analysis.
Grainy, high-frequency checkerboard noise:
Cause: Standard vanilla saliency captures local gradient fluctuations and gradient shattering inherent to deep non-linear networks, producing pixelated speckle patterns rather than coherent semantic regions.
Resolution: Apply SmoothGrad (averaging saliency across multiple Gaussian-perturbed copies of the input) or use path-integrated methods like captum.attr.IntegratedGradients to smooth high-frequency noise.
IndexError: Target out of range:
Cause: The integer supplied to target exceeds the maximum class dimension of the model’s output layer.
Resolution: Confirm that the target index is non-negative and strictly less than the total number of classes (from 0 to num_classes – 1). If targeting the model’s top predicted class dynamically, extract the index using target = model(input_tensor).argmax(dim=-1).item().
Dimension mismatch during visualization:
Cause: Passing a 4D batch tensor (N, C, H, W) directly to captum.attr.visualization.visualize_image_attr, which expects a 2D (H, W) or 3D (H, W, C) NumPy array.
Resolution: Squeeze the batch dimension and permute the axes using np.transpose(attr.squeeze(0).cpu().detach().numpy(), (1, 2, 0)) before passing the array to the visualization function.

Free alternatives (table)

While vanilla saliency maps provide a fast first-order gradient baseline, other open-source attribution algorithms in Captum offer theoretical advantages for non-linear models. The table below compares free, open-source attribution methods available within the PyTorch and Captum ecosystem.

Attribution Method	Captum Class	Mathematical Foundation	Baseline Required	Computational Overhead
Vanilla Saliency	`captum.attr.Saliency`	First-order gradient of target score w.r.t. input	No	1 forward + 1 backward pass
Integrated Gradients	`captum.attr.IntegratedGradients`	Path integral of gradients along a straight line from baseline to input	Yes (e.g., black or zero tensor)	50–200 forward/backward passes
Input x Gradient	`captum.attr.InputXGradient`	Elementwise product of input features and their gradients	No	1 forward + 1 backward pass
Guided Backpropagation	`captum.attr.GuidedBackprop`	Backpropagates only positive gradients through positive ReLU activations	No	1 forward + 1 backward pass
Layer Grad-CAM	`captum.attr.LayerGradCam`	Gradient-weighted spatial feature pooling in target convolutional layers	No	1 forward + 1 backward pass

FAQ

What is the difference between Saliency Maps and Grad-CAM?

Vanilla saliency maps calculate gradients directly with respect to raw input pixels, producing fine-grained but often noisy pixel-level attributions. In contrast, Grad-CAM (Gradient-weighted Class Activation Mapping) calculates gradients with respect to the activations of the final convolutional layer, producing a coarse, semantically coherent localization heatmap of discriminative regions.

Why does Captum return absolute values of gradients by default?

Captum’s captum.attr.Saliency sets abs=True by default because any input feature with a large gradient—whether positive or negative—substantially alters the target class output. In visual saliency, researchers are typically interested in the magnitude of influence regardless of sign. Setting abs=False returns signed directional gradients.

How does Saliency differ from Integrated Gradients?

Vanilla saliency evaluates the gradient only at the exact point of the input image, making it vulnerable to gradient saturation (where a feature is critical to the prediction but has a flat local gradient). Integrated Gradients addresses gradient saturation by integrating gradients along a linear path from a neutral baseline (such as an all-black image) to the input, satisfying the axioms of Completeness and Implementation Invariance.

Can saliency maps be used for tabular or time-series data?

Yes. The mathematical definition of saliency maps applies to any differentiable model. For tabular features or sequential time-series inputs, captum.attr.Saliency outputs feature-level attribution values indicating the sensitivity of the output prediction to each individual tabular column or time step.

Sources (official docs)

Captum Saliency API Reference. Meta Platforms, Inc. and PyTorch. Read November 9, 2026. https://captum.ai/api/saliency.html
Captum Getting Started and Installation. Meta Platforms, Inc. and PyTorch. Read November 9, 2026. https://captum.ai/docs/getting_started
Captum Visualization Utilities API Reference. Meta Platforms, Inc. and PyTorch. Read November 9, 2026. https://captum.ai/api/utilities.html
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013). Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. arXiv:1312.6034 [cs.CV]. Read November 9, 2026. https://arxiv.org/abs/1312.6034
Captum Integrated Gradients API Reference. Meta Platforms, Inc. and PyTorch. Read November 9, 2026. https://captum.ai/api/integrated_gradients.html

Saliency Maps: PyTorch and Captum Research Guide