Written and maintained by CASRAI Editorial Board
Last updated
Image recognition is the task of getting software to say what an image contains. This guide is for researchers, students and lab staff who want to understand the term precisely and run a working pretrained classifier today, using two official open-source libraries (PyTorch torchvision and Keras Applications), then read the output correctly.
Short answer: in current practice, image recognition usually means a convolutional or similar neural network that takes a preprocessed image and returns a probability for each label in a fixed list. The two examples below do exactly that, and each is taken from the library’s own documentation (read 2026-11-07).
What image recognition is: classification versus detection
The term covers more than one task. The ImageNet Large Scale Visual Recognition Challenge (ILSVRC), the benchmark that drove much of the field, defined two of them: object category classification (which categories are present in the image) and object detection (where each object is). The benchmark paper describes a dataset of hundreds of object categories and millions of images, and reports on human-versus-machine accuracy and lessons from annual runs from 2010 (Russakovsky et al., arXiv:1409.0575, read 2026-11-07).
- Classification returns labels with scores for the whole image. The walkthrough below does this.
- Detection returns labelled locations. It needs a detection model, not the classifiers used here.
For the benchmark dataset itself, see the CASRAI guide to ImageNet. For the broader field, see what is machine learning and what is artificial intelligence.
What image recognition is used for in research
In research settings the same recognition step sits inside larger imaging workflows: classifying or segmenting microscopy and medical images, screening images for duplication or manipulation, and extracting features from images for downstream statistics. CASRAI covers examples elsewhere: MONAI (a framework for medical imaging deep learning) and QuPath (digital pathology image analysis). A general ImageNet classifier is a starting point and a teaching tool; research use normally requires a model trained or validated on your own image type.
Install or access (official sources only)
Both libraries ship pretrained models, so no training data is needed to try recognition.
- PyTorch torchvision: install PyTorch by choosing your platform in the selector on the official PyTorch Get Started page. The model API is documented at docs.pytorch.org/vision/stable/models.html.
- Keras Applications: documented at keras.io/api/applications as deep learning models with pretrained weights for prediction, feature extraction and fine-tuning. The weights download automatically the first time you instantiate a model and are stored in
~/.keras/models/.
Step-by-step: classify an image
Option A: PyTorch torchvision (ResNet-50)
- Load the image with
decode_image. - Create the weights object, build the model with those weights, and switch to evaluation mode.
- Use the transforms that belong to those weights to resize and normalise the image, and add a batch dimension.
- Run the model, apply softmax, and look up the winning class name.
from torchvision.io import decode_image
from torchvision.models import resnet50, ResNet50_Weights
img = decode_image("test/assets/encode_jpeg/grace_hopper_517x606.jpg")
weights = ResNet50_Weights.DEFAULT
model = resnet50(weights=weights)
model.eval()
preprocess = weights.transforms()
batch = preprocess(img).unsqueeze(0)
prediction = model(batch).squeeze(0).softmax(0)
class_id = prediction.argmax().item()
score = prediction[class_id].item()
category_name = weights.meta["categories"][class_id]
print(f"{category_name}: {100 * score:.1f}%")Replace the file path with your own image. In the documentation, DEFAULT is described as the best available weights and is currently an alias for IMAGENET1K_V2 (read 2026-11-07).
Option B: Keras Applications (ResNet50)
- Instantiate
ResNet50(weights='imagenet'). - Load the image at 224 by 224 pixels, convert it to an array, and add a batch axis.
- Apply the model’s own
preprocess_input. - Predict, then decode the top results into readable labels.
import numpy as np
import keras
from keras.applications.resnet50 import ResNet50, preprocess_input, decode_predictions
model = ResNet50(weights='imagenet')
img_path = 'elephant.jpg'
img = keras.utils.load_img(img_path, target_size=(224, 224))
x = keras.utils.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)
preds = model.predict(x)
print(decode_predictions(preds, top=3)[0])The Keras documentation shows this pattern with an elephant photograph, and an example result of the form ('n02504013', 'Indian_elephant', 0.82658225). Each tuple is a class ID, a description and a probability.
Output and how to read it
- Probabilities, not certainty. The PyTorch example applies softmax so the scores across all labels sum to one; the printed percentage is the model’s confidence in its single best label, not a measured accuracy.
- The label list is fixed. The labels come from the benchmark’s categories (
weights.meta["categories"]in torchvision,decode_predictionsin Keras). A cell image or a chart will still receive one of those labels, because the model cannot answer outside its list. - Top-1 and top-5. Top-1 accuracy counts a prediction correct when the single highest-scoring label is right. Top-5 counts it correct when the right label is among the five highest. Top-5 is therefore always at least as high as top-1.
| Keras Applications model | Size | Top-1 accuracy | Top-5 accuracy | Parameters | Depth |
|---|---|---|---|---|---|
| ResNet50 | 98 MB | 74.9% | 92.1% | 25.6M | 107 |
Source: Keras Applications documentation (read 2026-11-07). The accuracies are measured on the ImageNet validation set. The same page lists inference time of 58.2 ms per step on CPU and 4.6 ms on GPU, averaged over batches of 32 on an AMD EPYC CPU and a Tesla A100, so compare speeds only on similar hardware.
Troubleshooting
- Accuracy is worse than expected. Use the preprocessing that matches the weights. The torchvision documentation notes that wrong preprocessing can reduce accuracy; use
weights.transforms()there andpreprocess_inputin Keras rather than writing your own resize and normalise steps. - Shape error about dimensions. Models expect a batch. The examples add one with
unsqueeze(0)in PyTorch andnp.expand_dims(x, axis=0)in Keras. - Unstable or odd results in PyTorch. Call
model.eval()before inference, as the example does. - First Keras run is slow or needs a network connection. The pretrained weights are downloaded on first instantiation and cached in
~/.keras/models/. - Confident but wrong labels on research images. The model only knows the benchmark’s categories. Fine-tuning on your own labelled images, or a domain-specific model such as those in MONAI, is the usual route.
Free alternatives
| Library | What it provides | Pretrained weights | Official documentation |
|---|---|---|---|
| PyTorch torchvision | Model API with matching transforms and category metadata | Selected through a weights object, for example ResNet50_Weights.DEFAULT |
torchvision models |
| Keras Applications | Models for prediction, feature extraction and fine-tuning, with decode_predictions |
Downloaded when you pass weights='imagenet' |
keras.io/api/applications |
FAQ
What is the difference between image recognition and image classification?
Classification is one image recognition task: assign labels to a whole image. Recognition in the wider sense also includes detection, which locates each object, as defined in the ILSVRC benchmark tasks.
Can I try image recognition for free?
Yes. The two walkthroughs above use the free open-source libraries PyTorch torchvision and Keras Applications with their published pretrained weights, and need only your own image file.
Is image recognition the same as machine learning?
It is an application of machine learning. Modern image recognition uses trained neural networks; see what is machine learning for the general field.
Can a pretrained model identify anything in my images?
No. It can only return labels from the category list it was trained on. For specialised images, fine-tune or use a domain model.
Sources
- PyTorch, torchvision models documentation: docs.pytorch.org/vision/stable/models.html (read 2026-11-07).
- Keras, Keras Applications: keras.io/api/applications (read 2026-11-07).
- Russakovsky, O. et al., ImageNet Large Scale Visual Recognition Challenge: arXiv:1409.0575 (read 2026-11-07).
