Computer vision is the field of AI that teaches computers to understand what is in images and video. When your phone unlocks with your face, a photo app finds pictures of your friend, or a bank app reads a cheque, computer vision is at work. This guide explains the basic tasks, how they work and where they are used.
How a computer "sees"
To a computer, an image is a grid of numbers. A colour photo has three layers (red, green and blue), and each pixel in each layer holds a value, typically from 0 to 255. Computer vision turns these numbers into meaning, such as "this region is a face" or "this is the number 7".
Early methods used hand-designed filters to find edges and corners. Since about 2012, convolutional neural networks (CNNs) and more recently vision transformers have learned useful visual features automatically from many examples.
The main tasks
Image classification
Decide what the whole image shows: cat, dog, pizza, X-ray with or without a suspicious area.
Object detection
Find objects and draw boxes around them, with a label for each. A car's camera system might detect pedestrians, vehicles and traffic signs in the same frame.
Image segmentation
Label every pixel. Semantic segmentation marks "road", "sky" and "building"; instance segmentation separates individual objects of the same type.
Face detection and recognition
Detection finds where faces are. Recognition tries to identify who they belong to, which raises privacy concerns.
Optical character recognition (OCR)
Read printed or handwritten text from images and scanned documents.
Pose estimation
Locate key points on the body, such as shoulders, elbows and knees, used in fitness apps and animation.
Depth and 3D understanding
Estimate distance or build 3D models from one or more images.
Video analysis
Track objects over time, recognise actions or detect unusual events.
How a classification model learns
- Collect thousands of labelled images.
- Resize and normalise them so they have a consistent format.
- Train a CNN: early layers learn simple patterns like edges, middle layers learn textures and parts, and deeper layers learn whole objects.
- Evaluate on images the model has never seen.
Often you do not train from scratch. Transfer learning starts from a model pretrained on a large dataset and fine-tunes it on your smaller dataset, which saves time and data.
Real-world uses
- Healthcare: helping radiologists spot patterns in scans
- Agriculture: detecting plant disease from leaf photos or monitoring fields from drones
- Retail: counting shelf stock and scanning products
- Manufacturing: inspecting items for defects
- Transport: driver assistance, number plate reading and traffic counting
- Security and attendance: face-based check-in systems, which must be used carefully and legally
- Documents: scanning IDs, invoices and forms
- Accessibility: describing images for visually impaired users
Try it: image classification in Python
This example uses a pretrained model from PyTorch's torchvision to label a photo.
import torch
from torchvision import models
from torchvision.models import ResNet50_Weights
from PIL import Image
weights = ResNet50_Weights.DEFAULT
model = models.resnet50(weights=weights).eval()
preprocess = weights.transforms()
img = Image.open("photo.jpg").convert("RGB")
batch = preprocess(img).unsqueeze(0)
with torch.no_grad():
probs = model(batch).softmax(dim=1)[0]
top = probs.topk(3)
for score, idx in zip(top.values, top.indices):
print(f"{weights.meta['categories'][idx]}: {score:.1%}")The model prints its three best guesses with confidence scores. Remember that the labels come from a fixed list of categories, so it cannot name things outside that list.
A simple project to practise
A good first project is a plant-leaf or product-photo classifier. Collect 50 to 100 photos for each of two or three categories using your phone, split them into training and test folders, and fine-tune a small pretrained network. Measure accuracy on the test folder, then look at the images it gets wrong. You will quickly learn why lighting, background and photo angle matter as much as the model itself, and why data quality is usually the biggest lever.
Challenges and limits
- Lighting, angle and occlusion change how objects look.
- Dataset bias: if training images do not cover different skin tones, regions or conditions, accuracy will be uneven.
- Adversarial examples: small, deliberate changes can fool a model.
- Privacy and consent: cameras and recognition systems can be used to track people. Laws and ethics matter.
- Labelling cost: high-quality labelled data takes time and money to produce.
Frequently Asked Questions
Is computer vision the same as image processing?
Image processing changes or enhances images (resizing, blurring, sharpening). Computer vision tries to extract meaning from them. They often work together.
Do I need a GPU to learn computer vision?
Not to start. You can use small models, pretrained networks or free cloud notebooks. A GPU becomes important for training large models.
Which library should a beginner use?
OpenCV is great for basic image operations, and PyTorch or TensorFlow for deep learning.
Is face recognition accurate?
Accuracy varies by system, data and conditions. Because errors can have serious consequences, such systems should be used with safeguards and consent.
Conclusion
Computer vision gives machines the ability to classify, locate and interpret what appears in images and video. With pretrained models and open-source libraries, beginners can build useful tools today, as long as they test fairly and respect privacy.