Data science & AI · Reference
What is deep learning?
Deep learning is a branch of machine learning that uses neural networks with many layers to learn increasingly abstract representations of data, enabling strong performance on tasks such as image recognition and language understanding.
What makes learning "deep"
The depth in deep learning refers to the number of layers in a neural network through which data is transformed. Each successive layer builds a more abstract representation: in image tasks, early layers may respond to edges, later layers to shapes, and deeper layers to whole objects. This is representation learning — the network discovers useful features automatically rather than relying on humans to design them, which is the key practical advantage over earlier machine-learning methods on perception and language.
How deep networks are trained
Deep networks are trained by backpropagation, an algorithm that computes how each parameter contributes to the output error and adjusts it accordingly, combined with gradient-descent optimisation over many examples.
Common architectures include convolutional neural networks for images, recurrent networks for sequences, and the transformer for language and beyond. Training deep models typically requires large labelled or self-supervised datasets and substantial compute, often on graphics processing units (GPUs).
Why deep learning scaled
Deep neural networks are an old idea, but they became dominant only around 2012, when a deep convolutional network sharply improved image-classification accuracy on the ImageNet benchmark. Three factors converged: far larger labelled datasets, parallel hardware (GPUs) fast enough to train large models, and algorithmic refinements such as better activation functions and regularisation. This combination of data plus compute let deep models surpass methods that depended on hand-crafted features.
Deep learning in research
Deep learning is now a standard research tool in fields from structural biology to astronomy, where it is used for pattern recognition in large datasets. Methodologically, deep models pose challenges: they are data-hungry, can be opaque ("black box"), and may fail unpredictably outside their training distribution. Responsible use involves rigorous held-out evaluation, attention to dataset bias, and — where decisions matter — interpretability analysis and uncertainty estimation. Results are reported as empirical findings to be independently reproduced.
Key facts
At a glance
- Field: subfield of machine learning
- Core unit: multi-layer (deep) neural networks
- Key idea: representation (feature) learning from raw data
- Training algorithm: backpropagation with gradient descent
- Breakthrough: ImageNet image classification, 2012
- Enablers: large datasets plus GPU compute
Common questions
FAQ
What is the difference between machine learning and deep learning?+
Deep learning is a subset of machine learning that specifically uses many-layered neural networks and learns features automatically. Other machine-learning methods often rely on hand-engineered features and simpler models.
Why did deep learning become so successful?+
Deep learning scaled because three things converged after around 2012: much larger datasets, fast parallel hardware (GPUs), and algorithmic improvements. Together these let deep networks outperform methods using hand-crafted features on vision and language tasks.
What is representation learning?+
Representation learning is the automatic discovery of useful features from raw data. In a deep network, each layer learns a more abstract representation, so engineers do not have to design features by hand.
The step most authors miss
Doing CRediT right? Don’t stop at the statement.
A CRediT statement credits you inside one paper. The recognition CRediT was built for happens when those roles are tied to you, persistently. Sign in with your ORCID — free — and claim your CRediT contributions on casrai.org, the home of the standard. They become a verified, portable part of your identity, not a line that disappears into one PDF.
Free: claim your contributions, then export a journal-ready CRediT statement, schema.org structured data, JATS XML, CSV or BibTeX — and preview your public profile. A membership publishes that profile publicly and verifies the journals you serve.
