Data science & AI · Reference
What is a neural network?
A neural network is a computational model made of interconnected nodes, or artificial neurons, arranged in layers that transform input data into an output by adjusting the strengths of their connections during training.
Neurons, weights and layers
A neural network is built from artificial neurons. Each neuron computes a weighted sum of its inputs, adds a bias, and passes the result through a non-linear activation function such as the ReLU or sigmoid. Neurons are arranged in layers: an input layer receives the data, one or more hidden layers transform it, and an output layer produces the result. The weights on the connections are the network's adjustable parameters; the non-linear activations are what let the network represent complex, non-linear relationships.
How a neural network is trained
Training begins by passing data forward through the network to produce an output, which is compared with the desired output using a loss function.
Backpropagation then computes how each weight contributed to the error, and an optimiser such as gradient descent nudges every weight to reduce that error. Repeating this over many examples gradually fits the network. Backpropagation was popularised for neural networks by Rumelhart, Hinton, and Williams in 1986, and remains the foundation of training today.
Architectures and depth
Networks differ in how their neurons are connected. A simple feedforward network passes data in one direction. Convolutional networks share weights to process grid-like data such as images; recurrent networks have loops to handle sequences; and transformer networks use attention mechanisms. Networks with many hidden layers are called deep, which is the basis of deep learning.
Neural networks in research
Neural networks are loosely inspired by biological neurons but are mathematical function approximators, not models of the brain. In research they are valued for learning patterns directly from data, but their many parameters make them prone to overfitting and hard to interpret. Sound practice includes held-out evaluation, regularisation, and — where it matters — analysing which inputs drive a prediction. Reproducibility requires reporting architecture, hyperparameters, and training procedure.
Key facts
At a glance
- Definition: model of interconnected artificial neurons in layers
- Connection parameters: weights (plus a bias per neuron)
- Non-linearity: provided by activation functions
- Training algorithm: backpropagation with gradient descent
- Backpropagation popularised: Rumelhart, Hinton & Williams, 1986
- Deep network: a network with many hidden layers
Common questions
FAQ
How does a neural network learn?+
A neural network learns by adjusting the weights on its connections. It passes data forward to make a prediction, measures the error, and uses backpropagation with an optimiser to change each weight so the error shrinks, repeating this over many training examples.
What is an activation function?+
An activation function is the non-linear function each neuron applies to its weighted input, such as ReLU or sigmoid. Without non-linear activations, a multi-layer network could only represent linear relationships.
Are artificial neural networks like the human brain?+
Only loosely. They borrow the metaphor of interconnected neurons, but they are mathematical function approximators trained by optimisation, not accurate models of biological brains.
Going deeper
Related on CASRAI
- What is deep learning? →
- What is machine learning? →
- What is a GAN? →
- What is artificial intelligence? →
- Computer science, data science & AI →
Sources
The step most authors miss
Doing CRediT right? Don’t stop at the statement.
A CRediT statement credits you inside one paper. The recognition CRediT was built for happens when those roles are tied to you, persistently. Sign in with your ORCID — free — and claim your CRediT contributions on casrai.org, the home of the standard. They become a verified, portable part of your identity, not a line that disappears into one PDF.
Free: claim your contributions, then export a journal-ready CRediT statement, schema.org structured data, JATS XML, CSV or BibTeX — and preview your public profile. A membership publishes that profile publicly and verifies the journals you serve.
