Written and maintained by CASRAI Editorial Board
Published · Last updated
On this page
A neural network is built by stacking layers, and the main types of neural networks differ in which layers they stack and what shape of data those layers expect. This guide sorts the common types (fully connected, convolutional, recurrent, attention-based) by the layer you would actually call in the current Keras and PyTorch documentation, so the choice of architecture is tied to the input data you hold. The layer details below were last checked against those documents on 2026-10-29.
Ask CASRAI · free to tryQuestion about this guide? Ask CASRAI answers with linked sources. 2 free questions a day, no account.
What it is used for in research
Researchers reach for a particular neural network type because their data has a particular shape: a grid of pixels, a sequence of measurements over time, or a set of tokens. The library documentation makes that link explicit, because each layer states the input shape it accepts. For the broader field around these models, see What Is Machine Learning? and What Is Artificial Intelligence?. For a model family built on attention, see the dictionary entry for a large language model.
Types of neural networks and the layers that implement them
The Keras layers index groups its built-in layers into families. Four of them correspond to the architectures most often named in introductions to the subject.
Fully connected (feedforward) networks
The simplest type stacks fully connected layers. In Keras these sit in the core layers family, which lists Dense, EinsumDense, Embedding, Activation, Masking, Lambda, Identity and Input. Dense is the layer to start from when no spatial or sequential structure needs to be exploited.
Convolutional neural networks (CNNs)
Convolutional networks use convolution layers. Keras lists Conv1D, Conv2D, Conv3D, SeparableConv1D, SeparableConv2D, DepthwiseConv1D, DepthwiseConv2D and the transposed variants Conv1DTranspose, Conv2DTranspose and Conv3DTranspose. The number in the name is the number of spatial or temporal dimensions the kernel slides over. The Conv2D page describes a layer that creates “a convolution kernel that is convolved with the layer input over a 2D spatial (or temporal) dimension (height and width) to produce a tensor of outputs.”
Recurrent neural networks (RNNs), LSTM and GRU
Recurrent networks process an input one timestep at a time. The Keras recurrent family lists LSTM, GRU, SimpleRNN, the wrappers Bidirectional and TimeDistributed, and the convolution-recurrent hybrids ConvLSTM1D, ConvLSTM2D and ConvLSTM3D. The Keras LSTM page cites Hochreiter 1997 for the long short-term memory design. PyTorch’s torch.nn.LSTM is described as applying “a multi-layer long short-term memory (LSTM) RNN to an input sequence.”
Attention-based networks and the Transformer
The Transformer departed from both of the types above. Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser and Polosukhin, in “Attention Is All You Need” (arXiv:1706.03762, first submitted 12 June 2017), propose a network “based solely on attention mechanisms, dispensing with recurrence and convolutions entirely.” Keras provides the attention family as MultiHeadAttention, GroupQueryAttention, Attention and AdditiveAttention. Its MultiHeadAttention page says the layer implements multi-headed attention as described in that paper, and that if query, key and value are the same, the layer performs self-attention. Related research-side vocabulary: mixture-of-experts (MoE).
Match the data shape to the layer
The table follows the Keras layer documentation. It states what each layer accepts, not which one performs best.
| Layer family | Keras layers (examples) | Input the documentation specifies |
|---|---|---|
| Fully connected | Dense, Embedding |
Set out on the Dense and Embedding pages in the core layers family |
| Convolutional | Conv1D, Conv2D, Conv3D |
Conv2D: a 4D tensor, (batch_size, height, width, channels) when data_format is “channels_last” |
| Recurrent | LSTM, GRU, SimpleRNN |
LSTM: a 3D tensor with shape (batch, timesteps, feature) |
| Attention | MultiHeadAttention, Attention |
query, key and value tensors; same tensor for all three gives self-attention |
Install or access (official source only)
The Keras getting-started page gives the install command pip install --upgrade keras. Keras 3 also needs a backend framework: JAX, TensorFlow or PyTorch. The same page warns that if you install TensorFlow 2.15, you should reinstall Keras 3 afterwards, because tensorflow==2.15 overwrites it.
For PyTorch, the install page asks you to select your preferences and run the command it generates. The layer reference for torch.nn.LSTM is the torch.nn.LSTM documentation.
Step-by-step: choose and build a layer stack
- Write down the shape of one training example. A height by width by channels array points to the convolution family; a (timesteps, feature) array points to the recurrent or attention families.
- Open the Keras layer page for the family and read its input shape. For
Conv2Dwith data_format “channels_last” the input is (batch_size, height, width, channels) and the output is (batch_size, new_height, new_width, filters). - Create the layer with its required arguments.
Conv2Drequiresfiltersandkernel_size;LSTMrequiresunits;MultiHeadAttentionrequiresnum_headsandkey_dim. - Run a small tensor through the layer and print the output shape before building the full model. The Keras documentation does exactly this:
x = np.random.rand(4, 10, 10, 128) y = keras.layers.Conv2D(32, 3, activation='relu')(x) print(y.shape) # (4, 8, 8, 32) - In PyTorch, set the three
torch.nn.LSTMsizes that define the recurrent stack:input_sizeis “the number of expected features in the input x”,hidden_sizeis “the number of features in the hidden state h”, andnum_layersis the “number of recurrent layers”, where setting num_layers=2 means stacking two LSTMs together (default 1).
Output and how to read it
The first output to read is the tensor shape, because it tells you whether the layer did what you intended. In the Conv2D example above, a 10 by 10 input with a 3 by 3 kernel and the default padding “valid” (“no padding”) comes out 8 by 8, and the last dimension becomes the number of filters (32). With padding “same”, the layer pads evenly left/right or up/down of the input instead. For LSTM the documented input is (batch, timesteps, feature); an optional mask of shape (samples, timesteps) marks timesteps to ignore, where a True entry means the timestep is used.
Troubleshooting
- ValueError from Conv2D. The page lists a ValueError when both strides greater than 1 and dilation_rate greater than 1 are set, and notes that strides above 1 are incompatible with dilation_rate above 1.
- Convolution results differ across backends. The Conv2D page notes that results are identical across backends up to 1e-7 precision in float32 in general, but Conv2D can show larger variation, especially with large inputs or kernel sizes, or when switching backends (TensorFlow vs JAX) or hardware. Record the backend and hardware with your results; see computational reproducibility and sharing a reproducible conda environment.
- LSTM runs slowly on a GPU. The Keras LSTM page says the layer uses a fast cuDNN implementation under the TensorFlow backend only if the arguments meet its requirements, which include activation equal to tanh, recurrent_activation equal to sigmoid and recurrent_dropout equal to 0. Read the full list on that page.
- Keras behaves unexpectedly after installing TensorFlow 2.15. Reinstall Keras 3, as the getting-started page instructs.
Free alternatives
| Library | Where the layers are documented | Recurrent layer | Attention layer |
|---|---|---|---|
| Keras 3 | keras.io/api/layers (core, convolution, recurrent, attention families) | keras.layers.LSTM |
keras.layers.MultiHeadAttention |
| PyTorch | docs.pytorch.org torch.nn reference | torch.nn.LSTM |
see the torch.nn reference |
Keras 3 can run on a PyTorch backend, so the two are not mutually exclusive.
FAQ
What are the main types of neural networks?
Fully connected, convolutional, recurrent (including LSTM and GRU) and attention-based (the Transformer). Keras documents each as a layer family: core, convolution, recurrent and attention layers.
What replaced recurrence in the Transformer?
Attention. The 2017 paper describes a network based solely on attention mechanisms, dispensing with recurrence and convolutions entirely.
Which layer do I use for image data?
The convolution family, for example Conv2D, whose documented input is a 4D tensor of batch size, height, width and channels (with channels_last).
Which layer do I use for sequences?
The recurrent family (LSTM, GRU, SimpleRNN) takes (batch, timesteps, feature); the attention family is the alternative the Transformer introduced.
Sources
All sources below were checked on 2026-10-29.
- Keras API layers index: https://keras.io/api/layers/
- Keras Conv2D layer: https://keras.io/api/layers/convolution_layers/convolution2d/
- Keras LSTM layer: https://keras.io/api/layers/recurrent_layers/lstm/
- Keras MultiHeadAttention layer: https://keras.io/api/layers/attention_layers/multi_head_attention/
- Getting started with Keras: https://keras.io/getting_started/
- PyTorch documentation, torch.nn.LSTM: https://docs.pytorch.org/docs/stable/generated/torch.nn.LSTM.html
- PyTorch, Start Locally: https://pytorch.org/get-started/locally/
- Vaswani et al., “Attention Is All You Need”: https://arxiv.org/abs/1706.03762







