Tag: supervised learning

  • Supervised vs Unsupervised Learning Explained

    Supervised learning is a machine-learning paradigm in which a model is trained on labelled examples — inputs paired with known correct outputs — so that it can predict the output for new, unseen inputs. Its counterpart, unsupervised learning, works with unlabelled data and seeks to discover structure, patterns or groupings without being told what the “right” answer is. The presence or absence of labels is the defining distinction between the two.

    Both belong to the wider field of machine learning, and choosing between them depends on whether you have labelled data and what question you are asking. A third paradigm, reinforcement learning, sits apart from both.

    Supervised learning: learning from labels

    In supervised learning, each training example carries a label: the email is “spam” or “not spam”, the image is “tumour” or “benign”, the house has a known sale price. The algorithm learns a mapping from inputs (features) to outputs (labels), and its performance is judged by how accurately it predicts labels for data it has not seen. Two main task types exist. Classification predicts a discrete category (spam or not, species A, B or C). Regression predicts a continuous quantity (a price, a temperature, a concentration).

    Common supervised methods include linear and logistic regression, decision trees, support vector machines and neural networks. The key practical requirement — and often the key cost — is obtaining enough accurately labelled data, which may require expert annotation.

    Unsupervised learning: finding structure

    Unsupervised learning has no labels. Instead, the algorithm looks for inherent structure in the data. Clustering groups similar items together — for example, segmenting samples into subtypes without prior categories, using methods such as k-means or hierarchical clustering. Dimensionality reduction, such as principal component analysis, compresses many variables into fewer while preserving structure, aiding visualisation and downstream analysis. Because there is no ground-truth label, evaluating unsupervised results is harder and often relies on domain judgement.

    Reinforcement learning: a third paradigm

    Reinforcement learning differs from both. Here an agent learns by interacting with an environment, taking actions and receiving rewards or penalties, and gradually improving a policy to maximise cumulative reward. It is neither labelled-example learning nor pure pattern discovery; it learns from consequences over time. Reinforcement learning underlies advances in game-playing and robotics, and is noted here for completeness rather than treated in depth.

    Comparing the paradigms

    Feature Supervised learning Unsupervised learning
    Data Labelled (input–output pairs) Unlabelled
    Goal Predict known outputs Discover hidden structure
    Main tasks Classification, regression Clustering, dimensionality reduction
    Example methods Logistic regression, decision trees, SVMs k-means, hierarchical clustering, PCA
    Evaluation Accuracy against held-out labels Often qualitative; needs domain judgement

    Research uses and good practice

    In research, supervised learning suits prediction and classification where labelled outcomes exist — diagnosing from images, predicting properties from features. Unsupervised learning suits exploration — finding subgroups, detecting anomalies, reducing dimensionality before further analysis. The two are often combined: unsupervised methods can pre-process or explore data that a supervised model then uses.

    Whichever paradigm is used, the outputs are research outputs that require careful reporting: how data were labelled or collected, how the model was validated, and how results were evaluated. Sharing data, code and methods using consistent terminology supports reproducibility, and our guidance for authors covers documenting such computational work. For foundational background, see our overview of machine learning concepts and methods and the explainer on neural networks and deep learning.

    Frequently asked questions

    What is the main difference between supervised and unsupervised learning?

    Labels. Supervised learning trains on labelled examples — inputs paired with known correct outputs — to make predictions, whereas unsupervised learning works with unlabelled data to discover structure such as clusters or lower-dimensional representations without any predefined answer.

    What are classification and regression?

    They are the two main supervised tasks. Classification predicts a discrete category, such as spam or not spam. Regression predicts a continuous value, such as a price or temperature. Both learn a mapping from input features to known output labels.

    Where does reinforcement learning fit?

    It is a separate, third paradigm. An agent learns by acting in an environment and receiving rewards or penalties, improving its policy over time to maximise cumulative reward. It learns from consequences rather than from labelled examples or pure pattern discovery.

    Can the two approaches be combined?

    Yes, frequently. Unsupervised methods such as clustering or PCA can explore or pre-process data, and a supervised model can then make predictions from the result. Many research pipelines use both, so they are complementary rather than mutually exclusive.

  • What Is Machine Learning? Concepts and Methods

    Machine learning (ML) is the subfield of artificial intelligence concerned with algorithms that learn patterns from data and improve at a task with experience, rather than being explicitly programmed with rules. Instead of an engineer writing the logic, the engineer specifies a model and an objective, and the model adjusts its internal parameters to fit examples. The central scientific question is not whether a model fits the data it has seen, but whether it generalises to data it has not.

    Features, labels and the learning objective

    A machine-learning problem is usually framed in terms of features (the input variables describing each example) and, for supervised tasks, labels (the target output to be predicted). For a model predicting house prices, features might include floor area and location, and the label is the sale price. Learning means searching for model parameters that minimise a loss function measuring the gap between predictions and the truth.

    Machine learning is one paradigm within the broader discipline described in our explainer on artificial intelligence definition and history. Where symbolic AI encodes knowledge by hand, ML infers it statistically from examples.

    The three main paradigms

    Machine learning is conventionally divided into three families, distinguished by what kind of feedback the algorithm receives.

    Type Data used Goal Typical examples
    Supervised learning Labelled examples (features + targets) Predict a label for new inputs Classification, regression
    Unsupervised learning Unlabelled data Discover structure Clustering, dimensionality reduction
    Reinforcement learning Rewards from an environment Learn a policy that maximises long-term reward Control, game playing, sequential decisions

    Supervised learning trains on examples paired with correct answers and learns to predict those answers for unseen inputs; classification predicts categories and regression predicts continuous values. Unsupervised learning works with unlabelled data and seeks hidden structure, for instance grouping similar items (clustering) or compressing many variables into a few (dimensionality reduction). Reinforcement learning learns by trial and error: an agent takes actions, receives rewards or penalties, and gradually improves a policy that maximises cumulative reward.

    The train, validation and test split

    To estimate how well a model will generalise, data is partitioned into three disjoint sets. The training set is used to fit the model’s parameters. The validation set is used to tune choices the algorithm does not learn directly, such as model size or learning rate (the hyperparameters), and to compare candidate models. The test set is held back and used only once, at the end, to give an unbiased estimate of performance on unseen data.

    The cardinal rule is that the test set must not influence training or model selection. Repeatedly peeking at the test set leaks information and inflates reported performance, a subtle but common source of irreproducible results. We discuss safeguards at length in our guide to reproducibility of machine learning research.

    Overfitting and generalisation

    Overfitting occurs when a model learns the noise and idiosyncrasies of its training data rather than the underlying pattern, performing well on training examples but poorly on new ones. The opposite failure, underfitting, occurs when a model is too simple to capture the real structure. The art of machine learning lies in finding the balance, the so-called bias-variance trade-off, that yields the best generalisation to unseen data. Techniques such as regularisation, early stopping and cross-validation all serve this goal.

    Why method reporting matters

    Because performance depends so heavily on the data split, the loss function and the hyperparameters, a machine-learning result is only as credible as its reporting. Standardised vocabulary, captured in the casrai.org research dictionary, helps authors describe their methods consistently, and contribution frameworks such as CRediT help assign credit for the data, software and analysis work involved. Coverage of these issues continues in our AI and ML research outputs category.

    Frequently asked questions

    What is the difference between supervised and unsupervised learning?

    Supervised learning trains on data with known correct answers (labels) and predicts those answers for new inputs. Unsupervised learning works with unlabelled data and instead discovers structure, such as clusters or compressed representations, without a target to predict.

    Why split data into training, validation and test sets?

    The training set fits the model, the validation set tunes hyperparameters and compares models, and the held-out test set gives an unbiased estimate of real-world performance. Mixing these roles inflates results and undermines reproducibility.

    What is overfitting?

    Overfitting is when a model memorises the noise in its training data and therefore performs well on that data but poorly on new examples. The goal of machine learning is generalisation, not memorisation.

    Is machine learning the same as artificial intelligence?

    No. Machine learning is a subfield of artificial intelligence focused on learning from data. AI also includes symbolic reasoning, search and planning that do not learn from examples.