Dictionary domainTrack C
AI and ML research outputs
Model cards, system cards, datasheets, benchmarks, evaluation suites.
For implementers
Operational deployment checklist for AI and ML research outputs: prerequisites, five deploy steps, integration notes for Pure, Symplectic Elements, Worktribe, DSpace, and more, plus the pitfalls that recur in the field.
Terms in this domain
51 terms
EU AI Act Article 53 (General-Purpose AI Model Obligations)
Article 53 of the EU AI Act (Regulation (EU) 2024/1689) sets the baseline obligations that apply to every provider of a general-purpose AI (GPAI) model placed on the EU market -- a model trained on large amounts of data using self-supervision at scale, showing significant generality, and capable of competently performing a wide range of distinct tasks (Article 3(63)). A GPAI provider must, at minimum: (1) draw up and keep current technical documentation of the model per Annex XI, available on request to the AI Office and national authorities; (2) draw up and make available to downstream AI-system providers the information and documentation set out in Annex XII needed to understand the model's capabilities and limitations; (3) put in place a policy to comply with EU copyright law, including respecting rightsholders' text-and-data-mining opt-outs reserved under Article 4(3) of Directive (EU) 2019/790; and (4) draw up and publish a sufficiently detailed summary of the content used to train the model, following a template the AI Office provides. These obligations became applicable on 2 August 2025 (Article 113(b)). Models released under a genuinely free and open-source licence, with publicly available weights, architecture, and usage information, are exempt from obligations (1) and (2) above -- but never from the copyright-policy or training-data-summary duties, and never at all if the model is also classified as carrying 'systemic risk.'
EU AI Act Article 6 (High-Risk Classification Rules)
Article 6 of the EU AI Act (Regulation (EU) 2024/1689) is the classification test that determines whether an AI system counts as 'high-risk.' An AI system is high-risk if either: (1) under Article 6(1), it is a safety component of, or is itself, a product covered by EU harmonisation legislation listed in Annex I and that product requires third-party conformity assessment; or (2) under Article 6(2), it falls into one of the eight functional use-case categories listed in Annex III (biometrics; critical infrastructure; education/vocational training; employment/worker management; essential private and public services; law enforcement; migration/asylum/border control; administration of justice and democratic processes). Article 6(3) provides a narrow, documented exception for Annex III systems that perform only a narrow procedural task, improve a completed human activity's result, detect patterns without replacing human assessment, or perform preparatory tasks -- unless the system profiles natural persons, in which case it is always high-risk. Classification under Article 6 is the trigger, not the substance: it determines whether the Title III, Chapter 2 obligations (risk management, data governance, technical documentation, human oversight, and more) apply at all.
NEJM AI
NEJM AI is a peer-reviewed monthly journal published by NEJM Group (a division of the Massachusetts Medical Society, the same publisher as the flagship New England Journal of Medicine) that publishes original research, reviews, perspectives, and policy analysis on the application of artificial intelligence and machine learning to medicine and health care. It launched in December 2023, with its inaugural issue in January 2024. NEJM AI is editorially and operationally distinct from NEJM itself: it has its own submission portal (ai.nejm.org), its own editorial board and peer-review process, and a narrower topical scope than the flagship journal's general internal medicine remit. A manuscript is within NEJM AI's scope if its central contribution is an AI/ML method, dataset, or system applied to a biomedical or clinical problem, or an analysis of AI's technical, regulatory, ethical, or health-system implications — not simply a clinical study that happens to use software.
National AI Research Resource (NAIRR)
The National AI Research Resource (NAIRR) is a US federal initiative -- currently operating as the NAIRR Pilot -- that provides researchers, educators, and students with shared access to AI computational infrastructure (supercomputing and cloud allocations), AI-ready datasets, pretrained models, and educational resources, rather than direct cash funding. It is led by the National Science Foundation (NSF) in partnership with more than a dozen other federal agencies, including the Department of Energy (DOE) and National Institutes of Health (NIH), who co-lead a security-focused 'NAIRR Secure' track for sensitive data. Eligible US-based researchers, educators, and graduate students (with a faculty sponsor) apply for time-limited resource allocations -- typically a 3-month Start-Up award or a 12-month Research award -- through a competitive, proposal-based process rather than an open-access account. A project 'uses NAIRR' when it draws on the pilot's pooled portfolio of partner-provided compute, storage, or model resources under an awarded allocation; a project that purchases or funds its own AI hardware, or that uses a single university's internal HPC cluster with no NAIRR allocation, is not a NAIRR activity even if the research itself is AI-related.
Scite (Smart Citations)
Scite is a commercial citation-analysis platform whose core feature, Smart Citations, uses a trained natural-language-processing (deep learning) model to classify each individual citing statement of a paper as supporting, contrasting, or mentioning the claim it cites, and displays the actual citation-context sentence alongside that classification. A citation is a Scite Smart Citation, specifically, when it comes with (a) the excerpted sentence from the citing paper that references the cited work, and (b) a supporting/contrasting/mentioning label assigned to that specific citing statement -- not just a tally of how many times the work has been cited.
EU AI Act Article 10 (Data and Data Governance)
Article 10 of the EU AI Act (Regulation (EU) 2024/1689), in Title III, Chapter 2, sets the data-quality and data-governance obligations for the training, validation, and testing datasets used to build high-risk AI systems. A system falls under Article 10 only after it has already met the Act's separate 'high-risk' classification test (an Annex III listed use case, or a safety component covered by Annex I sectoral product law). For a high-risk system built using techniques that involve training models on data, Article 10(2) requires documented data-governance practices covering collection and origin, preparation operations (annotation, labelling, cleaning, updating, enrichment), bias examination, bias-mitigation measures, and identification of data gaps; Article 10(3)-(4) require the training, validation, and testing datasets to be relevant, sufficiently representative, and to the best extent possible free of errors and complete for the system's intended purpose and deployment context. For a high-risk system that does not use such training techniques, Article 10(6) narrows the same requirements to the testing dataset only. Article 10(5) separately permits limited, safeguarded processing of special-category personal data solely to detect and correct bias. It does not apply to AI systems or models developed and used solely for scientific research and development prior to being placed on the market or put into service, per the Article 2(6)/2(8) research exemption.
Consensus (AI Academic Search Engine)
Consensus is a named AI-powered academic search engine (built by the company Consensus, at consensus.app) that retrieves peer-reviewed papers relevant to a natural-language research question and generates a synthesis of what the retrieved literature says, rather than returning a plain ranked list of results. For questions phrased as a yes/no/maybe claim, it additionally displays a 'Consensus Meter' -- a visual indicator of how the retrieved papers' findings line up (agree, disagree, or mixed) on that specific claim, generated from the paper set Consensus itself retrieved and summarized. It is a specific product in the 'literature summarization and evidence-synthesis' category, distinct from general-purpose AI chatbots (which do not search a dedicated indexed academic corpus or cite retrieved papers by default) and from citation-graph or general-purpose scholarly search tools such as Semantic Scholar (which surface and rank papers but do not generate a claim-level agreement synthesis across them).
AI Research Tool
An AI research tool is software that applies machine learning — typically a large language model (LLM), an embedding-based semantic search index, or both — to a specific stage of the research workflow: finding and screening literature, extracting or summarizing data from papers, mapping citation relationships, drafting or revising manuscript text, or analyzing research data. "AI research tool" is a category label, not the name of any single product: it covers named tools with genuinely different scopes (a citation-mapping tool like Connected Papers does not do what a writing-assistance tool like Paperpal does), and a page or citation that treats it as one interchangeable thing is usually mis-scoped. What makes a tool an instance of this category, rather than a general-purpose AI assistant that happens to get used for research, is that it is built or marketed specifically around a research task — an academic search index, a citation graph, a manuscript-formatting model trained on published literature — rather than being a general chat interface pointed at an arbitrary prompt.
Synthetic benchmark
A benchmark whose evaluation items are wholly or partially generated by another model or procedural method, rather than collected from natural human-produced sources, used to probe specific capabilities or to scale evaluation cheaply.
RLHF (Reinforcement Learning from Human Feedback)
A training methodology in which a language model is fine-tuned using a reward signal derived from human preferences over pairs (or larger sets) of candidate model outputs, typically by first training a reward model and then optimising the policy against it via PPO or a related algorithm.
Constitutional AI (concept)
A training methodology in which a model is trained to align its outputs with a written set of principles ('a constitution'), with the model itself used to critique and revise candidate responses against those principles in place of direct human feedback at scale.
Prompt injection
An attack on a language-model-based system in which adversarial instructions, embedded in untrusted input (a document, web page, tool output, image), cause the model to act in ways that diverge from its developer's or user's intent.
Jailbreak (LLM)
A prompt or interaction pattern that causes a language model to bypass its safety training and produce outputs the model was tuned to refuse, such as harmful instructions, restricted content, or violations of provider policy.
Red-teaming
The practice of deliberately adversarial testing of an AI system by skilled testers attempting to elicit failures, unsafe outputs, or policy violations, in order to discover weaknesses before deployment.
AI safety case
A structured, evidence-based argument that an AI system is acceptably safe to deploy in a defined context, modelled on safety cases from established engineering disciplines (nuclear, aviation, medical devices).
AI evaluation card
A structured documentation artefact specifically describing an evaluation of an AI system, separate from the model card, including the evaluation methodology, datasets, metrics, results, and known limitations of the evaluation itself.
Reproducible AI experiment
An AI experiment for which sufficient artefacts and metadata are released (data, code, seed, environment, hyperparameters, training procedure) that an independent investigator can re-run it and obtain numerically equivalent or statistically indistinguishable results.
Open-source model (criteria)
A model meeting the criteria articulated by the Open Source Initiative's Open Source AI Definition: open data information, open code, and open weights, with each released under terms compatible with the OSI's freedoms to use, study, modify, and share.
Open weights model
A model whose trained parameter values are publicly released and downloadable, typically under a named licence, distinct from but often described as 'open' even when training data and code are not released.
Model weight licence
The licence terms governing the use, modification, and redistribution of a model's trained weights, which may differ from the licence on the training code and the licence on the training data.
Model checkpoint
A saved snapshot of a model's parameters (and optionally optimiser state) at a specific point in training, identified by a step number or version tag and serialised to a file format such as safetensors or .pt.
Model evaluation suite
A defined collection of benchmarks, tasks, and metrics, with standardised prompting and decoding rules, used to characterise a model's capabilities and behaviour across a range of dimensions.
Model fine-tune lineage
The specific portion of model lineage that records the sequence of fine-tuning operations applied to a base model: dataset, method (SFT, DPO, RLHF, LoRA), hyperparameters, and resulting checkpoint identifier.
Model lineage
The chain of provenance for a model recording its base model, the fine-tuning datasets and procedures applied, and any further derivatives, such that any deployed model can be traced back to its constituent training operations.
Inference carbon footprint
The greenhouse-gas emissions associated with serving inference requests from a deployed model, typically expressed per-request (e.g., gCO2e per query) or in aggregate (kgCO2e per month).
Training carbon footprint
The total greenhouse-gas emissions, expressed in kilograms or tonnes of CO2-equivalent, attributable to training a machine-learning model, estimated from energy consumption and the carbon intensity of the electricity supply.
Compute (FLOPs estimate)
The total floating-point operations consumed by training a model, conventionally reported as a single number (e.g., 3.0 x 10^25 FLOPs) used as a regulatory and scientific proxy for training-run scale.
Training data composition
The mixture of data sources, by domain, language, modality, and provenance, used to train a model, including the proportions and any filtering or deduplication applied.
Parameter count
The total number of learnable scalar weights in a machine-learning model, conventionally reported as a count (e.g., 7B = 7 x 10^9 parameters) and disclosed as a basic model metadata field.
Mixture-of-experts (MoE)
A neural-network architecture in which a learned router directs each input (or token) to a small subset of specialist sub-networks ('experts'), so that the model has a large total parameter count but uses only a fraction per forward pass.
Frontier model
A foundation model whose capabilities meet or exceed the most advanced publicly known systems at the time of training, often defined operationally by training-compute thresholds or by performance on canonical benchmarks.
Foundation model
A large machine-learning model trained on broad data at scale and adaptable to a wide range of downstream tasks through fine-tuning, prompting, or retrieval augmentation.
MMLU benchmark
The Massive Multitask Language Understanding benchmark, a 57-subject multiple-choice test covering elementary, high-school, college, and professional knowledge, designed to probe broad-coverage language-model knowledge.
HELM benchmark
The Holistic Evaluation of Language Models benchmark, a multi-metric framework evaluating language models across accuracy, calibration, robustness, fairness, bias, toxicity, and efficiency on a fixed set of scenarios.
BIG-bench
The Beyond the Imitation Game benchmark, a community-contributed collection of more than 200 tasks designed to probe capabilities of large language models that may be missed by narrower benchmarks.
MLCommons benchmark
A benchmark published by the MLCommons consortium for measuring AI system performance under standardised workloads, datasets, and submission rules, with the principal suites being MLPerf Training, MLPerf Inference, and MLPerf HPC.
Hugging Face Hub (concept)
A web-based platform and ecosystem for sharing machine-learning models, datasets, and demonstration applications ('Spaces'), with conventions for model cards, dataset cards, and versioned repositories.
NIST AI RMF (Risk Management Framework)
The US National Institute of Standards and Technology's voluntary framework for managing risks associated with AI systems across the AI lifecycle, structured around the functions Govern, Map, Measure, and Manage.
ISO/IEC 42001 (AI management system)
An international standard, published in 2023, specifying requirements for establishing, implementing, maintaining, and continually improving an AI Management System within an organisation, structured analogously to ISO 9001 (quality) and ISO/IEC 27001 (information security).
AI conformance assessment
A formal evaluation, conducted by the AI system provider or a notified third-party body, demonstrating that an AI system meets the applicable regulatory or standard-based requirements before being placed on the market.
AI assurance
The process of measuring, evaluating, and communicating the trustworthiness of AI systems through evidence-based mechanisms such as audits, certifications, impact assessments, and conformity declarations.
Trustworthy AI
AI systems exhibiting properties (lawful, ethical, technically robust) that warrant the trust of users, affected parties, and society, as articulated in the EU High-Level Expert Group's framework and adopted in subsequent regulation.
Responsible AI
An umbrella term covering the design, development, deployment, and governance practices intended to ensure AI systems are ethical, fair, transparent, accountable, robust, secure, and respectful of privacy.
Use card
A documentation artefact recording an intended deployment context for a model or system, including the user population, the decisions the model informs, the supervision regime, and out-of-scope uses.
Algorithm card
A documentation artefact describing the algorithmic method or family (e.g., a particular gradient-boosting estimator, a clustering algorithm) independent of any particular trained instance, including inductive biases, assumptions, complexity, and intended use cases.
Bias audit (model)
An audit specifically focused on disparate model performance across demographic, geographic, or contextual sub-groups, including testing for direct, proxy, and intersectional disparities.
Model audit
A structured assessment of a machine-learning model by an independent party against pre-specified criteria covering performance, robustness, fairness, security, privacy, and conformance with stated policy.
Data statement (NLP)
A standardised description of an NLP dataset covering curation rationale, language variety, speaker and annotator demographics, speech situation, text characteristics, and recording quality.
Datasheet for datasets
A structured document accompanying a machine-learning dataset that records its motivation, composition, collection process, pre-processing, intended uses, distribution, and maintenance, modelled on electronic-component datasheets.
System card
A documentation artefact describing an AI-enabled system in its production configuration, including the constituent models, the pre- and post-processing pipeline, safety filters, monitoring, and operational guardrails.
Model card
A short, structured document accompanying a machine-learning model that records its intended use, training data, evaluation methodology, performance characteristics across population sub-groups, and known limitations.
