CASRAI Dictionary

Tag: GenAI disclosure

  • AI Model Documentation: Datasheets and Model Cards

    Model cards are short, structured documents that report what an AI model does, how it was evaluated, and the conditions under which it should and should not be used. Together with datasheets for datasets, which document the data a model is trained and tested on, they form the backbone of responsible-AI documentation. Both were proposed to bring the same rigour to AI artefacts that established disciplines bring to materials and reagents, and both directly support reproducibility, accountability and the integrity of the research record.

    Model cards (Mitchell et al. 2019)

    Model cards were introduced by Mitchell and colleagues in 2019 as a framework for transparent model reporting. A model card accompanies a trained model and records, in a consistent format, the essential facts a user needs to decide whether the model is appropriate for their purpose. Crucially, model cards emphasise disaggregated evaluation: reporting performance not only in aggregate but across relevant subgroups, so that uneven performance is visible rather than hidden behind a single headline number.

    A typical model card covers model details (who built it, version, architecture), intended use and out-of-scope uses, evaluation data and metrics, performance across conditions, and ethical considerations, limitations and caveats. By stating intended and prohibited uses explicitly, a model card reduces the risk of a model being deployed in a context it was never validated for.

    Datasheets for datasets (Gebru et al.)

    Datasheets for datasets, proposed by Gebru and colleagues, apply the same documentation philosophy to data. A datasheet answers questions about a dataset’s whole life cycle: the motivation for creating it, its composition (what the instances represent, how many, whether sensitive data is present), the collection process, any preprocessing, cleaning or labelling, intended and discouraged uses, distribution terms, and arrangements for maintenance. Because so many problems in machine learning originate in the data, documenting it is often more consequential than documenting the model.

    Artefact Documents Key contents
    Model card A trained model Intended use, evaluation, disaggregated performance, limitations
    Datasheet for datasets A dataset Motivation, composition, collection, preprocessing, uses, maintenance

    How they support reproducibility and accountability

    Documentation turns an opaque artefact into an auditable one. A model card tells a future researcher exactly which model version and evaluation protocol produced a published result, while a datasheet records the data provenance needed to interpret or rebuild that result. This is the documentation layer that complements the engineering practices in our guide to reproducibility of machine learning research: code and seeds make a result re-runnable, while cards and datasheets make it interpretable and accountable.

    These artefacts also support the broader disclosure expectations now common in scholarly publishing. When generative AI features in a study, documenting the model and its data complements the editorial requirements covered in our explainer on generative AI and research disclosure norms and across our GenAI disclosure coverage.

    Embedding documentation in the research record

    For documentation to be useful it must be findable and citable as part of the scholarly record, not buried in a code repository. Treating model cards and datasheets as first-class research outputs supports proper credit assignment through frameworks such as CRediT and consistent description through the casrai.org research dictionary. Doing so recognises the substantial work of data curation and evaluation that these documents describe.

    Frequently asked questions

    What is a model card?

    A model card is a structured document, proposed by Mitchell et al. in 2019, that reports an AI model’s intended use, evaluation results (including across subgroups), limitations and ethical considerations, so users can judge whether it suits their purpose.

    What is a datasheet for datasets?

    A datasheet, proposed by Gebru et al., documents a dataset’s motivation, composition, collection and preprocessing, intended uses and maintenance, capturing the data provenance needed to interpret or reproduce results.

    How do model cards differ from datasheets?

    Model cards document a trained model; datasheets document the dataset behind it. Used together, they describe both the artefact and the data that shaped it.

    Why does AI documentation matter for reproducibility?

    It records which model version, evaluation protocol and data produced a result, turning an opaque artefact into an auditable one that others can interpret, scrutinise and rebuild.

  • What Is Generative AI and Research Disclosure Norms?

    Generative AI refers to machine-learning systems that produce new content, such as text, images, audio or code, by modelling the patterns of their training data and sampling from them. Unlike predictive models that output a label or a number, a generative model outputs an artefact. The most prominent examples are large language models (LLMs) for text and diffusion models for images. For research, the rise of these tools has prompted clear disclosure norms from editorial bodies, the most important being that AI cannot be listed as an author.

    What generative AI is

    Modern generative systems are typically foundation models: large models trained on broad data at scale, then adapted to many downstream tasks. Large language models are built on the transformer architecture introduced in 2017, which uses an attention mechanism to weigh relationships between tokens in a sequence and predict the next token. Diffusion models generate images by learning to reverse a gradual noising process, starting from random noise and denoising it step by step into a coherent image. The underlying machinery is the neural network described in our explainer on neural networks and deep learning.

    How generative AI differs from predictive ML

    The distinction is one of output. Predictive (discriminative) machine learning answers questions about given inputs: is this email spam, what is this house worth, which category does this image belong to? Generative AI instead produces novel outputs that did not exist before. A useful framing is that predictive models estimate a label given an input, whereas generative models estimate the distribution of the data itself and sample new examples from it. The foundations of the predictive paradigm are covered in our guide to machine learning concepts and methods.

    Aspect Predictive ML Generative AI
    Typical output Label, score or value New text, image, audio or code
    Goal Predict a target for an input Produce novel content
    Examples Spam filter, price regression LLMs, diffusion image models

    Emerging research-disclosure norms

    As researchers began using generative tools to draft, edit and analyse, journals and editorial bodies responded with guidance. Two positions are now widely shared across the scholarly publishing ecosystem.

    AI cannot be an author. The International Committee of Medical Journal Editors (ICMJE) and the Committee on Publication Ethics (COPE) hold that authorship entails responsibility and accountability that a non-human tool cannot bear, including approving the final version and being answerable for the integrity of the work. A generative model therefore cannot meet authorship criteria and must not be listed as an author or co-author.

    Use must be disclosed. Where generative AI has been used in producing a manuscript, authors are expected to disclose how it was used, typically in the methods or acknowledgements, so that reviewers and readers can assess it. Authors remain fully responsible for the accuracy and integrity of everything in the submission, including any AI-assisted content. These norms are tracked across our GenAI disclosure coverage, and they extend to confidential contexts such as peer review, as set out in our policy on generative AI in peer review, disclosure and confidentiality.

    Documenting generative-AI use in the research record

    Good disclosure is specific. Stating which tool was used, for what purpose (for example language editing versus drafting analysis), and what human verification followed, makes the record auditable. This dovetails with structured documentation practices such as model cards and datasheets, discussed in our piece on AI model documentation, and with the controlled vocabulary maintained in the casrai.org research dictionary.

    Frequently asked questions

    Can generative AI be listed as an author on a paper?

    No. ICMJE and COPE positions hold that authorship requires accountability for the work that a non-human tool cannot bear. Generative AI cannot be an author or co-author, and its use should instead be disclosed.

    How is generative AI different from predictive machine learning?

    Predictive ML outputs a label, score or value for a given input, while generative AI produces new content such as text or images. Generative models learn the distribution of the data and sample from it.

    Where should authors disclose generative-AI use?

    Typically in the methods or acknowledgements, stating which tool was used and for what purpose. Authors remain fully responsible for the accuracy and integrity of all AI-assisted content.

    What is a foundation model?

    A foundation model is a large model trained on broad data at scale and then adapted to many downstream tasks. Large language models and diffusion image models are common examples.