Skip to main content
v2026.11,986 entries · CC-BY 4.0

SDMX: Standard, Formats and Fetching Data with Python

SDMX explained: the ISO 17369 standard, its 2.1, 3.0 and 3.1 versions, formats, key syntax and a worked Python fetch of ECB data.

Written and maintained by CASRAI Editorial Board

Published · Last updated

On this page

SDMX (Statistical Data and Metadata eXchange) is an international standard for describing statistical data and the metadata that explains it, published as ISO 17369 and sponsored by eight international organisations, including the Bank for International Settlements. In research practice it is the common format behind the data services of central banks and statistical agencies: you ask a provider’s SDMX web service for a series, and you receive the observations together with the dimension codes needed to interpret them. This guide explains what the standard defines, which versions and formats exist, how an SDMX query key is built, and how to fetch and read European Central Bank exchange-rate data with the open-source Python package sdmx1.

What SDMX is used for in research

The SDMX standard exists so that statistical organisations can exchange and publish data in one agreed structure instead of each inventing its own spreadsheet layout. According to the SDMX community’s own description, it covers both the statistical data and the metadata (definitions, code lists, structures) that describe it. For a researcher the practical consequence is that a time series from one provider and a time series from another can be requested, parsed and compared with the same tooling.

The sdmx1 documentation lists a long set of data sources the package can talk to, among them the Australian Bureau of Statistics (ABS), the Bank for International Settlements (BIS), the European Central Bank (ECB), Eurostat (ESTAT), the International Labour Organization (ILO), the International Monetary Fund (IMF), INSEE, ISTAT, the OECD, Statistics Canada, UNESCO, UNICEF, the United Nations Statistics Division and the World Bank. If your project draws macroeconomic, demographic or financial indicators from these bodies, SDMX is often the machine-readable route. For sources that publish static files instead of a web service, see the guide to documenting and citing government datasets; for the World Bank’s own interfaces, see World Bank data, WDI, DataBank and the API.

Because the standard separates data from metadata, an SDMX download is more self-describing than a bare CSV, which supports the documentation habits behind computational reproducibility: the query you ran, the dataflow and the dimension codes can all be recorded in your code and your methods section.

Versions, formats and the standard itself

The SDMX technical specifications are published on sdmx.org. The standards page records three relevant release points: version 2.1 (consolidated in 2020, with support for the Validation and Transformation Language 2.0 added in July 2020), version 3.0 (September 2021) and version 3.1 (technical specifications released in May 2025). The data table below sets these side by side with what the sdmx1 package documents.

The specification is organised in numbered sections. Section 1 is the Framework, Section 2 the Information Model (expressed in UML), Section 5 the Registry Specification and Section 6 the Technical Notes. Section 3 (the SDMX-ML XML format) and Section 7 (the web-service API) are maintained on GitHub, and Section 4 (the older SDMX-EDI format) was deprecated in version 3.0.

Which data format will I receive?

SDMX data can be exchanged as SDMX-ML (XML), SDMX-JSON or SDMX-CSV. The REST specification maintained by the SDMX technical working group selects a format through the HTTP media type: its data documentation names application/vnd.sdmx.data+json;version=2.1.0 as the default, with application/vnd.sdmx.data+xml;version=3.1.0 and application/vnd.sdmx.data+csv;version=2.1.0 as alternatives. When you use a client library you normally never set these yourself.

How to install or access it

The package documented here is sdmx1, whose official documentation is at sdmx1.readthedocs.io. Its installation page states that it requires Python 3.9 or later and is installed with pip:

pip install sdmx1

The same page lists an optional extra that adds request caching through requests-cache:

pip install sdmx1[cache]

The core dependencies are pandas, requests and lxml. According to its documentation, the package implements SDMX 2.1 (ISO 17369:2013) and 3.0. Note that the SDMX 3.1 specification released in May 2025 is not named in the package’s feature statement, so check the sdmx1 release notes if your provider has already moved to 3.1.

Step-by-step: fetch ECB exchange-rate data

The sdmx1 walkthrough uses the ECB’s exchange-rate dataflow, identified as EXR. These steps follow that walkthrough.

  1. Import the package and create a client. The client is named after the data source: import sdmx, then ecb = sdmx.Client("ECB").
  2. List the available dataflows. Call flow_msg = ecb.dataflow(). With no identifier, this returns the definitions of every ECB dataflow. Convert them to a table with dataflows = sdmx.to_pandas(flow_msg.dataflow). The documentation’s example listing contained 105 dataflows on the day it was generated, so treat that number as a snapshot, not a constant.
  3. Define the key. A key selects which series you want by giving a value for each dimension. In the walkthrough: key = dict(CURRENCY_DENOM="EUR", FREQ="M", EXR_SUFFIX="A"). Dimensions you leave out are not restricted.
  4. Define the time range. Pass query parameters as a dictionary: params = dict(startPeriod="2019-01", endPeriod="2019-06").
  5. Request the data. data = ecb.data("EXR", key=key, params=params).data[0] returns the first data set in the response message.
  6. Convert it for analysis. sdmx.to_pandas(data) returns a pandas Series.

The walkthrough also shows that the same selection can be written as a dot-separated key string, in which + separates alternative values for one dimension: key = ".USD+JPY..." is documented as equivalent to dict(CURRENCY=["USD", "JPY"]).

Decoding a REST key by hand

Under the hood the client builds an HTTP request. The SDMX REST data documentation gives the URL pattern protocol://ws-entry-point/data/{context}/{agencyID}/{resourceID}/{version}/{key}, where the context is one of datastructure, dataflow or provisionagreement. Its worked example is /data/dataflow/ECB/EXR/1.0.0/M.USD.EUR.SP00.A. The key is a dot-separated list with one position per dimension, in the order the data structure definition declares them. In the ECB exchange-rate output shown in the sdmx1 walkthrough those dimensions are FREQ, CURRENCY, CURRENCY_DENOM, EXR_TYPE and EXR_SUFFIX, so the five positions of M.USD.EUR.SP00.A read left to right as a value for each of them.

The REST documentation sets out these rules for keys:

  • An asterisk (*) is a wildcard for one dimension.
  • Trailing dimensions you omit are treated as wildcards.
  • A comma separates several keys in one request.

The same documentation describes further query parameters in the newer REST specification: firstNObservations and lastNObservations to limit the number of observations, dimensionAtObservation (with values TIME_PERIOD, a dimension name or AllDimensions) to control layout, and a filter such as c[TIME_PERIOD]=ge:2009-05-01+le:2009-05-31 for a time window. For CSV responses it lists a labels option (id, name or both). The sdmx1 walkthrough you followed above uses the older startPeriod and endPeriod parameters, so match the parameter names to what your provider’s service supports.

Output and how to read it

For a multi-dimensional dataset, sdmx.to_pandas(data) returns a pandas Series whose index is a MultiIndex with one level per dimension. In the walkthrough’s ECB example the levels are FREQ, CURRENCY, CURRENCY_DENOM, EXR_TYPE, EXR_SUFFIX and TIME_PERIOD, and the result holds 288 values. Each row therefore tells you exactly which series and which period a number belongs to, with no guessing from column headings.

Two points matter when you analyse the result:

  • Time handling. The documentation shows sdmx.to_pandas(data, datetime="TIME_PERIOD") producing a DataFrame with a DatetimeIndex, but it also labels the datetime argument as deprecated (see the migration notes for version 2.23.0) and warns that parsing datetimes can reduce performance on large datasets. Check the current migration notes before relying on that argument in a long-lived script.
  • Look up the codes. A code such as a currency or frequency is only a label. The docs show how to obtain the data structure definition from the dataflow (dsd = exr_flow.structure), list its dimensions (dsd.dimensions.components), and read the code list for a dimension such as FREQ. Pass the structure back into a query with ecb.data("EXR", key=key, params=params, dsd=dsd).

Which combinations actually exist?

Not every combination of dimension values has data. The walkthrough describes a content constraint returned with the dataflow metadata (exr_msg.constraint.EXR_CONSTRAINTS), whose content region lists the valid members for each dimension; its example shows that CURRENCY and CURRENCY_DENOM have different valid sets. Reading the constraint before building a key avoids empty results.

Troubleshooting

The following items are documented behaviours of sdmx1, not general advice.

  • The server returned an error but no exception was raised. The documentation states that the package raises only HTTP errors with status codes from 400 to 499. Codes of 500 and above do not raise, so inspect Message.response for the HTTP status and check the result yourself.
  • You need the provider’s explanation. When a server returns an error, Message.footer and Footer.text may contain explanatory notes.
  • The query returns nothing or the wrong series. Check the key against the data structure definition and the content constraint, as described above, because the key positions must follow the dimension order.
  • Repeated runs hit the server every time. The package uses requests-cache automatically when it is installed (the documented example builds a client with backend="sqlite", fast_save=True and expire_after=600). A separate cache for parsed messages is disabled by default and is switched on with use_cache=True in the client constructor.

Free alternatives

sdmx1 is open source and free to install. The other routes below are documented by their own official pages.

Route What the official documentation says Best suited to
sdmx1 (Python) Implements SDMX 2.1 and 3.0; lists many named data sources; converts messages to pandas objects Python and pandas workflows
rsdmx (R) CRAN title “Tools for Reading SDMX Data and Metadata”; version 0.6-5 published 2025-02-27; GPL licence; focuses on the SDMX-ML format R workflows with XML responses
Direct REST requests The SDMX REST data documentation defines the URL, key and parameter syntax and the media types Checking exactly what a provider returns, or tools without a client library

Frequently asked questions

What does SDMX stand for?

Statistical Data and Metadata eXchange. It is an ISO standard (ISO 17369) for describing statistical data and metadata, sponsored by eight international organisations.

Is SDMX a file format or a web service?

It is both a family of formats and a set of specifications. Data can be expressed as SDMX-ML, SDMX-JSON or SDMX-CSV, and a separate REST specification defines how a client requests data from a provider’s service.

Which SDMX version should I target?

Use the version your data provider’s service supports. The standards page lists 2.1, 3.0 and 3.1, and sdmx1 documents support for 2.1 and 3.0, so confirm the provider’s version first.

How do I find the dimensions to put in a key?

Read the data structure definition attached to the dataflow, as shown in the sdmx1 walkthrough, and use its dimension order. A wildcard or omitted trailing dimension leaves that dimension unrestricted.

Can I use SDMX data in R?

Yes. The CRAN package rsdmx reads SDMX data and metadata, with a stated focus on the SDMX-ML format.

How should I cite SDMX-based data in a paper?

Cite the data provider, the dataflow identifier, the date you retrieved it, and the query you ran. The guide on citing government datasets covers the dataset-citation elements.

Sources

  • SDMX, About SDMX, https://sdmx.org/about-sdmx/ (checked 2026-11-14)
  • SDMX, Standards (technical specifications and versions), https://sdmx.org/standards/ (checked 2026-11-14)
  • sdmx1 documentation, https://sdmx1.readthedocs.io/en/latest/ (checked 2026-11-14)
  • sdmx1 documentation, Installation, https://sdmx1.readthedocs.io/en/latest/install.html (checked 2026-11-14)
  • sdmx1 documentation, Walkthrough, https://sdmx1.readthedocs.io/en/latest/walkthrough.html (checked 2026-11-14)
  • SDMX REST specification, Data queries, https://github.com/sdmx-twg/sdmx-rest/blob/master/doc/data.md (checked 2026-11-14)
  • CRAN, rsdmx package page, https://cran.r-project.org/web/packages/rsdmx/index.html (checked 2026-11-14)

Data: SDMX specification versions and documented sdmx1 support

Compiled from the SDMX standards page and the sdmx1 package documentation: for each SDMX technical specification release named on sdmx.org, its release timing and whether the sdmx1 documentation names that version as implemented.

Period covered: 2020/2025.

SDMX specification versions and documented sdmx1 support
SDMX versionRelease (per sdmx.org)Named as implemented in sdmx1 documentation
2.1Consolidated 2020 (VTL 2.0 support added July 2020)Yes (ISO 17369:2013)
3.0September 2021Yes
3.1Technical specifications released May 2025Not named

Sources

Compiled by CASRAI from the sources above. Free to reuse under CC BY 4.0 with attribution.

Follow CASRAI

Research-administration guidance, standards updates and independent tool reviews.

Ask CASRAI · free to try

Ask about SDMX: Standard, Formats and Fetching Data with Python

2 free questions a day, no account.

An AI assistant for research: funder rules, compliance, methods and publishing. It links the sources behind its answers and labels answers that come from a web search.

Answers draw on CASRAI's guides and dictionary plus the federal and funder documents we index: Federal Register, Grants.gov, Regulations.gov and UKRI.

Works on this site and inside Claude, Cursor and the AI tools you already use.

Everything CASRAI publishes — this page, the dictionary, the guides and the news — stays open to read, with no account and no card.

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →