Codecov: What It Is and How to Set It Up for Research Code

Codecov is a hosted service that receives test-coverage reports from your continuous integration (CI) runs and shows which lines of your code the tests executed. For research software, the useful result is a coverage figure on every pull request that tells you which new or changed lines no test has run yet. This guide explains what that figure does and does not measure, the two documented ways to upload reports from GitHub, a gcov-to-Codecov chain for C and C++ code, and free alternatives. Sources were read on 2026-11-07.

What is Codecov used for in research software?

Codecov is used to collect coverage reports from CI, store them per commit, and report coverage back on pull requests. According to Codecov’s quick start, after an upload runs on a pull request, Codecov adds a patch coverage status check and a pull request comment. Patch coverage is the coverage of the lines changed in that pull request, which is the figure a reviewer needs when judging whether a change was tested.

Codecov does not run your tests and does not measure coverage itself. Your test runner and a coverage tool produce a report file, and Codecov processes it on its servers. Its documentation states that it ingests .xml, .json and .txt report formats.

What is code coverage testing, and what is its purpose?

Code coverage testing measures how much of a program is exercised by its test suite. The GCC manual describes the purpose precisely: a test suite verifies that a program works as expected, while a coverage program checks how much of the program the test suite exercises, so developers can decide which test cases to add.

The unit of measurement matters. The GCC manual says gcov accumulates statistics by line at its lowest resolution, and records how many times each line executed. A line marked as executed shows that the line ran during the tests. It does not show that the test checked the result. For that reason a coverage report finds untested code but cannot prove tested code is correct. See What Is Unit Testing? for the assertions that make coverage meaningful.

Should code coverage be 100%?

No single percentage is required by Codecov, gcov or the sources read for this page, and the figure should not be treated as a target by itself. This is an editorial interpretation, not a rule from a standard. Because coverage counts executed lines rather than checked results, a suite can reach a high figure while asserting little. A more useful working practice is to read the uncovered lines, decide for each whether it needs a test, and use the pull-request patch figure to keep new code from adding untested lines.

Install or access (official sources only)

1. Sign up at app.codecov.io.
2. For GitHub, install the Codecov GitHub app for your organisation or account. The quick start states that GitHub users must install it.
3. Use “setup repo” to get a repository upload token and store it as a GitHub Actions secret named CODECOV_TOKEN.
4. Add the upload step to your workflow using one of the two routes below.

Step by step: upload coverage from GitHub Actions

Route 1: codecov/codecov-action

The official GitHub Action is published at github.com/codecov/codecov-action under an MIT licence. Its README example uses version 5 and requires actions/checkout to run first; the runner needs bash, curl, git and gpg.

- uses: codecov/codecov-action@v5 with: token: ${{ secrets.CODECOV_TOKEN }} files: ./coverage1.xml,./coverage2.xml fail_ci_if_error: true

fail_ci_if_error defaults to false, so an upload failure does not fail the job unless you set it. The README also documents an OpenID Connect alternative to the token (use_oidc: true with the id-token: write permission).

Route 2: the Codecov CLI

The quick start’s example instead downloads the Codecov CLI, for Linux:

curl -Os https://cli.codecov.io/latest/linux/codecov chmod +x codecov ./codecov --verbose upload-process --fail-on-error -t ${{ secrets.CODECOV_TOKEN }} -n 'service'-${{ github.run_id }} -F service -f coverage-service.xml

The documented operating-system options in the download path are alpine, linux, macos and windows. In that example the step is preceded by pytest --cov ., which produces the coverage data for a Python project.

Route	How coverage is sent	Failure behaviour
codecov/codecov-action	files: input lists report files	fail_ci_if_error defaults to false
Codecov CLI	upload-process with -f report file	Fails on error when --fail-on-error is passed

How to use gcov (C and C++ code)

gcov is the coverage tool shipped with GCC. Its manual calls it a test coverage program used in concert with GCC. The workflow in the manual is:

1. Compile with the --coverage option. The manual states this is required, and that if you want to prove every line executed you should not compile with optimization at the same time, because the optimizer can merge lines so separate counts cannot be calculated.
2. Run your program or test suite. Running code built with -fprofile-arcs writes a .gcda count-data file for each object file; the .gcno notes file is written at compile time with -ftest-coverage.
3. Run gcov [options] files on your source files. gcov creates a sourcefile.gcov log showing how many times each line executed.

gcov works only with code compiled by GCC and works best when each line holds one statement; complex macros that expand into loops or control structures make the statistics less helpful because only the line of the macro call is reported. Options include -b (branch probabilities) and -c (branch counts as numbers rather than percentages).

From gcov to Codecov

Codecov’s supported-formats page lists gcov text output and lcov, which it describes as the graphical version of gcov, among the formats it processes. The gcovr project, which describes itself as a utility for managing the use of gcov and generating summarized coverage results, can produce text, HTML and other report types. Generate a supported report from your gcov data, then upload it with either route above.

Output and how to read it

• Pull request check: the patch coverage status check reports coverage for the changed lines.
• Pull request comment: Codecov comments on the pull request after the upload completes.
• Dashboard: shows coverage for the repository. If it is empty, the quick start says to open Configuration, then General, and confirm the default branch.
• gcov log: in a .gcov file, each line carries an execution count; a line with count zero never ran in your tests.

Troubleshooting

• Dashboard is empty: confirm the default branch under Configuration, General.
• Upload fails silently: the Action’s fail_ci_if_error is false by default; set it to true to see failures. The CLI example uses --verbose and --fail-on-error.
• Python .coverage file not accepted: Codecov lists .coverage (Python), .html, .exec, .ec and .xccov among report formats it does not support. Export an XML report from your coverage tool instead.
• Action fails to start: check that actions/checkout runs first and that the runner has bash, curl, git and gpg.
• gcov counts look odd: recompile without optimization and keep one statement per line.

Free alternatives

These tools measure or report coverage without Codecov. Each row is taken from the tool’s own documentation.

Tool	What its documentation says it does
gcov	Test coverage program used with GCC; writes per-line execution counts to .gcov files.
gcovr	Utility for managing gcov and generating summarized coverage results as text, HTML, CSV and other reports.
coverage.py	Measures code coverage of Python programs, noting which parts of the code ran and which could have run but did not.

These produce local reports; Codecov adds hosted storage, history and pull-request feedback. Pair coverage with continuous integration so it runs on every change, and see reproducibility infrastructure for how tested code fits into computational reproducibility. The research software engineer role often owns this tooling.

FAQ

What is Codecov used for?

Receiving coverage reports from CI runs and showing, per commit and per pull request, which lines your tests executed.

How do I use gcov?

Compile with --coverage and without optimization, run the program, then run gcov on the source files to get .gcov files with per-line execution counts.

What is code coverage testing and what is its purpose?

It measures how much of a program the test suite exercises, so developers can see which parts are untested and add test cases.

Should code coverage be 100%?

No source read here sets a required percentage. Coverage shows lines executed, not results verified, so treat it as a way to find untested code rather than as a goal in itself.

Sources

• Codecov documentation, Quick Start: docs.codecov.com/docs/quick-start (read 2026-11-07)
• Codecov documentation, Supported Coverage Report Formats: docs.codecov.com/docs/supported-report-formats (read 2026-11-07)
• codecov/codecov-action README: github.com/codecov/codecov-action (read 2026-11-07)
• GCC manual, gcov chapter: gcc.gnu.org/onlinedocs/gcc/Gcov.html (read 2026-11-07)
• gcovr documentation: gcovr.com; coverage.py documentation: coverage.readthedocs.io (read 2026-11-07)