Rooting a Phylogenetic Tree: Outgroup, Midpoint and Non-Reversible Methods

Rooting a phylogenetic tree means choosing the branch where the root sits, which gives the tree a direction in time: it tells you which lineages split earlier and which taxa form clades. Most software infers unrooted trees, so you add the root afterwards with an outgroup, a midpoint, or a non-reversible model. This guide shows the commands for each in R (ape, phangorn) and IQ-TREE, how to read the result, and how to fix the usual problems.

What rooting is used for in research

An unrooted tree shows relationships but not direction. Until a root is placed, you cannot say which group is ancestral or call a set of taxa a clade (see phylogenetic tree and clade). Rooting changes where the root is drawn, not the unrooted branching pattern, so it is a separate step from building the tree (see how to make a phylogenetic tree). The IQ-TREE documentation states that time-reversible substitution models infer unrooted trees, so a root must come from an outgroup or from a non-reversible model.

Which rooting method should you use?

Method	What you supply	Tool and function
Outgroup rooting	One or more taxa known to lie outside the group of interest	ape root(); IQ-TREE -o
Midpoint rooting	Nothing beyond the tree with branch lengths	phangorn midpoint()
Non-reversible model	An alignment; no outgroup	IQ-TREE --model-joint UNREST or NONREV

The IQ-TREE documentation notes that choosing a good outgroup is an entire topic of its own, so justify the choice from independent evidence and report it.

Install or access

• ape and phangorn: R packages distributed through CRAN, the R package repository. In R, run install.packages(c('ape','phangorn')).
• IQ-TREE: download from the project site, iqtree.github.io. See also the CASRAI IQ-TREE guide.

Step-by-step: root a tree

Outgroup rooting with ape

1. Load the tree and check its state: library(ape); tr <- read.tree('tree.nwk'); is.rooted(tr).
2. Root on the outgroup: rt <- root(tr, outgroup = 'TaxonX', resolve.root = TRUE). The outgroup argument is a numeric or character vector naming the new outgroup. With a single tip, the tree is rooted at the node below that tip; with several, ape unroots the tree and uses the most recent common ancestor of the ingroup as the new root, as described in the ape help page.
3. Alternatively, root at a known internal node with root(tr, node = 12).
4. Keep support values aligned by adding edgelabel = TRUE, which treats node labels as edge labels so they follow the branch when the tree is rerooted.
5. Confirm: is.rooted(rt) should return TRUE. To undo, use unroot(rt).

resolve.root = TRUE resolves the new root as a bifurcating node, which adds a zero-length branch.

Midpoint rooting with phangorn

1. Load both packages: library(ape); library(phangorn).
2. Run rt <- midpoint(tr). The phangorn help describes this as midpoint rooting of a tree. Midpoint rooting places the root at the midpoint of the longest path between two tips, so it needs branch lengths and assumes similar evolutionary rates across lineages.
3. Control support labels with the node.labels argument, which accepts 'support' (the default), 'label', or deletion.

Outgroup rooting in IQ-TREE

Add -o followed by the outgroup taxon name to the inference command, for example iqtree3 -s alignment.phy -o TaxonX -B 1000 -T AUTO. The documentation states the output tree in the .treefile is rooted accordingly. Without -o, IQ-TREE uses the first taxon in the alignment as the outgroup, which is a display default and not evidence of the biological root.

Rooting without an outgroup in IQ-TREE

1. Infer the tree with reversible models and save the best partitioning scheme.
2. Rerun with a non-reversible joint model: --model-joint UNREST for DNA (12 parameters) or --model-joint NONREV for amino acids (379 parameters), as the IQ-TREE rooting documentation specifies.
3. Read the root position from the resulting tree and its rootstrap values (below).

Output and how to read it

A rooted tree has a root node from which two descendant lineages emerge, and every internal node now has a direction: its descendants are tips and clades below it (see how to read a phylogenetic tree). In IQ-TREE, the rootstrap is the proportion of rooted bootstrap trees that have the root on a given branch, so a high value on one branch supports that root position and a spread across branches means the root is uncertain.

Testing the root

IQ-TREE v2.1.3 or later offers --root-test, which reroots on every branch and runs topology tests. Combined with -zb 1000 -au it gives the approximately unbiased (AU) test, and results are written to <prefix>.roottest.csv. Root positions with a low AU p-value can be rejected. The documented amino-acid example is iqtree3 -s bovidae.phy -p rev_aa.best_scheme.nex --model-joint NONREV -st NT2AA --root-test -zb 1000 -au -te nonrev_aa.treefile --prefix nonrev_aa_test.

Troubleshooting

• Support values sit on the wrong branches after rerooting. In ape use edgelabel = TRUE. For midpoint rooting, choose node.labels deliberately; the phangorn help says the default assumes support values computed from splits, and support computed from clades should be recalculated.
• Outgroup not found. The outgroup must match tip labels exactly, including case and underscores.
• Outgroup is not a single clade. root() returns an error when the outgroup is not monophyletic in the tree. Check that the outgroup taxa form a clade, or choose different outgroup taxa.
• Root changes between methods. Midpoint and outgroup roots can differ; report the method used and, in IQ-TREE, the rootstrap or AU results.

Free alternatives

Tool	Rooting features documented	Documentation
ape (R)	root() by outgroup or node, unroot(), is.rooted()	ape root help
phangorn (R)	midpoint()	phangorn midpoint help
IQ-TREE	-o outgroup, UNREST/NONREV models, rootstrap, --root-test	IQ-TREE rooting

FAQ

What is the difference between a rooted and an unrooted tree?

A rooted tree has a single root that gives direction from ancestor to descendants; an unrooted tree shows only how taxa are related. Time-reversible models infer unrooted trees, so the root is added afterwards.

How do I root a tree in R?

Use root(tr, outgroup = 'TaxonX', resolve.root = TRUE) from ape for an outgroup, or midpoint(tr) from phangorn for midpoint rooting. Check with is.rooted().

Can I root a tree without an outgroup?

Yes. Midpoint rooting needs no outgroup, and IQ-TREE can infer the root from a non-reversible model (UNREST for DNA, NONREV for amino acids), with rootstrap and --root-test to assess it.

Does rooting change the tree topology?

Rooting places the root on a branch of the unrooted tree; it does not change the unrooted branching pattern. It does change which groups read as clades.

Sources

• ape root help, rdrr.io/cran/ape/man/root.html (read 2026-11-08).
• phangorn midpoint help, rdrr.io/cran/phangorn/man/midpoint.html (read 2026-11-08).
• IQ-TREE command reference, iqtree.github.io/doc/Command-Reference (read 2026-11-08).
• IQ-TREE rooting and rootstrap, iqtree.github.io/doc/Rootstrap (read 2026-11-08).