What Is Natural Language Processing (NLP)?

Natural language processing (NLP) is the area of computing that works with human-language text, turning running prose into units a program can count, label and compare. For a researcher it is a method for analysing literature, interview transcripts, survey free-text and other corpora at a scale no one can read by hand. This guide explains the standard task chain, then walks through a reproducible setup with two widely used Python toolkits, spaCy and NLTK, using the commands from their official documentation (checked 2026-10-29).

What natural language processing is used for in research

NLP in research is a chain of tasks, each feeding the next. The free textbook Speech and Language Processing by Dan Jurafsky and James H. Martin (3rd edition draft, online manuscript of August 19, 2026) is the standard reference, and three of its chapters map directly onto that chain:

1. Tokenization: splitting raw text into words, punctuation and other units. Textbook chapter 2, “Words and Tokens”.
2. Text classification: assigning a label to a whole text, for example sorting open-ended survey answers into categories. Chapter 4, “Logistic Regression and Text Classification”.
3. Sequence labeling: assigning a label to each token, such as a part of speech or a named entity (a person, organisation or place mentioned in the text). Chapter 18, “Sequence Labeling for Parts of Speech and Named Entities”.

Reading order for a newcomer: install a toolkit, run a short text through it, then read the matching chapter to understand what each output column means. NLP sits within the broader field of artificial intelligence and draws on linguistics; its classification methods are the same family used in machine learning.

Install or access (official sources only)

Both toolkits are installed with Python. Check your Python version first, because the two projects state different supported ranges (see Troubleshooting).

spaCy

The spaCy installation page (spacy.io/usage) states that spaCy is compatible with 64-bit CPython 3.7+ and runs on Unix/Linux, macOS and Windows, and recommends installing in a virtual environment.

python -m venv .env
source .env/bin/activate      # macOS/Linux
.env\Scripts\activate         # Windows
pip install -U pip setuptools wheel
pip install -U spacy

The same page gives a conda route:

conda create -n venv
conda activate venv
conda install -c conda-forge spacy

NLTK

The NLTK installation page (nltk.org/install.html) states that NLTK requires Python 3.9, 3.10, 3.11, 3.12 or 3.13, and gives this install command:

pip install --user -U nltk

Step-by-step: set up a first pipeline with spaCy

1. Create and activate a virtual environment, then install spaCy with the pip commands above.
2. Download a trained pipeline. The spaCy documentation notes that after installation you typically want to download one: python -m spacy download en_core_web_sm
3. Check that the installed pipeline is compatible with your spaCy version: python -m spacy validate. The documentation recommends running spaCy commands with python -m so the correct version is used.
4. Load the pipeline and process a text. In Python, spacy.load returns a Language object conventionally named nlp; calling it on a string returns a Doc:

   import spacy
   nlp = spacy.load("en_core_web_sm")
   doc = nlp("Apple is looking at buying U.K. startup for $1 billion.")

5. Print the tokens and the named entities:

   for token in doc:
       print(token.text, token.pos_, token.dep_)
   for ent in doc.ents:
       print(ent.text, ent.label_)

Setting up NLTK data

NLTK keeps its corpora and models in separate downloadable data packages. The documentation describes the “popular” subset as a starting point when you are unsure which datasets you need. From a terminal: python -m nltk.downloader popular. From the Python interpreter: import nltk; nltk.download('popular').

Output and how to read it

The spaCy documentation (“spaCy 101”, spacy.io/usage/spacy-101) explains that spaCy first tokenizes the text and that the Doc object owns the sequence of tokens and all their annotations. Iterating over a Doc gives one token at a time. For the example sentence “Apple is looking at buying U.K. startup for $1 billion.” the documentation tabulates each token’s attributes: Text, Lemma, POS, Tag and Dep.

Output	Where it appears	How to read it
Tokens	Iterating over the Doc	The units the text was segmented into: words, punctuation and so on. Each carries its attributes (Text, Lemma, POS, Tag, Dep).
Named entities	doc.ents	Each entity has Text, Start, End and Label. In the documentation’s example, Apple is labeled ORG, U.K. is labeled GPE and $1 billion is labeled MONEY.

For a research use, treat these labels as machine annotations to be checked, not as ground truth: the textbook’s chapter 18 is the place to learn how sequence labelers are defined and evaluated. Record the toolkit version and the pipeline name (here en_core_web_sm) alongside your results so the analysis can be reproduced.

Troubleshooting

• Install fails or the import errors. Check python --version against the supported ranges: CPython 3.7+ (64-bit) for spaCy, and 3.9 to 3.13 for NLTK, as each project’s installation page states.
• The pipeline will not load or warns about versions. Run python -m spacy validate, which the spaCy documentation describes as verifying that all installed pipeline packages are compatible with your spaCy version.
• NLTK cannot find a corpus or model. The data packages are downloaded separately from the library; run the popular download shown above.
• Wrong spaCy version runs. Invoke commands as python -m spacy ..., as the documentation recommends, so the interpreter you activated is the one used.

Free alternatives compared

	spaCy	NLTK
Python requirement (per official install page)	64-bit CPython 3.7+	Python 3.9, 3.10, 3.11, 3.12 or 3.13
Install route	pip install -U spacy or conda install -c conda-forge spacy	pip install --user -U nltk
Extra download after install	A trained pipeline, e.g. python -m spacy download en_core_web_sm	Data packages, e.g. python -m nltk.downloader popular
Check command	python -m spacy validate	Not given on the install page

Frequently asked questions

What is natural language processing?

NLP is the computational processing of human-language text. In practice it is a sequence of tasks such as tokenization, text classification and labeling each token with a part of speech or entity type, as laid out in the Jurafsky and Martin textbook.

Is NLP the same as machine learning?

No. NLP names the problem area (language data); machine learning supplies many of the methods. The textbook’s chapter 4 treats text classification through logistic regression, a standard machine-learning method. See the AI terms glossary for related vocabulary and the dictionary entry on large language models for the neural models now common in this field.

What should I document when I use NLP in a study?

Record the toolkit and its version, the pipeline or data package, and the text you processed. For dataset-level documentation of language data, see the dictionary entry on data statements for NLP.

Which reference should I read first?

Start with the textbook chapters named above (2, 4 and 18), which correspond to the three tasks in this guide.

Sources

All sources checked on 2026-10-29.

• spaCy, Install spaCy: spacy.io/usage (checked 2026-10-29).
• spaCy 101: spacy.io/usage/spacy-101 (checked 2026-10-29).
• NLTK, Installing NLTK: nltk.org/install.html (checked 2026-10-29).
• Dan Jurafsky and James H. Martin, Speech and Language Processing, 3rd edition draft, online manuscript released August 19, 2026: web.stanford.edu/~jurafsky/slp3 (checked 2026-10-29).