Data Analysis Tool in Python: pandas and SciPy Guide

A Python data analysis tool is usually a small stack of libraries rather than one program. For tabular research data the core pair is pandas, which loads a file and summarises it by group, and SciPy, whose scipy.stats module runs the statistical test. This guide installs both from their official instructions, then runs one complete pipeline (load, describe, group, test) and shows how to read each output, including the one assumption that ttest_ind makes silently.

Python data analysis tools and what each is used for in research

The phrases “python data analysis tool”, “python data analysis tools” and “data analysis tools in python” all point at the same working set. For the pipeline below, two pieces do the work:

pandas: load a table and summarise it

pandas reads a CSV file into a table (a DataFrame), shows its first rows, produces summary statistics with describe() and computes group-wise statistics with groupby(). Cleaning and reshaping that table before analysis is covered in data wrangling, data cleaning steps and renaming columns in a pandas DataFrame.

SciPy: test whether two groups differ

scipy.stats.ttest_ind calculates the t-test for the means of two independent samples of scores. The concept behind it is explained in the t-test guide; the wider module is covered in scipy.stats. Plotting and machine learning are separate steps: see scatter plots in Python and the scikit-learn tutorial.

Install pandas and SciPy

Install from the official sources only. The pandas installation page states that pandas is distributed through conda-forge, PyPI and as source on GitHub, and recommends installing and running it from a virtual environment. The SciPy installation page gives the same advice for its pip workflow, which starts by creating and activating a virtual environment.

Route	pandas	SciPy
pip (inside an activated virtual environment)	pip install pandas	python -m pip install scipy
conda	conda install -c conda-forge pandas	conda install scipy (or mamba install scipy)

For conda or mamba, both installation pages point to Miniforge. To keep an environment reproducible for colleagues, see sharing a reproducible conda environment, using conda, pip and mamba in one environment and Anaconda for Jupyter Notebook.

Step by step: load, summarise and compare groups

The first three steps follow the pandas “calculate summary statistics” tutorial, which uses the Titanic passenger table (891 rows). Any CSV with a Sex column and a numeric Age column works the same way.

1. Create and activate a virtual environment, then install pandas and SciPy with the commands above.
2. Load the file and look at it:
   import pandas as pd
titanic = pd.read_csv("data/titanic.csv")
titanic.head()
head() shows the first 5 rows; the tutorial’s table has 12 columns.
3. Summarise the columns you care about: titanic[["Age", "Fare"]].describe()
4. Compare groups: titanic[["Sex", "Age"]].groupby("Sex").mean(). To average every numeric column per group, use titanic.groupby("Sex").mean(numeric_only=True). Selecting one column, as in titanic.groupby("Sex")["Age"].mean(), returns a Series (dtype float64) instead of a table.
5. Test the difference between two independent samples. The call signature in the SciPy documentation is scipy.stats.ttest_ind(a, b, *, axis=0, equal_var=True, nan_policy='propagate', alternative='two-sided', trim=0, method=None, keepdims=False). Worked example with two small samples:
   from scipy import stats
a = [3, 4, 4, 5]
b = [9, 10, 11, 12, 40]
stats.ttest_ind(a, b)
stats.ttest_ind(a, b, equal_var=False)

Output and how to read it

describe(): check count against the number of rows

The pandas tutorial’s output for the Titanic table is:

Statistic	Age	Fare
count	714	891
mean	29.699118	32.204208
std	14.526497	49.693429
min	0.42	0
25%	20.125	7.9104
50%	28.0	14.4542
75%	38.0	31.0
max	80.0	512.3292

Read count first. The table has 891 rows, but Age has a count of 714, so Age is missing for the other rows and every Age statistic describes only those 714. Fare’s count of 891 matches the table. See descriptive statistics for what the remaining rows mean.

groupby(): one summary per group

In the tutorial output, mean Age is 27.915709 for female passengers and 30.726645 for male passengers. The result is a summary per group, not a test: it does not say whether the difference is larger than chance.

ttest_ind(): the statistic, the p-value and the equal_var default

The function returns a t-statistic and a p-value for the chosen alternative; the default is 'two-sided'. The SciPy documentation’s own example, two samples drawn from the same distribution, prints TtestResult(statistic=-0.4390847099199348, pvalue=0.6606952038870015, df=998.0).

The argument that changes results is equal_var. It defaults to True; with equal_var=False SciPy performs Welch’s t-test, which does not assume equal population variance. On the two small samples in step 5 (means 4.0 and 16.4), computed with SciPy 1.18.0 on 2026-10-29:

Call	t-statistic	p-value (two-sided)	df
ttest_ind(a, b) (equal_var=True)	-1.844	0.1077	7.0
ttest_ind(a, b, equal_var=False) (Welch)	-2.089	0.1043	4.038

The two calls return different numbers, so report which one you used. Related reading: ANOVA in Python with statsmodels for more than two groups.

Troubleshooting

• Count is lower than the number of rows. The column has missing values (Age: 714 of 891 in the tutorial data). Decide how to handle them before comparing groups; see data cleaning steps.
• You want means for every numeric column by group. Use titanic.groupby("Sex").mean(numeric_only=True), the form shown in the pandas tutorial.
• A t-test gives different numbers from another package. Check whether that package ran the equal-variance test or Welch’s test, then set equal_var to match.
• Install problems or package conflicts. Both official installation pages recommend working in a virtual environment; start a fresh one and install pandas and SciPy into it with the commands above.

Free alternatives

Python is one route to the same pipeline. Each row below is taken from the tool’s own site, checked 2026-10-29.

Tool	What its own site says	Official site
R	“R is a language and environment for statistical computing and graphics.”; available as Free Software under the GNU General Public License.	r-project.org
JASP	A free cross-platform program with a graphical user interface, supported by the University of Amsterdam, offering standard analyses in classical and Bayesian form.	jasp-stats.org
jamovi	“A free statistical platform, built by the community”; its About page says it was founded to develop a free and open statistical platform.	jamovi.org

For comparisons on CASRAI see SPSS vs R, jamovi vs JASP and how to use RStudio.

FAQ

What is a Python data analysis tool?

Usually a small set of libraries rather than one program. In this guide, pandas loads and summarises a table and scipy.stats runs the test.

How do I install pandas and SciPy?

Create and activate a virtual environment, then run pip install pandas and python -m pip install scipy, or use the conda commands in the install table above.

Why does ttest_ind give a different result from another package?

ttest_ind defaults to equal_var=True. With equal_var=False SciPy performs Welch’s t-test, which does not assume equal population variance, so the numbers differ. Set the option to match the other package and report which one you used.

Why is the count in describe() lower than the number of rows?

The column has missing values. In the pandas tutorial, Age has a count of 714 against 891 rows.

Sources (official documentation)

• pandas, Installation, checked 2026-10-29.
• pandas, Calculate summary statistics, checked 2026-10-29.
• SciPy, scipy.stats.ttest_ind, checked 2026-10-29.
• SciPy, Installing SciPy, checked 2026-10-29.
• The R Project, What is R?; JASP; About jamovi, checked 2026-10-29.