Installing modules and Virtual environments

The standard modules are only a tiny fraction of the vast amount of libraries available for Python. The rich third-party ecosystem is one of the reasons why Python is so popular. You can find plenty of well made modules for web development, finance applications, scientific computing, machine learning, bioinformatics, data science, GUI, games, etc. There are plenty of alternatives for standard modules as well. Quoting from pypi.org: Python Packaging Index:

The Python Package Index (PyPI) is a repository of software for the Python programming language. PyPI helps you find and install software developed and shared by the Python community.

This chapter will discuss how to use pip for installing modules. You'll also see how to create virtual environments using the venv module.

pip

Modern Python versions come with the pip installer program. The below code shows how to install the latest version of a module (along with dependencies, if any) from PyPI. See the pip user guide for documentation and other details like how to use pip on Windows. The --user option limits the availability of the module to the current user, see packaging.python: Installing to the User Site for more details.

# use py instead of python3.13 for Windows
$ python3.13 -m pip install --user regex
Collecting regex
  Downloading ...
Installing collected packages: regex
Successfully installed regex-2024.11.6

warning Make sure that the package to be installed supports your Python version.

Here's an example with the regex module that makes use of a subexpression call to recursively match nested sets of parentheses.

>>> import regex

>>> eqn = '((3+a) * ((r-2)*(t+2)/6) + 42 * (a(b(c(d(e)))))'
>>> regex.findall(r'\((?:[^()]++|(?0))++\)', eqn)
['(3+a)', '((r-2)*(t+2)/6)', '(a(b(c(d(e)))))']

info See packaging.python: Installing from PyPI for details like constraining package version number, upgrading packages, etc.

info uninstall instead of install --user in the above example will remove the package. See also stackoverflow: how to uninstall a package for details and gotchas.

warning warning warning Unless you really know what you are doing, do NOT ever use pip as an admin user. Problematic packages are an issue, see Malicious packages found to be typo-squatting and Hunting for Malicious Packages on PyPI for examples. See also security.stackexchange: PyPI security measures.

venv

Virtual environments allow you to work with specific Python and package versions without interfering with other projects. You can use the built-in module venv to easily create and manage virtual environments.

The flow I use is summarized below. If you are using an IDE, it will likely have options to create and manage virtual environments.

# this is needed only once
# 'new_project' is the name of the folder, can be new or already existing
$ python3.13 -m venv new_project

$ cd new_project/
$ source bin/activate
(new_project) $ # pip install <modules>
(new_project) $ # do some scripting
(new_project) $ deactivate
# you're now out of the virtual environment

Here, new_project is the name of the folder containing the virtual environment. If the folder doesn't already exist, a new folder will be created. The command source bin/activate enables the virtual environment. Among other things, python or python3 will point to the version you used to create this environment, which is python3.13 in the above example. The prompt will change to the name of the folder, which is an easy way to know that you are inside a virtual environment (unless your normal prompt is something similar). pip install will be restricted to this environment.

Once your work is done, use the deactivate command to exit the virtual environment. If you delete the folder, your installed modules in that environment will be lost as well.

info The commands shown above will differ for Windows. See this article on Virtual Environments for details.

Here are some more resources about virtual environments:

Creating your own package

The packaging.python: Packaging Python Projects tutorial walks you through packaging a simple Python project. See also: