Installing modules and Virtual environments
The standard modules are only a tiny fraction of the vast amount of libraries available for Python. The rich third-party ecosystem is one of the reasons why Python is so popular. You can find plenty of well made modules for web development, finance applications, scientific computing, machine learning, bioinformatics, data science, GUI, games, etc. There are plenty of alternatives for standard modules as well. Quoting from pypi.org: Python Packaging Index:
The Python Package Index (PyPI) is a repository of software for the Python programming language. PyPI helps you find and install software developed and shared by the Python community.
This chapter will discuss how to use pip
for installing modules. You'll also see how to create virtual environments using the venv
module.
pip
Modern Python versions come with the pip
installer program. The below code shows how to install the latest version of a module (along with dependencies, if any) from PyPI. See the pip user guide for documentation and other details like how to use pip
on Windows. The --user
option limits the availability of the module to the current user, see packaging.python: Installing to the User Site for more details.
# use py instead of python3.13 for Windows
$ python3.13 -m pip install --user regex
Collecting regex
Downloading ...
Installing collected packages: regex
Successfully installed regex-2024.11.6
Make sure that the package to be installed supports your Python version.
Here's an example with the regex
module that makes use of a subexpression call to recursively match nested sets of parentheses.
>>> import regex
>>> eqn = '((3+a) * ((r-2)*(t+2)/6) + 42 * (a(b(c(d(e)))))'
>>> regex.findall(r'\((?:[^()]++|(?0))++\)', eqn)
['(3+a)', '((r-2)*(t+2)/6)', '(a(b(c(d(e)))))']
See packaging.python: Installing from PyPI for details like constraining package version number, upgrading packages, etc.
uninstall
instead ofinstall --user
in the above example will remove the package. See also stackoverflow: how to uninstall a package for details and gotchas.
Unless you really know what you are doing, do NOT ever use
pip
as an admin user. Problematic packages are an issue, see Malicious packages found to be typo-squatting and Hunting for Malicious Packages on PyPI for examples. See also security.stackexchange: PyPI security measures.
venv
Virtual environments allow you to work with specific Python and package versions without interfering with other projects. You can use the built-in module venv
to easily create and manage virtual environments.
The flow I use is summarized below. If you are using an IDE, it will likely have options to create and manage virtual environments.
# this is needed only once
# 'new_project' is the name of the folder, can be new or already existing
$ python3.13 -m venv new_project
$ cd new_project/
$ source bin/activate
(new_project) $ # pip install <modules>
(new_project) $ # do some scripting
(new_project) $ deactivate
# you're now out of the virtual environment
Here, new_project
is the name of the folder containing the virtual environment. If the folder doesn't already exist, a new folder will be created. The command source bin/activate
enables the virtual environment. Among other things, python
or python3
will point to the version you used to create this environment, which is python3.13
in the above example. The prompt will change to the name of the folder, which is an easy way to know that you are inside a virtual environment (unless your normal prompt is something similar). pip install
will be restricted to this environment.
Once your work is done, use the deactivate
command to exit the virtual environment. If you delete the folder, your installed modules in that environment will be lost as well.
The commands shown above will differ for Windows. See this article on Virtual Environments for details.
Here are some more resources about virtual environments:
- meribold: Virtual Environments Demystified
- calmcode.io: virtualenv
- realpython: Python Virtual Environments Primer
- stackoverflow: What is the difference between venv, pyvenv, pyenv, virtualenv, virtualenvwrapper, pipenv, etc?
Creating your own package
The packaging.python: Packaging Python Projects tutorial walks you through packaging a simple Python project. See also: