Installing modules and Virtual environments

The standard modules are only a tiny fraction of the vast amount of libraries available for Python. The rich third-party ecosystem is one of the reasons why Python is so popular. You can find plenty of well made modules for web development, finance applications, scientific computing, machine learning, bioinformatics, data science, GUI, games, etc. There are plenty of alternatives for standard modules as well. Quoting from pypi.org: Python Packaging Index:

The Python Package Index (PyPI) is a repository of software for the Python programming language. PyPI helps you find and install software developed and shared by the Python community.

This chapter will discuss how to use pip for installing modules. You'll also see how to create virtual environments using the venv module.

pip

Modern Python versions come with the pip installer program. The below code shows how to install the latest version of a module (along with dependencies, if any) from PyPI. See pip user guide for documentation and other details like how to use pip on Windows. --user option limits the availability of the module to the current user, see packaging.python: Installing to the User Site for more details.

# use py instead of python3.9 for Windows
$ python3.9 -m pip install --user regex
Collecting regex
  Downloading ...
Installing collected packages: regex
Successfully installed regex-2021.4.4

warning Make sure that the package you want to install supports your Python version.

Here's an example with regex module that makes use of possessive quantifiers, a feature not yet supported by the re module.

>>> import regex

# numbers >= 100 with optional leading zeros
# same as: r'\b0*[1-9]\d{2,}\b'
>>> regex.findall(r'\b0*+\d{3,}\b', '0501 035 154 12 26 98234')
['0501', '154', '98234']

info See packaging.python: Installing from PyPI for details like constraining package version number, upgrading packages, etc.

info uninstall instead of install --user in the above example will remove the package. See also stackoverflow: how to uninstall a package for details and gotchas.

warning warning warning Unless you really know what you are doing, do NOT ever use pip as a root/admin user. Problematic packages are an issue, see Malicious packages found to be typo-squatting and Hunting for Malicious Packages on PyPI for examples. See also security.stackexchange: PyPI security measures.

venv

Virtual environments allow you to work with specific Python and package versions without interfering with other projects. Modern Python versions come with built-in module venv to easily create and manage virtual environments.

The flow I use is summarized below. If you are using an IDE, it will likely have options to create and manage virtual environments.

# this is needed only once
# 'new_project' is the name of the folder, can be new or already existing
$ python3.9 -m venv new_project

$ cd new_project/
$ source bin/activate
(new_project) $ # pip install <modules>
(new_project) $ # do some scripting
(new_project) $ deactivate
$ # you're now out of the virtual environment

Here, new_project is the name of the folder containing the virtual environment. If the folder doesn't already exist, a new folder will be created. source bin/activate will enable the virtual environment. Among other things, python or python3 will point to the version you used to create the environment, which is python3.9 in the above example. The prompt will change to the name of the folder, which is an easy way to know that you are inside a virtual environment (unless your normal prompt is something similar). pip install will be restricted to this environment.

Once your work is done, use deactivate command to exit the virtual environment. If you delete the folder, your installed modules in that environment will be lost as well. See also:

Creating your own package

The packaging.python: Packaging Python Projects tutorial walks you through packaging a simple Python project. See also: