Python Workflow

Setting up virtual environments and package management on macOS.

When working on multiple python projects, it’s important to create isolated environments for each project to avoid dependency conflicts. There are several tools available to manage virtual environments in Python. Here I will cover the built-in venv module, as well as the popular third-party package manager conda.

1 venv

Reference: Python Virtual Environments - Full Tutorial for Beginners

1.1 Create a Virtual Environment

On mac using Python 3, you can create a virtual environment under current working directory by running the following command in your terminal:

python3 -m venv env

1.2 Activate the virtual environment

source env/bin/activate

Simply type deactivate to exit the virtual environment when you’re done working.

1.3 Setup the environment

Use pip to install the pacakages you need for your project. For example: pip install matplotlib. Use pip list to show all installed packages in the virtual environment. You can use the requirements.txt file to install all the dependencies for your project. Refer to the next Section for more details.

1.4 Share your dependencies

To share your project’s dependencies with others, you can create a requirements.txt file by running pip freeze > requirements.txt. This file will list all the packages and their versions installed in your virtual environment.

To create a virtual environment from a requirements.txt file, run: pip install -r requirements.txt

2 conda

While venv is a built-in module for creating virtual environments in Python, conda is a popular third-party package manager that also provides powerful environment management features. One advantage of using conda over venv is that it can manage both Python packages and Python itself, allowing you to create environments with different Python versions. Conda can be downloaded from Anaconda.

2.1 Manage conda environments

In terminal, run conda env list to see all the conda environments you have created. Different from venv, conda environments are often stored in a central location instead of the project directory.

To remove a conda environment, run conda env remove -n myenv to remove the conda environment named myenv. Or conda env remove -p ./env if you created the environment in current directory.

2.2 Create a conda environment

Run conda create -n myenv python=3.13 to create a new conda environment named myenv with Python 3.13 installed. Run conda create -p ./env python=3.13 to create a conda environment in current directory.

If you want the environment to use packages from conda-forge, make sure you have configured conda beforehand (see Section 2.4 for details):

2.3 Activate the conda environment

Run conda activate myenv to activate the conda environment you just created. Or conda activate ./env if you created the environment in current directory.

Simply run conda deactivate to exit the conda environment when you’re done working.

2.4 Setup the environment

Run conda list to show all installed packages in the conda environment.

To install packages, it’s recommended to use conda-forge as primary channel in conda environment (e.g., conda install -c conda-forge matplotlib). You can also set conda-forge as the default channel and give it higher priority by running the following commands (do this before creating the environment):

conda config --add channels conda-forge
conda config --set channel_priority strict

Then you can simply run conda install matplotlib to install packages from conda-forge channel.

You can use pip to install packages in a conda environment when the package is not available in conda repositories.

2.5 Share your dependencies

To share your conda environment’s dependencies with others, you can create an environment.yml file by running: conda env export > environment.yml. You can also do conda env export --from-history > environment.yml to only include the packages you explicitly installed in the environment.yml file.

To create a conda environment from an environment.yml file, run: conda env create -f environment.yml.

To update an existing environment from a modified environment.yml without recreating it from scratch, run conda env update -f environment.yml --prune. The --prune flag removes packages that are no longer listed in the doc.