Installation, IDEs etc.

Installation, IDEs etc.

Python can be installed right away from the official site or use conda distros.

• Python.org

• Anaconda - Full distro with hundreds of shipped packages (not needed in reality)

• Miniconda - minimal distro with basic packages

  • Recommended as it comes with conda package and env manager

After installation pip tool (Python package manager) can be used to manage 3rd party packages

Main commands:

• install, upgrade package and install requirements from a file:

pip install jupyter virtualenv
pip install -U numpy
pip install -r requirements.txt

• uninstall package:

pip uninstall requests

• list and export all installed packages:

pip freeze
pip freeze > requirements.txt

Similarly you can manage packages via conda (if installed *conda distro):

conda list                          # Print currently installed packages
conda install requests pandas       # Install new packages 
conda remove scipy                  # Remove packages (alias: uninstall)
conda update python                 # Update package (alias: upgrade), even Python itself:

python                          3.9.13-h9a09f29_0_cpython --> 3.10.6-h9a09f29_0_cpython

Virtual environments

Virtual environment is the way of isolation the Python installation from other environments allowing to maintain the same versions of all packages as it was planned/required/designed for. This allows to mimic the working environment on other system and contribute in the development. Also with this we can run tests of different Python programs with different dependencies locally, on remote machines or clouds.

There are quite a few possible solutions for virtualizing the environment:

• Virtualenv

  • Almost default solution (some part of it were integrated into Python 3.3 as module venv). Very simple but robust way of creating an isolated environment at a given place.

• Conda

  • The most popular modern method of virtual environment on production is using specialized Python distribution and it's own environment manager. You can create, export or delete environments.

  • It is the a de facto standard for data science or AI/ML-related projects.

  • It has faster mimicking alternative called mamba

• Poetry

  • Modern and slick packaging, dependency resolver and virtual env management tool. You can do some magic things like having separate dev/prod requirements, project relocation, publishing helpers.

virtualenv

Virtual environment creation tool which allows to keep different version of Python with different version of packages.

You need to install it via pip:

pip install virtualenv

Usage:

virtualenv -p /path/to/existing/python venv   # Create "venv" environment using Python from "/path/to/existing/python"
source venv/bin/activate                      # Activation of this env in Unix             
venv\Scripts\activate                         # Activation of this env in Windows
...
deactivate                                    # Deactivation (reverting to system-wide Python)

virtualenvwrapper

Better tool to manage different python virtual environment is virtualenvwrapper.

• UNIX:

  • virtualenvwrapper site

• Windows clone:

  • virtualenvwrapper-win

Main feature is controlling all available environments with easy fast switching between them.

Installation:

pip install virtualenvwrapper (or pip install virtualenvwrapper-win)

There are two related terms: projects and virtual environments.

• Virtual environment

  • Environment that lives inside $WORKON_HOME (%WORKON_HOME%)

• Project

  • Directory with the code related to specific venv.

  • If venv was created with association to specific project dir (-a <path/to/project/) - after switching to it virtualenvwrapper will chande current directory to project.

virtualenvwrapper VENV commands:

• Create new venv:

  • mkvirtualenv [mkvirtualenv-options] [virtualenv-options] VENV_NAME

  • mkvirtualenv options:

    • -a project_path Associate existing path as project directory

    • -i package Install package in new environment

    • -r requirements_file

    • VENV_NAME (must be last) - the name of resulted venv

• List of all venvs:

  • workon RECOMMENDED

  • lsvirtualenv

• Switch to another venv:

  • workon VENV_NAME

• Deactivate current venv:

  • deactivate

virtualenvwrapper PROJECTS commands:

• Associate project with current active venv:

  • setprojectdir <path/to/project/>

• Create new project environment:

  • mkproject

    • If the environment variable $PROJECT_HOME (%PROJECT_HOME%) is set, create a new project directory in PROJECT_HOME

• Change into project directory:

  • cdproject

  • cd- - change into the directory you were before running cdproject

How to add existing virtualenv to virtualenvwrapper

Easiest way is to dump all packages from virtualenv environment and recreate with virtualenvwrapper:

# dumping part
cd /home/user/projects/old_project/
source .venv/bin/activate (`.venv` is the dir with venv)
pip freeze > requirements.txt
which python > python_bin.txt
deactivate
rm -rf .venv

# virtualenvwrapper part
mkvirtualenv -a /home/user/projects/old_project/ -p `cat python_bin.txt` -r requirements.txt SOME_PROJ

Conda

Here I will cover main conda environment management commands only, full list: conda help and conda env --help

Action	Command
List all environments and locations	conda env list
Activate another environment	conda activate ENVNAME
Reactivate base environment	conda activate base
Create an empty new env	conda env create -n ENVNAME
Clone existing environment	conda create --clone ENVNAME
Create new env from file (.yaml)	conda env create -n ENVNAME --file ENV.yml
Create new env from file (.txt)	conda env create -n ENVNAME --file ENV.txt
Export env to file (.yaml)	conda env export ENVNAME > ENV.yml

Problems with Python installation on Windows

Mostly problems are related to incorrectly ENV variables. Some links with help to deal with this:

• Py2

• Py3

Can't install some module via pip

Sometimes to install additional package is impossible to do via pip because some pre-installed libraries (like MS Studio C++ or mysql/postgresql libraries) needed. So it's better to download and install precompiled packages from here:

• gohlke

Install of .whl package:

pip install <whl_file>

Problem with installing packages on Windows: Microsoft Visual C++ is missing

When you see this during installing some package with pip:

...
    error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": https://visualstudio.microsoft.com/downloads/

this means it is required to download and install Microsoft Visual C++ Build Tools:

• go to Downloads page > "Tools for Visual Studio 2019"

• Download and run installer for Build Tools for Visual Studio 2019

• Select checkbox for Build Tools and click "Install"

Best IDEs:

• Microsoft Visual Studio Code <-- recommended

  • Python extension <-- required

• JetBrains PyCharm (Community Edition is OK) or Fleet

• Sublime Text (with lot of Python plugins)

• Notepad++

• Spyder

• Eclipse with pydev

• vim