While installing packages in requirements.txt using Conda through the following command
conda install --yes --file requirements.txt
If a package in requirements.txt is not available, then it throws a "No package error" such as the one shown below:
Using Anaconda Cloud api site https://api.anaconda.org Fetching package metadata: .... Error: No packages found in current linux-64 channels matching: nimfa ==1.2.3 You can search for this package on anaconda.org with anaconda search -t conda nimfa ==1.2.3
Instead of throwing an error, is it possible to change this behavior such that it installs all the available packages in requirements.txt and throws a warning for those that are not available?
I would like this because, the package nimfa which the error says is not available, can be pip installed. So if I can change the behavior of conda install --yes --file requirements.txt
to just throw a warning for unavailable packages, I can follow it up with the command pip install -r requirments.txt
in .travis.yml so TravisCI attempts to install it from either place where it is available.
I ended up just iterating over the lines of the file
$ while read requirement; do conda install --yes $requirement; done < requirements.txt
Edit: If you would like to install a package using pip if it is not available through conda, give this a go:
$ while read requirement; do conda install --yes $requirement || pip install $requirement; done < requirements.txt
Edit: If you are using Windows (credit goes to @Clay):
$ FOR /F "delims=~" %f in (requirements.txt) DO conda install --yes "%f" || pip install "%f"
You can do this as mentioned in this
Export to .yml file
conda env export > freeze.yml
To reproduce:
conda env create -f freeze.yml
Pbms's answer here is the right way to do it, assuming you have an existing environment to copy off of. Conda is fully capable of installing both Conda packages and pip packages, as listed in environment.yml
. I wanted to document the whole process in more detail. Note that I am using folder-based environments, which is why I added --prefix [path to environment folder]
to most of the commands.
Say you installed an environment for an existing project to a folder called env
in the current folder, like this:
conda create --prefix ./env
You'd generate environment.yml
for that project's environment like this:
conda env export --prefix ./env > environment.yml
You'd create a new environment within some other folder by copying environment.yml
to there and then running this from there:
conda env create --prefix ./env --file environment.yml
You'd get an already-existing environment to match environment.yml
by once again copying environment.yml
to there and then running this from there:
conda env update --prefix ./env --file environment.yml --prune
With the environment in question active, you'd verify the state of its packages like this:
conda list
This is an abridged version of what that command might print (note that the pip packages are marked pypi
):
# Name Version Build Channel
pip 19.2.2 py37_0
python 3.7.4 h5263a28_0
numpy 1.16.4 py37h19fb1c0_0
pandas 0.25.1 py37ha925a31_0
pyodbc 4.0.27 py37ha925a31_0
ibm-db 3.0.1 pypi_0 pypi
ibm-db-sa 0.3.5 pypi_0 pypi
Finally, this is an abridged version of what environment.yml
might look like (note that the pip packages are listed in their own category):
dependencies:
- pip=19.2.2=py37_0
- python=3.7.4=h5263a28_0
- numpy=1.16.4=py37h19fb1c0_0
- pandas=0.25.1=py37ha925a31_0
- pyodbc=4.0.27=py37ha925a31_0
- pip:
- ibm-db==3.0.1
- ibm-db-sa==0.3.5
Be aware that using Conda and pip together can cause some heartburn because they can unknowingly blow away each other's dependencies. You are supposed to install all of your Conda packages first and then all of your pip packages afterward, rather than alternating between the two. If your environment breaks, the official recommendation is to delete and recreate it (from your environment.yml
file). For more details, see this guide:
https://www.anaconda.com/using-pip-in-a-conda-environment/
For those looking, I used this as @TillHoffmann 's solution for the fish shell:
$ while read requirement; conda install --yes $requirement; end < requirements.txt
And
$ while read requirement; conda install --yes $requirement;or pip install $requirement; end < requirements.txt
Success story sharing
pip install
packages which are not available usingconda install
, while iterating?FOR /F "delims=~" %f in (requirements.txt) DO conda install --yes "%f" || pip install "%f"
Works very well. Thanks!while read req; do if [[ $req != "#"* ]]; then conda install --yes $requirement || pip install $requirement; fi; done < requirements.txt