I’ve got a package which I’ve previously successfully built on ReadTheDocs, but this is no longer the case. My imports are as follows:
import pandas as pd import numpy as np import scipy import sys from annoy import AnnoyIndex from packaging import version from scipy.spatial import cKDTree from scipy.sparse import coo_matrix from umap.umap_ import fuzzy_simplicial_set from sklearn.neighbors import KDTree from sklearn.neighbors import DistanceMetric from sklearn.linear_model import Ridge
I cover these packages via mock importing in Sphinx.
autodoc_mock_imports = ['pandas','numpy','scipy','annoy','packaging','scipy.spatial','scipy.sparse','sklearn','sklearn.neighbors','sklearn.linear_model','umap','umap.umap_']
However, in spite of that, I appear to encounter issues when building ReadTheDocs side. It seems that despite being asked to mock
sklearn up, the thing’s trying to install it and failing.
Installed /home/docs/checkouts/readthedocs.org/user_builds/bbknn/envs/latest/lib/python3.7/site-packages/bbknn-1.4.0-py3.7.egg Processing dependencies for bbknn==1.4.0 Searching for scikit-learn Reading https://pypi.org/simple/scikit-learn/ Downloading https://files.pythonhosted.org/packages/db/e2/9c0bde5f81394b627f623557690536b12017b84988a4a1f98ec826edab9e/scikit-learn-0.24.0.tar.gz#sha256=076369634ee72b5a5941440661e2f306ff4ac30903802dc52031c7e9199ac640 Best match: scikit-learn 0.24.0 Processing scikit-learn-0.24.0.tar.gz Writing /tmp/easy_install-1f85jnk0/scikit-learn-0.24.0/setup.cfg Running scikit-learn-0.24.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-1f85jnk0/scikit-learn-0.24.0/egg-dist-tmp-mfl2n2vi Partial import of sklearn during the build process. Traceback (most recent call last): File "/tmp/easy_install-1f85jnk0/scikit-learn-0.24.0/setup.py", line 201, in check_package_status module = importlib.import_module(package) File "/home/docs/checkouts/readthedocs.org/user_builds/bbknn/envs/latest/lib/python3.7/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1006, in _gcd_import File "<frozen importlib._bootstrap>", line 983, in _find_and_load File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked ModuleNotFoundError: No module named 'numpy' Traceback (most recent call last): File "/home/docs/checkouts/readthedocs.org/user_builds/bbknn/envs/latest/lib/python3.7/site-packages/setuptools/sandbox.py", line 154, in save_modules yield saved File "/home/docs/checkouts/readthedocs.org/user_builds/bbknn/envs/latest/lib/python3.7/site-packages/setuptools/sandbox.py", line 195, in setup_context yield File "/home/docs/checkouts/readthedocs.org/user_builds/bbknn/envs/latest/lib/python3.7/site-packages/setuptools/sandbox.py", line 250, in run_setup _execfile(setup_script, ns) File "/home/docs/checkouts/readthedocs.org/user_builds/bbknn/envs/latest/lib/python3.7/site-packages/setuptools/sandbox.py", line 45, in _execfile exec(code, globals, locals) File "/tmp/easy_install-1f85jnk0/scikit-learn-0.24.0/setup.py", line 306, in <module> File "/tmp/easy_install-1f85jnk0/scikit-learn-0.24.0/setup.py", line 292, in setup_package File "/tmp/easy_install-1f85jnk0/scikit-learn-0.24.0/setup.py", line 227, in check_package_status ImportError: numpy is not installed. scikit-learn requires numpy >= 1.13.3. Installation instructions are available on the scikit-learn website: http://scikit-learn.org/stable/install.html
Why is this happening? Could this be related to the way I require dependencies in
setup.py, which somehow circumvents the mocks?
I have hotfix-generated a docstring by removing
install_requires, but that’s not an ideal solution. Anybody got anything?
Based on feedback from a helpful user, I eventually arrived at a less hack’y solution. Unfortunately, the discussion went AWOL because of an unhelpful user, who responded with
pip install -U numpy, waited for me to figure it out, edited their answer and requested I accept it. Upon being denied, the answer and comment thread vanished. I don’t even remember your name, helpful user, so I can’t credit you for the tip.
Apparently ReadTheDocs uses an old pip, and requiring
pip>=19.0 makes scikit-learn not install from source. As such, I added that line to
docs/requirements.txt, which I had previously set up to be a ReadTheDocs requirement file. This resulted in some progress – now rather than scikit-learn complaining about numpy, it was numba. Still, some synapses connected, and I just handled any dependency problems that arose via
docs/requirements.txt, the final contents of which are:
pip>=19.0 numpy llvmlite==0.35.0
This got the setup to clear. However, just because setup cleared, does not mean that
autodoc_mock_imports can be stripped away for whatever reason. Not sure why this is.
As such, the quick and dirty solution from the OP wherein you just strip stuff out of
install_requires is likely the easiest way around this. However, if you feel like faffing around to get it to be cleaner, and not requiring a dedicated non-functional commit to have documentation build, having a ReadTheDocs requirement file and installing numpy through it is likely to solve a lot of woes. No idea why llvmlite was also this fussy, and somehow worked when in the requirements file but not otherwise, but got there in the end.