Would you accept a PR for automatic macOS and Linux wheel generation?

[jbarlow83]

I'm interested in switching my project, OCRmyPDF, to PyMuPDF as the PDF backend since it's far superior to PyPDF2 and the other alternatives, and would probably eliminate my dependency on Ghostscript as well.

However, PyMuPDF installation on non-Windows is a bit of a pain right now, ultimately because Artifex doesn't provide shared libraries for MuPDF.

Would you accept a PR that added a Travis CI compile and deploy step to automatically build binary wheels for PyMuPDF? Basically I want to make sure you don't have some philosophical objection or already have work in progress before I spend time on it. Would boil down if, there is a tagged release, installed the appropriate Homebrew packages, generate a binary wheel and upload it to PyPI. The benefit is that this would enable mac users to "pip install pymupdf" without needing homebrew + gcc + mupdf-tools installed.

[jbarlow83]

Updated initial comment because I found a straightforward way to make manylinux1 wheels too.

[JorjMcKie]

Hi there,
I would be more than happy if we could incorporate an elegant way of installing on non-Windows systems!

I am not the repo owner, but I am sure @rk700 will have no objections either!

My skills on non-Windows systems is too low to contribute much - except maybe testing any new installlation procedure on Linux virtual machines (Oracle Virtual Box).

[jbarlow83]

Great!

[rk700]

@jbarlow83 Thank you so much for the help! Now Travis CI is integrated and the wheels can be built successfully.

However it failed when uploading wheels to PyPI: https://travis-ci.org/rk700/PyMuPDF/builds/355674679

I'll take a deeper look at the issue.

[jbarlow83]

I have one of my projects uploading to legacy PyPI since it saved hassle. May be this will help.

... twine --repository-url https://upload.pypi.org/legacy/ ...

You could change the upload so that non-tagged builds go to PyPI, as a way of confirming things work without having to tag everything, e.g.

if [ "$TRAVIS_TAG" == "" ]; then
   ... twine  --repository-url https://testpypi.pypi.org/legacy/ ...
else
   # release
   ... twine  --repository-url https://upload.pypi.org/legacy/ ...
fi

[jbarlow83]

I checked the Travis logs again. Did you set the environment variable TWINE_PASSWORD in Travis settings to the password for rk700's PyPI account? That would explain the auth failure.

On this screen:

[JorjMcKie]

First of all, thank you very much for your contribution to our repo!
It gave me a kick to look around for other options on how to use Travis.

A first observation on the generated wheels (whether they have yet been successfully passed on to PyPI or not): they look really big compared to my homegrown Windows wheels: Linux 26.x MB versus less than 3 MB on Windows!
Once I can get my hands on one of the Linux / MAC wheels I will curiously investigate what is going on.

My first guess:
I am configuring my local MuPDF via /mupdf/include/mupdf/fitz/config.h, before I generate MuPDF object and PyMuPDF binaries. This mainly boils down to / kicking out the plethora of NOTO fonts present in the original MuPDF source. They are of no use in PyMuPDF. And I know from my Windows environment, that massaging config.h brings down the PyMuPDF binary size down to below 5 MB.

As you have been so resourceful: would you have an idea on how to force Travis to also use a modified config.h?

[jbarlow83]

You're welcome... I've learned a lot about Travis on a few different projects so it really didn't take that long.

Yes, fonts could easily explain the size difference.

I should probably explain how it actually works.

The install: block in .travis.yml downloads $MUPDF and extracts it to a folder named mupdf/. Then cibuildwheel kicks off a Docker container running on Travis, which has access the current directory, and runs the build, leaving the results in the same folder. The reason for the Docker container is that Travis runs Ubuntu 14.04, and manylinux1 wheels must be built on CentOS 5 for compatibility. The CIBW_BEFORE_BUILD environment passes commands to the Docker container (i.e. before it build the wheels, run make on mupdf).

So right after this command

mkdir mupdf && wget -q $MUPDF -O - | tar zx -C mupdf --strip-components=1

...you can patch mupdf any way you see fit, and cibuildwheel will build the modified code the same way for macOS and Linux.

I think the easiest option will be to include a copy of config.h in PyMuPDF. Travis will have a git checkout of PyMuPDF in the current working directory, so you just copy it to the mupdf/ directory we downloaded and set up.

install:
  - $PIP install cibuildwheel==0.7.0 twine
  - mkdir mupdf && wget -q $MUPDF -O - | tar zx -C mupdf --strip-components=1
  - cp fitz/_mupdf_config.h mupdf/include/mupdf/fitz/config.h

Or you could do a diff/patch of config.h, so it's more robust to upstream changes.