Development Notes

Notes for maintaining neuprint-python.

Packaging and Release

neuprint-python is packaged for both conda (on the flyem-forge channel) and pip (on PyPI).

The package version is automatically inferred from the git tag. To prepare a release, follow these steps:

cd neuprint-python

# Update the change log!
code docs/source/changelog.rst
git commit -m "Updated changelog" docs/source/changelog.rst

# Do the tests still pass?
pytest .

# Do the docs still build?
cd docs && make html && cd -

# Tag the git repo with the new version
NEW_TAG=0.3.1
git tag -a ${NEW_TAG} -m ${NEW_TAG}
git push --tags origin

# Build and upload the conda package
conda build conda-recipe
anaconda upload -u flyem-forge $(conda info --base)/conda-bld/noarch/neuprint-python-${NEW_TAG}-py_0.tar.bz2

# Build and upload the PyPI package
./upload-to-pypi.sh

# Deploy the docs
./docs/deploy-docs.sh

Dependencies

If you need to add dependencies to neuprint-python, edit dependencies.txt (which is used by the conda recipe). You should also update environment.yml so that our binder container will acquire the new dependencies when users try out the interactive tutorial. After publishing a new conda package with the updated dependencies, follow these steps on a Linux machine:

#!/bin/bash
# update-deps.sh

set -e

# Create an environment with the binder dependencies
TUTORIAL_DEPS="ipywidgets bokeh holoviews hvplot"
SIMULATION_DEPS="ngspice umap-learn scikit-learn matplotlib"
BINDER_DEPS="neuprint-python jupyterlab ${TUTORIAL_DEPS} ${SIMULATION_DEPS}"
conda create -y -n neuprint-python -c flyem-forge -c conda-forge ${BINDER_DEPS}

# Export to environment.yml, but relax the neuprint-python version requirement
conda env export -n neuprint-python > environment.yml
sed --in-place 's/neuprint-python=.*/neuprint-python/g' environment.yml

git commit -m "Updated environment.yml for binder" environment.yml
git push origin master

Documentation

The docs are built with Sphinx. See docs/requirements.txt for the docs dependencies. To build the docs locally:

cd neuprint-python/docs
make html
open build/html/index.html

We publish the docs via github pages. Use the script docs/deploy-docs.sh to build and publish the docs to GitHub in the gh-pages branch. (At some point in the future, we may automate this via a CI system.)

./docs/deploy-docs.sh

Interactive Tutorial

The documentation contains a tutorial which can be launched interactively via binder. To update the tutorial contents, simply edit the .ipynb file and re-build the docs.

If the binder setup is broken, make sure the dependencies are configured properly as described above.

It takes a few minutes to initialize the binder container for the first time after a new release. Consider sparing your users from that by clicking the binder button yourself after each release.

Tests

The tests require pytest, and they rely on the public hemibrain:v1.2.1 dataset on neuprint.janelia.org, which means you must define NEUPRINT_APPLICATION_CREDENTIALS in your environment before running them.

To run the tests:

cd neuprint-python
PYTHONPATH=. pytest neuprint/tests