* Docker improvements
This addresses much of what was brought up in #359. Specifically, it:
- Significantly improves testing for the Docker image, adding a
`docker/test_docker.py` file using the regular pytest machinery to
set up and run docker images for testing
- Hopefully addresses a variety of permissions issues, by being explicit
about what access pypiserver needs and asking for it, only erroring
if that access is not available
- Requires RX permissions on `/data` (R to read files, X to list files
and to be able to cd into the directory. This is important since
`/data` is the `WORKDIR`)
- Requires RWX permissions on `/data/packages`, so that we can list
packages, write packages, and read packages.
- When running in the default configuration (as root on Linux or
as the pypiserver-named rootish user on Mac), with no volumes
mounted, these requirements are all satisfied
- Volume mounts still must be readable by the pypiserver user (UID
9898) in order for the container to run. However, we now error early
if this is not the case, and direct users to a useful issue.
- If the container is run as a non-root, non-pypiserver user (e.g.
because someone ran `docker run --user=<user_id>`, we try to run
pypiserver as that user). Provided that user has access to the
necessary directories, it should run fine.
- Fixes issues with running help and similar commands
- Updates the Docker image to use `PYPISERVER_PORT` for port
specification, while still falling back to `PORT` for backwards
compatibility
- Moves some docker-related things into a `/docker` directory
- Adds a `Makefile` for building a test fixture package sdist and wheel,
so that test code can call `make mypkg` and not need to worry about it
potentially building multiple times
The only issue #359 raises that's not addressed here is the one of
running pypiserver in the Docker container using some non-default server
for performance. I would like to do some benchmarking before deciding on
what to do there.
I gave test_server.py some much needed attention. This file now take ~30 seconds on my machine to run (down from 130 seconds), and I cleaned up the code a little. Let's see how this goes in CI
Commits:
-------------
* minimize time.sleep, convert to pathlib
* refactor, dry code
* run black
Co-authored-by: Matthew Planchard <mplanchard@users.noreply.github.com>
* Cleanup setup.py
* remove explicit inheritance from object
* convert most string interpolations to f-strings
Co-authored-by: Pelle Koster <pelle.koster@nginfra.nl>
* run black on codebase
* add black check to travis ci
* add pyproject.toml, revert black on bottle.py
Co-authored-by: Pelle Koster <pelle.koster@nginfra.nl>
Resolves#237
Previously, we were not running any sort of URL escaping on values
passed in from the client that were used for redirects. This allowed
injection attacks via URL encoded newlines in the original request.
This update ensures that all user-supplied paths that are used as
components of redirects are passed through `urllib.parse.quote()`
(or the python 2 equivalent) prior to being used in a redirect
response.
Also specified 127.0.0.1 rather than 0.0.0.0 (the default) in server
tests to avoid triggering firewall dialogs when testing on MacOS
Resolves#226
Cleaned up `test_server.py` a bit:
* Call `twine` via `Popen` instead of using their internal API, which
should help us from having to fix twine tests as often
* Cleaned up fixtures
* Some PEP8 updates and general formatting
* Grouped test fixtures and functions all before the tests themselves
Resolves#205 - pypi.python.org shutting down
* Updated the default fallback URL to `pypi.org/simple` rather than
`pypi.python.org/simple`
* Scrubbed references and links to `pypi.python.org`
* Fixed tests breaking due to the removal of `pip.main()` in pip 10.0 -
see pypa/pip#5080 for more info
In addition to including tests from @jameshiebert with slight
updates, I have also resolved an issue with newer versions of twine,
which require an extra parameter in the calls to `upload` and
`register`.
* Updated .travis.yml to fix PEP 440 warnings
* Fixed twine calls
We were getting test failures on multiple branches in
`test_server.py`. I first investigated a warning message
popping up in every test run:
```
PEP440Warning,
/home/travis/virtualenv/python3.5.2/lib/python3.5/site-packages/pkg_resources/__init__.py:2510: PEP440Warning: 'setuptools (git-0.4.0)' is being parsed as a legacy, non PEP 440, version. You may find odd behavior and sort order. In particular it will be sorted as less than 0.0. It is recommend to migrate to PEP 440 compatible versions.
```
Moving the installation of setuptools, pip, sphinx, and tox
into the `install` key for Travis resolved that issue, but
`test_server.py` tests were still failing. It turns out that
Twine 1.7.0 added support for SSL cert specification and,
in the process, changed the call signature for the `upload`
and `register` internal methods.
This PR fixes the calls so that they align with Twine's new
function signature. Note that tests now fail on Twine <1.7.0,
so I have also updated the dev requirements file.
Dmitrii Orlov