This PR is a pretty substantial refactor of the entrypoints of pypiserver (`__main__` and `__init__`) to use the argparse-based config added in #339.
- Updated `RunConfig` and `UpdateConfig` classes to have exclusive init kwargs, instead of taking an namespace. This turned out to be much easier when working with the library-style app initialization in `__init__`, both for direct instantiation and via paste config
- Added an `iter_packages()` method to the `RunConfig` to iterate over packages specified by the configuration (note @elfjes, I think that replacing this with e.g. a `backend` reference will be a nice way to tie in #348)
- Added a general-purpose method to map legacy keyword arguments to the `app()` and `paste_app_factory()` functions to updated forms
- Refactored the `paste_app_factory()` to not mutate the incoming dictionary
- Removed all argument-parsing and config-related code from `__main__` and `core`
- Moved `_logwrite` from `__init__` to `__main__`, since that was the only place it was being used after the updates to `core`
- Updated `digest_file` to use `hashlib.new(algo)` instead of `getattr(hashlib, algo)`, because the former supports more algorithms
- Updated `setup.py` to, instead of calling `eval()` on the entirety of `__init__`, to instead just evaluate the line that defines the version
- Assigned the config to a `._pypiserver_config` attribute on the `Bottle` instance to reduce hacky test workarounds
- Fixed the tox config, which I broke in #339
* Config: add auth & absolute path resolution
* Config: check pkg dirs on config creation
* Instantiate config with kwargs, not namespace
* WIP: still pulling the threads
* Init seems to be working
* tests passing locally, still need to update cache
* Fix tox command
* unused import
* Fix typing
* Be more selective in exec() in setup.py
* Require accurate casing for hash algos
* Remove old comment
* Comments, minor updates and simplifications
* move _logwrite to a more reasonable place
* Update config to work with cache
* Type cachemanager listdir in core
* Update config module docstring, rename method
* Add more comments re: paste config
* Add comments to main, remove unneded check
* Remove commented code
* Use {posargs} instead of [] for clarity in tox
* Add dupe check for kwarg updater
* Remove unused references on app instance
* Fix typo
* Remove redundancy in log level parsing
Thanks @elfjes for pointing out that I'd missed this! I also went ahead
and bumped the version in the README to 2.0.0dev1, so that it's clear
that what's in master shouldn't be what people expect from pypi or in the
docker image.
Adds an argparse config that, while adding subcommands (`pypi-server run` and `pypi-server update`), retains full commandline backwards compatibility with the existing config parsing logic.
There's a bit of hackery required to do this, so this also issues a warning if using the non-subcommand arguments, allowing us to potentially remove support for the old form in our next next major version bump (i.e. 3.0).
Also adds a `.pyproject.toml` with a black config, and a mypy config block to `setup.cfg`.
`mypy` is now called in `tox`, currently only for `config.py`, because nothing else typechecks successfully.
----
* WIP: argparse config
* Complete config
* Test all the config options
* Another test and a note re: being unused
* mypy config, call mypy in tox
* No mypy on pypy
* Fix tox config
* Add venv to black ignore
* fix tox config (again)
* Fix formatting, simplify error handling
* FMT: Run black on changed files
* Cleanup setup.py
* remove explicit inheritance from object
* convert most string interpolations to f-strings
Co-authored-by: Pelle Koster <pelle.koster@nginfra.nl>
* run black on codebase
* add black check to travis ci
* add pyproject.toml, revert black on bottle.py
Co-authored-by: Pelle Koster <pelle.koster@nginfra.nl>
@stephen-dexda pointed out in #341 that our update in #330 changed
the `chown` operation to apply to the entire `/data` directory,
rather than just `/data/packages`. For anyone who was previously
relying on a workflow like mounting a read-only secrets directory
into `/data` to host authentication information, this broke their
workflow.
This fix sets `entrypoint.sh` to only `chown` `/data/packages`, which
should ensure that the permissions issues resolved by #330 (e.g. #309)
remain fixed, while also fixing the issue in #341.
* log to stdout
* add stdout logging to config and test it
* remove non-implemented parameter from docs
* configure log stream based on config, somehow this change got lost
* fix unittests for other python versions
* option to specify log stream
* Be more explicit in usage text
Co-authored-by: Matthew Planchard <mplanchard@users.noreply.github.com>
* remove redundant arguments
* be more consistent in usage text
* add test for disabling stream logging
* fix side-effect of unittests
Co-authored-by: Matthew Planchard <mplanchard@users.noreply.github.com>
* Add the option to specify list of modules we don't want to update
Signed-off-by: Peter Slovak <peter.slovak@websupport.sk>
* Fix docs
Signed-off-by: Peter Slovak <peter.slovak@websupport.sk>
* Minimize the number of strip() calls
Co-authored-by: Matthew Planchard <mplanchard@users.noreply.github.com>
* Log an exception when we fail to open/read the package blacklist file
* Abort server startup if we fail to read the blacklist file
Co-authored-by: Matthew Planchard <mplanchard@users.noreply.github.com>
Include watchdog as a dependency in the docker image pip requirements.
This is very useful in situations where your packages are mounted over
the network which may be a typical use case for the image.
As passlib is included as a default dependency this seems like a natural
addition.
* Drop support for Python 3.4
Python 3.4 is no longer supported. Even pyenv is failing to install it
for me, because apparently the currnet version of `ensurepip` bombs for
3.4. Pypiserver may still work on 3.4, but testing on it has becomes
more of a hassle than it's worth.
* Fix @mplanchard's email address
Just realized my email address in the authors' file has been wrong for
pretty much forever.
* Remove GL CI
GitLab CI is nice, but doesn't support forks, and so isn't going to be
viable for this project.
CherryPy changed the import location.
This attempts the new or falls back to the old location.
Co-authored-by: Tiemen Schuijbroek <t.j.l.schuijbroek@ratio-case.nl>
* Slightly clarify the relationship to warehouse.
Closes: #307
* Wording tweaks and split out the related software into its own section.
* Separate the note in a note directive
Co-authored-by: Matthew Planchard <mplanchard@users.noreply.github.com>
Co-authored-by: Matthew Planchard <mplanchard@users.noreply.github.com>
`HTTPError(code, message, format_args)` appears to be equivalent to `HTTPError(status=code, body=message, exception=format_args)` which is not what we want here. The string formatting can't be deferred like in logging library usage.
The gitlab tests started failing with GL's newest "docker in docker"
image due to the lack of some core C library headers. This ensures that
everything needed for twine (i.e. cryptography) is present before doing
a pip install for twine.
Resolves#264
Generally, pip doesn't need to be available for `pypiserver` to work
correctly, but the `-U` command to update packages requires it to be
importable. This ensures the `pip` module will be available in the final
image.
This reverts commit de6a9ce19440975759e979fc47ba870192c99370.
Not going to be able to use GL because they do not support forks, so
reverting associated changes.
I didn't realize GitLab does not support running PRs on forks for
mirrored GitHub repos. I have switched other projects to Azure
pipelines, but just to get things going, I'm going to pull .travis.yml
back in here.
* [268] Ensure remove_pkg removes all pkg instances
Reported by @esciara in #268.
Previously, the `remove_pkg` command was only removing the first
matching package that it found so if, for example, there were a .tar.gz
file and a .whl file, it would only remove one of them.
Of course, it could be run in succession to accomplish full removal, but
the expected behavior is that removal will remove the package entirely.
Here, I've grouped `remove_pkg` related tests into a test class, added
some tests that verify the expected behavior, and updated the
`remove_pkg` method to remove all matching packages.
* CHANGES.rst
