Commit Graph

716 Commits

Author SHA1 Message Date
Matthew Planchard
4f4f1ffa26
Merge remote-tracking branch 'origin/master' into maint/update-elfjes-storage-be-to-gh-actions 2020-11-15 20:16:55 -06:00
Matthew Planchard
d0694d9e15
Switch to GH actions (#361)
* Switch to GH actions

Removes the travis config and adds a GH actions config. Resolves #360.

As part of this:

- only runs the README check once, instead of for each python version
- only runs mypy once, removing it from tox
- unifies the pypy and cpython tests in tox by separating dev and test
  requirements, and only installing the latter for running tests in tox

* Update README w/badge & link to GH actions tests
2020-11-15 20:08:58 -06:00
Matthew Planchard
70566b986e
Merge branch 'master' into backend_interface 2020-11-15 16:15:14 -06:00
PelleK
4b1bd1c9db
Refactor test_server to increase speed (#354)
I gave test_server.py some much needed attention. This file now take ~30 seconds on my machine to run (down from 130 seconds), and I cleaned up the code a little. Let's see how this goes in CI

Commits:
-------------
* minimize time.sleep, convert to pathlib
* refactor, dry code
* run black

Co-authored-by: Matthew Planchard <mplanchard@users.noreply.github.com>
2020-11-15 15:57:53 -06:00
PelleK
4e1fd1eedc
Fix version incompatibility that breaks twine in docker test (#356)
* Fix version incompatibility that breaks twine in docker test

* upgrade pip before using feature flag
2020-11-15 15:08:57 -06:00
Pelle Koster
52912a931d fix more silly mistakes 2020-10-30 15:22:25 +01:00
Pelle Koster
46c6a481eb fix mypy 2020-10-30 15:14:58 +01:00
Pelle Koster
d8961fea5c fix cli, add tests 2020-10-30 15:05:08 +01:00
Pelle Koster
669b8b6146 backend as a cli argument 2020-10-30 14:24:08 +01:00
Pelle Koster
4fe62381b0 fix tab indentation 2020-10-30 12:19:49 +01:00
Pelle Koster
f4728b3884 fix merge mistakes 2020-10-30 12:11:39 +01:00
Pelle Koster
528736d3d4 exclude bottle.py from coverage 2020-10-30 11:49:12 +01:00
Pelle Koster
a8ddaedeb4 Merge branch 'master' of github.com:/pypiserver/pypiserver into backend_interface 2020-10-30 11:48:32 +01:00
Matthew Planchard
15d81147ac
Add python 3.9 testing (#351)
* Add python 3.9 testing

* Add 3.9 proper

According to this, it's available now: https://travis-ci.community/t/python-3-9-0-build/10091/18
2020-10-28 07:11:06 -05:00
Matthew Planchard
c668b1814a
Use argparse config throughout app (#349)
This PR is a pretty substantial refactor of the entrypoints of pypiserver (`__main__` and `__init__`) to use the argparse-based config added in #339.

- Updated `RunConfig` and `UpdateConfig` classes to have exclusive init kwargs, instead of taking an namespace. This turned out to be much easier when working with the library-style app initialization in `__init__`, both for direct instantiation and via paste config
- Added an `iter_packages()` method to the `RunConfig` to iterate over packages specified by the configuration (note @elfjes, I think that replacing this with e.g. a `backend` reference will be a nice way to tie in #348)
- Added a general-purpose method to map legacy keyword arguments to the `app()` and `paste_app_factory()` functions to updated forms
- Refactored the `paste_app_factory()` to not mutate the incoming dictionary
- Removed all argument-parsing and config-related code from `__main__` and `core`
- Moved `_logwrite` from `__init__` to `__main__`, since that was the only place it was being used after the updates to `core`
- Updated `digest_file` to use `hashlib.new(algo)` instead of `getattr(hashlib, algo)`, because the former supports more algorithms
- Updated `setup.py` to, instead of calling `eval()` on the entirety of `__init__`, to instead just evaluate the line that defines the version
- Assigned the config to a `._pypiserver_config` attribute on the `Bottle` instance to reduce hacky test workarounds
- Fixed the tox config, which I broke in #339 

* Config: add auth & absolute path resolution

* Config: check pkg dirs on config creation

* Instantiate config with kwargs, not namespace

* WIP: still pulling the threads

* Init seems to be working

* tests passing locally, still need to update cache

* Fix tox command

* unused import

* Fix typing

* Be more selective in exec() in setup.py

* Require accurate casing for hash algos

* Remove old comment

* Comments, minor updates and simplifications

* move _logwrite to a more reasonable place

* Update config to work with cache

* Type cachemanager listdir in core

* Update config module docstring, rename method

* Add more comments re: paste config

* Add comments to main, remove unneded check

* Remove commented code

* Use {posargs} instead of [] for clarity in tox

* Add dupe check for kwarg updater

* Remove unused references on app instance

* Fix typo

* Remove redundancy in log level parsing
2020-10-25 18:48:28 -05:00
Pelle Koster
abc1ae0c47 rm dead code 2020-10-20 22:22:56 +02:00
Pelle Koster
666c5d6a9e fix unittest 2020-10-20 22:19:05 +02:00
Pelle Koster
055d49fe50 check for existing package recursively, bugfix, some more pathlib 2020-10-20 22:09:24 +02:00
Pelle Koster
176a7179e0 bugfix when checking invalid whl file 2020-10-20 20:30:59 +02:00
Pelle Koster
2af9649b7a minor changes 2020-10-19 22:47:56 +02:00
Pelle Koster
559546c1bb add package count method to backend 2020-10-19 22:16:43 +02:00
Pelle Koster
dee063a196 add package count method to backend 2020-10-19 22:15:29 +02:00
Pelle Koster
b0fa1fe9f8 add more type hints to pass mypy, fix tox.ini 2020-10-19 22:00:34 +02:00
PelleK
aefa3161fc
Apply suggestions from code review
Co-authored-by: Matthew Planchard <mplanchard@users.noreply.github.com>
2020-10-19 20:56:36 +02:00
Pelle Koster
0e32bd4b9e fix typos, run black 2020-10-15 18:11:12 +02:00
Pelle Koster
a162cd1c2f Merge branch 'master' of github.com:elfjes/pypiserver into backend_interface 2020-10-15 07:53:07 +02:00
Pelle Koster
e95c8ad1a7 WIP file backends as plugin 2020-10-14 23:08:27 +02:00
Pelle Koster
50a77964e4 better testability of cache 2020-10-14 22:33:26 +02:00
Pelle Koster
b4e288f1df WIP make cache better testable 2020-10-14 22:18:14 +02:00
Pelle Koster
dd12820f18 WIP caching backend 2020-10-14 21:01:37 +02:00
Pelle Koster
c959372a43 add digester func as attribute to pkgfile 2020-10-11 22:06:37 +02:00
Matthew Planchard
47d6efe196
Restore ability to drop hashing in new config (#347)
Thanks @elfjes for pointing out that I'd missed this! I also went ahead
and bumped the version in the README to 2.0.0dev1, so that it's clear
that what's in master shouldn't be what people expect from pypi or in the
docker image.
2020-10-11 14:16:57 -05:00
Pelle Koster
3e99067ec5 rename prefix to project, since that's more descriptive 2020-10-11 12:00:09 +02:00
Pelle Koster
c396cbc293 further implement the current filestorage as simple file backend 2020-10-10 22:31:07 +02:00
Pelle Koster
42f5e5af9a rename pkg_utils to pkg_helpers to prevent confusion with stdlib pkgutil 2020-10-10 16:14:57 +02:00
Pelle Koster
0dac621a15 move some functions around in preparation for backend module 2020-10-10 16:06:16 +02:00
Matthew Planchard
8014fa56fc
Merge branch 'v1.4.x' 2020-10-10 08:21:35 -05:00
Matthew Planchard
776d319eb1
chore(ver): bump 1.4.1-->1.4.2 v1.4.2 2020-10-10 08:15:56 -05:00
Matthew Planchard
ab8b33e5fb
CHORE: prep for v1.4.2 2020-10-10 08:14:50 -05:00
PelleK
e0bff63ab9
fix docker entrypoint script, improve docker build speed/caching (#344)
Co-authored-by: Pelle Koster <pelle.koster@nginfra.nl>
2020-10-10 08:12:06 -05:00
Matthew Planchard
0594c33e53
Backwards-compatible argparse config (not yet in use) (#339)
Adds an argparse config that, while adding subcommands (`pypi-server run` and `pypi-server update`), retains full commandline backwards compatibility with the existing config parsing logic.

There's a bit of hackery required to do this, so this also issues a warning if using the non-subcommand arguments, allowing us to potentially remove support for the old form in our next next major version bump (i.e. 3.0).

Also adds a `.pyproject.toml` with a black config, and a mypy config block to `setup.cfg`.

`mypy` is now called in `tox`, currently only for `config.py`, because nothing else typechecks successfully.

----

* WIP: argparse config

* Complete config

* Test all the config options

* Another test and a note re: being unused

* mypy config, call mypy in tox

* No mypy on pypy

* Fix tox config

* Add venv to black ignore

* fix tox config (again)

* Fix formatting, simplify error handling

* FMT: Run black on changed files
2020-10-08 19:37:39 -05:00
PelleK
d886bc2eba
Cleanup code to python 3.6 (#342)
* Cleanup setup.py

* remove explicit inheritance from object

* convert most string interpolations to f-strings

Co-authored-by: Pelle Koster <pelle.koster@nginfra.nl>
2020-10-07 20:45:51 -05:00
Matthew Planchard
b44edb61ce
CHORE: pull CHANGES.rst from v1.4.x 2020-10-05 21:15:33 -05:00
PelleK
8101cf9192
Run black on codebase (#336)
* run black on codebase

* add black check to travis ci

* add pyproject.toml, revert black on bottle.py

Co-authored-by: Pelle Koster <pelle.koster@nginfra.nl>
2020-10-05 21:04:22 -05:00
Matthew Planchard
5ca5351d80
chore(ver): bump 1.4.0-->1.4.1 v1.4.1 2020-10-05 20:51:01 -05:00
Matthew Planchard
b322aebb87
FIX: bash -> sh in entryoint 2020-10-05 20:26:28 -05:00
Matthew Planchard
1f696e56c1
DOC: Update changelog and readme for v1.4.1 2020-10-05 20:20:53 -05:00
Matthew Planchard
75ec4e95c0
FIX: only chown /data/packages in entrypoint
@stephen-dexda pointed out in #341 that our update in #330 changed
the `chown` operation to apply to the entire `/data` directory,
rather than just `/data/packages`. For anyone who was previously
relying on a workflow like mounting a read-only secrets directory
into `/data` to host authentication information, this broke their
workflow.

This fix sets `entrypoint.sh` to only `chown` `/data/packages`, which
should ensure that the permissions issues resolved by #330 (e.g. #309)
remain fixed, while also fixing the issue in #341.
2020-10-05 20:13:16 -05:00
Matthew Planchard
4ab210c82b
MAINT: drop standalone, drop py 2.7 and 3.5 (#338) 2020-10-03 22:25:14 -05:00
Matthew Planchard
b208103951
chore(ver): bump 1.3.2-->1.4.0 v1.4.0 2020-10-03 17:45:40 -05:00