diff --git a/CHANGES.rst b/CHANGES.rst index 1d9f9b1..a7249f5 100644 --- a/CHANGES.rst +++ b/CHANGES.rst @@ -3,22 +3,22 @@ Changelog 1.2.0 (2016-XX-XX) ------------------ -"New Day", with less rigorous support for `python-2 < 2.7` and `python-3 < 3.3`. +"New Day": less rigorous support for ``python-2 < 2.7`` and ``python-3 < 3.3``. - #38, #139, #140: Package normalizations and :pep:`503` updates (thanks to @dpkp): - - URLs are redirected unless they end in `'/'` (expect packages themselves). - - Package names are normalized: lower-case with all runs of the characters, - and `[-_]` replaced with a single `'-'` character. + - Package names are normalized: convert all characters to lower-case + and replace any of ``[-_.]`` with a dash(``'-'``). - The simple index only lists normalized package names. - Any request for a non-normalized package name is redirected to the normalized name. + - URLs are redirected unless they end in ``'/'`` (expect packages themselves). -- #114: Added `pip search` support (thanks to @blade2005) +- #80, #114: Added ``pip search`` support (thanks to @blade2005) - #117, #122, #124/#127/#128: FIX startup regressions introduced by - previous `v1.1.10` (thanks to @virtuald, @Oneplus, @michaelkuty, @harcher81, + previous ``v1.1.10`` (thanks to @virtuald, @Oneplus, @michaelkuty, @harcher81, @8u1a). -- #113: FIX over-writing of packages even when without `--overwrite` flag +- #113: FIX over-writing of packages even when without ``--overwrite`` flag (thanks to @blade2005). - #107, #108, #110: Test against *python-3.5*. - #112, #118, #119: Fixes for *paste*, *gunicorn* and other *WSGI* servers @@ -27,9 +27,8 @@ Changelog in dependent libraries (thanks @dpkp). - #116: Add cache for speeding up GPG signatures (thanks to @virtuald). - Provide sample ``systemd`` *unit-file* (thanks to @ssbarnea). -- Add usage instructions for related project ``pypi-uploader`` and mention also - project - (thanks @ssbarnea & @bibby). +- Add usage instructions for related project ``pypi-uploader`` + (thanks to @ssbarnea & @bibby). - #129, #131: Other minor fixes and improvements (thanks to @bibby, @Oneplus, @8u1a). - Detailed changes recorded in `Github's milestone 1.2.0 diff --git a/README.rst b/README.rst index 67e708c..50b4167 100644 --- a/README.rst +++ b/README.rst @@ -10,8 +10,8 @@ pypiserver - minimal PyPI server for use with pip/easy_install |pypi-ver| |travis-status| |dependencies| |downloads-count| |python-ver| \ |proj-license| -:Version: 1.2.0.dev1 -:Date: 2016-06-17 +:Version: 1.2.0b0 +:Date: 2016-06-23 :Source: https://github.com/pypiserver/pypiserver :PyPI: https://pypi.python.org/pypi/pypiserver :Travis: https://travis-ci.org/pypiserver/pypiserver @@ -30,12 +30,12 @@ either with *pip*, *setuptools*, *twine* or simply copied with *scp*. Quickstart: Installation and Usage ================================== -*pypiserver* `> 1.2.x` works with python `2.7` and `3.3 --> 3.5` or *pypy*. -Python `3.0 --> 3.2` may also work, but it is not being tested for these +*pypiserver* ``> 1.2.x`` works with python ``2.7`` and ``3.3+`` or *pypy*. +Python ``3.0 --> 3.2`` may also work, but it is not being tested for these versions. -For legacy python versions, use `pypiserver-1.1.x` series. +For legacy python versions, use ``pypiserver-1.1.x`` series. -Run the following commands to get your PyPI server up and running:: +Run the following commands to get your *pypiserver* up and running:: ## Installation. pip install pypiserver ## Or: pypiserver[passlib,watchdog] @@ -54,11 +54,11 @@ From the client computer, type this:: See also `Client-side configurations`_ for avoiding tedious typing. .. Note:: - The above commands work on a unix-like operating system with a posix shell. - The ``~`` character expands to user's home directory. + The above commands work on a unix-like operating system with a posix shell. + The ``'~'`` character expands to user's home directory. - If you're using windows, you'll have to use their "windows counterparts". - The same is true for the rest of this documentation. + If you're using Windows, you'll have to use their "Windows counterparts". + The same is true for the rest of this documentation. Uploading packages from sources, remotely @@ -68,26 +68,27 @@ you may also upload them remotely with a ``python setup.py upload`` command. Currently only password-protected uploads are supported! #. First make sure you have the *passlib* module installed (note that - `passlib>=1.6` is required), which is needed for parsing the apache + `passlib>=1.6` is required), which is needed for parsing the Apache *htpasswd* file specified by the `-P`, `--passwords` option (see next steps):: pip install passlib -#. Create the apache *htpasswd* file with at least one user/password pair +#. Create the Apache *htpasswd* file with at least one user/password pair with this command (you'll be prompted for a password):: htpasswd -sc htpasswd.txt - .. Tip:: - Read this SO question for running `htpasswd` cmd under *Windows*: + .. Tip:: Read this SO question for running `htpasswd` cmd + under *Windows*: - http://serverfault.com/questions/152950/how-to-create-and-edit-htaccess-and-htpasswd-locally-on-my-computer-and-then-u + http://serverfault.com/questions/152950/how-to-create-and-edit-htaccess-and-htpasswd-locally-on-my-computer-and-then-u - or if you have bogus passwords for an internal service you may use this - public service: + or if you have bogus passwords that you don't care because they are for + an internal service (which is still "bad", from a security prespective...) + you may use this public service: - http://www.htaccesstools.com/htpasswd-generator/ + http://www.htaccesstools.com/htpasswd-generator/ It is also possible to disable authentication even for uploads. To avoid lazy security decisions, read help for ``-P`` and ``-a`` options. @@ -119,26 +120,27 @@ Currently only password-protected uploads are supported! python setup.py sdist upload -r local .. Tip:: - To avoid storing you passwords on disk in clear text, you may either: - - Use the `register` command with the `-r` option, like that:: + To avoid storing you passwords on disk, in clear text, you may either: - python setup.py sdist register -r local upload -r local + - use the ``register`` *setuptools*'s command with the ``-r`` option, + like that:: - - Use `twine `_ library which - breaks the procedure in two steps. + python setup.py sdist register -r local upload -r local - Furthermore, it allows you to pre-sign your files with PGP-Signatures - and upload also the generated `.asc` files to *pypiserver*:: + - use `twine`_ library, which + breaks the procedure in two steps. In addition, it supports signing + your files with PGP-Signatures and uploading the generated `.asc` files + to *pypiserver*:: - twine upload -r local --sign -identity user_name ./foo-1.zip + twine upload -r local --sign -identity user_name ./foo-1.zip .. Tip:: You can also upload packages using `pypi-uploader`_, which obviates the need to download packages locally prior to uploading them to - pypiserver. You can install it with ``pip install pypi-uploader``, and use - it like so, assuming you have a ``pypi_local`` source set up in your - ``~/.pypirc``:: + pypiserver. You can install it with ``pip install pypi-uploader``, and + assuming you have a ``pypi_local`` source set up in your ``~/.pypirc``, + use it like this:: pypiupload packages mock==1.0.1 requests==2.2.1 -i pypi_local pypiupload requirements requirements.txt -i pypi_local @@ -147,33 +149,33 @@ Currently only password-protected uploads are supported! Client-side configurations -------------------------- Always specifying the the pypi url on the command line is a bit -cumbersome. Since pypi-server redirects pip/easy_install to the -pypi.python.org index if it doesn't have a requested package, it's a +cumbersome. Since *pypiserver* redirects ``pip/easy_install`` to the +``pypi.python.org`` index if it doesn't have a requested package, it's a good idea to configure them to always use your local pypi index. -`pip` -~~~~~ -For *pip* this can be done by setting the environment variable -`PIP_EXTRA_INDEX_URL` in your `.bashrc`/`.profile`/`.zshrc`:: +Configuring *pip* +~~~~~~~~~~~~~~~~~ +For ``pip`` command this can be done by setting the environment variable +``PIP_EXTRA_INDEX_URL`` in your ``.bashr/.profile/.zshrc``:: export PIP_EXTRA_INDEX_URL=http://localhost:8080/simple/ -or by adding the following lines to `~/.pip/pip.conf`:: +or by adding the following lines to ``~/.pip/pip.conf``:: [global] extra-index-url = http://localhost:8080/simple/ .. Note:: - If you have installed *pypi-server* on a remote url without *https* - you wil receive an "untrusted" warning from *pip*, urging you to append - the `--trusted-host` option. You can also include this option permanently - in your configuration-files or environment variables. + If you have installed *pypiserver* on a remote url without *https* + you wil receive an "untrusted" warning from *pip*, urging you to append + the ``--trusted-host`` option. You can also include this option permanently + in your configuration-files or environment variables. -`easy_install` -~~~~~~~~~~~~~~ -For *easy_install* it can be configured with the following setting in -`~/.pydistutils.cfg`:: +Configuring *easy_install* +~~~~~~~~~~~~~~~~~~~~~~~~~~ +For ``easy_install`` command you may set the following configuration in +``~/.pydistutils.cfg``:: [easy_install] index_url = http://localhost:8080/simple/ @@ -211,12 +213,12 @@ The git repository contains a ``pypi-server-standalone.py`` script, which is a single python file that can be executed without any other dependencies. -Run the following commands to download the script with `wget`:: +Run the following commands to download the script with ``wget``:: wget https://raw.github.com/pypiserver/pypiserver/standalone/pypi-server-standalone.py chmod +x pypi-server-standalone.py -or with `curl`:: +or with ``curl``:: curl -O https://raw.github.com/pypiserver/pypiserver/standalone/pypi-server-standalone.py chmod +x pypi-server-standalone.py @@ -225,20 +227,20 @@ You can then start-up the server with:: ./pypi-server-standalone.py -Feel free to rename the script and move it into your `$PATH`. +Feel free to rename the script and move it into your ``$PATH``. -Running on heroku/dotcloud --------------------------- +Running on *heroku/dotcloud* +---------------------------- https://github.com/dexterous/pypiserver-on-the-cloud contains -instructions on how to run pypiserver on one of the supported cloud +instructions on how to run *pypiserver* on one of the supported cloud service providers. Detailed Usage ============== -Running ``pypi-server -h`` will print a detailed usage message:: +Enter ``pypi-server -h`` in the cmd-line to print a detailed usage message:: pypi-server [OPTIONS] [PACKAGES_DIRECTORY...] start PyPI compatible package server serving packages from @@ -359,37 +361,37 @@ Running ``pypi-server -h`` will print a detailed usage message:: Managing the package directory ------------------------------ -The `pypi-server` command has the `-U` option that searches for updates of +The ``pypi-server`` command has the ``-U`` option that searches for updates of available packages. It scans the package directory for available packages and searches on pypi.python.org for updates. Without further options ``pypi-server -U`` will just print a list of commands which must be run in order to get the latest version of each package. Output looks like:: - $ ./pypi-server -U - checking 106 packages for newer version + $ ./pypi-server -U + checking 106 packages for newer version - .........u.e...........e..u............. - .....e..............................e... - .......................... + .........u.e...........e..u............. + .....e..............................e... + .......................... - no releases found on pypi for PyXML, Pymacs, mercurial, setuptools + no releases found on pypi for PyXML, Pymacs, mercurial, setuptools - # update raven from 1.4.3 to 1.4.4 - pip -q install --no-deps --extra-index-url http://pypi.python.org/simple -d /home/ralf/packages/mirror raven==1.4.4 + # update raven from 1.4.3 to 1.4.4 + pip -q install --no-deps --extra-index-url http://pypi.python.org/simple -d /home/ralf/packages/mirror raven==1.4.4 - # update greenlet from 0.3.3 to 0.3.4 - pip -q install --no-deps --extra-index-url http://pypi.python.org/simple -d /home/ralf/packages/mirror greenlet==0.3.4 + # update greenlet from 0.3.3 to 0.3.4 + pip -q install --no-deps --extra-index-url http://pypi.python.org/simple -d /home/ralf/packages/mirror greenlet==0.3.4 It first prints for each package a single character after checking the -available versions on pypi. A dot(`.`) means the package is up-to-date, `u` -means the package can be updated and `e` means the list of releases on +available versions on pypi. A dot(`.`) means the package is up-to-date, ``'u'`` +means the package can be updated and ``'e'`` means the list of releases on pypi is empty. After that it shows a *pip* command line which can be used to update a one package. Either copy and paste that or run ``pypi-server -Ux`` in order to really execute those commands. You need to have *pip* installed for that to work however. -Specifying an additional `-u` option will also allow alpha, beta and +Specifying an additional ``-u`` option will also allow alpha, beta and release candidates to be downloaded. Without this option these releases won't be considered. @@ -407,7 +409,7 @@ pypiserver's directory caching functionality. The only requirement is to install the ``watchdog`` package, or it can be installed by installing ``pypiserver`` using the ``cache`` extras option:: - pip install pypiserver[cache] + pip install pypiserver[cache] If you are using a static webserver such as *Apache* or *nginx* as a reverse-proxy for pypiserver, additional speedup can be gained by @@ -421,36 +423,35 @@ packages-directly directly (take care not to expose "sensitive" files):: } If you have packages that are very large, you may find it helpful to -disable hashing of files (set `--hash-algo=off`, or `hash_algo=None` when +disable hashing of files (set ``--hash-algo=off``, or ``hash_algo=None`` when using wsgi). Managing Automated Startup -------------------------- - There are a variety of options for handling the automated starting of -pypiserver upon system startup. Two of the most common are ``systemd`` and -``supervisor``. +pypiserver upon system startup. Two of the most common are *systemd* and +*supervisor*. -systemd -~~~~~~~ -``systemd`` is installed by default on most modern Linux systems and as such is an -excellent option for managing the pypiserver process. An example config file -for ``systemd`` can be seen below:: +Running as a *systemd* service +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +*systemd* is installed by default on most modern Linux systems and as such, +it is an excellent option for managing the pypiserver process. An example +config file for ``systemd`` can be seen below:: [Unit] - Description=pypi-server + Description=A minimal PyPI server for use with pip/easy_install. After=network.target [Service] Type=simple # systemd requires absolute path here too. - PIDFile=/var/run/pypi-server.pid + PIDFile=/var/run/pypiserver.pid User=www-data Group=www-data - ExecStart=/usr/local/bin/pypi-server -p 8080 -a update,download --log-file /var/log/pypi-server.log --P /etc/nginx/.htpasswd /var/www/pypi + ExecStart=/usr/local/bin/pypi-server -p 8080 -a update,download --log-file /var/log/pypiserver.log --P /etc/nginx/.htpasswd /var/www/pypi ExecStop=/bin/kill -TERM $MAINPID ExecReload=/bin/kill -HUP $MAINPID Restart=always @@ -463,16 +464,20 @@ for ``systemd`` can be seen below:: [Install] WantedBy=multi-user.target -Adjusting the paths and adding this file as ``pypi-server.service`` into your +Adjusting the paths and adding this file as ``pypiserver.service`` into your ``systemd/system`` directory will allow management of the pypiserver process with -``systemctl``, e.g. ``systemctl start pypi-server``. +``systemctl``, e.g. ``systemctl start pypiserver``. -supervisor -~~~~~~~~~~ +More useful information about *systemd* can be found at +https://www.digitalocean.com/community/tutorials/how-to-use-systemctl-to-manage-systemd-services-and-units -``supervisor`` has the benefit of being a pure python package and as such -provides excellent cross-platform support for process management. An example -config file for ``supervisor`` is below:: + +Launching through *supervisor* +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +`supervisor `_ has the benefit of being a pure python +package and as such, it provides excellent cross-platform support for process +management. An example configuration file for ``supervisor`` is given below:: [program:pypi] command=/home/pypi/pypi-venv/bin/pypi-server -p 7001 -P /home/pypi/.htaccess /home/pypi/packages @@ -480,8 +485,8 @@ config file for ``supervisor`` is below:: user=pypi autostart=true autorestart=true - stderr_logfile=/var/log/pypi-server.err.log - stdout_logfile=/var/log/pypi-server.out.log + stderr_logfile=/var/log/pypiserver.err.log + stdout_logfile=/var/log/pypiserver.out.log From there, the process can be managed via ``supervisord`` using ``supervisorctl``. @@ -497,7 +502,7 @@ Using a different WSGI server - If none of the above servers matches your needs, pypiserver also exposes an API to get the internal WSGI app, which you can then run - under any WSGI server you like. `pypiserver.app` has the following + under any WSGI server you like. ``pypiserver.app`` has the following interface:: def app(root=None, @@ -543,58 +548,55 @@ In case you're using *apache2* with *mod_wsgi*, the following config-file paste/pastedeploy ~~~~~~~~~~~~~~~~~ -*paste* allows to run multiple WSGI applications under different URL -paths. Therefore it's possible to serve different set of packages on -different paths. +`paste `_ allows to run multiple WSGI applications +under different URL paths. Therefore it's possible to serve different set +of packages on different paths. -The following example `paste.ini` could be used to serve stable and +The following example ``paste.ini`` could be used to serve stable and unstable packages on different paths:: - [composite:main] - use = egg:Paste#urlmap - /unstable/ = unstable - / = stable + [composite:main] + use = egg:Paste#urlmap + /unstable/ = unstable + / = stable - [app:stable] - use = egg:pypiserver#main - root = ~/stable-packages + [app:stable] + use = egg:pypiserver#main + root = ~/stable-packages - [app:unstable] - use = egg:pypiserver#main - root = ~/stable-packages - ~/unstable-packages + [app:unstable] + use = egg:pypiserver#main + root = ~/stable-packages + ~/unstable-packages - [server:main] - use = egg:gunicorn#main - host = 0.0.0.0 - port = 9000 - workers = 5 - accesslog = - + [server:main] + use = egg:gunicorn#main + host = 0.0.0.0 + port = 9000 + workers = 5 + accesslog = - .. Note:: - You need to install some more dependencies for this to work, - e.g. run:: + You need to install some more dependencies for this to work, like:: - pip install paste pastedeploy gunicorn pypiserver + pip install paste pastedeploy gunicorn pypiserver - The server can then be started with:: + The server can then start with:: - gunicorn_paster paste.ini + gunicorn_paster paste.ini Sources ======= -Use:: +To create a copy of the repository, use:: - git clone https://github.com/pypiserver/pypiserver.git + git clone https://github.com/pypiserver/pypiserver.git + cd pypiserver -to create a copy of the repository, then:: - - git pull - -inside the copy to receive any later changes. +To receive any later changes, in the above folder use:: + git pull Known Limitations @@ -604,20 +606,17 @@ implements just enough to make ``easy_install`` and ``pip install`` to work. The following limitations are known: -- It doesn't implement the XMLRPC json API interface: pip search - will not work (a patch has been suggested: - issue https://github.com/pypiserver/pypiserver/issues/80). - Command ``pypi -U`` that compares uploaded packages with *pypi* to see if - they are outdated does not respect a http-proxy environment variable + they are outdated, does not respect a http-proxy environment variable (see https://github.com/pypiserver/pypiserver/issues/19). - It accepts documentation uploads but does not save them to disk (see https://github.com/pypiserver/pypiserver/issues/47 for a discussion) - It does not handle misspelled packages as *pypi-repo* does, - therefore it is suggested to use it with `--extra-index-url` instead - of `--index-url` (see https://github.com/pypiserver/pypiserver/issues/38) + therefore it is suggested to use it with ``--extra-index-url`` instead + of ``--index-url`` (see https://github.com/pypiserver/pypiserver/issues/38). -Please use github's `bugtracker `_ +Please use Github's `bugtracker `_ for other bugs you find. @@ -641,11 +640,15 @@ among the most popular alternatives: - `flask-pypi-proxy `_ A proxy for PyPI that also enables also uploading custom packages. +- `twine`_: + A command-line utility for interacting with PyPI or *pypiserver*. + - `pypi-uploader`_: - Provides an easy way to upload packages to your pypiserver from pypi without + A command-line utility to upload packages to your *pypiserver* from pypi without having to store them locally first. -- Check this SO question: ` How to roll my own pypi `_ +- Check this SO question: ` How to roll my own pypi + `_ @@ -653,12 +656,13 @@ License ======= *pypiserver* contains a copy of bottle_ which is available under the *MIT* license, and the remaining part is distributed under the *zlib/libpng* license. -See the `LICENSE.txt` file. +See the ``LICENSE.txt`` file. .. _bottle: http://bottlepy.org .. _PyPI: http://pypi.python.org +.. _twine: https://pypi.python.org/pypi/twine .. _pypi-uploader: https://pypi.python.org/pypi/pypi-uploader .. |travis-status| image:: https://travis-ci.org/pypiserver/pypiserver.svg :alt: Travis build status diff --git a/pypiserver/__init__.py b/pypiserver/__init__.py index 0dcc106..885226e 100644 --- a/pypiserver/__init__.py +++ b/pypiserver/__init__.py @@ -1,8 +1,8 @@ import re as _re -version = __version__ = "1.2.0.dev1" +version = __version__ = "1.2.0b0" __version_info__ = tuple(_re.split('[.-]', __version__)) -__updated__ = "2016-06-17" +__updated__ = "2016-06-23" __title__ = "pypiserver" __summary__ = "A minimal PyPI server for use with pip/easy_install."