FIX: Fallback URL, remove calls to pip.main

Resolves #205 - pypi.python.org shutting down

* Updated the default fallback URL to `pypi.org/simple` rather than
`pypi.python.org/simple`
* Scrubbed references and links to `pypi.python.org`
* Fixed tests breaking due to the removal of `pip.main()` in pip 10.0 -
see pypa/pip#5080 for more info

.gitignore

@@ -11,6 +11,7 @@
 .*.swo
 .*.swn
 .~
+.envrc
 .DS_Store
 .ropeproject
 ID
@@ -23,6 +24,7 @@ __pycache__/
 /pypi-server-standalone.py
 /.project
 /.pydevproject
+/.pytest_cache
 /.tox/
 /*.egg-info/
 /.standalone
@@ -38,6 +40,7 @@ __pycache__/
 
 # IDE stuff
 .idea/
+.vscode/
 
 # Venvs
 .venv/

CHANGES.rst

@@ -1,6 +1,15 @@
 Changelog
 =========
 
+1.2.2 (2018-06-xx)
+------------------
+
+- FIX: update fallback URL to https://pypi.org/simple since pypi.python.org
+  has shut down
+- FIX: updated tests to use ``Popen`` rather than ``pip.main()`` given its
+  removal in pip version 10.0
+- DOC: scrubbed docs of links to pypi.python.org
+
 1.2.1 (2017-11-29)
 ------------------
 

README.rst

@@ -13,7 +13,7 @@ pypiserver - minimal PyPI server for use with pip/easy_install
 :Version:     1.2.2dev0
 :Date:        2017-11-29 11:49:30
 :Source:      https://github.com/pypiserver/pypiserver
-:PyPI:        https://pypi.python.org/pypi/pypiserver
+:PyPI:        https://pypi.org/project/pypiserver/
 :Travis:      https://travis-ci.org/pypiserver/pypiserver
 :Maintainers: Kostis Anagnostopoulos <ankostis@gmail.com>,
               Matthew Planchard <mplanchard@gmail.com>
@@ -114,7 +114,7 @@ For legacy python versions, use ``pypiserver-1.1.x`` series.
 
       --fallback-url FALLBACK_URL
         for packages not found in the local index, this URL will be used to
-        redirect to (default: https://pypi.python.org/simple)
+        redirect to (default: https://pypi.org/simple)
 
       --server METHOD
         use METHOD to run the server. Valid values include paste,
@@ -170,7 +170,7 @@ For legacy python versions, use ``pypiserver-1.1.x`` series.
 
     pypi-server -U [OPTIONS] [PACKAGES_DIRECTORY...]
       update packages in PACKAGES_DIRECTORY. This command searches
-      pypi.python.org for updates and shows a pip command line which
+      pypi.org for updates and shows a pip command line which
       updates the package.
 
     The following additional options can be specified with -U:
@@ -194,7 +194,7 @@ Client-side Configurations
 ==========================
 Always specifying the the pypi url on the command line is a bit
 cumbersome. Since *pypiserver* redirects ``pip/easy_install`` to the
-``pypi.python.org`` index if it doesn't have a requested package, it's a
+``pypi.org`` index if it doesn't have a requested package, it's a
 good idea to configure them to always use your local pypi index.
 
 Configuring *pip*
@@ -397,7 +397,7 @@ Managing the package directory
 ------------------------------
 The ``pypi-server`` command has the ``-U`` option that searches for updates of
 available packages. It scans the package directory for available
-packages and searches on pypi.python.org for updates. Without further
+packages and searches on pypi.org for updates. Without further
 options ``pypi-server -U`` will just print a list of commands which must
 be run in order to get the latest version of each package. Output
 looks like::
@@ -412,10 +412,10 @@ looks like::
     no releases found on pypi for PyXML, Pymacs, mercurial, setuptools
 
     # update raven from 1.4.3 to 1.4.4
-    pip -q install --no-deps  --extra-index-url https://pypi.python.org/simple -d /home/ralf/packages/mirror raven==1.4.4
+    pip -q install --no-deps  --extra-index-url https://pypi.org/simple -d /home/ralf/packages/mirror raven==1.4.4
 
     # update greenlet from 0.3.3 to 0.3.4
-    pip -q install --no-deps  --extra-index-url https://pypi.python.org/simple -d /home/ralf/packages/mirror greenlet==0.3.4
+    pip -q install --no-deps  --extra-index-url https://pypi.org/simple -d /home/ralf/packages/mirror greenlet==0.3.4
 
 It first prints for each package a single character after checking the
 available versions on pypi. A dot(`.`) means the package is up-to-date, ``'u'``
@@ -777,10 +777,10 @@ There are lots of other projects, which allow you to run your own
 PyPI server. If *pypiserver* doesn't work for you, the following are
 among the most popular alternatives:
 
-- `devpi-server <https://pypi.python.org/pypi/devpi-server>`_:
+- `devpi-server <https://pypi.org/project/devpi/>`_:
-  a reliable fast pypi.python.org caching server, part of
+  a reliable fast pypi.org caching server, part of
   the comprehensive `github-style pypi index server and packaging meta tool
-  <https://pypi.python.org/pypi/devpi>`_.
+  <https://pypi.org/project/devpi/>`_.
   (version: 2.1.4, access date: 8/3/2015)
 
 - `pip2pi <https://github.com/wolever/pip2pi>`_
@@ -811,25 +811,25 @@ See the ``LICENSE.txt`` file.
 
 
 .. _bottle: http://bottlepy.org
-.. _PyPI: https://pypi.python.org
+.. _PyPI: https://pypi.org
-.. _twine: https://pypi.python.org/pypi/twine
+.. _twine: https://pypi.org/project/twine/
-.. _pypi-uploader: https://pypi.python.org/pypi/pypi-uploader
+.. _pypi-uploader: https://pypi.org/project/pypi-uploader/
-.. _python-pam: https://pypi.python.org/pypi/python-pam/
+.. _python-pam: https://pypi.org/project/python-pam/
 .. |travis-status| image:: https://travis-ci.org/pypiserver/pypiserver.svg
     :alt: Travis build status
     :scale: 100%
     :target: https://travis-ci.org/pypiserver/pypiserver
 
 .. |pypi-ver| image::  https://img.shields.io/pypi/v/pypiserver.svg
-    :target: https://pypi.python.org/pypi/pypiserver/
+    :target: https://pypi.org/project/pypiserver/
     :alt: Latest Version in PyPI
 
 .. |python-ver| image:: https://img.shields.io/pypi/pyversions/pypiserver.svg
-    :target: https://pypi.python.org/pypi/pypiserver/
+    :target: https://pypi.org/project/pypiserver/
     :alt: Supported Python versions
 
 .. |downloads-count| image:: https://img.shields.io/pypi/dm/pypiserver.svg?period=week
-    :target: https://pypi.python.org/pypi/pypiserver/
+    :target: https://pypi.org/project/pypiserver/
     :alt: Downloads
 
 .. |proj-license| image:: https://img.shields.io/badge/license-BSD%2Bzlib%2Flibpng-blue.svg

bin/gen-standalone.sh

@@ -26,7 +26,7 @@ sudo update-ca-certificates || echo "Failed updating certs (run on travis contai
 unzip -jo $wheel pypiserver/__main__.py -d ./dist
 zip -d $wheel pypiserver/__main__.py
 zip -mj $wheel ./dist/__main__.py
-wget https://pypi.python.org/packages/2.7/p/passlib/passlib-1.6.5-py2.py3-none-any.whl#md5=03de8f28697eaa67835758a60386c9fa \
+wget https://files.pythonhosted.org/packages/2d/a7/1a3363e5d531d438267a79d43d4b8d224655adef489e98fc96678fe16ed1/passlib-1.6.5-py2.py3-none-any.whl \
         -O ./dist/passlib-1.6.5-py2.py3-none-any.whl
 zip -mj $wheel ./dist/passlib-*.whl
 gitversion=$(git describe --tags)

pypiserver/__main__.py

@@ -129,7 +129,7 @@ def usage():
 
   pypi-server -U [OPTIONS] [PACKAGES_DIRECTORY...]
     update packages in PACKAGES_DIRECTORY. This command searches
-    pypi.python.org for updates and shows a pip command line which
+    pypi.org for updates and shows a pip command line which
     updates the package.
 
   The following additional options can be specified with -U:
@@ -145,7 +145,7 @@ def usage():
     -u
       allow updating to unstable version (alpha, beta, rc, dev versions)
 
-  Visit https://pypi.python.org/pypi/pypiserver for more information.
+  Visit https://pypi.org/project/pypiserver/ for more information.
   """)
 
 

pypiserver/core.py

@@ -68,7 +68,7 @@ def configure(**kwds):
             "Could not load welcome-file(%s)!", c.welcome_file, exc_info=1)
 
     if c.fallback_url is None:
-        c.fallback_url = "https://pypi.python.org/simple"
+        c.fallback_url = "https://pypi.org/simple"
 
     if c.hash_algo:
         try:

pypiserver/manage.py

@@ -104,7 +104,7 @@ def find_updates(pkgset, stable_only=True):
         "checking %s packages for newer version\n" % len(latest_pkgs),)
     need_update = set()
 
-    pypi = make_pypi_client("https://pypi.python.org/pypi/")
+    pypi = make_pypi_client("https://pypi.org/pypi/")
 
     for count, pkg in enumerate(latest_pkgs):
         if count % 40 == 0:
@@ -140,7 +140,7 @@ def update(pkgset, destdir=None, dry_run=False, stable_only=True):
         sys.stdout.write("# update %s from %s to %s\n" %
                          (pkg.pkgname, pkg.replaces.version, pkg.version))
 
-        cmd = ["pip", "-q", "install", "--no-deps", "-i", "https://pypi.python.org/simple",
+        cmd = ["pip", "-q", "install", "--no-deps", "-i", "https://pypi.org/simple",
                "-d", destdir or os.path.dirname(pkg.replaces.fn),
                "%s==%s" % (pkg.pkgname, pkg.version)]
 

pypiserver/welcome.html

@@ -16,5 +16,5 @@ easy_install -i {{URL}}simple/ PACKAGE
 or via the <a href="{{SIMPLE}}">simple</a> index.</p>
 
 <p>This instance is running version {{VERSION}} of the
-  <a href="https://pypi.python.org/pypi/pypiserver">pypiserver</a> software.</p>
+  <a href="https://pypi.org/project/pypiserver/">pypiserver</a> software.</p>
 </body></html>

tests/test_app.py

@@ -177,8 +177,7 @@ def test_favicon(testapp):
 def test_fallback(root, _app, testapp):
     assert _app.config.redirect_to_fallback
     resp = testapp.get("/simple/pypiserver/", status=302)
-    assert resp.headers[
+    assert resp.headers["Location"] == "https://pypi.org/simple/pypiserver/"
-        "Location"] == "https://pypi.python.org/simple/pypiserver/"
 
 
 def test_no_fallback(root, _app, testapp):
@@ -313,7 +312,7 @@ def test_root_no_relative_paths(testpriv):
     resp = testpriv.get("/priv/")
     hrefs = [x["href"] for x in resp.html("a")]
     assert hrefs == ['/priv/packages/', '/priv/simple/',
-                     'https://pypi.python.org/pypi/pypiserver']
+                     'https://pypi.org/project/pypiserver/']
 
 
 def test_simple_index_list_no_duplicates(root, testapp):
@@ -381,8 +380,8 @@ def test_upload_badAction(root, testapp):
     assert "Unsupported ':action' field: BAD" in hp.unescape(resp.text)
 
 
-@pytest.mark.parametrize(("package"), [f[0] 
+@pytest.mark.parametrize(("package"), [f[0]
-        for f in test_core.files 
+        for f in test_core.files
         if f[1] and '/' not in f[0]])
 def test_upload(package, root, testapp):
     resp = testapp.post("/", params={':action': 'file_upload'},
@@ -393,13 +392,13 @@ def test_upload(package, root, testapp):
     assert uploaded_pkgs[0].lower() == package.lower()
 
 
-@pytest.mark.parametrize(("package"), [f[0] 
+@pytest.mark.parametrize(("package"), [f[0]
-        for f in test_core.files 
+        for f in test_core.files
         if f[1] and '/' not in f[0]])
 def test_upload_with_signature(package, root, testapp):
     resp = testapp.post("/", params={':action': 'file_upload'},
             upload_files=[
-                    ('content', package, b''), 
+                    ('content', package, b''),
                     ('gpg_signature', '%s.asc' % package, b'')])
     assert resp.status_int == 200
     uploaded_pkgs = [f.basename.lower() for f in root.listdir()]
@@ -433,7 +432,7 @@ def test_remove_pkg_missingNaveVersion(name, version, root, testapp):
     params = {':action': 'remove_pkg', 'name': name, 'version': version}
     params = dict((k, v) for k,v in params.items() if v is not None)
     resp = testapp.post("/", expect_errors=1, params=params)
-    
+
     assert resp.status == '400 Bad Request'
     assert msg %(name, version) in hp.unescape(resp.text)
 

tests/test_main.py

@@ -86,8 +86,7 @@ def test_fallback_url(main):
 
 def test_fallback_url_default(main):
     main([])
-    assert main.app.module.config.fallback_url == \
+    assert main.app.module.config.fallback_url == "https://pypi.org/simple"
-        "https://pypi.python.org/simple"
 
 
 def test_hash_algo_default(main):

tests/test_server.py

@@ -6,8 +6,8 @@ The tests below are using 3 ways to startup pypi-servers:
 
 - "open": a per-module server instance without any authed operations,
   serving a single `wheel` package, on a fixed port.
-- "open": a per-module server instance with authed 'download/upload' operations,
+- "open": a per-module server instance with authed 'download/upload'
-  serving a single `wheel` package, on a fixed port.
+  operations, serving a single `wheel` package, on a fixed port.
 - "new_server": starting a new server with any configurations on each test.
 
 """
@@ -21,13 +21,14 @@ import subprocess
 import sys
 import tempfile
 import time
+from shlex import split
+from subprocess import Popen
 from textwrap import dedent
 try:
     from urllib.request import urlopen
 except ImportError:
     from urllib import urlopen
 
-import pip
 from py import path  # @UnresolvedImport
 import pytest
 
@@ -166,16 +167,19 @@ def _build_url(port, user='', pswd=''):
 
 
 def _run_pip(cmd):
-    ncmd = ("--disable-pip-version-check --retries 0 --timeout 5"
+    ncmd = (
-            " --no-input %s"
+        "pip --disable-pip-version-check --retries 0 --timeout 5 --no-input %s"
-            ) % cmd
+    ) % cmd
     print('PIP: %s' % ncmd)
-    return pip.main(ncmd.split())
+    proc = Popen(split(ncmd))
+    proc.communicate()
+    return proc.returncode
 
 
 def _run_pip_install(cmd, port, install_dir, user=None, pswd=None):
     url = _build_url(port, user, pswd)
-    ncmd = '-vv install --download %s -i %s %s' % (install_dir, url, cmd)
+    # ncmd = '-vv install --download %s -i %s %s' % (install_dir, url, cmd)
+    ncmd = '-vv download -d %s -i %s %s' % (install_dir, url, cmd)
     return _run_pip(ncmd)