v2.18.2

redundant comma

.gitignore

@@ -5,6 +5,7 @@ nosetests.xml
 junit-report.xml
 pylint.txt
 toy.py
+.cache/
 cover/
 build/
 docs/_build

3.0-HISTORY.rst

@@ -1,68 +0,0 @@
-3.0.0 (2017-xx-xx)
-++++++++++++++++++
-
-- Simplified logic for determining Content-Length and Transfer-Encoding.
-  Requests will now avoid setting both headers on the same request, and
-  raise an exception if this is done manually by a user.
-
-- Remove the HTTPProxyAuth class in favor of supporting proxy auth via
-  the proxies parameter.
-
-- Relax how Requests strips bodies from redirects. 3.0.0 only supports body
-  removal on 301/302 POST redirects and all 303 redirects.
-
-- Remove support for non-string/bytes parameters in ``_basic_auth_str``.
-
-- Prevent ``Session.merge_environment`` from erroneously setting the
-  ``verify`` parameter to ``None`` instead of ``True``.
-
-- Streaming responses with ``Response.iter_lines`` or ``Response.iter_content``
-  now requires an encoding to be set if one isn't provided by the server.
-
-- Raise exception if multiple locations are returned during a redirect.
-
-- Update ConnectionPool connections when TLS/SSL settings change.
-
-- Remove simplejson import and only use standard json module.
-
-- Strip surrounding whitespace from urls.
-
-- MissingSchema and InvalidSchema renamed to MissingScheme and InvalidScheme
-  respectively.
-
-- Change merge order for environment settings to avoid excluding Session-level
-  settings.
-
-- Encode redirect URIs as latin-1 before performing redirects in Python 3 to
-  avoid mangling during the requoting process.
-
-- Remove the ``__bool__`` and ``__nonzero__`` methods from a ``Response``
-  object.
-
-  This has been a planned feature for over a year. The behaviour is surprising
-  to most people and breaks most of the assumptions that people have about
-  Response objects. This resolves issue `#2002`_
-
-- Skip over empty chunks in iterators. Empty chunks could prematurely signal
-  the end of a request body's transmission, skipping them allows all of the
-  data through. See `#2631`_ for more details.
-
-- Rename the ``req`` argument from ``Session.resolve_redirects`` method
-  to ``request``.
-
-- Rename the ``resp`` argument from ``Session.resolve_redirects`` to
-  ``response``.
-
-- New ``PreparedRequest.send`` method. Now, you can
-  ``Request().prepare().send()``.
-
-- All porcelain API functions (e.g. ``requests.get``, etc) now accept an
-  optional ``session`` parameter. If provided, the session given will be used
-  for the request, in place of one being created for you.
-
-- URLs are now automatically stripped of leading/trailing whitespace.
-
-- ``Response.raise_for_status()`` now returns the response object for good responses
-
-.. _#2002: https://github.com/kennethreitz/requests/issues/2002
-.. _#2631: https://github.com/kennethreitz/requests/issues/2631

AUTHORS.rst

@@ -160,19 +160,16 @@ Patches and Suggestions
 - Muhammad Yasoob Ullah Khalid <yasoob.khld@gmail.com> (`@yasoob <https://github.com/yasoob>`_)
 - Paul van der Linden (`@pvanderlinden <https://github.com/pvanderlinden>`_)
 - Colin Dickson (`@colindickson <https://github.com/colindickson>`_)
-- Sabari Kumar Murugesan (`@neosab <https://github.com/neosab>`_)
 - Smiley Barry (`@smiley <https://github.com/smiley>`_)
 - Shagun Sodhani (`@shagunsodhani <https://github.com/shagunsodhani>`_)
 - Robin Linderborg (`@vienno <https://github.com/vienno>`_)
 - Brian Samek (`@bsamek <https://github.com/bsamek>`_)
 - Dmitry Dygalo (`@Stranger6667 <https://github.com/Stranger6667>`_)
-- Tomáš Heger (`@geckon <https://github.com/geckon>`_)
 - piotrjurkiewicz
 - Jesse Shapiro <jesse@jesseshapiro.net> (`@haikuginger <https://github.com/haikuginger>`_)
 - Nate Prewitt <nate.prewitt@gmail.com> (`@nateprewitt <https://github.com/nateprewitt>`_)
 - Maik Himstedt
 - Michael Hunsinger
-- Jeremy Cline <jcline@redhat.com> (`@jeremycline <https://github.com/jeremycline>`_)
 - Brian Bamsch <bbamsch32@gmail.com> (`@bbamsch <https://github.com/bbamsch>`_)
 - Om Prakash Kumar <omprakash070@gmail.com> (`@iamprakashom <https://github.com/iamprakashom>`_)
 - Philipp Konrad <gardiac2002@gmail.com> (`@gardiac2002 <https://github.com/gardiac2002>`_)
@@ -186,3 +183,5 @@ Patches and Suggestions
 - Shmuel Amar (`@shmuelamar <https://github.com/shmuelamar>`_)
 - Gary Wu (`@garywu <https://github.com/garywu>`_)
 - Ryan Pineo (`@ryanpineo <https://github.com/ryanpineo>`_)
+- Ed Morley (`@edmorley <https://github.com/edmorley>`_)
+- Matt Liu <liumatt@gmail.com> (`@mlcrazy <https://github.com/mlcrazy>`_)

HISTORY.rst

@@ -3,13 +3,39 @@
 Release History
 ---------------
 
-dev
-+++
+2.18.2 (2017-07-25)
++++++++++++++++++++
+
+**Bugfixes**
+
+- ``requests.help`` no longer fails on Python 2.6 due to the absence of
+  ``ssl.OPENSSL_VERSION_NUMBER``.
+
+**Dependencies**
+
+- We now support urllib3 v1.22.
+
+2.18.1 (2017-06-14)
++++++++++++++++++++
+
+**Bugfixes**
+
+- Fix an error in the packaging whereby the *.whl contained incorrect data that
+  regressed the fix in v2.17.3.
+
+2.18.0 (2017-06-14)
++++++++++++++++++++
+
+**Improvements**
+
+- ``Response`` is now a context manager, so can be used directly in a ``with`` statement
+  without first having to be wrapped by ``contextlib.closing()``.
 
 **Bugfixes**
 
 - Resolve installation failure if multiprocessing is not available
 - Resolve tests crash if multiprocessing is not able to determine the number of CPU cores
+- Resolve error swallowing in utils set_environ generator
 
 
 2.17.3 (2017-05-29)
@@ -174,11 +200,6 @@ Or, even better::
 - Updated bundled idna to v2.5.
 - Updated bundled certifi to 2017.4.17.
 
-- Altered how ``SessionRedirectMixin.resolve_redirects`` and ``Session.send``
-  process redirect history.  Developers who subclass ``resolve_redirects`` will
-  find a different ``.history`` attribute - the first element now contains the
-  original response, and the last element now contains the active response.
-
 2.13.0 (2017-01-24)
 +++++++++++++++++++
 
@@ -1481,4 +1502,3 @@ This is not a backwards compatible change.
 
 * Frustration
 * Conception
-

MANIFEST.in

@@ -1,2 +1,2 @@
-include README.rst LICENSE NOTICE HISTORY.rst pytest.ini
+include README.rst LICENSE NOTICE HISTORY.rst pytest.ini requirements.txt
 recursive-include tests *.py

Makefile

@@ -8,7 +8,7 @@ ci:
 	py.test -n 8 --boxed --junitxml=report.xml
 
 test-readme:
-	python setup.py check -r -s
+	@python setup.py check --restructuredtext --strict && ([ $$? -eq 0 ] && echo "README.rst and HISTORY.rst ok") || echo "Invalid markup in README.rst or HISTORY.rst!"
 
 flake8:
 	flake8 --ignore=E501,F401,E128,E402,E731,F821 requests

README.rst

@@ -93,7 +93,7 @@ To install Requests, simply:
     $ pip install requests
     ✨🍰✨
 
-Satisfaction, guaranteed.
+Satisfaction guaranteed.
 
 Documentation
 -------------

docs/_static/custom.css

@@ -0,0 +1,39 @@
+#carbonads {
+  display: block;
+  overflow: hidden;
+  padding: 1em;
+  background-color: #eeeeee;
+  text-align: center;
+  border: solid 1px #cccccc;
+  margin: 1.5em 0 2em;
+  border-radius: 2px;
+  line-height: 1.5;
+}
+
+#carbonads a {
+  border-bottom: 0;
+}
+
+#carbonads span {
+  display: block;
+  overflow: hidden;
+}
+
+.carbon-img {
+  display: block;
+  margin: 0 auto 1em;
+  text-align: center;
+}
+
+.carbon-text {
+  display: block;
+  margin-bottom: 1em;
+}
+
+.carbon-poweredby {
+  display: block;
+  text-transform: uppercase;
+  letter-spacing: 1px;
+  line-height: 1;
+  font-size: 10px;
+}

docs/_templates/sidebarintro.html

@@ -14,16 +14,7 @@
   human beings.
 </p>
 
-
-<h3>The Hitchhiker's Guide to Python</h3>
-
-<p>This guide is now available in tangible book form!</p>
-
-<a href="https://www.amazon.com/Hitchhikers-Guide-Python-Practices-Development/dp/1491933178/ref=as_li_ss_il?ie=UTF8&linkCode=li2&tag=bookforkind-20&linkId=804806ebdacaf3b56567347f3afbdbca" target="_blank"><img border="0" src="https://ws-na.amazon-adsystem.com/widgets/q?_encoding=UTF8&ASIN=1491933178&Format=_SL160_&ID=AsinImage&MarketPlace=US&ServiceVersion=20070822&WS=1&tag=bookforkind-20" ></a><img src="//ir-na.amazon-adsystem.com/e/ir?t=bookforkind-20&l=li2&o=1&a=1491933178" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" />
-
-<p>All proceeds are being directly donated to the <a href="https://djangogirls.org">DjangoGirls</a> organization.</p>
-
-
+<script async type="text/javascript" src="//cdn.carbonads.com/carbon.js?zoneid=1673&serve=C6AILKT&placement=pythonrequestsorg" id="_carbonads_js"></script>
 
 <h3>Stay Informed</h3>
 <p>Receive updates on new releases and upcoming projects.</p>

docs/_templates/sidebarlogo.html

@@ -21,14 +21,7 @@
 
 <hr/>
 
-<h3>The Hitchhiker's Guide to Python</h3>
-
-<p>This guide is now available in tangible book form!</p>
-
-<a href="https://www.amazon.com/Hitchhikers-Guide-Python-Practices-Development/dp/1491933178/ref=as_li_ss_il?ie=UTF8&linkCode=li2&tag=bookforkind-20&linkId=804806ebdacaf3b56567347f3afbdbca" target="_blank"><img border="0" src="https://ws-na.amazon-adsystem.com/widgets/q?_encoding=UTF8&ASIN=1491933178&Format=_SL160_&ID=AsinImage&MarketPlace=US&ServiceVersion=20070822&WS=1&tag=bookforkind-20" ></a><img src="//ir-na.amazon-adsystem.com/e/ir?t=bookforkind-20&l=li2&o=1&a=1491933178" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" />
-
-<p>All proceeds are being directly donated to the <a href="https://djangogirls.org">DjangoGirls</a> organization.</p>
-
+<script async type="text/javascript" src="//cdn.carbonads.com/carbon.js?zoneid=1673&serve=C6AILKT&placement=pythonrequestsorg" id="_carbonads_js"></script>
 
 <p>If you enjoy using this project, <a href="https://saythanks.io/to/kennethreitz">Say Thanks!</a></p>
 

docs/api.rst

@@ -74,6 +74,7 @@ Authentication
 
 .. autoclass:: requests.auth.AuthBase
 .. autoclass:: requests.auth.HTTPBasicAuth
+.. autoclass:: requests.auth.HTTPProxyAuth
 .. autoclass:: requests.auth.HTTPDigestAuth
 
 
@@ -251,7 +252,7 @@ API Changes
       }
 
       # In requests 1.x, this was legal, in requests 2.x,
-      #  this raises requests.exceptions.MissingScheme
+      #  this raises requests.exceptions.MissingSchema
       requests.get("http://example.org", proxies=proxies)
 
 

docs/user/advanced.rst

@@ -301,15 +301,11 @@ release the connection back to the pool unless you consume all the data or call
 :meth:`Response.close <requests.Response.close>`. This can lead to
 inefficiency with connections. If you find yourself partially reading request
 bodies (or not reading them at all) while using ``stream=True``, you should
-consider using ``contextlib.closing`` (`documented here`_), like this::
+make the request within a ``with`` statement to ensure it's always closed::
 
-    from contextlib import closing
-
-    with closing(requests.get('http://httpbin.org/get', stream=True)) as r:
+    with requests.get('http://httpbin.org/get', stream=True) as r:
         # Do things with the response here.
 
-.. _`documented here`: http://docs.python.org/2/library/contextlib.html#contextlib.closing
-
 .. _keep-alive:
 
 Keep-Alive

docs/user/quickstart.rst

@@ -371,14 +371,10 @@ But, since our ``status_code`` for ``r`` was ``200``, when we call
 ``raise_for_status()`` we get::
 
     >>> r.raise_for_status()
-    <Response [200]>
+    None
 
 All is well.
 
-.. note:: ``raise_for_status`` returns the response object for a successful response. This eases chaining in trivial cases, where we want bad codes to raise an exception, but use the response otherwise:
-
-    >>> value = requests.get('http://httpbin.org/ip').raise_for_status().json()['origin']
-
 
 Response Headers
 ----------------

requests/__init__.py

@@ -40,34 +40,44 @@ is at <http://python-requests.org>.
 :license: Apache 2.0, see LICENSE for more details.
 """
 
-# Check urllib3 for compatibility.
 import urllib3
-urllib3_version = urllib3.__version__.split('.')
-# Sometimes, urllib3 only reports its version as 16.1.
-if len(urllib3_version) == 2:
-    urllib3_version.append('0')
-major, minor, patch = urllib3_version
-major, minor, patch = int(major), int(minor), int(patch)
-# urllib3 >= 1.21.1, < 1.22
-try:
+import chardet
+import warnings
+from .exceptions import RequestsDependencyWarning
+
+
+def check_compatibility(urllib3_version, chardet_version):
+    urllib3_version = urllib3_version.split('.')
+    assert urllib3_version != ['dev']  # Verify urllib3 isn't installed from git.
+
+    # Sometimes, urllib3 only reports its version as 16.1.
+    if len(urllib3_version) == 2:
+        urllib3_version.append('0')
+
+    # Check urllib3 for compatibility.
+    major, minor, patch = urllib3_version  # noqa: F811
+    major, minor, patch = int(major), int(minor), int(patch)
+    # urllib3 >= 1.21.1, < 1.22
     assert major == 1
     assert minor >= 21
     assert minor <= 22
-except AssertionError:
-    raise RuntimeError('Requests dependency \'urllib3\' must be version >= 1.21.1, < 1.22!')
 
-
-# Check chardet for compatibility.
-import chardet
-major, minor, patch = chardet.__version__.split('.')[:3]
-major, minor, patch = int(major), int(minor), int(patch)
-# chardet >= 3.0.2, < 3.1.0
-try:
+    # Check chardet for compatibility.
+    major, minor, patch = chardet_version.split('.')[:3]
+    major, minor, patch = int(major), int(minor), int(patch)
+    # chardet >= 3.0.2, < 3.1.0
     assert major == 3
     assert minor < 1
     assert patch >= 2
-except AssertionError:
-    raise RuntimeError('Requests dependency \'chardet\' must be version >= 3.0.2, < 3.1.0!')
+
+
+# Check imported dependencies for compatibility.
+try:
+    check_compatibility(urllib3.__version__, chardet.__version__)
+except (AssertionError, ValueError):
+    warnings.warn("urllib3 ({0}) or chardet ({1}) doesn't match a supported "
+                  "version!".format(urllib3.__version__, chardet.__version__),
+                  RequestsDependencyWarning)
 
 # Attempt to enable urllib3's SNI support, if possible
 try:
@@ -76,8 +86,6 @@ try:
 except ImportError:
     pass
 
-import warnings
-
 # urllib3's DependencyWarnings should be silenced.
 from urllib3.exceptions import DependencyWarning
 warnings.simplefilter('ignore', DependencyWarning)

requests/__version__.py

@@ -5,8 +5,8 @@
 __title__ = 'requests'
 __description__ = 'Python HTTP for Humans.'
 __url__ = 'http://python-requests.org'
-__version__ = '3.0.0'
-__build__ = 0x030000
+__version__ = '2.18.2'
+__build__ = 0x021802
 __author__ = 'Kenneth Reitz'
 __author_email__ = 'me@kennethreitz.org'
 __license__ = 'Apache 2.0'

requests/adapters.py

@@ -10,10 +10,6 @@ and maintain connections.
 
 import os.path
 import socket
-try:
-    from threading import RLock
-except ImportError:  # threading is an optional module and may not be present.
-    from dummy_threading import RLock
 
 from urllib3.poolmanager import PoolManager, proxy_from_url
 from urllib3.response import HTTPResponse
@@ -38,14 +34,14 @@ from .utils import (DEFAULT_CA_BUNDLE_PATH, get_encoding_from_headers,
 from .structures import CaseInsensitiveDict
 from .cookies import extract_cookies_to_jar
 from .exceptions import (ConnectionError, ConnectTimeout, ReadTimeout, SSLError,
-                         ProxyError, RetryError, InvalidScheme)
+                         ProxyError, RetryError, InvalidSchema)
 from .auth import _basic_auth_str
 
 try:
     from urllib3.contrib.socks import SOCKSProxyManager
 except ImportError:
     def SOCKSProxyManager(*args, **kwargs):
-        raise InvalidScheme("Missing dependencies for SOCKS support.")
+        raise InvalidSchema("Missing dependencies for SOCKS support.")
 
 DEFAULT_POOLBLOCK = False
 DEFAULT_POOLSIZE = 10
@@ -126,7 +122,6 @@ class HTTPAdapter(BaseAdapter):
         self._pool_connections = pool_connections
         self._pool_maxsize = pool_maxsize
         self._pool_block = pool_block
-        self._pool_kw_lock = RLock()
 
         self.init_poolmanager(pool_connections, pool_maxsize, block=pool_block)
 
@@ -139,7 +134,6 @@ class HTTPAdapter(BaseAdapter):
         # self.poolmanager uses a lambda function, which isn't pickleable.
         self.proxy_manager = {}
         self.config = {}
-        self._pool_kw_lock = RLock()
 
         for attr, value in state.items():
             setattr(self, attr, value)
@@ -204,21 +198,19 @@ class HTTPAdapter(BaseAdapter):
 
         return manager
 
-    def _update_poolmanager_ssl_kw(self, verify, cert):
-        """Update the :class:`PoolManager <urllib3.poolmanager.PoolManager>`
-        connection_pool_kw with the necessary SSL configuration. This method
-        should not be called from user code, and is only exposed for use when
-        subclassing the :class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
+    def cert_verify(self, conn, url, verify, cert):
+        """Verify a SSL certificate. This method should not be called from user
+        code, and is only exposed for use when subclassing the
+        :class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
 
-        :param verify: Whether we should actually verify the certificate;
-                       optionally a path to a CA certificate bundle or
-                       directory of CA certificates.
-        :param cert: The path to the client certificate and key, if any.
-                     This can either be the path to the certificate and
-                     key concatenated in a single file, or as a tuple of
-                     (cert_file, key_file).
+        :param conn: The urllib3 connection object associated with the cert.
+        :param url: The requested URL.
+        :param verify: Either a boolean, in which case it controls whether we verify
+            the server's TLS certificate, or a string, in which case it must be a path
+            to a CA bundle to use
+        :param cert: The SSL certificate to verify.
         """
-        if verify:
+        if url.lower().startswith('https') and verify:
 
             cert_loc = None
 
@@ -233,35 +225,30 @@ class HTTPAdapter(BaseAdapter):
                 raise IOError("Could not find a suitable TLS CA certificate bundle, "
                               "invalid path: {0}".format(cert_loc))
 
-            self.poolmanager.connection_pool_kw['cert_reqs'] = 'CERT_REQUIRED'
+            conn.cert_reqs = 'CERT_REQUIRED'
 
             if not os.path.isdir(cert_loc):
-                self.poolmanager.connection_pool_kw['ca_certs'] = cert_loc
-                self.poolmanager.connection_pool_kw['ca_cert_dir'] = None
+                conn.ca_certs = cert_loc
             else:
-                self.poolmanager.connection_pool_kw['ca_cert_dir'] = cert_loc
-                self.poolmanager.connection_pool_kw['ca_certs'] = None
+                conn.ca_cert_dir = cert_loc
         else:
-            self.poolmanager.connection_pool_kw['cert_reqs'] = 'CERT_NONE'
-            self.poolmanager.connection_pool_kw['ca_certs'] = None
-            self.poolmanager.connection_pool_kw['ca_cert_dir'] = None
+            conn.cert_reqs = 'CERT_NONE'
+            conn.ca_certs = None
+            conn.ca_cert_dir = None
 
         if cert:
             if not isinstance(cert, basestring):
-                self.poolmanager.connection_pool_kw['cert_file'] = cert[0]
-                self.poolmanager.connection_pool_kw['key_file'] = cert[1]
+                conn.cert_file = cert[0]
+                conn.key_file = cert[1]
             else:
-                self.poolmanager.connection_pool_kw['cert_file'] = cert
-                self.poolmanager.connection_pool_kw['key_file'] = None
-
-            cert_file = self.poolmanager.connection_pool_kw['cert_file']
-            key_file = self.poolmanager.connection_pool_kw['key_file']
-            if cert_file and not os.path.exists(cert_file):
+                conn.cert_file = cert
+                conn.key_file = None
+            if conn.cert_file and not os.path.exists(conn.cert_file):
                 raise IOError("Could not find the TLS certificate file, "
-                              "invalid path: {0}".format(cert_file))
-            if key_file and not os.path.exists(key_file):
+                              "invalid path: {0}".format(conn.cert_file))
+            if conn.key_file and not os.path.exists(conn.key_file):
                 raise IOError("Could not find the TLS key file, "
-                              "invalid path: {0}".format(key_file))
+                              "invalid path: {0}".format(conn.key_file))
 
     def build_response(self, req, resp):
         """Builds a :class:`Response <requests.Response>` object from a urllib3
@@ -300,7 +287,7 @@ class HTTPAdapter(BaseAdapter):
 
         return response
 
-    def get_connection(self, url, proxies=None, verify=None, cert=None):
+    def get_connection(self, url, proxies=None):
         """Returns a urllib3 connection for the given URL. This should not be
         called from user code, and is only exposed for use when subclassing the
         :class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
@@ -309,21 +296,17 @@ class HTTPAdapter(BaseAdapter):
         :param proxies: (optional) A Requests-style dictionary of proxies used on this request.
         :rtype: urllib3.ConnectionPool
         """
-        with self._pool_kw_lock:
-            if url.lower().startswith('https'):
-                self._update_poolmanager_ssl_kw(verify, cert)
+        proxy = select_proxy(url, proxies)
 
-            proxy = select_proxy(url, proxies)
-
-            if proxy:
-                proxy = prepend_scheme_if_needed(proxy, 'http')
-                proxy_manager = self.proxy_manager_for(proxy)
-                conn = proxy_manager.connection_from_url(url)
-            else:
-                # Only scheme should be lower case
-                parsed = urlparse(url)
-                url = parsed.geturl()
-                conn = self.poolmanager.connection_from_url(url)
+        if proxy:
+            proxy = prepend_scheme_if_needed(proxy, 'http')
+            proxy_manager = self.proxy_manager_for(proxy)
+            conn = proxy_manager.connection_from_url(url)
+        else:
+            # Only scheme should be lower case
+            parsed = urlparse(url)
+            url = parsed.geturl()
+            conn = self.poolmanager.connection_from_url(url)
 
         return conn
 
@@ -418,8 +401,10 @@ class HTTPAdapter(BaseAdapter):
         :param proxies: (optional) The proxies dictionary to apply to the request.
         :rtype: requests.Response
         """
-        conn = self.get_connection(request.url, proxies, verify, cert)
 
+        conn = self.get_connection(request.url, proxies)
+
+        self.cert_verify(conn, request.url, verify, cert)
         url = self.request_url(request, proxies)
         self.add_headers(request)
 
@@ -452,8 +437,7 @@ class HTTPAdapter(BaseAdapter):
                     preload_content=False,
                     decode_content=False,
                     retries=self.max_retries,
-                    timeout=timeout,
-                    enforce_content_length=True
+                    timeout=timeout
                 )
 
             # Send the request.
@@ -474,10 +458,7 @@ class HTTPAdapter(BaseAdapter):
                     low_conn.endheaders()
 
                     for i in request.body:
-                        chunk_size = len(i)
-                        if chunk_size == 0:
-                            continue
-                        low_conn.send(hex(chunk_size)[2:].encode('utf-8'))
+                        low_conn.send(hex(len(i))[2:].encode('utf-8'))
                         low_conn.send(b'\r\n')
                         low_conn.send(i)
                         low_conn.send(b'\r\n')
@@ -497,9 +478,7 @@ class HTTPAdapter(BaseAdapter):
                         pool=conn,
                         connection=low_conn,
                         preload_content=False,
-                        decode_content=False,
-                        enforce_content_length=True,
-                        request_method=request.method
+                        decode_content=False
                     )
                 except:
                     # If we hit any problems here, clean up the connection.

requests/api.py

@@ -13,12 +13,11 @@ This module implements the Requests API.
 from . import sessions
 
 
-def request(method, url, session=None, **kwargs):
+def request(method, url, **kwargs):
     """Constructs and sends a :class:`Request <Request>`.
 
     :param method: method for the new :class:`Request` object.
     :param url: URL for the new :class:`Request` object.
-    :param session: :class:`Session` object to use for this request. If none is given, one will be provided.
     :param params: (optional) Dictionary or bytes to be sent in the query string for the :class:`Request`.
     :param data: (optional) Dictionary or list of tuples ``[(key, value)]`` (will be form-encoded), bytes, or file-like object to send in the body of the :class:`Request`.
     :param json: (optional) json data to send in the body of the :class:`Request`.
@@ -55,10 +54,7 @@ def request(method, url, session=None, **kwargs):
     # By using the 'with' statement we are sure the session is closed, thus we
     # avoid leaving sockets open which can trigger a ResourceWarning in some
     # cases, and look like a memory leak in others.
-
-    session = sessions.Session() if session is None else session
-
-    with session:
+    with sessions.Session() as session:
         return session.request(method=method, url=url, **kwargs)
 
 
@@ -77,7 +73,7 @@ def get(url, params=None, **kwargs):
 
 
 def options(url, **kwargs):
-    r"""Sends a OPTIONS request.
+    r"""Sends an OPTIONS request.
 
     :param url: URL for the new :class:`Request` object.
     :param \*\*kwargs: Optional arguments that ``request`` takes.

requests/auth.py

@@ -12,6 +12,7 @@ import re
 import time
 import hashlib
 import threading
+import warnings
 
 from base64 import b64encode
 
@@ -27,13 +28,33 @@ CONTENT_TYPE_MULTI_PART = 'multipart/form-data'
 def _basic_auth_str(username, password):
     """Returns a Basic Auth string."""
 
+    # "I want us to put a big-ol' comment on top of it that
+    # says that this behaviour is dumb but we need to preserve
+    # it because people are relying on it."
+    #    - Lukasa
+    #
+    # These are here solely to maintain backwards compatibility
+    # for things like ints. This will be removed in 3.0.0.
     if not isinstance(username, basestring):
-        raise TypeError('username must be of type str or bytes, '
-                        'instead it was %s' % type(username))
+        warnings.warn(
+            "Non-string usernames will no longer be supported in Requests "
+            "3.0.0. Please convert the object you've passed in ({0!r}) to "
+            "a string or bytes object in the near future to avoid "
+            "problems.".format(username),
+            category=DeprecationWarning,
+        )
+        username = str(username)
 
     if not isinstance(password, basestring):
-        raise TypeError('password must be of type str or bytes, '
-                        'instead it was %s' % type(password))
+        warnings.warn(
+            "Non-string passwords will no longer be supported in Requests "
+            "3.0.0. Please convert the object you've passed in ({0!r}) to "
+            "a string or bytes object in the near future to avoid "
+            "problems.".format(password),
+            category=DeprecationWarning,
+        )
+        password = str(password)
+    # -- End Removal --
 
     if isinstance(username, str):
         username = username.encode('latin1')
@@ -76,6 +97,14 @@ class HTTPBasicAuth(AuthBase):
         return r
 
 
+class HTTPProxyAuth(HTTPBasicAuth):
+    """Attaches HTTP Proxy Authentication to a given Request object."""
+
+    def __call__(self, r):
+        r.headers['Proxy-Authorization'] = _basic_auth_str(self.username, self.password)
+        return r
+
+
 class HTTPDigestAuth(AuthBase):
     """Attaches HTTP Digest Authentication to the given Request object."""
 

requests/compat.py

@@ -25,6 +25,12 @@ is_py2 = (_ver[0] == 2)
 #: Python 3.x?
 is_py3 = (_ver[0] == 3)
 
+try:
+    import simplejson as json
+except (ImportError, SyntaxError):
+    # simplejson does not support Python 3.2, it throws a SyntaxError
+    # because of u'...' Unicode literals.
+    import json
 
 # ---------
 # Specifics

requests/cookies.py

@@ -414,7 +414,7 @@ class RequestsCookieJar(cookielib.CookieJar, collections.MutableMapping):
 
     def copy(self):
         """Return a copy of this RequestsCookieJar."""
-        new_cj = RequestsCookieJar(self._policy)
+        new_cj = RequestsCookieJar()
         new_cj.update(self)
         return new_cj
 

requests/exceptions.py

@@ -69,12 +69,12 @@ class TooManyRedirects(RequestException):
     """Too many redirects."""
 
 
-class MissingScheme(RequestException, ValueError):
-    """The URL scheme (e.g. http or https) is missing."""
+class MissingSchema(RequestException, ValueError):
+    """The URL schema (e.g. http or https) is missing."""
 
 
-class InvalidScheme(RequestException, ValueError):
-    """See defaults.py for valid schemes."""
+class InvalidSchema(RequestException, ValueError):
+    """See defaults.py for valid schemas."""
 
 
 class InvalidURL(RequestException, ValueError):
@@ -104,10 +104,6 @@ class RetryError(RequestException):
 class UnrewindableBodyError(RequestException):
     """Requests encountered an error when trying to rewind a body"""
 
-
-class InvalidBodyError(RequestException, ValueError):
-    """An invalid request body was specified"""
-
 # Warnings
 
 
@@ -119,3 +115,8 @@ class RequestsWarning(Warning):
 class FileModeWarning(RequestsWarning, DeprecationWarning):
     """A file was opened in text mode, but Requests determined its binary length."""
     pass
+
+
+class RequestsDependencyWarning(RequestsWarning):
+    """An imported dependency doesn't match the expected version range."""
+    pass

requests/help.py

@@ -23,7 +23,7 @@ else:
 
 
 def _implementation():
-    """Return a dict with the Python implementation and verison.
+    """Return a dict with the Python implementation and version.
 
     Provide both the name and the version of the Python implementation
     currently running. For example, on CPython 2.7.5 it will return
@@ -85,12 +85,16 @@ def info():
         'version': getattr(cryptography, '__version__', ''),
     }
 
+    # OPENSSL_VERSION_NUMBER doesn't exist in the Python 2.6 ssl module.
+    system_ssl = getattr(ssl, 'OPENSSL_VERSION_NUMBER', None)
+    system_ssl_info = {
+        'version': '%x' % system_ssl if system_ssl is not None else ''
+    }
+
     return {
         'platform': platform_info,
         'implementation': implementation_info,
-        'system_ssl': {
-            'version': '%x' % ssl.OPENSSL_VERSION_NUMBER,
-        },
+        'system_ssl': system_ssl_info,
         'using_pyopenssl': pyopenssl is not None,
         'pyOpenSSL': pyopenssl_info,
         'urllib3': urllib3_info,

requests/models.py

@@ -9,7 +9,6 @@ This module contains the primary objects that power Requests.
 
 import collections
 import datetime
-import codecs
 import sys
 
 # Import encoding now, to avoid implicit import later.
@@ -27,23 +26,20 @@ from io import UnsupportedOperation
 from .hooks import default_hooks
 from .structures import CaseInsensitiveDict
 
-import requests
 from .auth import HTTPBasicAuth
 from .cookies import cookiejar_from_dict, get_cookie_header, _copy_cookie_jar
 from .exceptions import (
-    HTTPError, MissingScheme, InvalidURL, ChunkedEncodingError,
-    ContentDecodingError, ConnectionError, StreamConsumedError,
-    InvalidHeader, InvalidBodyError)
+    HTTPError, MissingSchema, InvalidURL, ChunkedEncodingError,
+    ContentDecodingError, ConnectionError, StreamConsumedError)
 from ._internal_utils import to_native_string, unicode_is_ascii
 from .utils import (
     guess_filename, get_auth_from_url, requote_uri,
     stream_decode_response_unicode, to_key_val_list, parse_header_links,
-    iter_slices, guess_json_utf, super_len, check_header_validity,
-    is_stream)
+    iter_slices, guess_json_utf, super_len, check_header_validity)
 from .compat import (
     cookielib, urlunparse, urlsplit, urlencode, str, bytes,
     is_py2, chardet, builtin_str, basestring)
-import json as complexjson
+from .compat import json as complexjson
 from .status_codes import codes
 
 #: The set of HTTP status codes that indicate an automatically
@@ -335,9 +331,8 @@ class PreparedRequest(RequestEncodingMixin, RequestHooksMixin):
     def prepare_method(self, method):
         """Prepares the given HTTP method."""
         self.method = method
-        if self.method is None:
-            raise ValueError('Request method cannot be "None"')
-        self.method = to_native_string(self.method.upper())
+        if self.method is not None:
+            self.method = to_native_string(self.method.upper())
 
     @staticmethod
     def _get_idna_encoded_host(host):
@@ -361,8 +356,8 @@ class PreparedRequest(RequestEncodingMixin, RequestHooksMixin):
         else:
             url = unicode(url) if is_py2 else str(url)
 
-        # Ignore any leading and trailing whitespace characters.
-        url = url.strip()
+        # Remove leading whitespaces from url
+        url = url.lstrip()
 
         # Don't do any URL preparation for non-HTTP schemes like `mailto`,
         # `data` etc to work around exceptions from `url_parse`, which
@@ -378,10 +373,10 @@ class PreparedRequest(RequestEncodingMixin, RequestHooksMixin):
             raise InvalidURL(*e.args)
 
         if not scheme:
-            error = ("Invalid URL {0!r}: No scheme supplied. Perhaps you meant http://{0}?")
+            error = ("Invalid URL {0!r}: No schema supplied. Perhaps you meant http://{0}?")
             error = error.format(to_native_string(url, 'utf8'))
 
-            raise MissingScheme(error)
+            raise MissingSchema(error)
 
         if not host:
             raise InvalidURL("Invalid URL %r: No host supplied" % url)
@@ -464,7 +459,17 @@ class PreparedRequest(RequestEncodingMixin, RequestHooksMixin):
             if not isinstance(body, bytes):
                 body = body.encode('utf-8')
 
-        if is_stream(data):
+        is_stream = all([
+            hasattr(data, '__iter__'),
+            not isinstance(data, (basestring, list, tuple, collections.Mapping))
+        ])
+
+        try:
+            length = super_len(data)
+        except (TypeError, AttributeError, UnsupportedOperation):
+            length = None
+
+        if is_stream:
             body = data
 
             if getattr(body, 'tell', None) is not None:
@@ -481,6 +486,10 @@ class PreparedRequest(RequestEncodingMixin, RequestHooksMixin):
             if files:
                 raise NotImplementedError('Streamed bodies and files are mutually exclusive.')
 
+            if length:
+                self.headers['Content-Length'] = builtin_str(length)
+            else:
+                self.headers['Transfer-Encoding'] = 'chunked'
         else:
             # Multi-part file uploads.
             if files:
@@ -493,40 +502,27 @@ class PreparedRequest(RequestEncodingMixin, RequestHooksMixin):
                     else:
                         content_type = 'application/x-www-form-urlencoded'
 
+            self.prepare_content_length(body)
+
             # Add content-type if it wasn't explicitly provided.
             if content_type and ('content-type' not in self.headers):
                 self.headers['Content-Type'] = content_type
 
-        self.prepare_content_length(body)
         self.body = body
 
     def prepare_content_length(self, body):
-        """Prepares Content-Length header.
-
-        If the length of the body of the request can be computed, Content-Length
-        is set using ``super_len``. If user has manually set either a
-        Transfer-Encoding or Content-Length header when it should not be set
-        (they should be mutually exclusive) an InvalidHeader
-        error will be raised.
-        """
+        """Prepare Content-Length header based on request method and body"""
         if body is not None:
             length = super_len(body)
-
             if length:
+                # If length exists, set it. Otherwise, we fallback
+                # to Transfer-Encoding: chunked.
                 self.headers['Content-Length'] = builtin_str(length)
-            elif is_stream(body):
-                self.headers['Transfer-Encoding'] = 'chunked'
-            else:
-                raise InvalidBodyError('Non-null body must have length or be streamable.')
         elif self.method not in ('GET', 'HEAD') and self.headers.get('Content-Length') is None:
             # Set Content-Length to 0 for methods that can have a body
             # but don't provide one. (i.e. not GET or HEAD)
             self.headers['Content-Length'] = '0'
 
-        if 'Transfer-Encoding' in self.headers and 'Content-Length' in self.headers:
-            raise InvalidHeader('Conflicting Headers: Both Transfer-Encoding and '
-                                'Content-Length are set.')
-
     def prepare_auth(self, auth, url=''):
         """Prepares the given HTTP auth data."""
 
@@ -578,14 +574,6 @@ class PreparedRequest(RequestEncodingMixin, RequestHooksMixin):
         for event in hooks:
             self.register_hook(event, hooks[event])
 
-    def send(self, session=None, **send_kwargs):
-        """Sends the PreparedRequest to the given Session.
-        If none is provided, one is created for you."""
-        session = requests.Session() if session is None else session
-
-        with session:
-            return session.send(self, **send_kwargs)
-
 
 class Response(object):
     """The :class:`Response <Response>` object, which contains a
@@ -620,8 +608,7 @@ class Response(object):
         #: Final URL location of Response.
         self.url = None
 
-        #: Encoding to decode with when accessing r.text or
-        #: r.iter_content(decode_unicode=True)
+        #: Encoding to decode with when accessing r.text.
         self.encoding = None
 
         #: A list of :class:`Response <Response>` objects from
@@ -647,6 +634,12 @@ class Response(object):
         #: is a response.
         self.request = None
 
+    def __enter__(self):
+        return self
+
+    def __exit__(self, *args):
+        self.close()
+
     def __getstate__(self):
         # Consume everything; accessing the content attribute makes
         # sure the content has been fully read.
@@ -669,6 +662,26 @@ class Response(object):
     def __repr__(self):
         return '<Response [%s]>' % (self.status_code)
 
+    def __bool__(self):
+        """Returns True if :attr:`status_code` is less than 400.
+
+        This attribute checks if the status code of the response is between
+        400 and 600 to see if there was a client error or a server error. If
+        the status code, is between 200 and 400, this will return True. This
+        is **not** a check to see if the response code is ``200 OK``.
+        """
+        return self.ok
+
+    def __nonzero__(self):
+        """Returns True if :attr:`status_code` is less than 400.
+
+        This attribute checks if the status code of the response is between
+        400 and 600 to see if there was a client error or a server error. If
+        the status code, is between 200 and 400, this will return True. This
+        is **not** a check to see if the response code is ``200 OK``.
+        """
+        return self.ok
+
     def __iter__(self):
         """Allows you to use a response as an iterator."""
         return self.iter_content(128)
@@ -723,8 +736,8 @@ class Response(object):
         chunks are received. If stream=False, data is returned as
         a single chunk.
 
-        If using decode_unicode, the encoding must be set to a valid encoding
-        enumeration before invoking iter_content.
+        If decode_unicode is True, content will be decoded using the best
+        available encoding based on the response.
         """
 
         def generate():
@@ -734,10 +747,7 @@ class Response(object):
                     for chunk in self.raw.stream(chunk_size, decode_content=True):
                         yield chunk
                 except ProtocolError as e:
-                    if self.headers.get('Transfer-Encoding') == 'chunked':
-                        raise ChunkedEncodingError(e)
-                    else:
-                        raise ConnectionError(e)
+                    raise ChunkedEncodingError(e)
                 except DecodeError as e:
                     raise ContentDecodingError(e)
                 except ReadTimeoutError as e:
@@ -764,16 +774,6 @@ class Response(object):
         chunks = reused_chunks if self._content_consumed else stream_chunks
 
         if decode_unicode:
-            if self.encoding is None:
-                raise TypeError(
-                    'encoding must be set before consuming streaming '
-                    'responses'
-                )
-
-            # check encoding value here, don't wait for the generator to be
-            # consumed before raising an exception
-            codecs.lookup(self.encoding)
-
             chunks = stream_decode_response_unicode(chunks, self)
 
         return chunks
@@ -785,67 +785,23 @@ class Response(object):
 
         .. note:: This method is not reentrant safe.
         """
-        carriage_return = u'\r' if decode_unicode else b'\r'
-        line_feed = u'\n' if decode_unicode else b'\n'
 
         pending = None
-        last_chunk_ends_with_cr = False
 
-        for chunk in self.iter_content(chunk_size=chunk_size,
-                                       decode_unicode=decode_unicode):
-            # Skip any null responses: if there is pending data it is necessarily an
-            # incomplete chunk, so if we don't have more data we don't want to bother
-            # trying to get it. Unconsumed pending data will be yielded anyway in the
-            # end of the loop if the stream ends.
-            if not chunk:
-                continue
+        for chunk in self.iter_content(chunk_size=chunk_size, decode_unicode=decode_unicode):
 
-            # Consume any pending data
             if pending is not None:
                 chunk = pending + chunk
-                pending = None
 
-            # Either split on a line, or split on a specified delimiter
             if delimiter:
                 lines = chunk.split(delimiter)
             else:
-                # Python splitlines() supports the universal newline (PEP 278).
-                # That means, '\r', '\n', and '\r\n' are all treated as end of
-                # line. If the last chunk ends with '\r', and the current chunk
-                # starts with '\n', they should be merged and treated as only
-                # *one* new line separator '\r\n' by splitlines().
-                # This rule only applies when splitlines() is used.
-
-                # The last chunk ends with '\r', so the '\n' at chunk[0]
-                # is just the second half of a '\r\n' pair rather than a
-                # new line break. Just skip it.
-                skip_first_char = last_chunk_ends_with_cr and chunk.startswith(line_feed)
-                last_chunk_ends_with_cr = chunk.endswith(carriage_return)
-                if skip_first_char:
-                    chunk = chunk[1:]
-                    # it's possible that after stripping the '\n' then chunk becomes empty
-                    if not chunk:
-                        continue
                 lines = chunk.splitlines()
 
-            # Calling `.split(delimiter)` will always end with whatever text
-            # remains beyond the delimiter, or '' if the delimiter is the end
-            # of the text.  On the other hand, `.splitlines()` doesn't include
-            # a '' if the text ends in a line delimiter.
-            #
-            # For example:
-            #
-            #     'abc\ndef\n'.split('\n')  ~> ['abc', 'def', '']
-            #     'abc\ndef\n'.splitlines() ~> ['abc', 'def']
-            #
-            # So if we have a specified delimiter, we always pop the final
-            # item and prepend it to the next chunk.
-            #
-            # If we're using `splitlines()`, we only do this if the chunk
-            # ended midway through a line.
-            incomplete_line = lines[-1] and lines[-1][-1] == chunk[-1]
-            if delimiter or incomplete_line:
+            if lines and lines[-1] and chunk and lines[-1][-1] == chunk[-1]:
                 pending = lines.pop()
+            else:
+                pending = None
 
             for line in lines:
                 yield line
@@ -956,8 +912,7 @@ class Response(object):
         return l
 
     def raise_for_status(self):
-        """Raises stored :class:`HTTPError`, if one occurred.
-        Otherwise, returns the response object (self)."""
+        """Raises stored :class:`HTTPError`, if one occurred."""
 
         http_error_msg = ''
         if isinstance(self.reason, bytes):
@@ -981,8 +936,6 @@ class Response(object):
         if http_error_msg:
             raise HTTPError(http_error_msg, response=self)
 
-        return self
-
     def close(self):
         """Releases the connection back to the pool. Once this method has been
         called the underlying ``raw`` object must not be accessed again.

requests/sessions.py

@@ -14,24 +14,22 @@ from collections import Mapping
 from datetime import timedelta
 
 from .auth import _basic_auth_str
-from .compat import cookielib, OrderedDict, urljoin, urlparse, is_py3, str
+from .compat import cookielib, is_py3, OrderedDict, urljoin, urlparse
 from .cookies import (
-    cookiejar_from_dict, extract_cookies_to_jar, RequestsCookieJar,
-    merge_cookies, _copy_cookie_jar)
+    cookiejar_from_dict, extract_cookies_to_jar, RequestsCookieJar, merge_cookies)
 from .models import Request, PreparedRequest, DEFAULT_REDIRECT_LIMIT
 from .hooks import default_hooks, dispatch_hook
 from ._internal_utils import to_native_string
 from .utils import to_key_val_list, default_headers
 from .exceptions import (
-    TooManyRedirects, InvalidScheme, ChunkedEncodingError,
-    ConnectionError, ContentDecodingError, InvalidHeader)
+    TooManyRedirects, InvalidSchema, ChunkedEncodingError, ContentDecodingError)
 
 from .structures import CaseInsensitiveDict
 from .adapters import HTTPAdapter
 
 from .utils import (
     requote_uri, get_environ_proxies, get_netrc_auth, should_bypass_proxies,
-    get_auth_from_url, is_valid_location, rewind_body
+    get_auth_from_url, rewind_body
 )
 
 from .status_codes import codes
@@ -52,7 +50,7 @@ else:
 def merge_setting(request_setting, session_setting, dict_class=OrderedDict):
     """Determines appropriate setting for a given request, taking into account
     the explicit setting on that request, and the setting in the session. If a
-    setting is a dictionary, they will be merged together using `dict_class`.
+    setting is a dictionary, they will be merged together using `dict_class`
     """
 
     if session_setting is None:
@@ -97,14 +95,10 @@ def merge_hooks(request_hooks, session_hooks, dict_class=OrderedDict):
 
 class SessionRedirectMixin(object):
 
-    def get_redirect_target(self, response):
+    def get_redirect_target(self, resp):
         """Receives a Response. Returns a redirect URI or ``None``"""
-        if response.is_redirect:
-            if not is_valid_location(response):
-                raise InvalidHeader('Response contains multiple Location headers. '
-                                    'Unable to perform redirect.')
-
-            location = response.headers['location']
+        if resp.is_redirect:
+            location = resp.headers['location']
             # Currently the underlying http module on py3 decode headers
             # in latin1, but empirical evidence suggests that latin1 is very
             # rarely used with non-ASCII characters in HTTP headers.
@@ -116,56 +110,55 @@ class SessionRedirectMixin(object):
             return to_native_string(location, 'utf8')
         return None
 
-    def resolve_redirects(self, response, request, stream=False, timeout=None,
-                          verify=True, cert=None, proxies=None,
-                          yield_requests=False, **adapter_kwargs):
-        """Given a Response, yields Responses until 'Location' header-based
-        redirection ceases, or the Session.max_redirects limit has been
-        reached.
-        """
+    def resolve_redirects(self, resp, req, stream=False, timeout=None,
+                          verify=True, cert=None, proxies=None, yield_requests=False, **adapter_kwargs):
+        """Receives a Response. Returns a generator of Responses or Requests."""
 
-        history = [response]  # keep track of history; seed it with the original response
+        hist = []  # keep track of history
 
-        location_url = self.get_redirect_target(response)
+        url = self.get_redirect_target(resp)
+        while url:
+            prepared_request = req.copy()
 
-        while location_url:
-            prepared_request = request.copy()
+            # Update history and keep track of redirects.
+            # resp.history must ignore the original request in this loop
+            hist.append(resp)
+            resp.history = hist[1:]
 
             try:
-                response.content  # Consume socket so it can be released
-            except (ChunkedEncodingError, ConnectionError, ContentDecodingError, RuntimeError):
-                response.raw.read(decode_content=False)
+                resp.content  # Consume socket so it can be released
+            except (ChunkedEncodingError, ContentDecodingError, RuntimeError):
+                resp.raw.read(decode_content=False)
 
-            if len(response.history) >= self.max_redirects:
-                raise TooManyRedirects('Exceeded %s redirects.' % self.max_redirects, response=response)
+            if len(resp.history) >= self.max_redirects:
+                raise TooManyRedirects('Exceeded %s redirects.' % self.max_redirects, response=resp)
 
             # Release the connection back into the pool.
-            response.close()
+            resp.close()
 
             # Handle redirection without scheme (see: RFC 1808 Section 4)
-            if location_url.startswith('//'):
-                parsed_rurl = urlparse(response.url)
-                location_url = '%s:%s' % (to_native_string(parsed_rurl.scheme), location_url)
+            if url.startswith('//'):
+                parsed_rurl = urlparse(resp.url)
+                url = '%s:%s' % (to_native_string(parsed_rurl.scheme), url)
 
             # The scheme should be lower case...
-            parsed = urlparse(location_url)
-            location_url = parsed.geturl()
+            parsed = urlparse(url)
+            url = parsed.geturl()
 
             # Facilitate relative 'location' headers, as allowed by RFC 7231.
             # (e.g. '/path/to/resource' instead of 'http://domain.tld/path/to/resource')
             # Compliant with RFC3986, we percent encode the url.
             if not parsed.netloc:
-                location_url = urljoin(response.url, requote_uri(location_url))
+                url = urljoin(resp.url, requote_uri(url))
             else:
-                location_url = requote_uri(location_url)
+                url = requote_uri(url)
 
-            prepared_request.url = to_native_string(location_url)
+            prepared_request.url = to_native_string(url)
 
-            method_changed = self.rebuild_method(prepared_request, response)
+            self.rebuild_method(prepared_request, resp)
 
-            # https://github.com/kennethreitz/requests/issues/2590
-            # If method is changed to GET we need to remove body and associated headers.
-            if method_changed and prepared_request.method == 'GET':
+            # https://github.com/requests/requests/issues/1084
+            if resp.status_code not in (codes.temporary_redirect, codes.permanent_redirect):
                 # https://github.com/requests/requests/issues/3490
                 purged_headers = ('Content-Length', 'Content-Type', 'Transfer-Encoding')
                 for header in purged_headers:
@@ -181,13 +174,13 @@ class SessionRedirectMixin(object):
             # Extract any cookies sent on the response to the cookiejar
             # in the new request. Because we've mutated our copied prepared
             # request, use the old one that we haven't yet touched.
-            extract_cookies_to_jar(prepared_request._cookies, request, response.raw)
+            extract_cookies_to_jar(prepared_request._cookies, req, resp.raw)
             merge_cookies(prepared_request._cookies, self.cookies)
             prepared_request.prepare_cookies(prepared_request._cookies)
 
             # Rebuild auth and proxy information.
             proxies = self.rebuild_proxies(prepared_request, proxies)
-            self.rebuild_auth(prepared_request, response)
+            self.rebuild_auth(prepared_request, resp)
 
             # A failed tell() sets `_body_position` to `object()`. This non-None
             # value ensures `rewindable` will be True, allowing us to raise an
@@ -202,14 +195,14 @@ class SessionRedirectMixin(object):
                 rewind_body(prepared_request)
 
             # Override the original request.
-            request = prepared_request
+            req = prepared_request
 
             if yield_requests:
-                yield request
+                yield req
             else:
 
-                response = self.send(
-                    request,
+                resp = self.send(
+                    req,
                     stream=stream,
                     timeout=timeout,
                     verify=verify,
@@ -218,21 +211,16 @@ class SessionRedirectMixin(object):
                     allow_redirects=False,
                     **adapter_kwargs
                 )
-                # copy our history tracker into the response
-                response.history = history[:]
-                # append the new response to the history tracker for the next iteration
-                history.append(response)
 
-                extract_cookies_to_jar(self.cookies, prepared_request, response.raw)
+                extract_cookies_to_jar(self.cookies, prepared_request, resp.raw)
 
                 # extract redirect url, if any, for the next loop
-                location_url = self.get_redirect_target(response)
-                yield response
+                url = self.get_redirect_target(resp)
+                yield resp
 
     def rebuild_auth(self, prepared_request, response):
         """When being redirected we may want to strip authentication from the
-        request to avoid leaking credentials. This method intelligently
-        removes
+        request to avoid leaking credentials. This method intelligently removes
         and reapplies authentication where possible to avoid credential loss.
         """
         headers = prepared_request.headers
@@ -255,11 +243,11 @@ class SessionRedirectMixin(object):
         return
 
     def rebuild_proxies(self, prepared_request, proxies):
-        """This method re-evaluates the proxy configuration by
-        considering the environment variables. If we are redirected to a
-        URL covered by NO_PROXY, we strip the proxy configuration.
-        Otherwise, we set missing proxy keys for this URL (in case they
-        were stripped by a previous redirect).
+        """This method re-evaluates the proxy configuration by considering the
+        environment variables. If we are redirected to a URL covered by
+        NO_PROXY, we strip the proxy configuration. Otherwise, we set missing
+        proxy keys for this URL (in case they were stripped by a previous
+        redirect).
 
         This method also replaces the Proxy-Authorization header where
         necessary.
@@ -298,26 +286,24 @@ class SessionRedirectMixin(object):
     def rebuild_method(self, prepared_request, response):
         """When being redirected we may want to change the method of the request
         based on certain specs or browser behavior.
-
-        :rtype bool:
-        :return: boolean expressing if the method changed during rebuild.
         """
-        method = original_method = prepared_request.method
+        method = prepared_request.method
 
         # http://tools.ietf.org/html/rfc7231#section-6.4.4
         if response.status_code == codes.see_other and method != 'HEAD':
             method = 'GET'
 
-        # If a POST is responded to with a 301 or 302, turn it into a GET. This has
-        # become a common pattern in browsers and was introduced into later versions
-        # of HTTP RFCs. While some browsers transform other methods to GET, little of
-        # that has been standardized. For that reason, we're using curl as a model
-        # which only supports POST->GET.
-        if response.status_code in (codes.found, codes.moved) and method == 'POST':
+        # Do what the browsers do, despite standards...
+        # First, turn 302s into GETs.
+        if response.status_code == codes.found and method != 'HEAD':
+            method = 'GET'
+
+        # Second, if a POST is responded to with a 301, turn it into a GET.
+        # This bizarre behaviour is explained in Issue 1704.
+        if response.status_code == codes.moved and method == 'POST':
             method = 'GET'
 
         prepared_request.method = method
-        return method != original_method
 
 
 class Session(SessionRedirectMixin):
@@ -413,7 +399,7 @@ class Session(SessionRedirectMixin):
         :class:`Session`.
 
         :param request: :class:`Request` instance to prepare with this
-            Session's settings.
+            session's settings.
         :rtype: requests.PreparedRequest
         """
         cookies = request.cookies or {}
@@ -423,8 +409,8 @@ class Session(SessionRedirectMixin):
             cookies = cookiejar_from_dict(cookies)
 
         # Merge with session cookies
-        session_cookies = _copy_cookie_jar(self.cookies)
-        merged_cookies = merge_cookies(session_cookies, cookies)
+        merged_cookies = merge_cookies(
+            merge_cookies(RequestsCookieJar(), self.cookies), cookies)
 
         # Set environment's basic authentication if not explicitly set.
         auth = request.auth
@@ -450,7 +436,7 @@ class Session(SessionRedirectMixin):
             params=None, data=None, headers=None, cookies=None, files=None,
             auth=None, timeout=None, allow_redirects=True, proxies=None,
             hooks=None, stream=None, verify=None, cert=None, json=None):
-        """Constructs a :class:`Request <Request>`, prepares it, and sends it.
+        """Constructs a :class:`Request <Request>`, prepares it and sends it.
         Returns :class:`Response <Response>` object.
 
         :param method: method for the new :class:`Request` object.
@@ -611,8 +597,7 @@ class Session(SessionRedirectMixin):
         if isinstance(request, Request):
             raise ValueError('You can only send PreparedRequests.')
 
-        # Set up variables needed for resolve_redirects and dispatching of
-        # hooks
+        # Set up variables needed for resolve_redirects and dispatching of hooks
         allow_redirects = kwargs.pop('allow_redirects', True)
         stream = kwargs.get('stream')
         hooks = request.hooks
@@ -630,7 +615,7 @@ class Session(SessionRedirectMixin):
         elapsed = preferred_clock() - start
         r.elapsed = timedelta(seconds=elapsed)
 
-        # Response manipulation hooks.
+        # Response manipulation hooks
         r = dispatch_hook('response', hooks, r, **kwargs)
 
         # Persist cookies
@@ -645,12 +630,16 @@ class Session(SessionRedirectMixin):
         # Redirect resolving generator.
         gen = self.resolve_redirects(r, request, **kwargs)
 
-        # Resolve redirects, if allowed.
+        # Resolve redirects if allowed.
         history = [resp for resp in gen] if allow_redirects else []
 
-        # If there is a history, replace ``r`` with the last response
+        # Shuffle things around if there's history.
         if history:
+            # Insert the first (original) request at the start
+            history.insert(0, r)
+            # Get the last request made
             r = history.pop()
+            r.history = history
 
         # If redirects aren't being followed, store the response on the Request for Response.next().
         if not allow_redirects:
@@ -670,37 +659,25 @@ class Session(SessionRedirectMixin):
 
         :rtype: dict
         """
-        # Merge all the kwargs except for proxies.
-        stream = merge_setting(stream, self.stream)
-        verify = merge_setting(verify, self.verify)
-        cert = merge_setting(cert, self.cert)
         # Gather clues from the surrounding environment.
-        # We do this after merging the Session values to make sure we don't
-        # accidentally exclude them.
         if self.trust_env:
+            # Set environment's proxies.
+            no_proxy = proxies.get('no_proxy') if proxies is not None else None
+            env_proxies = get_environ_proxies(url, no_proxy=no_proxy)
+            for (k, v) in env_proxies.items():
+                proxies.setdefault(k, v)
+
             # Look for requests environment configuration and be compatible
             # with cURL.
             if verify is True or verify is None:
                 verify = (os.environ.get('REQUESTS_CA_BUNDLE') or
-                          os.environ.get('CURL_CA_BUNDLE') or
-                          verify)
+                          os.environ.get('CURL_CA_BUNDLE'))
 
-        # Now we handle proxies.
-        # Proxies need to be built up backwards. This is because None values
-        # can delete proxy information, which can then be re-added by a more
-        # specific layer. So we begin by getting the environment's proxies,
-        # then add the Session, then add the request.
-        no_proxy = proxies.get('no_proxy') if proxies is not None else None
-        if no_proxy is None:
-            no_proxy = self.proxies.get('no_proxy')
-
-        env_proxies = {}
-
-        if self.trust_env:
-            env_proxies = get_environ_proxies(url, no_proxy=no_proxy) or {}
-
-        new_proxies = merge_setting(self.proxies, env_proxies)
-        proxies = merge_setting(proxies, new_proxies)
+        # Merge all the kwargs.
+        proxies = merge_setting(proxies, self.proxies)
+        stream = merge_setting(stream, self.stream)
+        verify = merge_setting(verify, self.verify)
+        cert = merge_setting(cert, self.cert)
 
         return {'verify': verify, 'proxies': proxies, 'stream': stream,
                 'cert': cert}
@@ -717,17 +694,17 @@ class Session(SessionRedirectMixin):
                 return adapter
 
         # Nothing matches :-/
-        raise InvalidScheme("No connection adapters were found for '%s'" % url)
+        raise InvalidSchema("No connection adapters were found for '%s'" % url)
 
     def close(self):
-        """Closes all adapters and, as such, the Session."""
+        """Closes all adapters and as such the session"""
         for v in self.adapters.values():
             v.close()
 
     def mount(self, prefix, adapter):
         """Registers a connection adapter to a prefix.
 
-        Adapters are sorted in descending order by key length.
+        Adapters are sorted in descending order by prefix length.
         """
         self.adapters[prefix] = adapter
         keys_to_move = [k for k in self.adapters if len(k) < len(prefix)]

requests/utils.py

@@ -27,7 +27,7 @@ from ._internal_utils import to_native_string
 from .compat import parse_http_list as _parse_list_header
 from .compat import (
     quote, urlparse, bytes, str, OrderedDict, unquote, getproxies,
-    proxy_bypass, urlunparse, basestring, integer_types, is_py2, is_py3,
+    proxy_bypass, urlunparse, basestring, integer_types, is_py3,
     proxy_bypass_environment, getproxies_environment)
 from .cookies import cookiejar_from_dict
 from .structures import CaseInsensitiveDict
@@ -431,6 +431,11 @@ def get_encoding_from_headers(headers):
 def stream_decode_response_unicode(iterator, r):
     """Stream decodes a iterator."""
 
+    if r.encoding is None:
+        for item in iterator:
+            yield item
+        return
+
     decoder = codecs.getincrementaldecoder(r.encoding)(errors='replace')
     for chunk in iterator:
         rv = decoder.decode(chunk)
@@ -498,26 +503,7 @@ def unquote_unreserved(uri):
 
     :rtype: str
     """
-    # This convert function is used to optionally convert the output of `chr`.
-    # In Python 3, `chr` returns a unicode string, while in Python 2 it returns
-    # a bytestring. Here we deal with that by optionally converting.
-    def convert(is_bytes, c):
-        if is_py2 and not is_bytes:
-            return c.decode('ascii')
-        elif is_py3 and is_bytes:
-            return c.encode('ascii')
-        else:
-            return c
-
-    # Handle both bytestrings and unicode strings.
-    is_bytes = isinstance(uri, bytes)
-    splitchar = u'%'
-    base = u''
-    if is_bytes:
-        splitchar = splitchar.encode('ascii')
-        base = base.encode('ascii')
-
-    parts = uri.split(splitchar)
+    parts = uri.split('%')
     for i in range(1, len(parts)):
         h = parts[i][0:2]
         if len(h) == 2 and h.isalnum():
@@ -527,12 +513,12 @@ def unquote_unreserved(uri):
                 raise InvalidURL("Invalid percent-escape sequence: '%s'" % h)
 
             if c in UNRESERVED_SET:
-                parts[i] = convert(is_bytes, c) + parts[i][2:]
+                parts[i] = c + parts[i][2:]
             else:
-                parts[i] = splitchar + parts[i]
+                parts[i] = '%' + parts[i]
         else:
-            parts[i] = splitchar + parts[i]
-    return base.join(parts)
+            parts[i] = '%' + parts[i]
+    return ''.join(parts)
 
 
 def requote_uri(uri):
@@ -626,18 +612,18 @@ def set_environ(env_name, value):
     the environment variable 'env_name'.
 
     If 'value' is None, do nothing"""
-    if value is not None:
+    value_changed = value is not None
+    if value_changed:
         old_value = os.environ.get(env_name)
         os.environ[env_name] = value
     try:
         yield
     finally:
-        if value is None:
-            return
-        if old_value is None:
-            del os.environ[env_name]
-        else:
-            os.environ[env_name] = old_value
+        if value_changed:
+            if old_value is None:
+                del os.environ[env_name]
+            else:
+                os.environ[env_name] = old_value
 
 
 def should_bypass_proxies(url, no_proxy):
@@ -789,19 +775,6 @@ def parse_header_links(value):
     return links
 
 
-def is_valid_location(response):
-    """Verify that multiple Location headers weren't
-    returned from the last response.
-    """
-    headers = getattr(response.raw, 'headers', None)
-    if headers is not None:
-        getlist = getattr(headers, 'getlist', None)
-        if getlist is not None:
-            return len(getlist('location')) <= 1
-    # If response.raw isn't urllib3-like we can't reliably check this
-    return True
-
-
 # Null bytes; no need to recreate these on each call to guess_json_utf
 _null = '\x00'.encode('ascii')  # encoding to ASCII for Python 3
 _null2 = _null * 2
@@ -929,10 +902,3 @@ def rewind_body(prepared_request):
                                         "body for redirect.")
     else:
         raise UnrewindableBodyError("Unable to rewind request body for redirect.")
-
-
-def is_stream(data):
-    """Given data, determines if it should be sent as a stream."""
-    is_iterable = getattr(data, '__iter__', False)
-    is_io_type = not isinstance(data, (basestring, list, tuple, collections.Mapping))
-    return is_iterable and is_io_type

setup.py

@@ -44,7 +44,7 @@ packages = ['requests']
 requires = [
     'chardet>=3.0.2,<3.1.0',
     'idna>=2.5,<2.6',
-    'urllib3>=1.21.1,<1.22',
+    'urllib3>=1.21.1,<1.23',
     'certifi>=2017.4.17'
 
 ]

tests/test_help.py

@@ -0,0 +1,21 @@
+# -*- encoding: utf-8
+
+import sys
+
+import pytest
+
+from requests.help import info
+
+
+@pytest.mark.skipif(sys.version_info[:2] != (2,6), reason="Only run on Python 2.6")
+def test_system_ssl_py26():
+    """OPENSSL_VERSION_NUMBER isn't provided in Python 2.6, verify we don't
+    blow up in this case.
+    """
+    assert info()['system_ssl'] == {'version': ''}
+
+
+@pytest.mark.skipif(sys.version_info < (2,7), reason="Only run on Python 2.7+")
+def test_system_ssl():
+    """Verify we're actually setting system_ssl when it should be available."""
+    assert info()['system_ssl']['version'] != ''

tests/test_lowlevel.py

@@ -23,23 +23,6 @@ def test_chunked_upload():
     assert r.status_code == 200
     assert r.request.headers['Transfer-Encoding'] == 'chunked'
 
-def test_incorrect_content_length():
-    """Test ConnectionError raised for incomplete responses"""
-    close_server = threading.Event()
-    server = Server.text_response_server(
-            "HTTP/1.1 200 OK\r\n" +
-            "Content-Length: 50\r\n\r\n" +
-            "Hello World."
-        )
-    with server as (host, port):
-        url = 'http://{0}:{1}/'.format(host, port)
-        r = requests.Request('GET', url).prepare()
-        s = requests.Session()
-        with pytest.raises(requests.exceptions.ConnectionError) as e:
-            resp = s.send(r)
-        assert "12 bytes read, 38 more expected" in str(e)
-        close_server.set()  # release server block
-
 
 def test_digestauth_401_count_reset_on_redirect():
     """Ensure we correctly reset num_401_calls after a successful digest auth,

tests/test_requests.py

@@ -1,4 +1,3 @@
-#!/usr/bin/env python
 # -*- coding: utf-8 -*-
 
 """Tests for Requests."""
@@ -14,7 +13,6 @@ import warnings
 import io
 import requests
 import pytest
-import pytest_httpbin
 from requests.adapters import HTTPAdapter
 from requests.auth import HTTPDigestAuth, _basic_auth_str
 from requests.compat import (
@@ -23,9 +21,9 @@ from requests.compat import (
 from requests.cookies import (
     cookiejar_from_dict, morsel_to_cookie)
 from requests.exceptions import (
-    ConnectionError, ConnectTimeout, InvalidScheme, InvalidURL,
-    MissingScheme, ReadTimeout, Timeout, RetryError, TooManyRedirects,
-    ProxyError, InvalidHeader, UnrewindableBodyError, InvalidBodyError)
+    ConnectionError, ConnectTimeout, InvalidSchema, InvalidURL,
+    MissingSchema, ReadTimeout, Timeout, RetryError, TooManyRedirects,
+    ProxyError, InvalidHeader, UnrewindableBodyError)
 from requests.models import PreparedRequest
 from requests.structures import CaseInsensitiveDict
 from requests.sessions import SessionRedirectMixin
@@ -36,21 +34,6 @@ from .compat import StringIO, u
 from .utils import override_environ
 from urllib3.util import Timeout as Urllib3Timeout
 
-class SendRecordingAdapter(HTTPAdapter):
-    """
-    A basic subclass of the HTTPAdapter that records the arguments used to
-    ``send``.
-    """
-    def __init__(self, *args, **kwargs):
-        super(SendRecordingAdapter, self).__init__(*args, **kwargs)
-
-        self.send_calls = []
-
-    def send(self, *args, **kwargs):
-        self.send_calls.append((args, kwargs))
-        return super(SendRecordingAdapter, self).send(*args, **kwargs)
-
-
 # Requests to this URL should always fail with a connection timeout (nothing
 # listening on that port)
 TARPIT = 'http://10.255.255.1'
@@ -86,10 +69,10 @@ class TestRequests:
 
     @pytest.mark.parametrize(
         'exception, url', (
-            (MissingScheme, 'hiwpefhipowhefopw'),
-            (InvalidScheme, 'localhost:3128'),
-            (InvalidScheme, 'localhost.localdomain:3128/'),
-            (InvalidScheme, '10.122.1.1:3128/'),
+            (MissingSchema, 'hiwpefhipowhefopw'),
+            (InvalidSchema, 'localhost:3128'),
+            (InvalidSchema, 'localhost.localdomain:3128/'),
+            (InvalidSchema, '10.122.1.1:3128/'),
             (InvalidURL, 'http://'),
         ))
     def test_invalid_url(self, exception, url):
@@ -97,7 +80,7 @@ class TestRequests:
             requests.get(url)
 
     def test_basic_building(self):
-        req = requests.Request(method='GET')
+        req = requests.Request()
         req.url = 'http://kennethreitz.org/'
         req.data = {'life': '42'}
 
@@ -228,106 +211,48 @@ class TestRequests:
         else:
             pytest.fail('Expected custom max number of redirects to be respected but was not')
 
-    @pytest.mark.parametrize(
-        'method, body, expected', (
-            ('GET', None, 'GET'),
-            ('HEAD', None, 'HEAD'),
-            ('POST', 'test', 'GET'),
-            ('PUT', 'put test', 'PUT'),
-            ('PATCH', 'patch test', 'PATCH'),
-            ('DELETE', '', 'DELETE')
-        )
-    )
-    def test_http_301_for_redirectable_methods(self, httpbin, method, body, expected):
-        """Tests all methods except OPTIONS for expected redirect behaviour.
-
-        OPTIONS responses can behave differently depending on the server, so
-        we don't have anything uniform to test except how httpbin responds
-        to them. For that reason they aren't included here.
-        """
-        params = {'url': '/%s' % expected.lower(), 'status_code': '301'}
-        r = requests.request(method, httpbin('redirect-to'), data=body, params=params)
-
-        assert r.request.url == httpbin(expected.lower())
-        assert r.request.method == expected
+    def test_http_301_changes_post_to_get(self, httpbin):
+        r = requests.post(httpbin('status', '301'))
+        assert r.status_code == 200
+        assert r.request.method == 'GET'
         assert r.history[0].status_code == 301
         assert r.history[0].is_redirect
 
-        if expected in ('GET', 'HEAD'):
-            assert r.request.body is None
-        else:
-            assert r.json()['data'] == body
+    def test_http_301_doesnt_change_head_to_get(self, httpbin):
+        r = requests.head(httpbin('status', '301'), allow_redirects=True)
+        print(r.content)
+        assert r.status_code == 200
+        assert r.request.method == 'HEAD'
+        assert r.history[0].status_code == 301
+        assert r.history[0].is_redirect
 
-    @pytest.mark.parametrize(
-        'method, body, expected', (
-            ('GET', None, 'GET'),
-            ('HEAD', None, 'HEAD'),
-            ('POST', 'test', 'GET'),
-            ('PUT', 'put test', 'PUT'),
-            ('PATCH', 'patch test', 'PATCH'),
-            ('DELETE', '', 'DELETE')
-        )
-    )
-    def test_http_302_for_redirectable_methods(self, httpbin, method, body, expected):
-        """Tests all methods except OPTIONS for expected redirect behaviour.
-
-        OPTIONS responses can behave differently depending on the server, so
-        we don't have anything uniform to test except how httpbin responds
-        to them. For that reason they aren't included here.
-        """
-        params = {'url': '/%s' % expected.lower()}
-        r = requests.request(method, httpbin('redirect-to'), data=body, params=params)
-
-        assert r.request.url == httpbin(expected.lower())
-        assert r.request.method == expected
+    def test_http_302_changes_post_to_get(self, httpbin):
+        r = requests.post(httpbin('status', '302'))
+        assert r.status_code == 200
+        assert r.request.method == 'GET'
         assert r.history[0].status_code == 302
         assert r.history[0].is_redirect
 
-        if expected in ('GET', 'HEAD'):
-            assert r.request.body is None
-        else:
-            assert r.json()['data'] == body
+    def test_http_302_doesnt_change_head_to_get(self, httpbin):
+        r = requests.head(httpbin('status', '302'), allow_redirects=True)
+        assert r.status_code == 200
+        assert r.request.method == 'HEAD'
+        assert r.history[0].status_code == 302
+        assert r.history[0].is_redirect
 
-    @pytest.mark.parametrize(
-        'method, body, expected', (
-            ('GET', None, 'GET'),
-            ('HEAD', None, 'HEAD'),
-            ('POST', 'test', 'GET'),
-            ('PUT', 'put test', 'GET'),
-            ('PATCH', 'patch test', 'GET'),
-            ('DELETE', '', 'GET')
-        )
-    )
-    def test_http_303_for_redirectable_methods(self, httpbin, method, body, expected):
-        """Tests all methods except OPTIONS for expected redirect behaviour.
-
-        OPTIONS responses can behave differently depending on the server, so
-        we don't have anything uniform to test except how httpbin responds
-        to them. For that reason they aren't included here.
-        """
-        params = {'url': '/%s' % expected.lower(), 'status_code': '303'}
-        r = requests.request(method, httpbin('redirect-to'), data=body, params=params)
-
-        assert r.request.url == httpbin(expected.lower())
-        assert r.request.method == expected
+    def test_http_303_changes_post_to_get(self, httpbin):
+        r = requests.post(httpbin('status', '303'))
+        assert r.status_code == 200
+        assert r.request.method == 'GET'
         assert r.history[0].status_code == 303
         assert r.history[0].is_redirect
 
-        assert r.request.body is None
-
-    def test_multiple_location_headers(self, httpbin):
-        headers = [('Location', 'http://example.com'),
-                   ('Location', 'https://example.com/1')]
-        params = '&'.join(['%s=%s' % (k, v) for k, v in headers])
-        ses = requests.Session()
-        req = requests.Request('GET', httpbin('response-headers?%s' % params))
-        prep = ses.prepare_request(req)
-        resp = ses.send(prep)
-        # change response to redirect
-        resp.status_code = 302
-        with pytest.raises(InvalidHeader):
-            # next triggers yield on generator
-            next(ses.resolve_redirects(resp, prep))
+    def test_http_303_doesnt_change_head_to_get(self, httpbin):
+        r = requests.head(httpbin('status', '303'), allow_redirects=True)
+        assert r.status_code == 200
+        assert r.request.method == 'HEAD'
+        assert r.history[0].status_code == 303
+        assert r.history[0].is_redirect
 
     def test_header_and_body_removal_on_redirect(self, httpbin):
         purged_headers = ('Content-Length', 'Content-Type')
@@ -484,35 +409,6 @@ class TestRequests:
         assert cookies['foo'] == 'bar'
         assert cookies['cookie'] == 'tasty'
 
-    @pytest.mark.parametrize(
-        'jar', (
-            requests.cookies.RequestsCookieJar(),
-            cookielib.CookieJar()
-        ))
-    def test_custom_cookie_policy_persistence(self, httpbin, jar):
-        """Verify a custom CookiePolicy is propagated on each session request."""
-
-        class TestCookiePolicy(cookielib.DefaultCookiePolicy):
-            """Policy to restrict all cookies from localhost (127.0.0.1)."""
-            def __init__(self):
-                cookielib.DefaultCookiePolicy.__init__(self, blocked_domains=['127.0.0.1'])
-
-        # Establish session with jar and set some cookies.
-        s = requests.Session()
-        s.cookies = jar
-        s.get(httpbin('cookies/set?k1=v1&k2=v2'))
-        assert len(s.cookies) == 2
-
-        # Set different policy.
-        s.cookies.set_policy(TestCookiePolicy())
-        assert isinstance(s.cookies._policy, TestCookiePolicy)
-
-        # No cookies were sent to our blocked domain and none were set.
-        resp = s.get(httpbin('cookies/set?k3=v3'))
-        assert 'Cookie' not in resp.request.headers
-        assert len(s.cookies) == 2
-        assert 'k3' not in s.cookies
-
     def test_requests_in_history_are_not_overridden(self, httpbin):
         resp = requests.get(httpbin('redirect/3'))
         urls = [r.url for r in resp.history]
@@ -590,6 +486,8 @@ class TestRequests:
         'username, password', (
             ('user', 'pass'),
             (u'имя'.encode('utf-8'), u'пароль'.encode('utf-8')),
+            (42, 42),
+            (None, None),
         ))
     def test_set_basicauth(self, httpbin, username, password):
         auth = (username, password)
@@ -600,18 +498,6 @@ class TestRequests:
 
         assert p.headers['Authorization'] == _basic_auth_str(username, password)
 
-    @pytest.mark.parametrize(
-        'username, password', (
-            ('user', 1234),
-            (None, 'test'),
-        ))
-    def test_non_str_basicauth(self, username, password):
-        """Ensure we only allow string or bytes values for basicauth"""
-        with pytest.raises(TypeError) as e:
-            requests.auth._basic_auth_str(username, password)
-
-        assert 'must be of type str or bytes' in str(e)
-
     def test_basicauth_encodes_byte_strings(self):
         """Ensure b'test' formats as the byte string "test" rather
         than the unicode string "b'test'" in Python 3.
@@ -861,10 +747,6 @@ class TestRequests:
         r = requests.get(httpbin('status', '500'))
         assert not r.ok
 
-    def test_raise_for_status_returns_self(self, httpbin):
-        r = requests.get(httpbin('status', '200'))
-        assert r.raise_for_status() is r
-
     def test_decompress_gzip(self, httpbin):
         r = requests.get(httpbin('gzip'))
         r.content.decode('ascii')
@@ -943,8 +825,8 @@ class TestRequests:
             files={'file': ('test_requests.py', open(__file__, 'rb'))})
         assert r.status_code == 200
 
-    @pytest.mark.parametrize('data',
-        (
+    @pytest.mark.parametrize(
+        'data', (
             {'stuff': u('ëlïxr')},
             {'stuff': u('ëlïxr').encode('utf-8')},
             {'stuff': 'elixr'},
@@ -1278,24 +1160,9 @@ class TestRequests:
         r = requests.Response()
         r.raw = io.BytesIO(b'the content')
         r.encoding = 'ascii'
-
         chunks = r.iter_content(decode_unicode=True)
         assert all(isinstance(chunk, str) for chunk in chunks)
 
-    @pytest.mark.parametrize(
-        'encoding, exception', (
-            (None, TypeError),
-            ('invalid encoding', LookupError),
-        ))
-    def test_decode_unicode_encoding(self, encoding, exception):
-        # raise an exception if encoding isn't set
-        r = requests.Response()
-        r.raw = io.BytesIO(b'the content')
-        r.encoding = encoding
-
-        with pytest.raises(exception):
-            chunks = r.iter_content(decode_unicode=True)
-
     def test_response_reason_unicode(self):
         # check for unicode HTTP status
         r = requests.Response()
@@ -1348,109 +1215,6 @@ class TestRequests:
         assert r.request.url == pr.request.url
         assert r.request.headers == pr.request.headers
 
-
-    def test_response_lines(self):
-        """
-        iter_lines should be able to handle data dribbling in which delimiters
-        might not be lined up ideally.
-        """
-        mock_chunks = [
-            b'This \r\n',
-            b'',
-            b'is\r',
-            b'\n',
-            b'a',
-            b' ',
-            b'',
-            b'',
-            b'test.',
-            b'\r',
-            b'\n',
-            b'end.',
-        ]
-        mock_data = b''.join(mock_chunks)
-        unicode_mock_data = mock_data.decode('utf-8')
-
-        def mock_iter_content(*args, **kwargs):
-            if kwargs.get("decode_unicode"):
-                return (e.decode('utf-8') for e in mock_chunks)
-            return (e for e in mock_chunks)
-
-        r = requests.Response()
-        r._content_consumed = True
-        r.iter_content = mock_iter_content
-
-        # decode_unicode=None, output raw bytes
-        assert list(r.iter_lines(delimiter=b'\r\n')) == mock_data.split(b'\r\n')
-
-        # decode_unicode=True, output unicode strings
-        assert list(r.iter_lines(decode_unicode=True, delimiter=u'\r\n')) == unicode_mock_data.split(u'\r\n')
-
-        # When delimiter is None, we should yield the same result as splitlines()
-        # which supports the universal newline.
-        # '\r', '\n', and '\r\n' are all treated as one line break.
-
-        # decode_unicode=None, output raw bytes
-        result = list(r.iter_lines())
-        assert result == mock_data.splitlines()
-
-        # decode_unicode=True, output unicode strings
-        result = list(r.iter_lines(decode_unicode=True))
-        assert result == unicode_mock_data.splitlines()
-
-        # If we change all the line breaks to `\r`, we should be okay.
-        # decode_unicode=None, output raw bytes
-        mock_chunks = [chunk.replace(b'\n', b'\r') for chunk in mock_chunks]
-        mock_data = b''.join(mock_chunks)
-        assert list(r.iter_lines()) == mock_data.splitlines()
-
-        # decode_unicode=True, output unicode strings
-        unicode_mock_data = mock_data.decode('utf-8')
-        assert list(r.iter_lines(decode_unicode=True)) == unicode_mock_data.splitlines()
-
-
-    @pytest.mark.parametrize(
-        'content, expected_no_delimiter, expected_delimiter', (
-            ([b''], [], []),
-            ([b'line\n'], [u'line'], [u'line\n']),
-            ([b'line', b'\n'], [u'line'], [u'line\n']),
-            ([b'line\r\n'], [u'line'], [u'line', u'']),
-            # Empty chunk in the end of stream, same behavior as the previous
-            ([b'line\r\n', b''], [u'line'], [u'line', u'']),
-            ([b'line', b'\r\n'], [u'line'], [u'line', u'']),
-            ([b'a\r', b'\nb\r'], [u'a', u'b'], [u'a', u'b\r']),
-            ([b'a\r', b'\n', b'\nb'], [u'a', u'', u'b'], [u'a', u'\nb']),
-            ([b'a\n', b'\nb'], [u'a', u'', u'b'], [u'a\n\nb']),
-            ([b'a\r\n', b'\rb\n'], [u'a', u'', u'b'], [u'a', u'\rb\n']),
-            ([b'a\nb', b'c'], [u'a', u'bc'], [u'a\nbc']),
-            ([b'a\n', b'\rb', b'\r\nc'], [u'a', u'', u'b', u'c'], [u'a\n\rb', u'c']),
-            ([b'a\r\nb', b'', b'c'], [u'a', u'bc'], [u'a', u'bc'])  # Empty chunk with pending data
-        ))
-    def test_response_lines_parametrized(self, content, expected_no_delimiter, expected_delimiter):
-        """
-        Test a lot of potential chunk splits to ensure consistency of
-        iter_lines(delimiter=x), as well as the legacy behavior of
-        iter_lines() without delimiter
-        https://github.com/kennethreitz/requests/pull/2431#issuecomment-72333964
-        """
-        mock_chunks = content
-        def mock_iter_content(*args, **kwargs):
-            if kwargs.get("decode_unicode"):
-                return (e.decode('utf-8') for e in mock_chunks)
-            return (e for e in mock_chunks)
-
-        r = requests.Response()
-        r._content_consumed = True
-        r.iter_content = mock_iter_content
-
-        # decode_unicode=True, output unicode strings
-        assert list(r.iter_lines(decode_unicode=True)) == expected_no_delimiter
-        assert list(r.iter_lines(decode_unicode=True, delimiter='\r\n')) == expected_delimiter
-
-        # decode_unicode=None, output raw bytes
-        assert list(r.iter_lines()) == [line.encode('utf-8') for line in expected_no_delimiter]
-        assert list(r.iter_lines(delimiter=b'\r\n')) == [line.encode('utf-8') for line in expected_delimiter]
-
     def test_prepared_request_is_pickleable(self, httpbin):
         p = requests.Request('GET', httpbin('get')).prepare()
 
@@ -1723,10 +1487,9 @@ class TestRequests:
 
     def test_manual_redirect_with_partial_body_read(self, httpbin):
         s = requests.Session()
-        req = requests.Request('GET', httpbin('redirect/2')).prepare()
-        r1 = s.send(req, allow_redirects=False, stream=True)
+        r1 = s.get(httpbin('redirect/2'), allow_redirects=False, stream=True)
         assert r1.is_redirect
-        rg = s.resolve_redirects(r1, req, stream=True)
+        rg = s.resolve_redirects(r1, r1.request, stream=True)
 
         # read only the first eight bytes of the response body,
         # then follow the redirect
@@ -1896,15 +1659,20 @@ class TestRequests:
         prep = r.prepare()
         assert 'stuff=elixr' == prep.body
 
-    @pytest.mark.parametrize('decode_unicode', (True, False))
-    def test_response_iter_lines(self, httpbin, decode_unicode):
+    def test_response_iter_lines(self, httpbin):
         r = requests.get(httpbin('stream/4'), stream=True)
         assert r.status_code == 200
-        r.encoding = 'utf-8'
-        it = r.iter_lines(decode_unicode=decode_unicode)
+
+        it = r.iter_lines()
         next(it)
         assert len(list(it)) == 3
 
+    def test_response_context_manager(self, httpbin):
+        with requests.get(httpbin('stream/4'), stream=True) as response:
+            assert isinstance(response, requests.Response)
+
+        assert response.raw.closed
+
     def test_unconsumed_session_response_closes_connection(self, httpbin):
         s = requests.session()
 
@@ -1923,59 +1691,6 @@ class TestRequests:
         next(r.iter_lines())
         assert len(list(r.iter_lines())) == 3
 
-    def test_environment_comes_after_session(self, httpbin):
-        """The Session arguments should come before environment arguments."""
-        # We get proxies from the environment and verify from the argument.
-        s = requests.Session()
-        a = SendRecordingAdapter()
-        s.mount('http://', a)
-
-        # Both of these arguments are safe fallbacks that we can easily
-        # detect, but which will allow the request to succeed.
-        s.verify = False
-        s.proxies = {'http': None}
-
-        old_proxy = os.environ.get('HTTP_PROXY')
-        old_bundle = os.environ.get('REQUESTS_CA_BUNDLE')
-
-        try:
-            os.environ['HTTP_PROXY'] = '10.10.10.10:3128'
-            os.environ['REQUESTS_CA_BUNDLE'] = '/path/to/nowhere'
-
-            s.get(httpbin('get'), timeout=5)
-        finally:
-            if old_proxy is not None:
-                os.environ['HTTP_PROXY'] = old_proxy
-            else:
-                del os.environ['HTTP_PROXY']
-
-            if old_bundle is not None:
-                os.environ['REQUESTS_CA_BUNDLE'] = old_bundle
-            else:
-                del os.environ['REQUESTS_CA_BUNDLE']
-
-        call = a.send_calls[0]
-        assert call[1]['verify'] == False
-
-        proxies = call[1]['proxies']
-        with pytest.raises(KeyError):
-            proxies['http']
-
-    @pytest.fixture(autouse=True)
-    def test_merge_environment_settings_verify(self, monkeypatch):
-        """Assert CA environment settings are merged as expected when missing"""
-        session = requests.Session()
-        monkeypatch.delenv('CURL_CA_BUNDLE', raising=False)
-        monkeypatch.delenv('REQUESTS_CA_BUNDLE', raising=False)
-
-        assert session.trust_env is True
-        assert session.verify is True
-        assert 'REQUESTS_CA_BUNDLE' not in os.environ
-        assert 'CURL_CA_BUNDLE' not in os.environ
-        merged_settings = session.merge_environment_settings(
-            'http://example.com', {}, False, True, None)
-        assert merged_settings['verify'] is True
-
     def test_session_close_proxy_clear(self, mocker):
         proxies = {
           'one': mocker.Mock(),
@@ -2018,33 +1733,6 @@ class TestRequests:
         resp.close()
         assert resp.raw.closed
 
-    def test_updating_ca_cert(self, httpbin_secure):
-        """Assert that requests use the latest configured CA certificates."""
-        session = requests.session()
-        session.verify = pytest_httpbin.certs.where()
-        session.get(httpbin_secure('/'))
-        session.verify = True
-        with pytest.raises(requests.exceptions.SSLError) as e:
-            session.get(httpbin_secure('/'))
-        assert 'certificate verify failed' in str(e)
-
-    def test_updating_client_cert(self, httpbin_secure):
-        """Assert that requests use the latest configured client certificates."""
-        ca_file = pytest_httpbin.certs.where()
-        cert_dir = os.path.dirname(ca_file)
-        # All we need is a valid certificate and key to make a request. httpbin_secure
-        # won't check the signature or subject name, so it's okay that these happen to
-        # be the server's certificate and key.
-        cert = os.path.join(cert_dir, 'cert.pem')
-        key = os.path.join(cert_dir, 'key.pem')
-        session = requests.session()
-        session.verify = ca_file
-        resp = session.get(httpbin_secure('/'))
-        resp_with_cert = session.get(httpbin_secure('/'), cert=(cert, key))
-        assert resp_with_cert.raw._pool.cert_file == cert
-        assert resp_with_cert.raw._pool.key_file == key
-        assert resp.raw._pool is not resp_with_cert.raw._pool
-
     def test_empty_stream_with_auth_does_not_set_content_length_header(self, httpbin):
         """Ensure that a byte stream with size 0 will not set both a Content-Length
         and Transfer-Encoding header.
@@ -2080,61 +1768,6 @@ class TestRequests:
         assert 'Transfer-Encoding' in prepared_request.headers
         assert 'Content-Length' not in prepared_request.headers
 
-    def test_chunked_upload_with_manually_set_content_length_header_raises_error(self, httpbin):
-        """Ensure that if a user manually sets a content length header, when
-        the data is chunked, that an InvalidHeader error is raised.
-        """
-        data = (i for i in [b'a', b'b', b'c']) 
-        url = httpbin('post')
-        with pytest.raises(InvalidHeader):
-            r = requests.post(url, data=data, headers={'Content-Length': 'foo'})
-
-    def test_content_length_with_manually_set_transfer_encoding_raises_error(self, httpbin):
-        """Ensure that if a user manually sets a Transfer-Encoding header when
-        data is not chunked that an InvalidHeader error is raised.
-        """
-        data = 'test data'
-        url = httpbin('post')
-        with pytest.raises(InvalidHeader):
-            r = requests.post(url, data=data, headers={'Transfer-Encoding': 'chunked'})
-
-    def test_null_body_does_not_raise_error(self, httpbin):
-        url = httpbin('post')
-        try:
-            requests.post(url, data=None)
-        except InvalidHeader:
-            pytest.fail('InvalidHeader error raised unexpectedly.')
-
-    @pytest.mark.parametrize(
-        'body, expected', (
-            (None, ('Content-Length', '0')),
-            ('test_data', ('Content-Length', '9')),
-            (io.BytesIO(b'test_data'), ('Content-Length', '9')),
-            (StringIO.StringIO(''), ('Transfer-Encoding', 'chunked'))
-        ))
-    def test_prepare_content_length(self, httpbin, body, expected):
-        """Test prepare_content_length creates expected header."""
-        prep = requests.PreparedRequest()
-        prep.headers = {}
-        prep.method = 'POST'
-
-        # Ensure Content-Length is set appropriately.
-        key, value = expected
-        prep.prepare_content_length(body)
-        assert prep.headers[key] == value
-
-    def test_prepare_content_length_with_bad_body(self, httpbin):
-        """Test prepare_content_length raises exception with unsendable body."""
-        # Initialize minimum required PreparedRequest.
-        prep = requests.PreparedRequest()
-        prep.headers = {}
-        prep.method = 'POST'
-
-        with pytest.raises(InvalidBodyError) as e:
-            # Send object that isn't iterable and has no accessible content.
-            prep.prepare_content_length(object())
-            assert "Non-null body must have length or be streamable." in str(e)
-
     def test_custom_redirect_mixin(self, httpbin):
         """Tests a custom mixin to overwrite ``get_redirect_target``.
 
@@ -2320,18 +1953,6 @@ class TestCaseInsensitiveDict:
         cid['changed'] = True
         assert cid != cid_copy
 
-    def test_url_surrounding_whitespace(self, httpbin):
-        """Test case with URLs surrounded by whitespace characters."""
-        get_url = httpbin('get')
-        # All surrounding whitespaces are supposed to be ignored:
-        assert requests.get(get_url + ' ').status_code == 200
-        assert requests.get(' ' + get_url).status_code == 200
-        assert requests.get(get_url + ' \t ').status_code == 200
-        assert requests.get('  \t' + get_url).status_code == 200
-        assert requests.get(get_url + '\n').status_code == 200
-        # The whitespaces can't be in the middle of the URL though:
-        assert requests.get(get_url + ' abc').status_code == 404
-
 
 class TestMorselToCookieExpires:
     """Tests for morsel_to_cookie when morsel contains expires."""
@@ -2474,7 +2095,6 @@ class RedirectSession(SessionRedirectMixin):
         self.max_redirects = 30
         self.cookies = {}
         self.trust_env = False
-        self.location = '/'
 
     def send(self, *args, **kwargs):
         self.calls.append(SendCall(args, kwargs))
@@ -2489,7 +2109,7 @@ class RedirectSession(SessionRedirectMixin):
         except IndexError:
             r.status_code = 200
 
-        r.headers = CaseInsensitiveDict({'Location': self.location})
+        r.headers = CaseInsensitiveDict({'Location': '/'})
         r.raw = self._build_raw()
         r.request = request
         return r
@@ -2604,16 +2224,6 @@ def test_prepared_copy(kwargs):
         assert getattr(p, attr) == getattr(copy, attr)
 
 
-def test_prepare_requires_a_request_method():
-    req = requests.Request()
-    with pytest.raises(ValueError):
-        req.prepare()
-
-    prepped = PreparedRequest()
-    with pytest.raises(ValueError):
-        prepped.prepare()
-
-
 def test_urllib3_retries(httpbin):
     from urllib3.util import Retry
     s = requests.Session()
@@ -2759,7 +2369,7 @@ class TestPreparingURLs(object):
     )
     def test_parameters_for_nonstandard_schemes(self, input, params, expected):
         """
-        Setting paramters for nonstandard schemes is allowed if those schemes
+        Setting parameters for nonstandard schemes is allowed if those schemes
         begin with "http", and is forbidden otherwise.
         """
         r = requests.Request('GET', url=input, params=params)

tests/test_utils.py

@@ -1,6 +1,7 @@
 # -*- coding: utf-8 -*-
 
 import os
+import copy
 from io import BytesIO
 
 import pytest
@@ -17,7 +18,7 @@ from requests.utils import (
     requote_uri, select_proxy, should_bypass_proxies, super_len,
     to_key_val_list, to_native_string,
     unquote_header_value, unquote_unreserved,
-    urldefragauth, add_dict_to_cookiejar)
+    urldefragauth, add_dict_to_cookiejar, set_environ)
 from requests._internal_utils import unicode_is_ascii
 
 from .compat import StringIO, cStringIO
@@ -651,4 +652,29 @@ def test_should_bypass_proxies_win_registry(url, expected, override,
     monkeypatch.setenv('NO_PROXY', '')
     monkeypatch.setattr(winreg, 'OpenKey', OpenKey)
     monkeypatch.setattr(winreg, 'QueryValueEx', QueryValueEx)
-    assert should_bypass_proxies(url, no_proxy=None) == expected
+
+
+@pytest.mark.parametrize(
+    'env_name, value', (
+            ('no_proxy', '192.168.0.0/24,127.0.0.1,localhost.localdomain'),
+            ('no_proxy', None),
+            ('a_new_key', '192.168.0.0/24,127.0.0.1,localhost.localdomain'),
+            ('a_new_key', None),
+    ))
+def test_set_environ(env_name, value):
+    """Tests set_environ will set environ values and will restore the environ."""
+    environ_copy = copy.deepcopy(os.environ)
+    with set_environ(env_name, value):
+        assert os.environ.get(env_name) == value
+
+    assert os.environ == environ_copy
+
+
+def test_set_environ_raises_exception():
+    """Tests set_environ will raise exceptions in context when the
+    value parameter is None."""
+    with pytest.raises(Exception) as exception:
+        with set_environ('test1', None):
+            raise Exception('Expected exception')
+
+    assert 'Expected exception' in str(exception.value)