This should be 3.0

Persist Session-level CookiePolicy

3.0-HISTORY.rst

@@ -0,0 +1,68 @@
+3.0.0 (2017-xx-xx)
+++++++++++++++++++
+
+- Simplified logic for determining Content-Length and Transfer-Encoding.
+  Requests will now avoid setting both headers on the same request, and
+  raise an exception if this is done manually by a user.
+
+- Remove the HTTPProxyAuth class in favor of supporting proxy auth via
+  the proxies parameter.
+
+- Relax how Requests strips bodies from redirects. 3.0.0 only supports body
+  removal on 301/302 POST redirects and all 303 redirects.
+
+- Remove support for non-string/bytes parameters in ``_basic_auth_str``.
+
+- Prevent ``Session.merge_environment`` from erroneously setting the
+  ``verify`` parameter to ``None`` instead of ``True``.
+
+- Streaming responses with ``Response.iter_lines`` or ``Response.iter_content``
+  now requires an encoding to be set if one isn't provided by the server.
+
+- Raise exception if multiple locations are returned during a redirect.
+
+- Update ConnectionPool connections when TLS/SSL settings change.
+
+- Remove simplejson import and only use standard json module.
+
+- Strip surrounding whitespace from urls.
+
+- MissingSchema and InvalidSchema renamed to MissingScheme and InvalidScheme
+  respectively.
+
+- Change merge order for environment settings to avoid excluding Session-level
+  settings.
+
+- Encode redirect URIs as latin-1 before performing redirects in Python 3 to
+  avoid mangling during the requoting process.
+
+- Remove the ``__bool__`` and ``__nonzero__`` methods from a ``Response``
+  object.
+
+  This has been a planned feature for over a year. The behaviour is surprising
+  to most people and breaks most of the assumptions that people have about
+  Response objects. This resolves issue `#2002`_
+
+- Skip over empty chunks in iterators. Empty chunks could prematurely signal
+  the end of a request body's transmission, skipping them allows all of the
+  data through. See `#2631`_ for more details.
+
+- Rename the ``req`` argument from ``Session.resolve_redirects`` method
+  to ``request``.
+
+- Rename the ``resp`` argument from ``Session.resolve_redirects`` to
+  ``response``.
+
+- New ``PreparedRequest.send`` method. Now, you can
+  ``Request().prepare().send()``.
+
+- All porcelain API functions (e.g. ``requests.get``, etc) now accept an
+  optional ``session`` parameter. If provided, the session given will be used
+  for the request, in place of one being created for you.
+
+- URLs are now automatically stripped of leading/trailing whitespace.
+
+- ``Response.raise_for_status()`` now returns the response object for good responses
+
+.. _#2002: https://github.com/kennethreitz/requests/issues/2002
+.. _#2631: https://github.com/kennethreitz/requests/issues/2631

AUTHORS.rst

@@ -160,16 +160,19 @@ Patches and Suggestions
 - Muhammad Yasoob Ullah Khalid <yasoob.khld@gmail.com> (`@yasoob <https://github.com/yasoob>`_)
 - Paul van der Linden (`@pvanderlinden <https://github.com/pvanderlinden>`_)
 - Colin Dickson (`@colindickson <https://github.com/colindickson>`_)
+- Sabari Kumar Murugesan (`@neosab <https://github.com/neosab>`_)
 - Smiley Barry (`@smiley <https://github.com/smiley>`_)
 - Shagun Sodhani (`@shagunsodhani <https://github.com/shagunsodhani>`_)
 - Robin Linderborg (`@vienno <https://github.com/vienno>`_)
 - Brian Samek (`@bsamek <https://github.com/bsamek>`_)
 - Dmitry Dygalo (`@Stranger6667 <https://github.com/Stranger6667>`_)
+- Tomáš Heger (`@geckon <https://github.com/geckon>`_)
 - piotrjurkiewicz
 - Jesse Shapiro <jesse@jesseshapiro.net> (`@haikuginger <https://github.com/haikuginger>`_)
 - Nate Prewitt <nate.prewitt@gmail.com> (`@nateprewitt <https://github.com/nateprewitt>`_)
 - Maik Himstedt
 - Michael Hunsinger
+- Jeremy Cline <jcline@redhat.com> (`@jeremycline <https://github.com/jeremycline>`_)
 - Brian Bamsch <bbamsch32@gmail.com> (`@bbamsch <https://github.com/bbamsch>`_)
 - Om Prakash Kumar <omprakash070@gmail.com> (`@iamprakashom <https://github.com/iamprakashom>`_)
 - Philipp Konrad <gardiac2002@gmail.com> (`@gardiac2002 <https://github.com/gardiac2002>`_)

HISTORY.rst

@@ -174,6 +174,11 @@ Or, even better::
 - Updated bundled idna to v2.5.
 - Updated bundled certifi to 2017.4.17.
 
+- Altered how ``SessionRedirectMixin.resolve_redirects`` and ``Session.send``
+  process redirect history.  Developers who subclass ``resolve_redirects`` will
+  find a different ``.history`` attribute - the first element now contains the
+  original response, and the last element now contains the active response.
+
 2.13.0 (2017-01-24)
 +++++++++++++++++++
 

docs/api.rst

@@ -74,7 +74,6 @@ Authentication
 
 .. autoclass:: requests.auth.AuthBase
 .. autoclass:: requests.auth.HTTPBasicAuth
-.. autoclass:: requests.auth.HTTPProxyAuth
 .. autoclass:: requests.auth.HTTPDigestAuth
 
 
@@ -252,7 +251,7 @@ API Changes
       }
 
       # In requests 1.x, this was legal, in requests 2.x,
-      #  this raises requests.exceptions.MissingSchema
+      #  this raises requests.exceptions.MissingScheme
       requests.get("http://example.org", proxies=proxies)
 
 

docs/user/quickstart.rst

@@ -371,10 +371,14 @@ But, since our ``status_code`` for ``r`` was ``200``, when we call
 ``raise_for_status()`` we get::
 
     >>> r.raise_for_status()
-    None
+    <Response [200]>
 
 All is well.
 
+.. note:: ``raise_for_status`` returns the response object for a successful response. This eases chaining in trivial cases, where we want bad codes to raise an exception, but use the response otherwise:
+
+    >>> value = requests.get('http://httpbin.org/ip').raise_for_status().json()['origin']
+
 
 Response Headers
 ----------------

requests/__version__.py

@@ -5,8 +5,8 @@
 __title__ = 'requests'
 __description__ = 'Python HTTP for Humans.'
 __url__ = 'http://python-requests.org'
-__version__ = '2.17.3'
-__build__ = 0x021703
+__version__ = '3.0.0'
+__build__ = 0x030000
 __author__ = 'Kenneth Reitz'
 __author_email__ = 'me@kennethreitz.org'
 __license__ = 'Apache 2.0'

requests/adapters.py

@@ -10,6 +10,10 @@ and maintain connections.
 
 import os.path
 import socket
+try:
+    from threading import RLock
+except ImportError:  # threading is an optional module and may not be present.
+    from dummy_threading import RLock
 
 from urllib3.poolmanager import PoolManager, proxy_from_url
 from urllib3.response import HTTPResponse
@@ -34,14 +38,14 @@ from .utils import (DEFAULT_CA_BUNDLE_PATH, get_encoding_from_headers,
 from .structures import CaseInsensitiveDict
 from .cookies import extract_cookies_to_jar
 from .exceptions import (ConnectionError, ConnectTimeout, ReadTimeout, SSLError,
-                         ProxyError, RetryError, InvalidSchema)
+                         ProxyError, RetryError, InvalidScheme)
 from .auth import _basic_auth_str
 
 try:
     from urllib3.contrib.socks import SOCKSProxyManager
 except ImportError:
     def SOCKSProxyManager(*args, **kwargs):
-        raise InvalidSchema("Missing dependencies for SOCKS support.")
+        raise InvalidScheme("Missing dependencies for SOCKS support.")
 
 DEFAULT_POOLBLOCK = False
 DEFAULT_POOLSIZE = 10
@@ -122,6 +126,7 @@ class HTTPAdapter(BaseAdapter):
         self._pool_connections = pool_connections
         self._pool_maxsize = pool_maxsize
         self._pool_block = pool_block
+        self._pool_kw_lock = RLock()
 
         self.init_poolmanager(pool_connections, pool_maxsize, block=pool_block)
 
@@ -134,6 +139,7 @@ class HTTPAdapter(BaseAdapter):
         # self.poolmanager uses a lambda function, which isn't pickleable.
         self.proxy_manager = {}
         self.config = {}
+        self._pool_kw_lock = RLock()
 
         for attr, value in state.items():
             setattr(self, attr, value)
@@ -198,19 +204,21 @@ class HTTPAdapter(BaseAdapter):
 
         return manager
 
-    def cert_verify(self, conn, url, verify, cert):
-        """Verify a SSL certificate. This method should not be called from user
-        code, and is only exposed for use when subclassing the
-        :class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
+    def _update_poolmanager_ssl_kw(self, verify, cert):
+        """Update the :class:`PoolManager <urllib3.poolmanager.PoolManager>`
+        connection_pool_kw with the necessary SSL configuration. This method
+        should not be called from user code, and is only exposed for use when
+        subclassing the :class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
 
-        :param conn: The urllib3 connection object associated with the cert.
-        :param url: The requested URL.
-        :param verify: Either a boolean, in which case it controls whether we verify
-            the server's TLS certificate, or a string, in which case it must be a path
-            to a CA bundle to use
-        :param cert: The SSL certificate to verify.
+        :param verify: Whether we should actually verify the certificate;
+                       optionally a path to a CA certificate bundle or
+                       directory of CA certificates.
+        :param cert: The path to the client certificate and key, if any.
+                     This can either be the path to the certificate and
+                     key concatenated in a single file, or as a tuple of
+                     (cert_file, key_file).
         """
-        if url.lower().startswith('https') and verify:
+        if verify:
 
             cert_loc = None
 
@@ -225,30 +233,35 @@ class HTTPAdapter(BaseAdapter):
                 raise IOError("Could not find a suitable TLS CA certificate bundle, "
                               "invalid path: {0}".format(cert_loc))
 
-            conn.cert_reqs = 'CERT_REQUIRED'
+            self.poolmanager.connection_pool_kw['cert_reqs'] = 'CERT_REQUIRED'
 
             if not os.path.isdir(cert_loc):
-                conn.ca_certs = cert_loc
+                self.poolmanager.connection_pool_kw['ca_certs'] = cert_loc
+                self.poolmanager.connection_pool_kw['ca_cert_dir'] = None
             else:
-                conn.ca_cert_dir = cert_loc
+                self.poolmanager.connection_pool_kw['ca_cert_dir'] = cert_loc
+                self.poolmanager.connection_pool_kw['ca_certs'] = None
         else:
-            conn.cert_reqs = 'CERT_NONE'
-            conn.ca_certs = None
-            conn.ca_cert_dir = None
+            self.poolmanager.connection_pool_kw['cert_reqs'] = 'CERT_NONE'
+            self.poolmanager.connection_pool_kw['ca_certs'] = None
+            self.poolmanager.connection_pool_kw['ca_cert_dir'] = None
 
         if cert:
             if not isinstance(cert, basestring):
-                conn.cert_file = cert[0]
-                conn.key_file = cert[1]
+                self.poolmanager.connection_pool_kw['cert_file'] = cert[0]
+                self.poolmanager.connection_pool_kw['key_file'] = cert[1]
             else:
-                conn.cert_file = cert
-                conn.key_file = None
-            if conn.cert_file and not os.path.exists(conn.cert_file):
+                self.poolmanager.connection_pool_kw['cert_file'] = cert
+                self.poolmanager.connection_pool_kw['key_file'] = None
+
+            cert_file = self.poolmanager.connection_pool_kw['cert_file']
+            key_file = self.poolmanager.connection_pool_kw['key_file']
+            if cert_file and not os.path.exists(cert_file):
                 raise IOError("Could not find the TLS certificate file, "
-                              "invalid path: {0}".format(conn.cert_file))
-            if conn.key_file and not os.path.exists(conn.key_file):
+                              "invalid path: {0}".format(cert_file))
+            if key_file and not os.path.exists(key_file):
                 raise IOError("Could not find the TLS key file, "
-                              "invalid path: {0}".format(conn.key_file))
+                              "invalid path: {0}".format(key_file))
 
     def build_response(self, req, resp):
         """Builds a :class:`Response <requests.Response>` object from a urllib3
@@ -287,7 +300,7 @@ class HTTPAdapter(BaseAdapter):
 
         return response
 
-    def get_connection(self, url, proxies=None):
+    def get_connection(self, url, proxies=None, verify=None, cert=None):
         """Returns a urllib3 connection for the given URL. This should not be
         called from user code, and is only exposed for use when subclassing the
         :class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
@@ -296,17 +309,21 @@ class HTTPAdapter(BaseAdapter):
         :param proxies: (optional) A Requests-style dictionary of proxies used on this request.
         :rtype: urllib3.ConnectionPool
         """
-        proxy = select_proxy(url, proxies)
+        with self._pool_kw_lock:
+            if url.lower().startswith('https'):
+                self._update_poolmanager_ssl_kw(verify, cert)
 
-        if proxy:
-            proxy = prepend_scheme_if_needed(proxy, 'http')
-            proxy_manager = self.proxy_manager_for(proxy)
-            conn = proxy_manager.connection_from_url(url)
-        else:
-            # Only scheme should be lower case
-            parsed = urlparse(url)
-            url = parsed.geturl()
-            conn = self.poolmanager.connection_from_url(url)
+            proxy = select_proxy(url, proxies)
+
+            if proxy:
+                proxy = prepend_scheme_if_needed(proxy, 'http')
+                proxy_manager = self.proxy_manager_for(proxy)
+                conn = proxy_manager.connection_from_url(url)
+            else:
+                # Only scheme should be lower case
+                parsed = urlparse(url)
+                url = parsed.geturl()
+                conn = self.poolmanager.connection_from_url(url)
 
         return conn
 
@@ -401,10 +418,8 @@ class HTTPAdapter(BaseAdapter):
         :param proxies: (optional) The proxies dictionary to apply to the request.
         :rtype: requests.Response
         """
+        conn = self.get_connection(request.url, proxies, verify, cert)
 
-        conn = self.get_connection(request.url, proxies)
-
-        self.cert_verify(conn, request.url, verify, cert)
         url = self.request_url(request, proxies)
         self.add_headers(request)
 
@@ -437,7 +452,8 @@ class HTTPAdapter(BaseAdapter):
                     preload_content=False,
                     decode_content=False,
                     retries=self.max_retries,
-                    timeout=timeout
+                    timeout=timeout,
+                    enforce_content_length=True
                 )
 
             # Send the request.
@@ -458,7 +474,10 @@ class HTTPAdapter(BaseAdapter):
                     low_conn.endheaders()
 
                     for i in request.body:
-                        low_conn.send(hex(len(i))[2:].encode('utf-8'))
+                        chunk_size = len(i)
+                        if chunk_size == 0:
+                            continue
+                        low_conn.send(hex(chunk_size)[2:].encode('utf-8'))
                         low_conn.send(b'\r\n')
                         low_conn.send(i)
                         low_conn.send(b'\r\n')
@@ -478,7 +497,9 @@ class HTTPAdapter(BaseAdapter):
                         pool=conn,
                         connection=low_conn,
                         preload_content=False,
-                        decode_content=False
+                        decode_content=False,
+                        enforce_content_length=True,
+                        request_method=request.method
                     )
                 except:
                     # If we hit any problems here, clean up the connection.

requests/api.py

@@ -13,11 +13,12 @@ This module implements the Requests API.
 from . import sessions
 
 
-def request(method, url, **kwargs):
+def request(method, url, session=None, **kwargs):
     """Constructs and sends a :class:`Request <Request>`.
 
     :param method: method for the new :class:`Request` object.
     :param url: URL for the new :class:`Request` object.
+    :param session: :class:`Session` object to use for this request. If none is given, one will be provided.
     :param params: (optional) Dictionary or bytes to be sent in the query string for the :class:`Request`.
     :param data: (optional) Dictionary or list of tuples ``[(key, value)]`` (will be form-encoded), bytes, or file-like object to send in the body of the :class:`Request`.
     :param json: (optional) json data to send in the body of the :class:`Request`.
@@ -54,7 +55,10 @@ def request(method, url, **kwargs):
     # By using the 'with' statement we are sure the session is closed, thus we
     # avoid leaving sockets open which can trigger a ResourceWarning in some
     # cases, and look like a memory leak in others.
-    with sessions.Session() as session:
+
+    session = sessions.Session() if session is None else session
+
+    with session:
         return session.request(method=method, url=url, **kwargs)
 
 

requests/auth.py

@@ -12,7 +12,6 @@ import re
 import time
 import hashlib
 import threading
-import warnings
 
 from base64 import b64encode
 
@@ -28,33 +27,13 @@ CONTENT_TYPE_MULTI_PART = 'multipart/form-data'
 def _basic_auth_str(username, password):
     """Returns a Basic Auth string."""
 
-    # "I want us to put a big-ol' comment on top of it that
-    # says that this behaviour is dumb but we need to preserve
-    # it because people are relying on it."
-    #    - Lukasa
-    #
-    # These are here solely to maintain backwards compatibility
-    # for things like ints. This will be removed in 3.0.0.
     if not isinstance(username, basestring):
-        warnings.warn(
-            "Non-string usernames will no longer be supported in Requests "
-            "3.0.0. Please convert the object you've passed in ({0!r}) to "
-            "a string or bytes object in the near future to avoid "
-            "problems.".format(username),
-            category=DeprecationWarning,
-        )
-        username = str(username)
+        raise TypeError('username must be of type str or bytes, '
+                        'instead it was %s' % type(username))
 
     if not isinstance(password, basestring):
-        warnings.warn(
-            "Non-string passwords will no longer be supported in Requests "
-            "3.0.0. Please convert the object you've passed in ({0!r}) to "
-            "a string or bytes object in the near future to avoid "
-            "problems.".format(password),
-            category=DeprecationWarning,
-        )
-        password = str(password)
-    # -- End Removal --
+        raise TypeError('password must be of type str or bytes, '
+                        'instead it was %s' % type(password))
 
     if isinstance(username, str):
         username = username.encode('latin1')
@@ -97,14 +76,6 @@ class HTTPBasicAuth(AuthBase):
         return r
 
 
-class HTTPProxyAuth(HTTPBasicAuth):
-    """Attaches HTTP Proxy Authentication to a given Request object."""
-
-    def __call__(self, r):
-        r.headers['Proxy-Authorization'] = _basic_auth_str(self.username, self.password)
-        return r
-
-
 class HTTPDigestAuth(AuthBase):
     """Attaches HTTP Digest Authentication to the given Request object."""
 

requests/compat.py

@@ -25,12 +25,6 @@ is_py2 = (_ver[0] == 2)
 #: Python 3.x?
 is_py3 = (_ver[0] == 3)
 
-try:
-    import simplejson as json
-except (ImportError, SyntaxError):
-    # simplejson does not support Python 3.2, it throws a SyntaxError
-    # because of u'...' Unicode literals.
-    import json
 
 # ---------
 # Specifics

requests/cookies.py

@@ -414,7 +414,7 @@ class RequestsCookieJar(cookielib.CookieJar, collections.MutableMapping):
 
     def copy(self):
         """Return a copy of this RequestsCookieJar."""
-        new_cj = RequestsCookieJar()
+        new_cj = RequestsCookieJar(self._policy)
         new_cj.update(self)
         return new_cj
 

requests/exceptions.py

@@ -69,12 +69,12 @@ class TooManyRedirects(RequestException):
     """Too many redirects."""
 
 
-class MissingSchema(RequestException, ValueError):
-    """The URL schema (e.g. http or https) is missing."""
+class MissingScheme(RequestException, ValueError):
+    """The URL scheme (e.g. http or https) is missing."""
 
 
-class InvalidSchema(RequestException, ValueError):
-    """See defaults.py for valid schemas."""
+class InvalidScheme(RequestException, ValueError):
+    """See defaults.py for valid schemes."""
 
 
 class InvalidURL(RequestException, ValueError):
@@ -104,6 +104,10 @@ class RetryError(RequestException):
 class UnrewindableBodyError(RequestException):
     """Requests encountered an error when trying to rewind a body"""
 
+
+class InvalidBodyError(RequestException, ValueError):
+    """An invalid request body was specified"""
+
 # Warnings
 
 

requests/models.py

@@ -9,6 +9,7 @@ This module contains the primary objects that power Requests.
 
 import collections
 import datetime
+import codecs
 import sys
 
 # Import encoding now, to avoid implicit import later.
@@ -26,20 +27,23 @@ from io import UnsupportedOperation
 from .hooks import default_hooks
 from .structures import CaseInsensitiveDict
 
+import requests
 from .auth import HTTPBasicAuth
 from .cookies import cookiejar_from_dict, get_cookie_header, _copy_cookie_jar
 from .exceptions import (
-    HTTPError, MissingSchema, InvalidURL, ChunkedEncodingError,
-    ContentDecodingError, ConnectionError, StreamConsumedError)
+    HTTPError, MissingScheme, InvalidURL, ChunkedEncodingError,
+    ContentDecodingError, ConnectionError, StreamConsumedError,
+    InvalidHeader, InvalidBodyError)
 from ._internal_utils import to_native_string, unicode_is_ascii
 from .utils import (
     guess_filename, get_auth_from_url, requote_uri,
     stream_decode_response_unicode, to_key_val_list, parse_header_links,
-    iter_slices, guess_json_utf, super_len, check_header_validity)
+    iter_slices, guess_json_utf, super_len, check_header_validity,
+    is_stream)
 from .compat import (
     cookielib, urlunparse, urlsplit, urlencode, str, bytes,
     is_py2, chardet, builtin_str, basestring)
-from .compat import json as complexjson
+import json as complexjson
 from .status_codes import codes
 
 #: The set of HTTP status codes that indicate an automatically
@@ -331,8 +335,9 @@ class PreparedRequest(RequestEncodingMixin, RequestHooksMixin):
     def prepare_method(self, method):
         """Prepares the given HTTP method."""
         self.method = method
-        if self.method is not None:
-            self.method = to_native_string(self.method.upper())
+        if self.method is None:
+            raise ValueError('Request method cannot be "None"')
+        self.method = to_native_string(self.method.upper())
 
     @staticmethod
     def _get_idna_encoded_host(host):
@@ -356,8 +361,8 @@ class PreparedRequest(RequestEncodingMixin, RequestHooksMixin):
         else:
             url = unicode(url) if is_py2 else str(url)
 
-        # Remove leading whitespaces from url
-        url = url.lstrip()
+        # Ignore any leading and trailing whitespace characters.
+        url = url.strip()
 
         # Don't do any URL preparation for non-HTTP schemes like `mailto`,
         # `data` etc to work around exceptions from `url_parse`, which
@@ -373,10 +378,10 @@ class PreparedRequest(RequestEncodingMixin, RequestHooksMixin):
             raise InvalidURL(*e.args)
 
         if not scheme:
-            error = ("Invalid URL {0!r}: No schema supplied. Perhaps you meant http://{0}?")
+            error = ("Invalid URL {0!r}: No scheme supplied. Perhaps you meant http://{0}?")
             error = error.format(to_native_string(url, 'utf8'))
 
-            raise MissingSchema(error)
+            raise MissingScheme(error)
 
         if not host:
             raise InvalidURL("Invalid URL %r: No host supplied" % url)
@@ -459,17 +464,7 @@ class PreparedRequest(RequestEncodingMixin, RequestHooksMixin):
             if not isinstance(body, bytes):
                 body = body.encode('utf-8')
 
-        is_stream = all([
-            hasattr(data, '__iter__'),
-            not isinstance(data, (basestring, list, tuple, collections.Mapping))
-        ])
-
-        try:
-            length = super_len(data)
-        except (TypeError, AttributeError, UnsupportedOperation):
-            length = None
-
-        if is_stream:
+        if is_stream(data):
             body = data
 
             if getattr(body, 'tell', None) is not None:
@@ -486,10 +481,6 @@ class PreparedRequest(RequestEncodingMixin, RequestHooksMixin):
             if files:
                 raise NotImplementedError('Streamed bodies and files are mutually exclusive.')
 
-            if length:
-                self.headers['Content-Length'] = builtin_str(length)
-            else:
-                self.headers['Transfer-Encoding'] = 'chunked'
         else:
             # Multi-part file uploads.
             if files:
@@ -502,27 +493,40 @@ class PreparedRequest(RequestEncodingMixin, RequestHooksMixin):
                     else:
                         content_type = 'application/x-www-form-urlencoded'
 
-            self.prepare_content_length(body)
-
             # Add content-type if it wasn't explicitly provided.
             if content_type and ('content-type' not in self.headers):
                 self.headers['Content-Type'] = content_type
 
+        self.prepare_content_length(body)
         self.body = body
 
     def prepare_content_length(self, body):
-        """Prepare Content-Length header based on request method and body"""
+        """Prepares Content-Length header.
+
+        If the length of the body of the request can be computed, Content-Length
+        is set using ``super_len``. If user has manually set either a
+        Transfer-Encoding or Content-Length header when it should not be set
+        (they should be mutually exclusive) an InvalidHeader
+        error will be raised.
+        """
         if body is not None:
             length = super_len(body)
+
             if length:
-                # If length exists, set it. Otherwise, we fallback
-                # to Transfer-Encoding: chunked.
                 self.headers['Content-Length'] = builtin_str(length)
+            elif is_stream(body):
+                self.headers['Transfer-Encoding'] = 'chunked'
+            else:
+                raise InvalidBodyError('Non-null body must have length or be streamable.')
         elif self.method not in ('GET', 'HEAD') and self.headers.get('Content-Length') is None:
             # Set Content-Length to 0 for methods that can have a body
             # but don't provide one. (i.e. not GET or HEAD)
             self.headers['Content-Length'] = '0'
 
+        if 'Transfer-Encoding' in self.headers and 'Content-Length' in self.headers:
+            raise InvalidHeader('Conflicting Headers: Both Transfer-Encoding and '
+                                'Content-Length are set.')
+
     def prepare_auth(self, auth, url=''):
         """Prepares the given HTTP auth data."""
 
@@ -574,6 +578,14 @@ class PreparedRequest(RequestEncodingMixin, RequestHooksMixin):
         for event in hooks:
             self.register_hook(event, hooks[event])
 
+    def send(self, session=None, **send_kwargs):
+        """Sends the PreparedRequest to the given Session.
+        If none is provided, one is created for you."""
+        session = requests.Session() if session is None else session
+
+        with session:
+            return session.send(self, **send_kwargs)
+
 
 class Response(object):
     """The :class:`Response <Response>` object, which contains a
@@ -608,7 +620,8 @@ class Response(object):
         #: Final URL location of Response.
         self.url = None
 
-        #: Encoding to decode with when accessing r.text.
+        #: Encoding to decode with when accessing r.text or
+        #: r.iter_content(decode_unicode=True)
         self.encoding = None
 
         #: A list of :class:`Response <Response>` objects from
@@ -656,26 +669,6 @@ class Response(object):
     def __repr__(self):
         return '<Response [%s]>' % (self.status_code)
 
-    def __bool__(self):
-        """Returns True if :attr:`status_code` is less than 400.
-
-        This attribute checks if the status code of the response is between
-        400 and 600 to see if there was a client error or a server error. If
-        the status code, is between 200 and 400, this will return True. This
-        is **not** a check to see if the response code is ``200 OK``.
-        """
-        return self.ok
-
-    def __nonzero__(self):
-        """Returns True if :attr:`status_code` is less than 400.
-
-        This attribute checks if the status code of the response is between
-        400 and 600 to see if there was a client error or a server error. If
-        the status code, is between 200 and 400, this will return True. This
-        is **not** a check to see if the response code is ``200 OK``.
-        """
-        return self.ok
-
     def __iter__(self):
         """Allows you to use a response as an iterator."""
         return self.iter_content(128)
@@ -730,8 +723,8 @@ class Response(object):
         chunks are received. If stream=False, data is returned as
         a single chunk.
 
-        If decode_unicode is True, content will be decoded using the best
-        available encoding based on the response.
+        If using decode_unicode, the encoding must be set to a valid encoding
+        enumeration before invoking iter_content.
         """
 
         def generate():
@@ -741,7 +734,10 @@ class Response(object):
                     for chunk in self.raw.stream(chunk_size, decode_content=True):
                         yield chunk
                 except ProtocolError as e:
-                    raise ChunkedEncodingError(e)
+                    if self.headers.get('Transfer-Encoding') == 'chunked':
+                        raise ChunkedEncodingError(e)
+                    else:
+                        raise ConnectionError(e)
                 except DecodeError as e:
                     raise ContentDecodingError(e)
                 except ReadTimeoutError as e:
@@ -768,6 +764,16 @@ class Response(object):
         chunks = reused_chunks if self._content_consumed else stream_chunks
 
         if decode_unicode:
+            if self.encoding is None:
+                raise TypeError(
+                    'encoding must be set before consuming streaming '
+                    'responses'
+                )
+
+            # check encoding value here, don't wait for the generator to be
+            # consumed before raising an exception
+            codecs.lookup(self.encoding)
+
             chunks = stream_decode_response_unicode(chunks, self)
 
         return chunks
@@ -779,23 +785,67 @@ class Response(object):
 
         .. note:: This method is not reentrant safe.
         """
+        carriage_return = u'\r' if decode_unicode else b'\r'
+        line_feed = u'\n' if decode_unicode else b'\n'
 
         pending = None
+        last_chunk_ends_with_cr = False
 
-        for chunk in self.iter_content(chunk_size=chunk_size, decode_unicode=decode_unicode):
+        for chunk in self.iter_content(chunk_size=chunk_size,
+                                       decode_unicode=decode_unicode):
+            # Skip any null responses: if there is pending data it is necessarily an
+            # incomplete chunk, so if we don't have more data we don't want to bother
+            # trying to get it. Unconsumed pending data will be yielded anyway in the
+            # end of the loop if the stream ends.
+            if not chunk:
+                continue
 
+            # Consume any pending data
             if pending is not None:
                 chunk = pending + chunk
+                pending = None
 
+            # Either split on a line, or split on a specified delimiter
             if delimiter:
                 lines = chunk.split(delimiter)
             else:
+                # Python splitlines() supports the universal newline (PEP 278).
+                # That means, '\r', '\n', and '\r\n' are all treated as end of
+                # line. If the last chunk ends with '\r', and the current chunk
+                # starts with '\n', they should be merged and treated as only
+                # *one* new line separator '\r\n' by splitlines().
+                # This rule only applies when splitlines() is used.
+
+                # The last chunk ends with '\r', so the '\n' at chunk[0]
+                # is just the second half of a '\r\n' pair rather than a
+                # new line break. Just skip it.
+                skip_first_char = last_chunk_ends_with_cr and chunk.startswith(line_feed)
+                last_chunk_ends_with_cr = chunk.endswith(carriage_return)
+                if skip_first_char:
+                    chunk = chunk[1:]
+                    # it's possible that after stripping the '\n' then chunk becomes empty
+                    if not chunk:
+                        continue
                 lines = chunk.splitlines()
 
-            if lines and lines[-1] and chunk and lines[-1][-1] == chunk[-1]:
+            # Calling `.split(delimiter)` will always end with whatever text
+            # remains beyond the delimiter, or '' if the delimiter is the end
+            # of the text.  On the other hand, `.splitlines()` doesn't include
+            # a '' if the text ends in a line delimiter.
+            #
+            # For example:
+            #
+            #     'abc\ndef\n'.split('\n')  ~> ['abc', 'def', '']
+            #     'abc\ndef\n'.splitlines() ~> ['abc', 'def']
+            #
+            # So if we have a specified delimiter, we always pop the final
+            # item and prepend it to the next chunk.
+            #
+            # If we're using `splitlines()`, we only do this if the chunk
+            # ended midway through a line.
+            incomplete_line = lines[-1] and lines[-1][-1] == chunk[-1]
+            if delimiter or incomplete_line:
                 pending = lines.pop()
-            else:
-                pending = None
 
             for line in lines:
                 yield line
@@ -906,7 +956,8 @@ class Response(object):
         return l
 
     def raise_for_status(self):
-        """Raises stored :class:`HTTPError`, if one occurred."""
+        """Raises stored :class:`HTTPError`, if one occurred.
+        Otherwise, returns the response object (self)."""
 
         http_error_msg = ''
         if isinstance(self.reason, bytes):
@@ -930,6 +981,8 @@ class Response(object):
         if http_error_msg:
             raise HTTPError(http_error_msg, response=self)
 
+        return self
+
     def close(self):
         """Releases the connection back to the pool. Once this method has been
         called the underlying ``raw`` object must not be accessed again.

requests/sessions.py

@@ -14,22 +14,24 @@ from collections import Mapping
 from datetime import timedelta
 
 from .auth import _basic_auth_str
-from .compat import cookielib, is_py3, OrderedDict, urljoin, urlparse
+from .compat import cookielib, OrderedDict, urljoin, urlparse, is_py3, str
 from .cookies import (
-    cookiejar_from_dict, extract_cookies_to_jar, RequestsCookieJar, merge_cookies)
+    cookiejar_from_dict, extract_cookies_to_jar, RequestsCookieJar,
+    merge_cookies, _copy_cookie_jar)
 from .models import Request, PreparedRequest, DEFAULT_REDIRECT_LIMIT
 from .hooks import default_hooks, dispatch_hook
 from ._internal_utils import to_native_string
 from .utils import to_key_val_list, default_headers
 from .exceptions import (
-    TooManyRedirects, InvalidSchema, ChunkedEncodingError, ContentDecodingError)
+    TooManyRedirects, InvalidScheme, ChunkedEncodingError,
+    ConnectionError, ContentDecodingError, InvalidHeader)
 
 from .structures import CaseInsensitiveDict
 from .adapters import HTTPAdapter
 
 from .utils import (
     requote_uri, get_environ_proxies, get_netrc_auth, should_bypass_proxies,
-    get_auth_from_url, rewind_body
+    get_auth_from_url, is_valid_location, rewind_body
 )
 
 from .status_codes import codes
@@ -50,7 +52,7 @@ else:
 def merge_setting(request_setting, session_setting, dict_class=OrderedDict):
     """Determines appropriate setting for a given request, taking into account
     the explicit setting on that request, and the setting in the session. If a
-    setting is a dictionary, they will be merged together using `dict_class`
+    setting is a dictionary, they will be merged together using `dict_class`.
     """
 
     if session_setting is None:
@@ -95,10 +97,14 @@ def merge_hooks(request_hooks, session_hooks, dict_class=OrderedDict):
 
 class SessionRedirectMixin(object):
 
-    def get_redirect_target(self, resp):
+    def get_redirect_target(self, response):
         """Receives a Response. Returns a redirect URI or ``None``"""
-        if resp.is_redirect:
-            location = resp.headers['location']
+        if response.is_redirect:
+            if not is_valid_location(response):
+                raise InvalidHeader('Response contains multiple Location headers. '
+                                    'Unable to perform redirect.')
+
+            location = response.headers['location']
             # Currently the underlying http module on py3 decode headers
             # in latin1, but empirical evidence suggests that latin1 is very
             # rarely used with non-ASCII characters in HTTP headers.
@@ -110,55 +116,56 @@ class SessionRedirectMixin(object):
             return to_native_string(location, 'utf8')
         return None
 
-    def resolve_redirects(self, resp, req, stream=False, timeout=None,
-                          verify=True, cert=None, proxies=None, yield_requests=False, **adapter_kwargs):
-        """Receives a Response. Returns a generator of Responses or Requests."""
+    def resolve_redirects(self, response, request, stream=False, timeout=None,
+                          verify=True, cert=None, proxies=None,
+                          yield_requests=False, **adapter_kwargs):
+        """Given a Response, yields Responses until 'Location' header-based
+        redirection ceases, or the Session.max_redirects limit has been
+        reached.
+        """
 
-        hist = []  # keep track of history
+        history = [response]  # keep track of history; seed it with the original response
 
-        url = self.get_redirect_target(resp)
-        while url:
-            prepared_request = req.copy()
+        location_url = self.get_redirect_target(response)
 
-            # Update history and keep track of redirects.
-            # resp.history must ignore the original request in this loop
-            hist.append(resp)
-            resp.history = hist[1:]
+        while location_url:
+            prepared_request = request.copy()
 
             try:
-                resp.content  # Consume socket so it can be released
-            except (ChunkedEncodingError, ContentDecodingError, RuntimeError):
-                resp.raw.read(decode_content=False)
+                response.content  # Consume socket so it can be released
+            except (ChunkedEncodingError, ConnectionError, ContentDecodingError, RuntimeError):
+                response.raw.read(decode_content=False)
 
-            if len(resp.history) >= self.max_redirects:
-                raise TooManyRedirects('Exceeded %s redirects.' % self.max_redirects, response=resp)
+            if len(response.history) >= self.max_redirects:
+                raise TooManyRedirects('Exceeded %s redirects.' % self.max_redirects, response=response)
 
             # Release the connection back into the pool.
-            resp.close()
+            response.close()
 
             # Handle redirection without scheme (see: RFC 1808 Section 4)
-            if url.startswith('//'):
-                parsed_rurl = urlparse(resp.url)
-                url = '%s:%s' % (to_native_string(parsed_rurl.scheme), url)
+            if location_url.startswith('//'):
+                parsed_rurl = urlparse(response.url)
+                location_url = '%s:%s' % (to_native_string(parsed_rurl.scheme), location_url)
 
             # The scheme should be lower case...
-            parsed = urlparse(url)
-            url = parsed.geturl()
+            parsed = urlparse(location_url)
+            location_url = parsed.geturl()
 
             # Facilitate relative 'location' headers, as allowed by RFC 7231.
             # (e.g. '/path/to/resource' instead of 'http://domain.tld/path/to/resource')
             # Compliant with RFC3986, we percent encode the url.
             if not parsed.netloc:
-                url = urljoin(resp.url, requote_uri(url))
+                location_url = urljoin(response.url, requote_uri(location_url))
             else:
-                url = requote_uri(url)
+                location_url = requote_uri(location_url)
 
-            prepared_request.url = to_native_string(url)
+            prepared_request.url = to_native_string(location_url)
 
-            self.rebuild_method(prepared_request, resp)
+            method_changed = self.rebuild_method(prepared_request, response)
 
-            # https://github.com/requests/requests/issues/1084
-            if resp.status_code not in (codes.temporary_redirect, codes.permanent_redirect):
+            # https://github.com/kennethreitz/requests/issues/2590
+            # If method is changed to GET we need to remove body and associated headers.
+            if method_changed and prepared_request.method == 'GET':
                 # https://github.com/requests/requests/issues/3490
                 purged_headers = ('Content-Length', 'Content-Type', 'Transfer-Encoding')
                 for header in purged_headers:
@@ -174,13 +181,13 @@ class SessionRedirectMixin(object):
             # Extract any cookies sent on the response to the cookiejar
             # in the new request. Because we've mutated our copied prepared
             # request, use the old one that we haven't yet touched.
-            extract_cookies_to_jar(prepared_request._cookies, req, resp.raw)
+            extract_cookies_to_jar(prepared_request._cookies, request, response.raw)
             merge_cookies(prepared_request._cookies, self.cookies)
             prepared_request.prepare_cookies(prepared_request._cookies)
 
             # Rebuild auth and proxy information.
             proxies = self.rebuild_proxies(prepared_request, proxies)
-            self.rebuild_auth(prepared_request, resp)
+            self.rebuild_auth(prepared_request, response)
 
             # A failed tell() sets `_body_position` to `object()`. This non-None
             # value ensures `rewindable` will be True, allowing us to raise an
@@ -195,14 +202,14 @@ class SessionRedirectMixin(object):
                 rewind_body(prepared_request)
 
             # Override the original request.
-            req = prepared_request
+            request = prepared_request
 
             if yield_requests:
-                yield req
+                yield request
             else:
 
-                resp = self.send(
-                    req,
+                response = self.send(
+                    request,
                     stream=stream,
                     timeout=timeout,
                     verify=verify,
@@ -211,16 +218,21 @@ class SessionRedirectMixin(object):
                     allow_redirects=False,
                     **adapter_kwargs
                 )
+                # copy our history tracker into the response
+                response.history = history[:]
+                # append the new response to the history tracker for the next iteration
+                history.append(response)
 
-                extract_cookies_to_jar(self.cookies, prepared_request, resp.raw)
+                extract_cookies_to_jar(self.cookies, prepared_request, response.raw)
 
                 # extract redirect url, if any, for the next loop
-                url = self.get_redirect_target(resp)
-                yield resp
+                location_url = self.get_redirect_target(response)
+                yield response
 
     def rebuild_auth(self, prepared_request, response):
         """When being redirected we may want to strip authentication from the
-        request to avoid leaking credentials. This method intelligently removes
+        request to avoid leaking credentials. This method intelligently
+        removes
         and reapplies authentication where possible to avoid credential loss.
         """
         headers = prepared_request.headers
@@ -243,11 +255,11 @@ class SessionRedirectMixin(object):
         return
 
     def rebuild_proxies(self, prepared_request, proxies):
-        """This method re-evaluates the proxy configuration by considering the
-        environment variables. If we are redirected to a URL covered by
-        NO_PROXY, we strip the proxy configuration. Otherwise, we set missing
-        proxy keys for this URL (in case they were stripped by a previous
-        redirect).
+        """This method re-evaluates the proxy configuration by
+        considering the environment variables. If we are redirected to a
+        URL covered by NO_PROXY, we strip the proxy configuration.
+        Otherwise, we set missing proxy keys for this URL (in case they
+        were stripped by a previous redirect).
 
         This method also replaces the Proxy-Authorization header where
         necessary.
@@ -286,24 +298,26 @@ class SessionRedirectMixin(object):
     def rebuild_method(self, prepared_request, response):
         """When being redirected we may want to change the method of the request
         based on certain specs or browser behavior.
+
+        :rtype bool:
+        :return: boolean expressing if the method changed during rebuild.
         """
-        method = prepared_request.method
+        method = original_method = prepared_request.method
 
         # http://tools.ietf.org/html/rfc7231#section-6.4.4
         if response.status_code == codes.see_other and method != 'HEAD':
             method = 'GET'
 
-        # Do what the browsers do, despite standards...
-        # First, turn 302s into GETs.
-        if response.status_code == codes.found and method != 'HEAD':
-            method = 'GET'
-
-        # Second, if a POST is responded to with a 301, turn it into a GET.
-        # This bizarre behaviour is explained in Issue 1704.
-        if response.status_code == codes.moved and method == 'POST':
+        # If a POST is responded to with a 301 or 302, turn it into a GET. This has
+        # become a common pattern in browsers and was introduced into later versions
+        # of HTTP RFCs. While some browsers transform other methods to GET, little of
+        # that has been standardized. For that reason, we're using curl as a model
+        # which only supports POST->GET.
+        if response.status_code in (codes.found, codes.moved) and method == 'POST':
             method = 'GET'
 
         prepared_request.method = method
+        return method != original_method
 
 
 class Session(SessionRedirectMixin):
@@ -399,7 +413,7 @@ class Session(SessionRedirectMixin):
         :class:`Session`.
 
         :param request: :class:`Request` instance to prepare with this
-            session's settings.
+            Session's settings.
         :rtype: requests.PreparedRequest
         """
         cookies = request.cookies or {}
@@ -409,8 +423,8 @@ class Session(SessionRedirectMixin):
             cookies = cookiejar_from_dict(cookies)
 
         # Merge with session cookies
-        merged_cookies = merge_cookies(
-            merge_cookies(RequestsCookieJar(), self.cookies), cookies)
+        session_cookies = _copy_cookie_jar(self.cookies)
+        merged_cookies = merge_cookies(session_cookies, cookies)
 
         # Set environment's basic authentication if not explicitly set.
         auth = request.auth
@@ -436,7 +450,7 @@ class Session(SessionRedirectMixin):
             params=None, data=None, headers=None, cookies=None, files=None,
             auth=None, timeout=None, allow_redirects=True, proxies=None,
             hooks=None, stream=None, verify=None, cert=None, json=None):
-        """Constructs a :class:`Request <Request>`, prepares it and sends it.
+        """Constructs a :class:`Request <Request>`, prepares it, and sends it.
         Returns :class:`Response <Response>` object.
 
         :param method: method for the new :class:`Request` object.
@@ -597,7 +611,8 @@ class Session(SessionRedirectMixin):
         if isinstance(request, Request):
             raise ValueError('You can only send PreparedRequests.')
 
-        # Set up variables needed for resolve_redirects and dispatching of hooks
+        # Set up variables needed for resolve_redirects and dispatching of
+        # hooks
         allow_redirects = kwargs.pop('allow_redirects', True)
         stream = kwargs.get('stream')
         hooks = request.hooks
@@ -615,7 +630,7 @@ class Session(SessionRedirectMixin):
         elapsed = preferred_clock() - start
         r.elapsed = timedelta(seconds=elapsed)
 
-        # Response manipulation hooks
+        # Response manipulation hooks.
         r = dispatch_hook('response', hooks, r, **kwargs)
 
         # Persist cookies
@@ -630,16 +645,12 @@ class Session(SessionRedirectMixin):
         # Redirect resolving generator.
         gen = self.resolve_redirects(r, request, **kwargs)
 
-        # Resolve redirects if allowed.
+        # Resolve redirects, if allowed.
         history = [resp for resp in gen] if allow_redirects else []
 
-        # Shuffle things around if there's history.
+        # If there is a history, replace ``r`` with the last response
         if history:
-            # Insert the first (original) request at the start
-            history.insert(0, r)
-            # Get the last request made
             r = history.pop()
-            r.history = history
 
         # If redirects aren't being followed, store the response on the Request for Response.next().
         if not allow_redirects:
@@ -659,25 +670,37 @@ class Session(SessionRedirectMixin):
 
         :rtype: dict
         """
+        # Merge all the kwargs except for proxies.
+        stream = merge_setting(stream, self.stream)
+        verify = merge_setting(verify, self.verify)
+        cert = merge_setting(cert, self.cert)
         # Gather clues from the surrounding environment.
+        # We do this after merging the Session values to make sure we don't
+        # accidentally exclude them.
         if self.trust_env:
-            # Set environment's proxies.
-            no_proxy = proxies.get('no_proxy') if proxies is not None else None
-            env_proxies = get_environ_proxies(url, no_proxy=no_proxy)
-            for (k, v) in env_proxies.items():
-                proxies.setdefault(k, v)
-
             # Look for requests environment configuration and be compatible
             # with cURL.
             if verify is True or verify is None:
                 verify = (os.environ.get('REQUESTS_CA_BUNDLE') or
-                          os.environ.get('CURL_CA_BUNDLE'))
+                          os.environ.get('CURL_CA_BUNDLE') or
+                          verify)
 
-        # Merge all the kwargs.
-        proxies = merge_setting(proxies, self.proxies)
-        stream = merge_setting(stream, self.stream)
-        verify = merge_setting(verify, self.verify)
-        cert = merge_setting(cert, self.cert)
+        # Now we handle proxies.
+        # Proxies need to be built up backwards. This is because None values
+        # can delete proxy information, which can then be re-added by a more
+        # specific layer. So we begin by getting the environment's proxies,
+        # then add the Session, then add the request.
+        no_proxy = proxies.get('no_proxy') if proxies is not None else None
+        if no_proxy is None:
+            no_proxy = self.proxies.get('no_proxy')
+
+        env_proxies = {}
+
+        if self.trust_env:
+            env_proxies = get_environ_proxies(url, no_proxy=no_proxy) or {}
+
+        new_proxies = merge_setting(self.proxies, env_proxies)
+        proxies = merge_setting(proxies, new_proxies)
 
         return {'verify': verify, 'proxies': proxies, 'stream': stream,
                 'cert': cert}
@@ -694,10 +717,10 @@ class Session(SessionRedirectMixin):
                 return adapter
 
         # Nothing matches :-/
-        raise InvalidSchema("No connection adapters were found for '%s'" % url)
+        raise InvalidScheme("No connection adapters were found for '%s'" % url)
 
     def close(self):
-        """Closes all adapters and as such the session"""
+        """Closes all adapters and, as such, the Session."""
         for v in self.adapters.values():
             v.close()
 

requests/utils.py

@@ -27,7 +27,7 @@ from ._internal_utils import to_native_string
 from .compat import parse_http_list as _parse_list_header
 from .compat import (
     quote, urlparse, bytes, str, OrderedDict, unquote, getproxies,
-    proxy_bypass, urlunparse, basestring, integer_types, is_py3,
+    proxy_bypass, urlunparse, basestring, integer_types, is_py2, is_py3,
     proxy_bypass_environment, getproxies_environment)
 from .cookies import cookiejar_from_dict
 from .structures import CaseInsensitiveDict
@@ -431,11 +431,6 @@ def get_encoding_from_headers(headers):
 def stream_decode_response_unicode(iterator, r):
     """Stream decodes a iterator."""
 
-    if r.encoding is None:
-        for item in iterator:
-            yield item
-        return
-
     decoder = codecs.getincrementaldecoder(r.encoding)(errors='replace')
     for chunk in iterator:
         rv = decoder.decode(chunk)
@@ -503,7 +498,26 @@ def unquote_unreserved(uri):
 
     :rtype: str
     """
-    parts = uri.split('%')
+    # This convert function is used to optionally convert the output of `chr`.
+    # In Python 3, `chr` returns a unicode string, while in Python 2 it returns
+    # a bytestring. Here we deal with that by optionally converting.
+    def convert(is_bytes, c):
+        if is_py2 and not is_bytes:
+            return c.decode('ascii')
+        elif is_py3 and is_bytes:
+            return c.encode('ascii')
+        else:
+            return c
+
+    # Handle both bytestrings and unicode strings.
+    is_bytes = isinstance(uri, bytes)
+    splitchar = u'%'
+    base = u''
+    if is_bytes:
+        splitchar = splitchar.encode('ascii')
+        base = base.encode('ascii')
+
+    parts = uri.split(splitchar)
     for i in range(1, len(parts)):
         h = parts[i][0:2]
         if len(h) == 2 and h.isalnum():
@@ -513,12 +527,12 @@ def unquote_unreserved(uri):
                 raise InvalidURL("Invalid percent-escape sequence: '%s'" % h)
 
             if c in UNRESERVED_SET:
-                parts[i] = c + parts[i][2:]
+                parts[i] = convert(is_bytes, c) + parts[i][2:]
             else:
-                parts[i] = '%' + parts[i]
+                parts[i] = splitchar + parts[i]
         else:
-            parts[i] = '%' + parts[i]
-    return ''.join(parts)
+            parts[i] = splitchar + parts[i]
+    return base.join(parts)
 
 
 def requote_uri(uri):
@@ -775,6 +789,19 @@ def parse_header_links(value):
     return links
 
 
+def is_valid_location(response):
+    """Verify that multiple Location headers weren't
+    returned from the last response.
+    """
+    headers = getattr(response.raw, 'headers', None)
+    if headers is not None:
+        getlist = getattr(headers, 'getlist', None)
+        if getlist is not None:
+            return len(getlist('location')) <= 1
+    # If response.raw isn't urllib3-like we can't reliably check this
+    return True
+
+
 # Null bytes; no need to recreate these on each call to guess_json_utf
 _null = '\x00'.encode('ascii')  # encoding to ASCII for Python 3
 _null2 = _null * 2
@@ -902,3 +929,10 @@ def rewind_body(prepared_request):
                                         "body for redirect.")
     else:
         raise UnrewindableBodyError("Unable to rewind request body for redirect.")
+
+
+def is_stream(data):
+    """Given data, determines if it should be sent as a stream."""
+    is_iterable = getattr(data, '__iter__', False)
+    is_io_type = not isinstance(data, (basestring, list, tuple, collections.Mapping))
+    return is_iterable and is_io_type

tests/test_lowlevel.py

@@ -23,6 +23,23 @@ def test_chunked_upload():
     assert r.status_code == 200
     assert r.request.headers['Transfer-Encoding'] == 'chunked'
 
+def test_incorrect_content_length():
+    """Test ConnectionError raised for incomplete responses"""
+    close_server = threading.Event()
+    server = Server.text_response_server(
+            "HTTP/1.1 200 OK\r\n" +
+            "Content-Length: 50\r\n\r\n" +
+            "Hello World."
+        )
+    with server as (host, port):
+        url = 'http://{0}:{1}/'.format(host, port)
+        r = requests.Request('GET', url).prepare()
+        s = requests.Session()
+        with pytest.raises(requests.exceptions.ConnectionError) as e:
+            resp = s.send(r)
+        assert "12 bytes read, 38 more expected" in str(e)
+        close_server.set()  # release server block
+
 
 def test_digestauth_401_count_reset_on_redirect():
     """Ensure we correctly reset num_401_calls after a successful digest auth,

tests/test_requests.py

@@ -14,6 +14,7 @@ import warnings
 import io
 import requests
 import pytest
+import pytest_httpbin
 from requests.adapters import HTTPAdapter
 from requests.auth import HTTPDigestAuth, _basic_auth_str
 from requests.compat import (
@@ -22,9 +23,9 @@ from requests.compat import (
 from requests.cookies import (
     cookiejar_from_dict, morsel_to_cookie)
 from requests.exceptions import (
-    ConnectionError, ConnectTimeout, InvalidSchema, InvalidURL,
-    MissingSchema, ReadTimeout, Timeout, RetryError, TooManyRedirects,
-    ProxyError, InvalidHeader, UnrewindableBodyError)
+    ConnectionError, ConnectTimeout, InvalidScheme, InvalidURL,
+    MissingScheme, ReadTimeout, Timeout, RetryError, TooManyRedirects,
+    ProxyError, InvalidHeader, UnrewindableBodyError, InvalidBodyError)
 from requests.models import PreparedRequest
 from requests.structures import CaseInsensitiveDict
 from requests.sessions import SessionRedirectMixin
@@ -35,6 +36,21 @@ from .compat import StringIO, u
 from .utils import override_environ
 from urllib3.util import Timeout as Urllib3Timeout
 
+class SendRecordingAdapter(HTTPAdapter):
+    """
+    A basic subclass of the HTTPAdapter that records the arguments used to
+    ``send``.
+    """
+    def __init__(self, *args, **kwargs):
+        super(SendRecordingAdapter, self).__init__(*args, **kwargs)
+
+        self.send_calls = []
+
+    def send(self, *args, **kwargs):
+        self.send_calls.append((args, kwargs))
+        return super(SendRecordingAdapter, self).send(*args, **kwargs)
+
+
 # Requests to this URL should always fail with a connection timeout (nothing
 # listening on that port)
 TARPIT = 'http://10.255.255.1'
@@ -70,10 +86,10 @@ class TestRequests:
 
     @pytest.mark.parametrize(
         'exception, url', (
-            (MissingSchema, 'hiwpefhipowhefopw'),
-            (InvalidSchema, 'localhost:3128'),
-            (InvalidSchema, 'localhost.localdomain:3128/'),
-            (InvalidSchema, '10.122.1.1:3128/'),
+            (MissingScheme, 'hiwpefhipowhefopw'),
+            (InvalidScheme, 'localhost:3128'),
+            (InvalidScheme, 'localhost.localdomain:3128/'),
+            (InvalidScheme, '10.122.1.1:3128/'),
             (InvalidURL, 'http://'),
         ))
     def test_invalid_url(self, exception, url):
@@ -81,7 +97,7 @@ class TestRequests:
             requests.get(url)
 
     def test_basic_building(self):
-        req = requests.Request()
+        req = requests.Request(method='GET')
         req.url = 'http://kennethreitz.org/'
         req.data = {'life': '42'}
 
@@ -212,48 +228,106 @@ class TestRequests:
         else:
             pytest.fail('Expected custom max number of redirects to be respected but was not')
 
-    def test_http_301_changes_post_to_get(self, httpbin):
-        r = requests.post(httpbin('status', '301'))
-        assert r.status_code == 200
-        assert r.request.method == 'GET'
+    @pytest.mark.parametrize(
+        'method, body, expected', (
+            ('GET', None, 'GET'),
+            ('HEAD', None, 'HEAD'),
+            ('POST', 'test', 'GET'),
+            ('PUT', 'put test', 'PUT'),
+            ('PATCH', 'patch test', 'PATCH'),
+            ('DELETE', '', 'DELETE')
+        )
+    )
+    def test_http_301_for_redirectable_methods(self, httpbin, method, body, expected):
+        """Tests all methods except OPTIONS for expected redirect behaviour.
+
+        OPTIONS responses can behave differently depending on the server, so
+        we don't have anything uniform to test except how httpbin responds
+        to them. For that reason they aren't included here.
+        """
+        params = {'url': '/%s' % expected.lower(), 'status_code': '301'}
+        r = requests.request(method, httpbin('redirect-to'), data=body, params=params)
+
+        assert r.request.url == httpbin(expected.lower())
+        assert r.request.method == expected
         assert r.history[0].status_code == 301
         assert r.history[0].is_redirect
 
-    def test_http_301_doesnt_change_head_to_get(self, httpbin):
-        r = requests.head(httpbin('status', '301'), allow_redirects=True)
-        print(r.content)
-        assert r.status_code == 200
-        assert r.request.method == 'HEAD'
-        assert r.history[0].status_code == 301
-        assert r.history[0].is_redirect
+        if expected in ('GET', 'HEAD'):
+            assert r.request.body is None
+        else:
+            assert r.json()['data'] == body
 
-    def test_http_302_changes_post_to_get(self, httpbin):
-        r = requests.post(httpbin('status', '302'))
-        assert r.status_code == 200
-        assert r.request.method == 'GET'
+    @pytest.mark.parametrize(
+        'method, body, expected', (
+            ('GET', None, 'GET'),
+            ('HEAD', None, 'HEAD'),
+            ('POST', 'test', 'GET'),
+            ('PUT', 'put test', 'PUT'),
+            ('PATCH', 'patch test', 'PATCH'),
+            ('DELETE', '', 'DELETE')
+        )
+    )
+    def test_http_302_for_redirectable_methods(self, httpbin, method, body, expected):
+        """Tests all methods except OPTIONS for expected redirect behaviour.
+
+        OPTIONS responses can behave differently depending on the server, so
+        we don't have anything uniform to test except how httpbin responds
+        to them. For that reason they aren't included here.
+        """
+        params = {'url': '/%s' % expected.lower()}
+        r = requests.request(method, httpbin('redirect-to'), data=body, params=params)
+
+        assert r.request.url == httpbin(expected.lower())
+        assert r.request.method == expected
         assert r.history[0].status_code == 302
         assert r.history[0].is_redirect
 
-    def test_http_302_doesnt_change_head_to_get(self, httpbin):
-        r = requests.head(httpbin('status', '302'), allow_redirects=True)
-        assert r.status_code == 200
-        assert r.request.method == 'HEAD'
-        assert r.history[0].status_code == 302
-        assert r.history[0].is_redirect
+        if expected in ('GET', 'HEAD'):
+            assert r.request.body is None
+        else:
+            assert r.json()['data'] == body
 
-    def test_http_303_changes_post_to_get(self, httpbin):
-        r = requests.post(httpbin('status', '303'))
-        assert r.status_code == 200
-        assert r.request.method == 'GET'
+    @pytest.mark.parametrize(
+        'method, body, expected', (
+            ('GET', None, 'GET'),
+            ('HEAD', None, 'HEAD'),
+            ('POST', 'test', 'GET'),
+            ('PUT', 'put test', 'GET'),
+            ('PATCH', 'patch test', 'GET'),
+            ('DELETE', '', 'GET')
+        )
+    )
+    def test_http_303_for_redirectable_methods(self, httpbin, method, body, expected):
+        """Tests all methods except OPTIONS for expected redirect behaviour.
+
+        OPTIONS responses can behave differently depending on the server, so
+        we don't have anything uniform to test except how httpbin responds
+        to them. For that reason they aren't included here.
+        """
+        params = {'url': '/%s' % expected.lower(), 'status_code': '303'}
+        r = requests.request(method, httpbin('redirect-to'), data=body, params=params)
+
+        assert r.request.url == httpbin(expected.lower())
+        assert r.request.method == expected
         assert r.history[0].status_code == 303
         assert r.history[0].is_redirect
 
-    def test_http_303_doesnt_change_head_to_get(self, httpbin):
-        r = requests.head(httpbin('status', '303'), allow_redirects=True)
-        assert r.status_code == 200
-        assert r.request.method == 'HEAD'
-        assert r.history[0].status_code == 303
-        assert r.history[0].is_redirect
+        assert r.request.body is None
+
+    def test_multiple_location_headers(self, httpbin):
+        headers = [('Location', 'http://example.com'),
+                   ('Location', 'https://example.com/1')]
+        params = '&'.join(['%s=%s' % (k, v) for k, v in headers])
+        ses = requests.Session()
+        req = requests.Request('GET', httpbin('response-headers?%s' % params))
+        prep = ses.prepare_request(req)
+        resp = ses.send(prep)
+        # change response to redirect
+        resp.status_code = 302
+        with pytest.raises(InvalidHeader):
+            # next triggers yield on generator
+            next(ses.resolve_redirects(resp, prep))
 
     def test_header_and_body_removal_on_redirect(self, httpbin):
         purged_headers = ('Content-Length', 'Content-Type')
@@ -410,6 +484,35 @@ class TestRequests:
         assert cookies['foo'] == 'bar'
         assert cookies['cookie'] == 'tasty'
 
+    @pytest.mark.parametrize(
+        'jar', (
+            requests.cookies.RequestsCookieJar(),
+            cookielib.CookieJar()
+        ))
+    def test_custom_cookie_policy_persistence(self, httpbin, jar):
+        """Verify a custom CookiePolicy is propagated on each session request."""
+
+        class TestCookiePolicy(cookielib.DefaultCookiePolicy):
+            """Policy to restrict all cookies from localhost (127.0.0.1)."""
+            def __init__(self):
+                cookielib.DefaultCookiePolicy.__init__(self, blocked_domains=['127.0.0.1'])
+
+        # Establish session with jar and set some cookies.
+        s = requests.Session()
+        s.cookies = jar
+        s.get(httpbin('cookies/set?k1=v1&k2=v2'))
+        assert len(s.cookies) == 2
+
+        # Set different policy.
+        s.cookies.set_policy(TestCookiePolicy())
+        assert isinstance(s.cookies._policy, TestCookiePolicy)
+
+        # No cookies were sent to our blocked domain and none were set.
+        resp = s.get(httpbin('cookies/set?k3=v3'))
+        assert 'Cookie' not in resp.request.headers
+        assert len(s.cookies) == 2
+        assert 'k3' not in s.cookies
+
     def test_requests_in_history_are_not_overridden(self, httpbin):
         resp = requests.get(httpbin('redirect/3'))
         urls = [r.url for r in resp.history]
@@ -487,8 +590,6 @@ class TestRequests:
         'username, password', (
             ('user', 'pass'),
             (u'имя'.encode('utf-8'), u'пароль'.encode('utf-8')),
-            (42, 42),
-            (None, None),
         ))
     def test_set_basicauth(self, httpbin, username, password):
         auth = (username, password)
@@ -499,6 +600,18 @@ class TestRequests:
 
         assert p.headers['Authorization'] == _basic_auth_str(username, password)
 
+    @pytest.mark.parametrize(
+        'username, password', (
+            ('user', 1234),
+            (None, 'test'),
+        ))
+    def test_non_str_basicauth(self, username, password):
+        """Ensure we only allow string or bytes values for basicauth"""
+        with pytest.raises(TypeError) as e:
+            requests.auth._basic_auth_str(username, password)
+
+        assert 'must be of type str or bytes' in str(e)
+
     def test_basicauth_encodes_byte_strings(self):
         """Ensure b'test' formats as the byte string "test" rather
         than the unicode string "b'test'" in Python 3.
@@ -748,6 +861,10 @@ class TestRequests:
         r = requests.get(httpbin('status', '500'))
         assert not r.ok
 
+    def test_raise_for_status_returns_self(self, httpbin):
+        r = requests.get(httpbin('status', '200'))
+        assert r.raise_for_status() is r
+
     def test_decompress_gzip(self, httpbin):
         r = requests.get(httpbin('gzip'))
         r.content.decode('ascii')
@@ -826,8 +943,8 @@ class TestRequests:
             files={'file': ('test_requests.py', open(__file__, 'rb'))})
         assert r.status_code == 200
 
-    @pytest.mark.parametrize(
-        'data', (
+    @pytest.mark.parametrize('data',
+        (
             {'stuff': u('ëlïxr')},
             {'stuff': u('ëlïxr').encode('utf-8')},
             {'stuff': 'elixr'},
@@ -1161,9 +1278,24 @@ class TestRequests:
         r = requests.Response()
         r.raw = io.BytesIO(b'the content')
         r.encoding = 'ascii'
+
         chunks = r.iter_content(decode_unicode=True)
         assert all(isinstance(chunk, str) for chunk in chunks)
 
+    @pytest.mark.parametrize(
+        'encoding, exception', (
+            (None, TypeError),
+            ('invalid encoding', LookupError),
+        ))
+    def test_decode_unicode_encoding(self, encoding, exception):
+        # raise an exception if encoding isn't set
+        r = requests.Response()
+        r.raw = io.BytesIO(b'the content')
+        r.encoding = encoding
+
+        with pytest.raises(exception):
+            chunks = r.iter_content(decode_unicode=True)
+
     def test_response_reason_unicode(self):
         # check for unicode HTTP status
         r = requests.Response()
@@ -1216,6 +1348,109 @@ class TestRequests:
         assert r.request.url == pr.request.url
         assert r.request.headers == pr.request.headers
 
+
+    def test_response_lines(self):
+        """
+        iter_lines should be able to handle data dribbling in which delimiters
+        might not be lined up ideally.
+        """
+        mock_chunks = [
+            b'This \r\n',
+            b'',
+            b'is\r',
+            b'\n',
+            b'a',
+            b' ',
+            b'',
+            b'',
+            b'test.',
+            b'\r',
+            b'\n',
+            b'end.',
+        ]
+        mock_data = b''.join(mock_chunks)
+        unicode_mock_data = mock_data.decode('utf-8')
+
+        def mock_iter_content(*args, **kwargs):
+            if kwargs.get("decode_unicode"):
+                return (e.decode('utf-8') for e in mock_chunks)
+            return (e for e in mock_chunks)
+
+        r = requests.Response()
+        r._content_consumed = True
+        r.iter_content = mock_iter_content
+
+        # decode_unicode=None, output raw bytes
+        assert list(r.iter_lines(delimiter=b'\r\n')) == mock_data.split(b'\r\n')
+
+        # decode_unicode=True, output unicode strings
+        assert list(r.iter_lines(decode_unicode=True, delimiter=u'\r\n')) == unicode_mock_data.split(u'\r\n')
+
+        # When delimiter is None, we should yield the same result as splitlines()
+        # which supports the universal newline.
+        # '\r', '\n', and '\r\n' are all treated as one line break.
+
+        # decode_unicode=None, output raw bytes
+        result = list(r.iter_lines())
+        assert result == mock_data.splitlines()
+
+        # decode_unicode=True, output unicode strings
+        result = list(r.iter_lines(decode_unicode=True))
+        assert result == unicode_mock_data.splitlines()
+
+        # If we change all the line breaks to `\r`, we should be okay.
+        # decode_unicode=None, output raw bytes
+        mock_chunks = [chunk.replace(b'\n', b'\r') for chunk in mock_chunks]
+        mock_data = b''.join(mock_chunks)
+        assert list(r.iter_lines()) == mock_data.splitlines()
+
+        # decode_unicode=True, output unicode strings
+        unicode_mock_data = mock_data.decode('utf-8')
+        assert list(r.iter_lines(decode_unicode=True)) == unicode_mock_data.splitlines()
+
+
+    @pytest.mark.parametrize(
+        'content, expected_no_delimiter, expected_delimiter', (
+            ([b''], [], []),
+            ([b'line\n'], [u'line'], [u'line\n']),
+            ([b'line', b'\n'], [u'line'], [u'line\n']),
+            ([b'line\r\n'], [u'line'], [u'line', u'']),
+            # Empty chunk in the end of stream, same behavior as the previous
+            ([b'line\r\n', b''], [u'line'], [u'line', u'']),
+            ([b'line', b'\r\n'], [u'line'], [u'line', u'']),
+            ([b'a\r', b'\nb\r'], [u'a', u'b'], [u'a', u'b\r']),
+            ([b'a\r', b'\n', b'\nb'], [u'a', u'', u'b'], [u'a', u'\nb']),
+            ([b'a\n', b'\nb'], [u'a', u'', u'b'], [u'a\n\nb']),
+            ([b'a\r\n', b'\rb\n'], [u'a', u'', u'b'], [u'a', u'\rb\n']),
+            ([b'a\nb', b'c'], [u'a', u'bc'], [u'a\nbc']),
+            ([b'a\n', b'\rb', b'\r\nc'], [u'a', u'', u'b', u'c'], [u'a\n\rb', u'c']),
+            ([b'a\r\nb', b'', b'c'], [u'a', u'bc'], [u'a', u'bc'])  # Empty chunk with pending data
+        ))
+    def test_response_lines_parametrized(self, content, expected_no_delimiter, expected_delimiter):
+        """
+        Test a lot of potential chunk splits to ensure consistency of
+        iter_lines(delimiter=x), as well as the legacy behavior of
+        iter_lines() without delimiter
+        https://github.com/kennethreitz/requests/pull/2431#issuecomment-72333964
+        """
+        mock_chunks = content
+        def mock_iter_content(*args, **kwargs):
+            if kwargs.get("decode_unicode"):
+                return (e.decode('utf-8') for e in mock_chunks)
+            return (e for e in mock_chunks)
+
+        r = requests.Response()
+        r._content_consumed = True
+        r.iter_content = mock_iter_content
+
+        # decode_unicode=True, output unicode strings
+        assert list(r.iter_lines(decode_unicode=True)) == expected_no_delimiter
+        assert list(r.iter_lines(decode_unicode=True, delimiter='\r\n')) == expected_delimiter
+
+        # decode_unicode=None, output raw bytes
+        assert list(r.iter_lines()) == [line.encode('utf-8') for line in expected_no_delimiter]
+        assert list(r.iter_lines(delimiter=b'\r\n')) == [line.encode('utf-8') for line in expected_delimiter]
+
     def test_prepared_request_is_pickleable(self, httpbin):
         p = requests.Request('GET', httpbin('get')).prepare()
 
@@ -1488,9 +1723,10 @@ class TestRequests:
 
     def test_manual_redirect_with_partial_body_read(self, httpbin):
         s = requests.Session()
-        r1 = s.get(httpbin('redirect/2'), allow_redirects=False, stream=True)
+        req = requests.Request('GET', httpbin('redirect/2')).prepare()
+        r1 = s.send(req, allow_redirects=False, stream=True)
         assert r1.is_redirect
-        rg = s.resolve_redirects(r1, r1.request, stream=True)
+        rg = s.resolve_redirects(r1, req, stream=True)
 
         # read only the first eight bytes of the response body,
         # then follow the redirect
@@ -1660,11 +1896,12 @@ class TestRequests:
         prep = r.prepare()
         assert 'stuff=elixr' == prep.body
 
-    def test_response_iter_lines(self, httpbin):
+    @pytest.mark.parametrize('decode_unicode', (True, False))
+    def test_response_iter_lines(self, httpbin, decode_unicode):
         r = requests.get(httpbin('stream/4'), stream=True)
         assert r.status_code == 200
-
-        it = r.iter_lines()
+        r.encoding = 'utf-8'
+        it = r.iter_lines(decode_unicode=decode_unicode)
         next(it)
         assert len(list(it)) == 3
 
@@ -1686,6 +1923,59 @@ class TestRequests:
         next(r.iter_lines())
         assert len(list(r.iter_lines())) == 3
 
+    def test_environment_comes_after_session(self, httpbin):
+        """The Session arguments should come before environment arguments."""
+        # We get proxies from the environment and verify from the argument.
+        s = requests.Session()
+        a = SendRecordingAdapter()
+        s.mount('http://', a)
+
+        # Both of these arguments are safe fallbacks that we can easily
+        # detect, but which will allow the request to succeed.
+        s.verify = False
+        s.proxies = {'http': None}
+
+        old_proxy = os.environ.get('HTTP_PROXY')
+        old_bundle = os.environ.get('REQUESTS_CA_BUNDLE')
+
+        try:
+            os.environ['HTTP_PROXY'] = '10.10.10.10:3128'
+            os.environ['REQUESTS_CA_BUNDLE'] = '/path/to/nowhere'
+
+            s.get(httpbin('get'), timeout=5)
+        finally:
+            if old_proxy is not None:
+                os.environ['HTTP_PROXY'] = old_proxy
+            else:
+                del os.environ['HTTP_PROXY']
+
+            if old_bundle is not None:
+                os.environ['REQUESTS_CA_BUNDLE'] = old_bundle
+            else:
+                del os.environ['REQUESTS_CA_BUNDLE']
+
+        call = a.send_calls[0]
+        assert call[1]['verify'] == False
+
+        proxies = call[1]['proxies']
+        with pytest.raises(KeyError):
+            proxies['http']
+
+    @pytest.fixture(autouse=True)
+    def test_merge_environment_settings_verify(self, monkeypatch):
+        """Assert CA environment settings are merged as expected when missing"""
+        session = requests.Session()
+        monkeypatch.delenv('CURL_CA_BUNDLE', raising=False)
+        monkeypatch.delenv('REQUESTS_CA_BUNDLE', raising=False)
+
+        assert session.trust_env is True
+        assert session.verify is True
+        assert 'REQUESTS_CA_BUNDLE' not in os.environ
+        assert 'CURL_CA_BUNDLE' not in os.environ
+        merged_settings = session.merge_environment_settings(
+            'http://example.com', {}, False, True, None)
+        assert merged_settings['verify'] is True
+
     def test_session_close_proxy_clear(self, mocker):
         proxies = {
           'one': mocker.Mock(),
@@ -1728,6 +2018,33 @@ class TestRequests:
         resp.close()
         assert resp.raw.closed
 
+    def test_updating_ca_cert(self, httpbin_secure):
+        """Assert that requests use the latest configured CA certificates."""
+        session = requests.session()
+        session.verify = pytest_httpbin.certs.where()
+        session.get(httpbin_secure('/'))
+        session.verify = True
+        with pytest.raises(requests.exceptions.SSLError) as e:
+            session.get(httpbin_secure('/'))
+        assert 'certificate verify failed' in str(e)
+
+    def test_updating_client_cert(self, httpbin_secure):
+        """Assert that requests use the latest configured client certificates."""
+        ca_file = pytest_httpbin.certs.where()
+        cert_dir = os.path.dirname(ca_file)
+        # All we need is a valid certificate and key to make a request. httpbin_secure
+        # won't check the signature or subject name, so it's okay that these happen to
+        # be the server's certificate and key.
+        cert = os.path.join(cert_dir, 'cert.pem')
+        key = os.path.join(cert_dir, 'key.pem')
+        session = requests.session()
+        session.verify = ca_file
+        resp = session.get(httpbin_secure('/'))
+        resp_with_cert = session.get(httpbin_secure('/'), cert=(cert, key))
+        assert resp_with_cert.raw._pool.cert_file == cert
+        assert resp_with_cert.raw._pool.key_file == key
+        assert resp.raw._pool is not resp_with_cert.raw._pool
+
     def test_empty_stream_with_auth_does_not_set_content_length_header(self, httpbin):
         """Ensure that a byte stream with size 0 will not set both a Content-Length
         and Transfer-Encoding header.
@@ -1763,6 +2080,61 @@ class TestRequests:
         assert 'Transfer-Encoding' in prepared_request.headers
         assert 'Content-Length' not in prepared_request.headers
 
+    def test_chunked_upload_with_manually_set_content_length_header_raises_error(self, httpbin):
+        """Ensure that if a user manually sets a content length header, when
+        the data is chunked, that an InvalidHeader error is raised.
+        """
+        data = (i for i in [b'a', b'b', b'c']) 
+        url = httpbin('post')
+        with pytest.raises(InvalidHeader):
+            r = requests.post(url, data=data, headers={'Content-Length': 'foo'})
+
+    def test_content_length_with_manually_set_transfer_encoding_raises_error(self, httpbin):
+        """Ensure that if a user manually sets a Transfer-Encoding header when
+        data is not chunked that an InvalidHeader error is raised.
+        """
+        data = 'test data'
+        url = httpbin('post')
+        with pytest.raises(InvalidHeader):
+            r = requests.post(url, data=data, headers={'Transfer-Encoding': 'chunked'})
+
+    def test_null_body_does_not_raise_error(self, httpbin):
+        url = httpbin('post')
+        try:
+            requests.post(url, data=None)
+        except InvalidHeader:
+            pytest.fail('InvalidHeader error raised unexpectedly.')
+
+    @pytest.mark.parametrize(
+        'body, expected', (
+            (None, ('Content-Length', '0')),
+            ('test_data', ('Content-Length', '9')),
+            (io.BytesIO(b'test_data'), ('Content-Length', '9')),
+            (StringIO.StringIO(''), ('Transfer-Encoding', 'chunked'))
+        ))
+    def test_prepare_content_length(self, httpbin, body, expected):
+        """Test prepare_content_length creates expected header."""
+        prep = requests.PreparedRequest()
+        prep.headers = {}
+        prep.method = 'POST'
+
+        # Ensure Content-Length is set appropriately.
+        key, value = expected
+        prep.prepare_content_length(body)
+        assert prep.headers[key] == value
+
+    def test_prepare_content_length_with_bad_body(self, httpbin):
+        """Test prepare_content_length raises exception with unsendable body."""
+        # Initialize minimum required PreparedRequest.
+        prep = requests.PreparedRequest()
+        prep.headers = {}
+        prep.method = 'POST'
+
+        with pytest.raises(InvalidBodyError) as e:
+            # Send object that isn't iterable and has no accessible content.
+            prep.prepare_content_length(object())
+            assert "Non-null body must have length or be streamable." in str(e)
+
     def test_custom_redirect_mixin(self, httpbin):
         """Tests a custom mixin to overwrite ``get_redirect_target``.
 
@@ -1948,6 +2320,18 @@ class TestCaseInsensitiveDict:
         cid['changed'] = True
         assert cid != cid_copy
 
+    def test_url_surrounding_whitespace(self, httpbin):
+        """Test case with URLs surrounded by whitespace characters."""
+        get_url = httpbin('get')
+        # All surrounding whitespaces are supposed to be ignored:
+        assert requests.get(get_url + ' ').status_code == 200
+        assert requests.get(' ' + get_url).status_code == 200
+        assert requests.get(get_url + ' \t ').status_code == 200
+        assert requests.get('  \t' + get_url).status_code == 200
+        assert requests.get(get_url + '\n').status_code == 200
+        # The whitespaces can't be in the middle of the URL though:
+        assert requests.get(get_url + ' abc').status_code == 404
+
 
 class TestMorselToCookieExpires:
     """Tests for morsel_to_cookie when morsel contains expires."""
@@ -2090,6 +2474,7 @@ class RedirectSession(SessionRedirectMixin):
         self.max_redirects = 30
         self.cookies = {}
         self.trust_env = False
+        self.location = '/'
 
     def send(self, *args, **kwargs):
         self.calls.append(SendCall(args, kwargs))
@@ -2104,7 +2489,7 @@ class RedirectSession(SessionRedirectMixin):
         except IndexError:
             r.status_code = 200
 
-        r.headers = CaseInsensitiveDict({'Location': '/'})
+        r.headers = CaseInsensitiveDict({'Location': self.location})
         r.raw = self._build_raw()
         r.request = request
         return r
@@ -2219,6 +2604,16 @@ def test_prepared_copy(kwargs):
         assert getattr(p, attr) == getattr(copy, attr)
 
 
+def test_prepare_requires_a_request_method():
+    req = requests.Request()
+    with pytest.raises(ValueError):
+        req.prepare()
+
+    prepped = PreparedRequest()
+    with pytest.raises(ValueError):
+        prepped.prepare()
+
+
 def test_urllib3_retries(httpbin):
     from urllib3.util import Retry
     s = requests.Session()