v2.12.5

Prepare changelog for 2.12.5

.coveragerc

@@ -0,0 +1,2 @@
+[run]
+omit = requests/packages/*

.travis.yml

@@ -0,0 +1,18 @@
+language: python
+python:
+  - "2.6"
+  - "2.7"
+  - "3.3"
+  - "3.4"
+  - "3.5"
+  - "3.6"
+  - "3.7-dev"
+  # - "pypy"  -- appears to hang
+  # - "pypy3"
+# command to install dependencies
+install: "pip install -r requirements.txt"
+# command to run tests
+script:
+  - make coverage
+after_success:
+  - codecov

AUTHORS.rst

@@ -173,3 +173,5 @@ Patches and Suggestions
 - Om Prakash Kumar <omprakash070@gmail.com> (`@iamprakashom <https://github.com/iamprakashom>`_)
 - Philipp Konrad <gardiac2002@gmail.com> (`@gardiac2002 <https://github.com/gardiac2002>`_)
 - Hussain Tamboli <hussaintamboli18@gmail.com> (`@hussaintamboli <https://github.com/hussaintamboli>`_)
+- Casey Davidson (`@davidsoncasey <https://github.com/davidsoncasey>`_)
+- Andrii Soldatenko (`@a_soldatenko <https://github.com/andriisoldatenko>`_)

HISTORY.rst

@@ -3,6 +3,35 @@
 Release History
 ---------------
 
+2.12.5 (2017-01-18)
++++++++++++++++++++
+
+**Bugfixes**
+
+- Fixed an issue with JSON encoding detection, specifically detecting
+  big-endian UTF-32 with BOM.
+
+2.12.4 (2016-12-14)
++++++++++++++++++++
+
+**Bugfixes**
+
+- Fixed regression from 2.12.2 where non-string types were rejected in the
+  basic auth parameters. While support for this behaviour has been readded,
+  the behaviour is deprecated and will be removed in the future.
+
+2.12.3 (2016-12-01)
++++++++++++++++++++
+
+**Bugfixes**
+
+- Fixed regression from v2.12.1 for URLs with schemes that begin with "http".
+  These URLs have historically been processed as though they were HTTP-schemed
+  URLs, and so have had parameters added. This was removed in v2.12.2 in an
+  overzealous attempt to resolve problems with IDNA-encoding those URLs. This
+  change was reverted: the other fixes for IDNA-encoding have been judged to
+  be sufficient to return to the behaviour Requests had before v2.12.0.
+
 2.12.2 (2016-11-30)
 +++++++++++++++++++
 

Makefile

@@ -9,10 +9,7 @@ test:
 	py.test tests
 
 coverage:
-	py.test --verbose --cov-report term --cov=requests tests
-
-ci: init
-	py.test --junitxml=junit.xml
+	py.test --cov-config .coveragerc --verbose --cov-report term --cov-report xml --cov=requests tests
 
 certs:
 	curl http://ci.kennethreitz.org/job/ca-bundle/lastSuccessfulBuild/artifact/cacerts.pem -o requests/cacert.pem
@@ -44,7 +41,6 @@ idna:
 	    git checkout `git describe --abbrev=0 --tags` && \
 	    cd .. && \
 	    mv idna/idna requests/packages/ && \
-	    find requests/packages/idna -type f -exec sed -i "" "s/^from idna/from /" {} \; && \
 	    rm -fr idna
 
 publish:

README.rst

@@ -3,7 +3,19 @@ Requests: HTTP for Humans
 
 .. image:: https://img.shields.io/pypi/v/requests.svg
     :target: https://pypi.python.org/pypi/requests
+
+.. image:: https://travis-ci.org/kennethreitz/requests.svg?branch=master
+    :target: https://travis-ci.org/kennethreitz/requests
+
+.. image:: https://codecov.io/github/kennethreitz/requests/coverage.svg?branch=master
+    :target: https://codecov.io/github/kennethreitz/requests
+    :alt: codecov.io
     
+.. image:: https://img.shields.io/badge/SayThanks.io-☼-1EAEDB.svg
+    :target: https://saythanks.io/to/kennethreitz
+
+
+
 Requests is the only *Non-GMO* HTTP library for Python, safe for human
 consumption.
 
@@ -65,7 +77,7 @@ Requests is ready for today's web.
 - Chunked Requests
 - Thread-safety
 
-Requests officially supports Python 2.6–2.7 & 3.3–3.5, and runs great on PyPy.
+Requests officially supports Python 2.6–2.7 & 3.3–3.7, and runs great on PyPy.
 
 Installation
 ------------

docs/community/faq.rst

@@ -59,6 +59,7 @@ supported:
 * Python 3.3
 * Python 3.4
 * Python 3.5
+* Python 3.6
 * PyPy
 
 What are "hostname doesn't match" errors?

docs/dev/todo.rst

@@ -41,6 +41,7 @@ Requests currently supports the following versions of Python:
 - Python 3.3
 - Python 3.4
 - Python 3.5
+- Python 3.6
 - PyPy
 
 Google AppEngine is not officially supported although support is available

docs/index.rst

@@ -43,7 +43,7 @@ User Testimonials
 
 The NSA, Her Majesty's Government, Amazon, Google, Twilio, Runscope, Mozilla, Heroku,
 PayPal, NPR, Obama for America, Transifex, Native Instruments, The Washington
-Post, Twitter, SoundCloud, Kippt, Readability, Sony, and Federal U.S.
+Post, Twitter, SoundCloud, Kippt, Sony, and Federal U.S.
 Institutions that prefer to be unnamed claim to use Requests internally.
 
 **Armin Ronacher**
@@ -56,7 +56,7 @@ Institutions that prefer to be unnamed claim to use Requests internally.
 
 **Daniel Greenfeld**
     Nuked a 1200 LOC spaghetti code library with 10 lines of code thanks to
-    @kennethreitz's request library. Today has been AWESOME.
+    `@kennethreitz <https://twitter.com/kennethreitz>`_'s request library. Today has been AWESOME.
 
 **Kenny Meyers**
     Python HTTP: When in doubt, or when not in doubt, use Requests. Beautiful,
@@ -87,7 +87,7 @@ Requests is ready for today's web.
 - Chunked Requests
 - Thread-safety
 
-Requests officially supports Python 2.6–2.7 & 3.3–3.5, and runs great on PyPy.
+Requests officially supports Python 2.6–2.7 & 3.3–3.7, and runs great on PyPy.
 
 
 The User Guide

docs/user/advanced.rst

@@ -220,7 +220,7 @@ This list of trusted CAs can also be specified through the ``REQUESTS_CA_BUNDLE`
 
 Requests can also ignore verifying the SSL certificate if you set ``verify`` to False::
 
-    >>> requests.get('https://kennethreitz.com', verify=False)
+    >>> requests.get('https://kennethreitz.org', verify=False)
     <Response [200]>
 
 By default, ``verify`` is set to True. Option ``verify`` only applies to host certs.
@@ -229,7 +229,7 @@ You can also specify a local cert to use as client side certificate, as a single
 file (containing the private key and the certificate) or as a tuple of both
 file's path::
 
-    >>> requests.get('https://kennethreitz.com', cert=('/path/client.cert', '/path/client.key'))
+    >>> requests.get('https://kennethreitz.org', cert=('/path/client.cert', '/path/client.key'))
     <Response [200]>
 
 or persistent::
@@ -239,7 +239,7 @@ or persistent::
 
 If you specify a wrong path or an invalid cert, you'll get a SSLError::
 
-    >>> requests.get('https://kennethreitz.com', cert='/wrong_path/client.pem')
+    >>> requests.get('https://kennethreitz.org', cert='/wrong_path/client.pem')
     SSLError: [Errno 336265225] _ssl.c:347: error:140B0009:SSL routines:SSL_CTX_use_PrivateKey_file:PEM lib
 
 .. warning:: The private key to your local certificate *must* be unencrypted.

docs/user/authentication.rst

@@ -60,7 +60,7 @@ OAuth 1 Authentication
 ----------------------
 
 A common form of authentication for several web APIs is OAuth. The ``requests-oauthlib``
-library allows Requests users to easily make OAuth authenticated requests::
+library allows Requests users to easily make OAuth 1 authenticated requests::
 
     >>> import requests
     >>> from requests_oauthlib import OAuth1
@@ -76,6 +76,17 @@ For more information on how to OAuth flow works, please see the official `OAuth`
 For examples and documentation on requests-oauthlib, please see the `requests_oauthlib`_
 repository on GitHub
 
+OAuth 2 and OpenID Connect Authentication
+-----------------------------------------
+
+The ``requests-oauthlib`` library also handles OAuth 2, the authentication mechanism
+underpinning OpenID Connect. See the `requests-oauthlib OAuth2 documentation`_ for
+details of the various OAuth 2 credential management flows:
+
+* `Web Application Flow`_
+* `Mobile Application Flow`_
+* `Legacy Application Flow`_
+* `Backend Application Flow`_
 
 Other Authentication
 --------------------
@@ -123,6 +134,11 @@ Further examples can be found under the `Requests organization`_ and in the
 
 .. _OAuth: http://oauth.net/
 .. _requests_oauthlib: https://github.com/requests/requests-oauthlib
+.. _requests-oauthlib OAuth2 documentation: http://requests-oauthlib.readthedocs.io/en/latest/oauth2_workflow.html
+.. _Web Application Flow: http://requests-oauthlib.readthedocs.io/en/latest/oauth2_workflow.html#web-application-flow
+.. _Mobile Application Flow: http://requests-oauthlib.readthedocs.io/en/latest/oauth2_workflow.html#mobile-application-flow
+.. _Legacy Application Flow:  http://requests-oauthlib.readthedocs.io/en/latest/oauth2_workflow.html#legacy-application-flow
+.. _Backend Application Flow:  http://requests-oauthlib.readthedocs.io/en/latest/oauth2_workflow.html#backend-application-flow
 .. _Kerberos: https://github.com/requests/requests-kerberos
 .. _NTLM: https://github.com/requests/requests-ntlm
 .. _Requests organization: https://github.com/requests

requests/__init__.py

@@ -41,8 +41,8 @@ is at <http://python-requests.org>.
 """
 
 __title__ = 'requests'
-__version__ = '2.12.2'
-__build__ = 0x021202
+__version__ = '2.12.5'
+__build__ = 0x021205
 __author__ = 'Kenneth Reitz'
 __license__ = 'Apache 2.0'
 __copyright__ = 'Copyright 2016 Kenneth Reitz'

requests/api.py

@@ -33,7 +33,7 @@ def request(method, url, **kwargs):
         before giving up, as a float, or a :ref:`(connect timeout, read
         timeout) <timeouts>` tuple.
     :type timeout: float or tuple
-    :param allow_redirects: (optional) Boolean. Enable/disable GET/OPTIONS/POST/PUT/PATCH/DELETE/HEAD redirection.
+    :param allow_redirects: (optional) Boolean. Enable/disable GET/OPTIONS/POST/PUT/PATCH/DELETE/HEAD redirection. Defaults to ``True``.
     :type allow_redirects: bool
     :param proxies: (optional) Dictionary mapping protocol to the URL of the proxy.
     :param verify: (optional) whether the SSL cert will be verified. A CA_BUNDLE path can also be provided. Defaults to ``True``.

requests/auth.py

@@ -12,10 +12,11 @@ import re
 import time
 import hashlib
 import threading
+import warnings
 
 from base64 import b64encode
 
-from .compat import urlparse, str
+from .compat import urlparse, str, basestring
 from .cookies import extract_cookies_to_jar
 from ._internal_utils import to_native_string
 from .utils import parse_dict_header
@@ -27,7 +28,35 @@ CONTENT_TYPE_MULTI_PART = 'multipart/form-data'
 
 def _basic_auth_str(username, password):
     """Returns a Basic Auth string."""
-    
+
+    # "I want us to put a big-ol' comment on top of it that
+    # says that this behaviour is dumb but we need to preserve
+    # it because people are relying on it."
+    #    - Lukasa
+    #
+    # These are here solely to maintain backwards compatibility
+    # for things like ints. This will be removed in 3.0.0.
+    if not isinstance(username, basestring):
+        warnings.warn(
+            "Non-string usernames will no longer be supported in Requests "
+            "3.0.0. Please convert the object you've passed in ({0!r}) to "
+            "a string or bytes object in the near future to avoid "
+            "problems.".format(username),
+            category=DeprecationWarning,
+        )
+        username = str(username)
+
+    if not isinstance(password, basestring):
+        warnings.warn(
+            "Non-string passwords will no longer be supported in Requests "
+            "3.0.0. Please convert the object you've passed in ({0!r}) to "
+            "a string or bytes object in the near future to avoid "
+            "problems.".format(password),
+            category=DeprecationWarning,
+        )
+        password = str(password)
+    # -- End Removal --
+
     if isinstance(username, str):
         username = username.encode('latin1')
 

requests/models.py

@@ -347,9 +347,9 @@ class PreparedRequest(RequestEncodingMixin, RequestHooksMixin):
         url = url.lstrip()
 
         # Don't do any URL preparation for non-HTTP schemes like `mailto`,
-        # `data`, `http+unix` etc to work around exceptions from `url_parse`,
-        # which handles RFC 3986 only.
-        if ':' in url and not url.lower().startswith(('http://', 'https://')):
+        # `data` etc to work around exceptions from `url_parse`, which
+        # handles RFC 3986 only.
+        if ':' in url and not url.lower().startswith('http'):
             self.url = url
             return
 

requests/packages/urllib3/contrib/appengine.py

@@ -111,7 +111,7 @@ class AppEngineManager(RequestMethods):
         warnings.warn(
             "urllib3 is using URLFetch on Google App Engine sandbox instead "
             "of sockets. To use sockets directly instead of URLFetch see "
-            "https://urllib3.readthedocs.io/en/latest/contrib.html.",
+            "https://urllib3.readthedocs.io/en/latest/reference/urllib3.contrib.html.",
             AppEnginePlatformWarning)
 
         RequestMethods.__init__(self, headers)

requests/utils.py

@@ -714,7 +714,7 @@ def guess_json_utf(data):
     # easy as counting the nulls and from their location and count
     # determine the encoding. Also detect a BOM, if present.
     sample = data[:4]
-    if sample in (codecs.BOM_UTF32_LE, codecs.BOM32_BE):
+    if sample in (codecs.BOM_UTF32_LE, codecs.BOM_UTF32_BE):
         return 'utf-32'     # BOM included
     if sample[:3] == codecs.BOM_UTF8:
         return 'utf-8-sig'  # BOM included, MS style (discouraged)

requirements-to-freeze.txt

@@ -1,4 +1,7 @@
 pytest
 pytest-cov
 pytest-httpbin
-sphinx
+pytest-mock
+PySocks
+sphinx
+codecov

requirements.txt

@@ -1,24 +1,29 @@
-alabaster==0.7.7
-Babel==2.2.0
-coverage==4.0.3
-decorator==4.0.9
-docutils==0.12
-Flask==0.10.1
+alabaster==0.7.9
+Babel==2.3.4
+click==6.7
+codecov==2.0.5
+coverage==4.3.4
+decorator==4.0.11
+docutils==0.13.1
+Flask==0.12
+funcsigs==1.0.2
 httpbin==0.5.0
+imagesize==0.7.1
 itsdangerous==0.24
-Jinja2==2.8
+Jinja2==2.9.4
 MarkupSafe==0.23
-py==1.4.31
-Pygments==2.1.1
-PySocks==1.5.6
-pytest==2.8.7
-pytest-cov==2.2.1
-pytest-httpbin==0.2.0
-pytest-mock==0.11.0
-pytz==2015.7
+mock==2.0.0
+pbr==1.10.0
+py==1.4.32
+Pygments==2.1.3
+PySocks==1.6.5
+pytest==3.0.5
+pytest-cov==2.4.0
+pytest-httpbin==0.2.3
+pytest-mock==1.5.0
+pytz==2016.10
+requests==2.12.4
 six==1.10.0
 snowballstemmer==1.2.1
-Sphinx==1.3.5
-sphinx-rtd-theme==0.1.9
-Werkzeug==0.11.4
-wheel==0.29.0
+Sphinx==1.5.1
+Werkzeug==0.11.15

setup.py

@@ -47,7 +47,7 @@ packages = [
 ]
 
 requires = []
-test_requirements = ['pytest>=2.8.0', 'pytest-httpbin==0.0.7', 'pytest-cov']
+test_requirements = ['pytest>=2.8.0', 'pytest-httpbin==0.0.7', 'pytest-cov', 'pytest-mock']
 
 with open('requests/__init__.py', 'r') as fd:
     version = re.search(r'^__version__\s*=\s*[\'"]([^\'"]*)[\'"]',
@@ -88,6 +88,7 @@ setup(
         'Programming Language :: Python :: 3.3',
         'Programming Language :: Python :: 3.4',
         'Programming Language :: Python :: 3.5',
+        'Programming Language :: Python :: 3.6',
         'Programming Language :: Python :: Implementation :: CPython',
         'Programming Language :: Python :: Implementation :: PyPy'
     ),

tests/test_lowlevel.py

@@ -5,7 +5,7 @@ import pytest
 import threading
 import requests
 
-from tests.testserver.server import Server
+from tests.testserver.server import Server, consume_socket_content
 
 from .utils import override_environ
 
@@ -25,6 +25,117 @@ def test_chunked_upload():
     assert r.request.headers['Transfer-Encoding'] == 'chunked'
 
 
+def test_digestauth_401_count_reset_on_redirect():
+    """Ensure we correctly reset num_401_calls after a successful digest auth,
+    followed by a 302 redirect to another digest auth prompt.
+
+    See https://github.com/kennethreitz/requests/issues/1979.
+    """
+    text_401 = (b'HTTP/1.1 401 UNAUTHORIZED\r\n'
+                b'Content-Length: 0\r\n'
+                b'WWW-Authenticate: Digest nonce="6bf5d6e4da1ce66918800195d6b9130d"'
+                b', opaque="372825293d1c26955496c80ed6426e9e", '
+                b'realm="me@kennethreitz.com", qop=auth\r\n\r\n')
+
+    text_302 = (b'HTTP/1.1 302 FOUND\r\n'
+                b'Content-Length: 0\r\n'
+                b'Location: /\r\n\r\n')
+
+    text_200 = (b'HTTP/1.1 200 OK\r\n'
+                b'Content-Length: 0\r\n\r\n')
+
+    expected_digest = (b'Authorization: Digest username="user", '
+                       b'realm="me@kennethreitz.com", '
+                       b'nonce="6bf5d6e4da1ce66918800195d6b9130d", uri="/"')
+
+    auth = requests.auth.HTTPDigestAuth('user', 'pass')
+
+    def digest_response_handler(sock):
+        # Respond to initial GET with a challenge.
+        request_content = consume_socket_content(sock, timeout=0.5)
+        assert request_content.startswith(b"GET / HTTP/1.1")
+        sock.send(text_401)
+
+        # Verify we receive an Authorization header in response, then redirect.
+        request_content = consume_socket_content(sock, timeout=0.5)
+        assert expected_digest in request_content
+        sock.send(text_302)
+
+        # Verify Authorization isn't sent to the redirected host,
+        # then send another challenge.
+        request_content = consume_socket_content(sock, timeout=0.5)
+        assert b'Authorization:' not in request_content
+        sock.send(text_401)
+
+        # Verify Authorization is sent correctly again, and return 200 OK.
+        request_content = consume_socket_content(sock, timeout=0.5)
+        assert expected_digest in request_content
+        sock.send(text_200)
+
+        return request_content
+
+    close_server = threading.Event()
+    server = Server(digest_response_handler, wait_to_close_event=close_server)
+
+    with server as (host, port):
+        url = 'http://{0}:{1}/'.format(host, port)
+        r = requests.get(url, auth=auth)
+        # Verify server succeeded in authenticating.
+        assert r.status_code == 200
+        # Verify Authorization was sent in final request.
+        assert 'Authorization' in r.request.headers
+        assert r.request.headers['Authorization'].startswith('Digest ')
+        # Verify redirect happened as we expected.
+        assert r.history[0].status_code == 302
+        close_server.set()
+
+
+def test_digestauth_401_only_sent_once():
+    """Ensure we correctly respond to a 401 challenge once, and then
+    stop responding if challenged again.
+    """
+    text_401 = (b'HTTP/1.1 401 UNAUTHORIZED\r\n'
+                b'Content-Length: 0\r\n'
+                b'WWW-Authenticate: Digest nonce="6bf5d6e4da1ce66918800195d6b9130d"'
+                b', opaque="372825293d1c26955496c80ed6426e9e", '
+                b'realm="me@kennethreitz.com", qop=auth\r\n\r\n')
+
+    expected_digest = (b'Authorization: Digest username="user", '
+                       b'realm="me@kennethreitz.com", '
+                       b'nonce="6bf5d6e4da1ce66918800195d6b9130d", uri="/"')
+
+    auth = requests.auth.HTTPDigestAuth('user', 'pass')
+
+    def digest_failed_response_handler(sock):
+        # Respond to initial GET with a challenge.
+        request_content = consume_socket_content(sock, timeout=0.5)
+        assert request_content.startswith(b"GET / HTTP/1.1")
+        sock.send(text_401)
+
+        # Verify we receive an Authorization header in response, then
+        # challenge again.
+        request_content = consume_socket_content(sock, timeout=0.5)
+        assert expected_digest in request_content
+        sock.send(text_401)
+
+        # Verify the client didn't respond to second challenge.
+        request_content = consume_socket_content(sock, timeout=0.5)
+        assert request_content == b''
+
+        return request_content
+
+    close_server = threading.Event()
+    server = Server(digest_failed_response_handler, wait_to_close_event=close_server)
+
+    with server as (host, port):
+        url = 'http://{0}:{1}/'.format(host, port)
+        r = requests.get(url, auth=auth)
+        # Verify server didn't authenticate us.
+        assert r.status_code == 401
+        assert r.history[0].status_code == 401
+        close_server.set()
+
+
 _schemes_by_var_prefix = [
     ('http', ['http']),
     ('https', ['https']),

tests/test_requests.py

@@ -20,7 +20,7 @@ from requests.compat import (
     Morsel, cookielib, getproxies, str, urlparse,
     builtin_str, OrderedDict)
 from requests.cookies import (
-    cookiejar_from_dict, morsel_to_cookie, merge_cookies)
+    cookiejar_from_dict, morsel_to_cookie)
 from requests.exceptions import (
     ConnectionError, ConnectTimeout, InvalidSchema, InvalidURL,
     MissingSchema, ReadTimeout, Timeout, RetryError, TooManyRedirects,
@@ -484,6 +484,8 @@ class TestRequests:
         'username, password', (
             ('user', 'pass'),
             (u'имя'.encode('utf-8'), u'пароль'.encode('utf-8')),
+            (42, 42),
+            (None, None),
         ))
     def test_set_basicauth(self, httpbin, username, password):
         auth = (username, password)
@@ -494,6 +496,16 @@ class TestRequests:
 
         assert p.headers['Authorization'] == _basic_auth_str(username, password)
 
+    def test_basicauth_encodes_byte_strings(self):
+        """Ensure b'test' formats as the byte string "test" rather
+        than the unicode string "b'test'" in Python 3.
+        """
+        auth = (b'\xc5\xafsername', b'test\xc6\xb6')
+        r = requests.Request('GET', 'http://localhost', auth=auth)
+        p = r.prepare()
+
+        assert p.headers['Authorization'] == 'Basic xa9zZXJuYW1lOnRlc3TGtg=='
+
     @pytest.mark.parametrize(
         'url, exception', (
             # Connecting to an unknown domain should raise a ConnectionError
@@ -1693,6 +1705,42 @@ class TestRequests:
         resp.close()
         assert resp.raw.closed
 
+    def test_empty_stream_with_auth_does_not_set_content_length_header(self, httpbin):
+        """Ensure that a byte stream with size 0 will not set both a Content-Length
+        and Transfer-Encoding header.
+        """
+        auth = ('user', 'pass')
+        url = httpbin('post')
+        file_obj = io.BytesIO(b'')
+        r = requests.Request('POST', url, auth=auth, data=file_obj)
+        prepared_request = r.prepare()
+        assert 'Transfer-Encoding' in prepared_request.headers
+        assert 'Content-Length' not in prepared_request.headers
+
+    def test_stream_with_auth_does_not_set_transfer_encoding_header(self, httpbin):
+        """Ensure that a byte stream with size > 0 will not set both a Content-Length
+        and Transfer-Encoding header.
+        """
+        auth = ('user', 'pass')
+        url = httpbin('post')
+        file_obj = io.BytesIO(b'test data')
+        r = requests.Request('POST', url, auth=auth, data=file_obj)
+        prepared_request = r.prepare()
+        assert 'Transfer-Encoding' not in prepared_request.headers
+        assert 'Content-Length' in prepared_request.headers
+
+    def test_chunked_upload_does_not_set_content_length_header(self, httpbin):
+        """Ensure that requests with a generator body stream using
+        Transfer-Encoding: chunked, not a Content-Length header.
+        """
+        data = (i for i in [b'a', b'b', b'c'])
+        url = httpbin('post')
+        r = requests.Request('POST', url, data=data)
+        prepared_request = r.prepare()
+        assert 'Transfer-Encoding' in prepared_request.headers
+        assert 'Content-Length' not in prepared_request.headers
+
+
 class TestCaseInsensitiveDict:
 
     @pytest.mark.parametrize(
@@ -2177,18 +2225,71 @@ class TestPreparingURLs(object):
             r.prepare()
 
     @pytest.mark.parametrize(
-        'protocol, url',
+        'input, expected',
         (
-            ("http+unix://", b"http+unix://%2Fvar%2Frun%2Fsocket/path"),
-            ("http+unix://", u"http+unix://%2Fvar%2Frun%2Fsocket/path"),
-            ("mailto", b"mailto:user@example.org"),
-            ("mailto", u"mailto:user@example.org"),
-            ("data", b"data:SSDimaUgUHl0aG9uIQ=="),
+            (
+                b"http+unix://%2Fvar%2Frun%2Fsocket/path",
+                u"http+unix://%2fvar%2frun%2fsocket/path",
+            ),
+            (
+                u"http+unix://%2Fvar%2Frun%2Fsocket/path",
+                u"http+unix://%2fvar%2frun%2fsocket/path",
+            ),
+            (
+                b"mailto:user@example.org",
+                u"mailto:user@example.org",
+            ),
+            (
+                u"mailto:user@example.org",
+                u"mailto:user@example.org",
+            ),
+            (
+                b"data:SSDimaUgUHl0aG9uIQ==",
+                u"data:SSDimaUgUHl0aG9uIQ==",
+            )
         )
     )
-    def test_url_passthrough(self, protocol, url):
-        session = requests.Session()
-        session.mount(protocol, HTTPAdapter())
-        p = requests.Request('GET', url=url)
-        p.prepare()
-        assert p.url == url
+    def test_url_mutation(self, input, expected):
+        """
+        This test validates that we correctly exclude some URLs from
+        preparation, and that we handle others. Specifically, it tests that
+        any URL whose scheme doesn't begin with "http" is left alone, and
+        those whose scheme *does* begin with "http" are mutated.
+        """
+        r = requests.Request('GET', url=input)
+        p = r.prepare()
+        assert p.url == expected
+
+    @pytest.mark.parametrize(
+        'input, params, expected',
+        (
+            (
+                b"http+unix://%2Fvar%2Frun%2Fsocket/path",
+                {"key": "value"},
+                u"http+unix://%2fvar%2frun%2fsocket/path?key=value",
+            ),
+            (
+                u"http+unix://%2Fvar%2Frun%2Fsocket/path",
+                {"key": "value"},
+                u"http+unix://%2fvar%2frun%2fsocket/path?key=value",
+            ),
+            (
+                b"mailto:user@example.org",
+                {"key": "value"},
+                u"mailto:user@example.org",
+            ),
+            (
+                u"mailto:user@example.org",
+                {"key": "value"},
+                u"mailto:user@example.org",
+            ),
+        )
+    )
+    def test_parameters_for_nonstandard_schemes(self, input, params, expected):
+        """
+        Setting paramters for nonstandard schemes is allowed if those schemes
+        begin with "http", and is forbidden otherwise.
+        """
+        r = requests.Request('GET', url=input, params=params)
+        p = r.prepare()
+        assert p.url == expected

tests/test_utils.py

@@ -4,7 +4,7 @@ from io import BytesIO
 
 import pytest
 from requests import compat
-from requests.cookies import RequestsCookieJar, cookiejar_from_dict
+from requests.cookies import RequestsCookieJar
 from requests.structures import CaseInsensitiveDict
 from requests.utils import (
     address_in_network, dotted_netmask,
@@ -274,6 +274,17 @@ class TestGuessJSONUTF:
     def test_bad_utf_like_encoding(self):
         assert guess_json_utf(b'\x00\x00\x00\x00') is None
 
+    @pytest.mark.parametrize(
+        ('encoding', 'expected'), (
+            ('utf-16-be', 'utf-16'),
+            ('utf-16-le', 'utf-16'),
+            ('utf-32-be', 'utf-32'),
+            ('utf-32-le', 'utf-32')
+        ))
+    def test_guess_by_bom(self, encoding, expected):
+        data = u'\ufeff{}'.encode(encoding)
+        assert guess_json_utf(data) == expected
+
 
 USER = PASSWORD = "%!*'();:@&=+$,/?#[] "
 ENCODED_USER = compat.quote(USER, '')