Merge pull request #1500 from gavrie/master

Update urllib3 to d89d508

AUTHORS.rst

@@ -125,6 +125,11 @@ Patches and Suggestions
 - Dmitry Medvinsky <me@dmedvinsky.name>
 - Bryce Boe <bbzbryce@gmail.com> @bboe
 - Colin Dunklau <colin.dunklau@gmail.com> @cdunklau
+- Bob Carroll <bob.carroll@alum.rit.edu> @rcarz
 - Hugo Osvaldo Barrera <hugo@osvaldobarrera.com.ar> @hobarrera
 - Łukasz Langa <lukasz@langa.pl> @llanga
 - Dave Shawley <daveshawley@gmail.com>
+- James Clarke (jam)
+- Kevin Burke <kev@inburke.com>
+- Flavio Curella
+- David Pursehouse <david.pursehouse@gmail.com> @dpursehouse

HISTORY.rst

@@ -3,6 +3,18 @@
 History
 -------
 
+1.2.3 (2013-05-25)
+++++++++++++++++++
+
+- Simple packaging fix
+
+
+1.2.2 (2013-05-23)
+++++++++++++++++++
+
+- Simple packaging fix
+
+
 1.2.1 (2013-05-20)
 ++++++++++++++++++
 

README.rst

@@ -1,6 +1,8 @@
 Requests: HTTP for Humans
 =========================
 
+.. image:: https://badge.fury.io/py/requests.png
+    :target: http://badge.fury.io/py/requests
 
 .. image:: https://travis-ci.org/kennethreitz/requests.png?branch=master
         :target: https://travis-ci.org/kennethreitz/requests
@@ -57,7 +59,7 @@ Features
 Installation
 ------------
 
-To install requests, simply:
+To install Requests, simply:
 
 .. code-block:: bash
 
@@ -82,7 +84,7 @@ Contribute
 ----------
 
 #. Check for open issues or open a fresh issue to start a discussion around a feature idea or a bug. There is a Contributor Friendly tag for issues that should be ideal for people who are not very familiar with the codebase yet.
-#. Fork `the repository`_ on Github to start making your changes to the **master** branch (or branch off of it).
+#. Fork `the repository`_ on GitHub to start making your changes to the **master** branch (or branch off of it).
 #. Write a test which shows that the bug was fixed or that the feature works as expected.
 #. Send a pull request and bug the maintainer until it gets merged and published. :) Make sure to add yourself to AUTHORS_.
 

docs/_templates/sidebarintro.html

@@ -11,8 +11,7 @@
 
 <p>
   Requests is an elegant and simple HTTP library for Python, built for
-  human beings. You are currently looking at the documentation of the
-  development release.
+  human beings.
 </p>
 
 

docs/api.rst

@@ -186,7 +186,7 @@ Licensing
 
 One key difference that has nothing to do with the API is a change in the
 license from the ISC_ license to the `Apache 2.0`_ license. The Apache 2.0
-license ensures that contributions to requests are also covered by the Apache
+license ensures that contributions to Requests are also covered by the Apache
 2.0 license.
 
 .. _ISC: http://opensource.org/licenses/ISC

docs/dev/philosophy.rst

@@ -38,4 +38,4 @@ Linux Distro Packages
 
 Distributions have been made for many Linux repositories, including: Ubuntu, Debian, RHEL, and Arch.
 
-These distributions are sometimes divergent forks, or are otherwise not kept up-to-date with the latest code and bugfixes. PyPI (and its mirrors) and GitHub are the official distribution sources; alternatives are not supported by the requests project.
+These distributions are sometimes divergent forks, or are otherwise not kept up-to-date with the latest code and bugfixes. PyPI (and its mirrors) and GitHub are the official distribution sources; alternatives are not supported by the Requests project.

docs/dev/todo.rst

@@ -6,7 +6,7 @@ Requests is under active development, and contributions are more than welcome!
 #. Check for open issues or open a fresh issue to start a discussion around a bug.
    There is a Contributor Friendly tag for issues that should be ideal for people who are not very
    familiar with the codebase yet.
-#. Fork `the repository <https://github.com/kennethreitz/requests>`_ on Github and start making your
+#. Fork `the repository <https://github.com/kennethreitz/requests>`_ on GitHub and start making your
    changes to a new branch.
 #. Write a test which shows that the bug was fixed.
 #. Send a pull request and bug the maintainer until it gets merged and published. :)
@@ -46,10 +46,15 @@ Requests currently supports the following versions of Python:
 
 Support for Python 3.1 and 3.2 may be dropped at any time.
 
-Google App Engine will never be officially supported. Pull requests for compatibility will be accepted, as long as they don't complicate the codebase.
+Google App Engine will never be officially supported. Pull Requests for compatibility will be accepted, as long as they don't complicate the codebase.
 
 
 Are you crazy?
 --------------
 
 - SPDY support would be awesome. No C extensions.
+
+Downstream Repackaging
+----------------------
+
+If you are repackaging Requests, please note that you must also redistribute the ``cacerts.pem`` file in order to get correct SSL functionality.

docs/user/advanced.rst

@@ -13,7 +13,7 @@ The Session object allows you to persist certain parameters across
 requests. It also persists cookies across all requests made from the
 Session instance.
 
-A session object has all the methods of the main Requests API.
+A Session object has all the methods of the main Requests API.
 
 Let's persist some cookies across requests::
 
@@ -27,7 +27,7 @@ Let's persist some cookies across requests::
 
 
 Sessions can also be used to provide default data to the request methods. This
-is done by providing data to the properties on a session object::
+is done by providing data to the properties on a Session object::
 
     s = requests.Session()
     s.auth = ('user', 'pass')
@@ -49,9 +49,9 @@ Request and Response Objects
 ----------------------------
 
 Whenever a call is made to requests.*() you are doing two major things. First,
-you are constructing a ``Request`` object which will be sent of to a server
+you are constructing a ``Request`` object which will be sent off to a server
 to request or query some resource. Second, a ``Response`` object is generated
-once ``requests`` gets a response back from the server. The response object
+once ``requests`` gets a response back from the server. The Response object
 contains all of the information returned by the server and also contains the
 ``Request`` object you created originally. Here is a simple request to get some
 very important information from Wikipedia's servers::
@@ -120,7 +120,7 @@ Requests can verify SSL certificates for HTTPS requests, just like a web browser
     >>> requests.get('https://kennethreitz.com', verify=True)
     requests.exceptions.SSLError: hostname 'kennethreitz.com' doesn't match either of '*.herokuapp.com', 'herokuapp.com'
 
-I don't have SSL setup on this domain, so it fails. Excellent. Github does though::
+I don't have SSL setup on this domain, so it fails. Excellent. GitHub does though::
 
     >>> requests.get('https://github.com', verify=True)
     <Response [200]>
@@ -268,6 +268,8 @@ Then, we can make a request using our Pizza Auth::
     >>> requests.get('http://pizzabin.org/admin', auth=PizzaAuth('kenneth'))
     <Response [200]>
 
+.. _streaming-requests
+
 Streaming Requests
 ------------------
 
@@ -279,7 +281,7 @@ To use the Twitter Streaming API to track the keyword "requests"::
     import json
     import requests
 
-    r = requests.post('http://httpbin.org/stream/20', stream=True)
+    r = requests.get('http://httpbin.org/stream/20', stream=True)
 
     for line in r.iter_lines():
 
@@ -574,3 +576,19 @@ a good start would be to subclass the ``requests.adapters.BaseAdapter`` class.
 .. _`described here`: http://kennethreitz.org/exposures/the-future-of-python-http
 .. _`urllib3`: https://github.com/shazow/urllib3
 
+Blocking Or Non-Blocking?
+-------------------------
+
+With the default Transport Adapter in place, Requests does not provide any kind
+of non-blocking IO. The ``Response.content`` property will block until the
+entire response has been downloaded. If you require more granularity, the
+streaming features of the library (see :ref:`streaming-requests`) allow you to
+retrieve smaller quantities of the response at a time. However, these calls
+will still block.
+
+If you are concerned about the use of blocking IO, there are lots of projects
+out there that combine Requests with one of Python's asynchronicity frameworks.
+Two excellent examples are `grequests`_ and `requests-futures`_.
+
+.. _`grequests`: https://github.com/kennethreitz/grequests
+.. _`requests-futures`: https://github.com/ross/requests-futures

docs/user/authentication.rst

@@ -32,6 +32,17 @@ Providing the credentials in a tuple like this is exactly the same as the
 ``HTTPBasicAuth`` example above.
 
 
+netrc Authentication
+~~~~~~~~~~~~~~~~~~~~
+
+If no authentication method is given with the ``auth`` argument, Requests will
+attempt to get the authentication credentials for the URL's hostname from the
+user's netrc file.
+
+If credentials for the hostname are found, the request is sent with HTTP Basic
+Auth.
+
+
 Digest Authentication
 ---------------------
 
@@ -47,7 +58,8 @@ and Requests supports this out of the box as well::
 OAuth 1 Authentication
 ----------------------
 
-A common form of authentication for several web APIs is OAuth. The ``requests-oauthlib`` library allows Requests users to easily make OAuth authenticated requests::
+A common form of authentication for several web APIs is OAuth. The ``requests-oauthlib``
+library allows Requests users to easily make OAuth authenticated requests::
 
     >>> import requests
     >>> from requests_oauthlib import OAuth1
@@ -60,7 +72,8 @@ A common form of authentication for several web APIs is OAuth. The ``requests-oa
     <Response [200]>
 
 For more information on how to OAuth flow works, please see the official `OAuth`_ website.
-For examples and documentation on requests-oauthlib, please see the `requests_oauthlib`_ repository on GitHub
+For examples and documentation on requests-oauthlib, please see the `requests_oauthlib`_
+repository on GitHub
 
 
 Other Authentication
@@ -76,7 +89,7 @@ authentication. Some of the best have been brought together under the
 - NTLM_
 
 If you want to use any of these forms of authentication, go straight to their
-Github page and follow the instructions.
+GitHub page and follow the instructions.
 
 
 New Forms of Authentication
@@ -87,12 +100,24 @@ want, you can implement it yourself. Requests makes it easy to add your own
 forms of authentication.
 
 To do so, subclass :class:`requests.auth.AuthBase` and implement the
-``__call__()`` method. When an authentication handler is attached to a request,
+``__call__()`` method::
+
+    >>> import requests
+    >>> class MyAuth(requests.auth.AuthBase):
+    ...     def __call__(self, r):
+    ...         # Implement my authentication
+    ...         return r
+    ...
+    >>> url = 'http://httpbin.org/get'
+    >>> requests.get(url, auth=MyAuth())
+    <Response [200]>
+
+When an authentication handler is attached to a request,
 it is called during request setup. The ``__call__`` method must therefore do
 whatever is required to make the authentication work. Some forms of
 authentication will additionally add hooks to provide further functionality.
 
-Examples can be found under the `Requests organization`_ and in the
+Further examples can be found under the `Requests organization`_ and in the
 ``auth.py`` file.
 
 .. _OAuth: http://oauth.net/

docs/user/install.rst

@@ -10,7 +10,7 @@ The first step to using any software package is getting it properly installed.
 Distribute & Pip
 ----------------
 
-Installing requests is simple with `pip <http://www.pip-installer.org/>`_::
+Installing Requests is simple with `pip <http://www.pip-installer.org/>`_::
 
     $ pip install requests
 
@@ -22,11 +22,11 @@ But, you really `shouldn't do that <http://www.pip-installer.org/en/latest/other
 
 
 
-Cheeseshop Mirror
------------------
+Cheeseshop (PyPI) Mirror
+------------------------
 
-If the Cheeseshop is down, you can also install Requests from one of the
-mirrors. `Crate.io <http://crate.io>`_ is one of them::
+If the Cheeseshop (a.k.a. PyPI) is down, you can also install Requests from one
+of the mirrors. `Crate.io <http://crate.io>`_ is one of them::
 
     $ pip install -i http://simple.crate.io/ requests
 

docs/user/intro.rst

@@ -20,7 +20,7 @@ All contributions to Requests should keep these important rules in mind.
 .. _`apache2`:
 
 Apache2 License
------------
+---------------
 
 A large number of open source projects you find today are `GPL Licensed`_.
 While the GPL has its time and place, it should most certainly not be your

docs/user/quickstart.rst

@@ -69,7 +69,10 @@ following code::
 You can see that the URL has been correctly encoded by printing the URL::
 
     >>> print r.url
-    u'http://httpbin.org/get?key2=value2&key1=value1'
+    http://httpbin.org/get?key2=value2&key1=value1
+
+Note that any dictionary key whose value is ``None`` will not be added to the
+URL's query string.
 
 
 Response Content
@@ -81,7 +84,7 @@ again::
     >>> import requests
     >>> r = requests.get('https://github.com/timeline.json')
     >>> r.text
-    '[{"repository":{"open_issues":0,"url":"https://github.com/...
+    u'[{"repository":{"open_issues":0,"url":"https://github.com/...
 
 Requests will automatically decode content from the server. Most unicode
 charsets are seamlessly decoded.
@@ -114,7 +117,7 @@ You can also access the response body as bytes, for non-text requests::
 The ``gzip`` and ``deflate`` transfer-encodings are automatically decoded for you.
 
 For example, to create an image from binary data returned by a request, you can
-use the following code:
+use the following code::
 
     >>> from PIL import Image
     >>> from StringIO import StringIO
@@ -170,7 +173,7 @@ More complicated POST requests
 ------------------------------
 
 Typically, you want to send some form-encoded data — much like an HTML form.
-To do this, simply pass a dictionary to the `data` argument. Your
+To do this, simply pass a dictionary to the ``data`` argument. Your
 dictionary of data will automatically be form-encoded when the request is made::
 
     >>> payload = {'key1': 'value1', 'key2': 'value2'}
@@ -260,7 +263,7 @@ reference::
     >>> r.status_code == requests.codes.ok
     True
 
-If we made a bad request (non-200 response), we can raise it with
+If we made a bad request (a 4XX client error or 5XX server error response), we can raise it with
 :class:`Response.raise_for_status()`::
 
     >>> bad_r = requests.get('http://httpbin.org/status/404')
@@ -295,7 +298,7 @@ We can view the server's response headers using a Python dictionary::
         'server': 'nginx/1.0.4',
         'x-runtime': '148ms',
         'etag': '"e1ca502697e5c9317743dc078f67693f"',
-        'content-type': 'application/json; charset=utf-8'
+        'content-type': 'application/json'
     }
 
 The dictionary is special, though: it's made just for HTTP headers. According to
@@ -305,15 +308,10 @@ Headers are case-insensitive.
 So, we can access the headers using any capitalization we want::
 
     >>> r.headers['Content-Type']
-    'application/json; charset=utf-8'
+    'application/json'
 
     >>> r.headers.get('content-type')
-    'application/json; charset=utf-8'
-
-If a header doesn't exist in the Response, its value defaults to ``None``::
-
-    >>> r.headers['X-Random']
-    None
+    'application/json'
 
 
 Cookies
@@ -345,7 +343,7 @@ Requests will automatically perform location redirection while using the GET
 and OPTIONS verbs.
 
 GitHub redirects all HTTP requests to HTTPS. We can use the ``history`` method
-of the Response object to track redirection. Let's see what Github does::
+of the Response object to track redirection. Let's see what GitHub does::
 
     >>> r = requests.get('http://github.com')
     >>> r.url
@@ -355,8 +353,9 @@ of the Response object to track redirection. Let's see what Github does::
     >>> r.history
     [<Response [301]>]
 
-The :class:`Response.history` list contains a list of the
-:class:`Request` objects that were created in order to complete the request. The list is sorted from the oldest to the most recent request.
+The :class:`Response.history` list contains the :class:`Request` objects that
+were created in order to complete the request. The list is sorted from the
+oldest to the most recent request.
 
 If you're using GET or OPTIONS, you can disable redirection handling with the
 ``allow_redirects`` parameter::
@@ -380,7 +379,7 @@ redirection as well::
 Timeouts
 --------
 
-You can tell requests to stop waiting for a response after a given number of
+You can tell Requests to stop waiting for a response after a given number of
 seconds with the ``timeout`` parameter::
 
     >>> requests.get('http://github.com', timeout=0.001)
@@ -391,7 +390,7 @@ seconds with the ``timeout`` parameter::
 
 .. admonition:: Note:
 
-    ``timeout`` only effects the connection process itself, not the
+    ``timeout`` only affects the connection process itself, not the
     downloading of the response body.
 
 
@@ -399,15 +398,16 @@ Errors and Exceptions
 ---------------------
 
 In the event of a network problem (e.g. DNS failure, refused connection, etc),
-Requests will raise a :class:`ConnectionError` exception.
+Requests will raise a :class:`~requests.exceptions.ConnectionError` exception.
 
-In the event of the rare invalid HTTP response, Requests will raise
-an  :class:`HTTPError` exception.
+In the event of the rare invalid HTTP response, Requests will raise an
+:class:`~requests.exceptions.HTTPError` exception.
 
-If a request times out, a :class:`Timeout` exception is raised.
+If a request times out, a :class:`~requests.exceptions.Timeout` exception is
+raised.
 
 If a request exceeds the configured number of maximum redirections, a
-:class:`TooManyRedirects` exception is raised.
+:class:`~requests.exceptions.TooManyRedirects` exception is raised.
 
 All exceptions that Requests explicitly raises inherit from
 :class:`requests.exceptions.RequestException`.

invokefile.py

@@ -1,5 +0,0 @@
-from invoke import run, task
-
-@task
-def build():
-    print("Building!")

requests/__init__.py

@@ -42,8 +42,8 @@ is at <http://python-requests.org>.
 """
 
 __title__ = 'requests'
-__version__ = '1.2.1'
-__build__ = 0x010201
+__version__ = '1.2.3'
+__build__ = 0x010203
 __author__ = 'Kenneth Reitz'
 __license__ = 'Apache 2.0'
 __copyright__ = 'Copyright 2013 Kenneth Reitz'

requests/adapters.py

@@ -25,6 +25,7 @@ from .cookies import extract_cookies_to_jar
 from .exceptions import ConnectionError, Timeout, SSLError
 from .auth import _basic_auth_str
 
+DEFAULT_POOLBLOCK = False
 DEFAULT_POOLSIZE = 10
 DEFAULT_RETRIES = 0
 
@@ -53,6 +54,7 @@ class HTTPAdapter(BaseAdapter):
     :param pool_connections: The number of urllib3 connection pools to cache.
     :param pool_maxsize: The maximum number of connections to save in the pool.
     :param max_retries: The maximum number of retries each connection should attempt.
+    :param pool_block: Whether the connection pool should block for connections.
 
     Usage::
 
@@ -61,10 +63,12 @@ class HTTPAdapter(BaseAdapter):
       >>> a = requests.adapters.HTTPAdapter()
       >>> s.mount('http://', a)
     """
-    __attrs__ = ['max_retries', 'config', '_pool_connections', '_pool_maxsize']
+    __attrs__ = ['max_retries', 'config', '_pool_connections', '_pool_maxsize',
+                 '_pool_block']
 
     def __init__(self, pool_connections=DEFAULT_POOLSIZE,
-                 pool_maxsize=DEFAULT_POOLSIZE, max_retries=DEFAULT_RETRIES):
+                 pool_maxsize=DEFAULT_POOLSIZE, max_retries=DEFAULT_RETRIES,
+                 pool_block=DEFAULT_POOLBLOCK):
         self.max_retries = max_retries
         self.config = {}
 
@@ -72,8 +76,9 @@ class HTTPAdapter(BaseAdapter):
 
         self._pool_connections = pool_connections
         self._pool_maxsize = pool_maxsize
+        self._pool_block = pool_block
 
-        self.init_poolmanager(pool_connections, pool_maxsize)
+        self.init_poolmanager(pool_connections, pool_maxsize, block=pool_block)
 
     def __getstate__(self):
         return dict((attr, getattr(self, attr, None)) for attr in
@@ -83,21 +88,25 @@ class HTTPAdapter(BaseAdapter):
         for attr, value in state.items():
             setattr(self, attr, value)
 
-        self.init_poolmanager(self._pool_connections, self._pool_maxsize)
+        self.init_poolmanager(self._pool_connections, self._pool_maxsize,
+                              block=self._pool_block)
 
-    def init_poolmanager(self, connections, maxsize):
+    def init_poolmanager(self, connections, maxsize, block=DEFAULT_POOLBLOCK):
         """Initializes a urllib3 PoolManager. This method should not be called
         from user code, and is only exposed for use when subclassing the
         :class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
 
         :param connections: The number of urllib3 connection pools to cache.
         :param maxsize: The maximum number of connections to save in the pool.
+        :param block: Block when no free connections are available.
         """
         # save these values for pickling
         self._pool_connections = connections
         self._pool_maxsize = maxsize
+        self._pool_block = block
 
-        self.poolmanager = PoolManager(num_pools=connections, maxsize=maxsize)
+        self.poolmanager = PoolManager(num_pools=connections, maxsize=maxsize,
+                                       block=block)
 
     def cert_verify(self, conn, url, verify, cert):
         """Verify a SSL certificate. This method should not be called from user
@@ -109,7 +118,7 @@ class HTTPAdapter(BaseAdapter):
         :param verify: Whether we should actually verify the certificate.
         :param cert: The SSL certificate to verify.
         """
-        if url.startswith('https') and verify:
+        if url.lower().startswith('https') and verify:
 
             cert_loc = None
 
@@ -181,13 +190,13 @@ class HTTPAdapter(BaseAdapter):
         :param proxies: (optional) A Requests-style dictionary of proxies used on this request.
         """
         proxies = proxies or {}
-        proxy = proxies.get(urlparse(url).scheme)
+        proxy = proxies.get(urlparse(url.lower()).scheme)
 
         if proxy:
-            proxy = prepend_scheme_if_needed(proxy, urlparse(url).scheme)
+            proxy = prepend_scheme_if_needed(proxy, urlparse(url.lower()).scheme)
             conn = ProxyManager(self.poolmanager.connection_from_url(proxy))
         else:
-            conn = self.poolmanager.connection_from_url(url)
+            conn = self.poolmanager.connection_from_url(url.lower())
 
         return conn
 
@@ -205,7 +214,7 @@ class HTTPAdapter(BaseAdapter):
         If the message is being sent through a proxy, the full URL has to be
         used. Otherwise, we should only use the path portion of the URL.
 
-        This shoudl not be called from user code, and is only exposed for use
+        This should not be called from user code, and is only exposed for use
         when subclassing the
         :class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
 

requests/auth.py

@@ -153,7 +153,7 @@ class HTTPDigestAuth(AuthBase):
 
             setattr(self, 'num_401_calls', num_401_calls + 1)
             pat = re.compile(r'digest ', flags=re.IGNORECASE)
-            self.chal = parse_dict_header(pat.sub('', s_auth))
+            self.chal = parse_dict_header(pat.sub('', s_auth, count=1))
 
             # Consume content and release the original connection
             # to allow our new request to reuse the same one.

requests/compat.py

@@ -83,13 +83,14 @@ except ImportError:
 # ---------
 
 if is_py2:
-    from urllib import quote, unquote, quote_plus, unquote_plus, urlencode
+    from urllib import quote, unquote, quote_plus, unquote_plus, urlencode, getproxies, proxy_bypass
     from urlparse import urlparse, urlunparse, urljoin, urlsplit, urldefrag
     from urllib2 import parse_http_list
     import cookielib
     from Cookie import Morsel
     from StringIO import StringIO
     from .packages.urllib3.packages.ordered_dict import OrderedDict
+    from httplib import IncompleteRead
 
     builtin_str = str
     bytes = str
@@ -100,11 +101,12 @@ if is_py2:
 
 elif is_py3:
     from urllib.parse import urlparse, urlunparse, urljoin, urlsplit, urlencode, quote, unquote, quote_plus, unquote_plus, urldefrag
-    from urllib.request import parse_http_list
+    from urllib.request import parse_http_list, getproxies, proxy_bypass
     from http import cookiejar as cookielib
     from http.cookies import Morsel
     from io import StringIO
     from collections import OrderedDict
+    from http.client import IncompleteRead
 
     builtin_str = str
     str = str

requests/cookies.py

@@ -6,6 +6,7 @@ Compatibility code to be able to use `cookielib.CookieJar` with requests.
 requests.utils imports from here, so be careful with imports.
 """
 
+import time
 import collections
 from .compat import cookielib, urlparse, Morsel
 
@@ -73,6 +74,10 @@ class MockRequest(object):
     def origin_req_host(self):
         return self.get_origin_req_host()
 
+    @property
+    def host(self):
+        return self.get_host()
+
 
 class MockResponse(object):
     """Wraps a `httplib.HTTPMessage` to mimic a `urllib.addinfourl`.
@@ -258,6 +263,11 @@ class RequestsCookieJar(cookielib.CookieJar, collections.MutableMapping):
         """Deletes a cookie given a name. Wraps cookielib.CookieJar's remove_cookie_by_name()."""
         remove_cookie_by_name(self, name)
 
+    def set_cookie(self, cookie, *args, **kwargs):
+        if cookie.value.startswith('"') and cookie.value.endswith('"'):
+            cookie.value = cookie.value.replace('\\"', '')
+        return super(RequestsCookieJar, self).set_cookie(cookie, *args, **kwargs)
+
     def update(self, other):
         """Updates this jar with cookies from another CookieJar or dict-like"""
         if isinstance(other, cookielib.CookieJar):
@@ -354,19 +364,23 @@ def create_cookie(name, value, **kwargs):
 
 def morsel_to_cookie(morsel):
     """Convert a Morsel object into a Cookie containing the one k/v pair."""
+    expires = None
+    if morsel["max-age"]:
+        expires = time.time() + morsel["max-age"]
+    elif morsel['expires']:
+        expires = morsel['expires']
+        if type(expires) == type(""):
+            time_template = "%a, %d-%b-%Y %H:%M:%S GMT"
+            expires = time.mktime(time.strptime(expires, time_template))
     c = create_cookie(
         name=morsel.key,
         value=morsel.value,
         version=morsel['version'] or 0,
         port=None,
-        port_specified=False,
         domain=morsel['domain'],
-        domain_specified=bool(morsel['domain']),
-        domain_initial_dot=morsel['domain'].startswith('.'),
         path=morsel['path'],
-        path_specified=bool(morsel['path']),
         secure=bool(morsel['secure']),
-        expires=morsel['max-age'] or morsel['expires'],
+        expires=expires,
         discard=False,
         comment=morsel['comment'],
         comment_url=bool(morsel['comment']),

requests/exceptions.py

@@ -53,3 +53,7 @@ class InvalidSchema(RequestException, ValueError):
 
 class InvalidURL(RequestException, ValueError):
     """ The URL provided was somehow invalid. """
+
+
+class ChunkedEncodingError(RequestException):
+    """The server declared chunked encoding but sent an invalid chunk."""

requests/models.py

@@ -19,14 +19,16 @@ from .auth import HTTPBasicAuth
 from .cookies import cookiejar_from_dict, get_cookie_header
 from .packages.urllib3.filepost import encode_multipart_formdata
 from .packages.urllib3.util import parse_url
-from .exceptions import HTTPError, RequestException, MissingSchema, InvalidURL
+from .exceptions import (
+    HTTPError, RequestException, MissingSchema, InvalidURL,
+    ChunkedEncodingError)
 from .utils import (
     guess_filename, get_auth_from_url, requote_uri,
     stream_decode_response_unicode, to_key_val_list, parse_header_links,
     iter_slices, guess_json_utf, super_len)
 from .compat import (
-    cookielib, urlparse, urlunparse, urlsplit, urlencode, str, bytes, StringIO,
-    is_py2, chardet, json, builtin_str, basestring)
+    cookielib, urlunparse, urlsplit, urlencode, str, bytes, StringIO,
+    is_py2, chardet, json, builtin_str, basestring, IncompleteRead)
 
 CONTENT_CHUNK_SIZE = 10 * 1024
 ITER_CHUNK_SIZE = 512
@@ -105,7 +107,7 @@ class RequestEncodingMixin(object):
             for v in val:
                 if v is not None:
                     new_fields.append(
-                        (field.encode('utf-8') if isinstance(field, str) else field,
+                        (field.decode('utf-8') if isinstance(field, bytes) else field,
                          v.encode('utf-8') if isinstance(v, str) else v))
 
         for (k, v) in files:
@@ -209,7 +211,6 @@ class Request(RequestHooksMixin):
         self.params = params
         self.auth = auth
         self.cookies = cookies
-        self.hooks = hooks
 
     def __repr__(self):
         return '<Request [%s]>' % (self.method)
@@ -217,19 +218,17 @@ class Request(RequestHooksMixin):
     def prepare(self):
         """Constructs a :class:`PreparedRequest <PreparedRequest>` for transmission and returns it."""
         p = PreparedRequest()
-
-        p.prepare_method(self.method)
-        p.prepare_url(self.url, self.params)
-        p.prepare_headers(self.headers)
-        p.prepare_cookies(self.cookies)
-        p.prepare_body(self.data, self.files)
-        p.prepare_auth(self.auth, self.url)
-        # Note that prepare_auth must be last to enable authentication schemes
-        # such as OAuth to work on a fully prepared request.
-
-        # This MUST go after prepare_auth. Authenticators could add a hook
-        p.prepare_hooks(self.hooks)
-
+        p.prepare(
+            method=self.method,
+            url=self.url,
+            headers=self.headers,
+            files=self.files,
+            data=self.data,
+            params=self.params,
+            auth=self.auth,
+            cookies=self.cookies,
+            hooks=self.hooks,
+        )
         return p
 
 
@@ -264,9 +263,34 @@ class PreparedRequest(RequestEncodingMixin, RequestHooksMixin):
         #: dictionary of callback hooks, for internal usage.
         self.hooks = default_hooks()
 
+    def prepare(self, method=None, url=None, headers=None, files=None,
+                data=None, params=None, auth=None, cookies=None, hooks=None):
+        """Prepares the the entire request with the given parameters."""
+
+        self.prepare_method(method)
+        self.prepare_url(url, params)
+        self.prepare_headers(headers)
+        self.prepare_cookies(cookies)
+        self.prepare_body(data, files)
+        self.prepare_auth(auth, url)
+        # Note that prepare_auth must be last to enable authentication schemes
+        # such as OAuth to work on a fully prepared request.
+
+        # This MUST go after prepare_auth. Authenticators could add a hook
+        self.prepare_hooks(hooks)
+
     def __repr__(self):
         return '<PreparedRequest [%s]>' % (self.method)
 
+    def copy(self):
+        p = PreparedRequest()
+        p.method = self.method
+        p.url = self.url
+        p.headers = self.headers
+        p.body = self.body
+        p.hooks = self.hooks
+        return p
+
     def prepare_method(self, method):
         """Prepares the given HTTP method."""
         self.method = method
@@ -291,7 +315,7 @@ class PreparedRequest(RequestEncodingMixin, RequestHooksMixin):
             raise MissingSchema("Invalid URL %r: No schema supplied" % url)
 
         if not host:
-            raise InvalidURL("Invalid URL %t: No host supplied" % url)
+            raise InvalidURL("Invalid URL %r: No host supplied" % url)
 
         # Only want to apply IDNA to the hostname
         try:
@@ -352,7 +376,6 @@ class PreparedRequest(RequestEncodingMixin, RequestHooksMixin):
         body = None
         content_type = None
         length = None
-        is_stream = False
 
         is_stream = all([
             hasattr(data, '__iter__'),
@@ -364,7 +387,7 @@ class PreparedRequest(RequestEncodingMixin, RequestHooksMixin):
         try:
             length = super_len(data)
         except (TypeError, AttributeError):
-            length = False
+            length = None
 
         if is_stream:
             body = data
@@ -372,7 +395,7 @@ class PreparedRequest(RequestEncodingMixin, RequestHooksMixin):
             if files:
                 raise NotImplementedError('Streamed bodies and files are mutually exclusive.')
 
-            if length:
+            if length is not None:
                 self.headers['Content-Length'] = str(length)
             else:
                 self.headers['Transfer-Encoding'] = 'chunked'
@@ -537,11 +560,22 @@ class Response(object):
             return iter_slices(self._content, chunk_size)
 
         def generate():
-            while 1:
-                chunk = self.raw.read(chunk_size, decode_content=True)
-                if not chunk:
-                    break
-                yield chunk
+            try:
+                # Special case for urllib3.
+                try:
+                    for chunk in self.raw.stream(chunk_size,
+                                                 decode_content=True):
+                        yield chunk
+                except IncompleteRead as e:
+                    raise ChunkedEncodingError(e)
+            except AttributeError:
+                # Standard file-like object.
+                while 1:
+                    chunk = self.raw.read(chunk_size)
+                    if not chunk:
+                        break
+                    yield chunk
+
             self._content_consumed = True
 
         gen = generate()
@@ -683,4 +717,9 @@ class Response(object):
             raise HTTPError(http_error_msg, response=self)
 
     def close(self):
+        """Closes the underlying file descriptor and releases the connection
+        back to the pool.
+
+        *Note: Should not normally need to be called explicitly.*
+        """
         return self.raw.release_conn()

requests/packages/urllib3/_collections.py

@@ -5,7 +5,7 @@
 # the MIT License: http://www.opensource.org/licenses/mit-license.php
 
 from collections import MutableMapping
-from threading import Lock
+from threading import RLock
 
 try: # Python 2.7+
     from collections import OrderedDict
@@ -40,18 +40,18 @@ class RecentlyUsedContainer(MutableMapping):
         self.dispose_func = dispose_func
 
         self._container = self.ContainerCls()
-        self._lock = Lock()
+        self.lock = RLock()
 
     def __getitem__(self, key):
         # Re-insert the item, moving it to the end of the eviction line.
-        with self._lock:
+        with self.lock:
             item = self._container.pop(key)
             self._container[key] = item
             return item
 
     def __setitem__(self, key, value):
         evicted_value = _Null
-        with self._lock:
+        with self.lock:
             # Possibly evict the existing value of 'key'
             evicted_value = self._container.get(key, _Null)
             self._container[key] = value
@@ -65,21 +65,21 @@ class RecentlyUsedContainer(MutableMapping):
             self.dispose_func(evicted_value)
 
     def __delitem__(self, key):
-        with self._lock:
+        with self.lock:
             value = self._container.pop(key)
 
         if self.dispose_func:
             self.dispose_func(value)
 
     def __len__(self):
-        with self._lock:
+        with self.lock:
             return len(self._container)
 
     def __iter__(self):
         raise NotImplementedError('Iteration over this class is unlikely to be threadsafe.')
 
     def clear(self):
-        with self._lock:
+        with self.lock:
             # Copy pointers to all values, then wipe the mapping
             # under Python 2, this copies the list of values twice :-|
             values = list(self._container.values())
@@ -90,5 +90,5 @@ class RecentlyUsedContainer(MutableMapping):
                 self.dispose_func(value)
 
     def keys(self):
-        with self._lock:
+        with self.lock:
             return self._container.keys()

requests/packages/urllib3/connectionpool.py

@@ -26,7 +26,10 @@ except ImportError:
 
 try: # Compiled with SSL?
     HTTPSConnection = object
-    BaseSSLError = None
+
+    class BaseSSLError(BaseException):
+        pass
+
     ssl = None
 
     try: # Python 3
@@ -110,7 +113,7 @@ class VerifiedHTTPSConnection(HTTPSConnection):
             if self.assert_fingerprint:
                 assert_fingerprint(self.sock.getpeercert(binary_form=True),
                                    self.assert_fingerprint)
-            else:
+            elif self.assert_hostname is not False:
                 match_hostname(self.sock.getpeercert(),
                                self.assert_hostname or self.host)
 
@@ -152,8 +155,8 @@ class HTTPConnectionPool(ConnectionPool, RequestMethods):
         :class:`httplib.HTTPConnection`.
 
     :param timeout:
-        Socket timeout for each individual connection, can be a float. None
-        disables timeout.
+        Socket timeout in seconds for each individual connection, can be
+        a float. None disables timeout.
 
     :param maxsize:
         Number of connections to save that can be reused. More than 1 is useful
@@ -376,6 +379,7 @@ class HTTPConnectionPool(ConnectionPool, RequestMethods):
 
         :param timeout:
             If specified, overrides the default timeout for this one request.
+            It may be a float (in seconds).
 
         :param pool_timeout:
             If set and the pool is set to block=True, then this method will
@@ -410,10 +414,6 @@ class HTTPConnectionPool(ConnectionPool, RequestMethods):
 
         # Check host
         if assert_same_host and not self.is_same_host(url):
-            host = "%s://%s" % (self.scheme, self.host)
-            if self.port:
-                host = "%s:%d" % (host, self.port)
-
             raise HostChangedError(self, url, retries - 1)
 
         conn = None
@@ -513,6 +513,7 @@ class HTTPSConnectionPool(HTTPConnectionPool):
 
     :class:`.VerifiedHTTPSConnection` uses one of ``assert_fingerprint``,
     ``assert_hostname`` and ``host`` in this order to verify connections.
+    If ``assert_hostname`` is False, no verification is done.
 
     The ``key_file``, ``cert_file``, ``cert_reqs``, ``ca_certs`` and
     ``ssl_version`` are only used if :mod:`ssl` is available and are fed into

requests/packages/urllib3/contrib/ntlmpool.py

@@ -33,7 +33,7 @@ class NTLMConnectionPool(HTTPSConnectionPool):
     def __init__(self, user, pw, authurl, *args, **kwargs):
         """
         authurl is a random URL on the server that is protected by NTLM.
-        user is the Windows user, probably in the DOMAIN\username format.
+        user is the Windows user, probably in the DOMAIN\\username format.
         pw is the password for the user.
         """
         super(NTLMConnectionPool, self).__init__(*args, **kwargs)

requests/packages/urllib3/contrib/pyopenssl.py

@@ -106,6 +106,9 @@ class WrappedSocket(object):
         self.connection = connection
         self.socket = socket
 
+    def fileno(self):
+        return self.socket.fileno()
+
     def makefile(self, mode, bufsize=-1):
         return _fileobject(self.connection, mode, bufsize)
 
@@ -115,6 +118,9 @@ class WrappedSocket(object):
     def sendall(self, data):
         return self.connection.sendall(data)
 
+    def close(self):
+        return self.connection.shutdown()
+
     def getpeercert(self, binary_form=False):
         x509 = self.connection.get_peer_certificate()
         if not x509:

requests/packages/urllib3/filepost.py

@@ -1,5 +1,5 @@
 # urllib3/filepost.py
-# Copyright 2008-2012 Andrey Petrov and contributors (see CONTRIBUTORS.txt)
+# Copyright 2008-2013 Andrey Petrov and contributors (see CONTRIBUTORS.txt)
 #
 # This module is part of urllib3 and is released under
 # the MIT License: http://www.opensource.org/licenses/mit-license.php

requests/packages/urllib3/poolmanager.py

@@ -6,6 +6,11 @@
 
 import logging
 
+try:  # Python 3
+    from urllib.parse import urljoin
+except ImportError:
+    from urlparse import urljoin
+
 from ._collections import RecentlyUsedContainer
 from .connectionpool import HTTPConnectionPool, HTTPSConnectionPool
 from .connectionpool import connection_from_url, port_by_scheme
@@ -99,15 +104,16 @@ class PoolManager(RequestMethods):
 
         pool_key = (scheme, host, port)
 
-        # If the scheme, host, or port doesn't match existing open connections,
-        # open a new ConnectionPool.
-        pool = self.pools.get(pool_key)
-        if pool:
-            return pool
+        with self.pools.lock:
+          # If the scheme, host, or port doesn't match existing open connections,
+          # open a new ConnectionPool.
+          pool = self.pools.get(pool_key)
+          if pool:
+              return pool
 
-        # Make a fresh ConnectionPool of the desired type
-        pool = self._new_pool(scheme, host, port)
-        self.pools[pool_key] = pool
+          # Make a fresh ConnectionPool of the desired type
+          pool = self._new_pool(scheme, host, port)
+          self.pools[pool_key] = pool
         return pool
 
     def connection_from_url(self, url):
@@ -145,6 +151,10 @@ class PoolManager(RequestMethods):
         if not redirect_location:
             return response
 
+        # Support relative URLs for redirecting.
+        redirect_location = urljoin(url, redirect_location)
+
+        # RFC 2616, Section 10.3.4
         if response.status == 303:
             method = 'GET'
 
@@ -171,9 +181,9 @@ class ProxyManager(RequestMethods):
         """
         headers_ = {'Accept': '*/*'}
 
-        host = parse_url(url).host
-        if host:
-            headers_['Host'] = host
+        netloc = parse_url(url).netloc
+        if netloc:
+            headers_['Host'] = netloc
 
         if headers:
             headers_.update(headers)

requests/packages/urllib3/request.py

@@ -30,7 +30,7 @@ class RequestMethods(object):
     in the URL (such as GET, HEAD, DELETE).
 
     :meth:`.request_encode_body` is for sending requests whose fields are
-    encoded in the *body* of the request using multipart or www-orm-urlencoded
+    encoded in the *body* of the request using multipart or www-form-urlencoded
     (such as for POST, PUT, PATCH).
 
     :meth:`.request` is for making any kind of request, it will look up the

requests/packages/urllib3/response.py

@@ -1,5 +1,5 @@
 # urllib3/response.py
-# Copyright 2008-2012 Andrey Petrov and contributors (see CONTRIBUTORS.txt)
+# Copyright 2008-2013 Andrey Petrov and contributors (see CONTRIBUTORS.txt)
 #
 # This module is part of urllib3 and is released under
 # the MIT License: http://www.opensource.org/licenses/mit-license.php
@@ -7,9 +7,11 @@
 
 import logging
 import zlib
+import io
 
 from .exceptions import DecodeError
 from .packages.six import string_types as basestring, binary_type
+from .util import is_fp_closed
 
 
 log = logging.getLogger(__name__)
@@ -48,7 +50,7 @@ def _get_decoder(mode):
     return DeflateDecoder()
 
 
-class HTTPResponse(object):
+class HTTPResponse(io.IOBase):
     """
     HTTP Response container.
 
@@ -183,9 +185,11 @@ class HTTPResponse(object):
             try:
                 if decode_content and self._decoder:
                     data = self._decoder.decompress(data)
-            except (IOError, zlib.error):
-                raise DecodeError("Received response with content-encoding: %s, but "
-                                  "failed to decode it." % content_encoding)
+            except (IOError, zlib.error) as e:
+                raise DecodeError(
+                    "Received response with content-encoding: %s, but "
+                    "failed to decode it." % content_encoding,
+                    e)
 
             if flush_decoder and self._decoder:
                 buf = self._decoder.decompress(binary_type())
@@ -200,6 +204,29 @@ class HTTPResponse(object):
             if self._original_response and self._original_response.isclosed():
                 self.release_conn()
 
+    def stream(self, amt=2**16, decode_content=None):
+        """
+        A generator wrapper for the read() method. A call will block until
+        ``amt`` bytes have been read from the connection or until the
+        connection is closed.
+
+        :param amt:
+            How much of the content to read. The generator will return up to
+            much data per iteration, but may return less. This is particularly
+            likely when using compressed data. However, the empty string will
+            never be returned.
+
+        :param decode_content:
+            If True, will attempt to decode the body based on the
+            'content-encoding' header.
+        """
+        while not is_fp_closed(self._fp):
+            data = self.read(amt=amt, decode_content=decode_content)
+
+            if data:
+                yield data
+
+
     @classmethod
     def from_httplib(ResponseCls, r, **response_kw):
         """
@@ -239,3 +266,35 @@ class HTTPResponse(object):
 
     def getheader(self, name, default=None):
         return self.headers.get(name, default)
+
+    # Overrides from io.IOBase
+    def close(self):
+        if not self.closed:
+            self._fp.close()
+
+    @property
+    def closed(self):
+        if self._fp is None:
+            return True
+        elif hasattr(self._fp, 'closed'):
+            return self._fp.closed
+        elif hasattr(self._fp, 'isclosed'):  # Python 2
+            return self._fp.isclosed()
+        else:
+            return True
+
+    def fileno(self):
+        if self._fp is None:
+            raise IOError("HTTPResponse has no file to get a fileno from")
+        elif hasattr(self._fp, "fileno"):
+            return self._fp.fileno()
+        else:
+            raise IOError("The file-like object  this HTTPResponse is wrapped "
+                          "around has no file descriptor")
+
+    def flush(self):
+        if self._fp is not None and hasattr(self._fp, 'flush'):
+            return self._fp.flush()
+
+    def readable(self):
+        return True

requests/packages/urllib3/util.py

@@ -31,7 +31,6 @@ try:  # Test for SSL features
 except ImportError:
     pass
 
-
 from .packages import six
 from .exceptions import LocationParseError, SSLError
 
@@ -61,6 +60,13 @@ class Url(namedtuple('Url', ['scheme', 'auth', 'host', 'port', 'path', 'query',
 
         return uri
 
+    @property
+    def netloc(self):
+        """Network location including host and port"""
+        if self.port:
+            return '%s:%d' % (self.host, self.port)
+        return self.host
+
 
 def split_first(s, delims):
     """
@@ -114,7 +120,7 @@ def parse_url(url):
 
     # While this code has overlap with stdlib's urlparse, it is much
     # simplified for our needs and less annoying.
-    # Additionally, this imeplementations does silly things to be optimal
+    # Additionally, this implementations does silly things to be optimal
     # on CPython.
 
     scheme = None
@@ -143,7 +149,8 @@ def parse_url(url):
 
     # IPv6
     if url and url[0] == '[':
-        host, url = url[1:].split(']', 1)
+        host, url = url.split(']', 1)
+        host += ']'
 
     # Port
     if ':' in url:
@@ -341,6 +348,20 @@ def assert_fingerprint(cert, fingerprint):
                        .format(hexlify(fingerprint_bytes),
                                hexlify(cert_digest)))
 
+def is_fp_closed(obj):
+    """
+    Checks whether a given file-like object is closed.
+
+    :param obj:
+        The file-like object to check.
+    """
+    if hasattr(obj, 'fp'):
+        # Object is a container for another file-like object that gets released
+        # on exhaustion (e.g. HTTPResponse)
+        return obj.fp is None
+
+    return obj.closed
+
 
 if SSLContext is not None:  # Python 3.2+
     def ssl_wrap_socket(sock, keyfile=None, certfile=None, cert_reqs=None,

requests/sessions.py

@@ -9,14 +9,16 @@ requests (cookies, auth, proxies).
 
 """
 import os
+from collections import Mapping
 from datetime import datetime
 
 from .compat import cookielib, OrderedDict, urljoin, urlparse
 from .cookies import cookiejar_from_dict, extract_cookies_to_jar, RequestsCookieJar
 from .models import Request, PreparedRequest
 from .hooks import default_hooks, dispatch_hook
-from .utils import from_key_val_list, default_headers
+from .utils import to_key_val_list, default_headers
 from .exceptions import TooManyRedirects, InvalidSchema
+from .structures import CaseInsensitiveDict
 
 from .adapters import HTTPAdapter
 
@@ -32,49 +34,35 @@ REDIRECT_STATI = (
 DEFAULT_REDIRECT_LIMIT = 30
 
 
-def merge_kwargs(local_kwarg, default_kwarg):
-    """Merges kwarg dictionaries.
-
-    If a local key in the dictionary is set to None, it will be removed.
+def merge_setting(request_setting, session_setting, dict_class=OrderedDict):
+    """
+    Determines appropriate setting for a given request, taking into account the
+    explicit setting on that request, and the setting in the session. If a
+    setting is a dictionary, they will be merged together using `dict_class`
     """
 
-    if default_kwarg is None:
-        return local_kwarg
+    if session_setting is None:
+        return request_setting
 
-    if isinstance(local_kwarg, str):
-        return local_kwarg
+    if request_setting is None:
+        return session_setting
 
-    if local_kwarg is None:
-        return default_kwarg
+    # Bypass if not a dictionary (e.g. verify)
+    if not (
+            isinstance(session_setting, Mapping) and
+            isinstance(request_setting, Mapping)
+    ):
+        return request_setting
 
-    # Bypass if not a dictionary (e.g. timeout)
-    if not hasattr(default_kwarg, 'items'):
-        return local_kwarg
-
-    default_kwarg = from_key_val_list(default_kwarg)
-    local_kwarg = from_key_val_list(local_kwarg)
-
-    # Update new values in a case-insensitive way
-    def get_original_key(original_keys, new_key):
-        """
-        Finds the key from original_keys that case-insensitive matches new_key.
-        """
-        for original_key in original_keys:
-            if key.lower() == original_key.lower():
-                return original_key
-        return new_key
-
-    kwargs = default_kwarg.copy()
-    original_keys = kwargs.keys()
-    for key, value in local_kwarg.items():
-        kwargs[get_original_key(original_keys, key)] = value
+    merged_setting = dict_class(to_key_val_list(session_setting))
+    merged_setting.update(to_key_val_list(request_setting))
 
     # Remove keys that are set to None.
-    for (k, v) in local_kwarg.items():
+    for (k, v) in request_setting.items():
         if v is None:
-            del kwargs[k]
+            del merged_setting[k]
 
-    return kwargs
+    return merged_setting
 
 
 class SessionRedirectMixin(object):
@@ -83,15 +71,13 @@ class SessionRedirectMixin(object):
         """Receives a Response. Returns a generator of Responses."""
 
         i = 0
-        prepared_request = PreparedRequest()
-        prepared_request.body = req.body
-        prepared_request.headers = req.headers.copy()
-        prepared_request.hooks = req.hooks
-        prepared_request.method = req.method
-        prepared_request.url = req.url
 
         # ((resp.status_code is codes.see_other))
         while (('location' in resp.headers and resp.status_code in REDIRECT_STATI)):
+            prepared_request = PreparedRequest()
+            prepared_request.body = req.body
+            prepared_request.headers = req.headers.copy()
+            prepared_request.hooks = req.hooks
 
             resp.content  # Consume socket so it can be released
 
@@ -102,29 +88,36 @@ class SessionRedirectMixin(object):
             resp.close()
 
             url = resp.headers['location']
-            method = prepared_request.method
+            method = req.method
 
             # Handle redirection without scheme (see: RFC 1808 Section 4)
             if url.startswith('//'):
                 parsed_rurl = urlparse(resp.url)
                 url = '%s:%s' % (parsed_rurl.scheme, url)
 
+            # The scheme should be lower case...
+            if '://' in url:
+                scheme, uri = url.split('://', 1)
+                url = '%s://%s' % (scheme.lower(), uri)
+
             # Facilitate non-RFC2616-compliant 'location' headers
             # (e.g. '/path/to/resource' instead of 'http://domain.tld/path/to/resource')
+            # Compliant with RFC3986, we percent encode the url.
             if not urlparse(url).netloc:
-                # Compliant with RFC3986, we percent encode the url.
                 url = urljoin(resp.url, requote_uri(url))
+            else:
+                url = requote_uri(url)
 
             prepared_request.url = url
 
             # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.4
             if (resp.status_code == codes.see_other and
-                    prepared_request.method != 'HEAD'):
+                    method != 'HEAD'):
                 method = 'GET'
 
             # Do what the browsers do, despite standards...
             if (resp.status_code in (codes.moved, codes.found) and
-                    prepared_request.method not in ('GET', 'HEAD')):
+                    method not in ('GET', 'HEAD')):
                 method = 'GET'
 
             prepared_request.method = method
@@ -218,7 +211,10 @@ class Session(SessionRedirectMixin):
         #: Should we trust the environment?
         self.trust_env = True
 
-        # Set up a CookieJar to be used by default
+        #: A CookieJar containing all currently outstanding cookies set on this
+        #: session. By default it is a
+        #: :class:`RequestsCookieJar <requests.cookies.RequestsCookieJar>`, but
+        #: may be any other ``cookielib.CookieJar`` compatible object.
         self.cookies = cookiejar_from_dict({})
 
         # Default connection adapters.
@@ -232,6 +228,46 @@ class Session(SessionRedirectMixin):
     def __exit__(self, *args):
         self.close()
 
+    def prepare_request(self, request):
+        """Constructs a :class:`PreparedRequest <PreparedRequest>` for
+        transmission and returns it. The :class:`PreparedRequest` has settings
+        merged from the :class:`Request <Request>` instance and those of the
+        :class:`Session`.
+
+        :param request: :class:`Request` instance to prepare with this
+        session's settings.
+        """
+        cookies = request.cookies or {}
+
+        # Bootstrap CookieJar.
+        if not isinstance(cookies, cookielib.CookieJar):
+            cookies = cookiejar_from_dict(cookies)
+
+        # Merge with session cookies
+        merged_cookies = RequestsCookieJar()
+        merged_cookies.update(self.cookies)
+        merged_cookies.update(cookies)
+
+
+        # Set environment's basic authentication if not explicitly set.
+        auth = request.auth
+        if self.trust_env and not auth and not self.auth:
+            auth = get_netrc_auth(request.url)
+
+        p = PreparedRequest()
+        p.prepare(
+            method=request.method.upper(),
+            url=request.url,
+            files=request.files,
+            data=request.data,
+            headers=merge_setting(request.headers, self.headers, dict_class=CaseInsensitiveDict),
+            params=merge_setting(request.params, self.params),
+            auth=merge_setting(auth, self.auth),
+            cookies=merged_cookies,
+            hooks=merge_setting(request.hooks, self.hooks),
+        )
+        return p
+
     def request(self, method, url,
         params=None,
         data=None,
@@ -275,20 +311,22 @@ class Session(SessionRedirectMixin):
         :param cert: (optional) if String, path to ssl client cert file (.pem).
             If Tuple, ('cert', 'key') pair.
         """
+        # Create the Request.
+        req = Request(
+            method = method.upper(),
+            url = url,
+            headers = headers,
+            files = files,
+            data = data or {},
+            params = params or {},
+            auth = auth,
+            cookies = cookies,
+            hooks = hooks,
+        )
+        prep = self.prepare_request(req)
 
-        cookies = cookies or {}
         proxies = proxies or {}
 
-        # Bootstrap CookieJar.
-        if not isinstance(cookies, cookielib.CookieJar):
-            cookies = cookiejar_from_dict(cookies)
-
-        # Merge with session cookies
-        merged_cookies = RequestsCookieJar()
-        merged_cookies.update(self.cookies)
-        merged_cookies.update(cookies)
-        cookies = merged_cookies
-
         # Gather clues from the surrounding environment.
         if self.trust_env:
             # Set environment's proxies.
@@ -296,10 +334,6 @@ class Session(SessionRedirectMixin):
             for (k, v) in env_proxies.items():
                 proxies.setdefault(k, v)
 
-            # Set environment's basic authentication.
-            if not auth:
-                auth = get_netrc_auth(url)
-
             # Look for configuration.
             if not verify and verify is not False:
                 verify = os.environ.get('REQUESTS_CA_BUNDLE')
@@ -309,29 +343,10 @@ class Session(SessionRedirectMixin):
                 verify = os.environ.get('CURL_CA_BUNDLE')
 
         # Merge all the kwargs.
-        params = merge_kwargs(params, self.params)
-        headers = merge_kwargs(headers, self.headers)
-        auth = merge_kwargs(auth, self.auth)
-        proxies = merge_kwargs(proxies, self.proxies)
-        hooks = merge_kwargs(hooks, self.hooks)
-        stream = merge_kwargs(stream, self.stream)
-        verify = merge_kwargs(verify, self.verify)
-        cert = merge_kwargs(cert, self.cert)
-
-        # Create the Request.
-        req = Request()
-        req.method = method.upper()
-        req.url = url
-        req.headers = headers
-        req.files = files
-        req.data = data
-        req.params = params
-        req.auth = auth
-        req.cookies = cookies
-        req.hooks = hooks
-
-        # Prepare the Request.
-        prep = req.prepare()
+        proxies = merge_setting(proxies, self.proxies)
+        stream = merge_setting(stream, self.stream)
+        verify = merge_setting(verify, self.verify)
+        cert = merge_setting(cert, self.cert)
 
         # Send the request.
         send_kwargs = {
@@ -426,7 +441,7 @@ class Session(SessionRedirectMixin):
 
         # It's possible that users might accidentally send a Request object.
         # Guard against that specific failure case.
-        if getattr(request, 'prepare', None):
+        if not isinstance(request, PreparedRequest):
             raise ValueError('You can only send PreparedRequests.')
 
         # Set up variables needed for resolve_redirects and dispatching of
@@ -477,7 +492,7 @@ class Session(SessionRedirectMixin):
         """Returns the appropriate connnection adapter for the given URL."""
         for (prefix, adapter) in self.adapters.items():
 
-            if url.startswith(prefix):
+            if url.lower().startswith(prefix):
                 return adapter
 
         # Nothing matches :-/

requests/status_codes.py

@@ -18,7 +18,8 @@ _codes = {
     205: ('reset_content', 'reset'),
     206: ('partial_content', 'partial'),
     207: ('multi_status', 'multiple_status', 'multi_stati', 'multiple_stati'),
-    208: ('im_used',),
+    208: ('already_reported',),
+    226: ('im_used',),
 
     # Redirection.
     300: ('multiple_choices',),

requests/structures.py

@@ -103,7 +103,7 @@ class CaseInsensitiveDict(collections.MutableMapping):
 
     # Copy is required
     def copy(self):
-         return CaseInsensitiveDict(self._store.values())
+        return CaseInsensitiveDict(self._store.values())
 
     def __repr__(self):
         return '%s(%r)' % (self.__class__.__name__, dict(self.items()))

requests/utils.py

@@ -11,17 +11,18 @@ that are also useful for external consumption.
 
 import cgi
 import codecs
+import collections
 import os
 import platform
 import re
 import sys
-import zlib
 from netrc import netrc, NetrcParseError
 
 from . import __version__
 from . import certs
 from .compat import parse_http_list as _parse_list_header
 from .compat import quote, urlparse, bytes, str, OrderedDict, urlunparse
+from .compat import getproxies, proxy_bypass
 from .cookies import RequestsCookieJar, cookiejar_from_dict
 from .structures import CaseInsensitiveDict
 
@@ -135,7 +136,7 @@ def to_key_val_list(value):
     if isinstance(value, (str, bytes, bool, int)):
         raise ValueError('cannot encode objects that are not 2-tuples')
 
-    if isinstance(value, dict):
+    if isinstance(value, collections.Mapping):
         value = value.items()
 
     return list(value)
@@ -301,7 +302,7 @@ def stream_decode_response_unicode(iterator, r):
         rv = decoder.decode(chunk)
         if rv:
             yield rv
-    rv = decoder.decode('', final=True)
+    rv = decoder.decode(b'', final=True)
     if rv:
         yield rv
 
@@ -386,37 +387,34 @@ def requote_uri(uri):
 def get_environ_proxies(url):
     """Return a dict of environment proxies."""
 
-    proxy_keys = [
-        'all',
-        'http',
-        'https',
-        'ftp',
-        'socks'
-    ]
-
     get_proxy = lambda k: os.environ.get(k) or os.environ.get(k.upper())
 
     # First check whether no_proxy is defined. If it is, check that the URL
     # we're getting isn't in the no_proxy list.
     no_proxy = get_proxy('no_proxy')
-
+    netloc = urlparse(url).netloc
+    
     if no_proxy:
         # We need to check whether we match here. We need to see if we match
         # the end of the netloc, both with and without the port.
         no_proxy = no_proxy.split(',')
-        netloc = urlparse(url).netloc
-
+        
         for host in no_proxy:
             if netloc.endswith(host) or netloc.split(':')[0].endswith(host):
                 # The URL does match something in no_proxy, so we don't want
                 # to apply the proxies on this URL.
                 return {}
+                
+    # If the system proxy settings indicate that this URL should be bypassed,
+    # don't proxy.
+    if proxy_bypass(netloc):
+        return {}
 
     # If we get here, we either didn't have no_proxy set or we're not going
-    # anywhere that no_proxy applies to.
-    proxies = [(key, get_proxy(key + '_proxy')) for key in proxy_keys]
-    return dict([(key, val) for (key, val) in proxies if val])
-
+    # anywhere that no_proxy applies to, and the system settings don't require
+    # bypassing the proxy for the current URL.
+    return getproxies()
+    
 
 def default_user_agent():
     """Return a string representing the default user agent."""

setup.py

@@ -40,7 +40,6 @@ setup(
     package_dir={'requests': 'requests'},
     include_package_data=True,
     install_requires=requires,
-    setup_requires=['sphinx'],
     license=open('LICENSE').read(),
     zip_safe=False,
     classifiers=(

test_requests.py

@@ -12,8 +12,9 @@ import pickle
 import requests
 from requests.auth import HTTPDigestAuth
 from requests.adapters import HTTPAdapter
-from requests.compat import str, cookielib
+from requests.compat import str, cookielib, getproxies, urljoin, urlparse
 from requests.cookies import cookiejar_from_dict
+from requests.exceptions import InvalidURL, MissingSchema
 from requests.structures import CaseInsensitiveDict
 
 try:
@@ -22,11 +23,13 @@ except ImportError:
     import io as StringIO
 
 HTTPBIN = os.environ.get('HTTPBIN_URL', 'http://httpbin.org/')
+# Issue #1483: Make sure the URL always has a trailing slash
+HTTPBIN = HTTPBIN.rstrip('/') + '/'
 
 
 def httpbin(*suffix):
     """Returns url for HTTPBIN resource."""
-    return HTTPBIN + '/'.join(suffix)
+    return urljoin(HTTPBIN, '/'.join(suffix))
 
 
 class RequestsTestCase(unittest.TestCase):
@@ -53,7 +56,8 @@ class RequestsTestCase(unittest.TestCase):
         requests.post
 
     def test_invalid_url(self):
-        self.assertRaises(ValueError, requests.get, 'hiwpefhipowhefopw')
+        self.assertRaises(MissingSchema, requests.get, 'hiwpefhipowhefopw')
+        self.assertRaises(InvalidURL, requests.get, 'http://')
 
     def test_basic_building(self):
         req = requests.Request()
@@ -85,9 +89,23 @@ class RequestsTestCase(unittest.TestCase):
         self.assertEqual(request.url,
             "http://example.com/path?key=value&a=b#fragment")
 
+    def test_mixed_case_scheme_acceptable(self):
+        s = requests.Session()
+        s.proxies = getproxies()
+        parts = urlparse(httpbin('get'))
+        schemes = ['http://', 'HTTP://', 'hTTp://', 'HttP://',
+                   'https://', 'HTTPS://', 'hTTps://', 'HttPs://']
+        for scheme in schemes:
+            url = scheme + parts.netloc + parts.path
+            r = requests.Request('GET', url)
+            r = s.send(r.prepare())
+            self.assertEqual(r.status_code, 200,
+                             "failed for scheme %s" % scheme)
+
     def test_HTTP_200_OK_GET_ALTERNATIVE(self):
         r = requests.Request('GET', httpbin('get'))
         s = requests.Session()
+        s.proxies = getproxies()
 
         r = s.send(r.prepare())
 
@@ -140,6 +158,11 @@ class RequestsTestCase(unittest.TestCase):
         )
         assert 'foo' not in s.cookies
 
+    def test_cookie_quote_wrapped(self):
+        s = requests.session()
+        s.get(httpbin('cookies/set?foo="bar:baz"'))
+        self.assertTrue(s.cookies['foo'] == '"bar:baz"')
+
     def test_request_cookie_overrides_session_cookie(self):
         s = requests.session()
         s.cookies['foo'] = 'bar'
@@ -158,7 +181,13 @@ class RequestsTestCase(unittest.TestCase):
         assert r.json()['cookies']['foo'] == 'bar'
         # Make sure the session cj is still the custom one
         assert s.cookies is cj
-
+    
+    def test_requests_in_history_are_not_overridden(self):
+        resp = requests.get(httpbin('redirect/3'))
+        urls = [r.url for r in resp.history]
+        req_urls = [r.request.url for r in resp.history]
+        self.assertEquals(urls, req_urls)
+        
     def test_user_agent_transfers(self):
 
         heads = {
@@ -198,6 +227,34 @@ class RequestsTestCase(unittest.TestCase):
         r = s.get(url)
         self.assertEqual(r.status_code, 200)
 
+    def test_basicauth_with_netrc(self):
+        auth = ('user', 'pass')
+        wrong_auth = ('wronguser', 'wrongpass')
+        url = httpbin('basic-auth', 'user', 'pass')
+
+        def get_netrc_auth_mock(url):
+            return auth
+        requests.sessions.get_netrc_auth = get_netrc_auth_mock
+
+        # Should use netrc and work.
+        r = requests.get(url)
+        self.assertEqual(r.status_code, 200)
+
+        # Given auth should override and fail.
+        r = requests.get(url, auth=wrong_auth)
+        self.assertEqual(r.status_code, 401)
+
+        s = requests.session()
+        
+        # Should use netrc and work.
+        r = s.get(url)
+        self.assertEqual(r.status_code, 200)
+
+        # Given auth should override and fail.
+        s.auth = wrong_auth
+        r = s.get(url)
+        self.assertEqual(r.status_code, 401)
+
     def test_DIGEST_HTTP_200_OK_GET(self):
 
         auth = HTTPDigestAuth('user', 'pass')
@@ -342,6 +399,17 @@ class RequestsTestCase(unittest.TestCase):
                           files={'file': ('test_requests.py', open(__file__, 'rb'))})
         self.assertEqual(r.status_code, 200)
 
+    def test_unicode_multipart_post_fieldnames(self):
+        filename = os.path.splitext(__file__)[0] + '.py'
+        r = requests.Request(method='POST',
+                             url=httpbin('post'),
+                             data={'stuff'.encode('utf-8'): 'elixr'},
+                             files={'file': ('test_requests.py',
+                                             open(filename, 'rb'))})
+        prep = r.prepare()
+        self.assertTrue(b'name="stuff"' in prep.body)
+        self.assertFalse(b'name="b\'stuff\'"' in prep.body)
+
     def test_custom_content_type(self):
         r = requests.post(httpbin('post'),
                           data={'stuff': json.dumps({'a': 123})},
@@ -367,10 +435,28 @@ class RequestsTestCase(unittest.TestCase):
         prep = req.prepare()
 
         s = requests.Session()
+        s.proxies = getproxies()
         resp = s.send(prep)
 
         self.assertTrue(hasattr(resp, 'hook_working'))
 
+    def test_prepared_from_session(self):
+        class DummyAuth(requests.auth.AuthBase):
+            def __call__(self, r):
+                r.headers['Dummy-Auth-Test'] = 'dummy-auth-test-ok'
+                return r
+
+        req = requests.Request('GET', httpbin('headers'))
+        self.assertEqual(req.auth, None)
+
+        s = requests.Session()
+        s.auth = DummyAuth()
+
+        prep = s.prepare_request(req)
+        resp = s.send(prep)
+
+        self.assertTrue(resp.json()['headers']['Dummy-Auth-Test'], 'dummy-auth-test-ok')
+
     def test_links(self):
         r = requests.Response()
         r.headers = {
@@ -456,6 +542,7 @@ class RequestsTestCase(unittest.TestCase):
         s = requests.Session()
 
         s = pickle.loads(pickle.dumps(s))
+        s.proxies = getproxies()
 
         r = s.send(r.prepare())
         self.assertEqual(r.status_code, 200)
@@ -483,6 +570,13 @@ class RequestsTestCase(unittest.TestCase):
             'application/json'
         )
 
+    def test_uppercase_scheme_redirect(self):
+        parts = urlparse(httpbin('html'))
+        url = "HTTP://" + parts.netloc + parts.path
+        r = requests.get(httpbin('redirect-to'), params={'url': url})
+        self.assertEqual(r.status_code, 200)
+        self.assertEqual(r.url.lower(), url.lower())
+
     def test_transport_adapter_ordering(self):
         s = requests.Session()
         order = ['https://', 'http://']
@@ -521,6 +615,20 @@ class RequestsTestCase(unittest.TestCase):
         self.assertTrue('http://' in s2.adapters)
         self.assertTrue('https://' in s2.adapters)
 
+    def test_header_remove_is_case_insensitive(self):
+        # From issue #1321
+        s = requests.Session()
+        s.headers['foo'] = 'bar'
+        r = s.get(httpbin('get'), headers={'FOO': None})
+        assert 'foo' not in r.request.headers
+
+    def test_params_are_merged_case_sensitive(self):
+        s = requests.Session()
+        s.params['foo'] = 'bar'
+        r = s.get(httpbin('get'), params={'FOO': 'bar'})
+        assert r.json()['args'] == {'foo': 'bar', 'FOO': 'bar'}
+
+
     def test_long_authinfo_in_url(self):
         url = 'http://{0}:{1}@{2}:9000/path?query#frag'.format(
             'E8A3BE87-9E3F-4620-8858-95478E385B5B',