Compare commits

..

2 Commits

Author SHA1 Message Date
Kenneth Reitz
850c0bed7f v2.17.2 2017-05-29 17:33:16 -04:00
Kenneth Reitz
48684d3705 2.17.1 2017-05-29 17:32:43 -04:00
151 changed files with 7416 additions and 11723 deletions

View File

@@ -1,2 +1,2 @@
[run]
omit = requests/packages/*
omit = requests/packages/*

View File

@@ -1,5 +0,0 @@
# You can configure git to automatically use this file with the following config:
# git config --global blame.ignoreRevsFile .git-blame-ignore-revs
# Add automatic code formatting to Requests
2a6f290bc09324406708a4d404a88a45d848ddf9

81
.github/AI_POLICY.md vendored
View File

@@ -1,81 +0,0 @@
# Generative AI / LLM Policy
We appreciate that we can't realistically police how you author your pull requests, which includes whether you employ large-language model (LLM)-based development tools.
So, we don't.
However, due to both legal and human reasons, we have to establish boundaries.
> [!CAUTION]
> **TL;DR:**
> - We take the responsibility for this project very seriously and we expect you to take your responsibility for your contributions seriously, too.
> This used to be a given, but it changed now that a pull request is just one prompt away.
>
> - Every contribution has to be backed by a human who unequivocally owns the copyright for all changes.
> No LLM bots in `Co-authored-by:`s.
>
> - DoS-by-slop leads to a permanent ban.
>
> - Absolutely **no** unsupervised agentic tools like OpenClaw.
>
> ---
>
> By submitting a pull request, you certify that:
>
> - You are the author of the contribution or have the legal right to submit it.
> - You either hold the copyright to the changes or have explicit legal authorization to contribute them under this project's license.
> - You understand the code.
> - You accept full responsibility for it.
## Legal
There is ongoing legal uncertainty regarding the copyright status of LLM-generated works and their provenance.
Since we do not have a formal [Contributor License Agreement](https://en.wikipedia.org/wiki/Contributor_license_agreement) (CLA), you retain your copyright to your changes to this project.
Therefore, allowing contributions by LLMs has unpredictable consequences for the copyright status of this project even when leaving aside possible copyright violations due to plagiarism.
## Human
As the makers of software that is used by millions of people worldwide and with a reputation for high-quality maintenance, we take our responsibility to our users very seriously.
No matter what LLM vendors or boosters on LinkedIn tell you, we have to manually review every change before merging, because it's **our responsibility** to keep the project stable.
Please understand that by opening low-quality pull requests you're not helping anyone.
Worse, you're [poisoning the open source ecosystem](https://lwn.net/Articles/1058266/) that was precarious even before the arrival of LLM tools.
Having to wade through plausible-looking-but-low-quality pull requests and trying to determine which ones are legit is extremely demoralizing and has already burned out many good maintainers.
Put bluntly, we have no time or interest to become part of your vibe coding loop where you drop LLM slop at our door, we spend time and energy to review it, and you just feed it back into the LLM for another iteration.
This dynamic is especially pernicious because it poisons the well for mentoring new contributors which we are committed to.
## Summary
In practice, this means:
- Pull requests that have an LLM product listed as co-author can't be merged and will be closed without further discussion.
We cannot risk the copyright status of this project.
If you used LLM tools during development, you may still submit but you must remove any LLM co-author tags and take full ownership of every line.
- By submitting a pull request, **you** take full **technical and legal** responsibility for the contents of the pull request and promise that **you** hold the copyright for the changes submitted.
"An LLM wrote it" is **not** an acceptable response to questions or critique.
**If you cannot explain and defend the changes you submit, do not submit them** and open a high-quality bug report/feature request instead.
- Accounts that exercise bot-like behavior like automated mass pull requests will be permanently banned, whether they belong to a human or not.
- Do **not** post LLM-generated review comments we can prompt LLMs ourselves should we desire their wisdom.
Do **not** post summaries unless you've fact-checked them and take responsibility for 100% of their content.
Remember that *all* LLM output *looks* **plausible**.
When using these tools, it's **your** responsibility to ensure that it's also **correct** and has a reasonable signal-to-noise ratio.
---
# Credits, Attribution
The original version of this can be found in [pyca/service-identity][] and is
used with permission from Hynek Schlawack.
[pyca/service-identity]:
https://github.com/pyca/service-identity/blob/fa91bf55cfda64145aa3d202cc84059befb98af4/.github/AI_POLICY.md

8
.github/CODEOWNERS vendored
View File

@@ -1,8 +0,0 @@
# Restrict all files related to deploying to
# require lead maintainer approval.
.github/workflows/ @nateprewitt @sigmavirus24
.github/CODEOWNERS @nateprewitt @sigmavirus24
src/requests/__version__.py @nateprewitt @sigmavirus24
HISTORY.md @nateprewitt @sigmavirus24
pyproject.toml @nateprewitt @sigmavirus24

View File

@@ -1,6 +0,0 @@
# Treat each other well
Everyone participating in the _requests_ project, and in particular in the issue tracker,
pull requests, and social media activity, is expected to treat other people with respect
and more generally to follow the guidelines articulated in the
[Python Community Code of Conduct](https://www.python.org/psf/codeofconduct/).

1
.github/FUNDING.yml vendored
View File

@@ -1 +0,0 @@
custom: ['https://www.python.org/psf/sponsorship/']

View File

@@ -25,4 +25,4 @@ import requests
This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).
operating system, &c).

View File

@@ -1,36 +0,0 @@
---
name: Bug report
about: Create a report to help us improve
---
<!-- Summary. -->
## Expected Result
<!-- What you expected. -->
## Actual Result
<!-- What happened instead. -->
## Reproduction Steps
```python
import requests
```
## System Information
$ python -m requests.help
```json
{
"paste": "here"
}
```
<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->

View File

@@ -1,10 +0,0 @@
---
name: Request for Help
about: Guidance on using Requests.
labels:
- "Question/Not a bug"
- "actions/autoclose-qa"
---
Please refer to our [Stack Overflow tag](https://stackoverflow.com/questions/tagged/python-requests) for guidance.

View File

@@ -1,10 +0,0 @@
---
name: Feature request
about: Suggest an idea for this project
labels:
- "Feature Request"
- "actions/autoclose-feat"
---
Requests is not accepting feature requests at this time.

84
.github/SECURITY.md vendored
View File

@@ -1,84 +0,0 @@
# Vulnerability Disclosure
If you think you have found a potential security vulnerability in
requests, please open a [draft Security Advisory](https://github.com/psf/requests/security/advisories/new)
via GitHub. We will coordinate verification and next steps through
that secure medium.
If English is not your first language, please try to describe the
problem and its impact to the best of your ability. For greater detail,
please use your native language and we will try our best to translate it
using online services.
Please also include the code you used to find the problem and the
shortest amount of code necessary to reproduce it.
Please do not disclose this to anyone else. We will retrieve a CVE
identifier if necessary and give you full credit under whatever name or
alias you provide. We will only request an identifier when we have a fix
and can publish it in a release.
We will respect your privacy and will only publicize your involvement if
you grant us permission.
## Process
This following information discusses the process the requests project
follows in response to vulnerability disclosures. If you are disclosing
a vulnerability, this section of the documentation lets you know how we
will respond to your disclosure.
### Timeline
When you report an issue, one of the project members will respond to you
within two days *at the outside*. In most cases responses will be
faster, usually within 12 hours. This initial response will at the very
least confirm receipt of the report.
If we were able to rapidly reproduce the issue, the initial response
will also contain confirmation of the issue. If we are not, we will
often ask for more information about the reproduction scenario.
Our goal is to have a fix for any vulnerability released within two
weeks of the initial disclosure. This may potentially involve shipping
an interim release that simply disables function while a more mature fix
can be prepared, but will in the vast majority of cases mean shipping a
complete release as soon as possible.
Throughout the fix process we will keep you up to speed with how the fix
is progressing. Once the fix is prepared, we will notify you that we
believe we have a fix. Often we will ask you to confirm the fix resolves
the problem in your environment, especially if we are not confident of
our reproduction scenario.
At this point, we will prepare for the release. We will obtain a CVE
number if one is required, providing you with full credit for the
discovery. We will also decide on a planned release date, and let you
know when it is. This release date will *always* be on a weekday.
At this point we will reach out to our major downstream packagers to
notify them of an impending security-related patch so they can make
arrangements. In addition, these packagers will be provided with the
intended patch ahead of time, to ensure that they are able to promptly
release their downstream packages. Currently the list of people we
actively contact *ahead of a public release* is:
- Python Maintenance Team, Red Hat (python-maint@redhat.com)
- Daniele Tricoli, Debian (@eriol)
We will notify these individuals at least a week ahead of our planned
release date to ensure that they have sufficient time to prepare. If you
believe you should be on this list, please let one of the maintainers
know at one of the email addresses at the top of this article.
On release day, we will push the patch to our public repository, along
with an updated changelog that describes the issue and credits you. We
will then issue a PyPI release containing the patch.
At this point, we will publicise the release. This will involve mails to
mailing lists, Tweets, and all other communication mechanisms available
to the core team.
We will also explicitly mention which commits contain the fix to make it
easier for other distributors and users to easily patch their own
versions of requests if upgrading is not an option.

View File

@@ -1,30 +0,0 @@
version: 2
updates:
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
ignore:
# Ignore all patch releases as we can manually
# upgrade if we run into a bug and need a fix.
- dependency-name: "*"
update-types: ["version-update:semver-patch"]
cooldown:
default-days: 7
groups:
actions:
patterns:
- "*"
- package-ecosystem: "pre-commit"
directory: "/"
schedule:
interval: "weekly"
ignore:
- dependency-name: "*"
update-types: ["version-update:semver-patch"]
cooldown:
default-days: 7
groups:
pre-commit:
patterns:
- "*"

View File

@@ -1,38 +0,0 @@
name: 'Autoclose Issues'
on:
issues:
types:
- labeled
permissions: {}
jobs:
close_qa:
permissions:
issues: write
if: github.event.label.name == 'actions/autoclose-qa'
runs-on: ubuntu-latest
steps:
- env:
ISSUE_URL: ${{ github.event.issue.html_url }}
GH_TOKEN: ${{ github.token }}
run: |
gh issue close $ISSUE_URL \
--comment "As described in the template, we won't be able to answer questions on this issue tracker. Please use [Stack Overflow](https://stackoverflow.com/)" \
--reason completed
gh issue lock $ISSUE_URL --reason off_topic
close_feature_request:
permissions:
issues: write
if: github.event.label.name == 'actions/autoclose-feat'
runs-on: ubuntu-latest
steps:
- env:
ISSUE_URL: ${{ github.event.issue.html_url }}
GH_TOKEN: ${{ github.token }}
run: |
gh issue close $ISSUE_URL \
--comment "As described in the template, Requests is not accepting feature requests" \
--reason "not planned"
gh issue lock $ISSUE_URL --reason off_topic

View File

@@ -1,74 +0,0 @@
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
name: "CodeQL"
on:
push:
branches: [main]
pull_request:
# The branches below must be a subset of the branches above
branches: [main]
schedule:
- cron: '0 23 * * 0'
permissions:
contents: read
jobs:
analyze:
permissions:
actions: read # for github/codeql-action/init to get workflow details
contents: read # for actions/checkout to fetch code
security-events: write # for github/codeql-action/autobuild to send a status report
name: Analyze
runs-on: ubuntu-latest
strategy:
fail-fast: false
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
# We must fetch at least the immediate parents so that if this is
# a pull request then we can checkout the head.
fetch-depth: 2
persist-credentials: false
# If this run was triggered by a pull request event, then checkout
# the head of the pull request instead of the merge commit.
- run: git checkout HEAD^2
if: ${{ github.event_name == 'pull_request' }}
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@99df26d4f13ea111d4ec1a7dddef6063f76b97e9 # v4.37.0
with:
languages: "python"
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
# queries: ./path/to/local/query, your-org/your-repo/queries@main
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@99df26d4f13ea111d4ec1a7dddef6063f76b97e9 # v4.37.0
# Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl
# ✏️ If the Autobuild fails above, remove it and uncomment the following three lines
# and modify them (or add more) to build your code if your project
# uses a compiled language
#- run: |
# make bootstrap
# make release
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@99df26d4f13ea111d4ec1a7dddef6063f76b97e9 # v4.37.0

View File

@@ -1,24 +0,0 @@
name: Lint code
on: [push, pull_request]
permissions:
contents: read
jobs:
lint:
runs-on: ubuntu-24.04
timeout-minutes: 10
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.x"
- name: Run pre-commit
run: |
python -m pip install pre-commit==4.6.0
pre-commit run --show-diff-on-failure --color=always --all-files

View File

@@ -1,19 +0,0 @@
name: 'Lock Threads'
on:
schedule:
- cron: '0 0 * * *'
permissions:
issues: write
pull-requests: write
jobs:
action:
if: github.repository_owner == 'psf'
runs-on: ubuntu-latest
steps:
- uses: dessant/lock-threads@89ae32b08ed1a541efecbab17912962a5e38981c # v6.0.2
with:
issue-lock-inactive-days: 90
pr-lock-inactive-days: 90

View File

@@ -1,95 +0,0 @@
name: Publish to PyPI
on:
push:
tags:
- "v*"
workflow_dispatch:
inputs:
test-pypi-only:
description: "Publish to Test PyPI only"
type: boolean
default: true
permissions:
contents: read
jobs:
build:
name: "Build dists"
runs-on: "ubuntu-latest"
outputs:
artifact-id: ${{ steps.upload-artifact.outputs.artifact-id }}
steps:
- name: "Checkout repository"
uses: "actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0" # v7.0.0
with:
persist-credentials: false
- name: "Setup Python"
uses: "actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97" # v7.0.0
with:
python-version: "3.x"
- name: "Install dependencies"
run: python -m pip install build==1.4.0
- name: "Build dists"
run: |
SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct) \
python -m build
- name: "Upload dists"
uses: "actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f"
id: upload-artifact
with:
name: "dist"
path: "dist/"
if-no-files-found: error
retention-days: 5
publish:
name: "Publish"
if: startsWith(github.ref, 'refs/tags/')
needs: ["build"]
permissions:
id-token: write
runs-on: "ubuntu-latest"
environment:
name: "publish"
steps:
- name: "Download dists"
uses: "actions/download-artifact@70fc10c6e5e1ce46ad2ea6f2b72d43f7d47b13c3" # v8.0.0
with:
artifact-ids: ${{ needs.build.outputs.artifact-id }}
path: "dist/"
- name: "Publish dists to PyPI"
uses: "pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b" # v1.14.0
with:
attestations: true
publish-test-pypi:
name: "Publish to Test PyPI"
if: github.event_name == 'workflow_dispatch'
needs: ["build"]
permissions:
id-token: write
runs-on: "ubuntu-latest"
environment:
name: "testpypi"
steps:
- name: "Download dists"
uses: "actions/download-artifact@70fc10c6e5e1ce46ad2ea6f2b72d43f7d47b13c3" # v8.0.0
with:
artifact-ids: ${{ needs.build.outputs.artifact-id }}
path: "dist/"
- name: "Publish dists to Test PyPI"
uses: "pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b" # v1.14.0
with:
repository-url: https://test.pypi.org/legacy/
attestations: true

View File

@@ -1,83 +0,0 @@
name: Tests
on: [push, pull_request]
permissions:
contents: read
jobs:
build:
runs-on: ${{ matrix.os }}
timeout-minutes: 10
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14", "3.14t", "3.15-dev", "pypy-3.11"]
os: [ubuntu-22.04, macOS-latest, windows-latest]
# Pypy-3.11 can't install openssl-sys with rust
# which prevents us from testing in GHA.
exclude:
- { python-version: "pypy-3.11", os: "windows-latest" }
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
allow-prereleases: true
- name: Install dependencies
env:
PYO3_USE_ABI3_FORWARD_COMPATIBILITY: ${{ matrix.python-version == '3.15-dev' && '1' || '' }}
run: |
make
- name: Run tests
run: |
make ci
no_chardet:
name: "No Character Detection"
runs-on: ubuntu-latest
strategy:
fail-fast: true
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0
with:
persist-credentials: false
- name: 'Set up Python 3.10'
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97
with:
python-version: '3.10'
- name: Install dependencies
run: |
make
python -m pip uninstall -y "charset_normalizer" "chardet"
- name: Run tests
run: |
make ci
urllib3:
name: 'urllib3 1.x'
runs-on: 'ubuntu-latest'
strategy:
fail-fast: true
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0
with:
persist-credentials: false
- name: 'Set up Python 3.10'
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97
with:
python-version: '3.10'
- name: Install dependencies
run: |
make
python -m pip install "urllib3<2"
- name: Run tests
run: |
make ci

View File

@@ -1,32 +0,0 @@
name: Type Check
on: [push, pull_request]
permissions:
contents: read
jobs:
typecheck:
runs-on: ubuntu-24.04
timeout-minutes: 10
strategy:
matrix:
python-version: ["3.10", "3.14"]
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install pip==26.0.1
python -m pip install -e . --group typecheck
- name: Run pyright
run: python -m pyright src/requests/

View File

@@ -1,24 +0,0 @@
# Sourced from https://github.com/zizmorcore/zizmor-action
name: GitHub Actions Security Analysis with zizmor 🌈
on:
push:
branches: ["main"]
pull_request:
branches: ["**"]
permissions: {}
jobs:
zizmor:
runs-on: ubuntu-latest
permissions:
security-events: write
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- name: Run zizmor 🌈
uses: zizmorcore/zizmor-action@6599ee8b7a49aef6a770f63d261d214911a7ce02 # v0.6.0

15
.gitignore vendored
View File

@@ -5,7 +5,6 @@ nosetests.xml
junit-report.xml
pylint.txt
toy.py
.cache/
cover/
build/
docs/_build
@@ -14,24 +13,10 @@ requests.egg-info/
*.swp
*.egg
env/
.venv/
.eggs/
.tox/
.pytest_cache/
.vscode/
.eggs/
.workon
# in case you work with IntelliJ/PyCharm
.idea
*.iml
.python-version
t.py
t2.py
dist
/.mypy_cache/

View File

@@ -1,21 +0,0 @@
exclude: 'docs/|ext/'
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v6.0.0
hooks:
- id: check-case-conflict
- id: check-merge-conflict
- id: check-toml
- id: check-yaml
- id: end-of-file-fixer
- id: mixed-line-ending
args: [--fix=auto]
- id: trailing-whitespace
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.16.4
hooks:
- id: ruff-check
args: [--fix]
- id: ruff-format
exclude: tests/test_lowlevel.py

View File

@@ -1,29 +0,0 @@
# Read the Docs configuration file for Sphinx projects
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
# Required
version: 2
# Set the OS, Python version and other tools you might need
build:
os: ubuntu-22.04
tools:
python: "3.12"
# Build documentation in the "docs/" directory with Sphinx
sphinx:
configuration: docs/conf.py
builder: "dirhtml"
# Optionally build your docs in additional formats such as PDF and ePub
formats:
- pdf
- epub
# Optional but recommended, declare the Python requirements required
# to build your documentation
# See https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html
python:
install:
- path: .
- requirements: docs/requirements.txt

32
.travis.yml Normal file
View File

@@ -0,0 +1,32 @@
sudo: false
language: python
python:
- "2.6"
- "2.7"
- "3.3"
- "3.4"
- "3.5"
- "3.6"
# - "3.7-dev"
# - "pypy" -- appears to hang
# - "pypy3"
# command to install dependencies
install: "make"
# command to run tests
script:
- |
if [[ "$TRAVIS_PYTHON_VERSION" != "2.6" ]] ; then make test-readme; fi
- make ci
cache: pip
jobs:
include:
- stage: test
script:
- |
if [[ "$TRAVIS_PYTHON_VERSION" != "2.6" ]] ; then make test-readme; fi
- make ci
- stage: coverage
python: 3.6
script: codecov

43
AUTHORS.rst Normal file → Executable file
View File

@@ -1,17 +1,21 @@
Requests was lovingly created by Kenneth Reitz.
Requests is written and maintained by Kenneth Reitz and
various contributors:
Requests Maintainers
````````````````````
Keepers of the Four Crystals
````````````````````````````
- Ian Stapleton Cordasco <graffatcolmingov@gmail.com> `@sigmavirus24 <https://github.com/sigmavirus24>`_.
- Nate Prewitt `@nateprewitt <https://github.com/nateprewitt>`_.
Previous Maintainers
````````````````````
- Kenneth Reitz <me@kennethreitz.org> `@kennethreitz <https://github.com/kennethreitz>`_, reluctant Keeper of the Master Crystal.
- Kenneth Reitz <me@kennethreitz.org> `@kennethreitz <https://github.com/kennethreitz>`_, Keeper of the Master Crystal.
- Cory Benfield <cory@lukasa.co.uk> `@lukasa <https://github.com/lukasa>`_
- Seth M. Larson `@sethmlarson <https://github.com/sethmlarson>`_.
- Ian Cordasco <graffatcolmingov@gmail.com> `@sigmavirus24 <https://github.com/sigmavirus24>`_
- Nate Prewitt <nate.prewitt@gmail.com> `@nateprewitt <https://github.com/nateprewitt>`_
Urllib3
```````
- Andrey Petrov <andrey.petrov@shazow.net>
Patches and Suggestions
```````````````````````
@@ -124,7 +128,7 @@ Patches and Suggestions
- Bryce Boe <bbzbryce@gmail.com> (`@bboe <https://github.com/bboe>`_)
- Colin Dunklau <colin.dunklau@gmail.com> (`@cdunklau <https://github.com/cdunklau>`_)
- Bob Carroll <bob.carroll@alum.rit.edu> (`@rcarz <https://github.com/rcarz>`_)
- Hugo Osvaldo Barrera <hugo@barrera.io> (`@hobarrera <https://github.com/hobarrera>`_)
- Hugo Osvaldo Barrera <hugo@osvaldobarrera.com.ar> (`@hobarrera <https://github.com/hobarrera>`_)
- Łukasz Langa <lukasz@langa.pl>
- Dave Shawley <daveshawley@gmail.com>
- James Clarke (`@jam <https://github.com/jam>`_)
@@ -178,18 +182,3 @@ Patches and Suggestions
- David Fontenot (`@davidfontenot <https://github.com/davidfontenot>`_)
- Shmuel Amar (`@shmuelamar <https://github.com/shmuelamar>`_)
- Gary Wu (`@garywu <https://github.com/garywu>`_)
- Ryan Pineo (`@ryanpineo <https://github.com/ryanpineo>`_)
- Ed Morley (`@edmorley <https://github.com/edmorley>`_)
- Matt Liu <liumatt@gmail.com> (`@mlcrazy <https://github.com/mlcrazy>`_)
- Taylor Hoff <primdevs@protonmail.com> (`@PrimordialHelios <https://github.com/PrimordialHelios>`_)
- Arthur Vigil (`@ahvigil <https://github.com/ahvigil>`_)
- Nehal J Wani (`@nehaljwani <https://github.com/nehaljwani>`_)
- Demetrios Bairaktaris (`@DemetriosBairaktaris <https://github.com/demetriosbairaktaris>`_)
- Darren Dormer (`@ddormer <https://github.com/ddormer>`_)
- Rajiv Mayani (`@mayani <https://github.com/mayani>`_)
- Antti Kaihola (`@akaihola <https://github.com/akaihola>`_)
- "Dull Bananas" <dull.bananas0@gmail.com> (`@dullbananas <https://github.com/dullbananas>`_)
- Alessio Izzo (`@aless10 <https://github.com/aless10>`_)
- Sylvain Marié (`@smarie <https://github.com/smarie>`_)
- Hod Bin Noon (`@hodbn <https://github.com/hodbn>`_)
- Mike Fiedler (`@miketheman <https://github.com/miketheman>`_)

3
CODE_OF_CONDUCT.md Normal file
View File

@@ -0,0 +1,3 @@
Be cordial or be on your way.
https://www.kennethreitz.org/essays/be-cordial-or-be-on-your-way

View File

@@ -1,8 +1,10 @@
# Contribution Guidelines
Before opening any issues or proposing any pull requests, please read
our [Contributor's Guide](https://requests.readthedocs.io/en/latest/dev/contributing/)
as well as our [AI Policy](/.github/AI_POLICY.md).
Before opening any issues or proposing any pull requests, please do the
following:
1. Read our [Contributor's Guide](http://docs.python-requests.org/en/latest/dev/contributing/).
2. Understand our [development philosophy](http://docs.python-requests.org/en/latest/dev/philosophy/).
To get the greatest chance of helpful responses, please also observe the
following additional notes.
@@ -53,14 +55,3 @@ Please be aware of the following things when filing bug reports:
If you do not provide all of these things, it will take us much longer to
fix your problem. If we ask you to clarify these and you never respond, we
will close your issue without fixing it.
## Good Pull Requests
1. Avoid raising duplicate pull requests. *Please* use the GitHub pull request
search feature to check whether someone else has already filed a pull
request.
2. When filing a pull request, always include tests that fail without your
changes thus preventing the fix from regressing.
3. Always run the test suite locally
4. Write commit messages that explain _why_ a change was made beyond "Fixes
#NNNN". Why is this the right fix? What other options were ruled out? etc.

2102
HISTORY.md

File diff suppressed because it is too large Load Diff

1462
HISTORY.rst Normal file

File diff suppressed because it is too large Load Diff

182
LICENSE
View File

@@ -1,175 +1,13 @@
Copyright 2017 Kenneth Reitz
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
http://www.apache.org/licenses/LICENSE-2.0
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

View File

@@ -1,3 +1,2 @@
include README.md LICENSE NOTICE HISTORY.md requirements-dev.txt
include README.rst LICENSE NOTICE HISTORY.rst pytest.ini
recursive-include tests *.py
recursive-include tests/certs *

View File

@@ -1,27 +1,27 @@
.PHONY: docs
init:
python -m pip install -r requirements-dev.txt
pip install -r requirements.txt
test:
python -m pytest tests
# This runs all of the tests, on both Python 2 and Python 3.
detox
ci:
python -m pytest tests --junitxml=report.xml
py.test -n 8 --boxed --junitxml=report.xml
test-readme:
python setup.py check --restructuredtext --strict && ([ $$? -eq 0 ] && echo "README.rst and HISTORY.rst ok") || echo "Invalid markup in README.rst or HISTORY.rst!"
python setup.py check -r -s
flake8:
flake8 --ignore=E501 requests
coverage:
python -m pytest --cov-config .coveragerc --verbose --cov-report term --cov-report xml --cov=src/requests tests
py.test --cov-config .coveragerc --verbose --cov-report term --cov-report xml --cov=requests tests
.publishenv:
python -m venv .publishenv
.publishenv/bin/pip install 'twine>=1.5.0' build
publish: .publishenv
.publishenv/bin/python -m build
.publishenv/bin/python -m twine upload --skip-existing dist/*
publish:
pip install 'twine>=1.5.0'
python setup.py sdist bdist_wheel
twine upload dist/*
rm -fr build dist .egg requests.egg-info
docs:
cd docs && make html
@echo "\033[95m\n\nBuild successful! View the docs homepage at docs/_build/html/index.html.\n\033[0m"
@echo "\033[95m\n\nBuild successful! View the docs homepage at docs/_build/html/index.html.\n\033[0m"

2
NOTICE
View File

@@ -1,2 +0,0 @@
Requests
Copyright 2019 Kenneth Reitz

View File

@@ -1,76 +0,0 @@
# Requests
[![Version](https://img.shields.io/pypi/v/requests.svg?maxAge=86400)](https://pypi.org/project/requests/)
[![Supported Versions](https://img.shields.io/pypi/pyversions/requests.svg)](https://pypi.org/project/requests)
[![Downloads](https://static.pepy.tech/badge/requests/month)](https://pepy.tech/project/requests)
[![Contributors](https://img.shields.io/github/contributors/psf/requests.svg)](https://github.com/psf/requests/graphs/contributors)
[![Documentation](https://readthedocs.org/projects/requests/badge/?version=latest)](https://requests.readthedocs.io)
**Requests** is a simple, yet elegant, HTTP library.
```python
>>> import requests
>>> r = requests.get('https://httpbin.org/basic-auth/user/pass', auth=('user', 'pass'))
>>> r.status_code
200
>>> r.headers['content-type']
'application/json; charset=utf8'
>>> r.encoding
'utf-8'
>>> r.text
'{"authenticated": true, ...'
>>> r.json()
{'authenticated': True, ...}
```
Requests allows you to send HTTP/1.1 requests extremely easily. Theres no need to manually add query strings to your URLs, or to form-encode your `PUT` & `POST` data — but nowadays, just use the `json` method!
Requests is one of the most downloaded Python packages today, pulling in around `300M downloads / week` — according to GitHub, Requests is currently [depended upon](https://github.com/psf/requests/network/dependents?package_id=UGFja2FnZS01NzA4OTExNg%3D%3D) by `4,000,000+` repositories.
## Installing Requests and Supported Versions
Requests is available on PyPI:
```console
$ python -m pip install requests
```
Requests officially supports Python 3.10+.
## Supported Features & BestPractices
Requests is ready for the demands of building robust and reliable HTTPspeaking applications, for the needs of today.
- Keep-Alive & Connection Pooling
- International Domains and URLs
- Sessions with Cookie Persistence
- Browser-style TLS/SSL Verification
- Basic & Digest Authentication
- Familiar `dict`like Cookies
- Automatic Content Decompression and Decoding
- Multi-part File Uploads
- SOCKS Proxy Support
- Connection Timeouts
- Streaming Downloads
- Automatic honoring of `.netrc`
- Chunked HTTP Requests
## Cloning the repository
When cloning the Requests repository, you may need to add the `-c
fetch.fsck.badTimezone=ignore` flag to avoid an error about a bad commit timestamp (see
[this issue](https://github.com/psf/requests/issues/2690) for more background):
```shell
git clone -c fetch.fsck.badTimezone=ignore https://github.com/psf/requests.git
```
You can also apply this setting to your global Git config:
```shell
git config --global fetch.fsck.badTimezone ignore
```
---
[![Kenneth Reitz](https://raw.githubusercontent.com/psf/requests/main/ext/kr.png)](https://kennethreitz.org) [![Python Software Foundation](https://raw.githubusercontent.com/psf/requests/main/ext/psf.png)](https://www.python.org/psf)

114
README.rst Normal file
View File

@@ -0,0 +1,114 @@
Requests: HTTP for Humans
=========================
.. image:: https://img.shields.io/pypi/v/requests.svg
:target: https://pypi.python.org/pypi/requests
.. image:: https://img.shields.io/pypi/l/requests.svg
:target: https://pypi.python.org/pypi/requests
.. image:: https://img.shields.io/pypi/pyversions/requests.svg
:target: https://pypi.python.org/pypi/requests
.. image:: https://codecov.io/github/requests/requests/coverage.svg?branch=master
:target: https://codecov.io/github/requests/requests
:alt: codecov.io
.. image:: https://img.shields.io/github/contributors/requests/requests.svg
:target: https://github.com/requests/requests/graphs/contributors
.. image:: https://img.shields.io/badge/Say%20Thanks-!-1EAEDB.svg
:target: https://saythanks.io/to/kennethreitz
Requests is the only *Non-GMO* HTTP library for Python, safe for human
consumption.
**Warning:** Recreational use of the Python standard library for HTTP may result in dangerous side-effects,
including: security vulnerabilities, verbose code, reinventing the wheel,
constantly reading documentation, depression, headaches, or even death.
Behold, the power of Requests:
.. code-block:: python
>>> r = requests.get('https://api.github.com/user', auth=('user', 'pass'))
>>> r.status_code
200
>>> r.headers['content-type']
'application/json; charset=utf8'
>>> r.encoding
'utf-8'
>>> r.text
u'{"type":"User"...'
>>> r.json()
{u'disk_usage': 368627, u'private_gists': 484, ...}
See `the similar code, sans Requests <https://gist.github.com/973705>`_.
.. image:: https://raw.githubusercontent.com/requests/requests/master/docs/_static/requests-logo-small.png
:target: http://docs.python-requests.org/
Requests allows you to send *organic, grass-fed* HTTP/1.1 requests, without the
need for manual labor. There's no need to manually add query strings to your
URLs, or to form-encode your POST data. Keep-alive and HTTP connection pooling
are 100% automatic, thanks to `urllib3 <https://github.com/shazow/urllib3>`_.
Besides, all the cool kids are doing it. Requests is one of the most
downloaded Python packages of all time, pulling in over 11,000,000 downloads
every month. You don't want to be left out!
Feature Support
---------------
Requests is ready for today's web.
- International Domains and URLs
- Keep-Alive & Connection Pooling
- Sessions with Cookie Persistence
- Browser-style SSL Verification
- Basic/Digest Authentication
- Elegant Key/Value Cookies
- Automatic Decompression
- Automatic Content Decoding
- Unicode Response Bodies
- Multipart File Uploads
- HTTP(S) Proxy Support
- Connection Timeouts
- Streaming Downloads
- ``.netrc`` Support
- Chunked Requests
Requests officially supports Python 2.62.7 & 3.33.7, and runs great on PyPy.
Installation
------------
To install Requests, simply:
.. code-block:: bash
$ pip install requests
✨🍰✨
Satisfaction, guaranteed.
Documentation
-------------
Fantastic documentation is available at http://docs.python-requests.org/, for a limited time only.
How to Contribute
-----------------
#. Check for open issues or open a fresh issue to start a discussion around a feature idea or a bug. There is a `Contributor Friendly`_ tag for issues that should be ideal for people who are not very familiar with the codebase yet.
#. Fork `the repository`_ on GitHub to start making your changes to the **master** branch (or branch off of it).
#. Write a test which shows that the bug was fixed or that the feature works as expected.
#. Send a pull request and bug the maintainer until it gets merged and published. :) Make sure to add yourself to AUTHORS_.
.. _`the repository`: http://github.com/requests/requests
.. _AUTHORS: https://github.com/requests/requests/blob/master/AUTHORS.rst
.. _Contributor Friendly: https://github.com/requests/requests/issues?direction=desc&labels=Contributor+Friendly&page=1&sort=updated&state=open

229
_appveyor/install.ps1 Normal file
View File

@@ -0,0 +1,229 @@
# Sample script to install Python and pip under Windows
# Authors: Olivier Grisel, Jonathan Helmus, Kyle Kastner, and Alex Willmer
# License: CC0 1.0 Universal: http://creativecommons.org/publicdomain/zero/1.0/
$MINICONDA_URL = "http://repo.continuum.io/miniconda/"
$BASE_URL = "https://www.python.org/ftp/python/"
$GET_PIP_URL = "https://bootstrap.pypa.io/get-pip.py"
$GET_PIP_PATH = "C:\get-pip.py"
$PYTHON_PRERELEASE_REGEX = @"
(?x)
(?<major>\d+)
\.
(?<minor>\d+)
\.
(?<micro>\d+)
(?<prerelease>[a-z]{1,2}\d+)
"@
function Download ($filename, $url) {
$webclient = New-Object System.Net.WebClient
$basedir = $pwd.Path + "\"
$filepath = $basedir + $filename
if (Test-Path $filename) {
Write-Host "Reusing" $filepath
return $filepath
}
# Download and retry up to 3 times in case of network transient errors.
Write-Host "Downloading" $filename "from" $url
$retry_attempts = 2
for ($i = 0; $i -lt $retry_attempts; $i++) {
try {
$webclient.DownloadFile($url, $filepath)
break
}
Catch [Exception]{
Start-Sleep 1
}
}
if (Test-Path $filepath) {
Write-Host "File saved at" $filepath
} else {
# Retry once to get the error message if any at the last try
$webclient.DownloadFile($url, $filepath)
}
return $filepath
}
function ParsePythonVersion ($python_version) {
if ($python_version -match $PYTHON_PRERELEASE_REGEX) {
return ([int]$matches.major, [int]$matches.minor, [int]$matches.micro,
$matches.prerelease)
}
$version_obj = [version]$python_version
return ($version_obj.major, $version_obj.minor, $version_obj.build, "")
}
function DownloadPython ($python_version, $platform_suffix) {
$major, $minor, $micro, $prerelease = ParsePythonVersion $python_version
if (($major -le 2 -and $micro -eq 0) `
-or ($major -eq 3 -and $minor -le 2 -and $micro -eq 0) `
) {
$dir = "$major.$minor"
$python_version = "$major.$minor$prerelease"
} else {
$dir = "$major.$minor.$micro"
}
if ($prerelease) {
if (($major -le 2) `
-or ($major -eq 3 -and $minor -eq 1) `
-or ($major -eq 3 -and $minor -eq 2) `
-or ($major -eq 3 -and $minor -eq 3) `
) {
$dir = "$dir/prev"
}
}
if (($major -le 2) -or ($major -le 3 -and $minor -le 4)) {
$ext = "msi"
if ($platform_suffix) {
$platform_suffix = ".$platform_suffix"
}
} else {
$ext = "exe"
if ($platform_suffix) {
$platform_suffix = "-$platform_suffix"
}
}
$filename = "python-$python_version$platform_suffix.$ext"
$url = "$BASE_URL$dir/$filename"
$filepath = Download $filename $url
return $filepath
}
function InstallPython ($python_version, $architecture, $python_home) {
Write-Host "Installing Python" $python_version "for" $architecture "bit architecture to" $python_home
if (Test-Path $python_home) {
Write-Host $python_home "already exists, skipping."
return $false
}
if ($architecture -eq "32") {
$platform_suffix = ""
} else {
$platform_suffix = "amd64"
}
$installer_path = DownloadPython $python_version $platform_suffix
$installer_ext = [System.IO.Path]::GetExtension($installer_path)
Write-Host "Installing $installer_path to $python_home"
$install_log = $python_home + ".log"
if ($installer_ext -eq '.msi') {
InstallPythonMSI $installer_path $python_home $install_log
} else {
InstallPythonEXE $installer_path $python_home $install_log
}
if (Test-Path $python_home) {
Write-Host "Python $python_version ($architecture) installation complete"
} else {
Write-Host "Failed to install Python in $python_home"
Get-Content -Path $install_log
Exit 1
}
}
function InstallPythonEXE ($exepath, $python_home, $install_log) {
$install_args = "/quiet InstallAllUsers=1 TargetDir=$python_home"
RunCommand $exepath $install_args
}
function InstallPythonMSI ($msipath, $python_home, $install_log) {
$install_args = "/qn /log $install_log /i $msipath TARGETDIR=$python_home"
$uninstall_args = "/qn /x $msipath"
RunCommand "msiexec.exe" $install_args
if (-not(Test-Path $python_home)) {
Write-Host "Python seems to be installed else-where, reinstalling."
RunCommand "msiexec.exe" $uninstall_args
RunCommand "msiexec.exe" $install_args
}
}
function RunCommand ($command, $command_args) {
Write-Host $command $command_args
Start-Process -FilePath $command -ArgumentList $command_args -Wait -Passthru
}
function InstallPip ($python_home) {
$pip_path = $python_home + "\Scripts\pip.exe"
$python_path = $python_home + "\python.exe"
if (-not(Test-Path $pip_path)) {
Write-Host "Installing pip..."
$webclient = New-Object System.Net.WebClient
$webclient.DownloadFile($GET_PIP_URL, $GET_PIP_PATH)
Write-Host "Executing:" $python_path $GET_PIP_PATH
& $python_path $GET_PIP_PATH
} else {
Write-Host "pip already installed."
}
}
function DownloadMiniconda ($python_version, $platform_suffix) {
if ($python_version -eq "3.4") {
$filename = "Miniconda3-3.5.5-Windows-" + $platform_suffix + ".exe"
} else {
$filename = "Miniconda-3.5.5-Windows-" + $platform_suffix + ".exe"
}
$url = $MINICONDA_URL + $filename
$filepath = Download $filename $url
return $filepath
}
function InstallMiniconda ($python_version, $architecture, $python_home) {
Write-Host "Installing Python" $python_version "for" $architecture "bit architecture to" $python_home
if (Test-Path $python_home) {
Write-Host $python_home "already exists, skipping."
return $false
}
if ($architecture -eq "32") {
$platform_suffix = "x86"
} else {
$platform_suffix = "x86_64"
}
$filepath = DownloadMiniconda $python_version $platform_suffix
Write-Host "Installing" $filepath "to" $python_home
$install_log = $python_home + ".log"
$args = "/S /D=$python_home"
Write-Host $filepath $args
Start-Process -FilePath $filepath -ArgumentList $args -Wait -Passthru
if (Test-Path $python_home) {
Write-Host "Python $python_version ($architecture) installation complete"
} else {
Write-Host "Failed to install Python in $python_home"
Get-Content -Path $install_log
Exit 1
}
}
function InstallMinicondaPip ($python_home) {
$pip_path = $python_home + "\Scripts\pip.exe"
$conda_path = $python_home + "\Scripts\conda.exe"
if (-not(Test-Path $pip_path)) {
Write-Host "Installing pip..."
$args = "install --yes pip"
Write-Host $conda_path $args
Start-Process -FilePath "$conda_path" -ArgumentList $args -Wait -Passthru
} else {
Write-Host "pip already installed."
}
}
function main () {
InstallPython $env:PYTHON_VERSION $env:PYTHON_ARCH $env:PYTHON
InstallPip $env:PYTHON
}
main

56
appveyor.yml Normal file
View File

@@ -0,0 +1,56 @@
# AppVeyor.yml from https://github.com/ogrisel/python-appveyor-demo
# License: CC0 1.0 Universal: http://creativecommons.org/publicdomain/zero/1.0/
build: off
environment:
matrix:
- PYTHON: "C:\\Python27-x64"
PYTHON_VERSION: "2.7.x"
PYTHON_ARCH: "64"
TOXENV: "py27"
- PYTHON: "C:\\Python33-x64"
PYTHON_VERSION: "3.3.x"
PYTHON_ARCH: "64"
TOXENV: "py33"
- PYTHON: "C:\\Python34-x64"
PYTHON_VERSION: "3.4.x"
PYTHON_ARCH: "64"
TOXENV: "py34"
- PYTHON: "C:\\Python35-x64"
PYTHON_VERSION: "3.5.x"
PYTHON_ARCH: "64"
TOXENV: "py35"
- PYTHON: "C:\\Python36-x64"
PYTHON_VERSION: "3.6.x"
PYTHON_ARCH: "64"
TOXENV: "py36"
install:
# Install Python (from the official .msi of http://python.org) and pip when
# not already installed.
- ps: if (-not(Test-Path($env:PYTHON))) { & _appveyor\install.ps1 }
# Prepend newly installed Python to the PATH of this build (this cannot be
# done from inside the powershell script as it would require to restart
# the parent CMD process).
- "SET PATH=%PYTHON%;%PYTHON%\\Scripts;%PATH%"
# Check that we have the expected version and architecture for Python
- "python --version"
- "python -c \"import struct; print(struct.calcsize('P') * 8)\""
# Upgrade to the latest version of pip to avoid it displaying warnings
# about it being out of date.
- "pip install --disable-pip-version-check --user --upgrade pip"
- "C:\\MinGW\\bin\\mingw32-make"
test_script:
- "C:\\MinGW\\bin\\mingw32-make coverage"
on_success:
- "codecov -f coverage.xml"

View File

@@ -1 +0,0 @@

View File

@@ -9,7 +9,7 @@ BUILDDIR = _build
# User-friendly check for sphinx-build
ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1)
$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from https://www.sphinx-doc.org/)
$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/)
endif
# Internal variables.

View File

@@ -1,12 +0,0 @@
img.logo {margin-left: -20px!important;}
div#searchbox h3 {text-transform: capitalize;}
div.document {width: 1008px;}
div.highlight pre {padding: 11px 14px;}
@media screen and (max-width: 1008px) {
div.document {width: 100%!important;}
div.highlight pre {margin-right: -30px;}
}

116
docs/_static/konami.js vendored Normal file
View File

@@ -0,0 +1,116 @@
/*
* Konami-JS ~
* :: Now with support for touch events and multiple instances for
* :: those situations that call for multiple easter eggs!
* Code: http://konami-js.googlecode.com/
* Examples: http://www.snaptortoise.com/konami-js
* Copyright (c) 2009 George Mandis (georgemandis.com, snaptortoise.com)
* Version: 1.4.2 (9/2/2013)
* Licensed under the MIT License (http://opensource.org/licenses/MIT)
* Tested in: Safari 4+, Google Chrome 4+, Firefox 3+, IE7+, Mobile Safari 2.2.1 and Dolphin Browser
*/
var Konami = function (callback) {
var konami = {
addEvent: function (obj, type, fn, ref_obj) {
if (obj.addEventListener)
obj.addEventListener(type, fn, false);
else if (obj.attachEvent) {
// IE
obj["e" + type + fn] = fn;
obj[type + fn] = function () {
obj["e" + type + fn](window.event, ref_obj);
};
obj.attachEvent("on" + type, obj[type + fn]);
}
},
input: "",
pattern: "38384040373937396665",
load: function (link) {
this.addEvent(document, "keydown", function (e, ref_obj) {
if (ref_obj) konami = ref_obj; // IE
konami.input += e ? e.keyCode : event.keyCode;
if (konami.input.length > konami.pattern.length)
konami.input = konami.input.substr((konami.input.length - konami.pattern.length));
if (konami.input == konami.pattern) {
konami.code(link);
konami.input = "";
e.preventDefault();
return false;
}
}, this);
this.iphone.load(link);
},
code: function (link) {
window.location = link
},
iphone: {
start_x: 0,
start_y: 0,
stop_x: 0,
stop_y: 0,
tapTolerance: 8,
capture: false,
orig_keys: "",
keys: ["UP", "UP", "DOWN", "DOWN", "LEFT", "RIGHT", "LEFT", "RIGHT", "TAP", "TAP"],
code: function (link) {
konami.code(link);
},
touchCapture: function(evt) {
konami.iphone.start_x = evt.changedTouches[0].pageX;
konami.iphone.start_y = evt.changedTouches[0].pageY;
konami.iphone.capture = true;
},
load: function (link) {
this.orig_keys = this.keys;
konami.addEvent(document, "touchmove", function (e) {
if (e.touches.length == 1 && konami.iphone.capture == true) {
var touch = e.touches[0];
konami.iphone.stop_x = touch.pageX;
konami.iphone.stop_y = touch.pageY;
konami.iphone.check_direction();
}
});
konami.addEvent(document, "touchend", function (evt) {
konami.touchCapture(evt);
konami.iphone.check_direction(link);
}, false);
konami.addEvent(document, "touchstart", function (evt) {
konami.touchCapture(evt);
});
},
check_direction: function (link) {
var x_magnitude = Math.abs(this.start_x - this.stop_x);
var y_magnitude = Math.abs(this.start_y - this.stop_y);
var hasMoved = (x_magnitude > this.tapTolerance || y_magnitude > this.tapTolerance);
var result;
if (this.capture === true && hasMoved) {
this.capture = false;
var x = ((this.start_x - this.stop_x) < 0) ? "RIGHT" : "LEFT";
var y = ((this.start_y - this.stop_y) < 0) ? "DOWN" : "UP";
var result = (x_magnitude > y_magnitude) ? x : y;
}
else if (this.capture === false && !hasMoved) {
result = (this.tap == true) ? "TAP" : result;
result = "TAP";
}
if (result) {
if (result == this.keys[0]) this.keys = this.keys.slice(1, this.keys.length);
else this.keys = this.orig_keys;
}
if (this.keys.length == 0) {
this.keys = this.orig_keys;
this.code(link);
}
}
}
}
typeof callback === "string" && konami.load(callback);
if (typeof callback === "function") {
konami.code = callback;
konami.load();
}
return konami;
};

BIN
docs/_static/requests-logo-small.png vendored Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 155 KiB

54
docs/_templates/hacks.html vendored Normal file
View File

@@ -0,0 +1,54 @@
<!-- Alabaster (krTheme++) Hacks -->
<!-- CSS Adjustments (I'm very picky.) -->
<style type="text/css">
/* Rezzy requires precise alignment. */
img.logo {margin-left: -20px!important;}
/* "Quick Search" should be capitalized. */
div#searchbox h3 {text-transform: capitalize;}
/* Make the document a little wider, less code is cut-off. */
div.document {width: 1008px;}
/* Much-improved spacing around code blocks. */
div.highlight pre {padding: 11px 14px;}
/* Remain Responsive! */
@media screen and (max-width: 1008px) {
div.sphinxsidebar {display: none;}
div.document {width: 100%!important;}
/* Have code blocks escape the document right-margin. */
div.highlight pre {margin-right: -30px;}
}
</style>
<!-- Analytics tracking for Kenneth. -->
<script type="text/javascript">
var _gauges = _gauges || [];
(function() {
var t = document.createElement('script');
t.type = 'text/javascript';
t.async = true;
t.id = 'gauges-tracker';
t.setAttribute('data-site-id', '56ca7a57c88d9011080024dd');
t.setAttribute('data-track-path', 'https://track.gaug.es/track.gif');
t.src = 'https://d36ee2fcip1434.cloudfront.net/track.js';
var s = document.getElementsByTagName('script')[0];
s.parentNode.insertBefore(t, s);
})();
</script>
<!-- There are no more hacks. -->
<!-- இڿڰۣ-ڰۣ— -->
<!-- Love, Kenneth Reitz -->
<script src="{{ pathto('_static/', 1) }}/konami.js"></script>
<script>
var easter_egg = new Konami('http://fortunes.herokuapp.com/random/raw');
</script>
<!-- That was not a hack. That was art. -->

View File

@@ -1,33 +0,0 @@
<p class="logo">
<a href="{{ pathto(root_doc) }}">
<img class="logo" src="{{ pathto('_static/requests-sidebar.png', 1) }}" alt="{{ _('Requests logo') }}" />
</a>
</p>
<p>
<iframe src="https://ghbtns.com/github-btn.html?user=psf&repo=requests&type=watch&count=true&size=large"
allowtransparency="true" frameborder="0" scrolling="0" width="200px" height="35px"></iframe>
</p>
<p>
{{ _('Requests is an elegant and simple HTTP library for Python, built for human beings.') }}
</p>
<h3>{{ _('Useful Links') }}</h3>
<ul>
<li><a href="{{ pathto('user/quickstart') }}">{{ _('Quickstart') }}</a></li>
<li><a href="{{ pathto('user/advanced') }}">{{ _('Advanced Usage') }}</a></li>
<li><a href="{{ pathto('api') }}">{{ _('API Reference') }}</a></li>
<li><a href="{{ pathto('community/updates') + '#release-history' }}">{{ _('Release History') }}</a></li>
<li><a href="{{ pathto('dev/contributing') }}">{{ _('Contributors Guide') }}</a></li>
<p></p>
<li><a href="{{ pathto('community/recommended') }}">{{ _('Recommended Packages and Extensions') }}</a></li>
<p></p>
<li><a href="https://github.com/psf/requests">{{ _('Requests @ GitHub') }}</a></li>
<li><a href="https://pypi.org/project/requests/">{{ _('Requests @ PyPI') }}</a></li>
<li><a href="https://github.com/psf/requests/issues">{{ _('Issue Tracker') }}</a></li>
</ul>

80
docs/_templates/sidebarintro.html vendored Normal file
View File

@@ -0,0 +1,80 @@
<p class="logo">
<a href="{{ pathto(master_doc) }}">
<img class="logo" src="{{ pathto('_static/requests-sidebar.png', 1) }}" title="https://kennethreitz.org/tattoos"/>
</a>
</p>
<p>
<iframe src="http://ghbtns.com/github-btn.html?user=kennethreitz&repo=requests&type=watch&count=true&size=large"
allowtransparency="true" frameborder="0" scrolling="0" width="200px" height="35px"></iframe>
</p>
<p>
Requests is an elegant and simple HTTP library for Python, built for
human beings.
</p>
<h3>The Hitchhiker's Guide to Python</h3>
<p>This guide is now available in tangible book form!</p>
<a href="https://www.amazon.com/Hitchhikers-Guide-Python-Practices-Development/dp/1491933178/ref=as_li_ss_il?ie=UTF8&linkCode=li2&tag=bookforkind-20&linkId=804806ebdacaf3b56567347f3afbdbca" target="_blank"><img border="0" src="https://ws-na.amazon-adsystem.com/widgets/q?_encoding=UTF8&ASIN=1491933178&Format=_SL160_&ID=AsinImage&MarketPlace=US&ServiceVersion=20070822&WS=1&tag=bookforkind-20" ></a><img src="//ir-na.amazon-adsystem.com/e/ir?t=bookforkind-20&l=li2&o=1&a=1491933178" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" />
<p>All proceeds are being directly donated to the <a href="https://djangogirls.org">DjangoGirls</a> organization.</p>
<h3>Stay Informed</h3>
<p>Receive updates on new releases and upcoming projects.</p>
<p><iframe src="http://ghbtns.com/github-btn.html?user=kennethreitz&type=follow&count=false"
allowtransparency="true" frameborder="0" scrolling="0" width="200" height="20"></iframe></p>
<p><a href="https://twitter.com/kennethreitz" class="twitter-follow-button" data-show-count="false">Follow @kennethreitz</a> <script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script></p>
<p><a href="https://saythanks.io/to/kennethreitz">Say Thanks!</a></p>
<p><a href="http://tinyletter.com/kennethreitz">Join Mailing List</a>.</p>
<h3>Other Projects</h3>
<p>More <a href="http://kennethreitz.org/">Kenneth Reitz</a> projects:</p>
<ul>
<li><a href="http://edmsynths.com/">edmsynths.com</a></li>
<li><a href="http://pipenv.org/">pipenv</a></li>
<li><a href="http://pep8.org/">pep8.org</a></li>
<li><a href="http://httpbin.org/">httpbin.org</a></li>
<li><a href="http://python-guide.org">The Python Guide</a></li>
<li><a href="https://github.com/kennethreitz/maya">Maya: Datetimes for Humans</a></li>
<li><a href="https://github.com/kennethreitz/records">Records: SQL for Humans</a></li>
<li><a href="http://www.git-legit.org">Legit: Git for Humans</a></li>
<li><a href="http://docs.python-tablib.org/en/latest/">Tablib: Tabular Datasets</a></li>
</ul>
<h3>Useful Links</h3>
<ul>
<li><a href="http://docs.python-requests.org/en/latest/community/recommended/">Recommended Packages and Extensions</a></li>
<p></p>
<li><a href="http://github.com/requests/requests">Requests @ GitHub</a></li>
<li><a href="http://pypi.python.org/pypi/requests">Requests @ PyPI</a></li>
<li><a href="http://github.com/requests/requests/issues">Issue Tracker</a></li>
<li><a href="http://docs.python-requests.org/en/latest/community/updates/#software-updates">Release History</a></li>
</ul>
<h3>Translations</h3>
<ul>
<li><a href="http://docs.python-requests.org/">English</a></li>
<li><a href="http://fr.python-requests.org/">French</a></li>
<li><a href="http://de.python-requests.org/">German</a></li>
<li><a href="http://jp.python-requests.org/">Japanese</a></li>
<li><a href="http://cn.python-requests.org/">Chinese</a></li>
<li><a href="http://pt.python-requests.org/">Portuguese</a></li>
<li><a href="http://it.python-requests.org/">Italian</a></li>
<li><a href="http://es.python-requests.org/">Spanish</a></li>
</ul>

69
docs/_templates/sidebarlogo.html vendored Normal file
View File

@@ -0,0 +1,69 @@
<p class="logo">
<a href="{{ pathto(master_doc) }}">
<img class="logo" src="{{ pathto('_static/requests-sidebar.png', 1) }}" title="https://kennethreitz.org/tattoos"/>
</a>
</p>
<p>
<iframe src="http://ghbtns.com/github-btn.html?user=kennethreitz&repo=requests&type=watch&count=true&size=large"
allowtransparency="true" frameborder="0" scrolling="0" width="200px" height="35px"></iframe>
</p>
<p>
Requests is an elegant and simple HTTP library for Python, built for
human beings. You are currently looking at the documentation of the
development release.
</p>
<h3>Stay Informed</h3>
<p>Receive updates on new releases and upcoming projects.</p>
<p><a href="http://tinyletter.com/kennethreitz">Join Mailing List</a>.</p>
<hr/>
<h3>The Hitchhiker's Guide to Python</h3>
<p>This guide is now available in tangible book form!</p>
<a href="https://www.amazon.com/Hitchhikers-Guide-Python-Practices-Development/dp/1491933178/ref=as_li_ss_il?ie=UTF8&linkCode=li2&tag=bookforkind-20&linkId=804806ebdacaf3b56567347f3afbdbca" target="_blank"><img border="0" src="https://ws-na.amazon-adsystem.com/widgets/q?_encoding=UTF8&ASIN=1491933178&Format=_SL160_&ID=AsinImage&MarketPlace=US&ServiceVersion=20070822&WS=1&tag=bookforkind-20" ></a><img src="//ir-na.amazon-adsystem.com/e/ir?t=bookforkind-20&l=li2&o=1&a=1491933178" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" />
<p>All proceeds are being directly donated to the <a href="https://djangogirls.org">DjangoGirls</a> organization.</p>
<p>If you enjoy using this project, <a href="https://saythanks.io/to/kennethreitz">Say Thanks!</a></p>
<p><iframe src="http://ghbtns.com/github-btn.html?user=kennethreitz&type=follow&count=false"
allowtransparency="true" frameborder="0" scrolling="0" width="200" height="20"></iframe></p>
<p><a href="https://twitter.com/kennethreitz" class="twitter-follow-button" data-show-count="false">Follow @kennethreitz</a> <script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script></p>
<h3>Other Projects</h3>
<p>More <a href="http://kennethreitz.org/">Kenneth Reitz</a> projects:</p>
<ul>
<li><a href="http://edmsynths.com/">edmsynths.com</a></li>
<li><a href="http://pipenv.org/">pipenv</a></li>
<li><a href="http://pep8.org/">pep8.org</a></li>
<li><a href="http://httpbin.org/">httpbin.org</a></li>
<li><a href="http://python-guide.org">The Python Guide</a></li>
<li><a href="https://github.com/kennethreitz/maya">Maya: Datetimes for Humans</a></li>
<li><a href="https://github.com/kennethreitz/records">Records: SQL for Humans</a></li>
<li><a href="http://www.git-legit.org">Legit: Git for Humans</a></li>
<li><a href="http://docs.python-tablib.org/en/latest/">Tablib: Tabular Datasets</a></li>
</ul>
<h3>Translations</h3>
<ul>
<li><a href="http://docs.python-requests.org/">English</a></li>
<li><a href="http://fr.python-requests.org/">French</a></li>
<li><a href="http://de.python-requests.org/">German</a></li>
<li><a href="http://jp.python-requests.org/">Japanese</a></li>
<li><a href="http://cn.python-requests.org/">Chinese</a></li>
<li><a href="http://pt.python-requests.org/">Portuguese</a></li>
<li><a href="http://it.python-requests.org/">Italian</a></li>
<li><a href="http://es.python-requests.org/">Spanish</a></li>
</ul>

View File

@@ -31,11 +31,11 @@ Exceptions
.. autoexception:: requests.RequestException
.. autoexception:: requests.ConnectionError
.. autoexception:: requests.HTTPError
.. autoexception:: requests.URLRequired
.. autoexception:: requests.TooManyRedirects
.. autoexception:: requests.ConnectTimeout
.. autoexception:: requests.ReadTimeout
.. autoexception:: requests.Timeout
.. autoexception:: requests.JSONDecodeError
Request Sessions
@@ -55,7 +55,6 @@ Lower-Level Classes
.. autoclass:: Response
:inherited-members:
:exclude-members: is_permanent_redirect
Lower-Lower-Level Classes
@@ -110,7 +109,17 @@ Status Code Lookup
.. autoclass:: requests.codes
.. automodule:: requests.status_codes
::
>>> requests.codes['temporary_redirect']
307
>>> requests.codes.teapot
418
>>> requests.codes['\o/']
200
Migrating to 1.x
@@ -128,7 +137,7 @@ API Changes
::
import requests
r = requests.get('https://api.github.com/events')
r = requests.get('https://github.com/timeline.json')
r.json() # This *call* raises an exception if JSON decoding fails
* The ``Session`` API has changed. Sessions objects no longer take parameters.
@@ -140,7 +149,7 @@ API Changes
s = requests.Session() # formerly, session took parameters
s.auth = auth
s.headers.update(headers)
r = s.get('https://httpbin.org/headers')
r = s.get('http://httpbin.org/headers')
* All request hooks have been removed except 'response'.
@@ -157,7 +166,7 @@ API Changes
::
# in 0.x, passing prefetch=False would accomplish the same thing
r = requests.get('https://api.github.com/events', stream=True)
r = requests.get('https://github.com/timeline.json', stream=True)
for chunk in r.iter_content(8192):
...
@@ -182,11 +191,11 @@ API Changes
logging.basicConfig() # you need to initialize logging, otherwise you will not see anything from requests
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger("urllib3")
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True
requests.get('https://httpbin.org/headers')
requests.get('http://httpbin.org/headers')
@@ -198,8 +207,8 @@ license from the ISC_ license to the `Apache 2.0`_ license. The Apache 2.0
license ensures that contributions to Requests are also covered by the Apache
2.0 license.
.. _ISC: https://opensource.org/licenses/ISC
.. _Apache 2.0: https://opensource.org/licenses/Apache-2.0
.. _ISC: http://opensource.org/licenses/ISC
.. _Apache 2.0: http://opensource.org/licenses/Apache-2.0
Migrating to 2.x
@@ -214,7 +223,7 @@ For more details on the changes in this release including new APIs, links
to the relevant GitHub issues and some of the bug fixes, read Cory's blog_
on the subject.
.. _blog: https://lukasa.co.uk/2013/09/Requests_20/
.. _blog: http://lukasa.co.uk/2013/09/Requests_20/
API Changes

View File

@@ -11,9 +11,6 @@ Encoded Data?
Requests automatically decompresses gzip-encoded responses, and does
its best to decode response content to unicode when possible.
When either the `brotli <https://pypi.org/project/Brotli/>`_ or `brotlicffi <https://pypi.org/project/brotlicffi/>`_
package is installed, requests also decodes Brotli-encoded responses.
You can get direct access to the raw response (and even the socket),
if needed as well.
@@ -22,15 +19,14 @@ Custom User-Agents?
-------------------
Requests allows you to easily override User-Agent strings, along with
any other HTTP Header. See :ref:`documentation about headers <custom-headers>`.
any other HTTP Header.
Why not Httplib2?
-----------------
Chris Adams gave an excellent summary on
`Hacker News <https://news.ycombinator.com/item?id=2884406>`_:
`Hacker News <http://news.ycombinator.com/item?id=2884406>`_:
httplib2 is part of why you should use requests: it's far more respectable
as a client but not as well documented and it still takes way too much code
@@ -44,8 +40,8 @@ Chris Adams gave an excellent summary on
Disclosure: I'm listed in the requests AUTHORS file but can claim credit
for, oh, about 0.0001% of the awesomeness.
1. https://code.google.com/p/httplib2/issues/detail?id=96 is a good example:
an annoying bug that affected many people, there was a fix available for
1. http://code.google.com/p/httplib2/issues/detail?id=96 is a good example:
an annoying bug which affect many people, there was a fix available for
months, which worked great when I applied it in a fork and pounded a couple
TB of data through it, but it took over a year to make it into trunk and
even longer to make it onto PyPI where any other project which required "
@@ -55,18 +51,16 @@ Chris Adams gave an excellent summary on
Python 3 Support?
-----------------
Yes! Requests supports all `officially supported versions of Python <https://devguide.python.org/versions/>`_
and recent releases of PyPy.
Yes! Here's a list of Python platforms that are officially
supported:
Python 2 Support?
-----------------
No! As of Requests 2.28.0, Requests no longer supports Python 2.7. Users who
have been unable to migrate should pin to `requests<2.28`. Full information
can be found in `psf/requests#6023 <https://github.com/psf/requests/issues/6023>`_.
It is *highly* recommended users migrate to a supported Python 3.x version now since
Python 2.7 is no longer receiving bug fixes or security updates as of January 1, 2020.
* Python 2.6
* Python 2.7
* Python 3.3
* Python 3.4
* Python 3.5
* Python 3.6
* PyPy
What are "hostname doesn't match" errors?
-----------------------------------------
@@ -75,7 +69,7 @@ These errors occur when :ref:`SSL certificate verification <verification>`
fails to match the certificate the server responds with to the hostname
Requests thinks it's contacting. If you're certain the server's SSL setup is
correct (for example, because you can visit the site with your browser) and
you're using Python 2.7, a possible explanation is that you need
you're using Python 2.6 or 2.7, a possible explanation is that you need
Server-Name-Indication.
`Server-Name-Indication`_, or SNI, is an official extension to SSL where the
@@ -84,7 +78,10 @@ when servers are using `Virtual Hosting`_. When such servers are hosting
more than one SSL site they need to be able to return the appropriate
certificate based on the hostname the client is connecting to.
Python 3 already includes native support for SNI in their SSL modules.
Python3 and Python 2.7.9+ include native support for SNI in their SSL modules.
For information on using SNI with Requests on Python < 2.7.9 refer to this
`Stack Overflow answer`_.
.. _`Server-Name-Indication`: https://en.wikipedia.org/wiki/Server_Name_Indication
.. _`virtual hosting`: https://en.wikipedia.org/wiki/Virtual_hosting
.. _`Stack Overflow answer`: https://stackoverflow.com/questions/18578439/using-requests-with-tls-doesnt-give-sni-support/18579484#18579484

View File

@@ -1,10 +1,22 @@
Integrations
============
Python for iOS
--------------
Requests is built into the wonderful `Python for iOS <https://itunes.apple.com/us/app/python-2.7-for-ios/id485729872?mt=Python8>`_ runtime!
To give it a try, simply::
import requests
Articles & Talks
================
- `Daniel Greenfeld's Review of Requests <https://pydanny.blogspot.com/2011/05/python-http-requests-for-humans.html>`_
- `Issac Kelly's 'Consuming Web APIs' talk <https://issackelly.github.io/Consuming-Web-APIs-with-Python-Talk/slides/slides.html>`_
- `Blog post about Requests via Yum <https://arunsag.wordpress.com/2011/08/17/new-package-python-requests-http-for-humans/>`_
- `Russian blog post introducing Requests <https://habr.com/post/126262/>`_
- `Sending JSON in Requests <https://www.coglib.com/~icordasc/blog/2014/11/sending-json-in-requests.html>`_
- `Python for the Web <http://gun.io/blog/python-for-the-web/>`_ teaches how to use Python to interact with the web, using Requests.
- `Daniel Greenfeld's Review of Requests <http://pydanny.blogspot.com/2011/05/python-http-requests-for-humans.html>`_
- `My 'Python for Humans' talk <http://python-for-humans.heroku.com>`_ ( `audio <http://codeconf.s3.amazonaws.com/2011/pycodeconf/talks/PyCodeConf2011%20-%20Kenneth%20Reitz.m4a>`_ )
- `Issac Kelly's 'Consuming Web APIs' talk <http://issackelly.github.com/Consuming-Web-APIs-with-Python-Talk/slides/slides.html>`_
- `Blog post about Requests via Yum <http://arunsag.wordpress.com/2011/08/17/new-package-python-requests-http-for-humans/>`_
- `Russian blog post introducing Requests <http://habrahabr.ru/blogs/python/126262/>`_
- `Sending JSON in Requests <http://www.coglib.com/~icordasc/blog/2014/11/sending-json-in-requests.html>`_

View File

@@ -13,7 +13,7 @@ Certifi CA Bundle
validating the trustworthiness of SSL certificates while verifying the
identity of TLS hosts. It has been extracted from the Requests project.
.. _Certifi: https://github.com/certifi/python-certifi
.. _Certifi: http://certifi.io/en/latest/
CacheControl
------------
@@ -32,15 +32,7 @@ but do not belong in Requests proper. This library is actively maintained
by members of the Requests core team, and reflects the functionality most
requested by users within the community.
.. _Requests-Toolbelt: https://toolbelt.readthedocs.io/en/latest/index.html
Requests-Threads
----------------
`Requests-Threads`_ is a Requests session that returns the amazing Twisted's awaitable Deferreds instead of Response objects. This allows the use of ``async``/``await`` keyword usage on Python 3, or Twisted's style of programming, if desired.
.. _Requests-Threads: https://github.com/requests/requests-threads
.. _Requests-Toolbelt: http://toolbelt.readthedocs.io/en/latest/index.html
Requests-OAuthlib
-----------------
@@ -59,4 +51,7 @@ Betamax
`Betamax`_ records your HTTP interactions so the NSA does not have to.
A VCR imitation designed only for Python-Requests.
.. _betamax: https://github.com/betamaxpy/betamax
.. _betamax: https://github.com/sigmavirus24/betamax

View File

@@ -17,18 +17,19 @@ Breaking changes are changes that break backwards compatibility with prior
versions. If the project were to change the ``text`` attribute on a
``Response`` object to a method, that would only happen in a Major release.
Major releases may also include miscellaneous bug fixes. The core developers of
Requests are committed to providing a good user experience. This means we're
also committed to preserving backwards compatibility as much as possible. Major
releases will be infrequent and will need strong justifications before they are
considered.
Major releases may also include miscellaneous bug fixes and upgrades to
vendored packages. The core developers of Requests are committed to providing
a good user experience. This means we're also committed to preserving
backwards compatibility as much as possible. Major releases will be infrequent
and will need strong justifications before they are considered.
Minor Releases
--------------
A minor release will not include breaking changes but may include miscellaneous
bug fixes. If the previous version of Requests released was ``v10.2.7`` a minor
release would be versioned as ``v10.3.0``.
A minor release will not include breaking changes but may include
miscellaneous bug fixes and upgrades to vendored packages. If the previous
version of Requests released was ``v10.2.7`` a minor release would be
versioned as ``v10.3.0``.
Minor releases will be backwards compatible with releases that have the same
major version number. In other words, all versions that would start with

View File

@@ -5,27 +5,47 @@ Support
If you have questions or issues about Requests, there are several options:
Stack Overflow
--------------
StackOverflow
-------------
If your question does not contain sensitive (possibly proprietary)
information or can be properly anonymized, please ask a question on
`Stack Overflow <https://stackoverflow.com/questions/tagged/python-requests>`_
`StackOverflow <https://stackoverflow.com/questions/tagged/python-requests>`_
and use the tag ``python-requests``.
Send a Tweet
------------
If your question is less than 140 characters, feel free to send a tweet to
`@kennethreitz <https://twitter.com/kennethreitz>`_,
`@sigmavirus24 <https://twitter.com/sigmavirus24>`_, or
`@lukasaoz <https://twitter.com/lukasaoz>`_.
File an Issue
-------------
If you notice some unexpected behaviour in Requests, or want to see support
for a new feature,
`file an issue on GitHub <https://github.com/psf/requests/issues>`_.
`file an issue on GitHub <https://github.com/requests/requests/issues>`_.
Send a Tweet
------------
E-mail
------
If your question is less than 280 characters, feel free to send a tweet to
`@nateprewitt <https://twitter.com/nateprewitt>`_,
`@sethmlarson <https://twitter.com/sethmlarson>`_, or
`@sigmavirus24 <https://twitter.com/sigmavirus24>`_.
I'm more than happy to answer any personal or in-depth questions about
Requests. Feel free to email
`requests@kennethreitz.com <mailto:requests@kennethreitz.com>`_.
IRC
---
The official Freenode channel for Requests is
`#python-requests <irc://irc.freenode.net/python-requests>`_
The core developers of requests are on IRC throughout the day.
You can find them in ``#python-requests`` as:
- kennethreitz
- lukasa
- sigmavirus24

View File

@@ -12,7 +12,19 @@ GitHub
------
The best way to track the development of Requests is through
`the GitHub repo <https://github.com/psf/requests>`_.
`the GitHub repo <https://github.com/requests/requests>`_.
Twitter
-------
The author, Kenneth Reitz, often tweets about new features and releases of Requests.
Follow `@kennethreitz <https://twitter.com/kennethreitz>`_ for updates.
.. include:: ../../HISTORY.md
Release and Version History
===========================
.. include:: ../../HISTORY.rst

View File

@@ -1,5 +1,105 @@
Vulnerability Disclosure
========================
The latest vulnerability disclosure information can be found on GitHub in our
`Security Policy <https://github.com/psf/requests/blob/main/.github/SECURITY.md>`_.
If you think you have found a potential security vulnerability in requests,
please email `sigmavirus24 <mailto:graffatcolmingov@gmail.com>`_ and
`Lukasa <mailto:cory@lukasa.co.uk>`_ directly. **Do not file a public issue.**
Our PGP Key fingerprints are:
- 0161 BB7E B208 B5E0 4FDC 9F81 D9DA 0A04 9113 F853 (@sigmavirus24)
- 90DC AE40 FEA7 4B14 9B70 662D F25F 2144 EEC1 373D (@lukasa)
If English is not your first language, please try to describe the problem and
its impact to the best of your ability. For greater detail, please use your
native language and we will try our best to translate it using online services.
Please also include the code you used to find the problem and the shortest
amount of code necessary to reproduce it.
Please do not disclose this to anyone else. We will retrieve a CVE identifier
if necessary and give you full credit under whatever name or alias you provide.
We will only request an identifier when we have a fix and can publish it in a
release.
We will respect your privacy and will only publicize your involvement if you
grant us permission.
Process
-------
This following information discusses the process the requests project follows
in response to vulnerability disclosures. If you are disclosing a
vulnerability, this section of the documentation lets you know how we will
respond to your disclosure.
Timeline
~~~~~~~~
When you report an issue, one of the project members will respond to you within
two days *at the outside*. In most cases responses will be faster, usually
within 12 hours. This initial response will at the very least confirm receipt
of the report.
If we were able to rapidly reproduce the issue, the initial response will also
contain confirmation of the issue. If we are not, we will often ask for more
information about the reproduction scenario.
Our goal is to have a fix for any vulnerability released within two weeks of
the initial disclosure. This may potentially involve shipping an interim
release that simply disables function while a more mature fix can be prepared,
but will in the vast majority of cases mean shipping a complete release as soon
as possible.
Throughout the fix process we will keep you up to speed with how the fix is
progressing. Once the fix is prepared, we will notify you that we believe we
have a fix. Often we will ask you to confirm the fix resolves the problem in
your environment, especially if we are not confident of our reproduction
scenario.
At this point, we will prepare for the release. We will obtain a CVE number
if one is required, providing you with full credit for the discovery. We will
also decide on a planned release date, and let you know when it is. This
release date will *always* be on a weekday.
At this point we will reach out to our major downstream packagers to notify
them of an impending security-related patch so they can make arrangements. In
addition, these packagers will be provided with the intended patch ahead of
time, to ensure that they are able to promptly release their downstream
packages. Currently the list of people we actively contact *ahead of a public
release* is:
- Jeremy Cline, Red Hat (@jeremycline)
- Daniele Tricoli, Debian (@eriol)
We will notify these individuals at least a week ahead of our planned release
date to ensure that they have sufficient time to prepare. If you believe you
should be on this list, please let one of the maintainers know at one of the
email addresses at the top of this article.
On release day, we will push the patch to our public repository, along with an
updated changelog that describes the issue and credits you. We will then issue
a PyPI release containing the patch.
At this point, we will publicise the release. This will involve mails to
mailing lists, Tweets, and all other communication mechanisms available to the
core team.
We will also explicitly mention which commits contain the fix to make it easier
for other distributors and users to easily patch their own versions of requests
if upgrading is not an option.
Previous CVEs
-------------
- Fixed in 2.6.0
- `CVE 2015-2296 <http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=2015-2296>`_,
reported by Matthew Daley of `BugFuzz <https://bugfuzz.com/>`_.
- Fixed in 2.3.0
- `CVE 2014-1829 <http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=2014-1829>`_
- `CVE 2014-1830 <http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=2014-1830>`_

View File

@@ -18,11 +18,11 @@ import os
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
# sys.path.insert(0, os.path.abspath('.'))
#sys.path.insert(0, os.path.abspath('.'))
# Insert Requests' path into the system.
sys.path.insert(0, os.path.abspath(".."))
sys.path.insert(0, os.path.abspath("_themes"))
sys.path.insert(0, os.path.abspath('..'))
sys.path.insert(0, os.path.abspath('_themes'))
import requests
@@ -30,36 +30,36 @@ import requests
# -- General configuration ------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
# needs_sphinx = '1.0'
#needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
"sphinx.ext.autodoc",
"sphinx.ext.intersphinx",
"sphinx.ext.todo",
"sphinx.ext.viewcode",
'sphinx.ext.autodoc',
'sphinx.ext.intersphinx',
'sphinx.ext.todo',
'sphinx.ext.viewcode',
]
# Add any paths that contain templates here, relative to this directory.
templates_path = ["_templates"]
templates_path = ['_templates']
# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
# source_suffix = ['.rst', '.md']
source_suffix = ".rst"
source_suffix = '.rst'
# The encoding of source files.
# source_encoding = 'utf-8-sig'
#source_encoding = 'utf-8-sig'
# The master toctree document.
root_doc = "index"
master_doc = 'index'
# General information about the project.
project = u"Requests"
copyright = u"Kenneth Reitz and contributors"
author = u"Kenneth Reitz"
project = u'Requests'
copyright = u'MMXVII. A <a href="http://kennethreitz.com/pages/open-projects.html">Kenneth Reitz</a> Project'
author = u'Kenneth Reitz'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
@@ -79,17 +79,17 @@ language = None
# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
# today = ''
#today = ''
# Else, today_fmt is used as the format for a strftime call.
# today_fmt = '%B %d, %Y'
#today_fmt = '%B %d, %Y'
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = ["_build"]
exclude_patterns = ['_build']
# The reST default role (used for this markup: `text`) to use for all
# documents.
# default_role = None
#default_role = None
# If true, '()' will be appended to :func: etc. cross-reference text.
add_function_parentheses = False
@@ -100,16 +100,16 @@ add_module_names = True
# If true, sectionauthor and moduleauthor directives will be shown in the
# output. They are ignored by default.
# show_authors = False
#show_authors = False
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = "flask_theme_support.FlaskyStyle"
pygments_style = 'flask_theme_support.FlaskyStyle'
# A list of ignored prefixes for module index sorting.
# modindex_common_prefix = []
#modindex_common_prefix = []
# If true, keep warnings as "system message" paragraphs in the built documents.
# keep_warnings = False
#keep_warnings = False
# If true, `todo` and `todoList` produce output, else they produce nothing.
todo_include_todos = True
@@ -119,52 +119,51 @@ todo_include_todos = True
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
html_theme = "alabaster"
html_theme = 'alabaster'
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
html_theme_options = {
"show_powered_by": False,
"github_user": "psf",
"github_repo": "requests",
"github_banner": True,
"show_related": False,
"note_bg": "#FFF59C",
'show_powered_by': False,
'github_user': 'kennethreitz',
'github_repo': 'requests',
'github_banner': True,
'show_related': False
}
# Add any paths that contain custom themes here, relative to this directory.
# html_theme_path = []
#html_theme_path = []
# The name for this set of Sphinx documents. If None, it defaults to
# "<project> v<release> documentation".
# html_title = None
#html_title = None
# A shorter title for the navigation bar. Default is the same as html_title.
# html_short_title = None
#html_short_title = None
# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
# html_logo = None
#html_logo = None
# The name of an image file (within the static path) to use as favicon of the
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
# html_favicon = None
#html_favicon = None
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ["_static"]
html_static_path = ['_static']
# Add any extra paths that contain custom files (such as robots.txt or
# .htaccess) here, relative to this directory. These files are copied
# directly to the root of the documentation.
# html_extra_path = []
#html_extra_path = []
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.
# html_last_updated_fmt = '%b %d, %Y'
#html_last_updated_fmt = '%b %d, %Y'
# If true, SmartyPants will be used to convert quotes and dashes to
# typographically correct entities.
@@ -172,28 +171,24 @@ html_use_smartypants = False
# Custom sidebar templates, maps document names to template names.
html_sidebars = {
"index": ["sidebar.html", "sourcelink.html", "searchbox.html"],
"**": [
"sidebar.html",
"localtoc.html",
"relations.html",
"sourcelink.html",
"searchbox.html",
],
'index': ['sidebarintro.html', 'sourcelink.html', 'searchbox.html',
'hacks.html'],
'**': ['sidebarlogo.html', 'localtoc.html', 'relations.html',
'sourcelink.html', 'searchbox.html', 'hacks.html']
}
# Additional templates that should be rendered to pages, maps page names to
# template names.
# html_additional_pages = {}
#html_additional_pages = {}
# If false, no module index is generated.
# html_domain_indices = True
#html_domain_indices = True
# If false, no index is generated.
# html_use_index = True
#html_use_index = True
# If true, the index is split into individual pages for each letter.
# html_split_index = False
#html_split_index = False
# If true, links to the reST sources are added to the pages.
html_show_sourcelink = False
@@ -207,77 +202,84 @@ html_show_copyright = True
# If true, an OpenSearch description file will be output, and all pages will
# contain a <link> tag referring to it. The value of this option must be the
# base URL from which the finished HTML is served.
# html_use_opensearch = ''
#html_use_opensearch = ''
# This is the file name suffix for HTML files (e.g. ".xhtml").
# html_file_suffix = None
#html_file_suffix = None
# Language to be used for generating the HTML full-text search index.
# Sphinx supports the following languages:
# 'da', 'de', 'en', 'es', 'fi', 'fr', 'hu', 'it', 'ja'
# 'nl', 'no', 'pt', 'ro', 'ru', 'sv', 'tr'
# html_search_language = 'en'
#html_search_language = 'en'
# A dictionary with options for the search language support, empty by default.
# Now only 'ja' uses this config value
# html_search_options = {'type': 'default'}
#html_search_options = {'type': 'default'}
# The name of a javascript file (relative to the configuration directory) that
# implements a search results scorer. If empty, the default will be used.
# html_search_scorer = 'scorer.js'
#html_search_scorer = 'scorer.js'
# Output file base name for HTML help builder.
htmlhelp_basename = "Requestsdoc"
htmlhelp_basename = 'Requestsdoc'
# -- Options for LaTeX output ---------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#'preamble': '',
# Latex figure (float) alignment
#'figure_align': 'htbp',
# The paper size ('letterpaper' or 'a4paper').
#'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#'preamble': '',
# Latex figure (float) alignment
#'figure_align': 'htbp',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
(root_doc, "Requests.tex", u"Requests Documentation", u"Kenneth Reitz", "manual")
(master_doc, 'Requests.tex', u'Requests Documentation',
u'Kenneth Reitz', 'manual'),
]
# The name of an image file (relative to this directory) to place at the top of
# the title page.
# latex_logo = None
#latex_logo = None
# For "manual" documents, if this is true, then toplevel headings are parts,
# not chapters.
# latex_use_parts = False
#latex_use_parts = False
# If true, show page references after internal links.
# latex_show_pagerefs = False
#latex_show_pagerefs = False
# If true, show URL addresses after external links.
# latex_show_urls = False
#latex_show_urls = False
# Documents to append as an appendix to all manuals.
# latex_appendices = []
#latex_appendices = []
# If false, no module index is generated.
# latex_domain_indices = True
#latex_domain_indices = True
# -- Options for manual page output ---------------------------------------
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [(root_doc, "requests", u"Requests Documentation", [author], 1)]
man_pages = [
(master_doc, 'requests', u'Requests Documentation',
[author], 1)
]
# If true, show URL addresses after external links.
# man_show_urls = False
#man_show_urls = False
# -- Options for Texinfo output -------------------------------------------
@@ -286,28 +288,22 @@ man_pages = [(root_doc, "requests", u"Requests Documentation", [author], 1)]
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
(
root_doc,
"Requests",
u"Requests Documentation",
author,
"Requests",
"One line description of project.",
"Miscellaneous",
)
(master_doc, 'Requests', u'Requests Documentation',
author, 'Requests', 'One line description of project.',
'Miscellaneous'),
]
# Documents to append as an appendix to all manuals.
# texinfo_appendices = []
#texinfo_appendices = []
# If false, no module index is generated.
# texinfo_domain_indices = True
#texinfo_domain_indices = True
# How to display URL addresses: 'footnote', 'no', or 'inline'.
# texinfo_show_urls = 'footnote'
#texinfo_show_urls = 'footnote'
# If true, do not generate a @detailmenu in the "Top" node's menu.
# texinfo_no_detailmenu = False
#texinfo_no_detailmenu = False
# -- Options for Epub output ----------------------------------------------
@@ -319,67 +315,64 @@ epub_publisher = author
epub_copyright = copyright
# The basename for the epub file. It defaults to the project name.
# epub_basename = project
#epub_basename = project
# The HTML theme for the epub output. Since the default themes are not
# optimized for small screen space, using the same theme for HTML and epub
# output is usually not wise. This defaults to 'epub', a theme designed to save
# visual space.
# epub_theme = 'epub'
#epub_theme = 'epub'
# The language of the text. It defaults to the language option
# or 'en' if the language is not set.
# epub_language = ''
#epub_language = ''
# The scheme of the identifier. Typical schemes are ISBN or URL.
# epub_scheme = ''
#epub_scheme = ''
# The unique identifier of the text. This can be a ISBN number
# or the project homepage.
# epub_identifier = ''
#epub_identifier = ''
# A unique identification for the text.
# epub_uid = ''
#epub_uid = ''
# A tuple containing the cover image and cover page html template filenames.
# epub_cover = ()
#epub_cover = ()
# A sequence of (type, uri, title) tuples for the guide element of content.opf.
# epub_guide = ()
#epub_guide = ()
# HTML files that should be inserted before the pages created by sphinx.
# The format is a list of tuples containing the path and title.
# epub_pre_files = []
#epub_pre_files = []
# HTML files that should be inserted after the pages created by sphinx.
# The format is a list of tuples containing the path and title.
# epub_post_files = []
#epub_post_files = []
# A list of files that should not be packed into the epub file.
epub_exclude_files = ["search.html"]
epub_exclude_files = ['search.html']
# The depth of the table of contents in toc.ncx.
# epub_tocdepth = 3
#epub_tocdepth = 3
# Allow duplicate toc entries.
# epub_tocdup = True
#epub_tocdup = True
# Choose between 'default' and 'includehidden'.
# epub_tocscope = 'default'
#epub_tocscope = 'default'
# Fix unsupported image types using the Pillow.
# epub_fix_images = False
#epub_fix_images = False
# Scale large images.
# epub_max_image_width = 0
#epub_max_image_width = 0
# How to display URL addresses: 'footnote', 'no', or 'inline'.
# epub_show_urls = 'inline'
#epub_show_urls = 'inline'
# If false, no index is generated.
# epub_use_index = True
#epub_use_index = True
intersphinx_mapping = {
"python": ("https://docs.python.org/3/", None),
"urllib3": ("https://urllib3.readthedocs.io/en/latest", None),
}
intersphinx_mapping = {'urllib3': ('http://urllib3.readthedocs.io/en/latest', None)}

View File

@@ -1,4 +1,5 @@
Authors
=======
.. include:: ../../AUTHORS.rst

View File

@@ -11,28 +11,35 @@ contributing to the Requests project is *very* generous of you.
This document lays out guidelines and advice for contributing to this project.
If you're thinking of contributing, please start by reading this document and
getting a feel for how contributing to this project works. If you have any
questions, feel free to reach out to either `Nate Prewitt`_, `Ian Cordasco`_,
or `Seth Michael Larson`_, the primary maintainers.
questions, feel free to reach out to either `Ian Cordasco`_ or `Cory Benfield`_,
the primary maintainers.
.. _Ian Cordasco: https://www.coglib.com/~icordasc/
.. _Nate Prewitt: https://www.nateprewitt.com/
.. _Seth Michael Larson: https://sethmlarson.dev/
.. _Ian Cordasco: http://www.coglib.com/~icordasc/
.. _Cory Benfield: https://lukasa.co.uk/about
If you have non-technical feedback, philosophical ponderings, crazy ideas, or
other general thoughts about Requests or its position within the Python
ecosystem, the BDFL, `Kenneth Reitz`_, would love to hear from you.
The guide is split into sections based on the type of contribution you're
thinking of making, with a section that covers general guidelines for all
contributors.
Code of Conduct
---------------
.. _Kenneth Reitz: mailto:me@kennethreitz.org
The Python community is made up of members from around the globe with a diverse
set of skills, personalities, and experiences. It is through these differences
that our community experiences great successes and continued growth. When you're
working with members of the community, follow the
`Python Software Foundation Code of Conduct`_ to help steer your interactions
and keep Python a positive, successful, and growing community.
Be Cordial
----------
.. _Python Software Foundation Code of Conduct: https://policies.python.org/python.org/code-of-conduct/
**Be cordial or be on your way**. *—Kenneth Reitz*
Requests has one very important rule governing all forms of contribution,
including reporting bugs or requesting features. This golden rule is
"`be cordial or be on your way`_".
**All contributions are welcome**, as long as
everyone involved is treated with respect.
.. _be cordial or be on your way: http://kennethreitz.org/be-cordial-or-be-on-your-way/
.. _early-feedback:
@@ -76,7 +83,7 @@ When contributing code, you'll want to follow this checklist:
4. Make your change.
5. Run the entire test suite again, confirming that all tests pass *including
the ones you just added*.
6. Send a GitHub Pull Request to the main repository's ``main`` branch.
6. Send a GitHub Pull Request to the main repository's ``master`` branch.
GitHub Pull Requests are the expected method of code collaboration on this
project.
@@ -91,21 +98,6 @@ event that you object to the code review feedback, you should make your case
clearly and calmly. If, after doing so, the feedback is judged to still apply,
you must either apply the feedback or withdraw your contribution.
Code Style
~~~~~~~~~~
Requests uses a collection of tools to ensure the code base has a consistent
style as it grows. We have these orchestrated using a tool called
`pre-commit`_. This can be installed locally and run over your changes prior
to opening a PR, and will also be run as part of the CI approval process
before a change is merged.
You can find the full list of formatting requirements specified in the
`.pre-commit-config.yaml`_ at the top level directory of Requests.
.. _pre-commit: https://pre-commit.com/
.. _.pre-commit-config.yaml: https://github.com/psf/requests/blob/main/.pre-commit-config.yaml
New Contributors
~~~~~~~~~~~~~~~~
@@ -116,6 +108,55 @@ asking for help.
Please also check the :ref:`early-feedback` section.
Kenneth Reitz's Code Style™
~~~~~~~~~~~~~~~~~~~~~~~~~~~
The Requests codebase uses the `PEP 8`_ code style.
In addition to the standards outlined in PEP 8, we have a few guidelines:
- Line-length can exceed 79 characters, to 100, when convenient.
- Line-length can exceed 100 characters, when doing otherwise would be *terribly* inconvenient.
- Always use single-quoted strings (e.g. ``'#flatearth'``), unless a single-quote occurs within the string.
Additionally, one of the styles that PEP8 recommends for `line continuations`_
completely lacks all sense of taste, and is not to be permitted within
the Requests codebase::
# Aligned with opening delimiter.
foo = long_function_name(var_one, var_two,
var_three, var_four)
No. Just don't. Please.
Docstrings are to follow the following syntaxes::
def the_earth_is_flat():
"""NASA divided up the seas into thirty-three degrees."""
pass
::
def fibonacci_spiral_tool():
"""With my feet upon the ground I lose myself / between the sounds
and open wide to suck it in. / I feel it move across my skin. / I'm
reaching up and reaching out. / I'm reaching for the random or
whatever will bewilder me. / Whatever will bewilder me. / And
following our will and wind we may just go where no one's been. /
We'll ride the spiral to the end and may just go where no one's
been.
Spiral out. Keep going...
"""
pass
All functions, methods, and classes are to contain docstrings. Object data
model methods (e.g. ``__repr__``) are typically the exception to this rule.
Thanks for helping to make the world a better place!
.. _PEP 8: http://pep8.org
.. _line continuations: https://www.python.org/dev/peps/pep-0008/#indentation
Documentation Contributions
---------------------------
@@ -132,8 +173,8 @@ files and a semi-formal, yet friendly and approachable, prose style.
When presenting Python code, use single-quoted strings (``'hello'`` instead of
``"hello"``).
.. _reStructuredText: https://docutils.sourceforge.net/rst.html
.. _Sphinx: https://www.sphinx-doc.org/en/master/
.. _reStructuredText: http://docutils.sourceforge.net/rst.html
.. _Sphinx: http://sphinx-doc.org/index.html
.. _bug-reports:
@@ -146,7 +187,7 @@ through the `GitHub issues`_, **both open and closed**, to confirm that the bug
hasn't been reported before. Duplicate bug reports are a huge drain on the time
of other contributors, and should be avoided as much as possible.
.. _GitHub issues: https://github.com/psf/requests/issues
.. _GitHub issues: https://github.com/requests/requests/issues
Feature Requests
@@ -162,4 +203,4 @@ while keeping an open ear and mind.
If you believe there is a feature missing, feel free to raise a feature
request, but please do be aware that the overwhelming likelihood is that your
feature request will not be accepted.
feature request will not be accepted.

41
docs/dev/philosophy.rst Normal file
View File

@@ -0,0 +1,41 @@
Development Philosophy
======================
Requests is an open but opinionated library, created by an open but opinionated developer.
Management Style
~~~~~~~~~~~~~~~~
`Kenneth Reitz <http://kennethreitz.org>`_ is the BDFL. He has final say in any decision related to the Requests project. Kenneth is responsible for the direction and form of the library. In addition to making decisions based on technical merit, he is responsible for making decisions based on the development philosophy of Requests. Only Kenneth may merge code into Requests.
`Ian Cordasco <http://www.coglib.com/~icordasc/>`_ and `Cory Benfield <https://lukasa.co.uk/about/>`_ are the core contributors. They are responsible for triaging bug reports, reviewing pull requests and ensuring that Kenneth is kept up to speed with developments around the library. The day-to-day managing of the project is done by the core contributors. They are responsible for making judgements about whether or not a feature request is likely to be accepted by Kenneth. They do not have the authority to change code or merge code changes, though they may change documentation. Their word is not final.
Values
~~~~~~
- Simplicity is always better than functionality.
- Listen to everyone, then disregard it.
- The API is all that matters. Everything else is secondary.
- Fit the 90% use-case. Ignore the nay-sayers.
Semantic Versioning
~~~~~~~~~~~~~~~~~~~
For many years, the open source community has been plagued with version number dystonia. Numbers vary so greatly from project to project, they are practically meaningless.
Requests uses `Semantic Versioning <http://semver.org>`_. This specification seeks to put an end to this madness with a small set of practical guidelines for you and your colleagues to use in your next project.
Standard Library?
~~~~~~~~~~~~~~~~~
Requests has no *active* plans to be included in the standard library. This decision has been discussed at length with Guido as well as numerous core developers.
Essentially, the standard library is where a library goes to die. It is appropriate for a module to be included when active development is no longer necessary.
Linux Distro Packages
~~~~~~~~~~~~~~~~~~~~~
Distributions have been made for many Linux repositories, including: Ubuntu, Debian, RHEL, and Arch.
These distributions are sometimes divergent forks, or are otherwise not kept up-to-date with the latest code and bugfixes. PyPI (and its mirrors) and GitHub are the official distribution sources; alternatives are not supported by the Requests project.

71
docs/dev/todo.rst Normal file
View File

@@ -0,0 +1,71 @@
How to Help
===========
Requests is under active development, and contributions are more than welcome!
#. Check for open issues or open a fresh issue to start a discussion around a bug.
There is a Contributor Friendly tag for issues that should be ideal for people who are not very
familiar with the codebase yet.
#. Fork `the repository <https://github.com/requests/requests>`_ on GitHub and start making your
changes to a new branch.
#. Write a test which shows that the bug was fixed.
#. Send a pull request and bug the maintainer until it gets merged and published. :)
Make sure to add yourself to `AUTHORS <https://github.com/requests/requests/blob/master/AUTHORS.rst>`_.
Feature Freeze
--------------
As of v1.0.0, Requests has now entered a feature freeze. Requests for new
features and Pull Requests implementing those features will not be accepted.
Development Dependencies
------------------------
You'll need to install py.test in order to run the Requests' test suite::
$ pip install pipenv
$ pipenv lock
$ pipenv install --dev
$ pipenv run py.test tests
============================= test session starts ==============================
platform darwin -- Python 3.4.4, pytest-3.0.6, py-1.4.32, pluggy-0.4.0
...
collected 445 items
tests/test_hooks.py ...
tests/test_lowlevel.py ............
tests/test_requests.py ...........................................................
tests/test_structures.py ....................
tests/test_testserver.py ...........
tests/test_utils.py ..s...........................................................
============== 442 passed, 1 skipped, 2 xpassed in 46.48 seconds ===============
Runtime Environments
--------------------
Requests currently supports the following versions of Python:
- Python 2.6
- Python 2.7
- Python 3.3
- Python 3.4
- Python 3.5
- Python 3.6
- PyPy
Google AppEngine is not officially supported although support is available
with the `Requests-Toolbelt`_.
.. _Requests-Toolbelt: http://toolbelt.readthedocs.io/
Are you crazy?
--------------
- SPDY support would be awesome. No C extensions.
Downstream Repackaging
----------------------
If you are repackaging Requests, please note that you must also redistribute the ``cacerts.pem`` file in order to get correct SSL functionality.

View File

@@ -3,33 +3,34 @@
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Requests: HTTP for Humans
==========================
Requests: HTTP for Humans
=========================
Release v\ |version|. (:ref:`Installation <install>`)
.. image:: https://img.shields.io/pypi/l/requests.svg
:target: https://pypi.python.org/pypi/requests
.. image:: https://img.shields.io/pypi/v/requests.svg?maxAge=86400
:target: https://pypi.org/project/requests/
:alt: PyPI Version Badge
.. image:: https://img.shields.io/pypi/wheel/requests.svg
:target: https://pypi.python.org/pypi/requests
.. image:: https://img.shields.io/pypi/pyversions/requests.svg
:target: https://pypi.org/project/requests/
:alt: Supported Versions Badge
:target: https://pypi.python.org/pypi/requests
.. image:: https://static.pepy.tech/badge/requests/month
:target: https://pepy.tech/project/requests
:alt: Downloads Per Month Badge
.. image:: https://codecov.io/github/requests/requests/coverage.svg?branch=master
:target: https://codecov.io/github/requests/requests
:alt: codecov.io
.. image:: https://img.shields.io/github/contributors/psf/requests.svg
:target: https://github.com/psf/requests/graphs/contributors
:alt: Contributors Badge
.. image:: https://img.shields.io/badge/Say%20Thanks!-🦉-1EAEDB.svg
:target: https://saythanks.io/to/kennethreitz
.. image:: https://readthedocs.org/projects/requests/badge/?version=latest
:target: https://requests.readthedocs.io
:alt: Documentation Badge
**Requests** is an elegant and simple HTTP library for Python, built for human beings.
**Requests** is the only *Non-GMO* HTTP library for Python, safe for human
consumption.
*Warning: Recreational use of the Python standard library for HTTP may result in dangerous side-effects,
including: security vulnerabilities, verbose code, reinventing the wheel,
constantly reading documentation, depression, headaches, or even death.*
-------------------
@@ -43,17 +44,44 @@ Release v\ |version|. (:ref:`Installation <install>`)
>>> r.encoding
'utf-8'
>>> r.text
'{"type":"User"...'
u'{"type":"User"...'
>>> r.json()
{'private_gists': 419, 'total_private_repos': 77, ...}
{u'private_gists': 419, u'total_private_repos': 77, ...}
See `similar code, sans Requests <https://gist.github.com/973705>`_.
**Requests** allows you to send HTTP/1.1 requests extremely easily.
There's no need to manually add query strings to your
**Requests** allows you to send *organic, grass-fed* HTTP/1.1 requests, without the
need for manual labor. There's no need to manually add query strings to your
URLs, or to form-encode your POST data. Keep-alive and HTTP connection pooling
are 100% automatic, thanks to `urllib3 <https://github.com/urllib3/urllib3>`_.
are 100% automatic, thanks to `urllib3 <https://github.com/shazow/urllib3>`_.
User Testimonials
-----------------
Twitter, Spotify, Microsoft, Amazon, Lyft, BuzzFeed, Reddit, The NSA, Her Majesty's Government, Google, Twilio, Runscope, Mozilla, Heroku,
PayPal, NPR, Obama for America, Transifex, Native Instruments, The Washington
Post, SoundCloud, Kippt, Sony, and Federal U.S.
Institutions that prefer to be unnamed claim to use Requests internally.
**Armin Ronacher**
*Requests is the perfect example how beautiful an API can be with the
right level of abstraction.*
**Matt DeBoard**
*I'm going to get Kenneth Reitz's Python requests module tattooed
on my body, somehow. The whole thing.*
**Daniel Greenfeld**
*Nuked a 1200 LOC spaghetti code library with 10 lines of code thanks to
Kenneth Reitz's request library. Today has been AWESOME.*
**Kenny Meyers**
*Python HTTP: When in doubt, or when not in doubt, use Requests. Beautiful,
simple, Pythonic.*
Requests is one of the most downloaded Python packages of all time, pulling in
over 11,000,000 downloads every month. All the cool kids are doing it!
Beloved Features
----------------
@@ -76,7 +104,7 @@ Requests is ready for today's web.
- Chunked Requests
- ``.netrc`` Support
Requests officially supports Python 3.10+, and runs great on PyPy.
Requests officially supports Python 2.62.7 & 3.33.7, and runs great on PyPy.
The User Guide
@@ -89,6 +117,7 @@ instructions for getting the most out of Requests.
.. toctree::
:maxdepth: 2
user/intro
user/install
user/quickstart
user/advanced
@@ -102,19 +131,15 @@ This part of the documentation, which is mostly prose, details the
Requests ecosystem and community.
.. toctree::
:maxdepth: 2
:maxdepth: 1
community/recommended
community/faq
community/recommended
community/out-there
community/support
community/vulnerabilities
community/release-process
.. toctree::
:maxdepth: 1
community/updates
community/release-process
The API Documentation / Guide
-----------------------------
@@ -138,6 +163,8 @@ you.
:maxdepth: 3
dev/contributing
dev/philosophy
dev/todo
dev/authors
There are no more guides. You are now guideless.

View File

@@ -65,7 +65,7 @@ if errorlevel 9009 (
echo.may add the Sphinx directory to PATH.
echo.
echo.If you don't have Sphinx installed, grab it from
echo.https://www.sphinx-doc.org/
echo.http://sphinx-doc.org/
exit /b 1
)

View File

@@ -1,3 +0,0 @@
# Pinning to avoid unexpected breakages.
# Used by RTD to generate docs.
Sphinx==7.2.6

View File

@@ -23,8 +23,8 @@ Let's persist some cookies across requests::
s = requests.Session()
s.get('https://httpbin.org/cookies/set/sessioncookie/123456789')
r = s.get('https://httpbin.org/cookies')
s.get('http://httpbin.org/cookies/set/sessioncookie/123456789')
r = s.get('http://httpbin.org/cookies')
print(r.text)
# '{"cookies": {"sessioncookie": "123456789"}}'
@@ -38,7 +38,7 @@ is done by providing data to the properties on a Session object::
s.headers.update({'x-test': 'true'})
# both 'x-test' and 'x-test2' are sent
s.get('https://httpbin.org/headers', headers={'x-test2': 'true'})
s.get('http://httpbin.org/headers', headers={'x-test2': 'true'})
Any dictionaries that you pass to a request method will be merged with the
@@ -51,11 +51,11 @@ with the first request, but not the second::
s = requests.Session()
r = s.get('https://httpbin.org/cookies', cookies={'from-my': 'browser'})
r = s.get('http://httpbin.org/cookies', cookies={'from-my': 'browser'})
print(r.text)
# '{"cookies": {"from-my": "browser"}}'
r = s.get('https://httpbin.org/cookies')
r = s.get('http://httpbin.org/cookies')
print(r.text)
# '{"cookies": {}}'
@@ -67,7 +67,7 @@ If you want to manually add cookies to your session, use the
Sessions can also be used as context managers::
with requests.Session() as s:
s.get('https://httpbin.org/cookies/set/sessioncookie/123456789')
s.get('http://httpbin.org/cookies/set/sessioncookie/123456789')
This will make sure the session is closed as soon as the ``with`` block is
exited, even if unhandled exceptions occurred.
@@ -95,7 +95,7 @@ The ``Response`` object contains all of the information returned by the server a
also contains the ``Request`` object you created originally. Here is a simple
request to get some very important information from Wikipedia's servers::
>>> r = requests.get('https://en.wikipedia.org/wiki/Monty_Python')
>>> r = requests.get('http://en.wikipedia.org/wiki/Monty_Python')
If we want to access the headers the server sent back to us, we do this::
@@ -187,25 +187,6 @@ applied, replace the call to :meth:`Request.prepare()
print(resp.status_code)
When you are using the prepared request flow, keep in mind that it does not take into account the environment.
This can cause problems if you are using environment variables to change the behaviour of requests.
For example: Self-signed SSL certificates specified in ``REQUESTS_CA_BUNDLE`` will not be taken into account.
As a result an ``SSL: CERTIFICATE_VERIFY_FAILED`` is thrown.
You can get around this behaviour by explicitly merging the environment settings into your session::
from requests import Request, Session
s = Session()
req = Request('GET', url)
prepped = s.prepare_request(req)
# Merge environment settings into session
settings = s.merge_environment_settings(prepped.url, {}, None, None, None)
resp = s.send(prepped, **settings)
print(resp.status_code)
.. _verification:
SSL Cert Verification
@@ -233,22 +214,15 @@ or persistent::
s.verify = '/path/to/certfile'
.. note:: If ``verify`` is set to a path to a directory, the directory must have been processed using
the ``c_rehash`` utility supplied with OpenSSL.
the c_rehash utility supplied with OpenSSL.
This list of trusted CAs can also be specified through the ``REQUESTS_CA_BUNDLE`` environment variable.
If ``REQUESTS_CA_BUNDLE`` is not set, ``CURL_CA_BUNDLE`` will be used as fallback.
Requests can also ignore verifying the SSL certificate if you set ``verify`` to False::
>>> requests.get('https://kennethreitz.org', verify=False)
<Response [200]>
Note that when ``verify`` is set to ``False``, requests will accept any TLS
certificate presented by the server, and will ignore hostname mismatches
and/or expired certificates, which will make your application vulnerable to
man-in-the-middle (MitM) attacks. Setting verify to ``False`` may be useful
during local development or testing.
By default, ``verify`` is set to True. Option ``verify`` only applies to host certs.
Client Side Certificates
@@ -279,20 +253,21 @@ If you specify a wrong path or an invalid cert, you'll get a SSLError::
CA Certificates
---------------
Requests uses certificates from the package `certifi`_. This allows for users
to update their trusted certificates without changing the version of Requests.
By default, Requests bundles a set of root CAs that it trusts, sourced from the
`Mozilla trust store`_. However, these are only updated once for each Requests
version. This means that if you pin a Requests version your certificates can
become extremely out of date.
Before version 2.16, Requests bundled a set of root CAs that it trusted,
sourced from the `Mozilla trust store`_. The certificates were only updated
once for each Requests version. When ``certifi`` was not installed, this led to
extremely out-of-date certificate bundles when using significantly older
versions of Requests.
From Requests version 2.4.0 onwards, Requests will attempt to use certificates
from `certifi`_ if it is present on the system. This allows for users to update
their trusted certificates without having to change the code that runs on their
system.
For the sake of security we recommend upgrading certifi frequently!
.. _HTTP persistent connection: https://en.wikipedia.org/wiki/HTTP_persistent_connection
.. _connection pooling: https://urllib3.readthedocs.io/en/latest/reference/urllib3.connectionpool.html
.. _certifi: https://certifiio.readthedocs.io/
.. _connection pooling: http://urllib3.readthedocs.io/en/latest/reference/index.html#module-urllib3.connectionpool
.. _certifi: http://certifi.io/
.. _Mozilla trust store: https://hg.mozilla.org/mozilla-central/raw-file/tip/security/nss/lib/ckfw/builtins/certdata.txt
.. _body-content-workflow:
@@ -305,7 +280,7 @@ immediately. You can override this behaviour and defer downloading the response
body until you access the :attr:`Response.content <requests.Response.content>`
attribute with the ``stream`` parameter::
tarball_url = 'https://github.com/psf/requests/tarball/main'
tarball_url = 'https://github.com/requests/requests/tarball/master'
r = requests.get(tarball_url, stream=True)
At this point only the response headers have been downloaded and the connection
@@ -326,11 +301,15 @@ release the connection back to the pool unless you consume all the data or call
:meth:`Response.close <requests.Response.close>`. This can lead to
inefficiency with connections. If you find yourself partially reading request
bodies (or not reading them at all) while using ``stream=True``, you should
make the request within a ``with`` statement to ensure it's always closed::
consider using ``contextlib.closing`` (`documented here`_), like this::
with requests.get('https://httpbin.org/get', stream=True) as r:
from contextlib import closing
with closing(requests.get('http://httpbin.org/get', stream=True)) as r:
# Do things with the response here.
.. _`documented here`: http://docs.python.org/2/library/contextlib.html#contextlib.closing
.. _keep-alive:
Keep-Alive
@@ -356,11 +335,13 @@ file-like object for your body::
with open('massive-body', 'rb') as f:
requests.post('http://some.url/streamed', data=f)
.. warning:: It is strongly recommended that you open files in :ref:`binary
mode <tut-files>`. This is because Requests may attempt to provide
the ``Content-Length`` header for you, and if it does this value
will be set to the number of *bytes* in the file. Errors may occur
if you open the file in *text mode*.
.. warning:: It is strongly recommended that you open files in `binary mode`_.
This is because Requests may attempt to provide the
``Content-Length`` header for you, and if it does this value will
be set to the number of *bytes* in the file. Errors may occur if
you open the file in *text mode*.
.. _binary mode: https://docs.python.org/2/tutorial/inputoutput.html#reading-and-writing-files
.. _chunk-encoding:
@@ -398,10 +379,10 @@ upload image files to an HTML form with a multiple file field 'images'::
To do that, just set files to a list of tuples of ``(form_field_name, file_info)``::
>>> url = 'https://httpbin.org/post'
>>> url = 'http://httpbin.org/post'
>>> multiple_files = [
... ('images', ('foo.png', open('foo.png', 'rb'), 'image/png')),
... ('images', ('bar.png', open('bar.png', 'rb'), 'image/png'))]
('images', ('foo.png', open('foo.png', 'rb'), 'image/png')),
('images', ('bar.png', open('bar.png', 'rb'), 'image/png'))]
>>> r = requests.post(url, files=multiple_files)
>>> r.text
{
@@ -411,11 +392,13 @@ To do that, just set files to a list of tuples of ``(form_field_name, file_info)
...
}
.. warning:: It is strongly recommended that you open files in :ref:`binary
mode <tut-files>`. This is because Requests may attempt to provide
the ``Content-Length`` header for you, and if it does this value
will be set to the number of *bytes* in the file. Errors may occur
if you open the file in *text mode*.
.. warning:: It is strongly recommended that you open files in `binary mode`_.
This is because Requests may attempt to provide the
``Content-Length`` header for you, and if it does this value will
be set to the number of *bytes* in the file. Errors may occur if
you open the file in *text mode*.
.. _binary mode: https://docs.python.org/2/tutorial/inputoutput.html#reading-and-writing-files
.. _event-hooks:
@@ -436,7 +419,7 @@ You can assign a hook function on a per-request basis by passing a
``{hook_name: callback_function}`` dictionary to the ``hooks`` request
parameter::
hooks={'response': print_url}
hooks=dict(response=print_url)
That ``callback_function`` will receive a chunk of data as its first
argument.
@@ -446,49 +429,24 @@ argument.
def print_url(r, *args, **kwargs):
print(r.url)
Your callback function must handle its own exceptions. Any unhandled exception won't be passed silently and thus should be handled by the code calling Requests.
If an error occurs while executing your callback, a warning is given.
If the callback function returns a value, it is assumed that it is to
replace the data that was passed in. If the function doesn't return
anything, nothing else is affected.
::
def record_hook(r, *args, **kwargs):
r.hook_called = True
return r
anything, nothing else is effected.
Let's print some request method arguments at runtime::
>>> requests.get('https://httpbin.org/', hooks={'response': print_url})
https://httpbin.org/
>>> requests.get('http://httpbin.org', hooks=dict(response=print_url))
http://httpbin.org
<Response [200]>
You can add multiple hooks to a single request. Let's call two hooks at once::
>>> r = requests.get('https://httpbin.org/', hooks={'response': [print_url, record_hook]})
https://httpbin.org/
>>> r.hook_called
True
You can also add hooks to a ``Session`` instance. Any hooks you add will then
be called on every request made to the session. For example::
>>> s = requests.Session()
>>> s.hooks['response'].append(print_url)
>>> s.get('https://httpbin.org/')
https://httpbin.org/
<Response [200]>
A ``Session`` can have multiple hooks, which will be called in the order
they are added.
.. _custom-auth:
Custom Authentication
---------------------
Requests allows you to specify your own authentication mechanism.
Requests allows you to use specify your own authentication mechanism.
Any callable which is passed as the ``auth`` argument to a request method will
have the opportunity to modify the request before it is dispatched.
@@ -535,7 +493,7 @@ set ``stream`` to ``True`` and iterate over the response with
import json
import requests
r = requests.get('https://httpbin.org/stream/20', stream=True)
r = requests.get('http://httpbin.org/stream/20', stream=True)
for line in r.iter_lines():
@@ -549,7 +507,7 @@ When using `decode_unicode=True` with
:meth:`Response.iter_content() <requests.Response.iter_content>`, you'll want
to provide a fallback encoding in the event the server doesn't provide one::
r = requests.get('https://httpbin.org/stream/20', stream=True)
r = requests.get('http://httpbin.org/stream/20', stream=True)
if r.encoding is None:
r.encoding = 'utf-8'
@@ -590,55 +548,21 @@ If you need to use a proxy, you can configure individual requests with the
requests.get('http://example.org', proxies=proxies)
Alternatively you can configure it once for an entire
:class:`Session <requests.Session>`::
You can also configure proxies by setting the environment variables
``HTTP_PROXY`` and ``HTTPS_PROXY``.
import requests
proxies = {
'http': 'http://10.10.1.10:3128',
'https': 'http://10.10.1.10:1080',
}
session = requests.Session()
session.proxies.update(proxies)
session.get('http://example.org')
.. warning:: Setting ``session.proxies`` may behave differently than expected.
Values provided will be overwritten by environmental proxies
(those returned by `urllib.request.getproxies <https://docs.python.org/3/library/urllib.request.html#urllib.request.getproxies>`_).
To ensure the use of proxies in the presence of environmental proxies,
explicitly specify the ``proxies`` argument on all individual requests as
initially explained above.
See `#2018 <https://github.com/psf/requests/issues/2018>`_ for details.
When the proxies configuration is not overridden per request as shown above,
Requests relies on the proxy configuration defined by standard
environment variables ``http_proxy``, ``https_proxy``, ``no_proxy``,
and ``all_proxy``. Uppercase variants of these variables are also supported.
You can therefore set them to configure Requests (only set the ones relevant
to your needs)::
::
$ export HTTP_PROXY="http://10.10.1.10:3128"
$ export HTTPS_PROXY="http://10.10.1.10:1080"
$ export ALL_PROXY="socks5://10.10.1.10:3434"
$ python
>>> import requests
>>> requests.get('http://example.org')
To use HTTP Basic Auth with your proxy, use the `http://user:password@host/`
syntax in any of the above configuration entries::
To use HTTP Basic Auth with your proxy, use the `http://user:password@host/` syntax::
$ export HTTPS_PROXY="http://user:pass@10.10.1.10:1080"
$ python
>>> proxies = {'http': 'http://user:pass@10.10.1.10:3128/'}
.. warning:: Storing sensitive username and password information in an
environment variable or a version-controlled file is a security risk and is
highly discouraged.
proxies = {'http': 'http://user:pass@10.10.1.10:3128/'}
To give a proxy for a specific scheme and host, use the
`scheme://hostname` form for the key. This will match for
@@ -650,25 +574,6 @@ any request to the given scheme and exact hostname.
Note that proxy URLs must include the scheme.
Finally, note that using a proxy for https connections typically requires your
local machine to trust the proxy's root certificate. By default the list of
certificates trusted by Requests can be found with::
from requests.utils import DEFAULT_CA_BUNDLE_PATH
print(DEFAULT_CA_BUNDLE_PATH)
You override this default certificate bundle by setting the ``REQUESTS_CA_BUNDLE``
(or ``CURL_CA_BUNDLE``) environment variable to another file path::
$ export REQUESTS_CA_BUNDLE="/usr/local/myproxy_info/cacert.pem"
$ export https_proxy="http://10.10.1.10:1080"
$ python
>>> import requests
>>> requests.get('https://example.org')
.. _socks:
SOCKS
^^^^^
@@ -678,9 +583,11 @@ In addition to basic HTTP proxies, Requests also supports proxies using the
SOCKS protocol. This is an optional feature that requires that additional
third-party libraries be installed before use.
You can get the dependencies for this feature from ``pip``::
You can get the dependencies for this feature from ``pip``:
$ python -m pip install 'requests[socks]'
.. code-block:: bash
$ pip install requests[socks]
Once you've installed those dependencies, using a SOCKS proxy is just as easy
as using a HTTP one::
@@ -690,8 +597,6 @@ as using a HTTP one::
'https': 'socks5://user:pass@host:port'
}
Using the scheme ``socks5`` causes the DNS resolution to happen on the client, rather than on the proxy server. This is in line with curl, which uses the scheme to decide whether to do the DNS resolution on the client or proxy. If you want to resolve the domains on the proxy server, use ``socks5h`` as the scheme.
.. _compliance:
Compliance
@@ -708,24 +613,13 @@ Encodings
When you receive a response, Requests makes a guess at the encoding to
use for decoding the response when you access the :attr:`Response.text
<requests.Response.text>` attribute. Requests will first check for an
encoding in the HTTP header, and if none is present, will use
`charset_normalizer <https://pypi.org/project/charset_normalizer/>`_
or `chardet <https://github.com/chardet/chardet>`_ to attempt to
guess the encoding.
encoding in the HTTP header, and if none is present, will use `chardet
<http://pypi.python.org/pypi/chardet>`_ to attempt to guess the encoding.
If ``chardet`` is installed, ``requests`` uses it, however for python3
``chardet`` is no longer a mandatory dependency. The ``chardet``
library is an LGPL-licenced dependency and some users of requests
cannot depend on mandatory LGPL-licensed dependencies.
When you install ``requests`` without specifying ``[use_chardet_on_py3]`` extra,
and ``chardet`` is not already installed, ``requests`` uses ``charset-normalizer``
(MIT-licensed) to guess the encoding.
The only time Requests will not guess the encoding is if no explicit charset
The only time Requests will not do this is if no explicit charset
is present in the HTTP headers **and** the ``Content-Type``
header contains ``text``. In this situation, `RFC 2616
<https://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.7.1>`_ specifies
<http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.7.1>`_ specifies
that the default charset must be ``ISO-8859-1``. Requests follows the
specification in this case. If you require a different encoding, you can
manually set the :attr:`Response.encoding <requests.Response.encoding>`
@@ -748,7 +642,7 @@ from GitHub. Suppose we wanted commit ``a050faf`` on Requests. We would get it
like so::
>>> import requests
>>> r = requests.get('https://api.github.com/repos/psf/requests/git/commits/a050faf084662f3a352dd1a941f2c7c9f886d4ad')
>>> r = requests.get('https://api.github.com/repos/requests/requests/git/commits/a050faf084662f3a352dd1a941f2c7c9f886d4ad')
We should confirm that GitHub responded correctly. If it has, we want to work
out what type of content it is. Do this like so::
@@ -766,12 +660,12 @@ So, GitHub returns JSON. That's great, we can use the :meth:`r.json
>>> commit_data = r.json()
>>> print(commit_data.keys())
['committer', 'author', 'url', 'tree', 'sha', 'parents', 'message']
[u'committer', u'author', u'url', u'tree', u'sha', u'parents', u'message']
>>> print(commit_data['committer'])
{'date': '2012-05-10T11:10:50-07:00', 'email': 'me@kennethreitz.com', 'name': 'Kenneth Reitz'}
>>> print(commit_data[u'committer'])
{u'date': u'2012-05-10T11:10:50-07:00', u'email': u'me@kennethreitz.com', u'name': u'Kenneth Reitz'}
>>> print(commit_data['message'])
>>> print(commit_data[u'message'])
makin' history
So far, so simple. Well, let's investigate the GitHub API a little bit. Now,
@@ -803,37 +697,37 @@ we should probably avoid making ham-handed POSTS to it. Instead, let's play
with the Issues feature of GitHub.
This documentation was added in response to
`Issue #482 <https://github.com/psf/requests/issues/482>`_. Given that
`Issue #482 <https://github.com/requests/requests/issues/482>`_. Given that
this issue already exists, we will use it as an example. Let's start by getting it.
::
>>> r = requests.get('https://api.github.com/repos/psf/requests/issues/482')
>>> r = requests.get('https://api.github.com/repos/requests/requests/issues/482')
>>> r.status_code
200
>>> issue = json.loads(r.text)
>>> print(issue['title'])
>>> print(issue[u'title'])
Feature any http verb in docs
>>> print(issue['comments'])
>>> print(issue[u'comments'])
3
Cool, we have three comments. Let's take a look at the last of them.
::
>>> r = requests.get(r.url + '/comments')
>>> r = requests.get(r.url + u'/comments')
>>> r.status_code
200
>>> comments = r.json()
>>> print(comments[0].keys())
['body', 'url', 'created_at', 'updated_at', 'user', 'id']
[u'body', u'url', u'created_at', u'updated_at', u'user', u'id']
>>> print(comments[2]['body'])
>>> print(comments[2][u'body'])
Probably in the "advanced" section
Well, that seems like a silly place. Let's post a comment telling the poster
@@ -841,7 +735,7 @@ that he's silly. Who is the poster, anyway?
::
>>> print(comments[2]['user']['login'])
>>> print(comments[2][u'user'][u'login'])
kennethreitz
OK, so let's tell this Kenneth guy that we think this example should go in the
@@ -851,7 +745,7 @@ is to POST to the thread. Let's do it.
::
>>> body = json.dumps({u"body": u"Sounds great! I'll get right on it!"})
>>> url = u"https://api.github.com/repos/psf/requests/issues/482/comments"
>>> url = u"https://api.github.com/repos/requests/requests/issues/482/comments"
>>> r = requests.post(url=url, data=body)
>>> r.status_code
@@ -871,7 +765,7 @@ the very common Basic Auth.
201
>>> content = r.json()
>>> print(content['body'])
>>> print(content[u'body'])
Sounds great! I'll get right on it.
Brilliant. Oh, wait, no! I meant to add that it would take me a while, because
@@ -885,7 +779,7 @@ that.
5804413
>>> body = json.dumps({u"body": u"Sounds great! I'll get right on it once I feed my cat."})
>>> url = u"https://api.github.com/repos/psf/requests/issues/comments/5804413"
>>> url = u"https://api.github.com/repos/requests/requests/issues/comments/5804413"
>>> r = requests.patch(url=url, data=body, auth=auth)
>>> r.status_code
@@ -947,7 +841,7 @@ Link Headers
Many HTTP APIs feature Link headers. They make APIs more self describing and
discoverable.
GitHub uses these for `pagination <https://docs.github.com/en/rest/guides/using-pagination-in-the-rest-api>`_
GitHub uses these for `pagination <http://developer.github.com/v3/#pagination>`_
in their API, for example::
>>> url = 'https://api.github.com/users/kennethreitz/repos?page=1&per_page=10'
@@ -968,9 +862,11 @@ Requests will automatically parse these link headers and make them easily consum
Transport Adapters
------------------
As of v1.0.0, Requests has moved to a modular internal design using Transport
Adapters. These objects provide a mechanism to define interaction methods for an
HTTP service. In particular, they allow you to apply per-service configuration.
As of v1.0.0, Requests has moved to a modular internal design. Part of the
reason this was done was to implement Transport Adapters, originally
`described here`_. Transport Adapters provide a mechanism to define interaction
methods for an HTTP service. In particular, they allow you to apply per-service
configuration.
Requests ships with a single Transport Adapter, the :class:`HTTPAdapter
<requests.adapters.HTTPAdapter>`. This adapter provides the default Requests
@@ -987,16 +883,12 @@ it should apply to.
::
>>> s = requests.Session()
>>> s.mount('https://github.com/', MyAdapter())
>>> s.mount('http://www.github.com', MyAdapter())
The mount call registers a specific instance of a Transport Adapter to a
prefix. Once mounted, any HTTP request made using that session whose URL starts
with the given prefix will use the given Transport Adapter.
.. note:: The adapter will be chosen based on a longest prefix match. Be mindful
prefixes such as ``http://localhost`` will also match ``http://localhost.other.com``
or ``http://localhost@other.com``. It's recommended to terminate full hostnames with a ``/``.
Many of the details of implementing a Transport Adapter are beyond the scope of
this documentation, but take a look at the next example for a simple SSL use-
case. For more than that, you might look at subclassing the
@@ -1016,9 +908,9 @@ passed-through to `urllib3`. We'll make a Transport Adapter that instructs the
library to use SSLv3::
import ssl
from urllib3.poolmanager import PoolManager
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.poolmanager import PoolManager
class Ssl3HttpAdapter(HTTPAdapter):
@@ -1029,29 +921,8 @@ library to use SSLv3::
num_pools=connections, maxsize=maxsize,
block=block, ssl_version=ssl.PROTOCOL_SSLv3)
Example: Automatic Retries
^^^^^^^^^^^^^^^^^^^^^^^^^^
By default, Requests does not retry failed connections. However, it is possible
to implement automatic retries with a powerful array of features, including
backoff, within a Requests :class:`Session <requests.Session>` using the
`urllib3.util.Retry`_ class::
from urllib3.util import Retry
from requests import Session
from requests.adapters import HTTPAdapter
s = Session()
retries = Retry(
total=3,
backoff_factor=0.1,
status_forcelist=[502, 503, 504],
allowed_methods={'POST'},
)
s.mount('https://', HTTPAdapter(max_retries=retries))
.. _`urllib3`: https://github.com/urllib3/urllib3
.. _`urllib3.util.Retry`: https://urllib3.readthedocs.io/en/stable/reference/urllib3.util.html#urllib3.util.Retry
.. _`described here`: http://www.kennethreitz.org/essays/the-future-of-python-http
.. _`urllib3`: https://github.com/shazow/urllib3
.. _blocking-or-nonblocking:
@@ -1067,19 +938,17 @@ response at a time. However, these calls will still block.
If you are concerned about the use of blocking IO, there are lots of projects
out there that combine Requests with one of Python's asynchronicity frameworks.
Some excellent examples are `requests-threads`_, `grequests`_, `requests-futures`_, and `httpx`_.
Two excellent examples are `grequests`_ and `requests-futures`_.
.. _`requests-threads`: https://github.com/requests/requests-threads
.. _`grequests`: https://github.com/spyoungtech/grequests
.. _`grequests`: https://github.com/kennethreitz/grequests
.. _`requests-futures`: https://github.com/ross/requests-futures
.. _`httpx`: https://github.com/encode/httpx
Header Ordering
---------------
In unusual circumstances you may want to provide headers in an ordered manner. If you pass an ``OrderedDict`` to the ``headers`` keyword argument, that will provide the headers with an ordering. *However*, the ordering of the default headers used by Requests will be preferred, which means that if you override default headers in the ``headers`` keyword argument, they may appear out of order compared to other headers in that keyword argument.
If this is problematic, users should consider setting the default headers on a :class:`Session <requests.Session>` object, by setting :attr:`Session.headers <requests.Session.headers>` to a custom ``OrderedDict``. That ordering will always be preferred.
If this is problematic, users should consider setting the default headers on a :class:`Session <requests.Session>` object, by setting :attr:`Session <requests.Session.headers>` to a custom ``OrderedDict``. That ordering will always be preferred.
.. _timeouts:
@@ -1095,7 +964,7 @@ The **connect** timeout is the number of seconds Requests will wait for your
client to establish a connection to a remote machine (corresponding to the
`connect()`_) call on the socket. It's a good practice to set connect timeouts
to slightly larger than a multiple of 3, which is the default `TCP packet
retransmission window <https://datatracker.ietf.org/doc/html/rfc2988>`_.
retransmission window <http://www.hjp.at/doc/rfc/rfc2988.txt>`_.
Once your client has connected to the server and sent the HTTP request, the
**read** timeout is the number of seconds the client will wait for the server
@@ -1120,18 +989,4 @@ coffee.
r = requests.get('https://github.com', timeout=None)
.. note:: The connect timeout applies to each connection attempt to an IP address.
If multiple addresses exist for a domain name, the underlying ``urllib3`` will
try each address sequentially until one successfully connects.
This may lead to an effective total connection timeout *multiple* times longer
than the specified time, e.g. an unresponsive server having both IPv4 and IPv6
addresses will have its perceived timeout *doubled*, so take that into account
when setting the connection timeout.
.. note:: Neither the connect nor read timeouts are `wall clock`_. This means
that if you start a request, and look at the time, and then look at
the time when the request finishes or times out, the real-world time
may be greater than what you specified.
.. _`wall clock`: https://wiki.php.net/rfc/max_execution_wall_time
.. _`connect()`: https://linux.die.net/man/2/connect
.. _`connect()`: http://linux.die.net/man/2/connect

View File

@@ -19,14 +19,13 @@ the simplest kind, and Requests supports it straight out of the box.
Making requests with HTTP Basic Auth is very simple::
>>> from requests.auth import HTTPBasicAuth
>>> basic = HTTPBasicAuth('user', 'pass')
>>> requests.get('https://httpbin.org/basic-auth/user/pass', auth=basic)
>>> requests.get('https://api.github.com/user', auth=HTTPBasicAuth('user', 'pass'))
<Response [200]>
In fact, HTTP Basic Auth is so common that Requests provides a handy shorthand
for using it::
>>> requests.get('https://httpbin.org/basic-auth/user/pass', auth=('user', 'pass'))
>>> requests.get('https://api.github.com/user', auth=('user', 'pass'))
<Response [200]>
Providing the credentials in a tuple like this is exactly the same as the
@@ -44,16 +43,6 @@ set with `headers=`.
If credentials for the hostname are found, the request is sent with HTTP Basic
Auth.
Requests will search for the netrc file at `~/.netrc`, `~/_netrc`, or at the path
specified by the `NETRC` environment variable. `~` denotes the user's home
directory, which is `$HOME` on Unix based systems and `%USERPROFILE%` on Windows.
Usage of netrc file can be disabled by setting `trust_env` to `False` in the
Requests session::
>>> s = requests.Session()
>>> s.trust_env = False
>>> s.get('https://httpbin.org/basic-auth/user/pass')
Digest Authentication
---------------------
@@ -62,7 +51,7 @@ Another very popular form of HTTP Authentication is Digest Authentication,
and Requests supports this out of the box as well::
>>> from requests.auth import HTTPDigestAuth
>>> url = 'https://httpbin.org/digest-auth/auth/user/pass'
>>> url = 'http://httpbin.org/digest-auth/auth/user/pass'
>>> requests.get(url, auth=HTTPDigestAuth('user', 'pass'))
<Response [200]>
@@ -131,7 +120,7 @@ To do so, subclass :class:`AuthBase <requests.auth.AuthBase>` and implement the
... # Implement my authentication
... return r
...
>>> url = 'https://httpbin.org/get'
>>> url = 'http://httpbin.org/get'
>>> requests.get(url, auth=MyAuth())
<Response [200]>
@@ -143,13 +132,13 @@ authentication will additionally add hooks to provide further functionality.
Further examples can be found under the `Requests organization`_ and in the
``auth.py`` file.
.. _OAuth: https://oauth.net/
.. _OAuth: http://oauth.net/
.. _requests_oauthlib: https://github.com/requests/requests-oauthlib
.. _requests-oauthlib OAuth2 documentation: https://requests-oauthlib.readthedocs.io/en/latest/oauth2_workflow.html
.. _Web Application Flow: https://requests-oauthlib.readthedocs.io/en/latest/oauth2_workflow.html#web-application-flow
.. _Mobile Application Flow: https://requests-oauthlib.readthedocs.io/en/latest/oauth2_workflow.html#mobile-application-flow
.. _Legacy Application Flow: https://requests-oauthlib.readthedocs.io/en/latest/oauth2_workflow.html#legacy-application-flow
.. _Backend Application Flow: https://requests-oauthlib.readthedocs.io/en/latest/oauth2_workflow.html#backend-application-flow
.. _requests-oauthlib OAuth2 documentation: http://requests-oauthlib.readthedocs.io/en/latest/oauth2_workflow.html
.. _Web Application Flow: http://requests-oauthlib.readthedocs.io/en/latest/oauth2_workflow.html#web-application-flow
.. _Mobile Application Flow: http://requests-oauthlib.readthedocs.io/en/latest/oauth2_workflow.html#mobile-application-flow
.. _Legacy Application Flow: http://requests-oauthlib.readthedocs.io/en/latest/oauth2_workflow.html#legacy-application-flow
.. _Backend Application Flow: http://requests-oauthlib.readthedocs.io/en/latest/oauth2_workflow.html#backend-application-flow
.. _Kerberos: https://github.com/requests/requests-kerberos
.. _NTLM: https://github.com/requests/requests-ntlm
.. _Requests organization: https://github.com/requests

View File

@@ -1,36 +1,39 @@
.. _install:
Installation of Requests
========================
Installation
============
This part of the documentation covers the installation of Requests.
The first step to using any software package is getting it properly installed.
$ python -m pip install requests
--------------------------------
Pip Install Requests
--------------------
To install Requests, simply run this simple command in your terminal of choice::
$ python -m pip install requests
$ pip install requests
If you don't have `pip <https://pip.pypa.io>`_ installed (tisk tisk!),
`this Python installation guide <http://docs.python-guide.org/en/latest/starting/installation/>`_
can guide you through the process.
Get the Source Code
-------------------
Requests is actively developed on GitHub, where the code is
`always available <https://github.com/psf/requests>`_.
`always available <https://github.com/requests/requests>`_.
You can either clone the public repository::
$ git clone https://github.com/psf/requests.git
$ git clone git://github.com/requests/requests.git
Or, download the `tarball <https://github.com/psf/requests/tarball/main>`_::
Or, download the `tarball <https://github.com/requests/requests/tarball/master>`_::
$ curl -OL https://github.com/psf/requests/tarball/main
# optionally, zipball is also available (for Windows users).
$ curl -OL https://github.com/requests/requests/tarball/master
# optionally, zipball is also available (for Windows users).
Once you have a copy of the source, you can embed it in your own Python
package, or install it into your site-packages easily::
$ cd requests
$ python -m pip install .
$ python setup.py install

45
docs/user/intro.rst Normal file
View File

@@ -0,0 +1,45 @@
.. _introduction:
Introduction
============
Philosophy
----------
Requests was developed with a few :pep:`20` idioms in mind.
#. Beautiful is better than ugly.
#. Explicit is better than implicit.
#. Simple is better than complex.
#. Complex is better than complicated.
#. Readability counts.
All contributions to Requests should keep these important rules in mind.
.. _`apache2`:
Apache2 License
---------------
A large number of open source projects you find today are `GPL Licensed`_.
While the GPL has its time and place, it should most certainly not be your
go-to license for your next open source project.
A project that is released as GPL cannot be used in any commercial product
without the product itself also being offered as open source.
The MIT, BSD, ISC, and Apache2 licenses are great alternatives to the GPL
that allow your open-source software to be used freely in proprietary,
closed-source software.
Requests is released under terms of `Apache2 License`_.
.. _`GPL Licensed`: http://www.opensource.org/licenses/gpl-license.php
.. _`Apache2 License`: http://opensource.org/licenses/Apache-2.0
Requests License
----------------
.. include:: ../../LICENSE

View File

@@ -37,15 +37,15 @@ get all the information we need from this object.
Requests' simple API means that all forms of HTTP request are as obvious. For
example, this is how you make an HTTP POST request::
>>> r = requests.post('https://httpbin.org/post', data={'key': 'value'})
>>> r = requests.post('http://httpbin.org/post', data = {'key':'value'})
Nice, right? What about the other HTTP request types: PUT, DELETE, HEAD and
OPTIONS? These are all just as simple::
>>> r = requests.put('https://httpbin.org/put', data={'key': 'value'})
>>> r = requests.delete('https://httpbin.org/delete')
>>> r = requests.head('https://httpbin.org/get')
>>> r = requests.options('https://httpbin.org/get')
>>> r = requests.put('http://httpbin.org/put', data = {'key':'value'})
>>> r = requests.delete('http://httpbin.org/delete')
>>> r = requests.head('http://httpbin.org/get')
>>> r = requests.options('http://httpbin.org/get')
That's all well and good, but it's also only the start of what Requests can
do.
@@ -63,12 +63,12 @@ using the ``params`` keyword argument. As an example, if you wanted to pass
following code::
>>> payload = {'key1': 'value1', 'key2': 'value2'}
>>> r = requests.get('https://httpbin.org/get', params=payload)
>>> r = requests.get('http://httpbin.org/get', params=payload)
You can see that the URL has been correctly encoded by printing the URL::
>>> print(r.url)
https://httpbin.org/get?key2=value2&key1=value1
http://httpbin.org/get?key2=value2&key1=value1
Note that any dictionary key whose value is ``None`` will not be added to the
URL's query string.
@@ -77,9 +77,9 @@ You can also pass a list of items as a value::
>>> payload = {'key1': 'value1', 'key2': ['value2', 'value3']}
>>> r = requests.get('https://httpbin.org/get', params=payload)
>>> r = requests.get('http://httpbin.org/get', params=payload)
>>> print(r.url)
https://httpbin.org/get?key1=value1&key2=value2&key2=value3
http://httpbin.org/get?key1=value1&key2=value2&key2=value3
Response Content
----------------
@@ -91,7 +91,7 @@ again::
>>> r = requests.get('https://api.github.com/events')
>>> r.text
'[{"repository":{"open_issues":0,"url":"https://github.com/...
u'[{"repository":{"open_issues":0,"url":"https://github.com/...
Requests will automatically decode content from the server. Most unicode
charsets are seamlessly decoded.
@@ -108,7 +108,7 @@ using, and change it, using the ``r.encoding`` property::
If you change the encoding, Requests will use the new value of ``r.encoding``
whenever you call ``r.text``. You might want to do this in any situation where
you can apply special logic to work out what the encoding of the content will
be. For example, HTML and XML have the ability to specify their encoding in
be. For example, HTTP and XML have the ability to specify their encoding in
their body. In situations like this, you should use ``r.content`` to find the
encoding, and then set ``r.encoding``. This will let you use ``r.text`` with
the correct encoding.
@@ -128,9 +128,6 @@ You can also access the response body as bytes, for non-text requests::
The ``gzip`` and ``deflate`` transfer-encodings are automatically decoded for you.
The ``br`` transfer-encoding is automatically decoded for you if a Brotli library
like `brotli <https://pypi.org/project/brotli>`_ or `brotlicffi <https://pypi.org/project/brotlicffi>`_ is installed.
For example, to create an image from binary data returned by a request, you can
use the following code::
@@ -149,13 +146,11 @@ There's also a builtin JSON decoder, in case you're dealing with JSON data::
>>> r = requests.get('https://api.github.com/events')
>>> r.json()
[{'repository': {'open_issues': 0, 'url': 'https://github.com/...
[{u'repository': {u'open_issues': 0, u'url': 'https://github.com/...
In case the JSON decoding fails, ``r.json()`` raises an exception. For example, if
the response gets a 204 (No Content), or if the response contains invalid JSON,
attempting ``r.json()`` raises ``requests.exceptions.JSONDecodeError``. This wrapper exception
provides interoperability for multiple exceptions that may be thrown by different
python versions and json serialization libraries.
attempting ``r.json()`` raises ``ValueError: No JSON object could be decoded``.
It should be noted that the success of the call to ``r.json()`` does **not**
indicate the success of the response. Some servers may return a JSON object in a
@@ -174,10 +169,10 @@ server, you can access ``r.raw``. If you want to do this, make sure you set
>>> r = requests.get('https://api.github.com/events', stream=True)
>>> r.raw
<urllib3.response.HTTPResponse object at 0x101194810>
<requests.packages.urllib3.response.HTTPResponse object at 0x101194810>
>>> r.raw.read(10)
b'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03'
'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03'
In general, however, you should use a pattern like this to save what is being
streamed to a file::
@@ -192,16 +187,6 @@ download, the above is the preferred and recommended way to retrieve the
content. Note that ``chunk_size`` can be freely adjusted to a number that
may better fit your use cases.
.. note::
An important note about using ``Response.iter_content`` versus ``Response.raw``.
``Response.iter_content`` will automatically decode the ``gzip`` and ``deflate``
transfer-encodings. ``Response.raw`` is a raw stream of bytes -- it does not
transform the response content. If you really need access to the bytes as they
were returned, use ``Response.raw``.
.. _custom-headers:
Custom Headers
--------------
@@ -220,9 +205,7 @@ Note: Custom headers are given less precedence than more specific sources of inf
* Authorization headers set with `headers=` will be overridden if credentials
are specified in ``.netrc``, which in turn will be overridden by the ``auth=``
parameter. Requests will search for the netrc file at `~/.netrc`, `~/_netrc`,
or at the path specified by the `NETRC` environment variable.
Check details in :ref:`netrc authentication <authentication>`.
parameter.
* Authorization headers will be removed if you get redirected off-host.
* Proxy-Authorization headers will be overridden by proxy credentials provided in the URL.
* Content-Length headers will be overridden when we can determine the length of the content.
@@ -240,7 +223,7 @@ dictionary of data will automatically be form-encoded when the request is made::
>>> payload = {'key1': 'value1', 'key2': 'value2'}
>>> r = requests.post('https://httpbin.org/post', data=payload)
>>> r = requests.post("http://httpbin.org/post", data=payload)
>>> print(r.text)
{
...
@@ -251,16 +234,12 @@ dictionary of data will automatically be form-encoded when the request is made::
...
}
The ``data`` argument can also have multiple values for each key. This can be
done by making ``data`` either a list of tuples or a dictionary with lists
as values. This is particularly useful when the form has multiple elements that
use the same key::
You can also pass a list of tuples to the ``data`` argument. This is particularly
useful when the form has multiple elements that use the same key::
>>> payload_tuples = [('key1', 'value1'), ('key1', 'value2')]
>>> r1 = requests.post('https://httpbin.org/post', data=payload_tuples)
>>> payload_dict = {'key1': ['value1', 'value2']}
>>> r2 = requests.post('https://httpbin.org/post', data=payload_dict)
>>> print(r1.text)
>>> payload = (('key1', 'value1'), ('key1', 'value2'))
>>> r = requests.post('http://httpbin.org/post', data=payload)
>>> print(r.text)
{
...
"form": {
@@ -271,10 +250,6 @@ use the same key::
},
...
}
>>> # httpbin may embed non-deterministic metadata,
>>> # so we only compare our submitted data here.
>>> r1.json()['form'] == r2.json()['form']
True
There are times that you may want to send data that is not form-encoded. If
you pass in a ``string`` instead of a ``dict``, that data will be posted directly.
@@ -288,26 +263,21 @@ For example, the GitHub API v3 accepts JSON-Encoded POST/PATCH data::
>>> r = requests.post(url, data=json.dumps(payload))
Please note that the above code will NOT add the ``Content-Type`` header
(so in particular it will NOT set it to ``application/json``).
If you need that header set and you don't want to encode the ``dict`` yourself,
you can also pass it directly using the ``json`` parameter (added in version 2.4.2)
and it will be encoded automatically:
Instead of encoding the ``dict`` yourself, you can also pass it directly using
the ``json`` parameter (added in version 2.4.2) and it will be encoded automatically::
>>> url = 'https://api.github.com/some/endpoint'
>>> payload = {'some': 'data'}
>>> r = requests.post(url, json=payload)
Note, the ``json`` parameter is ignored if either ``data`` or ``files`` is passed.
POST a Multipart-Encoded File
-----------------------------
Requests makes it simple to upload Multipart-encoded files::
>>> url = 'https://httpbin.org/post'
>>> url = 'http://httpbin.org/post'
>>> files = {'file': open('report.xls', 'rb')}
>>> r = requests.post(url, files=files)
@@ -322,7 +292,7 @@ Requests makes it simple to upload Multipart-encoded files::
You can set the filename, content_type and headers explicitly::
>>> url = 'https://httpbin.org/post'
>>> url = 'http://httpbin.org/post'
>>> files = {'file': ('report.xls', open('report.xls', 'rb'), 'application/vnd.ms-excel', {'Expires': '0'})}
>>> r = requests.post(url, files=files)
@@ -337,7 +307,7 @@ You can set the filename, content_type and headers explicitly::
If you want, you can send strings to be received as files::
>>> url = 'https://httpbin.org/post'
>>> url = 'http://httpbin.org/post'
>>> files = {'file': ('report.csv', 'some,data,to,send\nanother,row,to,send\n')}
>>> r = requests.post(url, files=files)
@@ -359,11 +329,13 @@ support this, but there is a separate package which does -
For sending multiple files in one request refer to the :ref:`advanced <advanced>`
section.
.. warning:: It is strongly recommended that you open files in :ref:`binary
mode <tut-files>`. This is because Requests may attempt to provide
the ``Content-Length`` header for you, and if it does this value
will be set to the number of *bytes* in the file. Errors may occur
if you open the file in *text mode*.
.. warning:: It is strongly recommended that you open files in `binary mode`_.
This is because Requests may attempt to provide the
``Content-Length`` header for you, and if it does this value will
be set to the number of *bytes* in the file. Errors may occur if
you open the file in *text mode*.
.. _binary mode: https://docs.python.org/2/tutorial/inputoutput.html#reading-and-writing-files
Response Status Codes
@@ -371,7 +343,7 @@ Response Status Codes
We can check the response status code::
>>> r = requests.get('https://httpbin.org/get')
>>> r = requests.get('http://httpbin.org/get')
>>> r.status_code
200
@@ -385,7 +357,7 @@ If we made a bad request (a 4XX client error or 5XX server error response), we
can raise it with
:meth:`Response.raise_for_status() <requests.Response.raise_for_status>`::
>>> bad_r = requests.get('https://httpbin.org/status/404')
>>> bad_r = requests.get('http://httpbin.org/status/404')
>>> bad_r.status_code
404
@@ -421,7 +393,7 @@ We can view the server's response headers using a Python dictionary::
}
The dictionary is special, though: it's made just for HTTP headers. According to
`RFC 7230 <https://tools.ietf.org/html/rfc7230#section-3.2>`_, HTTP Header names
`RFC 7230 <http://tools.ietf.org/html/rfc7230#section-3.2>`_, HTTP Header names
are case-insensitive.
So, we can access the headers using any capitalization we want::
@@ -435,7 +407,7 @@ So, we can access the headers using any capitalization we want::
It is also special in that the server could have sent the same header multiple
times with different values, but requests combines them so they can be
represented in the dictionary within a single mapping, as per
`RFC 7230 <https://tools.ietf.org/html/rfc7230#section-3.2>`_:
`RFC 7230 <http://tools.ietf.org/html/rfc7230#section-3.2>`_:
A recipient MAY combine multiple header fields with the same field name
into one "field-name: field-value" pair, without changing the semantics
@@ -456,7 +428,7 @@ If a response contains some Cookies, you can quickly access them::
To send your own cookies to the server, you can use the ``cookies``
parameter::
>>> url = 'https://httpbin.org/cookies'
>>> url = 'http://httpbin.org/cookies'
>>> cookies = dict(cookies_are='working')
>>> r = requests.get(url, cookies=cookies)
@@ -471,7 +443,7 @@ also be passed in to requests::
>>> jar = requests.cookies.RequestsCookieJar()
>>> jar.set('tasty_cookie', 'yum', domain='httpbin.org', path='/cookies')
>>> jar.set('gross_cookie', 'blech', domain='httpbin.org', path='/elsewhere')
>>> url = 'https://httpbin.org/cookies'
>>> url = 'http://httpbin.org/cookies'
>>> r = requests.get(url, cookies=jar)
>>> r.text
'{"cookies": {"tasty_cookie": "yum"}}'
@@ -492,7 +464,7 @@ response.
For example, GitHub redirects all HTTP requests to HTTPS::
>>> r = requests.get('http://github.com/')
>>> r = requests.get('http://github.com')
>>> r.url
'https://github.com/'
@@ -507,7 +479,7 @@ For example, GitHub redirects all HTTP requests to HTTPS::
If you're using GET, OPTIONS, POST, PUT, PATCH or DELETE, you can disable
redirection handling with the ``allow_redirects`` parameter::
>>> r = requests.get('http://github.com/', allow_redirects=False)
>>> r = requests.get('http://github.com', allow_redirects=False)
>>> r.status_code
301
@@ -517,7 +489,7 @@ redirection handling with the ``allow_redirects`` parameter::
If you're using HEAD, you can enable redirection as well::
>>> r = requests.head('http://github.com/', allow_redirects=True)
>>> r = requests.head('http://github.com', allow_redirects=True)
>>> r.url
'https://github.com/'
@@ -534,7 +506,7 @@ seconds with the ``timeout`` parameter. Nearly all production code should use
this parameter in nearly all requests. Failure to do so can cause your program
to hang indefinitely::
>>> requests.get('https://github.com/', timeout=0.001)
>>> requests.get('http://github.com', timeout=0.001)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
requests.exceptions.Timeout: HTTPConnectionPool(host='github.com', port=80): Request timed out. (timeout=0.001)

View File

@@ -1 +0,0 @@
Copyright 2019 Kenneth Reitz. All rights reserved.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 9.2 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 188 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 188 KiB

View File

@@ -1,125 +0,0 @@
[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"
[project]
name = "requests"
description = "Python HTTP for Humans."
readme = "README.md"
license = {text = "Apache-2.0"}
authors = [
{ name = "Kenneth Reitz", email = "me@kennethreitz.org" },
]
maintainers = [
{name = "Ian Stapleton Cordasco", email="graffatcolmingov@gmail.com"},
{name = "Nate Prewitt", email="nate.prewitt@gmail.com"}
]
requires-python = ">=3.10"
dependencies = [
"charset_normalizer>=2,<4",
"idna>=2.5,<4",
"urllib3>=1.26,<3",
"certifi>=2023.5.7"
]
dynamic = ["version"]
classifiers = [
"Development Status :: 5 - Production/Stable",
"Environment :: Web Environment",
"Intended Audience :: Developers",
"License :: OSI Approved :: Apache Software License",
"Natural Language :: English",
"Operating System :: OS Independent",
"Programming Language :: Python",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.14",
"Programming Language :: Python :: 3.15",
"Programming Language :: Python :: 3 :: Only",
"Programming Language :: Python :: Implementation :: CPython",
"Programming Language :: Python :: Implementation :: PyPy",
"Programming Language :: Python :: Free Threading :: 2 - Beta",
"Topic :: Internet :: WWW/HTTP",
"Topic :: Software Development :: Libraries"
]
[project.urls]
Documentation = "https://requests.readthedocs.io"
Source = "https://github.com/psf/requests"
[project.optional-dependencies]
security = []
socks = ["PySocks>=1.5.6, !=1.5.7"]
use_chardet_on_py3 = ["chardet>=3.0.2,<8"]
[dependency-groups]
test = [
"requests[socks]",
"pytest-httpbin==2.1.0",
"httpbin~=0.10.0",
"pytest-cov",
"pytest-mock",
"pytest-xdist",
"pytest>=3",
"trustme",
]
typecheck = [
"pyright",
"typing_extensions",
"chardet",
"cryptography",
"PyOpenSSL",
]
[tool.setuptools]
license-files = ["LICENSE", "NOTICE"]
[tool.setuptools.dynamic]
version = {attr = "requests.__version__.__version__"}
[tool.setuptools.packages.find]
where = ["src"]
[tool.ruff]
target-version = "py310"
src = ["src/requests", "tests"]
exclude = ["docs/", "ext/"]
[tool.ruff.lint]
select = [
"E", # pycodestyle errors
"W", # pycodestyle warnings
"F", # pyflakes
"I", # isort
"UP", # pyupgrade
"T10", # flake8-debugger (replaces debug-statements hook)
]
ignore = ["E203", "E501", "UP031"]
per-file-ignores = {"src/requests/__init__.py" = ["E402", "F401"], "src/requests/compat.py" = ["E402", "F401"], "tests/compat.py" = ["F401"]}
[tool.ruff.lint.isort]
known-first-party = ["requests"]
[tool.ruff.format]
# Use black-compatible formatting
quote-style = "double"
indent-style = "space"
[tool.pytest.ini_options]
addopts = "--doctest-modules"
doctest_optionflags = "NORMALIZE_WHITESPACE ELLIPSIS"
minversion = "6.2"
testpaths = ["tests"]
[tool.pyright]
include = ["src/requests"]
typeCheckingMode = "strict"
# linting rules unrelated to type checking
reportPrivateUsage = false
reportPrivateImportUsage = false
reportUnnecessaryIsInstance = false
reportUnusedImport = false # duplicate of Ruff's F401 rule

2
pytest.ini Normal file
View File

@@ -0,0 +1,2 @@
[pytest]
addopts = -p no:warnings

113
requests/__init__.py Normal file
View File

@@ -0,0 +1,113 @@
# -*- coding: utf-8 -*-
# __
# /__) _ _ _ _ _/ _
# / ( (- (/ (/ (- _) / _)
# /
"""
Requests HTTP Library
~~~~~~~~~~~~~~~~~~~~~
Requests is an HTTP library, written in Python, for human beings. Basic GET
usage:
>>> import requests
>>> r = requests.get('https://www.python.org')
>>> r.status_code
200
>>> 'Python is a programming language' in r.content
True
... or POST:
>>> payload = dict(key1='value1', key2='value2')
>>> r = requests.post('http://httpbin.org/post', data=payload)
>>> print(r.text)
{
...
"form": {
"key2": "value2",
"key1": "value1"
},
...
}
The other HTTP methods are supported - see `requests.api`. Full documentation
is at <http://python-requests.org>.
:copyright: (c) 2017 by Kenneth Reitz.
:license: Apache 2.0, see LICENSE for more details.
"""
from .__version__ import __title__, __description__, __url__, __version__
from .__version__ import __build__, __author__, __author_email__, __license__
from .__version__ import __copyright__, __cake__
# Check urllib3 for compatibility.
import urllib3
urllib3_version = urllib3.__version__.split('.')
# Sometimes, urllib3 only reports its version as 16.1.
if len(urllib3_version) == 2:
urllib3_version.append('0')
major, minor, patch = urllib3_version
major, minor, patch = int(major), int(minor), int(patch)
# urllib3 >= 1.21.1, < 1.22
try:
assert major == 1
assert minor >= 21
assert minor <= 22
except AssertionError:
raise RuntimeError('Requests dependency \'urllib3\' must be version >= 1.21.1, < 1.22!')
# Check chardet for compatibility.
import chardet
major, minor, patch = chardet.__version__.split('.')[:3]
major, minor, patch = int(major), int(minor), int(patch)
# chardet >= 3.0.2, < 3.1.0
try:
assert major == 3
assert minor < 1
assert patch >= 2
except AssertionError:
raise RuntimeError('Requests dependency \'chardet\' must be version >= 3.0.2, < 3.1.0!')
# Attempt to enable urllib3's SNI support, if possible
try:
from urllib3.contrib import pyopenssl
pyopenssl.inject_into_urllib3()
except ImportError:
pass
import warnings
# urllib3's DependencyWarnings should be silenced.
from urllib3.exceptions import DependencyWarning
warnings.simplefilter('ignore', DependencyWarning)
from . import utils
from . import packages
from .models import Request, Response, PreparedRequest
from .api import request, get, head, post, patch, put, delete, options
from .sessions import session, Session
from .status_codes import codes
from .exceptions import (
RequestException, Timeout, URLRequired,
TooManyRedirects, HTTPError, ConnectionError,
FileModeWarning, ConnectTimeout, ReadTimeout
)
# Set default logging handler to avoid "No handler found" warnings.
import logging
try: # Python 2.7+
from logging import NullHandler
except ImportError:
class NullHandler(logging.Handler):
def emit(self, record):
pass
logging.getLogger(__name__).addHandler(NullHandler())
# FileModeWarnings go off per the default.
warnings.simplefilter('default', FileModeWarning, append=True)

16
requests/__version__.py Normal file
View File

@@ -0,0 +1,16 @@
# .-. .-. .-. . . .-. .-. .-. .-.
# |( |- |.| | | |- `-. | `-.
# ' ' `-' `-`.`-' `-' `-' ' `-'
__title__ = 'requests'
__description__ = 'Python HTTP for Humans.'
__url__ = 'http://python-requests.org'
__version__ = '2.17.2'
__build__ = 0x021702
__author__ = 'Kenneth Reitz'
__author_email__ = 'me@kennethreitz.org'
__license__ = 'Apache 2.0'
__copyright__ = 'Copyright 2017 Kenneth Reitz'
__cake__ = u'\u2728 \U0001f370 \u2728'

View File

@@ -1,3 +1,5 @@
# -*- coding: utf-8 -*-
"""
requests._internal_utils
~~~~~~~~~~~~~~
@@ -6,24 +8,10 @@ Provides utility functions that are consumed internally by Requests
which depend on extremely few external helpers (such as compat)
"""
import re
from .compat import builtin_str
_VALID_HEADER_NAME_RE_BYTE = re.compile(rb"^[^:\s][^:\r\n]*\Z")
_VALID_HEADER_NAME_RE_STR = re.compile(r"^[^:\s][^:\r\n]*\Z")
_VALID_HEADER_VALUE_RE_BYTE = re.compile(rb"^\S[^\r\n]*\Z|^\Z")
_VALID_HEADER_VALUE_RE_STR = re.compile(r"^\S[^\r\n]*\Z|^\Z")
_HEADER_VALIDATORS_STR = (_VALID_HEADER_NAME_RE_STR, _VALID_HEADER_VALUE_RE_STR)
_HEADER_VALIDATORS_BYTE = (_VALID_HEADER_NAME_RE_BYTE, _VALID_HEADER_VALUE_RE_BYTE)
HEADER_VALIDATORS = {
bytes: _HEADER_VALIDATORS_BYTE,
str: _HEADER_VALIDATORS_STR,
}
from .compat import is_py2, builtin_str, str
def to_native_string(string: str | bytes, encoding: str = "ascii") -> str:
def to_native_string(string, encoding='ascii'):
"""Given a string object, regardless of type, returns a representation of
that string in the native string type, encoding and decoding where
necessary. This assumes ASCII unless told otherwise.
@@ -31,12 +19,15 @@ def to_native_string(string: str | bytes, encoding: str = "ascii") -> str:
if isinstance(string, builtin_str):
out = string
else:
out = string.decode(encoding)
if is_py2:
out = string.encode(encoding)
else:
out = string.decode(encoding)
return out
def unicode_is_ascii(u_string: str) -> bool:
def unicode_is_ascii(u_string):
"""Determine if unicode string only contains ASCII characters.
:param str u_string: unicode string to check. Must be unicode
@@ -45,7 +36,7 @@ def unicode_is_ascii(u_string: str) -> bool:
"""
assert isinstance(u_string, str)
try:
u_string.encode("ascii")
u_string.encode('ascii')
return True
except UnicodeEncodeError:
return False

520
requests/adapters.py Normal file
View File

@@ -0,0 +1,520 @@
# -*- coding: utf-8 -*-
"""
requests.adapters
~~~~~~~~~~~~~~~~~
This module contains the transport adapters that Requests uses to define
and maintain connections.
"""
import os.path
import socket
from urllib3.poolmanager import PoolManager, proxy_from_url
from urllib3.response import HTTPResponse
from urllib3.util import Timeout as TimeoutSauce
from urllib3.util.retry import Retry
from urllib3.exceptions import ClosedPoolError
from urllib3.exceptions import ConnectTimeoutError
from urllib3.exceptions import HTTPError as _HTTPError
from urllib3.exceptions import MaxRetryError
from urllib3.exceptions import NewConnectionError
from urllib3.exceptions import ProxyError as _ProxyError
from urllib3.exceptions import ProtocolError
from urllib3.exceptions import ReadTimeoutError
from urllib3.exceptions import SSLError as _SSLError
from urllib3.exceptions import ResponseError
from .models import Response
from .compat import urlparse, basestring
from .utils import (DEFAULT_CA_BUNDLE_PATH, get_encoding_from_headers,
prepend_scheme_if_needed, get_auth_from_url, urldefragauth,
select_proxy)
from .structures import CaseInsensitiveDict
from .cookies import extract_cookies_to_jar
from .exceptions import (ConnectionError, ConnectTimeout, ReadTimeout, SSLError,
ProxyError, RetryError, InvalidSchema)
from .auth import _basic_auth_str
try:
from urllib3.contrib.socks import SOCKSProxyManager
except ImportError:
def SOCKSProxyManager(*args, **kwargs):
raise InvalidSchema("Missing dependencies for SOCKS support.")
DEFAULT_POOLBLOCK = False
DEFAULT_POOLSIZE = 10
DEFAULT_RETRIES = 0
DEFAULT_POOL_TIMEOUT = None
class BaseAdapter(object):
"""The Base Transport Adapter"""
def __init__(self):
super(BaseAdapter, self).__init__()
def send(self, request, stream=False, timeout=None, verify=True,
cert=None, proxies=None):
"""Sends PreparedRequest object. Returns Response object.
:param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
:param stream: (optional) Whether to stream the request content.
:param timeout: (optional) How long to wait for the server to send
data before giving up, as a float, or a :ref:`(connect timeout,
read timeout) <timeouts>` tuple.
:type timeout: float or tuple
:param verify: (optional) Either a boolean, in which case it controls whether we verify
the server's TLS certificate, or a string, in which case it must be a path
to a CA bundle to use
:param cert: (optional) Any user-provided SSL certificate to be trusted.
:param proxies: (optional) The proxies dictionary to apply to the request.
"""
raise NotImplementedError
def close(self):
"""Cleans up adapter specific items."""
raise NotImplementedError
class HTTPAdapter(BaseAdapter):
"""The built-in HTTP Adapter for urllib3.
Provides a general-case interface for Requests sessions to contact HTTP and
HTTPS urls by implementing the Transport Adapter interface. This class will
usually be created by the :class:`Session <Session>` class under the
covers.
:param pool_connections: The number of urllib3 connection pools to cache.
:param pool_maxsize: The maximum number of connections to save in the pool.
:param max_retries: The maximum number of retries each connection
should attempt. Note, this applies only to failed DNS lookups, socket
connections and connection timeouts, never to requests where data has
made it to the server. By default, Requests does not retry failed
connections. If you need granular control over the conditions under
which we retry a request, import urllib3's ``Retry`` class and pass
that instead.
:param pool_block: Whether the connection pool should block for connections.
Usage::
>>> import requests
>>> s = requests.Session()
>>> a = requests.adapters.HTTPAdapter(max_retries=3)
>>> s.mount('http://', a)
"""
__attrs__ = ['max_retries', 'config', '_pool_connections', '_pool_maxsize',
'_pool_block']
def __init__(self, pool_connections=DEFAULT_POOLSIZE,
pool_maxsize=DEFAULT_POOLSIZE, max_retries=DEFAULT_RETRIES,
pool_block=DEFAULT_POOLBLOCK):
if max_retries == DEFAULT_RETRIES:
self.max_retries = Retry(0, read=False)
else:
self.max_retries = Retry.from_int(max_retries)
self.config = {}
self.proxy_manager = {}
super(HTTPAdapter, self).__init__()
self._pool_connections = pool_connections
self._pool_maxsize = pool_maxsize
self._pool_block = pool_block
self.init_poolmanager(pool_connections, pool_maxsize, block=pool_block)
def __getstate__(self):
return dict((attr, getattr(self, attr, None)) for attr in
self.__attrs__)
def __setstate__(self, state):
# Can't handle by adding 'proxy_manager' to self.__attrs__ because
# self.poolmanager uses a lambda function, which isn't pickleable.
self.proxy_manager = {}
self.config = {}
for attr, value in state.items():
setattr(self, attr, value)
self.init_poolmanager(self._pool_connections, self._pool_maxsize,
block=self._pool_block)
def init_poolmanager(self, connections, maxsize, block=DEFAULT_POOLBLOCK, **pool_kwargs):
"""Initializes a urllib3 PoolManager.
This method should not be called from user code, and is only
exposed for use when subclassing the
:class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
:param connections: The number of urllib3 connection pools to cache.
:param maxsize: The maximum number of connections to save in the pool.
:param block: Block when no free connections are available.
:param pool_kwargs: Extra keyword arguments used to initialize the Pool Manager.
"""
# save these values for pickling
self._pool_connections = connections
self._pool_maxsize = maxsize
self._pool_block = block
self.poolmanager = PoolManager(num_pools=connections, maxsize=maxsize,
block=block, strict=True, **pool_kwargs)
def proxy_manager_for(self, proxy, **proxy_kwargs):
"""Return urllib3 ProxyManager for the given proxy.
This method should not be called from user code, and is only
exposed for use when subclassing the
:class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
:param proxy: The proxy to return a urllib3 ProxyManager for.
:param proxy_kwargs: Extra keyword arguments used to configure the Proxy Manager.
:returns: ProxyManager
:rtype: urllib3.ProxyManager
"""
if proxy in self.proxy_manager:
manager = self.proxy_manager[proxy]
elif proxy.lower().startswith('socks'):
username, password = get_auth_from_url(proxy)
manager = self.proxy_manager[proxy] = SOCKSProxyManager(
proxy,
username=username,
password=password,
num_pools=self._pool_connections,
maxsize=self._pool_maxsize,
block=self._pool_block,
**proxy_kwargs
)
else:
proxy_headers = self.proxy_headers(proxy)
manager = self.proxy_manager[proxy] = proxy_from_url(
proxy,
proxy_headers=proxy_headers,
num_pools=self._pool_connections,
maxsize=self._pool_maxsize,
block=self._pool_block,
**proxy_kwargs)
return manager
def cert_verify(self, conn, url, verify, cert):
"""Verify a SSL certificate. This method should not be called from user
code, and is only exposed for use when subclassing the
:class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
:param conn: The urllib3 connection object associated with the cert.
:param url: The requested URL.
:param verify: Either a boolean, in which case it controls whether we verify
the server's TLS certificate, or a string, in which case it must be a path
to a CA bundle to use
:param cert: The SSL certificate to verify.
"""
if url.lower().startswith('https') and verify:
cert_loc = None
# Allow self-specified cert location.
if verify is not True:
cert_loc = verify
if not cert_loc:
cert_loc = DEFAULT_CA_BUNDLE_PATH
if not cert_loc or not os.path.exists(cert_loc):
raise IOError("Could not find a suitable TLS CA certificate bundle, "
"invalid path: {0}".format(cert_loc))
conn.cert_reqs = 'CERT_REQUIRED'
if not os.path.isdir(cert_loc):
conn.ca_certs = cert_loc
else:
conn.ca_cert_dir = cert_loc
else:
conn.cert_reqs = 'CERT_NONE'
conn.ca_certs = None
conn.ca_cert_dir = None
if cert:
if not isinstance(cert, basestring):
conn.cert_file = cert[0]
conn.key_file = cert[1]
else:
conn.cert_file = cert
conn.key_file = None
if conn.cert_file and not os.path.exists(conn.cert_file):
raise IOError("Could not find the TLS certificate file, "
"invalid path: {0}".format(conn.cert_file))
if conn.key_file and not os.path.exists(conn.key_file):
raise IOError("Could not find the TLS key file, "
"invalid path: {0}".format(conn.key_file))
def build_response(self, req, resp):
"""Builds a :class:`Response <requests.Response>` object from a urllib3
response. This should not be called from user code, and is only exposed
for use when subclassing the
:class:`HTTPAdapter <requests.adapters.HTTPAdapter>`
:param req: The :class:`PreparedRequest <PreparedRequest>` used to generate the response.
:param resp: The urllib3 response object.
:rtype: requests.Response
"""
response = Response()
# Fallback to None if there's no status_code, for whatever reason.
response.status_code = getattr(resp, 'status', None)
# Make headers case-insensitive.
response.headers = CaseInsensitiveDict(getattr(resp, 'headers', {}))
# Set encoding.
response.encoding = get_encoding_from_headers(response.headers)
response.raw = resp
response.reason = response.raw.reason
if isinstance(req.url, bytes):
response.url = req.url.decode('utf-8')
else:
response.url = req.url
# Add new cookies from the server.
extract_cookies_to_jar(response.cookies, req, resp)
# Give the Response some context.
response.request = req
response.connection = self
return response
def get_connection(self, url, proxies=None):
"""Returns a urllib3 connection for the given URL. This should not be
called from user code, and is only exposed for use when subclassing the
:class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
:param url: The URL to connect to.
:param proxies: (optional) A Requests-style dictionary of proxies used on this request.
:rtype: urllib3.ConnectionPool
"""
proxy = select_proxy(url, proxies)
if proxy:
proxy = prepend_scheme_if_needed(proxy, 'http')
proxy_manager = self.proxy_manager_for(proxy)
conn = proxy_manager.connection_from_url(url)
else:
# Only scheme should be lower case
parsed = urlparse(url)
url = parsed.geturl()
conn = self.poolmanager.connection_from_url(url)
return conn
def close(self):
"""Disposes of any internal state.
Currently, this closes the PoolManager and any active ProxyManager,
which closes any pooled connections.
"""
self.poolmanager.clear()
for proxy in self.proxy_manager.values():
proxy.clear()
def request_url(self, request, proxies):
"""Obtain the url to use when making the final request.
If the message is being sent through a HTTP proxy, the full URL has to
be used. Otherwise, we should only use the path portion of the URL.
This should not be called from user code, and is only exposed for use
when subclassing the
:class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
:param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
:param proxies: A dictionary of schemes or schemes and hosts to proxy URLs.
:rtype: str
"""
proxy = select_proxy(request.url, proxies)
scheme = urlparse(request.url).scheme
is_proxied_http_request = (proxy and scheme != 'https')
using_socks_proxy = False
if proxy:
proxy_scheme = urlparse(proxy).scheme.lower()
using_socks_proxy = proxy_scheme.startswith('socks')
url = request.path_url
if is_proxied_http_request and not using_socks_proxy:
url = urldefragauth(request.url)
return url
def add_headers(self, request, **kwargs):
"""Add any headers needed by the connection. As of v2.0 this does
nothing by default, but is left for overriding by users that subclass
the :class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
This should not be called from user code, and is only exposed for use
when subclassing the
:class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
:param request: The :class:`PreparedRequest <PreparedRequest>` to add headers to.
:param kwargs: The keyword arguments from the call to send().
"""
pass
def proxy_headers(self, proxy):
"""Returns a dictionary of the headers to add to any request sent
through a proxy. This works with urllib3 magic to ensure that they are
correctly sent to the proxy, rather than in a tunnelled request if
CONNECT is being used.
This should not be called from user code, and is only exposed for use
when subclassing the
:class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
:param proxies: The url of the proxy being used for this request.
:rtype: dict
"""
headers = {}
username, password = get_auth_from_url(proxy)
if username:
headers['Proxy-Authorization'] = _basic_auth_str(username,
password)
return headers
def send(self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None):
"""Sends PreparedRequest object. Returns Response object.
:param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
:param stream: (optional) Whether to stream the request content.
:param timeout: (optional) How long to wait for the server to send
data before giving up, as a float, or a :ref:`(connect timeout,
read timeout) <timeouts>` tuple.
:type timeout: float or tuple or urllib3 Timeout object
:param verify: (optional) Either a boolean, in which case it controls whether
we verify the server's TLS certificate, or a string, in which case it
must be a path to a CA bundle to use
:param cert: (optional) Any user-provided SSL certificate to be trusted.
:param proxies: (optional) The proxies dictionary to apply to the request.
:rtype: requests.Response
"""
conn = self.get_connection(request.url, proxies)
self.cert_verify(conn, request.url, verify, cert)
url = self.request_url(request, proxies)
self.add_headers(request)
chunked = not (request.body is None or 'Content-Length' in request.headers)
if isinstance(timeout, tuple):
try:
connect, read = timeout
timeout = TimeoutSauce(connect=connect, read=read)
except ValueError as e:
# this may raise a string formatting error.
err = ("Invalid timeout {0}. Pass a (connect, read) "
"timeout tuple, or a single float to set "
"both timeouts to the same value".format(timeout))
raise ValueError(err)
elif isinstance(timeout, TimeoutSauce):
pass
else:
timeout = TimeoutSauce(connect=timeout, read=timeout)
try:
if not chunked:
resp = conn.urlopen(
method=request.method,
url=url,
body=request.body,
headers=request.headers,
redirect=False,
assert_same_host=False,
preload_content=False,
decode_content=False,
retries=self.max_retries,
timeout=timeout
)
# Send the request.
else:
if hasattr(conn, 'proxy_pool'):
conn = conn.proxy_pool
low_conn = conn._get_conn(timeout=DEFAULT_POOL_TIMEOUT)
try:
low_conn.putrequest(request.method,
url,
skip_accept_encoding=True)
for header, value in request.headers.items():
low_conn.putheader(header, value)
low_conn.endheaders()
for i in request.body:
low_conn.send(hex(len(i))[2:].encode('utf-8'))
low_conn.send(b'\r\n')
low_conn.send(i)
low_conn.send(b'\r\n')
low_conn.send(b'0\r\n\r\n')
# Receive the response from the server
try:
# For Python 2.7+ versions, use buffering of HTTP
# responses
r = low_conn.getresponse(buffering=True)
except TypeError:
# For compatibility with Python 2.6 versions and back
r = low_conn.getresponse()
resp = HTTPResponse.from_httplib(
r,
pool=conn,
connection=low_conn,
preload_content=False,
decode_content=False
)
except:
# If we hit any problems here, clean up the connection.
# Then, reraise so that we can handle the actual exception.
low_conn.close()
raise
except (ProtocolError, socket.error) as err:
raise ConnectionError(err, request=request)
except MaxRetryError as e:
if isinstance(e.reason, ConnectTimeoutError):
# TODO: Remove this in 3.0.0: see #2811
if not isinstance(e.reason, NewConnectionError):
raise ConnectTimeout(e, request=request)
if isinstance(e.reason, ResponseError):
raise RetryError(e, request=request)
if isinstance(e.reason, _ProxyError):
raise ProxyError(e, request=request)
raise ConnectionError(e, request=request)
except ClosedPoolError as e:
raise ConnectionError(e, request=request)
except _ProxyError as e:
raise ProxyError(e)
except (_SSLError, _HTTPError) as e:
if isinstance(e, _SSLError):
raise SSLError(e, request=request)
elif isinstance(e, ReadTimeoutError):
raise ReadTimeout(e, request=request)
else:
raise
return self.build_response(request, resp)

View File

@@ -1,3 +1,5 @@
# -*- coding: utf-8 -*-
"""
requests.api
~~~~~~~~~~~~
@@ -8,36 +10,22 @@ This module implements the Requests API.
:license: Apache2, see LICENSE for more details.
"""
from __future__ import annotations
from typing import TYPE_CHECKING
from . import sessions
from .models import Response
if TYPE_CHECKING:
from typing_extensions import Unpack
from . import _types as _t
def request(
method: str, url: _t.UriType, **kwargs: Unpack[_t.RequestKwargs]
) -> Response:
def request(method, url, **kwargs):
"""Constructs and sends a :class:`Request <Request>`.
:param method: method for the new :class:`Request` object: ``GET``, ``OPTIONS``, ``HEAD``, ``POST``, ``PUT``, ``PATCH``, or ``DELETE``.
:param method: method for the new :class:`Request` object.
:param url: URL for the new :class:`Request` object.
:param params: (optional) Dictionary, list of tuples or bytes to send
in the query string for the :class:`Request`.
:param data: (optional) Dictionary, list of tuples, bytes, or file-like
object to send in the body of the :class:`Request`.
:param json: (optional) A JSON serializable Python object to send in the body of the :class:`Request`.
:param params: (optional) Dictionary or bytes to be sent in the query string for the :class:`Request`.
:param data: (optional) Dictionary or list of tuples ``[(key, value)]`` (will be form-encoded), bytes, or file-like object to send in the body of the :class:`Request`.
:param json: (optional) json data to send in the body of the :class:`Request`.
:param headers: (optional) Dictionary of HTTP Headers to send with the :class:`Request`.
:param cookies: (optional) Dict or CookieJar object to send with the :class:`Request`.
:param files: (optional) Dictionary of ``'name': file-like-objects`` (or ``{'name': file-tuple}``) for multipart encoding upload.
``file-tuple`` can be a 2-tuple ``('filename', fileobj)``, 3-tuple ``('filename', fileobj, 'content_type')``
or a 4-tuple ``('filename', fileobj, 'content_type', custom_headers)``, where ``'content_type'`` is a string
or a 4-tuple ``('filename', fileobj, 'content_type', custom_headers)``, where ``'content-type'`` is a string
defining the content type of the given file and ``custom_headers`` a dict-like object containing additional headers
to add for the file.
:param auth: (optional) Auth tuple to enable Basic/Digest/Custom HTTP Auth.
@@ -59,8 +47,7 @@ def request(
Usage::
>>> import requests
>>> req = requests.request('GET', 'https://httpbin.org/get')
>>> req
>>> req = requests.request('GET', 'http://httpbin.org/get')
<Response [200]>
"""
@@ -71,24 +58,22 @@ def request(
return session.request(method=method, url=url, **kwargs)
def get(
url: _t.UriType, params: _t.ParamsType = None, **kwargs: Unpack[_t.GetKwargs]
) -> Response:
def get(url, params=None, **kwargs):
r"""Sends a GET request.
:param url: URL for the new :class:`Request` object.
:param params: (optional) Dictionary, list of tuples or bytes to send
in the query string for the :class:`Request`.
:param params: (optional) Dictionary or bytes to be sent in the query string for the :class:`Request`.
:param \*\*kwargs: Optional arguments that ``request`` takes.
:return: :class:`Response <Response>` object
:rtype: requests.Response
"""
return request("get", url, params=params, **kwargs)
kwargs.setdefault('allow_redirects', True)
return request('get', url, params=params, **kwargs)
def options(url: _t.UriType, **kwargs: Unpack[_t.RequestKwargs]) -> Response:
r"""Sends an OPTIONS request.
def options(url, **kwargs):
r"""Sends a OPTIONS request.
:param url: URL for the new :class:`Request` object.
:param \*\*kwargs: Optional arguments that ``request`` takes.
@@ -96,79 +81,66 @@ def options(url: _t.UriType, **kwargs: Unpack[_t.RequestKwargs]) -> Response:
:rtype: requests.Response
"""
return request("options", url, **kwargs)
kwargs.setdefault('allow_redirects', True)
return request('options', url, **kwargs)
def head(url: _t.UriType, **kwargs: Unpack[_t.RequestKwargs]) -> Response:
def head(url, **kwargs):
r"""Sends a HEAD request.
:param url: URL for the new :class:`Request` object.
:param \*\*kwargs: Optional arguments that ``request`` takes. If
`allow_redirects` is not provided, it will be set to `False` (as
opposed to the default :meth:`request` behavior).
:param \*\*kwargs: Optional arguments that ``request`` takes.
:return: :class:`Response <Response>` object
:rtype: requests.Response
"""
kwargs.setdefault("allow_redirects", False)
return request("head", url, **kwargs)
kwargs.setdefault('allow_redirects', False)
return request('head', url, **kwargs)
def post(
url: _t.UriType,
data: _t.DataType = None,
json: _t.JsonType = None,
**kwargs: Unpack[_t.PostKwargs],
) -> Response:
def post(url, data=None, json=None, **kwargs):
r"""Sends a POST request.
:param url: URL for the new :class:`Request` object.
:param data: (optional) Dictionary, list of tuples, bytes, or file-like
object to send in the body of the :class:`Request`.
:param json: (optional) A JSON serializable Python object to send in the body of the :class:`Request`.
:param data: (optional) Dictionary (will be form-encoded), bytes, or file-like object to send in the body of the :class:`Request`.
:param json: (optional) json data to send in the body of the :class:`Request`.
:param \*\*kwargs: Optional arguments that ``request`` takes.
:return: :class:`Response <Response>` object
:rtype: requests.Response
"""
return request("post", url, data=data, json=json, **kwargs)
return request('post', url, data=data, json=json, **kwargs)
def put(
url: _t.UriType, data: _t.DataType = None, **kwargs: Unpack[_t.DataKwargs]
) -> Response:
def put(url, data=None, **kwargs):
r"""Sends a PUT request.
:param url: URL for the new :class:`Request` object.
:param data: (optional) Dictionary, list of tuples, bytes, or file-like
object to send in the body of the :class:`Request`.
:param json: (optional) A JSON serializable Python object to send in the body of the :class:`Request`.
:param data: (optional) Dictionary (will be form-encoded), bytes, or file-like object to send in the body of the :class:`Request`.
:param json: (optional) json data to send in the body of the :class:`Request`.
:param \*\*kwargs: Optional arguments that ``request`` takes.
:return: :class:`Response <Response>` object
:rtype: requests.Response
"""
return request("put", url, data=data, **kwargs)
return request('put', url, data=data, **kwargs)
def patch(
url: _t.UriType, data: _t.DataType = None, **kwargs: Unpack[_t.DataKwargs]
) -> Response:
def patch(url, data=None, **kwargs):
r"""Sends a PATCH request.
:param url: URL for the new :class:`Request` object.
:param data: (optional) Dictionary, list of tuples, bytes, or file-like
object to send in the body of the :class:`Request`.
:param json: (optional) A JSON serializable Python object to send in the body of the :class:`Request`.
:param data: (optional) Dictionary (will be form-encoded), bytes, or file-like object to send in the body of the :class:`Request`.
:param json: (optional) json data to send in the body of the :class:`Request`.
:param \*\*kwargs: Optional arguments that ``request`` takes.
:return: :class:`Response <Response>` object
:rtype: requests.Response
"""
return request("patch", url, data=data, **kwargs)
return request('patch', url, data=data, **kwargs)
def delete(url: _t.UriType, **kwargs: Unpack[_t.RequestKwargs]) -> Response:
def delete(url, **kwargs):
r"""Sends a DELETE request.
:param url: URL for the new :class:`Request` object.
@@ -177,4 +149,4 @@ def delete(url: _t.UriType, **kwargs: Unpack[_t.RequestKwargs]) -> Response:
:rtype: requests.Response
"""
return request("delete", url, **kwargs)
return request('delete', url, **kwargs)

293
requests/auth.py Normal file
View File

@@ -0,0 +1,293 @@
# -*- coding: utf-8 -*-
"""
requests.auth
~~~~~~~~~~~~~
This module contains the authentication handlers for Requests.
"""
import os
import re
import time
import hashlib
import threading
import warnings
from base64 import b64encode
from .compat import urlparse, str, basestring
from .cookies import extract_cookies_to_jar
from ._internal_utils import to_native_string
from .utils import parse_dict_header
CONTENT_TYPE_FORM_URLENCODED = 'application/x-www-form-urlencoded'
CONTENT_TYPE_MULTI_PART = 'multipart/form-data'
def _basic_auth_str(username, password):
"""Returns a Basic Auth string."""
# "I want us to put a big-ol' comment on top of it that
# says that this behaviour is dumb but we need to preserve
# it because people are relying on it."
# - Lukasa
#
# These are here solely to maintain backwards compatibility
# for things like ints. This will be removed in 3.0.0.
if not isinstance(username, basestring):
warnings.warn(
"Non-string usernames will no longer be supported in Requests "
"3.0.0. Please convert the object you've passed in ({0!r}) to "
"a string or bytes object in the near future to avoid "
"problems.".format(username),
category=DeprecationWarning,
)
username = str(username)
if not isinstance(password, basestring):
warnings.warn(
"Non-string passwords will no longer be supported in Requests "
"3.0.0. Please convert the object you've passed in ({0!r}) to "
"a string or bytes object in the near future to avoid "
"problems.".format(password),
category=DeprecationWarning,
)
password = str(password)
# -- End Removal --
if isinstance(username, str):
username = username.encode('latin1')
if isinstance(password, str):
password = password.encode('latin1')
authstr = 'Basic ' + to_native_string(
b64encode(b':'.join((username, password))).strip()
)
return authstr
class AuthBase(object):
"""Base class that all auth implementations derive from"""
def __call__(self, r):
raise NotImplementedError('Auth hooks must be callable.')
class HTTPBasicAuth(AuthBase):
"""Attaches HTTP Basic Authentication to the given Request object."""
def __init__(self, username, password):
self.username = username
self.password = password
def __eq__(self, other):
return all([
self.username == getattr(other, 'username', None),
self.password == getattr(other, 'password', None)
])
def __ne__(self, other):
return not self == other
def __call__(self, r):
r.headers['Authorization'] = _basic_auth_str(self.username, self.password)
return r
class HTTPProxyAuth(HTTPBasicAuth):
"""Attaches HTTP Proxy Authentication to a given Request object."""
def __call__(self, r):
r.headers['Proxy-Authorization'] = _basic_auth_str(self.username, self.password)
return r
class HTTPDigestAuth(AuthBase):
"""Attaches HTTP Digest Authentication to the given Request object."""
def __init__(self, username, password):
self.username = username
self.password = password
# Keep state in per-thread local storage
self._thread_local = threading.local()
def init_per_thread_state(self):
# Ensure state is initialized just once per-thread
if not hasattr(self._thread_local, 'init'):
self._thread_local.init = True
self._thread_local.last_nonce = ''
self._thread_local.nonce_count = 0
self._thread_local.chal = {}
self._thread_local.pos = None
self._thread_local.num_401_calls = None
def build_digest_header(self, method, url):
"""
:rtype: str
"""
realm = self._thread_local.chal['realm']
nonce = self._thread_local.chal['nonce']
qop = self._thread_local.chal.get('qop')
algorithm = self._thread_local.chal.get('algorithm')
opaque = self._thread_local.chal.get('opaque')
hash_utf8 = None
if algorithm is None:
_algorithm = 'MD5'
else:
_algorithm = algorithm.upper()
# lambdas assume digest modules are imported at the top level
if _algorithm == 'MD5' or _algorithm == 'MD5-SESS':
def md5_utf8(x):
if isinstance(x, str):
x = x.encode('utf-8')
return hashlib.md5(x).hexdigest()
hash_utf8 = md5_utf8
elif _algorithm == 'SHA':
def sha_utf8(x):
if isinstance(x, str):
x = x.encode('utf-8')
return hashlib.sha1(x).hexdigest()
hash_utf8 = sha_utf8
KD = lambda s, d: hash_utf8("%s:%s" % (s, d))
if hash_utf8 is None:
return None
# XXX not implemented yet
entdig = None
p_parsed = urlparse(url)
#: path is request-uri defined in RFC 2616 which should not be empty
path = p_parsed.path or "/"
if p_parsed.query:
path += '?' + p_parsed.query
A1 = '%s:%s:%s' % (self.username, realm, self.password)
A2 = '%s:%s' % (method, path)
HA1 = hash_utf8(A1)
HA2 = hash_utf8(A2)
if nonce == self._thread_local.last_nonce:
self._thread_local.nonce_count += 1
else:
self._thread_local.nonce_count = 1
ncvalue = '%08x' % self._thread_local.nonce_count
s = str(self._thread_local.nonce_count).encode('utf-8')
s += nonce.encode('utf-8')
s += time.ctime().encode('utf-8')
s += os.urandom(8)
cnonce = (hashlib.sha1(s).hexdigest()[:16])
if _algorithm == 'MD5-SESS':
HA1 = hash_utf8('%s:%s:%s' % (HA1, nonce, cnonce))
if not qop:
respdig = KD(HA1, "%s:%s" % (nonce, HA2))
elif qop == 'auth' or 'auth' in qop.split(','):
noncebit = "%s:%s:%s:%s:%s" % (
nonce, ncvalue, cnonce, 'auth', HA2
)
respdig = KD(HA1, noncebit)
else:
# XXX handle auth-int.
return None
self._thread_local.last_nonce = nonce
# XXX should the partial digests be encoded too?
base = 'username="%s", realm="%s", nonce="%s", uri="%s", ' \
'response="%s"' % (self.username, realm, nonce, path, respdig)
if opaque:
base += ', opaque="%s"' % opaque
if algorithm:
base += ', algorithm="%s"' % algorithm
if entdig:
base += ', digest="%s"' % entdig
if qop:
base += ', qop="auth", nc=%s, cnonce="%s"' % (ncvalue, cnonce)
return 'Digest %s' % (base)
def handle_redirect(self, r, **kwargs):
"""Reset num_401_calls counter on redirects."""
if r.is_redirect:
self._thread_local.num_401_calls = 1
def handle_401(self, r, **kwargs):
"""
Takes the given response and tries digest-auth, if needed.
:rtype: requests.Response
"""
# If response is not 4xx, do not auth
# See https://github.com/requests/requests/issues/3772
if not 400 <= r.status_code < 500:
self._thread_local.num_401_calls = 1
return r
if self._thread_local.pos is not None:
# Rewind the file position indicator of the body to where
# it was to resend the request.
r.request.body.seek(self._thread_local.pos)
s_auth = r.headers.get('www-authenticate', '')
if 'digest' in s_auth.lower() and self._thread_local.num_401_calls < 2:
self._thread_local.num_401_calls += 1
pat = re.compile(r'digest ', flags=re.IGNORECASE)
self._thread_local.chal = parse_dict_header(pat.sub('', s_auth, count=1))
# Consume content and release the original connection
# to allow our new request to reuse the same one.
r.content
r.close()
prep = r.request.copy()
extract_cookies_to_jar(prep._cookies, r.request, r.raw)
prep.prepare_cookies(prep._cookies)
prep.headers['Authorization'] = self.build_digest_header(
prep.method, prep.url)
_r = r.connection.send(prep, **kwargs)
_r.history.append(r)
_r.request = prep
return _r
self._thread_local.num_401_calls = 1
return r
def __call__(self, r):
# Initialize per-thread state, if needed
self.init_per_thread_state()
# If we have a saved nonce, skip the 401
if self._thread_local.last_nonce:
r.headers['Authorization'] = self.build_digest_header(r.method, r.url)
try:
self._thread_local.pos = r.body.tell()
except AttributeError:
# In the case of HTTPDigestAuth being reused and the body of
# the previous request was a file-like object, pos has the
# file position of the previous body. Ensure it's set to
# None.
self._thread_local.pos = None
r.register_hook('response', self.handle_401)
r.register_hook('response', self.handle_redirect)
self._thread_local.num_401_calls = 1
return r
def __eq__(self, other):
return all([
self.username == getattr(other, 'username', None),
self.password == getattr(other, 'password', None)
])
def __ne__(self, other):
return not self == other

View File

@@ -1,4 +1,5 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
requests.certs
@@ -11,8 +12,7 @@ If you are packaging Requests, e.g., for a Linux distribution or a managed
environment, you can change the definition of where() to return a separately
packaged CA bundle.
"""
from certifi import where
if __name__ == "__main__":
if __name__ == '__main__':
print(where())

71
requests/compat.py Normal file
View File

@@ -0,0 +1,71 @@
# -*- coding: utf-8 -*-
"""
requests.compat
~~~~~~~~~~~~~~~
This module handles import compatibility issues between Python 2 and
Python 3.
"""
import chardet
import sys
# -------
# Pythons
# -------
# Syntax sugar.
_ver = sys.version_info
#: Python 2.x?
is_py2 = (_ver[0] == 2)
#: Python 3.x?
is_py3 = (_ver[0] == 3)
try:
import simplejson as json
except (ImportError, SyntaxError):
# simplejson does not support Python 3.2, it throws a SyntaxError
# because of u'...' Unicode literals.
import json
# ---------
# Specifics
# ---------
if is_py2:
from urllib import (
quote, unquote, quote_plus, unquote_plus, urlencode, getproxies,
proxy_bypass, proxy_bypass_environment, getproxies_environment)
from urlparse import urlparse, urlunparse, urljoin, urlsplit, urldefrag
from urllib2 import parse_http_list
import cookielib
from Cookie import Morsel
from StringIO import StringIO
from urllib3.packages.ordered_dict import OrderedDict
builtin_str = str
bytes = str
str = unicode
basestring = basestring
numeric_types = (int, long, float)
integer_types = (int, long)
elif is_py3:
from urllib.parse import urlparse, urlunparse, urljoin, urlsplit, urlencode, quote, unquote, quote_plus, unquote_plus, urldefrag
from urllib.request import parse_http_list, getproxies, proxy_bypass, proxy_bypass_environment, getproxies_environment
from http import cookiejar as cookielib
from http.cookies import Morsel
from io import StringIO
from collections import OrderedDict
builtin_str = str
str = str
bytes = bytes
basestring = (str, bytes)
numeric_types = (int, float)
integer_types = (int,)

View File

@@ -1,37 +1,32 @@
# -*- coding: utf-8 -*-
"""
requests.cookies
~~~~~~~~~~~~~~~~
Compatibility code to be able to use `http.cookiejar.CookieJar` with requests.
Compatibility code to be able to use `cookielib.CookieJar` with requests.
requests.utils imports from here, so be careful with imports.
"""
from __future__ import annotations
import calendar
import copy
import time
from collections.abc import Iterator, MutableMapping
from http.cookiejar import Cookie, CookieJar, CookiePolicy
from typing import TYPE_CHECKING, Any, TypeVar, overload
import calendar
import collections
from ._internal_utils import to_native_string
from ._types import is_prepared as _is_prepared
from .compat import Morsel, cookielib, urlparse, urlunparse
from .compat import cookielib, urlparse, urlunparse, Morsel
if TYPE_CHECKING:
from _typeshed import SupportsKeysAndGetItem
from .models import PreparedRequest
import threading
try:
import threading
except ImportError:
import dummy_threading as threading
class MockRequest:
"""Wraps a `requests.PreparedRequest` to mimic a `urllib2.Request`.
class MockRequest(object):
"""Wraps a `requests.Request` to mimic a `urllib2.Request`.
The code in `http.cookiejar.CookieJar` expects this interface in order to correctly
The code in `cookielib.CookieJar` expects this interface in order to correctly
manage cookie policies, i.e., determine whether a cookie can be set, given the
domains of the request and the cookie.
@@ -40,135 +35,121 @@ class MockRequest:
probably want `get_cookie_header`, defined below.
"""
type: str
def __init__(self, request: PreparedRequest) -> None:
assert _is_prepared(request)
def __init__(self, request):
self._r = request
self._new_headers: dict[str, str] = {}
self._new_headers = {}
self.type = urlparse(self._r.url).scheme
def get_type(self) -> str:
def get_type(self):
return self.type
def get_host(self) -> str:
def get_host(self):
return urlparse(self._r.url).netloc
def get_origin_req_host(self) -> str:
def get_origin_req_host(self):
return self.get_host()
def get_full_url(self) -> str:
def get_full_url(self):
# Only return the response's URL if the user hadn't set the Host
# header
if not self._r.headers.get("Host"):
if not self._r.headers.get('Host'):
return self._r.url
# If they did set it, retrieve it and reconstruct the expected domain
host = to_native_string(self._r.headers["Host"], encoding="utf-8")
host = to_native_string(self._r.headers['Host'], encoding='utf-8')
parsed = urlparse(self._r.url)
# Reconstruct the URL as we expect it
return urlunparse(
[
parsed.scheme,
host,
parsed.path,
parsed.params,
parsed.query,
parsed.fragment,
]
)
return urlunparse([
parsed.scheme, host, parsed.path, parsed.params, parsed.query,
parsed.fragment
])
def is_unverifiable(self) -> bool:
def is_unverifiable(self):
return True
def has_header(self, name: str) -> bool:
def has_header(self, name):
return name in self._r.headers or name in self._new_headers
def get_header(self, name: str, default: str | None = None) -> str | None:
return self._r.headers.get(name, self._new_headers.get(name, default)) # type: ignore[return-value]
def get_header(self, name, default=None):
return self._r.headers.get(name, self._new_headers.get(name, default))
def add_header(self, key: str, val: str) -> None:
"""cookiejar has no legitimate use for this method; add it back if you find one."""
raise NotImplementedError(
"Cookie headers should be added with add_unredirected_header()"
)
def add_header(self, key, val):
"""cookielib has no legitimate use for this method; add it back if you find one."""
raise NotImplementedError("Cookie headers should be added with add_unredirected_header()")
def add_unredirected_header(self, name: str, value: str) -> None:
def add_unredirected_header(self, name, value):
self._new_headers[name] = value
def get_new_headers(self) -> dict[str, str]:
def get_new_headers(self):
return self._new_headers
@property
def unverifiable(self) -> bool:
def unverifiable(self):
return self.is_unverifiable()
@property
def origin_req_host(self) -> str:
def origin_req_host(self):
return self.get_origin_req_host()
@property
def host(self) -> str:
def host(self):
return self.get_host()
class MockResponse:
class MockResponse(object):
"""Wraps a `httplib.HTTPMessage` to mimic a `urllib.addinfourl`.
...what? Basically, expose the parsed HTTP headers from the server response
the way `http.cookiejar` expects to see them.
the way `cookielib` expects to see them.
"""
def __init__(self, headers: Any) -> None:
"""Make a MockResponse for `cookiejar` to read.
def __init__(self, headers):
"""Make a MockResponse for `cookielib` to read.
:param headers: a httplib.HTTPMessage or analogous carrying the headers
"""
self._headers = headers
def info(self) -> Any:
def info(self):
return self._headers
def getheaders(self, name: str) -> Any:
def getheaders(self, name):
self._headers.getheaders(name)
def extract_cookies_to_jar(
jar: CookieJar, request: PreparedRequest, response: Any
) -> None:
def extract_cookies_to_jar(jar, request, response):
"""Extract the cookies from the response into a CookieJar.
:param jar: http.cookiejar.CookieJar (not necessarily a RequestsCookieJar)
:param jar: cookielib.CookieJar (not necessarily a RequestsCookieJar)
:param request: our own requests.Request object
:param response: urllib3.HTTPResponse object
"""
if not (hasattr(response, "_original_response") and response._original_response):
if not (hasattr(response, '_original_response') and
response._original_response):
return
# the _original_response field is the wrapped httplib.HTTPResponse object,
req = MockRequest(request)
# pull out the HTTPMessage with the headers and put it in the mock:
res = MockResponse(response._original_response.msg)
jar.extract_cookies(res, req) # type: ignore[arg-type]
jar.extract_cookies(res, req)
def get_cookie_header(jar: CookieJar, request: PreparedRequest) -> str | None:
def get_cookie_header(jar, request):
"""
Produce an appropriate Cookie header string to be sent with `request`, or None.
:rtype: str
"""
r = MockRequest(request)
jar.add_cookie_header(r) # type: ignore[arg-type]
return r.get_new_headers().get("Cookie")
jar.add_cookie_header(r)
return r.get_new_headers().get('Cookie')
def remove_cookie_by_name(
cookiejar: CookieJar, name: str, domain: str | None = None, path: str | None = None
) -> None:
def remove_cookie_by_name(cookiejar, name, domain=None, path=None):
"""Unsets a cookie by name, by default over all domains and paths.
Wraps CookieJar.clear(), is O(n).
"""
clearables: list[tuple[str, str, str]] = []
clearables = []
for cookie in cookiejar:
if cookie.name != name:
continue
@@ -188,8 +169,8 @@ class CookieConflictError(RuntimeError):
"""
class RequestsCookieJar(CookieJar, MutableMapping[str, str | None]): # type: ignore[misc]
"""Compatibility class; is a http.cookiejar.CookieJar, but exposes a dict
class RequestsCookieJar(cookielib.CookieJar, collections.MutableMapping):
"""Compatibility class; is a cookielib.CookieJar, but exposes a dict
interface.
This is the CookieJar we create by default for requests and sessions that
@@ -206,15 +187,7 @@ class RequestsCookieJar(CookieJar, MutableMapping[str, str | None]): # type: ig
.. warning:: dictionary operations that are normally O(1) may be O(n).
"""
_policy: CookiePolicy
def get( # type: ignore[override]
self,
name: str,
default: str | None = None,
domain: str | None = None,
path: str | None = None,
) -> str | None:
def get(self, name, default=None, domain=None, path=None):
"""Dict-like get() that also supports optional domain and path args in
order to resolve naming collisions from using one cookie jar over
multiple domains.
@@ -226,18 +199,14 @@ class RequestsCookieJar(CookieJar, MutableMapping[str, str | None]): # type: ig
except KeyError:
return default
def set(
self, name: str, value: str | Morsel[dict[str, str]] | None, **kwargs: Any
) -> Cookie | None:
def set(self, name, value, **kwargs):
"""Dict-like set() that also supports optional domain and path args in
order to resolve naming collisions from using one cookie jar over
multiple domains.
"""
# support client code that unsets cookies by assignment of a None value:
if value is None:
remove_cookie_by_name(
self, name, domain=kwargs.get("domain"), path=kwargs.get("path")
)
remove_cookie_by_name(self, name, domain=kwargs.get('domain'), path=kwargs.get('path'))
return
if isinstance(value, Morsel):
@@ -247,7 +216,7 @@ class RequestsCookieJar(CookieJar, MutableMapping[str, str | None]): # type: ig
self.set_cookie(c)
return c
def iterkeys(self) -> Iterator[str]:
def iterkeys(self):
"""Dict-like iterkeys() that returns an iterator of names of cookies
from the jar.
@@ -256,7 +225,7 @@ class RequestsCookieJar(CookieJar, MutableMapping[str, str | None]): # type: ig
for cookie in iter(self):
yield cookie.name
def keys(self) -> list[str]: # type: ignore[override]
def keys(self):
"""Dict-like keys() that returns a list of names of cookies from the
jar.
@@ -264,7 +233,7 @@ class RequestsCookieJar(CookieJar, MutableMapping[str, str | None]): # type: ig
"""
return list(self.iterkeys())
def itervalues(self) -> Iterator[str | None]:
def itervalues(self):
"""Dict-like itervalues() that returns an iterator of values of cookies
from the jar.
@@ -273,7 +242,7 @@ class RequestsCookieJar(CookieJar, MutableMapping[str, str | None]): # type: ig
for cookie in iter(self):
yield cookie.value
def values(self) -> list[str | None]: # type: ignore[override]
def values(self):
"""Dict-like values() that returns a list of values of cookies from the
jar.
@@ -281,7 +250,7 @@ class RequestsCookieJar(CookieJar, MutableMapping[str, str | None]): # type: ig
"""
return list(self.itervalues())
def iteritems(self) -> Iterator[tuple[str, str | None]]:
def iteritems(self):
"""Dict-like iteritems() that returns an iterator of name-value tuples
from the jar.
@@ -290,7 +259,7 @@ class RequestsCookieJar(CookieJar, MutableMapping[str, str | None]): # type: ig
for cookie in iter(self):
yield cookie.name, cookie.value
def items(self) -> list[tuple[str, str | None]]: # type: ignore[override]
def items(self):
"""Dict-like items() that returns a list of name-value tuples from the
jar. Allows client-code to call ``dict(RequestsCookieJar)`` and get a
vanilla python dict of key value pairs.
@@ -299,63 +268,56 @@ class RequestsCookieJar(CookieJar, MutableMapping[str, str | None]): # type: ig
"""
return list(self.iteritems())
def list_domains(self) -> list[str]:
def list_domains(self):
"""Utility method to list all the domains in the jar."""
domains: list[str] = []
domains = []
for cookie in iter(self):
if cookie.domain not in domains:
domains.append(cookie.domain)
return domains
def list_paths(self) -> list[str]:
def list_paths(self):
"""Utility method to list all the paths in the jar."""
paths: list[str] = []
paths = []
for cookie in iter(self):
if cookie.path not in paths:
paths.append(cookie.path)
return paths
def multiple_domains(self) -> bool:
def multiple_domains(self):
"""Returns True if there are multiple domains in the jar.
Returns False otherwise.
:rtype: bool
"""
domains: list[str] = []
domains = []
for cookie in iter(self):
if cookie.domain is not None and cookie.domain in domains: # type: ignore[reportUnnecessaryComparison] # defensive check
if cookie.domain is not None and cookie.domain in domains:
return True
domains.append(cookie.domain)
return False # there is only one domain in jar
def get_dict(
self, domain: str | None = None, path: str | None = None
) -> dict[str, str | None]:
def get_dict(self, domain=None, path=None):
"""Takes as an argument an optional domain and path and returns a plain
old Python dict of name-value pairs of cookies that meet the
requirements.
:rtype: dict
"""
dictionary: dict[str, str | None] = {}
dictionary = {}
for cookie in iter(self):
if (domain is None or cookie.domain == domain) and (
path is None or cookie.path == path
):
if (domain is None or cookie.domain == domain) and (path is None
or cookie.path == path):
dictionary[cookie.name] = cookie.value
return dictionary
def __iter__(self) -> Iterator[Cookie]: # type: ignore[override]
"""RequestCookieJar's __iter__ comes from CookieJar not MutableMapping."""
return super().__iter__()
def __contains__(self, name: object) -> bool:
def __contains__(self, name):
try:
return super().__contains__(name)
return super(RequestsCookieJar, self).__contains__(name)
except CookieConflictError:
return True
def __getitem__(self, name: str) -> str | None:
def __getitem__(self, name):
"""Dict-like __getitem__() for compatibility with client code. Throws
exception if there are more than one cookie with name. In that case,
use the more explicit get() method instead.
@@ -364,43 +326,33 @@ class RequestsCookieJar(CookieJar, MutableMapping[str, str | None]): # type: ig
"""
return self._find_no_duplicates(name)
def __setitem__(
self, name: str, value: str | Morsel[dict[str, str]] | None
) -> None:
def __setitem__(self, name, value):
"""Dict-like __setitem__ for compatibility with client code. Throws
exception if there is already a cookie of that name in the jar. In that
case, use the more explicit set() method instead.
"""
self.set(name, value)
def __delitem__(self, name: str) -> None:
"""Deletes a cookie given a name. Wraps ``http.cookiejar.CookieJar``'s
def __delitem__(self, name):
"""Deletes a cookie given a name. Wraps ``cookielib.CookieJar``'s
``remove_cookie_by_name()``.
"""
remove_cookie_by_name(self, name)
def set_cookie(self, cookie: Cookie, *args: Any, **kwargs: Any) -> None:
if (
(value := cookie.value) is not None
and value.startswith('"')
and value.endswith('"')
):
cookie.value = value.replace('\\"', "")
return super().set_cookie(cookie, *args, **kwargs)
def set_cookie(self, cookie, *args, **kwargs):
if hasattr(cookie.value, 'startswith') and cookie.value.startswith('"') and cookie.value.endswith('"'):
cookie.value = cookie.value.replace('\\"', '')
return super(RequestsCookieJar, self).set_cookie(cookie, *args, **kwargs)
def update( # type: ignore[override]
self, other: CookieJar | SupportsKeysAndGetItem[str, str]
) -> None:
def update(self, other):
"""Updates this jar with cookies from another CookieJar or dict-like"""
if isinstance(other, cookielib.CookieJar):
for cookie in other:
self.set_cookie(copy.copy(cookie))
else:
super().update(other)
super(RequestsCookieJar, self).update(other)
def _find(
self, name: str, domain: str | None = None, path: str | None = None
) -> str | None:
def _find(self, name, domain=None, path=None):
"""Requests uses this method internally to get cookie values.
If there are conflicting cookies, _find arbitrarily chooses one.
@@ -418,11 +370,9 @@ class RequestsCookieJar(CookieJar, MutableMapping[str, str | None]): # type: ig
if path is None or cookie.path == path:
return cookie.value
raise KeyError(f"name={name!r}, domain={domain!r}, path={path!r}")
raise KeyError('name=%r, domain=%r, path=%r' % (name, domain, path))
def _find_no_duplicates(
self, name: str, domain: str | None = None, path: str | None = None
) -> str:
def _find_no_duplicates(self, name, domain=None, path=None):
"""Both ``__get_item__`` and ``get`` call this function: it's never
used elsewhere in Requests.
@@ -439,50 +389,41 @@ class RequestsCookieJar(CookieJar, MutableMapping[str, str | None]): # type: ig
if cookie.name == name:
if domain is None or cookie.domain == domain:
if path is None or cookie.path == path:
if toReturn is not None:
# if there are multiple cookies that meet passed in criteria
raise CookieConflictError(
f"There are multiple cookies with name, {name!r}"
)
# we will eventually return this as long as no cookie conflict
toReturn = cookie.value
if toReturn is not None: # if there are multiple cookies that meet passed in criteria
raise CookieConflictError('There are multiple cookies with name, %r' % (name))
toReturn = cookie.value # we will eventually return this as long as no cookie conflict
if toReturn is not None:
if toReturn:
return toReturn
raise KeyError(f"name={name!r}, domain={domain!r}, path={path!r}")
raise KeyError('name=%r, domain=%r, path=%r' % (name, domain, path))
def __getstate__(self) -> dict[str, Any]:
def __getstate__(self):
"""Unlike a normal CookieJar, this class is pickleable."""
state = self.__dict__.copy()
# remove the unpickleable RLock object
state.pop("_cookies_lock")
state.pop('_cookies_lock')
return state
def __setstate__(self, state: dict[str, Any]) -> None:
def __setstate__(self, state):
"""Unlike a normal CookieJar, this class is pickleable."""
self.__dict__.update(state)
if "_cookies_lock" not in self.__dict__:
if '_cookies_lock' not in self.__dict__:
self._cookies_lock = threading.RLock()
def copy(self) -> RequestsCookieJar:
def copy(self):
"""Return a copy of this RequestsCookieJar."""
new_cj = RequestsCookieJar()
new_cj.set_policy(self.get_policy())
new_cj.update(self)
return new_cj
def get_policy(self) -> CookiePolicy:
"""Return the CookiePolicy instance used."""
return self._policy
def _copy_cookie_jar(jar: CookieJar | None) -> CookieJar | None: # type: ignore[reportUnusedFunction] # cross-module usage in models.py
def _copy_cookie_jar(jar):
if jar is None:
return None
if copy_method := getattr(jar, "copy", None):
if hasattr(jar, 'copy'):
# We're dealing with an instance of RequestsCookieJar
return copy_method()
return jar.copy()
# We're dealing with a generic CookieJar instance
new_jar = copy.copy(jar)
new_jar.clear()
@@ -491,103 +432,79 @@ def _copy_cookie_jar(jar: CookieJar | None) -> CookieJar | None: # type: ignore
return new_jar
def create_cookie(name: str, value: str, **kwargs: Any) -> Cookie:
def create_cookie(name, value, **kwargs):
"""Make a cookie from underspecified parameters.
By default, the pair of `name` and `value` will be set for the domain ''
and sent on every request (this is sometimes called a "supercookie").
"""
result: dict[str, Any] = {
"version": 0,
"name": name,
"value": value,
"port": None,
"domain": "",
"path": "/",
"secure": False,
"expires": None,
"discard": True,
"comment": None,
"comment_url": None,
"rest": {"HttpOnly": None},
"rfc2109": False,
}
result = dict(
version=0,
name=name,
value=value,
port=None,
domain='',
path='/',
secure=False,
expires=None,
discard=True,
comment=None,
comment_url=None,
rest={'HttpOnly': None},
rfc2109=False,)
badargs = set(kwargs) - set(result)
if badargs:
raise TypeError(
f"create_cookie() got unexpected keyword arguments: {list(badargs)}"
)
err = 'create_cookie() got unexpected keyword arguments: %s'
raise TypeError(err % list(badargs))
result.update(kwargs)
result["port_specified"] = bool(result["port"])
result["domain_specified"] = bool(result["domain"])
result["domain_initial_dot"] = result["domain"].startswith(".")
result["path_specified"] = bool(result["path"])
result['port_specified'] = bool(result['port'])
result['domain_specified'] = bool(result['domain'])
result['domain_initial_dot'] = result['domain'].startswith('.')
result['path_specified'] = bool(result['path'])
return cookielib.Cookie(**result)
def morsel_to_cookie(morsel: Morsel[Any]) -> Cookie:
def morsel_to_cookie(morsel):
"""Convert a Morsel object into a Cookie containing the one k/v pair."""
expires: int | None = None
if morsel["max-age"]:
expires = None
if morsel['max-age']:
try:
expires = int(time.time() + int(morsel["max-age"]))
expires = int(time.time() + int(morsel['max-age']))
except ValueError:
raise TypeError(f"max-age: {morsel['max-age']} must be integer")
elif morsel["expires"]:
time_template = "%a, %d-%b-%Y %H:%M:%S GMT"
expires = calendar.timegm(time.strptime(morsel["expires"], time_template))
raise TypeError('max-age: %s must be integer' % morsel['max-age'])
elif morsel['expires']:
time_template = '%a, %d-%b-%Y %H:%M:%S GMT'
expires = calendar.timegm(
time.strptime(morsel['expires'], time_template)
)
return create_cookie(
comment=morsel["comment"],
comment_url=bool(morsel["comment"]),
comment=morsel['comment'],
comment_url=bool(morsel['comment']),
discard=False,
domain=morsel["domain"],
domain=morsel['domain'],
expires=expires,
name=morsel.key,
path=morsel["path"],
path=morsel['path'],
port=None,
rest={"HttpOnly": morsel["httponly"]},
rest={'HttpOnly': morsel['httponly']},
rfc2109=False,
secure=bool(morsel["secure"]),
secure=bool(morsel['secure']),
value=morsel.value,
version=morsel["version"] or 0,
version=morsel['version'] or 0,
)
_CookieJarT = TypeVar("_CookieJarT", bound=CookieJar)
@overload
def cookiejar_from_dict(
cookie_dict: dict[str, str] | None,
cookiejar: None = None,
overwrite: bool = True,
) -> RequestsCookieJar: ...
@overload
def cookiejar_from_dict(
cookie_dict: dict[str, str] | None,
cookiejar: _CookieJarT,
overwrite: bool = True,
) -> _CookieJarT: ...
def cookiejar_from_dict(
cookie_dict: dict[str, str] | None,
cookiejar: CookieJar | None = None,
overwrite: bool = True,
) -> CookieJar:
def cookiejar_from_dict(cookie_dict, cookiejar=None, overwrite=True):
"""Returns a CookieJar from a key/value dictionary.
:param cookie_dict: Dict of key/values to insert into CookieJar.
:param cookiejar: (optional) A cookiejar to add the cookies to.
:param overwrite: (optional) If False, will not replace cookies
already in the jar with new ones.
:rtype: CookieJar
"""
if cookiejar is None:
cookiejar = RequestsCookieJar()
@@ -601,24 +518,22 @@ def cookiejar_from_dict(
return cookiejar
def merge_cookies(
cookiejar: CookieJar, cookies: dict[str, str] | CookieJar | None
) -> CookieJar:
def merge_cookies(cookiejar, cookies):
"""Add cookies to cookiejar and returns a merged CookieJar.
:param cookiejar: CookieJar object to add the cookies to.
:param cookies: Dictionary or CookieJar object to be added.
:rtype: CookieJar
"""
if not isinstance(cookiejar, cookielib.CookieJar): # runtime guard
raise ValueError("You can only merge into CookieJar")
if not isinstance(cookiejar, cookielib.CookieJar):
raise ValueError('You can only merge into CookieJar')
if isinstance(cookies, dict):
cookiejar = cookiejar_from_dict(cookies, cookiejar=cookiejar, overwrite=False)
cookiejar = cookiejar_from_dict(
cookies, cookiejar=cookiejar, overwrite=False)
elif isinstance(cookies, cookielib.CookieJar):
if update_method := getattr(cookiejar, "update", None):
update_method(cookies)
else:
try:
cookiejar.update(cookies)
except AttributeError:
for cookie_in_jar in cookies:
cookiejar.set_cookie(cookie_in_jar)

View File

@@ -1,66 +1,28 @@
# -*- coding: utf-8 -*-
"""
requests.exceptions
~~~~~~~~~~~~~~~~~~~
This module contains the set of Requests' exceptions.
"""
from __future__ import annotations
from typing import TYPE_CHECKING, Any
from urllib3.exceptions import HTTPError as BaseHTTPError
from .compat import JSONDecodeError as CompatJSONDecodeError
if TYPE_CHECKING:
from .models import PreparedRequest, Request, Response
class RequestException(IOError):
"""There was an ambiguous exception that occurred while handling your
request.
"""
response: Response | None
request: Request | PreparedRequest | None
def __init__(self, *args: Any, **kwargs: Any) -> None:
def __init__(self, *args, **kwargs):
"""Initialize RequestException with `request` and `response` objects."""
response: Response | None = kwargs.pop("response", None)
response = kwargs.pop('response', None)
self.response = response
self.request = kwargs.pop("request", None)
if response is not None and not self.request and hasattr(response, "request"):
self.request = response.request
super().__init__(*args, **kwargs)
class InvalidJSONError(RequestException):
"""A JSON error occurred."""
class JSONDecodeError(InvalidJSONError, CompatJSONDecodeError):
"""Couldn't decode the text into json"""
def __init__(self, *args: Any, **kwargs: Any) -> None:
"""
Construct the JSONDecodeError instance first with all
args. Then use it's args to construct the IOError so that
the json specific args aren't used as IOError specific args
and the error message from JSONDecodeError is preserved.
"""
CompatJSONDecodeError.__init__(self, *args)
InvalidJSONError.__init__(self, *self.args, **kwargs)
def __reduce__(self) -> tuple[Any, ...] | str:
"""
The __reduce__ method called when pickling the object must
be the one from the JSONDecodeError (be it json/simplejson)
as it expects all the arguments for instantiation, not just
one like the IOError, and the MRO would by default call the
__reduce__ method from the IOError due to the inheritance order.
"""
return CompatJSONDecodeError.__reduce__(self)
self.request = kwargs.pop('request', None)
if (response is not None and not self.request and
hasattr(response, 'request')):
self.request = self.response.request
super(RequestException, self).__init__(*args, **kwargs)
class HTTPError(RequestException):
@@ -108,11 +70,11 @@ class TooManyRedirects(RequestException):
class MissingSchema(RequestException, ValueError):
"""The URL scheme (e.g. http or https) is missing."""
"""The URL schema (e.g. http or https) is missing."""
class InvalidSchema(RequestException, ValueError):
"""The URL scheme provided is either invalid or unsupported."""
"""See defaults.py for valid schemas."""
class InvalidURL(RequestException, ValueError):
@@ -123,40 +85,32 @@ class InvalidHeader(RequestException, ValueError):
"""The header value provided was somehow invalid."""
class InvalidProxyURL(InvalidURL):
"""The proxy URL provided is invalid."""
class ChunkedEncodingError(RequestException):
"""The server declared chunked encoding but sent an invalid chunk."""
class ContentDecodingError(RequestException, BaseHTTPError):
"""Failed to decode response content."""
"""Failed to decode response content"""
class StreamConsumedError(RequestException, TypeError):
"""The content for this response was already consumed."""
"""The content for this response was already consumed"""
class RetryError(RequestException):
"""Custom retries logic failed"""
class UnrewindableBodyError(RequestException):
"""Requests encountered an error when trying to rewind a body."""
"""Requests encountered an error when trying to rewind a body"""
# Warnings
class RequestsWarning(Warning):
"""Base warning for Requests."""
pass
class FileModeWarning(RequestsWarning, DeprecationWarning):
"""A file was opened in text mode, but Requests determined its binary length."""
class RequestsDependencyWarning(RequestsWarning):
"""An imported dependency doesn't match the expected version range."""
pass

110
requests/help.py Normal file
View File

@@ -0,0 +1,110 @@
"""Module containing bug report helper(s)."""
from __future__ import print_function
import json
import platform
import sys
import ssl
import urllib3
import chardet
from . import __version__ as requests_version
try:
from .packages.urllib3.contrib import pyopenssl
except ImportError:
pyopenssl = None
OpenSSL = None
cryptography = None
else:
import OpenSSL
import cryptography
def _implementation():
"""Return a dict with the Python implementation and verison.
Provide both the name and the version of the Python implementation
currently running. For example, on CPython 2.7.5 it will return
{'name': 'CPython', 'version': '2.7.5'}.
This function works best on CPython and PyPy: in particular, it probably
doesn't work for Jython or IronPython. Future investigation should be done
to work out the correct shape of the code for those platforms.
"""
implementation = platform.python_implementation()
if implementation == 'CPython':
implementation_version = platform.python_version()
elif implementation == 'PyPy':
implementation_version = '%s.%s.%s' % (sys.pypy_version_info.major,
sys.pypy_version_info.minor,
sys.pypy_version_info.micro)
if sys.pypy_version_info.releaselevel != 'final':
implementation_version = ''.join([
implementation_version, sys.pypy_version_info.releaselevel
])
elif implementation == 'Jython':
implementation_version = platform.python_version() # Complete Guess
elif implementation == 'IronPython':
implementation_version = platform.python_version() # Complete Guess
else:
implementation_version = 'Unknown'
return {'name': implementation, 'version': implementation_version}
def info():
"""Generate information for a bug report."""
try:
platform_info = {
'system': platform.system(),
'release': platform.release(),
}
except IOError:
platform_info = {
'system': 'Unknown',
'release': 'Unknown',
}
implementation_info = _implementation()
urllib3_info = {'version': urllib3.__version__}
chardet_info = {'version': chardet.__version__}
pyopenssl_info = {
'version': None,
'openssl_version': '',
}
if OpenSSL:
pyopenssl_info = {
'version': OpenSSL.__version__,
'openssl_version': '%x' % OpenSSL.SSL.OPENSSL_VERSION_NUMBER,
}
cryptography_info = {
'version': getattr(cryptography, '__version__', ''),
}
return {
'platform': platform_info,
'implementation': implementation_info,
'system_ssl': {
'version': '%x' % ssl.OPENSSL_VERSION_NUMBER,
},
'using_pyopenssl': pyopenssl is not None,
'pyOpenSSL': pyopenssl_info,
'urllib3': urllib3_info,
'chardet': chardet_info,
'cryptography': cryptography_info,
'requests': {
'version': requests_version,
},
}
def main():
"""Pretty-print the bug information as JSON."""
print(json.dumps(info(), sort_keys=True, indent=2))
if __name__ == '__main__':
main()

34
requests/hooks.py Normal file
View File

@@ -0,0 +1,34 @@
# -*- coding: utf-8 -*-
"""
requests.hooks
~~~~~~~~~~~~~~
This module provides the capabilities for the Requests hooks system.
Available hooks:
``response``:
The response generated from a Request.
"""
HOOKS = ['response']
def default_hooks():
return dict((event, []) for event in HOOKS)
# TODO: response is the only one
def dispatch_hook(key, hooks, hook_data, **kwargs):
"""Dispatches a hook dictionary on a given piece of data."""
hooks = hooks or dict()
hooks = hooks.get(key)
if hooks:
if hasattr(hooks, '__call__'):
hooks = [hooks]
for hook in hooks:
_hook_data = hook(hook_data, **kwargs)
if _hook_data is not None:
hook_data = _hook_data
return hook_data

File diff suppressed because it is too large Load Diff

14
requests/packages.py Normal file
View File

@@ -0,0 +1,14 @@
import sys
# This code exists for backwards compatibility reasons.
# I don't like it either. Just look the other way. :)
for package in ('urllib3', 'idna', 'chardet'):
__import__(package)
# This traversal is apparently necessary such that the identities are
# preserved (requests.packages.urllib3.* is urllib3.*)
for mod in list(sys.modules):
if mod == package or mod.startswith(package + '.'):
sys.modules['requests.packages.' + mod] = sys.modules[mod]
# Kinda cool, though, right?

602
src/requests/sessions.py → requests/sessions.py Normal file → Executable file

File diff suppressed because it is too large Load Diff

91
requests/status_codes.py Normal file
View File

@@ -0,0 +1,91 @@
# -*- coding: utf-8 -*-
from .structures import LookupDict
_codes = {
# Informational.
100: ('continue',),
101: ('switching_protocols',),
102: ('processing',),
103: ('checkpoint',),
122: ('uri_too_long', 'request_uri_too_long'),
200: ('ok', 'okay', 'all_ok', 'all_okay', 'all_good', '\\o/', ''),
201: ('created',),
202: ('accepted',),
203: ('non_authoritative_info', 'non_authoritative_information'),
204: ('no_content',),
205: ('reset_content', 'reset'),
206: ('partial_content', 'partial'),
207: ('multi_status', 'multiple_status', 'multi_stati', 'multiple_stati'),
208: ('already_reported',),
226: ('im_used',),
# Redirection.
300: ('multiple_choices',),
301: ('moved_permanently', 'moved', '\\o-'),
302: ('found',),
303: ('see_other', 'other'),
304: ('not_modified',),
305: ('use_proxy',),
306: ('switch_proxy',),
307: ('temporary_redirect', 'temporary_moved', 'temporary'),
308: ('permanent_redirect',
'resume_incomplete', 'resume',), # These 2 to be removed in 3.0
# Client Error.
400: ('bad_request', 'bad'),
401: ('unauthorized',),
402: ('payment_required', 'payment'),
403: ('forbidden',),
404: ('not_found', '-o-'),
405: ('method_not_allowed', 'not_allowed'),
406: ('not_acceptable',),
407: ('proxy_authentication_required', 'proxy_auth', 'proxy_authentication'),
408: ('request_timeout', 'timeout'),
409: ('conflict',),
410: ('gone',),
411: ('length_required',),
412: ('precondition_failed', 'precondition'),
413: ('request_entity_too_large',),
414: ('request_uri_too_large',),
415: ('unsupported_media_type', 'unsupported_media', 'media_type'),
416: ('requested_range_not_satisfiable', 'requested_range', 'range_not_satisfiable'),
417: ('expectation_failed',),
418: ('im_a_teapot', 'teapot', 'i_am_a_teapot'),
421: ('misdirected_request',),
422: ('unprocessable_entity', 'unprocessable'),
423: ('locked',),
424: ('failed_dependency', 'dependency'),
425: ('unordered_collection', 'unordered'),
426: ('upgrade_required', 'upgrade'),
428: ('precondition_required', 'precondition'),
429: ('too_many_requests', 'too_many'),
431: ('header_fields_too_large', 'fields_too_large'),
444: ('no_response', 'none'),
449: ('retry_with', 'retry'),
450: ('blocked_by_windows_parental_controls', 'parental_controls'),
451: ('unavailable_for_legal_reasons', 'legal_reasons'),
499: ('client_closed_request',),
# Server Error.
500: ('internal_server_error', 'server_error', '/o\\', ''),
501: ('not_implemented',),
502: ('bad_gateway',),
503: ('service_unavailable', 'unavailable'),
504: ('gateway_timeout',),
505: ('http_version_not_supported', 'http_version'),
506: ('variant_also_negotiates',),
507: ('insufficient_storage',),
509: ('bandwidth_limit_exceeded', 'bandwidth'),
510: ('not_extended',),
511: ('network_authentication_required', 'network_auth', 'network_authentication'),
}
codes = LookupDict(name='status_codes')
for code, titles in _codes.items():
for title in titles:
setattr(codes, title, code)
if not title.startswith(('\\', '/')):
setattr(codes, title.upper(), code)

105
requests/structures.py Normal file
View File

@@ -0,0 +1,105 @@
# -*- coding: utf-8 -*-
"""
requests.structures
~~~~~~~~~~~~~~~~~~~
Data structures that power Requests.
"""
import collections
from .compat import OrderedDict
class CaseInsensitiveDict(collections.MutableMapping):
"""A case-insensitive ``dict``-like object.
Implements all methods and operations of
``collections.MutableMapping`` as well as dict's ``copy``. Also
provides ``lower_items``.
All keys are expected to be strings. The structure remembers the
case of the last key to be set, and ``iter(instance)``,
``keys()``, ``items()``, ``iterkeys()``, and ``iteritems()``
will contain case-sensitive keys. However, querying and contains
testing is case insensitive::
cid = CaseInsensitiveDict()
cid['Accept'] = 'application/json'
cid['aCCEPT'] == 'application/json' # True
list(cid) == ['Accept'] # True
For example, ``headers['content-encoding']`` will return the
value of a ``'Content-Encoding'`` response header, regardless
of how the header name was originally stored.
If the constructor, ``.update``, or equality comparison
operations are given keys that have equal ``.lower()``s, the
behavior is undefined.
"""
def __init__(self, data=None, **kwargs):
self._store = OrderedDict()
if data is None:
data = {}
self.update(data, **kwargs)
def __setitem__(self, key, value):
# Use the lowercased key for lookups, but store the actual
# key alongside the value.
self._store[key.lower()] = (key, value)
def __getitem__(self, key):
return self._store[key.lower()][1]
def __delitem__(self, key):
del self._store[key.lower()]
def __iter__(self):
return (casedkey for casedkey, mappedvalue in self._store.values())
def __len__(self):
return len(self._store)
def lower_items(self):
"""Like iteritems(), but with all lowercase keys."""
return (
(lowerkey, keyval[1])
for (lowerkey, keyval)
in self._store.items()
)
def __eq__(self, other):
if isinstance(other, collections.Mapping):
other = CaseInsensitiveDict(other)
else:
return NotImplemented
# Compare insensitively
return dict(self.lower_items()) == dict(other.lower_items())
# Copy is required
def copy(self):
return CaseInsensitiveDict(self._store.values())
def __repr__(self):
return str(dict(self.items()))
class LookupDict(dict):
"""Dictionary lookup object."""
def __init__(self, name=None):
self.name = name
super(LookupDict, self).__init__()
def __repr__(self):
return '<lookup \'%s\'>' % (self.name)
def __getitem__(self, key):
# We allow fall-through here, so values default to None
return self.__dict__.get(key, None)
def get(self, key, default=None):
return self.__dict__.get(key, default)

903
requests/utils.py Normal file
View File

@@ -0,0 +1,903 @@
# -*- coding: utf-8 -*-
"""
requests.utils
~~~~~~~~~~~~~~
This module provides utility functions that are used within Requests
that are also useful for external consumption.
"""
import cgi
import codecs
import collections
import contextlib
import io
import os
import platform
import re
import socket
import struct
import warnings
from .__version__ import __version__
from . import certs
# to_native_string is unused here, but imported here for backwards compatibility
from ._internal_utils import to_native_string
from .compat import parse_http_list as _parse_list_header
from .compat import (
quote, urlparse, bytes, str, OrderedDict, unquote, getproxies,
proxy_bypass, urlunparse, basestring, integer_types, is_py3,
proxy_bypass_environment, getproxies_environment)
from .cookies import cookiejar_from_dict
from .structures import CaseInsensitiveDict
from .exceptions import (
InvalidURL, InvalidHeader, FileModeWarning, UnrewindableBodyError)
NETRC_FILES = ('.netrc', '_netrc')
DEFAULT_CA_BUNDLE_PATH = certs.where()
if platform.system() == 'Windows':
# provide a proxy_bypass version on Windows without DNS lookups
def proxy_bypass_registry(host):
if is_py3:
import winreg
else:
import _winreg as winreg
try:
internetSettings = winreg.OpenKey(winreg.HKEY_CURRENT_USER,
r'Software\Microsoft\Windows\CurrentVersion\Internet Settings')
proxyEnable = winreg.QueryValueEx(internetSettings,
'ProxyEnable')[0]
proxyOverride = winreg.QueryValueEx(internetSettings,
'ProxyOverride')[0]
except OSError:
return False
if not proxyEnable or not proxyOverride:
return False
# make a check value list from the registry entry: replace the
# '<local>' string by the localhost entry and the corresponding
# canonical entry.
proxyOverride = proxyOverride.split(';')
# now check if we match one of the registry values.
for test in proxyOverride:
if test == '<local>':
if '.' not in host:
return True
test = test.replace(".", r"\.") # mask dots
test = test.replace("*", r".*") # change glob sequence
test = test.replace("?", r".") # change glob char
if re.match(test, host, re.I):
return True
return False
def proxy_bypass(host): # noqa
"""Return True, if the host should be bypassed.
Checks proxy settings gathered from the environment, if specified,
or the registry.
"""
if getproxies_environment():
return proxy_bypass_environment(host)
else:
return proxy_bypass_registry(host)
def dict_to_sequence(d):
"""Returns an internal sequence dictionary update."""
if hasattr(d, 'items'):
d = d.items()
return d
def super_len(o):
total_length = None
current_position = 0
if hasattr(o, '__len__'):
total_length = len(o)
elif hasattr(o, 'len'):
total_length = o.len
elif hasattr(o, 'fileno'):
try:
fileno = o.fileno()
except io.UnsupportedOperation:
pass
else:
total_length = os.fstat(fileno).st_size
# Having used fstat to determine the file length, we need to
# confirm that this file was opened up in binary mode.
if 'b' not in o.mode:
warnings.warn((
"Requests has determined the content-length for this "
"request using the binary size of the file: however, the "
"file has been opened in text mode (i.e. without the 'b' "
"flag in the mode). This may lead to an incorrect "
"content-length. In Requests 3.0, support will be removed "
"for files in text mode."),
FileModeWarning
)
if hasattr(o, 'tell'):
try:
current_position = o.tell()
except (OSError, IOError):
# This can happen in some weird situations, such as when the file
# is actually a special file descriptor like stdin. In this
# instance, we don't know what the length is, so set it to zero and
# let requests chunk it instead.
if total_length is not None:
current_position = total_length
else:
if hasattr(o, 'seek') and total_length is None:
# StringIO and BytesIO have seek but no useable fileno
try:
# seek to end of file
o.seek(0, 2)
total_length = o.tell()
# seek back to current position to support
# partially read file-like objects
o.seek(current_position or 0)
except (OSError, IOError):
total_length = 0
if total_length is None:
total_length = 0
return max(0, total_length - current_position)
def get_netrc_auth(url, raise_errors=False):
"""Returns the Requests tuple auth for a given url from netrc."""
try:
from netrc import netrc, NetrcParseError
netrc_path = None
for f in NETRC_FILES:
try:
loc = os.path.expanduser('~/{0}'.format(f))
except KeyError:
# os.path.expanduser can fail when $HOME is undefined and
# getpwuid fails. See http://bugs.python.org/issue20164 &
# https://github.com/requests/requests/issues/1846
return
if os.path.exists(loc):
netrc_path = loc
break
# Abort early if there isn't one.
if netrc_path is None:
return
ri = urlparse(url)
# Strip port numbers from netloc. This weird `if...encode`` dance is
# used for Python 3.2, which doesn't support unicode literals.
splitstr = b':'
if isinstance(url, str):
splitstr = splitstr.decode('ascii')
host = ri.netloc.split(splitstr)[0]
try:
_netrc = netrc(netrc_path).authenticators(host)
if _netrc:
# Return with login / password
login_i = (0 if _netrc[0] else 1)
return (_netrc[login_i], _netrc[2])
except (NetrcParseError, IOError):
# If there was a parsing error or a permissions issue reading the file,
# we'll just skip netrc auth unless explicitly asked to raise errors.
if raise_errors:
raise
# AppEngine hackiness.
except (ImportError, AttributeError):
pass
def guess_filename(obj):
"""Tries to guess the filename of the given object."""
name = getattr(obj, 'name', None)
if (name and isinstance(name, basestring) and name[0] != '<' and
name[-1] != '>'):
return os.path.basename(name)
def from_key_val_list(value):
"""Take an object and test to see if it can be represented as a
dictionary. Unless it can not be represented as such, return an
OrderedDict, e.g.,
::
>>> from_key_val_list([('key', 'val')])
OrderedDict([('key', 'val')])
>>> from_key_val_list('string')
ValueError: need more than 1 value to unpack
>>> from_key_val_list({'key': 'val'})
OrderedDict([('key', 'val')])
:rtype: OrderedDict
"""
if value is None:
return None
if isinstance(value, (str, bytes, bool, int)):
raise ValueError('cannot encode objects that are not 2-tuples')
return OrderedDict(value)
def to_key_val_list(value):
"""Take an object and test to see if it can be represented as a
dictionary. If it can be, return a list of tuples, e.g.,
::
>>> to_key_val_list([('key', 'val')])
[('key', 'val')]
>>> to_key_val_list({'key': 'val'})
[('key', 'val')]
>>> to_key_val_list('string')
ValueError: cannot encode objects that are not 2-tuples.
:rtype: list
"""
if value is None:
return None
if isinstance(value, (str, bytes, bool, int)):
raise ValueError('cannot encode objects that are not 2-tuples')
if isinstance(value, collections.Mapping):
value = value.items()
return list(value)
# From mitsuhiko/werkzeug (used with permission).
def parse_list_header(value):
"""Parse lists as described by RFC 2068 Section 2.
In particular, parse comma-separated lists where the elements of
the list may include quoted-strings. A quoted-string could
contain a comma. A non-quoted string could have quotes in the
middle. Quotes are removed automatically after parsing.
It basically works like :func:`parse_set_header` just that items
may appear multiple times and case sensitivity is preserved.
The return value is a standard :class:`list`:
>>> parse_list_header('token, "quoted value"')
['token', 'quoted value']
To create a header from the :class:`list` again, use the
:func:`dump_header` function.
:param value: a string with a list header.
:return: :class:`list`
:rtype: list
"""
result = []
for item in _parse_list_header(value):
if item[:1] == item[-1:] == '"':
item = unquote_header_value(item[1:-1])
result.append(item)
return result
# From mitsuhiko/werkzeug (used with permission).
def parse_dict_header(value):
"""Parse lists of key, value pairs as described by RFC 2068 Section 2 and
convert them into a python dict:
>>> d = parse_dict_header('foo="is a fish", bar="as well"')
>>> type(d) is dict
True
>>> sorted(d.items())
[('bar', 'as well'), ('foo', 'is a fish')]
If there is no value for a key it will be `None`:
>>> parse_dict_header('key_without_value')
{'key_without_value': None}
To create a header from the :class:`dict` again, use the
:func:`dump_header` function.
:param value: a string with a dict header.
:return: :class:`dict`
:rtype: dict
"""
result = {}
for item in _parse_list_header(value):
if '=' not in item:
result[item] = None
continue
name, value = item.split('=', 1)
if value[:1] == value[-1:] == '"':
value = unquote_header_value(value[1:-1])
result[name] = value
return result
# From mitsuhiko/werkzeug (used with permission).
def unquote_header_value(value, is_filename=False):
r"""Unquotes a header value. (Reversal of :func:`quote_header_value`).
This does not use the real unquoting but what browsers are actually
using for quoting.
:param value: the header value to unquote.
:rtype: str
"""
if value and value[0] == value[-1] == '"':
# this is not the real unquoting, but fixing this so that the
# RFC is met will result in bugs with internet explorer and
# probably some other browsers as well. IE for example is
# uploading files with "C:\foo\bar.txt" as filename
value = value[1:-1]
# if this is a filename and the starting characters look like
# a UNC path, then just return the value without quotes. Using the
# replace sequence below on a UNC path has the effect of turning
# the leading double slash into a single slash and then
# _fix_ie_filename() doesn't work correctly. See #458.
if not is_filename or value[:2] != '\\\\':
return value.replace('\\\\', '\\').replace('\\"', '"')
return value
def dict_from_cookiejar(cj):
"""Returns a key/value dictionary from a CookieJar.
:param cj: CookieJar object to extract cookies from.
:rtype: dict
"""
cookie_dict = {}
for cookie in cj:
cookie_dict[cookie.name] = cookie.value
return cookie_dict
def add_dict_to_cookiejar(cj, cookie_dict):
"""Returns a CookieJar from a key/value dictionary.
:param cj: CookieJar to insert cookies into.
:param cookie_dict: Dict of key/values to insert into CookieJar.
:rtype: CookieJar
"""
return cookiejar_from_dict(cookie_dict, cj)
def get_encodings_from_content(content):
"""Returns encodings from given content string.
:param content: bytestring to extract encodings from.
"""
warnings.warn((
'In requests 3.0, get_encodings_from_content will be removed. For '
'more information, please see the discussion on issue #2266. (This'
' warning should only appear once.)'),
DeprecationWarning)
charset_re = re.compile(r'<meta.*?charset=["\']*(.+?)["\'>]', flags=re.I)
pragma_re = re.compile(r'<meta.*?content=["\']*;?charset=(.+?)["\'>]', flags=re.I)
xml_re = re.compile(r'^<\?xml.*?encoding=["\']*(.+?)["\'>]')
return (charset_re.findall(content) +
pragma_re.findall(content) +
xml_re.findall(content))
def get_encoding_from_headers(headers):
"""Returns encodings from given HTTP Header Dict.
:param headers: dictionary to extract encoding from.
:rtype: str
"""
content_type = headers.get('content-type')
if not content_type:
return None
content_type, params = cgi.parse_header(content_type)
if 'charset' in params:
return params['charset'].strip("'\"")
if 'text' in content_type:
return 'ISO-8859-1'
def stream_decode_response_unicode(iterator, r):
"""Stream decodes a iterator."""
if r.encoding is None:
for item in iterator:
yield item
return
decoder = codecs.getincrementaldecoder(r.encoding)(errors='replace')
for chunk in iterator:
rv = decoder.decode(chunk)
if rv:
yield rv
rv = decoder.decode(b'', final=True)
if rv:
yield rv
def iter_slices(string, slice_length):
"""Iterate over slices of a string."""
pos = 0
if slice_length is None or slice_length <= 0:
slice_length = len(string)
while pos < len(string):
yield string[pos:pos + slice_length]
pos += slice_length
def get_unicode_from_response(r):
"""Returns the requested content back in unicode.
:param r: Response object to get unicode content from.
Tried:
1. charset from content-type
2. fall back and replace all unicode characters
:rtype: str
"""
warnings.warn((
'In requests 3.0, get_unicode_from_response will be removed. For '
'more information, please see the discussion on issue #2266. (This'
' warning should only appear once.)'),
DeprecationWarning)
tried_encodings = []
# Try charset from content-type
encoding = get_encoding_from_headers(r.headers)
if encoding:
try:
return str(r.content, encoding)
except UnicodeError:
tried_encodings.append(encoding)
# Fall back:
try:
return str(r.content, encoding, errors='replace')
except TypeError:
return r.content
# The unreserved URI characters (RFC 3986)
UNRESERVED_SET = frozenset(
"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
+ "0123456789-._~")
def unquote_unreserved(uri):
"""Un-escape any percent-escape sequences in a URI that are unreserved
characters. This leaves all reserved, illegal and non-ASCII bytes encoded.
:rtype: str
"""
parts = uri.split('%')
for i in range(1, len(parts)):
h = parts[i][0:2]
if len(h) == 2 and h.isalnum():
try:
c = chr(int(h, 16))
except ValueError:
raise InvalidURL("Invalid percent-escape sequence: '%s'" % h)
if c in UNRESERVED_SET:
parts[i] = c + parts[i][2:]
else:
parts[i] = '%' + parts[i]
else:
parts[i] = '%' + parts[i]
return ''.join(parts)
def requote_uri(uri):
"""Re-quote the given URI.
This function passes the given URI through an unquote/quote cycle to
ensure that it is fully and consistently quoted.
:rtype: str
"""
safe_with_percent = "!#$%&'()*+,/:;=?@[]~"
safe_without_percent = "!#$&'()*+,/:;=?@[]~"
try:
# Unquote only the unreserved characters
# Then quote only illegal characters (do not quote reserved,
# unreserved, or '%')
return quote(unquote_unreserved(uri), safe=safe_with_percent)
except InvalidURL:
# We couldn't unquote the given URI, so let's try quoting it, but
# there may be unquoted '%'s in the URI. We need to make sure they're
# properly quoted so they do not cause issues elsewhere.
return quote(uri, safe=safe_without_percent)
def address_in_network(ip, net):
"""This function allows you to check if an IP belongs to a network subnet
Example: returns True if ip = 192.168.1.1 and net = 192.168.1.0/24
returns False if ip = 192.168.1.1 and net = 192.168.100.0/24
:rtype: bool
"""
ipaddr = struct.unpack('=L', socket.inet_aton(ip))[0]
netaddr, bits = net.split('/')
netmask = struct.unpack('=L', socket.inet_aton(dotted_netmask(int(bits))))[0]
network = struct.unpack('=L', socket.inet_aton(netaddr))[0] & netmask
return (ipaddr & netmask) == (network & netmask)
def dotted_netmask(mask):
"""Converts mask from /xx format to xxx.xxx.xxx.xxx
Example: if mask is 24 function returns 255.255.255.0
:rtype: str
"""
bits = 0xffffffff ^ (1 << 32 - mask) - 1
return socket.inet_ntoa(struct.pack('>I', bits))
def is_ipv4_address(string_ip):
"""
:rtype: bool
"""
try:
socket.inet_aton(string_ip)
except socket.error:
return False
return True
def is_valid_cidr(string_network):
"""
Very simple check of the cidr format in no_proxy variable.
:rtype: bool
"""
if string_network.count('/') == 1:
try:
mask = int(string_network.split('/')[1])
except ValueError:
return False
if mask < 1 or mask > 32:
return False
try:
socket.inet_aton(string_network.split('/')[0])
except socket.error:
return False
else:
return False
return True
@contextlib.contextmanager
def set_environ(env_name, value):
"""Set the environment variable 'env_name' to 'value'
Save previous value, yield, and then restore the previous value stored in
the environment variable 'env_name'.
If 'value' is None, do nothing"""
if value is not None:
old_value = os.environ.get(env_name)
os.environ[env_name] = value
try:
yield
finally:
if value is None:
return
if old_value is None:
del os.environ[env_name]
else:
os.environ[env_name] = old_value
def should_bypass_proxies(url, no_proxy):
"""
Returns whether we should bypass proxies or not.
:rtype: bool
"""
get_proxy = lambda k: os.environ.get(k) or os.environ.get(k.upper())
# First check whether no_proxy is defined. If it is, check that the URL
# we're getting isn't in the no_proxy list.
no_proxy_arg = no_proxy
if no_proxy is None:
no_proxy = get_proxy('no_proxy')
netloc = urlparse(url).netloc
if no_proxy:
# We need to check whether we match here. We need to see if we match
# the end of the netloc, both with and without the port.
no_proxy = (
host for host in no_proxy.replace(' ', '').split(',') if host
)
ip = netloc.split(':')[0]
if is_ipv4_address(ip):
for proxy_ip in no_proxy:
if is_valid_cidr(proxy_ip):
if address_in_network(ip, proxy_ip):
return True
elif ip == proxy_ip:
# If no_proxy ip was defined in plain IP notation instead of cidr notation &
# matches the IP of the index
return True
else:
for host in no_proxy:
if netloc.endswith(host) or netloc.split(':')[0].endswith(host):
# The URL does match something in no_proxy, so we don't want
# to apply the proxies on this URL.
return True
# If the system proxy settings indicate that this URL should be bypassed,
# don't proxy.
# The proxy_bypass function is incredibly buggy on OS X in early versions
# of Python 2.6, so allow this call to fail. Only catch the specific
# exceptions we've seen, though: this call failing in other ways can reveal
# legitimate problems.
with set_environ('no_proxy', no_proxy_arg):
try:
bypass = proxy_bypass(netloc)
except (TypeError, socket.gaierror):
bypass = False
if bypass:
return True
return False
def get_environ_proxies(url, no_proxy=None):
"""
Return a dict of environment proxies.
:rtype: dict
"""
if should_bypass_proxies(url, no_proxy=no_proxy):
return {}
else:
return getproxies()
def select_proxy(url, proxies):
"""Select a proxy for the url, if applicable.
:param url: The url being for the request
:param proxies: A dictionary of schemes or schemes and hosts to proxy URLs
"""
proxies = proxies or {}
urlparts = urlparse(url)
if urlparts.hostname is None:
return proxies.get(urlparts.scheme, proxies.get('all'))
proxy_keys = [
urlparts.scheme + '://' + urlparts.hostname,
urlparts.scheme,
'all://' + urlparts.hostname,
'all',
]
proxy = None
for proxy_key in proxy_keys:
if proxy_key in proxies:
proxy = proxies[proxy_key]
break
return proxy
def default_user_agent(name="python-requests"):
"""
Return a string representing the default user agent.
:rtype: str
"""
return '%s/%s' % (name, __version__)
def default_headers():
"""
:rtype: requests.structures.CaseInsensitiveDict
"""
return CaseInsensitiveDict({
'User-Agent': default_user_agent(),
'Accept-Encoding': ', '.join(('gzip', 'deflate')),
'Accept': '*/*',
'Connection': 'keep-alive',
})
def parse_header_links(value):
"""Return a dict of parsed link headers proxies.
i.e. Link: <http:/.../front.jpeg>; rel=front; type="image/jpeg",<http://.../back.jpeg>; rel=back;type="image/jpeg"
:rtype: list
"""
links = []
replace_chars = ' \'"'
for val in re.split(', *<', value):
try:
url, params = val.split(';', 1)
except ValueError:
url, params = val, ''
link = {'url': url.strip('<> \'"')}
for param in params.split(';'):
try:
key, value = param.split('=')
except ValueError:
break
link[key.strip(replace_chars)] = value.strip(replace_chars)
links.append(link)
return links
# Null bytes; no need to recreate these on each call to guess_json_utf
_null = '\x00'.encode('ascii') # encoding to ASCII for Python 3
_null2 = _null * 2
_null3 = _null * 3
def guess_json_utf(data):
"""
:rtype: str
"""
# JSON always starts with two ASCII characters, so detection is as
# easy as counting the nulls and from their location and count
# determine the encoding. Also detect a BOM, if present.
sample = data[:4]
if sample in (codecs.BOM_UTF32_LE, codecs.BOM_UTF32_BE):
return 'utf-32' # BOM included
if sample[:3] == codecs.BOM_UTF8:
return 'utf-8-sig' # BOM included, MS style (discouraged)
if sample[:2] in (codecs.BOM_UTF16_LE, codecs.BOM_UTF16_BE):
return 'utf-16' # BOM included
nullcount = sample.count(_null)
if nullcount == 0:
return 'utf-8'
if nullcount == 2:
if sample[::2] == _null2: # 1st and 3rd are null
return 'utf-16-be'
if sample[1::2] == _null2: # 2nd and 4th are null
return 'utf-16-le'
# Did not detect 2 valid UTF-16 ascii-range characters
if nullcount == 3:
if sample[:3] == _null3:
return 'utf-32-be'
if sample[1:] == _null3:
return 'utf-32-le'
# Did not detect a valid UTF-32 ascii-range character
return None
def prepend_scheme_if_needed(url, new_scheme):
"""Given a URL that may or may not have a scheme, prepend the given scheme.
Does not replace a present scheme with the one provided as an argument.
:rtype: str
"""
scheme, netloc, path, params, query, fragment = urlparse(url, new_scheme)
# urlparse is a finicky beast, and sometimes decides that there isn't a
# netloc present. Assume that it's being over-cautious, and switch netloc
# and path if urlparse decided there was no netloc.
if not netloc:
netloc, path = path, netloc
return urlunparse((scheme, netloc, path, params, query, fragment))
def get_auth_from_url(url):
"""Given a url with authentication components, extract them into a tuple of
username,password.
:rtype: (str,str)
"""
parsed = urlparse(url)
try:
auth = (unquote(parsed.username), unquote(parsed.password))
except (AttributeError, TypeError):
auth = ('', '')
return auth
# Moved outside of function to avoid recompile every call
_CLEAN_HEADER_REGEX_BYTE = re.compile(b'^\\S[^\\r\\n]*$|^$')
_CLEAN_HEADER_REGEX_STR = re.compile(r'^\S[^\r\n]*$|^$')
def check_header_validity(header):
"""Verifies that header value is a string which doesn't contain
leading whitespace or return characters. This prevents unintended
header injection.
:param header: tuple, in the format (name, value).
"""
name, value = header
if isinstance(value, bytes):
pat = _CLEAN_HEADER_REGEX_BYTE
else:
pat = _CLEAN_HEADER_REGEX_STR
try:
if not pat.match(value):
raise InvalidHeader("Invalid return character or leading space in header: %s" % name)
except TypeError:
raise InvalidHeader("Header value %s must be of type str or bytes, "
"not %s" % (value, type(value)))
def urldefragauth(url):
"""
Given a url remove the fragment and the authentication part.
:rtype: str
"""
scheme, netloc, path, params, query, fragment = urlparse(url)
# see func:`prepend_scheme_if_needed`
if not netloc:
netloc, path = path, netloc
netloc = netloc.rsplit('@', 1)[-1]
return urlunparse((scheme, netloc, path, params, query, ''))
def rewind_body(prepared_request):
"""Move file pointer back to its recorded starting position
so it can be read again on redirect.
"""
body_seek = getattr(prepared_request.body, 'seek', None)
if body_seek is not None and isinstance(prepared_request._body_position, integer_types):
try:
body_seek(prepared_request._body_position)
except (IOError, OSError):
raise UnrewindableBodyError("An error occurred when rewinding request "
"body for redirect.")
else:
raise UnrewindableBodyError("Unable to rewind request body for redirect.")

View File

@@ -1,7 +0,0 @@
-e .[socks]
pytest>=2.8.0,<10
pytest-cov
pytest-httpbin==2.1.0
httpbin~=0.10.0
trustme
wheel

16
requirements.txt Normal file
View File

@@ -0,0 +1,16 @@
-e .[socks]
pytest>=2.8.0
codecov
pytest-httpbin==0.0.7
pytest-mock
pytest-cov
pytest-xdist
alabaster
readme_renderer
Sphinx<=1.5.5
PySocks
setuptools>=18.5
docutils
flake8
tox
detox

2
setup.cfg Normal file
View File

@@ -0,0 +1,2 @@
[bdist_wheel]
universal = 1

View File

@@ -1,9 +1,98 @@
#!/usr/bin/env python
import os
import re
import sys
if sys.version_info < (3, 10): # noqa: UP036
sys.stderr.write("Requests requires Python 3.10 or later.\n")
sys.exit(1)
from codecs import open
from setuptools import setup
from setuptools.command.test import test as TestCommand
from multiprocessing import cpu_count
here = os.path.abspath(os.path.dirname(__file__))
class PyTest(TestCommand):
user_options = [('pytest-args=', 'a', "Arguments to pass into py.test")]
def initialize_options(self):
TestCommand.initialize_options(self)
self.pytest_args = ['-n', str(cpu_count()), '--boxed']
def finalize_options(self):
TestCommand.finalize_options(self)
self.test_args = []
self.test_suite = True
def run_tests(self):
import pytest
errno = pytest.main(self.pytest_args)
sys.exit(errno)
# 'setup.py publish' shortcut.
if sys.argv[-1] == 'publish':
os.system('python setup.py sdist bdist_wheel')
os.system('twine upload dist/*')
sys.exit()
packages = ['requests']
requires = [
'chardet>=3.0.2,<3.1.0',
'idna>=2.5,<2.6',
'urllib3>=1.21.1,<1.22',
'certifi>=2017.4.17'
]
test_requirements = ['pytest-httpbin==0.0.7', 'pytest-cov', 'pytest-mock', 'pytest-xdist', 'PySocks>=1.5.6, !=1.5.7', 'pytest>=2.8.0']
about = {}
with open(os.path.join(here, 'requests', '__version__.py'), 'r', 'utf-8') as f:
exec(f.read(), about)
with open('README.rst', 'r', 'utf-8') as f:
readme = f.read()
with open('HISTORY.rst', 'r', 'utf-8') as f:
history = f.read()
setup(
name=about['__title__'],
version=about['__version__'],
description=about['__description__'],
long_description=readme + '\n\n' + history,
author=about['__author__'],
author_email=about['__author_email__'],
url=about['__url__'],
packages=packages,
package_data={'': ['LICENSE', 'NOTICE'], 'requests': ['*.pem']},
package_dir={'requests': 'requests'},
include_package_data=True,
install_requires=requires,
license=about['__license__'],
zip_safe=False,
classifiers=(
'Development Status :: 5 - Production/Stable',
'Intended Audience :: Developers',
'Natural Language :: English',
'License :: OSI Approved :: Apache Software License',
'Programming Language :: Python',
'Programming Language :: Python :: 2.6',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.3',
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: Implementation :: CPython',
'Programming Language :: Python :: Implementation :: PyPy'
),
cmdclass={'test': PyTest},
tests_require=test_requirements,
extras_require={
'security': ['pyOpenSSL>=0.14', 'cryptography>=1.3.4', 'idna>=2.0.0'],
'socks': ['PySocks>=1.5.6, !=1.5.7'],
'socks:sys_platform == "win32" and (python_version == "2.7" or python_version == "2.6")': ['win_inet_pton'],
},
)
setup()

View File

@@ -1,219 +0,0 @@
# __
# /__) _ _ _ _ _/ _
# / ( (- (/ (/ (- _) / _)
# /
"""
Requests HTTP Library
~~~~~~~~~~~~~~~~~~~~~
Requests is an HTTP library, written in Python, for human beings.
Basic GET usage:
>>> import requests
>>> r = requests.get('https://www.python.org')
>>> r.status_code
200
>>> b'Python is a programming language' in r.content
True
... or POST:
>>> payload = dict(key1='value1', key2='value2')
>>> r = requests.post('https://httpbin.org/post', data=payload)
>>> print(r.text)
{
...
"form": {
"key1": "value1",
"key2": "value2"
},
...
}
The other HTTP methods are supported - see `requests.api`. Full documentation
is at <https://requests.readthedocs.io>.
:copyright: (c) 2017 by Kenneth Reitz.
:license: Apache 2.0, see LICENSE for more details.
"""
from __future__ import annotations
import warnings
import urllib3
from .exceptions import RequestsDependencyWarning
try:
from charset_normalizer import __version__ as charset_normalizer_version
except ImportError:
charset_normalizer_version = None
try:
from chardet import __version__ as chardet_version
except ImportError:
chardet_version = None
def check_compatibility(
urllib3_version: str,
chardet_version: str | None,
charset_normalizer_version: str | None,
) -> None:
urllib3_version_list = urllib3_version.split(".")[:3]
assert urllib3_version_list != ["dev"] # Verify urllib3 isn't installed from git.
# Sometimes, urllib3 only reports its version as 16.1.
if len(urllib3_version_list) == 2:
urllib3_version_list.append("0")
# Check urllib3 for compatibility.
major, minor, patch = urllib3_version_list # noqa: F811
major, minor, patch = int(major), int(minor), int(patch)
# urllib3 >= 1.21.1
assert major >= 1
if major == 1:
assert minor >= 21
# Check charset_normalizer for compatibility.
if chardet_version:
major, minor, patch = chardet_version.split(".")[:3]
major, minor, patch = int(major), int(minor), int(patch)
# chardet_version >= 3.0.2, < 8.0.0
assert (3, 0, 2) <= (major, minor, patch) < (8, 0, 0)
elif charset_normalizer_version:
major, minor, patch = charset_normalizer_version.split(".")[:3]
major, minor, patch = int(major), int(minor), int(patch)
# charset_normalizer >= 2.0.0 < 4.0.0
assert (2, 0, 0) <= (major, minor, patch) < (4, 0, 0)
else:
warnings.warn(
"Unable to find acceptable character detection dependency "
"(chardet or charset_normalizer).",
RequestsDependencyWarning,
)
def _check_cryptography(cryptography_version: str) -> None:
# cryptography < 1.3.4
try:
cryptography_version_list = list(map(int, cryptography_version.split(".")))
except ValueError:
return
if cryptography_version_list < [1, 3, 4]:
warning = f"Old version of cryptography ({cryptography_version_list}) may cause slowdown."
warnings.warn(warning, RequestsDependencyWarning)
# Check imported dependencies for compatibility.
try:
check_compatibility(
urllib3.__version__,
chardet_version,
charset_normalizer_version,
)
except (AssertionError, ValueError):
warnings.warn(
f"urllib3 ({urllib3.__version__}) or chardet "
f"({chardet_version})/charset_normalizer ({charset_normalizer_version}) "
"doesn't match a supported version!",
RequestsDependencyWarning,
)
# Attempt to enable urllib3's fallback for SNI support
# if the standard library doesn't support SNI or the
# 'ssl' library isn't available.
try:
try:
import ssl
except ImportError:
ssl = None
if not getattr(ssl, "HAS_SNI", False):
from urllib3.contrib import pyopenssl
pyopenssl.inject_into_urllib3()
# Check cryptography version
from cryptography import (
__version__ as cryptography_version,
)
_check_cryptography(cryptography_version)
except ImportError:
pass
# urllib3's DependencyWarnings should be silenced.
from urllib3.exceptions import DependencyWarning
warnings.simplefilter("ignore", DependencyWarning)
# Set default logging handler to avoid "No handler found" warnings.
import logging
from logging import NullHandler
from . import packages, utils
from .__version__ import (
__author__,
__author_email__,
__build__,
__cake__,
__copyright__,
__description__,
__license__,
__title__,
__url__,
__version__,
)
from .api import delete, get, head, options, patch, post, put, request
from .exceptions import (
ConnectionError,
ConnectTimeout,
FileModeWarning,
HTTPError,
JSONDecodeError,
ReadTimeout,
RequestException,
Timeout,
TooManyRedirects,
URLRequired,
)
from .models import PreparedRequest, Request, Response
from .sessions import Session, session
from .status_codes import codes
__all__ = (
"ConnectionError",
"ConnectTimeout",
"HTTPError",
"JSONDecodeError",
"PreparedRequest",
"ReadTimeout",
"Request",
"RequestException",
"Response",
"Session",
"Timeout",
"TooManyRedirects",
"URLRequired",
"codes",
"delete",
"get",
"head",
"options",
"packages",
"patch",
"post",
"put",
"request",
"session",
"utils",
)
logging.getLogger(__name__).addHandler(NullHandler())
# FileModeWarnings go off per the default.
warnings.simplefilter("default", FileModeWarning, append=True)

View File

@@ -1,14 +0,0 @@
# .-. .-. .-. . . .-. .-. .-. .-.
# |( |- |.| | | |- `-. | `-.
# ' ' `-' `-`.`-' `-' `-' ' `-'
__title__ = "requests"
__description__ = "Python HTTP for Humans."
__url__ = "https://requests.readthedocs.io"
__version__ = "2.34.2"
__build__ = 0x023402
__author__ = "Kenneth Reitz"
__author_email__ = "me@kennethreitz.org"
__license__ = "Apache-2.0"
__copyright__ = "Copyright Kenneth Reitz"
__cake__ = "\u2728 \U0001f370 \u2728"

View File

@@ -1,188 +0,0 @@
"""
requests._types
~~~~~~~~~~~~~~~
This module contains type aliases used internally by the Requests library.
These types are not part of the public API and must not be relied upon
by external code.
"""
from __future__ import annotations
from collections.abc import Callable, Iterable, Mapping, MutableMapping, Sequence
from typing import (
TYPE_CHECKING,
Any,
Protocol,
TypeAlias,
TypeVar,
runtime_checkable,
)
_T_co = TypeVar("_T_co", covariant=True)
_KT_co = TypeVar("_KT_co", covariant=True)
_VT_co = TypeVar("_VT_co", covariant=True)
@runtime_checkable
class SupportsRead(Protocol[_T_co]):
def read(self, length: int = ..., /) -> _T_co: ...
def has_read(obj: Any) -> TypeIs[SupportsRead[str | bytes]]:
"""Check if obj supports read, including __getattr__ based proxies."""
return isinstance(obj, SupportsRead) or hasattr(obj, "read")
@runtime_checkable
class SupportsItems(Protocol[_KT_co, _VT_co]):
def items(self) -> Iterable[tuple[_KT_co, _VT_co]]: ...
# These are needed at runtime for default_hooks() return type
HookType: TypeAlias = Callable[["Response"], Any]
HooksInputType: TypeAlias = Mapping[str, Iterable[HookType] | HookType]
def is_prepared(request: PreparedRequest) -> TypeIs[_ValidatedRequest]:
"""Verify a PreparedRequest has been fully prepared."""
if TYPE_CHECKING:
return request.url is not None and request.method is not None
# noop at runtime to avoid AssertionError
return True
if TYPE_CHECKING:
from http.cookiejar import CookieJar
from typing import TypeAlias, TypedDict
from typing_extensions import (
Buffer, # TODO: move to collections.abc when Python >= 3.12
TypeIs, # TODO: move to typing when Python >= 3.13
)
from .auth import AuthBase
from .cookies import RequestsCookieJar
from .models import PreparedRequest, Response
from .structures import CaseInsensitiveDict
class _ValidatedRequest(PreparedRequest):
"""Subtype asserting a PreparedRequest has been fully prepared before calling.
The override suppression is required because mutable attribute types are
invariant (Liskov), but we only narrow after preparation is complete. This
is the explicit contract for Requests but Python's typing doesn't have a
better way to represent the requirement.
"""
url: str # type: ignore[reportIncompatibleVariableOverride]
method: str # type: ignore[reportIncompatibleVariableOverride]
# Type aliases for core API concepts (ordered by request() signature)
UriType: TypeAlias = str | bytes
_ParamsMappingKeyType: TypeAlias = str | bytes | int | float
_ParamsMappingValueType: TypeAlias = (
str | bytes | int | float | Iterable[str | bytes | int | float] | None
)
ParamsType: TypeAlias = (
SupportsItems[_ParamsMappingKeyType, _ParamsMappingValueType]
| tuple[tuple[_ParamsMappingKeyType, _ParamsMappingValueType], ...]
| Iterable[tuple[_ParamsMappingKeyType, _ParamsMappingValueType]]
| str
| bytes
| None
)
KVDataType: TypeAlias = Iterable[tuple[Any, Any]] | SupportsItems[Any, Any]
RawDataType: TypeAlias = KVDataType | str | bytes
StreamDataType: TypeAlias = SupportsRead[str | bytes]
EncodableDataType: TypeAlias = RawDataType | StreamDataType
DataType: TypeAlias = (
KVDataType
| Iterable[bytes | str]
| str
| bytes
| Buffer
| SupportsRead[str | bytes]
| None
)
BodyType: TypeAlias = (
bytes | str | Iterable[bytes | str] | SupportsRead[bytes | str] | None
)
HeadersType: TypeAlias = Mapping[str, str | bytes] | None
CookiesType: TypeAlias = RequestsCookieJar | Mapping[str, str]
# Building blocks for FilesType
_FileName: TypeAlias = str | None
_FileContent: TypeAlias = SupportsRead[str | bytes] | str | bytes
_FileSpecBasic: TypeAlias = tuple[_FileName, _FileContent]
_FileSpecWithContentType: TypeAlias = tuple[_FileName, _FileContent, str]
_FileSpecWithHeaders: TypeAlias = tuple[
_FileName, _FileContent, str, CaseInsensitiveDict[str] | Mapping[str, str]
]
_FileSpec: TypeAlias = (
_FileContent | _FileSpecBasic | _FileSpecWithContentType | _FileSpecWithHeaders
)
FilesType: TypeAlias = (
Mapping[str, _FileSpec] | Iterable[tuple[str, _FileSpec]] | None
)
AuthType: TypeAlias = (
tuple[str, str] | AuthBase | Callable[[PreparedRequest], PreparedRequest] | None
)
TimeoutType: TypeAlias = float | tuple[float | None, float | None] | None
ProxiesType: TypeAlias = MutableMapping[str, str]
HooksType: TypeAlias = dict[str, list[HookType]] | None
VerifyType: TypeAlias = bool | str
CertType: TypeAlias = str | tuple[str, str] | None
JsonType: TypeAlias = (
None
| bool
| int
| float
| str
| Sequence["JsonType"]
| Mapping[str, "JsonType"]
)
# TypedDicts for Unpack kwargs (PEP 692)
class BaseRequestKwargs(TypedDict, total=False):
headers: HeadersType
cookies: RequestsCookieJar | CookieJar | dict[str, str] | None
files: FilesType
auth: AuthType
timeout: TimeoutType
allow_redirects: bool
proxies: dict[str, str] | None
hooks: HooksInputType | None
stream: bool | None
verify: VerifyType | None
cert: CertType
class RequestKwargs(BaseRequestKwargs, total=False):
"""kwargs for request(), options(), head(), delete()."""
params: ParamsType
data: DataType
json: JsonType
class GetKwargs(BaseRequestKwargs, total=False):
data: DataType
json: JsonType
class PostKwargs(BaseRequestKwargs, total=False):
params: ParamsType
class DataKwargs(BaseRequestKwargs, total=False):
"""kwargs for put(), patch()."""
params: ParamsType
json: JsonType

View File

@@ -1,748 +0,0 @@
"""
requests.adapters
~~~~~~~~~~~~~~~~~
This module contains the transport adapters that Requests uses to define
and maintain connections.
"""
from __future__ import annotations
import os.path
import socket # noqa: F401
import typing
import warnings
from typing import Any
from urllib3.exceptions import (
ClosedPoolError,
ConnectTimeoutError,
LocationValueError,
MaxRetryError,
NewConnectionError,
ProtocolError,
ReadTimeoutError,
ResponseError,
)
from urllib3.exceptions import HTTPError as _HTTPError
from urllib3.exceptions import InvalidHeader as _InvalidHeader
from urllib3.exceptions import ProxyError as _ProxyError
from urllib3.exceptions import SSLError as _SSLError
from urllib3.poolmanager import PoolManager, proxy_from_url
from urllib3.util import Timeout as TimeoutSauce
from urllib3.util import parse_url
from urllib3.util.retry import Retry
from .auth import _basic_auth_str
from .compat import basestring, urlparse
from .cookies import extract_cookies_to_jar
from .exceptions import (
ConnectionError,
ConnectTimeout,
InvalidHeader,
InvalidProxyURL,
InvalidSchema,
InvalidURL,
ProxyError,
ReadTimeout,
RetryError,
SSLError,
)
from .models import Response
from .structures import CaseInsensitiveDict
from .utils import (
DEFAULT_CA_BUNDLE_PATH,
get_auth_from_url,
get_encoding_from_headers,
prepend_scheme_if_needed,
select_proxy,
urldefragauth,
)
try:
from urllib3.contrib.socks import SOCKSProxyManager # type: ignore[assignment]
except ImportError:
def SOCKSProxyManager(*args: Any, **kwargs: Any) -> None:
raise InvalidSchema("Missing dependencies for SOCKS support.")
if typing.TYPE_CHECKING:
from urllib3.connectionpool import HTTPConnectionPool
from urllib3.poolmanager import PoolManager as _PoolManager
from . import _types as _t
from .models import PreparedRequest
from ._types import is_prepared as _is_prepared
DEFAULT_POOLBLOCK = False
DEFAULT_POOLSIZE = 10
DEFAULT_RETRIES = 0
DEFAULT_POOL_TIMEOUT = None
def _urllib3_request_context(
request: PreparedRequest,
verify: bool | str | None,
client_cert: tuple[str, str] | str | None,
poolmanager: PoolManager,
) -> tuple[dict[str, Any], dict[str, Any]]:
host_params: dict[str, Any] = {}
pool_kwargs: dict[str, Any] = {}
parsed_request_url = urlparse(request.url)
scheme = parsed_request_url.scheme.lower()
port = parsed_request_url.port
cert_reqs = "CERT_REQUIRED"
if verify is False:
cert_reqs = "CERT_NONE"
elif isinstance(verify, str):
if not os.path.isdir(verify):
pool_kwargs["ca_certs"] = verify
else:
pool_kwargs["ca_cert_dir"] = verify
pool_kwargs["cert_reqs"] = cert_reqs
if client_cert is not None:
if isinstance(client_cert, tuple) and len(client_cert) == 2:
pool_kwargs["cert_file"] = client_cert[0]
pool_kwargs["key_file"] = client_cert[1]
else:
# According to our docs, we allow users to specify just the client
# cert path
pool_kwargs["cert_file"] = client_cert
host_params = {
"scheme": scheme,
"host": parsed_request_url.hostname,
"port": port,
}
return host_params, pool_kwargs
class BaseAdapter:
"""The Base Transport Adapter"""
def __init__(self) -> None:
super().__init__()
def send(
self,
request: PreparedRequest,
stream: bool = False,
timeout: _t.TimeoutType = None,
verify: _t.VerifyType = True,
cert: _t.CertType = None,
proxies: dict[str, str] | None = None,
) -> Response:
"""Sends PreparedRequest object. Returns Response object.
:param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
:param stream: (optional) Whether to stream the request content.
:param timeout: (optional) How long to wait for the server to send
data before giving up, as a float, or a :ref:`(connect timeout,
read timeout) <timeouts>` tuple.
:type timeout: float or tuple
:param verify: (optional) Either a boolean, in which case it controls whether we verify
the server's TLS certificate, or a string, in which case it must be a path
to a CA bundle to use
:param cert: (optional) Any user-provided SSL certificate to be trusted.
:param proxies: (optional) The proxies dictionary to apply to the request.
"""
raise NotImplementedError
def close(self) -> None:
"""Cleans up adapter specific items."""
raise NotImplementedError
class HTTPAdapter(BaseAdapter):
"""The built-in HTTP Adapter for urllib3.
Provides a general-case interface for Requests sessions to contact HTTP and
HTTPS urls by implementing the Transport Adapter interface. This class will
usually be created by the :class:`Session <Session>` class under the
covers.
:param pool_connections: The number of urllib3 connection pools to cache.
:param pool_maxsize: The maximum number of connections to save in the pool.
:param max_retries: The maximum number of retries each connection
should attempt. Note, this applies only to failed DNS lookups, socket
connections and connection timeouts, never to requests where data has
made it to the server. By default, Requests does not retry failed
connections. If you need granular control over the conditions under
which we retry a request, import urllib3's ``Retry`` class and pass
that instead.
:param pool_block: Whether the connection pool should block for connections.
Usage::
>>> import requests
>>> s = requests.Session()
>>> a = requests.adapters.HTTPAdapter(max_retries=3)
>>> s.mount('http://', a)
"""
__attrs__: list[str] = [
"max_retries",
"config",
"_pool_connections",
"_pool_maxsize",
"_pool_block",
]
max_retries: Retry
config: dict[str, Any]
proxy_manager: dict[str, Any]
_pool_connections: int
_pool_maxsize: int
_pool_block: bool
poolmanager: _PoolManager
def __init__(
self,
pool_connections: int = DEFAULT_POOLSIZE,
pool_maxsize: int = DEFAULT_POOLSIZE,
max_retries: int | Retry = DEFAULT_RETRIES,
pool_block: bool = DEFAULT_POOLBLOCK,
) -> None:
if max_retries == DEFAULT_RETRIES:
self.max_retries = Retry(0, read=False)
else:
self.max_retries = Retry.from_int(max_retries)
self.config = {}
self.proxy_manager = {}
super().__init__()
self._pool_connections = pool_connections
self._pool_maxsize = pool_maxsize
self._pool_block = pool_block
self.init_poolmanager(pool_connections, pool_maxsize, block=pool_block)
def __getstate__(self) -> dict[str, Any]:
return {attr: getattr(self, attr, None) for attr in self.__attrs__}
def __setstate__(self, state: dict[str, Any]) -> None:
# Can't handle by adding 'proxy_manager' to self.__attrs__ because
# self.poolmanager uses a lambda function, which isn't pickleable.
self.proxy_manager = {}
self.config = {}
for attr, value in state.items():
setattr(self, attr, value)
self.init_poolmanager(
self._pool_connections, self._pool_maxsize, block=self._pool_block
)
def init_poolmanager(
self,
connections: int,
maxsize: int,
block: bool = DEFAULT_POOLBLOCK,
**pool_kwargs: Any,
) -> None:
"""Initializes a urllib3 PoolManager.
This method should not be called from user code, and is only
exposed for use when subclassing the
:class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
:param connections: The number of urllib3 connection pools to cache.
:param maxsize: The maximum number of connections to save in the pool.
:param block: Block when no free connections are available.
:param pool_kwargs: Extra keyword arguments used to initialize the Pool Manager.
"""
# save these values for pickling
self._pool_connections = connections
self._pool_maxsize = maxsize
self._pool_block = block
self.poolmanager = PoolManager(
num_pools=connections,
maxsize=maxsize,
block=block,
**pool_kwargs,
)
def proxy_manager_for(self, proxy: str, **proxy_kwargs: Any) -> Any:
"""Return urllib3 ProxyManager for the given proxy.
This method should not be called from user code, and is only
exposed for use when subclassing the
:class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
:param proxy: The proxy to return a urllib3 ProxyManager for.
:param proxy_kwargs: Extra keyword arguments used to configure the Proxy Manager.
:returns: ProxyManager
:rtype: urllib3.ProxyManager
"""
if proxy in self.proxy_manager:
manager = self.proxy_manager[proxy]
elif proxy.lower().startswith("socks"):
username, password = get_auth_from_url(proxy)
manager = self.proxy_manager[proxy] = SOCKSProxyManager(
proxy,
username=username,
password=password,
num_pools=self._pool_connections,
maxsize=self._pool_maxsize,
block=self._pool_block,
**proxy_kwargs,
)
else:
proxy_headers = self.proxy_headers(proxy)
manager = self.proxy_manager[proxy] = proxy_from_url(
proxy,
proxy_headers=proxy_headers,
num_pools=self._pool_connections,
maxsize=self._pool_maxsize,
block=self._pool_block,
**proxy_kwargs,
)
return manager
def cert_verify(
self, conn: Any, url: str, verify: _t.VerifyType, cert: _t.CertType
) -> None:
"""Verify a SSL certificate. This method should not be called from user
code, and is only exposed for use when subclassing the
:class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
:param conn: The urllib3 connection object associated with the cert.
:param url: The requested URL.
:param verify: Either a boolean, in which case it controls whether we verify
the server's TLS certificate, or a string, in which case it must be a path
to a CA bundle to use
:param cert: The SSL certificate to verify.
"""
if url.lower().startswith("https") and verify:
cert_loc = None
# Allow self-specified cert location.
if verify is not True:
cert_loc = verify
if not cert_loc:
cert_loc = DEFAULT_CA_BUNDLE_PATH
if not cert_loc or not os.path.exists(cert_loc):
raise OSError(
f"Could not find a suitable TLS CA certificate bundle, "
f"invalid path: {cert_loc}"
)
conn.cert_reqs = "CERT_REQUIRED"
if not os.path.isdir(cert_loc):
conn.ca_certs = cert_loc
else:
conn.ca_cert_dir = cert_loc
else:
conn.cert_reqs = "CERT_NONE"
conn.ca_certs = None
conn.ca_cert_dir = None
if cert:
if not isinstance(cert, basestring):
conn.cert_file = cert[0]
conn.key_file = cert[1]
else:
conn.cert_file = cert
conn.key_file = None
if conn.cert_file and not os.path.exists(conn.cert_file):
raise OSError(
f"Could not find the TLS certificate file, "
f"invalid path: {conn.cert_file}"
)
if conn.key_file and not os.path.exists(conn.key_file):
raise OSError(
f"Could not find the TLS key file, invalid path: {conn.key_file}"
)
def build_response(self, req: PreparedRequest, resp: Any) -> Response:
"""Builds a :class:`Response <requests.Response>` object from a urllib3
response. This should not be called from user code, and is only exposed
for use when subclassing the
:class:`HTTPAdapter <requests.adapters.HTTPAdapter>`
:param req: The :class:`PreparedRequest <PreparedRequest>` used to generate the response.
:param resp: The urllib3 response object.
:rtype: requests.Response
"""
assert _is_prepared(req)
response = Response()
# Fallback to None if there's no status_code, for whatever reason.
response.status_code = getattr(resp, "status", None) # type: ignore[assignment]
# Make headers case-insensitive.
response.headers = CaseInsensitiveDict(getattr(resp, "headers", {}))
# Set encoding.
response.encoding = get_encoding_from_headers(response.headers)
response.raw = resp
response.reason = response.raw.reason
if isinstance(req.url, bytes):
response.url = req.url.decode("utf-8")
else:
response.url = req.url
# Add new cookies from the server.
extract_cookies_to_jar(response.cookies, req, resp)
# Give the Response some context.
response.request = req
response.connection = self
return response
def build_connection_pool_key_attributes(
self, request: PreparedRequest, verify: _t.VerifyType, cert: _t.CertType = None
) -> tuple[dict[str, Any], dict[str, Any]]:
"""Build the PoolKey attributes used by urllib3 to return a connection.
This looks at the PreparedRequest, the user-specified verify value,
and the value of the cert parameter to determine what PoolKey values
to use to select a connection from a given urllib3 Connection Pool.
The SSL related pool key arguments are not consistently set. As of
this writing, use the following to determine what keys may be in that
dictionary:
* If ``verify`` is ``True``, ``"ssl_context"`` will be set and will be the
default Requests SSL Context
* If ``verify`` is ``False``, ``"ssl_context"`` will not be set but
``"cert_reqs"`` will be set
* If ``verify`` is a string, (i.e., it is a user-specified trust bundle)
``"ca_certs"`` will be set if the string is not a directory recognized
by :py:func:`os.path.isdir`, otherwise ``"ca_cert_dir"`` will be
set.
* If ``"cert"`` is specified, ``"cert_file"`` will always be set. If
``"cert"`` is a tuple with a second item, ``"key_file"`` will also
be present
To override these settings, one may subclass this class, call this
method and use the above logic to change parameters as desired. For
example, if one wishes to use a custom :py:class:`ssl.SSLContext` one
must both set ``"ssl_context"`` and based on what else they require,
alter the other keys to ensure the desired behaviour.
:param request:
The PreparedRequest being sent over the connection.
:type request:
:class:`~requests.models.PreparedRequest`
:param verify:
Either a boolean, in which case it controls whether
we verify the server's TLS certificate, or a string, in which case it
must be a path to a CA bundle to use.
:param cert:
(optional) Any user-provided SSL certificate for client
authentication (a.k.a., mTLS). This may be a string (i.e., just
the path to a file which holds both certificate and key) or a
tuple of length 2 with the certificate file path and key file
path.
:returns:
A tuple of two dictionaries. The first is the "host parameters"
portion of the Pool Key including scheme, hostname, and port. The
second is a dictionary of SSLContext related parameters.
"""
return _urllib3_request_context(request, verify, cert, self.poolmanager)
def get_connection_with_tls_context(
self,
request: PreparedRequest,
verify: _t.VerifyType,
proxies: dict[str, str] | None = None,
cert: _t.CertType = None,
) -> HTTPConnectionPool:
"""Returns a urllib3 connection for the given request and TLS settings.
This should not be called from user code, and is only exposed for use
when subclassing the :class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
:param request:
The :class:`PreparedRequest <PreparedRequest>` object to be sent
over the connection.
:param verify:
Either a boolean, in which case it controls whether we verify the
server's TLS certificate, or a string, in which case it must be a
path to a CA bundle to use.
:param proxies:
(optional) The proxies dictionary to apply to the request.
:param cert:
(optional) Any user-provided SSL certificate to be used for client
authentication (a.k.a., mTLS).
:rtype:
urllib3.HTTPConnectionPool
"""
assert _is_prepared(request)
proxy = select_proxy(request.url, proxies)
try:
host_params, pool_kwargs = self.build_connection_pool_key_attributes(
request,
verify,
cert,
)
except ValueError as e:
raise InvalidURL(e, request=request)
if proxy:
proxy = prepend_scheme_if_needed(proxy, "http")
proxy_url = parse_url(proxy)
if not proxy_url.host:
raise InvalidProxyURL(
"Please check proxy URL. It is malformed "
"and could be missing the host."
)
proxy_manager = self.proxy_manager_for(proxy)
conn = proxy_manager.connection_from_host(
**host_params, pool_kwargs=pool_kwargs
)
else:
# Only scheme should be lower case
conn = self.poolmanager.connection_from_host(
**host_params, pool_kwargs=pool_kwargs
)
return conn
def get_connection(
self, url: str, proxies: dict[str, str] | None = None
) -> HTTPConnectionPool:
"""DEPRECATED: Users should move to `get_connection_with_tls_context`
for all subclasses of HTTPAdapter using Requests>=2.32.2.
Returns a urllib3 connection for the given URL. This should not be
called from user code, and is only exposed for use when subclassing the
:class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
:param url: The URL to connect to.
:param proxies: (optional) A Requests-style dictionary of proxies used on this request.
:rtype: urllib3.HTTPConnectionPool
"""
warnings.warn(
(
"`get_connection` has been deprecated in favor of "
"`get_connection_with_tls_context`. Custom HTTPAdapter subclasses "
"will need to migrate for Requests>=2.32.2. Please see "
"https://github.com/psf/requests/pull/6710 for more details."
),
DeprecationWarning,
)
proxy = select_proxy(url, proxies)
if proxy:
proxy = prepend_scheme_if_needed(proxy, "http")
proxy_url = parse_url(proxy)
if not proxy_url.host:
raise InvalidProxyURL(
"Please check proxy URL. It is malformed "
"and could be missing the host."
)
proxy_manager = self.proxy_manager_for(proxy)
conn = proxy_manager.connection_from_url(url)
else:
# Only scheme should be lower case
parsed = urlparse(url)
url = parsed.geturl()
conn = self.poolmanager.connection_from_url(url)
return conn
def close(self) -> None:
"""Disposes of any internal state.
Currently, this closes the PoolManager and any active ProxyManager,
which closes any pooled connections.
"""
self.poolmanager.clear()
for proxy in self.proxy_manager.values():
proxy.clear()
def request_url(
self, request: PreparedRequest, proxies: dict[str, str] | None
) -> str:
"""Obtain the url to use when making the final request.
If the message is being sent through a HTTP proxy, the full URL has to
be used. Otherwise, we should only use the path portion of the URL.
This should not be called from user code, and is only exposed for use
when subclassing the
:class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
:param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
:param proxies: A dictionary of schemes or schemes and hosts to proxy URLs.
:rtype: str
"""
assert _is_prepared(request)
proxy = select_proxy(request.url, proxies)
scheme = urlparse(request.url).scheme
is_proxied_http_request = proxy and scheme != "https"
using_socks_proxy = False
if proxy:
proxy_scheme = urlparse(proxy).scheme.lower()
using_socks_proxy = proxy_scheme.startswith("socks")
url = request.path_url
if is_proxied_http_request and not using_socks_proxy:
url = urldefragauth(request.url)
return url
def add_headers(self, request: PreparedRequest, **kwargs: Any) -> None:
"""Add any headers needed by the connection. As of v2.0 this does
nothing by default, but is left for overriding by users that subclass
the :class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
This should not be called from user code, and is only exposed for use
when subclassing the
:class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
:param request: The :class:`PreparedRequest <PreparedRequest>` to add headers to.
:param kwargs: The keyword arguments from the call to send().
"""
pass
def proxy_headers(self, proxy: str) -> dict[str, str]:
"""Returns a dictionary of the headers to add to any request sent
through a proxy. This works with urllib3 magic to ensure that they are
correctly sent to the proxy, rather than in a tunnelled request if
CONNECT is being used.
This should not be called from user code, and is only exposed for use
when subclassing the
:class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
:param proxy: The url of the proxy being used for this request.
:rtype: dict
"""
headers: dict[str, str] = {}
username, password = get_auth_from_url(proxy)
if username:
headers["Proxy-Authorization"] = _basic_auth_str(username, password)
return headers
def send(
self,
request: PreparedRequest,
stream: bool = False,
timeout: _t.TimeoutType = None,
verify: _t.VerifyType = True,
cert: _t.CertType = None,
proxies: dict[str, str] | None = None,
) -> Response:
"""Sends PreparedRequest object. Returns Response object.
:param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
:param stream: (optional) Whether to stream the request content.
:param timeout: (optional) How long to wait for the server to send
data before giving up, as a float, or a :ref:`(connect timeout,
read timeout) <timeouts>` tuple.
:type timeout: float or tuple or urllib3 Timeout object
:param verify: (optional) Either a boolean, in which case it controls whether
we verify the server's TLS certificate, or a string, in which case it
must be a path to a CA bundle to use
:param cert: (optional) Any user-provided SSL certificate to be trusted.
:param proxies: (optional) The proxies dictionary to apply to the request.
:rtype: requests.Response
"""
assert _is_prepared(request)
try:
conn = self.get_connection_with_tls_context(
request, verify, proxies=proxies, cert=cert
)
except LocationValueError as e:
raise InvalidURL(e, request=request)
self.cert_verify(conn, request.url, verify, cert)
url = self.request_url(request, proxies)
self.add_headers(
request,
stream=stream,
timeout=timeout,
verify=verify,
cert=cert,
proxies=proxies,
)
chunked = not (request.body is None or "Content-Length" in request.headers)
if isinstance(timeout, tuple):
try:
connect, read = timeout
resolved_timeout = TimeoutSauce(connect=connect, read=read)
except ValueError:
raise ValueError(
f"Invalid timeout {timeout}. Pass a (connect, read) timeout tuple, "
f"or a single float to set both timeouts to the same value."
)
elif isinstance(timeout, TimeoutSauce):
resolved_timeout = timeout
else:
resolved_timeout = TimeoutSauce(connect=timeout, read=timeout)
try:
resp = conn.urlopen(
method=request.method,
url=url,
body=request.body, # type: ignore[arg-type] # urllib3 stubs don't accept Iterable[bytes | str]
headers=request.headers, # type: ignore[arg-type] # urllib3#3072
redirect=False,
assert_same_host=False,
preload_content=False,
decode_content=False,
retries=self.max_retries,
timeout=resolved_timeout,
chunked=chunked,
)
except (ProtocolError, OSError) as err:
raise ConnectionError(err, request=request)
except MaxRetryError as e:
if isinstance(e.reason, ConnectTimeoutError):
# TODO: Remove this in 3.0.0: see #2811
if not isinstance(e.reason, NewConnectionError):
raise ConnectTimeout(e, request=request)
if isinstance(e.reason, ResponseError):
raise RetryError(e, request=request)
if isinstance(e.reason, _ProxyError):
raise ProxyError(e, request=request)
if isinstance(e.reason, _SSLError):
# This branch is for urllib3 v1.22 and later.
raise SSLError(e, request=request)
raise ConnectionError(e, request=request)
except ClosedPoolError as e:
raise ConnectionError(e, request=request)
except _ProxyError as e:
raise ProxyError(e)
except (_SSLError, _HTTPError) as e:
if isinstance(e, _SSLError):
# This branch is for urllib3 versions earlier than v1.22
raise SSLError(e, request=request)
elif isinstance(e, ReadTimeoutError):
raise ReadTimeout(e, request=request)
elif isinstance(e, _InvalidHeader):
raise InvalidHeader(e, request=request)
else:
raise
return self.build_response(request, resp)

Some files were not shown because too many files have changed in this diff Show More