Extensions for Python HTTP libraries that send custom proxy headers on HTTPS CONNECT and read proxy response headers back. Works with any proxy provider; works best with ProxyMesh.
When making HTTPS requests through a proxy, the connection is established via a CONNECT tunnel. During this process:
-
Sending proxy headers - Most Python HTTP libraries don't provide an easy way to send custom proxy headers (like
X-ProxyMesh-Country) to the proxy server during the CONNECT handshake. -
Reading proxy response headers - The proxy's CONNECT response headers are typically discarded, making it impossible to read proxy response headers (like
X-ProxyMesh-IP) that the proxy sends back.
This library solves both problems for popular Python HTTP libraries.
| Library | Module | Use Case |
|---|---|---|
| urllib3 | urllib3_proxy_manager |
Low-level HTTP client |
| requests | requests_adapter |
Simple HTTP requests |
| aiohttp | aiohttp_proxy |
Async HTTP client |
| httpx | httpx_proxy |
Modern HTTP client |
| pycurl | pycurl_proxy |
libcurl bindings |
| cloudscraper | cloudscraper_proxy |
Cloudflare bypass |
| autoscraper | autoscraper_proxy |
Automatic web scraping |
pip install python-proxy-headersOr install the client extra you need. Each extra pins a range that CI tests:
pip install 'python-proxy-headers[requests]'
pip install 'python-proxy-headers[urllib3]'
pip install 'python-proxy-headers[httpx]'
pip install 'python-proxy-headers[aiohttp]'
pip install 'python-proxy-headers[pycurl]'
pip install 'python-proxy-headers[cloudscraper]'
pip install 'python-proxy-headers[autoscraper]'The httpx extra is httpx>=0.28,<1. Importing python_proxy_headers.httpx_proxy on httpx 1.x raises ImportError. See the httpx docs. Proxy header names are specific to your provider; ProxyMesh documents theirs at docs.proxymesh.com.
Note: This package has no required dependencies. Install only the extra you need.
from python_proxy_headers.requests_adapter import ProxySession
with ProxySession(proxy_headers={'X-ProxyMesh-Country': 'US'}) as session:
session.proxies = {'https': 'http://user:pass@proxy.example.com:8080'}
response = session.get('https://httpbin.org/ip')
# Proxy response headers are merged into response.headers
print(response.headers.get('X-ProxyMesh-IP'))from python_proxy_headers.httpx_proxy import get
response = get(
'https://httpbin.org/ip',
proxy='http://user:pass@proxy.example.com:8080'
)
# Proxy CONNECT response headers are merged into response.headers
print(response.headers.get('X-ProxyMesh-IP'))import asyncio
from python_proxy_headers.aiohttp_proxy import ProxyClientSession
async def main():
async with ProxyClientSession() as session:
async with session.get(
'https://httpbin.org/ip',
proxy='http://user:pass@proxy.example.com:8080'
) as response:
# Proxy response headers merged into response.headers
print(response.headers.get('X-ProxyMesh-IP'))
asyncio.run(main())import pycurl
from python_proxy_headers.pycurl_proxy import set_proxy_headers, HeaderCapture
c = pycurl.Curl()
c.setopt(pycurl.URL, 'https://httpbin.org/ip')
c.setopt(pycurl.PROXY, 'http://proxy.example.com:8080')
# Add these two lines to any existing pycurl code
set_proxy_headers(c, {'X-ProxyMesh-Country': 'US'})
capture = HeaderCapture(c)
c.perform()
print(capture.proxy_headers) # Proxy response headers from the CONNECT response
c.close()from python_proxy_headers.cloudscraper_proxy import create_scraper
# Drop-in replacement for cloudscraper.create_scraper()
scraper = create_scraper(proxy_headers={'X-ProxyMesh-Country': 'US'})
scraper.proxies = {'https': 'http://proxy.example.com:8080'}
response = scraper.get('https://example.com')
# All CloudScraper features (Cloudflare bypass) preservedA test harness is included to verify proxy header functionality:
# Header checks and private-hook checks (no proxy credentials)
python test_header_security.py
python test_httpx_guard.py
python test_client_internals.py
# Set your proxy
export PROXY_URL='http://user:pass@proxy.example.com:8080'
# Test all modules
python test_proxy_headers.py
# Test specific modules
python test_proxy_headers.py requests httpx
# Verbose output (show header values)
python test_proxy_headers.py -vFor detailed documentation, API reference, and more examples:
- Full Documentation: python-proxy-headers.readthedocs.io
- Example Code: proxy-examples for Python
- javascript-proxy-headers - Send and read custom proxy headers on HTTPS CONNECT for Node.js HTTP clients
- ruby-proxy-headers - Send and read custom proxy headers on HTTPS CONNECT for Ruby HTTP clients
- scrapy-proxy-headers - Proxy header support for Scrapy
ProxyMesh created this library to help customers use custom headers to control proxy behavior, as documented at docs.proxymesh.com. It works with any proxy that supports custom headers on CONNECT.
MIT License