Authenticating aiohttp requests against an HTTP or SOCKS5 proxy
By the ProxyV4 Engineering Team · Updated:
aiohttp proxy authentication works by passing proxy="http://USER:PASS@IP:PORT" to a request or to ClientSession; aiohttp turns the user and password into a Proxy-Authorization header for you. The older proxy_auth=aiohttp.BasicAuth(...) argument still runs, but aiohttp 3.14 deprecated it, so new code should use the URL or an explicit header. SOCKS5 is not built in and needs the aiohttp-socks package.
Three places credentials can live
aiohttp gives you three ways to hand it a proxy login. They produce the same Proxy-Authorization: Basic ... header on the wire, so the choice is about readability, logging and which aiohttp versions you support.
| Method | Example | Status in 3.14 |
|---|---|---|
| Userinfo in the proxy URL | proxy="http://USER:PASS@IP:PORT" | Supported, shortest form |
| Explicit header | proxy_headers={"Proxy-Authorization": encode_basic_auth(u, p)} | Recommended replacement for proxy_auth |
proxy_auth=aiohttp.BasicAuth(u, p) | proxy_auth=BasicAuth("USER", "PASS") | Deprecated, removal planned for 4.0 |
The URL form, step by step
For most scripts the URL form is enough. Pass it per request with proxy= and print the exit IP to confirm traffic leaves through the proxy:
import asyncio
import aiohttp
PROXY = "http://USER:[email protected]:8080"
async def main():
timeout = aiohttp.ClientTimeout(total=20)
async with aiohttp.ClientSession(timeout=timeout) as session:
async with session.get("https://proxyv4.com/api/my-ip", proxy=PROXY) as r:
print(r.status, await r.json())
asyncio.run(main())Keep the proxy scheme as http:// even for https:// targets. aiohttp issues a CONNECT to the proxy and wraps the website's TLS inside the tunnel. If the password contains @, : or /, percent-encode it with urllib.parse.quote(password, safe='') before building the string, or switch to the header form, which never parses a URL.
Replacing aiohttp BasicAuth proxy_auth after 3.14
aiohttp 3.14 deprecated the BasicAuth class together with the auth and proxy_auth parameters, scheduling their removal for 4.0. Existing code keeps working but emits a DeprecationWarning, which fails test suites configured with -W error. The replacement is the encode_basic_auth() helper, which returns a ready-made Basic <base64> string, sent through proxy_headers.
Before and after
# aiohttp < 3.14 style (deprecated)
await session.get(url, proxy="http://203.0.113.10:8080",
proxy_auth=aiohttp.BasicAuth("USER", "PASS"))
# aiohttp 3.14+ style
from aiohttp import encode_basic_auth
await session.get(url, proxy="http://203.0.113.10:8080",
proxy_headers={"Proxy-Authorization": encode_basic_auth("USER", "PASS")})encode_basic_auth() raises ValueError if the login contains a colon, as the Basic scheme forbids it. It also encodes with UTF-8 by default, while BasicAuth defaulted to Latin-1, so a password with non-ASCII characters can change bytes during the migration. Pass encoding="latin1" if your proxy expected the old behavior.
Supporting older aiohttp versions
If a library you maintain must run on both sides of 3.14, the URL form is the portable choice: every 3.x release accepts credentials in the proxy URL. Alternatively, build the header yourself with base64.b64encode(f"{user}:{pw}".encode()).decode() and prefix it with Basic , which works on any version.
aiohttp proxy per request versus per session
Because proxy= is a per-call argument, one ClientSession can send each request through a different IP. That is the cheapest way to spread an async crawl across several static addresses: one connection pool, many exits.
import asyncio
import itertools
import aiohttp
PROXIES = [
"http://USER:[email protected]:8080",
"http://USER:[email protected]:8080",
"http://USER:[email protected]:8080",
]
async def fetch(session, url, proxy, sem):
async with sem:
async with session.get(url, proxy=proxy) as r:
return r.status
async def main(urls):
sem = asyncio.Semaphore(20)
cycle = itertools.cycle(PROXIES)
async with aiohttp.ClientSession(timeout=aiohttp.ClientTimeout(total=30)) as s:
results = await asyncio.gather(
*(fetch(s, u, next(cycle), sem) for u in urls), return_exceptions=True)
print(results)
asyncio.run(main(["https://proxyv4.com/api/my-ip"] * 6))Recent 3.x releases also accept proxy= and proxy_headers= on ClientSession itself as defaults. A value passed on an individual request overrides the session default, so you can set a main proxy for the session and route a few calls elsewhere.
trust_env, HTTP_PROXY and .netrc
Unlike requests, aiohttp ignores proxy environment variables unless you opt in with aiohttp.ClientSession(trust_env=True). With that flag set, it reads HTTP_PROXY, HTTPS_PROXY, WS_PROXY and WSS_PROXY plus NO_PROXY, strips any credentials out of the URL and sends them as a header. If the proxy URL carries no credentials, aiohttp looks for a matching machine entry in ~/.netrc.
Two details catch people. An explicit proxy= on the request wins over the environment, and environment proxies whose URL starts with https:// are skipped with a warning, so write HTTPS_PROXY=http://USER:PASS@IP:PORT with an http:// scheme.
aiohttp SOCKS5 proxy with aiohttp-socks
aiohttp only speaks HTTP proxies natively. For SOCKS5, install aiohttp-socks and give the session a ProxyConnector. The connector replaces the default TCPConnector, so the SOCKS proxy applies to the whole session and you must not also pass proxy= on requests.
pip install aiohttp-socksimport asyncio
import aiohttp
from aiohttp_socks import ProxyConnector
async def main():
connector = ProxyConnector.from_url(
"socks5://USER:[email protected]:1080", rdns=True)
async with aiohttp.ClientSession(connector=connector) as session:
async with session.get("https://proxyv4.com/api/my-ip") as r:
print(await r.json())
asyncio.run(main())rdns=True asks the proxy to resolve hostnames, so DNS lookups for target sites do not leave from your own network. Because the connector is per session, routing through several SOCKS5 IPs means one session per IP. If your proxies accept both protocols on the same port, the HTTP form with per-request proxy= is simpler to scale.
Sizing a pool of static IPs for an async crawler
An async crawler can open hundreds of connections from one process, and a single IP carrying all of them looks very different to a target than a handful of IPs carrying a few dozen each. The practical model is a fixed pool of addresses, a per-IP concurrency cap and a polite request rate, adjusted after a test run. The web scraping proxy page walks through that sizing.
ProxyV4 sells private datacenter IPv4 in seven countries and static ISP IPs in the United States and Vietnam, priced per IP per month with discounts that grow with quantity, so adding IPs to a pool is a linear cost you can see on the pricing page. Each IP has one tenant for the term, traffic is unmetered, and every proxy accepts username and password over HTTP(S) and SOCKS5. Renewing before expiry keeps the same addresses, which keeps your pool stable between runs.
It is not the right fit if your design depends on a fresh IP per request: rotating proxies are not on sale, and there are no per-GB or mobile plans. If an IP fails for a technical reason, such as not connecting or showing the wrong location, the refund policy credits your balance.
Exceptions aiohttp raises when the proxy fails
aiohttp's exception classes map cleanly onto the stage that failed. Catch them separately so your retry logic can tell a dead IP from a bad password.
ClientHttpProxyError: 407
For https:// targets a rejected login arrives during CONNECT as aiohttp.ClientHttpProxyError: 407, message='Proxy Authentication Required'. For plain http:// targets no tunnel is involved, so you get a response with r.status == 407. Check the credentials, the encoding and whether the header actually reached the request; the 407 troubleshooting page covers the rest.
ClientProxyConnectionError
Cannot connect to host 203.0.113.10:8080 ssl:default [Connect call failed ...] means aiohttp never reached the proxy. Verify the IP and port, remember that switching an order between HTTP and SOCKS5 modes can move the port, and test outbound access from the machine running the crawler.
Errors from aiohttp-socks
With ProxyConnector, failures come from the python-socks layer as ProxyError or ProxyConnectionError, often with a message about authentication or a general SOCKS server failure. A SOCKS URL pointed at an HTTP-only port also lands here, so confirm the port speaks SOCKS5.
Frequently asked questions
Is proxy_auth removed in aiohttp?
Not yet. It is deprecated since 3.14 and scheduled for removal in 4.0, so it still works with a DeprecationWarning. Move to credentials in the URL or proxy_headers with encode_basic_auth().
Can aiohttp use an HTTPS proxy URL?
Recent versions can speak TLS to a proxy that offers it, but most proxy ports, including standard authenticated ones, expect a plain http:// connection with CONNECT for HTTPS targets. Use http:// unless your provider says the port is TLS.
How do I check which IP aiohttp is using?
Request an IP echo endpoint through the proxy and compare the result with the proxy line, or open the my IP tool in a browser configured with the same proxy.
Does aiohttp-socks support username and password?
Yes. Put them in the URL passed to ProxyConnector.from_url, or pass username and password when constructing ProxyConnector directly.
Will my crawler keep the same IPs next month?
With ProxyV4, yes as long as you renew before expiry. You can renew all proxies in an order or a selection, and the addresses stay the same.
Stable IPs for an async crawler, priced per IP
Build your aiohttp pool from ProxyV4 private IPv4 or static ISP proxies, each with HTTP and SOCKS5 login and unmetered traffic. Order a single IP first, run your crawler through it, then add more.
Compare plans