How to Test a Proxy List in Bulk with Python
By the ProxyV4 Engineering Team · Updated:
To test proxies in bulk with Python, send one request through each proxy to an IP echo endpoint, run many of those checks at the same time with a fixed concurrency limit, and give every request a timeout so dead proxies fail fast. Then compare the IP the endpoint reports with the proxy's own IP. Below are two complete scripts: one with requests and a thread pool, and one with asyncio and aiohttp for larger lists.
How to test proxies in bulk with Python
Checking proxies one after another is slow because a dead proxy usually does not refuse the connection; it just never answers, and your script waits for the full timeout. A list with many dead entries can take far longer than its working entries would suggest. Running checks in parallel means those waits overlap.
A useful bulk check answers three questions for each line: does the proxy accept the connection and the credentials, does the request reach the internet through it, and is the exit IP the one you expect. The scripts below answer all three and report how long each successful check took, so slow proxies stand out too.
- Bounded concurrency. A thread pool size or an
asyncio.Semaphorecaps how many checks run at once, so you do not run out of sockets or flood the echo endpoint. - Timeouts on every request. A connect timeout and a read or total timeout turn hanging proxies into clear failures.
- Exit IP comparison. A 200 response alone does not prove the request left through the proxy you think it did.
Parse ip:port:user:pass lines safely
Most providers, including ProxyV4, deliver proxies as one line in the format ip:port:user:pass. Split it with a maximum of three splits, line.split(':', 3), so that a colon inside the password stays part of the password instead of breaking the line into five fields. When you build a proxy URL, percent-encode the username and password, because characters such as @, : or / would otherwise be read as part of the URL syntax and cause 407 errors.
from urllib.parse import quote
def parse_line(line):
"""Turn 'ip:port:user:pass' into (ip, proxy_url). Returns None for blank or comment lines."""
line = line.strip()
if not line or line.startswith("#"):
return None
ip, port, user, password = line.split(":", 3) # maxsplit=3 keeps any ':' inside the password
return ip, f"http://{quote(user, safe='')}:{quote(password, safe='')}@{ip}:{port}"If your list comes in another layout, such as user:pass@ip:port, convert it once with the free proxy format converter rather than writing a parser for every format.
Bulk proxy checker with requests and a thread pool
For lists up to a few hundred lines, threads are the simplest option, and requests is already installed in most environments. ThreadPoolExecutor(max_workers=WORKERS) is the concurrency limit: no more than that many checks run at once. Save your proxies one per line in proxies.txt and run python check_proxies.py proxies.txt.
# pip install requests
import sys
import time
from concurrent.futures import ThreadPoolExecutor, as_completed
from urllib.parse import quote
import requests
CHECK_URL = "https://proxyv4.com/api/my-ip" # returns JSON with the caller's "ip"
WORKERS = 20 # max proxies tested at the same time
TIMEOUT = (5, 10) # (connect, read) seconds
def parse_line(line):
line = line.strip()
if not line or line.startswith("#"):
return None
ip, port, user, password = line.split(":", 3)
return ip, f"http://{quote(user, safe='')}:{quote(password, safe='')}@{ip}:{port}"
def check(ip, proxy_url):
start = time.monotonic()
try:
r = requests.get(CHECK_URL, proxies={"http": proxy_url, "https": proxy_url}, timeout=TIMEOUT)
if r.status_code != 200:
return ip, False, f"HTTP {r.status_code}"
exit_ip = r.json().get("ip")
ms = round((time.monotonic() - start) * 1000)
if exit_ip != ip:
return ip, False, f"exit IP {exit_ip} differs"
return ip, True, f"{ms} ms"
except requests.exceptions.ProxyError as exc:
# requests wraps both a dead proxy and a rejected login in ProxyError; read the message
msg = str(exc)
if "407" in msg:
return ip, False, "407 auth rejected"
if "Timeout" in msg or "timed out" in msg:
return ip, False, "timeout (proxy did not answer)"
return ip, False, "proxy unreachable"
except requests.exceptions.Timeout:
return ip, False, "timeout"
except (requests.exceptions.RequestException, ValueError) as exc:
return ip, False, f"{exc.__class__.__name__}"
def main(path):
with open(path) as f:
proxies = [p for p in (parse_line(l) for l in f) if p]
with ThreadPoolExecutor(max_workers=WORKERS) as pool:
futures = [pool.submit(check, ip, url) for ip, url in proxies]
for fut in as_completed(futures):
ip, ok, info = fut.result()
print(f"{'OK ' if ok else 'FAIL'} {ip} {info}")
if __name__ == "__main__":
main(sys.argv[1] if len(sys.argv) > 1 else "proxies.txt")The checker calls https://proxyv4.com/api/my-ip, which returns JSON with the caller's ip and country code, and treats the proxy as working only if that IP matches the proxy's IP. If you test a provider whose exit IP differs from the entry IP by design (a gateway that rotates, for example), drop that comparison and just record the exit IP. More about the requests proxy settings is in our setup guide.
Reading the failure reasons
ProxyError with a 407 behind it means the proxy rejected the credentials: check for typos and special characters that were not encoded. A timeout means the proxy did not answer in time, which points at a dead host, a wrong port or a firewall on your side. An HTTP error from the echo endpoint means the proxy works but something between it and the endpoint failed. Our 407 error guide walks through the authentication case.
Asyncio proxy test with aiohttp for large lists
For thousands of lines, an asyncio version uses fewer resources than threads. One aiohttp.ClientSession is shared by every check, each request names its own proxy, and an asyncio.Semaphore limits how many are in flight. aiohttp takes proxy credentials through proxy_auth=aiohttp.BasicAuth(...), so no URL encoding is needed here.
# pip install aiohttp
import asyncio
import sys
import time
import aiohttp
CHECK_URL = "https://proxyv4.com/api/my-ip"
CONCURRENCY = 50
TIMEOUT = aiohttp.ClientTimeout(total=15, sock_connect=5)
def parse_line(line):
line = line.strip()
if not line or line.startswith("#"):
return None
ip, port, user, password = line.split(":", 3)
return ip, f"http://{ip}:{port}", aiohttp.BasicAuth(user, password)
async def check(session, sem, ip, proxy, auth):
async with sem:
start = time.monotonic()
try:
async with session.get(CHECK_URL, proxy=proxy, proxy_auth=auth) as r:
if r.status != 200:
return ip, False, f"HTTP {r.status}"
data = await r.json(content_type=None)
except asyncio.TimeoutError:
return ip, False, "timeout"
except aiohttp.ClientHttpProxyError as exc:
return ip, False, f"proxy refused: HTTP {exc.status}"
except (aiohttp.ClientError, ValueError) as exc:
return ip, False, exc.__class__.__name__
ms = round((time.monotonic() - start) * 1000)
if data.get("ip") != ip:
return ip, False, f"exit IP {data.get('ip')} differs"
return ip, True, f"{ms} ms"
async def main(path):
with open(path) as f:
proxies = [p for p in (parse_line(l) for l in f) if p]
sem = asyncio.Semaphore(CONCURRENCY)
async with aiohttp.ClientSession(timeout=TIMEOUT) as session:
tasks = [check(session, sem, *p) for p in proxies]
for coro in asyncio.as_completed(tasks):
ip, ok, info = await coro
print(f"{'OK ' if ok else 'FAIL'} {ip} {info}")
if __name__ == "__main__":
asyncio.run(main(sys.argv[1] if len(sys.argv) > 1 else "proxies.txt"))Note that aiohttp supports HTTP proxies only. To test the SOCKS5 side, use the aiohttp-socks package, or install requests[socks] and use socks5h:// URLs in the threaded script; the h makes DNS resolve through the proxy.
Choosing the concurrency limit and timeouts
There is no single right number. Start with a modest limit, such as the values in the scripts, and raise it only while the results stay consistent. If raising it produces more timeouts on proxies that passed before, you have hit a limit on your own machine or network, not on the proxies. On Linux and macOS a very high limit can also hit the open-file limit (ulimit -n).
Keep timeouts short enough that dead proxies fail quickly but long enough for distant locations. Do not retry inside the checker: a bulk test should report the state of the list now, and a flaky proxy is something you want to see. Run the test again later if you want a second sample.
When to use the free checker instead, and what to buy
Code is the right tool for scheduled checks, large lists and pipelines. For a quick look at a handful of proxies, the free online proxy checker does the same job in the browser without any setup.
If your list comes from free or shared sources, expect many failures and changing exit IPs; that is why bulk testers exist. Private proxies need less checking because each IP has a single tenant. ProxyV4 sells private datacenter IPv4 in seven countries and static ISP proxies in the United States and Vietnam, delivered as ip:port:user:pass lines that the scripts above read directly. Every proxy supports HTTP(S) and SOCKS5 with username and password; you choose HTTP only, SOCKS5 only or both on the same port per order. Traffic is unmetered, and the IP stays the same while the proxy is active and when you renew before expiry.
It is not the right product if you need a new IP on every request (rotating proxies are waitlist only), mobile carrier IPs, or IPv6, which is sold separately as IPv6 proxies. Check live prices, buy one IP, run it through your checker and your real workload, then scale.
Frequently asked questions
Why does my Python proxy checker hang?
Usually because a request has no timeout, so a proxy that never answers blocks the worker forever. Set a connect and a read timeout in requests, or a ClientTimeout in aiohttp.
How many proxies can I check at once?
It depends on your machine, network and the echo endpoint. Start with a few dozen at a time and increase while results stay stable; a sudden rise in timeouts means you have gone too far.
Can these scripts test SOCKS5 proxies?
The threaded script can, after pip install requests[socks] and a socks5h:// URL. aiohttp needs the aiohttp-socks package for SOCKS5.
Why split the line with split(':', 3)?
Passwords may contain colons. Limiting the split to three keeps everything after the third colon as the password.
What if a ProxyV4 proxy fails the test after purchase?
A proxy that fails for a technical reason, such as not connecting or showing the wrong location, is covered by the refund policy and credited to your account balance.
Bulk-test with code or the free checker
Private IPv4 and static ISP proxies arrive as ip:port:user:pass lines with HTTP(S) and SOCKS5 and unmetered traffic. Buy one IP and run it through your checker first.
Get a static IP