Bukatar Python ta hanyar ingantaccen proxy, mataki-mataki
Daga Tawagar Injiniyoyi ta ProxyV4 · Updated:
Don amfani da proxy a Python requests, tura proxies dict inda maɓallan 'http' da 'https' duka ke nuna http://user:pass@ip:port. Sanya wannan dict ɗin a kan requests.Session() domin sake amfani da haɗin proxy ɗaya ga kowace buƙata, wanda ya fi buɗe sabo a kowane kira sauri. A ƙasa: dict ɗin, session ɗin, kurakuran da za ka iya fuskanta, da kuma layin duba fitar IP na ProxyV4.
proxies = {'http': 'http://user:pass@ip:port', 'https': 'http://user:pass@ip:port'}- 1 IP$ 1.85
- 3+ IPs$ 1.70
- 10+ IPs$ 1.60
- 25+ IPs$ 1.50
- 50+ IPs$ 1.40
- 100+ IPs$ 1.30
- 200+ IPs$ 1.20
- 500+ IPs$ 1.10
- 1 IP$ 1.85
- 3+ IPs$ 1.70
- 10+ IPs$ 1.60
- 25+ IPs$ 1.50
- 50+ IPs$ 1.40
- 100+ IPs$ 1.30
- 200+ IPs$ 1.20
- 500+ IPs$ 1.10
Bayanin proxies dict
| Saitawa | Ƙima | Dalili |
|---|---|---|
Maɓallin 'http' | 'http://user:pass@ip:port' | Ana amfani da shi don adiresoshin http:// na yau da kullum |
Maɓallin 'https' | 'http://user:pass@ip:port' | Har yanzu adireshin proxy ne na http://: urllib3 tana neman ramin CONNECT daga proxy sannan tana gudanar da TLS a cikinsa |
timeout | timeout=(5, 15) | Daƙiƙa 5 don haɗawa, daƙiƙa 15 don karantawa, domin makoma da ba ta aiki ba ta daskare zaren aiki ba |
| Juyawa | itertools.cycle(proxy_list) | Yana bayar da tsayayyun IP ɗinku bi da bi |
Dalilin da ya sa Session ya fi amfani da requests.get() zalla
Kowane kiran requests.get() na sama yana gina sabon rukunin haɗi, don haka ana maimaita haɗin TCP zuwa ga proxy da musayar TLS da shafin a kowane lokaci. Session yana barin waɗannan haɗe-haɗen a raye tsakanin kiran da ake yi. Ga rubutun umarni da ke ɗauko daruruwan shafuka ta IP ɗaya, wannan bambancin yana bayyana sarai a jimillar lokacin aiki.
Saita s.proxies.update(proxies) sau ɗaya kuma kowane s.get() da s.post() zai wuce ta cikin proxy. Abu ɗaya da ya kamata a sani: idan an saita HTTP_PROXY ko HTTPS_PROXY a cikin muhalli, requests na iya haɗa su da saitunan session; tura proxies= a ainihin kiran lokacin da kake son tabbatar da wanda ake amfani da shi.
Kurakurai da yadda ake fahimtar su
ProxyErrortare da407 Proxy Authentication Required: ana iya samun proxy amma bayanin shiga a URL ɗinka bai dace da oda ba. Duba shafin kuskure na 407- Kalmomin sirri da ke ɗauke da
@,:ko/suna ɓata URL; sauya musu lamba da farko ta amfani daurllib.parse.quote(password, safe='') - Sauya adireshin URL na proxy zuwa
https://yakan lalata haɗin; riƙehttp://ga maɓallan biyu sai dai idan tashar proxy ɗinka tashar TLS ce ta musamman ConnectTimeoutyana nuni ne ga IP ko tasha, ba bayanan shiga ba; gwada damar haɗawa ta amfani da mai duba tasha
Tsari iri ɗaya a httpx da aiohttp
Tsarin URL ɗin yana aiki iri ɗaya. A cikin httpx, ƙirƙiri httpx.Client(proxy='http://user:pass@ip:port'); a cikin aiohttp, tura proxy= zuwa session.get(url, proxy=...). Binciken shafuka masu yawa tare da tsara lokaci da sake gwadawa yakan fi sauƙi a cikin Scrapy.
Mataki bayan mataki
- 1
Gina URL na proxy
Sake rubuta kowane layin
ip:port:user:passna ProxyV4 azamanhttp://user:pass@ip:port, ko kuma canza dukkan jerin da mai canza tsarin proxy. - 2
Ƙirƙiri dict ɗin
Rubuta
proxies = {'http': 'http://user:pass@ip:port', 'https': 'http://user:pass@ip:port'}. - 3
Haɗa shi da Session
Ƙirƙiri
s = requests.Session()kuma kiras.proxies.update(proxies)domin a bar haɗe-haɗe a raye tsakanin buƙatu. - 4
Buga fitar IP
Gudanar da
print(s.get('https://proxyv4.com/api/my-ip', timeout=10).json()); IP ɗin da ke fitowa ya zama na proxy ne.
Tambayoyi da aka fi yi
Me ya sa shigarwar 'https' take da URL na http://?
Tsarinsa yana bayyana yadda Python ke magana da proxy ne, ba da gidan yanar gizon ba. urllib3 tana haɗawa da proxy ta hanyar HTTP ta yau da kullum, ta aika da CONNECT, sannan ta yi TLS da shafin https:// ta cikin wannan ramin, don haka har yanzu zirga-zirgar shafin a ɓoye take.
Ta yaya zan rarraba buƙatu a kan tsayayyun proxies 25?
Sanya adiresoshin URL 25 a cikin jeri, ƙirƙiri pool = itertools.cycle(PROXIES), kuma ga kowace buƙata ɗauki p = next(pool) sannan ka tura proxies={'http': p, 'https': p}. Tsara ɗan dakatawa mai ma'ana ga kowane IP don kada a cika wa adireshi guda ɗaya aiki.
Ana auna zirga-zirgar bayanai daga rubutun umarni na Python?
A'a. Tsare-tsaren keɓaɓɓen IPv4 da tsayayyen ISP na ProxyV4 suna zuwa da bandwidth marar iyaka, don haka rubutun umarni masu ɗaukar lokaci mai tsawo ba sa fuskantar ƙayyadadden bayanai.
Tushe
Rukunin tsayayyun IP don rubutun umarni na Python
Keɓaɓɓen IPv4 a ƙasashe 7 tare da tsayayyen ISP a US da Vietnam. Sayi IP guda ɗaya ko rukuni; farashin kowane IP yana raguwa da yawa.
Duba farashin US na yanzu