Selenium WebDriver da proxies masu tantancewa: abin da ke aiki
Daga Tawagar Injiniyoyi ta ProxyV4 · Updated:
Tantancewar proxy a Selenium na buƙatar dabarar warwarewa, saboda tutar --proxy-server=http://ip:port ta ChromeDriver ba za ta iya ɗaukar sunan mai amfani da kalmar sirri ba; Chrome yana watsar da user:pass da ke ciki kuma yana nuna akwatin shiga wanda WebDriver ba zai iya cikewa ba. Hanyoyi biyu abin dogaro don gyara wannan su ne selenium-wire, wanda ke kula da shigar ProxyV4 a Python, da kuma wani ƙaramin ƙarin Chrome wanda ke amsa onAuthRequired a kowane yaren Selenium.
http://user:pass@ip:port (selenium-wire / Chrome proxy extension)- 1 IP$ 1.85
- 3+ IPs$ 1.70
- 10+ IPs$ 1.60
- 25+ IPs$ 1.50
- 50+ IPs$ 1.40
- 100+ IPs$ 1.30
- 200+ IPs$ 1.20
- 500+ IPs$ 1.10
- 1 IP$ 2.25
- 3+ IPs$ 2.10
- 10+ IPs$ 2.00
- 25+ IPs$ 1.90
- 50+ IPs$ 1.80
- 100+ IPs$ 1.70
- 200+ IPs$ 1.60
- 500+ IPs$ 1.50
Kwatanta zaɓuɓɓuka
| Hanya | Yadda kake saita shi | Wanda ya dace da shi |
|---|---|---|
selenium-wire (Python) | seleniumwire_options={'proxy': {'http': 'http://user:pass@ip:port', 'https': 'http://user:pass@ip:port'}} | Hanya mafi sauri a Python; ana iya canza proxy alhali burauzar a buɗe take |
| Ƙarin tantancewa | Wani ƙari mai nau'in zip wanda background script ɗinsa ke sarrafa chrome.webRequest.onAuthRequired | Java, C#, Python da kuma undetected-chromedriver baki ɗaya |
| Koma zuwa Playwright | browser.newContext({ proxy: { server, username, password } }) | Sababbin ayyuka; shigar proxy tana ciki a shirye, babu buƙatar wani ƙari |
Abin da ke faruwa da --proxy-server kawai
Chrome yana karɓar adireshin proxy daga layin umarni amma yana yin watsi da duk wani bayanin shiga da aka rubuta a ciki. Lokacin da proxy ya ba da amsar 407, Chrome yana buɗe akwatin shiga na asali na na'ura. Wannan akwatin yana wajen DOM na shafin, saboda haka find_element ba zai iya ganinsa ba kuma rubutun zai ci gaba da jira har sai lokaci ya ƙare. Don haka, kowace hanyar da ke aiki tana bayar da bayanan shigar a wani wuri daban ba a cikin tutar ba.
Lura cewa ba a ci gaba da kula da selenium-wire a halin yanzu. Har yanzu yana aiki da tsare-tsare da yawa, amma ga ayyukan da za su daɗe, amfani da dabarar ƙari ko Playwright shi ne zaɓi mafi aminci.
Kiyaye ingancin gudanarwar Selenium mai proxy
- Saita
options.page_load_strategy = 'eager'domin rubutun ya ci gaba da aiki da zarar DOM ta shirya maimakon jiran rubutun wasu na waje masu jinkiri - Don gudanarwa mara fuska a Chrome 109 ko na gaba yi amfani da
--headless=new; kari, ciki har da ƙarin tantancewa, suna aiki kamar yadda suke a cikin taga ta al'ada - Kashe lodin hotuna a saitunan Chrome lokacin da kake buƙatar rubutu kawai kamar farashi ko lakabi
- Sanya
driver.quit()a cikin sashintry / finallydomin idan aikin ya faɗi kada ya bar ayyukan Chrome a buɗe a baya
Kafin ka faɗaɗa aiki
Kowane zangon WebDriver cikakken burawza ne, don haka ka ba kowane zango nasa IP idan shafin yana danganta asusu ta adireshi. Ana ba da hayar keɓaɓɓen IPv4 na ProxyV4 ga abokin ciniki guda ɗaya kuma yana riƙe adireshinsa lokacin da ka sabunta, wanda ke sa asusun gwaji masu aiki na dogon lokaci su kasance a kan IP ɗaya. Fara gwada sabbin proxies ta mai duba proxy da farko don kada lalataccen layi ya fito a matsayin Selenium timeout mai rikitarwa.
Idan gudanarwa har yanzu tana buɗe shafuka daga ainihin IP ɗin sabar ko kuma ta tsaya a kuskuren tunnel, sashin Selenium na proxy da ba ya aiki yana lissafa dalilan da aka fi sani bisa tsari.
Mataki bayan mataki
- 1
Shigar da selenium-wire
A cikin yanayin aiki na zahiri, gudanar da
pip install selenium-wire. - 2
Bayyana proxy ɗin
Gina
sw_options = {'proxy': {'http': 'http://user:pass@ip:port', 'https': 'http://user:pass@ip:port'}}daga oda ta ProxyV4 ɗinka. - 3
Fara Chrome da waɗannan zaɓuɓɓukan
Ƙirƙiri driver tare da
driver = webdriver.Chrome(seleniumwire_options=sw_options), kana shigo dawebdriverdagaseleniumwire. - 4
Duba IP ɗin fita
Kira
driver.get('https://proxyv4.com/api/my-ip')kuma ka tabbatar da cewa shafin yana nuna adireshin proxy.
Tambayoyi da aka fi yi
Me ya sa ake watsi da --proxy-server=http://user:pass@ip:port a cikin ChromeOptions?
Tsarin Chrome ne ya cire bayanan shiga daga ƙimar --proxy-server. Ka samar da su ta hanyar selenium-wire, ƙarin onAuthRequired, ko kuma ka koma zuwa Playwright, wanda ke karɓar sunan mai amfani da kalmar sirri kai tsaye.
Shin selenium-wire zai iya canza proxy ba tare da sake kunna burauzar ba?
I. Saita sabon dict, driver.proxy = {'http': 'http://...', 'https': 'http://...'}, kuma buƙatun da ke biyo baya za su yi amfani da sabon proxy na sama.
Shin ƙarin tantancewa yana aiki a cikin Chrome mara fuska?
I, tare da --headless=new. Tsohon yanayin --headless bai kasance yana loda kari ta wannan hanyar ba.
Lokacin Selenium yana ƙarewa a kowane shafi ta hanyar proxy. Menene mataki na gaba?
Loda mashigar IP ɗin tare da cURL ta amfani da bayanan shiga iri ɗaya. Idan cURL ma ya gaza, layin ko tashar ce matsalar; jagorar ƙarewar lokaci na proxy ta lissafa abubuwan da za a duba.
Tushe
Tabbatattun IP don zangonin Selenium
Keɓaɓɓen IPv4 a ƙasashe 7 da tsayayyen ISP a US da Vietnam, tare da mai amfani da kalmar sirri, mara iyaka kuma abokin ciniki ɗaya a kowane IP.
Duba farashin US na yanzu