Selenium 循环中的超时异常

Posted 2023-02-25

【中文标题】Selenium 循环中的超时异常

【英文标题】：Timeout exception in a cicle for Selenium

【发布时间】：2020-12-16 01:41:14

【问题描述】：

我正在抓取此页面https://www.betexplorer.com/soccer/netherlands/eerste-divisie-2018-2019/results/ 我的代码提取所有匹配的 url，并使用 cicle for 将从每个 url 中提取数据。 （网址示例https://www.betexplorer.com/soccer/netherlands/eerste-divisie-2018-2019/den-bosch-g-a-eagles/YkOxU6sM/）这是我的代码：

for i in matches:
    driver.get(i)

    Country = WebDriverWait(driver, 2).until(EC.visibility_of_element_located((By.XPATH, "/html/body/div[4]/div[5]/div/div/div[1]/section/ul/li[3]/a"))).text
    leagueseason = WebDriverWait(driver, 2).until(EC.visibility_of_element_located((By.XPATH, "/html/body/div[4]/div[5]/div/div/div[1]/section/header/h1/a"))).text
    Date = WebDriverWait(driver, 2).until(EC.visibility_of_element_located((By.ID, 'match-date'))).text
    Date = Date.replace(".", "/").replace("-", "").replace(" ", "", 1)
    Home = WebDriverWait(driver, 2).until(EC.visibility_of_element_located((By.XPATH, "/html/body/div[4]/div[5]/div/div/div[1]/section/ul[2]/li[1]/h2/a"))).text
    Away = WebDriverWait(driver, 2).until(EC.visibility_of_element_located((By.XPATH, "/html/body/div[4]/div[5]/div/div/div[1]/section/ul[2]/li[3]/h2/a"))).text
    ft = WebDriverWait(driver, 2).until(EC.visibility_of_element_located((By.ID, 'js-score'))).text
    Res = ""
    
    try:
        extrainfo = driver.find_element_by_xpath("//*[@id='js-eventstage']").text
    except NoSuchElementException:
        extrainfo = " "

    try:
        driver.find_element_by_xpath("//td[a[.='bet365']]/following-sibling::td[span]")
        WebDriverWait(driver, 2).until(EC.element_to_be_clickable((By.XPATH, "(//td[a[.='bet365']]/following-sibling::td[@data-odd])[1]"))).click()
        oid = WebDriverWait(driver, 2).until(EC.visibility_of_element_located((By.XPATH, "(//td[a[.='bet365']]/following-sibling::td[@data-odd])[1]"))).get_attribute("data-oid")
        bid = WebDriverWait(driver, 2).until(EC.visibility_of_element_located((By.XPATH, "(//td[a[.='bet365']]/following-sibling::td[@data-odd])[1]"))).get_attribute("data-bid")
        var = oid+'-'+bid
    except NoSuchElementException:
        B365H = 'no bet365 odd'
    except TypeError:
        B365H = WebDriverWait(driver, 2).until(EC.visibility_of_element_located((By.XPATH, "(//td[a[.='bet365']]/following-sibling::td[@data-odd])[1]"))).get_attribute("data-odd")
    else:
        B365H = WebDriverWait(driver, 2).until(EC.visibility_of_element_located((By.XPATH, "//*[contains(@id,'%s')]/tr[last()]/td[@class='bold']" % var))).text

    
    try:
        driver.find_element_by_xpath("//td[a[.='bet365']]/following-sibling::td[span]")
        WebDriverWait(driver, 2).until(EC.element_to_be_clickable((By.XPATH, "(//td[a[.='bet365']]/following-sibling::td[@data-odd])[2]"))).click()
        oid = WebDriverWait(driver, 2).until(EC.visibility_of_element_located((By.XPATH, "(//td[a[.='bet365']]/following-sibling::td[@data-odd])[2]"))).get_attribute("data-oid")
        bid = WebDriverWait(driver, 2).until(EC.visibility_of_element_located((By.XPATH, "(//td[a[.='bet365']]/following-sibling::td[@data-odd])[2]"))).get_attribute("data-bid")
        var = oid+'-'+bid
    except NoSuchElementException:
        B365D = 'no bet365 odd'
    except TypeError:
        B365D = WebDriverWait(driver, 2).until(EC.visibility_of_element_located((By.XPATH, "(//td[a[.='bet365']]/following-sibling::td[@data-odd])[2]"))).get_attribute("data-odd")
    else:
        B365D = WebDriverWait(driver, 2).until(EC.visibility_of_element_located((By.XPATH, "//*[contains(@id,'%s')]/tr[last()]/td[@class='bold']" % var))).text


    
    try:
        driver.find_element_by_xpath("//td[a[.='bet365']]/following-sibling::td[span]")
        WebDriverWait(driver, 2).until(EC.element_to_be_clickable((By.XPATH, "(//td[a[.='bet365']]/following-sibling::td[@data-odd])[3]"))).click()
        oid = WebDriverWait(driver, 2).until(EC.visibility_of_element_located((By.XPATH, "(//td[a[.='bet365']]/following-sibling::td[@data-odd])[3]"))).get_attribute("data-oid")
        bid = WebDriverWait(driver, 2).until(EC.visibility_of_element_located((By.XPATH, "(//td[a[.='bet365']]/following-sibling::td[@data-odd])[3]"))).get_attribute("data-bid")
        var = oid+'-'+bid
    except NoSuchElementException:
        B365A = 'no bet365 odd'
    except TypeError:
        B365A = WebDriverWait(driver, 2).until(EC.visibility_of_element_located((By.XPATH, "(//td[a[.='bet365']]/following-sibling::td[@data-odd])[3]"))).get_attribute("data-odd")
    else:
        B365A = WebDriverWait(driver, 2).until(EC.visibility_of_element_located((By.XPATH, "//*[contains(@id,'%s')]/tr[last()]/td[@class='bold']" % var))).text

现在我会在页面未加载时添加TimeoutException，我尝试了这种方式：

try:
   #previous code
except TimeoutException:
   driver.get(i)

这样，当我遇到超时错误时，代码将跳过 url 抓取并继续下一个 url。 我该如何解决这个问题？我希望在超时的情况下它会重新加载页面并再次为该 url 抓取数据。

【问题讨论】：

欢迎来到！请阅读minimal reproducible example 并相应地编辑您的帖子。

你可以，但它进入一个while 循环，将在同一个 url 上继续，直到它成功，然后转到下一个 url

【参考方案1】：

你可以把它放到一个while循环中，一旦它成功完成，就会继续。所以像：

success = False
while success == False:
    try:
        # previous code
        success = True
    except TimeoutException:
        print ('Will try again...')

【讨论】：

谢谢你的回答，我试过了，当有超时它会完成代码

确保在for i循环中缩进