使用 Selenium webdriver 和 Python 获得完全渲染的 HTML

Posted 2023-02-15

技术标签:

【中文标题】使用 Selenium webdriver 和 Python 获得完全渲染的 HTML【英文标题】：Get fully rendered HTML using Selenium webdriver and Python 【发布时间】：2019-02-24 23:01:34 【问题描述】：

我正在尝试使用 Selenium webdriver 在 Python 中构建一个网络爬虫，但是当我从 webdriver 检索网站源代码时，我无法访问所需的信息。

我认为问题在于，一旦页面最初从服务器下载，内容就会通过 javascript 添加到页面中。当我运行browser.page_source 时，我得到了添加此内容之前页面的源代码。我想知道在添加了用 JavaScript 加载的内容后是否可以获取页面的源代码（换句话说，当我使用 Inspect Element 查看页面时看到的）。

这是我正在使用的基本 Python 脚本：

from selenium import webdriver

browser = webdriver.Chrome()
browser.get("https://www.opportunities.auckland.ac.nz")
print(browser.page_source)

当我运行上述脚本时，我得到了在浏览器中查看页面源时看到的页面的源代码（即，使用检查元素查看代码时没有可见的其他内容）。

我尝试过的事情

在不同的地方添加time.sleep(10)，以防我访问源时页面没有完全加载。在正文上使用get_attribute("innerhtml")。使用execute_script() 使JS 运行。使用execute_script()让JS脚本一个个运行。

如果有人能首先告诉我这是否可能，以及它是否为我指明了正确的方向，那就太好了。谢谢。

更新 1

我在尝试 Piotrek 的解决方案时得到以下输出：

Warning (from warnings module):
  File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/phantomjs/webdriver.py", line 49
warnings.warn('Selenium support for PhantomJS has been deprecated, please use headless '
UserWarning: Selenium support for PhantomJS has been deprecated, please use headless versions of Chrome or Firefox instead
<html><head></head><body></body></html>

不幸的是，这似乎不起作用。

【问题讨论】：

使用 DOM 检查您想要的源，并使用 WebDriverWait 和 presence_of_element_located 来获取仅在 JS 执行后才存在的元素。 【参考方案1】：

所需的元素在<iframe> 中，因此您必须使用 WebDriverWait 以使 iframe 可用，然后切换到它，然后再次使用 WebDriverWait 让元素可见。

您可以使用以下解决方案：

代码块：

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome()
driver.get("https://www.opportunities.auckland.ac.nz")
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.ID,"ptifrmtgtframe")))
WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//span[@id='HRS_APPL_WRK_HRS_PAGE_TITLE']")))
print(driver.page_source)

控制台输出：

<!DOCTYPE html><html xmlns="http://www.w3.org/1999/xhtml" class="pc chrome win psc_dir-ltr psc_form-xlarge" dir="ltr" lang="en"><!-- Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved. --><head>
<meta name="apple-mobile-web-app-capable" content="yes" />
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<script type="text/javascript" async="" src="https://ssl.google-analytics.com/ga.js"></script><script language="JavaScript">
var  totalTimeoutMilliseconds = 2073600000; 
var  warningTimeoutMilliseconds = 2073600000; 
var timeOutURL = 'https://www.opportunities.auckland.ac.nz/psp/ps/EMPLOYEE/HRMS/?cmd=expire';
var timeoutWarningPageURL = 'https://www.opportunities.auckland.ac.nz/psc/ps/EMPLOYEE/HRMS/s/WEBLIB_TIMEOUT.PT_TIMEOUTWARNING.FieldFormula.IScript_TIMEOUTWARNING';
var sRCRequestURL = 'https://www.opportunities.auckland.ac.nz/psc/ps/EMPLOYEE/HRMS/s/WEBLIB_PTRC.ISCRIPT1.FieldFormula.IScript_RC_REQUEST';
</script> 

<link id="PSSTYLEREQ_1_css" rel="stylesheet" type="text/css" href="/cs/ps/cache/PSSTYLEREQ_1.css" />
<link id="UOA_STYLE_SWAN_1_css" rel="stylesheet" type="text/css" href="/cs/ps/cache/UOA_STYLE_SWAN_1.css" />
<title>Applicant Home</title>
<script language="JavaScript">
try 
document.domain = "opportunities.auckland.ac.nz";

catch(err) ;
</script>
<script language="JavaScript">
var winName='win0';
var winParent = null;
var modalID = null;
var modalZoomName = null;
var baseKey_win0 = "\x1b\r\n";
var altKey_win0 = "05678\xbc\xbe\xbf\xde";
var ctrlKey_win0 = "JKM";
var saveWarningKeys_win0 = "";
var refererURL = 'https://www.opportunities.auckland.ac.nz/psp/ps/EMPLOYEE/HRMS/c/HRS_HRAM.HRS_CE.GBL?languageCd=ENG';
var isNewSaveWarn = true;
var bAccessibility_win0 =false;
var bFMode =false;
var bDoModal_win0 = false;
var bJSModal_win0 = false;
var bPromptPage_win0 = false;
var bTabOverTB_win0 = false;
var bTabOverPg_win0 = false;
var bTabOverNonPS_win0 = false;
var PIA_KEYSTRUCT=HRS_PERSON_ID:"0";
var strCurrUrl='https://www.opportunities.auckland.ac.nz/psp/ps/EMPLOYEE/HRMS/c/HRS_HRAM.HRS_CE.GBL?HRS_PERSON_ID=0&amp;PAGE=HRS_CE_HM_PRE';
var bIncInNavigation='F';
var szCrefID='HC_HRS_CE_GBL2';
var szCrefReged='T';
var szCrefVisible='F';
var szCrefLabel='Careers';
bGenDomInfo = false;
var szCalendarType = 'G';
var bMenuSrchPage = false;
var bWSRP=0;var szMenuSrchText = "-999999-";
var disablePNSubscriptions=1;var modalCloseUrl = '/cs/ps/cache/PT_TRANS_16PIX_1.png';
var modalCloseAlt = 'Close';
var modalResizeUrl= '/cs/ps/cache/PT_RESIZE_IMG_1.gif';
var modalResizeAlt = 'Drag to resize';
var modalMoveAlt = 'Drag to move';
</script>
<script language="JavaScript">
var gridList_win0 = [
['HRS_CE_JO_EXT_I$0',0,56,0,1,1]
];
var gridHeaderList_win0 = [
[['HRS_CE_JO_EXT_I$0#0',1]]
];
var gridRowSelRgbColor_win0 ='rgb(173,216,230)';
var gridFieldList_win0 = [
['HTMLAREA$%c',
'HRS_CE_JO_EXT_I$new$%c$$0',
'HRS_CE_JO_EXT_I$delete$%c$$0']
];
</script>
<script language="JavaScript">
var PFieldList_win0 = [['HRS_SCH_WRK_ESTABID',1],['HRS_ESTAB_I_DESCR',1]];
</script>
<script language="JavaScript">
try 
document.domain = "opportunities.auckland.ac.nz";

catch(err) ;
</script>
<script language="JavaScript" type="text/javascript" src="/cs/ps/cache/PT_COMMON_MIN_1.js">
</script>
<script language="JavaScript" type="text/javascript" src="/cs/ps/cache/PT_AJAX_NET_MIN_1.js">
</script>
<script language="JavaScript" type="text/javascript" src="/cs/ps/cache/PT_SAVEWARNINGSCRIPT_MIN_1.js">
</script>
<script language="JavaScript" type="text/javascript" src="/cs/ps/cache/PT_PAGESCRIPT_win0_MIN_1.js">
</script>
<script language="JavaScript" type="text/javascript" src="/cs/ps/cache/PT_EDITSCRIPT_win0_MIN_1.js">
</script>
<script language="JavaScript" type="text/javascript" src="/cs/ps/cache/PT_RC_JS_MIN_1.js">
</script>
<script language="JavaScript" type="text/javascript" src="/cs/ps/cache/PT_POPUPSCRIPT_win0_MIN_1.js">
</script>
<script language="JavaScript" type="text/javascript" src="/cs/ps/cache/PT_GRIDSCRIPT_win0_MIN_1.js">
</script>
<link id="UOA_HDR_LOGO_1_css" rel="stylesheet" type="text/css" href="/cs/ps/cache/UOA_HDR_LOGO_1.css" />
<link id="UOA_HDR_BANNER_1_css" rel="stylesheet" type="text/css" href="/cs/ps/cache/UOA_HDR_BANNER_1.css" />
<link id="UOA_FOOTER_BANNER_1_css" rel="stylesheet" type="text/css" href="/cs/ps/cache/UOA_FOOTER_BANNER_1.css" />
<script language="JavaScript" type="text/javascript" src="/cs/ps/cache/UOA_GA_JS_MIN_1.js">
</script>
<script language="JavaScript" type="text/javascript" src="/cs/ps/cache/PT_TYPEAHEAD_win0_MIN_1.js">
</script>
<script language="JavaScript">
var nResubmit=0;
var nLastAction=0;
var loader=null;
if (typeof(bCleanHtml) == 'undefined')
    var bCleanHtml = false;
setupTimeout2();
var postUrl_win0='https://www.opportunities.auckland.ac.nz/psc/ps/EMPLOYEE/HRMS/c/HRS_HRAM.HRS_CE.GBL';
function aAction_win0(form, id, params, bAjax, bPrompt)

setupTimeout2();
if ((id != '#ICSave'))
processing_win0(1,3000);
aAction0_win0(form, id, params, bAjax, bPrompt);

function submitAction_win0(form, id, event)

setupTimeout2();
if (!ptCommonObj2.isICQryDownload(form, id))
  processing_win0(1,3000); 
preSubmitProcess_win0(form, id);
var spellcheckpos = id.indexOf('$spellcheck');
if ((spellcheckpos != -1) &amp;&amp; (id.indexOf('#KEYA5') != -1))
    form.ICAction.value = id.substring(0,spellcheckpos);
else 
    form.ICAction.value=id;
var actionName = form.ICAction.value;
form.ICXPos.value=ptCommonObj2.getScrollX();
form.ICYPos.value=ptCommonObj2.getScrollY();
bcUpdater.storeBcDomData();
if ((typeof(bAutoSave) != 'undefined') &amp;&amp; bAutoSave)
    form.ICAutoSave.value = '1';
if (!ptCommonObj2.isAJAXReq(form, id) &amp;&amp; !ptCommonObj2.isPromptReq(id))
if (nLastAction == 1 &amp;&amp; nResubmit &gt; 0) return;
form.ICResubmit.value=nResubmit;
form.submit();
if (!ptCommonObj2.isICQryDownload(form, id))
    nResubmit++;

else if (ptCommonObj2.isPromptReq(id))
    pAction_win0(form, id, arguments[2]);
else
    aAction_win0(form, actionName);
cancelBubble(event);

</script>
<script language="javascript">
var pt_calHeadstyle=" class='PTCALHEAD' "
var pt_calWeekHeadstyle=" class='PTCALWEEKHEAD' "
function dateitemrefs()

this.pt_dateheader = "/cs/ps/cache/PT_PORTAL_IC_CLOSE_SD_SWAN_1.gif";
this.pt_datering = "/cs/ps/cache/PT_CURRENT_DATE_SD_CSS_1.gif";
this.pt_datesel = "/cs/ps/cache/PT_SELECTED_DATE_SD_CSS_1.gif";
this.pt_nextmonth = "/cs/ps/cache/PT_RIGHT_SCROLL_SD_SWAN_1.gif";
this.pt_prevmonth= "/cs/ps/cache/PT_LEFT_SCROLL_SD_SWAN_1.gif";
this.pt_daystitle_hijri = "/cs/ps/cache/PT_DATE_TITLE_S6_1.GIF";
this.pt_daystitle_thai = "/cs/ps/cache/PT_DATE_TITLE_S0_1.GIF";
this.pt_daystitle_s0 = "/cs/ps/cache/PT_DATE_TITLE_S0_1.GIF";
this.pt_daystitle_m1 = "/cs/ps/cache/PT_DATE_TITLE_M1_1.GIF";
this.pt_daystitle_t2 = "/cs/ps/cache/PT_DATE_TITLE_T2_1.GIF";
this.pt_daystitle_w3 = "/cs/ps/cache/PT_DATE_TITLE_W3_1.GIF";
this.pt_daystitle_t4 = "/cs/ps/cache/PT_DATE_TITLE_T4_1.GIF";
this.pt_daystitle_f5 = "/cs/ps/cache/PT_DATE_TITLE_F5_1.gif";
this.pt_daystitle_s6 = "/cs/ps/cache/PT_DATE_TITLE_S6_1.GIF";


function LoadCalendar()

var dateitems = new dateitemrefs();
loadImages(dateitems);

function DatePrompt_win0(dtfield,promptfield,yrfmt,bsubmit)

setupTimeout2();
openDate_win0(dtfield, promptfield, "DMY/"+yrfmt,bsubmit,"G",0);

</script>
<script language="JavaScript" ptdefer="defer">
if (typeof ptStyleSheet == "undefined") 
ptStyleSheet = 'UOA_STYLE_SWAN';

function onLoadExt_win0() 
modalZoomName = null;
var oWin=window;
var oDoc=document;
ptTAObj_win0.init(oWin,oDoc,oDoc.win0,'/cs/ps/cache/PT_SRT2UP_SEL_1.gif','/cs/ps/cache/PT_SRT2DN_SEL_1.gif','/cs/ps/cache/PT_PORTAL_IC_CLOSE_1.gif',500);
var pt_pageinfo = document.getElementById('pt_pageinfo_win0');
if (pt_pageinfo) 
pt_pageinfo.setAttribute('Page', 'HRS_CE_HM_PRE');
pt_pageinfo.setAttribute('Component', 'HRS_CE');
pt_pageinfo.setAttribute('Menu', 'HRS_HRAM');
pt_pageinfo.setAttribute('Mode', 'CLASSIC');

g_bAccessibilityMode=false;
var actn='';
setKeyEventHandler_win0();ptEvent.add(window,'scroll',positionWAIT_win0);
ptCommonObj2.generateABNSearchResults(document.win0);
getGblSrchPageNum(actn);
if (gSrchRsltPageNum &lt;= 5) getAllRelatedActions();

ptCommonObj2.moveUnivSrchDiv();
if (typeof(ptLongEditCounter) != 'undefined' &amp;&amp; ptLongEditCounter != null)
   ptLongEditCounter.onLoadLongEditCounter();
if (typeof(HelppopupObj_win0) != 'undefined' &amp;&amp; HelppopupObj_win0 != null)
 HelppopupObj_win0.StopPopup('win0');
doModalOnLoad_win0(false, false);
ResetGlyph_win0();
self.scroll(0,0);
ptGridObj_win0.initGriddd854();objToBeFocus = null;
if (typeof oWin == 'undefined') setEventHandlers_win0('ICFirstAnchor_win0', 'ICLastAnchor_win0', false);
 else
 oWin.setEventHandlers_win0('ICFirstAnchor_win0', 'ICLastAnchor_win0', false);
setFocus_win0('HRS_SCH_WRK_ESTABID',-1);
ptLoadingStatus_empty(0);
setupTimeout2();
processing_win0(0,3000);
ptConsole2.enable();

UpdateBreadCrumbs();
GenerateABN();
GenerateFakeBC();
bcUpdateForPTF();if (typeof(myAppsWindowOpenJS) != 'undefined' &amp;&amp; myAppsWindowOpenJS != null &amp;&amp; myAppsWindowOpenJS != '')
 
try eval(myAppsWindowOpenJS); catch(e) 
  myAppsWindowOpenJS=null;

ResetGlyph_win0();setPageletInfoInCtxmenu_win0("false");
GetDomData(win0, "TargetContent", "#GetDomInfo");

tryptEvent.load(onLoadExt_win0);catch(e)
</script></head>
<body tabindex="-1" class="PSPAGE">
<div id="pt_pageinfo_win0" page="HRS_CE_HM_PRE" component="HRS_CE" menu="HRS_HRAM" mode="CLASSIC"></div>
<form id="HRS_CE" name="win0" method="post" action="https://www.opportunities.auckland.ac.nz/psc/ps/EMPLOYEE/HRMS/c/HRS_HRAM.HRS_CE.GBL" autocomplete="off" class="PSForm">
<div class="ps_box-toolshiddens" id="win0divPSTOOLSHIDDENS"><div class="ps_modalmask_cover psc_hidden" id="pt_modalMaskCover"> </div><div class="ps_modalmask psc_hidden" id="pt_modalMask"> </div><div id="pt_modals" class="PSMODAL"><div id="ptModalShadow" class="popupDragFrame" style="cursor:nw-resize"> </div></div>
<div class="ps_ajax_console psc_hidden" id="pt_console"><input type="button" id="COPYCONSOLE" value="Copy" onclick="ptConsole2.copy();"  title="copy to clipboard" /><input type="button" id="CLEARCONSOLE" onclick="ptConsole2.clear();" value="Clear"  title="clear console" /><input type="button" id="HIDECONSOLE" onclick="ptConsole2.hide();" value="Hide"  title="hide console" /><input type="button" id="CLOSECONSOLE" onclick="ptConsole2.deactive();" value="Close"  title="close console" /></div><div class="ps_typeahead psc_hidden" id="pt_typeahead0" style="display: none;"><span id="pt_typeahead" class="spanTextDropdown"> </span></div><div id="WAIT_win0" style="display: none; position: absolute; right: 0px; z-index: 99991; visibility: hidden;"><img id="processing" align="left" src="/cs/ps/cache/PT_PROCESSING_1.gif"  title="Processing... please wait" /></div>
 <div id="SAVED_win0" style="display: none; position: absolute; right: 0px; z-index: 99991; visibility: hidden;"><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td><img src="/cs/ps/cache/PTPORTAL_POPBOX_LEFT_1.gif"   border="0"  /></td><td id="saveWait_win0"  align="center" valign="middle" background="/cs/ps/cache/PTPORTAL_POPBOX_CENTER_1.gif"><img src="/cs/ps/cache/PT_LOADER_1.gif"    /></td><td nowrap="nowrap" background="/cs/ps/cache/PTPORTAL_POPBOX_CENTER_1.gif"><span id="ptStatusText_win0" dir="ltr" class="PSPROCESSING"> </span></td><td><img src="/cs/ps/cache/PTPORTAL_POPBOX_RIGHT_1.gif"    /></td><td> </td></tr></tbody></table></div><div class="x" id="win0divPSHIDDENFIELDS" style="display:none"><input type="hidden" name="ICType" id="ICType" value="Panel" />
<input type="hidden" name="ICElementNum" id="ICElementNum" value="0" />
<input type="hidden" name="ICStateNum" id="ICStateNum" value="1" />
<input type="hidden" name="ICAction" id="ICAction" value="None" />
<input type="hidden" name="ICXPos" id="ICXPos" value="0" />
<input type="hidden" name="ICYPos" id="ICYPos" value="0" />
<input type="hidden" name="ResponsetoDiffFrame" id="ResponsetoDiffFrame" value="-1" />
<input type="hidden" name="TargetFrameName" id="TargetFrameName" value="None" />
<input type="hidden" name="FacetPath" id="FacetPath" value="None" />
<input type="hidden" name="ICFocus" id="ICFocus" value="" />
<input type="hidden" name="ICSaveWarningFilter" id="ICSaveWarningFilter" value="0" />
<input type="hidden" name="ICChanged" id="ICChanged" value="-1" />
<input type="hidden" name="ICAutoSave" id="ICAutoSave" value="0" />
<input type="hidden" name="ICResubmit" id="ICResubmit" value="0" />
<input type="hidden" name="ICSID" id="ICSID" value="iGf8pba3GL6SnwAoi3ZMca4P0dLE1k7Nqh632XOcc84=" />
<input type="hidden" name="ICActionPrompt" id="ICActionPrompt" value="false" />
<input type="hidden" name="ICTypeAheadID" id="ICTypeAheadID" value="" />
<input type="hidden" name="ICBcDomData" id="ICBcDomData" value="" />
<input type="hidden" name="ICFind" id="ICFind" value="" />
<input type="hidden" name="ICAddCount" id="ICAddCount" value="" />
<input type="hidden" name="ICAPPCLSDATA" id="ICAPPCLSDATA" value="" />
<input type="hidden" name="HRS_CE_JO_EXT_I$hnewpers$0" id="HRS_CE_JO_EXT_I$hnewpers$0" value="0|0|0|0|0|567|0#" />
</div>
</div>
<a name="ICFirstAnchor_win0" id="ICFirstAnchor_win0"></a>
<div class="ps_header_panel" id="PT_HEADER_PANEL"></div>
<div class="" id="win0divPAGEBAR"></div>
<div class="ps_box-pagetabs" id="win0divPSPANELTABS"></div>
<div class="ps_pagecontainer" id="win0divPAGECONTAINER"><table class="PSPAGECONTAINER" role="main" aria-labelledby="app_label"><tbody><tr><td>
<div class="ps_pspagecontainer" id="win0divPSPAGECONTAINER"><table role="presentation" border="0" id="ACE_width" cellpadding="0" cellspacing="0" class="PSPAGECONTAINER" cols="12" >
<tbody><tr>
<td  ></td>
<td ></td>
<td ></td>
<td ></td>
<td ></td>
<td ></td>
<td ></td>
<td ></td>
<td ></td>
<td ></td>
<td ></td>
<td ></td>
</tr>
<tr>
<td  colspan="7"></td>
<td colspan="5" rowspan="3" valign="top" align="left">
<div id="win0divHRS_APPL_WRK_HRS_APP_LOGIN"><span id="HRS_APPL_WRK_HRS_APP_LOGIN$span" class="PSHYPERLINK" title="Login"><a name="HRS_APPL_WRK_HRS_APP_LOGIN" id="HRS_APPL_WRK_HRS_APP_LOGIN" ptlinktgt="pt_replace" tabindex="64" onfocus="doFocus_win0(this,false,true);" href="https://www.opportunities.auckland.ac.nz/sso-login" target="_top" class="PSHYPERLINK">Sign in/Register</a></span>
</div></td>
</tr>
<tr>
<td  colspan="2"></td>
<td colspan="5" valign="top" align="left">
<div id="win0divHRS_APPL_WRK_HRS_PAGE_TITLE"><span role="heading" aria-level="1" class="PAPAGETITLE" id="HRS_APPL_WRK_HRS_PAGE_TITLE">Career Opportunities</span>
</div></td>
</tr>
<tr>
<td  colspan="2"></td>
<td rowspan="2" valign="top" align="left">
<div id="win0divHRS_APPL_WRK_HRS_APP_INFO_GPB"><table role="presentation" border="0" id="ACE_HRS_APPL_WRK_HRS_APP_INFO_GPB" cellpadding="0" cellspacing="0" cols="5"  class="PSGROUPBOX" style="border-style:none">
<tbody><tr>
<td  ></td>
<td ></td>
<td ></td>
<td ></td>
<td ></td>
</tr>
<tr>
<td  colspan="2"></td>
<td colspan="2" valign="top" align="left">
<div id="win0divHRS_CE_WRK_HRS_ERROR_LOGIN"><div style="width:480px; ">
<!-- Begin HTML Area Name Undisclosed -->

<!-- End HTML Area -->
</div>
</div></td>
</tr>
<tr>
<td ></td>
<td colspan="2" valign="top" align="left">
<div id="win0div$ICField11"><table role="presentation" border="0" id="ACE_$ICField11" cellpadding="0" cellspacing="0" cols="2"  class="PAERRORTEXT" style="border-style:none">
<tbody><tr>
<td  ></td>
<td ></td>
</tr>
<tr>
<td ></td>
<td valign="top" align="left">
<div id="win0divHRS_APPL_WRK_HRS_PAGE_INSTR"><span class="PAPAGEINSTRUCTIONS" style="word-wrap:break-word;overflow:hidden;display:block; width:420px;  overflow:auto;" id="HRS_APPL_WRK_HRS_PAGE_INSTR"> </span>
</div></td>
</tr>
</tbody></table>
</div></td>
</tr>
</tbody></table>
</div></td>
</tr>
<tr>
<td  colspan="2"></td>
<td colspan="4" rowspan="2"></td>
<td colspan="5" rowspan="3" nowrap="nowrap" valign="top" align="left">
<div id="win0divHRS_APP_CAR_DRV_HRS_RCMNT_LNK"><span id="HRS_APP_CAR_DRV_HRS_RCMNT_LNK$span" class="PSHYPERLINK"><a name="HRS_APP_CAR_DRV_HRS_RCMNT_LNK" id="HRS_APP_CAR_DRV_HRS_RCMNT_LNK" ptlinktgt="pt_new" tabindex="65" onfocus="doFocus_win0(this,false,true);" href="http://www.auckland.ac.nz/uoa/home/about/careers-at-auckland/faqs" onclick="window.open('http://www.auckland.ac.nz/uoa/home/about/careers-at-auckland/faqs','','');cancelBubble(event);return false;" class="PSHYPERLINK">Frequently Asked Questions</a></span>
</div></td>
</tr>
<tr>
<td  colspan="3"></td>
</tr>
<tr>
<td ></td>
<td colspan="3" rowspan="6" valign="top" align="left">
<div id="win0div$ICField153"><table role="presentation" border="0" id="ACE_$ICField153" cellpadding="0" cellspacing="0" cols="2"  class="PSFRAME" style="border-style:none">
<tbody><tr>
<td  ></td>
<td ></td>
</tr>
<tr>
<td ></td>
<td valign="top" align="left">
<div id="win0divHRS_APPL_WRK_HRS_JOB_POST_GPB"><table cellpadding="2" cellspacing="0" cols="1" class="PSGROUPBOXWBO" >
<tbody><tr><td class="PSGROUPBOXLABEL" align="left">Latest Career Opportunities</td></tr>
<tr><td >
<table role="presentation" border="0" id="ACE_HRS_APPL_WRK_HRS_JOB_POST_GPB" cellpadding="0" cellspacing="0" cols="3"  class="PSGROUPBOX" style="border-style:none">
<tbody><tr>
<td  ></td>
<td ></td>
<td ></td>
</tr>
<tr>
<td ></td>
<td colspan="2" valign="top" align="left">
<div id="win0div$ICField54"><table role="presentation" border="0" id="ACE_$ICField54" cellpadding="0" cellspacing="0" cols="4"  class="UOA_PSCHARTCOLOR1" style="border-style:none">
<tbody><tr>
<td  ></td>
<td ></td>
<td ></td>
<td ></td>
</tr>
<tr>
<td ></td>
<td nowrap="nowrap" valign="top" align="left">
<div id="win0divUOA_CG_SRCH_TITLE"><span id="UOA_CG_SRCH_TITLE$span" class="UOA_PABOLDTEXT" title="Title"><a name="UOA_CG_SRCH_TITLE" id="UOA_CG_SRCH_TITLE" ptlinktgt="pt_peoplecode" tabindex="71" onclick="javascript:cancelBubble(event);" href="javascript:submitAction_win0(document.win0,'UOA_CG_SRCH_TITLE');" class="UOA_PABOLDTEXT">Job Title</a></span>
</div></td>
<td nowrap="nowrap" valign="top" align="left">
<div id="win0divUOA_CG_SRCH_TITLE1"><span id="UOA_CG_SRCH_TITLE1$span" class="UOA_PABOLDTEXT" title="Title1"><a name="UOA_CG_SRCH_TITLE1" id="UOA_CG_SRCH_TITLE1" ptlinktgt="pt_peoplecode" tabindex="70" onclick="javascript:cancelBubble(event);" href="javascript:submitAction_win0(document.win0,'UOA_CG_SRCH_TITLE1');" class="UOA_PABOLDTEXT">Campus</a></span>
</div></td>
<td nowrap="nowrap" valign="top" align="left">
<div id="win0divUOA_CG_SRCH_TITLE2"><span id="UOA_CG_SRCH_TITLE2$span" class="UOA_PABOLDTEXT" title="Title2"><a name="UOA_CG_SRCH_TITLE2" id="UOA_CG_SRCH_TITLE2" ptlinktgt="pt_peoplecode" tabindex="69" onclick="javascript:cancelBubble(event);" href="javascript:submitAction_win0(document.win0,'UOA_CG_SRCH_TITLE2');" class="UOA_PABOLDTEXT">Close Date</a></span>
</div></td>
</tr>
</tbody></table>
</div></td>
</tr>
<tr>
<td  colspan="3"></td>
</tr>
<tr>
<td ></td>
<td valign="top" align="left">
<div id="win0divHRS_CE_JO_EXT_I$0"><table cellspacing="0" class="PSLEVEL1GRIDNBO" id="HRS_CE_JO_EXT_I$scroll$0" dir="ltr" cellpadding="0" cols="1"  ><tbody><tr><td align="center" valign="top"><div id="divgcHRS_CE_JO_EXT_I$0" style="width:632px;height:842px;" class="psprintgrid">
<table id="l0HRS_CE_JO_EXT_I$0" style="width:632px;" cellspacing="0" cellpadding="0"><tbody><tr><td>
<div id="divgbHRS_CE_JO_EXT_I$0" style="width:632px;height:842px; " class="psprintgrid"><table id="gbHRS_CE_JO_EXT_I$0" style="overflow:hidden;" class="psprintgrid" dir="ltr" border="0" cellpadding="0" cellspacing="0" >
<tbody><tr><td class="psprintRightTd" style="width:632px;" valign="top">
<div id="divgbrHRS_CE_JO_EXT_I$0" style="width:632px;height:812px; overflow-x:hidden;overflow-y:scroll;" class="psprintgrid" onscroll="ptGridObj_win0.doOnScroll('HRS_CE_JO_EXT_I$0',1);"><table id="tdgbrHRS_CE_JO_EXT_I$0" border="0" style="width:614px;" dir="ltr" cellpadding="2" cellspacing="0" cols="1"><tbody><tr id="trHRS_CE_JO_EXT_I$0_row1" onclick="HighLightTR('rgb(173,216,230)','','trHRS_CE_JO_EXT_I$0_row1');" onmouseover="hoverLightTR('rgb(196,194,189)','',0,'trHRS_CE_JO_EXT_I$0_row1');" onmouseout="hoverLightTR('rgb(196,194,189)','',1,'trHRS_CE_JO_EXT_I$0_row1');"><td align="center" id="tdHRS_CE_JO_EXT_I$0#0"  class="PSLEVEL1GRIDROW">
<div id="win0divHTMLAREA$0"><div style="width:563px; ">
<!-- Begin HTML Area Name Undisclosed -->
<!--
* Project:   Project Octane
* Developer: Remylou Sioson
* Date:      November 2011
* Details:   HTML for the layout of Job postings in Careers page.
-->
<table>
<tbody><tr>
<td class="PSHYPERLINK"  align="left"><a href="javascript:submitAction_win0(document.win0,'#ICSetFieldHRS_CE_HM_PRE.POSTING_TITLE$0.1');" title="Job Title">Business Analyst</a></td>
<td class="PSEDITBOX_DISPONLY"  align="left">Auckland City Campus</td><td class="PSEDITBOX_DISPONLY"  align="left">25/09/2018</td></tr>
<tr><td class="PSEDITBOX_DISPONLY" colspan="3"  align="left"><p><span style="font-family:arial,sans-serif"><span style="font-size:9.0pt">We are seeking a Business Analyst to support key projects and initiatives across the University, enabling key delivery teams, projects and initiatives to improve and innovate the University’s services.</span></span></p></td>
</tr>
</tbody></table>
<!-- End HTML Area -->
</div>
</div>
</td>
</tr><tr id="trHRS_CE_JO_EXT_I$0_row2" onclick="HighLightTR('rgb(173,216,230)','','trHRS_CE_JO_EXT_I$0_row2');" onmouseover="hoverLightTR('rgb(196,194,189)','',0,'trHRS_CE_JO_EXT_I$0_row2');" onmouseout="hoverLightTR('rgb(196,194,189)','',1,'trHRS_CE_JO_EXT_I$0_row2');"><td align="center"  class="PSLEVEL1GRIDROW">
<div id="win0divHTMLAREA$1"><div style="width:563px; ">
<!-- Begin HTML Area Name Undisclosed -->
<!--
* Project:   Project Octane
* Developer: Remylou Sioson
* Date:      November 2011
* Details:   HTML for the layout of Job postings in Careers page.
-->
<table>
<tbody><tr>
<td class="PSHYPERLINK"  align="left"><a href="javascript:submitAction_win0(document.win0,'#ICSetFieldHRS_CE_HM_PRE.POSTING_TITLE$1.1');" title="Job Title">Senior Lecturer in Clinical Pharmacology (part time)</a></td>
<td class="PSEDITBOX_DISPONLY"  align="left">Grafton Campus</td><td class="PSEDITBOX_DISPONLY"  align="left">07/10/2018</td></tr>
<tr><td class="PSEDITBOX_DISPONLY" colspan="3"  align="left"><p><span style="font-family:arial,sans-serif">Alongside our full time Senior Lecturer/Associate Professor opportunity, we seek to appoint a part time Senior Lecturer in Clinical Pharmacology at approximately 16 hours per week.<span style="font-size:9.0pt"> </span></span></p></td>
</tr>
</tbody></table>
<!-- End HTML Area -->
</div>
</div>
</td>
</tr><tr id="trHRS_CE_JO_EXT_I$0_row3" onclick="HighLightTR('rgb(173,216,230)','','trHRS_CE_JO_EXT_I$0_row3');" onmouseover="hoverLightTR('rgb(196,194,189)','',0,'trHRS_CE_JO_EXT_I$0_row3');" onmouseout="hoverLightTR('rgb(196,194,189)','',1,'trHRS_CE_JO_EXT_I$0_row3');"><td align="center"  class="PSLEVEL1GRIDROW">
<div id="win0divHTMLAREA$2"><div style="width:563px; ">
<!-- Begin HTML Area Name Undisclosed -->
<!--
* Project:   Project Octane
* Developer: Remylou Sioson
* Date:      November 2011
* Details:   HTML for the layout of Job postings in Careers page.
-->
<table>
<tbody><tr>
<td class="PSHYPERLINK"  align="left"><a href="javascript:submitAction_win0(document.win0,'#ICSetFieldHRS_CE_HM_PRE.POSTING_TITLE$2.1');" title="Job Title">Research Fellow</a></td>
<td class="PSEDITBOX_DISPONLY"  align="left">Tamaki Campus</td><td class="PSEDITBOX_DISPONLY"  align="left">24/09/2018</td></tr>
<tr><td class="PSEDITBOX_DISPONLY" colspan="3"  align="left"><p><span style="font-family:arial,sans-serif"><span style="font-size:9.0pt">On offer: an exciting opportunity to join the internationally-renowned NIHI nutrition team in the role of Research Fellow. </span></span></p></td>
</tr>
</tbody></table>
<!-- End HTML Area -->
</div>
</div>
</td>
</tr><tr id="trHRS_CE_JO_EXT_I$0_row4" onclick="HighLightTR('rgb(173,216,230)','','trHRS_CE_JO_EXT_I$0_row4');" onmouseover="hoverLightTR('rgb(196,194,189)','',0,'trHRS_CE_JO_EXT_I$0_row4');" onmouseout="hoverLightTR('rgb(196,194,189)','',1,'trHRS_CE_JO_EXT_I$0_row4');"><td align="center"  class="PSLEVEL1GRIDROW">
<div id="win0divHTMLAREA$3"><div style="width:563px; ">
<!-- Begin HTML Area Name Undisclosed -->
<!--
* Project:   Project Octane
* Developer: Remylou Sioson
* Date:      November 2011
* Details:   HTML for the layout of Job postings in Careers page.
-->
<table>
<tbody><tr>
<td class="PSHYPERLINK"  align="left"><a href="javascript:submitAction_win0(document.win0,'#ICSetFieldHRS_CE_HM_PRE.POSTING_TITLE$3.1');" title="Job Title">Regulatory Approvals Administrator- Fixed-term</a></td>
<td class="PSEDITBOX_DISPONLY"  align="left">Auckland City Campus</td><td class="PSEDITBOX_DISPONLY"  align="left">07/10/2018</td></tr>
<tr><td class="PSEDITBOX_DISPONLY" colspan="3"  align="left"><p><span style="font-size:12px"><span style="font-family:arial,helvetica,sans-serif"><em><span style="color:black">Are you talented administrator looking for a part-time, fixed-term role?</span></em></span></span></p><p><span style="font-size:12px"><span style="font-family:arial,helvetica,sans-serif"><em><span style="color:black">Are you interested in applying your skills in the dynamic tertiary environment?</span></em></span></span></p></td>
</tr>
</tbody></table>
<!-- End HTML Area -->
</div>
</div>
</td>
</tr><tr id="trHRS_CE_JO_EXT_I$0_row5" onclick="HighLightTR('rgb(173,216,230)','','trHRS_CE_JO_EXT_I$0_row5');" onmouseover="hoverLightTR('rgb(196,194,189)','',0,'trHRS_CE_JO_EXT_I$0_row5');" onmouseout="hoverLightTR('rgb(196,194,189)','',1,'trHRS_CE_JO_EXT_I$0_row5');"><td align="center"  class="PSLEVEL1GRIDROW">
<div id="win0divHTMLAREA$4"><div style="width:563px; ">

【讨论】：

谢谢，看起来这样可以在该页面上获取我需要的信息，所以我应该能够修改它来做我想做的事情。在这个答案中不必要地使用选项只会让事情变得混乱。我正在删除它们。【参考方案2】：

我曾经遇到过类似的问题，帮助我的是使用 PhantomJS() 而不是 Chrome()（尽管 Selenium 对 PhantomJS 的支持已被弃用）：

from selenium import webdriver

browser = webdriver.PhantomJS()
browser.get("https://www.opportunities.auckland.ac.nz")
print(browser.page_source)

【讨论】：

查看我对原始问题的编辑。这似乎不起作用。不过感谢您的建议。是的，我得到了同样的警告，但是如果你稍等片刻，page_source 会正确加载等待 15 秒让我得到源，但它没有加载额外的内容（即它与在浏览器中查看页面源相同）所以仍然没有乐趣。再次感谢您的建议。

以上是关于使用 Selenium webdriver 和 Python 获得完全渲染的 HTML的主要内容，如果未能解决你的问题，请参考以下文章

使用.p12证书验证Selenium WebDriver（Java）

如何使用 C# 在 Selenium WebDriver (Selenium 2) 中最大化浏览器窗口？

使用httpclient 调用selenium webdriver

selenium webdriver

webdriver

Selenium webdriver实现截图功能