selenium-webdriver(python) (十四) -- webdriver原理(转载虫师自动化)

Posted

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了selenium-webdriver(python) (十四) -- webdriver原理(转载虫师自动化)相关的知识,希望对你有一定的参考价值。

selenium-webdriver(python) (十四) -- webdriver原理

2013-08-22 12:55 by 虫师, 13926 阅读, 12 评论, 收藏, 编辑

 

之前看乙醇视频中提到,selenium ruby 实现有一个小后门,在代码中加上$DEBUG=1 ,再运行脚本的过程中,就可以看到客户端请求的信息与服务器端返回的数据;觉得这个功能很强大,可以帮助理解webdriver的运行原理。

后来查了半天,python并没有提供这样一个方便的后门,不过我们可以通过代理的方式获得这些交互信息;

 

一、需要安装java 虚拟机与selenium-server-standalone ,参考 《selenium + python自动化测试环境搭建》第78操作:

 

二、通过下面命令启动服务:

C:\\selenium>java -jar selenium-server-standalone-2.33.0.jar

在命令结尾加 >d:\\log.txt 可以将命令信息存入文件,但信息很少。

 

 

然后运行下面的自动化脚本:

技术分享
#coding = utf-8

import time
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

driver = webdriver.Remote(desired_capabilities=DesiredCapabilities.CHROME)
driver.get("http://www.youdao.com")
driver.find_element_by_name("q").send_keys("hello")
driver.find_element_by_name("q").send_keys("key.ENTER")

driver.close()
技术分享

 

 

webdriver原理:

1. WebDriver 启动目标浏览器,并绑定到指定端口。该启动的浏览器实例,做为web driverremote server

2. Client 端通过CommandExcuter 发送HTTPRequest remote server 的侦听端口(通信协议: the webriver wire protocol

3. Remote server 需要依赖原生的浏览器组件(如:IEDriver.dll,chromedriver.exe),来转化转化浏览器的native调用。

 

 

查看命令提示符下的运行日志:

咋一看很乱,慢慢分析一下就发现很有意思!结合上面的脚本分析

技术分享
---------------------------------------------------------------------------------------
启动代理进入监听状态
C:\\selenium>java -jar selenium-server-standalone-2.33.0.jar
八月 22, 2013 10:19:48 上午 org.openqa.grid.selenium.GridLauncher main
INFO: Launching a standalone server
10:19:48.734 INFO - Java: Oracle Corporation 23.21-b01
10:19:48.734 INFO - OS: Windows XP 5.1 x86
10:19:48.734 INFO - v2.33.0, with Core v2.33.0. Built from revision 4e90c97
10:19:48.843 INFO - RemoteWebDriver instances should connect to: http://127.0.0.
1:4444/wd/hub
10:19:48.843 INFO - Version Jetty/5.1.x
10:19:48.843 INFO - Started HttpContext[/selenium-server/driver,/selenium-server
/driver]
10:19:48.843 INFO - Started HttpContext[/selenium-server,/selenium-server]
10:19:48.843 INFO - Started HttpContext[/,/]
10:19:48.890 INFO - Started [email protected]176343
e
10:19:48.890 INFO - Started HttpContext[/wd,/wd]
10:19:48.906 INFO - Started SocketListener on 0.0.0.0:4444
10:19:48.906 INFO - Started [email protected]
--------------------------------------------------------------------------------------

创建新session 10:20:38.593 INFO - Executing: [new session: {platform=ANY, javascriptEnabled=tr ue, browserName=chrome, version=}] at URL: /session) 10:20:38.593 INFO - Creating a new session for Capabilities [{platform=ANY, java scriptEnabled=true, browserName=chrome, version=}]
webdrivr通过GET方式发送请求 [0.921][INFO]: received Webriver request: GET /status
向webdrver返回响应,返回码200表示成功 [0.921][INFO]: sending Webriver response: 200 { "sessionId": "", "status": 0, "value": { "build": { "version": "alpha" }, "os": { "arch": "x86", "name": "Windows NT", "version": "5.1 SP3" } } }
webdriver 再次以POST方式发送请求,并启动浏览器相关信息 [0.984][INFO]: received Webriver request: POST /session { "desiredCapabilities": { "browserName": "chrome", "javascriptEnabled": true, "platform": "ANY", "version": "" } } [0.984][INFO]: Launching chrome: "C:\\ocuments and Settings\\Administrator\\Local S ettings\\Application ata\\Google\\Chrome\\Application\\chrome.exe" --remote-debugging -port=4223 --no-first-run --enable-logging --logging-level=1 --user-data-dir="C: \\OCUME~1\\AMINI~1\\LOCALS~1\\Temp\\scoped_dir1808_7550" --load-extension="C:\\OCUME~1 \\AMINI~1\\LOCALS~1\\Temp\\scoped_dir1808_26821\\internal" --ignore-certificate-error s data:text/html;charset=utf-8, [1.773][INFO]: sending Webriver response: 303
webdriver再次以GET方法请求,这附加上了session的信息 [1.778][INFO]: received Webriver request: GET /session/32b33aa585ccbbf7ba7853588 2852af3
服务器先对sesssionID进行解析,确认是selenium调用的以及要访问的网址, [1.779][INFO]: sending Webriver response: 200 { "sessionId": "32b33aa585ccbbf7ba78535882852af3", "status": 0, "value": { "acceptSslCerts": true, "applicationCacheEnabled": false, "browserConnectionEnabled": false, "browserName": "chrome", "chrome": { "chromedriverVersion": "2.0" }, "cssSelectorsEnabled": true, "databaseEnabled": true, "handlesAlerts": true, "javascriptEnabled": true, "locationContextEnabled": true, "nativeEvents": true, "platform": "Windows NT", "rotatable": false, "takesScreenshot": true, "version": "27.0.1453.116", "webStorageEnabled": true } } 10:20:40.640 INFO - Done: /session 10:20:40.640 INFO - Executing: org.openqa.selenium.remote.server.handler.GetSess [email protected] at URL: /session/ac5b2c71-5b1a-469e-814c-fdd09a2061fc) 10:20:40.640 INFO - Done: /session/ac5b2c71-5b1a-469e-814c-fdd09a2061fc 10:20:40.656 INFO - Executing: [get: http://www.youdao.com] at URL: /session/ac5 b2c71-5b1a-469e-814c-fdd09a2061fc/url) webdriver正试向服务器请求youdao网站 [1.820][INFO]: received Webriver request: POST /session/32b33aa585ccbbf7ba785358 82852af3/url { "url": "http://www.youdao.com" } [1.822][INFO]: waiting for pending navigations... [1.829][INFO]: done waiting for pending navigations [2.073][INFO]: waiting for pending navigations... [2.900][INFO]: done waiting for pending navigations 获得服务器数据的应答 [2.900][INFO]: sending Webriver response: 200 { "sessionId": "32b33aa585ccbbf7ba78535882852af3", "status": 0, "value": null } 10:20:41.734 INFO - Done: /session/ac5b2c71-5b1a-469e-814c-fdd09a2061fc/url -------------------------------------------------------------------------------------- 下面接着发送定位输入框的信息 10:20:41.734 INFO - Executing: [find element: By.name: q] at URL: /session/ac5b2 c71-5b1a-469e-814c-fdd09a2061fc/element) [2.905][INFO]: received Webriver request: POST /session/32b33aa585ccbbf7ba785358 82852af3/element { "using": "name", "value": "q" } [2.905][INFO]: waiting for pending navigations... [2.905][INFO]: done waiting for pending navigations [2.922][INFO]: waiting for pending navigations... [2.922][INFO]: done waiting for pending navigations 得到服务器应答 [2.922][INFO]: sending Webriver response: 200 { "sessionId": "32b33aa585ccbbf7ba78535882852af3", "status": 0, "value": { "ELEMENT": "0.19427558477036655:1" } } 10:20:41.765 INFO - Done: /session/ac5b2c71-5b1a-469e-814c-fdd09a2061fc/element 10:20:41.765 INFO - Executing: [send keys: 0 org.openqa.selenium.support.events. [email protected], [h, e, l, l, o]] at URL: /s ession/ac5b2c71-5b1a-469e-814c-fdd09a2061fc/element/0/value) 向定位到的输入框写入hello [2.936][INFO]: received Webriver request: POST /session/32b33aa585ccbbf7ba785358 82852af3/element/0.19427558477036655:1/value { "id": "0.19427558477036655:1", "value": [ "h", "e", "l", "l", "o" ] } [2.936][INFO]: waiting for pending navigations... [2.936][INFO]: done waiting for pending navigations [3.002][INFO]: waiting for pending navigations... [3.002][INFO]: done waiting for pending navigations [3.002][INFO]: sending Webriver response: 200 { "sessionId": "32b33aa585ccbbf7ba78535882852af3", "status": 0, "value": null } 10:20:41.843 INFO - Done: /session/ac5b2c71-5b1a-469e-814c-fdd09a2061fc/element/ 0/value 再次发送定位输入框的请求 10:20:41.843 INFO - Executing: [find element: By.name: q] at URL: /session/ac5b2 c71-5b1a-469e-814c-fdd09a2061fc/element) [3.006][INFO]: received Webriver request: POST /session/32b33aa585ccbbf7ba785358 82852af3/element { "using": "name", "value": "q" } [3.006][INFO]: waiting for pending navigations... [3.006][INFO]: done waiting for pending navigations [3.016][INFO]: waiting for pending navigations... [3.016][INFO]: done waiting for pending navigations [3.016][INFO]: sending Webriver response: 200 { "sessionId": "32b33aa585ccbbf7ba78535882852af3", "status": 0, "value": { "ELEMENT": "0.19427558477036655:1" } } 10:20:41.859 INFO - Done: /session/ac5b2c71-5b1a-469e-814c-fdd09a2061fc/element 10:20:41.859 INFO - Executing: [send keys: 0 org.openqa.selenium.support.events. [email protected], [k, e, y, ., E, N, T, E, R] ] at URL: /session/ac5b2c71-5b1a-469e-814c-fdd09a2061fc/element/0/value) 对定位的到的输入框发送回车(ENTER)事件请求 [3.021][INFO]: received Webriver request: POST /session/32b33aa585ccbbf7ba785358 82852af3/element/0.19427558477036655:1/value { "id": "0.19427558477036655:1", "value": [ "k", "e", "y", ".", "E", "N", "T", "E", "R" ] } [3.021][INFO]: waiting for pending navigations... [3.021][INFO]: done waiting for pending navigations [3.064][INFO]: waiting for pending navigations... [3.064][INFO]: done waiting for pending navigations [3.064][INFO]: sending Webriver response: 200 { "sessionId": "32b33aa585ccbbf7ba78535882852af3", "status": 0, "value": null } 10:20:41.906 INFO - Done: /session/ac5b2c71-5b1a-469e-814c-fdd09a2061fc/element/ 0/value 10:20:41.906 INFO - Executing: [close window] at URL: /session/ac5b2c71-5b1a-469 e-814c-fdd09a2061fc/window) [3.068][INFO]: received Webriver request: ELETE /session/32b33aa585ccbbf7ba78535 882852af3/window [WARNING:chrome_desktop_impl.cc(88)] chrome detaches, user should take care of d irectory:C:\\DOCUME~1\\ADMINI~1\\LOCALS~1\\Temp\\scoped_dir1808_7550 and C:\\DOCUME~1ADMINI~1\\LOCALS~1\\Temp\\scoped_dir1808_26821 [5.318][INFO]: sending Webriver response: 200 { "sessionId": "32b33aa585ccbbf7ba78535882852af3", "status": 0, "value": null } 10:20:44.156 INFO - Done: /session/ac5b2c71-5b1a-469e-814c-fdd09a2061fc/window
技术分享

 

request 请求 response 应答

一次请求会对应一次应答

POST/GET  是请求(request)两种类型;关于两种请求方式的类别参考其它资料

200 203 是 HTTP请求返回的状态码,200表示成功;

sessionid :每一个访问服务器的客户端,都要先得到服务器端分配的一个sessionid ,就像通行证一样,只有得到sessionid的客户端才能向服务器请求想要的数据。

  其它还包括操作系统版本,浏览器类型、URL、字符类型等非常详细的记录。熟悉HTTP ,了解TCP 的三次握手四次挥手,相信你对浏览器的交互与webdriver原理会有更深入的认识。







以上是关于selenium-webdriver(python) (十四) -- webdriver原理(转载虫师自动化)的主要内容,如果未能解决你的问题,请参考以下文章

selenium-webdriver(python) 页面操作:滚动缩放

(转载)selenium-webdriver(python)

轻松自动化---selenium-webdriver(python)

python selenium-webdriver 元素操作之键盘操作

轻松自动化---selenium-webdriver(python)

selenium-webdriver(python) (十四) -- webdriver原理(转载虫师自动化)