HttpWebRequest 登录 - 在浏览器上启用 cookie

Posted 2023-02-19

技术标签:

【中文标题】HttpWebRequest 登录 - 在浏览器上启用 cookie【英文标题】：HttpWebRequest Login - Enabled cookies on Browser 【发布时间】：2021-10-13 10:48:27 【问题描述】：

我几天来一直在尝试使用 HttpWebRequest 登录网页。我已经完成了这样的项目，但我被这个项目卡住了。

我知道服务器可能需要一些 cookie 来处理请求，所以我创建了一个 CookieContainer。这是我的要求：

Public Function Login(ByVal loginPageAddress As String, ByVal loginData As NameValueCollection) As LoginStatus
        Dim container As CookieContainer = New CookieContainer
        Dim response As HttpWebResponse = Nothing
        Dim retval As LoginStatus = LoginStatus.NetworkError
        Try
            
            Dim request = CType(WebRequest.Create(loginPageAddress), HttpWebRequest)
            request.Method = "POST"
            request.ContentType = "application/x-www-form-urlencoded"
            request.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (Khtml, like Gecko) Chrome/91.0.4472.77 Safari/537.36"
            Dim query = String.Join("&", loginData.Cast(Of String)().[Select](Function(key) $"key=loginData(key)"))
            Dim buffer = System.Text.Encoding.ASCII.GetBytes(query)
            request.ContentLength = buffer.Length
            Dim requestStream = request.GetRequestStream()
            requestStream.Write(buffer, 0, buffer.Length)
            requestStream.Close()
            request.CookieContainer = New CookieContainer()
            container = request.CookieContainer
            response = request.GetResponse()
            Dim html As String = ""
            Using reader As New System.IO.StreamReader(response.GetResponseStream())
                html = reader.ReadToEnd()
            End Using
            Debug.Print(html)
            If html.Contains("Reports") Then
                retval = LoginStatus.OK
            Else
                retval = LoginStatus.CrendialsError
            End If
            response.Close()
        Catch ex As Exception
            If Not response Is Nothing Then
                response.Close()
            End If
            retval = LoginStatus.NetworkError
        End Try

        CookieContainer = container
        Return retval
    End Function

我没有运气，因为服务器响应“身份验证失败，请启用 cookie”

所以我想添加一个先前的请求来获取？cookie？。这样

Public Function Login(ByVal loginPageAddress As String, ByVal loginData As NameValueCollection) As LoginStatus
        Dim container As CookieContainer = New CookieContainer
        Dim response As HttpWebResponse = Nothing
        Dim retval As LoginStatus = LoginStatus.NetworkError
        Try
            'First request to get the cookies
            Dim requestCookies = CType(WebRequest.Create(loginPageAddress), HttpWebRequest)
            requestCookies.Method = "POST"
            requestCookies.ContentType = "application/x-www-form-urlencoded"
            requestCookies.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.77 Safari/537.36"
            Dim queryCookies = String.Join("&", loginData.Cast(Of String)().[Select](Function(key) $"key=loginData(key)"))
            Dim bufferCookies = System.Text.Encoding.ASCII.GetBytes(queryCookies)
            requestCookies.ContentLength = bufferCookies.Length
            Dim requestStreamCookies = requestCookies.GetRequestStream()
            requestStreamCookies.Write(bufferCookies, 0, bufferCookies.Length)
            requestStreamCookies.Close()
            requestCookies.CookieContainer = New CookieContainer()
   I tried to add this as well         'CookieContainer.Add(New Cookie("COOKIE_SUPPORT", "True"))

            container = requestCookies.CookieContainer
            response = requestCookies.GetResponse()
            Debug.Print("OK")

            'Login requests
            Dim request = CType(WebRequest.Create(loginPageAddress), HttpWebRequest)
            request.Method = "POST"
            request.ContentType = "application/x-www-form-urlencoded"
            request.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.77 Safari/537.36"
            Dim query = String.Join("&", loginData.Cast(Of String)().[Select](Function(key) $"key=loginData(key)"))
            Dim buffer = System.Text.Encoding.ASCII.GetBytes(query)
            request.ContentLength = buffer.Length
            Dim requestStream = request.GetRequestStream()
            requestStream.Write(buffer, 0, buffer.Length)
            requestStream.Close()
            request.CookieContainer = CookieContainer
            container = request.CookieContainer
            response = request.GetResponse()
            Dim html As String = ""
            Using reader As New System.IO.StreamReader(response.GetResponseStream())
                html = reader.ReadToEnd()
            End Using
            Debug.Print(html)
            If html.Contains("reports") Then
                retval = LoginStatus.OK
            Else
                retval = LoginStatus.CrendialsError
            End If
            response.Close()
        Catch ex As Exception
            If Not response Is Nothing Then
                response.Close()
            End If
            retval = LoginStatus.NetworkError
        End Try

        CookieContainer = container
        Return retval
    End Function

我用这个调用函数登录：

Public Function LoginWebsite() As String
        Dim retval As String = ""

        Dim loginAddress = "https://www.***********.com/home?p_auth=Fhc1hN0t&p_p_id=58&p_p_lifecycle=1&p_p_state=normal&p_p_mode=view&p_p_col_id=column-2&p_p_col_count=1&saveLastPath=0&_58_struts_action=%2Flogin%2Flogin&_58_doActionAfterLogin=false"
        Dim loginData = New NameValueCollection From 
        "_58_formDate", "1628523872372",
        "_58_redirect", "",
        "_58_login", "*******",
        "_58_password", "*******!"
       
        masterWebclient = New CookieAwareWebClient()
        Dim reply = masterWebclient.Login(loginAddress, loginData)
        If reply = LoginStatus.OK Then
            retval = "Logged In"
            isLoggedIn = True
        ElseIf reply = LoginStatus.CrendialsError Then
            retval = "User or password wrong"
        Else
            retval = "Network error"
        End If
        Return retval
    End Function

我所做的每一次尝试都是消极的。这是请求的标头。

Accept: text/html, application/xhtml+xml, image/jxr, */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US, en; q=0.8, es-ES; q=0.7, es; q=0.5, it-IT; q=0.3, it; q=0.2
Cache-Control: no-cache
Connection: Keep-Alive
Content-Length: 89
Content-Type: application/x-www-form-urlencoded
Cookie: NO_SPLASH_PUBLIC_SCREEN=true; JSESSIONID=98466D3D3A9259BDEBE8A294EA870CDC.GESPBEPSP02; COOKIE_SUPPORT=true; GUEST_LANGUAGE_ID=**; OptanonConsent=isIABGlobal=false&datestamp=Mon+Aug+09+2021+19%3A46%3A49+GMT%2B0100+(GMT+Daylight+Time)&version=6.16.0&consentId=fe6ac8d3-11a8-44d7-b7ae-23f47443f131&interactionCount=1&landingPath=NotLandingPage&groups=C0001%3A1%2CC0002%3A1%2CC0004%3A1&hosts=H110%3A1%2CH107%3A1%2CH1%3A1%2CH4%3A1%2CH10%3A1%2CH12%3A1; OptanonAlertBoxClosed=2021-08-09T18:46:49.040Z
Host: ******
Referer: https://*******/home?p_p_state=normal&p_p_mode=view&saveLastPath=0&_58_struts_action=%2Flogin%2Flogin&p_p_id=58&p_p_lifecycle=0&_58_redirect=%2F
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36 Edge/16.16299

这是表单数据：

_58_formDate: 1628524646492
_58_redirect: 
_58_login: *****
_58_password: ******

我真的被困在这几天了，如果有人能告诉我我做错了什么，请。

编辑：我曾尝试以这种方式添加请求 cookie，但没有成功。

Dim target As Uri = New Uri("https://********")
     request.CookieContainer.Add(target, New Cookie("NO_SPLASH_PUBLIC_SCREEN", "true"))
                request.CookieContainer.Add(target, New Cookie("LFR_SESSION_STATE_1282678", "1628523959195"))
                request.CookieContainer.Add(target, New Cookie("COOKIE_SUPPORT", "true"))
                request.CookieContainer.Add(target, New Cookie("GUEST_LANGUAGE_ID", "it_IT"))
                request.CookieContainer.Add(target, New Cookie("REMOTE_USER", "******"))
                request.CookieContainer.Add(target, New Cookie("PORTLET_LIFERAY", "TRUE"))
                request.CookieContainer.Add(target, New Cookie("NO_SPLASH_PRIVATE_SCREEN", "TRUE"))
                request.CookieContainer.Add(target, New Cookie("_gid", "GA1.2.1056978425.1628437222"))
                request.CookieContainer.Add(target, New Cookie("USER_UUID", "1crDatGEA9ueMkAYIKWtS8plWx72oazquLs3l9uCEEU"))
                request.CookieContainer.Add(target, New Cookie("TICKET", "17387333"))
                request.CookieContainer.Add(target, New Cookie("LFR_SESSION_STATE_1282678", "1628523959198"))
                request.CookieContainer.Add(target, New Cookie("JSESSIONID", "JSESSIONID=662DD1C24C7A41994F00DFAB7B91F9BD.GESPBEPSP01"))

编辑2：

建议我尝试使用基本身份验证。不工作，但可能我在代码上犯了一个错误。

   Dim request = CType(WebRequest.Create(loginPageAddress), HttpWebRequest)
    request.Method = "POST"
    request.ContentType = "application/x-www-form-urlencoded"
    Dim cred As New System.Net.NetworkCredential("*******", "*****","domain")  'where domain si may domain. 
    request.Credentials = cred
    request.CookieContainer = New CookieContainer()
    response = request.GetResponse()

【问题讨论】：

为什么不使用标准的身份验证程序并尝试重新发明***？我对此很陌生，我试图看看 vb.net 基本身份验证。我用我的身份验证代码编辑了这个问题，但不确定是否做对了。还是一样的问题首先我建议使用c#而不是VB。并先阅读官方文档docs.microsoft.com/en-us/aspnet/core/security/authentication/… 【参考方案1】：

我想，您尝试登录 Web 界面，但没有登录 API。所以这个服务器只是在另一个网页的某个地方发送cookie。在浏览器中查看正确的 Web 请求。

【讨论】：

以上是关于HttpWebRequest 登录 - 在浏览器上启用 cookie的主要内容，如果未能解决你的问题，请参考以下文章

HttpWebRequest 保存Cookies,模拟Session登录

使用HttpWebRequest时，为什么我在某些链接上出现“（304）Not Modified”错误？

HttpRequest 和HttpWebRequest的区别

asp 中有没有 httpwebrequest

c# WebBrowser登录后怎样直接用HttpWebRequest抓取数据

使用 HttpWebRequest 登录 Https 网站后的 Htmlagilitypack？