使用 cURL 使用用户名和密码登录 Gitlab

Posted 2023-03-10

技术标签:

【中文标题】使用 cURL 使用用户名和密码登录 Gitlab【英文标题】：Login to Gitlab with username and password using cURL 【发布时间】：2018-06-05 13:02:12 【问题描述】：

为了在 Docker 中针对 Gitlab 实例测试命令行工具，我想使用用户名和密码登录 Gitlab 并获取创建的会话来验证我的 API 请求。

因此我做了以下事情：

curl -i http://localhost:8080/users/sign_in -s

_gitlab_session

authenticity_token


curl 'http://localhost:8080/users/sign_in' \
  -H "_gitlab_session=$cookie" \
  -H 'Origin: http://localhost:8080' \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8' \
  -H 'Referer: http://localhost:8080/users/sign_in' \
  -H 'Connection: keep-alive' \
  --data-urlencode "authenticity_token=$token" \
  --data-urlencode "user[login]=root" \
  --data-urlencode "user[password]=12341234" \
  --data-urlencode "user[remember_me]=0"

但我得到的不是有效的用户登录，而是

422 - The change you requested was rejected.

在日志文件中，我看到了

==> gitlab-rails/production.log <==
Started POST "/users/sign_in" for 172.17.0.1 at 2017-12-23 00:22:16 +0000
Processing by SessionsController#create as HTML
Parameters: "authenticity_token"=>"[FILTERED]", "user"=>"login"=>"root", "password"=>"[FILTERED]", "remember_me"=>"0"
Can't verify CSRF token authenticity
Completed 422 Unprocessable Entity in 125ms (ActiveRecord: 7.8ms)

==> gitlab-rails/production_json.log <==
"method":"POST","path":"/users/sign_in","format":"html","controller":"SessionsController",  
"action":"create","status":422,"error":" 
ActionController::InvalidAuthenticityToken:ActionController::InvalidAuthenticityToken",
"duration":126.29,"view":0.0,"db":7.78,"time":"2017-12-23T00:22:16.039Z",
"params":"authenticity_token":"[FILTERED]","user":"login":"root","password":"
[FILTERED]","remember_me":"0","remote_ip":"172.17.0.1",
"user_id":1,"username":"root"

==> gitlab-rails/production.log <==

ActionController::InvalidAuthenticityToken (ActionController::InvalidAuthenticityToken):
  lib/gitlab/middleware/multipart.rb:93:in `call'
  lib/gitlab/request_profiler/middleware.rb:14:in `call'
  lib/gitlab/middleware/go.rb:18:in `call'
  lib/gitlab/etag_caching/middleware.rb:11:in `call'
  lib/gitlab/middleware/read_only.rb:31:in `call'
  lib/gitlab/request_context.rb:18:in `call'
  lib/gitlab/metrics/requests_rack_middleware.rb:27:in `call'

我假设我忘记在第二个请求中传递一个必要的成分 - 但我不知道是哪一个。

【问题讨论】：

错误是说您的 CSRF (Authenticity) 令牌无效。您能否仔细检查您是否正确提取它？在仔细检查之后，您可以尝试使用 cookie jar - mankier.com/1/curl#--cookie-jar - 而不是手动包含标题吗？这个脚本可能会激发你的灵感：gist.github.com/vitalyisaev2/215f890e75252cd36794221c2debf365（关于***.com/questions/47423024/…的完整讨论） 【参考方案1】：

$cookie 到底是什么？在我的测试中，登录页面有 4 个 cookie，其中 3 个看起来像令牌，如果成功登录还需要除 _gitlab_session cookie 之外的至少 1 个 cookie，我不会感到惊讶。但与其费心寻找所需cookie的正确组合，我认为让curl自动处理cookie会更好，例如--cookie-jar和--cookie（那么所有的cookie都应该是正确的） - 还有更多，那里登录页面上有 3 个不同的authenticity_token，你确定你取的是正确的吗？也许您的问题是您获取了错误的令牌。区分这 3 个标记非常困难，我建议改用脚本语言。

这是一个使用 hhb_curl 的 php 测试工作示例（只需替换第 10 和 11 行的用户名和密码）：

<?php
declare(strict_types = 1);
require_once ('hhb_.inc.php');
$hc = new hhb_curl ( '', true );
$html = $hc->exec ( 'https://gitlab.com/users/sign_in' )->getStdOut ();
$domd = @DOMDocument::loadHTML ( $html );
$inputs = getDOMDocumentFormInputs ( $domd, true, false ) ['new_user'];
// var_dump ( $inputs );
$inputs ['user[login]'] = '???';
$inputs ['user[password]'] = '???';
$html = $hc->setopt_array ( array (
        CURLOPT_POST => 1,
        CURLOPT_POSTFIELDS => http_build_query ( $inputs ) 
) )->exec ()->getStdOut ();
echo $html;
$domd = @DOMDocument::loadHTML ( $html );
$xp = new DOMXPath ( $domd );
foreach ( $xp->query ( '//div[contains(@class,"flash-alert")]' ) as $loginError ) 
    var_dump ( "login error: ", trim($loginError->textContent) );


function getDOMDocumentFormInputs(\DOMDocument $domd, bool $getOnlyFirstMatches = false, bool $getElements = true): array 
    // :DOMNodeList?
    if (! $getOnlyFirstMatches && ! $getElements) 
        throw new \InvalidArgumentException ( '!$getElements is currently only implemented for $getOnlyFirstMatches (cus im lazy and nobody has written the code yet)' );
    
    $forms = $domd->getElementsByTagName ( 'form' );
    $parsedForms = array ();
    $isDescendantOf = function (\DOMNode $decendant, \DOMNode $ele): bool 
        $parent = $decendant;
        while ( NULL !== ($parent = $parent->parentNode) ) 
            if ($parent === $ele) 
                return true;
            
        
        return false;
    ;
    // i can't use array_merge on DOMNodeLists :(
    $merged = function () use (&$domd): array 
        $ret = array ();
        foreach ( $domd->getElementsByTagName ( "input" ) as $input ) 
            $ret [] = $input;
        
        foreach ( $domd->getElementsByTagName ( "textarea" ) as $textarea ) 
            $ret [] = $textarea;
        
        foreach ( $domd->getElementsByTagName ( "button" ) as $button ) 
            $ret [] = $button;
        
        return $ret;
    ;
    $merged = $merged ();
    foreach ( $forms as $form ) 
        $inputs = function () use (&$domd, &$form, &$isDescendantOf, &$merged): array 
            $ret = array ();
            foreach ( $merged as $input ) 
                // hhb_var_dump ( $input->getAttribute ( "name" ), $input->getAttribute ( "id" ) );
                if ($input->hasAttribute ( "disabled" )) 
                    // ignore disabled elements?
                    continue;
                
                $name = $input->getAttribute ( "name" );
                if ($name === '') 
                    // echo "inputs with no name are ignored when submitted by mainstream browsers (presumably because of specs)... follow suite?", PHP_EOL;
                    continue;
                
                if (! $isDescendantOf ( $input, $form ) && $form->getAttribute ( "id" ) !== '' && $input->getAttribute ( "form" ) !== $form->getAttribute ( "id" )) 
                    // echo "this input does not belong to this form.", PHP_EOL;
                    continue;
                
                if (! array_key_exists ( $name, $ret )) 
                    $ret [$name] = array (
                            $input 
                    );
                 else 
                    $ret [$name] [] = $input;
                
            
            return $ret;
        ;
        $inputs = $inputs (); // sorry about that, Eclipse gets unstable on IIFE syntax.
        $hasName = true;
        $name = $form->getAttribute ( "id" );
        if ($name === '') 
            $name = $form->getAttribute ( "name" );
            if ($name === '') 
                $hasName = false;
            
        
        if (! $hasName) 
            $parsedForms [] = array (
                    $inputs 
            );
         else 
            if (! array_key_exists ( $name, $parsedForms )) 
                $parsedForms [$name] = array (
                        $inputs 
                );
             else 
                $parsedForms [$name] [] = $tmp;
            
        
    
    unset ( $form, $tmp, $hasName, $name, $i, $input );
    if ($getOnlyFirstMatches) 
        foreach ( $parsedForms as $key => $val ) 
            $parsedForms [$key] = $val [0];
        
        unset ( $key, $val );
        foreach ( $parsedForms as $key1 => $val1 ) 
            foreach ( $val1 as $key2 => $val2 ) 
                $parsedForms [$key1] [$key2] = $val2 [0];
            
        
    
    if ($getElements) 
        return $parsedForms;
    
    $ret = array ();
    foreach ( $parsedForms as $formName => $arr ) 
        $ret [$formName] = array ();
        foreach ( $arr as $ele ) 
            $ret [$formName] [$ele->getAttribute ( "name" )] = $ele->getAttribute ( "value" );
        
    
    return $ret;

当前输出

string(13) "login error: " string(26) "Invalid Login or password."

意思是???/???不是有效的用户名和密码组合。

【讨论】：

虽然这个答案很可能是正确的，但我不能在我的情况下使用 PHP。无论如何：感谢您的全面回答！请参阅下面的帖子，我最终是如何使用 bash / perl 解决它的。 @MichaelLihs perl 应该完全有能力做同样的事情:)（编辑：现在我可以看到你的帖子了）【参考方案2】：

借助其他答案和 cmets 的一些有用提示，我终于想出了这个解决方案：

gitlab_host="http://localhost:8080"
gitlab_user="root"
gitlab_password="12341234"

# curl for the login page to get a session cookie and the sources with the auth tokens
body_header=$(curl -c cookies.txt -i "$gitlab_host/users/sign_in" -s)

# grep the auth token for the user login for
#   not sure whether another token on the page will work, too - there are 3 of them
csrf_token=$(echo $body_header | perl -ne 'print "$1\n" if /new_user.*?authenticity_token"[[:blank:]]value="(.+?)"/' | sed -n 1p)

# send login credentials with curl, using cookies and token from previous request
curl -b cookies.txt -c cookies.txt -i "$gitlab_host/users/sign_in" \
    --data "user[login]=$gitlab_user&user[password]=$gitlab_password" \
    --data-urlencode "authenticity_token=$csrf_token"

# send curl GET request to personal access token page to get auth token
body_header=$(curl -H 'user-agent: curl' -b cookies.txt -i "$gitlab_host/profile/personal_access_tokens" -s)
csrf_token=$(echo $body_header | perl -ne 'print "$1\n" if /authenticity_token"[[:blank:]]value="(.+?)"/' | sed -n 1p)

# curl POST request to send the "generate personal access token form"
# the response will be a redirect, so we have to follow using `-L`
body_header=$(curl -L -b cookies.txt "$gitlab_host/profile/personal_access_tokens" \
    --data-urlencode "authenticity_token=$csrf_token" \
    --data 'personal_access_token[name]=golab-generated&personal_access_token[expires_at]=&personal_access_token[scopes][]=api')

# Scrape the personal access token from the response HTML
personal_access_token=$(echo $body_header | perl -ne 'print "$1\n" if /created-personal-access-token"[[:blank:]]value="(.+?)"/' | sed -n 1p)

根据GitLab API documentation，您现在可以使用会话 cookie 来验证 API 请求：

curl --header "Private-Token: $personal_access_token" https://gitlab.example.com/api/v4/projects

一些提示：

我首先对 curl -c file（从标头读取 cookie 并将它们写入文件）与 curl -b file（使用文件中的 cookie 并将它们与请求一起发送）感到困惑很遗憾，我没有找到适用于 sed 的正则表达式，所以我必须在这里使用 perl。在 Chrome 中使用开发者控制台并将 POST 请求复制为 curl 命令非常有帮助：https://www.alexkras.com/copy-any-api-call-as-curl-request-with-chrome-developer-tools/

【讨论】：

很好的反馈，+1 现在，您需要在 curl 命令中添加 -L 以跟踪重定向。感谢 Michael 的有用回复！这在 Gitlab 使用 LDAP 进行身份验证时不起作用。【参考方案3】：

在 gitlab 中创建 access_token 或 private_token，你可以在 URL 中使用它。

bash

【讨论】：

【参考方案4】：

我有一个使用 Python 工作的稳定版本，requests 和 beautifulsoup4：

from urllib.parse import urljoin

import requests
import bs4

base_uri = 'http://gitlab.example.com/'
username = 'me'
password = 'let_me_in'

# Use a session so cookies are retained between requests.
session = requests.session()

# Load the login page to get a CSRF token.
response = session.get(urljoin(base_uri, 'users/sign_in'))
response.raise_for_status()

# Extract the CSRF token from the login page.
soup = bs4.BeautifulSoup(response.text, 'html.parser')
csrf_param = soup.find('meta', dict(name='csrf-param'))['content']
csrf_token = soup.find('meta', dict(name='csrf-token'))['content']

# Login.
request_data = 
    'username': username,
    'password': password,
    csrf_param: csrf_token
response = session.post(response.url, data=request_data)
response.raise_for_status()

# Get a list of project.
response = session.get(urljoin(base_uri, 'api/v4/projects'))
print(response.json())

【讨论】：

【参考方案5】：

感谢@Michael Lihs 的工作。我正在使用 gitlab V13，我不得不修改他的解决方案以使其适合我。这对我有用：

gitlab_host="http://localhost:8080"
gitlab_user="root"
gitlab_password="12341234"
# step 1 create session and get authenticity_token
rm cookies.txt
body_header=$(curl -c cookies.txt -k -i "$gitlab_host/users/sign_in" -s)
csrf_token=$(echo $body_header | perl -ne 'print "$1\n" if /new_user.*?authenticity_token"[[:blank:]]value="(.+?)"/' | sed -n 1p)

# step 2 identification
curl -b cookies.txt -c cookies.txt -L -i -k "$gitlab_host/users/auth/ldapmain/callback" \
    -d username=$gitlab_user -d password="$gitlab_password" \
    --data-urlencode "authenticity_token=$csrf_token"

# step 3 download files
rm /tmp/x3.pdf && curl -k "$gitlab_host/my_group/my_project/-/raw/my_branch/my_fullpath_to_file" -b cookies.txt  --compressed > /tmp/x3.pdf

【讨论】：

以上是关于使用 cURL 使用用户名和密码登录 Gitlab的主要内容，如果未能解决你的问题，请参考以下文章

gitlab的ssh登录密码没有怎么用

如何在gitlab.com中获得docker注册表用户名和密码？

如何使用 curl 登录 Stack Exchange？

gitlab的日常使用

将公钥添加到 gitlab 后仍然 cli 需要登录名和密码

如何使用 curl 登录 Graphene Django 应用程序？