访问 $GITLAB/admin/runners 时出现内部服务器错误 500

Posted 2023-02-19

【中文标题】访问 $GITLAB/admin/runners 时出现内部服务器错误 500

【英文标题】：Internal Server Error 500 while accessing $GITLAB/admin/runners

【发布时间】：2019-06-10 12:46:06

【问题描述】：

我已经从备份中恢复了 Gitlab，现在每次我尝试访问 runners 网页时，都会收到内部服务器错误。

我已经尝试卸载所有配置的 gitlab-runners 并访问它，问题仍然存在。这是来自 gitlab-rails/production.log 的跟踪：

Started GET "/admin/runners" for 127.0.0.1 at 2019-01-16 07:17:12 -0500
Processing by Admin::RunnersController#index as html
Completed 500 Internal Server Error in 66ms (ActiveRecord: 5.7ms)

ActionView::Template::Error ():
    37: 
    38:     .col-sm-6
    39:       .bs-callout
    40:         = render partial: 'ci/runner/how_to_setup_runner',
    41:                  locals:  registration_token: Gitlab::CurrentSettings.runners_registration_token,
    42:                            type: 'shared',
    43:                            reset_token_url: reset_registration_token_admin_application_settings_path 

lib/gitlab/crypto_helper.rb:27:in `aes256_gcm_decrypt'
app/models/concerns/token_authenticatable_strategies/encrypted.rb:55:in `get_token'
app/models/concerns/token_authenticatable_strategies/base.rb:33:in `ensure_token!'
app/models/concerns/token_authenticatable.rb:43:in `block in add_authentication_token_field'
app/models/application_setting.rb:409:in `runners_registration_token'
lib/gitlab/current_settings.rb:19:in `method_missing'
app/views/admin/runners/index.html.haml:40:in `_app_views_admin_runners_index_html_haml___977288809002382166_69944849285200'
app/controllers/application_controller.rb:116:in `render'
lib/gitlab/i18n.rb:55:in `with_locale'
lib/gitlab/i18n.rb:61:in `with_user_locale'
app/controllers/application_controller.rb:420:in `set_locale'
lib/gitlab/middleware/multipart.rb:103:in `call'
lib/gitlab/request_profiler/middleware.rb:16:in `call'
lib/gitlab/middleware/go.rb:19:in `call'
lib/gitlab/etag_caching/middleware.rb:13:in `call'
lib/gitlab/middleware/correlation_id.rb:16:in `block in call'
lib/gitlab/correlation_id.rb:15:in `use_id'
lib/gitlab/middleware/correlation_id.rb:15:in `call'
lib/gitlab/middleware/read_only/controller.rb:40:in `call'
lib/gitlab/middleware/read_only.rb:18:in `call'
lib/gitlab/middleware/basic_health_check.rb:25:in `call'
lib/gitlab/request_context.rb:20:in `call'
lib/gitlab/metrics/requests_rack_middleware.rb:29:in `call'
lib/gitlab/middleware/release_env.rb:13:in `call'
Started GET "/-/metrics" for 127.0.0.1 at 2019-01-16 07:17:24 -0500
Processing by MetricsController#index as HTML
Completed 200 OK in 1ms (Views: 0.4ms | ActiveRecord: 0.0ms)

【问题讨论】：

那里没有足够的信息。看起来 Gitlab::CurrentSettings.runners_registration_token 有问题。在第 40 行放置一个 binding.pry 点，调用 Gitlab::CurrentSettings.runners_registration_token 并查看它返回的错误。

【参考方案1】：

我可以按照官方 gitlab 文档解决这个问题：

gitlab-rails dbconsole

-- Clear project tokens
UPDATE projects SET runners_token = null, runners_token_encrypted = null;

-- Clear group tokens
UPDATE namespaces SET runners_token = null, runners_token_encrypted = null;

-- Clear instance tokens
UPDATE application_settings SET runners_registration_token_encrypted = null;

-- Clear runner tokens
UPDATE ci_runners SET token = null, token_encrypted = null;

https://docs.gitlab.com/ee/raketasks/backup_restore.html#reset-runner-registration-tokens

编辑：

在清除现有管道作业（见上文）后，我仍然无法为一些已设置环境变量的迁移项目打开 ci 设置页面。在这种情况下，请尝试删除它们：

gitlab-rails dbconsole
SELECT * FROM ci_variables;
DELETE FROM ci_variables WHERE project_id='XX';

希望对您有所帮助。

【讨论】：

成功了！谢谢！对于 docker install 上的任何人，请使用docker exec -it gitlab_gitlab_1 bin/rails dbconsole 或更舒适的：docker exec -it gitlab_gitlab_1 bash，然后从 gitlab 目录 /home/git/gitlab：bin/rails dbconsole（和 postgres 密码可以在 /home/git/gitlab/config/database.yml 中找到）

【参考方案2】：

您进行了恢复，但可能没有恢复秘密存储（/etc/gitlab/gitlab-secrets.json 或 $GITLAB_HOME/config/secrets.json）。与gitlab 500 errors in the admin area相同的问题

你可以从 gitlab 服务器上的 rails 控制台这样重置它：

root@gitlab:/# gitlab-rails console
-------------------------------------------------------------------------------------
 GitLab:       11.5.1 (c90ae59)
 GitLab Shell: 8.4.1
 postgresql:   9.6.8
-------------------------------------------------------------------------------------
Both Deployment and its :status machine have defined a different default for "status". Use only one or the other for defining defaults to avoid unexpected behaviors.
Loading production environment (Rails 4.2.10)
irb(main):001:0> ApplicationSetting.current.reset_runners_registration_token!
=> true

【讨论】：

是的，我没有恢复配置。有没有办法可以重置它？我丢失了 secrets.json。

我已更新答案以显示重置令牌的方法

最好将其添加到文档中。

确保在之后键入 exit。我刚遇到这个问题，最初仍然出现 500 个错误，直到我输入“exit”退出 gitlab-rails 控制台。

除了：```错误：正在检查作业...禁止```

【参考方案3】：

我已经解决了，也许有用

    a. Reset CI/CD variables
    gitlab-rails dbconsole
    SELECT * FROM public."ci_group_variables";
    SELECT * FROM public."ci_variables";
    DELETE FROM ci_group_variables;
    DELETE FROM ci_variables;
    
    b. Reset runner registration tokens
    gitlab-rails dbconsole
    -- Clear project tokens
    UPDATE projects SET runners_token = null, runners_token_encrypted = null;
    -- Clear group tokens
    UPDATE namespaces SET runners_token = null, runners_token_encrypted = null;
    -- Clear instance tokens
    UPDATE application_settings SET runners_registration_token_encrypted = null;
    UPDATE application_settings SET encrypted_ci_jwt_signing_key = null;
    -- Clear runner tokens
    UPDATE ci_runners SET token = null, token_encrypted = null;
    
    c. Reset pending pipeline jobs
    sudo gitlab-rails dbconsole
    -- Clear build tokens
    UPDATE ci_builds SET token = null, token_encrypted = null;
    
    d. Fix project integrations
    gitlab-rails dbconsole
    -- truncate web_hooks table
    TRUNCATE web_hooks CASCADE;

来自gitlab