拒绝访问 - EMR Presto - 基于文件的授权

Posted 2023-03-21

【中文标题】拒绝访问 - EMR Presto - 基于文件的授权

【英文标题】：Access denied - EMR Presto - File Based Authorization

【发布时间】：2020-02-03 10:46:10

【问题描述】：

我在从 Presto (AWS EMR) 查询时遇到了一个奇怪的问题。我使用的是 Presto 0.194，一切正常，升级到 0.224 后，我无法运行查询。我正在使用 LDAP 身份验证进行 presto，还使用 ​​authentication.json 文件对 Hive 进行文件基础授权。我正在使用在旧版本中运行良好的相同 json 文件。任何帮助将不胜感激。

错误： 查询 20191005_104119_00006_3snge 失败：访问被拒绝：视图所有者“用户名”无法创建从中选择的视图...

config.propertis：

coordinator=true
node-scheduler.include-coordinator=false
discovery.uri=http://IP.ap-southeast-1.compute.internal:8889
http-server.threads.max=500
discovery-server.enabled=true
sink.max-buffer-size=1GB
query.max-memory=30GB
query.max-memory-per-node=6532645258B
query.max-total-memory-per-node=7839174309B
query.max-history=40
query.min-expire-age=30m
http-server.http.port=8889
http-server.log.path=/var/log/presto/http-request.log
http-server.log.max-size=67108864B
http-server.log.max-history=5
log.max-size=268435456B
log.max-history=5
query.execution-policy=phased
optimizer.dictionary-aggregation=true
optimizer.optimize-metadata-queries=true
colocated-joins-enabled=true
http-server.authentication.type=PASSWORD
http-server.https.enabled=true
http-server.https.port=9443
http-server.https.keystore.path=/etc/presto/presto_keystore.jks
http-server.https.keystore.key=passw0rd
node-scheduler.max-splits-per-node=125
optimizer.use-mark-distinct=false 

hive.properties：

hive.metastore-refresh-interval=1m
connector.name=hive-hadoop2
hive.metastore.uri=thrift://ip-10-0-2-141.ap-southeast- 
1.compute.internal:9083
hive.metastore-cache-ttl=20m
hive.config.resources=/etc/hadoop/conf/core- 
site.xml,/etc/hadoop/conf/hdfs-site.xml
hive.non-managed-table-writes-enabled = true
hive.s3-file-system-type = EMRFS
hive.hdfs.authentication.type = NONE
hive.hdfs.impersonation.enabled = true
hive.orc.bloom-filters.enabled=true
hive.recursive-directories=true
hive.s3select-pushdown.enabled=true
hive.security=file
security.config-file=/etc/presto/conf.dist/authorization.json

授权.json：

"schemas": [

  "user": "prestoSA",
  "owner": true
,

  "user": "marketing_jack",
  "owner": true
,

  "user": "system-apiquery",
  "owner": true
,

  "user": "redash",
  "owner": true
,

  "user": "system_.*",
  "schema": "prestosync_.*",
  "owner": true
,

  "user": "system_.*",
  "schema": "views_.*",
  "owner": true
,

  "user": "system_.*",
  "schema": "raw_.*",
  "owner": true

],
"tables": [

  "user": "prestoSA",
  "privileges": [
    "SELECT",
    "INSERT",
    "DELETE",
    "OWNERSHIP"
  ]
,

  "user": "redash",
  "privileges": [
    "SELECT"
  ]
,

  "schema": "raw_.*",
  "user": "system_.*",
  "privileges": [
    "SELECT",
    "INSERT",
    "DELETE",
    "OWNERSHIP"
  ]
,

  "schema": "production_.*",
  "user": "system_.*",
  "privileges": [
    "SELECT",
    "INSERT",
    "DELETE",
    "OWNERSHIP"
  ]
,

  "schema": "prestosync_.*",
  "user": "system_.*",
  "privileges": [
    "SELECT",
    "INSERT",
    "DELETE",
    "OWNERSHIP"
  ]
,

  "schema": "views_.*",
  "user": "system_.*",
  "privileges": [
    "SELECT",
    "INSERT",
    "DELETE",
    "OWNERSHIP"
  ]
,

  "schema": ".*dev",
  "user": "developer_.*",
  "privileges": [
    "SELECT"
  ]
,

  "schema": "raw_rin",
  "user": "developer_.*",
  "privileges": [
    "SELECT"
  ]
,

  "schema": ".*prod",
  "user": "developer_.*",
  "privileges": [
    "SELECT"
  ]
,

  "schema": "views_development_.*",
  "user": "marketing_.*",
  "privileges": [
    "SELECT"
  ]
,

  "schema": "views_prod",
  "user": "marketing_.*",
  "privileges": [
    "SELECT"
  ]
,

  "schema": "views_dev",
  "user": "sales_.*",
  "privileges": [
    "SELECT"
  ]
,

  "schema": "views_prod",
  "user": "sales_.*",
  "privileges": [
    "SELECT"
  ]
,

  "schema": "emr59_prod",
  "user": "marketing_.*",
  "privileges": [
    "SELECT"
  ]
,

  "schema": "views_dev",
  "user": "management_.*",
  "privileges": [
    "SELECT"
  ]
,

  "schema": "views_prod",
  "user": "management_.*",
  "privileges": [
    "SELECT"
  ]
,

  "schema": "views_dev",
  "user": "management_.*",
  "privileges": [
    "SELECT"
  ]
,

  "schema": "views_prod",
  "user": "management_.*",
  "privileges": [
    "SELECT"
  ]

]

访问控制属性：

access-control.name=file
security.config-file=/etc/presto/conf.dist/rules.json

rules.json:

"catalogs": [
 
  "user": "system_.*",
  "catalog": "(mysql|system)",
  "allow": true
,

  "user": "prestoSA",
  "catalog": "(mysql|system)",
  "allow": true
,

  "user": "redash",
  "catalog": "(mysql|system)",
  "allow": true
,

  "user": "developer_.*",
  "catalog": "(mysql|hive)",
  "allow": true
,

  "catalog": "hive",
  "allow": true
,

  "catalog": "system",
  "allow": false

]

【问题讨论】：

您没有上传任何authorization.json 文件，配置文件只是错误，我该如何帮助您？

@Lamanus 感谢您的回复。我添加了配置。

【参考方案1】：

错误：查询 20191005_104119_00006_3snge 失败：访问被拒绝：视图所有者“用户名”无法创建从中选择的视图...

这意味着username 在特定表上没有GRANT_SELECT 权限。

在 0.199 版本中影响您的特定更改： https://github.com/prestosql/presto/commit/6ed1ed88083baef1d29171364297631962adf05d 这是一个错误修复（创建视图应该需要不同的权限），因此更改不保持向后兼容性是有意的（尽管不方便）。

顺便说一句 对于不太可能对 SO 社区有益的一次性故障排除式问题，我建议使用Presto Community Slack 上的#troubleshooting 频道

【讨论】：

感谢 Piotr 的回复！ 1. 看来我需要邀请才能加入 Slack 工作区。 ([e-mail]) 2. 正如我在问题中提到的，我们对 hive 使用文件授权方法，并且所有权限都在 authorization.json 文件中可用。具有相同内容的相同文件在旧版本中工作。

@ahmokhtari re 1: prestosql.io/slack.html 有一个注册链接，但我已向您发送了个人邀请。回复 2：我添加了对提交的引用，该提交更改了要求您更新配置的行为。