翻译(zh-CN):Apache Griffin API

Posted ESOO

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了翻译(zh-CN):Apache Griffin API相关的知识,希望对你有一定的参考价值。

此页面列出了Apache Griffin提供的主要RESTful API。
Apache Griffin默认的BASE_PATH是 http://<your ip>:8080.

HTTP响应设计

我们遵循一般规则来设计Apache Griffin的REST API。在发送到客户端的HTTP响应中,状态代码(三位数字)伴随着一个简单描述代码含义的原因短语(也称为状态文本)。状态代码按编号范围分类,每类代码具有相同的基本含义。


100-199  被归类为信息。
200-299  成功。
300-399  是重定向。
400-499  是客户端错误。
500-599  是服务器错误。

有效的Apache Griffin响应

有效的HTTP响应设计如下:

POST201, "Created"created item
GET200, "OK"requested items
PUT204, "No Content"no content
DELETE204, "No Content"no content


注意:度量(metric)模块使用elasticsearch bulk api实现,因此响应不遵循上述规则。
Apache Griffin响应无效

异常响应的设计如下:

ANY400, "Bad Request"error detail
ANY500, "Internal Server Error"error detail

 


    "timestamp": 1517208444322,
    "status": 400,
    "error": "Bad Request",
    "code": 40009,
    "message": "Property 'measure.id' is invalid",
    "path": "/api/v1/jobs"

 

    "timestamp": 1517209428969,
    "status": 500,
    "error": "Internal Server Error",
    "message": "Failed to add metric values",
    "exception": "java.net.ConnectException",
    "path": "/api/v1/metrics/values"

 

说明:
    timestamp:创建响应的时间戳
    status:HTTP状态代码
    error:HTTP状态的原因短语
    code:自定义错误代码
    message:自定义错误消息
    exception:原因异常的完全限定名称
    path:请求的api

请注意:如果由于客户端错误导致“exception”字段可能不存在,并且服务器错误可能不存在“code”字段。

 

基本信息(Apache Griffin Basic)

获取 Griffin 版本

GET /api/v1/version

调用示例

curl -k -H "Accept: application/json" -X GET http://127.0.0.1:8080/api/v1/version
0.3.0

 

 

数据度量(Griffin Measures)

新增 measure

POST /api/v1/measures

Request Header

Content-Typeapplication/json

调用示例

这里有两种度量, Apache Griffin measure 和 external measure.  'dq.type' 可以为 'ACCURACY' 或者 'PROFILING'.

以下是定义分析度量(measure of profiling)的示例 :

curl -k -H "Content-Type: application/json" -H "Accept: application/json" \\
-X POST http://127.0.0.1:8080/api/v1/measures \\
-d '
    "name":"profiling_measure",
    "measure.type":"griffin",
    "dq.type":"PROFILING",
    "rule.description":
        "details":[
            
                "name":"age",
                "infos":"Total Count,Average"
            
        ]
    ,
    "process.type":"BATCH",
    "owner":"test",
    "description":"measure description",
    "data.sources":[
        
            "name":"source",
            "connectors":[
                
                    "name":"connector_name",
                    "type":"HIVE",
                    "version":"1.2",
                    "data.unit":"1hour",
                    "data.time.zone":"UTC(WET,GMT)",
                    "config":
                        "database":"default",
                        "table.name":"demo_src",
                        "where":"dt=#YYYYMMdd# AND hour=#HH#"
                    ,
                    "predicates":[
                        
                            "type":"file.exist",
                            "config":
                                "root.path":"hdfs:///griffin/demo_src",
                                "path":"/dt=#YYYYMMdd#/hour=#HH#/_DONE"
                            
                        
                    ]
                
            ]
        
    ],
    "evaluate.rule":
        "rules":[
            
                "dsl.type":"griffin-dsl",
                "dq.type":"PROFILING",
                "rule":"count(source.`age`) AS `age-count`,avg(source.`age`) AS `age-average`",
                "name":"profiling",
                "details":
            
        ]
    
'

以下是定义准确度度量的示例(measure of accuracy) :

curl -k -H "Content-Type: application/json" -H "Accept: application/json" \\
-X POST http://127.0.0.1:8080/api/v1/measures \\
-d '
    "name":"accuracy_measure",
    "measure.type":"griffin",
    "dq.type":"ACCURACY",
    "process.type":"BATCH",
    "owner":"test",
    "description":"measure description",
    "data.sources":[
        
            "name":"source",
            "connectors":[
                
                    "name":"connector_name_source",
                    "type":"HIVE",
                    "version":"1.2",
                    "data.unit":"1hour",
                    "data.time.zone":"UTC(WET,GMT)",
                    "config":
                        "database":"default",
                        "table.name":"demo_src",
                        "where":"dt=#YYYYMMdd# AND hour=#HH#"
                    ,
                    "predicates":[
                        
                            "type":"file.exist",
                            "config":
                                "root.path":"hdfs:///griffin/demo_src",
                                "path":"/dt=#YYYYMMdd#/hour=#HH#/_DONE"
                            
                        
                    ]
                
            ]
        ,
        
            "name":"target",
            "connectors":[
                
                    "name":"connector_name_target",
                    "type":"HIVE",
                    "version":"1.2",
                    "data.unit":"1hour",
                    "data.time.zone":"UTC(WET,GMT)",
                    "config":
                        "database":"default",
                        "table.name":"demo_tgt",
                        "where":"dt=#YYYYMMdd# AND hour=#HH#"
                    ,
                    "predicates":[
                        
                            "type":"file.exist",
                            "config":
                                "root.path":"hdfs:///griffin/demo_src",
                                "path":"/dt=#YYYYMMdd#/hour=#HH#/_DONE"
                            
                        
                    ]
                
            ]
        
    ],
    "evaluate.rule":
        "rules":[
            
                "dsl.type":"griffin-dsl",
                "dq.type":"ACCURACY",
                "name":"accuracy",
                "rule":"source.desc=target.desc"
            
        ]
    
'

以下是定义外部度量的示例( external measure) :

curl -k -H "Content-Type: application/json" -H "Accept: application/json" \\
-X POST http://127.0.0.1:8080/api/v1/measures \\
-d '
    "name": "external_name",
    "measure.type": "external",
    "dq.type": "ACCURACY",
    "description": "measure description",
    "organization": "orgName",
    "owner": "test",
    "metric.name": "metricName"
'

获取 measure

GET /api/v1/measures
GET /api/v1/measures/measure_id

调用示例

curl -k -H "Accept: application/json" -X GET http://127.0.0.1:8080/api/v1/measures
[
        "measure.type": "griffin",
        "id": 1,
        "name": "accuracy_measure",
        "owner": "test",
        "description": "measure description",
        "deleted": false,
        "dq.type": "ACCURACY",
        "sinks": ["ELASTICSEARCH", "HDFS"],
        "process.type": "BATCH",
        "data.sources": [
                "id": 4,
                "name": "source",
                "connectors": [
                        "id": 5,
                        "name": "connector_name_source",
                        "type": "HIVE",
                        "version": "1.2",
                        "predicates": [
                                "id": 6,
                                "type": "file.exist",
                                "config": 
                                    "root.path": "hdfs:///127.0.0.1/demo_src",
                                    "path": "/dt=#YYYYMMdd#/hour=#HH#/_DONE"
                                
                            
                        ],
                        "data.unit": "1hour",
                        "data.time.zone": "UTC(WET,GMT)",
                        "config": 
                            "database": "default",
                            "table.name": "demo_src",
                            "where": "dt=#YYYYMMdd# AND hour=#HH#"
                        
                    
                ],
                "baseline": false
            , 
                "id": 7,
                "name": "target",
                "connectors": [
                        "id": 8,
                        "name": "connector_name_target",
                        "type": "HIVE",
                        "version": "1.2",
                        "predicates": [
                                "id": 9,
                                "type": "file.exist",
                                "config": 
                                    "root.path": "hdfs:///127.0.0.1/demo_src",
                                    "path": "/dt=#YYYYMMdd#/hour=#HH#/_DONE"
                                
                            
                        ],
                        "data.unit": "1hour",
                        "data.time.zone": "UTC(WET,GMT)",
                        "config": 
                            "database": "default",
                            "table.name": "demo_tgt",
                            "where": "dt=#YYYYMMdd# AND hour=#HH#"
                        
                    
                ],
                "baseline": false
            
        ],
        "evaluate.rule": 
            "id": 2,
            "rules": [
                    "id": 3,
                    "rule": "source.desc=target.desc",
                    "dsl.type": "griffin-dsl",
                    "dq.type": "ACCURACY"
                
            ]
        ,
        "measure.type": "griffin"
    
]

删除 measure

DELETE /api/v1/measures/measure_id When deleting a measure,api will also delete related jobs.

调用示例

curl -k -H "Accept: application/json" -X DELETE http://127.0.0.1:8080/api/v1/measures/1

The response body should be empty if no error happens, and the HTTP status is (204, "No Content").

更新 measure

PUT /api/v1/measures

调用示例

Here is an example to update measure:

curl -k -H "Content-Type: application/json" -H "Accept: application/json" \\
-X PUT http://127.0.0.1:8080/api/v1/measures \\
-d '
    "measure.type": "griffin",
    "id": 19,
    "name": "profiling_measure_edited",
    "owner": "test",
    "description": "measure description",
    "deleted": false,
    "dq.type": "PROFILING",
    "sinks": ["ELASTICSEARCH", "HDFS"],
    "process.type": "BATCH",
    "rule.description": 
        "details": [
                "name": "age",
                "infos": "Total Count,Average"
            
        ]
    ,
    "data.sources": [
            "id": 22,
            "name": "source",
            "connectors": [
                    "id": 23,
                    "name": "connector_name",
                    "type": "HIVE",
                    "version": "1.2",
                    "predicates": [
                            "id": 24,
                            "type": "file.exist",
                            "config": 
                                "root.path": "hdfs:///griffin/demo_src",
                                "path": "/dt=#YYYYMMdd#/hour=#HH#/_DONE"
                            
                        
                    ],
                    "data.unit": "1hour",
                    "data.time.zone": "UTC(WET,GMT)",
                    "config": 
                        "database": "default",
                        "table.name": "demo_src",
                        "where": "dt=#YYYYMMdd# AND hour=#HH#"
                    
                
            ],
            "baseline": false
        
    ],
    "evaluate.rule": 
        "id": 20,
        "rules": [
                "id": 21,
                "rule": "count(source.`age`) AS `age-count`,avg(source.`age`) AS `age-average`",
                "dsl.type": "griffin-dsl",
                "dq.type": "PROFILING",
                "details": 
            
        ]
    ,
    "measure.type": "griffin"
'

以下是更新外部度量的示例(external measure) :

curl -k -H "Content-Type: application/json" -H "Accept: application/json" \\
-X PUT http://127.0.0.1:8080/api/v1/measures \\
-d '
    "measure.type": "external",
    "id": 25,
    "name": "external_name",
    "owner": "test",
    "description": "measure description edited",
    "organization": "orgName",
    "deleted": false,
    "dq.type": "ACCURACY",
    "sinks": ["ELASTICSEARCH", "HDFS"],
    "metric.name": "metricName",
    "measure.type": "external"
'

数据度量任务(Griffin Jobs)

新增 job

POST /api/v1/jobs

调用示例

curl -k -H "Content-Type: application/json" -H "Accept: application/json" \\
-X POST http://127.0.0.1:8080/api/v1/jobs \\
-d '
    "measure.id": 10,
    "job.name":"job_name_10",
    "job.type":"batch",
    "cron.expression": "0 0/4 * * * ?",
    "cron.time.zone": "GMT+8:00",
    "predicate.config": 
        "checkdonefile.schedule":
            "interval": "1m",
            "repeat": 2
        
    ,
    "data.segments": [
        
            "data.connector.name": "connector_name_source",
            "as.baseline":true,
            "segment.range": 
                "begin": "-1h",
                "length": "1h"
            
        ,
        
            "data.connector.name": "connector_name_target",
            "segment.range": 
                "begin": "-1h",
                "length": "1h"
            
        
    ]
'

按照id触发 job

POST /api/v1/jobs/trigger/job_id

In the current version triggering the job in this way leads to scheduling of a single job instance. The method returns immediately even if starting it may take time. The response contains triggerKey by which the instance could be found when it is started (see find instance by trigger key).

调用示例

curl -k -X POST http://127.0.0.1:8080/api/v1/jobs/trigger/101

    "triggerKey": "DEFAULT.6da64b5bd2ee-34e2cb23-11a2-4f92-9cbd-6cb3402cdb48",

获取所有 jobs

GET /api/v1/jobs

调用示例

curl -k -H "Content-Type: application/json" -H "Accept: application/json" \\
-X GET http://127.0.0.1:8080/api/v1/jobs
[
        "job.type": "batch",
        "id": 51,
        "measure.id": 10,
        "job.name": "job_name_10",
        "metric.name": "job_name_10",
        "quartz.name": "job_name_10_1547192473206",
        "quartz.group": "BA",
        "cron.expression": "0 0/4 * * * ?",
        "job.state": 
            "state": "NORMAL",
            "toStart": false,
            "toStop": true,
            "nextFireTime": 1547693040000,
            "previousFireTime": 1547692800000
        ,
        "cron.time.zone": "GMT+8:00",
        "predicate.config": 
            "checkdonefile.schedule": 
                "interval": "1m",
                "repeat": 2
            
        ,
        "data.segments": [
                "id": 52,
                "data.connector.name": "connector_name_source",
                "as.baseline": true,
                "segment.range": 
                    "id": 53,
                    "begin": "-1h",
                    "length": "1h"
                
            , 
                "id": 54,
                "data.connector.name": "connector_name_target",
                "as.baseline": false,
                "segment.range": 
                    "id": 55,
                    "begin": "-1h",
                    "length": "1h"
                
            
        ],
        "job.type": "batch"
    
]

根据id获取job信息

GET /api/v1/jobs/config?jobId=job_id

调用示例

curl -k -H "Content-Type: application/json" -H "Accept: application/json" \\
-X GET http://127.0.0.1:8080/api/v1/jobs/config?jobId=827

    "job.type": "batch",
    "id": 827,
    "measure.id": 10,
    "job.name": "job_name_10",
    "metric.name": "job_name_10",
    "quartz.name": "job_name_10_1547694147531",
    "quartz.group": "BA",
    "cron.expression": "0 0/4 * * * ?",
    "cron.time.zone": "GMT+8:00",
    "predicate.config": 
        "checkdonefile.schedule": 
            "interval": "1m",
            "repeat": 2
        
    ,
    "data.segments": [
            "id": 828,
            "data.connector.name": "connector_name_source",
            "as.baseline": true,
            "segment.range": 
                "id": 829,
                "begin": "-1h",
                "length": "1h"
            
        , 
            "id": 830,
            "data.connector.name": "connector_name_target",
            "as.baseline": false,
            "segment.range": 
                "id": 831,
                "begin": "-1h",
                "length": "1h"
            
        
    ],
    "job.type": "batch"

根据 id 删除 job

DELETE /api/v1/jobs/job_id

调用示例

curl -k -H "Content-Type: application/json" -H "Accept: application/json" \\
-X DELETE http://127.0.0.1:8080/api/v1/jobs/51

根据名称(name)删除 job

DELETE /api/v1/jobs?jobName=name

调用示例

curl -k -H "Content-Type: application/json" -H "Accept: application/json" \\
-X DELETE http://127.0.0.1:8080/api/v1/jobs?jobName=job_name_10

The response body should be empty if no error happens, and the HTTP status is (204, "No Content").

分页获取 job 实例

GET /api/v1/jobs/instances?jobId=id&page=pageNum&size=pageSize

Request Parameter

jobIdjob idLong1
page页码,从0开始int0
size页内数据量大小int10

调用示例

curl -k -G -X GET http://127.0.0.1:8080/api/v1/jobs/instances -d jobId=827 -d page=1 -d size=5
[
        "id": 1176,
        "sessionId": null,
        "state": "NOT_FOUND",
        "type": "BATCH",
        "predicateGroup": "PG",
        "predicateName": "job_name_10_predicate_1547776800012",
        "timestamp": 1547776800012,
        "expireTimestamp": 1548381600012
    , 
        "id": 1175,
        "sessionId": null,
        "state": "NOT_FOUND",
        "type": "BATCH",
        "predicateGroup": "PG",
        "predicateName": "job_name_10_predicate_1547776560018",
        "timestamp": 1547776560019,
        "expireTimestamp": 1548381360019
    
]

根据triggerKey查找 job 实例

这可以在按作业ID触发作业之后使用,以在调度时查找作业实例。 在当前版本中,触发的实例不超过一个,因此响应是具有单个元素的列表(如果未找到则为空列表)。

GET /api/v1/jobs/triggerKeys/triggerKey

curl http://127.0.0.1:8080/api/v1/jobs/triggerKeys/DEFAULT.6da64b5bd2ee-34e2cb23-11a2-4f92-9cbd-6cb3402cdb48
[
    
        "id":201,
        "sessionId":1,
        "state":"SUCCESS",
        "type":"BATCH",
        "appId":"application_1554199833471_0002",
        "appUri":"http://localhost:38088/cluster/app/application_1554199833471_0002",
        "predicateGroup":"PG",
        "predicateName":"acc1a_name_predicate_1554202748883",
        "triggerKey":"DEFAULT.6da64b5bd2ee-34e2cb23-11a2-4f92-9cbd-6cb3402cdb49",
        "timestamp":1554202748884,
        "expireTimestamp":1554807548884
    
]

获取job的健康数据

GET /api/v1/jobs/health

调用示例

curl -k -X GET http://127.0.0.1:8080/api/v1/jobs/health

	"healthyJobCount": 0,
	"jobCount": 1

下载样本记录

GET /api/v1/jobs/download?jobName=name&ts=timestamp

Request Parameter

jobNamejob名称String1
timestamp时间Long0

调用示例

curl -k -G -X GET http://127.0.0.1:8080/api/v1/jobs/download \\
-d jobName=job_name_10 -d timestamp=1547778857807

如果成功,此方法返回响应正文中的缺失记录,最大记录数为100。

根据id获取job实例

GET /api/v1/jobs/instances/jobInstanceId

调用示例

curl -k -G -X GET http://127.0.0.1:8080/api/v1/jobs/instances/1

如果成功,此方法将返回给定作业实例标识的作业实例描述。 如果没有找到给定id的实例,则返回Griffin Exception。

数据计量-图表展示度量结果(Metrics)

获取 metrics

GET /api/v1/metrics

调用示例

响应是按度量名称分组的度量图。示例:

curl -k -X GET http://127.0.0.1:8080/api/v1/metrics

    "measure_no_predicate_day": [
        
            "name": "job_no_predicate_day",
            "type": "accuracy",
            "owner": "test",
            "metricValues": [
                
                    "name": "job_no_predicate_day",
                    "tmst": 1517994480000,
                    "value": 
                        "total": 125000,
                        "miss": 0,
                        "matched": 125000
                    
                ,
                
                    "name": "job_no_predicate_day",
                    "tmst": 1517994240000,
                    "value": 
                        "total": 125000,
                        "miss": 0,
                        "matched": 125000
                    
                
            ]
        
    ],
    "measure_predicate_hour": [
        
            "name": "job_predicate_hour",
            "type": "accuracy",
            "owner": "test",
            "metricValues": []
        
    ]

增加 metric 值

POST /api/v1/metrics/values

Request Body

Metric Values度量值集合(list)MetricValue

调用示例

curl -k -H "Content-Type: application/json" -H "Accept: application/json" \\
-X POST http://127.0.0.1:8080/api/v1/metrics/values \\
-d '[
    
        "name" : "metricName",
        "tmst" : 1509599811123,
        "value" : 
            "__tmst" : 1509599811123,
            "miss" : 11,
            "total" : 125000,
            "matched" : 124989
        
   
]'

例如,如果成功,响应正文应该将'errors'字段设置为'false'


    "took": 32,
    "errors": false,
    "items": [
        
            "index": 
                "_index": "griffin",
                "_type": "accuracy",
                "_id": "AWFAs5pOJwYEbKWP7mhq",
                "_version": 1,
                "result": "created",
                "_shards": 
                    "total": 2,
                    "successful": 1,
                    "failed": 0
                ,
                "created": true,
                "status": 201
            
        
    ]

根据名称获取 metric 值

GET /api/v1/metrics/values?metricName=name&size=size&offset=offset&tmst=timestamp

Request 参数

metricNamemetric名称Stringjob_no_predicate_day
size返回数据行数大小int5
offset按时间戳按降序跳过的记录数量int0
tmst您想要获取的记录的开始时间戳long0

offset 和 tmst 是非必填项目.

调用示例

curl -k -G -X GET http://127.0.0.1:8080/api/v1/metrics/values -d metricName=job_no_predicate_day -d size=10
[
    
        "name": "job_no_predicate_day",
        "tmst": 1517994720000,
        "value": 
            "total": 125000,
            "miss": 0,
            "matched": 125000
        
    ,
    
        "name": "job_no_predicate_day",
        "tmst": 1517994480000,
        "value": 
            "total": 125000,
            "miss": 0,
            "matched": 125000
        
    ,
    
        "name": "job_no_predicate_day",
        "tmst": 1517994240000,
        "value": 
            "total": 125000,
            "miss": 0,
            "matched": 125000
        
    
]

根据名称(name)删除 metric

DELETE /api/v1/metrics/values?metricName=name

调用示例

例如,如果成功,响应正文应该将“failures”字段设置为空

curl -k -H "Accept: application/json" \\
-X DELETE http://127.0.0.1:8080/api/v1/metrics/values?metricName=job_no_predicate_day

    "took": 363,
    "timed_out": false,
    "total": 5,
    "deleted": 5,
    "batches": 1,
    "version_conflicts": 0,
    "noops": 0,
    "retries": 
        "bulk": 0,
        "search": 0
    ,
    "throttled_millis": 0,
    "requests_per_second": -1,
    "throttled_until_millis": 0,
    "failures": []

根据Job 实例 Id 获取 Metric

GET http://127.0.0.1:8080/api/v1/metrics/values/:jobInstanceId

调用示例

curl -k -G -X GET http://127.0.0.1:8080/api/v1/metrics/values/304

    "name": "some_job",
    "tmst": 1553526960000,
    "value": 
        "total": 74,
        "miss": 31,
        "matched": 43,
        "matchedFraction": 0.581081081081081
    ,
    "metadata": 
        "applicationId": "\\"application_1549876136110_0237\\"",
    

Hive元数据(Hive MetaStore)

获取表元数据

GET /api/v1/metadata/hive/table?db=&table=

Request 参数

dbDB名Stringdefault
table表名Stringdemo_src

调用示例

curl -k -H "Accept: application/json" \\
-G -X GET http://127.0.0.1:8080/api/v1/metadata/hive/table \\
-d db=default \\
-d table=demo_src

    "tableName": "demo_src",
    "dbName": "default",
    "owner": "root",
    "createTime": 1505986176,
    "lastAccessTime": 0,
    "retention": 0,
    "sd": 
        "cols": [
            
                "name": "id",
                "type": "bigint",
                "comment": null,
                "setName": true,
                "setType": true,
                "setComment": false
            ,
            
                "name": "age",
                "type": "int",
                "comment": null,
                "setName": true,
                "setType": true,
                "setComment": false
            ,
            
                "name": "desc",
                "type": "string",
                "comment": null,
                "setName": true,
                "setType": true,
                "setComment": false
            
        ],
        "location": "hdfs://sandbox:9000/griffin/data/batch/demo_src"
    ,
    "partitionKeys": [
        
            "name": "dt",
            "type": "string",
            "comment": null,
            "setName": true,
            "setType": true,
            "setComment": false
        ,
        
            "name": "hour",
            "type": "string",
            "comment": null,
            "setName": true,
            "setType": true,
            "setComment": false
        
    ]

获取所有表名

GET /api/v1/metadata/hive/tables/names?db=

Request 参数

dbdb名Stringdefault

调用示例

curl -k -H "Accept: application/json" \\
-X GET http://127.0.0.1:8080/api/v1/metadata/hive/table?db=default
[
  "demo_src",
  "demo_tgt"
]

获取所有数据库表元数据

GET /api/v1/metadata/hive/dbs/tables

调用示例

curl -k -H "Accept: application/json" \\
-X GET http://127.0.0.1:8080/api/v1/metadata/hive/dbs/tables

   "default": [
    
      "tableName": "demo_src",
      "dbName": "default",
      "owner": "root",
      "createTime": 1505986176,
      "lastAccessTime": 0,
      "sd": 
        "cols": [
          
            "name": "id",
            "type": "bigint",
            "comment": null,
            "setComment": false,
            "setType": true,
            "setName": true
          ,
          
            "name": "age",
            "type": "int",
            "comment": null,
            "setComment": false,
            "setType": true,
            "setName": true
          ,
          
            "name": "desc",
            "type": "string",
            "comment": null,
            "setComment": false,
            "setType": true,
            "setName": true
          
        ],
        "location": "hdfs://sandbox:9000/griffin/data/batch/demo_src"
      ,
      "partitionKeys": [
        
          "name": "dt",
          "type": "string",
          "comment": null,
          "setComment": false,
          "setType": true,
          "setName": true
        ,
        
          "name": "hour",
          "type": "string",
          "comment": null,
          "setComment": false,
          "setType": true,
          "setName": true
        
      ]
    ,
    
      "tableName": "demo_tgt",
      "dbName": "default",
      "owner": "root",
      "createTime": 1505986176,
      "lastAccessTime": 0,
      "sd": 
        "cols": [
          
            "name": "id",
            "type": "bigint",
            "comment": null,
            "setComment": false,
            "setType": true,
            "setName": true
          ,
          
            "name": "age",
            "type": "int",
            "comment": null,
            "setComment": false,
            "setType": true,
            "setName": true
          ,
          
            "name": "desc",
            "type": "string",
            "comment": null,
            "setComment": false,
            "setType": true,
            "setName": true
          
        ],
        "location": "hdfs://sandbox:9000/griffin/data/batch/demo_tgt"
      ,
      "partitionKeys": [
        
          "name": "dt",
          "type": "string",
          "comment": null,
          "setComment": false,
          "setType": true,
          "setName": true
        ,
        
          "name": "hour",
          "type": "string",
          "comment": null,
          "setComment": false,
          "setType": true,
          "setName": true
        
      ]
    
  ]


获取所有db名称

GET /api/v1/metadata/hive/dbs

调用示例

curl -k -H "Accept: application/json" \\
-X GET http://127.0.0.1:8080/api/v1/metadata/hive/dbs
[
    "default"
]

根据库名获取所有表元数据

GET /api/v1/metadata/hive/tables?db=name

Request 参数

dbdb名Stringdefault

调用示例

curl -k -H "Accept: application/json" \\
-X GET http://127.0.0.1:8080/api/v1/metadata/hive/tables?db=default
[
  
    "tableName": "demo_src",
    "dbName": "default",
    "owner": "root",
    "createTime": 1508216660,
    "lastAccessTime": 0,
    "retention": 0,
    "sd": 
      "cols": [
        
          "name": "id",
          "type": "bigint",
          "comment": null,
          "setName": true,
          "setType": true,
          "setComment": false
        ,
        
          "name": "age",
          "type": "int",
          "comment": null,
          "setName": true,
          "setType": true,
          "setComment": false
        ,
        
          "name": "desc",
          "type": "string",
          "comment": null,
          "setName": true,
          "setType": true,
          "setComment": false
        
      ],
      "location": "hdfs://sandbox:9000/griffin/data/batch/demo_src"
    ,
    "partitionKeys": [
      
        "name": "dt",
        "type": "string",
        "comment": null,
        "setName": true,
        "setType": true,
        "setComment": false
      ,
      
        "name": "hour",
        "type": "string",
        "comment": null,
        "setName": true,
        "setType": true,
        "setComment": false
      
    ]
  ,
  
    "tableName": "demo_tgt",
    "dbName": "default",
    "owner": "root",
    "createTime": 1508216660,
    "lastAccessTime": 0,
    "retention": 0,
    "sd": 
      "cols": [
        
          "name": "id",
          "type": "bigint",
          "comment": null,
          "setName": true,
          "setType": true,
          "setComment": false
        ,
        
          "name": "age",
          "type": "int",
          "comment": null,
          "setName": true,
          "setType": true,
          "setComment": false
        ,
        
          "name": "desc",
          "type": "string",
          "comment": null,
          "setName": true,
          "setType": true,
          "setComment": false
        
      ],
      "location": "hdfs://sandbox:9000/griffin/data/batch/demo_tgt"
 ,
    "partitionKeys": [
      
        "name": "dt",
        "type": "string",
        "comment": null,
        "setName": true,
        "setType": true,
        "setComment": false
      ,
      
        "name": "hour",
        "type": "string",
        "comment": null,
        "setName": true,
        "setType": true,
        "setComment": false
      
    ]
  
]

权限

用户认证

POST /api/v1/login/authenticate

Request 参数

map包含username 和 password  的map 数据Map"username":"user","password":"test"

调用示例

curl -k -H "Content-Type: application/json" -H "Accept: application/json" \\
-X POST http://127.0.0.1:8080/api/v1/login/authenticate \\
-d '"username":"user","password":"test"'

如果验证通过,将返回以下响应。


  "fullName": "Default",
  "ntAccount": "user",
  "status": 0

以上是关于翻译(zh-CN):Apache Griffin API的主要内容,如果未能解决你的问题,请参考以下文章

翻译(zh-CN):Apache Griffin API

翻译(zh-CN):Apache Griffin API

Apache griffin的搭建及应用

数仓之数据质量及Apache Griffin简介

Griffin调研笔记 --- 数据质量监控服务

Linux搭建数据质量监控 Griffin