从配置单元中的 JSON 数组获取 JSON 密钥时面临的问题
Posted
技术标签:
【中文标题】从配置单元中的 JSON 数组获取 JSON 密钥时面临的问题【英文标题】：Problem facing while getting JSON keys from an array of JSON's in hive 【发布时间】：2020-10-28 19:35:53 【问题描述】：我有一个嵌套的 json，我想提取值数组
"Event": "Mohan",
"Job ID": 1,
"Submission Time": 18032203,
"Stage Infos": [
"Stage ID": 2,
"Stage Attempt ID": 0,
"Stage Name": "Stage_name",
"Number of Tasks": 1394
,
"Stage ID": 3,
"Stage Attempt ID": 0,
"Stage Name": "Stage_name2",
"Number of Tasks": 1894
]
使用以下方式查询我：-
select j.event
,j.job_id
,j.submission_time
,get_json_object (spark_event,concat('$.Stage Infos[',e.i,'].Stage ID')) as stage_id
,get_json_object (spark_event,concat('$.Stage Infos[',e.i,'].Stage Attempt ID')) as atmpt_id
,get_json_object (spark_event,concat('$.Stage Infos[',e.i,'].Stage Name')) as stage_name
,get_json_object (spark_event,concat('$.Stage Infos[',e.i,'].Number of Tasks')) as number_tasks
from table_test t
lateral view json_tuple (spark_event,'Event','Job ID','Submission Time') j as event,job_id,submission_time
lateral view posexplode (split(get_json_object (spark_event,'$.Stage Infos[*].Stage ID'),'","')) e as i,x
当我运行上述查询时，我得到以下输出
event job_id submission_time stage_id atmpt_id stage_name number_tasks
Mohan 1 18032203 2 0 Stage_name 1394
但我想要如下所示的。
event job_id submission_time stage_id atmpt_id stage_name number_tasks
Mohan 1 18032203 2 0 Stage_name 1394
Mohan 1 18032203 3 0 Stage_name2 1894
【问题讨论】：
【参考方案1】：更改传递给 split 函数的值对我有用。请检查一次：
select
j.event,
j.job_id,
j.submission_time,
get_json_object (spark_event,concat('$.Stage Infos[',e.i,'].Stage ID')) as stage_id,
get_json_object (spark_event,concat('$.Stage Infos[',e.i,'].Stage Attempt ID')) as atmpt_id,
get_json_object (spark_event,concat('$.Stage Infos[',e.i,'].Stage Name')) as stage_name,
get_json_object (spark_event,concat('$.Stage Infos[',e.i,'].Number of Tasks')) as number_tasks from json_example t
lateral view json_tuple (spark_event,'Event','Job ID','Submission Time') j as event,job_id,submission_time
lateral view posexplode (split(get_json_object (spark_event,'$.Stage Infos[*]'),',')) e as i,x;
event job_id submission_time stage_id atmpt_id stage_name number_tasks
Mohan 1 18032203 2 0 Stage_name 1394
Mohan 1 18032203 3 0 Stage_name2 1894
【讨论】：
以上是关于从配置单元中的 JSON 数组获取 JSON 密钥时面临的问题的主要内容，如果未能解决你的问题，请参考以下文章