如何使用 jq 将 JSON 字符串格式化为表格？
Posted
技术标签:
【中文标题】如何使用 jq 将 JSON 字符串格式化为表格？【英文标题】：How to format a JSON string as a table using jq? 【发布时间】：2017-01-01 12:30:54 【问题描述】：刚开始使用 Bash 脚本，偶然发现 jq 可以使用 JSON。
我需要将如下所示的 JSON 字符串转换为表格以便在终端中输出。
[
"name": "George",
"id": 12,
"email": "george@domain.com"
,
"name": "Jack",
"id": 18,
"email": "jack@domain.com"
,
"name": "Joe",
"id": 19,
"email": "joe@domain.com"
]
我想在终端中显示的内容：
ID Name
=================
12 George
18 Jack
19 Joe
注意我不想显示每一行的 email 属性，所以 jq 命令应该包含一些过滤。以下为我提供了一个简单的名称和 ID 列表：
list=$(echo "$data" | jq -r '.[] | .name, .id')
printf "$list"
问题是，我不能像表格一样显示它。我知道 jq 有一些格式化选项，但不如我使用 printf 时的选项好。我想我想在一个数组中获取这些值，然后我可以循环自己进行格式化......？我尝试过的事情给了我不同的结果，但从来都不是我真正想要的。
有人能指出正确的方向吗？
【问题讨论】：
您能否添加一些jq -r ... 命令的示例输出？
您可以避免使用echo jq -r '...' <<<$data 或jr -r '...' < input-file.json。
你的问题是：我有一个字符串"name1 value1 name2 value2 name3 value3"我如何将它打印为表格？
【参考方案1】：
为什么不这样：
echo '[
"name": "George",
"id": 12,
"email": "george@domain.com"
,
"name": "Jack",
"id": 18,
"email": "jack@domain.com"
,
"name": "Joe",
"id": 19,
"email": "joe@domain.com"
]' | jq -r '.[] | "\(.id)\t\(.name)"'
输出
12 George
18 Jack
19 Joe
编辑 1： 对于细粒度格式，请使用awk等工具
echo '[
"name": "George",
"id": 12,
"email": "george@domain.com"
,
"name": "Jack",
"id": 18,
"email": "jack@domain.com"
,
"name": "Joe",
"id": 19,
"email": "joe@domain.com"
]' | jq -r '.[] | [.id, .name] | @csv' | awk -v FS="," 'BEGINprint "ID\tName";print "============"printf "%s\t%s%s",$1,$2,ORS'
ID Name
============
12 "George"
18 "Jack"
19 "Joe"
编辑2：回复
我无法直接获取包含数组的变量 来自jq？
为什么不呢？
一个涉及到的例子（实际上是从你的修改而来的）将电子邮件更改为数组演示了这一点
echo '[
"name": "George",
"id": 20,
"email": [ "george@domain1.com" , "george@domain2.com" ]
,
"name": "Jack",
"id": 18,
"email": [ "jack@domain3.com" , "jack@domain5.com" ]
,
"name": "Joe",
"id": 19,
"email": [ "joe@domain.com" ]
]' | jq -r '.[] | .email'
输出
[
"george@domain1.com",
"george@domain2.com"
]
[
"jack@domain3.com",
"jack@domain5.com"
]
[
"joe@domain.com"
]
【讨论】：
感谢您的回答。这在这种特殊情况下非常有效，id 的长度都相同。想象一下，我会切换字段的顺序，这会给我一些看起来根本不像一个方便的表格的东西。我真的在寻找可以在更多数据集上使用的解决方案。不过，感谢您的回答！ 好的，知道了。我无法直接从 jq 获取包含数组的变量？我总是要从一个字符串开始？ 感谢您的帮助，输出完全符合我的要求（名称周围的引号除外）。从一个字符串开始而不是像我们在 Python 中那样准备好使用数组感觉很奇怪。对我来说它感觉笨拙和肮脏，但我想只有我必须习惯 bash 的想法？我会尝试把它变成一个我可以重用的函数，这样我就可以将它用于更多具有不同标头的 JSON 字符串。 @Rein ：对于细粒度的格式，您需要以 csv 格式打印输出，然后使用awk，但请注意复杂的情况可能会失败。对于您的第二条评论，请参阅最后的编辑并与[ this ] 回答一起阅读。
感谢您的链接和解释，超级有用！【参考方案2】：
如果值不包含空格，这可能会有所帮助：
read -r -a data <<<'name1 value1 name2 value2'
echo "name value"
echo "=========="
for ((i=0; i<$#data[@]; i+=2)); do
echo $data[$i] $data[$((i+1))]
done
输出
name value
==========
name1 value1
name2 value2
【讨论】：
我开始意识到我无法直接从 jq 获取数组，对吗？所以要走的路是从中获取一个字符串（以可行的格式）并从那里开始？【参考方案3】：使用@tsv 过滤器有很多值得推荐的地方，主要是因为它以标准方式处理大量“边缘情况”：
.[] | [.id, .name] | @tsv
添加标题可以这样完成：
jq -r '["ID","NAME"], ["--","------"], (.[] | [.id, .name]) | @tsv'
结果：
ID NAME
-- ------
12 George
18 Jack
19 Joe
length*"-"
要自动生成虚线：
jq -r '(["ID","NAME"] | (., map(length*"-"))), (.[] | [.id, .name]) | @tsv'
【讨论】：
@tsv 过滤器甚至在 jq 的基本过滤器手册页上，嗯...我想知道我可能还错过了什么 :)【参考方案4】：手动定义标题是次优的！省略标题也是次优的。
TL;DR
数据
[ "name": "George", "id": 12, "email": "george@domain.com" ,
"name": "Jack", "id": 18, "email": "jack@domain.com" ,
"name": "Joe", "id": 19, "email": "joe@domain.com" ]
脚本
[.[]| with_entries( .key |= ascii_downcase ) ]
| (.[0] |keys_unsorted | @tsv)
, (.[] |map(.) |@tsv)
如何运行
$ < data jq -rf script | column -t
name id email
George 12 george@domain.com
Jack 18 jack@domain.com
Joe 19 joe@domain.com
我在总结来自亚马逊网络服务的一些数据时发现了这个问题。我正在处理的问题，如果你想要另一个例子：
$ aws ec2 describe-spot-instance-requests | tee /tmp/ins |
jq --raw-output '
# extract instances as a flat list.
[.SpotInstanceRequests | .[]
# remove unwanted data
|
State,
statusCode: .Status.Code,
type: .LaunchSpecification.InstanceType,
blockPrice: .ActualBlockHourlyPrice,
created: .CreateTime,
SpotInstanceRequestId
]
# lowercase keys
# (for predictable sorting, optional)
| [.[]| with_entries( .key |= ascii_downcase ) ]
| (.[0] |keys_unsorted | @tsv) # print headers
, (.[]|.|map(.) |@tsv) # print table
' | column -t
输出：
state statuscode type blockprice created spotinstancerequestid
closed instance-terminated-by-user t3.nano 0.002000 2019-02-24T15:21:36.000Z sir-r5bh7skq
cancelled bad-parameters t3.nano 0.002000 2019-02-24T14:51:47.000Z sir-1k9s5h3m
closed instance-terminated-by-user t3.nano 0.002000 2019-02-24T14:55:26.000Z sir-43x16b6n
cancelled bad-parameters t3.nano 0.002000 2019-02-24T14:29:23.000Z sir-2jsh5brn
active fulfilled t3.nano 0.002000 2019-02-24T15:37:26.000Z sir-z1e9591m
cancelled bad-parameters t3.nano 0.002000 2019-02-24T14:33:42.000Z sir-n7c15y5p
输入：
$ cat /tmp/ins
"SpotInstanceRequests": [
"Status":
"Message": "2019-02-24T15:29:38+0000 : 2019-02-24T15:29:38+0000 : Spot Instance terminated due to user-initiated termination.",
"Code": "instance-terminated-by-user",
"UpdateTime": "2019-02-24T15:31:03.000Z"
,
"ActualBlockHourlyPrice": "0.002000",
"ValidUntil": "2019-03-03T15:21:36.000Z",
"InstanceInterruptionBehavior": "terminate",
"Tags": [],
"InstanceId": "i-0414083bef5e91d94",
"BlockDurationMinutes": 60,
"SpotInstanceRequestId": "sir-r5bh7skq",
"State": "closed",
"ProductDescription": "Linux/UNIX",
"LaunchedAvailabilityZone": "eu-north-1a",
"LaunchSpecification":
"Placement":
"Tenancy": "default",
"AvailabilityZone": "eu-north-1a"
,
"ImageId": "ami-6d27a913",
"BlockDeviceMappings": [
"DeviceName": "/dev/sda1",
"VirtualName": "root",
"NoDevice": "",
"Ebs":
"Encrypted": false,
"DeleteOnTermination": true,
"VolumeType": "gp2",
"VolumeSize": 8
],
"EbsOptimized": false,
"SecurityGroups": [
"GroupName": "default"
],
"Monitoring":
"Enabled": false
,
"InstanceType": "t3.nano",
"AddressingType": "public",
"NetworkInterfaces": [
"DeviceIndex": 0,
"Description": "eth-zero",
"NetworkInterfaceId": "",
"DeleteOnTermination": true,
"SubnetId": "subnet-420ffc2b",
"AssociatePublicIpAddress": true
]
,
"Type": "one-time",
"CreateTime": "2019-02-24T15:21:36.000Z",
"SpotPrice": "0.008000"
,
"Status":
"Message": "Your Spot request failed due to bad parameters.",
"Code": "bad-parameters",
"UpdateTime": "2019-02-24T14:51:48.000Z"
,
"ActualBlockHourlyPrice": "0.002000",
"ValidUntil": "2019-03-03T14:51:47.000Z",
"InstanceInterruptionBehavior": "terminate",
"Tags": [],
"Fault":
"Message": "Invalid device name /dev/sda",
"Code": "InvalidBlockDeviceMapping"
,
"BlockDurationMinutes": 60,
"SpotInstanceRequestId": "sir-1k9s5h3m",
"State": "cancelled",
"ProductDescription": "Linux/UNIX",
"LaunchedAvailabilityZone": "eu-north-1a",
"LaunchSpecification":
"Placement":
"Tenancy": "default",
"AvailabilityZone": "eu-north-1a"
,
"ImageId": "ami-6d27a913",
"BlockDeviceMappings": [
"DeviceName": "/dev/sda",
"VirtualName": "root",
"NoDevice": "",
"Ebs":
"Encrypted": false,
"DeleteOnTermination": true,
"VolumeType": "gp2",
"VolumeSize": 8
],
"EbsOptimized": false,
"SecurityGroups": [
"GroupName": "default"
],
"Monitoring":
"Enabled": false
,
"InstanceType": "t3.nano",
"AddressingType": "public",
"NetworkInterfaces": [
"DeviceIndex": 0,
"Description": "eth-zero",
"NetworkInterfaceId": "",
"DeleteOnTermination": true,
"SubnetId": "subnet-420ffc2b",
"AssociatePublicIpAddress": true
]
,
"Type": "one-time",
"CreateTime": "2019-02-24T14:51:47.000Z",
"SpotPrice": "0.011600"
,
"Status":
"Message": "2019-02-24T15:02:17+0000 : 2019-02-24T15:02:17+0000 : Spot Instance terminated due to user-initiated termination.",
"Code": "instance-terminated-by-user",
"UpdateTime": "2019-02-24T15:03:34.000Z"
,
"ActualBlockHourlyPrice": "0.002000",
"ValidUntil": "2019-03-03T14:55:26.000Z",
"InstanceInterruptionBehavior": "terminate",
"Tags": [],
"InstanceId": "i-010442ac3cc85ec08",
"BlockDurationMinutes": 60,
"SpotInstanceRequestId": "sir-43x16b6n",
"State": "closed",
"ProductDescription": "Linux/UNIX",
"LaunchedAvailabilityZone": "eu-north-1a",
"LaunchSpecification":
"Placement":
"Tenancy": "default",
"AvailabilityZone": "eu-north-1a"
,
"ImageId": "ami-6d27a913",
"BlockDeviceMappings": [
"DeviceName": "/dev/sda1",
"VirtualName": "root",
"NoDevice": "",
"Ebs":
"Encrypted": false,
"DeleteOnTermination": true,
"VolumeType": "gp2",
"VolumeSize": 8
],
"EbsOptimized": false,
"SecurityGroups": [
"GroupName": "default"
],
"Monitoring":
"Enabled": false
,
"InstanceType": "t3.nano",
"AddressingType": "public",
"NetworkInterfaces": [
"DeviceIndex": 0,
"Description": "eth-zero",
"NetworkInterfaceId": "",
"DeleteOnTermination": true,
"SubnetId": "subnet-420ffc2b",
"AssociatePublicIpAddress": true
]
,
"Type": "one-time",
"CreateTime": "2019-02-24T14:55:26.000Z",
"SpotPrice": "0.011600"
,
"Status":
"Message": "Your Spot request failed due to bad parameters.",
"Code": "bad-parameters",
"UpdateTime": "2019-02-24T14:29:24.000Z"
,
"ActualBlockHourlyPrice": "0.002000",
"ValidUntil": "2019-03-03T14:29:23.000Z",
"InstanceInterruptionBehavior": "terminate",
"Tags": [],
"Fault":
"Message": "Addressing type must be 'public'",
"Code": "InvalidParameterCombination"
,
"BlockDurationMinutes": 60,
"SpotInstanceRequestId": "sir-2jsh5brn",
"State": "cancelled",
"ProductDescription": "Linux/UNIX",
"LaunchedAvailabilityZone": "eu-north-1a",
"LaunchSpecification":
"Placement":
"Tenancy": "default",
"AvailabilityZone": "eu-north-1a"
,
"ImageId": "ami-6d27a913",
"BlockDeviceMappings": [
"DeviceName": "/dev/sda",
"VirtualName": "root",
"NoDevice": "",
"Ebs":
"Encrypted": false,
"DeleteOnTermination": true,
"VolumeType": "gp2",
"VolumeSize": 8
],
"EbsOptimized": false,
"SecurityGroups": [
"GroupName": "default"
],
"Monitoring":
"Enabled": false
,
"InstanceType": "t3.nano",
"AddressingType": "",
"NetworkInterfaces": [
"DeviceIndex": 0,
"Description": "eth-zero",
"NetworkInterfaceId": "",
"DeleteOnTermination": true,
"SubnetId": "subnet-420ffc2b",
"AssociatePublicIpAddress": true
]
,
"Type": "one-time",
"CreateTime": "2019-02-24T14:29:23.000Z",
"SpotPrice": "0.011600"
,
"Status":
"Message": "Your spot request is fulfilled.",
"Code": "fulfilled",
"UpdateTime": "2019-02-24T15:37:28.000Z"
,
"ActualBlockHourlyPrice": "0.002000",
"ValidUntil": "2019-03-03T15:37:26.000Z",
"InstanceInterruptionBehavior": "terminate",
"Tags": [],
"InstanceId": "i-0a29e9de6d59d433f",
"BlockDurationMinutes": 60,
"SpotInstanceRequestId": "sir-z1e9591m",
"State": "active",
"ProductDescription": "Linux/UNIX",
"LaunchedAvailabilityZone": "eu-north-1a",
"LaunchSpecification":
"Placement":
"Tenancy": "default",
"AvailabilityZone": "eu-north-1a"
,
"ImageId": "ami-6d27a913",
"BlockDeviceMappings": [
"DeviceName": "/dev/sda1",
"VirtualName": "root",
"NoDevice": "",
"Ebs":
"Encrypted": false,
"DeleteOnTermination": true,
"VolumeType": "gp2",
"VolumeSize": 8
],
"EbsOptimized": false,
"SecurityGroups": [
"GroupName": "default"
],
"Monitoring":
"Enabled": false
,
"InstanceType": "t3.nano",
"AddressingType": "public",
"NetworkInterfaces": [
"DeviceIndex": 0,
"Description": "eth-zero",
"NetworkInterfaceId": "",
"DeleteOnTermination": true,
"SubnetId": "subnet-420ffc2b",
"AssociatePublicIpAddress": true
]
,
"Type": "one-time",
"CreateTime": "2019-02-24T15:37:26.000Z",
"SpotPrice": "0.008000"
,
"Status":
"Message": "Your Spot request failed due to bad parameters.",
"Code": "bad-parameters",
"UpdateTime": "2019-02-24T14:33:43.000Z"
,
"ActualBlockHourlyPrice": "0.002000",
"ValidUntil": "2019-03-03T14:33:42.000Z",
"InstanceInterruptionBehavior": "terminate",
"Tags": [],
"Fault":
"Message": "Invalid device name /dev/sda",
"Code": "InvalidBlockDeviceMapping"
,
"BlockDurationMinutes": 60,
"SpotInstanceRequestId": "sir-n7c15y5p",
"State": "cancelled",
"ProductDescription": "Linux/UNIX",
"LaunchedAvailabilityZone": "eu-north-1a",
"LaunchSpecification":
"Placement":
"Tenancy": "default",
"AvailabilityZone": "eu-north-1a"
,
"ImageId": "ami-6d27a913",
"BlockDeviceMappings": [
"DeviceName": "/dev/sda",
"VirtualName": "root",
"NoDevice": "",
"Ebs":
"Encrypted": false,
"DeleteOnTermination": true,
"VolumeType": "gp2",
"VolumeSize": 8
],
"EbsOptimized": false,
"SecurityGroups": [
"GroupName": "default"
],
"Monitoring":
"Enabled": false
,
"InstanceType": "t3.nano",
"AddressingType": "public",
"NetworkInterfaces": [
"DeviceIndex": 0,
"Description": "eth-zero",
"NetworkInterfaceId": "",
"DeleteOnTermination": true,
"SubnetId": "subnet-420ffc2b",
"AssociatePublicIpAddress": true
]
,
"Type": "one-time",
"CreateTime": "2019-02-24T14:33:42.000Z",
"SpotPrice": "0.011600"
]
【讨论】：
column -t 使标题与表格本身对齐。谢谢！
您可以使用column -ts $'\t' 来分割制表符而不是空格——否则带有空格的值将被分割成多列。来自unix.stackexchange.com/a/57235/140650【参考方案5】：
上述答案的问题在于，它们仅在字段宽度大致相同时才有效。
为避免此问题，可以使用 Linux column 命令：
// input.json
[
"name": "George",
"id": "a very very long field",
"email": "george@domain.com"
,
"name": "Jack",
"id": 18,
"email": "jack@domain.com"
,
"name": "Joe",
"id": 19,
"email": "joe@domain.com"
]
然后：
▶ jq -r '.[] | [.id, .name] | @tsv' input.json | column -ts $'\t'
a very very long field George
18 Jack
19 Joe
【讨论】：
【参考方案6】：我混合了所有响应以获得所有这些行为
创建头表 处理长字段 创建要重用的函数函数 bash
function jsonArrayToTable()
jq -r '(.[0] | ([keys[] | .] |(., map(length*"-")))), (.[] | ([keys[] as $k | .[$k]])) | @tsv' | column -t -s $'\t'
示例使用
echo '["key1":"V1.1", "key2":"V2.1", "keyA":"V1.2", "key2":"V2.2"]' | jsonArrayToTable
输出
key1 key2
---- ----
V1.1 V2.1
V2.2 V1.2
【讨论】：
【参考方案7】：如果你想生成一个html表而不是终端输出的表：
echo '[
"name": "George",
"id": 12,
"email": "george@domain.com"
,
"name": "Jack",
"id": 18,
"email": "jack@domain.com"
,
"name": "Joe",
"id": 19,
"email": "joe@domain.com"
]' | jq -r 'map("<tr><td>" + .name + "</td><td>" + (.id | tostring) + "</td></tr>") | ["<table>"] + . + ["</table>"] | .[]'
输出：
<table>
<tr><td>George</td><td>12</td></tr>
<tr><td>Jack</td><td>18</td></tr>
<tr><td>Joe</td><td>19</td></tr>
</table>
【讨论】：
【参考方案8】：更简单的实现：
jq -r '(.[0]|keys_unsorted|(.,map(length*"-"))),.[]|map(.)|@tsv'|column -ts $'\t'
您可以在~/.jq中添加以下jq函数：
def pretty_table:
(.[0]|keys_unsorted|(.,map(length*"-"))),.[]|map(.)|@tsv
;
然后运行：
cat apps.json | jq -r pretty_table | column -ts $'\t'
【讨论】：
以上是关于如何使用 jq 将 JSON 字符串格式化为表格？的主要内容，如果未能解决你的问题，请参考以下文章
如何在 jq 中将 JSON 对象转换为 key=value 格式？