从sql查询json格式中提取数据

Posted 2023-05-08

【中文标题】从sql查询json格式中提取数据

【英文标题】：extract data from sql query json format

【发布时间】：2021-03-30 07:16:28

【问题描述】：

我有一个表 sensor_measurements 和列 measure 和 measure_at (timestamp)

 select measured_at, pollutants
    from sensor_measurements;

给出：

October 22, 2019, 9:00 PM
["name": "NO", "units": "ppm", "concentration": 0.002161, "temporal_resolution": "1h",
 "name": "NO2", "units": "ppm", "concentration": 0.002, "temporal_resolution": "1h",
  "name": "TEMP", "units": "celsius", "concentration": 28, "temporal_resolution": "1h",
   "name": "HUM", "units": "percent", "concentration": 38, "temporal_resolution": "1h", 
   "name": "PM10", "units": "µg/m3", "concentration": 8, "temporal_resolution": "1h",
    "name": "PM25", "units": "µg/m3", "concentration": 7, "temporal_resolution": "1h"]
    
October 22, 2019, 10:00 PM
["name": "NO", "units": "ppm", "concentration": 0.002205, "temporal_resolution": "1h",
 "name": "NO2", "units": "ppm", "concentration": 0.008, "temporal_resolution": "1h",
  "name": "TEMP", "units": "celsius", "concentration": 28, "temporal_resolution": "1h", 
  "name": "HUM", "units": "percent", "concentration": 38, "temporal_resolution": "1h", 
  "name": "PM10", "units": "µg/m3", "concentration": 9, "temporal_resolution": "1h", 
  "name": "PM25", "units": "µg/m3", "concentration": 8, "temporal_resolution": "1h"]
  
October 22, 2019, 11:00 PM
["name": "NO", "units": "ppm", "concentration": 0.002209, "temporal_resolution": "1h",
 "name": "NO2", "units": "ppm", "concentration": 0.004, "temporal_resolution": "1h",
  "name": "TEMP", "units": "celsius", "concentration": 28, "temporal_resolution": "1h", 
  "name": "HUM", "units": "percent", "concentration": 38, "temporal_resolution": "1h", 
  "name": "PM10", "units": "µg/m3", "concentration": 8, "temporal_resolution": "1h", 
  "name": "PM25", "units": "µg/m3", "concentration": 7, "temporal_resolution": "1h"]
  
October 23, 2019, 12:00 AM
["name": "NO", "units": "ppm", "concentration": 0.002125, "temporal_resolution": "1h",
"name": "NO2", "units": "ppm", "concentration": 0.004, "temporal_resolution": "1h", 
"name": "TEMP", "units": "celsius", "concentration": 28, "temporal_resolution": "1h", 
"name": "HUM", "units": "percent", "concentration": 39, "temporal_resolution": "1h"]


October 23, 2019, 4:00 PM
["name": "NO", "units": "ppm", "concentration": 0.004563, "temporal_resolution": "1h", 
"name": "TEMP", "units": "celsius", "concentration": 34, "temporal_resolution": "1h", 
"name": "HUM", "units": "percent", "concentration": 28, "temporal_resolution": "1h"]

我想提取时间戳，pollutant 和它的值（浓度！

理想情况下，我想创建包含时间戳、污染物和值的三列，以便下载为 csv。

数据库类型是 PostgreSQl（在 metabase.com 中）

【问题讨论】：

请提供Sql引擎

@LeszekMazur：它在 metabase.com 中

您的元数据库套件是否以任何类型的 RDBM 为基础？您是否能够通过支持的 SQL 数据库访问数据 - metabase.com/learn/data-diet/analytics/…

如果生成了格式正确的 json 结构，我能够解析 - 示例 echo ' "measured_at": "October 23, 2019, 12:00 AM", "lines": [ "name" ：“NO”，“单位”：“ppm”，“浓度”：0.002125，“temporal_resolution”：“1h”，“名称”：“NO2”，“单位”：“ppm”，“浓度”：0.004 , "temporal_resolution": "1h", "name": "TEMP", "units": "celsius", "concentration": 28, "temporal_resolution": "1h", "name": "HUM" ，“单位”：“百分比”，“浓度”：39，“temporal_resolution”：“1h”]' | jq

@irnerd:它使用postgresql

【参考方案1】：

我是在 postgres 上完成的。 首先你必须有类型代表你的数据：

CREATE TYPE x as ("name" VARCHAR , "units" VARCHAR , "concentration" FLOAT, "temporal_resolution" VARCHAR );

接下来，您可以使用 json 作为连接表：

SELECT measured_at, name, concentration
FROM sensor_measurements
LEFT JOIN LATERAL json_populate_recordset(null::x, pollutants::json) ON true;

【讨论】：

嗨，看起来不错，只是我无法通过元数据库ERROR: cannot execute CREATE TYPE in a read-only transaction 执行create type。不过我有管理员权限。

所以尽量在事务之外做——一次，所以你做一次，在事务中每次都能得到结果