我如何只能使用 python 脚本从 json api 接收选定序列号的数据

Posted 2023-02-23

【中文标题】我如何只能使用 python 脚本从 json api 接收选定序列号的数据

【英文标题】：How can I only receive data from selected serial numbers from json api with python script

【发布时间】：2019-09-09 09:23:53

【问题描述】：

如何只接收来自选定序列号的数据 或者添加要排除的序列号列表？ 即）“serialNumber”：“121718037628”，已经死了大约2个月了 或者可能自动排除长时间未更新的数据 使用“lastReportDate”：1549920259，

使用 prometheus 抓取元数据

来自enphase微逆变器控制器的流看起来像这样 http://envoy/api/v1/production/inverters（本地网络）

[
  
    "serialNumber": "121718037628",
    "lastReportDate": 1549920259,
    "devType": 1,
    "lastReportWatts": 18,
    "maxReportWatts": 18
  ,
  
    "serialNumber": "121718037534",
    "lastReportDate": 1555635154,
    "devType": 1,
    "lastReportWatts": 108,
    "maxReportWatts": 187
  ,
  
    "serialNumber": "121718037683",
    "lastReportDate": 1555635148,
    "devType": 1,
    "lastReportWatts": 117,
    "maxReportWatts": 197
  ,

我已经尝试在

中添加这个

ignore_list = ['121718037628']
       for pattern in ignore_list:
           if fnmatch( pattern):
                should_ignore = True

并尝试过

[f for f in serial if not any(a in f.split() for a in faultyunit)]

这是原始的python代码

#!/usr/bin/env python -f

import os
import time
import json
import requests
import threading
from requests.auth import HTTPDigestAuth
from prometheus_client import start_http_server, Gauge


host = os.getenv('ENVOY_HOST')
password = os.getenv('ENVOY_PASS')

user = 'installer'
auth = HTTPDigestAuth(user, password)
marker = b'data: '


serials = 
    121718037663: '#1',
    121718037534: '#1',
    121718037513: '#1',
    121718037414: '#1',
    121718037683: '#1',
    121718037593: '',  
    121718037695: '#1',
    121718037872: '#1',
    121718037601: '#1',
    121718037876: '#2',
    121718037698: '#2',
    121718037881: '#2',
    121718037584: '#2',
    121718037703: '#2',




stream_gauges = 
    'p': Gauge('meter_active_power_watts', 'Active Power', ['type', 
'phase']),
    'q': Gauge('meter_reactive_power_watts', 'Reactive Power', 
['type', 'phase']),
    's': Gauge('meter_apparent_power_watts', 'Apparent Power', 
['type', 'phase']),
    'v': Gauge('meter_voltage_volts', 'Voltage', ['type', 'phase']),
    'i': Gauge('meter_current_amps', 'Current', ['type', 'phase']),
    'f': Gauge('meter_frequency_hertz', 'Frequency', ['type', 'phase']),
    'pf': Gauge('meter_power_factor_ratio', 'Power Factor', ['type', 'phase']),


production_gauges = 
    'activeCount': Gauge('production_active_count', 'Active Count', ['type']),
    'wNow': Gauge('power_now_watts', 'Active Count', ['type']),
    'whToday': Gauge('production_today_watthours', 'Total production today', ['type']),
    'whLastSevenDays': Gauge('production_7days_watthours', 'Total production last seven days', ['type']),
    'whLifetime': Gauge('production_lifetime_watthours', 'Total production lifetime', ['type']),


consumption_gauges = 
    'wNow': Gauge('consumption_now_watts', 'Active Count', ['type']),
    'whToday': Gauge('consumption_today_watthours', 'Total consumption today', ['type']),
    'whLastSevenDays': Gauge('consumption_7days_watthours', 'Total consumption last seven days', ['type']),
    'whLifetime': Gauge('consumption_lifetime_watthours', 'Total consumption lifetime', ['type']),


inverter_gauges = 
    'last': Gauge('inverter_last_report_watts', 'Last reported watts', ['serial', 'location']),
    'max': Gauge('inverter_max_report_watts', 'Max reported watts', ['serial', 'location']),



def scrape_stream():
    while True:
        try:
            url = 'http://%s/stream/meter' % host
            stream = requests.get(url, auth=auth, stream=True, timeout=5)
            for line in stream.iter_lines():
                if line.startswith(marker):
                    data = json.loads(line.replace(marker, b''))
                    print(data)
                    for meter_type in ['production', 'net-consumption', 'total-consumption']:
                        for phase in ['ph-a', 'ph-b']:
                            for key, value in data.get(meter_type, ).get(phase, ).items():
                                if key in stream_gauges:

stream_gauges[key].labels(type=meter_type, phase=phase).set(value)
        except requests.exceptions.RequestException as e:
            print('Exception fetching stream data: %s' % e)
            time.sleep(5)


def scrape_production_json():
    url = 'http://%s/production.json' % host
    data = requests.get(url).json()
    production = data['production']
    print(production)
    for each in production:
        mtype = each['type']
        for key in ['activeCount', 'wNow', 'whLifetime', 'whToday', 'whLastSevenDays']:
            value = each.get(key)
            if value is not None:
                production_gauges[key].labels(type=mtype).set(value)
    consumption = data['consumption']
    print(consumption)
    for each in consumption:
        mtype = each['measurementType']
        for key in ['wNow', 'whLifetime', 'whToday', 'whLastSevenDays']:
            value = each.get(key)
            if value is not None:
                consumption_gauges[key].labels(type=mtype).set(value)



def scrape_inverters():
    url = 'http://%s/api/v1/production/inverters' % host
    data = requests.get(url, auth=auth).json()
    print(data)
    for inverter in data:
        serial = int(inverter['serialNumber'])
        location = serials.get(serial, '')
        inverter_gauges['last'].labels(serial=serial, 
location=location).set(inverter['lastReportWatts'])
        inverter_gauges['max'].labels(serial=serial, 
location=location).set(inverter['maxReportWatts'])


def main():
    start_http_server(8000)
    stream_thread = threading.Thread(target=scrape_stream)
    stream_thread.setDaemon(True)
    stream_thread.start()
    while True:
        try:
            scrape_production_json()
            scrape_inverters()
        except Exception as e:
            print('Exception fetching scrape data: %s' % e)
        time.sleep(60)


if __name__ == '__main__':
    main()

我已经尝试添加这个

ignorelist = 
    121718037628,


for inverter in data:
     if inverter['serialNumber'] in ignorelist:
        continue
         serial = int(inverter['serialNumber'])

但是它会跳过所有的逆变器

也试过这种方法

json_response = [


]

但不确定如何获取它从 API 中读取的内容

ignore_set = '121718037628', 

for inverter in json_response:
    if inverter['serialNumber'] not in ignore_set:
        print(inverter)

终于解决了，谢谢大家的帮助

ignorelist = 
'121718037628','121718037534'


for inverter in data:
if inverter['serialNumber'] not in ignorelist:
    print(inverter)
    serial = int(inverter['serialNumber'])
    location = serials.get(serial, '')

【问题讨论】：

我刚醒来，所以没有关注 - 你是想从请求中排除或过滤掉响应？

感谢您的评论和早上好，我想我可以尝试查看排除，输出通常是这样的：inverter_last_report_wattslocation="",serial="121718037628" 18.0，我不是如果它是数字“121718037628”，则需要输出

但是如果它只从本节中包含的序列中获取数据：serials = 121718037663: '#1'，这将是非常棒的，因为它们是手动输入的以设置它们在图形上的数组

【参考方案1】：

也许检查序列在忽略列表中，然后进一步跳过设备处理

for inverter in data:
    if inverter['serialNumber'] not in ignorelist:
        serial = int(inverter['serialNumber'])

【讨论】：

谢谢我之前尝试过它会崩溃，除非我在设置忽略列表时做错了什么？ （对不起，我无法确定这个格式） ignorelist = 121718037628,

所以显然我因为 IndentationError 而浪费了我的一天，感谢您确认我在正确的轨道上

它现在如何忽略所有内容 `ignorelist = 121718037628, `（叹气我仍然不知道如何回复代码格式化如何在不发送的情况下删除一行）

谢谢，您是否有一个忽略列表作为字符串的示例，无论我做什么仍然会跳过所有逆变器：获取抓取数据的异常：未定义名称“循环”

【参考方案2】：

ignorelist = 
'121718037628','121718037534'


for inverter in data:
    if inverter['serialNumber'] not in ignorelist:
        print(inverter)
        serial = int(inverter['serialNumber'])
        location = serials.get(serial, '')