Groupby 聚合和缺失值组合

Posted 2023-02-16

【中文标题】Groupby 聚合和缺失值组合

【英文标题】：Groupby aggregations and missing combinations of values

【发布时间】：2021-02-06 18:13:31

【问题描述】：

我最近开始修改 Vega-Lite 模板，为一个名为 DVC 的开源数据科学软件制作混淆矩阵。你可以在my PR here看到模板，但我也会在下面重复一个简化版本：

    ...
    "data": 
        "values": [
            "actual": "Wake", "predicted": "Wake", "rev": "HEAD",
            "actual": "Wake", "predicted": "Deep", "rev": "HEAD",
            "actual": "Light", "predicted": "Wake", "rev": "HEAD",
            "actual": "REM", "predicted": "Light", "rev": "HEAD",
            ....
        ],
    ,
    "spec": 
        "transform": [
            
                "aggregate": ["op": "count", "as": "xy_count"],
                "groupby": ["actual", "predicted"],
            ,
            
                "joinaggregate": [
                    "op": "max", "field": "xy_count", "as": "max_count"
                ],
                "groupby": [],
            ,
            
                "calculate": "datum.xy_count / datum.max_count",
                "as": "percent_of_max",
            ,
        ],
        "encoding": 
            "x": "field": "predicted", "type": "nominal", "sort": "ascending",
            "y": "field": "actual", "type": "nominal", "sort": "ascending",
        ,
        "layer": [
            
                "mark": "rect",
                "width": 300,
                "height": 300,
                "encoding": 
                    "color": 
                        "field": "xy_count",
                        "type": "quantitative",
                        "title": "",
                        "scale": "domainMin": 0, "nice": True,
                    
                ,
            ,
            
                "mark": "text",
                "encoding": 
                    "text": 
                        "field": "xy_count",
                        "type": "quantitative"
                    ,
                    "color": 
                        "condition": 
                            "test": "datum.xy_count / datum.max_count > 0.5",
                            "value": "white"
                        ,
                        "value": "black"
                    
                
            
        ]
    

因此，由于我正在执行 groupby 聚合，因此混淆矩阵中可能存在没有条目的单元格。这是一个示例输出：link

如何用“后备”或其他内容填写这些单元格。我还研究了使用枢轴和估算，但不太明白。非常感谢帮助:)

【参考方案1】：

您可以通过在转换序列的末尾添加两个 Impute transforms 来做到这一点：

  "impute": "xy_count", "groupby": ["actual"], "key": "predicted", "keyvals": ["Deep", "Light", "Wake", "REM"], "value": 0,
  "impute": "xy_count", "groupby": ["predicted"], "key": "actual", "keyvals": ["Deep", "Light", "Wake", "REM"], "value": 0

keyvals 指定您希望在每个轴上估算哪些缺失值；如果每个 keyval 至少存在一个组，则可以将其省略。

【讨论】：

谢谢！这几乎完全正确，我只需要在“groupby”字段中添加“rev”，然后将这些插补直接放在第一个 groupby 之后，因为这似乎是最合乎逻辑的地方。我还删除了“keyvals”，因为这应该是一个通用模板，不依赖于我的特定值:)