在 SSIS 中使用 C# 从 JSON API 读取

Posted 2023-04-12

【中文标题】在 SSIS 中使用 C# 从 JSON API 读取

【英文标题】：Reading from JSON API with C# in SSIS

【发布时间】：2019-08-09 00:20:21

【问题描述】：

我想从 SSIS 中的 JSON API 读取一些数据并将其写入 SQL Server 中的表。我已经使用第 3 方解决了该任务，但解决方案并不那么优雅，所以现在我尝试使用 SSIS 的脚本组件在 Visual Studio 中自己编写脚本。

我在网上搜索了解决方案，并以这个结果结束。到目前为止，我对正在发生的事情相当有信心，但我缺乏最终的方向。我知道我需要以某种方式将输出映射到我在 SSIS 中创建的列。 我想我必须在CreateNewOutputRows() 周围做点什么，但我不确定是什么。有人可以帮我解决这个问题吗？此外，由于这或多或少是我的第一个 c# 脚本，如果有更简单的解决方案或者它在某些方面不合适等，我也将不胜感激。

首先，API 的输出看起来像这样(API documentation here)：

 "data":[  
        
         "created_at":"2016-03-12 09:45:00",
         "created_at_unix":1457772300,
         "shop_name":"DK - Some name",
         "location_id":1111,
         "custom_location_id":"2222",
         "custom_shop_id":"2222",
         "shop_id":3333,
         "count_in":"1",
         "count_out":"1",
         "timekey":3
      

到目前为止我得到的脚本是

using System;
using System.Data;
using Microsoft.SqlServer.Dts.Pipeline.Wrapper;
using Microsoft.SqlServer.Dts.Runtime.Wrapper;
using System.Linq;
using System.Net;
using System.Collections.Generic;
using Newtonsoft.Json;


[Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute]
public class ScriptMain : UserComponent

    public static void Main()
    
        //Test api
        var url = "https://login.vemcount.com/api/fetch_data/?data="api_key":"xxxxxxx","type":"company_id","id":"10","date_from":"2019-01-01 00:00","interval":"15min", "group_by":"shop_id"";
        var json = new WebClient().DownloadString(url);

        var root = JsonConvert.DeserializeObject<RootObject>(json);

        //Printing last record
        Console.WriteLine(root.data.Last().created_at);
    
    public class data
    

        public string created_at  get; set; 
        public int created_at_unix  get; set; 
        public string shop_name  get; set; 
        public int location_id  get; set; 
        public string custom_location_id  get; set; 
        public string custom_shop_id  get; set; 
        public int shop_id  get; set; 
        public string count_in  get; set; 
        public string count_out  get; set; 
        public int timekey  get; set; 
    

    public class RootObject
    
        public List<data> data  get; set; 
    
    public override void CreateNewOutputRows()
    

    

【参考方案1】：

您将把所有代码放入CreateNewOutputRows()。

首先，您必须手动将所有列添加到脚本组件。在这里的示例中，我只添加了 2 列：

代码进入 CreateNewOutPutRows()。我没有 Newtonsoft，只是在这里使用 javascriptSerializer 来展示一个工作示例，以便您了解如何连接它：

using System;
using System.Data;
using Microsoft.SqlServer.Dts.Pipeline.Wrapper;
using Microsoft.SqlServer.Dts.Runtime.Wrapper;
using System.Web.Script.Serialization;
using System.Collections.Generic;

[Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute]
public class ScriptMain : UserComponent

    public override void CreateNewOutputRows()
    
        string json = @"""data"":[""created_at"":""2016-03-12 09:45:00"",""created_at_unix"":1457772300,""shop_name"":""DK - Some name"",""location_id"":1111,""custom_location_id"":""2222"",""custom_shop_id"":""2222"",""shop_id"":3333,""count_in"":""1"",""count_out"":""1"",""timekey"":3,""created_at"":""2016-03-12 09:45:00"",""created_at_unix"":1457772300,""shop_name"":""test2"",""location_id"":1111,""custom_location_id"":""2222"",""custom_shop_id"":""2222"",""shop_id"":3333,""count_in"":""1"",""count_out"":""1"",""timekey"":3]";

        RootObject Test = new JavaScriptSerializer().Deserialize<RootObject>(json);

        /*
         * This is where data gets added to the output buffer.
         * After AddRow() you are basically mapping the column you manually added(on the left) to the data(on the right).
         * using a foreach loop to loop through the deserialize json
         */
        foreach (var item in Test.data)
        
            Output0Buffer.AddRow();
            Output0Buffer.createdat = item.created_at;
            Output0Buffer.shopname = item.shop_name;
        
    
    public class RootObject
    
        public List<data> data  get; set; 
    

    public class data
    

        public string created_at  get; set; 
        public int created_at_unix  get; set; 
        public string shop_name  get; set; 
        public int location_id  get; set; 
        public string custom_location_id  get; set; 
        public string custom_shop_id  get; set; 
        public int shop_id  get; set; 
        public string count_in  get; set; 
        public string count_out  get; set; 
        public int timekey  get; set; 
    

然后在此示例中，仅使用记录集目标并启用数据查看器，这样您就可以看到返回的个人行：

【讨论】：

很好的答案。我忘了提到我已经在脚本组件中创建了输出列，但这完美地展示了我的过程。 public override void CreateNewOutputRows() 部分是否覆盖默认反序列化器或？