如何使用 sql 表中的坐标作为 Weather URL API SSIS 脚本中的参数输入？

Posted 2023-05-08

【中文标题】如何使用 sql 表中的坐标作为 Weather URL API SSIS 脚本中的参数输入？

【英文标题】：How to use coordinates from a sql table as parameter input in Weather URL API SSIS script?

【发布时间】：2021-03-13 10:28:10

【问题描述】：

我创建了一个 SSIS 项目，它在脚本任务中使用 Web API 检索天气信息（JSON 格式）。我一直在关注本教程：Weather data API，如果您只想从一组固定的坐标中检索天气信息，它会非常有用。 我现在的目标是使用一个表格，我在其中存储了一些坐标作为 API URL 参数中的变量输入，而不是已经在 URL https://api.met.no/weatherapi/locationforecast/2.0/compact.json?lat=55.596&lon=15 中设置坐标

所以我这样做是为了什么

创建了一个收集天气信息的脚本任务：

#region Help:  Introduction to the script task
/* The Script Task allows you to perform virtually any operation that can be accomplished in
 * a .Net application within the context of an Integration Services control flow. 
 * 
 * Expand the other regions which have "Help" prefixes for examples of specific ways to use
 * Integration Services features within this script task. */
#endregion


#region Namespaces
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
using System.Data.SqlClient;
using System.Net;
#endregion

namespace ST_6f60bececd8f4f94afaf758869590918

    /// <summary>
    /// ScriptMain is the entry point class of the script.  Do not change the name, attributes,
    /// or parent of this class.
    /// </summary>
    [Microsoft.SqlServer.Dts.Tasks.ScriptTask.SSISScriptTaskEntryPointAttribute]
    public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
    
        #region Help:  Using Integration Services variables and parameters in a script
        /* To use a variable in this script, first ensure that the variable has been added to 
         * either the list contained in the ReadOnlyVariables property or the list contained in 
         * the ReadWriteVariables property of this script task, according to whether or not your
         * code needs to write to the variable.  To add the variable, save this script, close this instance of
         * Visual Studio, and update the ReadOnlyVariables and 
         * ReadWriteVariables properties in the Script Transformation Editor window.
         * To use a parameter in this script, follow the same steps. Parameters are always read-only.
         * 
         * Example of reading from a variable:
         *  DateTime startTime = (DateTime) Dts.Variables["System::StartTime"].Value;
         * 
         * Example of writing to a variable:
         *  Dts.Variables["User::myStringVariable"].Value = "new value";
         * 
         * Example of reading from a package parameter:
         *  int batchId = (int) Dts.Variables["$Package::batchId"].Value;
         *  
         * Example of reading from a project parameter:
         *  int batchId = (int) Dts.Variables["$Project::batchId"].Value;
         * 
         * Example of reading from a sensitive project parameter:
         *  int batchId = (int) Dts.Variables["$Project::batchId"].GetSensitiveValue();
         * */

        #endregion

        #region Help:  Firing Integration Services events from a script
        /* This script task can fire events for logging purposes.
         * 
         * Example of firing an error event:
         *  Dts.Events.FireError(18, "Process Values", "Bad value", "", 0);
         * 
         * Example of firing an information event:
         *  Dts.Events.FireInformation(3, "Process Values", "Processing has started", "", 0, ref fireAgain)
         * 
         * Example of firing a warning event:
         *  Dts.Events.FireWarning(14, "Process Values", "No values received for input", "", 0);
         * */
        #endregion

        #region Help:  Using Integration Services connection managers in a script
        /* Some types of connection managers can be used in this script task.  See the topic 
         * "Working with Connection Managers Programatically" for details.
         * 
         * Example of using an ADO.Net connection manager:
         *  object rawConnection = Dts.Connections["Sales DB"].AcquireConnection(Dts.Transaction);
         *  SqlConnection myADONETConnection = (SqlConnection)rawConnection;
         *  //Use the connection in some code here, then release the connection
         *  Dts.Connections["Sales DB"].ReleaseConnection(rawConnection);
         *
         * Example of using a File connection manager
         *  object rawConnection = Dts.Connections["Prices.zip"].AcquireConnection(Dts.Transaction);
         *  string filePath = (string)rawConnection;
         *  //Use the connection in some code here, then release the connection
         *  Dts.Connections["Prices.zip"].ReleaseConnection(rawConnection);
         * */
        #endregion


        /// <summary>
        /// This method is called when this script task executes in the control flow.
        /// Before returning from this method, set the value of Dts.TaskResult to indicate success or failure.
        /// To open Help, press F1.
        /// </summary>
        public void Main()
        
            string Longitude =  (string)Dts.Variables["User::Longitude"].Value.ToString();
            string Latitude =  (string)Dts.Variables["User::Latitude"].Value.ToString();
            var url = @"https://api.met.no/weatherapi/locationforecast/2.0/compact.json?lat=55.596&lon=15";
            System.Net.ServicePointManager.DefaultConnectionLimit = int.MaxValue;
            HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
            req.UseDefaultCredentials = true;
            req.Proxy.Credentials = System.Net.CredentialCache.DefaultCredentials;
            var syncClient = new WebClient();
            syncClient.Headers.Add("user-agent", "acmeweathersite.com [e-mail]");
            var content = syncClient.DownloadString(url);

            string connectionString = "Data Source=localhost;Initial Catalog=Weather;Integrated Security=True;";

            using (SqlConnection conn = new SqlConnection(connectionString))
            
                SqlCommand Storproc =
                  new SqlCommand(@"INSERT INTO [dbo].[Weather] (JSONData)
                                    select @JSONData", conn);
                Storproc.Parameters.AddWithValue("@JSONData", content.ToString());
                conn.Open();
                Storproc.ExecuteNonQuery();
                conn.Close();

            
            // TODO: Add your code here

            Dts.TaskResult = (int)ScriptResults.Success;
        

        #region ScriptResults declaration
        /// <summary>
        /// This enum provides a convenient shorthand within the scope of this class for setting the
        /// result of the script.
        /// 
        /// This code was generated automatically.
        /// </summary>
        enum ScriptResults
        
            Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
            Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
        ;
        #endregion

    

创建了一个带有坐标的 SQL 表：

create table Coordinates(
             Municipality nvarchar(50),
             Latitide nvarchar(50),
             Longitude nvarchar(50)
             )
INSERT INTO Coordinates (Municipality, Latitide, Longitude)
VALUES (114, 59.5166667, 17.9),
        (115, 59.5833333, 18.2),
        (117, 59.5, 18.45)

将坐标表添加为 SQL 任务：

最后我在Script任务代码中添加了两个变量：

 string Longitude =  (string)Dts.Variables["User::Longitude"].Value.ToString();
string Latitude =  (string)Dts.Variables["User::Latitude"].Value.ToString();
var url = @"https://api.met.no/weatherapi/locationforecast/2.0/compact.json?lat=Latitude&lon=Longitude";

但是当我执行包时，Foreach 循环容器只是永远循环，不会弹出错误，但也没有数据存储在数据库表中。感觉就像我错过了一些东西，但不确定是什么。当涉及到 SSIS 中的变量时非常新手，所以请原谅我缺乏知识。在我的示例中，我使用的是 SQL Server 2019。

【参考方案1】：

看起来你快到了。

需要检查的几件事：

在您的 SQL 中，您选择了 3 列，因此，这些列的“索引”将是：

Municipality -> index = 0
Latitide -> index = 1
Longitude -> index = 2

即，在变量映射中，您需要使用索引 1 和 2，而不是全部使用零。

https://docs.microsoft.com/en-us/sql/integration-services/control-flow/foreach-loop-container?view=sql-server-ver15

枚举项中定义的第一列的索引值为 0，第二列的索引值为 1，依此类推。

您对列的拼写似乎也不同（Latitide Vs Latitude）。交叉检查这一点。也就是说，如果你手动运行你的 sql 语句，你能看到数据吗？结果的列名是什么？

您还可以通过添加MessageBox 来检查脚本任务中的变量（用于调试目的）。 例如，

string longitude = (string)Dts.Variables["User::Longitude"].Value.ToString();
string latitide = (string)Dts.Variables["User::Latitide"].Value.ToString();
string municipality = (string)Dts.Variables["User::Municipality"].Value.ToString();

MessageBox.Show("longitude:" + longitude + ", latitide:" + latitide + ", municipality: " + municipality);

【讨论】：

感谢索引设置解决了这个问题。 :)。唯一的问题是 JSON 数据被存储为 3 个不同的行（每个坐标一个），而不是一个包含所有内容的大数组。因此，当我将现在存储在我的一个名为“Weather”的表中的数组数据解析为另一个名为“WeatherByHour”的 sql 结构化格式的表时，它似乎只使用第一行。

我在这里有将 JSON 数据转换为表格结构化数据的代码示例：codeshare.io/21wBoz