如何在流分析作业 ARM 模板中指定数据库表作为输出
Posted
技术标签:
【中文标题】如何在流分析作业 ARM 模板中指定数据库表作为输出【英文标题】：How to specify database table as output in Stream Analytics Job ARM template 【发布时间】：2018-09-25 17:02:50 【问题描述】：我正在通过 Azure 资源管理器 (ARM) 模板定义流分析作业。
我需要将来自事件中心输入的一些消息持久化到 SQL Server 数据库输出中。
运行此命令时：
az group deployment create \
--name "deployStreamAnalyticsJobs" \
--resource-group "CiTestRG" \
--template-file ./templates/stream-analytics-jobs-definition.json
然后我看到这个输出：
Deployment failed. Correlation ID: <ONE_GUID>.
"code": "422",
"message": "The required property 'type' is missing from the request.",
"details":
"code": "422",
"message": "The required property 'type' is missing from the request.",
"correlationId": "<ANOTHER_GUID>",
"requestId": "<YET_ANOTHER_GUID>"
但根据本文档：https://docs.microsoft.com/en-us/azure/templates/microsoft.streamanalytics/streamingjobs#MicrosoftSqlServerDatabaseOutput
我应该只提供数据库输出详细信息：
"datasource":
"type": "Microsoft.Sql/Server/Database",
"properties":
"server": "string",
"database": "string",
"user": "string",
"password": "string",
"table": "string"
我认为az 命令的错误消息试图说的是这些属性没有定义为string，但是为什么文档没有指定类型而只是将这些属性作为字符串值传递?
这是 ARM 模板的 JSON 定义：
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters":
"location":
"type": "string",
"defaultValue": "westeurope"
,
"hubName":
"type": "string",
"defaultValue": "myIoTHub"
,
"eventhubs_messages_name":
"defaultValue": "myEhName",
"type": "String"
,
"namespaces_oecollector_name":
"defaultValue": "myEventHub",
"type": "String"
,
"streamAnalyticsJobName":
"type": "string",
"defaultValue": "myStreamAnalyticsJob"
,
"resources": [
"type": "Microsoft.StreamAnalytics/StreamingJobs",
"apiVersion": "2016-03-01",
"name": "[parameters('streamAnalyticsJobName')]",
"location": "[resourceGroup().location]",
"properties":
"sku":
"name": "standard"
,
"outputErrorPolicy": "Drop",
"eventsOutOfOrderPolicy": "adjust",
"eventsOutOfOrderMaxDelayInSeconds": 0,
"eventsLateArrivalMaxDelayInSeconds": 86400,
"inputs": [
"Name": "EventHubOutputLable",
"Properties":
"DataSource":
"Type": "Microsoft.ServiceBus/EventHub",
"Properties":
"eventHubName": "[parameters('eventhubs_messages_name')]",
"serviceBusNamespace": "[parameters('namespaces_oecollector_name')]",
"sharedAccessPolicyKey": "[listKeys(resourceId('Microsoft.EventHub/namespaces/eventhubs/authorizationRules', parameters('namespaces_oecollector_name'), parameters('eventhubs_messages_name'), 'super-user'),'2017-04-01').primaryKey]",
"sharedAccessPolicyName": "RootManageSharedAccessKey"
,
"Serialization":
"Properties":
"Encoding": "UTF8"
,
"Type": "Json"
],
"outputs": [
"Name": "myOutputDb",
"Properties":
"DataSource":
"Type": "Microsoft.Sql/Server/Database",
"Properties":
"Server": "my-sql-server-name",
"Database": "my-database-name",
"User": "my-sa-user",
"Password": "my-password",
"Table": "MySchema.MyTable"
],
"transformation":
"name": "Transformation",
"properties":
"streamingUnits": 1,
"query": "WITH data AS (\r\n SELECT\r\n <THE_FIELDS_OF_THE_JSON_INPUT_AS_COLUMNS>\r\n FROM\r\n input\r\n WHERE\r\n topic = 'FOO'\r\n )\r\n \r\n SELECT * INTO myOutputDb FROM data"
]
如何让这个流分析作业从输入事件中心挑选消息并将它们存储到输出 SQL Server 数据库中？
【问题讨论】：
【参考方案1】：您的输入属性对象具有“数据源”和“序列化”，但缺少“类型”。没有这个，就无法知道输入是流输入还是引用数据输入。添加后，您的输入属性对象应如下所示：
"Properties":
"type": "stream",
"DataSource":
"Type": "Microsoft.ServiceBus/EventHub",
"Properties":
"eventHubName": "[parameters('eventhubs_messages_name')]",
"serviceBusNamespace": "[parameters('namespaces_oecollector_name')]",
"sharedAccessPolicyKey": "[listKeys(resourceId('Microsoft.EventHub/namespaces/eventhubs/authorizationRules', parameters('namespaces_oecollector_name'), parameters('eventhubs_messages_name'), 'super-user'),'2017-04-01').primaryKey]",
"sharedAccessPolicyName": "RootManageSharedAccessKey"
,
"Serialization":
"Properties":
"Encoding": "UTF8"
,
"Type": "Json"
【讨论】：
以上是关于如何在流分析作业 ARM 模板中指定数据库表作为输出的主要内容，如果未能解决你的问题，请参考以下文章