为什么bash动态变量和字符串串联会扰乱/覆盖我的循环输出？

Posted

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了为什么bash动态变量和字符串串联会扰乱/覆盖我的循环输出？相关的知识，希望对你有一定的参考价值。

我正在循环浏览包含2个json文件的文件夹，从“名称”键中提取值并将它们存储在变量中。

然后，我尝试使用这些变量执行某些操作（例如：与curl请求的结果进行比较，并在控制台上显示正在发生的事情。

我遇到的问题是，脚本的回显输出似乎使一些变量和字符串文本杂乱无章。它似乎只发生在第一个循环中，第二个循环看起来还可以。

我不确定为什么会这样。查看下面的脚本，脚本输出和源文件，有人可以看到为什么我的输出在第一个循环中被弄乱了吗？也许这是我对动态变量的使用？

下面的脚本，脚本输出和文件+内容...

脚本

###############################
# SETTINGS
###############################
connectorsDirectory="sources"
connectHost=localhost


###############################
# INIT
###############################

# Get a list of all currently running connectors
connectorListResponse=$(curl -s $connectHost:8083/connectors)
runningConnectorList=$(echo $connectorListResponse | sed -e 's/[(.*)]/1/')
echo $runningConnectorList
echo "==="
# Convert runningConnectorList into something parseable by removing square brackets and double-quotes
parseableRunningConnectorList=`echo $runningConnectorList | tr -d '[]"'`
echo $parseableRunningConnectorList
echo "===="
connectorName=""
genericConnectorName=""

###############################
# JOB
###############################


# Loop through JSON files in provided connectors directory
for filename in $connectorsDirectory/*.json; do
    [ -e "$filename" ] || continue

    echo "Reading $filename ..."

    # Find connector name specified in file
    #connectorName=$(cat $filename | sed -n 's|.*"name":"([^"]*)".*|1|p')
    connectorName=$(sed -nE '/name/{s/.*:s*"(.*)",/1/p;q}' $filename)
    #connectorNameSED='/name/{s/.*:s*"(.*)",/1/p;q}'
    echo "connectorName="$connectorName


    # Find "generic" connector name (sans .v### at END of string)
    genericConnectorName=$(echo "$connectorName" | sed 's/.v[0-9]*$//')

    echo "genericConnectorName="$genericConnectorName


    if [ -z "$connectorName" ]
    then
        echo "No connector found in this file."
    else
        echo "Found connector in file named: "$filename" (generic name = "$genericConnectorName")"
    fi
    echo "---"


    # Loop through running connectors and find those that start with our "generic" connector name

    # If there are any connectors running AND we also found a connector in the current file...
    if [ ! -z "$parseableRunningConnectorList" ] && [ ! -z "$connectorName" ]
    then
        echo "Turning off the following connectors..."
        # Loop through running connectors and delete them (turn them off)
        for runningConnectorName in $(echo $parseableRunningConnectorList | sed "s/,/ /g")
        do
            connectorToTurnOff=$(echo "$runningConnectorName" | grep ^$genericConnectorName)
            if [ ! -z $connectorToTurnOff ]
            then
                echo $connectorToTurnOff
                echo "TEST MODE: curl -X DELETE $connectHost:8083/connectors/$connectorToTurnOff"
                #curl -X DELETE $connectHost:8083/connectors/$connectorToTurnOff
                #add some logic here to concat found connectors
            fi
        done
    else
        # No running connectors, so nothing to turn off ahead of publishing new connector (ie: do nothing)
        echo "No connector in this file or no running connectors needing to be stopped."
    fi

    # Publish the connector found in the current file
    if [ -z $connectorName ]
    then
        echo "No connector job to publish in the current file."
    else
        echo "Publishing connect job: "$connectorName"..."
        echo "TEST MODE: curl -X POST -H "Content-Type: application/json" --data @$filename http://$connectHost:8083/connectors"
        echo "TEST MODE: Connect job "$connectorName" is now active."
        #curl -X POST -H "Content-Type: application/json" --data @$filename http://$connectHost:8083/connectors
        #echo "Connect job "$connectorName" is now active."
    fi
    #
    echo ""
    echo "#############################################################################"
    echo ""


done

脚本输出

[centos@localhost test]$ ./publish_connectors.sh
"some.sink.connector.running","some.source.connector.running","cars.sales.test_v2","some.other.source.connector.v001"
===
some.sink.connector.running,some.source.connector.running,cars.sales.test_v2,some.other.source.connector.v001
====
Reading sources/stanbridge.connect.source.test.v01.json ...
connectorName=stanbridge.connect.source.test.v001
genericConnectorName=stanbridge.connect.source.test.v001
)ound connector in file named: sources/stanbridge.connect.source.test.v01.json (generic name = stanbridge.connect.source.test.v001
---
Turning off the following connectors...
"...ishing connect job: "stanbridge.connect.source.test.v001
TEST MODE: curl -X POST -H "Content-Type: application/json" --data @sources/stanbridge.connect.source.test.v01.json http://localhost:8083/connectors
" is now active.ct job "stanbridge.connect.source.test.v001

#############################################################################

Reading sources/cars.sales.test.json ...
connectorName=cars.sales.test.v001
genericConnectorName=cars.sales.test
Found connector in file named: sources/cars.sales.test.json (generic name = cars.sales.test)
---
Turning off the following connectors...
Publishing connect job: "cars.sales.test.v001"...
TEST MODE: curl -X POST -H "Content-Type: application/json" --data @sources/cars.sales.test.json http://localhost:8083/connectors
TEST MODE: Connect job "cars.sales.test.v001" is now active.

#############################################################################

File1：sources / stanbridge.connect.source.test.v01.json

{
  "name": "stanbridge.connect.source.test.v001",
  "config": {
    "connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector",
    "group.id": "stanbridge.connect.source.test.v01xxyyzza",
    "key.converter": "io.confluent.connect.avro.AvroConverter",
    "value.converter": "io.confluent.connect.avro.AvroConverter",
    "key.converter.schema.registry.url": "https://my.schema.registry",
    "value.converter.schema.registry.url": "https://my.schema.registry",
    "key.converter.basic.auth.user.info": "someKeyString:someSecretString",
    "value.converter.basic.auth.user.info": "SsomeSString:someSecretString",
    "errors.log.enable": "true",
    "errors.log.include.messages": "true",
    "connection.url": "jdbc:postgresql://my.db.host/dbName",
    "connection.user": "postgres",
    "connection.password": "postgres",
    "mode": "incrementing",
    "incrementing.column.name": "audit_id",
    "query": "SELECT customer_id, customer_name, audit_id FROM cars.customers",
    "topic.prefix": "stanbridge.connect.source.test"
  }
}

File2：sources / cars.sales.test.json

{"name": "cars.sales.test.v001",
  "config":     {
    "connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector",
    "group.id": "cars.sales.test.v001",
    "value.converter": "io.confluent.connect.avro.AvroConverter",
    "value.converter.schema.registry.url": "https://my.schema.registry",
    "value.converter.basic.auth.credentials.source": "USER_INFO",
    "value.converter.schema.registry.basic.auth.user.info": "someKeyString:someSecretString",
    "key.converter": "io.confluent.connect.avro.AvroConverter",
    "key.converter.schema.registry.url": "https://my.schema.registry",
    "key.converter.basic.auth.credentials.source": "USER_INFO",
    "key.converter.schema.registry.basic.auth.user.info": "someKeyString:someSecretString",
    "errors.log.enable": "true",
    "errors.log.include.messages": "true",
    "tasks.max": "2",
    "batch.max.rows": "100",
    "connection.user": "postgres",
    "connection.password": "",
    "connection.url": "jdbc:postgresql://my.db.host/dbName",
    "query": "SELECT car_id, field1, field2, field3, date_modified_utc FROM cars.sales",
    "mode": "timestamp",
    "timestamp.column.name": "date_modified_utc",
    "poll.interval.ms": "5000",
    "timeout.ms": "25000",
    "topic.prefix": "cars.sales.dummy",
    "transforms": "createKey",
    "transforms.createKey.type": "org.apache.kafka.connect.transforms.ValueToKey",
    "transforms.createKey.fields": "car_id"
        }
}

在脚本输出中，请注意第一个循环中的以下内容...

  • sed不会删除.v001后缀：genericConnectorName = stanbridge.connect.source.test.v001直接在命令行中使用时可以使用此功能

  • 文本输出中的文本被覆盖，以：）文件名中的连接器>]

  • 行输出中的文本被覆盖，开头是：“ ... ishing连接作业：” stanbridge.connect.source.test.v001

  • 以以下内容开头的行输出中覆盖的文本现在处于活动状态。ct作业“ stanbridge.connect.source.test.v001

    • NB：我认为如果将整个脚本包括在内，那不容易理解，这将是最容易理解的。抱歉，如果这是不好的形式。

非常感谢您提供的任何帮助。谢谢！

Stanbridge

我正在遍历包含2个json文件的文件夹，从“名称”键中提取值并将它们存储在变量中。然后，我尝试使用这些变量执行一些操作（例如：...

答案

@ lucas FYI ...使用jq确实是答案。我不确定如何，但是尝试提取名称的脚本中的sed导致了问题，特别是：

以上是关于为什么bash动态变量和字符串串联会扰乱/覆盖我的循环输出？的主要内容，如果未能解决你的问题，请参考以下文章

Leetcode No.87 扰乱字符串（动态规划）

为啥此示例在字符串比较中使用空填充？ “编程珍珠”：一串串珍珠

LeetCode87. 扰乱字符串

如何在 bash 中使用带有特殊字符的变量？ [复制]

bash 中带有变量、大括号和哈希字符的 $0##... 语法是啥意思？

[leetcode] 87. 扰乱字符串