markdown サーバーの环境を変えないでJQコマンドっぽいことをやる方法を考える

Posted 2021-05-05

[
  {
    "id": "test_Aid",
    "pass": "test_Apass"
  },
  {
    "id": "test_Bid",
    "pass": "test_Bpass"
  },
  {
    "id": "test_Cid",
    "pass": "test_Cpass"
  }
]

{
  "id": "test_id",
  "pass": "test_pass"
}

id,pass,
test_Aid,test_Apass,
test_Bid,test_Bpass,
test_Cid,test_Cpass,

# python_jq
- [github page](https://shimajima-eiji.github.io/gist/python_jq/)
- [discription](https://nomuraya.work/techzine/0171)
- [back](../)

# get_CSVcount.sh
[view latest](https://gist.github.com/shimajima-eiji/401508ad48e248c6371e30bec1fcadb6#file-get_CSVcount-sh)

```
CSV_PATH='test.csv'  # 行数で管理できるものであれば何でもよい。TSVとか

# CSV_PATH以下のファイルの行数を取得。catではなくhead -n -1を採用したのはCSVヘッダーを除くため
csv_count=$( \
find ${CSV_PATH} -type f -name '*.csv' -exec head -n -1 {} \; | \
wc -l \
)
```

# get_JSONcount_01.sh
[view latest](https://gist.github.com/shimajima-eiji/401508ad48e248c6371e30bec1fcadb6#file-get_JSONcount_01-sh)

```
JSON_PATH='test.json'

json_count=$( \
find . -type f -name '*.json' -exec jq length {} \; | \
xargs echo | \
tr ' ' '+' | \
bc \
)

# Result: jqコマンドが見つかりません
```

# get_JSONcount_02.sh
[view latest](https://gist.github.com/shimajima-eiji/401508ad48e248c6371e30bec1fcadb6#file-get_JSONcount_02-sh)

```
JSON_PATH='test.json'
JSON_DIR=

pycmd=$(cat <<__PYCMD
import sys;
from pathlib import Path;
import json;
print(sys.argv[1]);
print(len(json.load(Path(sys.argv[1]).open())))
__PYCMD
)

python -c '${pycmd}' ${JSON_PATH}

# これをsshコマンドで実行させる。ssh関連は略
json_count=$( \
ssh ${host} "find ${JSON_DIR} -type f -exec python -c '${pycmd}' {} \;" | \
xargs echo | \
tr ' ' '+' | \
bc \
)
```

# test.csv
[view latest](https://gist.github.com/shimajima-eiji/401508ad48e248c6371e30bec1fcadb6#file-test-csv)

```
id,pass,
test_Aid,test_Apass,
test_Bid,test_Bpass,
test_Cid,test_Cpass,
```

# test.json
[view latest](https://gist.github.com/shimajima-eiji/401508ad48e248c6371e30bec1fcadb6#file-test-json)

```
{
  "id": "test_id",
  "pass": "test_pass"
}
```

# test_json_list.json
[view latest](https://gist.github.com/shimajima-eiji/401508ad48e248c6371e30bec1fcadb6#file-test_json_list-json)

```
[
  {
    "id": "test_Aid",
    "pass": "test_Apass"
  },
  {
    "id": "test_Bid",
    "pass": "test_Bpass"
  },
  {
    "id": "test_Cid",
    "pass": "test_Cpass"
  }
]
```

[back](../)

JSON_PATH='test.json'
JSON_DIR=

pycmd=$(cat <<__PYCMD
import sys;
from pathlib import Path;
import json;
print(sys.argv[1]);
print(len(json.load(Path(sys.argv[1]).open())))
__PYCMD
)

python -c '${pycmd}' ${JSON_PATH}

# これをsshコマンドで実行させる。ssh関連は略
json_count=$( \
ssh ${host} "find ${JSON_DIR} -type f -exec python -c '${pycmd}' {} \;" | \
xargs echo | \
tr ' ' '+' | \
bc \
)

JSON_PATH='test.json'

json_count=$( \
find . -type f -name '*.json' -exec jq length {} \; | \
xargs echo | \
tr ' ' '+' | \
bc \
)

# Result: jqコマンドが見つかりません

CSV_PATH='test.csv'  # 行数で管理できるものであれば何でもよい。TSVとか

# CSV_PATH以下のファイルの行数を取得。catではなくhead -n -1を採用したのはCSVヘッダーを除くため
csv_count=$( \
find ${CSV_PATH} -type f -name '*.csv' -exec head -n -1 {} \; | \
wc -l \
)

# python_jq
- [github page](https://shimajima-eiji.github.io/gist/python_jq/)
- [discription](https://nomuraya.work/techzine/0171)