azkaban job参数传递

Posted 2021-02-16 zzhuang

Parameter Passing
There is often a desire to pass these parameters to the executing job code. The method of passing these parameters is dependent on the jobtype that is run, but usually Azkaban writes these parameters to a temporary file that is readable by the job.
The path of the file is set in JOB_PROP_FILE environment variable. The format is the same key value pair property files. Certain built-in job types do this automatically for you. The java type, for instance, will invoke your Runnable and given a proper constructor, Azkaban can pass parameters to your code automatically.
Parameter Output
Properties can be exported to be passed to its dependencies. A second environment variable JOB_OUTPUT_PROP_FILE is set by Azkaban. If a job writes a file to that path, Azkaban will read this file and then pass the output to the next jobs in the flow.
The output file should be in json format. Certain built-in job types can handle this automatically, such as the java type.
以上是官方的使用描述，相对来说比较简单，给初用者带来一定的难度。下面举个例子说明。

JOB_OUTPUT_PROP_FILE和JOB_PROP_FILE都是一个环境变量，指向文件路径。上游节点把需要输出的值以json的格式写入JOB_OUTPUT_PROP_FILE文件，下游节点就可以在JOB_PROP_FILE中看到key-value形式的输出，用${key}的方式使用变量。
实际例子：
baseflow.flow
nodes:
- name: jobB
type: command
dependsOn:
- jobA
config:
command: sh commandB.sh "${firstName}"
- name: jobA
type: command
config:
command: sh commandA.sh

commandA.sh
#!/bin/bash
echo ‘{ "firstName":"John" , "lastName":"Doe" }‘ >> ${JOB_OUTPUT_PROP_FILE}

commandB.sh
#!/bin/bash
#cat ${JOB_PROP_FILE} >> /root/azkaban.txt
echo $1 >> /root/azkaban.txt

commandB.sh 脚本执行完成之后，将会把firstName变量值John的值写入文件/root/azkaban.txt文件。