markdown 星火学习

Posted 2021-05-06

# 启动spark交互式shell(python)

```zsh
# 安装pyspark
$ pip install pyspark

# 设置环境变量
$ export SPARK_HOME=/home/cloudera/py3project/venv/lib/python3.5/site-packages/pyspark
$ export PYSPARK_DRIVER_PYTHON=/home/cloudera/py3project/venv/bin/ipython
$ export PYSPARK_DRIVER_PYTHON_OPTS='notebook'

$ pyspark
```