python core-spark-filtering-data.py

Posted

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了python core-spark-filtering-data.py相关的知识，希望对你有一定的参考价值。

#Filtering the Data
orders = sc.textFile("/public/retail_db/orders")

ordersStatuses = orders.map(lambda order: order.split(",")[3])
for orderStatus in orderStatuses.collect(): print(orderStatus)

ordersFiltered = orders.\
filter(lambda order: order.split(",")[3] == "COMPLETE" or order.split(",")[3] == "CLOSED")

for order in ordersFiltered.take(10): print(order)

以上是关于python core-spark-filtering-data.py的主要内容，如果未能解决你的问题，请参考以下文章