Commit f6f075d7 by 吴济苍

[fix]调整pcp竞价数据兼容任务 监听频率从改为10min一次

parent dbbb751a
...@@ -34,7 +34,7 @@ def handle_listen(checkpoint): ...@@ -34,7 +34,7 @@ def handle_listen(checkpoint):
writing_sink = kafka_reader.writeStream \ writing_sink = kafka_reader.writeStream \
.option("checkpointLocation", f"/tmp/spark_kafka/checkpoint/{checkpoint}") \ .option("checkpointLocation", f"/tmp/spark_kafka/checkpoint/{checkpoint}") \
.foreachBatch(handle_foreach_save_hive) \ .foreachBatch(handle_foreach_save_hive) \
.trigger(processingTime=f'100 seconds').start() .trigger(processingTime=f'10 minutes').start()
writing_sink.awaitTermination() writing_sink.awaitTermination()
spark.stop() spark.stop()
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment