Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
A
Amazon-Selection-Data
Overview
Overview
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
abel_cjy
Amazon-Selection-Data
Commits
c3c84cf5
Commit
c3c84cf5
authored
May 06, 2026
by
fangxingjun
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
任务编排,测试通过
parent
05c13cdd
Show whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
6 additions
and
1 deletions
+6
-1
import_st_to_pg14.py
Pyspark_job/listen_program/import_st_to_pg14.py
+2
-0
export_dwt_asin_sync.py
Pyspark_job/sqoop_export/export_dwt_asin_sync.py
+4
-1
No files found.
Pyspark_job/listen_program/import_st_to_pg14.py
View file @
c3c84cf5
...
...
@@ -187,6 +187,7 @@ class ImportStToPg14(object):
spider_is_ready,
spider_state,
bg_name,
bg_dol_name,
bg_dol_state
)
VALUES
...
...
@@ -201,6 +202,7 @@ class ImportStToPg14(object):
'yes',
1,
'{self.site_name}_asin_export',
'ALL站点-asin同步-pg-api',
1
)
ON DUPLICATE KEY UPDATE
...
...
Pyspark_job/sqoop_export/export_dwt_asin_sync.py
View file @
c3c84cf5
...
...
@@ -36,6 +36,7 @@ def update_workflow_manager(site_name, date_type, date_info):
spider_is_ready,
spider_state,
bg_name,
bg_dol_name,
bg_dol_state,
finished_count
)
...
...
@@ -51,13 +52,15 @@ def update_workflow_manager(site_name, date_type, date_info):
'yes',
1,
'{site_name}_all_cal',
'ALL-月流程-ABA+反查+流量选品',
1,
0
)
ON DUPLICATE KEY UPDATE
spider_is_ready = VALUES(spider_is_ready),
spider_script = VALUES(spider_script),
spider_state = VALUES(spider_state);
spider_state = VALUES(spider_state),
bg_dol_name = VALUES(bg_dol_name);
"""
print
(
f
"workflow_manager进度表---重置爬虫的asin抓取进度: {update_sql_workflow_spider}"
)
engine_mysql
.
execute
(
update_sql_workflow_spider
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment