Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
A
Amazon-Selection-Data
Overview
Overview
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
abel_cjy
Amazon-Selection-Data
Commits
def9f63d
Commit
def9f63d
authored
Apr 29, 2026
by
fangxingjun
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
no message
parent
df8b11be
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
5 additions
and
3 deletions
+5
-3
export_dwt_asin_sync.py
Pyspark_job/sqoop_export/export_dwt_asin_sync.py
+5
-3
No files found.
Pyspark_job/sqoop_export/export_dwt_asin_sync.py
View file @
def9f63d
import
os
import
sys
import
time
import
traceback
sys
.
path
.
append
(
os
.
path
.
dirname
(
sys
.
path
[
0
]))
from
utils.secure_db_client
import
get_remote_engine
...
...
@@ -54,9 +55,10 @@ def update_workflow_manager(site_name, date_type, date_info):
"""
print
(
f
"workflow_manager进度表---重置爬虫的asin抓取进度: {update_sql_workflow}"
)
conn
.
execute
(
update_sql_workflow
)
break
except
Exception
as
e
:
print
(
f
"失败:workflow_manager进度表---重置爬虫的asin抓取进度: {update_sql_workflow}, {e}, 报错信息: {traceback.format_exc()}"
)
time
.
sleep
(
300
)
continue
...
...
@@ -97,5 +99,5 @@ if __name__ == '__main__':
date_type
=
sys
.
argv
[
2
]
# 参数2:类型:week/4_week/month/quarter/day
date_info
=
sys
.
argv
[
3
]
# 参数3:年-周/年-月/年-季/年-月-日, 比如: 2022-1
export_data
(
site_name
,
date_type
,
date_info
)
if
site_name
in
[
"us"
,
"uk"
,
"de"
]
and
date_type
==
"month"
:
update_workflow_manager
(
site_name
,
date_type
,
date_info
)
#
if site_name in ["us", "uk", "de"] and date_type == "month":
#
update_workflow_manager(site_name, date_type, date_info)
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment