Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
A
Amazon-Selection-Data
Overview
Overview
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
abel_cjy
Amazon-Selection-Data
Commits
ed081d26
Commit
ed081d26
authored
May 13, 2026
by
chenyuanjie
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
fix
parent
0c315139
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
3 additions
and
2 deletions
+3
-2
es_asin_profit_rate.py
Pyspark_job/export_es/es_asin_profit_rate.py
+3
-2
No files found.
Pyspark_job/export_es/es_asin_profit_rate.py
View file @
ed081d26
...
...
@@ -108,12 +108,13 @@ class EsAsinProfitRate(object):
)
# 从Doris获取asin_crawl_date和asin_price,用profit_key关联
df_crawl_date
=
DorisHelper
.
spark_import_with_flight
(
# 改用 Doris Spark Connector(HTTP fenodes)读取,规避 JDBC/FlightSQL 在 DATETIME/NULL 列上的兼容问题
df_crawl_date
=
DorisHelper
.
spark_import_with_connector
(
session
=
self
.
spark
,
table_identifier
=
f
"selection.{self.site_name}_asin_latest_detail"
,
read_fields
=
"asin,asin_crawl_date,asin_price"
)
.
withColumn
(
"asin_crawl_date"
,
F
.
substring
(
F
.
col
(
"asin_crawl_date"
),
1
,
10
)
"asin_crawl_date"
,
F
.
substring
(
F
.
col
(
"asin_crawl_date"
)
.
cast
(
"string"
)
,
1
,
10
)
)
.
withColumn
(
"profit_key"
,
F
.
concat_ws
(
"_"
,
F
.
col
(
"asin"
),
F
.
round
(
F
.
col
(
"asin_price"
),
2
))
)
.
select
(
'profit_key'
,
'asin_crawl_date'
)
\
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment