Compare commits
No commits in common. "codex/plan-recovery-20261105" and "main" have entirely different histories.
codex/plan
...
main
26
README.md
26
README.md
|
|
@ -1,7 +1,5 @@
|
|||
# akg-factor-bridge
|
||||
|
||||
2026年11月5日:补齐数据迟到后的有限自动恢复及构建、冻结的沪深范围保护。保持日期时效与真实版本时间,说明与实际发布状态见[恢复接续](docs/选股输入迟到恢复与沪深边界_2026-11-05.md)。
|
||||
|
||||
astock-kg(知识图谱基座)↔ `quant_factor_service`(通用因子平台)之间的**因子导出桥**。
|
||||
把基座的四路产出(分析师预期空间 / 热度 / 利好利空事件 / 板块传导)做成符合平台规范的
|
||||
子因子;**复合因子 `akg_score` 在桥内用自洽的景气度漏斗算出**,平台只负责注册/存储/
|
||||
|
|
@ -185,7 +183,7 @@ docker compose exec akg-factor-bridge python run.py freeze --date 2026-07-24
|
|||
`source_type` 不是这个字符串,事件因子会被过滤成空(会显式告警),按实际值改
|
||||
`EVENT_SOURCE_TYPES`。
|
||||
|
||||
## HTTP 接口(桥常驻 API,端口 8300)
|
||||
## HTTP 接口(桥常驻 API,端口 8300,全部只读除 refresh)
|
||||
|
||||
| 端点 | 作用 |
|
||||
|---|---|
|
||||
|
|
@ -193,10 +191,6 @@ docker compose exec akg-factor-bridge python run.py freeze --date 2026-07-24
|
|||
| `GET /plan/dates?limit=30` | 有计划文件的日期清单 |
|
||||
| `GET /plan?date=&format=json\|md&top=&obs_top=` | 当日选股计划(基座今日页 `/plan/today` 与 PMS 下单候选都读它)。顶层 `regime`(区制)与 `market`(两市成交额、广度、融资、恐贪四项)都只从当日快照读,快照缺失为空;每行的判决类字段 2026-09-03 起多 `basis`(判决依据)与 `logic`(因果论断带出处);2026-09-17 起名次改按质地带优先,每行多 `quality_band`(质地带:好 3 中 2 无评析 1 差 0)、`rank_axis`(quality/transmission)、`pointed`(是否被传导指向)、`rank_old`(旧序名次,供周报四周对照),开关 `PLAN_RANK_AXIS` 设 transmission 即逐字回旧 |
|
||||
| `POST /plan/refresh?date=&key=` | 使用既有 XXL key 排入完整修订;晨版缺失不代造 |
|
||||
| `GET /plan/user-candidates/search?q=` | 按名称或代码查找沪深 A 股 |
|
||||
| `GET /plan/user-candidates` | 人工关注登记与实时评析状态;只读,不提交评析任务 |
|
||||
| `POST /plan/user-candidates` | 经既有 `X-Job-Key` 鉴权登记人工关注;保存后由后台补齐评析并尝试计划修订 |
|
||||
| `POST /plan/user-candidates/{id}/remove` | 经既有 `X-Job-Key` 鉴权移除关注,保留原登记记录 |
|
||||
| `GET /plan/verdict?codes=300750,SH600438&date=` | 逐票『计划判决』(2026-08-18 PMS 对接加):decision(main/observe/reject/absent)+ 与命令行 `plan_reconcile` 逐字同口径的 verdict_text + score/rank/tier/赛道/图谱证据。单票数据异常或代码形态认不出只报该票 error,不崩整批 |
|
||||
| `GET\|POST /api/v1/xxl/daily-build?key=&steps=&date=` | XXL-JOB 平台触发盘前链(build→plan→push-pool,白名单步骤、单实例互斥、回调结案);`.env` 不配 `XXL_TRIGGER_KEY` 则整组禁用 |
|
||||
| `GET /review/latest` | 最新一份复盘周报的只读读数(2026-09-16 加,配合数据基座判收读数包):文件名、日期、大小、前二十行、三个标记行计数(`公司质地=`、`五日头`、`二十日头`)。没有周报回 404。纯文件读取,不连库 |
|
||||
|
|
@ -204,24 +198,6 @@ docker compose exec akg-factor-bridge python run.py freeze --date 2026-07-24
|
|||
逐票对账的命令行形态(同一段判决代码):
|
||||
`docker compose exec -T akg-factor-bridge python plan_reconcile.py 300750 [--date 2026-08-04]`
|
||||
|
||||
## 人工关注评析与吸筹参考规则(2026-11-04)
|
||||
|
||||
用户已明确选择沿用公司质地排序,吸筹作为参考,建仓由 PMS 的基本面、技术面、择时三源合议及后续规则判断。本次仅授权在 **155 模拟链路**配置 `CARD_ACCUM_CONFIRM_REQUIRED=0`。代码默认仍为 `1`,保留旧确认规则的回退能力;188 真实 PMS 不在本次切换范围。实际部署、参数生效与自然回执另记发布判收,不能用本地测试代替上线结论。
|
||||
|
||||
关闭确认要求后,潜在吸筹、其他非明确状态、缺评分或评分陈旧不再单独把股票降为“仅展示”或“关注”;原评分、状态与新鲜度仍保留为研究证据。不新增吸筹权重或综合分,质地排序与原始分数编码保持。券商覆盖、预期空间、证券资格、既有真实风险否决、高位派发和快照时点校验继续生效;人工来源只替代原发现入口。PMS 的三源合议、规则闸、研判、资金、组合及执行约束仍决定后续动作,研究完成或进入候选均不等于下单授权。
|
||||
|
||||
`plan-refresh` 只将本次获准的吸筹确认参数同步到新修订,其他冻结参数继续沿用原版本。即使研究输入未变,规则变化也必须追加带真实截止时刻、新参数指纹及明确原因的完整版本。`card_params.admission_rule` 区分确认规则与参考规则,`card_params.accum_confirm_required` 表明当前要求。旧日期、固定决策版本与晨版文件保持原样,不重写历史判决,也不倒填晨版。
|
||||
|
||||
人工关注走独立的真实买方评析链。登记后的后台任务与自然 `run.py plan-refresh` 周期先同步评析状态;GET 查询只批量读取真实报告。全部有效沪深人工关注加入原 `t_akg_review_targets` 目标名单,不受主榜前列、原分数或交易判决限制。缺少完整报告时,后台仅向配置的 `AKG_API_BASE` 提交原个股评析入口,要求买方评议和定性材料,使用 `force=false`,复用基座既有评析逻辑。
|
||||
|
||||
请求前先原子保存独立受理凭据,实际返回入队回执后才标记 `queued`。网络超时或受理结果不明时保持 `pending`,不立即重复提交;后续根据真实报告核验,并由原目标名单的自然批任务补齐。状态存于 `user_research.json`,不修改交易账本或历史计划。每轮默认最多提交 3 只、单次请求超时 3 秒;超过本轮额度的对象留待后续自然周期。移除或到期后停止新增提交与下游关注覆盖。
|
||||
|
||||
`GET /plan/user-candidates` 中,原 `status` 的 `pending/evaluated` 仅表示计划是否收录。独立的 `review_status` 为 `missing/pending/queued/completed/failed`,并附 `review_updated_at`、`review_error`、`review_data_date`、`company_review`、`company_review_text` 与 `review_ref`。任务标识不作为产品状态展示。
|
||||
|
||||
评析完成必须有真实固定版本引用、完整成功的买方评议、买方论点及不晚于当前时刻的报告生成时间。新鲜度按基座真实 `ran_at` 与 `checked_at` 中较新者检查,默认 7 天;轻量复核命中原指纹时可以更新核对时间,不能把它改写为新报告。`review_data_date` 始终取 `company_review.report_ran_at` 的北京时间日期。财报期是否落后仍由基座原有批任务判断;旧报告缺固定引用或评议未完成时明确等待补齐,不能伪造完成状态。
|
||||
|
||||
开发机验证:新增评析专项 19 项通过,包含吸筹参考规则的 35 个测试文件均经配置隔离、禁网护栏通过。候选镜像断网回归、真实评析受理与完成、自然日研究及盘中观察回执仍须在发布阶段分别核验。规则决定见[复盘决定台账 084](docs/复盘决定台账.md)。
|
||||
|
||||
## 里程碑状态(设计 §8)
|
||||
|
||||
- **G2 赛道门槛 ✅**:`config/frontier_tracks.yml` + `tracks.py`,三路映射
|
||||
|
|
|
|||
82
api.py
82
api.py
|
|
@ -20,7 +20,7 @@ import time
|
|||
import threading
|
||||
|
||||
import pandas as pd
|
||||
from fastapi import FastAPI, HTTPException, Request, Body, Header, BackgroundTasks
|
||||
from fastapi import FastAPI, HTTPException, Request
|
||||
from fastapi.responses import PlainTextResponse, JSONResponse, HTMLResponse
|
||||
|
||||
import card
|
||||
|
|
@ -96,58 +96,6 @@ def dividend_candidate_feed(code: str = None):
|
|||
raise HTTPException(status_code=503, detail="高股息资料暂不可用,请稍后再试") from exc
|
||||
|
||||
|
||||
@app.get('/plan/user-candidates/search')
|
||||
def search_user_candidates(q: str = ''):
|
||||
import user_candidates as uc
|
||||
query = q.strip().upper()[:40]
|
||||
if not query:
|
||||
return {'ok': True, 'items': []}
|
||||
names = uc.catalog()
|
||||
items = []
|
||||
for code, name in names.items():
|
||||
try:
|
||||
uc.normalize(code)
|
||||
except ValueError:
|
||||
continue
|
||||
if query in code or query in name.upper():
|
||||
items.append({'code': code, 'name': name})
|
||||
return {'ok': True, 'items': sorted(items, key=lambda x: x['code'])[:30]}
|
||||
|
||||
|
||||
@app.get('/plan/user-candidates')
|
||||
def list_user_candidates():
|
||||
import user_candidates as uc
|
||||
return uc.listing(config.PLAN_SNAPSHOT_DIR)
|
||||
|
||||
|
||||
@app.post('/plan/user-candidates')
|
||||
def add_user_candidates(background_tasks: BackgroundTasks, payload: dict = Body(...),
|
||||
x_job_key: str = Header(None)):
|
||||
from xxl import _require_key
|
||||
import user_candidates as uc
|
||||
_require_key(x_job_key, None)
|
||||
try:
|
||||
result = uc.save(config.PLAN_SNAPSHOT_DIR, payload, payload.get('actor') or 'PMS', names=uc.catalog())
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||
background_tasks.add_task(uc.refresh_saved)
|
||||
return result
|
||||
|
||||
|
||||
@app.post('/plan/user-candidates/{identity}/remove')
|
||||
def remove_user_candidate(identity: str, background_tasks: BackgroundTasks,
|
||||
payload: dict = Body(...), x_job_key: str = Header(None)):
|
||||
from xxl import _require_key
|
||||
import user_candidates as uc
|
||||
_require_key(x_job_key, None)
|
||||
try:
|
||||
result = uc.remove(config.PLAN_SNAPSHOT_DIR, identity, payload.get('actor') or 'PMS')
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||
background_tasks.add_task(uc.refresh_saved)
|
||||
return result
|
||||
|
||||
|
||||
@app.get("/plan/dates")
|
||||
def plan_dates(limit: int = 30):
|
||||
from pathlib import Path
|
||||
|
|
@ -180,13 +128,12 @@ _PLAN_CACHE_MAX = 8 # 不同参数组合最多留几份,防内存慢
|
|||
_PLAN_BUILD_LOCKS: dict = {} # 每个缓存键一把「正在算」的锁,见 _plan_cached 里的说明
|
||||
|
||||
|
||||
def _plan_build(date, top, obs_top, theme_cap, decision_id=None, include_user_candidates=False) -> dict:
|
||||
def _plan_build(date, top, obs_top, theme_cap, decision_id=None) -> dict:
|
||||
"""只读已发布版本。缺档明确报错,禁止用当前研究重构历史。"""
|
||||
return plan.from_snapshot(date, top, obs_top, theme_cap, decision_id=decision_id,
|
||||
include_user_candidates=include_user_candidates)
|
||||
return plan.from_snapshot(date, top, obs_top, theme_cap, decision_id=decision_id)
|
||||
|
||||
|
||||
def _plan_cached(date, top, obs_top, theme_cap, nocache: bool, decision_id=None, include_user_candidates=False):
|
||||
def _plan_cached(date, top, obs_top, theme_cap, nocache: bool, decision_id=None):
|
||||
"""返回 (data, 命中与否)。data 是 plan.collect 的结果(已去掉 _full)。
|
||||
|
||||
键里放**真实数据日**而不是请求里的 date:不传 date 时数据日随构建更新,明早换日后
|
||||
|
|
@ -196,10 +143,7 @@ def _plan_cached(date, top, obs_top, theme_cap, nocache: bool, decision_id=None,
|
|||
ds_key = published['date']
|
||||
pinned = (published.get('decision') or {}).get('id')
|
||||
state = pv.health(config.PLAN_SNAPSHOT_DIR, ds_key) if not decision_id else {}
|
||||
if not decision_id and not date:
|
||||
import user_candidates as uc
|
||||
state = dict(state, pending_codes=sorted(set(state.get('pending_codes') or []) | set(uc.live_pending(config.PLAN_SNAPSHOT_DIR, published))))
|
||||
key = (pinned or pv.digest(published), pv.digest(state), int(top), int(obs_top), int(theme_cap), bool(include_user_candidates))
|
||||
key = (pinned or pv.digest(published), pv.digest(state), int(top), int(obs_top), int(theme_cap))
|
||||
now = time.time()
|
||||
if not nocache and PLAN_CACHE_SEC > 0:
|
||||
with _PLAN_CACHE_LOCK:
|
||||
|
|
@ -223,8 +167,7 @@ def _plan_cached(date, top, obs_top, theme_cap, nocache: bool, decision_id=None,
|
|||
hit = _PLAN_CACHE.get(key)
|
||||
if hit and time.time() - hit[0] < PLAN_CACHE_SEC:
|
||||
return hit[1], True
|
||||
data = _plan_build(ds_key, top, obs_top, theme_cap, pinned,
|
||||
**({'include_user_candidates': True} if include_user_candidates else {}))
|
||||
data = _plan_build(ds_key, top, obs_top, theme_cap, pinned)
|
||||
data['refresh_status'] = state
|
||||
if PLAN_CACHE_SEC > 0:
|
||||
with _PLAN_CACHE_LOCK:
|
||||
|
|
@ -241,14 +184,14 @@ def _plan_cached(date, top, obs_top, theme_cap, nocache: bool, decision_id=None,
|
|||
@app.get("/plan")
|
||||
def get_plan(request: Request, date: str | None = None, format: str = "json",
|
||||
top: int = 20, obs_top: int = 10, theme_cap: int = 5, nocache: int = 0,
|
||||
decision_id: str | None = None, include_user_candidates: bool = False):
|
||||
decision_id: str | None = None):
|
||||
"""向下兼容承诺(2026-09-02 方案 2.7):main / observe 的装配、排序、裁剪与既有字段
|
||||
一字不动,每行只多联入判决类字段;顶层只新增 generated_at、plan_version、regime、
|
||||
card_counts、candidates、watch、segments_pointed、snapshot,2026-09-03 再加 market
|
||||
(环境段市场四项,与 regime 一样只从当日快照读,快照缺失为空字典)。PMS 按字段名取值、忽略未知键。"""
|
||||
t0 = time.time()
|
||||
try:
|
||||
data, cached = _plan_cached(date, top, obs_top, theme_cap, bool(nocache), decision_id, include_user_candidates)
|
||||
data, cached = _plan_cached(date, top, obs_top, theme_cap, bool(nocache), decision_id)
|
||||
except (RuntimeError, ValueError, OSError) as e:
|
||||
raise HTTPException(status_code=404, detail=str(e))
|
||||
data = dict(data) # 下面几个键按请求写,不污染缓存里那一份
|
||||
|
|
@ -273,12 +216,11 @@ def get_plan(request: Request, date: str | None = None, format: str = "json",
|
|||
|
||||
|
||||
@app.get('/plan/check')
|
||||
def check_plan_stock(code: str, include_user_candidates: bool = False):
|
||||
def check_plan_stock(code: str):
|
||||
"""采纳专用逐票读取。只读已发布计划,不装配、不查研究库。"""
|
||||
from common import to_prefix
|
||||
try:
|
||||
return JSONResponse(content=plan_versions.stock_check(config.PLAN_SNAPSHOT_DIR, to_prefix(code),
|
||||
include_user_candidates=include_user_candidates))
|
||||
return JSONResponse(content=plan_versions.stock_check(config.PLAN_SNAPSHOT_DIR, to_prefix(code)))
|
||||
except (OSError, ValueError, RuntimeError) as e:
|
||||
raise HTTPException(status_code=503, detail=str(e))
|
||||
|
||||
|
|
@ -290,15 +232,13 @@ def refresh(date: str | None = None, key: str | None = None):
|
|||
key=key, x_job_key=None)
|
||||
|
||||
|
||||
def _published_stock_rows(want, date, decision_id, include_user_candidates=False):
|
||||
def _published_stock_rows(want, date, decision_id):
|
||||
try:
|
||||
published = plan_versions.load(config.PLAN_SNAPSHOT_DIR, date, decision_id)
|
||||
except (OSError, ValueError):
|
||||
if decision_id or date:
|
||||
raise HTTPException(status_code=404, detail='指定计划版本不可用')
|
||||
return {}, {}
|
||||
import user_candidates as uc
|
||||
published = uc.consumer_view(published, include_user_candidates)
|
||||
rows = {r['code']: dict(r, decision=bucket, source='published',
|
||||
traceability=published.get('traceability') or 'legacy_unverified',
|
||||
decision_version=published.get('decision'), date=published['date'],
|
||||
|
|
|
|||
|
|
@ -4,7 +4,6 @@ from zoneinfo import ZoneInfo
|
|||
|
||||
import pandas as pd
|
||||
import db
|
||||
import market_scope
|
||||
|
||||
|
||||
def expected_data_day(now=None):
|
||||
|
|
@ -19,24 +18,15 @@ def expected_data_day(now=None):
|
|||
def check(now=None):
|
||||
"""数据未齐则在任何写入前失败;不能用缺失数据推断今天休市。"""
|
||||
expected = expected_data_day(now)
|
||||
# 日期索引先缩小到目标日及之后,再核验证券范围;不为找旧日期扫描全历史。
|
||||
params = (expected.isoformat(),)
|
||||
dates = {}
|
||||
failures = []
|
||||
for name, source, sql in (
|
||||
('传导', 'pg', 'SELECT MAX(scan_date) d FROM v_factor_transmission WHERE scan_date >= %s AND '
|
||||
+ market_scope.sql_filter('ts_code', postgres=True)),
|
||||
('收盘行情', 'price', None),
|
||||
('资金热度', 'heat', 'SELECT MAX(trade_date) d FROM stock_fund_heat_scores WHERE trade_date >= %s AND '
|
||||
+ market_scope.sql_filter('stock_code')),
|
||||
('传导', 'pg', 'SELECT MAX(scan_date) d FROM v_factor_transmission'),
|
||||
('收盘行情', 'price', 'SELECT MAX(`timestamp`) d FROM gp_day_data'),
|
||||
('资金热度', 'heat', 'SELECT MAX(trade_date) d FROM stock_fund_heat_scores'),
|
||||
):
|
||||
try:
|
||||
if source == 'price':
|
||||
import factors
|
||||
column = factors._price_code_col()
|
||||
sql = ('SELECT MAX(`timestamp`) d FROM gp_day_data WHERE `timestamp` >= %s AND '
|
||||
+ market_scope.sql_filter('`' + column + '`'))
|
||||
frame = db.read_pg(sql, params) if source == 'pg' else db.read_mysql(source, sql, params)
|
||||
frame = db.read_pg(sql) if source == 'pg' else db.read_mysql(source, sql)
|
||||
value = None if frame.empty else frame.iloc[0, 0]
|
||||
day = None if value is None or pd.isna(value) else pd.Timestamp(value).date()
|
||||
except Exception as exc:
|
||||
|
|
|
|||
34
card.py
34
card.py
|
|
@ -1,9 +1,5 @@
|
|||
"""候选卡:把一只票的各条证据线装配成判决与理由(纯函数,零 IO)。
|
||||
|
||||
2026-11-04新增参考模式:require_accum_confirm=False时,非明确、缺失和陈旧
|
||||
吸筹不再单独否决候选。资格、真实风险和公司质地分流保留,建仓交PMS三源合议。
|
||||
以下确认线描述为兼容旧模式,历史快照保持原判决。
|
||||
|
||||
## 为什么要有它
|
||||
|
||||
桥原来的产物是一个分数:主榜分 = 200 + 传导档位×20 + 组内分。分数排在最前的
|
||||
|
|
@ -99,7 +95,7 @@ def judge(ev: dict[str, Any], *, start_pct: float = 3.0,
|
|||
accum_max_age: int = 30, neg_tol: float = 0.0,
|
||||
logic_stale_days: int = 90, require_started: bool = True,
|
||||
company_review: dict | None = None, quality_gate: bool = True,
|
||||
stale_days: int = 120, require_accum_confirm: bool = True) -> dict[str, Any]:
|
||||
stale_days: int = 120) -> dict[str, Any]:
|
||||
"""一只票的证据字典 -> 判决字典。
|
||||
|
||||
ev 里认的键(缺键按缺失处理,不报错):
|
||||
|
|
@ -167,7 +163,7 @@ def judge(ev: dict[str, Any], *, start_pct: float = 3.0,
|
|||
elif confirm_stale:
|
||||
missing.append(f"吸筹评分是 {accum_age} 个交易日前的,太旧了"
|
||||
if isinstance(accum_age, int) else "吸筹评分有多旧算不出来")
|
||||
elif require_accum_confirm and confirm_negative and "派发" not in accum_state:
|
||||
elif confirm_negative and "派发" not in accum_state:
|
||||
state_short = accum_state.split("·")[0].strip()
|
||||
missing.append(f"吸筹只到「{state_short}」,没到明确吸筹这条线")
|
||||
if not gates["pointed"]:
|
||||
|
|
@ -192,33 +188,17 @@ def judge(ev: dict[str, Any], *, start_pct: float = 3.0,
|
|||
if confirm:
|
||||
reasons.append(f"明确吸筹(评分 {accum_score:.0f},{accum_age} 日前)"
|
||||
if accum_score is not None else f"明确吸筹({accum_age} 日前)")
|
||||
elif not require_accum_confirm and accum_state:
|
||||
detail = []
|
||||
if accum_score is not None:
|
||||
detail.append(f"评分 {accum_score:.0f}")
|
||||
if isinstance(accum_age, int):
|
||||
detail.append(f"{accum_age} 个交易日前")
|
||||
reasons.append("吸筹状态:" + accum_state + ("(" + ",".join(detail) + ")" if detail else ""))
|
||||
if gates["covered"]:
|
||||
reasons.append(f"券商覆盖,预期空间 {upside:+.0%}")
|
||||
|
||||
# 人工来源替代发现条件;pointed 仍保留真实传导状态,不改变排序和其他门槛。
|
||||
user_selected = bool(ev.get('user_selected'))
|
||||
if user_selected and not gates['pointed']:
|
||||
missing = [x for x in missing if x != '所在环节当日没有被传导指向']
|
||||
reasons.insert(0, '用户添加,按现有条件评估')
|
||||
effective = dict(gates, pointed=gates['pointed'] or user_selected)
|
||||
all_gates = all(effective.values())
|
||||
failed = [k for k, ok in effective.items() if not ok]
|
||||
all_gates = all(gates.values())
|
||||
failed = [k for k, ok in gates.items() if not ok]
|
||||
# 判决依据是这张卡上唯一一句"为什么",PMS 原样显示在页面上给人读。所以它必须是
|
||||
# 一句完整的中文,带上具体数字,不许出现门槛的变量名(started 这类)或信号的
|
||||
# 枚举值(SELL 这类)——那些是系统之间传递用的,不是给人看的。
|
||||
if risk:
|
||||
verdict = VERDICT_SHOW
|
||||
basis = ";".join(risk)
|
||||
elif all_gates and not require_accum_confirm:
|
||||
verdict = VERDICT_CANDIDATE
|
||||
basis = "选股资格通过,进入基本面、技术面和择时合议"
|
||||
elif all_gates and confirm:
|
||||
verdict = VERDICT_CANDIDATE
|
||||
basis = (f"三条门槛都过了,没有风险,且明确吸筹({accum_age} 个交易日前评的)"
|
||||
|
|
@ -250,13 +230,11 @@ def judge(ev: dict[str, Any], *, start_pct: float = 3.0,
|
|||
if quality_gate and cr and qband == 0 and verdict == VERDICT_CANDIDATE:
|
||||
verdict = VERDICT_WATCH
|
||||
why_q = ";".join((cr.get("quality_reasons") or [])[:2]) or "基座代码合成的质地档为差"
|
||||
company_gate = (f"公司质地评为差({why_q}),候选降为关注交人裁决" if require_accum_confirm else
|
||||
f"公司质地评为差({why_q}),按PMS现行合议和自决档处理关注候选")
|
||||
company_gate = f"公司质地评为差({why_q}),候选降为关注交人裁决"
|
||||
basis = f"{basis};{company_gate}"
|
||||
|
||||
return {"verdict": verdict, "reasons": reasons, "missing": missing, "risk": risk,
|
||||
"gates": gates, "confirm": confirm, "confirm_required": bool(require_accum_confirm),
|
||||
"failed_gates": failed, "basis": basis,
|
||||
"gates": gates, "confirm": confirm, "failed_gates": failed, "basis": basis,
|
||||
"company_gate": company_gate, "quality_band": qband,
|
||||
"logic": logic_lines(ev.get("logic")),
|
||||
"logic_quality": logic_quality(ev.get("logic"))}
|
||||
|
|
|
|||
16
common.py
16
common.py
|
|
@ -7,7 +7,6 @@ import pandas as pd
|
|||
|
||||
import config
|
||||
import db
|
||||
import market_scope as scope
|
||||
|
||||
|
||||
def to_prefix(ts_code: str) -> str:
|
||||
|
|
@ -25,9 +24,8 @@ def load_universe() -> set[str]:
|
|||
⚠️ 这是**当前态**:industry_pools 只存最新快照、每周一 refresh_pools 自动长大。
|
||||
用它过滤历史数据会引入成员性前视——见 universe_filter 与 config.SUBFACTOR_UNIVERSE。
|
||||
"""
|
||||
df = db.read_pg("SELECT ts_code FROM v_factor_universe WHERE " + scope.sql_filter("ts_code", postgres=True))
|
||||
df = scope.frame(df, "ts_code")
|
||||
return set(df["ts_code"])
|
||||
df = db.read_pg("SELECT ts_code FROM v_factor_universe")
|
||||
return set(df["ts_code"].astype(str).str.strip())
|
||||
|
||||
|
||||
def universe_filter(df: pd.DataFrame, col: str = "stock_code",
|
||||
|
|
@ -43,9 +41,6 @@ def universe_filter(df: pd.DataFrame, col: str = "stock_code",
|
|||
同时截面统计量会在每周一发生结构性跳变,IC 序列带周期性伪影。
|
||||
子因子表是「可独立观察的仪表」,过滤该发生在消费端。
|
||||
"""
|
||||
df = scope.frame(df, col, prefix=as_prefix)
|
||||
if df is None or df.empty:
|
||||
return df
|
||||
if config.SUBFACTOR_UNIVERSE == "pool":
|
||||
uni = load_universe()
|
||||
if as_prefix:
|
||||
|
|
@ -62,12 +57,10 @@ def trading_days(start: str, end: str) -> list:
|
|||
拿到空日历 → 静默返回空 → 只打印「无数据(跳过)」,不报错。
|
||||
改用行情表 gp_day_data(实测 5584 天,覆盖全历史)。
|
||||
"""
|
||||
import factors
|
||||
column = factors._price_code_col()
|
||||
df = db.read_mysql(
|
||||
"price",
|
||||
"SELECT DISTINCT `timestamp` AS trade_date FROM gp_day_data "
|
||||
"WHERE `timestamp` BETWEEN %s AND %s AND " + scope.sql_filter(f"`{column}`") + " ORDER BY `timestamp`",
|
||||
"WHERE `timestamp` BETWEEN %s AND %s ORDER BY `timestamp`",
|
||||
(start, end))
|
||||
if df.empty:
|
||||
print(f" ⚠️ 交易日历为空(gp_day_data 在 {start}~{end} 无数据)"
|
||||
|
|
@ -105,7 +98,6 @@ def write_factor(table: str, df: pd.DataFrame, mode: str = "daily") -> None:
|
|||
if df is None or df.empty:
|
||||
print(f" {table}: 无数据(跳过)")
|
||||
return
|
||||
df = scope.frame(df, "stock_code", prefix=True)
|
||||
df = df.dropna(subset=["trade_date", "stock_code", "factor_value"]).copy()
|
||||
df["stock_code"] = df["stock_code"].map(to_prefix)
|
||||
df["trade_date"] = pd.to_datetime(df["trade_date"]).dt.date
|
||||
|
|
@ -120,7 +112,7 @@ def write_factor(table: str, df: pd.DataFrame, mode: str = "daily") -> None:
|
|||
chunk = max(1, config.WRITE_CHUNK_ROWS)
|
||||
with db.factor_conn() as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(f"DELETE FROM {table} WHERE trade_date BETWEEN %s AND %s AND " + scope.sql_filter("stock_code"),
|
||||
cur.execute(f"DELETE FROM {table} WHERE trade_date BETWEEN %s AND %s",
|
||||
(dmin, dmax))
|
||||
conn.commit()
|
||||
for i in range(0, len(rows), chunk):
|
||||
|
|
|
|||
11
config.py
11
config.py
|
|
@ -183,11 +183,6 @@ XXL_TRIGGER_KEY = os.environ.get("XXL_TRIGGER_KEY", "")
|
|||
XXL_CALLBACK_READ_TIMEOUT = float(os.environ.get("XXL_CALLBACK_READ_TIMEOUT", "10"))
|
||||
XXL_CALLBACK_MAX_RETRIES = int(os.environ.get("XXL_CALLBACK_MAX_RETRIES", "3"))
|
||||
|
||||
# 自然修订缺今日基础版时的有界恢复;只构建因子/计划,不入池或额外投递模型。
|
||||
# 安全上限为每日三次实际构建、相邻至少十五分钟;可停用或收紧,不能放宽。
|
||||
PLAN_RECOVERY_MAX_ATTEMPTS = int(os.environ.get("PLAN_RECOVERY_MAX_ATTEMPTS", "3"))
|
||||
PLAN_RECOVERY_COOLDOWN_SECONDS = int(os.environ.get("PLAN_RECOVERY_COOLDOWN_SECONDS", "900"))
|
||||
|
||||
# --- 选股计划入池(08-03 定稿;规则与流程见 docs/选股计划入池_对接说明.md)------
|
||||
# 写 Mongo stock_groups 的独立分组,决策系统每晚扫描按分组并集覆盖 → 候选票自动
|
||||
# 获得夜间推理。入池范围与 PMS 候选**次序口径**一致:先筛强传导、再按绝对得分截断
|
||||
|
|
@ -229,9 +224,6 @@ CARD_START_PCT = float(os.environ.get("CARD_START_PCT", "3")) # 门槛二
|
|||
# 想开回去就把这个值设回 1 并重建容器,判决当天即恢复原样。
|
||||
CARD_REQUIRE_STARTED = os.environ.get("CARD_REQUIRE_STARTED", "1") == "1"
|
||||
CARD_ACCUM_MAX_AGE = int(os.environ.get("CARD_ACCUM_MAX_AGE", "30")) # 确认线:吸筹评分日龄上限(交易日)
|
||||
# 2026-11-04:155显式关闭旧吸筹准入确认,吸筹转为三源合议的参考。
|
||||
# 默认保留旧模式以兼容其他调用方;切换必须发布新计划版本,不能改历史快照。
|
||||
CARD_ACCUM_CONFIRM_REQUIRED = os.environ.get("CARD_ACCUM_CONFIRM_REQUIRED", "1").lower() not in ("0", "false", "no")
|
||||
# 计划快照落点(容器内路径;仓库根挂在 /app,故默认落在仓库 data/plan/)。
|
||||
# 每日 JSON 含主榜与观察档全部行与全部证据线,是复盘与对账的唯一底本。
|
||||
PLAN_SNAPSHOT_DIR = os.environ.get("PLAN_SNAPSHOT_DIR", "data/plan")
|
||||
|
|
@ -326,9 +318,6 @@ LOGIC_PATH_COMPANY = os.environ.get("LOGIC_PATH_COMPANY", "1") == "1"
|
|||
CARD_COMPANY_QUALITY_GATE = os.environ.get("CARD_COMPANY_QUALITY_GATE", "1") == "1" # 候选卡"质地差降一档"
|
||||
COMPANY_REVIEW_STALE_DAYS = int(os.environ.get("COMPANY_REVIEW_STALE_DAYS", "120")) # 报告运行日超过它算陈旧
|
||||
AKG_API_BASE = os.environ.get("AKG_API_BASE", "http://192.168.16.178:8000") # 报告链接前缀
|
||||
USER_RESEARCH_FRESH_DAYS = int(os.environ.get("USER_RESEARCH_FRESH_DAYS", "7")) # 基座核对时刻口径,不把复核当新报告
|
||||
USER_RESEARCH_REQUEST_TIMEOUT = float(os.environ.get("USER_RESEARCH_REQUEST_TIMEOUT", "3"))
|
||||
USER_RESEARCH_SUBMIT_LIMIT = int(os.environ.get("USER_RESEARCH_SUBMIT_LIMIT", "3")) # 每轮有界受理,后续自然调度继续
|
||||
LOGIC_SETTLE_CONFIRM_DAYS = int(os.environ.get("LOGIC_SETTLE_CONFIRM_DAYS", "2"))
|
||||
LOGIC_SETTLE_EXIT_DAYS = int(os.environ.get("LOGIC_SETTLE_EXIT_DAYS", "3"))
|
||||
# 读上一次落定态与近几日原始态时往前看几个自然日。要盖住国庆这种连休八天再加两个周末的空档;
|
||||
|
|
|
|||
|
|
@ -1,171 +0,0 @@
|
|||
{
|
||||
"verified_at": "2026-11-05T10:19:46.116833+08:00",
|
||||
"host": "155 simulation",
|
||||
"product_commits": [
|
||||
"475bfec480b4f27cca43337981f60adde88d9630",
|
||||
"9f8224e89bec8887ac11a963161fc3398a20e3cc"
|
||||
],
|
||||
"deployment": {
|
||||
"verified_at": "2026-11-05T02:18:44.142605+00:00",
|
||||
"target": {
|
||||
"akg_factor_bridge": {
|
||||
"id": "f6cfc7a6c9acf7421193cc0eab93ca6618096d987e42ff5e0134ee02cc70f7bd",
|
||||
"image": "sha256:c3c87ffa8ee7087f41d9cd0192ff9c86bdc37fb814da717115169c0d62f76496",
|
||||
"tag": "bridge:plan-recovery-9f8224e",
|
||||
"started": "2026-11-05T02:18:22.104228056Z"
|
||||
}
|
||||
},
|
||||
"other_unchanged": 17,
|
||||
"other_changed": [],
|
||||
"environment_unchanged": true,
|
||||
"old_history_unchanged": true,
|
||||
"original_source_unchanged": true
|
||||
},
|
||||
"recovery": {
|
||||
"attempts": 1,
|
||||
"completed_at": "2026-11-05T02:08:24.634400+00:00",
|
||||
"data_date": "2026-11-04",
|
||||
"decision_date": "2026-11-05",
|
||||
"error": null,
|
||||
"last_attempt_at": "2026-11-05T02:05:36.721028+00:00",
|
||||
"schema_version": 1,
|
||||
"status": "recovered"
|
||||
},
|
||||
"first_decision": {
|
||||
"ancestor_ids": [],
|
||||
"cutoff_at": "2026-11-05T02:07:00.338807+00:00",
|
||||
"id": "2026-11-04_bc51f887aea763cfd804a2c96be700dc3d7b3e4cc8be0e0fbe9c413668b832de",
|
||||
"input_hash": "b688a65e553cdb93341058cd4c3bfe6778d3c91ee82e6c0cd9fdf9f0b0d6c3a9",
|
||||
"params_hash": "0f97153c6267739c7d79524f343edbf4231db2cafec83bb2434466687f9b881b",
|
||||
"parent_id": null,
|
||||
"publication_kind": "recovery",
|
||||
"reason": "数据恢复后补发"
|
||||
},
|
||||
"first_morning_eligible": false,
|
||||
"freeze": {
|
||||
"files": 13,
|
||||
"all_counts_match": true,
|
||||
"errors": [],
|
||||
"warnings": 0,
|
||||
"scope_error_sum": 0
|
||||
},
|
||||
"runtime": {
|
||||
"PLAN_RANK_AXIS": "quality",
|
||||
"CARD_SORT_BY_QUALITY": true,
|
||||
"CARD_ACCUM_CONFIRM_REQUIRED": false,
|
||||
"PLAN_RECOVERY_MAX_ATTEMPTS": 3,
|
||||
"PLAN_RECOVERY_COOLDOWN_SECONDS": 900
|
||||
},
|
||||
"readiness_seconds": 0.441,
|
||||
"pms": {
|
||||
"read_only": true,
|
||||
"new_process": true,
|
||||
"checked_at": "2026-11-05T10:19:46.116833+08:00",
|
||||
"plan": {
|
||||
"date": "2026-11-04",
|
||||
"age_tdays": 1,
|
||||
"heat_date": "2026-11-04",
|
||||
"market_snapshot_days": [
|
||||
"2026-11-04"
|
||||
],
|
||||
"rank_axis": "quality",
|
||||
"returned": {
|
||||
"main": 300,
|
||||
"observe": 100
|
||||
},
|
||||
"decision": {
|
||||
"ancestor_ids": [
|
||||
"2026-11-04_bc51f887aea763cfd804a2c96be700dc3d7b3e4cc8be0e0fbe9c413668b832de"
|
||||
],
|
||||
"cutoff_at": "2026-11-05T02:14:31.303061+00:00",
|
||||
"id": "2026-11-04_b372b5615c49ec496bece5c27f355b8b30cabea24b38212c3a884458bcf5d88a",
|
||||
"input_hash": "c64eb8819705415fe0e5cc8303f6d6abd10ba5ed9c87d0abb690bb30cc28bb4a",
|
||||
"params_hash": "0f97153c6267739c7d79524f343edbf4231db2cafec83bb2434466687f9b881b",
|
||||
"parent_id": "2026-11-04_bc51f887aea763cfd804a2c96be700dc3d7b3e4cc8be0e0fbe9c413668b832de",
|
||||
"publication_kind": "revision",
|
||||
"reason": "评析更新"
|
||||
},
|
||||
"refresh_status": {
|
||||
"checked_at": "2026-11-05T02:15:14+00:00",
|
||||
"decision_id": "2026-11-04_b372b5615c49ec496bece5c27f355b8b30cabea24b38212c3a884458bcf5d88a",
|
||||
"error": null,
|
||||
"pending_codes": []
|
||||
}
|
||||
},
|
||||
"freshness": {
|
||||
"accepted": true,
|
||||
"limit_tdays": 1,
|
||||
"fetch_and_parse_seconds": 1.129
|
||||
},
|
||||
"candidates": {
|
||||
"considered": 300,
|
||||
"eligible": 86,
|
||||
"dropped": {
|
||||
"held": 1,
|
||||
"black": 0,
|
||||
"tier": 0,
|
||||
"score": 0,
|
||||
"sources": 0,
|
||||
"upside": 0,
|
||||
"st": 0,
|
||||
"theme": 0,
|
||||
"dup": 0,
|
||||
"capped": 36,
|
||||
"display": 213
|
||||
},
|
||||
"rank_mode": "upstream",
|
||||
"route_by_verdict": true,
|
||||
"selected": 50,
|
||||
"held_excluded_context": 1,
|
||||
"black_excluded_context": 0,
|
||||
"verdict_counts": {
|
||||
"候选": 38,
|
||||
"关注": 12
|
||||
},
|
||||
"outside_scope": 0
|
||||
},
|
||||
"manual": {
|
||||
"active": 6,
|
||||
"review_status": {
|
||||
"completed": 6
|
||||
},
|
||||
"report_dates": {
|
||||
"2026-11-04": 6
|
||||
},
|
||||
"coverage_notes": {
|
||||
"ok": 6
|
||||
},
|
||||
"stages": {
|
||||
"daily": {
|
||||
"COMPLETE": 6
|
||||
},
|
||||
"intraday": {
|
||||
"UNAVAILABLE": 3,
|
||||
"COMPLETE": 3
|
||||
}
|
||||
},
|
||||
"stage_dates": {
|
||||
"daily": {
|
||||
"2026-11-05 / 2026-11-04": 6
|
||||
},
|
||||
"intraday": {
|
||||
"2026-11-05 / 2026-11-05": 4,
|
||||
"2026-11-05 / None": 2
|
||||
}
|
||||
},
|
||||
"unavailable_reasons": {
|
||||
"模型没有返回合法研究结论": 1,
|
||||
"TimeoutError: ": 2
|
||||
},
|
||||
"review_errors": 0,
|
||||
"last_analysis_errors": 0
|
||||
},
|
||||
"table_prefix": ""
|
||||
},
|
||||
"natural_acceptance": {
|
||||
"recovery_task": "f5a16fb3",
|
||||
"manual_request_unchanged": "5b181fe6",
|
||||
"regime_append_completed": "3d52f1c0"
|
||||
},
|
||||
"offline_suites": 38
|
||||
}
|
||||
|
|
@ -1,103 +0,0 @@
|
|||
# 个股评析全池覆盖开发规格
|
||||
|
||||
日期为2026年10月28日。本文时间均为北京时间。
|
||||
|
||||
## 目标与授权
|
||||
|
||||
用户要求本轮覆盖整个产业池和高股息池。
|
||||
当前候选优先,未完成对象不因每日名单变化而丢失。
|
||||
本轮补普通个股评析,不自动扩大分红专项研究。
|
||||
股票资格、排序、模型判据和交易参数保持。
|
||||
|
||||
## 改前实测
|
||||
|
||||
178基座提交7ef107e,评析队列为零。
|
||||
当前目标148家,全部已有报告、定性及评议。
|
||||
产业池3833家,仅747家有报告。
|
||||
桥高股息池169家,仅9家有报告。
|
||||
两个池有交集,缺口不能直接相加。
|
||||
本轮只读预览并集3885家,已有747家,首次报告缺口3138家。
|
||||
高股息缺报告的160家均未进入留存的评析目标名单。
|
||||
|
||||
近72小时43次成功运行平均185.5秒。
|
||||
第95百分位246.4秒,最大286.5秒。
|
||||
近七天评议平均输出13465个token。
|
||||
原每日上限为评议150家、定性200家、评议输出300万token。
|
||||
|
||||
## 具体工作
|
||||
|
||||
1. 复用原每小时批任务和review队列,不新增任务或表。
|
||||
2. 自动读取全部产业池成员。
|
||||
3. 从桥已发布的高股息研究接口读取研究对象。
|
||||
不按投资资格eligible过滤,基座不重新计算股息筛选。
|
||||
4. 在现有job_runs保存累计研究名单,名单变化才追加快照。
|
||||
已纳入对象继续保留。来源读取失败保留旧名单并报告。
|
||||
旧快照读失败时停止扩展投递,不能用空名单覆盖。
|
||||
5. 新报告优先于名单外旧报告升级。
|
||||
当前候选保留优先名额,剩余名额持续补名单外对象。
|
||||
旧报告的复核不能无限挤占首次报告。
|
||||
6. 每批完整运行最多30家。每日评议、定性各600家。
|
||||
评议输出上限900万token,单次模型预算保持。
|
||||
每日额度不足时不继续投递完整运行。
|
||||
7. 用Redis短期凭据防止重复投递。
|
||||
失败对象延后重试,队列积压时暂停增加整批任务。
|
||||
重试不删除旧报告,不清空历史记录。
|
||||
8. 名单外补齐避开06:00至08:00、20:00至23:00。
|
||||
原候选任务仍按原调度执行。
|
||||
晚间行业催化与分红专项使用原队列和时刻。
|
||||
9. 日检分别显示有报告、定性、评议及首次报告缺口。
|
||||
无财务数据、材料薄和失败不能宣称完整研究已完成。
|
||||
|
||||
## 偏差发现与撤销
|
||||
|
||||
新增配置文件控制覆盖开关和每日额度。
|
||||
关闭覆盖开关即回到原目标集合与每日额度。
|
||||
旧报告、固定引用和累计名单保留。
|
||||
日检报告来源失败、剩余缺口和最近调度状态。
|
||||
同一对象反复失败时先暂停其重试,不阻塞后面的对象。
|
||||
|
||||
按每日有效完成约450至550家估算,首次覆盖约需一周。
|
||||
这只是按现有耗时计算的容量预估。
|
||||
上游缺财务数据及缺年报的对象单列,不能承诺完整结论。
|
||||
|
||||
## 分批验收预期
|
||||
|
||||
### 本地
|
||||
|
||||
- 合并去重和排序确定,重复读取不改变名单。
|
||||
- 高股息资格尚未通过的对象仍能进入研究。
|
||||
- 来源暂不可用时累计待办不丢,旧快照损坏时不覆盖。
|
||||
- 当前候选和首次报告均能获得名额。
|
||||
- 预算不足、已有在途及失败冷却时不重复投递。
|
||||
- 保护时段只运行原候选。
|
||||
- 旧指纹、有效标志、固定报告引用和财务期口径不变。
|
||||
- 全部离线unittest通过。
|
||||
|
||||
### 部署前只读预览
|
||||
|
||||
在178容器内核对两个池并集、首次报告缺口和首批代码。
|
||||
预览不保存名单,不登记运行,不投递任务,不调用模型。
|
||||
核对桥消费的company_review_runs及报告路径保持。
|
||||
|
||||
### 服务生效与生产判收
|
||||
|
||||
容器重建须取得用户同意。
|
||||
按仓库规则统一重建四类worker。
|
||||
beat调度不改,不需要重建。
|
||||
首批显式投递review队列,核对报告增量与预算读数。
|
||||
首批成功不等于全池覆盖完成。
|
||||
后续按实际缺口下降和失败清单判收。
|
||||
|
||||
## 开发者参考
|
||||
|
||||
原targets.collect仍供材料需求单和旧入口使用。
|
||||
扩大覆盖只在评析批任务启用,避免一次扩大数千张采集单。
|
||||
累计名单使用job_runs的既有tail JSON字段。
|
||||
临时投递凭据使用Redis,带过期和所有者校验。
|
||||
同一凭据只能从待执行原子转为执行中一次。
|
||||
失败冷却六小时;无数据沿用七天复查,三次评议失败沿用原放弃规则。
|
||||
预算沿用索引表的当日登记口径,含失败状态。
|
||||
每次模型调用仍受原单次输出上限约束。
|
||||
这不是服务商账单的逐次计费审计,不能把登记量称为实际费用。
|
||||
排序只决定研究先后,不形成证券评分。
|
||||
报告内容及下游排序由现有逻辑继续处理。
|
||||
|
|
@ -1,129 +0,0 @@
|
|||
# 个股评析全池覆盖交接
|
||||
|
||||
本轮日期为2026年10月28日,时间按北京时间。
|
||||
规格见《个股评析全池覆盖开发规格_2026-10-28.md》。
|
||||
|
||||
## 用户决定
|
||||
|
||||
用户要求本轮直接覆盖整个产业池和高股息池。
|
||||
本轮扩大普通个股评析,不扩大分红专项研究,也不改交易参数。
|
||||
|
||||
## 已实现
|
||||
|
||||
复用原每小时批任务、review队列及两个工作并发。
|
||||
每批最多30家,当前候选保留10个优先名额。
|
||||
没有候选待办时,30个名额均可补首次报告。
|
||||
高股息研究对象取桥已发布的现有接口。
|
||||
基座不重新计算股息筛选,也不按投资资格过滤研究。
|
||||
|
||||
累计范围沿用job_runs的tail字段,仅名单变化时追加快照。
|
||||
来源失败保留旧名单;旧名单损坏时停止扩展。
|
||||
在途凭据有所有者校验和六小时过期。
|
||||
失败冷却六小时,任务不反复抢同一批名额。
|
||||
每日评议和定性上限各600家,评议输出上限900万token。
|
||||
开关和额度由config/company_review_coverage.json热读。
|
||||
06:00至08:00、20:00至23:00仅处理原候选。
|
||||
其余时段允许补齐全池,原beat时刻保持。
|
||||
|
||||
材料需求单仍使用原targets.collect。
|
||||
本轮没有一次扩开数千张采集需求单。
|
||||
报告判据、财务期、幂等指纹和固定引用保持。
|
||||
批任务返回保留原有字段,新增覆盖和来源读数。
|
||||
|
||||
## 部署前实测证据
|
||||
|
||||
178原代码为7ef107e,旧服务正在运行。
|
||||
当前候选148家,全部已有评析、定性与评议。
|
||||
产业池3833家,高股息研究池169家。
|
||||
新范围去重后3885家,已有747家,缺3138家。
|
||||
首批只读预览为30家首次报告与60家原候选轻量复核。
|
||||
两路范围来源读取成功,review队列为零。
|
||||
预览未保存累计名单,未投任务,未调用模型。
|
||||
|
||||
本地全部离线unittest为264项,通过。
|
||||
新增26项覆盖轮转、预算、来源失败、在途及任务收尾测试。
|
||||
已有指纹、报告与财务历史边界测试保持通过。
|
||||
桥sources.company_reviews仍直接读原索引字段。
|
||||
报告路径与输出格式未改变。
|
||||
|
||||
## 部署前边界(已由下文上线记录更新)
|
||||
|
||||
代码尚未在178拉取,工作容器尚未重建。
|
||||
新范围尚未投递,不能把只读预览称为覆盖完成。
|
||||
服务器现有未提交文件保留。
|
||||
其他仓库只同步本文和规格副本,并追加桥决定台账。
|
||||
桥现有未提交台账与其他工作保持。
|
||||
|
||||
## 上线及撤销
|
||||
|
||||
用户同意后,在review队列与在途任务为空时统一重建四类worker。
|
||||
保留原worker副本数,不调整并发,不重建beat或前端。
|
||||
核验热读配置与只读预览后,显式向review队列投递首批。
|
||||
首批成功后,按自然小时批次继续推进。
|
||||
首次覆盖容量预估约一周,不保证缺材料对象得到完整结论。
|
||||
|
||||
撤销时在本地将配置enabled改为false,再提交并由178拉取。
|
||||
下一批恢复原名单和每日额度。
|
||||
已在途任务可完成,旧报告与累计名单保留。
|
||||
|
||||
## 可运行验证
|
||||
|
||||
```bash
|
||||
# 本地 · /Users/baobao/Documents/work/project/astock-kg
|
||||
PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=backend python3 -m unittest discover -s backend/tests -t backend
|
||||
git diff --check
|
||||
```
|
||||
|
||||
预期264项通过,差异检查无输出。
|
||||
|
||||
```bash
|
||||
# 基座机 · ~/project/astock-kg,部署后
|
||||
docker compose exec -T -e PYTHONPATH=/app backend python /app/scripts/company_review_coverage.py
|
||||
```
|
||||
|
||||
预期dry_run为true,dispatched为空。
|
||||
初始targets为3885、missing_reports为3138,后续随真实产出改变。
|
||||
source_errors应为空;无数据与材料薄须分开看。
|
||||
新增的日检行同时显示最近批次时刻及状态。
|
||||
|
||||
## 2026-10-28生产上线记录
|
||||
|
||||
本节时间均为北京时间。
|
||||
用户明确同意部署并启动补齐。
|
||||
15:35再次核查四条队列均为零,四类worker在途任务均为空。
|
||||
178从7ef107e快进到42191df,服务器已有未提交文件保持一致。
|
||||
15:35:52统一重建四类worker,镜像、各服务副本数及并发保持。
|
||||
15:36四个新节点均返回pong。
|
||||
beat、前端及数据库容器未重建。
|
||||
|
||||
部署后只读预览与部署前一致。
|
||||
范围3885家,有报告747家,首次缺口3138家。
|
||||
每日额度为评议600家、定性600家、评议输出900万token。
|
||||
名单来源无异常,预计首批为30家首次报告及60家轻量复核。
|
||||
|
||||
15:36:32显式向review队列投递批次。
|
||||
任务号为e20798d0-6186-4a96-9093-f35a133cf70f。
|
||||
15:36:33批任务成功,实际投递90项。
|
||||
其中30项带定性与评议,60项仅复核原候选。
|
||||
累计名单已保存,初始90项均持有在途凭据。
|
||||
首批投递成功不等于90项执行完成。
|
||||
|
||||
15:37部署后完整日检通过,19项正常、零警告、零错误。
|
||||
新增覆盖行和最近批次行均成功输出。
|
||||
15:38接口健康检查返回HTTP 200。
|
||||
|
||||
15:40:46首份新增报告000543.SZ成功落库。
|
||||
报告、评议、定性状态均为ok,在途凭据自动删除。
|
||||
最新报告和固定引用接口均返回HTTP 200,各51418字节。
|
||||
固定引用为3aae8fc1e87736811af30027715c12f64b9a656075608dbfff981338a97b07d1。
|
||||
15:41:43复查累计名单3885家,有报告748家,剩余3137家。
|
||||
在途89项、重试冷却零项,两路名单来源仍无异常。
|
||||
当日登记评议8家、定性8家、评议输出104020个token。
|
||||
|
||||
本轮已判收部署、批次投递、首份报告和凭据收尾。
|
||||
首批全部完成、下一次自然小时批次及全池覆盖尚未判收。
|
||||
后续由原定时任务继续推进,无需人工逐批投递。
|
||||
缺财务数据、材料薄及失败须分列,不能充作完整结论。
|
||||
|
||||
部署日志在178的~/akg_logs/review-coverage-deploy-20261028-1535/。
|
||||
目录保留前后服务状态、原镜像与命令、拉取记录和首批预览。
|
||||
|
|
@ -882,13 +882,3 @@
|
|||
研究池从该已发布版本取 rank 前 50 个被指向对象,保留实际 3 只持仓,上限仍为 100。补入 12 只,按原规则回收 1 只及池满移出 11 只。全部 16 只 PMS 候选与持仓在池内,候选关联决策版本和研究名次一致。增量补扫任务 5ce6930c-dd01-45ef-a68b-f830419d4858 已受理,未把受理写成研究完成。POOL_TIERS 的旧配置在 rank 来源不参与过滤,日志打印它不代表生效。
|
||||
|
||||
PMS 155 镜像 50 批、1165 例通过;桥 54 组契约及池、构建就绪、XXL 触发三个脚本通过。页面、桥健康及 QMT 账实核对正常,部署判收为真机见过。切换窗口一次高股息读取超时于 15:01:39 复查恢复。私有备份和验收在 155 的 /home/factor/pms_logs/quality-switch-20261026;旧镜像与分支保留,回退须追加版本而非覆盖历史。详见 PMS docs/开发接续_2026-10-12.md 第十节。未制造交易,实际自然建仓及收益表现另行观察。
|
||||
|
||||
## 084 · 2026-11-04 · 人工关注补齐真实评析,吸筹改为参考
|
||||
|
||||
- **日期与授权。** 2026-11-04,北京时间。用户明确选择“沿用当前质地排序,吸筹作为参考,建仓看三源合议”。本次仅在 155 模拟链路切换 `CARD_ACCUM_CONFIRM_REQUIRED=0`,不扩展到 188 真实 PMS。代码默认保持旧确认规则,回退仍有明确开关。
|
||||
- **改动。** 桥不再仅因吸筹未达到明确状态、缺评分或评分陈旧而阻断候选评估;原吸筹评分、状态及新鲜度继续作为证据。质地排序、原始分数编码和发现资格保持,不新造加权评分。证券范围、覆盖与估值条件、既有真实风险、高位派发及快照时点校验继续生效。PMS 的基本面、技术面、择时三源合议及后续规则、研判、资金、组合与执行约束继续决定建仓;本次不是关闭全部判决过滤。
|
||||
- **依据。** 代码和历史记录核对表明,桥从 9 月起将明确吸筹作为候选确认线;10 月质地排序切换保留了该判决。PMS 三源合议在上游“仅展示”过滤之后运行,因此无法评估已被旧确认线排除的对象。此次依据用户的新明确决定改变吸筹角色,不将其写成已经发生过的部署回退或历史修复。
|
||||
- **人工关注评析。** 登记后的后台任务与自然计划修订周期同步独立评析状态;查询入口不派模型任务。全部有效沪深人工关注进入原研究目标名单,不受原分数、名次和交易判决限制。后台复用基座真实评析入口,开启买方评议及定性材料,默认不强制重跑。提交前持久化受理凭据,收到真实入队回执才标记排队;超时或结果不明保持待核实,不立即重复投递。原登记的“已收录”与“评析完成”独立展示。
|
||||
- **完成与时间口径。** 完成要求真实固定报告引用、成功的买方评议、论点及有效生成时间。七天复核窗口使用真实运行或核对时刻,避免指纹命中后只有核对时间更新而永远无法完成;报告日期仍取原生成时间的北京时间日期。缺引用、材料不全、评议失败或来源不可用均不伪造完成。财报期与原评析重试规则继续由基座负责。
|
||||
- **版本与预期。** 规则切换即使输入不变也追加新的完整修订,记录真实截止时刻、参数指纹和变更原因。旧晨版、历史计划及固定版本查询不变。人工关注的研究覆盖与交易资格分开,移除或到期后停止新增关注研究;评析完成不等于可以自动建仓。预期自然回执能分别说明登记、受理、报告完成、日研究和盘中观察状态。
|
||||
- **验证与未完成事项。** 开发机新增评析专项 19 项通过;包含本次规则切换的 35 个测试文件均完成配置隔离、禁网验证,其中高股息旧测试的两处相对路径已修复并定向复测。此条记录开发交付和授权边界,不代表运行参数已切换。候选镜像断网回归、155 部署、真实评析受理与完成、自然日研究及盘中观察仍须在发布阶段逐项判收,不注入交易制造结果。
|
||||
|
|
|
|||
|
|
@ -1,53 +0,0 @@
|
|||
# 选股输入迟到恢复与沪深边界
|
||||
|
||||
2026年11月5日,北京时间。用户报告PMS选股计划过期,随后确认外部热度缺失已恢复,要求完成剩余恢复与重算。
|
||||
|
||||
## 原因与现场
|
||||
|
||||
09:36直接从155新版PMS读取上游:请求没有固定日期或版本参数,返回数据日、热度日和行情日均11-03,最后修订截止11-04 15:05。11-03至11-05是两个交易日,PMS上限1的拒绝正确,不是缓存或人工关注入口问题。
|
||||
|
||||
上游02:50构建因传导和热度旧日失败,07:10时传导已齐但热度仍11-03,写前检查阻止全部因子构建,故评分也停在旧日。后续每五分钟修订只接受当天已有基础版,持续no_morning_today,数据晚到后没有自动补建入口。08:45区制追加也缺当天基础版,但旧CLI把未发布仍报告成成功。
|
||||
|
||||
用户确认外部数据恢复后,09:44:25只读核对传导、沪深日线及热度均已齐11-04;热度批次130,沪深5208行。外部热度生产代码未改;原始批次写入时间的连接时区未证实,不把它解释为盘后停机证据。
|
||||
|
||||
准备重算发现,旧代码只在计划最终消费侧过滤沪深,因子构建和冻结仍可能读取、计算和归档北交所。按项目既定范围,先补齐输入边界再补跑,不为北交所另查异常、算分或生成数据。
|
||||
|
||||
## 实现边界
|
||||
|
||||
自然plan-refresh缺当日基础版时,先只读核对上一交易日三路输入;未齐仅等待,不给旧数据续期。真正齐备后复用同一跨进程发布锁,重新核对日期、现有版本和尝试预算,再完成默认daily因子构建、输入冻结及真实计划发布。明确指定历史date的入口不自动恢复,不通过显式日期绕过就绪检查。
|
||||
|
||||
恢复使用实际采集截止和完成时间,09:30之后由现有版本机制标记recovery,不倒填晨版。采集完成后、发布前再次核验允许时段和来源日期;已有当天成功版本不重复构建。旧格式或不合格的既有同日文件不自动覆盖。恢复本身不增加push-pool或模型调用,保留既有人工关注同步的一次自然执行。
|
||||
|
||||
恢复尝试先原子保存于data/plan/recovery/北京时间日期.json。每天最多3次实际构建,间隔至少900秒,重启不返还次数;未齐检查不耗构建次数。PLAN_RECOVERY_MAX_ATTEMPTS默认3,可降至0关闭;PLAN_RECOVERY_COOLDOWN_SECONDS默认900,可收紧间隔,不低于900。沿桥既有环境配置机制读取,不改变PMS运行参数。
|
||||
|
||||
普通daily构建也使用同一发布锁。冻结函数返回errors、缺核心记录或非法行数时,构建明确失败,不把部分因子/输入包装成完整计划;合法0行仍按原语义保留。区制追加返回独立publication_status,CLI不再把缺版、失败或窗外未发布伪报成功,原区制status字段含义保持。
|
||||
|
||||
证券入口统一支持沪深前缀、点后缀和六位裸码,显式交易所须与A股号段匹配。读取及样本构建前排除北交所、错配及非法代码;market和pool模式均适用。因子、冻结及嵌套证券成员均过滤,日期锚也仅看沪深;历史北交所记录不删除、不改写,因子区间删除只触及沪深。市场指数的两市成交额口径保持。
|
||||
|
||||
## 验证与实际发布
|
||||
|
||||
开发机空配置、禁网护栏全量38/38测试文件通过,末行ALL OFFLINE SUITES PASS。包括恢复28、日期就绪15、范围19及取数5项专项;已有版本、计划、质量排序、人工关注和估值回归全部通过。独立复核文件锁、次数/冷却、旧版本保留与跨窗口发布边界。开发日志/private/tmp/bridge-recovery-release-20261105/full-local-tests.log。
|
||||
|
||||
475bfec候选断网38/38测试文件通过后,于北京时间10:02完成155的akg_factor_bridge首次定向发布。原目录d9c4b92、原.env和data共享挂载保留,源码切独立发布目录,不重建PMS或188。
|
||||
|
||||
首次实机只读就绪检查发现,三路MAX加证券范围过滤后可能扫描大量旧行情,耗时达到数分钟。9f8224e补上日期大于等于目标日的参数化范围,仍读取更新日期以拒绝未来沪深数据,不放宽新鲜度,也不让范围外证券影响日期锚。开发机与第二次候选均38/38通过。第二次仅桥容器于10:18:22启动,10:18:44核验完成;17个其他容器标识、镜像和启动时间均保持,原源码、环境和历史晨版哈希保持。最终实机只读就绪检查0.441秒返回11-04。
|
||||
|
||||
## 自然恢复与PMS判收
|
||||
|
||||
以下时刻统一按北京时间说明。自然任务f5a16fb3在10:05:00启动plan-refresh,10:05:36.721028登记第一次实际构建,10:08:24.634400完成恢复。其日志10:08:38明确返回recovered、data_date为11-04、attempts为1,随后同轮修订unchanged。人工请求5b181fe6在10:05:36排队,直到自然任务结束才于10:08:38执行,并在10:08:53返回unchanged,没有再次构建。10:10和10:15自然刷新也保持正常。
|
||||
|
||||
首个完整版本的采集截止为10:07:00.338807,publication_kind为recovery,原因是“数据恢复后补发”,晨版统计资格为false。没有倒填07:10。完整主榜579行、观察档646行,包含6条有效人工登记;质地排序、吸筹参考规则保持。后续真实修订另存父子版本,首版固定指针保持。
|
||||
|
||||
10:16只读核验实际冻结的13个文件:清单与文件行数全部一致,errors和warnings均为空;热度5208行、收盘价5218行,全部证券代码及嵌套成员均符合沪深范围,未发现范围外或无法确认身份的证券记录。事件因子0行按原语义保留,不伪造为非空。
|
||||
|
||||
08:45缺版的区制追加已由授权补跑3d52f1c0于10:14:29至10:14:55完成。新版本保留真实10:14:31截止;区制为OK,数据日11-04,弱势数8,weak_day为true。10:15自然刷新unchanged,版本健康记录无错误、无待核验股票。
|
||||
|
||||
10:18:48和10:19:46在155模拟PMS的新只读进程验证:实例表前缀为空,直接读取当前上游后,计划、热度和行情数据日均11-04,交易日龄1,通过原上限1;没有调用带接收快照写入的get_plan。按真实参数及持仓、黑名单只读筛选,返回主榜300行、观察档100行,过滤后有86只合格、当前截取50只,其中“候选”38只、“关注”12只,均为沪深股票。排序模式为upstream,route_by_verdict仍为true。这是候选可读和筛选恢复,不是50只均获建仓批准;后续命令、三源合议及交易边界未改变。
|
||||
|
||||
## 尚未完成与证据
|
||||
|
||||
10:19:46读回6条人工关注买方评析均completed,真实报告日期为11-04,读取错误为0;每日研究6条均COMPLETE,执行日11-05、行情数据日11-04。盘中研究3条COMPLETE、3条UNAVAILABLE:1条模型未返回合法研究结论,2条超时;超时回执缺少行情数据日期,未按成功展示。它们属于当日盘中研判的真实未完成项,不等同桥计划过期,不通过重复构建或额外投递模型来制造完成。
|
||||
|
||||
桥基础版、区制和PMS计划新鲜度已恢复,无需重复补建。外部热度生产任务本身不在本轮修改范围;本次证明实际数据已齐及恢复链有效,不宣称外部停更原因已修复。盘中3条不可用回执继续按原有重试、超时和交易守成规则处理。
|
||||
|
||||
原始只读聚合证据保存在开发机私有目录/private/tmp/bridge-recovery-release-20261105:recovery-live-summary.json、pms-plan-live-summary-final.json。日志只摘取调度状态、版本指纹、数量与日期,不记录证券名单、模型正文或账户金额。性能补丁全量日志为readiness-bounded-full-tests.log;最终性能和容器基线由发布主流程另保存。探针只读文件、只读SQL及GET接口,禁止业务写入、模型和任务触发;发布瞬间的一次连接中断已在服务就绪后重新只读核实,不计为业务恢复失败。
|
||||
67
factors.py
67
factors.py
|
|
@ -14,7 +14,6 @@ import pandas as pd
|
|||
import config
|
||||
import common
|
||||
import db
|
||||
import market_scope as scope
|
||||
|
||||
FACTORS = {
|
||||
"akg_upside": "t_factor_akg_upside",
|
||||
|
|
@ -41,46 +40,34 @@ _EMPTY = pd.DataFrame(columns=["trade_date", "stock_code", "factor_value"])
|
|||
|
||||
|
||||
# ---------------------------------------------------------------- 热度
|
||||
def _read_heat_rows(start, end):
|
||||
"""先在沪深范围确定每日最新批次,再读证券行;不加载范围外热度。"""
|
||||
condition = scope.sql_filter("stock_code")
|
||||
batches = db.read_mysql("heat", "SELECT trade_date, MAX(batch_no) AS batch_no "
|
||||
"FROM stock_fund_heat_scores WHERE trade_date BETWEEN %s AND %s AND " + condition +
|
||||
" GROUP BY trade_date", (start, end))
|
||||
parts = []
|
||||
for row in batches.itertuples(index=False):
|
||||
if pd.isna(row.batch_no):
|
||||
continue
|
||||
values = db.read_mysql("heat", "SELECT trade_date, stock_code, batch_no, score "
|
||||
"FROM stock_fund_heat_scores WHERE trade_date = %s AND batch_no = %s AND " + condition,
|
||||
(row.trade_date, row.batch_no))
|
||||
parts.append(scope.frame(values, "stock_code", prefix=True))
|
||||
return pd.concat(parts, ignore_index=True) if parts else pd.DataFrame(
|
||||
columns=["trade_date", "stock_code", "batch_no", "score"])
|
||||
|
||||
|
||||
def build_heat(start, end):
|
||||
"""热度 = 沪深证券当日最新批次 score(0~1)。"""
|
||||
df = _read_heat_rows(start, end)
|
||||
"""热度 = stock_fund_heat_scores 最新批次 score(0~1)。stock_code 已前缀式。"""
|
||||
df = db.read_mysql("heat",
|
||||
"""SELECT s.trade_date, s.stock_code, s.score AS factor_value
|
||||
FROM stock_fund_heat_scores s
|
||||
JOIN (SELECT trade_date, MAX(batch_no) bn FROM stock_fund_heat_scores
|
||||
WHERE trade_date BETWEEN %s AND %s GROUP BY trade_date) m
|
||||
ON m.trade_date = s.trade_date AND m.bn = s.batch_no""",
|
||||
(start, end))
|
||||
if df.empty:
|
||||
return _EMPTY
|
||||
df["stock_code"] = df["stock_code"].astype(str).str.strip()
|
||||
df["factor_value"] = pd.to_numeric(df["factor_value"], errors="coerce")
|
||||
df = common.universe_filter(df, col="stock_code", as_prefix=True)
|
||||
df["factor_value"] = pd.to_numeric(df["score"], errors="coerce")
|
||||
return df[["trade_date", "stock_code", "factor_value"]]
|
||||
|
||||
|
||||
def _price_code_col() -> str:
|
||||
"""gp_day_data 的代码列名(平台实测 = symbol,非 ts_code)。
|
||||
|
||||
用 LIMIT 0 探列名,而不是拿全区间查询去试错——原来那种写法在 history 模式下
|
||||
用 LIMIT 1 探列名,而不是拿全区间查询去试错——原来那种写法在 history 模式下
|
||||
第一次试探就会拉一次全区间数据。"""
|
||||
cands = [config.PRICE_CODE_COL] + [c for c in ("symbol", "ts_code")
|
||||
if c != config.PRICE_CODE_COL]
|
||||
last = None
|
||||
for c in cands:
|
||||
try:
|
||||
scope.sql_filter(f"`{c}`") # 列名来自配置,拼 SQL 前校验。
|
||||
db.read_mysql("price", f"SELECT `{c}` FROM gp_day_data LIMIT 0")
|
||||
db.read_mysql("price", f"SELECT `{c}` FROM gp_day_data LIMIT 1")
|
||||
print(f" (现价用 gp_day_data.{c})")
|
||||
return c
|
||||
except Exception as e: # noqa: BLE001 —— 列名不对就换下一个候选
|
||||
|
|
@ -104,12 +91,12 @@ def _read_gp_price(start, end):
|
|||
parts.append(db.read_mysql(
|
||||
"price",
|
||||
f"SELECT `timestamp` AS trade_date, `{col}` AS ts_code, close "
|
||||
f"FROM gp_day_data WHERE `timestamp` BETWEEN %s AND %s AND " + scope.sql_filter(f"`{col}`"),
|
||||
f"FROM gp_day_data WHERE `timestamp` BETWEEN %s AND %s",
|
||||
(cur.date().isoformat(), hi.date().isoformat())))
|
||||
cur = hi + pd.Timedelta(days=1)
|
||||
if not parts:
|
||||
return pd.DataFrame(columns=["trade_date", "ts_code", "close"])
|
||||
return scope.frame(pd.concat(parts, ignore_index=True), "ts_code")
|
||||
return pd.concat(parts, ignore_index=True)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 预期空间
|
||||
|
|
@ -125,10 +112,9 @@ def build_upside(start, end):
|
|||
"""
|
||||
cons = db.read_pg(
|
||||
"SELECT ts_code, asof_date, target_mid_avg FROM v_factor_consensus "
|
||||
"WHERE asof_date <= %s AND " + scope.sql_filter("ts_code", postgres=True), (end,))
|
||||
"WHERE asof_date <= %s", (end,))
|
||||
if cons.empty:
|
||||
return _EMPTY
|
||||
cons = scope.frame(cons, "ts_code")
|
||||
cons["k"] = cons["ts_code"].map(common.to_prefix)
|
||||
# 注:consensus_daily 本就只对池成员聚合(基座 _pool_ts_codes),
|
||||
# 故 market 模式下 upside 覆盖不会真的变宽——这里过滤只为口径一致。
|
||||
|
|
@ -178,12 +164,12 @@ def build_event(start, end):
|
|||
ev = db.read_pg(
|
||||
"SELECT ts_code, disclosure_date, event_type, direction, "
|
||||
" confidence, doc_id, source_type "
|
||||
"FROM v_factor_events WHERE disclosure_date BETWEEN %s AND %s AND " + scope.sql_filter("ts_code", postgres=True), (look, end))
|
||||
"FROM v_factor_events WHERE disclosure_date BETWEEN %s AND %s", (look, end))
|
||||
except Exception as e: # noqa: BLE001 —— 视图还是 v1(无 doc_id/source_type)时退回
|
||||
print(f" (v_factor_events 无 doc_id/source_type,退回旧列——建议先更新视图: {e!r})")
|
||||
ev = db.read_pg(
|
||||
"SELECT ts_code, disclosure_date, event_type, direction "
|
||||
"FROM v_factor_events WHERE disclosure_date BETWEEN %s AND %s AND " + scope.sql_filter("ts_code", postgres=True), (look, end))
|
||||
"FROM v_factor_events WHERE disclosure_date BETWEEN %s AND %s", (look, end))
|
||||
if ev.empty:
|
||||
return _EMPTY
|
||||
ev = common.universe_filter(ev, col="ts_code", as_prefix=False).copy()
|
||||
|
|
@ -263,18 +249,17 @@ def build_transmission(start, end):
|
|||
try:
|
||||
tr = db.read_pg(
|
||||
f"SELECT {cols_new} FROM v_factor_transmission "
|
||||
f"WHERE scan_date BETWEEN %s AND %s AND " + scope.sql_filter("ts_code", postgres=True), (start, end))
|
||||
f"WHERE scan_date BETWEEN %s AND %s", (start, end))
|
||||
col_val = "n_sources"
|
||||
except Exception as e: # noqa: BLE001 —— 视图还是 v1 时退回,但明确告警
|
||||
print(f" ⚠️ v_factor_transmission 还是旧版(无 n_sources)——退回 n_paths,"
|
||||
f"该口径把重复路径计入了强度,请尽快更新视图: {e!r}")
|
||||
tr = db.read_pg(
|
||||
"SELECT scan_date, ts_code, n_paths, moved_ratio "
|
||||
"FROM v_factor_transmission WHERE scan_date BETWEEN %s AND %s AND " + scope.sql_filter("ts_code", postgres=True), (start, end))
|
||||
"FROM v_factor_transmission WHERE scan_date BETWEEN %s AND %s", (start, end))
|
||||
col_val = "n_paths"
|
||||
if tr.empty:
|
||||
return _EMPTY
|
||||
tr = scope.frame(tr, "ts_code")
|
||||
uni = common.load_universe() # 传导恒按池过滤(天生池内语义)
|
||||
tr = tr[tr["ts_code"].isin(uni)].copy()
|
||||
if tr.empty:
|
||||
|
|
@ -339,7 +324,7 @@ def _robust_z(v: pd.Series) -> pd.Series:
|
|||
def _heat_day_for(ds: str):
|
||||
"""热度 T+1 到达 → 取 <= ds 的最新热度日(与 probe / 设计 §3.4 末口径一致)。"""
|
||||
df = db.read_mysql("heat", "SELECT MAX(trade_date) d FROM stock_fund_heat_scores "
|
||||
"WHERE trade_date <= %s AND " + scope.sql_filter("stock_code"), (ds,))
|
||||
"WHERE trade_date <= %s", (ds,))
|
||||
v = None if df.empty else df.iloc[0, 0]
|
||||
return None if v is None or pd.isna(v) else pd.Timestamp(v).date().isoformat()
|
||||
|
||||
|
|
@ -348,8 +333,8 @@ def _col(df: pd.DataFrame, name: str) -> pd.Series:
|
|||
"""子因子 builder 输出 → 前缀码索引的一列(同股同日取最大)。"""
|
||||
if df is None or df.empty:
|
||||
return pd.Series(dtype=float, name=name)
|
||||
x = scope.frame(df, "stock_code", prefix=True)
|
||||
x["k"] = x["stock_code"]
|
||||
x = df.copy()
|
||||
x["k"] = x["stock_code"].map(common.to_prefix)
|
||||
return x.groupby("k")["factor_value"].max().astype(float).rename(name)
|
||||
|
||||
|
||||
|
|
@ -384,12 +369,10 @@ def _risk_set() -> set:
|
|||
if isinstance(ms, str):
|
||||
ms = json.loads(ms)
|
||||
for m in ms or []:
|
||||
ts = scope.normalize((m or {}).get("ts_code"), prefix=True)
|
||||
if not ts:
|
||||
continue
|
||||
ts = (m or {}).get("ts_code")
|
||||
name = str((m or {}).get("name") or "").replace(" ", "")
|
||||
if ts and (name.startswith(_RISK_PREFIXES) or name.endswith("退")):
|
||||
out.add(ts)
|
||||
out.add(common.to_prefix(ts))
|
||||
if out:
|
||||
print(f" (重大风险闸: {len(out)} 只 ST/退市族挡在档位外,"
|
||||
f"EXCLUDE_RISK_NAMES=0 可关)")
|
||||
|
|
@ -403,7 +386,7 @@ def _track_set():
|
|||
try:
|
||||
import tracks
|
||||
df, _missing = tracks.resolve_members(only_confirmed=True)
|
||||
return {code for x in df["ts_code"] if (code := scope.normalize(x, prefix=True))}
|
||||
return {common.to_prefix(x) for x in df["ts_code"]}
|
||||
except Exception as e: # noqa: BLE001 —— 解析失败按闸未启用降级,只告警不中止
|
||||
print(f" ⚠️ 赛道成员表解析失败,C 闸按未启用处理: {e!r}")
|
||||
return None
|
||||
|
|
|
|||
48
freeze.py
48
freeze.py
|
|
@ -36,7 +36,6 @@ import pandas as pd
|
|||
import config
|
||||
import db
|
||||
import version
|
||||
import market_scope as scope
|
||||
|
||||
try: # parquet 更省更快,但不强制装 pyarrow
|
||||
import pyarrow # noqa: F401
|
||||
|
|
@ -48,7 +47,7 @@ except Exception: # noqa: BLE001
|
|||
# ---------------------------------------------------------------- 落盘
|
||||
def _write(df: pd.DataFrame, path: Path) -> int:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
df = pd.DataFrame() if df is None else scope.frozen_frame(df)
|
||||
df = pd.DataFrame() if df is None else df
|
||||
# 空也要留痕:0 行本身是信息("那天上游真的没有数据" ≠ "那天没跑")
|
||||
if _FMT == "parquet":
|
||||
df.to_parquet(path, index=False)
|
||||
|
|
@ -68,49 +67,55 @@ def _git_rev() -> str:
|
|||
# ---------------------------------------------------------------- 各源抓取
|
||||
def _cap_universe(_day: str) -> pd.DataFrame:
|
||||
"""覆盖池当日成员(industry_pools 无历史版本 → 只能靠这里留时点)。"""
|
||||
return scope.frame(db.read_pg("SELECT ts_code FROM v_factor_universe WHERE " +
|
||||
scope.sql_filter("ts_code", postgres=True) + " ORDER BY ts_code"), "ts_code")
|
||||
return db.read_pg("SELECT ts_code FROM v_factor_universe ORDER BY ts_code")
|
||||
|
||||
|
||||
def _cap_transmission(day: str) -> pd.DataFrame:
|
||||
"""传导原始行仅冻结沪深证券;上游结构与日期保持。"""
|
||||
rows = db.read_pg("SELECT * FROM v_factor_transmission WHERE scan_date = %s AND " +
|
||||
scope.sql_filter("ts_code", postgres=True) + " ORDER BY ts_code", (day,))
|
||||
return scope.frozen_frame(rows)
|
||||
"""传导原始行。四路里唯一「过期即不可复原」的推断类信号,最该冻结。"""
|
||||
try:
|
||||
return db.read_pg(
|
||||
"SELECT * FROM v_factor_transmission WHERE scan_date = %s "
|
||||
"ORDER BY ts_code", (day,))
|
||||
except Exception: # noqa: BLE001 —— 旧视图无 rank/target 列时的兜底
|
||||
return db.read_pg(
|
||||
"SELECT * FROM v_factor_transmission WHERE scan_date = %s", (day,))
|
||||
|
||||
|
||||
def _cap_consensus(day: str) -> pd.DataFrame:
|
||||
"""当日可见的最新一致预期(as-of 口径与 build_upside 一致,每股留最新一行)。"""
|
||||
rows = db.read_pg(
|
||||
return db.read_pg(
|
||||
"SELECT DISTINCT ON (ts_code) ts_code, asof_date, target_mid_avg, "
|
||||
" eps_med, np_med, n_orgs, n_reports_90d "
|
||||
"FROM v_factor_consensus WHERE asof_date <= %s AND " +
|
||||
scope.sql_filter("ts_code", postgres=True) + " ORDER BY ts_code, asof_date DESC", (day,))
|
||||
return scope.frame(rows, "ts_code")
|
||||
"FROM v_factor_consensus WHERE asof_date <= %s "
|
||||
"ORDER BY ts_code, asof_date DESC", (day,))
|
||||
|
||||
|
||||
def _cap_price(day: str) -> pd.DataFrame:
|
||||
"""当日收盘(**前复权**,会被后续除权整体重写 → 必须冻结当时看到的值)。"""
|
||||
import factors # 复用同一套代码列探测,口径不分叉
|
||||
col = factors._price_code_col() # noqa: SLF001
|
||||
rows = db.read_mysql(
|
||||
return db.read_mysql(
|
||||
"price",
|
||||
f"SELECT `timestamp` AS trade_date, `{col}` AS code, close "
|
||||
f"FROM gp_day_data WHERE `timestamp` = %s AND " + scope.sql_filter(f"`{col}`"), (day,))
|
||||
return scope.frame(rows, "code")
|
||||
f"FROM gp_day_data WHERE `timestamp` = %s", (day,))
|
||||
|
||||
|
||||
def _cap_heat(day: str) -> pd.DataFrame:
|
||||
"""与因子计算同一沪深日频批次,空结果仍按原口径留痕。"""
|
||||
from factors import _read_heat_rows
|
||||
return _read_heat_rows(day, day)
|
||||
"""当日最新批次热度(T+1 到达,故 day 当天常为空——空也留痕)。"""
|
||||
return db.read_mysql(
|
||||
"heat",
|
||||
"""SELECT s.trade_date, s.stock_code, s.batch_no, s.score
|
||||
FROM stock_fund_heat_scores s
|
||||
JOIN (SELECT trade_date, MAX(batch_no) bn FROM stock_fund_heat_scores
|
||||
WHERE trade_date = %s GROUP BY trade_date) m
|
||||
ON m.trade_date = s.trade_date AND m.bn = s.batch_no""", (day,))
|
||||
|
||||
|
||||
def _cap_optional(sql: str):
|
||||
"""可选视图(第五/六个插槽视图还没建时不该让冻结失败)。"""
|
||||
def _f(_day: str) -> pd.DataFrame:
|
||||
try:
|
||||
return scope.frozen_frame(db.read_pg(sql))
|
||||
return db.read_pg(sql)
|
||||
except Exception: # noqa: BLE001
|
||||
return pd.DataFrame()
|
||||
return _f
|
||||
|
|
@ -123,8 +128,7 @@ _SOURCES = {
|
|||
"price_close": _cap_price,
|
||||
"heat": _cap_heat,
|
||||
# 第五/六个插槽视图建好后自动开始冻结(没建 → 空 DataFrame,不报错)
|
||||
"segment_members": _cap_optional("SELECT * FROM v_factor_segment_members WHERE " +
|
||||
scope.sql_filter("ts_code", postgres=True)),
|
||||
"segment_members": _cap_optional("SELECT * FROM v_factor_segment_members"),
|
||||
"segment_edges": _cap_optional("SELECT * FROM v_factor_segment_edges"),
|
||||
}
|
||||
|
||||
|
|
@ -172,7 +176,7 @@ def snapshot(day: str | None = None, extra_frames: dict | None = None) -> dict:
|
|||
|
||||
for name, fn in _SOURCES.items():
|
||||
try:
|
||||
df = scope.frozen_frame(fn(day))
|
||||
df = fn(day)
|
||||
frames[name] = df
|
||||
manifest["rows"][name] = _write(df, out / f"{name}{_EXT}")
|
||||
except Exception as e: # noqa: BLE001 —— 单源失败不拖累其余源
|
||||
|
|
|
|||
108
market_scope.py
108
market_scope.py
|
|
@ -1,108 +0,0 @@
|
|||
"""沪深 A 股样本边界;只规范化代码,不推断上市资格或补造资料。"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
|
||||
# 与项目证券入口一致:沪市 60/68、深市 00/30;显式市场必须与号段相符。
|
||||
_CODE = re.compile(r'^(?:(SH|SZ)([0-9]{6})|([0-9]{6})[.](SH|SZ)|([0-9]{6}))$')
|
||||
_SQL_CODE = ('^(SH(60|68)[0-9]{4}|(60|68)[0-9]{4}[.]SH|(60|68)[0-9]{4}'
|
||||
'|SZ(00|30)[0-9]{4}|(00|30)[0-9]{4}[.]SZ|(00|30)[0-9]{4})$')
|
||||
_CODE_FIELDS = ('ts_code', 'stock_code', 'symbol', 'code')
|
||||
_MEMBER_FIELDS = frozenset(('members', 'quiet', 'moved_members', 'constituents', 'stocks',
|
||||
'ts_codes', 'stock_codes', 'symbols'))
|
||||
_DROP = object()
|
||||
|
||||
|
||||
def normalize(value, *, prefix=False):
|
||||
"""前缀、点式、六位裸码 → 唯一代码;北交所、错市场、非法值返回 None。"""
|
||||
if value is None or isinstance(value, bool):
|
||||
return None
|
||||
hit = _CODE.fullmatch(str(value).strip().upper())
|
||||
if not hit:
|
||||
return None
|
||||
explicit = hit.group(1) or hit.group(4)
|
||||
number = hit.group(2) or hit.group(3) or hit.group(5)
|
||||
market = 'SH' if number.startswith(('60', '68')) else 'SZ' if number.startswith(('00', '30')) else None
|
||||
if market is None or (explicit and explicit != market):
|
||||
return None
|
||||
return market + number if prefix else number + '.' + market
|
||||
|
||||
|
||||
def sql_filter(column, *, postgres=False):
|
||||
"""仅接受本地列名,SQL 在读取前排除范围外证券;内存仍二次核验。"""
|
||||
if not re.fullmatch(r'(?:[A-Za-z_][A-Za-z0-9_]*[.])?`?[A-Za-z_][A-Za-z0-9_]*`?', column):
|
||||
raise ValueError('证券代码列名非法')
|
||||
cast = 'TEXT' if postgres else 'CHAR'
|
||||
op = '~' if postgres else 'REGEXP'
|
||||
return f"UPPER(TRIM(CAST({column} AS {cast}))) {op} '{_SQL_CODE}'"
|
||||
|
||||
|
||||
def frame(df, column, *, prefix=False):
|
||||
"""样本入口过滤后再计算;不修改调用方 DataFrame。"""
|
||||
if df is None or df.empty:
|
||||
return df
|
||||
if column not in df.columns:
|
||||
raise ValueError('证券样本缺少代码列:' + column)
|
||||
codes = df[column].map(lambda value: normalize(value, prefix=prefix))
|
||||
result = df.loc[codes.notna()].copy()
|
||||
result[column] = codes.loc[codes.notna()]
|
||||
return result
|
||||
|
||||
|
||||
def _nested(value, *, members=False):
|
||||
encoded = isinstance(value, str) and value.lstrip().startswith(('[', '{'))
|
||||
if encoded:
|
||||
try:
|
||||
value = json.loads(value)
|
||||
except ValueError:
|
||||
if members:
|
||||
raise ValueError("冻结成员不是合法 JSON") from None
|
||||
return value
|
||||
if isinstance(value, dict):
|
||||
code_fields = [key for key in _CODE_FIELDS if key in value]
|
||||
if code_fields:
|
||||
codes = [normalize(value[key]) for key in code_fields]
|
||||
if None in codes or len(set(codes)) != 1:
|
||||
return _DROP
|
||||
elif members:
|
||||
return _DROP # 未能确认证券身份的成员不新增归档。
|
||||
out = {}
|
||||
for key, val in value.items():
|
||||
if key in code_fields:
|
||||
out[key] = normalize(val, prefix=key == 'stock_code')
|
||||
continue
|
||||
clean = _nested(val, members=key in _MEMBER_FIELDS)
|
||||
if clean is not _DROP:
|
||||
out[key] = clean
|
||||
elif isinstance(value, (list, tuple)):
|
||||
out = []
|
||||
for item in value:
|
||||
clean = _nested(item, members=members)
|
||||
if clean is not _DROP:
|
||||
out.append(clean)
|
||||
elif members:
|
||||
out = normalize(value)
|
||||
if out is None:
|
||||
return _DROP
|
||||
else:
|
||||
out = value
|
||||
return json.dumps(out, ensure_ascii=False) if encoded else out
|
||||
|
||||
|
||||
def frozen_frame(df):
|
||||
"""过滤冻结中的证券行与嵌套成员;行业关系等非证券行保持。"""
|
||||
if df is None or df.empty:
|
||||
return df
|
||||
out = df.copy()
|
||||
for column in _CODE_FIELDS:
|
||||
if column in out.columns:
|
||||
out = frame(out, column, prefix=column == 'stock_code')
|
||||
for column in out.columns:
|
||||
if column in _CODE_FIELDS:
|
||||
continue
|
||||
def clean(value):
|
||||
result = _nested(value, members=column in _MEMBER_FIELDS)
|
||||
return None if result is _DROP else result
|
||||
out[column] = out[column].map(clean)
|
||||
return out
|
||||
82
plan.py
82
plan.py
|
|
@ -21,7 +21,6 @@ from __future__ import annotations # 注解不在定义时求值:开发机
|
|||
import datetime as dt
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
|
||||
import pandas as pd
|
||||
|
||||
|
|
@ -160,15 +159,12 @@ def _rerank_by_axis(ranked, cards: dict, axis: str, stale_days: int, has_tier: b
|
|||
def _key(item):
|
||||
k = item[0]
|
||||
s = float(item[1])
|
||||
available = pd.notna(s)
|
||||
c = cards.get(k) or {}
|
||||
band = c.get("quality_band")
|
||||
if band is None:
|
||||
band = card.quality_band(c.get("company_review"), stale_days)
|
||||
pointed = bool((c.get("gates") or {}).get("pointed"))
|
||||
if not available:
|
||||
group, s = -float('inf'), -float('inf')
|
||||
elif has_tier:
|
||||
if has_tier:
|
||||
tier = int((s - 190.0) // 20)
|
||||
group = s - 200.0 - tier * 20.0 # 剥掉档位:传导档位不进次序
|
||||
else:
|
||||
|
|
@ -377,7 +373,7 @@ def valuations_for(codes: list, ds: str) -> dict:
|
|||
|
||||
|
||||
def _assemble_cards(ds: str, codes: list, ev: dict, upside: pd.Series,
|
||||
mkt_days: set, risk: set | None = None, user_candidates=None) -> tuple[dict, list]:
|
||||
mkt_days: set, risk: set | None = None) -> tuple[dict, list]:
|
||||
"""候选卡装配(2026-09-02 方案第 2.2 节第三项):对档位表里的全部票(主榜与观察档,
|
||||
裁剪之前)读三路证据、逐票判决。规则在 card.py,取数在 sources.py,这里只做对齐。
|
||||
|
||||
|
|
@ -433,7 +429,7 @@ def _assemble_cards(ds: str, codes: list, ev: dict, upside: pd.Series,
|
|||
evd = {"pointed": bool(mv or e), "theme": theme, "n_sources": n_sources,
|
||||
"chain_fit": (mv or {}).get("chain_fit"),
|
||||
"pct0": d.get("pct0"), "covered": up is not None, "upside": up,
|
||||
"risk_name": k in risk, "user_selected": k in (user_candidates or {}),
|
||||
"risk_name": k in risk,
|
||||
"accum_state": n.get("accum_state"), "accum_score": n.get("accum_score"),
|
||||
"accum_age": n.get("accum_age"), "y_signal": n.get("signal"),
|
||||
"stale_snapshot": stale, "logic": logic.get(k) or []}
|
||||
|
|
@ -446,7 +442,6 @@ def _assemble_cards(ds: str, codes: list, ev: dict, upside: pd.Series,
|
|||
confirm_days=config.LOGIC_SETTLE_CONFIRM_DAYS, exit_days=config.LOGIC_SETTLE_EXIT_DAYS)
|
||||
j = card.judge(evd, start_pct=config.CARD_START_PCT,
|
||||
accum_max_age=config.CARD_ACCUM_MAX_AGE,
|
||||
require_accum_confirm=config.CARD_ACCUM_CONFIRM_REQUIRED,
|
||||
neg_tol=config.UPSIDE_NEG_TOLERANCE,
|
||||
logic_stale_days=config.LOGIC_STALE_DAYS,
|
||||
require_started=config.CARD_REQUIRE_STARTED,
|
||||
|
|
@ -506,7 +501,7 @@ def _assemble_cards(ds: str, codes: list, ev: dict, upside: pd.Series,
|
|||
|
||||
|
||||
def collect(date: str | None = None, top: int = 20, obs_top: int = 10,
|
||||
theme_cap: int = 5, user_candidates=None) -> dict:
|
||||
theme_cap: int = 5) -> dict:
|
||||
"""装配一天的计划为结构化字典。数据缺失抛 RuntimeError(api 侧转 404)。
|
||||
|
||||
2026-09-02 起附带候选卡:main / observe 的装配、排序、裁剪一字不动(下游 PMS 只读
|
||||
|
|
@ -534,35 +529,16 @@ def collect(date: str | None = None, top: int = 20, obs_top: int = 10,
|
|||
names = _names()
|
||||
ev, mkt_days = _evidence(ds)
|
||||
|
||||
import user_candidates as uc
|
||||
score = score[[bool(re.fullmatch(r'(SH(?:60|68)\d{4}|SZ(?:00|30)\d{4})', str(k))) for k in score.index]]
|
||||
selected = {uc.normalize(r['code']): r for r in (user_candidates or [])}
|
||||
if selected:
|
||||
# 完整证券名称随真实修订录制,重放时不回源,也不补造历史名称。
|
||||
manual_names = plan_inputs.read('user_candidates.catalog', uc.catalog)
|
||||
names = dict(names)
|
||||
for code in selected:
|
||||
if manual_names.get(code):
|
||||
names[code] = manual_names[code]
|
||||
ordinary = set(score.index)
|
||||
extras = [k for k in selected if k not in ordinary]
|
||||
main = score[score >= _MAIN_MIN].sort_index().sort_values(ascending=False, kind='mergesort')
|
||||
obs = score[score < _MAIN_MIN].sort_index().sort_values(ascending=False, kind='mergesort')
|
||||
if extras:
|
||||
main = pd.concat([main, pd.Series({k: float('nan') for k in extras})])
|
||||
score = pd.concat([score, pd.Series({k: float('nan') for k in extras})])
|
||||
generated_at = dt.datetime.now().isoformat(timespec="seconds")
|
||||
try: # 风险名单只读一次:候选卡与升降档原因共用
|
||||
import factors
|
||||
risk = plan_inputs.factor_read('_risk_set') or set()
|
||||
except Exception: # noqa: BLE001
|
||||
risk = set()
|
||||
if selected and config.EXCLUDE_RISK_NAMES:
|
||||
risk = set(risk) | {k for k in selected
|
||||
if str(names.get(k) or '').replace(' ', '').startswith(factors._RISK_PREFIXES)
|
||||
or str(names.get(k) or '').endswith('退')}
|
||||
cards, segments_pointed = _assemble_cards(
|
||||
ds, list(main.index) + list(obs.index), ev, upside, mkt_days, risk=risk, user_candidates=selected)
|
||||
ds, list(main.index) + list(obs.index), ev, upside, mkt_days, risk=risk)
|
||||
|
||||
# 排序轴(2026-09-17):拿到评析后在主榜与观察档这两个分数序列上重排一次。
|
||||
# 下面的 _pick 出接口、_full_rows 出复盘快照、快照又喂逐票逻辑状态入库与评析目标名单——
|
||||
|
|
@ -606,10 +582,10 @@ def collect(date: str | None = None, top: int = 20, obs_top: int = 10,
|
|||
evidence = {"theme": c["theme"], "n_sources": c.get("n_sources"),
|
||||
"moved_ratio": None, "source": "moved_view"}
|
||||
r = {"rank": rank, "code": k, "name": names.get(k),
|
||||
"score": round(float(s), 2) if pd.notna(s) else None,
|
||||
"score": round(float(s), 2),
|
||||
# 调整分只随行展示、供排序追溯;档位标签仍按原始 score 解码(工作包四)
|
||||
"score_adj": round(
|
||||
float(s) + _quality_bonus(c.get("company_review"), config.PLAN_QUALITY_BONUS), 2) if pd.notna(s) else None,
|
||||
float(s) + _quality_bonus(c.get("company_review"), config.PLAN_QUALITY_BONUS), 2),
|
||||
"evidence": evidence,
|
||||
"heat": _val(heat, k), "upside": _val(upside, k)}
|
||||
# 排序轴四键(2026-09-17):质地带、主轴名、是否被指向(候选卡门槛一)、旧序全量名次。
|
||||
|
|
@ -622,7 +598,7 @@ def collect(date: str | None = None, top: int = 20, obs_top: int = 10,
|
|||
r["pointed"] = bool((c.get("gates") or {}).get("pointed"))
|
||||
if config.PLAN_RANK_AXIS == "quality":
|
||||
r["rank_old"] = rank_old_map.get(k)
|
||||
if with_tier and pd.notna(s):
|
||||
if with_tier:
|
||||
r["tier"] = _tier_label(s)
|
||||
if c:
|
||||
# 2026-09-03 新增只联入:basis(判决依据一句话)、logic(因果论断带出处的文字行)、
|
||||
|
|
@ -655,11 +631,6 @@ def collect(date: str | None = None, top: int = 20, obs_top: int = 10,
|
|||
"confirm": c.get("confirm"), "failed_gates": c.get("failed_gates"),
|
||||
"chain_fit": c.get("chain_fit"),
|
||||
"logic_claims": c.get("logic_claims") or []})
|
||||
if k in selected:
|
||||
r['user_candidate'] = selected[k]
|
||||
r['user_required'] = k not in ordinary or not r['pointed']
|
||||
if not names.get(k):
|
||||
r.update(verdict=card.VERDICT_SHOW, basis='证券资料尚未齐全,暂不参与交易')
|
||||
return r
|
||||
|
||||
def _full_rows(ranked: pd.Series, with_tier: bool) -> list:
|
||||
|
|
@ -756,7 +727,6 @@ def collect(date: str | None = None, top: int = 20, obs_top: int = 10,
|
|||
for v in (card.VERDICT_CANDIDATE, card.VERDICT_WATCH, card.VERDICT_SHOW)}
|
||||
result = {
|
||||
"date": ds,
|
||||
"user_candidates": list(selected.values()),
|
||||
"generated_at": generated_at,
|
||||
"plan_version": version.git_short_rev(),
|
||||
"counts": {"main": int(len(main)), "observe": int(len(obs)),
|
||||
|
|
@ -776,22 +746,14 @@ def collect(date: str | None = None, top: int = 20, obs_top: int = 10,
|
|||
"transmission 时按 09-14 的质地调整分序",
|
||||
# ---- 候选卡(2026-09-02):分数与档位之外的另一份产物,不受 top / theme_cap 裁剪 ----
|
||||
"card_params": {"start_pct": config.CARD_START_PCT,
|
||||
"accum_confirm_required": config.CARD_ACCUM_CONFIRM_REQUIRED,
|
||||
"admission_rule": ("accum-confirm-v1" if config.CARD_ACCUM_CONFIRM_REQUIRED
|
||||
else "accum-reference-v2-20261104"),
|
||||
"require_started": config.CARD_REQUIRE_STARTED,
|
||||
"accum_max_age": config.CARD_ACCUM_MAX_AGE,
|
||||
"neg_tol": config.UPSIDE_NEG_TOLERANCE,
|
||||
"logic_per_stock": config.LOGIC_CLAIMS_PER_STOCK,
|
||||
"rules": ("候选=环节被指向∧当日涨幅达标∧券商覆盖且非ST∧明确吸筹∧无硬风险;"
|
||||
"关注=三门槛全过∧无硬风险∧确认线缺失或陈旧(系统无法判断,交人裁决);"
|
||||
"其余仅展示(只差覆盖、潜在吸筹等非明确状态都不升格,台账 013);"
|
||||
"因果论断只展示不进判决" if config.CARD_ACCUM_CONFIRM_REQUIRED else
|
||||
"质地排序不变,候选须满足现行选股资格、覆盖及风险条件;"
|
||||
"吸筹仅作参考,非明确、潜在、缺评分或陈旧本身不否决;"
|
||||
"高位派发及其他真实风险仍拦截,质地差仍按现行规则降为关注;"
|
||||
"建仓看基本面、技术面和择时三源合议,由PMS按命令、风控与现行自决档处理;"
|
||||
"因果论断只展示不进判决")},
|
||||
"rules": "候选=环节被指向∧当日涨幅达标∧券商覆盖且非ST∧明确吸筹∧无硬风险;"
|
||||
"关注=三门槛全过∧无硬风险∧确认线缺失或陈旧(系统无法判断,交人裁决);"
|
||||
"其余仅展示(只差覆盖、潜在吸筹等非明确状态都不升格,台账 013);"
|
||||
"因果论断只展示不进判决"},
|
||||
"card_counts": card_counts,
|
||||
"candidates": _by_verdict(card.VERDICT_CANDIDATE),
|
||||
"watch": _by_verdict(card.VERDICT_WATCH),
|
||||
|
|
@ -986,8 +948,8 @@ def render_md(d: dict) -> str:
|
|||
L.append("| # | 代码 | 名称 | 总分 | 档位 | 传导证据 | 热度 | 预期空间 |")
|
||||
L.append("|---|------|------|------|------|----------|------|----------|")
|
||||
for r in d["main"]:
|
||||
L.append(f"| {r['rank']} | {r['code']} | {r['name'] or '—'} | {_fmt_num(r['score'])} "
|
||||
f"| {r.get('tier') or '—'} | {_fmt_ev(r['evidence'])} "
|
||||
L.append(f"| {r['rank']} | {r['code']} | {r['name'] or '—'} | {r['score']:.1f} "
|
||||
f"| {r['tier']} | {_fmt_ev(r['evidence'])} "
|
||||
f"| {_fmt_num(r['heat'])} | {_fmt_pct(r['upside'])} |")
|
||||
L.append("")
|
||||
|
||||
|
|
@ -998,7 +960,7 @@ def render_md(d: dict) -> str:
|
|||
L.append("| # | 代码 | 名称 | 分 | 传导证据 | 热度 |")
|
||||
L.append("|---|------|------|----|----------|------|")
|
||||
for r in d["observe"]:
|
||||
L.append(f"| {r['rank']} | {r['code']} | {r['name'] or '—'} | {_fmt_num(r['score'])} "
|
||||
L.append(f"| {r['rank']} | {r['code']} | {r['name'] or '—'} | {r['score']:.1f} "
|
||||
f"| {_fmt_ev(r['evidence'])} | {_fmt_num(r['heat'])} |")
|
||||
L.append("")
|
||||
|
||||
|
|
@ -1084,7 +1046,7 @@ def pick_rows(rows, n: int, theme_cap: int, uncapped_no_theme=None) -> list:
|
|||
|
||||
|
||||
def from_snapshot(date: str | None = None, top: int = 20, obs_top: int = 10,
|
||||
theme_cap: int = 5, decision_id=None, include_user_candidates=False) -> dict:
|
||||
theme_cap: int = 5, decision_id=None) -> dict:
|
||||
"""从当日 JSON 快照读计划,按请求参数截取。**不做任何装配、不查任何库。**
|
||||
|
||||
为什么要这条路(2026-09-10,台账 057): collect 每次请求都现场装配整池,实测三十三秒,
|
||||
|
|
@ -1101,8 +1063,6 @@ def from_snapshot(date: str | None = None, top: int = 20, obs_top: int = 10,
|
|||
ds = date or (decision_id[:10] if decision_id else plan_versions.latest_date(config.PLAN_SNAPSHOT_DIR))
|
||||
path = os.path.join(config.PLAN_SNAPSHOT_DIR, f"plan_{ds}.json")
|
||||
snap = plan_versions.load(config.PLAN_SNAPSHOT_DIR, ds, decision_id)
|
||||
import user_candidates as uc
|
||||
snap = uc.consumer_view(snap, include_user_candidates)
|
||||
if str(snap.get("date") or "") != str(ds):
|
||||
raise RuntimeError(f"快照里的数据日 {snap.get('date')} 与请求的 {ds} 对不上: {path}")
|
||||
full_main, full_obs = snap.get("main"), snap.get("observe")
|
||||
|
|
@ -1135,7 +1095,7 @@ def generate(date: str | None = None, top: int = 20, obs_top: int = 10,
|
|||
return _generate(date, top, obs_top, theme_cap)
|
||||
|
||||
|
||||
def _generate(date, top, obs_top, theme_cap, *, before_publish=None):
|
||||
def _generate(date, top, obs_top, theme_cap):
|
||||
ds = date or _latest_date('t_factor_akg_score')
|
||||
# 既有夜间流水线也会触发全量构建。已发布日只能由独立 refresh 修订,
|
||||
# 不能重新取研究覆盖晨版或把旧格式快照升级成事后生成的真实历史。
|
||||
|
|
@ -1147,12 +1107,10 @@ def _generate(date, top, obs_top, theme_cap, *, before_publish=None):
|
|||
write_published_md(published)
|
||||
print(f"复用已发布计划 {ds},未重新采集输入")
|
||||
return os.path.join(config.PLAN_SNAPSHOT_DIR, f"plan_{ds}.md")
|
||||
import user_candidates as uc
|
||||
selected = uc.active(config.PLAN_SNAPSHOT_DIR)
|
||||
bundle = plan_inputs.Bundle(params=plan_inputs.effective_config(config.module))
|
||||
try:
|
||||
with bundle.use():
|
||||
data = collect(ds, top, obs_top, theme_cap, **({"user_candidates": selected} if selected else {}))
|
||||
data = collect(ds, top, obs_top, theme_cap)
|
||||
data['market'] = sources.market_context(data['date'])
|
||||
except RuntimeError as e:
|
||||
raise SystemExit(str(e))
|
||||
|
|
@ -1167,10 +1125,8 @@ def _generate(date, top, obs_top, theme_cap, *, before_publish=None):
|
|||
"main": full.get("main", []), "observe": full.get("observe", []),
|
||||
"shown_params": {"top": top, "obs_top": obs_top, "theme_cap": theme_cap}}
|
||||
jpath = os.path.join(config.PLAN_SNAPSHOT_DIR, f"plan_{data['date']}.json")
|
||||
if before_publish is not None:
|
||||
before_publish(data['date'])
|
||||
snap = plan_versions.publish(config.PLAN_SNAPSHOT_DIR, snap,
|
||||
{'records': bundle.records, 'market': data['market'], 'regime': data.get('regime'), 'user_candidates': selected},
|
||||
{'records': bundle.records, 'market': data['market'], 'regime': data.get('regime')},
|
||||
bundle.params, cutoff_at=bundle.cutoff_at, reason='晨版生成')
|
||||
text = write_published_md(snap)
|
||||
# ---- 逐票日频逻辑状态表(2026-09-07 第三件桥侧前置):只在这一次生成里写,/plan 的实时重算
|
||||
|
|
|
|||
149
plan_recovery.py
149
plan_recovery.py
|
|
@ -1,149 +0,0 @@
|
|||
"""自然修订缺当日基础版时,待真实输入齐备后有限补发。
|
||||
|
||||
只由 plan-refresh 的缺版分支进入。日期守卫、日更、采集及发布在同一文件锁内;
|
||||
不调用股票池、模型或评析同步,不续期旧输入,不重跑已有成功基础版。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime as dt
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import build_readiness
|
||||
import config
|
||||
import plan_refresh
|
||||
import plan_versions as pv
|
||||
|
||||
|
||||
def _now():
|
||||
return dt.datetime.now(dt.timezone.utc)
|
||||
|
||||
|
||||
def _baseline(root, day, now):
|
||||
"""只认可真实当前决策日的完整首版;既有不合格文件绝不覆盖。"""
|
||||
latest = pv.latest_date(root)
|
||||
if latest and latest > day:
|
||||
return 'blocked_existing_snapshot'
|
||||
try:
|
||||
current = pv.load(root, day)
|
||||
except FileNotFoundError:
|
||||
return 'missing'
|
||||
except (ValueError, TypeError):
|
||||
return 'blocked_existing_snapshot'
|
||||
try:
|
||||
if current['date'] != day or current.get('traceability') != 'captured_inputs':
|
||||
raise ValueError('计划来源不可核验')
|
||||
pv.envelope(root, current['decision']['id'])
|
||||
first = pv.load(root, day, morning=True)
|
||||
cutoff = dt.datetime.fromisoformat(first['decision']['cutoff_at'])
|
||||
if (cutoff.tzinfo is None or cutoff > now
|
||||
or cutoff.astimezone(plan_refresh.BJ).date() != now.astimezone(plan_refresh.BJ).date()):
|
||||
raise ValueError('首版不属于今日')
|
||||
except (KeyError, TypeError, ValueError, FileNotFoundError):
|
||||
return 'blocked_existing_snapshot'
|
||||
return 'already_ready'
|
||||
|
||||
|
||||
def _state(path, local_day, data_day):
|
||||
if not path.exists():
|
||||
return {'schema_version': 1, 'decision_date': local_day,
|
||||
'data_date': data_day, 'attempts': 0}
|
||||
state = json.loads(path.read_text())
|
||||
if (not isinstance(state, dict) or state.get('schema_version') != 1
|
||||
or state.get('decision_date') != local_day or state.get('data_date') != data_day
|
||||
or type(state.get('attempts')) is not int or state['attempts'] < 0):
|
||||
raise ValueError('恢复状态不可核验,不能重置重试次数')
|
||||
if state['attempts']:
|
||||
last = dt.datetime.fromisoformat(state['last_attempt_at'])
|
||||
if last.tzinfo is None:
|
||||
raise ValueError('恢复时刻缺少时区')
|
||||
return state
|
||||
|
||||
|
||||
def ensure_today(day=None, *, now=None):
|
||||
"""只接受自然最新日入口;返回可核验状态,失败不伪装成功。"""
|
||||
clock = (lambda: now) if now is not None else _now
|
||||
current = clock()
|
||||
if day is not None:
|
||||
return {'status': 'explicit_date', 'reason': '指定日期的维护入口不自动重建。'}
|
||||
if not plan_refresh.in_window(current):
|
||||
return {'status': 'outside_window'}
|
||||
expected = build_readiness.expected_data_day(current).isoformat()
|
||||
root = config.PLAN_SNAPSHOT_DIR
|
||||
# 有效首版已在时,不因当前来源暂时不可达而重建;后续仍由正常 refresh 修订。
|
||||
baseline = _baseline(root, expected, current)
|
||||
if baseline != 'missing':
|
||||
return {'status': baseline, 'data_date': expected}
|
||||
try:
|
||||
ready = build_readiness.check(current)
|
||||
if ready != expected:
|
||||
raise RuntimeError('输入日期与当前交易日不一致。')
|
||||
except RuntimeError as exc:
|
||||
return {'status': 'waiting_data', 'data_date': expected, 'reason': str(exc)}
|
||||
|
||||
with pv.writer(root):
|
||||
current = clock()
|
||||
if (not plan_refresh.in_window(current)
|
||||
or build_readiness.expected_data_day(current).isoformat() != expected):
|
||||
return {'status': 'outside_window'}
|
||||
baseline = _baseline(root, expected, current)
|
||||
if baseline != 'missing':
|
||||
return {'status': baseline, 'data_date': expected}
|
||||
local_day = current.astimezone(plan_refresh.BJ).date().isoformat()
|
||||
path = Path(root) / 'recovery' / (local_day + '.json')
|
||||
try:
|
||||
state = _state(path, local_day, expected)
|
||||
maximum = max(0, min(3, int(config.PLAN_RECOVERY_MAX_ATTEMPTS)))
|
||||
cooldown = max(900, int(config.PLAN_RECOVERY_COOLDOWN_SECONDS))
|
||||
if state['attempts'] >= maximum:
|
||||
return {'status': 'attempts_exhausted', 'attempts': state['attempts'],
|
||||
'reason': '当天自动构建次数已达上限,保留上次版本。'}
|
||||
if state['attempts']:
|
||||
last = dt.datetime.fromisoformat(state['last_attempt_at'])
|
||||
elapsed = (current - last).total_seconds()
|
||||
if elapsed < 0:
|
||||
raise ValueError('恢复时刻晚于当前时刻')
|
||||
if elapsed < cooldown:
|
||||
return {'status': 'cooldown', 'attempts': state['attempts'],
|
||||
'retry_after': (last + dt.timedelta(seconds=cooldown)).isoformat()}
|
||||
except (ValueError, TypeError, KeyError):
|
||||
return {'status': 'blocked_state', 'reason': '恢复记录不可核验,未重置次数或重建。'}
|
||||
# 等锁期间输入可能已变化;未齐只等待,不消耗实际构建次数。
|
||||
try:
|
||||
if build_readiness.check(current) != expected:
|
||||
raise RuntimeError('输入日期与当前交易日不一致。')
|
||||
except RuntimeError as exc:
|
||||
return {'status': 'waiting_data', 'data_date': expected, 'reason': str(exc)}
|
||||
state.update(attempts=state['attempts'] + 1, last_attempt_at=current.isoformat(),
|
||||
status='building', error=None)
|
||||
pv.atomic(path, state) # 在实际构建前持久化;进程退出不能退还本次预算。
|
||||
try:
|
||||
import run
|
||||
import plan
|
||||
built = run._cmd_build('all', 'daily', None, None, None)
|
||||
current = clock()
|
||||
if (not plan_refresh.in_window(current)
|
||||
or build_readiness.expected_data_day(current).isoformat() != expected):
|
||||
raise RuntimeError('构建完成时已离开允许恢复时段。')
|
||||
if built != expected or build_readiness.check(current) != expected:
|
||||
raise RuntimeError('构建完成后输入日期变化,未发布计划。')
|
||||
def before_publish(data_day):
|
||||
publishing = clock()
|
||||
if (data_day != expected or not plan_refresh.in_window(publishing)
|
||||
or build_readiness.expected_data_day(publishing).isoformat() != expected):
|
||||
raise RuntimeError('计划采集结束时已离开允许恢复时段或日期不符。')
|
||||
if build_readiness.check(publishing) != expected:
|
||||
raise RuntimeError('计划发布前输入日期变化,未发布计划。')
|
||||
# 持有与 generate 同一文件锁,直接调用内部实现避免递归 flock。
|
||||
# Bundle 自取实际采集截止时刻;09:30 后首版由 publish 标为 recovery。
|
||||
plan._generate(expected, 20, 10, 5, before_publish=before_publish)
|
||||
if _baseline(root, expected, clock()) != 'already_ready':
|
||||
raise RuntimeError('恢复发布未形成可核验的当前基础版。')
|
||||
except (Exception, SystemExit) as exc:
|
||||
state.update(status='failed', completed_at=clock().isoformat(),
|
||||
error=f'基础版恢复失败({type(exc).__name__}),保留上次已发布版本。')
|
||||
pv.atomic(path, state)
|
||||
return {'status': 'failed', 'attempts': state['attempts'], 'reason': state['error']}
|
||||
state.update(status='recovered', completed_at=clock().isoformat(), error=None)
|
||||
pv.atomic(path, state)
|
||||
return {'status': 'recovered', 'data_date': expected, 'attempts': state['attempts']}
|
||||
|
|
@ -43,31 +43,7 @@ def refresh(day=None, *, now=None, regime_value=None):
|
|||
pending = set(pv.health(root, day).get('pending_codes') or [])
|
||||
try:
|
||||
env = pv.envelope(root, old['decision']['id'])
|
||||
params = copy.deepcopy(env['params'])
|
||||
# 只允许本次明确批准的吸筹角色参数随真实修订改变。
|
||||
# 其余冻结参数原样重放,不顺带采用当前环境的排序或资格变化。
|
||||
previous_confirm = params.get('CARD_ACCUM_CONFIRM_REQUIRED', True)
|
||||
desired_confirm = bool(config.CARD_ACCUM_CONFIRM_REQUIRED)
|
||||
policy_changed = previous_confirm != desired_confirm
|
||||
if policy_changed:
|
||||
params['CARD_ACCUM_CONFIRM_REQUIRED'] = desired_confirm
|
||||
pending.update(r['code'] for b in ('main', 'observe') for r in old.get(b, []))
|
||||
inputs = copy.deepcopy(env['inputs'])
|
||||
import user_candidates as uc
|
||||
selected = uc.active(root, now)
|
||||
candidates_changed = selected != inputs.get('user_candidates', [])
|
||||
if candidates_changed:
|
||||
pending.update(r['code'] for r in selected + inputs.get('user_candidates', []))
|
||||
inputs['user_candidates'] = selected
|
||||
pv.health(root, day, pending_codes=sorted(pending), error=None)
|
||||
if selected and not any(r['name'] == 'user_candidates.catalog' for r in inputs['records'].values()):
|
||||
# 旧发布版尚未录制完整证券名单时,作为本次真实修订的新输入补齐。
|
||||
pending.update(r['code'] for r in selected)
|
||||
pv.health(root, day, pending_codes=sorted(pending), error=None)
|
||||
catalog_bundle = pi.Bundle(cutoff_at=now.isoformat())
|
||||
with catalog_bundle.use():
|
||||
pi.read('user_candidates.catalog', uc.catalog)
|
||||
inputs['records'].update(catalog_bundle.records)
|
||||
for rec in inputs['records'].values():
|
||||
if rec['name'] != 'sources.company_reviews':
|
||||
continue
|
||||
|
|
@ -87,28 +63,20 @@ def refresh(day=None, *, now=None, regime_value=None):
|
|||
return {k: v for k, v in (x or {}).items() if k != 'fetched_at'}
|
||||
if meaningful(inputs.get('regime')) != meaningful(regime_value):
|
||||
inputs['regime'] = regime_value
|
||||
if policy_changed:
|
||||
pending.update(r['code'] for b in ('main', 'observe') for r in old.get(b, []))
|
||||
pv.health(root, day, pending_codes=sorted(pending), error=None)
|
||||
if pv.digest(inputs) == old['decision']['input_hash'] and not policy_changed:
|
||||
if pv.digest(inputs) == old['decision']['input_hash']:
|
||||
pv.health(root, day, pending_codes=[], error=None, decision_id=old['decision']['id'])
|
||||
return {'status': 'unchanged', 'decision': old['decision']}
|
||||
bundle = pi.Bundle(records=None if candidates_changed else inputs['records'],
|
||||
params=params, cutoff_at=now.isoformat())
|
||||
if candidates_changed:
|
||||
# 原有调用继续读冻结输入;新增股票带来的新取数在本次真实修订中录制。
|
||||
bundle.records = copy.deepcopy(inputs['records'])
|
||||
bundle = pi.Bundle(records=inputs['records'], params=env['params'], cutoff_at=now.isoformat())
|
||||
shown = old.get('shown_params') or {'top': 20, 'obs_top': 10, 'theme_cap': 5}
|
||||
with bundle.use():
|
||||
data = plan.collect(day, **shown, **({'user_candidates': selected} if selected else {}))
|
||||
inputs['records'] = bundle.records
|
||||
data = plan.collect(day, **shown)
|
||||
full = data.pop('_full')
|
||||
data.update(main_shown=data['main'], observe_shown=data['observe'], shown_params=shown,
|
||||
main=full['main'], observe=full['observe'], market=inputs.get('market'),
|
||||
regime=inputs.get('regime'))
|
||||
reason = ('恢复吸筹确认规则' if desired_confirm else '吸筹改为三源合议参考') if policy_changed else (
|
||||
'用户候选更新' if candidates_changed else '评析更新' if pending else '市场区制更新')
|
||||
published = pv.publish(root, data, inputs, params, cutoff_at=now.isoformat(), reason=reason)
|
||||
published = pv.publish(root, data, inputs, env['params'], cutoff_at=now.isoformat(),
|
||||
reason='评析更新' if pending else '市场区制更新')
|
||||
plan.write_published_md(published)
|
||||
return {'status': 'published', 'decision': published['decision']}
|
||||
except Exception as e:
|
||||
|
|
|
|||
|
|
@ -233,7 +233,7 @@ def publish(root, plan, inputs, params, *, cutoff_at, reason):
|
|||
return p
|
||||
|
||||
|
||||
def stock_check(root, code, include_user_candidates=False):
|
||||
def stock_check(root, code):
|
||||
"""只读完整发布文件的一只股票。不存在与取数失败严格区分;不按展示名次裁剪。"""
|
||||
if not re.fullmatch(r'(SH|SZ|BJ)\d{6}', code):
|
||||
raise ValueError('股票代码无效')
|
||||
|
|
@ -243,24 +243,20 @@ def stock_check(root, code, include_user_candidates=False):
|
|||
raise FileNotFoundError('无已发布计划')
|
||||
path = Path(root) / f'plan_{day}.json'
|
||||
health_path = Path(root) / 'versions' / day / 'health.json'
|
||||
candidates_path = Path(root) / 'user_candidates.json'
|
||||
def identity(p):
|
||||
try:
|
||||
s = p.stat()
|
||||
return (s.st_dev, s.st_ino, s.st_size, s.st_mtime_ns, s.st_ctime_ns)
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
before = (identity(path), identity(health_path), identity(candidates_path))
|
||||
before = (identity(path), identity(health_path))
|
||||
published = load(root, day)
|
||||
state = health(root, day)
|
||||
import user_candidates as uc
|
||||
state = dict(state, pending_codes=sorted(set(state.get('pending_codes') or []) | set(uc.live_pending(root, published))))
|
||||
if (day != latest_date(root) or before != (identity(path), identity(health_path), identity(candidates_path))):
|
||||
if (day != latest_date(root) or before != (identity(path), identity(health_path))):
|
||||
continue
|
||||
if (published.get('date') != day or not (published.get('decision') or {}).get('id')
|
||||
or not all(isinstance(published.get(b), list) for b in ('main', 'observe'))):
|
||||
raise ValueError('完整发布计划不可核验')
|
||||
published = uc.consumer_view(published, include_user_candidates)
|
||||
rows = {b: [r for r in published[b] if r.get('code') == code] for b in ('main', 'observe')}
|
||||
count = sum(map(len, rows.values()))
|
||||
if count > 1:
|
||||
|
|
|
|||
|
|
@ -102,7 +102,7 @@ def append_to_snapshot(day: str) -> dict:
|
|||
path = snapshot_path(day)
|
||||
if not os.path.exists(path):
|
||||
print(f"快照 {path} 不存在,环境段未写入(状态 {reg['status']})")
|
||||
return {**reg, 'publication_status': 'no_morning'}
|
||||
return reg
|
||||
with open(path, "r", encoding="utf-8") as f:
|
||||
doc = json.load(f)
|
||||
# 区制也发布完整修订;旧版不就地改写。旧格式须先生成可追溯晨版。
|
||||
|
|
@ -110,10 +110,10 @@ def append_to_snapshot(day: str) -> dict:
|
|||
result = plan_refresh.refresh(day, regime_value=reg)
|
||||
if result['status'] not in ('published', 'unchanged'):
|
||||
print(f"环境段未发布:{result['status']}")
|
||||
return {**reg, 'publication_status': result['status']}
|
||||
return reg
|
||||
print(f"环境段已写入 {path}:status={reg['status']} weak_count={reg['weak_count']} "
|
||||
f"weak_day={reg['weak_day']}")
|
||||
return {**reg, 'publication_status': result['status']}
|
||||
return reg
|
||||
|
||||
|
||||
def read_section(day: str, key: str):
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
"""个股深度评析的目标名单(2026-09-09 方案附录甲):每早出计划时把主榜与观察档写进 153 库一张小表,
|
||||
数据基座的 00:45 任务读它选目标(178 的容器打不到 155 的接口,只能走库)。
|
||||
|
||||
表 t_akg_review_targets:plan_date、ts_code(点后缀式 600000.SH)、source(main / observe / manual)、rank、name。
|
||||
表 t_akg_review_targets:plan_date、ts_code(点后缀式 600000.SH)、source(main / observe)、rank、name。
|
||||
幂等:删该日行再整批插。写失败不拦计划(try 包住)。"""
|
||||
from __future__ import annotations
|
||||
|
||||
|
|
@ -35,17 +35,13 @@ def to_dot(code: str) -> str:
|
|||
return s
|
||||
|
||||
|
||||
def build_rows(ds: str, main_rows: list, obs_rows: list, user_rows=None) -> list[dict]:
|
||||
from user_candidates import normalize
|
||||
def build_rows(ds: str, main_rows: list, obs_rows: list) -> list[dict]:
|
||||
seen: set = set()
|
||||
out: list[dict] = []
|
||||
for src, rows in (("main", main_rows), ("observe", obs_rows), ("manual", user_rows)):
|
||||
for src, rows in (("main", main_rows), ("observe", obs_rows)):
|
||||
for i, r in enumerate(rows or [], 1):
|
||||
try:
|
||||
code = to_dot(normalize(r.get('code')))
|
||||
except ValueError:
|
||||
continue
|
||||
if code in seen:
|
||||
code = to_dot(str(r.get("code") or ""))
|
||||
if len(code) != 9 or code in seen:
|
||||
continue
|
||||
seen.add(code)
|
||||
out.append({"plan_date": ds, "ts_code": code, "source": src, "rank": i, "name": (r.get("name") or "")[:64]})
|
||||
|
|
@ -93,18 +89,15 @@ def pick_main_targets(main_rows: list, axis: str, top_score: int, top_rank: int)
|
|||
return union
|
||||
|
||||
|
||||
def persist(ds: str, snap: dict, write=None, user_rows=None) -> dict:
|
||||
def persist(ds: str, snap: dict, write=None) -> dict:
|
||||
"""目标是主榜(按 REVIEW_TARGETS_AXIS 取,both 是原始分前 TOP_MAIN ∪ 新序前 TOP_MAIN_RANK)加观察档前
|
||||
TOP_OBS 只(快照里的 main / observe 已是新序全部行,每行带 score 与 rank),约一百二十到一百五十家。"""
|
||||
main_sel = pick_main_targets(snap.get("main") or [], config.REVIEW_TARGETS_AXIS, TOP_MAIN, TOP_MAIN_RANK)
|
||||
rows = build_rows(ds, main_sel, (snap.get("observe") or [])[:TOP_OBS],
|
||||
snap.get('user_candidates', []) if user_rows is None else user_rows)
|
||||
rows = build_rows(ds, main_sel, (snap.get("observe") or [])[:TOP_OBS])
|
||||
(write or save)(ds, rows)
|
||||
n_main = sum(1 for r in rows if r["source"] == "main")
|
||||
n_manual = sum(1 for r in rows if r['source'] == 'manual')
|
||||
n_obs = len(rows) - n_main - n_manual
|
||||
print(f" 评析目标名单 {ds}:主榜 {n_main},观察 {n_obs},人工关注 {n_manual},写入 {TABLE}")
|
||||
return {"date": ds, "rows": len(rows), "main": n_main, 'observe': n_obs, 'manual': n_manual}
|
||||
print(f" 评析目标名单 {ds}:主榜 {n_main},观察 {len(rows) - n_main},写入 {TABLE}")
|
||||
return {"date": ds, "rows": len(rows), "main": n_main}
|
||||
|
||||
|
||||
def from_plan_file(ds: str, plan_dir: str | None = None, write=None) -> dict:
|
||||
|
|
|
|||
40
run.py
40
run.py
|
|
@ -122,16 +122,6 @@ def _latest_data_day() -> str:
|
|||
|
||||
|
||||
def cmd_build(which, mode, start, end, date, do_freeze=True):
|
||||
# 与计划发布/修订使用同一跨进程锁,避免恢复构建时被另一条日更链改写因子。
|
||||
if mode == 'daily':
|
||||
import plan_versions
|
||||
with plan_versions.writer(config.PLAN_SNAPSHOT_DIR):
|
||||
return _cmd_build(which, mode, start, end, date, do_freeze)
|
||||
return _cmd_build(which, mode, start, end, date, do_freeze)
|
||||
|
||||
|
||||
def _cmd_build(which, mode, start, end, date, do_freeze=True):
|
||||
"""内部构建;daily 调用方必须持有计划发布锁。"""
|
||||
if mode == "daily":
|
||||
d = date or _latest_data_day()
|
||||
start = end = d
|
||||
|
|
@ -159,18 +149,9 @@ def _cmd_build(which, mode, start, end, date, do_freeze=True):
|
|||
if do_freeze and mode == "daily":
|
||||
try:
|
||||
import freeze
|
||||
manifest = freeze.snapshot(start, extra_frames=frames)
|
||||
required = {'universe', 'transmission', 'consensus', 'price_close', 'heat'} | set(frames)
|
||||
rows = manifest.get('rows') if isinstance(manifest, dict) else None
|
||||
if (not isinstance(rows, dict) or required - set(rows)
|
||||
or manifest.get('errors') != {}):
|
||||
raise RuntimeError('冻结清单不完整或有来源/落盘错误')
|
||||
# 0 行是既有合法观测;只核验记录齐全,不新增选股资格门槛。
|
||||
if any(type(rows[name]) is not int or rows[name] < 0 for name in required):
|
||||
raise RuntimeError('冻结清单行数不可核验')
|
||||
freeze.snapshot(start, extra_frames=frames)
|
||||
except Exception as e: # noqa: BLE001 —— 因子已写,但完整输入缺失不得继续发布
|
||||
raise RuntimeError("输入冻结失败,停止计划及入池(因子已落库)") from e
|
||||
return start
|
||||
|
||||
|
||||
def main():
|
||||
|
|
@ -254,27 +235,10 @@ def main():
|
|||
day = a.date or plan._latest_date("t_factor_akg_score") # noqa: SLF001 —— 同仓自用
|
||||
if not day:
|
||||
raise SystemExit("t_factor_akg_score 还没有数据,没有当日快照可追加。")
|
||||
result = regime.append_to_snapshot(day)
|
||||
if result.get('publication_status') not in ('published', 'unchanged'):
|
||||
raise SystemExit('区制未发布:' + str(result.get('publication_status') or 'unknown'))
|
||||
regime.append_to_snapshot(day)
|
||||
elif a.cmd == 'plan-refresh':
|
||||
import plan_refresh
|
||||
import user_research
|
||||
try:
|
||||
user_research.sync()
|
||||
except Exception:
|
||||
import logging
|
||||
logging.getLogger(__name__).exception('人工关注评析同步失败,继续正常计划修订')
|
||||
result = plan_refresh.refresh(a.date)
|
||||
if result['status'] in ('no_morning', 'no_morning_today'):
|
||||
import plan_recovery
|
||||
recovery = plan_recovery.ensure_today(a.date)
|
||||
print(recovery)
|
||||
if recovery['status'] in ('already_ready', 'recovered'):
|
||||
result = plan_refresh.refresh(a.date)
|
||||
else:
|
||||
raise SystemExit('今日完整基础版尚未恢复:' + recovery['status']
|
||||
+ ';' + recovery.get('reason', '保留上次版本。'))
|
||||
print(result)
|
||||
if result['status'] in ('no_morning', 'no_morning_today'):
|
||||
raise SystemExit('今日完整晨版缺失,保留上次版本,修订未执行。')
|
||||
|
|
|
|||
38
sources.py
38
sources.py
|
|
@ -40,7 +40,6 @@ import pandas as pd
|
|||
import common
|
||||
import config
|
||||
import db
|
||||
import market_scope
|
||||
|
||||
# 上证指数与深证成指在指数日线表 zs_day_data 里的代码,两市成交额取二者当日 amount 之和。
|
||||
MARKET_INDEX_CODES = ("000001.SH", "399001.SZ")
|
||||
|
|
@ -58,13 +57,12 @@ def moved_members(ds: str) -> dict[str, dict]:
|
|||
df = db.read_pg(
|
||||
"SELECT ts_code, target, n_sources, chain_fit, members_total, moved, "
|
||||
"moved_ratio, mkt_trade_date FROM v_factor_transmission_moved "
|
||||
"WHERE scan_date = %s AND " + market_scope.sql_filter('ts_code', postgres=True), (ds,))
|
||||
"WHERE scan_date = %s", (ds,))
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f" (已动成员视图读取失败,候选卡的传导门槛整体缺席: {e!r})")
|
||||
return {}
|
||||
if df.empty:
|
||||
return {}
|
||||
df = market_scope.frame(df, 'ts_code')
|
||||
df["k"] = df["ts_code"].map(lambda s: common.to_prefix(str(s).strip()))
|
||||
df["n_sources"] = pd.to_numeric(df["n_sources"], errors="coerce").fillna(0)
|
||||
df["chain_fit"] = pd.to_numeric(df["chain_fit"], errors="coerce").fillna(0)
|
||||
|
|
@ -84,11 +82,10 @@ def stock_daily(ds: str) -> dict[str, dict]:
|
|||
try:
|
||||
df = db.read_pg(
|
||||
"SELECT ts_code, pct_change, net_z, heat_chg FROM v_factor_stock_daily "
|
||||
"WHERE trade_date = %s AND " + market_scope.sql_filter('ts_code', postgres=True), (ds,))
|
||||
"WHERE trade_date = %s", (ds,))
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f" (个股日行情视图读取失败,候选卡的已启动门槛整体缺席: {e!r})")
|
||||
return {}
|
||||
df = market_scope.frame(df, 'ts_code')
|
||||
out = {}
|
||||
for r in df.itertuples():
|
||||
k = common.to_prefix(str(r.ts_code).strip())
|
||||
|
|
@ -483,12 +480,10 @@ def market_context(ds: str, read_pg=None, read_mysql=None) -> dict:
|
|||
out["errors"]["turnover"] = repr(e)
|
||||
print(f" (两市成交额读取失败,环境段该项为空: {e!r})")
|
||||
|
||||
# 二、市场广度:读取与样本入口均限定沪深股票,不影响上面的指数成交额。
|
||||
# 二、市场广度:基座个股日行情视图当日全部行自算。
|
||||
try:
|
||||
rows = _records(rpg(
|
||||
"SELECT ts_code, pct_change FROM v_factor_stock_daily WHERE trade_date = %s AND "
|
||||
+ market_scope.sql_filter('ts_code', postgres=True), (ds,)))
|
||||
rows = [row for row in rows if market_scope.normalize(row.get('ts_code'))]
|
||||
"SELECT pct_change FROM v_factor_stock_daily WHERE trade_date = %s", (ds,)))
|
||||
out["breadth"] = _breadth(rows)
|
||||
if out["breadth"] is None:
|
||||
out["errors"]["breadth"] = "v_factor_stock_daily 当日无行"
|
||||
|
|
@ -791,14 +786,13 @@ def close_prices(ds: str, read_mysql=None, code_col: str | None = None) -> dict:
|
|||
code_col = factors._price_code_col() # noqa: SLF001 —— 同仓自用
|
||||
nxt = (dt.date.fromisoformat(ds) + dt.timedelta(days=1)).isoformat()
|
||||
df = reader("price", f"SELECT `{code_col}` AS ts_code, close FROM gp_day_data "
|
||||
f"WHERE `timestamp` >= %s AND `timestamp` < %s AND "
|
||||
+ market_scope.sql_filter('`' + code_col + '`'), (ds, nxt))
|
||||
f"WHERE `timestamp` >= %s AND `timestamp` < %s", (ds, nxt))
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f" (收盘价读取失败,安全边际这一行整体缺席: {e!r})")
|
||||
return {}
|
||||
out = {}
|
||||
for r in _records(df):
|
||||
k = market_scope.normalize(r.get("ts_code"), prefix=True)
|
||||
k = _prefix_any(r.get("ts_code"))
|
||||
px = _f(r.get("close"))
|
||||
if k and px and px > 0:
|
||||
out[k] = px
|
||||
|
|
@ -1180,7 +1174,7 @@ def catalysts_for_codes(seg_of: dict, cat_by_seg: dict, *, limit: int = 3) -> di
|
|||
# 公司深度评析(2026-09-09 接入方案,台账 053):数据基座 company_review_runs 索引表最新一行的摘要。
|
||||
# 质地由基座代码合成,这里只读不算;报告正文在对象存储,链接给人。表读不到整体缺席不断产。
|
||||
# ---------------------------------------------------------------------------
|
||||
def company_reviews(codes, ds: str, *, read_pg=None, cutoff_at=None, strict=False, include_check=False) -> dict:
|
||||
def company_reviews(codes, ds: str, *, read_pg=None, cutoff_at=None, strict=False) -> dict:
|
||||
"""{前缀码: {"period", "review_version", "overall", "groups", "valuation", "doubt_hard", "quality_reasons",
|
||||
"grade_counts", "grade_line", "thesis", "confidence", "invalidation", "antithesis", "catalyst", "brain_status",
|
||||
"ran_at", "age_days", "report_url"}};没报告的票不在字典里。"""
|
||||
|
|
@ -1191,8 +1185,7 @@ def company_reviews(codes, ds: str, *, read_pg=None, cutoff_at=None, strict=Fals
|
|||
dots = [_to_dot(c) for c in want]
|
||||
try:
|
||||
cutoff = cutoff_at or (str(ds) + 'T23:59:59.999999+08:00')
|
||||
checked_column = ', checked_at' if include_check else ''
|
||||
df = rp("SELECT DISTINCT ON (ts_code) ts_code, period, review_version, grades, summary, brain_used, brain_status, ran_at" + checked_column + " "
|
||||
df = rp("SELECT DISTINCT ON (ts_code) ts_code, period, review_version, grades, summary, brain_used, brain_status, ran_at "
|
||||
"FROM company_review_runs WHERE status = 'ok' AND ts_code = ANY(%s) AND ran_at <= %s "
|
||||
"ORDER BY ts_code, ran_at DESC, period DESC, review_version DESC", (dots, cutoff))
|
||||
except Exception as e: # noqa: BLE001
|
||||
|
|
@ -1216,14 +1209,14 @@ def company_reviews(codes, ds: str, *, read_pg=None, cutoff_at=None, strict=Fals
|
|||
except ValueError:
|
||||
grades = {}
|
||||
ran = r.get("ran_at")
|
||||
edge = dt.datetime.fromisoformat(str(cutoff).replace('Z', '+00:00'))
|
||||
if edge.tzinfo is None:
|
||||
edge = edge.replace(tzinfo=dt.timezone.utc)
|
||||
# 桩与某些代理不保证过滤;本地再守一遍时间边界,绝不把未来行归零当新鲜。
|
||||
if ran is not None:
|
||||
stamp = dt.datetime.fromisoformat(str(ran).replace('Z', '+00:00'))
|
||||
if stamp.tzinfo is None:
|
||||
stamp = stamp.replace(tzinfo=dt.timezone.utc)
|
||||
edge = dt.datetime.fromisoformat(str(cutoff).replace('Z', '+00:00'))
|
||||
if edge.tzinfo is None:
|
||||
edge = edge.replace(tzinfo=dt.timezone.utc)
|
||||
if stamp > edge:
|
||||
continue
|
||||
try:
|
||||
|
|
@ -1250,15 +1243,6 @@ def company_reviews(codes, ds: str, *, read_pg=None, cutoff_at=None, strict=Fals
|
|||
'unknown_n', 'numbers_unmatched', 'review_ref'):
|
||||
out[k][field] = sm.get(field)
|
||||
out[k]['report_ran_at'] = stamp.isoformat() if ran is not None else None
|
||||
if include_check:
|
||||
checked = r.get('checked_at')
|
||||
try:
|
||||
checked = dt.datetime.fromisoformat(str(checked).replace('Z', '+00:00'))
|
||||
if checked.tzinfo is None:
|
||||
checked = checked.replace(tzinfo=dt.timezone.utc)
|
||||
out[k]['checked_at'] = checked.isoformat() if checked <= edge else None
|
||||
except (TypeError, ValueError):
|
||||
out[k]['checked_at'] = None
|
||||
out[k]['report_version_url'] = (config.AKG_API_BASE + sm['report_version_path']
|
||||
if sm.get('report_version_path') else None)
|
||||
return out
|
||||
|
|
|
|||
|
|
@ -1,121 +0,0 @@
|
|||
"""离线:吸筹只作参考及同输入规则修订,全部来源使用内存替身。"""
|
||||
import copy
|
||||
import datetime as dt
|
||||
import tempfile
|
||||
import unittest
|
||||
from contextlib import ExitStack
|
||||
from unittest.mock import patch
|
||||
|
||||
import pandas as pd
|
||||
|
||||
import test_api_support
|
||||
import card
|
||||
import config
|
||||
import plan
|
||||
import plan_refresh
|
||||
import plan_versions as pv
|
||||
|
||||
|
||||
class ReferenceTests(unittest.TestCase):
|
||||
def test_published_rule_description_matches_confirmation_mode(self):
|
||||
legacy = ("候选=环节被指向∧当日涨幅达标∧券商覆盖且非ST∧明确吸筹∧无硬风险;"
|
||||
"关注=三门槛全过∧无硬风险∧确认线缺失或陈旧(系统无法判断,交人裁决);"
|
||||
"其余仅展示(只差覆盖、潜在吸筹等非明确状态都不升格,台账 013);"
|
||||
"因果论断只展示不进判决")
|
||||
with ExitStack() as stack:
|
||||
for name, value in (('_factor', pd.Series([240.], index=['SH600001'])),
|
||||
('_latest_date', None), ('_prev_date', None),
|
||||
('_names', {}), ('_evidence', ({}, set())),
|
||||
('_assemble_cards', ({}, []))):
|
||||
stack.enter_context(patch.object(plan, name, return_value=value))
|
||||
stack.enter_context(patch.object(plan.plan_inputs, 'factor_read', return_value=set()))
|
||||
stack.enter_context(patch.object(plan.version, 'git_short_rev', return_value='test'))
|
||||
for enabled in (True, False, True):
|
||||
with self.subTest(confirm_required=enabled), patch.object(config, 'CARD_ACCUM_CONFIRM_REQUIRED', enabled):
|
||||
params = plan.collect('2026-11-03')['card_params']
|
||||
self.assertEqual(params['accum_confirm_required'], enabled)
|
||||
if enabled:
|
||||
self.assertEqual(params['admission_rule'], 'accum-confirm-v1')
|
||||
self.assertEqual(params['rules'], legacy)
|
||||
else:
|
||||
self.assertEqual(params['admission_rule'], 'accum-reference-v2-20261104')
|
||||
for phrase in ('质地排序不变', '吸筹仅作参考', '非明确、潜在、缺评分或陈旧本身不否决',
|
||||
'现行选股资格', '高位派发及其他真实风险仍拦截',
|
||||
'基本面、技术面和择时三源合议', '现行自决档'):
|
||||
self.assertIn(phrase, params['rules'])
|
||||
self.assertNotIn('∧明确吸筹', params['rules'])
|
||||
self.assertNotIn('交人裁决', params['rules'])
|
||||
|
||||
def test_nonconfirmed_evidence_no_longer_blocks_qualified_stock(self):
|
||||
for state, age in [('潜在吸筹·低位企稳', 0), ('无吸筹迹象', 0), ('量能温和、信号不明', 1),
|
||||
('', None), ('明确吸筹', 45), ('明确吸筹', None)]:
|
||||
ev = dict(pointed=True, covered=True, upside=.2, pct0=3,
|
||||
accum_state=state, accum_age=age, accum_score=60)
|
||||
old = card.judge(ev)
|
||||
new = card.judge(ev, require_accum_confirm=False)
|
||||
self.assertNotEqual(old['verdict'], card.VERDICT_CANDIDATE)
|
||||
self.assertEqual(new['verdict'], card.VERDICT_CANDIDATE)
|
||||
self.assertFalse(new['confirm_required'])
|
||||
self.assertNotIn('confirm', new['failed_gates'])
|
||||
self.assertNotIn('没到明确吸筹', str(new))
|
||||
self.assertIn('基本面、技术面和择时', new['basis'])
|
||||
|
||||
def test_risks_and_qualification_remain_effective(self):
|
||||
base = dict(pointed=True, covered=True, upside=.2, pct0=3, accum_state='潜在吸筹', accum_age=0)
|
||||
for change in ({'pointed': False}, {'covered': False}, {'upside': -.2}, {'risk_name': True},
|
||||
{'stale_snapshot': True}, {'y_signal': 'SELL'}, {'y_signal': 'AVOID'},
|
||||
{'y_signal': 'DROPPED'}, {'accum_state': '高位派发'}):
|
||||
self.assertEqual(card.judge(dict(base, **change), require_accum_confirm=False)['verdict'],
|
||||
card.VERDICT_SHOW)
|
||||
poor = card.judge(base, require_accum_confirm=False,
|
||||
company_review={'overall': '差', 'age_days': 1})
|
||||
self.assertEqual(poor['verdict'], card.VERDICT_WATCH)
|
||||
|
||||
def test_policy_change_publishes_new_version_without_changing_history(self):
|
||||
with tempfile.TemporaryDirectory() as root:
|
||||
now = dt.datetime.fromisoformat('2026-11-04T10:30:00+08:00')
|
||||
old = {'date': '2026-11-03', 'plan_version': 'test',
|
||||
'main': [{'code': 'SH600001', 'verdict': '仅展示'}], 'observe': [],
|
||||
'shown_params': {'top': 20, 'obs_top': 10, 'theme_cap': 5}}
|
||||
inputs = {'records': {}, 'user_candidates': [], 'market': {}, 'regime': None}
|
||||
first = pv.publish(root, old, inputs, {}, cutoff_at='2026-11-04T07:10:00+08:00', reason='晨版')
|
||||
original = copy.deepcopy(first)
|
||||
def collect(*a, **kw):
|
||||
self.assertFalse(plan.config.CARD_ACCUM_CONFIRM_REQUIRED)
|
||||
data = copy.deepcopy(old)
|
||||
data['_full'] = {'main': [dict(old['main'][0], verdict='候选')], 'observe': []}
|
||||
return data
|
||||
with patch.object(config, 'PLAN_SNAPSHOT_DIR', root), \
|
||||
patch.object(config, 'CARD_ACCUM_CONFIRM_REQUIRED', False), \
|
||||
patch.object(plan_refresh, 'in_window', return_value=True), \
|
||||
patch.object(plan, 'collect', side_effect=collect), \
|
||||
patch.object(plan, 'write_published_md'):
|
||||
result = plan_refresh.refresh(now=now)
|
||||
self.assertEqual(result['status'], 'published')
|
||||
newest = pv.load(root)
|
||||
self.assertEqual(newest['decision']['reason'], '吸筹改为三源合议参考')
|
||||
self.assertEqual(newest['decision']['input_hash'], first['decision']['input_hash'])
|
||||
self.assertNotEqual(newest['decision']['params_hash'], first['decision']['params_hash'])
|
||||
self.assertEqual(newest['decision']['parent_id'], first['decision']['id'])
|
||||
self.assertEqual(pv.load(root, decision_id=first['decision']['id']), original)
|
||||
self.assertEqual(pv.load(root, morning=True), original)
|
||||
self.assertEqual(plan_refresh.refresh(now=now)['status'], 'unchanged')
|
||||
|
||||
def test_rebuild_failure_keeps_old_plan_and_pending_codes(self):
|
||||
with tempfile.TemporaryDirectory() as root:
|
||||
old = {'date': '2026-11-03', 'plan_version': 'test',
|
||||
'main': [{'code': 'SH600001', 'verdict': '仅展示'}], 'observe': []}
|
||||
inputs = {'records': {}, 'user_candidates': [], 'market': {}, 'regime': None}
|
||||
first = pv.publish(root, old, inputs, {}, cutoff_at='2026-11-04T07:10:00+08:00', reason='晨版')
|
||||
with patch.object(config, 'PLAN_SNAPSHOT_DIR', root), \
|
||||
patch.object(config, 'CARD_ACCUM_CONFIRM_REQUIRED', False), \
|
||||
patch.object(plan_refresh, 'in_window', return_value=True), \
|
||||
patch.object(plan, 'collect', side_effect=ValueError('材料损坏')):
|
||||
with self.assertRaises(ValueError):
|
||||
plan_refresh.refresh(now=dt.datetime.fromisoformat('2026-11-04T10:30:00+08:00'))
|
||||
self.assertEqual(pv.load(root), first)
|
||||
self.assertEqual(pv.health(root, old['date'])['pending_codes'], ['SH600001'])
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
|
|
@ -20,8 +20,7 @@ except ImportError:
|
|||
fa.FastAPI = fa.APIRouter = Router
|
||||
fa.Request = type('Request', (), {})
|
||||
fa.HTTPException = HTTPException
|
||||
fa.Body = fa.Header = fa.Query = lambda default=None, **kw: default
|
||||
fa.BackgroundTasks = type('BackgroundTasks', (), {'add_task': lambda self, *a, **kw: None})
|
||||
fa.Header = fa.Query = lambda default=None, **kw: default
|
||||
sys.modules['fastapi'] = fa
|
||||
resp = types.ModuleType('fastapi.responses')
|
||||
resp.PlainTextResponse = lambda content, **kw: content
|
||||
|
|
|
|||
|
|
@ -1,8 +1,6 @@
|
|||
"""本地离线:python -m unittest test_build_readiness;来源与写入全部替身。"""
|
||||
import datetime as dt
|
||||
import socket
|
||||
import re
|
||||
import sqlite3
|
||||
import sys
|
||||
import types
|
||||
import unittest
|
||||
|
|
@ -48,7 +46,7 @@ class ReadinessTest(unittest.TestCase):
|
|||
readiness.check(NOW)
|
||||
|
||||
def test_missing_heat_fails_even_if_scan_current(self):
|
||||
with patch.object(readiness.db, 'read_pg', return_value=pd.DataFrame({'d': ['2026-10-16']})), patch.object(readiness.db, 'read_mysql', side_effect=lambda source, sql, params=None: pd.DataFrame({'d': ['2026-10-15' if source == 'heat' else '2026-10-16']})):
|
||||
with patch.object(readiness.db, 'read_pg', return_value=pd.DataFrame({'d': ['2026-10-16']})), patch.object(readiness.db, 'read_mysql', side_effect=lambda source, sql: pd.DataFrame({'d': ['2026-10-15' if source == 'heat' else '2026-10-16']})):
|
||||
with self.assertRaisesRegex(RuntimeError, '资金热度日期 2026-10-15'):
|
||||
readiness.check(NOW)
|
||||
|
||||
|
|
@ -58,55 +56,6 @@ class ReadinessTest(unittest.TestCase):
|
|||
readiness.check(NOW)
|
||||
self.assertNotIn('secret-connection', str(result.exception))
|
||||
|
||||
def test_readiness_aggregates_only_shanghai_shenzhen_sources(self):
|
||||
# SQLite 只替换方言运算符,实际执行三路 MAX 查询,混入域外较新行不能造齐备。
|
||||
conn = sqlite3.connect(':memory:')
|
||||
self.addCleanup(conn.close)
|
||||
conn.create_function('regexp', 2, lambda pattern, value: bool(re.search(pattern, str(value or ''))))
|
||||
conn.executescript('''
|
||||
CREATE TABLE v_factor_transmission(ts_code TEXT, scan_date TEXT);
|
||||
CREATE TABLE gp_day_data(symbol TEXT, timestamp TEXT);
|
||||
CREATE TABLE stock_fund_heat_scores(stock_code TEXT, trade_date TEXT);
|
||||
CREATE INDEX transmission_day ON v_factor_transmission(scan_date);
|
||||
CREATE INDEX price_day ON gp_day_data(timestamp);
|
||||
CREATE INDEX heat_day ON stock_fund_heat_scores(trade_date);
|
||||
INSERT INTO v_factor_transmission VALUES ('600001.SH','2026-10-15'),('830001.BJ','2026-10-16');
|
||||
INSERT INTO gp_day_data VALUES ('600001','2026-10-15'),('830001','2026-10-16');
|
||||
INSERT INTO stock_fund_heat_scores VALUES ('SH600001','2026-10-15'),('BJ830001','2026-10-16');
|
||||
''')
|
||||
bounds = []
|
||||
def query(sql, params=None):
|
||||
translated = sql.replace(' ~ ', ' REGEXP ').replace('%s', '?')
|
||||
if 'MAX(' in sql:
|
||||
self.assertEqual(params, ('2026-10-16',))
|
||||
self.assertIn('>= %s AND', sql)
|
||||
plan = ' '.join(str(row[-1]) for row in conn.execute('EXPLAIN QUERY PLAN ' + translated, params))
|
||||
self.assertIn('SEARCH', plan)
|
||||
self.assertIn('INDEX', plan)
|
||||
bounds.append(params)
|
||||
cursor = conn.execute(translated, params or ())
|
||||
return pd.DataFrame(cursor.fetchall(), columns=[d[0] for d in cursor.description])
|
||||
with patch.object(readiness.db, 'read_pg', side_effect=query), patch.object(readiness.db, 'read_mysql', side_effect=lambda source, sql, params=None: query(sql, params)):
|
||||
with self.assertRaisesRegex(RuntimeError, '传导日期 缺失.*收盘行情日期 缺失.*资金热度日期 缺失'):
|
||||
readiness.check(NOW)
|
||||
conn.executescript('''
|
||||
UPDATE v_factor_transmission SET scan_date='2026-10-16' WHERE ts_code='600001.SH';
|
||||
UPDATE gp_day_data SET timestamp='2026-10-16' WHERE symbol='600001';
|
||||
UPDATE stock_fund_heat_scores SET trade_date='2026-10-16' WHERE stock_code='SH600001';
|
||||
UPDATE v_factor_transmission SET scan_date='2026-10-19' WHERE ts_code='830001.BJ';
|
||||
UPDATE gp_day_data SET timestamp='2026-10-19' WHERE symbol='830001';
|
||||
UPDATE stock_fund_heat_scores SET trade_date='2026-10-19' WHERE stock_code='BJ830001';
|
||||
''')
|
||||
self.assertEqual(readiness.check(NOW), '2026-10-16')
|
||||
conn.executescript('''
|
||||
UPDATE v_factor_transmission SET scan_date='2026-10-19' WHERE ts_code='600001.SH';
|
||||
UPDATE gp_day_data SET timestamp='2026-10-19' WHERE symbol='600001';
|
||||
UPDATE stock_fund_heat_scores SET trade_date='2026-10-19' WHERE stock_code='SH600001';
|
||||
''')
|
||||
with self.assertRaisesRegex(RuntimeError, '传导日期 2026-10-19.*收盘行情日期 2026-10-19.*资金热度日期 2026-10-19'):
|
||||
readiness.check(NOW)
|
||||
self.assertEqual(len(bounds), 9)
|
||||
|
||||
def test_stale_default_stops_before_any_writes(self):
|
||||
with patch.object(readiness, 'check', side_effect=RuntimeError('上游停更')), patch.object(run.common, 'write_factor') as write:
|
||||
with self.assertRaisesRegex(RuntimeError, '上游停更'):
|
||||
|
|
@ -134,34 +83,13 @@ class ReadinessTest(unittest.TestCase):
|
|||
with self.assertRaisesRegex(RuntimeError, '输入冻结失败'):
|
||||
run.cmd_build('test', 'daily', None, None, '2026-10-16')
|
||||
|
||||
def test_freeze_manifest_errors_are_not_success(self):
|
||||
frozen=types.ModuleType('freeze')
|
||||
rows = {name: 0 for name in ('universe', 'transmission', 'consensus', 'price_close', 'heat', 'factor_test')}
|
||||
for manifest in ({'rows': rows, 'errors': {'heat': 'source failed'}},
|
||||
{'rows': {'heat': 1}, 'errors': {}}, None,
|
||||
{'rows': dict(rows, heat=None), 'errors': {}}):
|
||||
frozen.snapshot=lambda *a, **kw: manifest
|
||||
with self.subTest(manifest=manifest), patch.dict(sys.modules, {'freeze': frozen}), patch.object(run.factors, 'BUILDERS', {'test': lambda a,b: pd.DataFrame()}), patch.object(run.factors, 'FACTORS', {'test': 't'}), patch.object(run.common, 'write_factor'):
|
||||
with self.assertRaisesRegex(RuntimeError, '输入冻结失败'):
|
||||
run.cmd_build('test', 'daily', None, None, '2026-10-16')
|
||||
|
||||
def test_complete_freeze_allows_zero_rows_and_optional_empty_inputs(self):
|
||||
frozen=types.ModuleType('freeze')
|
||||
rows = {name: 0 for name in ('universe', 'transmission', 'consensus', 'price_close', 'heat', 'factor_test', 'segment_members')}
|
||||
frozen.snapshot=lambda *a, **kw: {'rows': rows, 'errors': {}}
|
||||
with patch.dict(sys.modules, {'freeze': frozen}), patch.object(run.factors, 'BUILDERS', {'test': lambda a,b: pd.DataFrame()}), patch.object(run.factors, 'FACTORS', {'test': 't'}), patch.object(run.common, 'write_factor'):
|
||||
self.assertEqual(run.cmd_build('test', 'daily', None, None, '2026-10-16'), '2026-10-16')
|
||||
|
||||
def test_refresh_missing_morning_fails_cli(self):
|
||||
mod=types.ModuleType('plan_refresh')
|
||||
recovery=types.ModuleType('plan_recovery')
|
||||
recovery.ensure_today=lambda *a, **kw: {'status': 'waiting_data', 'reason': '输入未齐'}
|
||||
research=types.ModuleType('user_research'); research.sync=lambda: None
|
||||
for status in ('no_morning', 'no_morning_today', 'outside_window', 'unchanged'):
|
||||
mod.refresh=lambda *a, **kw: {'status': status}
|
||||
with self.subTest(status=status), patch.dict(sys.modules, {'plan_refresh': mod, 'plan_recovery': recovery, 'user_research': research}), patch.object(sys, 'argv', ['run.py','plan-refresh']):
|
||||
with self.subTest(status=status), patch.dict(sys.modules, {'plan_refresh': mod}), patch.object(sys, 'argv', ['run.py','plan-refresh']):
|
||||
if status.startswith('no_morning'):
|
||||
with self.assertRaisesRegex(SystemExit, '今日完整基础版尚未恢复.*waiting_data'): run.main()
|
||||
with self.assertRaisesRegex(SystemExit, '今日完整晨版缺失'): run.main()
|
||||
else: run.main()
|
||||
|
||||
if __name__ == '__main__':
|
||||
|
|
|
|||
|
|
@ -11,7 +11,7 @@ import test_api_support # noqa: F401
|
|||
def main():
|
||||
suite = unittest.TestSuite()
|
||||
for name in ('test_plan_versions', 'test_plan_reviews_live', 'test_plan_review_source', 'test_plan_stock_check',
|
||||
'test_plan_rank_switch', 'test_user_candidates', 'test_user_research'):
|
||||
'test_plan_rank_switch'):
|
||||
suite.addTests(unittest.defaultTestLoader.loadTestsFromModule(importlib.import_module(name)))
|
||||
for name in ('test_plan_cache', 'test_plan_snapshot', 'test_plan_rank_axis',
|
||||
'test_plan_quality_rank', 'test_plan_verdict', 'test_plan_logic_state',
|
||||
|
|
|
|||
|
|
@ -188,7 +188,7 @@ class PublishTest(unittest.TestCase):
|
|||
class IntegrationContractTest(unittest.TestCase):
|
||||
def test_business_defaults_unchanged_and_explicit_entry(self):
|
||||
import ast
|
||||
tree = ast.parse(Path(__file__).with_name('xxl.py').read_text())
|
||||
tree = ast.parse(Path('xxl.py').read_text())
|
||||
values = {}
|
||||
for node in tree.body:
|
||||
if isinstance(node, ast.Assign):
|
||||
|
|
@ -201,7 +201,7 @@ class IntegrationContractTest(unittest.TestCase):
|
|||
|
||||
def test_read_query_cannot_collect_or_publish(self):
|
||||
import ast
|
||||
tree = ast.parse(Path(__file__).with_name('api.py').read_text())
|
||||
tree = ast.parse(Path('api.py').read_text())
|
||||
fn = next(x for x in tree.body if isinstance(x, ast.FunctionDef) and x.name == 'dividend_research')
|
||||
calls = [n.func.attr for n in ast.walk(fn) if isinstance(n, ast.Call) and isinstance(n.func, ast.Attribute)]
|
||||
self.assertNotIn('collect', calls); self.assertNotIn('publish', calls)
|
||||
|
|
|
|||
|
|
@ -74,8 +74,8 @@ def _pg_ok(sql, params=None):
|
|||
s = " ".join(sql.split())
|
||||
if "v_factor_stock_daily" in s:
|
||||
assert params == (DS,)
|
||||
return [{"ts_code": '60000' + str(i) + '.SH', "pct_change": value}
|
||||
for i, value in enumerate((9.95, 3.0, 0.0, -1.5, None, 10.02, -4.0))]
|
||||
return [{"pct_change": 9.95}, {"pct_change": 3.0}, {"pct_change": 0.0}, {"pct_change": -1.5},
|
||||
{"pct_change": None}, {"pct_change": 10.02}, {"pct_change": -4.0}]
|
||||
if "v_factor_logic" in s:
|
||||
assert params[-1] == DS and "600000.SH" in params and "SZ000001" not in params
|
||||
return [
|
||||
|
|
|
|||
|
|
@ -1,190 +0,0 @@
|
|||
"""沪深输入与冻结边界:全部虚构数据,离线读写临时文件与 SQLite。"""
|
||||
import contextlib
|
||||
import json
|
||||
import re
|
||||
import socket
|
||||
import sqlite3
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
import pandas as pd
|
||||
import common
|
||||
import factors
|
||||
import freeze
|
||||
import market_scope as scope
|
||||
|
||||
DAY = '2026-11-04'
|
||||
BAD = ['430001', '830001.BJ', 'BJ920001', '920001', '600001.SZ', 'SH000001', '510001.SH', '399001.SZ', '600001x', None]
|
||||
|
||||
|
||||
def frame(codes, values=None):
|
||||
return pd.DataFrame({'trade_date':[DAY]*len(codes), 'stock_code':codes,
|
||||
'factor_value':values or list(range(len(codes)))})
|
||||
|
||||
|
||||
class MarketScopeTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
p=patch.object(socket.socket, 'connect', side_effect=AssertionError('禁止联网'));p.start();self.addCleanup(p.stop)
|
||||
for name in ('read_pg', 'read_mysql', 'factor_conn'):
|
||||
p=patch.object(common.db, name, side_effect=AssertionError('必须替换数据库依赖'));p.start();self.addCleanup(p.stop)
|
||||
|
||||
def test_normalizes_prefix_suffix_and_bare_code(self):
|
||||
for variants, expected in [(['SH600001','600001.SH','600001',' sh600001 '],'600001.SH'),
|
||||
(['SZ000001','000001.SZ','000001'],'000001.SZ'),
|
||||
(['688001','SH688001'],'688001.SH'),
|
||||
(['301001','301001.SZ'],'301001.SZ')]:
|
||||
for value in variants:
|
||||
self.assertEqual(scope.normalize(value), expected)
|
||||
self.assertEqual(scope.normalize(value,prefix=True), expected[-2:]+expected[:6])
|
||||
for value in BAD: self.assertIsNone(scope.normalize(value), value)
|
||||
|
||||
def test_sql_and_python_accept_identical_scope(self):
|
||||
sql = scope.sql_filter('stock_code')
|
||||
pattern=sql.split("REGEXP '",1)[1][:-1]
|
||||
for value in ['SH600001','600001.SH','600001','000001','SZ301001','688001.SH']+BAD:
|
||||
self.assertEqual(bool(re.fullmatch(pattern,str(value).strip().upper())), scope.normalize(value) is not None, value)
|
||||
self.assertIn(' ~ ',scope.sql_filter('ts_code',postgres=True))
|
||||
with self.assertRaises(ValueError): scope.sql_filter('stock_code;DROP TABLE x')
|
||||
|
||||
def test_market_mode_never_includes_out_of_scope_or_mutates_source(self):
|
||||
original=frame(['600001','SZ000001']+BAD); before=original.copy(deep=True)
|
||||
with patch.object(common.config,'SUBFACTOR_UNIVERSE','market'):
|
||||
result=common.universe_filter(original)
|
||||
self.assertEqual(list(result.stock_code),['SH600001','SZ000001'])
|
||||
pd.testing.assert_frame_equal(before,original)
|
||||
|
||||
def test_pool_mode_normalizes_both_sides_before_membership(self):
|
||||
with patch.object(common.config,'SUBFACTOR_UNIVERSE','pool'), patch.object(common.db,'read_pg',return_value=pd.DataFrame({'ts_code':['SH600001','000001','830001.BJ']})) as read:
|
||||
result=common.universe_filter(frame(['600001.SH','SZ000001','300001','BJ830001']))
|
||||
self.assertEqual(list(result.stock_code),['SH600001','SZ000001'])
|
||||
self.assertIn('WHERE',read.call_args[0][0]);self.assertIn(' ~ ',read.call_args[0][0])
|
||||
|
||||
def test_column_probe_does_not_read_arbitrary_security(self):
|
||||
with patch.object(common.db,'read_mysql',return_value=pd.DataFrame()) as read:
|
||||
self.assertEqual(factors._price_code_col(),common.config.PRICE_CODE_COL)
|
||||
self.assertTrue(read.call_args[0][1].endswith('LIMIT 0'))
|
||||
|
||||
def test_heat_latest_batch_read_and_output_are_scoped(self):
|
||||
calls=[]
|
||||
def read(which,sql,params):
|
||||
calls.append(sql)
|
||||
self.assertIn('REGEXP',sql);self.assertNotIn('JOIN',sql)
|
||||
if 'MAX(batch_no)' in sql: return pd.DataFrame({'trade_date':[DAY],'batch_no':[2]})
|
||||
return pd.DataFrame({'trade_date':[DAY]*3,'stock_code':['600001.SH','SZ000001','BJ830001'],'batch_no':[2]*3,'score':[.1,.2,999]})
|
||||
with patch.object(common.db,'read_mysql',side_effect=read),patch.object(common.config,'SUBFACTOR_UNIVERSE','market'):
|
||||
result=factors.build_heat(DAY,DAY)
|
||||
self.assertEqual(list(result.stock_code),['SH600001','SZ000001']);self.assertEqual(len(calls),2)
|
||||
|
||||
def test_heat_empty_qualified_batch_stays_empty(self):
|
||||
with patch.object(common.db,'read_mysql',return_value=pd.DataFrame(columns=['trade_date','batch_no'])) as read:
|
||||
self.assertTrue(factors.build_heat(DAY,DAY).empty)
|
||||
self.assertEqual(read.call_count,1)
|
||||
|
||||
def test_price_read_filters_and_normalizes_before_calculation(self):
|
||||
rows=pd.DataFrame({'trade_date':[DAY]*3,'ts_code':['SH600001','000001','830001.BJ'],'close':[10,20,999]})
|
||||
with patch.object(factors,'_price_code_col',return_value='symbol'),patch.object(common.db,'read_mysql',return_value=rows) as read:
|
||||
result=factors._read_gp_price(DAY,DAY)
|
||||
self.assertEqual(list(result.ts_code),['600001.SH','000001.SZ']);self.assertIn('REGEXP',read.call_args[0][1])
|
||||
|
||||
def test_upside_only_uses_scoped_consensus_and_prices(self):
|
||||
cons=pd.DataFrame({'ts_code':['SH600001','BJ830001'],'asof_date':[DAY]*2,'target_mid_avg':[12,1000]})
|
||||
prices=pd.DataFrame({'ts_code':['600001.SH'],'trade_date':[DAY],'close':[10]})
|
||||
with patch.object(common.db,'read_pg',return_value=cons) as read, patch.object(factors,'_read_gp_price',return_value=prices),patch.object(common.config,'SUBFACTOR_UNIVERSE','market'):
|
||||
result=factors.build_upside(DAY,DAY)
|
||||
self.assertEqual(list(result.stock_code),['SH600001']);self.assertAlmostEqual(result.iloc[0].factor_value,.2)
|
||||
self.assertIn(' ~ ',read.call_args[0][0])
|
||||
|
||||
def test_event_filter_precedes_single_document_cap(self):
|
||||
ev=pd.DataFrame({'ts_code':['BJ830001','SH600001'],'disclosure_date':[DAY]*2,'event_type':['股份回购']*2,'direction':['']*2,'confidence':[1,.8],'doc_id':[1,1],'source_type':['announcement']*2})
|
||||
with patch.object(common.db,'read_pg',return_value=ev) as read,patch.object(common.config,'SUBFACTOR_UNIVERSE','market'),patch.object(common.config,'EVENT_SOURCE_TYPES',{'announcement'}),patch.object(common.config,'EVENT_MAX_PER_DOC',1),patch.object(common,'trading_days',return_value=[pd.Timestamp(DAY)]):
|
||||
result=factors.build_event(DAY,DAY)
|
||||
self.assertEqual(list(result.stock_code),['600001.SH']);self.assertIn(' ~ ',read.call_args[0][0])
|
||||
|
||||
def test_transmission_filters_before_max_and_warning(self):
|
||||
tr=pd.DataFrame({'scan_date':[DAY]*3,'ts_code':['SH600001','600001.SH','BJ830001'],'n_sources':[1,2,999],'moved_ratio':[0,0,0]})
|
||||
with patch.object(common.db,'read_pg',return_value=tr) as read,patch.object(common,'load_universe',return_value={'600001.SH','830001.BJ'}),patch.object(factors,'_warn_upstream_truncation') as warn:
|
||||
result=factors.build_transmission(DAY,DAY)
|
||||
self.assertEqual(list(result.stock_code),['600001.SH']);self.assertEqual(result.iloc[0].factor_value,2)
|
||||
self.assertEqual(set(warn.call_args[0][0].ts_code),{'600001.SH'});self.assertIn(' ~ ',read.call_args[0][0])
|
||||
|
||||
def test_external_values_do_not_change_score_distribution(self):
|
||||
def score(add_bad):
|
||||
codes=['SH600001','SZ000001','SH600002']+(['BJ830001'] if add_bad else [])
|
||||
upside=frame(codes,[.1,.3,.8]+([1e12] if add_bad else []))
|
||||
heat=frame(codes,[.2,.4,.9]+([-1e12] if add_bad else []))
|
||||
transmission=frame(codes,[1,2,4]+([1e12] if add_bad else []))
|
||||
with patch.object(factors,'_days_of',return_value=[DAY]),patch.object(factors,'_track_set',return_value=None),patch.object(factors,'_risk_set',return_value=set()),patch.object(factors,'build_upside',return_value=upside),patch.object(factors,'_heat_day_for',return_value=DAY),patch.object(factors,'build_heat',return_value=heat),patch.object(factors,'build_transmission',return_value=transmission):
|
||||
return factors.build_score(DAY,DAY)
|
||||
pd.testing.assert_frame_equal(score(False),score(True))
|
||||
|
||||
def test_risk_members_exclude_bj_before_name_rule(self):
|
||||
pools=pd.DataFrame({'members':[json.dumps([{'ts_code':'BJ830001','name':'ST范围外'},{'ts_code':'600001','name':'ST有效'}])]})
|
||||
with patch.object(common.db,'read_pg',return_value=pools),patch.object(common.config,'EXCLUDE_RISK_NAMES',True):
|
||||
self.assertEqual(factors._risk_set(),{'SH600001'})
|
||||
|
||||
def test_calendar_date_query_excludes_out_of_scope(self):
|
||||
with patch.object(factors,'_price_code_col',return_value='symbol'),patch.object(common.db,'read_mysql',return_value=pd.DataFrame({'trade_date':[DAY]})) as read:
|
||||
self.assertEqual(len(common.trading_days(DAY,DAY)),1)
|
||||
self.assertIn('REGEXP',read.call_args[0][1])
|
||||
|
||||
def test_write_factor_preserves_historical_bj_and_other_dates(self):
|
||||
connection=sqlite3.connect(':memory:')
|
||||
self.addCleanup(connection.close)
|
||||
connection.create_function('REGEXP',2,lambda regex,value:int(bool(re.fullmatch(regex,str(value)))))
|
||||
connection.execute('CREATE TABLE t_factor_akg_score(trade_date TEXT,stock_code TEXT,factor_value REAL)')
|
||||
baseline=[(DAY,'BJ830001',77.),(DAY,'830002.BJ',78.),(DAY,'SH600001',2.),('2026-11-03','SH600001',3.)]
|
||||
connection.executemany('INSERT INTO t_factor_akg_score VALUES(?,?,?)',baseline);connection.commit()
|
||||
class Cursor:
|
||||
def __enter__(self):return self
|
||||
def __exit__(self,*a):pass
|
||||
def execute(self,sql,params):return connection.execute(sql.replace('%s','?'),params)
|
||||
def executemany(self,sql,rows):return connection.executemany(sql.replace('%s','?'),rows)
|
||||
class Conn:
|
||||
def cursor(self):return Cursor()
|
||||
def commit(self):connection.commit()
|
||||
@contextlib.contextmanager
|
||||
def conn():yield Conn()
|
||||
with patch.object(common.db,'factor_conn',conn),patch.object(common,'ensure_table'):
|
||||
common.write_factor('t_factor_akg_score',frame(['600001','BJ830001','830002.BJ'],[10,1000,1001]))
|
||||
rows=connection.execute('SELECT * FROM t_factor_akg_score ORDER BY trade_date,stock_code').fetchall()
|
||||
self.assertEqual(rows,sorted([baseline[0],baseline[1],baseline[3],(DAY,'SH600001',10.)]))
|
||||
|
||||
def test_out_of_scope_only_write_never_deletes_or_creates_table(self):
|
||||
with patch.object(common,'ensure_table') as ensure:
|
||||
common.write_factor('t_factor_akg_score',frame(BAD))
|
||||
ensure.assert_not_called()
|
||||
|
||||
def test_freeze_nested_members_and_extra_frames_have_no_out_of_scope(self):
|
||||
members=json.dumps([{'ts_code':'SH600001','name':'有效'},{'ts_code':'830001.BJ','name':'范围外'}, {'ts_code':'600001.SZ'}])
|
||||
source=pd.DataFrame({'ts_code':['600001.SH','830001.BJ'],'members':[members,members],'details':[{'quiet':['600001','BJ830001']},{}]})
|
||||
with tempfile.TemporaryDirectory() as root,patch.object(freeze.config,'FROZEN_ROOT',root),patch.object(freeze,'_FMT','csv.gz'),patch.object(freeze,'_EXT','.csv.gz'),patch.object(freeze,'_git_rev',return_value='test'),patch.dict(freeze._SOURCES,{'universe':lambda day:source},clear=True):
|
||||
manifest=freeze.snapshot(DAY,extra_frames={'factor_akg_score':frame(['600001','BJ830001'])})
|
||||
result=pd.read_csv(Path(root)/DAY/'universe.csv.gz')
|
||||
factor=pd.read_csv(Path(root)/DAY/'factor_akg_score.csv.gz')
|
||||
self.assertEqual(manifest['rows'],{'universe':1,'factor_akg_score':1})
|
||||
self.assertNotIn('830001',result.to_json());self.assertNotIn('830001',factor.to_json())
|
||||
self.assertEqual(json.loads(result.iloc[0].members),[{'ts_code':'600001.SH','name':'有效'}])
|
||||
self.assertEqual(manifest['errors'],{})
|
||||
|
||||
def test_industry_edges_remain_and_invalid_member_json_is_error(self):
|
||||
edge=pd.DataFrame({'src_segment':['产业甲'],'dst_segment':['产业乙'],'mechanism':['[事实] 保持']})
|
||||
pd.testing.assert_frame_equal(scope.frozen_frame(edge),edge)
|
||||
with self.assertRaises(ValueError):scope.frozen_frame(pd.DataFrame({'members':['[bad json']}))
|
||||
|
||||
def test_all_security_capture_queries_are_scoped(self):
|
||||
def pg(sql,params=None):
|
||||
self.assertIn(' ~ ',sql)
|
||||
return pd.DataFrame({'ts_code':['600001.SH','830001.BJ']})
|
||||
def mysql(which,sql,params=None):
|
||||
self.assertIn('REGEXP',sql)
|
||||
if 'MAX(batch_no)' in sql:return pd.DataFrame({'trade_date':[DAY],'batch_no':[1]})
|
||||
if which=='heat':return pd.DataFrame({'stock_code':['SH600001','BJ830001'],'trade_date':[DAY]*2,'score':[1,2],'batch_no':[1,1]})
|
||||
return pd.DataFrame({'code':['600001','830001'],'close':[1,2]})
|
||||
with patch.object(common.db,'read_pg',side_effect=pg),patch.object(common.db,'read_mysql',side_effect=mysql),patch.object(factors,'_price_code_col',return_value='symbol'):
|
||||
for fn in (freeze._cap_universe,freeze._cap_transmission,freeze._cap_consensus,freeze._cap_price,freeze._cap_heat):
|
||||
with self.subTest(fn=fn.__name__):self.assertEqual(len(fn(DAY)),1)
|
||||
|
||||
|
||||
if __name__=='__main__':unittest.main()
|
||||
|
|
@ -1,330 +0,0 @@
|
|||
"""有限恢复的隔离行为回归;所有来源/构建/模型替身,计划只写临时目录。"""
|
||||
import contextlib
|
||||
import datetime as dt
|
||||
import fcntl
|
||||
import json
|
||||
from pathlib import Path
|
||||
import socket
|
||||
import sys
|
||||
import tempfile
|
||||
import types
|
||||
import unittest
|
||||
from unittest.mock import Mock, patch
|
||||
|
||||
import config
|
||||
import plan
|
||||
import plan_recovery as recovery
|
||||
import plan_versions as pv
|
||||
import run
|
||||
|
||||
GENERATE = plan._generate
|
||||
|
||||
NOW = dt.datetime.fromisoformat('2026-11-05T10:05:00+08:00')
|
||||
DAY = '2026-11-04'
|
||||
|
||||
|
||||
class RecoveryTest(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.stack = contextlib.ExitStack()
|
||||
self.addCleanup(self.stack.close)
|
||||
self.root = Path(self.stack.enter_context(tempfile.TemporaryDirectory()))
|
||||
self.stack.enter_context(patch.object(config, 'PLAN_SNAPSHOT_DIR', str(self.root)))
|
||||
self.stack.enter_context(patch.object(config, 'PLAN_RECOVERY_MAX_ATTEMPTS', 3))
|
||||
self.stack.enter_context(patch.object(config, 'PLAN_RECOVERY_COOLDOWN_SECONDS', 900))
|
||||
self.stack.enter_context(patch.object(socket.socket, 'connect', side_effect=AssertionError('禁止联网')))
|
||||
self.ready = self.stack.enter_context(patch.object(recovery.build_readiness, 'check', return_value=DAY))
|
||||
self.build = self.stack.enter_context(patch.object(run, '_cmd_build', return_value=DAY))
|
||||
self.generate = self.stack.enter_context(patch.object(plan, '_generate', side_effect=self.publish))
|
||||
self.pool = Mock(side_effect=AssertionError('恢复不入池'))
|
||||
self.model = Mock(side_effect=AssertionError('恢复不额外发模型任务'))
|
||||
self.stack.enter_context(patch.dict(sys.modules, {
|
||||
'pool': types.SimpleNamespace(push=self.pool),
|
||||
'user_research': types.SimpleNamespace(sync=self.model),
|
||||
}))
|
||||
|
||||
def publish(self, day=DAY, *args, cutoff=NOW, **kwargs):
|
||||
if kwargs.get('before_publish'):
|
||||
kwargs['before_publish'](day)
|
||||
return pv.publish(str(self.root), {'date': day, 'main': [], 'observe': [],
|
||||
'plan_version': 'isolated'}, {'records': {}, 'market': {}}, {},
|
||||
cutoff_at=cutoff.isoformat(), reason='晨版生成')
|
||||
|
||||
def state(self, day='2026-11-05'):
|
||||
return json.loads((self.root / 'recovery' / (day + '.json')).read_text())
|
||||
|
||||
def test_missing_inputs_wait_without_build_or_budget(self):
|
||||
self.ready.side_effect = RuntimeError('资金热度日期 2026-11-03,预期 2026-11-04')
|
||||
result = recovery.ensure_today(now=NOW)
|
||||
self.assertEqual(result['status'], 'waiting_data')
|
||||
self.assertIn('2026-11-03', result['reason'])
|
||||
self.build.assert_not_called()
|
||||
self.generate.assert_not_called()
|
||||
self.assertFalse((self.root / 'recovery').exists())
|
||||
|
||||
def test_ready_builds_once_with_real_late_cutoff_and_no_pool_or_model(self):
|
||||
old = self.publish('2026-11-03', cutoff=NOW - dt.timedelta(days=1))
|
||||
old_path = self.root / 'plan_2026-11-03.json'
|
||||
previous = old_path.read_bytes()
|
||||
result = recovery.ensure_today(now=NOW)
|
||||
self.assertEqual(result['status'], 'recovered')
|
||||
self.build.assert_called_once_with('all', 'daily', None, None, None)
|
||||
today = pv.load(self.root, DAY)
|
||||
self.assertEqual(today['decision']['publication_kind'], 'recovery')
|
||||
self.assertEqual(today['decision']['cutoff_at'], NOW.isoformat())
|
||||
self.assertFalse(pv.morning_eligible(today))
|
||||
self.assertEqual(old_path.read_bytes(), previous)
|
||||
self.assertEqual(pv.load(self.root, '2026-11-03', morning=True), old)
|
||||
self.assertEqual(self.state()['attempts'], 1)
|
||||
self.assertEqual(self.state()['status'], 'recovered')
|
||||
self.pool.assert_not_called()
|
||||
self.model.assert_not_called()
|
||||
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'already_ready')
|
||||
self.build.assert_called_once()
|
||||
|
||||
def test_existing_recovery_does_not_rebuild_even_when_sources_unavailable(self):
|
||||
self.publish()
|
||||
self.ready.side_effect = AssertionError('已有有效版不重新取构建源')
|
||||
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'already_ready')
|
||||
self.build.assert_not_called()
|
||||
self.ready.assert_not_called()
|
||||
|
||||
def test_explicit_date_never_automatically_rebuilds(self):
|
||||
for day in ('2026-11-03', DAY):
|
||||
self.assertEqual(recovery.ensure_today(day, now=NOW)['status'], 'explicit_date')
|
||||
self.build.assert_not_called()
|
||||
self.ready.assert_not_called()
|
||||
|
||||
def test_outside_window_and_weekend_do_not_rebuild(self):
|
||||
for stamp in ('2026-11-05T07:19:59+08:00', '2026-11-05T15:10:01+08:00',
|
||||
'2026-11-07T10:00:00+08:00'):
|
||||
with self.subTest(stamp=stamp):
|
||||
result = recovery.ensure_today(now=dt.datetime.fromisoformat(stamp))
|
||||
self.assertEqual(result['status'], 'outside_window')
|
||||
self.build.assert_not_called()
|
||||
self.ready.assert_not_called()
|
||||
|
||||
def test_preexisting_legacy_or_wrong_decision_day_is_preserved(self):
|
||||
path = self.root / ('plan_' + DAY + '.json')
|
||||
for value in ({'date': DAY, 'main': []}, 'broken json'):
|
||||
path.write_text(json.dumps(value) if isinstance(value, dict) else value)
|
||||
before = path.read_bytes()
|
||||
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'blocked_existing_snapshot')
|
||||
self.assertEqual(path.read_bytes(), before)
|
||||
path.unlink()
|
||||
self.publish(cutoff=NOW - dt.timedelta(days=1))
|
||||
before = path.read_bytes()
|
||||
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'blocked_existing_snapshot')
|
||||
self.assertEqual(path.read_bytes(), before)
|
||||
self.build.assert_not_called()
|
||||
|
||||
def test_future_published_day_is_not_replaced_or_suppressed(self):
|
||||
self.publish('2026-11-06')
|
||||
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'blocked_existing_snapshot')
|
||||
self.build.assert_not_called()
|
||||
|
||||
def test_lock_recheck_observes_other_success(self):
|
||||
writer = pv.writer
|
||||
@contextlib.contextmanager
|
||||
def publish_before_lock(root):
|
||||
with writer(root):
|
||||
self.publish()
|
||||
yield
|
||||
with patch.object(pv, 'writer', side_effect=publish_before_lock):
|
||||
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'already_ready')
|
||||
self.build.assert_not_called()
|
||||
|
||||
def test_inputs_changed_while_waiting_lock_do_not_consume_attempt(self):
|
||||
self.ready.side_effect = [DAY, RuntimeError('热度已失效')]
|
||||
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'waiting_data')
|
||||
self.build.assert_not_called()
|
||||
self.assertFalse((self.root / 'recovery').exists())
|
||||
|
||||
def test_failed_build_persists_budget_and_obeys_cooldown_and_daily_cap(self):
|
||||
self.build.side_effect = RuntimeError('构建失败')
|
||||
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'failed')
|
||||
self.assertEqual(self.state()['status'], 'failed')
|
||||
self.assertEqual(recovery.ensure_today(now=NOW + dt.timedelta(minutes=14))['status'], 'cooldown')
|
||||
self.assertEqual(self.build.call_count, 1)
|
||||
self.assertEqual(recovery.ensure_today(now=NOW + dt.timedelta(minutes=15))['status'], 'failed')
|
||||
self.assertEqual(recovery.ensure_today(now=NOW + dt.timedelta(minutes=30))['status'], 'failed')
|
||||
self.assertEqual(recovery.ensure_today(now=NOW + dt.timedelta(minutes=45))['status'], 'attempts_exhausted')
|
||||
self.assertEqual(self.build.call_count, 3)
|
||||
self.assertEqual(self.state()['attempts'], 3)
|
||||
self.generate.assert_not_called()
|
||||
|
||||
def test_persisted_running_attempt_survives_process_restart(self):
|
||||
path = self.root / 'recovery' / '2026-11-05.json'
|
||||
pv.atomic(path, {'schema_version': 1, 'decision_date': '2026-11-05', 'data_date': DAY,
|
||||
'attempts': 2, 'status': 'building', 'last_attempt_at': NOW.isoformat()})
|
||||
self.assertEqual(recovery.ensure_today(now=NOW + dt.timedelta(minutes=1))['status'], 'cooldown')
|
||||
self.build.assert_not_called()
|
||||
|
||||
def test_day_switch_has_independent_bounded_budget(self):
|
||||
self.build.side_effect = RuntimeError('构建失败')
|
||||
for minutes in (0, 15, 30):
|
||||
recovery.ensure_today(now=NOW + dt.timedelta(minutes=minutes))
|
||||
self.ready.return_value = '2026-11-05'
|
||||
tomorrow = NOW + dt.timedelta(days=1)
|
||||
self.assertEqual(recovery.ensure_today(now=tomorrow)['status'], 'failed')
|
||||
self.assertEqual(self.state('2026-11-06')['attempts'], 1)
|
||||
self.assertEqual(self.state()['attempts'], 3)
|
||||
|
||||
def test_corrupt_attempt_state_does_not_reset_budget(self):
|
||||
path = self.root / 'recovery' / '2026-11-05.json'
|
||||
pv.atomic(path, {'attempts': 'unknown'})
|
||||
before = path.read_bytes()
|
||||
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'blocked_state')
|
||||
self.assertEqual(path.read_bytes(), before)
|
||||
self.build.assert_not_called()
|
||||
|
||||
def test_configuration_cannot_expand_safety_limits(self):
|
||||
self.build.side_effect = RuntimeError('构建失败')
|
||||
with patch.object(config, 'PLAN_RECOVERY_MAX_ATTEMPTS', 100), patch.object(config, 'PLAN_RECOVERY_COOLDOWN_SECONDS', 1):
|
||||
for minutes in (0, 15, 30):
|
||||
self.assertEqual(recovery.ensure_today(now=NOW + dt.timedelta(minutes=minutes))['status'], 'failed')
|
||||
self.assertEqual(recovery.ensure_today(now=NOW + dt.timedelta(minutes=45))['status'], 'attempts_exhausted')
|
||||
self.assertEqual(self.build.call_count, 3)
|
||||
|
||||
def test_disabled_recovery_does_not_construct(self):
|
||||
with patch.object(config, 'PLAN_RECOVERY_MAX_ATTEMPTS', 0):
|
||||
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'attempts_exhausted')
|
||||
self.build.assert_not_called()
|
||||
|
||||
def test_source_changes_after_build_do_not_publish(self):
|
||||
self.ready.side_effect = [DAY, DAY, RuntimeError('来源日期变化')]
|
||||
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'failed')
|
||||
self.build.assert_called_once()
|
||||
self.generate.assert_not_called()
|
||||
self.assertFalse((self.root / ('plan_' + DAY + '.json')).exists())
|
||||
|
||||
def test_build_finishes_after_window_does_not_publish(self):
|
||||
before = dt.datetime.fromisoformat('2026-11-05T15:09:00+08:00')
|
||||
after = dt.datetime.fromisoformat('2026-11-05T15:11:00+08:00')
|
||||
with patch.object(recovery, '_now', side_effect=[before, before, after, after]):
|
||||
self.assertEqual(recovery.ensure_today()['status'], 'failed')
|
||||
self.generate.assert_not_called()
|
||||
|
||||
def test_plan_failure_preserves_previous_snapshot(self):
|
||||
self.publish('2026-11-03', cutoff=NOW - dt.timedelta(days=1))
|
||||
path = self.root / 'plan_2026-11-03.json'
|
||||
before = path.read_bytes()
|
||||
self.generate.side_effect = SystemExit('采集失败')
|
||||
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'failed')
|
||||
self.assertEqual(path.read_bytes(), before)
|
||||
self.assertEqual(self.state()['attempts'], 1)
|
||||
|
||||
def test_published_plan_with_later_failure_is_not_rebuilt(self):
|
||||
def partly_published(*args, **kwargs):
|
||||
self.publish()
|
||||
raise OSError('Markdown 写入失败')
|
||||
self.generate.side_effect = partly_published
|
||||
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'failed')
|
||||
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'already_ready')
|
||||
self.build.assert_called_once()
|
||||
|
||||
def test_collection_finishes_after_window_does_not_publish(self):
|
||||
before = dt.datetime.fromisoformat('2026-11-05T15:09:00+08:00')
|
||||
after = dt.datetime.fromisoformat('2026-11-05T15:11:00+08:00')
|
||||
with patch.object(recovery, '_now', side_effect=[before, before, before, after, after]):
|
||||
self.assertEqual(recovery.ensure_today()['status'], 'failed')
|
||||
self.assertFalse((self.root / ('plan_' + DAY + '.json')).exists())
|
||||
self.assertEqual(self.state()['attempts'], 1)
|
||||
|
||||
def test_readiness_changes_during_collection_does_not_publish(self):
|
||||
self.ready.side_effect = [DAY, DAY, DAY, RuntimeError('发布前热度日期失效')]
|
||||
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'failed')
|
||||
self.assertFalse((self.root / ('plan_' + DAY + '.json')).exists())
|
||||
|
||||
def test_real_generate_checks_recovery_boundary_before_any_publication(self):
|
||||
import user_candidates
|
||||
guard = Mock(side_effect=RuntimeError('已过恢复时窗'))
|
||||
data = {'date': DAY, 'main': [], 'observe': [], 'plan_version': 'isolated'}
|
||||
with patch.object(user_candidates, 'active', return_value=[]), \
|
||||
patch.object(plan, 'collect', return_value=data), \
|
||||
patch.object(plan.sources, 'market_context', return_value={}), \
|
||||
patch.object(plan, 'write_published_md') as markdown, pv.writer(self.root):
|
||||
with self.assertRaisesRegex(RuntimeError, '已过恢复时窗'):
|
||||
GENERATE(DAY, 20, 10, 5, before_publish=guard)
|
||||
guard.assert_called_once_with(DAY)
|
||||
markdown.assert_not_called()
|
||||
self.assertFalse((self.root / ('plan_' + DAY + '.json')).exists())
|
||||
self.assertFalse((self.root / 'versions' / DAY).exists())
|
||||
|
||||
def test_recovery_keeps_real_cross_process_lock_through_build_and_publish(self):
|
||||
def assert_locked():
|
||||
with open(self.root / '.publish.lock', 'a') as other:
|
||||
with self.assertRaises(BlockingIOError):
|
||||
fcntl.flock(other, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||
def build(*args):
|
||||
assert_locked()
|
||||
return DAY
|
||||
def publish(*args, **kwargs):
|
||||
assert_locked()
|
||||
return self.publish(*args, **kwargs)
|
||||
self.build.side_effect = build
|
||||
self.generate.side_effect = publish
|
||||
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'recovered')
|
||||
|
||||
def test_cli_syncs_existing_workflow_once_then_refreshes_recovered_plan(self):
|
||||
refresh = types.SimpleNamespace(refresh=Mock(side_effect=[{'status': 'no_morning_today'}, {'status': 'published'}]))
|
||||
research = types.SimpleNamespace(sync=Mock())
|
||||
with patch.dict(sys.modules, {'plan_refresh': refresh, 'user_research': research}), \
|
||||
patch.object(recovery, 'ensure_today', return_value={'status': 'recovered'}) as recover, \
|
||||
patch.object(sys, 'argv', ['run.py', 'plan-refresh']):
|
||||
run.main()
|
||||
research.sync.assert_called_once_with()
|
||||
recover.assert_called_once_with(None)
|
||||
self.assertEqual(refresh.refresh.call_count, 2)
|
||||
self.pool.assert_not_called()
|
||||
|
||||
def test_cli_does_not_treat_waiting_cooldown_or_failure_as_success(self):
|
||||
for status in ('waiting_data', 'cooldown', 'attempts_exhausted', 'failed', 'blocked_state'):
|
||||
refresh = types.SimpleNamespace(refresh=Mock(return_value={'status': 'no_morning_today'}))
|
||||
research = types.SimpleNamespace(sync=Mock())
|
||||
with self.subTest(status=status), patch.dict(sys.modules, {'plan_refresh': refresh, 'user_research': research}), \
|
||||
patch.object(recovery, 'ensure_today', return_value={'status': status}), \
|
||||
patch.object(sys, 'argv', ['run.py', 'plan-refresh']):
|
||||
with self.assertRaisesRegex(SystemExit, status):
|
||||
run.main()
|
||||
research.sync.assert_called_once()
|
||||
refresh.refresh.assert_called_once()
|
||||
|
||||
def test_cli_existing_plan_only_runs_normal_refresh(self):
|
||||
refresh = types.SimpleNamespace(refresh=Mock(return_value={'status': 'unchanged'}))
|
||||
research = types.SimpleNamespace(sync=Mock())
|
||||
with patch.dict(sys.modules, {'plan_refresh': refresh, 'user_research': research}), \
|
||||
patch.object(recovery, 'ensure_today') as recover, \
|
||||
patch.object(sys, 'argv', ['run.py', 'plan-refresh']):
|
||||
run.main()
|
||||
recover.assert_not_called()
|
||||
research.sync.assert_called_once()
|
||||
|
||||
def test_daily_build_and_recovery_share_publish_lock(self):
|
||||
entered = []
|
||||
@contextlib.contextmanager
|
||||
def guard(root):
|
||||
entered.append('lock')
|
||||
yield
|
||||
entered.append('unlock')
|
||||
def build(*args):
|
||||
self.assertEqual(entered, ['lock'])
|
||||
return DAY
|
||||
self.build.side_effect = build
|
||||
with patch.object(pv, 'writer', side_effect=guard):
|
||||
self.assertEqual(run.cmd_build('all', 'daily', None, None, None), DAY)
|
||||
self.assertEqual(entered, ['lock', 'unlock'])
|
||||
|
||||
def test_regime_cli_propagates_unpublished_status(self):
|
||||
for status in ('no_morning', 'no_morning_today', 'outside_window', 'published', 'unchanged'):
|
||||
regime = types.SimpleNamespace(append_to_snapshot=Mock(return_value={'status': 'OK', 'publication_status': status}))
|
||||
with self.subTest(status=status), patch.dict(sys.modules, {'regime': regime}), \
|
||||
patch.object(sys, 'argv', ['run.py', 'regime-append', '--date', DAY]):
|
||||
if status in ('published', 'unchanged'):
|
||||
run.main()
|
||||
else:
|
||||
with self.assertRaisesRegex(SystemExit, status):
|
||||
run.main()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
|
|
@ -74,10 +74,8 @@ def main():
|
|||
urllib.request.urlopen = lambda url, timeout=0: _Resp(json.dumps(raw).encode("utf-8"))
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
config.PLAN_SNAPSHOT_DIR = d
|
||||
missing = regime.append_to_snapshot("2026-09-01")
|
||||
t("快照不存在时不创建空快照", missing["status"] == "OK"
|
||||
t("快照不存在时不创建空快照", regime.append_to_snapshot("2026-09-01")["status"] == "OK"
|
||||
and not os.path.exists(regime.snapshot_path("2026-09-01")))
|
||||
t("有效区制不冒充成功发布", missing['publication_status'] == 'no_morning')
|
||||
with open(regime.snapshot_path("2026-09-01"), "w", encoding="utf-8") as f:
|
||||
json.dump({"date": "2026-09-01", "main": []}, f)
|
||||
import datetime as dt
|
||||
|
|
@ -91,9 +89,8 @@ def main():
|
|||
path = Path(regime.snapshot_path("2026-09-01"))
|
||||
legacy = path.read_bytes()
|
||||
with patch('plan_refresh.refresh', side_effect=at_fixed_time):
|
||||
result = regime.append_to_snapshot("2026-09-01")
|
||||
regime.append_to_snapshot("2026-09-01")
|
||||
t("旧格式不补造完整晨版,原字节保持", path.read_bytes() == legacy)
|
||||
t("旧格式缺完整晨版如实返回未发布", result['publication_status'] == 'no_morning')
|
||||
path.unlink() # 仅删除测试临时夹具,另起完整晨版场景。
|
||||
morning = {'date': '2026-09-01', 'plan_version': 'test', 'main': [], 'observe': []}
|
||||
with pv.writer(d):
|
||||
|
|
|
|||
|
|
@ -1,69 +0,0 @@
|
|||
"""取数和样本入口的沪深范围回归,仅内存假数据,不访问服务。"""
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
import pandas as pd
|
||||
import sources
|
||||
|
||||
DAY = '2026-11-04'
|
||||
|
||||
|
||||
class SourcesScopeTest(unittest.TestCase):
|
||||
def test_moved_members_filters_sql_and_rechecks_returned_codes(self):
|
||||
rows = [{'ts_code': code, 'target': '测试环节', 'n_sources': 2, 'chain_fit': 1,
|
||||
'members_total': 3, 'moved': 1, 'mkt_trade_date': DAY}
|
||||
for code in ('600000.SH', 'sz000001', '920001.BJ', '600000.SZ')]
|
||||
with patch.object(sources.db, 'read_pg', return_value=pd.DataFrame(rows)) as reader:
|
||||
result = sources.moved_members(DAY)
|
||||
self.assertEqual(set(result), {'SH600000', 'SZ000001'})
|
||||
self.assertIn("UPPER(TRIM(CAST(ts_code AS TEXT))) ~", reader.call_args.args[0])
|
||||
self.assertEqual(reader.call_args.args[1], (DAY,))
|
||||
|
||||
def test_daily_stock_filters_before_exposing_evidence(self):
|
||||
rows = [{'ts_code': code, 'pct_change': 3, 'net_z': 1, 'heat_chg': 0.2}
|
||||
for code in ('688001.SH', '300001', 'BJ920001', '000001.SH')]
|
||||
with patch.object(sources.db, 'read_pg', return_value=pd.DataFrame(rows)) as reader:
|
||||
result = sources.stock_daily(DAY)
|
||||
self.assertEqual(set(result), {'SH688001', 'SZ300001'})
|
||||
self.assertIn("UPPER(TRIM(CAST(ts_code AS TEXT))) ~", reader.call_args.args[0])
|
||||
|
||||
def test_prices_filter_at_query_and_normalize_without_guessing_wrong_exchange(self):
|
||||
rows = [{'ts_code': code, 'close': 10} for code in
|
||||
('SH600000', '000001.SZ', '300001', '920001', '920001.BJ', '600000.SZ')]
|
||||
with patch.object(sources.db, 'read_mysql', return_value=rows) as reader:
|
||||
result = sources.close_prices(DAY, code_col='symbol')
|
||||
self.assertEqual(set(result), {'SH600000', 'SZ000001', 'SZ300001'})
|
||||
self.assertIn('CAST(`symbol` AS CHAR)', reader.call_args.args[1])
|
||||
self.assertEqual(reader.call_args.args[2], (DAY, '2026-11-05'))
|
||||
|
||||
def test_price_column_must_be_a_known_identifier(self):
|
||||
with patch.object(sources.db, 'read_mysql') as reader:
|
||||
self.assertEqual(sources.close_prices(DAY, code_col='symbol` OR 1=1'), {})
|
||||
reader.assert_not_called()
|
||||
|
||||
def test_breadth_excludes_other_markets_without_filtering_market_indices(self):
|
||||
seen = []
|
||||
def pg(sql, params):
|
||||
self.assertIn('SELECT ts_code, pct_change', sql)
|
||||
self.assertIn('CAST(ts_code AS TEXT)', sql)
|
||||
return [{'ts_code': '600000.SH', 'pct_change': 3},
|
||||
{'ts_code': '000001.SZ', 'pct_change': -1},
|
||||
{'ts_code': '920001.BJ', 'pct_change': 20},
|
||||
{'ts_code': '000001.SH', 'pct_change': 10}, {'pct_change': 10}]
|
||||
def mysql(source, sql, params=None):
|
||||
seen.append((sql, params))
|
||||
if 'zs_day_data' in sql:
|
||||
return [{'symbol': '000001.SH', 'd': DAY, 'amount': 100},
|
||||
{'symbol': '399001.SZ', 'd': DAY, 'amount': 200}]
|
||||
return []
|
||||
result = sources.market_context(DAY, read_pg=pg, read_mysql=mysql)
|
||||
self.assertEqual(result['breadth']['n'], 2)
|
||||
self.assertEqual(result['breadth']['up'], 1)
|
||||
self.assertEqual(result['breadth']['down'], 1)
|
||||
self.assertEqual(result['turnover']['amount'], 300)
|
||||
self.assertEqual(seen[0][1], ('000001.SH', '399001.SZ'))
|
||||
self.assertNotIn('REGEXP', seen[0][0])
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
|
|
@ -1,199 +0,0 @@
|
|||
"""用户候选登记、原有门槛与版本闭环的断网回归。"""
|
||||
import copy
|
||||
import datetime as dt
|
||||
import json
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch, Mock
|
||||
from contextlib import ExitStack
|
||||
import pandas as pd
|
||||
import test_api_support
|
||||
import user_candidates as uc
|
||||
import plan, config, card, sources, factors, judgement, logic_state_daily
|
||||
import plan_inputs as pi
|
||||
import plan_versions as pv
|
||||
import plan_refresh
|
||||
import user_research
|
||||
|
||||
NOW = dt.datetime.fromisoformat('2026-11-03T10:00:00+08:00')
|
||||
DAY = '2026-11-02'
|
||||
PAYLOAD = {'codes': ['600001.SH'], 'reason': '行业需求变化', 'expires_on': '2026-11-10'}
|
||||
USER_NAMES = dict(zip(
|
||||
['SZ300133', 'SZ300364', 'SZ300182', 'SH603533', 'SZ300291', 'SZ301052'],
|
||||
['华策影视', '中文在线', '捷成股份', '掌阅科技', '百纳千成', '果麦文化']))
|
||||
|
||||
class CandidateTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
self.root = self.temp.name
|
||||
self.addCleanup(self.temp.cleanup)
|
||||
self.reports = patch.object(user_research, 'load_reports', return_value={})
|
||||
self.reports.start()
|
||||
self.addCleanup(self.reports.stop)
|
||||
|
||||
def test_exchange_validation_batch_atomic_and_dedup(self):
|
||||
for code in ('830001.BJ', '920001', 'SH000001', 'SZ600001', '123', '510300.SH'):
|
||||
with self.assertRaises(ValueError):
|
||||
uc.save(self.root, dict(PAYLOAD, codes=['600001', code]), '甲', NOW)
|
||||
self.assertEqual(uc.read(self.root)['rows'], [])
|
||||
r = uc.save(self.root, dict(PAYLOAD, codes=['600001', 'SH600001', '600001.SH']), '甲', NOW)
|
||||
self.assertEqual(r['added'], 1)
|
||||
self.assertEqual(uc.save(self.root, PAYLOAD, '乙', NOW)['existing'], 1)
|
||||
self.assertEqual(uc.read(self.root)['rows'][0]['created_by'], '甲')
|
||||
|
||||
def test_pasted_names_mixed_codes_and_separators_save_canonical_codes(self):
|
||||
text = '华策影视 中文在线,捷成股份\n掌阅科技;百纳千成、果麦文化'
|
||||
result = uc.save(self.root, dict(PAYLOAD, codes=[text, '300133.SZ']), '甲', NOW, names=USER_NAMES)
|
||||
self.assertEqual(result['added'], 6)
|
||||
self.assertEqual([r['code'] for r in uc.read(self.root)['rows']], list(USER_NAMES))
|
||||
self.assertEqual([r['name'] for r in uc.listing(self.root, NOW)['rows']], list(USER_NAMES.values())[::-1])
|
||||
self.assertEqual(uc.save(self.root, dict(PAYLOAD, codes=text), '甲', NOW, names=USER_NAMES)['existing'], 6)
|
||||
|
||||
def test_unmatched_or_ambiguous_name_does_not_partially_save(self):
|
||||
names = dict(USER_NAMES, SH600001='同名', SZ000001='同名')
|
||||
for value, message in [('华策影视 不存在的股票', '未找到沪深 A 股'),
|
||||
('华策影视 同名', '对应多只股票'),
|
||||
('华策影视 中文', '未找到沪深 A 股'),
|
||||
('华策影视 920001', '仅支持沪深 A 股'),
|
||||
(' '.join(['华策影视'] * 31), '1 至 30')]:
|
||||
with self.assertRaisesRegex(ValueError, message):
|
||||
uc.save(self.root, dict(PAYLOAD, codes=[value]), '甲', NOW, names=names)
|
||||
self.assertFalse(Path(self.root, 'user_candidates.json').exists())
|
||||
|
||||
def test_catalog_reads_full_listed_shanghai_shenzhen_universe(self):
|
||||
import db
|
||||
rows = [{'gp_code_two': c[2:]+'.'+c[:2], 'gp_name': n} for c, n in USER_NAMES.items()]
|
||||
rows += [{'gp_code_two': '920001.BJ', 'gp_name': '排除示例'}, {'gp_code_two': '510300.SH', 'gp_name': '基金示例'}]
|
||||
with patch.object(db, 'read_mysql', return_value=pd.DataFrame(rows)) as read:
|
||||
self.assertEqual(uc.catalog(), USER_NAMES)
|
||||
self.assertEqual(read.call_args.args[0], 'heat')
|
||||
self.assertIn("list_status='L'", read.call_args.args[1])
|
||||
self.assertIn("gp_code_two LIKE '%.SH' OR gp_code_two LIKE '%.SZ'", read.call_args.args[1])
|
||||
|
||||
def test_search_and_add_use_full_catalog_without_industry_membership(self):
|
||||
import api
|
||||
with patch.object(config, 'XXL_TRIGGER_KEY', 'test-secret'), patch.object(config, 'PLAN_SNAPSHOT_DIR', self.root), \
|
||||
patch.object(uc, 'catalog', return_value=USER_NAMES), patch.object(plan, '_names', side_effect=AssertionError('不能依赖行业池')):
|
||||
self.assertEqual(api.search_user_candidates('中文在线')['items'], [{'code': 'SZ300364', 'name': '中文在线'}])
|
||||
background = Mock()
|
||||
expiry = (dt.datetime.now(uc.BJ).date() + dt.timedelta(days=1)).isoformat()
|
||||
result = api.add_user_candidates(background, dict(PAYLOAD, codes=list(USER_NAMES.values()), expires_on=expiry), x_job_key='test-secret')
|
||||
self.assertEqual(result['added'], 6)
|
||||
background.add_task.assert_called_once_with(uc.refresh_saved)
|
||||
|
||||
def test_expiry_remove_and_readd_do_not_erase_history(self):
|
||||
uc.save(self.root, PAYLOAD, '甲', NOW)
|
||||
old = uc.active(self.root, NOW)[0]
|
||||
pub = {'user_candidates': [old]}
|
||||
uc.remove(self.root, old['id'], '甲', NOW)
|
||||
self.assertEqual(uc.live_pending(self.root, pub, NOW), ['SH600001'])
|
||||
self.assertEqual(uc.remove(self.root, old['id'], '甲', NOW)['revision'], 2)
|
||||
uc.save(self.root, PAYLOAD, '甲', NOW)
|
||||
self.assertNotEqual(uc.active(self.root, NOW)[0]['id'], old['id'])
|
||||
self.assertEqual(len(uc.read(self.root)['rows']), 2)
|
||||
later = NOW + dt.timedelta(days=8)
|
||||
self.assertEqual(uc.active(self.root, later), [])
|
||||
|
||||
def test_manual_discovery_never_fakes_transmission_or_other_gates(self):
|
||||
evidence = {'pointed': False, 'user_selected': True, 'pct0': 4,
|
||||
'covered': True, 'upside': .2, 'accum_state': '明确吸筹', 'accum_age': 0}
|
||||
result = card.judge(evidence)
|
||||
self.assertEqual(result['verdict'], card.VERDICT_CANDIDATE)
|
||||
self.assertFalse(result['gates']['pointed'])
|
||||
for change in ({'covered': False}, {'upside': -.2}, {'risk_name': True},
|
||||
{'pct0': 0}, {'y_signal': 'SELL'}, {'accum_state': '潜在吸筹'}):
|
||||
self.assertEqual(card.judge(dict(evidence, **change))['verdict'], card.VERDICT_SHOW)
|
||||
self.assertEqual(card.judge(dict(evidence, user_selected=False))['verdict'], card.VERDICT_SHOW)
|
||||
|
||||
def test_mutations_require_key_and_do_not_publish_inline(self):
|
||||
import api
|
||||
with patch.object(config, 'XXL_TRIGGER_KEY', 'test-secret'), patch.object(config, 'PLAN_SNAPSHOT_DIR', self.root):
|
||||
with self.assertRaises(api.HTTPException):
|
||||
api.add_user_candidates(None, PAYLOAD, x_job_key='bad')
|
||||
self.assertFalse(Path(self.root, 'user_candidates.json').exists())
|
||||
|
||||
def test_old_consumers_exclude_user_source_before_top_and_cache_is_separate(self):
|
||||
import api
|
||||
source={'id':'a'*32,'code':'SH600001'}
|
||||
row={'code':'SH600001','score':250,'rank':1,'research_rank':1,'evidence':{},
|
||||
'user_candidate':source,'user_required':True,'verdict':'候选'}
|
||||
ordinary={'code':'SH600002','score':240,'rank':2,'research_rank':2,'evidence':{},'verdict':'候选'}
|
||||
published={'date':DAY,'decision':{'id':'v-opt-in'},'main':[row,ordinary],'observe':[],
|
||||
'user_candidates':[source],'candidates':[row,ordinary],'watch':[],
|
||||
'counts':{'main':2,'observe':0}}
|
||||
original=copy.deepcopy(published)
|
||||
with patch.object(config,'PLAN_SNAPSHOT_DIR',self.root),patch.object(pv,'load',return_value=published):
|
||||
default,_=api._plan_cached(DAY,1,0,0,True)
|
||||
opted,_=api._plan_cached(DAY,1,0,0,True,include_user_candidates=True)
|
||||
again,hit=api._plan_cached(DAY,1,0,0,False)
|
||||
self.assertEqual(default['main'][0]['code'],'SH600002')
|
||||
self.assertEqual(opted['main'][0]['code'],'SH600001')
|
||||
self.assertEqual(default,again)
|
||||
self.assertTrue(hit)
|
||||
self.assertNotIn('user_candidates',default)
|
||||
self.assertEqual(default['counts']['main'],1)
|
||||
self.assertEqual(len(default['candidates']),1)
|
||||
self.assertEqual(original,published)
|
||||
|
||||
def test_revision_add_remove_replay_and_frozen_morning(self):
|
||||
codes = ['SH600001', 'SH600002']
|
||||
with ExitStack() as stack:
|
||||
def stub(obj, name, **kw): return stack.enter_context(patch.object(obj, name, **kw))
|
||||
stub(config, 'PLAN_SNAPSHOT_DIR', new=self.root)
|
||||
stub(config, 'PLAN_RANK_AXIS', new='quality')
|
||||
stub(plan, '_factor', side_effect=lambda table, day: pd.Series(
|
||||
[240.,230.] if 'score' in table else [.3,.2] if 'upside' in table else [2.,2.], index=codes))
|
||||
stub(plan, '_latest_date', return_value=DAY)
|
||||
stub(plan, '_prev_date', return_value=None)
|
||||
stub(plan, '_names', return_value={'SH600001':'甲','SH600002':'乙'})
|
||||
stub(uc, 'catalog', return_value={'SH600001':'甲','SH600002':'乙','SH600003':'丙'})
|
||||
stub(plan, '_evidence', return_value=({}, {DAY}))
|
||||
stub(plan, '_segments_of', return_value={})
|
||||
stub(factors, '_risk_set', return_value=set())
|
||||
stub(judgement, 'load_previous', return_value=[])
|
||||
stub(judgement, 'recent_rows', return_value={})
|
||||
stub(logic_state_daily, 'history', return_value={})
|
||||
for name in ('moved_members','broker_reports','logic_claims','broker_actions','close_prices',
|
||||
'valuation_scenarios','analyst_events','event_day_fields','news_flashes',
|
||||
'segment_catalysts','catalysts_for_codes'):
|
||||
stub(sources, name, return_value={})
|
||||
stub(sources, 'stock_daily', return_value={c:{'pct0':4} for c in codes})
|
||||
stub(sources, 'night_conclusions', return_value={c:{'accum_state':'明确吸筹','accum_age':0} for c in codes})
|
||||
stub(sources, 'company_reviews', side_effect=lambda cs,*a,**kw: {c:{'overall':'好','age_days':0,'review_ref':'r'} for c in cs})
|
||||
bundle = pi.Bundle(params=pi.effective_config(config), cutoff_at='2026-11-03T07:10:00+08:00')
|
||||
with bundle.use():
|
||||
data = plan.collect(DAY)
|
||||
data.update(data.pop('_full'))
|
||||
with pv.writer(self.root):
|
||||
morning = pv.publish(self.root,data,{'records':bundle.records,'market':{},'regime':None},
|
||||
bundle.params,cutoff_at=bundle.cutoff_at,reason='晨版')
|
||||
frozen = copy.deepcopy(morning)
|
||||
uc.save(self.root,dict(PAYLOAD,codes=['600001','600003']), '甲', NOW)
|
||||
self.assertEqual(pv.stock_check(self.root,'SH600001')['refresh_status']['pending_codes'],['SH600001','SH600003'])
|
||||
result = plan_refresh.refresh(now=NOW)
|
||||
self.assertEqual(result['status'],'published')
|
||||
published = pv.load(self.root)
|
||||
rows = {r['code']:r for r in published['main']}
|
||||
self.assertEqual(rows['SH600001']['verdict'],'候选')
|
||||
self.assertFalse(rows['SH600001']['pointed'])
|
||||
self.assertEqual(rows['SH600001']['score'],240.)
|
||||
self.assertIsNone(rows['SH600003']['score'])
|
||||
self.assertEqual(rows['SH600003']['name'], '丙')
|
||||
self.assertNotIn('tier',rows['SH600003'])
|
||||
self.assertFalse(pv.stock_check(self.root,'SH600003')['found'])
|
||||
self.assertTrue(pv.stock_check(self.root,'SH600003',include_user_candidates=True)['found'])
|
||||
json.dumps(published,allow_nan=False)
|
||||
self.assertIn('SH600003', Path(self.root, f'plan_{DAY}.md').read_text())
|
||||
self.assertEqual(pv.load(self.root,DAY,morning=True),frozen)
|
||||
self.assertEqual(plan_refresh.refresh(now=NOW)['status'],'unchanged')
|
||||
for source in uc.active(self.root,NOW):
|
||||
uc.remove(self.root,source['id'],'甲',NOW)
|
||||
plan_refresh.refresh(now=NOW)
|
||||
after = pv.load(self.root)
|
||||
self.assertEqual(after['main'][0]['verdict'],'仅展示')
|
||||
self.assertNotIn('SH600003',[r['code'] for r in after['main']])
|
||||
self.assertEqual(pv.load(self.root,DAY,morning=True),frozen)
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
|
|
@ -1,244 +0,0 @@
|
|||
"""人工关注评析:所有外部来源与投递均用桩,状态文件仅在临时目录。"""
|
||||
import copy
|
||||
import datetime as dt
|
||||
import io
|
||||
import json
|
||||
from pathlib import Path
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import Mock, patch
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
|
||||
import test_plan_versions # 缺席数据库驱动用桩;统一由断网护栏启动。
|
||||
import config
|
||||
import sources
|
||||
import user_candidates as uc
|
||||
import user_research as ur
|
||||
import plan_versions as pv
|
||||
import review_targets as rt
|
||||
|
||||
NOW = dt.datetime.fromisoformat('2026-11-04T10:00:00+08:00')
|
||||
CODE = 'SZ300133'
|
||||
|
||||
|
||||
def report(**changes):
|
||||
return dict({'review_ref': 'a' * 64, 'brain_used': True, 'brain_status': 'ok',
|
||||
'thesis': '订单改善仍需后续验证', 'overall': '中',
|
||||
'report_ran_at': '2026-11-03T18:00:00+00:00',
|
||||
'checked_at': '2026-11-03T18:00:00+00:00'}, **changes)
|
||||
|
||||
|
||||
class ResearchTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.temp.cleanup)
|
||||
self.root = self.temp.name
|
||||
self.reader = Mock(return_value={})
|
||||
self.submit = Mock(return_value={'status': 'queued', 'task_id': 'private-task'})
|
||||
self.patches = [patch.object(config, 'PLAN_SNAPSHOT_DIR', self.root),
|
||||
patch.object(config, 'USER_RESEARCH_FRESH_DAYS', 7),
|
||||
patch.object(config, 'USER_RESEARCH_SUBMIT_LIMIT', 3)]
|
||||
for p in self.patches:
|
||||
p.start()
|
||||
self.addCleanup(p.stop)
|
||||
uc.save(self.root, {'codes': [CODE], 'reason': '自主关注', 'expires_on': '2026-11-10'}, 'test', NOW)
|
||||
|
||||
def sync(self, **kwargs):
|
||||
return ur.sync(self.root, now=kwargs.pop('now', NOW), reader=self.reader,
|
||||
submit=self.submit, target_write=lambda *a: None, **kwargs)
|
||||
|
||||
def test_complete_requires_real_reference_brain_thesis_and_time(self):
|
||||
self.assertTrue(ur.report_state(report(), NOW)[0])
|
||||
for change in ({'review_ref': None}, {'review_ref': 'old'}, {'brain_used': False},
|
||||
{'brain_status': 'half'}, {'thesis': ''}, {'report_ran_at': None},
|
||||
{'report_ran_at': '2026-11-05T10:00:00+08:00'}):
|
||||
with self.subTest(change=change):
|
||||
self.assertFalse(ur.report_state(report(**change), NOW)[0])
|
||||
|
||||
def test_recent_real_recheck_keeps_old_report_ready_without_redating(self):
|
||||
old = report(report_ran_at='2026-09-29T00:00:00+00:00')
|
||||
self.reader.return_value = {CODE: old}
|
||||
self.sync()
|
||||
self.submit.assert_not_called()
|
||||
view = ur.views(self.root, [CODE], now=NOW, reader=self.reader)[CODE]
|
||||
self.assertEqual(view['review_status'], 'completed')
|
||||
self.assertEqual(view['review_data_date'], '2026-09-29')
|
||||
self.assertNotIn('task_id', view)
|
||||
|
||||
def test_future_check_cannot_freshen_stale_report(self):
|
||||
old = report(report_ran_at='2026-09-29T00:00:00+00:00', checked_at='2026-11-05T00:00:00+00:00')
|
||||
self.assertFalse(ur.report_state(old, NOW)[0])
|
||||
|
||||
def test_claim_is_durable_before_submission_and_prevents_double_enqueue(self):
|
||||
def submit(code):
|
||||
self.assertEqual(ur.read(self.root)['rows'][code]['status'], 'pending')
|
||||
self.sync() # 并发调用能看到已持久化的 claim,不能再次派单。
|
||||
return {'status': 'queued', 'task_id': 'private-task'}
|
||||
self.submit.side_effect = submit
|
||||
self.sync()
|
||||
self.sync()
|
||||
self.assertEqual(self.submit.call_count, 1)
|
||||
row = ur.read(self.root)['rows'][CODE]
|
||||
self.assertEqual(row['status'], 'queued')
|
||||
self.assertEqual(row['task_id'], 'private-task')
|
||||
|
||||
def test_timeout_is_uncertain_not_failure_or_automatic_retry(self):
|
||||
self.submit.side_effect = TimeoutError('offline timeout')
|
||||
self.sync()
|
||||
self.sync(now=NOW + dt.timedelta(hours=2))
|
||||
self.assertEqual(self.submit.call_count, 1)
|
||||
row = ur.read(self.root)['rows'][CODE]
|
||||
self.assertEqual(row['status'], 'pending')
|
||||
self.assertIn('待核实', row['error'])
|
||||
|
||||
def test_completed_report_recovers_queued_or_uncertain_request(self):
|
||||
self.sync()
|
||||
self.reader.return_value = {CODE: report()}
|
||||
self.sync(now=NOW + dt.timedelta(minutes=5))
|
||||
self.assertEqual(ur.read(self.root)['rows'][CODE]['status'], 'completed')
|
||||
self.assertEqual(self.submit.call_count, 1)
|
||||
|
||||
def test_submission_done_is_not_report_completion(self):
|
||||
self.submit.return_value = {'status': 'done', 'result': {'status': 'ok'}}
|
||||
self.sync()
|
||||
self.assertEqual(ur.read(self.root)['rows'][CODE]['status'], 'pending')
|
||||
self.reader.return_value = {CODE: report()}
|
||||
self.sync(now=NOW + dt.timedelta(minutes=5))
|
||||
self.assertEqual(ur.read(self.root)['rows'][CODE]['status'], 'completed')
|
||||
|
||||
def test_old_no_reference_recheck_is_failed_not_fabricated_completed(self):
|
||||
self.reader.return_value = {CODE: report(review_ref=None)}
|
||||
self.sync()
|
||||
self.reader.return_value = {CODE: report(review_ref=None, checked_at='2026-11-04T10:01:00+08:00')}
|
||||
self.sync(now=NOW + dt.timedelta(minutes=5))
|
||||
row = ur.read(self.root)['rows'][CODE]
|
||||
self.assertEqual(row['status'], 'failed')
|
||||
self.assertIn('固定版本', row['error'])
|
||||
self.assertEqual(self.submit.call_count, 1)
|
||||
|
||||
def test_explicit_rejection_does_not_retry_every_schedule(self):
|
||||
self.submit.side_effect = urllib.error.HTTPError('offline', 403, '', {}, None)
|
||||
self.sync()
|
||||
self.sync()
|
||||
self.assertEqual(ur.read(self.root)['rows'][CODE]['status'], 'failed')
|
||||
self.assertEqual(self.submit.call_count, 1)
|
||||
|
||||
def test_source_failure_never_submits_and_get_reports_failure(self):
|
||||
self.reader.side_effect = RuntimeError('offline unavailable')
|
||||
self.assertEqual(self.sync()['status'], 'failed')
|
||||
self.submit.assert_not_called()
|
||||
self.assertEqual(ur.views(self.root, [CODE], now=NOW, reader=self.reader)[CODE]['review_status'], 'failed')
|
||||
|
||||
def test_readonly_listing_reads_live_reports_without_side_effects(self):
|
||||
before = {p.name: p.read_bytes() for p in Path(self.root).iterdir() if p.is_file()}
|
||||
with patch.object(ur, 'load_reports', return_value={CODE: report()}), \
|
||||
patch.object(ur, 'submit_review', side_effect=AssertionError('GET不可投递')):
|
||||
row = uc.listing(self.root, NOW)['rows'][0]
|
||||
self.assertEqual(row['status'], 'pending') # 尚无计划仍可有真实完成报告。
|
||||
self.assertEqual(row['review_status'], 'completed')
|
||||
self.assertEqual(row['review_ref'], 'a' * 64)
|
||||
self.assertEqual(before, {p.name: p.read_bytes() for p in Path(self.root).iterdir() if p.is_file()})
|
||||
|
||||
def test_removed_expired_and_out_of_scope_never_submit(self):
|
||||
uc.remove(self.root, uc.active(self.root, NOW)[0]['id'], 'test', NOW)
|
||||
self.sync()
|
||||
self.submit.assert_not_called()
|
||||
state = uc.read(self.root)
|
||||
state['rows'].append({'id': 'bad', 'code': 'BJ920001', 'expires_on': '2026-11-10'})
|
||||
pv.atomic(Path(self.root) / 'user_candidates.json', state)
|
||||
self.sync()
|
||||
self.submit.assert_not_called()
|
||||
self.assertEqual(ur.views(self.root, ['920001.BJ'], now=NOW, reader=self.reader), {})
|
||||
|
||||
def test_bounded_submissions_continue_next_natural_cycle(self):
|
||||
uc.save(self.root, {'codes': ['300364', '300182', '603533', '300291', '301052'],
|
||||
'reason': '自主关注', 'expires_on': '2026-11-10'}, 'test', NOW)
|
||||
self.assertEqual(self.sync()['submitted'], 3)
|
||||
self.assertEqual(self.sync()['submitted'], 3)
|
||||
self.assertEqual(self.sync()['submitted'], 0)
|
||||
self.assertEqual(self.submit.call_count, 6)
|
||||
|
||||
def test_stale_complete_gets_real_recheck_then_checked_at_recovers(self):
|
||||
self.reader.return_value = {CODE: report()}
|
||||
self.sync()
|
||||
later = NOW + dt.timedelta(days=8)
|
||||
# 延长关注仅供隔离测试,不改变报告时间。
|
||||
state = uc.read(self.root)
|
||||
state['rows'][0]['expires_on'] = '2026-11-30'
|
||||
pv.atomic(Path(self.root) / 'user_candidates.json', state)
|
||||
self.assertEqual(self.sync(now=later)['submitted'], 1)
|
||||
self.reader.return_value = {CODE: report(checked_at=later.isoformat())}
|
||||
self.sync(now=later + dt.timedelta(minutes=5))
|
||||
self.assertEqual(ur.read(self.root)['rows'][CODE]['status'], 'completed')
|
||||
|
||||
def test_targets_append_all_manual_without_score_verdict_or_prefix_cut(self):
|
||||
snap = {'main': [{'code': f'SH60{i:04d}', 'score': 300-i, 'rank': i+1} for i in range(150)],
|
||||
'observe': [], 'user_candidates': [dict(r, verdict='仅展示', score=None) for r in uc.active(self.root, NOW)]}
|
||||
got = []
|
||||
rt.persist('2026-11-03', snap, write=lambda day, rows: got.extend(rows))
|
||||
row = next(r for r in got if r['ts_code'] == '300133.SZ')
|
||||
self.assertEqual(row['source'], 'manual')
|
||||
self.assertEqual(len(got), rt.TOP_MAIN + 1)
|
||||
rows = rt.build_rows('2026-11-03', [], [], [{'code': 'BJ920001'}, {'code': CODE}, {'code': CODE}])
|
||||
self.assertEqual([r['ts_code'] for r in rows], ['300133.SZ'])
|
||||
|
||||
def test_target_sync_preserves_plan_files_and_removes_deleted_manual_rows(self):
|
||||
snap = {'date': '2026-11-03', 'main': [], 'observe': [], 'decision': {'id': 'read-only'}}
|
||||
path = Path(self.root) / 'plan_2026-11-03.json'
|
||||
pv.atomic(path, snap)
|
||||
before = path.read_bytes()
|
||||
got = []
|
||||
ur.sync(self.root, now=NOW, reader=self.reader, submit=self.submit,
|
||||
target_write=lambda day, rows: got.append(rows))
|
||||
self.assertEqual(got[-1][0]['ts_code'], '300133.SZ')
|
||||
self.assertEqual(path.read_bytes(), before)
|
||||
uc.remove(self.root, uc.active(self.root, NOW)[0]['id'], 'test', NOW)
|
||||
ur.sync(self.root, now=NOW, reader=self.reader, submit=self.submit,
|
||||
target_write=lambda day, rows: got.append(rows))
|
||||
self.assertEqual(got[-1], [])
|
||||
self.assertEqual(path.read_bytes(), before)
|
||||
|
||||
def test_submit_uses_fixed_base_and_flags_without_redirect(self):
|
||||
response = io.BytesIO(json.dumps({'status': 'queued', 'task_id': 'x'}).encode())
|
||||
opener = Mock()
|
||||
opener.open.return_value = response
|
||||
with patch.object(config, 'AKG_API_BASE', 'http://offline.invalid:8000'), \
|
||||
patch.object(ur.urllib.request, 'build_opener', return_value=opener):
|
||||
self.assertEqual(ur.submit_review(CODE)['status'], 'queued')
|
||||
req = opener.open.call_args.args[0]
|
||||
url = urllib.parse.urlsplit(req.full_url)
|
||||
self.assertEqual((url.netloc, url.path), ('offline.invalid:8000', '/admin/company_review'))
|
||||
self.assertEqual(req.method, 'POST')
|
||||
q = urllib.parse.parse_qs(url.query)
|
||||
self.assertEqual(q, {'ts_code': ['300133.SZ'], 'force': ['false'], 'sync': ['false'],
|
||||
'with_brain': ['true'], 'with_qual': ['true']})
|
||||
self.assertLessEqual(opener.open.call_args.kwargs['timeout'], 10)
|
||||
with self.assertRaises(urllib.error.HTTPError):
|
||||
ur._NoRedirect().redirect_request(req, None, 302, '', {}, 'http://elsewhere.invalid')
|
||||
|
||||
def test_checked_column_only_for_live_independent_research(self):
|
||||
import pandas as pd
|
||||
row = {'ts_code': '300133.SZ', 'period': '2026Q3', 'review_version': 'v',
|
||||
'grades': {}, 'summary': report(), 'brain_used': True, 'brain_status': 'ok',
|
||||
'ran_at': dt.datetime.fromisoformat('2026-09-29T00:00:00+00:00'),
|
||||
'checked_at': NOW}
|
||||
reader = Mock(return_value=pd.DataFrame([row]))
|
||||
got = sources.company_reviews([CODE], '2026-11-04', read_pg=reader,
|
||||
cutoff_at=NOW.isoformat(), include_check=True)
|
||||
self.assertIn('checked_at', reader.call_args.args[0])
|
||||
self.assertEqual(got[CODE]['checked_at'], NOW.isoformat())
|
||||
normal = sources.company_reviews([CODE], '2026-11-04', read_pg=reader, cutoff_at=NOW.isoformat())
|
||||
self.assertNotIn('checked_at', normal[CODE])
|
||||
self.assertNotIn(', checked_at', reader.call_args.args[0])
|
||||
|
||||
def test_background_research_failure_does_not_block_plan_refresh(self):
|
||||
import plan_refresh
|
||||
with patch.object(ur, 'sync', side_effect=RuntimeError('offline failure')), \
|
||||
patch.object(plan_refresh, 'refresh', return_value={'status': 'unchanged'}) as refresh:
|
||||
self.assertEqual(uc.refresh_saved()['status'], 'unchanged')
|
||||
refresh.assert_called_once()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
|
|
@ -128,7 +128,7 @@ def test_fetch_shared():
|
|||
return [{"ts_code": "002812.SZ", "close": "50.17"}, {"ts_code": "600000", "close": 10.5},
|
||||
{"ts_code": "430047", "close": 3.0}, {"ts_code": "300750.SZ", "close": None}]
|
||||
px = sources.close_prices(DS, read_mysql=_price_reader, code_col="symbol")
|
||||
t("收盘价按前缀码索引,沪深两种形态都认且排除范围外证券", px == {"SZ002812": 50.17, "SH600000": 10.5})
|
||||
t("收盘价按前缀码索引,两种代码形态都认", px == {"SZ002812": 50.17, "SH600000": 10.5, "BJ430047": 3.0})
|
||||
t("日期写成左闭右开区间", "`timestamp` >= %s AND `timestamp` < %s" in seen["psql"] and seen["pparams"] == (DS, "2026-09-03"))
|
||||
t("收盘价读失败返回空字典", sources.close_prices(DS, read_mysql=lambda *a: (_ for _ in ()).throw(OSError("x")), code_col="symbol") == {})
|
||||
|
||||
|
|
|
|||
|
|
@ -1,220 +0,0 @@
|
|||
"""用户补充的候选来源。与计划共用文件锁,先保存,再由修订任务发布。"""
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime as dt
|
||||
import json
|
||||
import re
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
import plan_versions as pv
|
||||
|
||||
BJ = ZoneInfo('Asia/Shanghai')
|
||||
|
||||
|
||||
def consumer_view(published, include_user_candidates=False):
|
||||
"""旧调用方只取系统来源;显式声明支持新来源后才返回人工候选。"""
|
||||
if include_user_candidates or not published.get('user_candidates'):
|
||||
return published
|
||||
out = dict(published)
|
||||
out.pop('user_candidates', None)
|
||||
for bucket in ('main', 'observe', 'main_shown', 'observe_shown', 'candidates', 'watch'):
|
||||
if bucket in out:
|
||||
out[bucket] = [{k: v for k, v in r.items() if k not in ('user_candidate', 'user_required')}
|
||||
for r in out[bucket] if not r.get('user_required')]
|
||||
if 'counts' in out:
|
||||
out['counts'] = dict(out['counts'], main=len(out.get('main', [])), observe=len(out.get('observe', [])))
|
||||
if 'card_counts' in out:
|
||||
rows = out.get('main', []) + out.get('observe', [])
|
||||
out['card_counts'] = {v: sum(r.get('verdict') == v for r in rows) for v in out['card_counts']}
|
||||
return out
|
||||
|
||||
|
||||
def normalize(value):
|
||||
value = str(value or '').strip().upper()
|
||||
if re.fullmatch(r'\d{6}\.(SH|SZ)', value):
|
||||
value = value[-2:] + value[:6]
|
||||
elif re.fullmatch(r'\d{6}', value):
|
||||
value = ('SH' if value.startswith(('60', '68')) else 'SZ') + value
|
||||
if not re.fullmatch(r'(SH(?:60|68)\d{4}|SZ(?:00|30)\d{4})', value):
|
||||
raise ValueError('仅支持沪深 A 股,请检查股票代码')
|
||||
return value
|
||||
|
||||
|
||||
def catalog():
|
||||
"""使用上市证券基础表,不能把部分行业池当作完整可搜索名单。"""
|
||||
import db
|
||||
frame = db.read_mysql('heat',
|
||||
"SELECT gp_code_two,gp_name FROM gp_code_all WHERE list_status='L' "
|
||||
"AND (gp_code_two LIKE '%.SH' OR gp_code_two LIKE '%.SZ')")
|
||||
out = {}
|
||||
for row in frame.to_dict('records'):
|
||||
try:
|
||||
code = normalize(row.get('gp_code_two'))
|
||||
except ValueError:
|
||||
continue
|
||||
name = str(row.get('gp_name') or '').strip()
|
||||
if name:
|
||||
out[code] = name
|
||||
if not out:
|
||||
raise ValueError('证券名称资料暂不可用,请稍后重试')
|
||||
return out
|
||||
|
||||
|
||||
def resolve(raw, names=None):
|
||||
"""批量名称只按完整简称精确匹配;全部解析成功后才允许保存。"""
|
||||
if isinstance(raw, str):
|
||||
raw = [raw]
|
||||
if not isinstance(raw, list) or any(not isinstance(v, str) for v in raw):
|
||||
raise ValueError('请输入股票名称或代码')
|
||||
tokens = [v for item in raw for v in re.split(r'[\s,,;;、]+', item.strip()) if v]
|
||||
if not 1 <= len(tokens) <= 30:
|
||||
raise ValueError('每次可添加 1 至 30 只股票')
|
||||
index = None
|
||||
codes = []
|
||||
for value in tokens:
|
||||
try:
|
||||
code = normalize(value)
|
||||
except ValueError:
|
||||
# 明确的证券代码继续走原交易所限制,不尝试按名称解释。
|
||||
if re.fullmatch(r'(?:(?:SH|SZ|BJ)?\d+(?:\.(?:SH|SZ|BJ))?)', value.upper()):
|
||||
raise
|
||||
if index is None:
|
||||
source = names() if callable(names) else names or {}
|
||||
index = {}
|
||||
for candidate, name in source.items():
|
||||
try:
|
||||
canonical = normalize(candidate)
|
||||
except ValueError:
|
||||
continue
|
||||
index.setdefault(str(name).strip().upper(), set()).add(canonical)
|
||||
matches = index.get(value.upper(), set())
|
||||
if not matches:
|
||||
raise ValueError(f'未找到沪深 A 股“{value[:40]}”,请检查完整名称或使用代码') from None
|
||||
if len(matches) != 1:
|
||||
raise ValueError(f'“{value[:40]}”对应多只股票,请使用代码') from None
|
||||
code = next(iter(matches))
|
||||
if code not in codes:
|
||||
codes.append(code)
|
||||
return codes
|
||||
|
||||
|
||||
def read(root):
|
||||
path = Path(root) / 'user_candidates.json'
|
||||
if not path.exists():
|
||||
return {'revision': 0, 'rows': []}
|
||||
state = json.loads(path.read_text())
|
||||
if not isinstance(state.get('rows'), list) or not isinstance(state.get('revision'), int):
|
||||
raise ValueError('添加记录暂不可用')
|
||||
return state
|
||||
|
||||
|
||||
def active(root, now=None):
|
||||
today = (now or dt.datetime.now(BJ)).astimezone(BJ).date().isoformat()
|
||||
return [dict(r) for r in read(root)['rows']
|
||||
if not r.get('removed_at') and r['expires_on'] >= today]
|
||||
|
||||
|
||||
def save(root, payload, actor, now=None, *, names=None):
|
||||
now = now or dt.datetime.now(BJ)
|
||||
codes = resolve(payload.get('codes'), names)
|
||||
reason = str(payload.get('reason') or '').strip()
|
||||
if not reason or len(reason) > 300:
|
||||
raise ValueError('请填写添加理由,最多 300 字')
|
||||
try:
|
||||
expires = dt.date.fromisoformat(str(payload.get('expires_on') or ''))
|
||||
except ValueError as exc:
|
||||
raise ValueError('请选择关注截止日期') from exc
|
||||
if not now.astimezone(BJ).date() <= expires <= now.astimezone(BJ).date() + dt.timedelta(days=90):
|
||||
raise ValueError('关注期限应在今天至未来 90 天内')
|
||||
with pv.writer(root):
|
||||
state = read(root)
|
||||
current = {r['code'] for r in active(root, now)}
|
||||
added = []
|
||||
for code in codes:
|
||||
if code in current:
|
||||
continue
|
||||
row = {'id': uuid.uuid4().hex, 'code': code, 'reason': reason,
|
||||
'expires_on': expires.isoformat(), 'created_at': now.isoformat(),
|
||||
'created_by': str(actor)[:100]}
|
||||
if isinstance(names, dict) and names.get(code):
|
||||
row['name'] = names[code]
|
||||
state['rows'].append(row)
|
||||
added.append(row)
|
||||
if added:
|
||||
state['revision'] += 1
|
||||
pv.atomic(Path(root) / 'user_candidates.json', state)
|
||||
mark_pending(root, [r['code'] for r in added])
|
||||
return {'ok': True, 'added': len(added), 'existing': len(codes) - len(added),
|
||||
'revision': state['revision']}
|
||||
|
||||
|
||||
def remove(root, identity, actor, now=None):
|
||||
now = now or dt.datetime.now(BJ)
|
||||
with pv.writer(root):
|
||||
state = read(root)
|
||||
row = next((r for r in state['rows'] if r['id'] == identity), None)
|
||||
if row is None:
|
||||
raise ValueError('添加记录不存在,请刷新后重试')
|
||||
if not row.get('removed_at'):
|
||||
row.update(removed_at=now.isoformat(), removed_by=str(actor)[:100])
|
||||
state['revision'] += 1
|
||||
pv.atomic(Path(root) / 'user_candidates.json', state)
|
||||
mark_pending(root, [row['code']])
|
||||
return {'ok': True, 'revision': state['revision']}
|
||||
|
||||
|
||||
def mark_pending(root, codes):
|
||||
day = pv.latest_date(root)
|
||||
if day:
|
||||
state = pv.health(root, day)
|
||||
pv.health(root, day, pending_codes=sorted(set(state.get('pending_codes') or []) | set(codes)))
|
||||
|
||||
|
||||
def live_pending(root, published, now=None):
|
||||
"""写后未发布、删除或到期时,即使后台失败也不能继续消费旧人工来源。"""
|
||||
current = {r['code']: r['id'] for r in active(root, now)}
|
||||
frozen = {r['code']: r['id'] for r in published.get('user_candidates', [])}
|
||||
return sorted(c for c in current.keys() | frozen.keys() if current.get(c) != frozen.get(c))
|
||||
|
||||
|
||||
def listing(root, now=None):
|
||||
now = now or dt.datetime.now(BJ)
|
||||
state = read(root)
|
||||
try:
|
||||
published = pv.load(root) if pv.latest_date(root) else {}
|
||||
except FileNotFoundError:
|
||||
published = {}
|
||||
material = {r['code']: r for b in ('main', 'observe') for r in published.get(b, [])}
|
||||
expected = {r['id'] for r in published.get('user_candidates', [])}
|
||||
today = now.astimezone(BJ).date().isoformat()
|
||||
import user_research
|
||||
research = user_research.views(root, [r['code'] for r in state['rows'][-300:]], now=now)
|
||||
rows = []
|
||||
for source in reversed(state['rows']):
|
||||
row = material.get(source['code']) or {}
|
||||
status = ('removed' if source.get('removed_at') else
|
||||
'expired' if source['expires_on'] < today else
|
||||
'pending' if source['id'] not in expected else 'evaluated')
|
||||
rows.append({**source, 'status': status, 'name': row.get('name') or source.get('name'),
|
||||
'verdict': row.get('verdict'), 'basis': row.get('basis'),
|
||||
'decision_id': (published.get('decision') or {}).get('id'),
|
||||
**research.get(source['code'], {})})
|
||||
return {'ok': True, 'rows': rows[:300], 'revision': state['revision']}
|
||||
|
||||
|
||||
def refresh_saved():
|
||||
"""请求结束后尝试发布;失败保留登记和待核验状态,定时修订会重试。"""
|
||||
import logging
|
||||
import plan_refresh
|
||||
import user_research
|
||||
try:
|
||||
user_research.sync()
|
||||
except Exception:
|
||||
logging.getLogger(__name__).exception('人工关注评析状态同步失败,保留登记等待自然调度')
|
||||
try:
|
||||
return plan_refresh.refresh()
|
||||
except Exception:
|
||||
logging.getLogger(__name__).exception('用户候选已保存,计划修订待重试')
|
||||
return {'status': 'pending'}
|
||||
273
user_research.py
273
user_research.py
|
|
@ -1,273 +0,0 @@
|
|||
"""人工关注的独立评析状态:先持久化受理凭据,再投基座;查询绝不派任务。
|
||||
|
||||
计划的 evaluated 只表示已收录,不能代替买方评议完成。网络结果不明时不重复
|
||||
投递,后续只读核验真实报告,并由原评析目标名单/批任务继续补齐。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import datetime as dt
|
||||
import fcntl
|
||||
import json
|
||||
import logging
|
||||
from pathlib import Path
|
||||
import re
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
import uuid
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
import plan_versions as pv
|
||||
|
||||
BJ = ZoneInfo('Asia/Shanghai')
|
||||
UTC = dt.timezone.utc
|
||||
_FILE = 'user_research.json'
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _time(value):
|
||||
try:
|
||||
value = value if isinstance(value, dt.datetime) else dt.datetime.fromisoformat(str(value).replace('Z', '+00:00'))
|
||||
return value.replace(tzinfo=UTC) if value.tzinfo is None else value
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def _now(now=None):
|
||||
return _time(now) or dt.datetime.now(UTC)
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def _writer(root):
|
||||
Path(root).mkdir(parents=True, exist_ok=True)
|
||||
with open(Path(root) / '.user-research.lock', 'a') as lock:
|
||||
fcntl.flock(lock, fcntl.LOCK_EX)
|
||||
yield
|
||||
|
||||
|
||||
def read(root):
|
||||
path = Path(root) / _FILE
|
||||
if not path.exists():
|
||||
return {'version': 1, 'rows': {}}
|
||||
value = json.loads(path.read_text())
|
||||
if value.get('version') != 1 or not isinstance(value.get('rows'), dict):
|
||||
raise ValueError('人工关注评析状态暂不可用')
|
||||
return value
|
||||
|
||||
|
||||
def _save(root, state):
|
||||
pv.atomic(Path(root) / _FILE, state)
|
||||
|
||||
|
||||
def _active(root, now):
|
||||
import user_candidates as uc
|
||||
rows = []
|
||||
for row in uc.active(root, now):
|
||||
try:
|
||||
rows.append(dict(row, code=uc.normalize(row['code'])))
|
||||
except (ValueError, KeyError):
|
||||
# 持久化来源也在入口守沪深范围,不能因旧文件或外部写入越界。
|
||||
continue
|
||||
return rows
|
||||
|
||||
|
||||
def report_state(report, now=None, fresh_days=7):
|
||||
"""只判报告完成事实;财报期是否落后仍由基座原有批任务判断。"""
|
||||
now = _now(now)
|
||||
report = report if isinstance(report, dict) else {}
|
||||
generated = _time(report.get('report_ran_at'))
|
||||
checked = _time(report.get('checked_at'))
|
||||
if not report:
|
||||
return False, '尚无买方评析报告', None
|
||||
if generated is None or generated > now:
|
||||
return False, '评析生成时间缺失或晚于当前时刻,待核实', None
|
||||
# 旧报告命中指纹时只更新 checked_at;不能把复核当新报告,也不能永远判旧。
|
||||
verified = max(t for t in (generated, checked) if t is not None and t <= now)
|
||||
if not report.get('brain_used') or report.get('brain_status') != 'ok':
|
||||
return False, '买方评议尚未完整完成', verified
|
||||
if not re.fullmatch(r'[0-9a-f]{64}', str(report.get('review_ref') or '')):
|
||||
return False, '报告缺少可核验的固定版本,等待重新归档', verified
|
||||
if not str(report.get('thesis') or '').strip():
|
||||
return False, '报告尚缺买方论点', verified
|
||||
if now - verified > dt.timedelta(days=fresh_days):
|
||||
return False, '评析需要重新核对资料', verified
|
||||
return True, None, verified
|
||||
|
||||
|
||||
def load_reports(codes, now=None):
|
||||
import sources
|
||||
now = _now(now)
|
||||
if not codes:
|
||||
return {}
|
||||
return sources.company_reviews(codes, now.astimezone(BJ).date().isoformat(),
|
||||
cutoff_at=now.isoformat(), strict=True, include_check=True)
|
||||
|
||||
|
||||
def _public(report, record, now, fresh_days):
|
||||
import card
|
||||
report = report if isinstance(report, dict) else None
|
||||
complete, reason, verified = report_state(report, now, fresh_days)
|
||||
status = (record or {}).get('status', 'missing')
|
||||
if status == 'completed' and not complete: # 上次完成不能掩盖这次缺报或陈旧。
|
||||
status = 'missing'
|
||||
if complete:
|
||||
status = 'completed'
|
||||
generated = _time((report or {}).get('report_ran_at'))
|
||||
error = None if complete else (record or {}).get('error') or reason
|
||||
return {'review_status': status, 'review_updated_at': (verified.isoformat() if complete and verified else (record or {}).get('updated_at')),
|
||||
'review_error': error,
|
||||
'review_data_date': generated.astimezone(BJ).date().isoformat() if generated and generated <= now else None,
|
||||
'company_review': report, 'company_review_text': card.company_review_view(report),
|
||||
'review_ref': (report or {}).get('review_ref')}
|
||||
|
||||
|
||||
def views(root, codes, *, now=None, reader=None):
|
||||
"""实时只读视图;既不落状态文件,也不触发请求或写研究目标表。"""
|
||||
import config
|
||||
import user_candidates as uc
|
||||
now = _now(now)
|
||||
safe = []
|
||||
for code in codes:
|
||||
try:
|
||||
safe.append(uc.normalize(code))
|
||||
except ValueError:
|
||||
continue
|
||||
safe = sorted(set(safe))
|
||||
state = read(root)
|
||||
try:
|
||||
reports = (reader or load_reports)(safe, now)
|
||||
except Exception:
|
||||
log.exception('人工关注评析读取失败')
|
||||
return {code: {'review_status': 'failed', 'review_updated_at': None,
|
||||
'review_error': '评析资料暂不可读取,请稍后核实', 'review_data_date': None,
|
||||
'company_review': None, 'company_review_text': '', 'review_ref': None}
|
||||
for code in safe}
|
||||
return {code: _public(reports.get(code), state['rows'].get(code), now,
|
||||
config.USER_RESEARCH_FRESH_DAYS) for code in safe}
|
||||
|
||||
|
||||
class _NoRedirect(urllib.request.HTTPRedirectHandler):
|
||||
def redirect_request(self, req, fp, code, msg, headers, newurl):
|
||||
raise urllib.error.HTTPError(req.full_url, code, '评析提交不接受跳转', headers, fp)
|
||||
|
||||
|
||||
def submit_review(code):
|
||||
"""唯一请求地址来自受管配置,代码参数无法改变域名、路径或强制重跑。"""
|
||||
import config
|
||||
import user_candidates as uc
|
||||
code = uc.normalize(code)
|
||||
base = urllib.parse.urlsplit(config.AKG_API_BASE)
|
||||
if (base.scheme not in ('http', 'https') or not base.hostname or base.username
|
||||
or base.password or base.query or base.fragment):
|
||||
raise ValueError('评析服务地址配置无效')
|
||||
endpoint = config.AKG_API_BASE.rstrip('/') + '/admin/company_review'
|
||||
query = urllib.parse.urlencode({'ts_code': code[2:] + '.' + code[:2],
|
||||
'force': 'false', 'sync': 'false',
|
||||
'with_brain': 'true', 'with_qual': 'true'})
|
||||
request = urllib.request.Request(endpoint + '?' + query, data=b'', method='POST')
|
||||
timeout = max(1.0, min(float(config.USER_RESEARCH_REQUEST_TIMEOUT), 10.0))
|
||||
opener = urllib.request.build_opener(_NoRedirect())
|
||||
with opener.open(request, timeout=timeout) as response:
|
||||
return json.loads(response.read(1024 * 1024).decode('utf-8'))
|
||||
|
||||
|
||||
def _targets(root, active, state, write=None):
|
||||
"""给原自然批任务补齐全部有效人工关注,不改变已发布计划内容。"""
|
||||
import review_targets
|
||||
if not active and not state.get('targets_key'):
|
||||
return
|
||||
day = pv.latest_date(root)
|
||||
if not day:
|
||||
return
|
||||
published = pv.load(root, day)
|
||||
key = pv.digest({'day': day, 'decision': published.get('decision'), 'active': active})
|
||||
if state.get('targets_key') == key:
|
||||
return
|
||||
review_targets.persist(day, published, write=write, user_rows=active)
|
||||
state['targets_key'] = key
|
||||
state.pop('targets_error', None)
|
||||
|
||||
|
||||
def sync(root=None, *, now=None, reader=None, submit=None, target_write=None):
|
||||
"""后台/自然调度入口。锁只护本地状态;网络前 claim 已经原子落盘。"""
|
||||
import config
|
||||
root = root or config.PLAN_SNAPSHOT_DIR
|
||||
now = _now(now)
|
||||
active = _active(root, now)
|
||||
codes = sorted({r['code'] for r in active})
|
||||
with _writer(root):
|
||||
state = read(root)
|
||||
try:
|
||||
_targets(root, active, state, target_write)
|
||||
except Exception:
|
||||
state['targets_error'] = '研究目标名单同步失败,后台会继续核对'
|
||||
log.exception('人工关注研究目标名单同步失败')
|
||||
_save(root, state)
|
||||
try:
|
||||
reports = (reader or load_reports)(codes, now)
|
||||
except Exception:
|
||||
log.exception('人工关注评析同步读取失败,本轮不投递')
|
||||
return {'status': 'failed', 'error': '评析资料暂不可读取,本轮未重复提交', 'submitted': 0}
|
||||
submitted = 0
|
||||
for code in codes:
|
||||
report = reports.get(code)
|
||||
complete, reason, verified = report_state(report, now, config.USER_RESEARCH_FRESH_DAYS)
|
||||
with _writer(root):
|
||||
state = read(root)
|
||||
previous = state['rows'].get(code, {})
|
||||
# 提交前再核对删除/到期,避免旧后台任务给已移除关注派新任务。
|
||||
if code not in {r['code'] for r in _active(root, now)}:
|
||||
continue
|
||||
if complete:
|
||||
state['rows'][code] = dict(previous, status='completed', error=None,
|
||||
updated_at=verified.isoformat(), observed_at=now.isoformat(),
|
||||
report_ref=report['review_ref'])
|
||||
_save(root, state)
|
||||
continue
|
||||
if previous.get('status') == 'completed' and (_time(previous.get('observed_at')) or dt.datetime.min.replace(tzinfo=UTC)) >= now:
|
||||
continue # 较早开始的并发读取不能把刚核实的完成状态反向改成待投递。
|
||||
claimed = _time(previous.get('claimed_at'))
|
||||
if previous.get('status') in ('pending', 'queued', 'failed') and claimed:
|
||||
# 已投过的旧无 ref/不完整报告,核对更新后仍不合格,明确留待补齐;不假造完成。
|
||||
if verified and verified > claimed and previous.get('status') != 'failed':
|
||||
state['rows'][code] = dict(previous, status='failed', error=reason,
|
||||
updated_at=now.isoformat())
|
||||
_save(root, state)
|
||||
continue
|
||||
if submitted >= max(1, config.USER_RESEARCH_SUBMIT_LIMIT):
|
||||
continue
|
||||
claim = uuid.uuid4().hex
|
||||
state['rows'][code] = dict(previous, status='pending', claim_id=claim,
|
||||
claimed_at=now.isoformat(), updated_at=now.isoformat(),
|
||||
error='评析请求正在受理,结果待核实')
|
||||
_save(root, state)
|
||||
submitted += 1
|
||||
status, error, task_id = 'pending', '评析请求的受理结果待核实,暂不重复提交', None
|
||||
try:
|
||||
result = (submit or submit_review)(code)
|
||||
if (isinstance(result, dict) and result.get('status') == 'queued'
|
||||
and isinstance(result.get('task_id'), str) and result['task_id'].strip()):
|
||||
status, error, task_id = 'queued', None, str(result['task_id'])
|
||||
elif isinstance(result, dict) and result.get('status') == 'done':
|
||||
result_status = (result.get('result') or {}).get('status')
|
||||
if result_status not in ('ok', 'skipped_fresh'):
|
||||
status, error = 'failed', '评析尚未生成完整报告,等待资料补齐'
|
||||
else:
|
||||
error = '评析已返回,等待核验固定报告'
|
||||
except urllib.error.HTTPError as exc:
|
||||
if 400 <= exc.code < 500:
|
||||
status, error = 'failed', '评析请求未被接受,请核对服务后重试'
|
||||
log.warning('人工关注评析请求失败 code=%s http=%s', code, exc.code)
|
||||
except ValueError:
|
||||
log.exception('人工关注评析响应或配置不可核验 code=%s', code)
|
||||
except Exception:
|
||||
log.exception('人工关注评析受理结果不明 code=%s', code)
|
||||
with _writer(root):
|
||||
state = read(root)
|
||||
previous = state['rows'].get(code, {})
|
||||
if previous.get('claim_id') == claim and previous.get('status') != 'completed':
|
||||
state['rows'][code] = dict(previous, status=status, error=error,
|
||||
task_id=task_id, updated_at=now.isoformat())
|
||||
_save(root, state)
|
||||
return {'status': 'checked', 'active': len(codes), 'submitted': submitted}
|
||||
Loading…
Reference in New Issue