Compare commits

...

15 Commits

44 changed files with 3573 additions and 148 deletions

View File

@ -120,3 +120,5 @@ MONGO_DB=stock_predictions
# 当晚 22:30 全量扫兜底)。key 填决策系统 .env 的 XXL_TRIGGER_KEY ---
# BIONIC_SCAN_URL=http://192.168.16.188:38000/api/v1/xxl/daily-scan
# BIONIC_SCAN_KEY=
# PLAN_RATING_PRIORITY_ENABLED=0 # 基座历史就绪后开启,同质地带内升优降后,30自然日

View File

@ -1,5 +1,7 @@
# akg-factor-bridge
2026年11月5日:补齐数据迟到后的有限自动恢复及构建、冻结的沪深范围保护。保持日期时效与真实版本时间,说明与实际发布状态见[恢复接续](docs/选股输入迟到恢复与沪深边界_2026-11-05.md)。
astock-kg(知识图谱基座)↔ `quant_factor_service`(通用因子平台)之间的**因子导出桥**。
把基座的四路产出(分析师预期空间 / 热度 / 利好利空事件 / 板块传导)做成符合平台规范的
子因子;**复合因子 `akg_score` 在桥内用自洽的景气度漏斗算出**,平台只负责注册/存储/
@ -183,7 +185,7 @@ docker compose exec akg-factor-bridge python run.py freeze --date 2026-07-24
`source_type` 不是这个字符串,事件因子会被过滤成空(会显式告警),按实际值改
`EVENT_SOURCE_TYPES`。
## HTTP 接口(桥常驻 API,端口 8300,全部只读除 refresh)
## HTTP 接口(桥常驻 API,端口 8300)
| 端点 | 作用 |
|---|---|
@ -191,6 +193,10 @@ docker compose exec akg-factor-bridge python run.py freeze --date 2026-07-24
| `GET /plan/dates?limit=30` | 有计划文件的日期清单 |
| `GET /plan?date=&format=json\|md&top=&obs_top=` | 当日选股计划(基座今日页 `/plan/today` 与 PMS 下单候选都读它)。顶层 `regime`(区制)与 `market`(两市成交额、广度、融资、恐贪四项)都只从当日快照读,快照缺失为空;每行的判决类字段 2026-09-03 起多 `basis`(判决依据)与 `logic`(因果论断带出处);2026-09-17 起名次改按质地带优先,每行多 `quality_band`(质地带:好 3 中 2 无评析 1 差 0)、`rank_axis`(quality/transmission)、`pointed`(是否被传导指向)、`rank_old`(旧序名次,供周报四周对照),开关 `PLAN_RANK_AXIS` 设 transmission 即逐字回旧 |
| `POST /plan/refresh?date=&key=` | 使用既有 XXL key 排入完整修订;晨版缺失不代造 |
| `GET /plan/user-candidates/search?q=` | 按名称或代码查找沪深 A 股 |
| `GET /plan/user-candidates` | 人工关注登记与实时评析状态;只读,不提交评析任务 |
| `POST /plan/user-candidates` | 经既有 `X-Job-Key` 鉴权登记人工关注;保存后由后台补齐评析并尝试计划修订 |
| `POST /plan/user-candidates/{id}/remove` | 经既有 `X-Job-Key` 鉴权移除关注,保留原登记记录 |
| `GET /plan/verdict?codes=300750,SH600438&date=` | 逐票『计划判决』(2026-08-18 PMS 对接加):decision(main/observe/reject/absent)+ 与命令行 `plan_reconcile` 逐字同口径的 verdict_text + score/rank/tier/赛道/图谱证据。单票数据异常或代码形态认不出只报该票 error,不崩整批 |
| `GET\|POST /api/v1/xxl/daily-build?key=&steps=&date=` | XXL-JOB 平台触发盘前链(build→plan→push-pool,白名单步骤、单实例互斥、回调结案);`.env` 不配 `XXL_TRIGGER_KEY` 则整组禁用 |
| `GET /review/latest` | 最新一份复盘周报的只读读数(2026-09-16 加,配合数据基座判收读数包):文件名、日期、大小、前二十行、三个标记行计数(`公司质地=`、`五日头`、`二十日头`)。没有周报回 404。纯文件读取,不连库 |
@ -198,6 +204,24 @@ docker compose exec akg-factor-bridge python run.py freeze --date 2026-07-24
逐票对账的命令行形态(同一段判决代码):
`docker compose exec -T akg-factor-bridge python plan_reconcile.py 300750 [--date 2026-08-04]`
## 人工关注评析与吸筹参考规则(2026-11-04)
用户已明确选择沿用公司质地排序,吸筹作为参考,建仓由 PMS 的基本面、技术面、择时三源合议及后续规则判断。本次仅授权在 **155 模拟链路**配置 `CARD_ACCUM_CONFIRM_REQUIRED=0`。代码默认仍为 `1`,保留旧确认规则的回退能力;188 真实 PMS 不在本次切换范围。实际部署、参数生效与自然回执另记发布判收,不能用本地测试代替上线结论。
关闭确认要求后,潜在吸筹、其他非明确状态、缺评分或评分陈旧不再单独把股票降为“仅展示”或“关注”;原评分、状态与新鲜度仍保留为研究证据。不新增吸筹权重或综合分,质地排序与原始分数编码保持。券商覆盖、预期空间、证券资格、既有真实风险否决、高位派发和快照时点校验继续生效;人工来源只替代原发现入口。PMS 的三源合议、规则闸、研判、资金、组合及执行约束仍决定后续动作,研究完成或进入候选均不等于下单授权。
`plan-refresh` 只将本次获准的吸筹确认参数同步到新修订,其他冻结参数继续沿用原版本。即使研究输入未变,规则变化也必须追加带真实截止时刻、新参数指纹及明确原因的完整版本。`card_params.admission_rule` 区分确认规则与参考规则,`card_params.accum_confirm_required` 表明当前要求。旧日期、固定决策版本与晨版文件保持原样,不重写历史判决,也不倒填晨版。
人工关注走独立的真实买方评析链。登记后的后台任务与自然 `run.py plan-refresh` 周期先同步评析状态;GET 查询只批量读取真实报告。全部有效沪深人工关注加入原 `t_akg_review_targets` 目标名单,不受主榜前列、原分数或交易判决限制。缺少完整报告时,后台仅向配置的 `AKG_API_BASE` 提交原个股评析入口,要求买方评议和定性材料,使用 `force=false`,复用基座既有评析逻辑。
请求前先原子保存独立受理凭据,实际返回入队回执后才标记 `queued`。网络超时或受理结果不明时保持 `pending`,不立即重复提交;后续根据真实报告核验,并由原目标名单的自然批任务补齐。状态存于 `user_research.json`,不修改交易账本或历史计划。每轮默认最多提交 3 只、单次请求超时 3 秒;超过本轮额度的对象留待后续自然周期。移除或到期后停止新增提交与下游关注覆盖。
`GET /plan/user-candidates` 中,原 `status` 的 `pending/evaluated` 仅表示计划是否收录。独立的 `review_status` 为 `missing/pending/queued/completed/failed`,并附 `review_updated_at`、`review_error`、`review_data_date`、`company_review`、`company_review_text` 与 `review_ref`。任务标识不作为产品状态展示。
评析完成必须有真实固定版本引用、完整成功的买方评议、买方论点及不晚于当前时刻的报告生成时间。新鲜度按基座真实 `ran_at` 与 `checked_at` 中较新者检查,默认 7 天;轻量复核命中原指纹时可以更新核对时间,不能把它改写为新报告。`review_data_date` 始终取 `company_review.report_ran_at` 的北京时间日期。财报期是否落后仍由基座原有批任务判断;旧报告缺固定引用或评议未完成时明确等待补齐,不能伪造完成状态。
开发机验证:新增评析专项 19 项通过,包含吸筹参考规则的 35 个测试文件均经配置隔离、禁网护栏通过。候选镜像断网回归、真实评析受理与完成、自然日研究及盘中观察回执仍须在发布阶段分别核验。规则决定见[复盘决定台账 084](docs/复盘决定台账.md)。
## 里程碑状态(设计 §8)
- **G2 赛道门槛 ✅**:`config/frontier_tracks.yml` + `tracks.py`,三路映射

82
api.py
View File

@ -20,7 +20,7 @@ import time
import threading
import pandas as pd
from fastapi import FastAPI, HTTPException, Request
from fastapi import FastAPI, HTTPException, Request, Body, Header, BackgroundTasks
from fastapi.responses import PlainTextResponse, JSONResponse, HTMLResponse
import card
@ -96,6 +96,58 @@ def dividend_candidate_feed(code: str = None):
raise HTTPException(status_code=503, detail="高股息资料暂不可用,请稍后再试") from exc
@app.get('/plan/user-candidates/search')
def search_user_candidates(q: str = ''):
import user_candidates as uc
query = q.strip().upper()[:40]
if not query:
return {'ok': True, 'items': []}
names = uc.catalog()
items = []
for code, name in names.items():
try:
uc.normalize(code)
except ValueError:
continue
if query in code or query in name.upper():
items.append({'code': code, 'name': name})
return {'ok': True, 'items': sorted(items, key=lambda x: x['code'])[:30]}
@app.get('/plan/user-candidates')
def list_user_candidates():
import user_candidates as uc
return uc.listing(config.PLAN_SNAPSHOT_DIR)
@app.post('/plan/user-candidates')
def add_user_candidates(background_tasks: BackgroundTasks, payload: dict = Body(...),
x_job_key: str = Header(None)):
from xxl import _require_key
import user_candidates as uc
_require_key(x_job_key, None)
try:
result = uc.save(config.PLAN_SNAPSHOT_DIR, payload, payload.get('actor') or 'PMS', names=uc.catalog())
except ValueError as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
background_tasks.add_task(uc.refresh_saved)
return result
@app.post('/plan/user-candidates/{identity}/remove')
def remove_user_candidate(identity: str, background_tasks: BackgroundTasks,
payload: dict = Body(...), x_job_key: str = Header(None)):
from xxl import _require_key
import user_candidates as uc
_require_key(x_job_key, None)
try:
result = uc.remove(config.PLAN_SNAPSHOT_DIR, identity, payload.get('actor') or 'PMS')
except ValueError as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
background_tasks.add_task(uc.refresh_saved)
return result
@app.get("/plan/dates")
def plan_dates(limit: int = 30):
from pathlib import Path
@ -128,12 +180,13 @@ _PLAN_CACHE_MAX = 8 # 不同参数组合最多留几份,防内存慢
_PLAN_BUILD_LOCKS: dict = {} # 每个缓存键一把「正在算」的锁,见 _plan_cached 里的说明
def _plan_build(date, top, obs_top, theme_cap, decision_id=None) -> dict:
def _plan_build(date, top, obs_top, theme_cap, decision_id=None, include_user_candidates=False) -> dict:
"""只读已发布版本。缺档明确报错,禁止用当前研究重构历史。"""
return plan.from_snapshot(date, top, obs_top, theme_cap, decision_id=decision_id)
return plan.from_snapshot(date, top, obs_top, theme_cap, decision_id=decision_id,
include_user_candidates=include_user_candidates)
def _plan_cached(date, top, obs_top, theme_cap, nocache: bool, decision_id=None):
def _plan_cached(date, top, obs_top, theme_cap, nocache: bool, decision_id=None, include_user_candidates=False):
"""返回 (data, 命中与否)。data 是 plan.collect 的结果(已去掉 _full)。
键里放**真实数据日**而不是请求里的 date:不传 date 时数据日随构建更新,明早换日后
@ -143,7 +196,10 @@ def _plan_cached(date, top, obs_top, theme_cap, nocache: bool, decision_id=None)
ds_key = published['date']
pinned = (published.get('decision') or {}).get('id')
state = pv.health(config.PLAN_SNAPSHOT_DIR, ds_key) if not decision_id else {}
key = (pinned or pv.digest(published), pv.digest(state), int(top), int(obs_top), int(theme_cap))
if not decision_id and not date:
import user_candidates as uc
state = dict(state, pending_codes=sorted(set(state.get('pending_codes') or []) | set(uc.live_pending(config.PLAN_SNAPSHOT_DIR, published))))
key = (pinned or pv.digest(published), pv.digest(state), int(top), int(obs_top), int(theme_cap), bool(include_user_candidates))
now = time.time()
if not nocache and PLAN_CACHE_SEC > 0:
with _PLAN_CACHE_LOCK:
@ -167,7 +223,8 @@ def _plan_cached(date, top, obs_top, theme_cap, nocache: bool, decision_id=None)
hit = _PLAN_CACHE.get(key)
if hit and time.time() - hit[0] < PLAN_CACHE_SEC:
return hit[1], True
data = _plan_build(ds_key, top, obs_top, theme_cap, pinned)
data = _plan_build(ds_key, top, obs_top, theme_cap, pinned,
**({'include_user_candidates': True} if include_user_candidates else {}))
data['refresh_status'] = state
if PLAN_CACHE_SEC > 0:
with _PLAN_CACHE_LOCK:
@ -184,14 +241,14 @@ def _plan_cached(date, top, obs_top, theme_cap, nocache: bool, decision_id=None)
@app.get("/plan")
def get_plan(request: Request, date: str | None = None, format: str = "json",
top: int = 20, obs_top: int = 10, theme_cap: int = 5, nocache: int = 0,
decision_id: str | None = None):
decision_id: str | None = None, include_user_candidates: bool = False):
"""向下兼容承诺(2026-09-02 方案 2.7):main / observe 的装配、排序、裁剪与既有字段
一字不动,每行只多联入判决类字段;顶层只新增 generated_at、plan_version、regime、
card_counts、candidates、watch、segments_pointed、snapshot,2026-09-03 再加 market
(环境段市场四项,与 regime 一样只从当日快照读,快照缺失为空字典)。PMS 按字段名取值、忽略未知键。"""
t0 = time.time()
try:
data, cached = _plan_cached(date, top, obs_top, theme_cap, bool(nocache), decision_id)
data, cached = _plan_cached(date, top, obs_top, theme_cap, bool(nocache), decision_id, include_user_candidates)
except (RuntimeError, ValueError, OSError) as e:
raise HTTPException(status_code=404, detail=str(e))
data = dict(data) # 下面几个键按请求写,不污染缓存里那一份
@ -216,11 +273,12 @@ def get_plan(request: Request, date: str | None = None, format: str = "json",
@app.get('/plan/check')
def check_plan_stock(code: str):
def check_plan_stock(code: str, include_user_candidates: bool = False):
"""采纳专用逐票读取。只读已发布计划,不装配、不查研究库。"""
from common import to_prefix
try:
return JSONResponse(content=plan_versions.stock_check(config.PLAN_SNAPSHOT_DIR, to_prefix(code)))
return JSONResponse(content=plan_versions.stock_check(config.PLAN_SNAPSHOT_DIR, to_prefix(code),
include_user_candidates=include_user_candidates))
except (OSError, ValueError, RuntimeError) as e:
raise HTTPException(status_code=503, detail=str(e))
@ -232,13 +290,15 @@ def refresh(date: str | None = None, key: str | None = None):
key=key, x_job_key=None)
def _published_stock_rows(want, date, decision_id):
def _published_stock_rows(want, date, decision_id, include_user_candidates=False):
try:
published = plan_versions.load(config.PLAN_SNAPSHOT_DIR, date, decision_id)
except (OSError, ValueError):
if decision_id or date:
raise HTTPException(status_code=404, detail='指定计划版本不可用')
return {}, {}
import user_candidates as uc
published = uc.consumer_view(published, include_user_candidates)
rows = {r['code']: dict(r, decision=bucket, source='published',
traceability=published.get('traceability') or 'legacy_unverified',
decision_version=published.get('decision'), date=published['date'],

View File

@ -4,6 +4,7 @@ from zoneinfo import ZoneInfo
import pandas as pd
import db
import market_scope
def expected_data_day(now=None):
@ -18,15 +19,24 @@ def expected_data_day(now=None):
def check(now=None):
"""数据未齐则在任何写入前失败;不能用缺失数据推断今天休市。"""
expected = expected_data_day(now)
# 日期索引先缩小到目标日及之后,再核验证券范围;不为找旧日期扫描全历史。
params = (expected.isoformat(),)
dates = {}
failures = []
for name, source, sql in (
('传导', 'pg', 'SELECT MAX(scan_date) d FROM v_factor_transmission'),
('收盘行情', 'price', 'SELECT MAX(`timestamp`) d FROM gp_day_data'),
('资金热度', 'heat', 'SELECT MAX(trade_date) d FROM stock_fund_heat_scores'),
('传导', 'pg', 'SELECT MAX(scan_date) d FROM v_factor_transmission WHERE scan_date >= %s AND '
+ market_scope.sql_filter('ts_code', postgres=True)),
('收盘行情', 'price', None),
('资金热度', 'heat', 'SELECT MAX(trade_date) d FROM stock_fund_heat_scores WHERE trade_date >= %s AND '
+ market_scope.sql_filter('stock_code')),
):
try:
frame = db.read_pg(sql) if source == 'pg' else db.read_mysql(source, sql)
if source == 'price':
import factors
column = factors._price_code_col()
sql = ('SELECT MAX(`timestamp`) d FROM gp_day_data WHERE `timestamp` >= %s AND '
+ market_scope.sql_filter('`' + column + '`'))
frame = db.read_pg(sql, params) if source == 'pg' else db.read_mysql(source, sql, params)
value = None if frame.empty else frame.iloc[0, 0]
day = None if value is None or pd.isna(value) else pd.Timestamp(value).date()
except Exception as exc:

39
card.py
View File

@ -1,5 +1,9 @@
"""候选卡:把一只票的各条证据线装配成判决与理由(纯函数,零 IO)。
2026-11-04新增参考模式:require_accum_confirm=False时,非明确、缺失和陈旧
吸筹不再单独否决候选。资格、真实风险和公司质地分流保留,建仓交PMS三源合议。
以下确认线描述为兼容旧模式,历史快照保持原判决。
## 为什么要有它
桥原来的产物是一个分数:主榜分 = 200 + 传导档位×20 + 组内分。分数排在最前的
@ -95,7 +99,7 @@ def judge(ev: dict[str, Any], *, start_pct: float = 3.0,
accum_max_age: int = 30, neg_tol: float = 0.0,
logic_stale_days: int = 90, require_started: bool = True,
company_review: dict | None = None, quality_gate: bool = True,
stale_days: int = 120) -> dict[str, Any]:
stale_days: int = 120, require_accum_confirm: bool = True) -> dict[str, Any]:
"""一只票的证据字典 -> 判决字典。
ev 里认的键(缺键按缺失处理,不报错):
@ -163,7 +167,7 @@ def judge(ev: dict[str, Any], *, start_pct: float = 3.0,
elif confirm_stale:
missing.append(f"吸筹评分是 {accum_age} 个交易日前的,太旧了"
if isinstance(accum_age, int) else "吸筹评分有多旧算不出来")
elif confirm_negative and "派发" not in accum_state:
elif require_accum_confirm and confirm_negative and "派发" not in accum_state:
state_short = accum_state.split("·")[0].strip()
missing.append(f"吸筹只到「{state_short}」,没到明确吸筹这条线")
if not gates["pointed"]:
@ -188,17 +192,33 @@ def judge(ev: dict[str, Any], *, start_pct: float = 3.0,
if confirm:
reasons.append(f"明确吸筹(评分 {accum_score:.0f},{accum_age} 日前)"
if accum_score is not None else f"明确吸筹({accum_age} 日前)")
elif not require_accum_confirm and accum_state:
detail = []
if accum_score is not None:
detail.append(f"评分 {accum_score:.0f}")
if isinstance(accum_age, int):
detail.append(f"{accum_age} 个交易日前")
reasons.append("吸筹状态:" + accum_state + ("(" + ",".join(detail) + ")" if detail else ""))
if gates["covered"]:
reasons.append(f"券商覆盖,预期空间 {upside:+.0%}")
all_gates = all(gates.values())
failed = [k for k, ok in gates.items() if not ok]
# 人工来源替代发现条件;pointed 仍保留真实传导状态,不改变排序和其他门槛。
user_selected = bool(ev.get('user_selected'))
if user_selected and not gates['pointed']:
missing = [x for x in missing if x != '所在环节当日没有被传导指向']
reasons.insert(0, '用户添加,按现有条件评估')
effective = dict(gates, pointed=gates['pointed'] or user_selected)
all_gates = all(effective.values())
failed = [k for k, ok in effective.items() if not ok]
# 判决依据是这张卡上唯一一句"为什么",PMS 原样显示在页面上给人读。所以它必须是
# 一句完整的中文,带上具体数字,不许出现门槛的变量名(started 这类)或信号的
# 枚举值(SELL 这类)——那些是系统之间传递用的,不是给人看的。
if risk:
verdict = VERDICT_SHOW
basis = ";".join(risk)
elif all_gates and not require_accum_confirm:
verdict = VERDICT_CANDIDATE
basis = "选股资格通过,进入基本面、技术面和择时合议"
elif all_gates and confirm:
verdict = VERDICT_CANDIDATE
basis = (f"三条门槛都过了,没有风险,且明确吸筹({accum_age} 个交易日前评的)"
@ -230,11 +250,13 @@ def judge(ev: dict[str, Any], *, start_pct: float = 3.0,
if quality_gate and cr and qband == 0 and verdict == VERDICT_CANDIDATE:
verdict = VERDICT_WATCH
why_q = ";".join((cr.get("quality_reasons") or [])[:2]) or "基座代码合成的质地档为差"
company_gate = f"公司质地评为差({why_q}),候选降为关注交人裁决"
company_gate = (f"公司质地评为差({why_q}),候选降为关注交人裁决" if require_accum_confirm else
f"公司质地评为差({why_q}),按PMS现行合议和自决档处理关注候选")
basis = f"{basis};{company_gate}"
return {"verdict": verdict, "reasons": reasons, "missing": missing, "risk": risk,
"gates": gates, "confirm": confirm, "failed_gates": failed, "basis": basis,
"gates": gates, "confirm": confirm, "confirm_required": bool(require_accum_confirm),
"failed_gates": failed, "basis": basis,
"company_gate": company_gate, "quality_band": qband,
"logic": logic_lines(ev.get("logic")),
"logic_quality": logic_quality(ev.get("logic"))}
@ -268,7 +290,7 @@ def logic_lines(claims, limit: int = 3) -> list[str]:
return out
def sort_key(row: dict[str, Any], sort_by_quality=None) -> tuple:
def sort_key(row: dict[str, Any], sort_by_quality=None, rating_enabled=False) -> tuple:
"""卡内序:先判决(候选 < 关注 < 仅展示),2026-09-17 起在判决与涨幅之间插入质地带降序
(开关 CARD_SORT_BY_QUALITY,默认开),最后按数据日涨幅降序。开关关掉时元组与旧实现
(先判决再涨幅)逐字相同,逐字回旧。不算总分。"""
@ -277,6 +299,9 @@ def sort_key(row: dict[str, Any], sort_by_quality=None) -> tuple:
pct0 = _num(row.get("pct0"))
pct_key = -(pct0 if pct0 is not None else -1e9)
if config.CARD_SORT_BY_QUALITY if sort_by_quality is None else sort_by_quality:
if rating_enabled:
import rating_priority
return (v, -int(row.get("quality_band") or 0), -rating_priority.key(row.get('company_review')), pct_key)
return (v, -int(row.get("quality_band") or 0), pct_key)
return (v, pct_key)

View File

@ -7,6 +7,7 @@ import pandas as pd
import config
import db
import market_scope as scope
def to_prefix(ts_code: str) -> str:
@ -24,8 +25,9 @@ def load_universe() -> set[str]:
⚠️ 这是**当前态**:industry_pools 只存最新快照、每周一 refresh_pools 自动长大。
用它过滤历史数据会引入成员性前视——见 universe_filter 与 config.SUBFACTOR_UNIVERSE。
"""
df = db.read_pg("SELECT ts_code FROM v_factor_universe")
return set(df["ts_code"].astype(str).str.strip())
df = db.read_pg("SELECT ts_code FROM v_factor_universe WHERE " + scope.sql_filter("ts_code", postgres=True))
df = scope.frame(df, "ts_code")
return set(df["ts_code"])
def universe_filter(df: pd.DataFrame, col: str = "stock_code",
@ -41,6 +43,9 @@ def universe_filter(df: pd.DataFrame, col: str = "stock_code",
同时截面统计量会在每周一发生结构性跳变,IC 序列带周期性伪影。
子因子表是「可独立观察的仪表」,过滤该发生在消费端。
"""
df = scope.frame(df, col, prefix=as_prefix)
if df is None or df.empty:
return df
if config.SUBFACTOR_UNIVERSE == "pool":
uni = load_universe()
if as_prefix:
@ -57,10 +62,12 @@ def trading_days(start: str, end: str) -> list:
拿到空日历 → 静默返回空 → 只打印「无数据(跳过)」,不报错。
改用行情表 gp_day_data(实测 5584 天,覆盖全历史)。
"""
import factors
column = factors._price_code_col()
df = db.read_mysql(
"price",
"SELECT DISTINCT `timestamp` AS trade_date FROM gp_day_data "
"WHERE `timestamp` BETWEEN %s AND %s ORDER BY `timestamp`",
"WHERE `timestamp` BETWEEN %s AND %s AND " + scope.sql_filter(f"`{column}`") + " ORDER BY `timestamp`",
(start, end))
if df.empty:
print(f" ⚠️ 交易日历为空(gp_day_data 在 {start}~{end} 无数据)"
@ -98,6 +105,7 @@ def write_factor(table: str, df: pd.DataFrame, mode: str = "daily") -> None:
if df is None or df.empty:
print(f" {table}: 无数据(跳过)")
return
df = scope.frame(df, "stock_code", prefix=True)
df = df.dropna(subset=["trade_date", "stock_code", "factor_value"]).copy()
df["stock_code"] = df["stock_code"].map(to_prefix)
df["trade_date"] = pd.to_datetime(df["trade_date"]).dt.date
@ -112,7 +120,7 @@ def write_factor(table: str, df: pd.DataFrame, mode: str = "daily") -> None:
chunk = max(1, config.WRITE_CHUNK_ROWS)
with db.factor_conn() as conn:
with conn.cursor() as cur:
cur.execute(f"DELETE FROM {table} WHERE trade_date BETWEEN %s AND %s",
cur.execute(f"DELETE FROM {table} WHERE trade_date BETWEEN %s AND %s AND " + scope.sql_filter("stock_code"),
(dmin, dmax))
conn.commit()
for i in range(0, len(rows), chunk):

View File

@ -183,6 +183,11 @@ XXL_TRIGGER_KEY = os.environ.get("XXL_TRIGGER_KEY", "")
XXL_CALLBACK_READ_TIMEOUT = float(os.environ.get("XXL_CALLBACK_READ_TIMEOUT", "10"))
XXL_CALLBACK_MAX_RETRIES = int(os.environ.get("XXL_CALLBACK_MAX_RETRIES", "3"))
# 自然修订缺今日基础版时的有界恢复;只构建因子/计划,不入池或额外投递模型。
# 安全上限为每日三次实际构建、相邻至少十五分钟;可停用或收紧,不能放宽。
PLAN_RECOVERY_MAX_ATTEMPTS = int(os.environ.get("PLAN_RECOVERY_MAX_ATTEMPTS", "3"))
PLAN_RECOVERY_COOLDOWN_SECONDS = int(os.environ.get("PLAN_RECOVERY_COOLDOWN_SECONDS", "900"))
# --- 选股计划入池(08-03 定稿;规则与流程见 docs/选股计划入池_对接说明.md)------
# 写 Mongo stock_groups 的独立分组,决策系统每晚扫描按分组并集覆盖 → 候选票自动
# 获得夜间推理。入池范围与 PMS 候选**次序口径**一致:先筛强传导、再按绝对得分截断
@ -224,6 +229,9 @@ CARD_START_PCT = float(os.environ.get("CARD_START_PCT", "3")) # 门槛二
# 想开回去就把这个值设回 1 并重建容器,判决当天即恢复原样。
CARD_REQUIRE_STARTED = os.environ.get("CARD_REQUIRE_STARTED", "1") == "1"
CARD_ACCUM_MAX_AGE = int(os.environ.get("CARD_ACCUM_MAX_AGE", "30")) # 确认线:吸筹评分日龄上限(交易日)
# 2026-11-04:155显式关闭旧吸筹准入确认,吸筹转为三源合议的参考。
# 默认保留旧模式以兼容其他调用方;切换必须发布新计划版本,不能改历史快照。
CARD_ACCUM_CONFIRM_REQUIRED = os.environ.get("CARD_ACCUM_CONFIRM_REQUIRED", "1").lower() not in ("0", "false", "no")
# 计划快照落点(容器内路径;仓库根挂在 /app,故默认落在仓库 data/plan/)。
# 每日 JSON 含主榜与观察档全部行与全部证据线,是复盘与对账的唯一底本。
PLAN_SNAPSHOT_DIR = os.environ.get("PLAN_SNAPSHOT_DIR", "data/plan")
@ -318,8 +326,15 @@ LOGIC_PATH_COMPANY = os.environ.get("LOGIC_PATH_COMPANY", "1") == "1"
CARD_COMPANY_QUALITY_GATE = os.environ.get("CARD_COMPANY_QUALITY_GATE", "1") == "1" # 候选卡"质地差降一档"
COMPANY_REVIEW_STALE_DAYS = int(os.environ.get("COMPANY_REVIEW_STALE_DAYS", "120")) # 报告运行日超过它算陈旧
AKG_API_BASE = os.environ.get("AKG_API_BASE", "http://192.168.16.178:8000") # 报告链接前缀
USER_RESEARCH_FRESH_DAYS = int(os.environ.get("USER_RESEARCH_FRESH_DAYS", "7")) # 基座核对时刻口径,不把复核当新报告
USER_RESEARCH_REQUEST_TIMEOUT = float(os.environ.get("USER_RESEARCH_REQUEST_TIMEOUT", "3"))
USER_RESEARCH_SUBMIT_LIMIT = int(os.environ.get("USER_RESEARCH_SUBMIT_LIMIT", "3")) # 每轮有界受理,后续自然调度继续
LOGIC_SETTLE_CONFIRM_DAYS = int(os.environ.get("LOGIC_SETTLE_CONFIRM_DAYS", "2"))
LOGIC_SETTLE_EXIT_DAYS = int(os.environ.get("LOGIC_SETTLE_EXIT_DAYS", "3"))
# 读上一次落定态与近几日原始态时往前看几个自然日。要盖住国庆这种连休八天再加两个周末的空档;
# 超过这个窗口没记过行的票按第一次见到处理,落定态等于原始态。
LOGIC_STATE_LOOKBACK_DAYS = int(os.environ.get("LOGIC_STATE_LOOKBACK_DAYS", "30"))
# 基座评级历史就绪后开启;保留候选,只在同质地带内改变优先级。
PLAN_RATING_PRIORITY_ENABLED = os.environ.get("PLAN_RATING_PRIORITY_ENABLED", "0") == "1"
PLAN_RATING_PRIORITY_DAYS = 30

View File

@ -0,0 +1,171 @@
{
"verified_at": "2026-11-05T10:19:46.116833+08:00",
"host": "155 simulation",
"product_commits": [
"475bfec480b4f27cca43337981f60adde88d9630",
"9f8224e89bec8887ac11a963161fc3398a20e3cc"
],
"deployment": {
"verified_at": "2026-11-05T02:18:44.142605+00:00",
"target": {
"akg_factor_bridge": {
"id": "f6cfc7a6c9acf7421193cc0eab93ca6618096d987e42ff5e0134ee02cc70f7bd",
"image": "sha256:c3c87ffa8ee7087f41d9cd0192ff9c86bdc37fb814da717115169c0d62f76496",
"tag": "bridge:plan-recovery-9f8224e",
"started": "2026-11-05T02:18:22.104228056Z"
}
},
"other_unchanged": 17,
"other_changed": [],
"environment_unchanged": true,
"old_history_unchanged": true,
"original_source_unchanged": true
},
"recovery": {
"attempts": 1,
"completed_at": "2026-11-05T02:08:24.634400+00:00",
"data_date": "2026-11-04",
"decision_date": "2026-11-05",
"error": null,
"last_attempt_at": "2026-11-05T02:05:36.721028+00:00",
"schema_version": 1,
"status": "recovered"
},
"first_decision": {
"ancestor_ids": [],
"cutoff_at": "2026-11-05T02:07:00.338807+00:00",
"id": "2026-11-04_bc51f887aea763cfd804a2c96be700dc3d7b3e4cc8be0e0fbe9c413668b832de",
"input_hash": "b688a65e553cdb93341058cd4c3bfe6778d3c91ee82e6c0cd9fdf9f0b0d6c3a9",
"params_hash": "0f97153c6267739c7d79524f343edbf4231db2cafec83bb2434466687f9b881b",
"parent_id": null,
"publication_kind": "recovery",
"reason": "数据恢复后补发"
},
"first_morning_eligible": false,
"freeze": {
"files": 13,
"all_counts_match": true,
"errors": [],
"warnings": 0,
"scope_error_sum": 0
},
"runtime": {
"PLAN_RANK_AXIS": "quality",
"CARD_SORT_BY_QUALITY": true,
"CARD_ACCUM_CONFIRM_REQUIRED": false,
"PLAN_RECOVERY_MAX_ATTEMPTS": 3,
"PLAN_RECOVERY_COOLDOWN_SECONDS": 900
},
"readiness_seconds": 0.441,
"pms": {
"read_only": true,
"new_process": true,
"checked_at": "2026-11-05T10:19:46.116833+08:00",
"plan": {
"date": "2026-11-04",
"age_tdays": 1,
"heat_date": "2026-11-04",
"market_snapshot_days": [
"2026-11-04"
],
"rank_axis": "quality",
"returned": {
"main": 300,
"observe": 100
},
"decision": {
"ancestor_ids": [
"2026-11-04_bc51f887aea763cfd804a2c96be700dc3d7b3e4cc8be0e0fbe9c413668b832de"
],
"cutoff_at": "2026-11-05T02:14:31.303061+00:00",
"id": "2026-11-04_b372b5615c49ec496bece5c27f355b8b30cabea24b38212c3a884458bcf5d88a",
"input_hash": "c64eb8819705415fe0e5cc8303f6d6abd10ba5ed9c87d0abb690bb30cc28bb4a",
"params_hash": "0f97153c6267739c7d79524f343edbf4231db2cafec83bb2434466687f9b881b",
"parent_id": "2026-11-04_bc51f887aea763cfd804a2c96be700dc3d7b3e4cc8be0e0fbe9c413668b832de",
"publication_kind": "revision",
"reason": "评析更新"
},
"refresh_status": {
"checked_at": "2026-11-05T02:15:14+00:00",
"decision_id": "2026-11-04_b372b5615c49ec496bece5c27f355b8b30cabea24b38212c3a884458bcf5d88a",
"error": null,
"pending_codes": []
}
},
"freshness": {
"accepted": true,
"limit_tdays": 1,
"fetch_and_parse_seconds": 1.129
},
"candidates": {
"considered": 300,
"eligible": 86,
"dropped": {
"held": 1,
"black": 0,
"tier": 0,
"score": 0,
"sources": 0,
"upside": 0,
"st": 0,
"theme": 0,
"dup": 0,
"capped": 36,
"display": 213
},
"rank_mode": "upstream",
"route_by_verdict": true,
"selected": 50,
"held_excluded_context": 1,
"black_excluded_context": 0,
"verdict_counts": {
"候选": 38,
"关注": 12
},
"outside_scope": 0
},
"manual": {
"active": 6,
"review_status": {
"completed": 6
},
"report_dates": {
"2026-11-04": 6
},
"coverage_notes": {
"ok": 6
},
"stages": {
"daily": {
"COMPLETE": 6
},
"intraday": {
"UNAVAILABLE": 3,
"COMPLETE": 3
}
},
"stage_dates": {
"daily": {
"2026-11-05 / 2026-11-04": 6
},
"intraday": {
"2026-11-05 / 2026-11-05": 4,
"2026-11-05 / None": 2
}
},
"unavailable_reasons": {
"模型没有返回合法研究结论": 1,
"TimeoutError: ": 2
},
"review_errors": 0,
"last_analysis_errors": 0
},
"table_prefix": ""
},
"natural_acceptance": {
"recovery_task": "f5a16fb3",
"manual_request_unchanged": "5b181fe6",
"regime_append_completed": "3d52f1c0"
},
"offline_suites": 38
}

View File

@ -0,0 +1,335 @@
{
"scope": "178基座五项服务与155选股、PMS web及worker。188未操作。",
"timezone": "Asia/Shanghai; raw ISO timestamps retain their explicit offsets",
"products": {
"akg": "972c9342db8c80c7530556caa68237a3011b6033",
"bridge": "0945fa2b29835886ddfdcb3ab00a088e6dafa721",
"pms": "38f91d2bddd27ff2727c2209df687a5a73278bd5"
},
"candidate_tests": {
"akg": {
"count": 318,
"network": "none",
"passed": true
},
"bridge": {
"groups": 39,
"network": "none",
"passed": true
},
"pms": {
"groups": 78,
"network": "none",
"passed": true
},
"postgres": {
"passed": true,
"checks": [
"concurrent_duplicate_once",
"per_stock_lock_serializes",
"concurrent_changes_preserve_chain",
"unchanged_grade_keeps_window",
"old_and_conflicting_refs_rejected",
"index_failure_rolls_back_history",
"history_failure_preserves_index",
"migration_idempotent_baseline_not_upgrade",
"duplicate_keeps_report_time"
],
"network": "none",
"database": "isolated temporary PostgreSQL",
"history_rows": 6
}
},
"deployment_178": {
"verified_at": "2026-11-05T05:46:27.104237+00:00",
"image": "sha256:143bfba15a1517cabba4b44f55aebcfef3121c3361622636c29603d3dd1de14e",
"other_unchanged": 7,
"targets": {
"akg-backend": {
"id": "c463ee28e817cf405afb8b2c5ee26bfb9a56e94b2ff5a4726e2598c9cfad4a89",
"image": "sha256:143bfba15a1517cabba4b44f55aebcfef3121c3361622636c29603d3dd1de14e",
"started": "2026-11-05T05:46:27.800874841Z"
},
"astock-kg-worker-targeted-1": {
"id": "27c85447dbdbe5c90750002a8a66ab9229f52bd04e55ce92b69a6d51da5efd82",
"image": "sha256:143bfba15a1517cabba4b44f55aebcfef3121c3361622636c29603d3dd1de14e",
"started": "2026-11-05T05:46:27.813174746Z"
},
"astock-kg-worker-review-1": {
"id": "3e8cc55bd11dbef799bc287bfb002dadd0a4d3e4c3052c0e3a58a1f8fa4f20f9",
"image": "sha256:143bfba15a1517cabba4b44f55aebcfef3121c3361622636c29603d3dd1de14e",
"started": "2026-11-05T05:46:27.797196496Z"
},
"astock-kg-worker-interactive-1": {
"id": "46980f04d3c17ccae699f5f17686dd607cce30a05d34a23c84978cf83c1a6ee9",
"image": "sha256:143bfba15a1517cabba4b44f55aebcfef3121c3361622636c29603d3dd1de14e",
"started": "2026-11-05T05:46:27.80914945Z"
},
"astock-kg-worker-2": {
"id": "2da4aec205d864d7ea9be1064b7a4fbfa78cb9e692a49c8045f4eef4cec0303d",
"image": "sha256:143bfba15a1517cabba4b44f55aebcfef3121c3361622636c29603d3dd1de14e",
"started": "2026-11-05T05:46:27.805032719Z"
}
}
},
"deployment_155": {
"at": "2026-11-05T05:58:52.284979+00:00",
"other_unchanged": 15,
"targets": {
"akg_factor_bridge": {
"id": "77dab496e24f4887eb8d1e1a3dc6358e1b67b1d7f56d6d6b6ea6279c07d85744",
"image": "sha256:702fcfaaaa2ba5a03baf7061b848d70e6a56be6d84fa3a653f48e77d6ebe55dc",
"started": "2026-11-05T05:51:21.977552868Z"
},
"pms-worker": {
"id": "9b3d5e24bb60ae32d3b3554d735c0ceeae8b633227cc05c42b709bd8b5e4a0de",
"image": "sha256:ba3ca2ea3d6a6b1eeb69570cc1f951ebdbbfdc51e2e64767b14ae9806881e5d6",
"started": "2026-11-05T05:49:21.719216493Z"
},
"pms-web": {
"id": "25cdb34ff1654f97f6dd5a5a8657277c2591a48c235033f4125e4f6c02468a86",
"image": "sha256:ba3ca2ea3d6a6b1eeb69570cc1f951ebdbbfdc51e2e64767b14ae9806881e5d6",
"started": "2026-11-05T05:49:21.721777241Z"
}
},
"history_unchanged": true
},
"runtime_178": {
"source_matches": true,
"company_review_registered": true,
"health": {
"status": "ok",
"service": "akg-backend"
}
},
"baseline_initialization": {
"created": 3764,
"existing": 0,
"invalid": 134
},
"natural_history_after": {
"counts": {
"baseline": 3768
},
"latest_recorded_at": "2026-11-05 05:57:02.685585+00:00"
},
"natural_plan": {
"at": "2026-11-05T05:57:36.001968+00:00",
"enabled": true,
"days": 30,
"axis": "quality",
"accum_confirm": false,
"date": "2026-11-04",
"decision": {
"cutoff_at": "2026-11-05T05:55:02.267061+00:00",
"id": "2026-11-04_19549ac407bb1dd73133a66fc6f9db0675cc62a2eea1749c1d96cb4b5d4f99df",
"input_hash": "d8a3d995861fbdd2cd0fbf43099a3a4f0e9120b83c31d2d2cce75cdd5d881a47",
"params_hash": "26553346a3d0f980056059cbc9bbd1c6d43dccc654c88d326e9856e701a14718",
"parent_id": "2026-11-04_81eb846be363fa5ea6c1088b82b76f608bd40121c184a5345c4f3c3e2cda34d5",
"publication_kind": "revision",
"reason": "启用评级变动三十日排序"
},
"published_policy": {
"days": 30,
"enabled": true
},
"params_enabled": true,
"priority_counts": {
"baseline": 1193,
"missing_history": 33
},
"metadata_count": 1226,
"rows": 1226,
"ordered_quality_priority": true,
"health": {
"checked_at": "2026-11-05T05:55:34+00:00",
"decision_id": "2026-11-04_19549ac407bb1dd73133a66fc6f9db0675cc62a2eea1749c1d96cb4b5d4f99df",
"error": null,
"pending_codes": []
},
"candidate_count": 39,
"watch_count": 50,
"manual_count": 8,
"raw_scores_hash": "f5667f5909df6b8bb50f44ffd6b7ab43ec6e0c6594fef11857897f48d47b4936"
},
"natural_pms_consumption": {
"id": 616,
"at": "2026-11-05 13:57:08",
"kind": "consumed",
"decision": "2026-11-04_19549ac407bb1dd73133a66fc6f9db0675cc62a2eea1749c1d96cb4b5d4f99df",
"rank_mode": "upstream",
"rows": 50,
"with_rating_metadata": 50,
"priority_states": {
"baseline": 50
},
"research_ranks": [
1,
2,
4,
5,
6,
7,
9,
38,
39,
40,
41,
42,
43,
44,
45,
47,
48,
49,
50,
51,
52,
53,
54,
56,
57,
58,
59,
60,
61,
62,
63,
64,
65,
68,
180,
181,
182,
184,
185,
186,
187,
189,
192,
193,
198,
199,
200,
203,
204,
208
],
"same_order_as_pre_switch": true
},
"pms_health": {
"finished_at": "2026-11-05T13:58:28.253674+08:00",
"guards": {
"blocked_calls": 0
},
"identity": {
"table_prefix": "",
"table_prefix_empty": true
},
"parameters": {
"effective": {
"PMS_CONSENSUS_ROUTE": true,
"PMS_DISPATCH_MODE": "ws",
"PMS_EXEC_REPRICE_ENABLED": true,
"PMS_GLOBAL_BUY_HALT": false,
"PMS_GLOBAL_EXEC_HALT": false,
"PMS_JUDGE_ASYNC_ENABLED": true,
"PMS_MONITOR_WATCH_SCOPE": "pms-sim-155",
"PMS_MONITOR_WATCH_TTL_SEC": 600,
"PMS_QMT_WS_ENABLED": true,
"PMS_RECON_SOURCE": "ws_only",
"PMS_RECON_WS_SNAPSHOT_MAX_AGE_SEC": 900,
"PMS_ROTATION_CONTROL": "run",
"PMS_ROTATION_MODE": "auto",
"PMS_ROTATION_RULE_VERSION": "micro50-v3-20261103",
"PMS_SELF_DECIDE": "full",
"PMS_SELF_DECIDE_DAILY_MAX": 3,
"PMS_SELF_DECIDE_TRIAL": true
},
"read_ok": true,
"stored_keys": [
"PMS_DISPATCH_MODE",
"PMS_QMT_WS_ENABLED",
"PMS_QMT_WS_URL",
"PMS_GLOBAL_BUY_HALT",
"PMS_GLOBAL_EXEC_HALT",
"PMS_SELF_DECIDE",
"PMS_SELF_DECIDE_TRIAL",
"PMS_MONITOR_WATCH_SCOPE",
"PMS_EXEC_REPRICE_ENABLED",
"PMS_JUDGE_ASYNC_ENABLED",
"PMS_ROTATION_MODE",
"PMS_ROTATION_CONTROL",
"PMS_ROTATION_RULE_VERSION",
"PMS_RECON_SOURCE"
]
},
"probe": "pms-correction-readonly-v1",
"qmt": {
"ack_equals_persisted_watermark": true,
"acked_seq": 1643413,
"conn_state": "ONLINE",
"heartbeat_age_sec": -1.4,
"last_error_present": false,
"last_seq": 1643413,
"orphan_count": 0,
"pending_ledger_count": 0,
"resync_flag": false,
"server_seq": 1641994,
"stat_safe": {
"ack_seq_degraded": null,
"gap_skips": null,
"last_rx_at": "13:58:25",
"order_update_conflicts": null,
"pending_gap": 0,
"reconnects": 0,
"seq_hwm": 1643413,
"sig_invalid": null
},
"watermark_window_contiguous": true,
"watermark_window_expected": 200,
"watermark_window_rows": 200,
"window_scope": "最多200条,不能据此证明全部历史连续"
},
"readonly": true,
"reconciliation": {
"accounting_ready": true,
"cash_difference_known": true,
"cash_matches_existing_settlement_tolerance": true,
"cash_ready": true,
"fix_count": 0,
"quantity_diff_count": 0,
"quantity_error_count": 0,
"quantity_ok": true,
"quantity_source": "ws",
"valuation_missing_count": 0,
"valuation_ready": true
},
"sampled_at": "2026-11-05T13:58:26.624510+08:00",
"snapshots": {
"funds_age_sec": 5.1,
"funds_available_known": true,
"funds_frozen_known": true,
"funds_present": true,
"positions_age_sec": 5.1,
"positions_present": true,
"positions_qty_column_known": true,
"positions_row_count": 5
}
},
"not_naturally_verified": [
"实际评级上调或下调后的排序变化",
"真实30天窗口届满后的修订"
],
"evidence_sha256": {
"178-full-candidate.log": "681a04f55a9e6ace821bb57fc6970f89dc98cb2ca8be2a2e70d1ab57c3e7244d",
"178-pg-candidate.json": "e963f9a25f6cd1b54d89311cf92db58fbc4e1683084cccbf382347e031525c5a",
"178-baseline-init.json": "d389589e58757c1aa04b409c7b513544ad9843a9c42be7c78337d5126889ad1f",
"178-runtime.json": "c34fbac94f7fc315b8c63621b0f24a128e121330743879b7ee34f3e9c544219d",
"155-verified-final.json": "db695e73d322304c340c88df9347556ebfac6ce1fb299dda81c637504af98d75",
"bridge-plan-after.json": "dbe6d630a6e114c10f7b736b11578f5808f4bde974713e78c88293eee52b1804",
"pms-consumption-after.json": "5d857e908db056c00e2b3ff82c5dbdec664dbf28fd3679359710a140daa72ee9",
"pms-health-final.json": "9316edc3d45beb90a064d29be4f417688a93173918ed9216ebf754b26c24bebc",
"history-summary-after.json": "c06302e32f5efa614d71464e4ecffbf03e4cbeb45e6b4aa4ded12652ed8d3483"
}
}

View File

@ -0,0 +1,103 @@
# 个股评析全池覆盖开发规格
日期为2026年10月28日。本文时间均为北京时间。
## 目标与授权
用户要求本轮覆盖整个产业池和高股息池。
当前候选优先,未完成对象不因每日名单变化而丢失。
本轮补普通个股评析,不自动扩大分红专项研究。
股票资格、排序、模型判据和交易参数保持。
## 改前实测
178基座提交7ef107e,评析队列为零。
当前目标148家,全部已有报告、定性及评议。
产业池3833家,仅747家有报告。
桥高股息池169家,仅9家有报告。
两个池有交集,缺口不能直接相加。
本轮只读预览并集3885家,已有747家,首次报告缺口3138家。
高股息缺报告的160家均未进入留存的评析目标名单。
近72小时43次成功运行平均185.5秒。
第95百分位246.4秒,最大286.5秒。
近七天评议平均输出13465个token。
原每日上限为评议150家、定性200家、评议输出300万token。
## 具体工作
1. 复用原每小时批任务和review队列,不新增任务或表。
2. 自动读取全部产业池成员。
3. 从桥已发布的高股息研究接口读取研究对象。
不按投资资格eligible过滤,基座不重新计算股息筛选。
4. 在现有job_runs保存累计研究名单,名单变化才追加快照。
已纳入对象继续保留。来源读取失败保留旧名单并报告。
旧快照读失败时停止扩展投递,不能用空名单覆盖。
5. 新报告优先于名单外旧报告升级。
当前候选保留优先名额,剩余名额持续补名单外对象。
旧报告的复核不能无限挤占首次报告。
6. 每批完整运行最多30家。每日评议、定性各600家。
评议输出上限900万token,单次模型预算保持。
每日额度不足时不继续投递完整运行。
7. 用Redis短期凭据防止重复投递。
失败对象延后重试,队列积压时暂停增加整批任务。
重试不删除旧报告,不清空历史记录。
8. 名单外补齐避开06:00至08:00、20:00至23:00。
原候选任务仍按原调度执行。
晚间行业催化与分红专项使用原队列和时刻。
9. 日检分别显示有报告、定性、评议及首次报告缺口。
无财务数据、材料薄和失败不能宣称完整研究已完成。
## 偏差发现与撤销
新增配置文件控制覆盖开关和每日额度。
关闭覆盖开关即回到原目标集合与每日额度。
旧报告、固定引用和累计名单保留。
日检报告来源失败、剩余缺口和最近调度状态。
同一对象反复失败时先暂停其重试,不阻塞后面的对象。
按每日有效完成约450至550家估算,首次覆盖约需一周。
这只是按现有耗时计算的容量预估。
上游缺财务数据及缺年报的对象单列,不能承诺完整结论。
## 分批验收预期
### 本地
- 合并去重和排序确定,重复读取不改变名单。
- 高股息资格尚未通过的对象仍能进入研究。
- 来源暂不可用时累计待办不丢,旧快照损坏时不覆盖。
- 当前候选和首次报告均能获得名额。
- 预算不足、已有在途及失败冷却时不重复投递。
- 保护时段只运行原候选。
- 旧指纹、有效标志、固定报告引用和财务期口径不变。
- 全部离线unittest通过。
### 部署前只读预览
在178容器内核对两个池并集、首次报告缺口和首批代码。
预览不保存名单,不登记运行,不投递任务,不调用模型。
核对桥消费的company_review_runs及报告路径保持。
### 服务生效与生产判收
容器重建须取得用户同意。
按仓库规则统一重建四类worker。
beat调度不改,不需要重建。
首批显式投递review队列,核对报告增量与预算读数。
首批成功不等于全池覆盖完成。
后续按实际缺口下降和失败清单判收。
## 开发者参考
原targets.collect仍供材料需求单和旧入口使用。
扩大覆盖只在评析批任务启用,避免一次扩大数千张采集单。
累计名单使用job_runs的既有tail JSON字段。
临时投递凭据使用Redis,带过期和所有者校验。
同一凭据只能从待执行原子转为执行中一次。
失败冷却六小时;无数据沿用七天复查,三次评议失败沿用原放弃规则。
预算沿用索引表的当日登记口径,含失败状态。
每次模型调用仍受原单次输出上限约束。
这不是服务商账单的逐次计费审计,不能把登记量称为实际费用。
排序只决定研究先后,不形成证券评分。
报告内容及下游排序由现有逻辑继续处理。

View File

@ -0,0 +1,129 @@
# 个股评析全池覆盖交接
本轮日期为2026年10月28日,时间按北京时间。
规格见《个股评析全池覆盖开发规格_2026-10-28.md》。
## 用户决定
用户要求本轮直接覆盖整个产业池和高股息池。
本轮扩大普通个股评析,不扩大分红专项研究,也不改交易参数。
## 已实现
复用原每小时批任务、review队列及两个工作并发。
每批最多30家,当前候选保留10个优先名额。
没有候选待办时,30个名额均可补首次报告。
高股息研究对象取桥已发布的现有接口。
基座不重新计算股息筛选,也不按投资资格过滤研究。
累计范围沿用job_runs的tail字段,仅名单变化时追加快照。
来源失败保留旧名单;旧名单损坏时停止扩展。
在途凭据有所有者校验和六小时过期。
失败冷却六小时,任务不反复抢同一批名额。
每日评议和定性上限各600家,评议输出上限900万token。
开关和额度由config/company_review_coverage.json热读。
06:00至08:00、20:00至23:00仅处理原候选。
其余时段允许补齐全池,原beat时刻保持。
材料需求单仍使用原targets.collect。
本轮没有一次扩开数千张采集需求单。
报告判据、财务期、幂等指纹和固定引用保持。
批任务返回保留原有字段,新增覆盖和来源读数。
## 部署前实测证据
178原代码为7ef107e,旧服务正在运行。
当前候选148家,全部已有评析、定性与评议。
产业池3833家,高股息研究池169家。
新范围去重后3885家,已有747家,缺3138家。
首批只读预览为30家首次报告与60家原候选轻量复核。
两路范围来源读取成功,review队列为零。
预览未保存累计名单,未投任务,未调用模型。
本地全部离线unittest为264项,通过。
新增26项覆盖轮转、预算、来源失败、在途及任务收尾测试。
已有指纹、报告与财务历史边界测试保持通过。
桥sources.company_reviews仍直接读原索引字段。
报告路径与输出格式未改变。
## 部署前边界(已由下文上线记录更新)
代码尚未在178拉取,工作容器尚未重建。
新范围尚未投递,不能把只读预览称为覆盖完成。
服务器现有未提交文件保留。
其他仓库只同步本文和规格副本,并追加桥决定台账。
桥现有未提交台账与其他工作保持。
## 上线及撤销
用户同意后,在review队列与在途任务为空时统一重建四类worker。
保留原worker副本数,不调整并发,不重建beat或前端。
核验热读配置与只读预览后,显式向review队列投递首批。
首批成功后,按自然小时批次继续推进。
首次覆盖容量预估约一周,不保证缺材料对象得到完整结论。
撤销时在本地将配置enabled改为false,再提交并由178拉取。
下一批恢复原名单和每日额度。
已在途任务可完成,旧报告与累计名单保留。
## 可运行验证
```bash
# 本地 · /Users/baobao/Documents/work/project/astock-kg
PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=backend python3 -m unittest discover -s backend/tests -t backend
git diff --check
```
预期264项通过,差异检查无输出。
```bash
# 基座机 · ~/project/astock-kg,部署后
docker compose exec -T -e PYTHONPATH=/app backend python /app/scripts/company_review_coverage.py
```
预期dry_run为true,dispatched为空。
初始targets为3885、missing_reports为3138,后续随真实产出改变。
source_errors应为空;无数据与材料薄须分开看。
新增的日检行同时显示最近批次时刻及状态。
## 2026-10-28生产上线记录
本节时间均为北京时间。
用户明确同意部署并启动补齐。
15:35再次核查四条队列均为零,四类worker在途任务均为空。
178从7ef107e快进到42191df,服务器已有未提交文件保持一致。
15:35:52统一重建四类worker,镜像、各服务副本数及并发保持。
15:36四个新节点均返回pong。
beat、前端及数据库容器未重建。
部署后只读预览与部署前一致。
范围3885家,有报告747家,首次缺口3138家。
每日额度为评议600家、定性600家、评议输出900万token。
名单来源无异常,预计首批为30家首次报告及60家轻量复核。
15:36:32显式向review队列投递批次。
任务号为e20798d0-6186-4a96-9093-f35a133cf70f。
15:36:33批任务成功,实际投递90项。
其中30项带定性与评议,60项仅复核原候选。
累计名单已保存,初始90项均持有在途凭据。
首批投递成功不等于90项执行完成。
15:37部署后完整日检通过,19项正常、零警告、零错误。
新增覆盖行和最近批次行均成功输出。
15:38接口健康检查返回HTTP 200。
15:40:46首份新增报告000543.SZ成功落库。
报告、评议、定性状态均为ok,在途凭据自动删除。
最新报告和固定引用接口均返回HTTP 200,各51418字节。
固定引用为3aae8fc1e87736811af30027715c12f64b9a656075608dbfff981338a97b07d1。
15:41:43复查累计名单3885家,有报告748家,剩余3137家。
在途89项、重试冷却零项,两路名单来源仍无异常。
当日登记评议8家、定性8家、评议输出104020个token。
本轮已判收部署、批次投递、首份报告和凭据收尾。
首批全部完成、下一次自然小时批次及全池覆盖尚未判收。
后续由原定时任务继续推进,无需人工逐批投递。
缺财务数据、材料薄及失败须分列,不能充作完整结论。
部署日志在178的~/akg_logs/review-coverage-deploy-20261028-1535/。
目录保留前后服务状态、原镜像与命令、拉取记录和首批预览。

View File

@ -882,3 +882,31 @@
研究池从该已发布版本取 rank 前 50 个被指向对象,保留实际 3 只持仓,上限仍为 100。补入 12 只,按原规则回收 1 只及池满移出 11 只。全部 16 只 PMS 候选与持仓在池内,候选关联决策版本和研究名次一致。增量补扫任务 5ce6930c-dd01-45ef-a68b-f830419d4858 已受理,未把受理写成研究完成。POOL_TIERS 的旧配置在 rank 来源不参与过滤,日志打印它不代表生效。
PMS 155 镜像 50 批、1165 例通过;桥 54 组契约及池、构建就绪、XXL 触发三个脚本通过。页面、桥健康及 QMT 账实核对正常,部署判收为真机见过。切换窗口一次高股息读取超时于 15:01:39 复查恢复。私有备份和验收在 155 的 /home/factor/pms_logs/quality-switch-20261026;旧镜像与分支保留,回退须追加版本而非覆盖历史。详见 PMS docs/开发接续_2026-10-12.md 第十节。未制造交易,实际自然建仓及收益表现另行观察。
## 084 · 2026-11-04 · 人工关注补齐真实评析,吸筹改为参考
- **日期与授权。** 2026-11-04,北京时间。用户明确选择“沿用当前质地排序,吸筹作为参考,建仓看三源合议”。本次仅在 155 模拟链路切换 `CARD_ACCUM_CONFIRM_REQUIRED=0`,不扩展到 188 真实 PMS。代码默认保持旧确认规则,回退仍有明确开关。
- **改动。** 桥不再仅因吸筹未达到明确状态、缺评分或评分陈旧而阻断候选评估;原吸筹评分、状态及新鲜度继续作为证据。质地排序、原始分数编码和发现资格保持,不新造加权评分。证券范围、覆盖与估值条件、既有真实风险、高位派发及快照时点校验继续生效。PMS 的基本面、技术面、择时三源合议及后续规则、研判、资金、组合与执行约束继续决定建仓;本次不是关闭全部判决过滤。
- **依据。** 代码和历史记录核对表明,桥从 9 月起将明确吸筹作为候选确认线;10 月质地排序切换保留了该判决。PMS 三源合议在上游“仅展示”过滤之后运行,因此无法评估已被旧确认线排除的对象。此次依据用户的新明确决定改变吸筹角色,不将其写成已经发生过的部署回退或历史修复。
- **人工关注评析。** 登记后的后台任务与自然计划修订周期同步独立评析状态;查询入口不派模型任务。全部有效沪深人工关注进入原研究目标名单,不受原分数、名次和交易判决限制。后台复用基座真实评析入口,开启买方评议及定性材料,默认不强制重跑。提交前持久化受理凭据,收到真实入队回执才标记排队;超时或结果不明保持待核实,不立即重复投递。原登记的“已收录”与“评析完成”独立展示。
- **完成与时间口径。** 完成要求真实固定报告引用、成功的买方评议、论点及有效生成时间。七天复核窗口使用真实运行或核对时刻,避免指纹命中后只有核对时间更新而永远无法完成;报告日期仍取原生成时间的北京时间日期。缺引用、材料不全、评议失败或来源不可用均不伪造完成。财报期与原评析重试规则继续由基座负责。
- **版本与预期。** 规则切换即使输入不变也追加新的完整修订,记录真实截止时刻、参数指纹和变更原因。旧晨版、历史计划及固定版本查询不变。人工关注的研究覆盖与交易资格分开,移除或到期后停止新增关注研究;评析完成不等于可以自动建仓。预期自然回执能分别说明登记、受理、报告完成、日研究和盘中观察状态。
- **验证与未完成事项。** 开发机新增评析专项 19 项通过;包含本次规则切换的 35 个测试文件均完成配置隔离、禁网验证,其中高股息旧测试的两处相对路径已修复并定向复测。此条记录开发交付和授权边界,不代表运行参数已切换。候选镜像断网回归、155 部署、真实评析受理与完成、自然日研究及盘中观察仍须在发布阶段逐项判收,不注入交易制造结果。
## 085 · 2026-11-05 · 买方评级变动留痕,同质地档三十日升优降后
- 日期:2026-11-05,北京时间。用户明确选择降级保留候选,只降低排序优先级。
- 改动:正式有效评析版本只追加留痕,保存前后等级、引用和真实时刻。首次覆盖为基线,同级不续期,最新升降覆盖当前排序方向。30个自然日内同质地档升级优先、降级后置,届满恢复普通排序;原始分数和建仓资格不变。
- 依据:当前评析索引会覆盖同报告期旧内容,仅比较持仓或相邻日榜无法完整记录变动。用户要求沿用质地排序,人工关注继续研究,建仓通过三源合议。此次排序不覆盖已上线的降级清仓与人工限制。
- 预期:每个正式固定报告只保存一次,失败或缺资料不伪装降级。选股固定截止时间和输入,五分钟自然修订识别期限届满。PMS继承上游名次,纯排序更新不重复发起模型研判。旧晨版和历史账本保留。
- 复核日期:发布当天核查表、镜像、参数与基线;首个自然升降事件到达后核查证据链及排序。部署与自然判收分别记录,不新增长期观察任务。详见同名规格与接续文档。
### 085 发布补记 · 2026-11-05 13:58 北京时间
178基座972c934经318项候选断网回归和独立PostgreSQL九项检查,于13:46:27完成五项服务定向发布。等待在途任务自然完成,保留原仓与环境。初始化3764条基线,随后正常队列新增4条,未倒造过去升降。
155选股0945fa2与PMS38f91d2经39组和78组候选断网回归后上线。排序开关已开启,质地轴、原分数及吸筹参考保持。13:55正常调度追加评级排序版本,1226条携带依据,8只人工关注保留,旧晨版不变。13:57:08 PMS记录616自然消费同版,50条沿用上游名次。PMS唯一WS与其他15容器保持,188未操作。
13:58账实检查通过,无待入账、孤儿或序号缺口。真实升降和30日届满尚未发生,仍单列未自然判收。完整镜像、回退、证据和核验命令见[发布接续](评级变动与三十日排序发布接续_2026-11-05.md)。

View File

@ -0,0 +1,96 @@
# 评级变动与三十日排序发布接续
最新核验时间:2026-11-05 13:58:52,北京时间。用户已批准本次发布,178基座五项服务及155联动已完成。评级排序已开启,正常调度的新计划和PMS消费已判收。真实升降及30天届满尚未发生,不冒充自然验收。
下文13:00至13:13的服务与待确认记录保留历史。当前镜像、基线、自然结果和边界见末尾13:52、13:58两节及[发布证据](evidence/rating_priority_release_20261105.json)。
## 当前结论与提交
用户决定为保留候选、降低排序优先级。主轴仍为质地好、中、无评析、差;同档内近30个自然日升级优先、降级后置,再走原排序。首次覆盖是基线,同级报告不续期,最新有效变化决定方向。人工关注继续分析,建仓三源合议与已有降级清仓独立执行。
基座产品提交为972c9342db8c80c7530556caa68237a3011b6033,基于178当前06bb822。选股提交为0945fa2b29835886ddfdcb3ab00a088e6dafa721,基于3d513a2。PMS提交为38f91d2bddd27ff2727c2209df687a5a73278bd5,基于0d3f115。三个产品提交已推送到各自任务分支,并通过git ls-remote核对。规格和接续的后续文档提交不改变候选产品内容。
开发机全量配置隔离与禁网验证:基座318项,选股39组,PMS78组,全部通过。基座新增10项涵盖全部升降方向、无效报告、同级不续期、版本幂等、索引与历史回滚、真实旧索引基线及初始化续跑。选股新增7项包含真实装配、期限届满修订、旧晨版不变、关闭开关、旧冻结参数不继承现值、来源失败和沪深入口;PMS新增1项验证上游顺序与依据透传、纯排序不重问模型。
本地日志:/tmp/akg-rating-local-test.log、/tmp/bridge-rating-local-test.log、/tmp/pms-rating-local-test.log。13:00时事务回滚仅有内存SQLite验证,独立PostgreSQL演练当时待授权执行;最终九项演练结果见13:52节。测试不使用现网数据。
## 13:00发布前只读记录
178主机为tlai4090,代码06bb822d419e389cbdeb0e604fe80380ff458cbc。存在旧数据文件删除及未跟踪文件,未修改。backend与四类worker实际镜像sha256:a2ee9f3c9cb1be438b9d85656b54f9182865d1b9642c752a93ccc5213d07e58b,源码挂载原backend/app及workers。四类worker各一实例,常规worker实例名后缀为2,不能假定为1。原源码、环境文件和其他共享进程须保留。
178尚不存在company_review_rating_history表。13:00附近review队列有两项真实company_review在途,其余三类worker当时空闲。完整资料与买方评议均ok的当前评析索引为4928行,该数是报告索引行数,不等同唯一证券数或可交易候选数。必须在执行前重新读取在途及容器基线,禁止拿本次快照当未来空闲证明。
155主机factorevaluation。web、worker、唯一WS仍为pms:risk-exit-64449c6;原仓HEAD64449c6。选股服务仍为bridge:plan-recovery-9f8224e,镜像sha256:c3c87ffa8ee7087f41d9cd0192ff9c86bdc37fb814da717115169c0d62f76496;原仓HEADd9c4b92。其他服务与原私有配置保持。本次尚未发布新排序规则,尚未产生可判收的自然评级变动。
## 发布前确定的实施顺序
1. 178需要本次容器发布确认。仅针对backend、worker、worker-interactive、worker-targeted、worker-review五项服务。候选代码972c934,独立发布目录,原源码与.env保持。基座只追加历史与基线,不改评级算法,不批量重跑模型。
2. 获准后先构建基座候选,使用本次归档的新代码与新测试,完成无业务配置、断网全量回归。不能在现用容器里运行镜像内旧测试。准备脚本在开发机/private/tmp/rating-priority-release-20261105/akg-candidate-plan.py,尚未上传或执行。随后以隔离临时PostgreSQL演练DDL、重复初始化、重复引用、事务回滚及并发证券锁。
3. 保存目标容器、有效配置和其他容器标识基线。定向暂停四类worker消费,等在途及预取任务正常结束,不强杀模型任务。按原Compose项目以--no-deps --no-build --force-recreate更新上述五项服务。只改候选镜像与独立app、workers、scripts挂载,配置和依赖保持。
4. 在新backend执行显式基线初始化。命令为python /app/scripts/company_review_rating_history.py --initialize-baselines,PYTHONPATH=/app。逐证券事务保存当前有效正式报告,初始记录方向必须为baseline,原报告时间和本次记录时间分开。脚本可重复执行,已有历史保持;失败停止排查,不吞错填0。
5. 确认历史表、实际源码、自然任务注册和读取权限。通过155选股容器只读核验历史可读取、与当前报告引用匹配,然后仅发布155选股服务及PMS的web、worker。PMS唯一WS、beat、采集和微盘监控无需因本次修改重建;188所有服务不在本次范围。
6. 155选股覆盖设置PLAN_RATING_PRIORITY_ENABLED=1,窗口固定30。原始分数、吸筹规则、质地轴及PMS所有现行参数保持。等下一次正常五分钟修订,保留旧晨版和冻结输入,记录实际新增版本,不伪造模型或交易测试。
7. 核验候选保留、排序与PMS消费一致,纯排序版本不引发额外研判。核验PMS连续确认、待入账、孤儿、资金及策略数量现金差。其他容器标识、镜像、启动时间不变。真实评级升降与到期尚未发生则明确记为未自然判收。
## 回退与只读核验
排序回退只关闭新开关并追加当前时刻修订;评级历史表、原报告、旧计划、账本与任务队列均保留。必要时回退上述目标服务镜像及挂载,不能撤销已经产生的退出指令或清理历史。没有新增长期观察或调度任务。
获准并发布后可在178 ~/project/astock-kg执行:
```bash
# 178 · ~/project/astock-kg,只读统计
docker compose exec -T -e PYTHONPATH=/app backend python /app/scripts/company_review_rating_history.py
```
预期输出基线、同级、升级、降级的真实计数与最近记录时刻。未发生升降时计数为0是有效结果,不补造事件。单票用--code 002979.SZ读取前后评级和固定引用,范围仅沪深。
发布前确认依据:astock-kg/AGENTS.md明确要求“容器的重启、重建及关联操作必须先经用户同意”。既有155授权不扩展为178操作。用户本轮已明确批准,实际发布结果见下文,不再等待或重复请示。
## 2026-11-05 13:11候选回归补记
155 PMS候选pms:rating-priority-38f91d2,镜像sha256:ba3ca2ea3d6a6b1eeb69570cc1f951ebdbbfdc51e2e64767b14ae9806881e5d6,已通过78组断网全量。证据为155的/tmp/rating-priority-release-20261105/pms-test.log和pms-candidate.json。
选股1278c07候选曾通过39组断网全量。最终复核追加0945fa2,保证旧冻结参数缺新开关时不继承现值;开发机39组复验通过。1278c07镜像不能作为最终发布版本,须以0945fa2的候选证据为准。开发机暂存目录/private/tmp/rating-priority-release-20261105保存归档与构建脚本。选股工作树的未跟踪data目录为本次离线测试输出,不纳入产品归档。
## 2026-11-05 13:13最终候选与待确认范围
155最终选股候选bridge:rating-priority-0945fa2,镜像sha256:702fcfaaaa2ba5a03baf7061b848d70e6a56be6d84fa3a653f48e77d6ebe55dc,39组断网全量通过。证据为155的/tmp/rating-priority-release-20261105/bridge-test.log和bridge-candidate.json。PMS78组候选回归也已完成,两个候选均未应用到运行服务。
待用户确认的唯一新增发布范围为178基座972c934:隔离候选回归和临时PostgreSQL演练通过后,温和排空在途,仅更新backend及四类worker,并初始化真实当前基线。原源码、配置、其他容器和188保持。随后按既有授权发布155选股及PMS web、worker并开启新排序。没有自然评级升降验收,不将代码或候选测试写成上线。
## 2026-11-05 13:52 已批准并完成服务发布
用户本轮明确回复“批准本次发布”,范围为178基座972c934的backend及四类worker、基线初始化,随后按既有授权完成155联动。此前“待确认”是历史状态,本节起以实际发布与后续判收为准。
178候选akg:rating-history-972c934,镜像sha256:143bfba15a1517cabba4b44f55aebcfef3121c3361622636c29603d3dd1de14e。318项全量断网回归通过,独立临时PostgreSQL的九项检查通过,包括并发重复只登记一次、证券锁等待、并发变动链连续、索引与历史失败回滚、迁移重复运行及同级不续期。临时数据库无网络、无端口,使用独立内存数据目录,测试后容器与套接字卷已删除。
候选准备曾发现镜像摘要被构建器解释为远程镜像名,已改用本机已核对标签。全量测试还发现旧测试读取根目录DDL文件及镜像内启用的全池额度配置;最终候选补齐测试文件,并以独立空配置隔离测试,业务代码未改变。失败证据保留于178的/tmp/rating-priority-release-20261105/akg-build-reference-failed.log和akg-test-isolation-failed.log;最终有效证据为akg-test.log、akg-pg-test.log。最初akg-candidate-plan.py是历史准备稿,实际由开发机编排SSH及远端shell/Docker,不在178宿主机运行Python。
178温和暂停四个目标worker消费,等待两项在途及两项已预取评析全部自然结束。13:46:27仅重建backend、worker、worker-interactive、worker-targeted、worker-review。独立源码在/home/tlai/project/astock-kg-releases/972c934,原仓仍为06bb822,原.env和配置不变,其余7个运行容器的标识、镜像和启动时间一致。五项服务镜像与挂载均核对通过,backend源码哈希、健康接口及正式评析任务注册通过;四个消费队列恢复,既有真实队列继续处理。
13:47:25完成初始化:新增3764条baseline,已有0条,134只资料不完整暂不登记。预检时有效3762只,排空期间真实评析继续完成,故最终以迁移实际读数3764为准。所有初始记录都是基线,未倒造过去升降。155通过实际选股连接只读访问历史表成功。178私有备份为/home/tlai/project/astock-kg/backups/rating-history-20261105-972c934,保留原有效配置、容器基线、已核对覆盖和部署意图。
155于13:49:21仅更新pms-web、pms-worker为pms:rating-priority-38f91d2,镜像sha256:ba3ca2ea3d6a6b1eeb69570cc1f951ebdbbfdc51e2e64767b14ae9806881e5d6。更新前目标worker在途、预取、定时待执行均为0。13:51:21选股服务空闲后更新为bridge:rating-priority-0945fa2,镜像sha256:702fcfaaaa2ba5a03baf7061b848d70e6a56be6d84fa3a653f48e77d6ebe55dc。选股独立目录为/home/factor/project/akg-factor-bridge-releases/0945fa2,原仓仍d9c4b92,原.env、共享data及历史晨版保持。
155只将PLAN_RATING_PRIORITY_ENABLED设为1;窗口30天,质地轴quality和CARD_ACCUM_CONFIRM_REQUIRED=false保持。PMS其余实际参数不改,包括唯一WS、beat、限制价采集、微盘监控在内的15个非目标运行容器均未重建,标识、镜像和启动时间一致。188完全未操作。155私有备份为/home/factor/project/tradingSystem/backups/rating-priority-20261105。
13:51:49发布后PMS检查:数量差0、现金差在既有0.02元结算容差内,待入账0、孤儿0,确认1643306与落库水位一致,最近200条连续;资金与持仓快照年龄29.1秒。未补账、未注入信号或委托。新worker已恢复celery队列。自然新计划与PMS消费的验收另记,不能用这组服务健康代替。
## 2026-11-05 13:58 自然计划与PMS消费判收
13:55:02的正常五分钟调度追加新修订。版本为2026-11-04_19549ac407bb1dd73133a66fc6f9db0675cc62a2eea1749c1d96cb4b5d4f99df,原因为“启用评级变动三十日排序”。11月4日是日线数据日期,实际发布在11月5日盘中,没有倒填晨版。13:55:34修订健康检查无错误,待修订列表为空。
该冻结版本的排序开关为true,窗口30天,仍以quality为主轴。1226条主榜与观察榜均携带评级依据,其中1193条为baseline,33条明确标记missing_history,没有将缺资料算作降级。实际候选39只、观察50只、人工关注8只。原始分哈希f5667f5909df6b8bb50f44ffd6b7ab43ec6e0c6594fef11857897f48d47b4936与切换前一致。同质地档内的优先级顺序检查通过;此时没有升降,因此没有虚构名次移动。
PMS自然消费记录616生成于13:57:08,读取上述同一决策版本。50条消费记录全部包含评级依据,采用upstream名次,research_rank递增,顺序与切换前记录614一致。没有手动写消费记录、重跑计划或投递模型任务来制造验收。纯排序不重问模型的保证来自隔离测试;本次自然初始化未发生实际评级升降或窗口届满,不能据此声称所有未来变化均已在线见过。
178既有队列自然继续产出。只读统计在13:58核实baseline总数3768,最新记录为13:57:02;较初始化增加4条首次有效报告基线。尚无same、up或down事件,不用基线充当升降。
13:58:26至13:58:28的PMS最终只读检查:确认和落库水位均1643413,最近200条连续,待缺口0、待入账0、孤儿0。数量差0,现金在既有0.02元结算容差内,估值齐备。资金与持仓快照均为5.1秒,实际持仓5只,未执行任何修正。13:58:52复核155全部非目标15容器的标识、镜像与启动时间一致,历史晨版及旧冻结文件哈希不变。
本次发布与自然计划消费完成。未自然判收的是第一次真实评级上调或下调,以及30天窗口真正届满后的计划修订;对应边界已通过隔离测试。没有新建长期观察任务。PMS唯一WS保持原容器和连接,188未操作,账本、队列、原报告、人工关注及历史计划保留。四仓文档同步保存,同目录evidence/rating_priority_release_20261105.json仅包含脱敏核验结果,不包含有效配置全文或密钥。

View File

@ -0,0 +1,27 @@
# 评级变动记录与三十日排序
## 用户确定的行为
用户选择下调后保留候选,只降低排序优先级。沿用现有质地排序,在同一质地档内,最近30个自然日上调的优先,未发生有效变动的居中,下调的靠后。随后仍按原被指向、组内分、原始分和代码排序。原分数、档位和候选资格不改,不用人为加减原始分污染其含义。
窗口从正式报告实际落库时刻计算,满30日失效。再次同评级出报告不续期;再次发生有效升降则采用最新一次变化。首次有效报告只建基线,不算上调。材料缺失、超时、失败、陈旧或无法追溯版本不构成升降。改变的是自动选股优先级,不替代三源合议、持仓退出纪律、人工命令和其他交易约束。
人工关注继续保留原分析与真实信号覆盖。无需为升降另起信息流分组,不改已经确认的页面样式。现有历史计划保持不可变,规则和到期变化只能形成新的真实修订。
## 代码核查与实现原则
数据基座company_review_runs按证券、报告期和算法版覆盖写入,不能完整还原每次评级变化。仅在PMS记录持仓或比较相邻日榜会遗漏未持仓证券、跨期变化及两次刷新之间的变动。因此正式报告落库时同步追加不可改写的评级版本记录,与原索引更新处于同一事务。沪深入口校验,按证券加锁,保留前后报告引用、前后等级、真实生成与记录时刻及变化方向。
新记录通过独立company_review_rating_history表保存。每份有效固定报告引用只保存一次。同引用内容冲突或旧版本重放不能覆盖新历史。首次写入可从当前实际索引保存一份基线,但不倒造旧升降事件。缺失的历史无法从当前评级推算,不给旧档案补写变化日期。记录覆盖原评析save_run入口,无新模型任务或定时任务;手工只读核验和日检增加相应计数。发布时显式初始化当前有效基线,逐票事务可中断后续跑。基线保留真实原报告时间,并另记本次记录时间,不算升级或降级。
选股系统分别单表读取当前报告和其评级历史。历史必须与当前报告引用匹配;最近有效变化及前后引用随录制输入保存。计划使用固定截止时刻计算30日状态,固定版本查询与重放不看今天时钟。自然五分钟修订要能识别期限届满,即使报告内容不变也生成新修订;旧晨版保持。旧源没有历史时不凭空加权,并显式标记缺历史。
PMS只透传排序依据并沿用上游研究排名。排序期满本身不改变研究结论,不因纯排名字段变化重复调用模型。买方整体评级下降的已上线清仓规则保持独立。
## 验证预期与发布边界
离线覆盖三种上调、三种下调、同评级不续期、首次基线、重复报告、版本冲突、失败和材料不全、跨期、并发事务、30日临界、再次升降、时区、历史重放、期满修订与排名一致性。检查降级股票仍保留候选,同档升级前移、降级后移,跨档仍质地优先;原始分与门槛不变。测试禁止连接真实数据或模型。
先完成本地实现、提交和隔离回归。155必要发布沿用已有授权。178数据基座容器操作需要确认适用授权后再实施,不能以155授权扩展。按实际服务与挂载核对生效范围,保留在途模型任务,不强杀。基座保存历史先就绪,再开启155排序规则。未得到真实评级变化时只报告部署与基线,不伪造自然升降验收。
撤销时仅关闭新排序开关并追加新版计划,保留评级历史与既有交易账本;已经触发的PMS退出指令不能被排序回退取消。验收和发布事实另行追加,不用本规格冒充线上状态。

View File

@ -0,0 +1,53 @@
# 选股输入迟到恢复与沪深边界
2026年11月5日,北京时间。用户报告PMS选股计划过期,随后确认外部热度缺失已恢复,要求完成剩余恢复与重算。
## 原因与现场
09:36直接从155新版PMS读取上游:请求没有固定日期或版本参数,返回数据日、热度日和行情日均11-03,最后修订截止11-04 15:05。11-03至11-05是两个交易日,PMS上限1的拒绝正确,不是缓存或人工关注入口问题。
上游02:50构建因传导和热度旧日失败,07:10时传导已齐但热度仍11-03,写前检查阻止全部因子构建,故评分也停在旧日。后续每五分钟修订只接受当天已有基础版,持续no_morning_today,数据晚到后没有自动补建入口。08:45区制追加也缺当天基础版,但旧CLI把未发布仍报告成成功。
用户确认外部数据恢复后,09:44:25只读核对传导、沪深日线及热度均已齐11-04;热度批次130,沪深5208行。外部热度生产代码未改;原始批次写入时间的连接时区未证实,不把它解释为盘后停机证据。
准备重算发现,旧代码只在计划最终消费侧过滤沪深,因子构建和冻结仍可能读取、计算和归档北交所。按项目既定范围,先补齐输入边界再补跑,不为北交所另查异常、算分或生成数据。
## 实现边界
自然plan-refresh缺当日基础版时,先只读核对上一交易日三路输入;未齐仅等待,不给旧数据续期。真正齐备后复用同一跨进程发布锁,重新核对日期、现有版本和尝试预算,再完成默认daily因子构建、输入冻结及真实计划发布。明确指定历史date的入口不自动恢复,不通过显式日期绕过就绪检查。
恢复使用实际采集截止和完成时间,09:30之后由现有版本机制标记recovery,不倒填晨版。采集完成后、发布前再次核验允许时段和来源日期;已有当天成功版本不重复构建。旧格式或不合格的既有同日文件不自动覆盖。恢复本身不增加push-pool或模型调用,保留既有人工关注同步的一次自然执行。
恢复尝试先原子保存于data/plan/recovery/北京时间日期.json。每天最多3次实际构建,间隔至少900秒,重启不返还次数;未齐检查不耗构建次数。PLAN_RECOVERY_MAX_ATTEMPTS默认3,可降至0关闭;PLAN_RECOVERY_COOLDOWN_SECONDS默认900,可收紧间隔,不低于900。沿桥既有环境配置机制读取,不改变PMS运行参数。
普通daily构建也使用同一发布锁。冻结函数返回errors、缺核心记录或非法行数时,构建明确失败,不把部分因子/输入包装成完整计划;合法0行仍按原语义保留。区制追加返回独立publication_status,CLI不再把缺版、失败或窗外未发布伪报成功,原区制status字段含义保持。
证券入口统一支持沪深前缀、点后缀和六位裸码,显式交易所须与A股号段匹配。读取及样本构建前排除北交所、错配及非法代码;market和pool模式均适用。因子、冻结及嵌套证券成员均过滤,日期锚也仅看沪深;历史北交所记录不删除、不改写,因子区间删除只触及沪深。市场指数的两市成交额口径保持。
## 验证与实际发布
开发机空配置、禁网护栏全量38/38测试文件通过,末行ALL OFFLINE SUITES PASS。包括恢复28、日期就绪15、范围19及取数5项专项;已有版本、计划、质量排序、人工关注和估值回归全部通过。独立复核文件锁、次数/冷却、旧版本保留与跨窗口发布边界。开发日志/private/tmp/bridge-recovery-release-20261105/full-local-tests.log。
475bfec候选断网38/38测试文件通过后,于北京时间10:02完成155的akg_factor_bridge首次定向发布。原目录d9c4b92、原.env和data共享挂载保留,源码切独立发布目录,不重建PMS或188。
首次实机只读就绪检查发现,三路MAX加证券范围过滤后可能扫描大量旧行情,耗时达到数分钟。9f8224e补上日期大于等于目标日的参数化范围,仍读取更新日期以拒绝未来沪深数据,不放宽新鲜度,也不让范围外证券影响日期锚。开发机与第二次候选均38/38通过。第二次仅桥容器于10:18:22启动,10:18:44核验完成;17个其他容器标识、镜像和启动时间均保持,原源码、环境和历史晨版哈希保持。最终实机只读就绪检查0.441秒返回11-04。
## 自然恢复与PMS判收
以下时刻统一按北京时间说明。自然任务f5a16fb3在10:05:00启动plan-refresh,10:05:36.721028登记第一次实际构建,10:08:24.634400完成恢复。其日志10:08:38明确返回recovered、data_date为11-04、attempts为1,随后同轮修订unchanged。人工请求5b181fe6在10:05:36排队,直到自然任务结束才于10:08:38执行,并在10:08:53返回unchanged,没有再次构建。10:10和10:15自然刷新也保持正常。
首个完整版本的采集截止为10:07:00.338807,publication_kind为recovery,原因是“数据恢复后补发”,晨版统计资格为false。没有倒填07:10。完整主榜579行、观察档646行,包含6条有效人工登记;质地排序、吸筹参考规则保持。后续真实修订另存父子版本,首版固定指针保持。
10:16只读核验实际冻结的13个文件:清单与文件行数全部一致,errors和warnings均为空;热度5208行、收盘价5218行,全部证券代码及嵌套成员均符合沪深范围,未发现范围外或无法确认身份的证券记录。事件因子0行按原语义保留,不伪造为非空。
08:45缺版的区制追加已由授权补跑3d52f1c0于10:14:29至10:14:55完成。新版本保留真实10:14:31截止;区制为OK,数据日11-04,弱势数8,weak_day为true。10:15自然刷新unchanged,版本健康记录无错误、无待核验股票。
10:18:48和10:19:46在155模拟PMS的新只读进程验证:实例表前缀为空,直接读取当前上游后,计划、热度和行情数据日均11-04,交易日龄1,通过原上限1;没有调用带接收快照写入的get_plan。按真实参数及持仓、黑名单只读筛选,返回主榜300行、观察档100行,过滤后有86只合格、当前截取50只,其中“候选”38只、“关注”12只,均为沪深股票。排序模式为upstream,route_by_verdict仍为true。这是候选可读和筛选恢复,不是50只均获建仓批准;后续命令、三源合议及交易边界未改变。
## 尚未完成与证据
10:19:46读回6条人工关注买方评析均completed,真实报告日期为11-04,读取错误为0;每日研究6条均COMPLETE,执行日11-05、行情数据日11-04。盘中研究3条COMPLETE、3条UNAVAILABLE:1条模型未返回合法研究结论,2条超时;超时回执缺少行情数据日期,未按成功展示。它们属于当日盘中研判的真实未完成项,不等同桥计划过期,不通过重复构建或额外投递模型来制造完成。
桥基础版、区制和PMS计划新鲜度已恢复,无需重复补建。外部热度生产任务本身不在本轮修改范围;本次证明实际数据已齐及恢复链有效,不宣称外部停更原因已修复。盘中3条不可用回执继续按原有重试、超时和交易守成规则处理。
原始只读聚合证据保存在开发机私有目录/private/tmp/bridge-recovery-release-20261105:recovery-live-summary.json、pms-plan-live-summary-final.json。日志只摘取调度状态、版本指纹、数量与日期,不记录证券名单、模型正文或账户金额。性能补丁全量日志为readiness-bounded-full-tests.log;最终性能和容器基线由发布主流程另保存。探针只读文件、只读SQL及GET接口,禁止业务写入、模型和任务触发;发布瞬间的一次连接中断已在服务就绪后重新只读核实,不计为业务恢复失败。

View File

@ -14,6 +14,7 @@ import pandas as pd
import config
import common
import db
import market_scope as scope
FACTORS = {
"akg_upside": "t_factor_akg_upside",
@ -40,34 +41,46 @@ _EMPTY = pd.DataFrame(columns=["trade_date", "stock_code", "factor_value"])
# ---------------------------------------------------------------- 热度
def _read_heat_rows(start, end):
"""先在沪深范围确定每日最新批次,再读证券行;不加载范围外热度。"""
condition = scope.sql_filter("stock_code")
batches = db.read_mysql("heat", "SELECT trade_date, MAX(batch_no) AS batch_no "
"FROM stock_fund_heat_scores WHERE trade_date BETWEEN %s AND %s AND " + condition +
" GROUP BY trade_date", (start, end))
parts = []
for row in batches.itertuples(index=False):
if pd.isna(row.batch_no):
continue
values = db.read_mysql("heat", "SELECT trade_date, stock_code, batch_no, score "
"FROM stock_fund_heat_scores WHERE trade_date = %s AND batch_no = %s AND " + condition,
(row.trade_date, row.batch_no))
parts.append(scope.frame(values, "stock_code", prefix=True))
return pd.concat(parts, ignore_index=True) if parts else pd.DataFrame(
columns=["trade_date", "stock_code", "batch_no", "score"])
def build_heat(start, end):
"""热度 = stock_fund_heat_scores 最新批次 score(0~1)。stock_code 已前缀式。"""
df = db.read_mysql("heat",
"""SELECT s.trade_date, s.stock_code, s.score AS factor_value
FROM stock_fund_heat_scores s
JOIN (SELECT trade_date, MAX(batch_no) bn FROM stock_fund_heat_scores
WHERE trade_date BETWEEN %s AND %s GROUP BY trade_date) m
ON m.trade_date = s.trade_date AND m.bn = s.batch_no""",
(start, end))
"""热度 = 沪深证券当日最新批次 score(0~1)。"""
df = _read_heat_rows(start, end)
if df.empty:
return _EMPTY
df["stock_code"] = df["stock_code"].astype(str).str.strip()
df["factor_value"] = pd.to_numeric(df["factor_value"], errors="coerce")
df = common.universe_filter(df, col="stock_code", as_prefix=True)
df["factor_value"] = pd.to_numeric(df["score"], errors="coerce")
return df[["trade_date", "stock_code", "factor_value"]]
def _price_code_col() -> str:
"""gp_day_data 的代码列名(平台实测 = symbol,非 ts_code)。
用 LIMIT 1 探列名,而不是拿全区间查询去试错——原来那种写法在 history 模式下
用 LIMIT 0 探列名,而不是拿全区间查询去试错——原来那种写法在 history 模式下
第一次试探就会拉一次全区间数据。"""
cands = [config.PRICE_CODE_COL] + [c for c in ("symbol", "ts_code")
if c != config.PRICE_CODE_COL]
last = None
for c in cands:
try:
db.read_mysql("price", f"SELECT `{c}` FROM gp_day_data LIMIT 1")
scope.sql_filter(f"`{c}`") # 列名来自配置,拼 SQL 前校验。
db.read_mysql("price", f"SELECT `{c}` FROM gp_day_data LIMIT 0")
print(f" (现价用 gp_day_data.{c})")
return c
except Exception as e: # noqa: BLE001 —— 列名不对就换下一个候选
@ -91,12 +104,12 @@ def _read_gp_price(start, end):
parts.append(db.read_mysql(
"price",
f"SELECT `timestamp` AS trade_date, `{col}` AS ts_code, close "
f"FROM gp_day_data WHERE `timestamp` BETWEEN %s AND %s",
f"FROM gp_day_data WHERE `timestamp` BETWEEN %s AND %s AND " + scope.sql_filter(f"`{col}`"),
(cur.date().isoformat(), hi.date().isoformat())))
cur = hi + pd.Timedelta(days=1)
if not parts:
return pd.DataFrame(columns=["trade_date", "ts_code", "close"])
return pd.concat(parts, ignore_index=True)
return scope.frame(pd.concat(parts, ignore_index=True), "ts_code")
# ---------------------------------------------------------------- 预期空间
@ -112,9 +125,10 @@ def build_upside(start, end):
"""
cons = db.read_pg(
"SELECT ts_code, asof_date, target_mid_avg FROM v_factor_consensus "
"WHERE asof_date <= %s", (end,))
"WHERE asof_date <= %s AND " + scope.sql_filter("ts_code", postgres=True), (end,))
if cons.empty:
return _EMPTY
cons = scope.frame(cons, "ts_code")
cons["k"] = cons["ts_code"].map(common.to_prefix)
# 注:consensus_daily 本就只对池成员聚合(基座 _pool_ts_codes),
# 故 market 模式下 upside 覆盖不会真的变宽——这里过滤只为口径一致。
@ -164,12 +178,12 @@ def build_event(start, end):
ev = db.read_pg(
"SELECT ts_code, disclosure_date, event_type, direction, "
" confidence, doc_id, source_type "
"FROM v_factor_events WHERE disclosure_date BETWEEN %s AND %s", (look, end))
"FROM v_factor_events WHERE disclosure_date BETWEEN %s AND %s AND " + scope.sql_filter("ts_code", postgres=True), (look, end))
except Exception as e: # noqa: BLE001 —— 视图还是 v1(无 doc_id/source_type)时退回
print(f" (v_factor_events 无 doc_id/source_type,退回旧列——建议先更新视图: {e!r})")
ev = db.read_pg(
"SELECT ts_code, disclosure_date, event_type, direction "
"FROM v_factor_events WHERE disclosure_date BETWEEN %s AND %s", (look, end))
"FROM v_factor_events WHERE disclosure_date BETWEEN %s AND %s AND " + scope.sql_filter("ts_code", postgres=True), (look, end))
if ev.empty:
return _EMPTY
ev = common.universe_filter(ev, col="ts_code", as_prefix=False).copy()
@ -249,17 +263,18 @@ def build_transmission(start, end):
try:
tr = db.read_pg(
f"SELECT {cols_new} FROM v_factor_transmission "
f"WHERE scan_date BETWEEN %s AND %s", (start, end))
f"WHERE scan_date BETWEEN %s AND %s AND " + scope.sql_filter("ts_code", postgres=True), (start, end))
col_val = "n_sources"
except Exception as e: # noqa: BLE001 —— 视图还是 v1 时退回,但明确告警
print(f" ⚠️ v_factor_transmission 还是旧版(无 n_sources)——退回 n_paths,"
f"该口径把重复路径计入了强度,请尽快更新视图: {e!r}")
tr = db.read_pg(
"SELECT scan_date, ts_code, n_paths, moved_ratio "
"FROM v_factor_transmission WHERE scan_date BETWEEN %s AND %s", (start, end))
"FROM v_factor_transmission WHERE scan_date BETWEEN %s AND %s AND " + scope.sql_filter("ts_code", postgres=True), (start, end))
col_val = "n_paths"
if tr.empty:
return _EMPTY
tr = scope.frame(tr, "ts_code")
uni = common.load_universe() # 传导恒按池过滤(天生池内语义)
tr = tr[tr["ts_code"].isin(uni)].copy()
if tr.empty:
@ -324,7 +339,7 @@ def _robust_z(v: pd.Series) -> pd.Series:
def _heat_day_for(ds: str):
"""热度 T+1 到达 → 取 <= ds 的最新热度日(与 probe / 设计 §3.4 末口径一致)。"""
df = db.read_mysql("heat", "SELECT MAX(trade_date) d FROM stock_fund_heat_scores "
"WHERE trade_date <= %s", (ds,))
"WHERE trade_date <= %s AND " + scope.sql_filter("stock_code"), (ds,))
v = None if df.empty else df.iloc[0, 0]
return None if v is None or pd.isna(v) else pd.Timestamp(v).date().isoformat()
@ -333,8 +348,8 @@ def _col(df: pd.DataFrame, name: str) -> pd.Series:
"""子因子 builder 输出 → 前缀码索引的一列(同股同日取最大)。"""
if df is None or df.empty:
return pd.Series(dtype=float, name=name)
x = df.copy()
x["k"] = x["stock_code"].map(common.to_prefix)
x = scope.frame(df, "stock_code", prefix=True)
x["k"] = x["stock_code"]
return x.groupby("k")["factor_value"].max().astype(float).rename(name)
@ -369,10 +384,12 @@ def _risk_set() -> set:
if isinstance(ms, str):
ms = json.loads(ms)
for m in ms or []:
ts = (m or {}).get("ts_code")
ts = scope.normalize((m or {}).get("ts_code"), prefix=True)
if not ts:
continue
name = str((m or {}).get("name") or "").replace(" ", "")
if ts and (name.startswith(_RISK_PREFIXES) or name.endswith("退")):
out.add(common.to_prefix(ts))
out.add(ts)
if out:
print(f" (重大风险闸: {len(out)} 只 ST/退市族挡在档位外,"
f"EXCLUDE_RISK_NAMES=0 可关)")
@ -386,7 +403,7 @@ def _track_set():
try:
import tracks
df, _missing = tracks.resolve_members(only_confirmed=True)
return {common.to_prefix(x) for x in df["ts_code"]}
return {code for x in df["ts_code"] if (code := scope.normalize(x, prefix=True))}
except Exception as e: # noqa: BLE001 —— 解析失败按闸未启用降级,只告警不中止
print(f" ⚠️ 赛道成员表解析失败,C 闸按未启用处理: {e!r}")
return None

View File

@ -36,6 +36,7 @@ import pandas as pd
import config
import db
import version
import market_scope as scope
try: # parquet 更省更快,但不强制装 pyarrow
import pyarrow # noqa: F401
@ -47,7 +48,7 @@ except Exception: # noqa: BLE001
# ---------------------------------------------------------------- 落盘
def _write(df: pd.DataFrame, path: Path) -> int:
path.parent.mkdir(parents=True, exist_ok=True)
df = pd.DataFrame() if df is None else df
df = pd.DataFrame() if df is None else scope.frozen_frame(df)
# 空也要留痕:0 行本身是信息("那天上游真的没有数据" ≠ "那天没跑")
if _FMT == "parquet":
df.to_parquet(path, index=False)
@ -67,55 +68,49 @@ def _git_rev() -> str:
# ---------------------------------------------------------------- 各源抓取
def _cap_universe(_day: str) -> pd.DataFrame:
"""覆盖池当日成员(industry_pools 无历史版本 → 只能靠这里留时点)。"""
return db.read_pg("SELECT ts_code FROM v_factor_universe ORDER BY ts_code")
return scope.frame(db.read_pg("SELECT ts_code FROM v_factor_universe WHERE " +
scope.sql_filter("ts_code", postgres=True) + " ORDER BY ts_code"), "ts_code")
def _cap_transmission(day: str) -> pd.DataFrame:
"""传导原始行。四路里唯一「过期即不可复原」的推断类信号,最该冻结。"""
try:
return db.read_pg(
"SELECT * FROM v_factor_transmission WHERE scan_date = %s "
"ORDER BY ts_code", (day,))
except Exception: # noqa: BLE001 —— 旧视图无 rank/target 列时的兜底
return db.read_pg(
"SELECT * FROM v_factor_transmission WHERE scan_date = %s", (day,))
"""传导原始行仅冻结沪深证券;上游结构与日期保持。"""
rows = db.read_pg("SELECT * FROM v_factor_transmission WHERE scan_date = %s AND " +
scope.sql_filter("ts_code", postgres=True) + " ORDER BY ts_code", (day,))
return scope.frozen_frame(rows)
def _cap_consensus(day: str) -> pd.DataFrame:
"""当日可见的最新一致预期(as-of 口径与 build_upside 一致,每股留最新一行)。"""
return db.read_pg(
rows = db.read_pg(
"SELECT DISTINCT ON (ts_code) ts_code, asof_date, target_mid_avg, "
" eps_med, np_med, n_orgs, n_reports_90d "
"FROM v_factor_consensus WHERE asof_date <= %s "
"ORDER BY ts_code, asof_date DESC", (day,))
"FROM v_factor_consensus WHERE asof_date <= %s AND " +
scope.sql_filter("ts_code", postgres=True) + " ORDER BY ts_code, asof_date DESC", (day,))
return scope.frame(rows, "ts_code")
def _cap_price(day: str) -> pd.DataFrame:
"""当日收盘(**前复权**,会被后续除权整体重写 → 必须冻结当时看到的值)。"""
import factors # 复用同一套代码列探测,口径不分叉
col = factors._price_code_col() # noqa: SLF001
return db.read_mysql(
rows = db.read_mysql(
"price",
f"SELECT `timestamp` AS trade_date, `{col}` AS code, close "
f"FROM gp_day_data WHERE `timestamp` = %s", (day,))
f"FROM gp_day_data WHERE `timestamp` = %s AND " + scope.sql_filter(f"`{col}`"), (day,))
return scope.frame(rows, "code")
def _cap_heat(day: str) -> pd.DataFrame:
"""当日最新批次热度(T+1 到达,故 day 当天常为空——空也留痕)。"""
return db.read_mysql(
"heat",
"""SELECT s.trade_date, s.stock_code, s.batch_no, s.score
FROM stock_fund_heat_scores s
JOIN (SELECT trade_date, MAX(batch_no) bn FROM stock_fund_heat_scores
WHERE trade_date = %s GROUP BY trade_date) m
ON m.trade_date = s.trade_date AND m.bn = s.batch_no""", (day,))
"""与因子计算同一沪深日频批次,空结果仍按原口径留痕。"""
from factors import _read_heat_rows
return _read_heat_rows(day, day)
def _cap_optional(sql: str):
"""可选视图(第五/六个插槽视图还没建时不该让冻结失败)。"""
def _f(_day: str) -> pd.DataFrame:
try:
return db.read_pg(sql)
return scope.frozen_frame(db.read_pg(sql))
except Exception: # noqa: BLE001
return pd.DataFrame()
return _f
@ -128,7 +123,8 @@ _SOURCES = {
"price_close": _cap_price,
"heat": _cap_heat,
# 第五/六个插槽视图建好后自动开始冻结(没建 → 空 DataFrame,不报错)
"segment_members": _cap_optional("SELECT * FROM v_factor_segment_members"),
"segment_members": _cap_optional("SELECT * FROM v_factor_segment_members WHERE " +
scope.sql_filter("ts_code", postgres=True)),
"segment_edges": _cap_optional("SELECT * FROM v_factor_segment_edges"),
}
@ -176,7 +172,7 @@ def snapshot(day: str | None = None, extra_frames: dict | None = None) -> dict:
for name, fn in _SOURCES.items():
try:
df = fn(day)
df = scope.frozen_frame(fn(day))
frames[name] = df
manifest["rows"][name] = _write(df, out / f"{name}{_EXT}")
except Exception as e: # noqa: BLE001 —— 单源失败不拖累其余源

108
market_scope.py Normal file
View File

@ -0,0 +1,108 @@
"""沪深 A 股样本边界;只规范化代码,不推断上市资格或补造资料。"""
from __future__ import annotations
import json
import re
# 与项目证券入口一致:沪市 60/68、深市 00/30;显式市场必须与号段相符。
_CODE = re.compile(r'^(?:(SH|SZ)([0-9]{6})|([0-9]{6})[.](SH|SZ)|([0-9]{6}))$')
_SQL_CODE = ('^(SH(60|68)[0-9]{4}|(60|68)[0-9]{4}[.]SH|(60|68)[0-9]{4}'
'|SZ(00|30)[0-9]{4}|(00|30)[0-9]{4}[.]SZ|(00|30)[0-9]{4})$')
_CODE_FIELDS = ('ts_code', 'stock_code', 'symbol', 'code')
_MEMBER_FIELDS = frozenset(('members', 'quiet', 'moved_members', 'constituents', 'stocks',
'ts_codes', 'stock_codes', 'symbols'))
_DROP = object()
def normalize(value, *, prefix=False):
"""前缀、点式、六位裸码 → 唯一代码;北交所、错市场、非法值返回 None。"""
if value is None or isinstance(value, bool):
return None
hit = _CODE.fullmatch(str(value).strip().upper())
if not hit:
return None
explicit = hit.group(1) or hit.group(4)
number = hit.group(2) or hit.group(3) or hit.group(5)
market = 'SH' if number.startswith(('60', '68')) else 'SZ' if number.startswith(('00', '30')) else None
if market is None or (explicit and explicit != market):
return None
return market + number if prefix else number + '.' + market
def sql_filter(column, *, postgres=False):
"""仅接受本地列名,SQL 在读取前排除范围外证券;内存仍二次核验。"""
if not re.fullmatch(r'(?:[A-Za-z_][A-Za-z0-9_]*[.])?`?[A-Za-z_][A-Za-z0-9_]*`?', column):
raise ValueError('证券代码列名非法')
cast = 'TEXT' if postgres else 'CHAR'
op = '~' if postgres else 'REGEXP'
return f"UPPER(TRIM(CAST({column} AS {cast}))) {op} '{_SQL_CODE}'"
def frame(df, column, *, prefix=False):
"""样本入口过滤后再计算;不修改调用方 DataFrame。"""
if df is None or df.empty:
return df
if column not in df.columns:
raise ValueError('证券样本缺少代码列:' + column)
codes = df[column].map(lambda value: normalize(value, prefix=prefix))
result = df.loc[codes.notna()].copy()
result[column] = codes.loc[codes.notna()]
return result
def _nested(value, *, members=False):
encoded = isinstance(value, str) and value.lstrip().startswith(('[', '{'))
if encoded:
try:
value = json.loads(value)
except ValueError:
if members:
raise ValueError("冻结成员不是合法 JSON") from None
return value
if isinstance(value, dict):
code_fields = [key for key in _CODE_FIELDS if key in value]
if code_fields:
codes = [normalize(value[key]) for key in code_fields]
if None in codes or len(set(codes)) != 1:
return _DROP
elif members:
return _DROP # 未能确认证券身份的成员不新增归档。
out = {}
for key, val in value.items():
if key in code_fields:
out[key] = normalize(val, prefix=key == 'stock_code')
continue
clean = _nested(val, members=key in _MEMBER_FIELDS)
if clean is not _DROP:
out[key] = clean
elif isinstance(value, (list, tuple)):
out = []
for item in value:
clean = _nested(item, members=members)
if clean is not _DROP:
out.append(clean)
elif members:
out = normalize(value)
if out is None:
return _DROP
else:
out = value
return json.dumps(out, ensure_ascii=False) if encoded else out
def frozen_frame(df):
"""过滤冻结中的证券行与嵌套成员;行业关系等非证券行保持。"""
if df is None or df.empty:
return df
out = df.copy()
for column in _CODE_FIELDS:
if column in out.columns:
out = frame(out, column, prefix=column == 'stock_code')
for column in out.columns:
if column in _CODE_FIELDS:
continue
def clean(value):
result = _nested(value, members=column in _MEMBER_FIELDS)
return None if result is _DROP else result
out[column] = out[column].map(clean)
return out

95
plan.py
View File

@ -21,10 +21,12 @@ from __future__ import annotations # 注解不在定义时求值:开发机
import datetime as dt
import json
import os
import re
import pandas as pd
import card
import rating_priority
import common
import config
import judgement
@ -159,17 +161,21 @@ def _rerank_by_axis(ranked, cards: dict, axis: str, stale_days: int, has_tier: b
def _key(item):
k = item[0]
s = float(item[1])
available = pd.notna(s)
c = cards.get(k) or {}
band = c.get("quality_band")
if band is None:
band = card.quality_band(c.get("company_review"), stale_days)
pointed = bool((c.get("gates") or {}).get("pointed"))
if has_tier:
if not available:
group, s = -float('inf'), -float('inf')
elif has_tier:
tier = int((s - 190.0) // 20)
group = s - 200.0 - tier * 20.0 # 剥掉档位:传导档位不进次序
else:
group = s # 观察档无档位,带内直接按原始分
return (-int(band), not pointed, -group, -s, k) # 每层升序排,负号即降序
priority = rating_priority.key(c.get('company_review')) if config.PLAN_RATING_PRIORITY_ENABLED else 0
return (-int(band), -priority, not pointed, -group, -s, k) # 每层升序排,负号即降序
order = [k for k, _ in sorted(ranked.items(), key=_key)]
return ranked.reindex(order)
@ -373,7 +379,7 @@ def valuations_for(codes: list, ds: str) -> dict:
def _assemble_cards(ds: str, codes: list, ev: dict, upside: pd.Series,
mkt_days: set, risk: set | None = None) -> tuple[dict, list]:
mkt_days: set, risk: set | None = None, user_candidates=None) -> tuple[dict, list]:
"""候选卡装配(2026-09-02 方案第 2.2 节第三项):对档位表里的全部票(主榜与观察档,
裁剪之前)读三路证据、逐票判决。规则在 card.py,取数在 sources.py,这里只做对齐。
@ -429,7 +435,7 @@ def _assemble_cards(ds: str, codes: list, ev: dict, upside: pd.Series,
evd = {"pointed": bool(mv or e), "theme": theme, "n_sources": n_sources,
"chain_fit": (mv or {}).get("chain_fit"),
"pct0": d.get("pct0"), "covered": up is not None, "upside": up,
"risk_name": k in risk,
"risk_name": k in risk, "user_selected": k in (user_candidates or {}),
"accum_state": n.get("accum_state"), "accum_score": n.get("accum_score"),
"accum_age": n.get("accum_age"), "y_signal": n.get("signal"),
"stale_snapshot": stale, "logic": logic.get(k) or []}
@ -442,6 +448,7 @@ def _assemble_cards(ds: str, codes: list, ev: dict, upside: pd.Series,
confirm_days=config.LOGIC_SETTLE_CONFIRM_DAYS, exit_days=config.LOGIC_SETTLE_EXIT_DAYS)
j = card.judge(evd, start_pct=config.CARD_START_PCT,
accum_max_age=config.CARD_ACCUM_MAX_AGE,
require_accum_confirm=config.CARD_ACCUM_CONFIRM_REQUIRED,
neg_tol=config.UPSIDE_NEG_TOLERANCE,
logic_stale_days=config.LOGIC_STALE_DAYS,
require_started=config.CARD_REQUIRE_STARTED,
@ -466,7 +473,7 @@ def _assemble_cards(ds: str, codes: list, ev: dict, upside: pd.Series,
"pressure": n.get("pressure"), "date": n.get("conclusion_date")}
if n else None),
}
for i, (k, c) in enumerate(sorted(cards.items(), key=lambda kv: (*card.sort_key(kv[1], config.CARD_SORT_BY_QUALITY), kv[0])), 1):
for i, (k, c) in enumerate(sorted(cards.items(), key=lambda kv: (*card.sort_key(kv[1], config.CARD_SORT_BY_QUALITY, config.PLAN_RATING_PRIORITY_ENABLED), kv[0])), 1):
c["card_rank"] = i
segs: dict = {}
@ -501,7 +508,7 @@ def _assemble_cards(ds: str, codes: list, ev: dict, upside: pd.Series,
def collect(date: str | None = None, top: int = 20, obs_top: int = 10,
theme_cap: int = 5) -> dict:
theme_cap: int = 5, user_candidates=None) -> dict:
"""装配一天的计划为结构化字典。数据缺失抛 RuntimeError(api 侧转 404)。
2026-09-02 起附带候选卡:main / observe 的装配、排序、裁剪一字不动(下游 PMS 只读
@ -529,16 +536,35 @@ def collect(date: str | None = None, top: int = 20, obs_top: int = 10,
names = _names()
ev, mkt_days = _evidence(ds)
import user_candidates as uc
score = score[[bool(re.fullmatch(r'(SH(?:60|68)\d{4}|SZ(?:00|30)\d{4})', str(k))) for k in score.index]]
selected = {uc.normalize(r['code']): r for r in (user_candidates or [])}
if selected:
# 完整证券名称随真实修订录制,重放时不回源,也不补造历史名称。
manual_names = plan_inputs.read('user_candidates.catalog', uc.catalog)
names = dict(names)
for code in selected:
if manual_names.get(code):
names[code] = manual_names[code]
ordinary = set(score.index)
extras = [k for k in selected if k not in ordinary]
main = score[score >= _MAIN_MIN].sort_index().sort_values(ascending=False, kind='mergesort')
obs = score[score < _MAIN_MIN].sort_index().sort_values(ascending=False, kind='mergesort')
if extras:
main = pd.concat([main, pd.Series({k: float('nan') for k in extras})])
score = pd.concat([score, pd.Series({k: float('nan') for k in extras})])
generated_at = dt.datetime.now().isoformat(timespec="seconds")
try: # 风险名单只读一次:候选卡与升降档原因共用
import factors
risk = plan_inputs.factor_read('_risk_set') or set()
except Exception: # noqa: BLE001
risk = set()
if selected and config.EXCLUDE_RISK_NAMES:
risk = set(risk) | {k for k in selected
if str(names.get(k) or '').replace(' ', '').startswith(factors._RISK_PREFIXES)
or str(names.get(k) or '').endswith('退')}
cards, segments_pointed = _assemble_cards(
ds, list(main.index) + list(obs.index), ev, upside, mkt_days, risk=risk)
ds, list(main.index) + list(obs.index), ev, upside, mkt_days, risk=risk, user_candidates=selected)
# 排序轴(2026-09-17):拿到评析后在主榜与观察档这两个分数序列上重排一次。
# 下面的 _pick 出接口、_full_rows 出复盘快照、快照又喂逐票逻辑状态入库与评析目标名单——
@ -582,10 +608,10 @@ def collect(date: str | None = None, top: int = 20, obs_top: int = 10,
evidence = {"theme": c["theme"], "n_sources": c.get("n_sources"),
"moved_ratio": None, "source": "moved_view"}
r = {"rank": rank, "code": k, "name": names.get(k),
"score": round(float(s), 2),
"score": round(float(s), 2) if pd.notna(s) else None,
# 调整分只随行展示、供排序追溯;档位标签仍按原始 score 解码(工作包四)
"score_adj": round(
float(s) + _quality_bonus(c.get("company_review"), config.PLAN_QUALITY_BONUS), 2),
float(s) + _quality_bonus(c.get("company_review"), config.PLAN_QUALITY_BONUS), 2) if pd.notna(s) else None,
"evidence": evidence,
"heat": _val(heat, k), "upside": _val(upside, k)}
# 排序轴四键(2026-09-17):质地带、主轴名、是否被指向(候选卡门槛一)、旧序全量名次。
@ -598,7 +624,7 @@ def collect(date: str | None = None, top: int = 20, obs_top: int = 10,
r["pointed"] = bool((c.get("gates") or {}).get("pointed"))
if config.PLAN_RANK_AXIS == "quality":
r["rank_old"] = rank_old_map.get(k)
if with_tier:
if with_tier and pd.notna(s):
r["tier"] = _tier_label(s)
if c:
# 2026-09-03 新增只联入:basis(判决依据一句话)、logic(因果论断带出处的文字行)、
@ -623,7 +649,9 @@ def collect(date: str | None = None, top: int = 20, obs_top: int = 10,
industry_catalyst_text=c.get("industry_catalyst_text"),
# 公司深度(2026-09-09 接入方案):质地档与评议摘要,原值给程序,整句给人;
# 质地进逻辑状态第五路与判决收敛(只降不升),评议文字只给人看与送研判。
company_review=c.get("company_review"), company_review_text=c.get("company_review_text"),
company_review=({key: value for key, value in c['company_review'].items() if key != 'rating_priority'} if c.get('company_review') else None),
rating_priority=(c.get('company_review') or {}).get('rating_priority'),
company_review_text=c.get("company_review_text"),
company_gate=c.get("company_gate"),
card={"pct0": c.get("pct0"), "net_z": c.get("net_z"),
"heat_chg": c.get("heat_chg"), "accum": c.get("accum"),
@ -631,6 +659,11 @@ def collect(date: str | None = None, top: int = 20, obs_top: int = 10,
"confirm": c.get("confirm"), "failed_gates": c.get("failed_gates"),
"chain_fit": c.get("chain_fit"),
"logic_claims": c.get("logic_claims") or []})
if k in selected:
r['user_candidate'] = selected[k]
r['user_required'] = k not in ordinary or not r['pointed']
if not names.get(k):
r.update(verdict=card.VERDICT_SHOW, basis='证券资料尚未齐全,暂不参与交易')
return r
def _full_rows(ranked: pd.Series, with_tier: bool) -> list:
@ -727,6 +760,7 @@ def collect(date: str | None = None, top: int = 20, obs_top: int = 10,
for v in (card.VERDICT_CANDIDATE, card.VERDICT_WATCH, card.VERDICT_SHOW)}
result = {
"date": ds,
"user_candidates": list(selected.values()),
"generated_at": generated_at,
"plan_version": version.git_short_rev(),
"counts": {"main": int(len(main)), "observe": int(len(obs)),
@ -746,14 +780,22 @@ def collect(date: str | None = None, top: int = 20, obs_top: int = 10,
"transmission 时按 09-14 的质地调整分序",
# ---- 候选卡(2026-09-02):分数与档位之外的另一份产物,不受 top / theme_cap 裁剪 ----
"card_params": {"start_pct": config.CARD_START_PCT,
"accum_confirm_required": config.CARD_ACCUM_CONFIRM_REQUIRED,
"admission_rule": ("accum-confirm-v1" if config.CARD_ACCUM_CONFIRM_REQUIRED
else "accum-reference-v2-20261104"),
"require_started": config.CARD_REQUIRE_STARTED,
"accum_max_age": config.CARD_ACCUM_MAX_AGE,
"neg_tol": config.UPSIDE_NEG_TOLERANCE,
"logic_per_stock": config.LOGIC_CLAIMS_PER_STOCK,
"rules": "候选=环节被指向∧当日涨幅达标∧券商覆盖且非ST∧明确吸筹∧无硬风险;"
"关注=三门槛全过∧无硬风险∧确认线缺失或陈旧(系统无法判断,交人裁决);"
"其余仅展示(只差覆盖、潜在吸筹等非明确状态都不升格,台账 013);"
"因果论断只展示不进判决"},
"rules": ("候选=环节被指向∧当日涨幅达标∧券商覆盖且非ST∧明确吸筹∧无硬风险;"
"关注=三门槛全过∧无硬风险∧确认线缺失或陈旧(系统无法判断,交人裁决);"
"其余仅展示(只差覆盖、潜在吸筹等非明确状态都不升格,台账 013);"
"因果论断只展示不进判决" if config.CARD_ACCUM_CONFIRM_REQUIRED else
"质地排序不变,候选须满足现行选股资格、覆盖及风险条件;"
"吸筹仅作参考,非明确、潜在、缺评分或陈旧本身不否决;"
"高位派发及其他真实风险仍拦截,质地差仍按现行规则降为关注;"
"建仓看基本面、技术面和择时三源合议,由PMS按命令、风控与现行自决档处理;"
"因果论断只展示不进判决")},
"card_counts": card_counts,
"candidates": _by_verdict(card.VERDICT_CANDIDATE),
"watch": _by_verdict(card.VERDICT_WATCH),
@ -769,7 +811,10 @@ def collect(date: str | None = None, top: int = 20, obs_top: int = 10,
out[b] = out.get(b, 0) + 1
return out
result["rank_axis"] = axis
result["rating_priority_policy"] = {"enabled": config.PLAN_RATING_PRIORITY_ENABLED, "days": config.PLAN_RATING_PRIORITY_DAYS}
result["rank_rule"] = "质地带(好3中2无评析1差0)降序 → 带内被指向优先 → 组内分 → 原始分 → 代码"
if config.PLAN_RATING_PRIORITY_ENABLED:
result['rank_rule'] = result['rank_rule'].replace('带内被指向优先', '带内近30天评级上调优先、下调后置,再按被指向优先')
result["quality_band_counts"] = {"main": _band_counts(result["_full"]["main"]),
"observe": _band_counts(result["_full"]["observe"])}
return result
@ -948,8 +993,8 @@ def render_md(d: dict) -> str:
L.append("| # | 代码 | 名称 | 总分 | 档位 | 传导证据 | 热度 | 预期空间 |")
L.append("|---|------|------|------|------|----------|------|----------|")
for r in d["main"]:
L.append(f"| {r['rank']} | {r['code']} | {r['name'] or '—'} | {r['score']:.1f} "
f"| {r['tier']} | {_fmt_ev(r['evidence'])} "
L.append(f"| {r['rank']} | {r['code']} | {r['name'] or '—'} | {_fmt_num(r['score'])} "
f"| {r.get('tier') or '—'} | {_fmt_ev(r['evidence'])} "
f"| {_fmt_num(r['heat'])} | {_fmt_pct(r['upside'])} |")
L.append("")
@ -960,7 +1005,7 @@ def render_md(d: dict) -> str:
L.append("| # | 代码 | 名称 | 分 | 传导证据 | 热度 |")
L.append("|---|------|------|----|----------|------|")
for r in d["observe"]:
L.append(f"| {r['rank']} | {r['code']} | {r['name'] or '—'} | {r['score']:.1f} "
L.append(f"| {r['rank']} | {r['code']} | {r['name'] or '—'} | {_fmt_num(r['score'])} "
f"| {_fmt_ev(r['evidence'])} | {_fmt_num(r['heat'])} |")
L.append("")
@ -1046,7 +1091,7 @@ def pick_rows(rows, n: int, theme_cap: int, uncapped_no_theme=None) -> list:
def from_snapshot(date: str | None = None, top: int = 20, obs_top: int = 10,
theme_cap: int = 5, decision_id=None) -> dict:
theme_cap: int = 5, decision_id=None, include_user_candidates=False) -> dict:
"""从当日 JSON 快照读计划,按请求参数截取。**不做任何装配、不查任何库。**
为什么要这条路(2026-09-10,台账 057): collect 每次请求都现场装配整池,实测三十三秒,
@ -1063,6 +1108,8 @@ def from_snapshot(date: str | None = None, top: int = 20, obs_top: int = 10,
ds = date or (decision_id[:10] if decision_id else plan_versions.latest_date(config.PLAN_SNAPSHOT_DIR))
path = os.path.join(config.PLAN_SNAPSHOT_DIR, f"plan_{ds}.json")
snap = plan_versions.load(config.PLAN_SNAPSHOT_DIR, ds, decision_id)
import user_candidates as uc
snap = uc.consumer_view(snap, include_user_candidates)
if str(snap.get("date") or "") != str(ds):
raise RuntimeError(f"快照里的数据日 {snap.get('date')} 与请求的 {ds} 对不上: {path}")
full_main, full_obs = snap.get("main"), snap.get("observe")
@ -1095,7 +1142,7 @@ def generate(date: str | None = None, top: int = 20, obs_top: int = 10,
return _generate(date, top, obs_top, theme_cap)
def _generate(date, top, obs_top, theme_cap):
def _generate(date, top, obs_top, theme_cap, *, before_publish=None):
ds = date or _latest_date('t_factor_akg_score')
# 既有夜间流水线也会触发全量构建。已发布日只能由独立 refresh 修订,
# 不能重新取研究覆盖晨版或把旧格式快照升级成事后生成的真实历史。
@ -1107,10 +1154,12 @@ def _generate(date, top, obs_top, theme_cap):
write_published_md(published)
print(f"复用已发布计划 {ds},未重新采集输入")
return os.path.join(config.PLAN_SNAPSHOT_DIR, f"plan_{ds}.md")
import user_candidates as uc
selected = uc.active(config.PLAN_SNAPSHOT_DIR)
bundle = plan_inputs.Bundle(params=plan_inputs.effective_config(config.module))
try:
with bundle.use():
data = collect(ds, top, obs_top, theme_cap)
data = collect(ds, top, obs_top, theme_cap, **({"user_candidates": selected} if selected else {}))
data['market'] = sources.market_context(data['date'])
except RuntimeError as e:
raise SystemExit(str(e))
@ -1125,8 +1174,10 @@ def _generate(date, top, obs_top, theme_cap):
"main": full.get("main", []), "observe": full.get("observe", []),
"shown_params": {"top": top, "obs_top": obs_top, "theme_cap": theme_cap}}
jpath = os.path.join(config.PLAN_SNAPSHOT_DIR, f"plan_{data['date']}.json")
if before_publish is not None:
before_publish(data['date'])
snap = plan_versions.publish(config.PLAN_SNAPSHOT_DIR, snap,
{'records': bundle.records, 'market': data['market'], 'regime': data.get('regime')},
{'records': bundle.records, 'market': data['market'], 'regime': data.get('regime'), 'user_candidates': selected},
bundle.params, cutoff_at=bundle.cutoff_at, reason='晨版生成')
text = write_published_md(snap)
# ---- 逐票日频逻辑状态表(2026-09-07 第三件桥侧前置):只在这一次生成里写,/plan 的实时重算

View File

@ -113,7 +113,11 @@ class Bundle:
try:
actual = dict(kw)
if name == 'sources.company_reviews':
actual.update(cutoff_at=self.cutoff_at, strict=True)
import config
actual.update(cutoff_at=self.cutoff_at, strict=True,
include_rating=self.params.get('PLAN_RATING_PRIORITY_ENABLED', False) if self.replay else
self.params.get('PLAN_RATING_PRIORITY_ENABLED', config.PLAN_RATING_PRIORITY_ENABLED),
rating_days=self.params.get('PLAN_RATING_PRIORITY_DAYS', 30))
result = fn(*args, **actual)
except Exception as e:
self.records[key] = {'name': name, 'error': type(e).__name__, 'args': pack([args, kw])}
@ -146,6 +150,9 @@ class Config:
bundle = _active.get()
if bundle is not None and name in bundle.params:
return bundle.params[name]
# 老冻结版本从未启用新规则,不能继承今天服务器的开关。
if bundle is not None and bundle.replay and name == 'PLAN_RATING_PRIORITY_ENABLED':
return False
return getattr(self.module, name)

149
plan_recovery.py Normal file
View File

@ -0,0 +1,149 @@
"""自然修订缺当日基础版时,待真实输入齐备后有限补发。
只由 plan-refresh 的缺版分支进入。日期守卫、日更、采集及发布在同一文件锁内;
不调用股票池、模型或评析同步,不续期旧输入,不重跑已有成功基础版。
"""
from __future__ import annotations
import datetime as dt
import json
from pathlib import Path
import build_readiness
import config
import plan_refresh
import plan_versions as pv
def _now():
return dt.datetime.now(dt.timezone.utc)
def _baseline(root, day, now):
"""只认可真实当前决策日的完整首版;既有不合格文件绝不覆盖。"""
latest = pv.latest_date(root)
if latest and latest > day:
return 'blocked_existing_snapshot'
try:
current = pv.load(root, day)
except FileNotFoundError:
return 'missing'
except (ValueError, TypeError):
return 'blocked_existing_snapshot'
try:
if current['date'] != day or current.get('traceability') != 'captured_inputs':
raise ValueError('计划来源不可核验')
pv.envelope(root, current['decision']['id'])
first = pv.load(root, day, morning=True)
cutoff = dt.datetime.fromisoformat(first['decision']['cutoff_at'])
if (cutoff.tzinfo is None or cutoff > now
or cutoff.astimezone(plan_refresh.BJ).date() != now.astimezone(plan_refresh.BJ).date()):
raise ValueError('首版不属于今日')
except (KeyError, TypeError, ValueError, FileNotFoundError):
return 'blocked_existing_snapshot'
return 'already_ready'
def _state(path, local_day, data_day):
if not path.exists():
return {'schema_version': 1, 'decision_date': local_day,
'data_date': data_day, 'attempts': 0}
state = json.loads(path.read_text())
if (not isinstance(state, dict) or state.get('schema_version') != 1
or state.get('decision_date') != local_day or state.get('data_date') != data_day
or type(state.get('attempts')) is not int or state['attempts'] < 0):
raise ValueError('恢复状态不可核验,不能重置重试次数')
if state['attempts']:
last = dt.datetime.fromisoformat(state['last_attempt_at'])
if last.tzinfo is None:
raise ValueError('恢复时刻缺少时区')
return state
def ensure_today(day=None, *, now=None):
"""只接受自然最新日入口;返回可核验状态,失败不伪装成功。"""
clock = (lambda: now) if now is not None else _now
current = clock()
if day is not None:
return {'status': 'explicit_date', 'reason': '指定日期的维护入口不自动重建。'}
if not plan_refresh.in_window(current):
return {'status': 'outside_window'}
expected = build_readiness.expected_data_day(current).isoformat()
root = config.PLAN_SNAPSHOT_DIR
# 有效首版已在时,不因当前来源暂时不可达而重建;后续仍由正常 refresh 修订。
baseline = _baseline(root, expected, current)
if baseline != 'missing':
return {'status': baseline, 'data_date': expected}
try:
ready = build_readiness.check(current)
if ready != expected:
raise RuntimeError('输入日期与当前交易日不一致。')
except RuntimeError as exc:
return {'status': 'waiting_data', 'data_date': expected, 'reason': str(exc)}
with pv.writer(root):
current = clock()
if (not plan_refresh.in_window(current)
or build_readiness.expected_data_day(current).isoformat() != expected):
return {'status': 'outside_window'}
baseline = _baseline(root, expected, current)
if baseline != 'missing':
return {'status': baseline, 'data_date': expected}
local_day = current.astimezone(plan_refresh.BJ).date().isoformat()
path = Path(root) / 'recovery' / (local_day + '.json')
try:
state = _state(path, local_day, expected)
maximum = max(0, min(3, int(config.PLAN_RECOVERY_MAX_ATTEMPTS)))
cooldown = max(900, int(config.PLAN_RECOVERY_COOLDOWN_SECONDS))
if state['attempts'] >= maximum:
return {'status': 'attempts_exhausted', 'attempts': state['attempts'],
'reason': '当天自动构建次数已达上限,保留上次版本。'}
if state['attempts']:
last = dt.datetime.fromisoformat(state['last_attempt_at'])
elapsed = (current - last).total_seconds()
if elapsed < 0:
raise ValueError('恢复时刻晚于当前时刻')
if elapsed < cooldown:
return {'status': 'cooldown', 'attempts': state['attempts'],
'retry_after': (last + dt.timedelta(seconds=cooldown)).isoformat()}
except (ValueError, TypeError, KeyError):
return {'status': 'blocked_state', 'reason': '恢复记录不可核验,未重置次数或重建。'}
# 等锁期间输入可能已变化;未齐只等待,不消耗实际构建次数。
try:
if build_readiness.check(current) != expected:
raise RuntimeError('输入日期与当前交易日不一致。')
except RuntimeError as exc:
return {'status': 'waiting_data', 'data_date': expected, 'reason': str(exc)}
state.update(attempts=state['attempts'] + 1, last_attempt_at=current.isoformat(),
status='building', error=None)
pv.atomic(path, state) # 在实际构建前持久化;进程退出不能退还本次预算。
try:
import run
import plan
built = run._cmd_build('all', 'daily', None, None, None)
current = clock()
if (not plan_refresh.in_window(current)
or build_readiness.expected_data_day(current).isoformat() != expected):
raise RuntimeError('构建完成时已离开允许恢复时段。')
if built != expected or build_readiness.check(current) != expected:
raise RuntimeError('构建完成后输入日期变化,未发布计划。')
def before_publish(data_day):
publishing = clock()
if (data_day != expected or not plan_refresh.in_window(publishing)
or build_readiness.expected_data_day(publishing).isoformat() != expected):
raise RuntimeError('计划采集结束时已离开允许恢复时段或日期不符。')
if build_readiness.check(publishing) != expected:
raise RuntimeError('计划发布前输入日期变化,未发布计划。')
# 持有与 generate 同一文件锁,直接调用内部实现避免递归 flock。
# Bundle 自取实际采集截止时刻;09:30 后首版由 publish 标为 recovery。
plan._generate(expected, 20, 10, 5, before_publish=before_publish)
if _baseline(root, expected, clock()) != 'already_ready':
raise RuntimeError('恢复发布未形成可核验的当前基础版。')
except (Exception, SystemExit) as exc:
state.update(status='failed', completed_at=clock().isoformat(),
error=f'基础版恢复失败({type(exc).__name__}),保留上次已发布版本。')
pv.atomic(path, state)
return {'status': 'failed', 'attempts': state['attempts'], 'reason': state['error']}
state.update(status='recovered', completed_at=clock().isoformat(), error=None)
pv.atomic(path, state)
return {'status': 'recovered', 'data_date': expected, 'attempts': state['attempts']}

View File

@ -43,7 +43,34 @@ def refresh(day=None, *, now=None, regime_value=None):
pending = set(pv.health(root, day).get('pending_codes') or [])
try:
env = pv.envelope(root, old['decision']['id'])
params = copy.deepcopy(env['params'])
# 只允许本次明确批准的吸筹角色参数随真实修订改变。
# 其余冻结参数原样重放,不顺带采用当前环境的排序或资格变化。
previous_confirm = params.get('CARD_ACCUM_CONFIRM_REQUIRED', True)
desired_confirm = bool(config.CARD_ACCUM_CONFIRM_REQUIRED)
policy_changed = previous_confirm != desired_confirm
if policy_changed:
params['CARD_ACCUM_CONFIRM_REQUIRED'] = desired_confirm
pending.update(r['code'] for b in ('main', 'observe') for r in old.get(b, []))
rating_changed = params.get('PLAN_RATING_PRIORITY_ENABLED', False) != config.PLAN_RATING_PRIORITY_ENABLED
params['PLAN_RATING_PRIORITY_ENABLED'] = config.PLAN_RATING_PRIORITY_ENABLED
params['PLAN_RATING_PRIORITY_DAYS'] = config.PLAN_RATING_PRIORITY_DAYS
inputs = copy.deepcopy(env['inputs'])
import user_candidates as uc
selected = uc.active(root, now)
candidates_changed = selected != inputs.get('user_candidates', [])
if candidates_changed:
pending.update(r['code'] for r in selected + inputs.get('user_candidates', []))
inputs['user_candidates'] = selected
pv.health(root, day, pending_codes=sorted(pending), error=None)
if selected and not any(r['name'] == 'user_candidates.catalog' for r in inputs['records'].values()):
# 旧发布版尚未录制完整证券名单时,作为本次真实修订的新输入补齐。
pending.update(r['code'] for r in selected)
pv.health(root, day, pending_codes=sorted(pending), error=None)
catalog_bundle = pi.Bundle(cutoff_at=now.isoformat())
with catalog_bundle.use():
pi.read('user_candidates.catalog', uc.catalog)
inputs['records'].update(catalog_bundle.records)
for rec in inputs['records'].values():
if rec['name'] != 'sources.company_reviews':
continue
@ -52,7 +79,8 @@ def refresh(day=None, *, now=None, regime_value=None):
# 读取失败时无法确定哪些股票变化,保守暂停整个本次观察范围。
pending.update(args[0])
pv.health(root, day, pending_codes=sorted(pending), error=None)
after = sources.company_reviews(*args, **dict(kw, cutoff_at=now.isoformat(), strict=True))
after = sources.company_reviews(*args, **dict(kw, cutoff_at=now.isoformat(), strict=True,
include_rating=params['PLAN_RATING_PRIORITY_ENABLED'], rating_days=params['PLAN_RATING_PRIORITY_DAYS']))
pending.difference_update(args[0])
pending.update(c for c in set(before) | set(after) if before.get(c) != after.get(c))
rec.pop('error', None)
@ -63,20 +91,32 @@ def refresh(day=None, *, now=None, regime_value=None):
return {k: v for k, v in (x or {}).items() if k != 'fetched_at'}
if meaningful(inputs.get('regime')) != meaningful(regime_value):
inputs['regime'] = regime_value
if policy_changed:
pending.update(r['code'] for b in ('main', 'observe') for r in old.get(b, []))
pv.health(root, day, pending_codes=sorted(pending), error=None)
if pv.digest(inputs) == old['decision']['input_hash']:
if pv.digest(inputs) == old['decision']['input_hash'] and not policy_changed and not rating_changed:
pv.health(root, day, pending_codes=[], error=None, decision_id=old['decision']['id'])
return {'status': 'unchanged', 'decision': old['decision']}
bundle = pi.Bundle(records=inputs['records'], params=env['params'], cutoff_at=now.isoformat())
bundle = pi.Bundle(records=None if candidates_changed else inputs['records'],
params=params, cutoff_at=now.isoformat())
if candidates_changed:
# 原有调用继续读冻结输入;新增股票带来的新取数在本次真实修订中录制。
bundle.records = copy.deepcopy(inputs['records'])
shown = old.get('shown_params') or {'top': 20, 'obs_top': 10, 'theme_cap': 5}
with bundle.use():
data = plan.collect(day, **shown)
data = plan.collect(day, **shown, **({'user_candidates': selected} if selected else {}))
inputs['records'] = bundle.records
full = data.pop('_full')
data.update(main_shown=data['main'], observe_shown=data['observe'], shown_params=shown,
main=full['main'], observe=full['observe'], market=inputs.get('market'),
regime=inputs.get('regime'))
published = pv.publish(root, data, inputs, env['params'], cutoff_at=now.isoformat(),
reason='评析更新' if pending else '市场区制更新')
reason = ('恢复吸筹确认规则' if desired_confirm else '吸筹改为三源合议参考') if policy_changed else (
'用户候选更新' if candidates_changed else '评析更新' if pending else '市场区制更新')
if rating_changed:
reason = '启用评级变动三十日排序' if config.PLAN_RATING_PRIORITY_ENABLED else '关闭评级变动排序'
elif config.PLAN_RATING_PRIORITY_ENABLED and pending:
reason = '评析或评级排序期限更新'
published = pv.publish(root, data, inputs, params, cutoff_at=now.isoformat(), reason=reason)
plan.write_published_md(published)
return {'status': 'published', 'decision': published['decision']}
except Exception as e:

View File

@ -233,7 +233,7 @@ def publish(root, plan, inputs, params, *, cutoff_at, reason):
return p
def stock_check(root, code):
def stock_check(root, code, include_user_candidates=False):
"""只读完整发布文件的一只股票。不存在与取数失败严格区分;不按展示名次裁剪。"""
if not re.fullmatch(r'(SH|SZ|BJ)\d{6}', code):
raise ValueError('股票代码无效')
@ -243,20 +243,24 @@ def stock_check(root, code):
raise FileNotFoundError('无已发布计划')
path = Path(root) / f'plan_{day}.json'
health_path = Path(root) / 'versions' / day / 'health.json'
candidates_path = Path(root) / 'user_candidates.json'
def identity(p):
try:
s = p.stat()
return (s.st_dev, s.st_ino, s.st_size, s.st_mtime_ns, s.st_ctime_ns)
except FileNotFoundError:
return None
before = (identity(path), identity(health_path))
before = (identity(path), identity(health_path), identity(candidates_path))
published = load(root, day)
state = health(root, day)
if (day != latest_date(root) or before != (identity(path), identity(health_path))):
import user_candidates as uc
state = dict(state, pending_codes=sorted(set(state.get('pending_codes') or []) | set(uc.live_pending(root, published))))
if (day != latest_date(root) or before != (identity(path), identity(health_path), identity(candidates_path))):
continue
if (published.get('date') != day or not (published.get('decision') or {}).get('id')
or not all(isinstance(published.get(b), list) for b in ('main', 'observe'))):
raise ValueError('完整发布计划不可核验')
published = uc.consumer_view(published, include_user_candidates)
rows = {b: [r for r in published[b] if r.get('code') == code] for b in ('main', 'observe')}
count = sum(map(len, rows.values()))
if count > 1:

45
rating_priority.py Normal file
View File

@ -0,0 +1,45 @@
"""评级变化只影响同质地档排序;计算严格使用计划截止时间,时间不混入评分。"""
import datetime as dt
import json
import re
GRADES={'差':0,'中':1,'好':2}
def stamp(value):
result=dt.datetime.fromisoformat(str(value).replace('Z','+00:00'))
if result.tzinfo is None: raise ValueError('评级时刻必须包含时区')
return result
def evaluate(review, history, cutoff_at, days=30):
result={'priority':0,'status':'missing_history','window_days':days,'change':None,'expires_at':None}
if not isinstance(history,dict): return result
if (review.get('qual')!='ok' or review.get('brain_status')!='ok'
or review.get('overall') not in GRADES):
return {**result,'status':'invalid_review'}
if history.get('review_ref')!=review.get('review_ref') or history.get('grade')!=review.get('overall'):
return {**result,'status':'history_mismatch'}
try:
edge=stamp(cutoff_at)
head=stamp(history['report_ran_at'])
if head>edge or stamp(history['recorded_at'])>edge:
return {**result,'status':'future_history'}
change=history.get('last_change_json')
if isinstance(change,str): change=json.loads(change)
result['history_event_id']=history['event_id']
if not change: return {**result,'status':'baseline'}
before,after=change['before'],change['after']
direction='up' if GRADES[after]>GRADES[before] else 'down' if GRADES[after]<GRADES[before] else 'same'
if (direction not in ('up','down') or direction!=change['direction'] or after!=history['grade']
or any(not re.fullmatch('[0-9a-f]{64}',str(change.get(k) or '')) for k in ('event_id','before_ref','after_ref'))):
raise ValueError('升降记录不完整')
at=stamp(change['at'])
if at>head: raise ValueError('升降发生在报告之后')
expires=at+dt.timedelta(days=days)
active=at<=edge<expires
return {**result,'priority':(1 if direction=='up' else -1) if active else 0,
'status':'active' if active else 'expired','change':change,'expires_at':expires.isoformat()}
except (ValueError,TypeError,KeyError):
return {**result,'status':'invalid_history'}
def key(review):
value=(review or {}).get('rating_priority') or {}
return int(value.get('priority') or 0) if value.get('status')=='active' else 0

View File

@ -102,7 +102,7 @@ def append_to_snapshot(day: str) -> dict:
path = snapshot_path(day)
if not os.path.exists(path):
print(f"快照 {path} 不存在,环境段未写入(状态 {reg['status']})")
return reg
return {**reg, 'publication_status': 'no_morning'}
with open(path, "r", encoding="utf-8") as f:
doc = json.load(f)
# 区制也发布完整修订;旧版不就地改写。旧格式须先生成可追溯晨版。
@ -110,10 +110,10 @@ def append_to_snapshot(day: str) -> dict:
result = plan_refresh.refresh(day, regime_value=reg)
if result['status'] not in ('published', 'unchanged'):
print(f"环境段未发布:{result['status']}")
return reg
return {**reg, 'publication_status': result['status']}
print(f"环境段已写入 {path}:status={reg['status']} weak_count={reg['weak_count']} "
f"weak_day={reg['weak_day']}")
return reg
return {**reg, 'publication_status': result['status']}
def read_section(day: str, key: str):

View File

@ -1,7 +1,7 @@
"""个股深度评析的目标名单(2026-09-09 方案附录甲):每早出计划时把主榜与观察档写进 153 库一张小表,
数据基座的 00:45 任务读它选目标(178 的容器打不到 155 的接口,只能走库)。
表 t_akg_review_targets:plan_date、ts_code(点后缀式 600000.SH)、source(main / observe)、rank、name。
表 t_akg_review_targets:plan_date、ts_code(点后缀式 600000.SH)、source(main / observe / manual)、rank、name。
幂等:删该日行再整批插。写失败不拦计划(try 包住)。"""
from __future__ import annotations
@ -35,13 +35,17 @@ def to_dot(code: str) -> str:
return s
def build_rows(ds: str, main_rows: list, obs_rows: list) -> list[dict]:
def build_rows(ds: str, main_rows: list, obs_rows: list, user_rows=None) -> list[dict]:
from user_candidates import normalize
seen: set = set()
out: list[dict] = []
for src, rows in (("main", main_rows), ("observe", obs_rows)):
for src, rows in (("main", main_rows), ("observe", obs_rows), ("manual", user_rows)):
for i, r in enumerate(rows or [], 1):
code = to_dot(str(r.get("code") or ""))
if len(code) != 9 or code in seen:
try:
code = to_dot(normalize(r.get('code')))
except ValueError:
continue
if code in seen:
continue
seen.add(code)
out.append({"plan_date": ds, "ts_code": code, "source": src, "rank": i, "name": (r.get("name") or "")[:64]})
@ -89,15 +93,18 @@ def pick_main_targets(main_rows: list, axis: str, top_score: int, top_rank: int)
return union
def persist(ds: str, snap: dict, write=None) -> dict:
def persist(ds: str, snap: dict, write=None, user_rows=None) -> dict:
"""目标是主榜(按 REVIEW_TARGETS_AXIS 取,both 是原始分前 TOP_MAIN ∪ 新序前 TOP_MAIN_RANK)加观察档前
TOP_OBS 只(快照里的 main / observe 已是新序全部行,每行带 score 与 rank),约一百二十到一百五十家。"""
main_sel = pick_main_targets(snap.get("main") or [], config.REVIEW_TARGETS_AXIS, TOP_MAIN, TOP_MAIN_RANK)
rows = build_rows(ds, main_sel, (snap.get("observe") or [])[:TOP_OBS])
rows = build_rows(ds, main_sel, (snap.get("observe") or [])[:TOP_OBS],
snap.get('user_candidates', []) if user_rows is None else user_rows)
(write or save)(ds, rows)
n_main = sum(1 for r in rows if r["source"] == "main")
print(f" 评析目标名单 {ds}:主榜 {n_main},观察 {len(rows) - n_main},写入 {TABLE}")
return {"date": ds, "rows": len(rows), "main": n_main}
n_manual = sum(1 for r in rows if r['source'] == 'manual')
n_obs = len(rows) - n_main - n_manual
print(f" 评析目标名单 {ds}:主榜 {n_main},观察 {n_obs},人工关注 {n_manual},写入 {TABLE}")
return {"date": ds, "rows": len(rows), "main": n_main, 'observe': n_obs, 'manual': n_manual}
def from_plan_file(ds: str, plan_dir: str | None = None, write=None) -> dict:

40
run.py
View File

@ -122,6 +122,16 @@ def _latest_data_day() -> str:
def cmd_build(which, mode, start, end, date, do_freeze=True):
# 与计划发布/修订使用同一跨进程锁,避免恢复构建时被另一条日更链改写因子。
if mode == 'daily':
import plan_versions
with plan_versions.writer(config.PLAN_SNAPSHOT_DIR):
return _cmd_build(which, mode, start, end, date, do_freeze)
return _cmd_build(which, mode, start, end, date, do_freeze)
def _cmd_build(which, mode, start, end, date, do_freeze=True):
"""内部构建;daily 调用方必须持有计划发布锁。"""
if mode == "daily":
d = date or _latest_data_day()
start = end = d
@ -149,9 +159,18 @@ def cmd_build(which, mode, start, end, date, do_freeze=True):
if do_freeze and mode == "daily":
try:
import freeze
freeze.snapshot(start, extra_frames=frames)
manifest = freeze.snapshot(start, extra_frames=frames)
required = {'universe', 'transmission', 'consensus', 'price_close', 'heat'} | set(frames)
rows = manifest.get('rows') if isinstance(manifest, dict) else None
if (not isinstance(rows, dict) or required - set(rows)
or manifest.get('errors') != {}):
raise RuntimeError('冻结清单不完整或有来源/落盘错误')
# 0 行是既有合法观测;只核验记录齐全,不新增选股资格门槛。
if any(type(rows[name]) is not int or rows[name] < 0 for name in required):
raise RuntimeError('冻结清单行数不可核验')
except Exception as e: # noqa: BLE001 —— 因子已写,但完整输入缺失不得继续发布
raise RuntimeError("输入冻结失败,停止计划及入池(因子已落库)") from e
return start
def main():
@ -235,10 +254,27 @@ def main():
day = a.date or plan._latest_date("t_factor_akg_score") # noqa: SLF001 —— 同仓自用
if not day:
raise SystemExit("t_factor_akg_score 还没有数据,没有当日快照可追加。")
regime.append_to_snapshot(day)
result = regime.append_to_snapshot(day)
if result.get('publication_status') not in ('published', 'unchanged'):
raise SystemExit('区制未发布:' + str(result.get('publication_status') or 'unknown'))
elif a.cmd == 'plan-refresh':
import plan_refresh
import user_research
try:
user_research.sync()
except Exception:
import logging
logging.getLogger(__name__).exception('人工关注评析同步失败,继续正常计划修订')
result = plan_refresh.refresh(a.date)
if result['status'] in ('no_morning', 'no_morning_today'):
import plan_recovery
recovery = plan_recovery.ensure_today(a.date)
print(recovery)
if recovery['status'] in ('already_ready', 'recovered'):
result = plan_refresh.refresh(a.date)
else:
raise SystemExit('今日完整基础版尚未恢复:' + recovery['status']
+ ';' + recovery.get('reason', '保留上次版本。'))
print(result)
if result['status'] in ('no_morning', 'no_morning_today'):
raise SystemExit('今日完整晨版缺失,保留上次版本,修订未执行。')

View File

@ -40,6 +40,7 @@ import pandas as pd
import common
import config
import db
import market_scope
# 上证指数与深证成指在指数日线表 zs_day_data 里的代码,两市成交额取二者当日 amount 之和。
MARKET_INDEX_CODES = ("000001.SH", "399001.SZ")
@ -57,12 +58,13 @@ def moved_members(ds: str) -> dict[str, dict]:
df = db.read_pg(
"SELECT ts_code, target, n_sources, chain_fit, members_total, moved, "
"moved_ratio, mkt_trade_date FROM v_factor_transmission_moved "
"WHERE scan_date = %s", (ds,))
"WHERE scan_date = %s AND " + market_scope.sql_filter('ts_code', postgres=True), (ds,))
except Exception as e: # noqa: BLE001
print(f" (已动成员视图读取失败,候选卡的传导门槛整体缺席: {e!r})")
return {}
if df.empty:
return {}
df = market_scope.frame(df, 'ts_code')
df["k"] = df["ts_code"].map(lambda s: common.to_prefix(str(s).strip()))
df["n_sources"] = pd.to_numeric(df["n_sources"], errors="coerce").fillna(0)
df["chain_fit"] = pd.to_numeric(df["chain_fit"], errors="coerce").fillna(0)
@ -82,10 +84,11 @@ def stock_daily(ds: str) -> dict[str, dict]:
try:
df = db.read_pg(
"SELECT ts_code, pct_change, net_z, heat_chg FROM v_factor_stock_daily "
"WHERE trade_date = %s", (ds,))
"WHERE trade_date = %s AND " + market_scope.sql_filter('ts_code', postgres=True), (ds,))
except Exception as e: # noqa: BLE001
print(f" (个股日行情视图读取失败,候选卡的已启动门槛整体缺席: {e!r})")
return {}
df = market_scope.frame(df, 'ts_code')
out = {}
for r in df.itertuples():
k = common.to_prefix(str(r.ts_code).strip())
@ -480,10 +483,12 @@ def market_context(ds: str, read_pg=None, read_mysql=None) -> dict:
out["errors"]["turnover"] = repr(e)
print(f" (两市成交额读取失败,环境段该项为空: {e!r})")
# 二、市场广度:基座个股日行情视图当日全部行自算。
# 二、市场广度:读取与样本入口均限定沪深股票,不影响上面的指数成交额。
try:
rows = _records(rpg(
"SELECT pct_change FROM v_factor_stock_daily WHERE trade_date = %s", (ds,)))
"SELECT ts_code, pct_change FROM v_factor_stock_daily WHERE trade_date = %s AND "
+ market_scope.sql_filter('ts_code', postgres=True), (ds,)))
rows = [row for row in rows if market_scope.normalize(row.get('ts_code'))]
out["breadth"] = _breadth(rows)
if out["breadth"] is None:
out["errors"]["breadth"] = "v_factor_stock_daily 当日无行"
@ -786,13 +791,14 @@ def close_prices(ds: str, read_mysql=None, code_col: str | None = None) -> dict:
code_col = factors._price_code_col() # noqa: SLF001 —— 同仓自用
nxt = (dt.date.fromisoformat(ds) + dt.timedelta(days=1)).isoformat()
df = reader("price", f"SELECT `{code_col}` AS ts_code, close FROM gp_day_data "
f"WHERE `timestamp` >= %s AND `timestamp` < %s", (ds, nxt))
f"WHERE `timestamp` >= %s AND `timestamp` < %s AND "
+ market_scope.sql_filter('`' + code_col + '`'), (ds, nxt))
except Exception as e: # noqa: BLE001
print(f" (收盘价读取失败,安全边际这一行整体缺席: {e!r})")
return {}
out = {}
for r in _records(df):
k = _prefix_any(r.get("ts_code"))
k = market_scope.normalize(r.get("ts_code"), prefix=True)
px = _f(r.get("close"))
if k and px and px > 0:
out[k] = px
@ -1174,18 +1180,19 @@ def catalysts_for_codes(seg_of: dict, cat_by_seg: dict, *, limit: int = 3) -> di
# 公司深度评析(2026-09-09 接入方案,台账 053):数据基座 company_review_runs 索引表最新一行的摘要。
# 质地由基座代码合成,这里只读不算;报告正文在对象存储,链接给人。表读不到整体缺席不断产。
# ---------------------------------------------------------------------------
def company_reviews(codes, ds: str, *, read_pg=None, cutoff_at=None, strict=False) -> dict:
def company_reviews(codes, ds: str, *, read_pg=None, cutoff_at=None, strict=False, include_check=False, include_rating=None, rating_days=30) -> dict:
"""{前缀码: {"period", "review_version", "overall", "groups", "valuation", "doubt_hard", "quality_reasons",
"grade_counts", "grade_line", "thesis", "confidence", "invalidation", "antithesis", "catalyst", "brain_status",
"ran_at", "age_days", "report_url"}};没报告的票不在字典里。"""
rp = read_pg or db.read_pg
want = sorted({c for c in (codes or []) if c})
want = sorted({_prefix_any(str(c)) for c in (codes or []) if re.fullmatch(r'(?:SH(?:60|68)|SZ(?:00|30))\d{4}', _prefix_any(str(c)))})
if not want:
return {}
dots = [_to_dot(c) for c in want]
try:
cutoff = cutoff_at or (str(ds) + 'T23:59:59.999999+08:00')
df = rp("SELECT DISTINCT ON (ts_code) ts_code, period, review_version, grades, summary, brain_used, brain_status, ran_at "
checked_column = ', checked_at' if include_check else ''
df = rp("SELECT DISTINCT ON (ts_code) ts_code, period, review_version, grades, summary, brain_used, brain_status, ran_at" + checked_column + " "
"FROM company_review_runs WHERE status = 'ok' AND ts_code = ANY(%s) AND ran_at <= %s "
"ORDER BY ts_code, ran_at DESC, period DESC, review_version DESC", (dots, cutoff))
except Exception as e: # noqa: BLE001
@ -1209,14 +1216,14 @@ def company_reviews(codes, ds: str, *, read_pg=None, cutoff_at=None, strict=Fals
except ValueError:
grades = {}
ran = r.get("ran_at")
edge = dt.datetime.fromisoformat(str(cutoff).replace('Z', '+00:00'))
if edge.tzinfo is None:
edge = edge.replace(tzinfo=dt.timezone.utc)
# 桩与某些代理不保证过滤;本地再守一遍时间边界,绝不把未来行归零当新鲜。
if ran is not None:
stamp = dt.datetime.fromisoformat(str(ran).replace('Z', '+00:00'))
if stamp.tzinfo is None:
stamp = stamp.replace(tzinfo=dt.timezone.utc)
edge = dt.datetime.fromisoformat(str(cutoff).replace('Z', '+00:00'))
if edge.tzinfo is None:
edge = edge.replace(tzinfo=dt.timezone.utc)
if stamp > edge:
continue
try:
@ -1225,6 +1232,8 @@ def company_reviews(codes, ds: str, *, read_pg=None, cutoff_at=None, strict=Fals
except (TypeError, ValueError, AttributeError):
ran_day, age = None, None
k = _prefix_any(str(r.get("ts_code")))
if k not in want:
continue
brain = sm.get("brain") or {}
if age is not None and age < 0:
age = 0 # 报告运行日晚于数据日(今早刚跑):算零天,不算负数
@ -1243,8 +1252,28 @@ def company_reviews(codes, ds: str, *, read_pg=None, cutoff_at=None, strict=Fals
'unknown_n', 'numbers_unmatched', 'review_ref'):
out[k][field] = sm.get(field)
out[k]['report_ran_at'] = stamp.isoformat() if ran is not None else None
if include_check:
checked = r.get('checked_at')
try:
checked = dt.datetime.fromisoformat(str(checked).replace('Z', '+00:00'))
if checked.tzinfo is None:
checked = checked.replace(tzinfo=dt.timezone.utc)
out[k]['checked_at'] = checked.isoformat() if checked <= edge else None
except (TypeError, ValueError):
out[k]['checked_at'] = None
out[k]['report_version_url'] = (config.AKG_API_BASE + sm['report_version_path']
if sm.get('report_version_path') else None)
enabled = config.PLAN_RATING_PRIORITY_ENABLED if include_rating is None else include_rating
if enabled and out:
import rating_priority
# 两张表分开读取,不联表。读取失败不能当作评级下降或静默发布新榜。
history = rp("SELECT DISTINCT ON (ts_code) ts_code, event_id, review_ref, grade, report_ran_at, "
"recorded_at, last_change_json FROM company_review_rating_history "
"WHERE ts_code = ANY(%s) AND report_ran_at <= %s AND recorded_at <= %s "
"ORDER BY ts_code, report_ran_at DESC, recorded_at DESC, event_id DESC", (dots, cutoff, cutoff))
by_code = {_prefix_any(str(r['ts_code'])): r for r in history.to_dict('records')}
for code, value in out.items():
value['rating_priority'] = rating_priority.evaluate(value, by_code.get(code), cutoff, rating_days)
return out

121
test_accum_reference.py Normal file
View File

@ -0,0 +1,121 @@
"""离线:吸筹只作参考及同输入规则修订,全部来源使用内存替身。"""
import copy
import datetime as dt
import tempfile
import unittest
from contextlib import ExitStack
from unittest.mock import patch
import pandas as pd
import test_api_support
import card
import config
import plan
import plan_refresh
import plan_versions as pv
class ReferenceTests(unittest.TestCase):
def test_published_rule_description_matches_confirmation_mode(self):
legacy = ("候选=环节被指向∧当日涨幅达标∧券商覆盖且非ST∧明确吸筹∧无硬风险;"
"关注=三门槛全过∧无硬风险∧确认线缺失或陈旧(系统无法判断,交人裁决);"
"其余仅展示(只差覆盖、潜在吸筹等非明确状态都不升格,台账 013);"
"因果论断只展示不进判决")
with ExitStack() as stack:
for name, value in (('_factor', pd.Series([240.], index=['SH600001'])),
('_latest_date', None), ('_prev_date', None),
('_names', {}), ('_evidence', ({}, set())),
('_assemble_cards', ({}, []))):
stack.enter_context(patch.object(plan, name, return_value=value))
stack.enter_context(patch.object(plan.plan_inputs, 'factor_read', return_value=set()))
stack.enter_context(patch.object(plan.version, 'git_short_rev', return_value='test'))
for enabled in (True, False, True):
with self.subTest(confirm_required=enabled), patch.object(config, 'CARD_ACCUM_CONFIRM_REQUIRED', enabled):
params = plan.collect('2026-11-03')['card_params']
self.assertEqual(params['accum_confirm_required'], enabled)
if enabled:
self.assertEqual(params['admission_rule'], 'accum-confirm-v1')
self.assertEqual(params['rules'], legacy)
else:
self.assertEqual(params['admission_rule'], 'accum-reference-v2-20261104')
for phrase in ('质地排序不变', '吸筹仅作参考', '非明确、潜在、缺评分或陈旧本身不否决',
'现行选股资格', '高位派发及其他真实风险仍拦截',
'基本面、技术面和择时三源合议', '现行自决档'):
self.assertIn(phrase, params['rules'])
self.assertNotIn('∧明确吸筹', params['rules'])
self.assertNotIn('交人裁决', params['rules'])
def test_nonconfirmed_evidence_no_longer_blocks_qualified_stock(self):
for state, age in [('潜在吸筹·低位企稳', 0), ('无吸筹迹象', 0), ('量能温和、信号不明', 1),
('', None), ('明确吸筹', 45), ('明确吸筹', None)]:
ev = dict(pointed=True, covered=True, upside=.2, pct0=3,
accum_state=state, accum_age=age, accum_score=60)
old = card.judge(ev)
new = card.judge(ev, require_accum_confirm=False)
self.assertNotEqual(old['verdict'], card.VERDICT_CANDIDATE)
self.assertEqual(new['verdict'], card.VERDICT_CANDIDATE)
self.assertFalse(new['confirm_required'])
self.assertNotIn('confirm', new['failed_gates'])
self.assertNotIn('没到明确吸筹', str(new))
self.assertIn('基本面、技术面和择时', new['basis'])
def test_risks_and_qualification_remain_effective(self):
base = dict(pointed=True, covered=True, upside=.2, pct0=3, accum_state='潜在吸筹', accum_age=0)
for change in ({'pointed': False}, {'covered': False}, {'upside': -.2}, {'risk_name': True},
{'stale_snapshot': True}, {'y_signal': 'SELL'}, {'y_signal': 'AVOID'},
{'y_signal': 'DROPPED'}, {'accum_state': '高位派发'}):
self.assertEqual(card.judge(dict(base, **change), require_accum_confirm=False)['verdict'],
card.VERDICT_SHOW)
poor = card.judge(base, require_accum_confirm=False,
company_review={'overall': '差', 'age_days': 1})
self.assertEqual(poor['verdict'], card.VERDICT_WATCH)
def test_policy_change_publishes_new_version_without_changing_history(self):
with tempfile.TemporaryDirectory() as root:
now = dt.datetime.fromisoformat('2026-11-04T10:30:00+08:00')
old = {'date': '2026-11-03', 'plan_version': 'test',
'main': [{'code': 'SH600001', 'verdict': '仅展示'}], 'observe': [],
'shown_params': {'top': 20, 'obs_top': 10, 'theme_cap': 5}}
inputs = {'records': {}, 'user_candidates': [], 'market': {}, 'regime': None}
first = pv.publish(root, old, inputs, {}, cutoff_at='2026-11-04T07:10:00+08:00', reason='晨版')
original = copy.deepcopy(first)
def collect(*a, **kw):
self.assertFalse(plan.config.CARD_ACCUM_CONFIRM_REQUIRED)
data = copy.deepcopy(old)
data['_full'] = {'main': [dict(old['main'][0], verdict='候选')], 'observe': []}
return data
with patch.object(config, 'PLAN_SNAPSHOT_DIR', root), \
patch.object(config, 'CARD_ACCUM_CONFIRM_REQUIRED', False), \
patch.object(plan_refresh, 'in_window', return_value=True), \
patch.object(plan, 'collect', side_effect=collect), \
patch.object(plan, 'write_published_md'):
result = plan_refresh.refresh(now=now)
self.assertEqual(result['status'], 'published')
newest = pv.load(root)
self.assertEqual(newest['decision']['reason'], '吸筹改为三源合议参考')
self.assertEqual(newest['decision']['input_hash'], first['decision']['input_hash'])
self.assertNotEqual(newest['decision']['params_hash'], first['decision']['params_hash'])
self.assertEqual(newest['decision']['parent_id'], first['decision']['id'])
self.assertEqual(pv.load(root, decision_id=first['decision']['id']), original)
self.assertEqual(pv.load(root, morning=True), original)
self.assertEqual(plan_refresh.refresh(now=now)['status'], 'unchanged')
def test_rebuild_failure_keeps_old_plan_and_pending_codes(self):
with tempfile.TemporaryDirectory() as root:
old = {'date': '2026-11-03', 'plan_version': 'test',
'main': [{'code': 'SH600001', 'verdict': '仅展示'}], 'observe': []}
inputs = {'records': {}, 'user_candidates': [], 'market': {}, 'regime': None}
first = pv.publish(root, old, inputs, {}, cutoff_at='2026-11-04T07:10:00+08:00', reason='晨版')
with patch.object(config, 'PLAN_SNAPSHOT_DIR', root), \
patch.object(config, 'CARD_ACCUM_CONFIRM_REQUIRED', False), \
patch.object(plan_refresh, 'in_window', return_value=True), \
patch.object(plan, 'collect', side_effect=ValueError('材料损坏')):
with self.assertRaises(ValueError):
plan_refresh.refresh(now=dt.datetime.fromisoformat('2026-11-04T10:30:00+08:00'))
self.assertEqual(pv.load(root), first)
self.assertEqual(pv.health(root, old['date'])['pending_codes'], ['SH600001'])
if __name__ == '__main__':
unittest.main()

View File

@ -20,7 +20,8 @@ except ImportError:
fa.FastAPI = fa.APIRouter = Router
fa.Request = type('Request', (), {})
fa.HTTPException = HTTPException
fa.Header = fa.Query = lambda default=None, **kw: default
fa.Body = fa.Header = fa.Query = lambda default=None, **kw: default
fa.BackgroundTasks = type('BackgroundTasks', (), {'add_task': lambda self, *a, **kw: None})
sys.modules['fastapi'] = fa
resp = types.ModuleType('fastapi.responses')
resp.PlainTextResponse = lambda content, **kw: content

View File

@ -1,6 +1,8 @@
"""本地离线:python -m unittest test_build_readiness;来源与写入全部替身。"""
import datetime as dt
import socket
import re
import sqlite3
import sys
import types
import unittest
@ -46,7 +48,7 @@ class ReadinessTest(unittest.TestCase):
readiness.check(NOW)
def test_missing_heat_fails_even_if_scan_current(self):
with patch.object(readiness.db, 'read_pg', return_value=pd.DataFrame({'d': ['2026-10-16']})), patch.object(readiness.db, 'read_mysql', side_effect=lambda source, sql: pd.DataFrame({'d': ['2026-10-15' if source == 'heat' else '2026-10-16']})):
with patch.object(readiness.db, 'read_pg', return_value=pd.DataFrame({'d': ['2026-10-16']})), patch.object(readiness.db, 'read_mysql', side_effect=lambda source, sql, params=None: pd.DataFrame({'d': ['2026-10-15' if source == 'heat' else '2026-10-16']})):
with self.assertRaisesRegex(RuntimeError, '资金热度日期 2026-10-15'):
readiness.check(NOW)
@ -56,6 +58,55 @@ class ReadinessTest(unittest.TestCase):
readiness.check(NOW)
self.assertNotIn('secret-connection', str(result.exception))
def test_readiness_aggregates_only_shanghai_shenzhen_sources(self):
# SQLite 只替换方言运算符,实际执行三路 MAX 查询,混入域外较新行不能造齐备。
conn = sqlite3.connect(':memory:')
self.addCleanup(conn.close)
conn.create_function('regexp', 2, lambda pattern, value: bool(re.search(pattern, str(value or ''))))
conn.executescript('''
CREATE TABLE v_factor_transmission(ts_code TEXT, scan_date TEXT);
CREATE TABLE gp_day_data(symbol TEXT, timestamp TEXT);
CREATE TABLE stock_fund_heat_scores(stock_code TEXT, trade_date TEXT);
CREATE INDEX transmission_day ON v_factor_transmission(scan_date);
CREATE INDEX price_day ON gp_day_data(timestamp);
CREATE INDEX heat_day ON stock_fund_heat_scores(trade_date);
INSERT INTO v_factor_transmission VALUES ('600001.SH','2026-10-15'),('830001.BJ','2026-10-16');
INSERT INTO gp_day_data VALUES ('600001','2026-10-15'),('830001','2026-10-16');
INSERT INTO stock_fund_heat_scores VALUES ('SH600001','2026-10-15'),('BJ830001','2026-10-16');
''')
bounds = []
def query(sql, params=None):
translated = sql.replace(' ~ ', ' REGEXP ').replace('%s', '?')
if 'MAX(' in sql:
self.assertEqual(params, ('2026-10-16',))
self.assertIn('>= %s AND', sql)
plan = ' '.join(str(row[-1]) for row in conn.execute('EXPLAIN QUERY PLAN ' + translated, params))
self.assertIn('SEARCH', plan)
self.assertIn('INDEX', plan)
bounds.append(params)
cursor = conn.execute(translated, params or ())
return pd.DataFrame(cursor.fetchall(), columns=[d[0] for d in cursor.description])
with patch.object(readiness.db, 'read_pg', side_effect=query), patch.object(readiness.db, 'read_mysql', side_effect=lambda source, sql, params=None: query(sql, params)):
with self.assertRaisesRegex(RuntimeError, '传导日期 缺失.*收盘行情日期 缺失.*资金热度日期 缺失'):
readiness.check(NOW)
conn.executescript('''
UPDATE v_factor_transmission SET scan_date='2026-10-16' WHERE ts_code='600001.SH';
UPDATE gp_day_data SET timestamp='2026-10-16' WHERE symbol='600001';
UPDATE stock_fund_heat_scores SET trade_date='2026-10-16' WHERE stock_code='SH600001';
UPDATE v_factor_transmission SET scan_date='2026-10-19' WHERE ts_code='830001.BJ';
UPDATE gp_day_data SET timestamp='2026-10-19' WHERE symbol='830001';
UPDATE stock_fund_heat_scores SET trade_date='2026-10-19' WHERE stock_code='BJ830001';
''')
self.assertEqual(readiness.check(NOW), '2026-10-16')
conn.executescript('''
UPDATE v_factor_transmission SET scan_date='2026-10-19' WHERE ts_code='600001.SH';
UPDATE gp_day_data SET timestamp='2026-10-19' WHERE symbol='600001';
UPDATE stock_fund_heat_scores SET trade_date='2026-10-19' WHERE stock_code='SH600001';
''')
with self.assertRaisesRegex(RuntimeError, '传导日期 2026-10-19.*收盘行情日期 2026-10-19.*资金热度日期 2026-10-19'):
readiness.check(NOW)
self.assertEqual(len(bounds), 9)
def test_stale_default_stops_before_any_writes(self):
with patch.object(readiness, 'check', side_effect=RuntimeError('上游停更')), patch.object(run.common, 'write_factor') as write:
with self.assertRaisesRegex(RuntimeError, '上游停更'):
@ -83,13 +134,34 @@ class ReadinessTest(unittest.TestCase):
with self.assertRaisesRegex(RuntimeError, '输入冻结失败'):
run.cmd_build('test', 'daily', None, None, '2026-10-16')
def test_freeze_manifest_errors_are_not_success(self):
frozen=types.ModuleType('freeze')
rows = {name: 0 for name in ('universe', 'transmission', 'consensus', 'price_close', 'heat', 'factor_test')}
for manifest in ({'rows': rows, 'errors': {'heat': 'source failed'}},
{'rows': {'heat': 1}, 'errors': {}}, None,
{'rows': dict(rows, heat=None), 'errors': {}}):
frozen.snapshot=lambda *a, **kw: manifest
with self.subTest(manifest=manifest), patch.dict(sys.modules, {'freeze': frozen}), patch.object(run.factors, 'BUILDERS', {'test': lambda a,b: pd.DataFrame()}), patch.object(run.factors, 'FACTORS', {'test': 't'}), patch.object(run.common, 'write_factor'):
with self.assertRaisesRegex(RuntimeError, '输入冻结失败'):
run.cmd_build('test', 'daily', None, None, '2026-10-16')
def test_complete_freeze_allows_zero_rows_and_optional_empty_inputs(self):
frozen=types.ModuleType('freeze')
rows = {name: 0 for name in ('universe', 'transmission', 'consensus', 'price_close', 'heat', 'factor_test', 'segment_members')}
frozen.snapshot=lambda *a, **kw: {'rows': rows, 'errors': {}}
with patch.dict(sys.modules, {'freeze': frozen}), patch.object(run.factors, 'BUILDERS', {'test': lambda a,b: pd.DataFrame()}), patch.object(run.factors, 'FACTORS', {'test': 't'}), patch.object(run.common, 'write_factor'):
self.assertEqual(run.cmd_build('test', 'daily', None, None, '2026-10-16'), '2026-10-16')
def test_refresh_missing_morning_fails_cli(self):
mod=types.ModuleType('plan_refresh')
recovery=types.ModuleType('plan_recovery')
recovery.ensure_today=lambda *a, **kw: {'status': 'waiting_data', 'reason': '输入未齐'}
research=types.ModuleType('user_research'); research.sync=lambda: None
for status in ('no_morning', 'no_morning_today', 'outside_window', 'unchanged'):
mod.refresh=lambda *a, **kw: {'status': status}
with self.subTest(status=status), patch.dict(sys.modules, {'plan_refresh': mod}), patch.object(sys, 'argv', ['run.py','plan-refresh']):
with self.subTest(status=status), patch.dict(sys.modules, {'plan_refresh': mod, 'plan_recovery': recovery, 'user_research': research}), patch.object(sys, 'argv', ['run.py','plan-refresh']):
if status.startswith('no_morning'):
with self.assertRaisesRegex(SystemExit, '今日完整晨版缺失'): run.main()
with self.assertRaisesRegex(SystemExit, '今日完整基础版尚未恢复.*waiting_data'): run.main()
else: run.main()
if __name__ == '__main__':

View File

@ -11,7 +11,7 @@ import test_api_support # noqa: F401
def main():
suite = unittest.TestSuite()
for name in ('test_plan_versions', 'test_plan_reviews_live', 'test_plan_review_source', 'test_plan_stock_check',
'test_plan_rank_switch'):
'test_plan_rank_switch', 'test_user_candidates', 'test_user_research'):
suite.addTests(unittest.defaultTestLoader.loadTestsFromModule(importlib.import_module(name)))
for name in ('test_plan_cache', 'test_plan_snapshot', 'test_plan_rank_axis',
'test_plan_quality_rank', 'test_plan_verdict', 'test_plan_logic_state',

View File

@ -188,7 +188,7 @@ class PublishTest(unittest.TestCase):
class IntegrationContractTest(unittest.TestCase):
def test_business_defaults_unchanged_and_explicit_entry(self):
import ast
tree = ast.parse(Path('xxl.py').read_text())
tree = ast.parse(Path(__file__).with_name('xxl.py').read_text())
values = {}
for node in tree.body:
if isinstance(node, ast.Assign):
@ -201,7 +201,7 @@ class IntegrationContractTest(unittest.TestCase):
def test_read_query_cannot_collect_or_publish(self):
import ast
tree = ast.parse(Path('api.py').read_text())
tree = ast.parse(Path(__file__).with_name('api.py').read_text())
fn = next(x for x in tree.body if isinstance(x, ast.FunctionDef) and x.name == 'dividend_research')
calls = [n.func.attr for n in ast.walk(fn) if isinstance(n, ast.Call) and isinstance(n.func, ast.Attribute)]
self.assertNotIn('collect', calls); self.assertNotIn('publish', calls)

View File

@ -74,8 +74,8 @@ def _pg_ok(sql, params=None):
s = " ".join(sql.split())
if "v_factor_stock_daily" in s:
assert params == (DS,)
return [{"pct_change": 9.95}, {"pct_change": 3.0}, {"pct_change": 0.0}, {"pct_change": -1.5},
{"pct_change": None}, {"pct_change": 10.02}, {"pct_change": -4.0}]
return [{"ts_code": '60000' + str(i) + '.SH', "pct_change": value}
for i, value in enumerate((9.95, 3.0, 0.0, -1.5, None, 10.02, -4.0))]
if "v_factor_logic" in s:
assert params[-1] == DS and "600000.SH" in params and "SZ000001" not in params
return [

190
test_market_scope.py Normal file
View File

@ -0,0 +1,190 @@
"""沪深输入与冻结边界:全部虚构数据,离线读写临时文件与 SQLite。"""
import contextlib
import json
import re
import socket
import sqlite3
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
import pandas as pd
import common
import factors
import freeze
import market_scope as scope
DAY = '2026-11-04'
BAD = ['430001', '830001.BJ', 'BJ920001', '920001', '600001.SZ', 'SH000001', '510001.SH', '399001.SZ', '600001x', None]
def frame(codes, values=None):
return pd.DataFrame({'trade_date':[DAY]*len(codes), 'stock_code':codes,
'factor_value':values or list(range(len(codes)))})
class MarketScopeTests(unittest.TestCase):
def setUp(self):
p=patch.object(socket.socket, 'connect', side_effect=AssertionError('禁止联网'));p.start();self.addCleanup(p.stop)
for name in ('read_pg', 'read_mysql', 'factor_conn'):
p=patch.object(common.db, name, side_effect=AssertionError('必须替换数据库依赖'));p.start();self.addCleanup(p.stop)
def test_normalizes_prefix_suffix_and_bare_code(self):
for variants, expected in [(['SH600001','600001.SH','600001',' sh600001 '],'600001.SH'),
(['SZ000001','000001.SZ','000001'],'000001.SZ'),
(['688001','SH688001'],'688001.SH'),
(['301001','301001.SZ'],'301001.SZ')]:
for value in variants:
self.assertEqual(scope.normalize(value), expected)
self.assertEqual(scope.normalize(value,prefix=True), expected[-2:]+expected[:6])
for value in BAD: self.assertIsNone(scope.normalize(value), value)
def test_sql_and_python_accept_identical_scope(self):
sql = scope.sql_filter('stock_code')
pattern=sql.split("REGEXP '",1)[1][:-1]
for value in ['SH600001','600001.SH','600001','000001','SZ301001','688001.SH']+BAD:
self.assertEqual(bool(re.fullmatch(pattern,str(value).strip().upper())), scope.normalize(value) is not None, value)
self.assertIn(' ~ ',scope.sql_filter('ts_code',postgres=True))
with self.assertRaises(ValueError): scope.sql_filter('stock_code;DROP TABLE x')
def test_market_mode_never_includes_out_of_scope_or_mutates_source(self):
original=frame(['600001','SZ000001']+BAD); before=original.copy(deep=True)
with patch.object(common.config,'SUBFACTOR_UNIVERSE','market'):
result=common.universe_filter(original)
self.assertEqual(list(result.stock_code),['SH600001','SZ000001'])
pd.testing.assert_frame_equal(before,original)
def test_pool_mode_normalizes_both_sides_before_membership(self):
with patch.object(common.config,'SUBFACTOR_UNIVERSE','pool'), patch.object(common.db,'read_pg',return_value=pd.DataFrame({'ts_code':['SH600001','000001','830001.BJ']})) as read:
result=common.universe_filter(frame(['600001.SH','SZ000001','300001','BJ830001']))
self.assertEqual(list(result.stock_code),['SH600001','SZ000001'])
self.assertIn('WHERE',read.call_args[0][0]);self.assertIn(' ~ ',read.call_args[0][0])
def test_column_probe_does_not_read_arbitrary_security(self):
with patch.object(common.db,'read_mysql',return_value=pd.DataFrame()) as read:
self.assertEqual(factors._price_code_col(),common.config.PRICE_CODE_COL)
self.assertTrue(read.call_args[0][1].endswith('LIMIT 0'))
def test_heat_latest_batch_read_and_output_are_scoped(self):
calls=[]
def read(which,sql,params):
calls.append(sql)
self.assertIn('REGEXP',sql);self.assertNotIn('JOIN',sql)
if 'MAX(batch_no)' in sql: return pd.DataFrame({'trade_date':[DAY],'batch_no':[2]})
return pd.DataFrame({'trade_date':[DAY]*3,'stock_code':['600001.SH','SZ000001','BJ830001'],'batch_no':[2]*3,'score':[.1,.2,999]})
with patch.object(common.db,'read_mysql',side_effect=read),patch.object(common.config,'SUBFACTOR_UNIVERSE','market'):
result=factors.build_heat(DAY,DAY)
self.assertEqual(list(result.stock_code),['SH600001','SZ000001']);self.assertEqual(len(calls),2)
def test_heat_empty_qualified_batch_stays_empty(self):
with patch.object(common.db,'read_mysql',return_value=pd.DataFrame(columns=['trade_date','batch_no'])) as read:
self.assertTrue(factors.build_heat(DAY,DAY).empty)
self.assertEqual(read.call_count,1)
def test_price_read_filters_and_normalizes_before_calculation(self):
rows=pd.DataFrame({'trade_date':[DAY]*3,'ts_code':['SH600001','000001','830001.BJ'],'close':[10,20,999]})
with patch.object(factors,'_price_code_col',return_value='symbol'),patch.object(common.db,'read_mysql',return_value=rows) as read:
result=factors._read_gp_price(DAY,DAY)
self.assertEqual(list(result.ts_code),['600001.SH','000001.SZ']);self.assertIn('REGEXP',read.call_args[0][1])
def test_upside_only_uses_scoped_consensus_and_prices(self):
cons=pd.DataFrame({'ts_code':['SH600001','BJ830001'],'asof_date':[DAY]*2,'target_mid_avg':[12,1000]})
prices=pd.DataFrame({'ts_code':['600001.SH'],'trade_date':[DAY],'close':[10]})
with patch.object(common.db,'read_pg',return_value=cons) as read, patch.object(factors,'_read_gp_price',return_value=prices),patch.object(common.config,'SUBFACTOR_UNIVERSE','market'):
result=factors.build_upside(DAY,DAY)
self.assertEqual(list(result.stock_code),['SH600001']);self.assertAlmostEqual(result.iloc[0].factor_value,.2)
self.assertIn(' ~ ',read.call_args[0][0])
def test_event_filter_precedes_single_document_cap(self):
ev=pd.DataFrame({'ts_code':['BJ830001','SH600001'],'disclosure_date':[DAY]*2,'event_type':['股份回购']*2,'direction':['']*2,'confidence':[1,.8],'doc_id':[1,1],'source_type':['announcement']*2})
with patch.object(common.db,'read_pg',return_value=ev) as read,patch.object(common.config,'SUBFACTOR_UNIVERSE','market'),patch.object(common.config,'EVENT_SOURCE_TYPES',{'announcement'}),patch.object(common.config,'EVENT_MAX_PER_DOC',1),patch.object(common,'trading_days',return_value=[pd.Timestamp(DAY)]):
result=factors.build_event(DAY,DAY)
self.assertEqual(list(result.stock_code),['600001.SH']);self.assertIn(' ~ ',read.call_args[0][0])
def test_transmission_filters_before_max_and_warning(self):
tr=pd.DataFrame({'scan_date':[DAY]*3,'ts_code':['SH600001','600001.SH','BJ830001'],'n_sources':[1,2,999],'moved_ratio':[0,0,0]})
with patch.object(common.db,'read_pg',return_value=tr) as read,patch.object(common,'load_universe',return_value={'600001.SH','830001.BJ'}),patch.object(factors,'_warn_upstream_truncation') as warn:
result=factors.build_transmission(DAY,DAY)
self.assertEqual(list(result.stock_code),['600001.SH']);self.assertEqual(result.iloc[0].factor_value,2)
self.assertEqual(set(warn.call_args[0][0].ts_code),{'600001.SH'});self.assertIn(' ~ ',read.call_args[0][0])
def test_external_values_do_not_change_score_distribution(self):
def score(add_bad):
codes=['SH600001','SZ000001','SH600002']+(['BJ830001'] if add_bad else [])
upside=frame(codes,[.1,.3,.8]+([1e12] if add_bad else []))
heat=frame(codes,[.2,.4,.9]+([-1e12] if add_bad else []))
transmission=frame(codes,[1,2,4]+([1e12] if add_bad else []))
with patch.object(factors,'_days_of',return_value=[DAY]),patch.object(factors,'_track_set',return_value=None),patch.object(factors,'_risk_set',return_value=set()),patch.object(factors,'build_upside',return_value=upside),patch.object(factors,'_heat_day_for',return_value=DAY),patch.object(factors,'build_heat',return_value=heat),patch.object(factors,'build_transmission',return_value=transmission):
return factors.build_score(DAY,DAY)
pd.testing.assert_frame_equal(score(False),score(True))
def test_risk_members_exclude_bj_before_name_rule(self):
pools=pd.DataFrame({'members':[json.dumps([{'ts_code':'BJ830001','name':'ST范围外'},{'ts_code':'600001','name':'ST有效'}])]})
with patch.object(common.db,'read_pg',return_value=pools),patch.object(common.config,'EXCLUDE_RISK_NAMES',True):
self.assertEqual(factors._risk_set(),{'SH600001'})
def test_calendar_date_query_excludes_out_of_scope(self):
with patch.object(factors,'_price_code_col',return_value='symbol'),patch.object(common.db,'read_mysql',return_value=pd.DataFrame({'trade_date':[DAY]})) as read:
self.assertEqual(len(common.trading_days(DAY,DAY)),1)
self.assertIn('REGEXP',read.call_args[0][1])
def test_write_factor_preserves_historical_bj_and_other_dates(self):
connection=sqlite3.connect(':memory:')
self.addCleanup(connection.close)
connection.create_function('REGEXP',2,lambda regex,value:int(bool(re.fullmatch(regex,str(value)))))
connection.execute('CREATE TABLE t_factor_akg_score(trade_date TEXT,stock_code TEXT,factor_value REAL)')
baseline=[(DAY,'BJ830001',77.),(DAY,'830002.BJ',78.),(DAY,'SH600001',2.),('2026-11-03','SH600001',3.)]
connection.executemany('INSERT INTO t_factor_akg_score VALUES(?,?,?)',baseline);connection.commit()
class Cursor:
def __enter__(self):return self
def __exit__(self,*a):pass
def execute(self,sql,params):return connection.execute(sql.replace('%s','?'),params)
def executemany(self,sql,rows):return connection.executemany(sql.replace('%s','?'),rows)
class Conn:
def cursor(self):return Cursor()
def commit(self):connection.commit()
@contextlib.contextmanager
def conn():yield Conn()
with patch.object(common.db,'factor_conn',conn),patch.object(common,'ensure_table'):
common.write_factor('t_factor_akg_score',frame(['600001','BJ830001','830002.BJ'],[10,1000,1001]))
rows=connection.execute('SELECT * FROM t_factor_akg_score ORDER BY trade_date,stock_code').fetchall()
self.assertEqual(rows,sorted([baseline[0],baseline[1],baseline[3],(DAY,'SH600001',10.)]))
def test_out_of_scope_only_write_never_deletes_or_creates_table(self):
with patch.object(common,'ensure_table') as ensure:
common.write_factor('t_factor_akg_score',frame(BAD))
ensure.assert_not_called()
def test_freeze_nested_members_and_extra_frames_have_no_out_of_scope(self):
members=json.dumps([{'ts_code':'SH600001','name':'有效'},{'ts_code':'830001.BJ','name':'范围外'}, {'ts_code':'600001.SZ'}])
source=pd.DataFrame({'ts_code':['600001.SH','830001.BJ'],'members':[members,members],'details':[{'quiet':['600001','BJ830001']},{}]})
with tempfile.TemporaryDirectory() as root,patch.object(freeze.config,'FROZEN_ROOT',root),patch.object(freeze,'_FMT','csv.gz'),patch.object(freeze,'_EXT','.csv.gz'),patch.object(freeze,'_git_rev',return_value='test'),patch.dict(freeze._SOURCES,{'universe':lambda day:source},clear=True):
manifest=freeze.snapshot(DAY,extra_frames={'factor_akg_score':frame(['600001','BJ830001'])})
result=pd.read_csv(Path(root)/DAY/'universe.csv.gz')
factor=pd.read_csv(Path(root)/DAY/'factor_akg_score.csv.gz')
self.assertEqual(manifest['rows'],{'universe':1,'factor_akg_score':1})
self.assertNotIn('830001',result.to_json());self.assertNotIn('830001',factor.to_json())
self.assertEqual(json.loads(result.iloc[0].members),[{'ts_code':'600001.SH','name':'有效'}])
self.assertEqual(manifest['errors'],{})
def test_industry_edges_remain_and_invalid_member_json_is_error(self):
edge=pd.DataFrame({'src_segment':['产业甲'],'dst_segment':['产业乙'],'mechanism':['[事实] 保持']})
pd.testing.assert_frame_equal(scope.frozen_frame(edge),edge)
with self.assertRaises(ValueError):scope.frozen_frame(pd.DataFrame({'members':['[bad json']}))
def test_all_security_capture_queries_are_scoped(self):
def pg(sql,params=None):
self.assertIn(' ~ ',sql)
return pd.DataFrame({'ts_code':['600001.SH','830001.BJ']})
def mysql(which,sql,params=None):
self.assertIn('REGEXP',sql)
if 'MAX(batch_no)' in sql:return pd.DataFrame({'trade_date':[DAY],'batch_no':[1]})
if which=='heat':return pd.DataFrame({'stock_code':['SH600001','BJ830001'],'trade_date':[DAY]*2,'score':[1,2],'batch_no':[1,1]})
return pd.DataFrame({'code':['600001','830001'],'close':[1,2]})
with patch.object(common.db,'read_pg',side_effect=pg),patch.object(common.db,'read_mysql',side_effect=mysql),patch.object(factors,'_price_code_col',return_value='symbol'):
for fn in (freeze._cap_universe,freeze._cap_transmission,freeze._cap_consensus,freeze._cap_price,freeze._cap_heat):
with self.subTest(fn=fn.__name__):self.assertEqual(len(fn(DAY)),1)
if __name__=='__main__':unittest.main()

330
test_plan_recovery.py Normal file
View File

@ -0,0 +1,330 @@
"""有限恢复的隔离行为回归;所有来源/构建/模型替身,计划只写临时目录。"""
import contextlib
import datetime as dt
import fcntl
import json
from pathlib import Path
import socket
import sys
import tempfile
import types
import unittest
from unittest.mock import Mock, patch
import config
import plan
import plan_recovery as recovery
import plan_versions as pv
import run
GENERATE = plan._generate
NOW = dt.datetime.fromisoformat('2026-11-05T10:05:00+08:00')
DAY = '2026-11-04'
class RecoveryTest(unittest.TestCase):
def setUp(self):
self.stack = contextlib.ExitStack()
self.addCleanup(self.stack.close)
self.root = Path(self.stack.enter_context(tempfile.TemporaryDirectory()))
self.stack.enter_context(patch.object(config, 'PLAN_SNAPSHOT_DIR', str(self.root)))
self.stack.enter_context(patch.object(config, 'PLAN_RECOVERY_MAX_ATTEMPTS', 3))
self.stack.enter_context(patch.object(config, 'PLAN_RECOVERY_COOLDOWN_SECONDS', 900))
self.stack.enter_context(patch.object(socket.socket, 'connect', side_effect=AssertionError('禁止联网')))
self.ready = self.stack.enter_context(patch.object(recovery.build_readiness, 'check', return_value=DAY))
self.build = self.stack.enter_context(patch.object(run, '_cmd_build', return_value=DAY))
self.generate = self.stack.enter_context(patch.object(plan, '_generate', side_effect=self.publish))
self.pool = Mock(side_effect=AssertionError('恢复不入池'))
self.model = Mock(side_effect=AssertionError('恢复不额外发模型任务'))
self.stack.enter_context(patch.dict(sys.modules, {
'pool': types.SimpleNamespace(push=self.pool),
'user_research': types.SimpleNamespace(sync=self.model),
}))
def publish(self, day=DAY, *args, cutoff=NOW, **kwargs):
if kwargs.get('before_publish'):
kwargs['before_publish'](day)
return pv.publish(str(self.root), {'date': day, 'main': [], 'observe': [],
'plan_version': 'isolated'}, {'records': {}, 'market': {}}, {},
cutoff_at=cutoff.isoformat(), reason='晨版生成')
def state(self, day='2026-11-05'):
return json.loads((self.root / 'recovery' / (day + '.json')).read_text())
def test_missing_inputs_wait_without_build_or_budget(self):
self.ready.side_effect = RuntimeError('资金热度日期 2026-11-03,预期 2026-11-04')
result = recovery.ensure_today(now=NOW)
self.assertEqual(result['status'], 'waiting_data')
self.assertIn('2026-11-03', result['reason'])
self.build.assert_not_called()
self.generate.assert_not_called()
self.assertFalse((self.root / 'recovery').exists())
def test_ready_builds_once_with_real_late_cutoff_and_no_pool_or_model(self):
old = self.publish('2026-11-03', cutoff=NOW - dt.timedelta(days=1))
old_path = self.root / 'plan_2026-11-03.json'
previous = old_path.read_bytes()
result = recovery.ensure_today(now=NOW)
self.assertEqual(result['status'], 'recovered')
self.build.assert_called_once_with('all', 'daily', None, None, None)
today = pv.load(self.root, DAY)
self.assertEqual(today['decision']['publication_kind'], 'recovery')
self.assertEqual(today['decision']['cutoff_at'], NOW.isoformat())
self.assertFalse(pv.morning_eligible(today))
self.assertEqual(old_path.read_bytes(), previous)
self.assertEqual(pv.load(self.root, '2026-11-03', morning=True), old)
self.assertEqual(self.state()['attempts'], 1)
self.assertEqual(self.state()['status'], 'recovered')
self.pool.assert_not_called()
self.model.assert_not_called()
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'already_ready')
self.build.assert_called_once()
def test_existing_recovery_does_not_rebuild_even_when_sources_unavailable(self):
self.publish()
self.ready.side_effect = AssertionError('已有有效版不重新取构建源')
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'already_ready')
self.build.assert_not_called()
self.ready.assert_not_called()
def test_explicit_date_never_automatically_rebuilds(self):
for day in ('2026-11-03', DAY):
self.assertEqual(recovery.ensure_today(day, now=NOW)['status'], 'explicit_date')
self.build.assert_not_called()
self.ready.assert_not_called()
def test_outside_window_and_weekend_do_not_rebuild(self):
for stamp in ('2026-11-05T07:19:59+08:00', '2026-11-05T15:10:01+08:00',
'2026-11-07T10:00:00+08:00'):
with self.subTest(stamp=stamp):
result = recovery.ensure_today(now=dt.datetime.fromisoformat(stamp))
self.assertEqual(result['status'], 'outside_window')
self.build.assert_not_called()
self.ready.assert_not_called()
def test_preexisting_legacy_or_wrong_decision_day_is_preserved(self):
path = self.root / ('plan_' + DAY + '.json')
for value in ({'date': DAY, 'main': []}, 'broken json'):
path.write_text(json.dumps(value) if isinstance(value, dict) else value)
before = path.read_bytes()
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'blocked_existing_snapshot')
self.assertEqual(path.read_bytes(), before)
path.unlink()
self.publish(cutoff=NOW - dt.timedelta(days=1))
before = path.read_bytes()
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'blocked_existing_snapshot')
self.assertEqual(path.read_bytes(), before)
self.build.assert_not_called()
def test_future_published_day_is_not_replaced_or_suppressed(self):
self.publish('2026-11-06')
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'blocked_existing_snapshot')
self.build.assert_not_called()
def test_lock_recheck_observes_other_success(self):
writer = pv.writer
@contextlib.contextmanager
def publish_before_lock(root):
with writer(root):
self.publish()
yield
with patch.object(pv, 'writer', side_effect=publish_before_lock):
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'already_ready')
self.build.assert_not_called()
def test_inputs_changed_while_waiting_lock_do_not_consume_attempt(self):
self.ready.side_effect = [DAY, RuntimeError('热度已失效')]
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'waiting_data')
self.build.assert_not_called()
self.assertFalse((self.root / 'recovery').exists())
def test_failed_build_persists_budget_and_obeys_cooldown_and_daily_cap(self):
self.build.side_effect = RuntimeError('构建失败')
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'failed')
self.assertEqual(self.state()['status'], 'failed')
self.assertEqual(recovery.ensure_today(now=NOW + dt.timedelta(minutes=14))['status'], 'cooldown')
self.assertEqual(self.build.call_count, 1)
self.assertEqual(recovery.ensure_today(now=NOW + dt.timedelta(minutes=15))['status'], 'failed')
self.assertEqual(recovery.ensure_today(now=NOW + dt.timedelta(minutes=30))['status'], 'failed')
self.assertEqual(recovery.ensure_today(now=NOW + dt.timedelta(minutes=45))['status'], 'attempts_exhausted')
self.assertEqual(self.build.call_count, 3)
self.assertEqual(self.state()['attempts'], 3)
self.generate.assert_not_called()
def test_persisted_running_attempt_survives_process_restart(self):
path = self.root / 'recovery' / '2026-11-05.json'
pv.atomic(path, {'schema_version': 1, 'decision_date': '2026-11-05', 'data_date': DAY,
'attempts': 2, 'status': 'building', 'last_attempt_at': NOW.isoformat()})
self.assertEqual(recovery.ensure_today(now=NOW + dt.timedelta(minutes=1))['status'], 'cooldown')
self.build.assert_not_called()
def test_day_switch_has_independent_bounded_budget(self):
self.build.side_effect = RuntimeError('构建失败')
for minutes in (0, 15, 30):
recovery.ensure_today(now=NOW + dt.timedelta(minutes=minutes))
self.ready.return_value = '2026-11-05'
tomorrow = NOW + dt.timedelta(days=1)
self.assertEqual(recovery.ensure_today(now=tomorrow)['status'], 'failed')
self.assertEqual(self.state('2026-11-06')['attempts'], 1)
self.assertEqual(self.state()['attempts'], 3)
def test_corrupt_attempt_state_does_not_reset_budget(self):
path = self.root / 'recovery' / '2026-11-05.json'
pv.atomic(path, {'attempts': 'unknown'})
before = path.read_bytes()
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'blocked_state')
self.assertEqual(path.read_bytes(), before)
self.build.assert_not_called()
def test_configuration_cannot_expand_safety_limits(self):
self.build.side_effect = RuntimeError('构建失败')
with patch.object(config, 'PLAN_RECOVERY_MAX_ATTEMPTS', 100), patch.object(config, 'PLAN_RECOVERY_COOLDOWN_SECONDS', 1):
for minutes in (0, 15, 30):
self.assertEqual(recovery.ensure_today(now=NOW + dt.timedelta(minutes=minutes))['status'], 'failed')
self.assertEqual(recovery.ensure_today(now=NOW + dt.timedelta(minutes=45))['status'], 'attempts_exhausted')
self.assertEqual(self.build.call_count, 3)
def test_disabled_recovery_does_not_construct(self):
with patch.object(config, 'PLAN_RECOVERY_MAX_ATTEMPTS', 0):
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'attempts_exhausted')
self.build.assert_not_called()
def test_source_changes_after_build_do_not_publish(self):
self.ready.side_effect = [DAY, DAY, RuntimeError('来源日期变化')]
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'failed')
self.build.assert_called_once()
self.generate.assert_not_called()
self.assertFalse((self.root / ('plan_' + DAY + '.json')).exists())
def test_build_finishes_after_window_does_not_publish(self):
before = dt.datetime.fromisoformat('2026-11-05T15:09:00+08:00')
after = dt.datetime.fromisoformat('2026-11-05T15:11:00+08:00')
with patch.object(recovery, '_now', side_effect=[before, before, after, after]):
self.assertEqual(recovery.ensure_today()['status'], 'failed')
self.generate.assert_not_called()
def test_plan_failure_preserves_previous_snapshot(self):
self.publish('2026-11-03', cutoff=NOW - dt.timedelta(days=1))
path = self.root / 'plan_2026-11-03.json'
before = path.read_bytes()
self.generate.side_effect = SystemExit('采集失败')
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'failed')
self.assertEqual(path.read_bytes(), before)
self.assertEqual(self.state()['attempts'], 1)
def test_published_plan_with_later_failure_is_not_rebuilt(self):
def partly_published(*args, **kwargs):
self.publish()
raise OSError('Markdown 写入失败')
self.generate.side_effect = partly_published
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'failed')
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'already_ready')
self.build.assert_called_once()
def test_collection_finishes_after_window_does_not_publish(self):
before = dt.datetime.fromisoformat('2026-11-05T15:09:00+08:00')
after = dt.datetime.fromisoformat('2026-11-05T15:11:00+08:00')
with patch.object(recovery, '_now', side_effect=[before, before, before, after, after]):
self.assertEqual(recovery.ensure_today()['status'], 'failed')
self.assertFalse((self.root / ('plan_' + DAY + '.json')).exists())
self.assertEqual(self.state()['attempts'], 1)
def test_readiness_changes_during_collection_does_not_publish(self):
self.ready.side_effect = [DAY, DAY, DAY, RuntimeError('发布前热度日期失效')]
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'failed')
self.assertFalse((self.root / ('plan_' + DAY + '.json')).exists())
def test_real_generate_checks_recovery_boundary_before_any_publication(self):
import user_candidates
guard = Mock(side_effect=RuntimeError('已过恢复时窗'))
data = {'date': DAY, 'main': [], 'observe': [], 'plan_version': 'isolated'}
with patch.object(user_candidates, 'active', return_value=[]), \
patch.object(plan, 'collect', return_value=data), \
patch.object(plan.sources, 'market_context', return_value={}), \
patch.object(plan, 'write_published_md') as markdown, pv.writer(self.root):
with self.assertRaisesRegex(RuntimeError, '已过恢复时窗'):
GENERATE(DAY, 20, 10, 5, before_publish=guard)
guard.assert_called_once_with(DAY)
markdown.assert_not_called()
self.assertFalse((self.root / ('plan_' + DAY + '.json')).exists())
self.assertFalse((self.root / 'versions' / DAY).exists())
def test_recovery_keeps_real_cross_process_lock_through_build_and_publish(self):
def assert_locked():
with open(self.root / '.publish.lock', 'a') as other:
with self.assertRaises(BlockingIOError):
fcntl.flock(other, fcntl.LOCK_EX | fcntl.LOCK_NB)
def build(*args):
assert_locked()
return DAY
def publish(*args, **kwargs):
assert_locked()
return self.publish(*args, **kwargs)
self.build.side_effect = build
self.generate.side_effect = publish
self.assertEqual(recovery.ensure_today(now=NOW)['status'], 'recovered')
def test_cli_syncs_existing_workflow_once_then_refreshes_recovered_plan(self):
refresh = types.SimpleNamespace(refresh=Mock(side_effect=[{'status': 'no_morning_today'}, {'status': 'published'}]))
research = types.SimpleNamespace(sync=Mock())
with patch.dict(sys.modules, {'plan_refresh': refresh, 'user_research': research}), \
patch.object(recovery, 'ensure_today', return_value={'status': 'recovered'}) as recover, \
patch.object(sys, 'argv', ['run.py', 'plan-refresh']):
run.main()
research.sync.assert_called_once_with()
recover.assert_called_once_with(None)
self.assertEqual(refresh.refresh.call_count, 2)
self.pool.assert_not_called()
def test_cli_does_not_treat_waiting_cooldown_or_failure_as_success(self):
for status in ('waiting_data', 'cooldown', 'attempts_exhausted', 'failed', 'blocked_state'):
refresh = types.SimpleNamespace(refresh=Mock(return_value={'status': 'no_morning_today'}))
research = types.SimpleNamespace(sync=Mock())
with self.subTest(status=status), patch.dict(sys.modules, {'plan_refresh': refresh, 'user_research': research}), \
patch.object(recovery, 'ensure_today', return_value={'status': status}), \
patch.object(sys, 'argv', ['run.py', 'plan-refresh']):
with self.assertRaisesRegex(SystemExit, status):
run.main()
research.sync.assert_called_once()
refresh.refresh.assert_called_once()
def test_cli_existing_plan_only_runs_normal_refresh(self):
refresh = types.SimpleNamespace(refresh=Mock(return_value={'status': 'unchanged'}))
research = types.SimpleNamespace(sync=Mock())
with patch.dict(sys.modules, {'plan_refresh': refresh, 'user_research': research}), \
patch.object(recovery, 'ensure_today') as recover, \
patch.object(sys, 'argv', ['run.py', 'plan-refresh']):
run.main()
recover.assert_not_called()
research.sync.assert_called_once()
def test_daily_build_and_recovery_share_publish_lock(self):
entered = []
@contextlib.contextmanager
def guard(root):
entered.append('lock')
yield
entered.append('unlock')
def build(*args):
self.assertEqual(entered, ['lock'])
return DAY
self.build.side_effect = build
with patch.object(pv, 'writer', side_effect=guard):
self.assertEqual(run.cmd_build('all', 'daily', None, None, None), DAY)
self.assertEqual(entered, ['lock', 'unlock'])
def test_regime_cli_propagates_unpublished_status(self):
for status in ('no_morning', 'no_morning_today', 'outside_window', 'published', 'unchanged'):
regime = types.SimpleNamespace(append_to_snapshot=Mock(return_value={'status': 'OK', 'publication_status': status}))
with self.subTest(status=status), patch.dict(sys.modules, {'regime': regime}), \
patch.object(sys, 'argv', ['run.py', 'regime-append', '--date', DAY]):
if status in ('published', 'unchanged'):
run.main()
else:
with self.assertRaisesRegex(SystemExit, status):
run.main()
if __name__ == '__main__':
unittest.main()

128
test_rating_priority.py Normal file
View File

@ -0,0 +1,128 @@
"""离线:30日边界、证据与同档排序,真实刷新路径及固定版本重放。"""
import copy
import datetime as dt
from contextlib import ExitStack
import tempfile
import unittest
from unittest.mock import patch
import pandas as pd
import test_plan_versions as fixture
import sources, config, plan, plan_inputs as pi, plan_versions as pv, plan_refresh
import rating_priority as rating
import card
AT='2026-10-06T01:00:00+00:00'
EDGE='2026-11-05T09:00:00+08:00'
def review(grade='中'):
return {'overall':grade,'qual':'ok','brain_status':'ok','review_ref':'b'*64,'age_days':1}
def history():
return {'ts_code':'600001.SH','review_ref':'b'*64,'grade':'中','event_id':'b'*64,
'report_ran_at':AT,'recorded_at':AT,'last_change_json':{'event_id':'b'*64,
'direction':'down','before':'好','after':'中','before_ref':'a'*64,'after_ref':'b'*64,'at':AT}}
class PriorityTest(unittest.TestCase):
def test_boundary_timezone_same_grade_and_latest(self):
before=dt.datetime.fromisoformat(EDGE)-dt.timedelta(microseconds=1)
a=rating.evaluate(review(),history(),before.isoformat())
self.assertEqual(a['priority'],-1)
self.assertEqual(rating.evaluate(review(),history(),EDGE)['priority'],0)
same=history();same.update(review_ref='c'*64,report_ran_at='2026-11-04T01:00:00+00:00')
r={**review(),'review_ref':'c'*64}
self.assertEqual(rating.evaluate(r,same,EDGE)['status'],'expired')
self.assertEqual(a,rating.evaluate(review(),history(),before.astimezone(dt.timezone.utc).isoformat()))
up=history();up['grade']='好';up['last_change_json'].update(direction='up',before='中',after='好',at='2026-11-04T01:00:00+00:00')
up['report_ran_at']='2026-11-04T01:00:00+00:00'
self.assertEqual(rating.evaluate(review('好'),up,EDGE)['priority'],1)
def test_baseline_missing_invalid_and_future(self):
self.assertEqual(rating.evaluate(review(),None,EDGE)['status'],'missing_history')
h=history();h['last_change_json']=None
self.assertEqual(rating.evaluate(review(),h,EDGE)['status'],'baseline')
for modified in ({'qual':'error'},{'brain_status':'timeout'},{'review_ref':'c'*64},{'overall':'证据不足'}):
self.assertEqual(rating.evaluate({**review(),**modified},history(),EDGE)['priority'],0)
for field in ('recorded_at','report_ran_at'):
self.assertEqual(rating.evaluate(review(),{**history(),field:'2027-01-01T00:00:00Z'},EDGE)['status'],'future_history')
h=history();h['last_change_json']['before_ref']='invalid'
self.assertEqual(rating.evaluate(review(),h,EDGE)['status'],'invalid_history')
def test_quality_first_candidates_retained_scores_unchanged_and_flag_off(self):
ranked=pd.Series({'good':200.,'up':201.,'neutral':209.,'down':210.})
cards={k:{'verdict':'候选','quality_band':3 if k=='good' else 2,'gates':{'pointed':True},
'company_review':{'rating_priority':{'status':'active','priority':{'up':1,'down':-1}.get(k,0)}}} for k in ranked.index}
with patch.object(config,'PLAN_RATING_PRIORITY_ENABLED',True):
out=plan._rerank_by_axis(ranked,cards,'quality',120,True)
self.assertEqual(list(out.index),['good','up','neutral','down'])
self.assertEqual(out.to_dict(),ranked.to_dict())
self.assertTrue(all(c['verdict']=='候选' for c in cards.values()))
order=sorted(cards,key=lambda k:card.sort_key(cards[k],True,True))
self.assertEqual(order,['good','up','neutral','down'])
with patch.object(config,'PLAN_RATING_PRIORITY_ENABLED',False):
self.assertEqual(list(plan._rerank_by_axis(ranked,cards,'quality',120,True).index),['good','neutral','up','down'])
def test_source_separate_tables_scope_failure_and_match(self):
calls=[]
def read(sql,args):
calls.append((sql,args));self.assertNotIn('JOIN',sql)
self.assertEqual(args[0],['600001.SH'])
if 'company_review_runs' in sql:
return pd.DataFrame([{'ts_code':'600001.SH','summary':review(),'brain_status':'ok','ran_at':AT}])
return pd.DataFrame([history()])
rows=sources.company_reviews(['SH600001','920001.BJ'],'2026-11-04',read_pg=read,cutoff_at=EDGE,include_rating=True)
self.assertEqual(len(calls),2)
self.assertEqual(rows['SH600001']['rating_priority']['status'],'expired')
with self.assertRaises(OSError):
sources.company_reviews(['SH600001'],'2026-11-04',read_pg=lambda sql,args: read(sql,args) if 'company_review_runs' in sql else (_ for _ in ()).throw(OSError('history unavailable')),cutoff_at=EDGE,include_rating=True)
def test_old_frozen_params_do_not_inherit_current_enabled_switch(self):
with patch.object(config, 'PLAN_RATING_PRIORITY_ENABLED', True):
with pi.Bundle(records={}, params={}).use():
self.assertFalse(plan.config.PLAN_RATING_PRIORITY_ENABLED)
with pi.Bundle(params={}).use():
self.assertTrue(plan.config.PLAN_RATING_PRIORITY_ENABLED)
def test_frozen_replay_does_not_read_or_expire(self):
source=lambda *a,**kw: {'SH600001':{**review(),'rating_priority':rating.evaluate(review(),history(),kw['cutoff_at'])}}
bundle=pi.Bundle(params={'PLAN_RATING_PRIORITY_ENABLED':True},cutoff_at='2026-11-05T08:59:00+08:00')
value=bundle.call('sources.company_reviews',source,(['SH600001'],'2026-11-04'),{})
replay=pi.Bundle(records=bundle.records,params=bundle.params,cutoff_at='2030-01-01T00:00:00Z')
again=replay.call('sources.company_reviews',lambda *a,**kw:self.fail('回源'),(['SH600001'],'2026-11-04'),{})
self.assertEqual(value,again)
self.assertEqual(again['SH600001']['rating_priority']['priority'],-1)
class RefreshTest(unittest.TestCase):
def test_expiration_publishes_once_and_keeps_morning(self):
import factors, judgement, logic_state_daily
codes=['SH600001','SH600002'];day='2026-11-04';cut='2026-11-05T08:59:00+08:00'
with tempfile.TemporaryDirectory() as root, ExitStack() as stack:
def stub(obj,name,**kw):return stack.enter_context(patch.object(obj,name,**kw))
stub(config,'PLAN_SNAPSHOT_DIR',new=root)
stub(config,'PLAN_RATING_PRIORITY_ENABLED',new=True)
stub(config,'PLAN_RANK_AXIS',new='quality')
stub(plan,'_factor',side_effect=lambda table,day:pd.Series([240.,230.] if 'score' in table else [.3,.2] if 'upside' in table else [2.,2.],index=codes))
stub(plan,'_latest_date',return_value=day);stub(plan,'_prev_date',return_value=None)
stub(plan,'_names',return_value=dict(zip(codes,['甲','乙'])))
stub(plan,'_evidence',return_value=({c:('材料',3,.1,10,1) for c in codes},{day}))
stub(plan,'_segments_of',return_value={});stub(factors,'_risk_set',return_value=set())
stub(judgement,'load_previous',return_value=[]);stub(judgement,'recent_rows',return_value={})
stub(logic_state_daily,'history',return_value={})
for name in ('moved_members','broker_reports','logic_claims','broker_actions','close_prices','valuation_scenarios','analyst_events','event_day_fields','news_flashes','segment_catalysts','catalysts_for_codes'):
stub(sources,name,return_value={})
stub(sources,'stock_daily',return_value={c:{'pct0':3} for c in codes})
stub(sources,'night_conclusions',return_value={c:{'accum_state':'明确吸筹','accum_age':0} for c in codes})
def read(*a,**kw):
return {c:{**review(),'rating_priority':rating.evaluate(review(),history() if i==0 else None,kw['cutoff_at'])} for i,c in enumerate(codes)}
stub(sources,'company_reviews',side_effect=read)
b=pi.Bundle(params=pi.effective_config(config),cutoff_at=cut)
with b.use(): data=plan.collect(day)
data.update(data.pop('_full'))
with pv.writer(root):morning=pv.publish(root,data,{'records':b.records,'market':{},'regime':None},b.params,cutoff_at=cut,reason='晨版')
self.assertEqual(morning['main'][0]['code'],codes[1])
stub(plan_refresh,'in_window',return_value=True)
self.assertEqual(plan_refresh.refresh(now=dt.datetime.fromisoformat(EDGE))['status'],'published')
latest=pv.load(root,day)
self.assertEqual(latest['main'][0]['code'],codes[0])
self.assertEqual(latest['main'][0]['rating_priority']['status'],'expired')
self.assertNotIn('rating_priority',latest['main'][0]['company_review'])
self.assertEqual(pv.load(root,day,morning=True),morning)
self.assertEqual(plan_refresh.refresh(now=dt.datetime.fromisoformat(EDGE)+dt.timedelta(minutes=5))['status'],'unchanged')
stub(config,'PLAN_RATING_PRIORITY_ENABLED',new=False)
self.assertEqual(plan_refresh.refresh(now=dt.datetime.fromisoformat(EDGE)+dt.timedelta(minutes=10))['status'],'published')
self.assertFalse(pv.load(root,day)['rating_priority_policy']['enabled'])
if __name__=='__main__':unittest.main()

View File

@ -74,8 +74,10 @@ def main():
urllib.request.urlopen = lambda url, timeout=0: _Resp(json.dumps(raw).encode("utf-8"))
with tempfile.TemporaryDirectory() as d:
config.PLAN_SNAPSHOT_DIR = d
t("快照不存在时不创建空快照", regime.append_to_snapshot("2026-09-01")["status"] == "OK"
missing = regime.append_to_snapshot("2026-09-01")
t("快照不存在时不创建空快照", missing["status"] == "OK"
and not os.path.exists(regime.snapshot_path("2026-09-01")))
t("有效区制不冒充成功发布", missing['publication_status'] == 'no_morning')
with open(regime.snapshot_path("2026-09-01"), "w", encoding="utf-8") as f:
json.dump({"date": "2026-09-01", "main": []}, f)
import datetime as dt
@ -89,8 +91,9 @@ def main():
path = Path(regime.snapshot_path("2026-09-01"))
legacy = path.read_bytes()
with patch('plan_refresh.refresh', side_effect=at_fixed_time):
regime.append_to_snapshot("2026-09-01")
result = regime.append_to_snapshot("2026-09-01")
t("旧格式不补造完整晨版,原字节保持", path.read_bytes() == legacy)
t("旧格式缺完整晨版如实返回未发布", result['publication_status'] == 'no_morning')
path.unlink() # 仅删除测试临时夹具,另起完整晨版场景。
morning = {'date': '2026-09-01', 'plan_version': 'test', 'main': [], 'observe': []}
with pv.writer(d):

69
test_sources_scope.py Normal file
View File

@ -0,0 +1,69 @@
"""取数和样本入口的沪深范围回归,仅内存假数据,不访问服务。"""
import unittest
from unittest.mock import patch
import pandas as pd
import sources
DAY = '2026-11-04'
class SourcesScopeTest(unittest.TestCase):
def test_moved_members_filters_sql_and_rechecks_returned_codes(self):
rows = [{'ts_code': code, 'target': '测试环节', 'n_sources': 2, 'chain_fit': 1,
'members_total': 3, 'moved': 1, 'mkt_trade_date': DAY}
for code in ('600000.SH', 'sz000001', '920001.BJ', '600000.SZ')]
with patch.object(sources.db, 'read_pg', return_value=pd.DataFrame(rows)) as reader:
result = sources.moved_members(DAY)
self.assertEqual(set(result), {'SH600000', 'SZ000001'})
self.assertIn("UPPER(TRIM(CAST(ts_code AS TEXT))) ~", reader.call_args.args[0])
self.assertEqual(reader.call_args.args[1], (DAY,))
def test_daily_stock_filters_before_exposing_evidence(self):
rows = [{'ts_code': code, 'pct_change': 3, 'net_z': 1, 'heat_chg': 0.2}
for code in ('688001.SH', '300001', 'BJ920001', '000001.SH')]
with patch.object(sources.db, 'read_pg', return_value=pd.DataFrame(rows)) as reader:
result = sources.stock_daily(DAY)
self.assertEqual(set(result), {'SH688001', 'SZ300001'})
self.assertIn("UPPER(TRIM(CAST(ts_code AS TEXT))) ~", reader.call_args.args[0])
def test_prices_filter_at_query_and_normalize_without_guessing_wrong_exchange(self):
rows = [{'ts_code': code, 'close': 10} for code in
('SH600000', '000001.SZ', '300001', '920001', '920001.BJ', '600000.SZ')]
with patch.object(sources.db, 'read_mysql', return_value=rows) as reader:
result = sources.close_prices(DAY, code_col='symbol')
self.assertEqual(set(result), {'SH600000', 'SZ000001', 'SZ300001'})
self.assertIn('CAST(`symbol` AS CHAR)', reader.call_args.args[1])
self.assertEqual(reader.call_args.args[2], (DAY, '2026-11-05'))
def test_price_column_must_be_a_known_identifier(self):
with patch.object(sources.db, 'read_mysql') as reader:
self.assertEqual(sources.close_prices(DAY, code_col='symbol` OR 1=1'), {})
reader.assert_not_called()
def test_breadth_excludes_other_markets_without_filtering_market_indices(self):
seen = []
def pg(sql, params):
self.assertIn('SELECT ts_code, pct_change', sql)
self.assertIn('CAST(ts_code AS TEXT)', sql)
return [{'ts_code': '600000.SH', 'pct_change': 3},
{'ts_code': '000001.SZ', 'pct_change': -1},
{'ts_code': '920001.BJ', 'pct_change': 20},
{'ts_code': '000001.SH', 'pct_change': 10}, {'pct_change': 10}]
def mysql(source, sql, params=None):
seen.append((sql, params))
if 'zs_day_data' in sql:
return [{'symbol': '000001.SH', 'd': DAY, 'amount': 100},
{'symbol': '399001.SZ', 'd': DAY, 'amount': 200}]
return []
result = sources.market_context(DAY, read_pg=pg, read_mysql=mysql)
self.assertEqual(result['breadth']['n'], 2)
self.assertEqual(result['breadth']['up'], 1)
self.assertEqual(result['breadth']['down'], 1)
self.assertEqual(result['turnover']['amount'], 300)
self.assertEqual(seen[0][1], ('000001.SH', '399001.SZ'))
self.assertNotIn('REGEXP', seen[0][0])
if __name__ == '__main__':
unittest.main()

199
test_user_candidates.py Normal file
View File

@ -0,0 +1,199 @@
"""用户候选登记、原有门槛与版本闭环的断网回归。"""
import copy
import datetime as dt
import json
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch, Mock
from contextlib import ExitStack
import pandas as pd
import test_api_support
import user_candidates as uc
import plan, config, card, sources, factors, judgement, logic_state_daily
import plan_inputs as pi
import plan_versions as pv
import plan_refresh
import user_research
NOW = dt.datetime.fromisoformat('2026-11-03T10:00:00+08:00')
DAY = '2026-11-02'
PAYLOAD = {'codes': ['600001.SH'], 'reason': '行业需求变化', 'expires_on': '2026-11-10'}
USER_NAMES = dict(zip(
['SZ300133', 'SZ300364', 'SZ300182', 'SH603533', 'SZ300291', 'SZ301052'],
['华策影视', '中文在线', '捷成股份', '掌阅科技', '百纳千成', '果麦文化']))
class CandidateTests(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory()
self.root = self.temp.name
self.addCleanup(self.temp.cleanup)
self.reports = patch.object(user_research, 'load_reports', return_value={})
self.reports.start()
self.addCleanup(self.reports.stop)
def test_exchange_validation_batch_atomic_and_dedup(self):
for code in ('830001.BJ', '920001', 'SH000001', 'SZ600001', '123', '510300.SH'):
with self.assertRaises(ValueError):
uc.save(self.root, dict(PAYLOAD, codes=['600001', code]), '甲', NOW)
self.assertEqual(uc.read(self.root)['rows'], [])
r = uc.save(self.root, dict(PAYLOAD, codes=['600001', 'SH600001', '600001.SH']), '甲', NOW)
self.assertEqual(r['added'], 1)
self.assertEqual(uc.save(self.root, PAYLOAD, '乙', NOW)['existing'], 1)
self.assertEqual(uc.read(self.root)['rows'][0]['created_by'], '甲')
def test_pasted_names_mixed_codes_and_separators_save_canonical_codes(self):
text = '华策影视 中文在线,捷成股份\n掌阅科技;百纳千成、果麦文化'
result = uc.save(self.root, dict(PAYLOAD, codes=[text, '300133.SZ']), '甲', NOW, names=USER_NAMES)
self.assertEqual(result['added'], 6)
self.assertEqual([r['code'] for r in uc.read(self.root)['rows']], list(USER_NAMES))
self.assertEqual([r['name'] for r in uc.listing(self.root, NOW)['rows']], list(USER_NAMES.values())[::-1])
self.assertEqual(uc.save(self.root, dict(PAYLOAD, codes=text), '甲', NOW, names=USER_NAMES)['existing'], 6)
def test_unmatched_or_ambiguous_name_does_not_partially_save(self):
names = dict(USER_NAMES, SH600001='同名', SZ000001='同名')
for value, message in [('华策影视 不存在的股票', '未找到沪深 A 股'),
('华策影视 同名', '对应多只股票'),
('华策影视 中文', '未找到沪深 A 股'),
('华策影视 920001', '仅支持沪深 A 股'),
(' '.join(['华策影视'] * 31), '1 至 30')]:
with self.assertRaisesRegex(ValueError, message):
uc.save(self.root, dict(PAYLOAD, codes=[value]), '甲', NOW, names=names)
self.assertFalse(Path(self.root, 'user_candidates.json').exists())
def test_catalog_reads_full_listed_shanghai_shenzhen_universe(self):
import db
rows = [{'gp_code_two': c[2:]+'.'+c[:2], 'gp_name': n} for c, n in USER_NAMES.items()]
rows += [{'gp_code_two': '920001.BJ', 'gp_name': '排除示例'}, {'gp_code_two': '510300.SH', 'gp_name': '基金示例'}]
with patch.object(db, 'read_mysql', return_value=pd.DataFrame(rows)) as read:
self.assertEqual(uc.catalog(), USER_NAMES)
self.assertEqual(read.call_args.args[0], 'heat')
self.assertIn("list_status='L'", read.call_args.args[1])
self.assertIn("gp_code_two LIKE '%.SH' OR gp_code_two LIKE '%.SZ'", read.call_args.args[1])
def test_search_and_add_use_full_catalog_without_industry_membership(self):
import api
with patch.object(config, 'XXL_TRIGGER_KEY', 'test-secret'), patch.object(config, 'PLAN_SNAPSHOT_DIR', self.root), \
patch.object(uc, 'catalog', return_value=USER_NAMES), patch.object(plan, '_names', side_effect=AssertionError('不能依赖行业池')):
self.assertEqual(api.search_user_candidates('中文在线')['items'], [{'code': 'SZ300364', 'name': '中文在线'}])
background = Mock()
expiry = (dt.datetime.now(uc.BJ).date() + dt.timedelta(days=1)).isoformat()
result = api.add_user_candidates(background, dict(PAYLOAD, codes=list(USER_NAMES.values()), expires_on=expiry), x_job_key='test-secret')
self.assertEqual(result['added'], 6)
background.add_task.assert_called_once_with(uc.refresh_saved)
def test_expiry_remove_and_readd_do_not_erase_history(self):
uc.save(self.root, PAYLOAD, '甲', NOW)
old = uc.active(self.root, NOW)[0]
pub = {'user_candidates': [old]}
uc.remove(self.root, old['id'], '甲', NOW)
self.assertEqual(uc.live_pending(self.root, pub, NOW), ['SH600001'])
self.assertEqual(uc.remove(self.root, old['id'], '甲', NOW)['revision'], 2)
uc.save(self.root, PAYLOAD, '甲', NOW)
self.assertNotEqual(uc.active(self.root, NOW)[0]['id'], old['id'])
self.assertEqual(len(uc.read(self.root)['rows']), 2)
later = NOW + dt.timedelta(days=8)
self.assertEqual(uc.active(self.root, later), [])
def test_manual_discovery_never_fakes_transmission_or_other_gates(self):
evidence = {'pointed': False, 'user_selected': True, 'pct0': 4,
'covered': True, 'upside': .2, 'accum_state': '明确吸筹', 'accum_age': 0}
result = card.judge(evidence)
self.assertEqual(result['verdict'], card.VERDICT_CANDIDATE)
self.assertFalse(result['gates']['pointed'])
for change in ({'covered': False}, {'upside': -.2}, {'risk_name': True},
{'pct0': 0}, {'y_signal': 'SELL'}, {'accum_state': '潜在吸筹'}):
self.assertEqual(card.judge(dict(evidence, **change))['verdict'], card.VERDICT_SHOW)
self.assertEqual(card.judge(dict(evidence, user_selected=False))['verdict'], card.VERDICT_SHOW)
def test_mutations_require_key_and_do_not_publish_inline(self):
import api
with patch.object(config, 'XXL_TRIGGER_KEY', 'test-secret'), patch.object(config, 'PLAN_SNAPSHOT_DIR', self.root):
with self.assertRaises(api.HTTPException):
api.add_user_candidates(None, PAYLOAD, x_job_key='bad')
self.assertFalse(Path(self.root, 'user_candidates.json').exists())
def test_old_consumers_exclude_user_source_before_top_and_cache_is_separate(self):
import api
source={'id':'a'*32,'code':'SH600001'}
row={'code':'SH600001','score':250,'rank':1,'research_rank':1,'evidence':{},
'user_candidate':source,'user_required':True,'verdict':'候选'}
ordinary={'code':'SH600002','score':240,'rank':2,'research_rank':2,'evidence':{},'verdict':'候选'}
published={'date':DAY,'decision':{'id':'v-opt-in'},'main':[row,ordinary],'observe':[],
'user_candidates':[source],'candidates':[row,ordinary],'watch':[],
'counts':{'main':2,'observe':0}}
original=copy.deepcopy(published)
with patch.object(config,'PLAN_SNAPSHOT_DIR',self.root),patch.object(pv,'load',return_value=published):
default,_=api._plan_cached(DAY,1,0,0,True)
opted,_=api._plan_cached(DAY,1,0,0,True,include_user_candidates=True)
again,hit=api._plan_cached(DAY,1,0,0,False)
self.assertEqual(default['main'][0]['code'],'SH600002')
self.assertEqual(opted['main'][0]['code'],'SH600001')
self.assertEqual(default,again)
self.assertTrue(hit)
self.assertNotIn('user_candidates',default)
self.assertEqual(default['counts']['main'],1)
self.assertEqual(len(default['candidates']),1)
self.assertEqual(original,published)
def test_revision_add_remove_replay_and_frozen_morning(self):
codes = ['SH600001', 'SH600002']
with ExitStack() as stack:
def stub(obj, name, **kw): return stack.enter_context(patch.object(obj, name, **kw))
stub(config, 'PLAN_SNAPSHOT_DIR', new=self.root)
stub(config, 'PLAN_RANK_AXIS', new='quality')
stub(plan, '_factor', side_effect=lambda table, day: pd.Series(
[240.,230.] if 'score' in table else [.3,.2] if 'upside' in table else [2.,2.], index=codes))
stub(plan, '_latest_date', return_value=DAY)
stub(plan, '_prev_date', return_value=None)
stub(plan, '_names', return_value={'SH600001':'甲','SH600002':'乙'})
stub(uc, 'catalog', return_value={'SH600001':'甲','SH600002':'乙','SH600003':'丙'})
stub(plan, '_evidence', return_value=({}, {DAY}))
stub(plan, '_segments_of', return_value={})
stub(factors, '_risk_set', return_value=set())
stub(judgement, 'load_previous', return_value=[])
stub(judgement, 'recent_rows', return_value={})
stub(logic_state_daily, 'history', return_value={})
for name in ('moved_members','broker_reports','logic_claims','broker_actions','close_prices',
'valuation_scenarios','analyst_events','event_day_fields','news_flashes',
'segment_catalysts','catalysts_for_codes'):
stub(sources, name, return_value={})
stub(sources, 'stock_daily', return_value={c:{'pct0':4} for c in codes})
stub(sources, 'night_conclusions', return_value={c:{'accum_state':'明确吸筹','accum_age':0} for c in codes})
stub(sources, 'company_reviews', side_effect=lambda cs,*a,**kw: {c:{'overall':'好','age_days':0,'review_ref':'r'} for c in cs})
bundle = pi.Bundle(params=pi.effective_config(config), cutoff_at='2026-11-03T07:10:00+08:00')
with bundle.use():
data = plan.collect(DAY)
data.update(data.pop('_full'))
with pv.writer(self.root):
morning = pv.publish(self.root,data,{'records':bundle.records,'market':{},'regime':None},
bundle.params,cutoff_at=bundle.cutoff_at,reason='晨版')
frozen = copy.deepcopy(morning)
uc.save(self.root,dict(PAYLOAD,codes=['600001','600003']), '甲', NOW)
self.assertEqual(pv.stock_check(self.root,'SH600001')['refresh_status']['pending_codes'],['SH600001','SH600003'])
result = plan_refresh.refresh(now=NOW)
self.assertEqual(result['status'],'published')
published = pv.load(self.root)
rows = {r['code']:r for r in published['main']}
self.assertEqual(rows['SH600001']['verdict'],'候选')
self.assertFalse(rows['SH600001']['pointed'])
self.assertEqual(rows['SH600001']['score'],240.)
self.assertIsNone(rows['SH600003']['score'])
self.assertEqual(rows['SH600003']['name'], '丙')
self.assertNotIn('tier',rows['SH600003'])
self.assertFalse(pv.stock_check(self.root,'SH600003')['found'])
self.assertTrue(pv.stock_check(self.root,'SH600003',include_user_candidates=True)['found'])
json.dumps(published,allow_nan=False)
self.assertIn('SH600003', Path(self.root, f'plan_{DAY}.md').read_text())
self.assertEqual(pv.load(self.root,DAY,morning=True),frozen)
self.assertEqual(plan_refresh.refresh(now=NOW)['status'],'unchanged')
for source in uc.active(self.root,NOW):
uc.remove(self.root,source['id'],'甲',NOW)
plan_refresh.refresh(now=NOW)
after = pv.load(self.root)
self.assertEqual(after['main'][0]['verdict'],'仅展示')
self.assertNotIn('SH600003',[r['code'] for r in after['main']])
self.assertEqual(pv.load(self.root,DAY,morning=True),frozen)
if __name__ == '__main__':
unittest.main()

244
test_user_research.py Normal file
View File

@ -0,0 +1,244 @@
"""人工关注评析:所有外部来源与投递均用桩,状态文件仅在临时目录。"""
import copy
import datetime as dt
import io
import json
from pathlib import Path
import tempfile
import unittest
from unittest.mock import Mock, patch
import urllib.error
import urllib.parse
import test_plan_versions # 缺席数据库驱动用桩;统一由断网护栏启动。
import config
import sources
import user_candidates as uc
import user_research as ur
import plan_versions as pv
import review_targets as rt
NOW = dt.datetime.fromisoformat('2026-11-04T10:00:00+08:00')
CODE = 'SZ300133'
def report(**changes):
return dict({'review_ref': 'a' * 64, 'brain_used': True, 'brain_status': 'ok',
'thesis': '订单改善仍需后续验证', 'overall': '中',
'report_ran_at': '2026-11-03T18:00:00+00:00',
'checked_at': '2026-11-03T18:00:00+00:00'}, **changes)
class ResearchTests(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory()
self.addCleanup(self.temp.cleanup)
self.root = self.temp.name
self.reader = Mock(return_value={})
self.submit = Mock(return_value={'status': 'queued', 'task_id': 'private-task'})
self.patches = [patch.object(config, 'PLAN_SNAPSHOT_DIR', self.root),
patch.object(config, 'USER_RESEARCH_FRESH_DAYS', 7),
patch.object(config, 'USER_RESEARCH_SUBMIT_LIMIT', 3)]
for p in self.patches:
p.start()
self.addCleanup(p.stop)
uc.save(self.root, {'codes': [CODE], 'reason': '自主关注', 'expires_on': '2026-11-10'}, 'test', NOW)
def sync(self, **kwargs):
return ur.sync(self.root, now=kwargs.pop('now', NOW), reader=self.reader,
submit=self.submit, target_write=lambda *a: None, **kwargs)
def test_complete_requires_real_reference_brain_thesis_and_time(self):
self.assertTrue(ur.report_state(report(), NOW)[0])
for change in ({'review_ref': None}, {'review_ref': 'old'}, {'brain_used': False},
{'brain_status': 'half'}, {'thesis': ''}, {'report_ran_at': None},
{'report_ran_at': '2026-11-05T10:00:00+08:00'}):
with self.subTest(change=change):
self.assertFalse(ur.report_state(report(**change), NOW)[0])
def test_recent_real_recheck_keeps_old_report_ready_without_redating(self):
old = report(report_ran_at='2026-09-29T00:00:00+00:00')
self.reader.return_value = {CODE: old}
self.sync()
self.submit.assert_not_called()
view = ur.views(self.root, [CODE], now=NOW, reader=self.reader)[CODE]
self.assertEqual(view['review_status'], 'completed')
self.assertEqual(view['review_data_date'], '2026-09-29')
self.assertNotIn('task_id', view)
def test_future_check_cannot_freshen_stale_report(self):
old = report(report_ran_at='2026-09-29T00:00:00+00:00', checked_at='2026-11-05T00:00:00+00:00')
self.assertFalse(ur.report_state(old, NOW)[0])
def test_claim_is_durable_before_submission_and_prevents_double_enqueue(self):
def submit(code):
self.assertEqual(ur.read(self.root)['rows'][code]['status'], 'pending')
self.sync() # 并发调用能看到已持久化的 claim,不能再次派单。
return {'status': 'queued', 'task_id': 'private-task'}
self.submit.side_effect = submit
self.sync()
self.sync()
self.assertEqual(self.submit.call_count, 1)
row = ur.read(self.root)['rows'][CODE]
self.assertEqual(row['status'], 'queued')
self.assertEqual(row['task_id'], 'private-task')
def test_timeout_is_uncertain_not_failure_or_automatic_retry(self):
self.submit.side_effect = TimeoutError('offline timeout')
self.sync()
self.sync(now=NOW + dt.timedelta(hours=2))
self.assertEqual(self.submit.call_count, 1)
row = ur.read(self.root)['rows'][CODE]
self.assertEqual(row['status'], 'pending')
self.assertIn('待核实', row['error'])
def test_completed_report_recovers_queued_or_uncertain_request(self):
self.sync()
self.reader.return_value = {CODE: report()}
self.sync(now=NOW + dt.timedelta(minutes=5))
self.assertEqual(ur.read(self.root)['rows'][CODE]['status'], 'completed')
self.assertEqual(self.submit.call_count, 1)
def test_submission_done_is_not_report_completion(self):
self.submit.return_value = {'status': 'done', 'result': {'status': 'ok'}}
self.sync()
self.assertEqual(ur.read(self.root)['rows'][CODE]['status'], 'pending')
self.reader.return_value = {CODE: report()}
self.sync(now=NOW + dt.timedelta(minutes=5))
self.assertEqual(ur.read(self.root)['rows'][CODE]['status'], 'completed')
def test_old_no_reference_recheck_is_failed_not_fabricated_completed(self):
self.reader.return_value = {CODE: report(review_ref=None)}
self.sync()
self.reader.return_value = {CODE: report(review_ref=None, checked_at='2026-11-04T10:01:00+08:00')}
self.sync(now=NOW + dt.timedelta(minutes=5))
row = ur.read(self.root)['rows'][CODE]
self.assertEqual(row['status'], 'failed')
self.assertIn('固定版本', row['error'])
self.assertEqual(self.submit.call_count, 1)
def test_explicit_rejection_does_not_retry_every_schedule(self):
self.submit.side_effect = urllib.error.HTTPError('offline', 403, '', {}, None)
self.sync()
self.sync()
self.assertEqual(ur.read(self.root)['rows'][CODE]['status'], 'failed')
self.assertEqual(self.submit.call_count, 1)
def test_source_failure_never_submits_and_get_reports_failure(self):
self.reader.side_effect = RuntimeError('offline unavailable')
self.assertEqual(self.sync()['status'], 'failed')
self.submit.assert_not_called()
self.assertEqual(ur.views(self.root, [CODE], now=NOW, reader=self.reader)[CODE]['review_status'], 'failed')
def test_readonly_listing_reads_live_reports_without_side_effects(self):
before = {p.name: p.read_bytes() for p in Path(self.root).iterdir() if p.is_file()}
with patch.object(ur, 'load_reports', return_value={CODE: report()}), \
patch.object(ur, 'submit_review', side_effect=AssertionError('GET不可投递')):
row = uc.listing(self.root, NOW)['rows'][0]
self.assertEqual(row['status'], 'pending') # 尚无计划仍可有真实完成报告。
self.assertEqual(row['review_status'], 'completed')
self.assertEqual(row['review_ref'], 'a' * 64)
self.assertEqual(before, {p.name: p.read_bytes() for p in Path(self.root).iterdir() if p.is_file()})
def test_removed_expired_and_out_of_scope_never_submit(self):
uc.remove(self.root, uc.active(self.root, NOW)[0]['id'], 'test', NOW)
self.sync()
self.submit.assert_not_called()
state = uc.read(self.root)
state['rows'].append({'id': 'bad', 'code': 'BJ920001', 'expires_on': '2026-11-10'})
pv.atomic(Path(self.root) / 'user_candidates.json', state)
self.sync()
self.submit.assert_not_called()
self.assertEqual(ur.views(self.root, ['920001.BJ'], now=NOW, reader=self.reader), {})
def test_bounded_submissions_continue_next_natural_cycle(self):
uc.save(self.root, {'codes': ['300364', '300182', '603533', '300291', '301052'],
'reason': '自主关注', 'expires_on': '2026-11-10'}, 'test', NOW)
self.assertEqual(self.sync()['submitted'], 3)
self.assertEqual(self.sync()['submitted'], 3)
self.assertEqual(self.sync()['submitted'], 0)
self.assertEqual(self.submit.call_count, 6)
def test_stale_complete_gets_real_recheck_then_checked_at_recovers(self):
self.reader.return_value = {CODE: report()}
self.sync()
later = NOW + dt.timedelta(days=8)
# 延长关注仅供隔离测试,不改变报告时间。
state = uc.read(self.root)
state['rows'][0]['expires_on'] = '2026-11-30'
pv.atomic(Path(self.root) / 'user_candidates.json', state)
self.assertEqual(self.sync(now=later)['submitted'], 1)
self.reader.return_value = {CODE: report(checked_at=later.isoformat())}
self.sync(now=later + dt.timedelta(minutes=5))
self.assertEqual(ur.read(self.root)['rows'][CODE]['status'], 'completed')
def test_targets_append_all_manual_without_score_verdict_or_prefix_cut(self):
snap = {'main': [{'code': f'SH60{i:04d}', 'score': 300-i, 'rank': i+1} for i in range(150)],
'observe': [], 'user_candidates': [dict(r, verdict='仅展示', score=None) for r in uc.active(self.root, NOW)]}
got = []
rt.persist('2026-11-03', snap, write=lambda day, rows: got.extend(rows))
row = next(r for r in got if r['ts_code'] == '300133.SZ')
self.assertEqual(row['source'], 'manual')
self.assertEqual(len(got), rt.TOP_MAIN + 1)
rows = rt.build_rows('2026-11-03', [], [], [{'code': 'BJ920001'}, {'code': CODE}, {'code': CODE}])
self.assertEqual([r['ts_code'] for r in rows], ['300133.SZ'])
def test_target_sync_preserves_plan_files_and_removes_deleted_manual_rows(self):
snap = {'date': '2026-11-03', 'main': [], 'observe': [], 'decision': {'id': 'read-only'}}
path = Path(self.root) / 'plan_2026-11-03.json'
pv.atomic(path, snap)
before = path.read_bytes()
got = []
ur.sync(self.root, now=NOW, reader=self.reader, submit=self.submit,
target_write=lambda day, rows: got.append(rows))
self.assertEqual(got[-1][0]['ts_code'], '300133.SZ')
self.assertEqual(path.read_bytes(), before)
uc.remove(self.root, uc.active(self.root, NOW)[0]['id'], 'test', NOW)
ur.sync(self.root, now=NOW, reader=self.reader, submit=self.submit,
target_write=lambda day, rows: got.append(rows))
self.assertEqual(got[-1], [])
self.assertEqual(path.read_bytes(), before)
def test_submit_uses_fixed_base_and_flags_without_redirect(self):
response = io.BytesIO(json.dumps({'status': 'queued', 'task_id': 'x'}).encode())
opener = Mock()
opener.open.return_value = response
with patch.object(config, 'AKG_API_BASE', 'http://offline.invalid:8000'), \
patch.object(ur.urllib.request, 'build_opener', return_value=opener):
self.assertEqual(ur.submit_review(CODE)['status'], 'queued')
req = opener.open.call_args.args[0]
url = urllib.parse.urlsplit(req.full_url)
self.assertEqual((url.netloc, url.path), ('offline.invalid:8000', '/admin/company_review'))
self.assertEqual(req.method, 'POST')
q = urllib.parse.parse_qs(url.query)
self.assertEqual(q, {'ts_code': ['300133.SZ'], 'force': ['false'], 'sync': ['false'],
'with_brain': ['true'], 'with_qual': ['true']})
self.assertLessEqual(opener.open.call_args.kwargs['timeout'], 10)
with self.assertRaises(urllib.error.HTTPError):
ur._NoRedirect().redirect_request(req, None, 302, '', {}, 'http://elsewhere.invalid')
def test_checked_column_only_for_live_independent_research(self):
import pandas as pd
row = {'ts_code': '300133.SZ', 'period': '2026Q3', 'review_version': 'v',
'grades': {}, 'summary': report(), 'brain_used': True, 'brain_status': 'ok',
'ran_at': dt.datetime.fromisoformat('2026-09-29T00:00:00+00:00'),
'checked_at': NOW}
reader = Mock(return_value=pd.DataFrame([row]))
got = sources.company_reviews([CODE], '2026-11-04', read_pg=reader,
cutoff_at=NOW.isoformat(), include_check=True)
self.assertIn('checked_at', reader.call_args.args[0])
self.assertEqual(got[CODE]['checked_at'], NOW.isoformat())
normal = sources.company_reviews([CODE], '2026-11-04', read_pg=reader, cutoff_at=NOW.isoformat())
self.assertNotIn('checked_at', normal[CODE])
self.assertNotIn(', checked_at', reader.call_args.args[0])
def test_background_research_failure_does_not_block_plan_refresh(self):
import plan_refresh
with patch.object(ur, 'sync', side_effect=RuntimeError('offline failure')), \
patch.object(plan_refresh, 'refresh', return_value={'status': 'unchanged'}) as refresh:
self.assertEqual(uc.refresh_saved()['status'], 'unchanged')
refresh.assert_called_once()
if __name__ == '__main__':
unittest.main()

View File

@ -128,7 +128,7 @@ def test_fetch_shared():
return [{"ts_code": "002812.SZ", "close": "50.17"}, {"ts_code": "600000", "close": 10.5},
{"ts_code": "430047", "close": 3.0}, {"ts_code": "300750.SZ", "close": None}]
px = sources.close_prices(DS, read_mysql=_price_reader, code_col="symbol")
t("收盘价按前缀码索引,两种代码形态都认", px == {"SZ002812": 50.17, "SH600000": 10.5, "BJ430047": 3.0})
t("收盘价按前缀码索引,沪深两种形态都认且排除范围外证券", px == {"SZ002812": 50.17, "SH600000": 10.5})
t("日期写成左闭右开区间", "`timestamp` >= %s AND `timestamp` < %s" in seen["psql"] and seen["pparams"] == (DS, "2026-09-03"))
t("收盘价读失败返回空字典", sources.close_prices(DS, read_mysql=lambda *a: (_ for _ in ()).throw(OSError("x")), code_col="symbol") == {})

220
user_candidates.py Normal file
View File

@ -0,0 +1,220 @@
"""用户补充的候选来源。与计划共用文件锁,先保存,再由修订任务发布。"""
from __future__ import annotations
import datetime as dt
import json
import re
import uuid
from pathlib import Path
from zoneinfo import ZoneInfo
import plan_versions as pv
BJ = ZoneInfo('Asia/Shanghai')
def consumer_view(published, include_user_candidates=False):
"""旧调用方只取系统来源;显式声明支持新来源后才返回人工候选。"""
if include_user_candidates or not published.get('user_candidates'):
return published
out = dict(published)
out.pop('user_candidates', None)
for bucket in ('main', 'observe', 'main_shown', 'observe_shown', 'candidates', 'watch'):
if bucket in out:
out[bucket] = [{k: v for k, v in r.items() if k not in ('user_candidate', 'user_required')}
for r in out[bucket] if not r.get('user_required')]
if 'counts' in out:
out['counts'] = dict(out['counts'], main=len(out.get('main', [])), observe=len(out.get('observe', [])))
if 'card_counts' in out:
rows = out.get('main', []) + out.get('observe', [])
out['card_counts'] = {v: sum(r.get('verdict') == v for r in rows) for v in out['card_counts']}
return out
def normalize(value):
value = str(value or '').strip().upper()
if re.fullmatch(r'\d{6}\.(SH|SZ)', value):
value = value[-2:] + value[:6]
elif re.fullmatch(r'\d{6}', value):
value = ('SH' if value.startswith(('60', '68')) else 'SZ') + value
if not re.fullmatch(r'(SH(?:60|68)\d{4}|SZ(?:00|30)\d{4})', value):
raise ValueError('仅支持沪深 A 股,请检查股票代码')
return value
def catalog():
"""使用上市证券基础表,不能把部分行业池当作完整可搜索名单。"""
import db
frame = db.read_mysql('heat',
"SELECT gp_code_two,gp_name FROM gp_code_all WHERE list_status='L' "
"AND (gp_code_two LIKE '%.SH' OR gp_code_two LIKE '%.SZ')")
out = {}
for row in frame.to_dict('records'):
try:
code = normalize(row.get('gp_code_two'))
except ValueError:
continue
name = str(row.get('gp_name') or '').strip()
if name:
out[code] = name
if not out:
raise ValueError('证券名称资料暂不可用,请稍后重试')
return out
def resolve(raw, names=None):
"""批量名称只按完整简称精确匹配;全部解析成功后才允许保存。"""
if isinstance(raw, str):
raw = [raw]
if not isinstance(raw, list) or any(not isinstance(v, str) for v in raw):
raise ValueError('请输入股票名称或代码')
tokens = [v for item in raw for v in re.split(r'[\s,,;;、]+', item.strip()) if v]
if not 1 <= len(tokens) <= 30:
raise ValueError('每次可添加 1 至 30 只股票')
index = None
codes = []
for value in tokens:
try:
code = normalize(value)
except ValueError:
# 明确的证券代码继续走原交易所限制,不尝试按名称解释。
if re.fullmatch(r'(?:(?:SH|SZ|BJ)?\d+(?:\.(?:SH|SZ|BJ))?)', value.upper()):
raise
if index is None:
source = names() if callable(names) else names or {}
index = {}
for candidate, name in source.items():
try:
canonical = normalize(candidate)
except ValueError:
continue
index.setdefault(str(name).strip().upper(), set()).add(canonical)
matches = index.get(value.upper(), set())
if not matches:
raise ValueError(f'未找到沪深 A 股“{value[:40]}”,请检查完整名称或使用代码') from None
if len(matches) != 1:
raise ValueError(f'“{value[:40]}”对应多只股票,请使用代码') from None
code = next(iter(matches))
if code not in codes:
codes.append(code)
return codes
def read(root):
path = Path(root) / 'user_candidates.json'
if not path.exists():
return {'revision': 0, 'rows': []}
state = json.loads(path.read_text())
if not isinstance(state.get('rows'), list) or not isinstance(state.get('revision'), int):
raise ValueError('添加记录暂不可用')
return state
def active(root, now=None):
today = (now or dt.datetime.now(BJ)).astimezone(BJ).date().isoformat()
return [dict(r) for r in read(root)['rows']
if not r.get('removed_at') and r['expires_on'] >= today]
def save(root, payload, actor, now=None, *, names=None):
now = now or dt.datetime.now(BJ)
codes = resolve(payload.get('codes'), names)
reason = str(payload.get('reason') or '').strip()
if not reason or len(reason) > 300:
raise ValueError('请填写添加理由,最多 300 字')
try:
expires = dt.date.fromisoformat(str(payload.get('expires_on') or ''))
except ValueError as exc:
raise ValueError('请选择关注截止日期') from exc
if not now.astimezone(BJ).date() <= expires <= now.astimezone(BJ).date() + dt.timedelta(days=90):
raise ValueError('关注期限应在今天至未来 90 天内')
with pv.writer(root):
state = read(root)
current = {r['code'] for r in active(root, now)}
added = []
for code in codes:
if code in current:
continue
row = {'id': uuid.uuid4().hex, 'code': code, 'reason': reason,
'expires_on': expires.isoformat(), 'created_at': now.isoformat(),
'created_by': str(actor)[:100]}
if isinstance(names, dict) and names.get(code):
row['name'] = names[code]
state['rows'].append(row)
added.append(row)
if added:
state['revision'] += 1
pv.atomic(Path(root) / 'user_candidates.json', state)
mark_pending(root, [r['code'] for r in added])
return {'ok': True, 'added': len(added), 'existing': len(codes) - len(added),
'revision': state['revision']}
def remove(root, identity, actor, now=None):
now = now or dt.datetime.now(BJ)
with pv.writer(root):
state = read(root)
row = next((r for r in state['rows'] if r['id'] == identity), None)
if row is None:
raise ValueError('添加记录不存在,请刷新后重试')
if not row.get('removed_at'):
row.update(removed_at=now.isoformat(), removed_by=str(actor)[:100])
state['revision'] += 1
pv.atomic(Path(root) / 'user_candidates.json', state)
mark_pending(root, [row['code']])
return {'ok': True, 'revision': state['revision']}
def mark_pending(root, codes):
day = pv.latest_date(root)
if day:
state = pv.health(root, day)
pv.health(root, day, pending_codes=sorted(set(state.get('pending_codes') or []) | set(codes)))
def live_pending(root, published, now=None):
"""写后未发布、删除或到期时,即使后台失败也不能继续消费旧人工来源。"""
current = {r['code']: r['id'] for r in active(root, now)}
frozen = {r['code']: r['id'] for r in published.get('user_candidates', [])}
return sorted(c for c in current.keys() | frozen.keys() if current.get(c) != frozen.get(c))
def listing(root, now=None):
now = now or dt.datetime.now(BJ)
state = read(root)
try:
published = pv.load(root) if pv.latest_date(root) else {}
except FileNotFoundError:
published = {}
material = {r['code']: r for b in ('main', 'observe') for r in published.get(b, [])}
expected = {r['id'] for r in published.get('user_candidates', [])}
today = now.astimezone(BJ).date().isoformat()
import user_research
research = user_research.views(root, [r['code'] for r in state['rows'][-300:]], now=now)
rows = []
for source in reversed(state['rows']):
row = material.get(source['code']) or {}
status = ('removed' if source.get('removed_at') else
'expired' if source['expires_on'] < today else
'pending' if source['id'] not in expected else 'evaluated')
rows.append({**source, 'status': status, 'name': row.get('name') or source.get('name'),
'verdict': row.get('verdict'), 'basis': row.get('basis'),
'decision_id': (published.get('decision') or {}).get('id'),
**research.get(source['code'], {})})
return {'ok': True, 'rows': rows[:300], 'revision': state['revision']}
def refresh_saved():
"""请求结束后尝试发布;失败保留登记和待核验状态,定时修订会重试。"""
import logging
import plan_refresh
import user_research
try:
user_research.sync()
except Exception:
logging.getLogger(__name__).exception('人工关注评析状态同步失败,保留登记等待自然调度')
try:
return plan_refresh.refresh()
except Exception:
logging.getLogger(__name__).exception('用户候选已保存,计划修订待重试')
return {'status': 'pending'}

273
user_research.py Normal file
View File

@ -0,0 +1,273 @@
"""人工关注的独立评析状态:先持久化受理凭据,再投基座;查询绝不派任务。
计划的 evaluated 只表示已收录,不能代替买方评议完成。网络结果不明时不重复
投递,后续只读核验真实报告,并由原评析目标名单/批任务继续补齐。
"""
from __future__ import annotations
import contextlib
import datetime as dt
import fcntl
import json
import logging
from pathlib import Path
import re
import urllib.error
import urllib.parse
import urllib.request
import uuid
from zoneinfo import ZoneInfo
import plan_versions as pv
BJ = ZoneInfo('Asia/Shanghai')
UTC = dt.timezone.utc
_FILE = 'user_research.json'
log = logging.getLogger(__name__)
def _time(value):
try:
value = value if isinstance(value, dt.datetime) else dt.datetime.fromisoformat(str(value).replace('Z', '+00:00'))
return value.replace(tzinfo=UTC) if value.tzinfo is None else value
except (TypeError, ValueError):
return None
def _now(now=None):
return _time(now) or dt.datetime.now(UTC)
@contextlib.contextmanager
def _writer(root):
Path(root).mkdir(parents=True, exist_ok=True)
with open(Path(root) / '.user-research.lock', 'a') as lock:
fcntl.flock(lock, fcntl.LOCK_EX)
yield
def read(root):
path = Path(root) / _FILE
if not path.exists():
return {'version': 1, 'rows': {}}
value = json.loads(path.read_text())
if value.get('version') != 1 or not isinstance(value.get('rows'), dict):
raise ValueError('人工关注评析状态暂不可用')
return value
def _save(root, state):
pv.atomic(Path(root) / _FILE, state)
def _active(root, now):
import user_candidates as uc
rows = []
for row in uc.active(root, now):
try:
rows.append(dict(row, code=uc.normalize(row['code'])))
except (ValueError, KeyError):
# 持久化来源也在入口守沪深范围,不能因旧文件或外部写入越界。
continue
return rows
def report_state(report, now=None, fresh_days=7):
"""只判报告完成事实;财报期是否落后仍由基座原有批任务判断。"""
now = _now(now)
report = report if isinstance(report, dict) else {}
generated = _time(report.get('report_ran_at'))
checked = _time(report.get('checked_at'))
if not report:
return False, '尚无买方评析报告', None
if generated is None or generated > now:
return False, '评析生成时间缺失或晚于当前时刻,待核实', None
# 旧报告命中指纹时只更新 checked_at;不能把复核当新报告,也不能永远判旧。
verified = max(t for t in (generated, checked) if t is not None and t <= now)
if not report.get('brain_used') or report.get('brain_status') != 'ok':
return False, '买方评议尚未完整完成', verified
if not re.fullmatch(r'[0-9a-f]{64}', str(report.get('review_ref') or '')):
return False, '报告缺少可核验的固定版本,等待重新归档', verified
if not str(report.get('thesis') or '').strip():
return False, '报告尚缺买方论点', verified
if now - verified > dt.timedelta(days=fresh_days):
return False, '评析需要重新核对资料', verified
return True, None, verified
def load_reports(codes, now=None):
import sources
now = _now(now)
if not codes:
return {}
return sources.company_reviews(codes, now.astimezone(BJ).date().isoformat(),
cutoff_at=now.isoformat(), strict=True, include_check=True)
def _public(report, record, now, fresh_days):
import card
report = report if isinstance(report, dict) else None
complete, reason, verified = report_state(report, now, fresh_days)
status = (record or {}).get('status', 'missing')
if status == 'completed' and not complete: # 上次完成不能掩盖这次缺报或陈旧。
status = 'missing'
if complete:
status = 'completed'
generated = _time((report or {}).get('report_ran_at'))
error = None if complete else (record or {}).get('error') or reason
return {'review_status': status, 'review_updated_at': (verified.isoformat() if complete and verified else (record or {}).get('updated_at')),
'review_error': error,
'review_data_date': generated.astimezone(BJ).date().isoformat() if generated and generated <= now else None,
'company_review': report, 'company_review_text': card.company_review_view(report),
'review_ref': (report or {}).get('review_ref')}
def views(root, codes, *, now=None, reader=None):
"""实时只读视图;既不落状态文件,也不触发请求或写研究目标表。"""
import config
import user_candidates as uc
now = _now(now)
safe = []
for code in codes:
try:
safe.append(uc.normalize(code))
except ValueError:
continue
safe = sorted(set(safe))
state = read(root)
try:
reports = (reader or load_reports)(safe, now)
except Exception:
log.exception('人工关注评析读取失败')
return {code: {'review_status': 'failed', 'review_updated_at': None,
'review_error': '评析资料暂不可读取,请稍后核实', 'review_data_date': None,
'company_review': None, 'company_review_text': '', 'review_ref': None}
for code in safe}
return {code: _public(reports.get(code), state['rows'].get(code), now,
config.USER_RESEARCH_FRESH_DAYS) for code in safe}
class _NoRedirect(urllib.request.HTTPRedirectHandler):
def redirect_request(self, req, fp, code, msg, headers, newurl):
raise urllib.error.HTTPError(req.full_url, code, '评析提交不接受跳转', headers, fp)
def submit_review(code):
"""唯一请求地址来自受管配置,代码参数无法改变域名、路径或强制重跑。"""
import config
import user_candidates as uc
code = uc.normalize(code)
base = urllib.parse.urlsplit(config.AKG_API_BASE)
if (base.scheme not in ('http', 'https') or not base.hostname or base.username
or base.password or base.query or base.fragment):
raise ValueError('评析服务地址配置无效')
endpoint = config.AKG_API_BASE.rstrip('/') + '/admin/company_review'
query = urllib.parse.urlencode({'ts_code': code[2:] + '.' + code[:2],
'force': 'false', 'sync': 'false',
'with_brain': 'true', 'with_qual': 'true'})
request = urllib.request.Request(endpoint + '?' + query, data=b'', method='POST')
timeout = max(1.0, min(float(config.USER_RESEARCH_REQUEST_TIMEOUT), 10.0))
opener = urllib.request.build_opener(_NoRedirect())
with opener.open(request, timeout=timeout) as response:
return json.loads(response.read(1024 * 1024).decode('utf-8'))
def _targets(root, active, state, write=None):
"""给原自然批任务补齐全部有效人工关注,不改变已发布计划内容。"""
import review_targets
if not active and not state.get('targets_key'):
return
day = pv.latest_date(root)
if not day:
return
published = pv.load(root, day)
key = pv.digest({'day': day, 'decision': published.get('decision'), 'active': active})
if state.get('targets_key') == key:
return
review_targets.persist(day, published, write=write, user_rows=active)
state['targets_key'] = key
state.pop('targets_error', None)
def sync(root=None, *, now=None, reader=None, submit=None, target_write=None):
"""后台/自然调度入口。锁只护本地状态;网络前 claim 已经原子落盘。"""
import config
root = root or config.PLAN_SNAPSHOT_DIR
now = _now(now)
active = _active(root, now)
codes = sorted({r['code'] for r in active})
with _writer(root):
state = read(root)
try:
_targets(root, active, state, target_write)
except Exception:
state['targets_error'] = '研究目标名单同步失败,后台会继续核对'
log.exception('人工关注研究目标名单同步失败')
_save(root, state)
try:
reports = (reader or load_reports)(codes, now)
except Exception:
log.exception('人工关注评析同步读取失败,本轮不投递')
return {'status': 'failed', 'error': '评析资料暂不可读取,本轮未重复提交', 'submitted': 0}
submitted = 0
for code in codes:
report = reports.get(code)
complete, reason, verified = report_state(report, now, config.USER_RESEARCH_FRESH_DAYS)
with _writer(root):
state = read(root)
previous = state['rows'].get(code, {})
# 提交前再核对删除/到期,避免旧后台任务给已移除关注派新任务。
if code not in {r['code'] for r in _active(root, now)}:
continue
if complete:
state['rows'][code] = dict(previous, status='completed', error=None,
updated_at=verified.isoformat(), observed_at=now.isoformat(),
report_ref=report['review_ref'])
_save(root, state)
continue
if previous.get('status') == 'completed' and (_time(previous.get('observed_at')) or dt.datetime.min.replace(tzinfo=UTC)) >= now:
continue # 较早开始的并发读取不能把刚核实的完成状态反向改成待投递。
claimed = _time(previous.get('claimed_at'))
if previous.get('status') in ('pending', 'queued', 'failed') and claimed:
# 已投过的旧无 ref/不完整报告,核对更新后仍不合格,明确留待补齐;不假造完成。
if verified and verified > claimed and previous.get('status') != 'failed':
state['rows'][code] = dict(previous, status='failed', error=reason,
updated_at=now.isoformat())
_save(root, state)
continue
if submitted >= max(1, config.USER_RESEARCH_SUBMIT_LIMIT):
continue
claim = uuid.uuid4().hex
state['rows'][code] = dict(previous, status='pending', claim_id=claim,
claimed_at=now.isoformat(), updated_at=now.isoformat(),
error='评析请求正在受理,结果待核实')
_save(root, state)
submitted += 1
status, error, task_id = 'pending', '评析请求的受理结果待核实,暂不重复提交', None
try:
result = (submit or submit_review)(code)
if (isinstance(result, dict) and result.get('status') == 'queued'
and isinstance(result.get('task_id'), str) and result['task_id'].strip()):
status, error, task_id = 'queued', None, str(result['task_id'])
elif isinstance(result, dict) and result.get('status') == 'done':
result_status = (result.get('result') or {}).get('status')
if result_status not in ('ok', 'skipped_fresh'):
status, error = 'failed', '评析尚未生成完整报告,等待资料补齐'
else:
error = '评析已返回,等待核验固定报告'
except urllib.error.HTTPError as exc:
if 400 <= exc.code < 500:
status, error = 'failed', '评析请求未被接受,请核对服务后重试'
log.warning('人工关注评析请求失败 code=%s http=%s', code, exc.code)
except ValueError:
log.exception('人工关注评析响应或配置不可核验 code=%s', code)
except Exception:
log.exception('人工关注评析受理结果不明 code=%s', code)
with _writer(root):
state = read(root)
previous = state['rows'].get(code, {})
if previous.get('claim_id') == claim and previous.get('status') != 'completed':
state['rows'][code] = dict(previous, status=status, error=error,
task_id=task_id, updated_at=now.isoformat())
_save(root, state)
return {'status': 'checked', 'active': len(codes), 'submitted': submitted}