高股息研究:分离每日初筛与月度资料复查(台账 080)
一、按用户节奏建立独立研究入口 股息每日检查,ST和退市名称先排除,身份不明者隔离。财务分红材料按月复用,新进入者补查。季度研究需求单列,不把需求清单冒充已完成评析。真实月表171只高股息,排除1只风险名称、隔离6只,164只中已有评析9只。 二、保留研究依据与失败边界 保存固定输入、各视角结论、反方、原报告读取副本和实际材料期。无专项分红证据时持续性保持待核验。原子归档、重复幂等和日检失败保留旧结果。真实同月复查2.438秒,164家财务和9份报告复用。 三、隔离现有交易流程 新增研究调度入口使用独立锁,不进入默认计划链,不占五分钟修订锁。不调用模型、不修改交易参数或PMS。36项新增与63项联合禁网回归通过,原XXL13项通过,研究页面搜索与筛选核验。尚未部署或启用日程,不宣称生产判收。
This commit is contained in:
parent
4bcd13fa6d
commit
c7db8d911e
|
|
@ -15,3 +15,6 @@ data/frozen/**/*.csv.gz
|
|||
|
||||
# 评审修复前的备份与补丁包(确认无误后可整目录删掉)
|
||||
.review_backup_*/
|
||||
|
||||
# 独立高股息研究归档和运行状态,不提交原始数据
|
||||
data/dividend_watch/
|
||||
|
|
|
|||
29
README.md
29
README.md
|
|
@ -206,3 +206,32 @@ docker compose exec akg-factor-bridge python run.py freeze --date 2026-07-24
|
|||
`build all` 一并日更;两锚三档 + 两段式组内分。
|
||||
- **G4 回填与仪表盘(未完成)**:upside 历史重建走「给基座 `sync_consensus` 加
|
||||
`asof` 参数」,动工前先与基座侧排期。
|
||||
|
||||
|
||||
## 高股息阶段检测(独立研究)
|
||||
|
||||
`dividend_watch.py scan`读取上一交易日对应的`gp_stock_factor_pro_YYYYMM`。
|
||||
先排除ST及退市名称,使用滚动股息不低于5%的研究入口。
|
||||
每天接收已有新评析,财务分红事实每月更新,新入池立即补查。
|
||||
季度复核需求单独列出;需求清单不等于已完成评析。
|
||||
|
||||
```bash
|
||||
# 155桥机 · ~/project/akg-factor-bridge;部署后执行
|
||||
docker exec akg_factor_bridge python dividend_watch.py scan
|
||||
docker exec akg_factor_bridge python dividend_watch.py health
|
||||
```
|
||||
|
||||
报告:`GET /research/dividend`。日检:`GET /research/dividend/health`。
|
||||
独立XXL触发:`/api/v1/xxl/dividend-watch`,沿用既有鉴权及完成回调。
|
||||
建议交易日北京时间08:05;使用独立锁,不占五分钟计划修订锁。
|
||||
当前本地实现与真实只读试跑完成,线上日程尚未创建或启用。
|
||||
|
||||
输出在`data/dividend_watch/`,包含不可变输入、完整研究及可读报告。
|
||||
没有分红公告和支付能力专项证据时,持续性保持待核验。
|
||||
不调用模型,不改每日计划,不写交易股票池或PMS。
|
||||
详见`docs/高股息阶段检测开发规格_2026-10-21.md`和同日交接。
|
||||
|
||||
```bash
|
||||
# 开发机 · akg-factor-bridge;预期36项通过,完全离线
|
||||
PYTHONDONTWRITEBYTECODE=1 python3 -m unittest test_dividend_watch test_dividend_jobs
|
||||
```
|
||||
|
|
|
|||
26
api.py
26
api.py
|
|
@ -21,7 +21,7 @@ import threading
|
|||
|
||||
import pandas as pd
|
||||
from fastapi import FastAPI, HTTPException, Request
|
||||
from fastapi.responses import PlainTextResponse, JSONResponse
|
||||
from fastapi.responses import PlainTextResponse, JSONResponse, HTMLResponse
|
||||
|
||||
import card
|
||||
import sources
|
||||
|
|
@ -34,11 +34,13 @@ import plan_reconcile
|
|||
import regime
|
||||
import review_latest
|
||||
from xxl import router as xxl_router
|
||||
from dividend_jobs import router as dividend_router
|
||||
|
||||
_access = logging.getLogger("plan.access")
|
||||
|
||||
app = FastAPI(title="akg-factor-bridge · 每日选股计划", version="0.1")
|
||||
app.include_router(xxl_router)
|
||||
app.include_router(dividend_router)
|
||||
|
||||
|
||||
@app.get("/health")
|
||||
|
|
@ -56,6 +58,28 @@ def health():
|
|||
return {"ok": True, "latest_plan_date": d, **depth}
|
||||
|
||||
|
||||
@app.get("/research/dividend", response_class=HTMLResponse)
|
||||
def dividend_research():
|
||||
"""独立研究页只读归档,不在请求内采集或调用模型。"""
|
||||
import dividend_watch as watch
|
||||
from build_readiness import expected_data_day
|
||||
bundle = watch.read_latest(watch.DEFAULT_ROOT)
|
||||
if not bundle:
|
||||
raise HTTPException(status_code=404, detail="高股息研究尚未发布")
|
||||
status = watch.health(watch.DEFAULT_ROOT, expected_data_day())
|
||||
page = watch.report(bundle['result'])
|
||||
if not status['ok']:
|
||||
page = page.replace('<main>', '<p role="alert">本次检查尚未成功。以下保留上次研究,请留意行情日期。</p><main>')
|
||||
return HTMLResponse(page)
|
||||
|
||||
|
||||
@app.get("/research/dividend/health")
|
||||
def dividend_research_health():
|
||||
import dividend_watch as watch
|
||||
from build_readiness import expected_data_day
|
||||
return watch.health(watch.DEFAULT_ROOT, expected_data_day())
|
||||
|
||||
|
||||
@app.get("/plan/dates")
|
||||
def plan_dates(limit: int = 30):
|
||||
from pathlib import Path
|
||||
|
|
|
|||
|
|
@ -0,0 +1,59 @@
|
|||
"""高股息独立调度入口;不占每日计划及五分钟修订的运行锁。"""
|
||||
import datetime as dt
|
||||
import threading
|
||||
import uuid
|
||||
|
||||
from fastapi import APIRouter, Header, HTTPException, Query
|
||||
from xxl import _require_key, _run_step, _notify
|
||||
|
||||
router = APIRouter(prefix='/api/v1/xxl', tags=['dividend-research'])
|
||||
_lock = threading.Lock()
|
||||
_jobs = {}
|
||||
_running = None
|
||||
|
||||
|
||||
def _run(job_id, callback):
|
||||
global _running
|
||||
job = _jobs[job_id]
|
||||
ok = False
|
||||
note = ''
|
||||
try:
|
||||
rc, output = _run_step(['dividend_watch.py', 'scan'])
|
||||
ok = rc == 0
|
||||
note = output[-2000:]
|
||||
except Exception as exc:
|
||||
note = type(exc).__name__
|
||||
finally:
|
||||
with _lock:
|
||||
job.update(state='success' if ok else 'failed', note=note,
|
||||
ended_at=dt.datetime.now(dt.timezone.utc).isoformat())
|
||||
_running = None
|
||||
_notify(job, callback, ok, note)
|
||||
job['closed'] = True
|
||||
|
||||
|
||||
@router.api_route('/dividend-watch', methods=['GET', 'POST'])
|
||||
def trigger(callbackUrl: str = Query(None), key: str = Query(None), x_job_key: str = Header(None)):
|
||||
"""交易日08:05,只读采集并写独立研究文件。使用既有XXL鉴权与回调。"""
|
||||
global _running
|
||||
_require_key(x_job_key, key)
|
||||
with _lock:
|
||||
if _running:
|
||||
raise HTTPException(status_code=409, detail='高股息检查已在运行')
|
||||
identity = str(uuid.uuid4())
|
||||
_jobs[identity] = {'task_id': identity, 'state': 'running', 'closed': False,
|
||||
'started_at': dt.datetime.now(dt.timezone.utc).isoformat(), 'tail': []}
|
||||
_running = identity
|
||||
for old in list(_jobs):
|
||||
if len(_jobs) > 20 and _jobs[old].get('closed'):
|
||||
del _jobs[old]
|
||||
threading.Thread(target=_run, args=(identity, callbackUrl), daemon=True).start()
|
||||
return {'status': 'accepted', 'task_id': identity}
|
||||
|
||||
|
||||
@router.get('/dividend-watch/status/{task_id}')
|
||||
def status(task_id: str, key: str = Query(None), x_job_key: str = Header(None)):
|
||||
_require_key(x_job_key, key)
|
||||
if task_id not in _jobs:
|
||||
raise HTTPException(status_code=404, detail='任务记录不可用,请查研究日检;不可推断成功')
|
||||
return _jobs[task_id]
|
||||
|
|
@ -0,0 +1,122 @@
|
|||
"""高股息观察取数。只读、单表查询,不调用模型或交易接口。"""
|
||||
import copy
|
||||
import datetime as dt
|
||||
import json
|
||||
import re
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
|
||||
from dividend_watch import BJ, candidates, day, digest, encode, instant, refresh_codes, unique
|
||||
|
||||
|
||||
def rows(cursor):
|
||||
names = [d[0] for d in cursor.description]
|
||||
return [dict(zip(names, row)) for row in cursor.fetchall()]
|
||||
|
||||
|
||||
def public_day(value):
|
||||
if value is None:
|
||||
return None
|
||||
raw = str(value)
|
||||
try:
|
||||
return dt.datetime.strptime(raw, '%Y%m%d').date() if re.fullmatch(r'\d{8}', raw) else day(raw)
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
def public_rows(records, cutoff):
|
||||
"""原始流量为累计值,保留报告类型,不在这里相加或假定为单季。"""
|
||||
accepted, quarantined = [], []
|
||||
for row in sorted(records, key=encode):
|
||||
dates = [public_day(row[k]) for k in ('ann_date', 'f_ann_date', 'imp_ann_date') if row.get(k)]
|
||||
if not dates or any(d is None for d in dates) or max(dates) > cutoff:
|
||||
quarantined.append(row)
|
||||
else:
|
||||
accepted.append(row)
|
||||
return {'records': accepted, 'quarantined': quarantined,
|
||||
'completeness_verified': False}
|
||||
|
||||
|
||||
def report_read(code, review, previous, base):
|
||||
"""旧报告不能固定追溯:前后核对最新索引,只保存这次读取的原文副本。"""
|
||||
if not re.fullmatch(r'\d{6}\.(SH|SZ|BJ)', code):
|
||||
raise ValueError('代码无效')
|
||||
identity = digest({k: review.get(k) for k in ('ran_at', 'summary', 'grades')})
|
||||
old = (previous or {}).get('report_capture') or {}
|
||||
if old.get('identity') == identity and old.get('status') == 'ok':
|
||||
return old
|
||||
ref = (review.get('summary') or {}).get('review_ref')
|
||||
suffix = '?ref=' + urllib.parse.quote(ref) if ref else ''
|
||||
url = base.rstrip('/') + f'/company/{code}/review'
|
||||
def get(path):
|
||||
with urllib.request.urlopen(path, timeout=15) as response:
|
||||
return response.read().decode('utf-8')
|
||||
def check_index():
|
||||
current = json.loads(get(url + '/summary'))
|
||||
if instant(current['ran_at']) != instant(review['ran_at']):
|
||||
raise ValueError('最新报告与冻结索引不同')
|
||||
try:
|
||||
if not ref:
|
||||
check_index()
|
||||
text = get(url + suffix)
|
||||
if not ref:
|
||||
check_index()
|
||||
return {'status': 'ok', 'identity': identity, 'markdown': text,
|
||||
'content_hash': digest(text), 'url': url + suffix,
|
||||
'traceability': 'fixed_reference' if ref else 'captured_current_legacy_report'}
|
||||
except Exception as exc:
|
||||
return {'status': 'unavailable', 'identity': identity, 'error_type': type(exc).__name__,
|
||||
'traceability': 'summary_only', 'url': url + suffix}
|
||||
|
||||
|
||||
def collect(expected, previous=None):
|
||||
import db
|
||||
import config
|
||||
data_day = day(expected)
|
||||
table = 'gp_stock_factor_pro_' + data_day.strftime('%Y%m')
|
||||
cutoff = dt.datetime.now(dt.timezone.utc)
|
||||
out = {'captured_at': cutoff.isoformat(), 'data_date': str(data_day),
|
||||
'expected_date': str(data_day), 'table': table}
|
||||
with db._mysql_cm('heat') as connection:
|
||||
with connection.cursor() as q:
|
||||
# 不用 MAX 日静默退回陈旧数据,精确查询上一完整交易日。
|
||||
q.execute(f'SELECT ts_code,symbol,trade_date,close,dv_ratio,dv_ttm,pe_ttm,pb,create_time FROM {table} WHERE trade_date=%s ORDER BY ts_code', (str(data_day),))
|
||||
out['factors'] = rows(q)
|
||||
q.execute('SELECT gp_code_two,gp_name,list_status,list_date,industry FROM gp_code_all ORDER BY gp_code_two')
|
||||
out['names'] = rows(q)
|
||||
if not out['factors'] or not out['names']:
|
||||
raise ValueError('预期数据日行情或名称为空')
|
||||
want = candidates(out)
|
||||
with db.akg_pg_conn() as connection:
|
||||
q = connection.execute("SELECT DISTINCT ON (ts_code) ts_code,period,review_version,grades,summary,ran_at,period_fi,period_rf,n_unknown,warnings,report_key,json_key FROM company_review_runs WHERE status='ok' AND ts_code=ANY(%s) AND ran_at<=%s ORDER BY ts_code,ran_at DESC,period DESC,review_version DESC", (want, cutoff))
|
||||
out['reviews'] = rows(q)
|
||||
old_reviews = unique((previous or {}).get('reviews', []), 'ts_code')
|
||||
for review in out['reviews']:
|
||||
code = review['ts_code']
|
||||
review['report_capture'] = report_read(code, review, old_reviews.get(code), config.AKG_API_BASE)
|
||||
review['report_url'] = review['report_capture']['url']
|
||||
prior_facts = (previous or {}).get('fundamentals') or {}
|
||||
out['fundamentals'] = {code: copy.deepcopy(prior_facts[code]) for code in want if code in prior_facts}
|
||||
due = refresh_codes(out, previous)
|
||||
if due:
|
||||
marks = ','.join(['%s']*len(due))
|
||||
lower = (data_day.year-5)*10000 + 101
|
||||
specs = (
|
||||
('financials', 'gp_fina_indicator', 'ts_code,ann_date,end_date,eps,dt_eps,roe,roe_dt,roic,netprofit_yoy,ocf_to_profit,debt_to_assets,or_yoy', 'ts_code,end_date,ann_date'),
|
||||
('cashflow', 'gp_cashflow', 'ts_code,ann_date,f_ann_date,end_date,comp_type,report_type,update_flag,net_profit,n_cashflow_act,n_cashflow_inv_act,c_pay_dist_dpcp_int_exp,free_cashflow', 'ts_code,end_date,ann_date,report_type'),
|
||||
('dividends', 'gp_dividend', 'ts_code,end_date,ann_date,div_proc,cash_div_tax,stk_div,ex_date,pay_date,imp_ann_date,base_date,base_share', 'ts_code,end_date,ann_date,ex_date,div_proc'),
|
||||
)
|
||||
raw = {code: {} for code in due}
|
||||
with db._mysql_cm('heat') as connection:
|
||||
with connection.cursor() as q:
|
||||
for key, name, fields, order in specs:
|
||||
q.execute(f'SELECT {fields} FROM {name} WHERE ts_code IN ({marks}) AND end_date>=%s ORDER BY {order}', tuple(due)+(lower,))
|
||||
by_code = {code: [] for code in due}
|
||||
for row in rows(q):
|
||||
by_code[row['ts_code']].append(row)
|
||||
for code in due:
|
||||
raw[code][key] = public_rows(by_code[code], cutoff.astimezone(BJ).date())
|
||||
for code in due:
|
||||
out['fundamentals'][code] = {'checked_at': cutoff.isoformat(), 'sources': raw[code]}
|
||||
# 统一 Decimal 与日期序列化后再计算指纹,真实取数与离线复演相同。
|
||||
return json.loads(encode(out))
|
||||
|
|
@ -0,0 +1,400 @@
|
|||
"""高股息独立研究;本地 python3 -m unittest test_dividend_watch。
|
||||
|
||||
桥机容器:python dividend_watch.py scan;只读上游,只写 data/dividend_watch。
|
||||
日检:python dividend_watch.py health。历史复演须使用 --input,不回填当前入口。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import collections
|
||||
import datetime as dt
|
||||
import fcntl
|
||||
import hashlib
|
||||
import html
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import tempfile
|
||||
import unicodedata
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
BJ = ZoneInfo('Asia/Shanghai')
|
||||
VERSION = '1'
|
||||
DEFAULT_ROOT = 'data/dividend_watch'
|
||||
LABELS = {'priority': '质地较好,优先核验', 'mixed': '观点有分歧',
|
||||
'risk': '已有明确风险', 'missing_review': '待补个股评析',
|
||||
'identity_unknown': '名称或上市状态待核验', 'excluded': '风险名称或非上市,已排除',
|
||||
'below': '滚动股息不足5%', 'yield_unknown': '滚动股息缺失'}
|
||||
|
||||
|
||||
def encode(value):
|
||||
return json.dumps(value, ensure_ascii=False, sort_keys=True, separators=(',', ':'),
|
||||
allow_nan=False, default=str).encode()
|
||||
|
||||
|
||||
def digest(value):
|
||||
return hashlib.sha256(encode(value)).hexdigest()
|
||||
|
||||
|
||||
def implementation_fingerprint():
|
||||
base = Path(__file__).parent
|
||||
return digest({name: hashlib.sha256((base/name).read_bytes()).hexdigest()
|
||||
for name in ('dividend_watch.py', 'dividend_sources.py')})
|
||||
|
||||
|
||||
def instant(value):
|
||||
result = dt.datetime.fromisoformat(str(value).replace('Z', '+00:00'))
|
||||
if result.tzinfo is None:
|
||||
raise ValueError('截止或来源时间缺少时区')
|
||||
return result
|
||||
|
||||
|
||||
def day(value):
|
||||
return dt.date.fromisoformat(str(value)[:10])
|
||||
|
||||
|
||||
def number(value):
|
||||
try:
|
||||
n = float(value)
|
||||
return n if math.isfinite(n) else None
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def risk_name(name):
|
||||
n = re.sub(r'\s+', '', unicodedata.normalize('NFKC', str(name or ''))).upper()
|
||||
while n.startswith(('XD', 'XR', 'DR')):
|
||||
n = n[2:]
|
||||
return n.startswith(('*ST', 'ST', 'SST', 'S*ST', '退市')) or n.endswith('退')
|
||||
|
||||
|
||||
def unique(rows, key):
|
||||
result = {}
|
||||
for row in rows:
|
||||
value = row.get(key)
|
||||
if not value or value in result:
|
||||
raise ValueError(f'来源身份缺失或重复:{key}={value}')
|
||||
result[value] = row
|
||||
return result
|
||||
|
||||
|
||||
def candidates(source):
|
||||
names = unique(source['names'], 'gp_code_two')
|
||||
factors = unique(source['factors'], 'ts_code')
|
||||
return sorted(code for code, f in factors.items()
|
||||
if (number(f.get('dv_ttm')) or 0) >= 5
|
||||
and names.get(code, {}).get('gp_name')
|
||||
and names[code].get('list_status') == 'L'
|
||||
and not risk_name(names[code]['gp_name']))
|
||||
|
||||
|
||||
def refresh_codes(source, previous):
|
||||
"""月初全查,新进入者当日补;接收新评析不等于重做财务材料。"""
|
||||
local = instant(source['captured_at']).astimezone(BJ)
|
||||
facts = (previous or {}).get('fundamentals') or {}
|
||||
want = candidates(source)
|
||||
due = [c for c in want if c not in facts or
|
||||
instant(facts[c]['checked_at']).astimezone(BJ).strftime('%Y-%m') != local.strftime('%Y-%m')]
|
||||
return due
|
||||
|
||||
|
||||
def required_period(on):
|
||||
"""按定期披露期限检查材料覆盖,不推断个别公司实际发布日期。"""
|
||||
if on.month >= 11:
|
||||
return f'{on.year}Q3'
|
||||
if on.month >= 9:
|
||||
return f'{on.year}Q2'
|
||||
if on.month >= 5:
|
||||
return f'{on.year}Q1'
|
||||
return f'{on.year-1}Q3'
|
||||
|
||||
|
||||
def assemble(source):
|
||||
cutoff = instant(source['captured_at'])
|
||||
data_day = day(source['data_date'])
|
||||
if data_day >= cutoff.astimezone(BJ).date():
|
||||
raise ValueError('本流程使用上一完整交易日,不接受当日未完成行情')
|
||||
if data_day != day(source['expected_date']):
|
||||
raise ValueError('股息数据日期落后或不匹配')
|
||||
if source['table'] != 'gp_stock_factor_pro_' + data_day.strftime('%Y%m'):
|
||||
raise ValueError('行情日期与月表不符')
|
||||
factors = unique(source['factors'], 'ts_code')
|
||||
names = unique(source['names'], 'gp_code_two')
|
||||
reviews = unique(source.get('reviews', []), 'ts_code')
|
||||
if not factors or not names:
|
||||
raise ValueError('全市场行情或名称源为空')
|
||||
for r in factors.values():
|
||||
if day(r['trade_date']) != data_day or not re.fullmatch(r'\d{6}\.(SH|SZ|BJ)', r['ts_code']):
|
||||
raise ValueError('混入异日行情或非法代码')
|
||||
for r in reviews.values():
|
||||
if instant(r['ran_at']) > cutoff:
|
||||
raise ValueError('评析晚于冻结截止时间')
|
||||
rows = []
|
||||
for code, f in sorted(factors.items()):
|
||||
identity = names.get(code, {})
|
||||
name = identity.get('gp_name') or ''
|
||||
y = number(f.get('dv_ttm'))
|
||||
rv = reviews.get(code, {})
|
||||
summary = rv.get('summary') or {}
|
||||
if not isinstance(summary, dict):
|
||||
raise ValueError('评析摘要格式异常')
|
||||
reason = []
|
||||
if risk_name(name) or (identity.get('list_status') and identity['list_status'] != 'L'):
|
||||
status = 'excluded'
|
||||
elif not name or not identity.get('list_status'):
|
||||
status = 'identity_unknown'
|
||||
elif y is None:
|
||||
status = 'yield_unknown'
|
||||
elif y < 5:
|
||||
status = 'below'
|
||||
elif not rv:
|
||||
status = 'missing_review'
|
||||
elif summary.get('overall') == '差' or summary.get('doubt_hard'):
|
||||
status = 'risk'
|
||||
elif summary.get('overall') == '好':
|
||||
status = 'priority'
|
||||
else:
|
||||
status = 'mixed'
|
||||
material_periods = {k: rv.get(k) or summary.get(k) for k in ('period_fi', 'period_rf')}
|
||||
if rv:
|
||||
if not all(material_periods.values()):
|
||||
reason.append('财务材料期未完整记录')
|
||||
elif len(set(material_periods.values())) > 1:
|
||||
reason.append('两路财务材料期不一致,需补齐较旧一路')
|
||||
required = required_period(cutoff.astimezone(BJ).date())
|
||||
if any(p and p < required for p in material_periods.values()):
|
||||
reason.append(f'财务指标尚未覆盖应披露的{required},新评析不能替代新材料')
|
||||
if (cutoff - instant(rv['ran_at'])).days > 31:
|
||||
reason.append('个股评析超过一个月,需复核')
|
||||
review_local = instant(rv['ran_at']).astimezone(BJ)
|
||||
current = cutoff.astimezone(BJ)
|
||||
if (review_local.year, (review_local.month-1)//3) != (current.year, (current.month-1)//3):
|
||||
reason.append('本季度尚无完整评析更新')
|
||||
if summary.get('numbers_unmatched'):
|
||||
reason.append('评析数字仍有未核验项')
|
||||
facts = (source.get('fundamentals') or {}).get(code)
|
||||
if status in ('priority', 'mixed', 'risk', 'missing_review'):
|
||||
reason.extend(['普通分红与特别分红需按公告区分', '分红记录完整性及支付能力待核验'])
|
||||
if not facts:
|
||||
reason.append('本月财务与分红事实未读取')
|
||||
elif instant(facts['checked_at']).astimezone(BJ).strftime('%Y-%m') != cutoff.astimezone(BJ).strftime('%Y-%m'):
|
||||
reason.append('财务与分红事实已到月度复查期')
|
||||
else:
|
||||
from dividend_sources import public_day
|
||||
required = required_period(cutoff.astimezone(BJ).date())
|
||||
year, qtr = required.split('Q')
|
||||
required_end = f'{year}-' + {'1': '03-31', '2': '06-30', '3': '09-30'}[qtr]
|
||||
for key, label in (('financials', '财务指标'), ('cashflow', '现金流报表'), ('dividends', '分红实施材料')):
|
||||
entries = facts['sources'][key]['records']
|
||||
periods = [public_day(r.get('end_date')) for r in entries]
|
||||
periods = [str(p) for p in periods if p]
|
||||
if not periods:
|
||||
reason.append(label + '缺失')
|
||||
elif key != 'dividends' and max(periods) < required_end:
|
||||
reason.append(label + f'仅到{max(periods)},需补{required}')
|
||||
if status == 'missing_review':
|
||||
reason.insert(0, '需补个股多视角评析及反方观点')
|
||||
if status == 'identity_unknown':
|
||||
reason.append('无法确认最新名称与上市状态,暂不纳入研究池')
|
||||
rows.append({'code': code, 'name': name, 'status': status,
|
||||
'yield_ttm_pct': y, 'yield_last_year_pct': number(f.get('dv_ratio')),
|
||||
'close': number(f.get('close')), 'quality': summary.get('overall'),
|
||||
'groups': summary.get('groups') or {}, 'perspectives': rv.get('grades') or {},
|
||||
'perspectives_text': summary.get('grade_line') or '',
|
||||
'thesis': summary.get('thesis'), 'antithesis': summary.get('antithesis'),
|
||||
'warnings': rv.get('warnings') or [], 'quality_reasons': summary.get('quality_reasons') or [],
|
||||
'unknown_n': summary.get('unknown_n'), 'material_periods': material_periods,
|
||||
'review_at': rv.get('ran_at'), 'review_ref': summary.get('review_ref'),
|
||||
'review_url': rv.get('report_url'), 'report_capture': rv.get('report_capture'),
|
||||
'fundamentals': facts, 'needs': reason,
|
||||
'sustainability': '待核验', 'investment_approved': False})
|
||||
counts = dict(collections.Counter(r['status'] for r in rows))
|
||||
high = [r for r in rows if (r['yield_ttm_pct'] or 0) >= 5]
|
||||
screen = {'market_rows': len(rows), 'high_yield': len(high),
|
||||
'excluded_high_yield': sum(r['status'] == 'excluded' for r in high),
|
||||
'identity_unknown_high_yield': sum(r['status'] == 'identity_unknown' for r in high),
|
||||
'research_pool': sum(r['status'] in ('priority','mixed','risk','missing_review') for r in high)}
|
||||
return {'schema_version': 1, 'code_version': VERSION, 'implementation_fingerprint': implementation_fingerprint(),
|
||||
'screening': screen, 'data_date': source['data_date'],
|
||||
'cutoff_at': source['captured_at'], 'source_table': source['table'],
|
||||
'research_only': True, 'counts': counts, 'rows': rows,
|
||||
'research_requests': [{'code': r['code'], 'name': r['name'], 'needs': r['needs']}
|
||||
for r in rows if r['status'] in ('priority', 'mixed', 'risk', 'missing_review')]}
|
||||
|
||||
|
||||
def fact_summary(facts):
|
||||
if not facts:
|
||||
return None
|
||||
from dividend_sources import public_day
|
||||
labels = {'financials': '财务指标', 'cashflow': '现金流报表', 'dividends': '分红记录'}
|
||||
result = {'本次读取': facts['checked_at'], '说明': '记录条数不代表材料完整。分红性质及支付能力仍须核验。'}
|
||||
for key, item in facts['sources'].items():
|
||||
periods = [public_day(r.get('end_date')) for r in item['records']]
|
||||
periods = [str(p) for p in periods if p]
|
||||
result[labels.get(key, key)] = {'记录数': len(item['records']), '最新材料期': max(periods) if periods else None,
|
||||
'日期不明或晚于截止的隔离条数': len(item['quarantined'])}
|
||||
return result
|
||||
|
||||
|
||||
def next_checks(row):
|
||||
if row['status'] == 'identity_unknown':
|
||||
return '先确认最新证券名称与上市状态。'
|
||||
parts = []
|
||||
if row['status'] == 'missing_review':
|
||||
parts.append('补齐个股评析与反方观点')
|
||||
if any('材料期' in n or re.search(r'需补\d{4}Q', n) or '财务指标尚未覆盖' in n for n in row['needs']):
|
||||
parts.append('补齐最新财报与现金流资料')
|
||||
parts.append('核实普通分红及其支付能力')
|
||||
return ';'.join(parts) + '。'
|
||||
|
||||
|
||||
def report(result):
|
||||
esc = lambda v: html.escape(str(v if v is not None else '未知'))
|
||||
counts = result['counts']
|
||||
screening = result['screening']
|
||||
stats = ' · '.join(f'{LABELS[k]} {counts.get(k, 0)}' for k in ('priority', 'mixed', 'risk', 'missing_review'))
|
||||
content = []
|
||||
for r in sorted(result['rows'], key=lambda r: (list(LABELS).index(r['status']), r['code'])):
|
||||
if r['status'] in ('below', 'yield_unknown', 'excluded'):
|
||||
continue
|
||||
if r['status'] == 'identity_unknown' and (r['yield_ttm_pct'] or 0) < 5:
|
||||
continue
|
||||
perspectives = r['perspectives_text'] or '视角资料待补齐'
|
||||
detail = esc(json.dumps({'各视角': r['perspectives'], '材料期': r['material_periods'],
|
||||
'完整核查事项': r['needs'],
|
||||
'来源': r['review_url'], '财务与分红事实': fact_summary(r['fundamentals']),
|
||||
'原报告留存': {k: v for k, v in (r['report_capture'] or {}).items() if k != 'markdown'}}, ensure_ascii=False, indent=2))
|
||||
content.append(f'<article data-status="{esc(r["status"])}"><h2>{esc(r["name"] or "名称待核")} '
|
||||
f'<small>{esc(r["code"])}</small><b>{r["yield_ttm_pct"]:.2f}%</b></h2>'
|
||||
f'<p class="tag">{esc(LABELS[r["status"]])} · 持续性待核验</p>'
|
||||
f'<p><strong>依据</strong> {esc(r["thesis"] or "尚缺个股评析")}</p>'
|
||||
f'<p><strong>反方</strong> {esc(r["antithesis"] or "尚未形成反方研究,不能视为无风险")}</p>'
|
||||
f'<p><strong>待查</strong> {esc(next_checks(r))}</p>'
|
||||
f'<details><summary>各方观点、材料与来源</summary><p>{esc(perspectives)}</p><pre>{detail}</pre></details></article>')
|
||||
return '<!doctype html><meta charset="utf-8"><title>高股息研究观察</title>' + '''<style>
|
||||
body{font:15px/1.7 system-ui,sans-serif;background:#f5f6f8;color:#263342;margin:32px auto;max-width:1080px;padding:0 24px}
|
||||
h1{font-size:28px}h2{font-size:20px;margin:0}small{font-size:13px;color:#7c8794;font-weight:400}b{float:right;color:#146a60}
|
||||
article{background:white;border:1px solid #e2e6eb;border-radius:9px;padding:20px;margin:14px 0}p{margin:8px 0}.tag{color:#8b6428}
|
||||
pre{white-space:pre-wrap;overflow-wrap:anywhere;font-size:12px}details{color:#59687a}input,select{padding:8px;border:1px solid #ccd3dc;border-radius:5px}
|
||||
</style>''' + f'<h1>高股息研究观察</h1><p>行情 {esc(result["data_date"])} · 检查 {esc(instant(result["cutoff_at"]).astimezone(BJ).strftime("%Y-%m-%d %H:%M"))} 北京时间</p><p>高股息 {screening["high_yield"]} 只 · 风险名称排除 {screening["excluded_high_yield"]} 只 · 身份待核 {screening["identity_unknown_high_yield"]} 只 · 进入研究 {screening["research_pool"]} 只</p><p>{esc(stats)}</p>' + '<p>滚动股息不低于5%。以下是研究线索,持续分红尚需公告和支付能力证据。</p><input id="search" placeholder="搜索名称、代码或风险"><select id="filter"><option value="">全部研究状态</option>' + ''.join(f'<option value="{k}">{v}</option>' for k,v in LABELS.items() if k not in ('below','yield_unknown','excluded')) + '</select><main>' + ''.join(content) + '''</main><script>
|
||||
function filter(){const q=document.querySelector('#search').value.toLowerCase(),s=document.querySelector('#filter').value;document.querySelectorAll('article').forEach(a=>a.hidden=(!a.textContent.toLowerCase().includes(q)||(s&&a.dataset.status!==s)));}
|
||||
document.querySelector('#search').addEventListener('input',filter);document.querySelector('#filter').addEventListener('change',filter);
|
||||
</script>'''
|
||||
|
||||
|
||||
def atomic(path, data):
|
||||
path = Path(path)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
fd, tmp = tempfile.mkstemp(dir=path.parent, prefix='.tmp-')
|
||||
try:
|
||||
with os.fdopen(fd, 'wb') as f:
|
||||
f.write(data); f.flush(); os.fsync(f.fileno())
|
||||
os.replace(tmp, path)
|
||||
finally:
|
||||
if os.path.exists(tmp):
|
||||
os.unlink(tmp)
|
||||
|
||||
|
||||
def read_latest(root):
|
||||
p = Path(root)/'latest.json'
|
||||
if not p.exists():
|
||||
return None
|
||||
info = json.loads(p.read_text())
|
||||
if not re.fullmatch('[0-9a-f]{64}', info['id']):
|
||||
raise ValueError('研究入口引用无效')
|
||||
raw = (Path(root)/'versions'/info['id']/'bundle.json').read_bytes()
|
||||
if hashlib.sha256(raw).hexdigest() != info['sha256']:
|
||||
raise ValueError('研究归档校验失败')
|
||||
return json.loads(raw)
|
||||
|
||||
|
||||
def publish(root, source):
|
||||
root = Path(root); root.mkdir(parents=True, exist_ok=True)
|
||||
source = json.loads(encode(source))
|
||||
for key, field in (('factors', 'ts_code'), ('names', 'gp_code_two'), ('reviews', 'ts_code')):
|
||||
source[key] = sorted(source.get(key, []), key=lambda row: str(row.get(field) or ''))
|
||||
result = assemble(source)
|
||||
# 采集时刻仅是观察时间。去掉它实现同日同输入重复运行幂等。
|
||||
effective = {k: v for k,v in source.items() if k != 'captured_at'}
|
||||
identity = digest({'source': effective, 'code_version': VERSION,
|
||||
'implementation_fingerprint': result['implementation_fingerprint']})
|
||||
with (root/'publish.lock').open('a') as lock:
|
||||
fcntl.flock(lock, fcntl.LOCK_EX)
|
||||
old = read_latest(root)
|
||||
if old and old['id'] == identity:
|
||||
return old, False
|
||||
if old and day(old['result']['data_date']) > day(result['data_date']):
|
||||
raise ValueError('不允许旧行情替换最新研究')
|
||||
directory = root/'versions'/identity
|
||||
bundle = {'id': identity, 'parent_id': old['id'] if old else None,
|
||||
'source': source, 'result': result}
|
||||
path = directory/'bundle.json'
|
||||
if path.exists():
|
||||
# 中断后沿用已完整写下的首份归档,不覆盖其截止和父引用。
|
||||
bundle = json.loads(path.read_text())
|
||||
if bundle['id'] != identity:
|
||||
raise ValueError('研究归档身份不符')
|
||||
else:
|
||||
atomic(path, encode(bundle))
|
||||
atomic(directory/'report.html', report(bundle['result']).encode())
|
||||
atomic(root/'latest.json', encode({'id': identity, 'sha256': hashlib.sha256(path.read_bytes()).hexdigest()}))
|
||||
return bundle, True
|
||||
|
||||
|
||||
def health(root, expected):
|
||||
try:
|
||||
bundle = read_latest(root)
|
||||
attempt_path = Path(root)/'last_check.json'
|
||||
attempt = json.loads(attempt_path.read_text()) if attempt_path.exists() else {}
|
||||
ok = bool(bundle and day(bundle['result']['data_date']) == day(expected)
|
||||
and attempt.get('ok') and attempt.get('id') == bundle['id'])
|
||||
return {'ok': ok, 'expected_date': str(expected),
|
||||
'data_date': bundle['result']['data_date'] if bundle else None,
|
||||
'counts': bundle['result']['counts'] if bundle else {}, 'last_check': attempt}
|
||||
except Exception as exc:
|
||||
return {'ok': False, 'error_type': type(exc).__name__}
|
||||
|
||||
|
||||
def main():
|
||||
p = argparse.ArgumentParser(description=__doc__)
|
||||
p.add_argument('command', choices=('scan', 'health'))
|
||||
p.add_argument('--input', help='离线冻结输入,复演必须指定独立 --out')
|
||||
p.add_argument('--out', default=DEFAULT_ROOT)
|
||||
p.add_argument('--date', help='日检预期数据日;扫描时只能等于真实上一交易日')
|
||||
a = p.parse_args()
|
||||
if a.input:
|
||||
if a.out == DEFAULT_ROOT or a.command != 'scan':
|
||||
p.error('离线复演只允许 scan 且须指定独立 --out')
|
||||
source = json.loads(Path(a.input).read_text())
|
||||
expected = source['expected_date']
|
||||
else:
|
||||
from build_readiness import expected_data_day
|
||||
expected = expected_data_day().isoformat()
|
||||
if a.date and a.command == 'scan' and a.date != expected:
|
||||
p.error('自动观察不能回填历史日期')
|
||||
if a.command == 'health':
|
||||
status = health(a.out, a.date or expected)
|
||||
print(json.dumps(status, ensure_ascii=False))
|
||||
return 0 if status['ok'] else 1
|
||||
root = Path(a.out); root.mkdir(parents=True, exist_ok=True)
|
||||
with (root/'scan.lock').open('a') as lock:
|
||||
fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||
try:
|
||||
if not a.input:
|
||||
from dividend_sources import collect
|
||||
prior = read_latest(root)
|
||||
source = collect(expected, prior['source'] if prior else None)
|
||||
bundle, changed = publish(root, source)
|
||||
atomic(root/'last_check.json', encode({'ok': True, 'at': dt.datetime.now(BJ).isoformat(), 'id': bundle['id']}))
|
||||
print(json.dumps({'changed': changed, 'id': bundle['id'], 'counts': bundle['result']['counts']}, ensure_ascii=False))
|
||||
return 0
|
||||
except Exception as exc:
|
||||
atomic(root/'last_check.json', encode({'ok': False, 'at': dt.datetime.now(BJ).isoformat(), 'error_type': type(exc).__name__}))
|
||||
print('研究检查失败,保留原归档:' + type(exc).__name__)
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
raise SystemExit(main())
|
||||
|
|
@ -817,3 +817,12 @@
|
|||
- **依据。** 原表211行均匹配上一年度字段。供应商滚动值108家不低于5%、95家低于5%、8家缺值。已发现格力、海尔及山东出版实施记录遗漏,203家库内财务只到2026一季度。单靠旧股息率不足以支持持续买入。
|
||||
- **预期。** 建立有反方证据的研究记录,分开检验基本面筛选与条件持续投入。16家目的抽样不代表全市场效果;现有候选门槛、排序、交易时段、仓位和执行保持。
|
||||
- **复核日期。** 本轮211行与880个公式已核验。下一批完成样本分红及半年报核查后,再冻结正常与压力情景。尚未启动组合或历史策略效果评价。
|
||||
|
||||
|
||||
## 080 · 2026-10-21 · 高股息每日初筛与低频研究分开
|
||||
|
||||
- **日期。** 2026-10-21,北京时间。
|
||||
- **改动。** 按用户要求,ST及退市名称先排除,名称或上市状态不明者隔离。股息用对应月表的dv_ttm每日检查,财务分红事实月度复用,新入池立即补查。季度列出完整评析复核需求。独立研究任务不占计划修订锁,不加入默认构建链。
|
||||
- **依据。** 10-20月表5568行中,171只滚动股息不低于5%。排除ST八菱1只,6只名称缺失,余164只仅9只有评析,其中好2、中3、差4。旧财务材料不能因新报告时间变成新材料。
|
||||
- **预期。** 高股息入口、公司研究与分红持续性分开。所有持续性暂待专项核验。原211只及16只样本记录保留,079中ST仅作旧样本的安排不延续到新池。现有排序、门槛、交易参数不动。
|
||||
- **复核日期。** 本地36项新增测试和63项联合禁网回归通过,原XXL13项回归通过。真实同月复查2.438秒,164家财务缓存和9份报告复用。未部署、未启用日程,季度研究和自动补齐评析未完成。
|
||||
|
|
|
|||
|
|
@ -25,4 +25,5 @@ except ImportError:
|
|||
resp = types.ModuleType('fastapi.responses')
|
||||
resp.PlainTextResponse = lambda content, **kw: content
|
||||
resp.JSONResponse = lambda content, **kw: content
|
||||
resp.HTMLResponse = lambda content, **kw: content
|
||||
sys.modules['fastapi.responses'] = resp
|
||||
|
|
|
|||
|
|
@ -0,0 +1,59 @@
|
|||
"""本地:python3 -m unittest test_dividend_jobs;线程、子进程和回调使用替身。"""
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
import test_api_support
|
||||
import dividend_jobs as jobs
|
||||
import xxl
|
||||
|
||||
|
||||
class JobsTest(unittest.TestCase):
|
||||
def setUp(self):
|
||||
jobs._jobs.clear(); jobs._running = None
|
||||
self.key = patch.object(xxl.config, 'XXL_TRIGGER_KEY', 'test-key'); self.key.start()
|
||||
def tearDown(self):
|
||||
self.key.stop(); jobs._jobs.clear(); jobs._running = None
|
||||
|
||||
def test_auth_required(self):
|
||||
with self.assertRaises(jobs.HTTPException) as e:
|
||||
jobs.trigger(callbackUrl=None, key='bad', x_job_key=None)
|
||||
self.assertEqual(e.exception.status_code, 401)
|
||||
|
||||
def test_duplicate_rejected(self):
|
||||
with patch.object(jobs.threading, 'Thread'):
|
||||
accepted = jobs.trigger(callbackUrl=None, key='test-key', x_job_key=None)
|
||||
with self.assertRaises(jobs.HTTPException) as e:
|
||||
jobs.trigger(callbackUrl=None, key='test-key', x_job_key=None)
|
||||
self.assertEqual(e.exception.status_code, 409)
|
||||
self.assertEqual(jobs._running, accepted['task_id'])
|
||||
|
||||
def test_plan_running_does_not_block_research(self):
|
||||
with patch.object(xxl, '_running_id', 'plan'), patch.object(jobs.threading, 'Thread'):
|
||||
result = jobs.trigger(callbackUrl=None, key='test-key', x_job_key=None)
|
||||
self.assertEqual(result['status'], 'accepted')
|
||||
self.assertEqual(xxl._running_id, 'plan')
|
||||
|
||||
def run_result(self, outcome):
|
||||
with patch.object(jobs.threading, 'Thread'):
|
||||
result = jobs.trigger(callbackUrl='http://callback.invalid', key='test-key', x_job_key=None)
|
||||
with patch.object(jobs, '_run_step', return_value=outcome) as run, patch.object(jobs, '_notify') as notify, patch.object(xxl, '_running_id', 'plan'):
|
||||
jobs._run(result['task_id'], 'http://callback.invalid')
|
||||
self.assertEqual(xxl._running_id, 'plan')
|
||||
self.assertEqual(run.call_args.args[0], ['dividend_watch.py', 'scan'])
|
||||
self.assertEqual(notify.call_args.args[2], outcome[0] == 0)
|
||||
self.assertIsNone(jobs._running)
|
||||
return jobs._jobs[result['task_id']]
|
||||
|
||||
def test_success_after_completion(self):
|
||||
self.assertEqual(self.run_result((0,'done'))['state'], 'success')
|
||||
|
||||
def test_failure_callback_not_success(self):
|
||||
self.assertEqual(self.run_result((1,'source unavailable'))['state'], 'failed')
|
||||
|
||||
def test_status_after_restart_unknown(self):
|
||||
with self.assertRaises(jobs.HTTPException) as e:
|
||||
jobs.status('lost', key='test-key', x_job_key=None)
|
||||
self.assertEqual(e.exception.status_code,404)
|
||||
|
||||
|
||||
if __name__ == '__main__': unittest.main()
|
||||
|
|
@ -0,0 +1,210 @@
|
|||
"""本地:python3 -m unittest test_dividend_watch;纯标准库,无网络或数据库。"""
|
||||
import copy
|
||||
import datetime as dt
|
||||
import json
|
||||
from pathlib import Path
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
import dividend_watch as d
|
||||
import dividend_sources as s
|
||||
|
||||
|
||||
def source():
|
||||
return {'captured_at': '2026-10-21T08:00:00+08:00', 'expected_date': '2026-10-20',
|
||||
'data_date': '2026-10-20', 'table': 'gp_stock_factor_pro_202610',
|
||||
'factors': [{'ts_code': '600001.SH', 'trade_date': '2026-10-20', 'dv_ttm': 5, 'dv_ratio': 9, 'close': 10}],
|
||||
'names': [{'gp_code_two': '600001.SH', 'gp_name': '示例公司', 'list_status': 'L'}],
|
||||
'reviews': [], 'fundamentals': {}}
|
||||
|
||||
|
||||
def reviewed(q='好'):
|
||||
x = source()
|
||||
x['reviews'] = [{'ts_code': '600001.SH', 'ran_at': '2026-10-20T09:00:00+08:00',
|
||||
'period_fi': '2026Q1', 'period_rf': '2026Q2', 'grades': {'buffett': '好', 'earnings_quality': '差'},
|
||||
'summary': {'overall': q, 'thesis': '盈利稳定', 'antithesis': '现金转化下降', 'unknown_n': 3}}]
|
||||
return x
|
||||
|
||||
|
||||
class ClassificationTest(unittest.TestCase):
|
||||
def test_risk_variants(self):
|
||||
for name in ['ST公司', '*ST公司', 'ST公司', 'S*ST公司', 'SST公司', 'XDst公司', 'XR* ST公司', '退市公司', '公司退']:
|
||||
with self.subTest(name=name): self.assertTrue(d.risk_name(name))
|
||||
self.assertFalse(d.risk_name('厦门国贸'))
|
||||
|
||||
def test_st_before_any_yield(self):
|
||||
x = source(); x['names'][0]['gp_name'] = '*ST公司'; x['factors'][0]['dv_ttm'] = None
|
||||
self.assertEqual(d.assemble(x)['rows'][0]['status'], 'excluded')
|
||||
self.assertEqual(d.candidates(x), [])
|
||||
|
||||
def test_missing_name_never_passes(self):
|
||||
x = source(); x['names'][0]['gp_name'] = ''
|
||||
self.assertEqual(d.assemble(x)['rows'][0]['status'], 'identity_unknown')
|
||||
self.assertEqual(d.candidates(x), [])
|
||||
|
||||
def test_listing_status_unknown_and_delisted(self):
|
||||
for v, expected in [(None, 'identity_unknown'), ('D', 'excluded')]:
|
||||
x = source(); x['names'][0]['list_status'] = v
|
||||
self.assertEqual(d.assemble(x)['rows'][0]['status'], expected)
|
||||
|
||||
def test_boundary_and_missing_not_previous_year(self):
|
||||
for value, expected in [(5, 'missing_review'), (4.9999, 'below'), (None, 'yield_unknown'), ('NaN', 'yield_unknown')]:
|
||||
x = source(); x['factors'][0]['dv_ttm'] = value
|
||||
self.assertEqual(d.assemble(x)['rows'][0]['status'], expected)
|
||||
|
||||
def test_good_not_sustainability_approval(self):
|
||||
r = d.assemble(reviewed())['rows'][0]
|
||||
self.assertEqual(r['status'], 'priority'); self.assertFalse(r['investment_approved'])
|
||||
self.assertEqual(r['sustainability'], '待核验')
|
||||
|
||||
def test_bad_and_divided(self):
|
||||
self.assertEqual(d.assemble(reviewed('差'))['rows'][0]['status'], 'risk')
|
||||
self.assertEqual(d.assemble(reviewed('中'))['rows'][0]['status'], 'mixed')
|
||||
|
||||
def test_opposing_view_and_material_age_preserved(self):
|
||||
r = d.assemble(reviewed())['rows'][0]
|
||||
self.assertEqual(r['antithesis'], '现金转化下降')
|
||||
self.assertEqual(r['perspectives']['earnings_quality'], '差')
|
||||
self.assertTrue(any('2026Q2' in n for n in r['needs']))
|
||||
|
||||
def test_quarter_review_due(self):
|
||||
x = reviewed(); x['reviews'][0]['ran_at'] = '2026-09-30T09:00:00+08:00'
|
||||
self.assertIn('本季度尚无完整评析更新', d.assemble(x)['rows'][0]['needs'])
|
||||
|
||||
def test_future_review_rejected(self):
|
||||
x = reviewed(); x['reviews'][0]['ran_at'] = '2026-10-21T09:00:00+08:00'
|
||||
with self.assertRaises(ValueError): d.assemble(x)
|
||||
|
||||
def test_duplicate_source_rejected(self):
|
||||
for key in ('factors', 'names'):
|
||||
x = source(); x[key].append(copy.deepcopy(x[key][0]))
|
||||
with self.assertRaises(ValueError): d.assemble(x)
|
||||
|
||||
def test_stale_mixed_dates_and_month_table(self):
|
||||
for change in ({'expected_date': '2026-10-21'}, {'table': 'gp_stock_factor_pro_202609'}, {'data_date': '2026-10-21'}):
|
||||
x = source(); x.update(change)
|
||||
with self.assertRaises(ValueError): d.assemble(x)
|
||||
|
||||
def test_timezone_required(self):
|
||||
x = source(); x['captured_at'] = '2026-10-21T08:00:00'
|
||||
with self.assertRaises(ValueError): d.assemble(x)
|
||||
|
||||
def test_november_uses_october_data_table(self):
|
||||
x = source(); x.update(captured_at='2026-11-02T08:00:00+08:00', expected_date='2026-10-30',data_date='2026-10-30')
|
||||
x['factors'][0]['trade_date'] = '2026-10-30'
|
||||
self.assertEqual(d.assemble(x)['source_table'], 'gp_stock_factor_pro_202610')
|
||||
|
||||
def test_deterministic_rows(self):
|
||||
x = source(); second = dict(x['factors'][0], ts_code='600002.SH'); x['factors'].append(second)
|
||||
a = d.assemble(x); x['factors'].reverse()
|
||||
self.assertEqual(a, d.assemble(x))
|
||||
|
||||
def test_report_escapes_untrusted_prose(self):
|
||||
x = reviewed(); x['reviews'][0]['summary']['thesis'] = '<img src=x onerror=alert(1)>'
|
||||
text = d.report(d.assemble(x))
|
||||
self.assertNotIn('<img', text); self.assertIn('<img', text)
|
||||
|
||||
|
||||
class CadenceTest(unittest.TestCase):
|
||||
def test_monthly_cache_reuse(self):
|
||||
old = {'fundamentals': {'600001.SH': {'checked_at': '2026-10-01T08:00:00+08:00'}}}
|
||||
self.assertEqual(d.refresh_codes(source(), old), [])
|
||||
|
||||
def test_new_month_and_new_candidate_due(self):
|
||||
self.assertEqual(d.refresh_codes(source(), None), ['600001.SH'])
|
||||
old = {'fundamentals': {'600001.SH': {'checked_at': '2026-09-30T08:00:00+08:00'}}}
|
||||
self.assertEqual(d.refresh_codes(source(), old), ['600001.SH'])
|
||||
|
||||
def test_announcement_cutoff_and_missing(self):
|
||||
rows = [{'ann_date': 20261021}, {'ann_date': 20261022}, {'ann_date': None}, {'ann_date': 20261020, 'f_ann_date': 20261022}]
|
||||
r = s.public_rows(rows, dt.date(2026,10,21))
|
||||
self.assertEqual(len(r['records']), 1); self.assertEqual(len(r['quarantined']), 3)
|
||||
self.assertFalse(r['completeness_verified'])
|
||||
|
||||
def test_financial_source_order_stable(self):
|
||||
rows = [{'ann_date': 20260101, 'x': 1}, {'ann_date': 20260101, 'x': 2}]
|
||||
self.assertEqual(s.public_rows(rows, dt.date(2026,10,21)), s.public_rows(rows[::-1], dt.date(2026,10,21)))
|
||||
|
||||
def test_report_cache_no_network(self):
|
||||
v = reviewed()['reviews'][0]
|
||||
identity = d.digest({k: v.get(k) for k in ('ran_at','summary','grades')})
|
||||
old = {'report_capture': {'identity': identity, 'status': 'ok', 'markdown': '原文'}}
|
||||
with patch('urllib.request.urlopen', side_effect=AssertionError('no network')):
|
||||
self.assertEqual(s.report_read('600001.SH', v, old, 'http://unused'), old['report_capture'])
|
||||
|
||||
def test_unavailable_report_not_claimed_full(self):
|
||||
with patch('urllib.request.urlopen', side_effect=OSError('down')):
|
||||
r = s.report_read('600001.SH', reviewed()['reviews'][0], None, 'http://unused')
|
||||
self.assertEqual(r['traceability'], 'summary_only')
|
||||
self.assertNotIn('markdown', r)
|
||||
|
||||
|
||||
class PublishTest(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory(); self.root = Path(self.tmp.name)
|
||||
def tearDown(self): self.tmp.cleanup()
|
||||
|
||||
def test_repeated_input_idempotent(self):
|
||||
x = source(); a, changed = d.publish(self.root, x); self.assertTrue(changed)
|
||||
x['captured_at'] = '2026-10-21T09:00:00+08:00'
|
||||
b, changed = d.publish(self.root, x); self.assertFalse(changed); self.assertEqual(a, b)
|
||||
|
||||
def test_changed_view_keeps_old_archive(self):
|
||||
a, _ = d.publish(self.root, reviewed()); old = (self.root/'versions'/a['id']/'bundle.json').read_bytes()
|
||||
b, _ = d.publish(self.root, reviewed('差'))
|
||||
self.assertEqual(b['parent_id'], a['id'])
|
||||
self.assertEqual((self.root/'versions'/a['id']/'bundle.json').read_bytes(), old)
|
||||
|
||||
def test_bad_source_keeps_pointer(self):
|
||||
a, _ = d.publish(self.root, source()); x = source(); x['factors'] = []
|
||||
with self.assertRaises(ValueError): d.publish(self.root, x)
|
||||
self.assertEqual(d.read_latest(self.root)['id'], a['id'])
|
||||
|
||||
def test_interrupted_publish_and_recovery(self):
|
||||
a, _ = d.publish(self.root, source()); write = d.atomic
|
||||
def fail_pointer(path, data):
|
||||
if Path(path).name == 'latest.json': raise OSError('interrupted')
|
||||
return write(path, data)
|
||||
with patch.object(d, 'atomic', side_effect=fail_pointer):
|
||||
with self.assertRaises(OSError): d.publish(self.root, reviewed())
|
||||
self.assertEqual(d.read_latest(self.root)['id'], a['id'])
|
||||
b, _ = d.publish(self.root, reviewed()); self.assertNotEqual(a['id'], b['id'])
|
||||
|
||||
def test_corruption_detected(self):
|
||||
a, _ = d.publish(self.root, source())
|
||||
(self.root/'versions'/a['id']/'bundle.json').write_text('{}')
|
||||
self.assertFalse(d.health(self.root, '2026-10-20')['ok'])
|
||||
|
||||
def test_health_requires_successful_check(self):
|
||||
bundle, _ = d.publish(self.root, source()); self.assertFalse(d.health(self.root, '2026-10-20')['ok'])
|
||||
d.atomic(self.root/'last_check.json', d.encode({'ok':True, 'id': bundle['id']}))
|
||||
self.assertTrue(d.health(self.root, '2026-10-20')['ok'])
|
||||
self.assertFalse(d.health(self.root, '2026-10-21')['ok'])
|
||||
d.atomic(self.root/'last_check.json', d.encode({'ok':False}))
|
||||
self.assertFalse(d.health(self.root, '2026-10-20')['ok'])
|
||||
|
||||
|
||||
class IntegrationContractTest(unittest.TestCase):
|
||||
def test_business_defaults_unchanged_and_explicit_entry(self):
|
||||
import ast
|
||||
tree = ast.parse(Path('xxl.py').read_text())
|
||||
values = {}
|
||||
for node in tree.body:
|
||||
if isinstance(node, ast.Assign):
|
||||
for target in node.targets:
|
||||
if isinstance(target, ast.Name) and target.id in ('DEFAULT_STEPS', 'STEP_ORDER', 'STEP_CMDS'):
|
||||
values[target.id] = ast.literal_eval(node.value)
|
||||
self.assertEqual(values['DEFAULT_STEPS'], ('build','plan','push-pool'))
|
||||
self.assertNotIn('dividend-watch', values['STEP_ORDER'])
|
||||
self.assertNotIn('dividend-watch', values['STEP_CMDS'])
|
||||
|
||||
def test_read_query_cannot_collect_or_publish(self):
|
||||
import ast
|
||||
tree = ast.parse(Path('api.py').read_text())
|
||||
fn = next(x for x in tree.body if isinstance(x, ast.FunctionDef) and x.name == 'dividend_research')
|
||||
calls = [n.func.attr for n in ast.walk(fn) if isinstance(n, ast.Call) and isinstance(n.func, ast.Attribute)]
|
||||
self.assertNotIn('collect', calls); self.assertNotIn('publish', calls)
|
||||
|
||||
|
||||
if __name__ == '__main__': unittest.main()
|
||||
Loading…
Reference in New Issue