diff --git a/DEVLOG.md b/DEVLOG.md index 5a6e23f..224f5ae 100644 --- a/DEVLOG.md +++ b/DEVLOG.md @@ -2792,3 +2792,12 @@ PMS最终代码56db0fd已推送,155 web/WS/beat为execution-2b1c447,worker 核验发现16:16盘后回补导致雷赛智能100股缺账,早于本次发布。用户确认盘中实际成交48.29元,沿既有回放当天入original,现金和数量归零差异。另按seq1599512原始FILLED回执对同一子单作审计更正,保留seq1598347原EXPIRED,不改通用终态保护。父目标300、累计100、剩余200;费用仍未知。没有新委托、伪造回报或自动跨策略认领。 16:47确认1600016、160条连续、资金16.9秒,待入账/孤儿/在途均0,账实一致。自然异步仍0,明日快照尚待夜间生成;已更新11-04 10:00一次性v3核验,加入当日缓存及迟报防重检查。实际配置、备份、测试日志和回退见发布接续及docs/evidence/pms_price_limits_release_20261103.json。后续仅文档推送不重建镜像。 + + +## 2026-11-05 QMT工作台成交修正接入 + +按用户提供的trade_correction协议实现原成交号不变、修正编号幂等及配套amended委托回报。修正与累计回报先收存并等待连续水位,原成交先入账;后续按原策略归属、当前有效版本和完整批次证据核验后,在同账户事务内调整本金、数量、成本、父子进度及网格档位,原始成交与费用事实保留。终态不恢复,数量下降不自动补单;缺失新持仓快照只等待,身份冲突、跨旧仓轮次或无法证明现账时挂起审计。 + +补齐普通成交的批次来源凭据、终态累计数量审计、资金快照截止、旧委托状态并发写回和网格旧任务/旧提议版本保护。挂起结果沿回放与日终摘要显示,不以ok=true遮蔽待核实。普通amended=false报文兼容,无关坏回执不污染其它修正。新专项覆盖批次投影、报文乱序重复、资金事务、普通入账凭据及网格并发;开发机全量ALL SUITES PASS,最后坏行隔离另做定向验证,最终提交仍需候选断网全量。 + +155发布前数量、现金一致,无待入账、孤儿或真实修正报文。计划仅重建web、worker和唯一WS,保留配置、其他服务及188。实际镜像和自然判收将在本次[成交修正接续](docs/QMT成交修正接入与发布_2026-11-05.md)追加,本条不代表已发布。 diff --git a/QMT_WS_PROTOCOL.md b/QMT_WS_PROTOCOL.md index c8f3d17..e3caadb 100644 --- a/QMT_WS_PROTOCOL.md +++ b/QMT_WS_PROTOCOL.md @@ -268,7 +268,7 @@ QMT 撤在途部分,已成交部分保留。撤单完成后该指令进入 `CA ### 5.4 `order_update` —— 委托状态变更 -每次状态发生变化推一条,**终态必须推且只推一次**。 +每次状态发生变化推一条,普通执行终态必须推送。终态后的成交修正另按§5.5.1处理,不表示重新开放执行。 ```json { @@ -284,7 +284,7 @@ QMT 撤在途部分,已成交部分保留。撤单完成后该指令进入 `CA | `cum_qty` | **本委托**累计成交股数。口径明确为本委托,不是当日、不是该股票持仓 | | `cum_avg_price` | **本委托**累计成交均价,2 位小数 | | `leaves_qty` | 未成交剩余 = `qty - cum_qty`。撤单/过期后应为 0 | -| `final` | 是否终态。终态后该 `instruction_id` 不再有任何 `order_update` | +| `final` | 是否执行终态。普通状态不能使终态回退;终态后的修正配套回报须带§5.5.1标记 | > 这三个字段是为了消灭原 `trading_log` 示例里 `total_filled=5000` 而 `order_quantity=600` 那种口径歧义。凡是累计量,一律以「本委托」为准。 @@ -304,17 +304,42 @@ QMT 撤在途部分,已成交部分保留。撤单完成后该指令进入 `CA | 字段 | 说明 | |---|---| -| `trade_no` | 该笔成交的唯一编号。**格式固定为 `{broker_order_id}#{n}`**,`n` 为该委托内成交序号,从 1 起递增。券商原生只提供委托号(`order_id`),一次委托多次成交时委托号相同、无法区分,故由 QMT 按此规则拼出笔号。跨重启唯一性由「委托号唯一 + n 从该委托已回报笔数续接」保证 | +| `trade_no` | 该笔成交的稳定唯一编号。普通券商成交沿用`{broker_order_id}#{n}`,工作台可使用`T-DESK-…`。PMS将其作为不透明标识,不根据格式推断策略。修正时不换号,跨重启仍须唯一 | | `qty` / `price` | **本笔**成交股数与价格 | | `amount` | `price × qty`,QMT 给出,PMS 会校验一致性(差异 > 0.01 元告警) | | `fee` | 本笔手续费(佣金 + 印花税 + 过户费合计)。QMT 侧按费率**估算**,与实际扣费可能有误差 | -| `fee_estimated` | `true` 表示 `fee` 为估算值。PMS 据此决定是否把它计入成本 | +| `fee_estimated` | `true`表示`fee`为估算值,用于费用追溯;不改变费用与成交成本分开记录的口径 | -**关于手续费的处理口径**(QMT 侧已说明逐笔费用未必精确):PMS **不把 `fee` 摊进持仓成本**,成本价只由 `price × qty` 决定;费用单独记为一笔现金流出。这样估算误差不会污染摊薄成本,进而不会污染安全垫(cushion)与保垫减仓的判断——这是 PMS 里最不能被噪声干扰的一条计算链。真实费用在日终用资金快照反推校准(总资产变动 − 成交净额 = 当日实际费用),只入现金账,不回溯改成本。 +**关于手续费的处理口径**:PMS不把`fee`摊进成交成本,费用单独记为现金流出。费用为空表示未知。双策略场景不能用账户总资产变化倒推并分摊逐笔费用;真实费用须按实际明细另行核对。历史日总额校准接口不能替代安全的分策略费用后补流程,成交修正也不承担费用后补。 -**PMS 的账本以 `trade` 为唯一入账依据**,`order_update` 只用于状态跟踪与最终核对(用终态的 `cum_qty` 校验逐笔加总是否相等,不等则告警)。这样即使某条 `order_update` 丢失也不会记错账。 +**普通成交只以 `trade` 入账**。`order_update`不能产生新成交。已成交事实的更正只经§5.5.1的独立审计流程,原`trade`保留;有效逐笔合计仍须与对应委托累计量核对。 -去重是双层的:**第一层按 `seq`**(补发时同一条消息 `seq` 不变,见 §6.1),**第二层按 `trade_no`**。任一层命中即丢弃。第二层的意义是防住 `seq` 实现出 bug 的场景,两层都命中才入账。 +普通成交去重是双层的:**第一层按 `seq`**(补发时同一条消息 `seq` 不变,见 §6.1),**第二层按 `trade_no`**。任一层命中均不重复入账,两层都未命中才处理。新序号的重复消息仍留占位记录,保证连续确认不产生缺口。 + +### 5.5.1 `trade_correction` —— 成交修正(工作台) + +交易员纠正已推送成交的价量时,不改变`trade_no`。QMT发送一条修正,并发送配套的`order_update`。 + +```json +{ + "correction_id": "AMD-20261104-a1b2c3d4e5f6", + "instruction_id": "INS-...", "broker_order_id": "DESK-...", + "trade_no": "T-DESK-abc123", + "ts_code": "600000.SH", "side": "buy", + "before": { "qty": 100, "price": 10.50, "amount": 1050.00 }, + "after": { "qty": 100, "price": 10.48, "amount": 1048.00 }, + "reason": "desk_trader_fix", + "correct_time": 1769500200000 +} +``` + +配套`order_update`包含`amended: true`、`reason: "trade_correction"`和相同`correction_id`。原信封、签名、序号与连续确认规则不变。两种消息均先持久化,PMS在worker中配对核验,收到顺序不构成业务依赖。 + +`correction_id`是幂等键。同编号同内容只处理一次;同编号异内容保留冲突,不能覆盖先前事实。原成交、修正前后及每次处理结果均留存。再次修正同一成交,`before`必须对应前次已应用的有效版本,而非最初原文。 + +累计成交可能减少,但QMT保证不超过原指令数量。PMS还会核验子委托和父指令总量。终态保留,已终结的委托剩余量保持零。数量修正不自动授权补差或重新执行原父指令;相关执行须待复核。普通`order_update`不能绕过修正链改变已审计的终态累计事实。 + +修正只能改变已核实的成交本金、数量及相关账本派生值。费用另按真实明细处理,缺少费用仍表示未知;不能用含费持仓成本覆盖成交价格,也不能由价差推算手续费。完整可核验的处理边界、事务及发布状态见[成交修正接入说明](docs/QMT成交修正接入与发布_2026-11-05.md)。 ### 5.6 `persist_result` —— 落库结果 @@ -420,7 +445,7 @@ QMT 侧为每条上行消息分配单调递增 `seq` 并**持久化到 Redis** └──────────┘ ``` -**终态**:`FILLED` / `CANCELLED` / `EXPIRED` / `REJECTED`。终态唯一、不可再变、必须推送一次 `order_update{final:true}`。 +**终态**:`FILLED` / `CANCELLED` / `EXPIRED` / `REJECTED`。普通执行状态不可从终态回退,必须推送`order_update{final:true}`。之后纠正已成交事实时使用§5.5.1,不恢复委托执行。 `CANCELLED` 与 `EXPIRED` 均可能带部分成交(`cum_qty > 0`),这是正常情况,PMS 会如实入账。 @@ -547,3 +572,4 @@ QMT 侧如有本表未覆盖的拒绝场景,请在定稿时补充,**不要 | **V1.0** | 2026-07-28 | **定稿**。回填 R1/R2:旧通道**即刻停用**、端点定为 `ws://192.168.16.98:8080`;补 §10.1.1 部署前检查清单(白名单、公钥交换、Redis 持久化、密钥只走 `.env`);调整 §9 灰度策略(旧通道已停,无从并行,改为小仓位实跑) | | V1.0.1 | 2026-07-30 | 依 07-30 联调实测澄清两处,**不改消息集与状态机**:①§7.3 `BAD_PARAM` 的含义由「价格精度超限」扩为「价格不合法(精度超限或限价越界)」,并补一段与 `LIMIT_UP`/`LIMIT_DOWN` 的分界说明——原表对「限价超出当日涨跌停区间」没有落点,实测该单既未被拒也未成交,一路挂到 `EXPIRED` ②本表顺序修正:V1.0 那行原排在 V0.9.2 之上,导致对端读到「定稿」即止、漏掉 V0.9.2 引入的 `ack_seq`(§4.5),进而形成 ack→reject 自激循环。**顺序修正不涉及任何条款变更** | | V1.0.2 | 2026-11-02 | 下单业务体新增`strategy_type`,按父指令账本区分主观量化与微盘轮动;其余下单字段、撤单、信封与握手版本、签名算法及状态机不变。新增字段参与完整payload签名,历史测试向量不变 | +| V1.0.3 | 2026-11-05 | 接入工作台`trade_correction`及带修正标记的配套委托回报,按修正编号幂等并保留原成交,终态不恢复执行;信封版本与签名算法不变 | diff --git a/README.md b/README.md index 15b7063..725dc7f 100644 --- a/README.md +++ b/README.md @@ -10,6 +10,8 @@ 当前工作从[AGENTS.md](AGENTS.md)和[开发入口](docs/开发入口.md)开始。本文保留系统说明及历史初始化、联调步骤,涉及旧参数、影子模式、测试委托和回退的内容须结合最新接续与会话授权判断。 +2026年11月5日成交修正接入:QMT的 `trade_correction` 与配套修正委托回报成对核验,按原策略归属调整本金、持仓及批次,同时保留原成交和审计。数量修正不恢复自动续单;缺少完整证据时挂起核实,费用未知仍保留未知。实现、发布和自然判收分别见[成交修正接续](docs/QMT成交修正接入与发布_2026-11-05.md)。 + 2026年11月4日开发补充:信号页独立保留有效人工关注,并提供已有信号、买方评析和每日、盘中观察分析入口。真实评析完成后,独立分钟任务发布研究名单;研究覆盖不从交易筛选结果提取。用户确认沿用公司质地排序,吸筹作为参考,建仓继续由基本面、技术面、择时三源合议及原研判、风险和资金约束决定。实现与边界见[修复规格](docs/人工关注分析链修复规格_2026-11-04.md),字段见[上游接口契约](UPSTREAM_PLAN_API.md)。开发验证通过不等于已发布或自然分析已经完成。 | 文件 | 内容 | diff --git a/app/core/recon.py b/app/core/recon.py index 7aee277..78e3600 100644 --- a/app/core/recon.py +++ b/app/core/recon.py @@ -223,7 +223,8 @@ def map_trades_to_book(trades: list, parent_actions=None) -> dict: "lot_type": (ACTION_TO_LOT.get(str(action or "").upper(), "BASE") if side == "buy" else None), "instruction_id": parent if action else None, - "order_id": pl.get("trade_no"), "seq": seq, "alerts": []}) + "order_id": pl.get("trade_no"), "seq": seq, "alerts": [], + "source_payload": dict(pl), "child_instruction_id": iid}) fee = pl.get("fee") if fee is not None and float(fee) != 0: diff --git a/app/core/trade_correction.py b/app/core/trade_correction.py new file mode 100644 index 0000000..44999d5 --- /dev/null +++ b/app/core/trade_correction.py @@ -0,0 +1,330 @@ +"""成交更正的纯投影器。原账可完整重演时才返回修改计划,不访问数据库或发出动作。""" +from copy import deepcopy +from datetime import date, datetime, timezone, timedelta +from decimal import Decimal, InvalidOperation, ROUND_HALF_UP +import re + +from app.core.cushion import sell_allocation +from app.core.recon import parent_instruction_id + +TZ = timezone(timedelta(hours=8)) +PRICE_UNIT, MONEY_UNIT = Decimal('0.001'), Decimal('0.01') +LOT_TYPES = {'BASE', 'FILL', 'ADD', 'DCA', 'T0'} +LOT_FIELDS = ('qty', 'closed_qty', 'open_price', 'close_avg_price', 'realized_pnl', 'status') +MAX_QTY = 2147483647 + + +def _need(value, reason): + if not value: + raise ValueError(reason) + + +def _decimal(value, label): + _need(value is not None and not isinstance(value, bool), label+'缺失或格式错误') + try: + number = Decimal(str(value)) + except (InvalidOperation, TypeError, ValueError): + raise ValueError(label+'不是有效数字') from None + _need(number.is_finite(), label+'必须为有限数字') + return number + + +def _quantize(value, unit): + try: + return value.quantize(unit, rounding=ROUND_HALF_UP) + except InvalidOperation: + raise ValueError('成交数字超出可核验精度') from None + + +def _qty(value, label='成交数量'): + number = _decimal(value, label) + _need(0 <= number <= MAX_QTY and number == number.to_integral_value(), label+'须为非负整数') + return int(number) + + +def _identity(value, label): + number = _decimal(value, label) + _need(0 < number < 2 ** 63 and number == number.to_integral_value(), label+'须为正整数') + return int(number) + + +def _price(value): + number = _decimal(value, '成交价格') + _need(0 <= number < Decimal('10000000') and number == _quantize(number, PRICE_UNIT), + '成交价格须非负且不超过三位小数') + return number + + +def _money(value): + number = _quantize(value, MONEY_UNIT) + _need(abs(number) < Decimal('1000000000000'), '成交金额超出账本范围') + return number + + +def _economic(payload): + _need(isinstance(payload, dict), '成交字段须为对象') + qty, price = _qty(payload.get('qty')), _price(payload.get('price')) + amount = _decimal(payload.get('amount'), '成交金额') + _need(amount >= 0 and amount == _money(amount), '成交金额须非负且不超过两位小数') + _need(qty == 0 or price > 0, '非零成交数量必须有正价格') + _need(amount == _money(price * qty), '成交数量、价格与金额不一致') + return {'qty': qty, 'price': price, 'amount': amount} + + +def _day(value, label): + try: + if isinstance(value, datetime): + return _stamp(value).date() + if isinstance(value, date): + return value + return date.fromisoformat(str(value)) + except (TypeError, ValueError): + raise ValueError(label+'缺失或格式错误') from None + + +def _stamp(value): + try: + stamp = value if isinstance(value, datetime) else datetime.fromisoformat(str(value).replace('Z', '+00:00')) + return stamp.replace(tzinfo=TZ) if stamp.tzinfo is None else stamp.astimezone(TZ) + except (TypeError, ValueError): + raise ValueError('入账时间缺失或格式错误') from None + + +def _trade_day(row): + raw = row['original_payload'].get('trade_time') + if raw is None: + # 只有服务层显式提供的真实交易日可替代原时间,不拿接收日猜交易日。 + return _day(row.get('trade_day'), '真实交易日期') + try: + value = _decimal(raw, '成交时间') + _need(value > 0 and value == value.to_integral_value(), '成交时间须为正毫秒整数') + return datetime.fromtimestamp(int(value) / 1000, TZ).date() + except (OverflowError, OSError, ValueError): + raise ValueError('真实成交时间不可核验') from None + + +def _code(value): + value = str(value or '').strip().upper() + _need(re.fullmatch(r'(?:60\d{4}\.SH|68\d{4}\.SH|00\d{4}\.SZ|30\d{4}\.SZ)', value), + '更正范围须为沪深股票') + return value + + +def _parent(payload): + value = payload.get('parent_id') or parent_instruction_id(payload.get('instruction_id')) + _need(isinstance(value, str) and value, '成交缺少父指令归属') + return value + + +def _fields(lot): + qty, closed = _qty(lot.get('qty'), '批次剩余数量'), _qty(lot.get('closed_qty'), '批次已平数量') + close = lot.get('close_avg_price') + close = _price(close) if close is not None else None + pnl = _decimal(lot.get('realized_pnl'), '批次已实现盈亏') + _need(pnl == _money(pnl), '批次盈亏金额精度不可核验') + status = str(lot.get('status') or '') + _need(status in ('OPEN', 'CLOSED'), '批次状态不可核验') + return {'qty': qty, 'closed_qty': closed, 'open_price': _price(lot.get('open_price')), + 'close_avg_price': close, 'realized_pnl': pnl, 'status': status} + + +def _export(fields): + out = dict(fields) + for key in ('open_price', 'close_avg_price'): + if out.get(key) is not None: + out[key] = format(_quantize(out[key], PRICE_UNIT), 'f') + if 'realized_pnl' in out: + out['realized_pnl'] = format(_money(out['realized_pnl']), 'f') + return out + + +def _export_trade(value): + return {'qty': value['qty'], 'price': format(_quantize(value['price'], PRICE_UNIT), 'f'), + 'amount': format(_money(value['amount']), 'f')} + + +def _prepare(lots, trades, lot_links): + _need(isinstance(lots, list) and lots and isinstance(trades, list) and trades, + '缺少完整批次或逐笔成交') + codes, books, by_lot = set(), set(), {} + for lot in lots: + _need(isinstance(lot, dict), '批次格式错误') + lid = _identity(lot.get('id'), '批次编号') + _need(lid > 0 and lid not in by_lot, '批次编号缺失或重复') + _need(lot.get('lot_type') in LOT_TYPES, '存在RECON或未知批次,不能自动更正') + _need(lot.get('instruction_id'), '存在外部批次,不能自动更正') + _need(not lot.get('archived_at'), '已归档批次不能自动更正') + _fields(lot) + _day(lot.get('open_date'), '批次开仓日期') + codes.add(_code(lot.get('ts_code'))) + if 'book_id' in lot: + _need(lot['book_id'] in ('original', 'micro30'), '批次策略归属不可核验') + books.add(lot['book_id']) + by_lot[lid] = lot + rows, seqs, numbers = [], set(), set() + for source in trades: + _need(isinstance(source, dict), '逐笔记录格式错误') + row = deepcopy(source) + payload = row.get('payload') + original = row.get('original_payload', payload) + _need(isinstance(payload, dict) and isinstance(original, dict), '缺少原始或有效成交') + _economic(payload) + _need(_economic(original)['qty'] > 0, '原始成交数量必须为正数') + seq = _identity(row.get('seq'), '入账序号') + no = payload.get('trade_no') + _need(seq > 0 and seq not in seqs and isinstance(no, str) and no and no not in numbers, + '成交序号或成交号缺失、重复') + _need(payload.get('side') in ('buy', 'sell'), '成交方向不可核验') + _need(all(payload.get(k) == original.get(k) for k in + ('trade_no', 'ts_code', 'side', 'instruction_id', 'broker_order_id')) + and _parent(payload) == _parent(original), '有效成交与原成交身份冲突') + _need(not row.get('archived_at') and not payload.get('archived_at'), '已归档成交不能自动更正') + codes.add(_code(payload.get('ts_code'))) + for owner in (row, payload, original): + if 'book_id' in owner: + _need(owner['book_id'] in ('original', 'micro30'), '成交策略归属不可核验') + books.add(owner['book_id']) + seqs.add(seq); numbers.add(no) + row.update(seq=seq, payload=payload, original_payload=original) + rows.append(row) + _need(len(codes) == 1, '更正输入混有不同证券') + _need(len(books) <= 1 and books <= {'original', 'micro30'}, '更正输入混有不同或未知策略账本') + links = dict(lot_links or {}) + buy_numbers = {r['payload']['trade_no'] for r in rows if r['payload']['side'] == 'buy'} + _need(set(links) <= buy_numbers, '批次映射包含不属于本次证据的成交') + used, mapped = set(), {} + for row in rows: + current, original = row['payload'], row['original_payload'] + if current['side'] != 'buy': + continue + no = current['trade_no'] + if no in links: + lid = _identity(links[no], '映射批次编号') + candidates = [by_lot[lid]] if lid in by_lot else [] + else: + value = _economic(original) + stamp = _stamp(row.get('processed_at')) + candidates = [l for l in lots if l['instruction_id'] == _parent(original) + and _qty(l['qty']) + _qty(l['closed_qty']) == value['qty'] + and _price(l['open_price']) == value['price'] + and _stamp(l.get('created_at')) == stamp] + _need(len(candidates) == 1, '原始买入成交不能唯一映射批次') + lot = candidates[0] + lid = int(lot['id']) + _need(lid not in used and lot['instruction_id'] == _parent(current), '成交批次重复使用或父指令不符') + used.add(lid); mapped[no] = lid + _need(used == set(by_lot), '批次缺少完整逐笔来源,不能假设零基线') + return sorted(rows, key=lambda r: r['seq']), by_lot, mapped + + +def _project(rows, by_lot, mapped): + state = {lid: {'qty': 0, 'closed_qty': 0, 'open_price': Decimal(0), + 'close_avg_price': None, 'realized_pnl': Decimal(0), 'status': 'CLOSED'} for lid in by_lot} + allocations, t_profit = [], Decimal(0) + for row in rows: + p = row['payload']; value = _economic(p) + qty, price = value['qty'], value['price'] + if p['side'] == 'buy': + state[mapped[p['trade_no']]].update(qty=qty, open_price=price, status='OPEN' if qty else 'CLOSED') + _need(sum(l['qty'] for l in state.values()) <= MAX_QTY, '逐笔持仓数量超出账本范围') + continue + candidates = [{'lot_id': lid, 'lot_type': by_lot[lid]['lot_type'], 'qty': l['qty'], + 'open_date': int(_day(by_lot[lid]['open_date'], '批次开仓日期').strftime('%Y%m%d'))} + for lid, l in sorted(state.items(), key=lambda pair: (str(by_lot[pair[0]]['open_date']), pair[0])) + if l['qty'] > 0] + trade_t_profit = Decimal(0) + for allocation in sell_allocation(candidates, qty): + lid, take = allocation['lot_id'], allocation['qty'] + lot = state[lid] + old_pnl = lot['realized_pnl'] + if by_lot[lid]['lot_type'] == 'T0': + trade_t_profit += (price - lot['open_price']) * take + closed = lot['closed_qty'] + take + # MySQL逐笔UPDATE会把均价写入三位列、盈亏写入两位列,下一笔使用这些已落库值。 + lot['close_avg_price'] = _quantize(((lot['close_avg_price'] or Decimal(0)) * lot['closed_qty'] + + price * take) / closed, PRICE_UNIT) + lot['realized_pnl'] = _money(old_pnl + (price - lot['open_price']) * take) + lot.update(qty=lot['qty'] - take, closed_qty=closed) + lot['status'] = 'OPEN' if lot['qty'] else 'CLOSED' + allocations.append({'seq': row['seq'], 'trade_no': p['trade_no'], 'lot_id': lid, 'qty': take, + 'price': format(_quantize(price, PRICE_UNIT), 'f'), + 'realized_pnl_delta': format(lot['realized_pnl'] - old_pnl, '.2f')}) + # 持仓T收益是在一笔卖出全部T批核销后合计写入两位列,与逐批盈亏分别舍入不同。 + t_profit = _money(t_profit + trade_t_profit) + _need(sum(l['qty'] for l in state.values()) <= MAX_QTY, '更正后持仓数量超出账本范围') + return state, allocations, t_profit + + +def _round_boundaries(rows): + """记录清零后再次买入的真实仓次边界;零量更正也保留原买入身份。""" + balance, bought = 0, False + boundaries, before = [], {} + for row in rows: + p, seq = row['payload'], row['seq'] + qty = _qty(p['qty']) + before[seq] = balance + if p['side'] == 'buy': + if bought and balance == 0 and qty > 0: + boundaries.append(seq) + bought = True + balance += qty + else: + balance -= qty + return boundaries, before + + +def compile_correction(*, lots, trades, target_trade_no, after, today, lot_links=None): + """证明当前有效逐笔与批次一致,再替换一笔并重演;费用与策略资金由服务层另行核验。 + + trades中的payload为已应用既有更正后的有效成交,original_payload保留原始trade。 + lot_links为{trade_no: lot_id};没有明确映射的旧数据须按原父指令、数量、价格、时间唯一配对。 + 原始trade_time为毫秒;缺失时只接受服务层显式提供的真实trade_day,不用接收时间猜测。 + """ + rows, by_lot, mapped = _prepare(lots, trades, lot_links) + current, allocations_before, t_before = _project(rows, by_lot, mapped) + for lid, lot in by_lot.items(): + _need(_fields(lot) == current[lid], '当前批次与有效逐笔重演不一致,拒绝更正') + targets = [r for r in rows if r['payload']['trade_no'] == target_trade_no] + _need(len(targets) == 1, '目标成交缺失或不唯一') + target = targets[0] + boundaries, _ = _round_boundaries(rows) + _need(not any(target['seq'] < boundary for boundary in boundaries), + '目标成交之后已清零并建立新仓,旧轮次更正须人工核对,不能回灌当前仓次') + _need(isinstance(after, dict) and set(after) == {'qty', 'price', 'amount'}, + '更正仅接受数量、价格和金额,不能修改成交身份或费用') + old, new = _economic(target['payload']), _economic(after) + current_day, actual_day = _day(today, '核验日期'), _trade_day(target) + _need(actual_day <= current_day, '成交日期在未来,拒绝更正') + quantity_delta = new['qty'] - old['qty'] + if quantity_delta: + book_day = (_day(by_lot[mapped[target_trade_no]]['open_date'], '批次开仓日期') + if target['payload']['side'] == 'buy' else + _day(target['book_day'], '原入账日期') if target.get('book_day') else + _stamp(target.get('processed_at')).date()) + _need(book_day == actual_day, '原成交跨日入账,数量更正的T+1基线待核实') + target['payload'].update(after) + projected, allocations, t_after = _project(rows, by_lot, mapped) + _, revised_before = _round_boundaries(rows) + _need(all(revised_before[boundary] == 0 for boundary in boundaries), + '更正会使旧清零边界带库存进入新仓,须人工核对') + changed, exported = [], [] + for lid in sorted(by_lot): + old_fields, new_fields = _export(current[lid]), _export(projected[lid]) + exported.append(dict(deepcopy(by_lot[lid]), **new_fields)) + if old_fields != new_fields: + changed.append({'id': lid, 'before': old_fields, 'after': new_fields}) + def net(state): + return sum((l['qty'] * l['open_price'] - l['realized_pnl'] for l in state.values()), Decimal(0)) + side = target['payload']['side'] + avail_delta = -quantity_delta if side == 'sell' else quantity_delta if actual_day < current_day else 0 + cash_delta = (new['amount'] - old['amount']) * (1 if side == 'sell' else -1) + return {'lots': exported, 'changes': changed, 'lot_links': mapped, + 'allocations': allocations, 'allocations_before': allocations_before, + 'qty_before': sum(l['qty'] for l in current.values()), + 'qty_after': sum(l['qty'] for l in projected.values()), + 'amount_before': format(old['amount'], '.2f'), 'amount_after': format(new['amount'], '.2f'), + 'cash_delta': format(cash_delta, '.2f'), 'avail_delta': avail_delta, + 't_profit_delta': format(t_after - t_before, '.2f'), + 't_profit_before': format(t_before, '.2f'), 't_profit_after': format(t_after, '.2f'), + 'net_cost_before': format(net(current), 'f'), 'net_cost_after': format(net(projected), 'f'), + 'trade_before': _export_trade(old), 'trade_after': _export_trade(new)} diff --git a/app/core/ws_codec.py b/app/core/ws_codec.py index a0e4cf9..ae87ccf 100644 --- a/app/core/ws_codec.py +++ b/app/core/ws_codec.py @@ -60,14 +60,14 @@ DOWNSTREAM_TYPES = (T_HELLO, T_PLACE, T_CANCEL, T_Q_POS, T_Q_FUNDS, T_Q_ORDERS, T_HELLO_ACK, T_ACK, T_REJECT = "hello_ack", "ack", "reject" T_ORDER_UPDATE, T_TRADE, T_PERSIST = "order_update", "trade", "persist_result" +T_TRADE_CORRECTION = "trade_correction" T_SNAPSHOT, T_POS_UPDATE, T_FUNDS_UPDATE, T_PONG = ("snapshot", "position_update", "funds_update", "pong") -UPSTREAM_TYPES = (T_HELLO_ACK, T_ACK, T_REJECT, T_ORDER_UPDATE, T_TRADE, T_PERSIST, +UPSTREAM_TYPES = (T_HELLO_ACK, T_ACK, T_REJECT, T_ORDER_UPDATE, T_TRADE, T_TRADE_CORRECTION, T_PERSIST, T_SNAPSHOT, T_POS_UPDATE, T_FUNDS_UPDATE, T_PONG) -# 只有 trade 需要 worker 入账 (§5.5「账本以 trade 为唯一入账依据」), 其余上行消息 -# 由常驻进程自己消化完就落库存档。这个集合是 inbox 里 processed 初值的判据。 -NEEDS_LEDGER = (T_TRADE,) +# 成交与成交修正由worker入账。修正型order_update还须按payload分类,见needs_ledger。 +NEEDS_LEDGER = (T_TRADE, T_TRADE_CORRECTION) # 指令状态 (§7.1)。终态唯一、不可再变。 ST_ACCEPTED, ST_SUBMITTED, ST_PARTIAL = "ACCEPTED", "SUBMITTED", "PARTIAL" @@ -419,11 +419,29 @@ def ack_seq_payload(seq: int) -> dict: # ================================================================ 上行 payload 读取 +def is_correction(type_: str, payload: dict) -> bool: + """单独 amended=false 是普通回报;修正编号或其它修正标记必须核验。""" + payload = payload or {} + return type_ == T_TRADE_CORRECTION or (type_ == T_ORDER_UPDATE and + (('amended' in payload and payload['amended'] is not False) or + 'correction_id' in payload or payload.get('reason') == T_TRADE_CORRECTION)) + + +def needs_ledger(type_: str, payload: dict) -> bool: + return type_ in NEEDS_LEDGER or is_correction(type_, payload) + + def dedup_key(type_: str, payload: dict): - """第二层去重键 (§5.5)。目前只有 trade 有 —— trade_no 唯一, 命中即丢弃。 + """第二层业务键:原成交按trade_no,修正两类证据各按correction_id。 第一层是 seq (补发时同一条消息 seq 不变)。两层都设是为了防住 seq 实现出 bug 的场景。 """ + if is_correction(type_, payload): + cid = (payload or {}).get('correction_id') + if not isinstance(cid, str) or not cid.strip(): + return None + prefix = 'correction:' if type_ == T_TRADE_CORRECTION else 'amendment:' + return prefix + hashlib.sha256(cid.encode('utf-8')).hexdigest() if type_ == T_TRADE: tn = (payload or {}).get("trade_no") return f"trade:{tn}" if tn else None diff --git a/app/repo/execution_repo.py b/app/repo/execution_repo.py index 5e00956..f9ed63a 100644 --- a/app/repo/execution_repo.py +++ b/app/repo/execution_repo.py @@ -20,6 +20,44 @@ def pending_trades(child_ids): key = 'c'+str(index) args[key] = iid keys.append(':'+key) - row = db.fetch_one("SELECT COUNT(*) AS n FROM pms_qmt_inbox WHERE msg_type='trade' " - "AND processed IN (:p,:orphan) AND corr_id IN ("+','.join(keys)+')', args) + ids = ','.join(keys) + row = db.fetch_one("SELECT COUNT(*) AS n FROM pms_qmt_inbox WHERE ((msg_type='trade' " + "AND processed IN (:p,:orphan)) OR ("+_correction_sql()+" AND "+_pending_correction_sql()+")) AND (corr_id IN ("+ids+ + ") OR JSON_UNQUOTE(JSON_EXTRACT(payload_json,'$.instruction_id')) IN ("+ids+'))', args) + return int((row or {}).get('n') or 0) + + +def _correction_sql(): + return ("(msg_type='trade_correction' OR (msg_type='order_update' AND " + "((JSON_CONTAINS_PATH(payload_json,'one','$.amended') AND " + "NOT COALESCE(JSON_CONTAINS(payload_json,'false','$.amended'),0)) OR " + "JSON_CONTAINS_PATH(payload_json,'one','$.correction_id') OR " + "JSON_UNQUOTE(JSON_EXTRACT(payload_json,'$.reason'))='trade_correction')))") + + +def _pending_correction_sql(): + # 升级前未知类型或amended被通道存成2,不等于修正已经入账;重复占位的note不同。 + return ("(processed IN (0,3) OR (processed=2 AND process_note IN " + "('trade_correction 由通道进程消化, 不入账','order_update 由通道进程消化, 不入账')))") + + +def pending_corrections(ts_code=None, child_ids=None): + """账户买入读全部未核实修正;按证券或子单查询供卖出及已有父指令守卫。""" + args = {} + where = [_correction_sql(), _pending_correction_sql()] + identity = [] + if ts_code: + args['code'] = ts_code + identity.append("JSON_UNQUOTE(JSON_EXTRACT(payload_json,'$.ts_code'))=:code") + if child_ids: + keys = [] + for index, iid in enumerate(child_ids): + key = 'c'+str(index) + args[key] = iid + keys.append(':'+key) + ids = ','.join(keys) + identity.append("(corr_id IN ("+ids+") OR JSON_UNQUOTE(JSON_EXTRACT(payload_json,'$.instruction_id')) IN ("+ids+'))') + if identity: + where.append('('+' OR '.join(identity)+')') + row = db.fetch_one('SELECT COUNT(*) AS n FROM pms_qmt_inbox WHERE '+' AND '.join(where), args) return int((row or {}).get('n') or 0) diff --git a/app/repo/pms_repo.py b/app/repo/pms_repo.py index 9049e80..26a6495 100644 --- a/app/repo/pms_repo.py +++ b/app/repo/pms_repo.py @@ -473,7 +473,7 @@ def update_instruction(instruction_id: str, *, status=None, exec_qty=None, exec_ # 独立刷新任务仅通过 execution_repo 写缓存;旧的整份进度不能覆盖新依据。 # 同理,异步撤单/撤换标记不能被不知情的执行回写抹掉。显式传键可更新标记。 body = ':pg' - for key in ('cancel_requested', 'replacement_requested'): + for key in ('cancel_requested', 'replacement_requested', 'trade_correction_hold'): if key not in progress: body = ("CASE WHEN JSON_CONTAINS_PATH(COALESCE(progress_json,JSON_OBJECT())," f"'one','$.{key}') THEN JSON_SET(JSON_EXTRACT({body},'$')," @@ -1025,19 +1025,46 @@ def insert_strategy(*, strategy_id, ts_code, stype, autonomy="auto", params=None def update_strategy(strategy_id: str, **fields) -> int: - """更新策略。params/state 传 dict 会自动 json 化到 *_json 列。""" + """更新策略;state 按成交更正版本比较后写入,旧运行任务不得覆盖新状态。""" upd = dict(fields) + unset = object() + expected = upd.pop("correction_expected", unset) if "params" in upd: upd["params_json"] = _dumps(upd.pop("params")) if "state" in upd: upd["state_json"] = _dumps(upd.pop("state")) + state_write = "state_json" in upd + if state_write: + state = _loads(upd["state_json"]) + if not isinstance(state, dict): + raise ValueError("策略状态必须是对象") + correction = state.get("trade_correction") or {} + if not isinstance(correction, dict): + raise ValueError("策略成交更正版本异常") + if expected is unset: + expected = correction.get("correction_id") + if expected is not None and (not isinstance(expected, str) or not expected): + raise ValueError("策略成交更正版本异常") + elif expected is not unset: + raise ValueError("成交更正版本比较必须同时保存策略状态") clause, p = _set_clause(upd, STRATEGY_COLS) if not clause: return 0 p["sid"] = strategy_id p["ts"] = _NOW() - return execute( - f"UPDATE pms_strategy SET {clause}, updated_at = :ts WHERE strategy_id = :sid", p) + where = "strategy_id = :sid" + if state_write: + epoch = "JSON_EXTRACT(state_json, '$.trade_correction.correction_id')" + if expected is None: + where += f" AND ({epoch} IS NULL OR {epoch} = JSON_EXTRACT('null', '$'))" + else: + where += f" AND {epoch} = JSON_EXTRACT(:correction_expected, '$')" + p["correction_expected"] = json.dumps(expected, ensure_ascii=False) + changed = execute( + f"UPDATE pms_strategy SET {clause}, updated_at = :ts WHERE {where}", p) + if state_write and changed != 1: + raise RuntimeError("策略成交更正版本已变化或策略不存在,拒绝覆盖旧状态") + return changed # ================================================================ 软归档 (从在办/在途视图移除) diff --git a/app/repo/qmt_repo.py b/app/repo/qmt_repo.py index 04b7a9d..54688fb 100644 --- a/app/repo/qmt_repo.py +++ b/app/repo/qmt_repo.py @@ -14,6 +14,7 @@ from __future__ import annotations import json from datetime import datetime +from app.core import ws_codec from app.db.session import execute, fetch_all, fetch_one @@ -48,10 +49,10 @@ def is_smoke(parent_id) -> bool: # pms_qmt_inbox.processed 的取值。0/1/2 是原有语义; 3 是 2026-07-30 新增的「挂起待人工」。 -# 0 待入账 只有 trade 会是这个值 (见 ws_codec 的 NEEDS_LEDGER) +# 0 待入账 trade、trade_correction及修正型order_update。 # 1 已入账 成交已落进批次账本 # 2 已消化 通道自处理完毕、无需入账 (含联调单跳过、重复 trade_no 占位行) -# 3 挂起待人工 该入账但**不敢入**: 出口表反查不到这笔成交对应的委托。 +# 3 挂起待人工 归属未知、修正身份或内容冲突,不能安全入账。 # 绝不能顺手当成 2 —— 2 的语义是「确认过不用管」, 3 是「有账没记, 等人看」。 # 混在一起, 事后翻 inbox 就分不出哪些是真丢账。 PROC_PENDING, PROC_BOOKED, PROC_DIGESTED, PROC_ORPHAN = 0, 1, 2, 3 @@ -166,10 +167,23 @@ def beat(stat=None) -> int: # ================================================================ pms_qmt_inbox PUT_NEW, PUT_DUP_SEQ, PUT_DUP_KEY = "NEW", "DUP_SEQ", "DUP_KEY" +PUT_CONFLICT = "CONFLICT" def inbox_put(*, seq: int, msg_id: str, msg_type: str, payload: dict, msg_ts: int, corr_id=None, dedup_key=None, processed: int = 2, note=None) -> str: + """修正消息的持久化与委托发送占用同一账户锁,先落库再推进确认水位。""" + from config.settings import settings + from contextlib import nullcontext + from app.repo import rotation_repo + guard = rotation_repo.account_guard() if settings.INFRA_STRATEGY_BOOK_SCHEMA and ws_codec.is_correction(msg_type, payload) else nullcontext() + with guard: + return _inbox_put(seq=seq, msg_id=msg_id, msg_type=msg_type, payload=payload, + msg_ts=msg_ts, corr_id=corr_id, dedup_key=dedup_key, processed=processed, note=note) + + +def _inbox_put(*, seq, msg_id, msg_type, payload, msg_ts, corr_id=None, dedup_key=None, + processed=2, note=None): """落一条上行消息。返回 NEW / DUP_SEQ / DUP_KEY。 双层去重 (§6.1 + §5.5): 主键 seq 是第一层, 唯一索引 dedup_key (trade_no) 是第二层。 @@ -193,11 +207,28 @@ def inbox_put(*, seq: int, msg_id: str, msg_type: str, payload: dict, msg_ts: in return PUT_DUP_SEQ verdict = PUT_NEW + correction = ws_codec.is_correction(msg_type, payload) + if correction: + # 仓储也计算业务键,避免调用方遗漏后将修正当普通存档静默消化。 + corr_id = payload.get('instruction_id') + dedup_key = ws_codec.dedup_key(msg_type, payload) + processed = PROC_PENDING + if dedup_key is None: + processed, verdict = PROC_ORPHAN, PUT_CONFLICT + note = '成交修正缺少有效 correction_id,挂起核对,不改变委托和账本' if dedup_key and fetch_one("SELECT seq FROM pms_qmt_inbox WHERE dedup_key = :k", {"k": str(dedup_key)[:80]}): verdict = PUT_DUP_KEY note = f"重复成交 (dedup_key={dedup_key}), 不入账, 仅占位以推进 seq 水位" + if correction: + old = fetch_one('SELECT msg_type, payload_json FROM pms_qmt_inbox WHERE dedup_key=:k', + {'k': dedup_key}) + if not old or old['msg_type'] != msg_type or _loads(old['payload_json'], None) != payload: + verdict = PUT_CONFLICT + note = '同一 correction_id 对应不同修正内容,挂起核对,原回执保留' dedup_key, processed = None, 2 + if verdict == PUT_CONFLICT: + processed = PROC_ORPHAN p = {"s": seq, "mi": str(msg_id)[:64], "mt": str(msg_type)[:24], "ci": (str(corr_id)[:64] if corr_id else None), @@ -213,6 +244,26 @@ def inbox_put(*, seq: int, msg_id: str, msg_type: str, payload: dict, msg_ts: in return verdict +def inbox_by_key(key): + row = fetch_one('SELECT * FROM pms_qmt_inbox WHERE dedup_key=:key', {'key': key}) + if row: + row['payload'] = _loads(row.get('payload_json'), {}) + return row + + +def inbox_for_order(iid, msg_type=None, limit=10000): + """按原始 payload 身份兼容旧 corr_id;达到上限由调用方拒绝不完整核验。""" + params = {'iid': iid, 'n': int(limit)} + condition = "(corr_id=:iid OR JSON_UNQUOTE(JSON_EXTRACT(payload_json,'$.instruction_id'))=:iid)" + if msg_type: + condition += ' AND msg_type=:type' + params['type'] = msg_type + rows = fetch_all('SELECT * FROM pms_qmt_inbox WHERE '+condition+' ORDER BY seq LIMIT :n', params) + for row in rows: + row['payload'] = _loads(row.get('payload_json'), {}) + return rows + + def inbox_recover_watermark(stored_last_seq: int, limit: int = 20000) -> int: """用 inbox 重算真正的连续水位。 @@ -231,9 +282,12 @@ def inbox_recover_watermark(stored_last_seq: int, limit: int = 20000) -> int: return cur -def inbox_pending(limit: int = 500) -> list: - rows = fetch_all("SELECT * FROM pms_qmt_inbox WHERE processed = 0 ORDER BY seq ASC " - "LIMIT :n", {"n": int(limit)}) +def inbox_pending(limit: int = 500, msg_type=None) -> list: + condition, params = 'processed = 0', {'n': int(limit)} + if msg_type: + condition += ' AND msg_type=:type' + params['type'] = msg_type + rows = fetch_all('SELECT * FROM pms_qmt_inbox WHERE '+condition+' ORDER BY seq ASC LIMIT :n', params) for r in rows: r["payload"] = _loads(r.get("payload_json"), {}) return rows diff --git a/app/repo/trade_correction_repo.py b/app/repo/trade_correction_repo.py new file mode 100644 index 0000000..c855cee --- /dev/null +++ b/app/repo/trade_correction_repo.py @@ -0,0 +1,110 @@ +"""成交修正的只读取证。单次查询只访问一张表,满上限不得冒充完整历史。""" +import json + +from app.db import session as db +from app.core import ws_codec + + +def _rows(rows, limit=None, *, strict=True): + if limit is not None and len(rows) >= limit: + raise ValueError('成交修正证据超过核验上限,需人工核对') + for row in rows: + try: + row['payload'] = json.loads(row.get('payload_json') or '{}') + if not isinstance(row['payload'], dict): + raise ValueError('成交回执不是对象') + except (TypeError, ValueError): + if strict: + raise + # 保留原文和seq,让worker只挂起坏行,不让它阻塞本批其它更正。 + row['payload'] = None + return rows + + +def pending(limit=100): + # 兼容升级前旧WS将未知类型存成2的原消息,不扫描已消化的重复占位行。 + return _rows(db.fetch_all( + "SELECT * FROM pms_qmt_inbox WHERE msg_type='trade_correction' " + "AND (processed=0 OR (processed=2 AND process_note=" + "'trade_correction 由通道进程消化, 不入账')) ORDER BY seq ASC LIMIT :n", + {'n': max(1, min(int(limit), 500))}), strict=False) + + +def get_inbox(seq, lock=False): + row = db.fetch_one('SELECT * FROM pms_qmt_inbox WHERE seq=:seq'+ + (' FOR UPDATE' if lock else ''), {'seq': int(seq)}) + return _rows([row], strict=False)[0] if row else None + + +def get_trade(trade_no): + key = ws_codec.dedup_key('trade', {'trade_no': trade_no}) + row = db.fetch_one('SELECT * FROM pms_qmt_inbox WHERE dedup_key=:key', + {'key': key[:80] if key else None}) + if row: + row = _rows([row])[0] + if row['msg_type'] != 'trade' or row['payload'].get('trade_no') != trade_no: + raise ValueError('成交编号与原回执去重键冲突') + return row + + +def _matches_correction(row, correction_id, kind): + payload = row.get('payload') + key = ws_codec.dedup_key(kind, {'correction_id': correction_id, 'amended': True}) + # 原文坏损但业务键仍指向本编号时也是冲突证据,不可默默筛掉。 + return (isinstance(payload, dict) and payload.get('correction_id') == correction_id + or bool(key) and row.get('dedup_key') == key) + + +def _correction_rows(kind, correction_id, iid): + # 同编号不同正文不能因主去重行存在而被忽略;也读取空dedup的冲突占位。 + limit = 10001 + rows = _rows(db.fetch_all( + 'SELECT * FROM pms_qmt_inbox WHERE msg_type=:kind ORDER BY seq ASC LIMIT :n', + {'kind': kind, 'n': limit}), limit, strict=False) + return [r for r in rows if _matches_correction(r, correction_id, kind)] + + +def amendments(correction_id, iid): + # 正常order_update量大,先走child corr_id索引;新类型收包强制corr_id=iid。 + limit = 2001 + rows = _rows(db.fetch_all( + "SELECT * FROM pms_qmt_inbox WHERE msg_type='order_update' AND corr_id=:iid " + 'ORDER BY seq ASC LIMIT :n', {'iid': iid, 'n': limit}), limit, strict=False) + # 冲突占位可能篡改instruction_id,不能只在正常child范围内找冲突。 + conflicts = _rows(db.fetch_all( + "SELECT * FROM pms_qmt_inbox WHERE msg_type='order_update' AND processed=3 " + 'ORDER BY seq ASC LIMIT :n', {'n': limit}), limit, strict=False) + by_seq = {r['seq']: r for r in [*rows, *conflicts] + if _matches_correction(r, correction_id, 'order_update')} + return sorted(by_seq.values(), key=lambda r: int(r['seq'])) + + +def corrections(correction_id, iid): + return _correction_rows('trade_correction', correction_id, iid) + + +def pending_for_trade(trade_no, before_seq): + limit = 10001 + rows = _rows(db.fetch_all( + "SELECT * FROM pms_qmt_inbox WHERE msg_type='trade_correction' AND seq<:seq " + "AND (processed IN (0,3) OR (processed=2 AND process_note=" + "'trade_correction 由通道进程消化, 不入账')) ORDER BY seq ASC LIMIT :n", + {'seq': int(before_seq), 'n': limit}), limit, strict=False) + return [r for r in rows if (r.get('payload') or {}).get('trade_no') == trade_no] + + +def orders_for_code(code, limit=10001): + rows = db.fetch_all('SELECT * FROM pms_qmt_order WHERE ts_code=:code ' + 'ORDER BY id ASC LIMIT :n', {'code': code, 'n': int(limit)}) + if len(rows) >= limit: + raise ValueError('该股委托历史超过核验上限') + return rows + + +def trades_for_orders(child_ids, limit=20001): + # 历史信封的corr_id可能不是子委托;以持久原文instruction_id核对,不猜所属策略。 + ids = set(child_ids) + rows = _rows(db.fetch_all("SELECT * FROM pms_qmt_inbox WHERE msg_type='trade' " + 'ORDER BY seq ASC LIMIT :n', {'n': int(limit)}), int(limit)) + return [r for r in rows if r['payload'].get('instruction_id') in ids + and r.get('dedup_key') is not None] diff --git a/app/scheduler.py b/app/scheduler.py index f184528..3d21537 100644 --- a/app/scheduler.py +++ b/app/scheduler.py @@ -370,11 +370,21 @@ def daily_settle(): pre = ledger_service.replay_fills() except Exception as e: logger.error("[daily_settle] 尾窗成交入账失败 (对账将按差异吸收): %s", e) + pre = {"ok": False, "errors": ["尾窗成交入账失败:"+type(e).__name__]} out = ledger_service.daily_settle() out.setdefault("steps", {})["pre_replay"] = { "ok": pre.get("ok"), "actions": pre.get("actions"), "ws_actions": (pre.get("ws") or {}).get("actions"), - "errors": (pre.get("errors") or [])[:3]} + "errors": (pre.get("errors") or [])[:3], + "corrections": {key: (pre.get("corrections") or {}).get(key, 0) + for key in ("applied", "waiting", "held")}, + "alerts": (pre.get("alerts") or [])[:3]} + # 业务挂起不混成程序错误;但不能把尾窗待核实的更正报成日终健康。 + if pre.get("ok") is False: + out["ok"] = False + out.setdefault("errors", []).extend(pre.get("errors") or []) + if pre.get("alerts"): + out.setdefault("alerts", []).extend(pre["alerts"]) try: out["steps"]["windows"] = executor.sweep_windows() # 窗口耗尽收口 + 方案成交回写 except Exception as e: diff --git a/app/services/daily_buy_review.py b/app/services/daily_buy_review.py index efc3c6b..5affd0c 100644 --- a/app/services/daily_buy_review.py +++ b/app/services/daily_buy_review.py @@ -32,6 +32,10 @@ def _command_blocker(ins): def blocker(ins, *, now=None): + from app.services import execution_orders + correction = execution_orders.correction_blocker(ins) if ins else '' + if correction: + return correction if not applies(ins): return None if ins.get('status') == 'WAIT_REVIEW': @@ -59,12 +63,10 @@ def _orders(ins): rows = qmt_repo.orders_for_parent(ins['instruction_id']) if len({r['instruction_id'] for r in rows}) != len(rows): raise ValueError('出口子单重复,等待核对') + from app.services import execution_orders filled = 0 for row in rows: - qty, done = int(row['qty']), int(row.get('cum_qty') or 0) - if qty <= 0 or not 0 <= done <= qty: - raise ValueError('出口数量无效,等待核对') - filled += qty if row['status'] == qmt_repo.OS_FILLED else done + filled += execution_orders.effective_filled(row) return rows, filled @@ -192,7 +194,7 @@ def _queue(ins, result, now): def process(*, now=None): - from app.services import dispatcher, proposal_service + from app.services import dispatcher, proposal_service, execution_orders now = now or datetime.now() out = {'held': [], 'approved': [], 'queued': [], 'ended': [], 'waiting': [], 'errors': []} if not param_store.get_bool('PMS_DAILY_BUY_REVIEW', False): @@ -215,6 +217,9 @@ def process(*, now=None): continue iid = row['instruction_id'] try: + if execution_orders.correction_blocker(row): + out['waiting'].append(iid) + continue orders, filled = _orders(row) if any(o['status'] not in qmt_repo.FINAL for o in orders): cancelled = dispatcher.cancel(instruction_id=iid, dispatch_ref=row.get('dispatch_ref')) @@ -288,6 +293,11 @@ def decide_confirmation(p, decision, reason, revision=None, *, now=None): now = now or datetime.now() h = p.get('hard_numbers') or {} ins = pms_repo.get_instruction(h.get('daily_parent')) + from app.services import execution_orders + if decision == 'ACCEPTED' and ins: + blocked = execution_orders.correction_blocker(ins) + if blocked: + return {'ok': False, 'error': blocked} if decision == 'ACCEPTED' and p['status'] == 'ACCEPTED': return {'ok': True, 'already_accepted': True, 'instruction_id': h['daily_parent']} if (not ins or ins['status'] != 'WAIT_REVIEW' or p['status'] != 'WAIT_USER' diff --git a/app/services/dispatcher.py b/app/services/dispatcher.py index 521d6d2..851f490 100644 --- a/app/services/dispatcher.py +++ b/app/services/dispatcher.py @@ -145,6 +145,9 @@ def dispatch(*, instruction_id: str, ts_code: str, side: str, qty: int, limit_pr from app.services import execution_orders, execution_basis pid = _parent_of(instruction_id, parent_id) parent = pms_repo.get_instruction(pid) + correction = execution_orders.correction_blocker(parent, ts_code=ts_code, side=str(side).lower()) + if correction: + return {'ok': False, 'ref': None, 'mode': m, 'note': '', 'error': correction} if parent and (parent.get('progress') or {}).get('cancel_requested'): return {'ok': False, 'ref': None, 'mode': m, 'note': '', 'error': '父指令撤销处理中,禁止继续下发'} if settings.INFRA_STRATEGY_BOOK_SCHEMA and str(side).lower() == 'buy': @@ -171,6 +174,10 @@ def dispatch(*, instruction_id: str, ts_code: str, side: str, qty: int, limit_pr return {'ok': False, 'ref': None, 'mode': m, 'note': '', 'error': blocked} if m == MODE_WS: with execution_orders.account_guard(): + parent = pms_repo.get_instruction(pid) + blocked = execution_orders.correction_blocker(parent, ts_code=ts_code, side=str(side).lower()) + if blocked: + raise ValueError(blocked) if execution_orders.enabled() and not qmt_repo.get_order(instruction_id): from app.core.execution_policy import supported_code if not supported_code(ts_code): diff --git a/app/services/execution_conflicts.py b/app/services/execution_conflicts.py index 48f67c5..ceba2be 100644 --- a/app/services/execution_conflicts.py +++ b/app/services/execution_conflicts.py @@ -18,6 +18,14 @@ def buy_blocker(ts_code): def queued_buy_blocker(row): """发出前再次检查。异常由通道保留队列等待,不按无冲突继续。""" from config.settings import settings + from app.services import execution_orders + from app.core import strategy_book as books + with books.scope('all'): + correction_parent = pms_repo.get_instruction(row.get('parent_id')) + correction = execution_orders.correction_blocker(correction_parent, + ts_code=row.get('ts_code'), side=row.get('side')) + if correction: + return correction if settings.INFRA_STRATEGY_BOOK_SCHEMA: from app.core import strategy_book as books from app.services import param_store, rotation_service diff --git a/app/services/execution_orders.py b/app/services/execution_orders.py index ca67dbe..3574256 100644 --- a/app/services/execution_orders.py +++ b/app/services/execution_orders.py @@ -1,5 +1,7 @@ """共用报价及撤换。撤单请求不释放数量,实际终态及成交入账后才可重报。""" from contextlib import nullcontext +import hashlib +from datetime import datetime from app.core import execution_policy as policy, strategy_book as books from app.repo import pms_repo, qmt_repo, execution_repo from app.services import param_store @@ -17,6 +19,99 @@ def account_guard(): return nullcontext() +def _receipt_time(value): + try: + parsed = value if isinstance(value, datetime) else datetime.fromisoformat(str(value).replace('Z', '+00:00')) + return policy.local_now(parsed) # 本端数据库 DATETIME 按已核实的北京时间解释。 + except (ValueError, TypeError): + return None + + +def order_correction(order, *, require_current=False): + """只接受修正事务已保存且与出口累计数一致的审计投影。""" + if not settings.INFRA_STRATEGY_BOOK_SCHEMA: + return None + from app.repo import rotation_repo + iid = order['instruction_id'] + value, _ = rotation_repo.get('order-correction:'+hashlib.sha256(iid.encode()).hexdigest()) + if value is None: + return None + if (not isinstance(value, dict) or value.get('instruction_id') != iid or + not 0 <= int(value.get('cum_qty', -1)) <= int(order['qty']) or + (require_current and int(value['cum_qty']) != int(order.get('cum_qty') or 0)) or + int(value.get('correction_seq') or 0) <= 0 or _receipt_time(value.get('received_at')) is None): + raise ValueError('委托成交修正审计与当前累计量不一致,等待核对') + return value + + +def effective_filled(order): + qty, done = int(order['qty']), int(order.get('cum_qty') or 0) + if qty <= 0 or not 0 <= done <= qty: + raise ValueError('子单累计成交数量异常') + if order.get('status') == 'FILLED' and done != qty and not order_correction(order, require_current=True): + raise ValueError('全成子单累计成交不完整,且没有已核实的成交修正') + return done + + +def settlement_cutoff(order): + """释放资金必须使用修正消息之后的快照,不能沿用早于修正的原终态时间。""" + final = _receipt_time(order.get('final_at')) + if final is None: + return None + audit = order_correction(order) + if audit: + return max(final, _receipt_time(audit['received_at'])) + return final + + +def correction_blocker(parent=None, *, ts_code=None, side=None): + parent = parent or {} + if (parent.get('progress') or {}).get('trade_correction_hold'): + return '父指令成交数量已修正,停止自动续单和改价;后续须另行明确安排' + progress = parent.get('progress') or {} + if parent.get('origin_type') == 'strategy' and parent.get('origin_id'): + with books.scope('all'): + strategy = pms_repo.get_strategy(parent['origin_id']) + if progress.get('stype') == 'GRID' or (strategy or {}).get('type') == 'GRID': + if not strategy or strategy.get('status') != 'ACTIVE' or strategy.get('archived_at'): + return '来源网格策略已停止或不可核实,旧指令不再下发' + state = strategy.get('state') or {} + if state.get('correction_hold'): + return '来源网格正在核实成交修正,停止旧指令下发' + epoch = (state.get('trade_correction') or {}).get('correction_id') + if epoch != progress.get('strategy_correction_id'): + return '来源网格的成交已修正,旧版本指令不能继续下发' + side = side or parent.get('side') + code = ts_code or parent.get('ts_code') + if not execution_repo.pending_corrections(): + return '' + if side == 'buy': + return '账户存在待核实成交修正,暂停新增买入和释放资金占用' + else: + if not code: + return '成交修正涉及证券无法核实,暂停新增委托' + # 修正原文证券字段缺失或冲突时,仍按本端出口身份拦住该股的新父指令。 + children = qmt_repo.list_orders(ts_code=code, limit=10001) + if len(children) >= 10001: + return '该股出口历史超过核验上限,等待成交修正核对' + if execution_repo.pending_corrections(ts_code=code, child_ids=[r['instruction_id'] for r in children]): + return '该证券存在待核实成交修正,暂停新增委托和改价' + return '' + + +def prepare_send(instruction_id): + """与修正收件串行检查,成功标记之后才允许WS实际发送。""" + with account_guard(), books.scope('all'): + order = qmt_repo.get_order(instruction_id) + if not order: + return {'marked': False, 'blocked': '委托不存在'} + parent = pms_repo.get_instruction(order['parent_id']) + blocked = correction_blocker(parent, ts_code=order['ts_code'], side=order['side']) + if blocked: + return {'marked': False, 'blocked': blocked} + return {'marked': qmt_repo.mark_send_attempt(instruction_id), 'blocked': ''} + + def quote_order(side, quote, now=None, user_limit=None, anchor_price=None): return policy.price_order(side=side, quote=quote, now=now, user_limit=user_limit, anchor_price=anchor_price, params={ @@ -40,11 +135,7 @@ def replacement_state(parent_id): if iid in seen: raise ValueError('重复子单编号') seen.add(iid) - qty, done = int(order['qty']), int(order.get('cum_qty') or 0) - if qty <= 0 or done < 0 or done > qty: - raise ValueError('子单累计成交数量异常') - if order.get('status') == 'FILLED' and done != qty: - raise ValueError('全成子单累计成交不完整') + done = effective_filled(order) filled += done if order.get('status') not in qmt_repo.FINAL: live.append(order) @@ -130,6 +221,8 @@ def should_replace(parent, desired, now=None): """仅比较同轮报价;失败重试次数和间隔由实际子单历史决定。""" if desired.get('action') != 'FIRE': return False + if correction_blocker(parent): + return False orders = qmt_repo.orders_for_parent(parent['instruction_id']) live = [o for o in orders if o.get('status') not in qmt_repo.FINAL] if not live or any(o.get('cancel_state') in ('REQUESTED', 'SENT') for o in live): diff --git a/app/services/executor.py b/app/services/executor.py index 1ce1b30..d2cf20d 100644 --- a/app/services/executor.py +++ b/app/services/executor.py @@ -201,6 +201,11 @@ def run_tick(*, now=None, dry_run: bool = False) -> dict: out['errors'].append(cancelled.get('error') or cancelled.get('message')) continue from app.services import execution_orders + correction_wait = execution_orders.correction_blocker(ins) + if correction_wait: + out['waited'].append({'instruction_id': ins['instruction_id'], 'code': code, + 'reason': correction_wait}) + continue if execution_orders.enabled() and dispatcher.mode() == dispatcher.MODE_WS: replacement = execution_orders.replacement_state(ins['instruction_id']) if not replacement['ready']: @@ -608,6 +613,9 @@ def _parent_available(ins: dict, children: list) -> int: 在途子单再预留尚未成交的数量;终态只释放未成交部分。必须跨日查询, 也必须包含已入出口但没写回 progress 的子单。查询失败由调用方报错,禁止下发。 """ + from app.services import execution_orders + if execution_orders.correction_blocker(ins): + return 0 target = int(ins['qty']) booked = int(ins.get('exec_qty') or 0) if dispatcher.mode() != dispatcher.MODE_WS: @@ -620,13 +628,8 @@ def _parent_available(ins: dict, children: list) -> int: raise ValueError('同一父指令的出口子单重复,暂停下发') seen.add(ref) qty = int(order['qty']) - done = int(order.get('cum_qty') or 0) - if qty <= 0 or done < 0 or done > qty: - raise ValueError('出口委托数量异常,暂停下发') + done = execution_orders.effective_filled(order) status = str(order.get('status') or '').upper() - # FILLED 却缺累计成交数属于不完整回报,不能据此释放整笔额度。 - if status == qmt_repo.OS_FILLED: - done = qty filled += done if status not in qmt_repo.FINAL: pending += qty - done diff --git a/app/services/ledger_service.py b/app/services/ledger_service.py index a5c4b0d..5f5c067 100644 --- a/app/services/ledger_service.py +++ b/app/services/ledger_service.py @@ -132,7 +132,7 @@ def consume_ws_trades(*, limit: int = 500) -> dict: """ out = {"ok": True, "trades": 0, "actions": 0, "fees": 0, "alerts": [], "errors": []} try: - rows = qmt_repo.inbox_pending(limit=limit) + rows = qmt_repo.inbox_pending(limit=limit, msg_type='trade') except Exception as e: # ws 三表没建 (影子期正常) 或库不可用 —— 不该让整个回放任务失败 out["skipped"] = f"inbox 不可读: {type(e).__name__}" @@ -382,10 +382,19 @@ def replay_fills(*, limit: int = 500) -> dict: out["ws"] = consume_ws_trades(limit=limit) if out["ws"].get("errors"): out["errors"].extend(out["ws"]["errors"]) + # 原成交先入账,修正再核对双报文;修正不会走新成交的批次或费用入口。 + from app.services import trade_correction + out['corrections'] = trade_correction.consume(limit=limit) + out['errors'].extend(out['corrections'].get('errors') or []) + correction_ok = out['corrections'].get('ok') is not False + for alert in out['corrections'].get('alerts') or []: + visible = dict(alert, message=alert.get('message') or alert.get('reason') or '成交更正待人工核实') + out['alerts'].append(visible) + logger.warning('[成交更正告警] %s', visible['message']) state, note = _cursor_load(pms_repo.get_param(CURSOR_KEY)) if state is None: # 从未设过 / 旧格式 —— 冷启动, 只对齐游标不入账 r = _seed_cursor(out, note_extra=note) - r["ok"] = not r.get("errors") + r["ok"] = not r.get("errors") and correction_ok return r since = str(state["t"]) @@ -398,14 +407,15 @@ def replay_fills(*, limit: int = 500) -> dict: try: fills = downstream_repo.fetch_filled_orders(since_time=since, limit=limit) except Exception as e: - out.update({"ok": False, "errors": [f"读 trading_order 失败: {type(e).__name__}: {e}"]}) + out["errors"].append(f"读 trading_order 失败: {type(e).__name__}: {e}") + out["ok"] = False return out seen = state["seen"] fresh = [f for f in fills if _fill_key(f) not in seen] out["fills"] = len(fresh) if not fresh: out["cursor"] = state["t"] - out["ok"] = not out["errors"] # 早退也要按 errors 定 ok (ws 那半的错不能吞) + out["ok"] = not out["errors"] and correction_ok # 早退也要按 errors 定 ok (ws 那半的错不能吞) return out # 2026-10-15:155 模拟仓三项差异均来自同一 ws 委托被汇总表再次回放。 @@ -508,9 +518,10 @@ def replay_fills(*, limit: int = 500) -> dict: except Exception: state["seen"] = seen _cursor_save(state, out) - out["ok"] = not out["errors"] + out["ok"] = not out["errors"] and correction_ok for a in out["alerts"]: - logger.warning("[回放告警] %s", a.get("message")) + if a.get("type") != "TRADE_CORRECTION_HELD": + logger.warning("[回放告警] %s", a.get("message") or a.get("reason")) return out diff --git a/app/services/rotation_execution.py b/app/services/rotation_execution.py index 3f86cce..d4d4330 100644 --- a/app/services/rotation_execution.py +++ b/app/services/rotation_execution.py @@ -22,16 +22,19 @@ def _later_snapshot(received_at, final_at): def release_settled_reservations(account, funds): """逐单释放已确认撤回的未成交金额,不重复扣QMT冻结、不预支卖出收入。""" from app.services import execution_orders + from app.repo import execution_repo + if execution_repo.pending_corrections(): + return initial = account.setdefault('original_reservations', {}) for iid, held in list(account['orders'].items()): initial.setdefault(iid, float(held)) order = qmt_repo.get_order(iid) - if not order or order['status'] not in TERMINAL or not _later_snapshot(funds.get('received_at'), order.get('final_at')): + if not order or order['status'] not in TERMINAL or not _later_snapshot(funds.get('received_at'), execution_orders.settlement_cutoff(order)): continue settled = execution_orders.replacement_state(order['parent_id']) if settled['pending_trades'] or settled['booked'] < settled['confirmed_filled'] or settled['missing_orders']: continue - qty, done, price = int(order['qty']), int(order.get('cum_qty') or 0), float(order['limit_price']) + qty, done, price = int(order['qty']), execution_orders.effective_filled(order), float(order['limit_price']) original = initial[iid] if qty <= 0 or not 0 <= done <= qty or price <= 0 or original+1e-6 < qty*price: raise ValueError('委托原始资金保留不可核实') @@ -42,6 +45,9 @@ def release_settled_reservations(account, funds): def _reservation_fills_booked(orders): from app.services import execution_orders + from app.repo import execution_repo + if execution_repo.pending_corrections(): + return False for order in orders: state = execution_orders.replacement_state(order['parent_id']) if state['pending_trades'] or state['booked'] < state['confirmed_filled'] or state['missing_orders']: @@ -51,7 +57,7 @@ def _reservation_fills_booked(orders): def enqueue_with_reservation(payload, parent_id): """所有策略共用一个账户锁;出口行与预算保留同一事务提交。""" - from app.services import rotation_service as service + from app.services import rotation_service as service, execution_orders with store.locked('account-execution', 'funds_reservation', {'epoch_seq': 0, 'available': 0., 'orders': {}}) as account: code, side, qty, price = payload['ts_code'], payload['side'], int(payload['qty']), float(payload['limit_price']) old = qmt_repo.get_order(payload['instruction_id']) @@ -65,6 +71,9 @@ def enqueue_with_reservation(payload, parent_id): parent = pms_repo.get_instruction(parent_id) if not parent: raise ValueError('委托没有可核实的策略父指令') + blocked = execution_orders.correction_blocker(parent, ts_code=code, side=side) + if blocked: + raise ValueError(blocked) book = parent['book_id'] if side == 'buy': if param_store.get_bool('PMS_GLOBAL_BUY_HALT', False): @@ -117,7 +126,7 @@ def enqueue_with_reservation(payload, parent_id): terminal.append(order) # 全部委托终态,且新的资金快照晚于全部终态,才开启新预算周期。 settled = not reserved or (len(terminal) == len(reserved) - and all(_later_snapshot(funds['received_at'], o['final_at']) for o in terminal) + and all(_later_snapshot(funds['received_at'], execution_orders.settlement_cutoff(o)) for o in terminal) and _reservation_fills_booked(terminal)) if settled and funds['seq'] > account['epoch_seq']: account.update(epoch_seq=funds['seq'], available=float(available), orders={}, original_reservations={}, market_value=float(parsed.get('market_value') or 0)) @@ -327,6 +336,7 @@ def cancel_expired(now): def book_trade(act, fees, seq): """策略归属、批次、费用、指令进度和inbox标记同成同败。""" from app.services import ledger_service as ledger + from app.services import trade_lineage parent = act.get('instruction_id') if not parent: raise ValueError('逐笔成交缺少父指令归属') @@ -344,6 +354,7 @@ def book_trade(act, fees, seq): raise ValueError('成交超过父指令剩余数量,等待核对') with books.scope(instruction['book_id']): before = pms_repo.get_position(act['ts_code']) or {} + lots_before = trade_lineage.lots(act['ts_code']) qty = int(act['qty']) if act['kind'] != 'BUY' and qty > int(before.get('total_qty') or 0): raise ValueError('卖出成交超出所属策略持仓,等待核对') @@ -363,5 +374,7 @@ def book_trade(act, fees, seq): record_fill(instruction['book_id'], 'buy' if act['kind'] == 'BUY' else 'sell', qty*float(act['price']), -sum(float(f['amount']) for f in fees), str(seq)) ledger.recompute_position(act['ts_code']) - qmt_repo.inbox_mark([seq], processed=qmt_repo.PROC_BOOKED, note='按策略归属原子入账') + trade_lineage.record(act, instruction, seq, lots_before, trade_lineage.lots(act['ts_code'])) + if not qmt_repo.inbox_mark([seq], processed=qmt_repo.PROC_BOOKED, note='按策略归属原子入账'): + raise RuntimeError('逐笔成交入账标记未写入') return True diff --git a/app/services/strategy_runner.py b/app/services/strategy_runner.py index cd37990..0351d72 100644 --- a/app/services/strategy_runner.py +++ b/app/services/strategy_runner.py @@ -45,6 +45,7 @@ signal_digest / intraday_exec 互相覆盖 (与 portfolio.neg_streak_map 用独 from __future__ import annotations import logging +from contextlib import contextmanager from datetime import datetime from app.core import tradedays as td @@ -128,6 +129,25 @@ def _macro_gate_active() -> bool: # ================================================================ 指令 / 提议下发 +def _correction_id(st: dict): + return ((st.get("state") or {}).get("trade_correction") or {}).get("correction_id") + + +@contextmanager +def _current_strategy(st: dict): + """只在落单阶段持账户锁;行情与策略求值不持锁、不跨线程。""" + from app.services.execution_orders import account_guard + with account_guard(): + fresh = pms_repo.get_strategy(st["strategy_id"]) + if not fresh or fresh.get("status") != "ACTIVE" or fresh.get("archived_at"): + raise RuntimeError("策略已停用,停止生成新指令或提议") + if (fresh.get("state") or {}).get("correction_hold"): + raise RuntimeError("策略成交数量已更正,等待核对后续安排") + if _correction_id(fresh) != _correction_id(st): + raise RuntimeError("策略成交更正版本已变化,请按最新状态重新评估") + yield fresh + + def _emit_instruction(st: dict, dec: dict, *, forced: bool = False) -> str: """按决策发一张短窗口命令指令 (window_tdays=1, is_command=True), 交 run_tick 执行。 @@ -146,11 +166,14 @@ def _emit_instruction(st: dict, dec: dict, *, forced: bool = False) -> str: # 网格/跟踪止盈到价即成交、当日没成交则作废, 不强制 prog = {"is_command": is_cmd, "deadline": str(_today()), "children": [], "origin": "strategy", "strategy_id": st["strategy_id"], "stype": st.get("type"), + "strategy_correction_id": _correction_id(st), "leg": dec["leg"], "reason": dec.get("reason"), "forced": bool(forced)} - pms_repo.insert_instruction( - instruction_id=iid, origin_type="strategy", origin_id=st["strategy_id"], - ts_code=code, action=dec["action"], side=dec["side"], qty=int(dec["qty"]), - limit_price=None, window_tdays=1, status="PROPOSED", progress=prog) + with _current_strategy(st): + if pms_repo.insert_instruction( + instruction_id=iid, origin_type="strategy", origin_id=st["strategy_id"], + ts_code=code, action=dec["action"], side=dec["side"], qty=int(dec["qty"]), + limit_price=None, window_tdays=1, status="PROPOSED", progress=prog) != 1: + raise RuntimeError("策略指令未成功保存") return iid @@ -162,33 +185,41 @@ def _emit_proposal(st: dict, dec: dict) -> str: now = datetime.now() pid = f"STP{_today()}{now.strftime('%H%M%S')}_{code.replace('.', '')}"[:40] hard = {"kind": "strategy", "strategy_id": st["strategy_id"], "stype": st.get("type"), + "strategy_correction_id": _correction_id(st), "side": dec["side"], "leg": dec["leg"], "action": dec["action"], "qty": int(dec["qty"]), "reason": dec.get("reason"), "entry": dec.get("entry")} expire = now.replace(hour=15, minute=0, second=0, microsecond=0) - pms_repo.insert_proposal(proposal_id=pid, ts_code=code, action=dec["action"], - qty=int(dec["qty"]), hard_numbers=hard, expire_at=expire, - judge_verdict="STRATEGY", - judge_reason=f"{st.get('type')} · {dec.get('reason')}") + with _current_strategy(st): + if pms_repo.insert_proposal(proposal_id=pid, ts_code=code, action=dec["action"], + qty=int(dec["qty"]), hard_numbers=hard, expire_at=expire, + judge_verdict="STRATEGY", + judge_reason=f"{st.get('type')} · {dec.get('reason')}") != 1: + raise RuntimeError("策略提议未成功保存") return pid def emit_from_spec(spec: dict) -> dict: """confirm 提议被采纳后的回调 (main._decide 调): 按 hard_numbers 里存的动作参数发指令。 返回 {ok, instruction_id} 或 {ok:false, error}。""" + from app.services.execution_orders import account_guard try: - st = pms_repo.get_strategy(spec.get("strategy_id")) - if not st: - return {"ok": False, "error": "策略不存在或已撤下"} - if st.get("status") != "ACTIVE": - return {"ok": False, "error": f"策略处于 {st.get('status')}, 不再发指令"} - dec = {"side": spec["side"], "action": spec["action"], "qty": int(spec["qty"]), - "leg": spec.get("leg") or "open", "reason": spec.get("reason"), - "entry": spec.get("entry")} - iid = _emit_instruction(st, dec) - state = dict(st.get("state") or {}) - state["pending"] = {"iid": iid, "leg": dec["leg"], "dir": spec.get("dir"), - "qty": dec["qty"], "entry": dec.get("entry")} - pms_repo.update_strategy(st["strategy_id"], state=state) + with account_guard(): + st = pms_repo.get_strategy(spec.get("strategy_id")) + if not st: + return {"ok": False, "error": "策略不存在或已撤下"} + if st.get("status") != "ACTIVE": + return {"ok": False, "error": f"策略处于 {st.get('status')}, 不再发指令"} + if spec.get("strategy_correction_id") != _correction_id(st): + return {"ok": False, "error": "提议对应的成交更正版本已变化,请按最新状态重新评估"} + dec = {"side": spec["side"], "action": spec["action"], "qty": int(spec["qty"]), + "leg": spec.get("leg") or "open", "reason": spec.get("reason"), + "entry": spec.get("entry")} + iid = _emit_instruction(st, dec) + state = dict(st.get("state") or {}) + state["pending"] = {"iid": iid, "leg": dec["leg"], "dir": spec.get("dir"), + "qty": dec["qty"], "entry": dec.get("entry")} + if pms_repo.update_strategy(st["strategy_id"], state=state) != 1: + raise RuntimeError("策略状态未成功保存") return {"ok": True, "instruction_id": iid} except Exception as e: # noqa: BLE001 logger.exception("[strategy] 采纳提议发指令失败") @@ -701,26 +732,31 @@ def tick(*, now=None, dry_run: bool = False) -> dict: if dec: if not dry_run: - _apply_grid_state(state, dec) - if (st.get("autonomy") or "auto") == "confirm": - pid = _emit_proposal(st, dec) - state["pending"] = {"pid": pid, "leg": dec["leg"], "dir": dec.get("dir"), - "qty": dec["qty"], "entry": dec.get("entry")} - out["queued"].append({"strategy_id": st["strategy_id"], "proposal_id": pid, - "reason": dec["reason"]}) - else: - iid = _emit_instruction(st, dec, forced=False) - state["pending"] = {"iid": iid, "leg": dec["leg"], "dir": dec.get("dir"), - "qty": dec["qty"], "entry": dec.get("entry"), - "grid_sell_idx": dec.get("grid_sell_idx"), - "grid_restore": dec.get("_grid_restore")} - out["fired"].append({"strategy_id": st["strategy_id"], "instruction_id": iid, - "side": dec["side"], "qty": dec["qty"], "reason": dec["reason"]}) + with _current_strategy(st): + _apply_grid_state(state, dec) + if (st.get("autonomy") or "auto") == "confirm": + pid = _emit_proposal(st, dec) + state["pending"] = {"pid": pid, "leg": dec["leg"], "dir": dec.get("dir"), + "qty": dec["qty"], "entry": dec.get("entry")} + if pms_repo.update_strategy(st["strategy_id"], state=state) != 1: + raise RuntimeError("策略状态未成功保存") + out["queued"].append({"strategy_id": st["strategy_id"], "proposal_id": pid, + "reason": dec["reason"]}) + else: + iid = _emit_instruction(st, dec, forced=False) + state["pending"] = {"iid": iid, "leg": dec["leg"], "dir": dec.get("dir"), + "qty": dec["qty"], "entry": dec.get("entry"), + "grid_sell_idx": dec.get("grid_sell_idx"), + "grid_restore": dec.get("_grid_restore")} + if pms_repo.update_strategy(st["strategy_id"], state=state) != 1: + raise RuntimeError("策略状态未成功保存") + out["fired"].append({"strategy_id": st["strategy_id"], "instruction_id": iid, + "side": dec["side"], "qty": dec["qty"], "reason": dec["reason"]}) else: out["fired"].append({"strategy_id": st["strategy_id"], "dry_run": True, "side": dec["side"], "qty": dec["qty"], "reason": dec["reason"]}) - if not dry_run: + if not dry_run and not dec: pms_repo.update_strategy(st["strategy_id"], state=state) except Exception as e: # noqa: BLE001 —— 单策略异常不拖垮整轮 logger.exception("[strategy] 评估失败 %s", st.get("strategy_id")) @@ -804,10 +840,12 @@ def force_t0_close(*, now=None) -> dict: if int(dec["qty"]) < LOT: out["skipped"].append({"strategy_id": st["strategy_id"], "why": "可平量不足 (avail 不够)"}) continue - iid = _emit_instruction(st, dec, forced=True) - state["pending"] = {"iid": iid, "leg": "close", "dir": open_leg.get("dir"), - "qty": dec["qty"], "entry": open_leg.get("entry")} - pms_repo.update_strategy(st["strategy_id"], state=state) + with _current_strategy(st): + iid = _emit_instruction(st, dec, forced=True) + state["pending"] = {"iid": iid, "leg": "close", "dir": open_leg.get("dir"), + "qty": dec["qty"], "entry": open_leg.get("entry")} + if pms_repo.update_strategy(st["strategy_id"], state=state) != 1: + raise RuntimeError("策略状态未成功保存") out["closed"].append({"strategy_id": st["strategy_id"], "instruction_id": iid, "side": dec["side"], "qty": dec["qty"]}) except Exception as e: # noqa: BLE001 diff --git a/app/services/trade_correction.py b/app/services/trade_correction.py new file mode 100644 index 0000000..b20bea9 --- /dev/null +++ b/app/services/trade_correction.py @@ -0,0 +1,500 @@ +"""两条QMT证据配对后的成交更正。原始成交不改写,投影与资金差额同成同败。""" +from __future__ import annotations + +import copy +import datetime as dt +import hashlib +import json +from collections import defaultdict +from decimal import Decimal, InvalidOperation, ROUND_HALF_UP +from zoneinfo import ZoneInfo + +from app.core import command_spec, strategy_book as books, trade_correction as core, ws_codec +from app.db import session as db +from app.repo import pms_repo, qmt_repo, rotation_repo as store, trade_correction_repo as repo +from config.settings import settings + +TZ = ZoneInfo('Asia/Shanghai') +IDENTITY = ('trade_no', 'instruction_id', 'broker_order_id', 'ts_code', 'side') +ECONOMIC = ('qty', 'price', 'amount') +LEGACY_NOTE = 'trade_correction 由通道进程消化, 不入账' + + +class Waiting(Exception): + """配对或原成交尚未入账,原消息留在待处理,不把暂缺证据判成冲突。""" + + +def _need(ok, reason): + if not ok: + raise ValueError(reason) + + +def _d(value): + _need(value is not None and not isinstance(value, bool), '金额或数量缺失') + try: + number = Decimal(str(value)) + except (InvalidOperation, ValueError, TypeError): + raise ValueError('金额或数量格式错误') from None + _need(number.is_finite(), '金额或数量不是有限值') + return number + + +def _q(value): + number = _d(value) + _need(number >= 0 and number == number.to_integral_value(), '数量须为非负整数') + return int(number) + + +def _body(value): + return json.dumps(value, sort_keys=True, separators=(',', ':'), ensure_ascii=False, default=str, allow_nan=False) + + +def _digest(value): + return hashlib.sha256(_body(value).encode()).hexdigest() + + +def _key(prefix, value): + return prefix+hashlib.sha256(str(value).encode()).hexdigest() + + +def _stamp(value): + try: + parsed = value if isinstance(value, dt.datetime) else dt.datetime.fromisoformat(str(value).replace('Z', '+00:00')) + return parsed.replace(tzinfo=TZ) if parsed.tzinfo is None else parsed.astimezone(TZ) + except (TypeError, ValueError): + raise ValueError('更正证据接收时间不可核实') from None + + +def _values(payload): + _need(isinstance(payload, dict) and all(k in payload for k in ECONOMIC), '成交数量价格金额不完整') + return (_q(payload['qty']), _d(payload['price']), _d(payload['amount'])) + + +def _write(result, label): + if not result: + raise RuntimeError(label+'未写入,回滚本次更正') + + +def _effective(row): + original = copy.deepcopy(row['payload']) + document, _ = store.get(_key('trade-effective:', original['trade_no'])) + if not document: + return original + _need(document.get('source_seq') == row['seq'] and document.get('original_digest') == _digest(original), + '有效成交版本与原始成交证据不符') + payload = document.get('effective') + _need(isinstance(payload, dict) and all(payload.get(k) == original.get(k) for k in IDENTITY), + '有效成交版本改变了成交身份') + _need(all(payload.get(k) == original.get(k) for k in ('fee', 'fee_estimated', 'trade_time')), + '有效成交版本改变了原费用或成交时间') + _values(payload) + return copy.deepcopy(payload) + + +def _pair(row): + payload = row.get('payload') + _need(isinstance(payload, dict), '成交更正报文须为对象') + cid, iid = payload.get('correction_id'), payload.get('instruction_id') + _need(isinstance(cid, str) and 0 < len(cid) <= 128 and isinstance(iid, str) and iid, + '成交更正缺少有效更正编号或子委托编号') + _need(all(payload.get(k) not in (None, '') for k in IDENTITY), '更正缺少原成交身份') + _need(isinstance(payload.get('before'), dict) and isinstance(payload.get('after'), dict) + and set(payload['before']) == set(ECONOMIC) and set(payload['after']) == set(ECONOMIC), + '更正前后只允许完整的数量、价格与金额') + _need(isinstance(payload.get('reason'), str) and payload['reason'].strip(), '成交更正原因缺失') + correct_time = payload.get('correct_time') + _need(isinstance(correct_time, int) and not isinstance(correct_time, bool) and 0 < correct_time < 253402271999999, + '更正时间须为正整数毫秒,不能替换原成交日期') + originals = repo.corrections(cid, iid) + _need(all(_digest(r['payload']) == _digest(payload) for r in originals), '同一更正编号对应不同更正内容') + _need(not any(r['processed'] == qmt_repo.PROC_ORPHAN for r in originals), '同一更正编号已有待人工核实记录') + amendments = repo.amendments(cid, iid) + if not amendments: + raise Waiting('等待同一更正编号的amended委托回报') + amendment = amendments[0] + other = amendment['payload'] + _need(isinstance(other, dict), '配套委托回报原文损坏,须人工核实后处理成交更正') + _need(all(_digest(r['payload']) == _digest(other) for r in amendments), '同一更正编号对应不同委托累计回报') + _need(not any(r['processed'] == qmt_repo.PROC_ORPHAN for r in amendments), '配对累计回报已有待人工核实记录') + _need(other.get('amended') is True and other.get('reason') == 'trade_correction' + and other.get('correction_id') == cid and other.get('instruction_id') == iid, + '委托回报不是同一成交更正的配对证据') + _need(other.get('broker_order_id') == payload['broker_order_id'], '更正与委托回报的券商委托号不一致') + for name in ('ts_code', 'side'): + _need(name not in other or other[name] == payload[name], '配对委托回报证券或方向冲突') + return payload, amendment, originals, amendments + + +def _load_evidence(original, order, parent): + """分别读取单表,在服务层按本端父指令归属组合完整证据。""" + code, book = parent['ts_code'], parent['book_id'] + orders = repo.orders_for_code(code) + parents, selected = {}, [] + with books.scope('all'): + for child in orders: + if qmt_repo.is_smoke(child.get('parent_id')): + continue + pid = child.get('parent_id') + current = parents.get(pid) or pms_repo.get_instruction(pid) + _need(current and current.get('book_id') in ('original', 'micro30'), '同股历史委托策略归属不完整') + _need(current.get('ts_code') == code and current.get('side') == child.get('side'), '历史父子委托身份冲突') + parents[pid] = current + if current['book_id'] == book: + selected.append(child) + by_child = {r['instruction_id']: r for r in selected} + _need(order['instruction_id'] in by_child, '目标子委托不在所属策略证据中') + rows = repo.trades_for_orders(list(by_child)) + trades, links = [], {} + for row in rows: + if row['processed'] == qmt_repo.PROC_PENDING: + raise Waiting('同股所属策略仍有原成交等待入账') + _need(row['processed'] == qmt_repo.PROC_BOOKED, '同股所属策略存在未核实的原成交') + raw = copy.deepcopy(row['payload']) + child = by_child[raw['instruction_id']] + _need(raw.get('broker_order_id') == child.get('broker_order_id') and raw.get('ts_code') == code + and raw.get('side') == child.get('side'), '原成交与本端委托身份冲突') + effective = _effective(row) + effective['parent_id'] = raw['parent_id'] = child['parent_id'] + trades.append({**row, 'payload': effective, 'original_payload': raw, 'book_id': book}) + journal, _ = store.get(_key('trade-lineage:', raw['trade_no'])) + if journal: + _need(journal.get('seq') == row['seq'] and journal.get('book_id') == book + and journal.get('parent_id') == child['parent_id'] + and journal.get('original_payload_digest') == _digest(row['payload']), '成交批次来源凭据冲突') + if raw['side'] == 'buy': + _need(journal.get('lot_id'), '买入来源凭据缺少批次编号') + links[raw['trade_no']] = journal['lot_id'] + _need(any(t['seq'] == original['seq'] for t in trades), '目标原成交不在完整入账证据中') + return selected, parents, trades, links + + +def _aggregate(trades, child_ids, replacement=None): + qty, amount = 0, Decimal(0) + for row in trades: + payload = row['payload'] + if payload['instruction_id'] not in child_ids: + continue + value = replacement[1] if replacement and payload['trade_no'] == replacement[0] else payload + q, _, money = _values(value) + qty += q + amount += money + return qty, amount + + +def _position_fields(position, projection): + _need(_q(position.get('total_qty')) == projection['qty_before'], '持仓数量与批次投影不一致') + _need(_d(position.get('realized_t_profit', 0)) == _d(projection['t_profit_before']), '持仓做T收益与原逐笔投影不一致') + available = _q(position.get('avail_qty'))+int(projection['avail_delta']) + quantity = projection['qty_after'] + _need(0 <= available <= quantity, '更正后的T+1可卖量越界') + by_type = defaultdict(int) + for lot in projection['lots']: + by_type[lot['lot_type']] += int(lot['qty']) + net = _d(projection['net_cost_after']) + average = max(Decimal(0), net/quantity).quantize(Decimal('.001'), rounding=ROUND_HALF_UP) if quantity else None + result = {'total_qty': quantity, 'avail_qty': available, 'base_qty': by_type['BASE'], + 'fill_qty': by_type['FILL'], 'add_qty': by_type['ADD'], 'dca_qty': by_type['DCA'], 't0_qty': by_type['T0'], + 'avg_cost': average, 'realized_t_profit': _d(position.get('realized_t_profit', 0))+_d(projection['t_profit_delta']), + 'cushion_pct': None, 'cushion_state': 'NONE', 'cushion_peak': 0, 'pct_of_scale': None} + # 成本基准改变,旧峰值仅保留审计;等待正常行情重建,不借旧值产生动作。 + if quantity == 0: + result.update(status='CLOSED', cushion_peak=0) + elif position.get('status') == 'CLOSED': + result['status'] = 'HOLDING' + return result + + +def _verify_physical(code, book, projection, received_at): + if projection['qty_before'] == projection['qty_after']: + return + snap = qmt_repo.latest_snapshot('positions') + if not snap or snap.get('age_sec') is None or not 0 <= snap['age_sec'] <= 60: + raise Waiting('数量更正等待一分钟内QMT持仓快照') + if _stamp(snap.get('received_at')) < received_at: + raise Waiting('数量更正等待配对回报之后的新QMT持仓快照') + payload = snap.get('payload') or {} + items = next((payload[k] for k in ('items', 'positions', 'rows', 'data', 'list') if isinstance(payload.get(k), list)), None) + _need(items is not None, 'QMT持仓快照没有完整列表,不能猜成空仓') + for item in items: + _need(isinstance(item, dict), 'QMT持仓快照行格式不合法') + low = {str(k).lower(): v for k, v in item.items()} + raw_qty = next((low[k] for k in ws_codec._SNAP_QTY if k in low and low[k] not in (None, '')), None) + _q(raw_qty) # 不接受兼容解析器int(float(...))把小数数量截成整数。 + parsed = ws_codec.parse_positions_snapshot(payload) + _need(not items or (parsed.get('columns', {}).get('qty') and len(parsed['rows']) == len(items)), 'QMT持仓快照数量或证券列无法识别') + physical = 0 + seen = set() + for row in parsed['rows']: + normalized = command_spec.normalize_code(row.get('ts_code')) + _need(normalized and normalized not in seen and row.get('qty') is not None, 'QMT持仓快照证券重复或数量缺失') + seen.add(normalized) + if normalized == code: + physical = _q(row['qty']) + with books.scope('all'): + positions = [r for r in pms_repo.list_positions() if r['ts_code'] == code] + _need(all(r.get('book_id') in ('original', 'micro30') for r in positions), '同股持仓策略归属不可核实') + total = sum(_q(r['total_qty']) for r in positions if r['book_id'] != book)+projection['qty_after'] + _need(total == physical, '更正后双策略数量与QMT实际持仓不一致') + + +def _cash_projection(original, book, side, projection, amended, order): + value, _ = store.get('capital:live') + _need(isinstance(value, dict) and isinstance(value.get('cash'), dict), '分策略现金尚未初始化') + entry = value.get('fills', {}).get(str(original['seq'])) + raw = original['payload'] + fee = Decimal(0) if raw.get('fee') is None else _d(raw['fee']) + amount = _d(raw['price'])*_q(raw['qty']) + _need(entry and entry.get('book') == book and entry.get('side') == side + and abs(_d(entry.get('amount'))-amount) <= Decimal('.000001') + and abs(_d(entry.get('amount'))-_d(raw.get('amount'))) <= Decimal('.000001') + and abs(_d(entry.get('fee'))-fee) <= Decimal('.000001'), '原成交本金或费用与策略现金凭据不符') + updated = copy.deepcopy(value) + cash = _d(value['cash'].get(book))+_d(projection['cash_delta']) + _need(cash >= 0, '更正差额会使所属策略现金透支') + active = qmt_repo.list_orders(statuses=qmt_repo.LIVE, limit=10001) + _need(len(active) < 10001, '在途委托超过资金核验上限') + reserved = Decimal(0) + from app.services.rotation_service import configuration + from app.core.micro_rotation import fee as estimate_fee + with books.scope('all'): + for child in active: + if child.get('side') != 'buy' or child['instruction_id'] == order['instruction_id'] and amended.get('final') is True: + continue + parent = pms_repo.get_instruction(child['parent_id']) + _need(parent and parent.get('book_id') in ('original', 'micro30'), '在途资金策略归属未知') + if parent['book_id'] != book: + continue + cum = amended['cum_qty'] if child['instruction_id'] == order['instruction_id'] else child.get('cum_qty', 0) + remaining = _q(child['qty'])-_q(cum) + _need(remaining >= 0, '在途委托剩余数量越界') + if remaining: + principal = _d(child['limit_price'])*remaining + reserved += principal+_d(estimate_fee(float(principal), 'buy', **configuration()['costs'])) + _need(cash >= reserved, '更正后策略现金不足以覆盖既有在途买入占用') + updated['cash'][book] = float(cash) + return value, updated + + +def _grid_updates(code, parent_map, orders, projection, quantity_changed, cid): + strategies = pms_repo.list_strategies(ts_code=code, statuses=['ACTIVE', 'PAUSED'], include_archived=True, limit=1001) + _need(len(strategies) < 1001, '同股策略数超过核验上限') + result = [] + for strategy in strategies: + _need(strategy.get('type') == 'GRID', '同股活动策略不是可重建网格,须人工核对后更正') + state = copy.deepcopy(strategy.get('state') or {}) + previous_correction = state.get('trade_correction') or {} + _need(isinstance(previous_correction, dict), '网格更正版本不可核实') + _need(not state.get('pending'), '网格仍有待结算指令,须核对后更正') + sid = strategy['strategy_id'] + related = {pid for pid, parent in parent_map.items() if parent.get('origin_id') == sid} + _need(all(r['status'] in qmt_repo.FINAL for r in orders if r['parent_id'] in related), '网格仍有活动子委托,须核对后更正') + levels = defaultdict(lambda: [0, Decimal(0)]) + for lot in projection['lots']: + parent = parent_map.get(lot.get('instruction_id')) + _need(parent, '批次缺少完整父指令来源') + if parent.get('origin_id') != sid or not int(lot['qty']): + continue + leg = (parent.get('progress') or {}).get('leg', '') + _need(isinstance(leg, str) and leg.startswith('grid_buy:') and leg.split(':', 1)[1].lstrip('-').isdigit(), '网格批次档位来源不可核实') + level = leg.split(':', 1)[1] + levels[level][0] += int(lot['qty']) + levels[level][1] += int(lot['qty'])*_d(lot['open_price']) + state['filled_levels'] = {k: {'qty': v[0], 'price': float(v[1]/v[0])} for k, v in levels.items()} + state['invested'] = float(sum((v[1] for v in levels.values()), Decimal(0))) + state['trade_correction'] = {'correction_id': cid, 'source': '已核实真实剩余批次'} + fields = {'state': state, 'correction_expected': previous_correction.get('correction_id')} + if quantity_changed: + state['correction_hold'] = '成交数量已更正,网格档位须人工复核后恢复' + fields['status'] = 'PAUSED' + result.append((sid, fields)) + return result + + +def _order_fields(order, amended, quantity, received): + if any(k not in amended or amended[k] is None for k in ('status', 'final', 'leaves_qty')): + raise Waiting('配对委托回报缺少状态、终态或剩余量,等待核实') + status, final = amended['status'], amended['final'] + _need(status in ws_codec.PROTOCOL_STATUSES and isinstance(final, bool), '配对委托状态格式不合法') + _need(final == (status in ws_codec.FINAL_STATUSES), '配对委托状态与终态标记冲突') + leaves = _q(amended['leaves_qty']) + _need(leaves == (0 if final else _q(order['qty'])-quantity), '配对委托剩余量与更正量冲突') + fields = {'cum_qty': quantity, 'cum_avg_price': amended.get('cum_avg_price'), 'leaves_qty': leaves} + if order['status'] in qmt_repo.FINAL: + _need(final and leaves == 0, '已终态委托不能因更正重新进入在途') + return fields # 对端可重分类终态;本地历史执行终态和final_at均保持。 + _need(order['status'] in qmt_repo.LIVE, '本端委托状态不可核实') + _need(status != 'FILLED' or quantity == _q(order['qty']), '活动单转全成时累计数量未满') + _need(status != 'REJECTED' or quantity == 0, '拒单终态不能带成交数量') + _need(status != 'PARTIAL' or 0 < quantity < _q(order['qty']), '部分成交状态的累计量不合法') + fields['status'] = status + if final: + fields['final_at'] = received.replace(tzinfo=None) + return fields + + +def _apply(row): + state = qmt_repo.get_state() or {} + watermark = state.get('last_seq') + if not isinstance(watermark, int) or watermark < int(row['seq']): + raise Waiting('更正回报等待连续持久水位追平') + payload, amendment, corrections, amendments = _pair(row) + if int(amendment['seq']) > watermark: + raise Waiting('配对累计回报等待连续持久水位追平') + cid, no, iid = payload['correction_id'], payload['trade_no'], payload['instruction_id'] + if repo.pending_for_trade(no, before_seq=row['seq']): + raise Waiting('同一原成交的前序更正仍待完成或人工核实') + fingerprint = _digest(payload) + audit_key = _key('trade-correction:', cid) + audit, _ = store.get(audit_key) + if audit: + _need(audit.get('payload_fingerprint') == fingerprint and audit.get('amendment_fingerprint') == _digest(amendment['payload']), '更正编号已处理但正文不同') + duplicates = [r['seq'] for r in corrections+amendments if r['processed'] in (0, 2)] + if duplicates: + _write(qmt_repo.inbox_mark(duplicates, processed=qmt_repo.PROC_DIGESTED, note='同一更正已原子入账,重复消息不再改账'), '重复更正标记') + return 'duplicate' + _need(settings.INFRA_STRATEGY_BOOK_SCHEMA, '未启用分策略账本,成交更正须人工核对') + original = repo.get_trade(no) + if not original or original['processed'] == qmt_repo.PROC_PENDING: + raise Waiting('等待原成交入账,更正不阻止原成交消费') + _need(original['processed'] == qmt_repo.PROC_BOOKED, '原成交没有已入账凭据') + _need(all(payload[k] == original['payload'].get(k) for k in IDENTITY), '更正身份与原成交不同') + effective = _effective(original) + _need(_values(payload['before']) == _values(effective), '更正before不是当前有效成交版本') + order = qmt_repo.get_order(iid) + _need(order and not qmt_repo.is_smoke(order.get('parent_id')), '更正子委托缺失或属于联调单') + _need(all(order.get(k) == payload[k] for k in ('instruction_id', 'broker_order_id', 'ts_code', 'side')), '更正身份与出口委托不一致') + with books.scope('all'): + parent = pms_repo.get_instruction(order['parent_id']) + _need(parent and parent.get('book_id') in ('original', 'micro30') and parent.get('ts_code') == payload['ts_code'] + and parent.get('side') == payload['side'], '父指令证券、方向或策略归属不可核实') + book, code = parent['book_id'], parent['ts_code'] + with books.scope(book): + orders, parents, trades, links = _load_evidence(original, order, parent) + lots = pms_repo.list_lots(code, status=None, limit=20001) + _need(len(lots) < 20001, '批次证据超过核验上限') + projection = core.compile_correction(lots=lots, trades=trades, target_trade_no=no, + after=payload['after'], today=dt.datetime.now(TZ).date(), lot_links=links) + current = pms_repo.get_position(code) + _need(current and not current.get('archived_at'), '目标策略持仓不存在或已归档') + position_fields = _position_fields(current, projection) + child_after, child_amount = _aggregate(trades, {iid}, (no, payload['after'])) + am = amendment['payload'] + _need(_q(am.get('cum_qty')) == child_after and child_after <= _q(order['qty']), '配对累计成交量与更正后逐笔合计不一致或超过子单') + avg = child_amount/child_after if child_after else Decimal(0) + # QMT旧协议累计均价为两位,新回报允许三位;仅接受正常舍入,不容忍任意差值。 + _need((not child_after and am.get('cum_avg_price') is None) or (am.get('cum_avg_price') is not None and _d(am['cum_avg_price']) in {avg, avg.quantize(Decimal('.01'), rounding=ROUND_HALF_UP), avg.quantize(Decimal('.001'), rounding=ROUND_HALF_UP)}), '配对累计均价与更正后逐笔金额不一致') + order_fields = _order_fields(order, am, child_after, max(_stamp(row['received_at']), _stamp(amendment['received_at']))) + children = {r['instruction_id'] for r in orders if r['parent_id'] == parent['instruction_id']} + old_qty, _ = _aggregate(trades, children) + new_qty, new_amount = _aggregate(trades, children, (no, payload['after'])) + _need(old_qty == _q(parent['exec_qty']) and new_qty <= _q(parent['qty']), '父指令执行量与旧逐笔合计不符或更正后超量') + quantity_changed = _q(effective['qty']) != _q(payload['after']['qty']) + received = max(_stamp(row['received_at']), _stamp(amendment['received_at'])) + _verify_physical(code, book, projection, received) + cash_before, cash_after = _cash_projection(original, book, payload['side'], projection, am, order) + projected_orders = [{**r, **order_fields} if r['instruction_id'] == iid else r for r in orders] + strategy_updates = _grid_updates(code, parents, projected_orders, projection, quantity_changed, cid) + progress = copy.deepcopy(parent.get('progress') or {}) + progress['trade_correction'] = {'correction_id': cid, 'before_exec_qty': old_qty, 'exec_qty': new_qty, + 'exec_amount': str(new_amount), 'received_at': received.isoformat()} + if quantity_changed: + progress['trade_correction_hold'] = '成交数量更正后停止自动续单,等待人工决定剩余目标' + for child in progress.get('children', []): + if child.get('ref') == iid: + child.update(actual_exec_qty=child_after, actual_exec_amount=str(child_amount)) + # 所有事实核对完成后才开始写入;任何软失败必须抛出以回滚整个savepoint。 + for change in projection['changes']: + _write(pms_repo.update_lot(change['id'], **{k: change['after'][k] for k in core.LOT_FIELDS}), '更正批次') + _write(pms_repo.update_position(code, **position_fields), '更正持仓') + _write(pms_repo.update_instruction(parent['instruction_id'], exec_qty=new_qty, + exec_avg_price=float(new_amount/new_qty) if new_qty else 0, progress=progress), '更正父指令') + if parent.get('origin_type') == 'plan': + plans = pms_repo.list_plans(ts_code=code, limit=10001) + _need(len(plans) < 10001 and any(p['plan_id'] == parent.get('origin_id') for p in plans), '来源方案不存在或超过核验上限') + _write(pms_repo.update_plan(parent['origin_id'], filled_qty=new_qty), '更正方案进度') + for sid, fields in strategy_updates: + _write(pms_repo.update_strategy(sid, **fields), '更正网格状态') + _write(qmt_repo.update_order(iid, **order_fields), '更正子委托累计量') + with store.locked('capital:live', 'capital', {}) as capital: + _need(_body(capital) == _body(cash_before), '资金核验后凭据已变化') + capital.clear(); capital.update(cash_after) + for trade_no, lot_id in projection['lot_links'].items(): + journal_key = _key('trade-lineage:', trade_no) + if store.get(journal_key)[0]: + continue + trade = next(t for t in trades if t['payload']['trade_no'] == trade_no) + raw = repo.get_trade(trade_no) + store.put_once(journal_key, 'trade_lineage', {'trade_no': trade_no, 'seq': trade['seq'], + 'parent_id': trade['payload']['parent_id'], 'instruction_id': trade['payload']['instruction_id'], + 'book_id': book, 'ts_code': code, 'side': 'buy', 'lot_id': lot_id, + 'original_payload_digest': _digest(raw['payload'])}) + updated_effective = {**effective, **payload['after']} + with store.locked(_key('trade-effective:', no), 'trade_effective', {}) as doc: + doc.update(source_seq=original['seq'], original_digest=_digest(original['payload']), effective=updated_effective, + last_correction_id=cid, lot_id=projection['lot_links'].get(no)) + with store.locked(_key('order-correction:', iid), 'order_correction', {}) as doc: + doc.update(instruction_id=iid, cum_qty=child_after, correction_seq=row['seq'], amendment_seq=amendment['seq'], received_at=received.isoformat(), + correction_id=cid, cum_avg_price_before=order.get('cum_avg_price'), cum_avg_price_after=am['cum_avg_price']) + store.put_once(audit_key, 'trade_correction', {'correction_id': cid, 'payload_fingerprint': fingerprint, + 'amendment_fingerprint': _digest(am), 'original_seq': original['seq'], 'correction_seq': row['seq'], + 'amendment_seq': amendment['seq'], 'book_id': book, 'ts_code': code, 'before': projection['trade_before'], + 'after': projection['trade_after'], 'cash_delta': projection['cash_delta'], 'lot_changes': projection['changes'], + 'allocations_before': projection['allocations_before'], 'allocations_after': projection['allocations'], + 'parent_status_preserved': parent['status'], 'order_status_before': order['status'], + 'order_status_after': order_fields.get('status', order['status']), 'qmt_amended_status': am['status'], 'quantity_hold': quantity_changed, 'received_at': received.isoformat(), + 'position_before': {k: current.get(k) for k in position_fields}, 'position_after': position_fields, + 'cushion_note': '旧成本基准峰值失效,待正常行情重建', + 'cash_before': cash_before['cash'][book], 'cash_after': cash_after['cash'][book], + 'original_fee': original['payload'].get('fee'), 'original_fee_estimated': original['payload'].get('fee_estimated')}) + seqs = [r['seq'] for r in corrections+amendments if r['processed'] in (0, 2)] + _write(qmt_repo.inbox_mark(seqs, processed=qmt_repo.PROC_BOOKED, note='成交更正及累计回报已按策略归属原子入账'), '更正回执标记') + return 'applied' + + +def _hold(row, reason): + with store.account_guard(), db.transaction(), db.savepoint(): + current = repo.get_inbox(row['seq'], lock=True) + if not current or current['processed'] not in (0, 2): + return False + payload = current.get('payload') if isinstance(current.get('payload'), dict) else {} + raw = current.get('payload_json') + raw = raw if isinstance(raw, str) else json.dumps(payload, ensure_ascii=False, default=str) + raw_fingerprint = hashlib.sha256(raw.encode()).hexdigest() + _write(qmt_repo.inbox_mark([row['seq']], processed=qmt_repo.PROC_ORPHAN, + note='成交更正待人工核实:'+reason), '更正挂起标记') + store.put_once(_key('trade-correction-hold:', str(row['seq'])), 'trade_correction_hold', + {'seq': row['seq'], 'correction_id': payload.get('correction_id') if isinstance(payload.get('correction_id'), str) else None, + 'raw_payload_fingerprint': raw_fingerprint, 'reason': reason}) + return True + + +def consume(limit=100): + out = {'ok': True, 'applied': 0, 'waiting': 0, 'held': 0, 'errors': [], 'alerts': []} + try: + pending = repo.pending(limit=limit) + except Exception as exc: + out.update(ok=False, errors=['更正收件证据读取失败:'+type(exc).__name__]) + return out + for candidate in pending: + try: + with store.account_guard(), db.transaction(), db.savepoint(): + row = repo.get_inbox(candidate['seq'], lock=True) + if not row or not (row['processed'] == 0 or row['processed'] == 2 and row.get('process_note') == LEGACY_NOTE): + continue + result = _apply(row) + if result == 'applied': + out['applied'] += 1 + except Waiting: + out['waiting'] += 1 + except ValueError as exc: + reason = str(exc) + try: + if _hold(candidate, reason): + out['held'] += 1 + out['alerts'].append({'type': 'TRADE_CORRECTION_HELD', 'seq': candidate['seq'], 'reason': reason}) + except Exception as failure: + out['errors'].append('更正挂起审计未完成:'+type(failure).__name__) + except Exception as exc: + out['errors'].append('成交更正事务未完成,保留待重试:'+type(exc).__name__) + out['ok'] = not out['errors'] and not out['held'] + return out diff --git a/app/services/trade_lineage.py b/app/services/trade_lineage.py new file mode 100644 index 0000000..06b01f2 --- /dev/null +++ b/app/services/trade_lineage.py @@ -0,0 +1,47 @@ +"""逐笔成交与批次的原子归属凭据。只记录真实入账,不重放或重写原回执。""" +import hashlib +import json + +from app.repo import pms_repo, rotation_repo + + +def key(trade_no): + return 'trade-lineage:'+hashlib.sha256(str(trade_no).encode()).hexdigest() + + +def lots(code): + rows = pms_repo.list_lots(code, status=None, limit=2001) + if len(rows) >= 2001: + raise ValueError('批次超过完整核验上限,不能截断入账依据') + return rows + + +def record(act, instruction, seq, before, after): + trade_no = act.get('order_id') + if not trade_no: + raise ValueError('逐笔入账缺少成交编号') + old = {r['id']: r for r in before} + new = {r['id']: r for r in after} + fields = ('qty', 'closed_qty', 'open_price', 'close_avg_price', 'realized_pnl', 'status') + changed = [i for i, row in new.items() if i not in old or + any(str(row.get(f)) != str(old[i].get(f)) for f in fields)] + lot_id = None + if act['kind'] == 'BUY': + added = set(new)-set(old) + if len(added) != 1: + raise ValueError('买入成交无法唯一对应新增批次') + lot_id = next(iter(added)) + if int(new[lot_id]['qty']) != int(act['qty']): + raise ValueError('新增批次数量与逐笔成交不符') + source = act.get('source_payload') + digest = (hashlib.sha256(json.dumps(source, ensure_ascii=False, sort_keys=True, + separators=(',', ':'), default=str, allow_nan=False).encode()).hexdigest() + if source else None) + rotation_repo.put_once(key(trade_no), 'trade_lineage', { + 'trade_no': trade_no, 'seq': int(seq), 'parent_id': instruction['instruction_id'], + 'instruction_id': act.get('child_instruction_id'), + 'book_id': instruction['book_id'], 'ts_code': act['ts_code'], + 'side': 'buy' if act['kind'] == 'BUY' else 'sell', 'lot_id': lot_id, + 'original_payload_digest': digest, + 'changes': [{'lot_id': i, 'before': {f: old[i].get(f) for f in fields} if i in old else None, + 'after': {f: new[i].get(f) for f in fields}} for i in changed]}) diff --git a/app/ws/runner.py b/app/ws/runner.py index fa72bda..a85a885 100644 --- a/app/ws/runner.py +++ b/app/ws/runner.py @@ -512,12 +512,13 @@ class WsRunner: 协议的补发机制正是为这种情况准备的, 用它比自己攒重试队列稳妥得多。 """ type_, pl, seq = env["type"], env.get("payload") or {}, int(env["seq"]) - needs_ledger = type_ in wsc.NEEDS_LEDGER + needs_ledger = wsc.needs_ledger(type_, pl) note = None if needs_ledger else f"{type_} 由通道进程消化, 不入账" try: r = await _db(qmt_repo.inbox_put, seq=seq, msg_id=str(env.get("msg_id") or ""), msg_type=type_, payload=pl, msg_ts=int(env.get("ts") or 0), - corr_id=env.get("corr_id") or pl.get("instruction_id"), + corr_id=(pl.get('instruction_id') if wsc.is_correction(type_, pl) + else env.get("corr_id") or pl.get("instruction_id")), dedup_key=wsc.dedup_key(type_, pl), processed=0 if needs_ledger else 2, note=note) except Exception as e: @@ -525,6 +526,8 @@ class WsRunner: if r == qmt_repo.PUT_DUP_KEY: logger.warning("成交 %s 重复推送 (seq=%s), 已占位不入账", pl.get("trade_no"), seq) + elif r == qmt_repo.PUT_CONFLICT: + logger.error('成交修正消息冲突或缺少身份,seq=%s,已挂起等待核对', seq) had_pending = bool(self._pending_seq) self._last_seq, self._pending_seq = wsc.next_watermark( self._last_seq, self._pending_seq, seq) @@ -632,21 +635,43 @@ class WsRunner: type_, iid, self._stat["orphan_updates"]) async def _apply_side_effects(self, type_: str, pl: dict, env: dict): + if (type_ in (wsc.T_ACK, wsc.T_REJECT, wsc.T_ORDER_UPDATE) + and not wsc.is_correction(type_, pl) + and (pl.get('instruction_id') or env.get('corr_id'))): + try: + await _db(self._apply_order_side_effects_locked, type_, pl, env) + except Exception as exc: + logger.warning('委托状态持锁更新失败 %s %s: %s', type_, + pl.get('instruction_id') or env.get('corr_id'), _brief_err(exc)) + else: + self._apply_side_effects_sync(type_, pl, env) + + def _apply_order_side_effects_locked(self, type_, pl, env): + from app.services import execution_orders + # 必须在同一线程内完成事务,不能让ContextVar连接跨多个to_thread调用。 + with execution_orders.account_guard(): + self._apply_side_effects_sync(type_, pl, env) + + def _apply_side_effects_sync(self, type_: str, pl: dict, env: dict): """把上行消息落到 pms_qmt_order 的状态上。**只动通道状态, 不动账本**。""" iid = pl.get("instruction_id") or env.get("corr_id") + if wsc.is_correction(type_, pl): + # 包括终态后的 amended 回报。两条证据配对、记账和累计量变更由 worker 原子完成。 + logger.info('[trade_correction] %s seq=%s 已收取,等待修正核验入账', iid, env.get('seq')) + return try: if type_ == wsc.T_ACK: # 终态保护 (2026-08-28 审查修, 与 _on_order_update 同一条纪律): # _boot 重发换来的 ack{duplicate:true} 是新 seq 的新消息, 不带 status 时 # 这里的 ACCEPTED 兜底会把已 FILLED 的委托改回在途 —— 幻影单从此常驻 # queue_depth 与撤单清单。已终态的委托, ack 一律只记日志不动状态。 - row = await _db(qmt_repo.get_order, iid) or {} + row = qmt_repo.get_order(iid) or {} cur = str(row.get("status") or "").upper() if cur and wsc.is_final(cur): logger.info("[ack] %s 已是终态 %s, ack(duplicate=%s) 不回退状态", iid, cur, bool(pl.get("duplicate"))) else: - n = await _db(qmt_repo.update_order, iid, + n = qmt_repo.update_order(iid, status=str(pl.get("status") or wsc.ST_ACCEPTED).upper(), broker_order_id=pl.get("broker_order_id")) self._note_orphan(n, type_, iid) @@ -675,22 +700,22 @@ class WsRunner: n, json.dumps(pl, ensure_ascii=False)[:500]) self._maybe_degrade_ack(pl) else: - row = await _db(qmt_repo.get_order, iid) or {} + row = qmt_repo.get_order(iid) or {} cur = str(row.get('status') or '').upper() # 撤单拒绝不等于原委托拒绝。先记录拒绝原因,保留数量与原状态。 cancelling = row.get('cancel_state') in ('REQUESTED', 'SENT') if wsc.is_final(cur) or cancelling or code == 'ALREADY_FINAL' or pl.get('cancel_id'): - self._note_orphan(await _db(qmt_repo.update_order, iid, + self._note_orphan(qmt_repo.update_order(iid, reject_code=code[:32], reject_reason=str(pl.get('reason') or '')[:300]), type_, iid) final_status = str(pl.get('status') or pl.get('current_status') or '').upper() if not wsc.is_final(cur) and wsc.is_final(final_status) and pl.get('cum_qty') is not None: - await self._on_order_update(iid, {**pl, 'status': final_status, 'final': True}) + self._on_order_update_sync(iid, {**pl, 'status': final_status, 'final': True}, sequence=env.get('seq')) else: logger.warning('[reject] %s 撤单或终态响应 %s,保留委托状态 %s 和成交数量,等待最终核对', iid, code, cur) return - n = await _db(qmt_repo.update_order, iid, status=qmt_repo.OS_REJECTED, + n = qmt_repo.update_order(iid, status=qmt_repo.OS_REJECTED, reject_code=code[:32], reject_reason=str(pl.get("reason") or "")[:300], final_at=datetime.now()) @@ -701,7 +726,7 @@ class WsRunner: lvl("[reject] %s %s: %s (retryable=%s, 不自动重发)", iid, code, pl.get("reason"), pl.get("retryable")) elif type_ == wsc.T_ORDER_UPDATE: - await self._on_order_update(iid, pl) + self._on_order_update_sync(iid, pl, sequence=env.get('seq')) elif type_ == wsc.T_TRADE: self._stat["trades"] += 1 if not wsc.trade_amount_ok(pl): @@ -733,10 +758,29 @@ class WsRunner: # 通道状态更新失败不该拖垮连接 —— 账本不靠它, 靠 inbox 里的 trade。 logger.warning("通道状态更新失败 %s %s: %s", type_, iid, _brief_err(e)) - async def _on_order_update(self, iid: str, pl: dict): + async def _on_order_update(self, iid: str, pl: dict, sequence=None): + if wsc.is_correction(wsc.T_ORDER_UPDATE, pl): + return + await _db(self._on_order_update_locked, iid, pl, sequence) + + def _on_order_update_locked(self, iid, pl, sequence): + from app.services import execution_orders + with execution_orders.account_guard(): + self._on_order_update_sync(iid, pl, sequence) + + def _on_order_update_sync(self, iid: str, pl: dict, sequence=None): + if wsc.is_correction(wsc.T_ORDER_UPDATE, pl): + return status = str(pl.get("status") or "").upper() - row = await _db(qmt_repo.get_order, iid) or {} + row = qmt_repo.get_order(iid) or {} cur = str(row.get("status") or "").upper() + if row: + from app.services import execution_orders + audit = execution_orders.order_correction(row) + if audit and (wsc.is_final(cur) or sequence is not None and int(sequence) <= + max(int(audit['correction_seq']), int(audit.get('amendment_seq') or 0))): + logger.info('[order_update] %s 已有已核实成交修正,普通终态回报不覆盖修正版本', iid) + return done = pl.get('cum_qty') try: done = int(done) @@ -776,7 +820,7 @@ class WsRunner: "撤单可能在到期后才到, 按到期处理", iid) logger.info("[order_update] %s 终态 %s, 累计成交 %s 股", iid, status, fields["cum_qty"]) - self._note_orphan(await _db(qmt_repo.update_order, iid, **fields), + self._note_orphan(qmt_repo.update_order(iid, **fields), wsc.T_ORDER_UPDATE, iid) # ============================================================ 确认 @@ -921,7 +965,12 @@ class WsRunner: qty=int(r["qty"]), limit_price=r["limit_price"], valid_until=int(r["valid_until"]), intent=r.get("intent") or "OPEN", note=r.get("note") or "", strategy_type=strategy_type) - if not await _db(qmt_repo.mark_send_attempt, iid): + from app.services import execution_orders + send_check = await _db(execution_orders.prepare_send, iid) + if send_check.get('blocked'): + await self._stop_outbox_order(iid, send_check['blocked']) + continue + if not send_check['marked']: continue # 同时发生撤单或状态改变,不再发送。 await self._send(wsc.T_PLACE, payload, seed, corr_id=iid) except Exception as e: diff --git a/docs/QMT成交修正接入与发布_2026-11-05.md b/docs/QMT成交修正接入与发布_2026-11-05.md new file mode 100644 index 0000000..28b7a96 --- /dev/null +++ b/docs/QMT成交修正接入与发布_2026-11-05.md @@ -0,0 +1,51 @@ +# QMT成交修正接入与发布 + +2026年11月5日,北京时间。本次承接用户提供的工作台成交修正协议。原成交号不变,新增修正编号,并配套带修正标记的委托累计回报。 + +## 目标和边界 + +PMS负责按原策略归属修正数量、现金和成本解释。QMT原成交、历次修正及配套回报均保留。修正不是一笔新成交,不重放原成交,不重复计费,不生成委托,不改变真实持仓或费用数据。 + +修正和配套回报均先收存,再由原成交回放任务配对处理。原成交先入账,修正随后核验。待配对的消息不能占满普通成交消费批次,也不能释放旧委托占用。相同修正编号、相同正文只执行一次;正文冲突必须保留并告警。 + +修正前值必须等于当前有效成交版本。身份同时核对成交号、子委托、券商委托号、证券、方向及父指令账本。北交所不在范围内。单条SQL只访问一张表,跨表核验在服务层进行。所有写入同账户锁、同事务完成。 + +批次投影先用原有效逐笔重演,确认能还原当前批次,再应用修正重演后续卖出分配。只有完整证据才自动改账。缺失原成交、无法唯一对应的旧批次、外部或对账补入的起点、除权差异和归档旧仓不猜测归属。新普通成交原子记录成交与批次的关联,减少今后历史匹配歧义。 + +现金仅按已核实的本金差额调整原策略,不跨策略认领、透支或划拨。费用保持原事实,未知不改成零。历史报表和历史划拨净值不因此被宣称已重新计算。 + +终态保持终态,终结委托剩余量保持零。数量变化为原父指令留下续单阻断,不自动补差;价格更正不产生新的交易授权。网格等另有内部档位状态,须核实并同步,无法证明一致时整笔挂起,不只改一半账。 + +## 验证计划 + +开发机通过独立内存和隔离数据库检查价格与数量更正、买卖分配、多次更正、重复及乱序、冲突、事务失败回滚、双策略隔离和终态不复活。协议签名向量仍须通过。候选镜像断网全量检查后,只更新155实际受影响的PMS服务。WS始终只保留一个连接,不复位序号、不忽略验签。 + +发布后只读核对真实配置、镜像、连续确认、待入账和资金数量对账。没有自然修正报文时,明确记录未做自然修正判收,不发送测试成交或伪造修正补验收。188真实PMS不操作。 + +## 当前现场 + +08:54至08:55只读核验155,尚无trade_correction或带amended的委托回报。连续确认推进22条,最近200条连续,无暂存缺口、待入账或孤儿。数量和现金一致;盘前一只持仓尚缺估值行情,现金核对正常。 + +当时web为watch-flow-67b0277,worker和beat为watch-bf5e3ed,唯一WS为execution-2b1c447,行情采集为limits-0b3c50e。各服务源码与其镜像提交一致。以上为发布前事实,不代表本次功能已上线。 + +后续开发、候选检查、实际发布和自然判收分别追加,不把隔离回放算成真实业务处理。 + +## 实现补充 + +配套回报必须已越过连续持久水位,才可改变账本;同一成交的前一次更正尚未完成时,后一次留待顺序处理。数量变化还需一分钟内、且晚于两条修正回报的新QMT持仓快照,核对双策略数量合计。暂缺配对、原成交或新快照只等待;身份、前值、金额、归属或已入账证据冲突则挂起并留审计。 + +修正期间保守阻止新增买入及受影响证券的新委托,普通成交仍正常入账。终态委托不复活,旧累计回报不能覆盖已核实的修正,修正前的资金快照不能释放新的占用。数量更正永久阻断原父指令自动续单;网格同步重建档位,数量改变时暂停并等待人工安排。旧网格任务和旧提议通过修正版本校验失效,不能覆盖新状态或继续发单。 + +旧仓已经归零、后来另起持仓轮次时,不自动把旧轮修正计入新仓。旧历史只在完整重演能精确还原现账、批次唯一对应时处理;证据达到读取上限、缺少费用本金凭据或其它活动策略无法重建时挂起。更正不是费用后补入口,含手续费成本不能直接填作成交价。 + +本次复用既有收件和文档表,无表结构迁移、不新增业务开关。受影响发布目标为155的web、worker与唯一WS;beat、行情采集、微盘监控及188不在本次重建范围。 + +## 回退边界 + +只回退本次三个目标服务,保留账本、收件、修正审计和队列。尚无已应用修正时,可以恢复备份的三个原镜像和私有覆盖。若已经应用数量更正,旧版本仍可能把FILLED当作原委托全量,不能直接恢复旧执行代码;须先停止新增执行,保留接收与账本事实,核对未结委托,再发布理解修正版本的兼容修复。不得用恢复数据库或删除修正记录来回退。 + +## 开发验证 + +开发机独立环境全量回归末行为ALL SUITES PASS,日志/private/tmp/pms-correction-evidence-20261105/full-local-tests.log。最后坏回执隔离补审使用对应专项复验,最终候选须重新完成全量,不混用旧代码指纹。页面文件未修改,原页面三道护栏随全量通过。新增检查使用内存凭据、隔离SQLite事务和交错线程;不连接真实服务。 + +最终专项共134例:批次投影36、传输与执行29、服务事务36、普通成交凭据与仓储13、网格版本与并发20,全部通过。已有回放回归另增加6例,核对等待、挂起、正常成交和日终摘要;该套18例通过。 diff --git a/docs/开发入口.md b/docs/开发入口.md index 0a7159d..02ddbf1 100644 --- a/docs/开发入口.md +++ b/docs/开发入口.md @@ -15,6 +15,7 @@ | PMS通用迁移与既有修复 | [开发接续](开发接续_2026-10-12.md),按主题查对应节;旧时点未完成项可能已有后续结论 | | 盘中改价、异步研判与微盘递补发布 | [执行链路发布接续](执行报价与异步研判发布接续_2026-11-03.md),区分部署、开关与自然判收;[实施规格](执行报价与异步研判实施规格_2026-11-03.md);[当日限制价采集边界](当日限制价采集边界_2026-11-03.md) | | 策略资金缺行情提示、成本三位精度与含费口径 | [估值与成本精度修复](策略估值与成本精度修复_2026-11-04.md),含实际QMT回执、展示降级和导出舍入边界 | +| QMT工作台成交修正与配套委托回报 | [成交修正接入与发布](QMT成交修正接入与发布_2026-11-05.md),含原成交保留、修正幂等、数量变化阻断续单和证据不足挂起边界 | | 人工关注显示已剔除、每日选股重复与研究覆盖 | [人工关注分析链修复规格](人工关注分析链修复规格_2026-11-04.md),说明独立研究、回执与交易边界;[只读巡检及当日更正](人工关注与选股覆盖巡检_2026-11-04.md)保留历史时点证据;首次部署见[发布接续](人工关注与吸筹参考发布接续_2026-11-04.md),最新界面融合与真实研究进度见[后续跟进](人工关注后续跟进与信号融合_2026-11-04.md) | | 架构与接口 | [README模块地图](../README.md)、[总体设计](../POSITION_MGMT_DESIGN.md);改接口再读[QMT协议](../QMT_WS_PROTOCOL.md)、[bionic接口](../BIONIC_PMS_INTERFACE.md)或[计划接口](../UPSTREAM_PLAN_API.md) | | 后端、前端、测试、研究 | [后端规则](../app/AGENTS.md)、[页面规则](../app/web/AGENTS.md)、[脚本规则](../scripts/AGENTS.md)、[研究规则](../research/AGENTS.md),仅选择所涉及目录 | diff --git a/scripts/run_tests.py b/scripts/run_tests.py index 3ed62de..2dfa285 100644 --- a/scripts/run_tests.py +++ b/scripts/run_tests.py @@ -6,6 +6,9 @@ test_batch54_units.py 用户候选来源、到期核验、权限、显式取用与技术面补读 (9例) +成交修正专项:批次投影36例、传输与执行保护29例、服务事务36例、 +普通成交凭据与仓储13例、网格更正版本与并发20例。 + 新增限制价采集专项:core 25例、provider 21例、targets 18例、collector 28例, 缓存消费23例、独立进程入口6例。 策略资金缺行情展示与严格划拨专项18例;成本均价展示与导出精度专项14例。 @@ -30,7 +33,7 @@ test_batch42_units.py 提议先准备后采纳、模型复用、并发更新与发布审计 (24 例) test_batch41_units.py 跨流程买卖互斥、清仓收尾、出口撤单与真实进展及人工清仓T+1 (27 例) test_batch40_units.py 逐笔历史修复、进度暂停、并发与事务回滚与均价失效 (22 例) - test_batch39_units.py 成交双来源归属、迟到、部分成交与失败守成 (12 例) + test_batch39_units.py 成交双来源归属、迟到、部分成交、失败守成与更正挂起摘要 (18 例) test_batch38_units.py 每日计划顺序、版本核验、重新确认、并发采纳、历史保留及采纳失败说明、明确否决撤下 (30 例) 含在途报价变化不重复确认、真实承诺变化仍重核。 test_core_units.py 仓位规划器 / 安全垫与成本账 (14 例) @@ -196,6 +199,9 @@ import sys HERE = os.path.dirname(os.path.abspath(__file__)) ROOT = os.path.dirname(HERE) SUITES = ["test_rotation_accounting_units.py", "test_cost_precision.py", "test_execution_basis_decimal.py", + "test_trade_correction_units.py", "test_trade_correction_transport.py", + "test_trade_correction_service.py", "test_trade_lineage_units.py", + "test_trade_correction_strategy.py", "test_monitor_watch_units.py", "test_price_limits_core.py", "test_price_limit_provider.py", "test_price_limit_targets.py", "test_price_limit_collector.py", diff --git a/scripts/test_batch10_units.py b/scripts/test_batch10_units.py index d2fe142..5743778 100644 --- a/scripts/test_batch10_units.py +++ b/scripts/test_batch10_units.py @@ -50,6 +50,7 @@ ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) # 这些函数**写失败时不抛异常**, 只把失败写在返回值里。调用点必须接住。 # key = 函数名, value = 允许的调用者模块名 (None = 不限, 用于 _db(f, ...) 这种间接调用) SOFT_FAIL = { + "consume": {"trade_correction"}, "sync": {"monitor_watch"}, "mark_send_attempt": {"qmt_repo"}, "request_order_cancel": {"qmt_repo"}, diff --git a/scripts/test_batch39_units.py b/scripts/test_batch39_units.py index e386ad6..c3d2757 100644 --- a/scripts/test_batch39_units.py +++ b/scripts/test_batch39_units.py @@ -1,10 +1,11 @@ """本地:python scripts/unit_test_guard.py scripts/test_batch39_units.py。禁止联网。""" import unit_test_guard # noqa: F401 +import inspect import json import unittest from unittest.mock import patch from test_wiring import install_fakes -from app.services import ledger_service as ls +from app.services import ledger_service as ls, trade_correction, executor from app.repo import downstream_repo as ds, qmt_repo as q CODE = '600000.SH' @@ -118,6 +119,70 @@ class ReplayOwnershipTest(unittest.TestCase): self.assertFalse(ls.replay_fills()['ok']) self.assertEqual(self.qty(), 0) + def correction_result(self, *, held=0, waiting=0, errors=None): + return {'ok': not held and not errors, 'applied': 0, 'waiting': waiting, 'held': held, + 'errors': errors or [], 'alerts': ([{'type':'TRADE_CORRECTION_HELD', 'seq':77, + 'reason':'成交前版本冲突,待人工核实'}] if held else [])} + + def test_correction_hold_survives_cold_start_and_logs_real_reason(self): + self.fake.params.pop(ls.CURSOR_KEY, None) + with patch.object(trade_correction, 'consume', return_value=self.correction_result(held=1)), \ + self.assertLogs('pms.ledger', level='WARNING') as logs: + result = ls.replay_fills() + self.assertFalse(result['ok']); self.assertEqual(result['errors'], []) + self.assertEqual(result['alerts'][0]['message'], '成交前版本冲突,待人工核实') + self.assertTrue(any('成交前版本冲突' in line for line in logs.output)) + + def test_correction_hold_survives_no_fresh_summary_rows(self): + self.fills = [] + with patch.object(trade_correction, 'consume', return_value=self.correction_result(held=1)): + result = ls.replay_fills() + self.assertFalse(result['ok']); self.assertEqual(result['corrections']['held'], 1) + self.assertEqual(result['errors'], []); self.assertEqual(len(result['alerts']), 1) + + def test_correction_hold_does_not_stop_original_trade_booking(self): + self.trade() + with patch.object(trade_correction, 'consume', return_value=self.correction_result(held=1)): + result = ls.replay_fills() + self.assertFalse(result['ok']); self.assertEqual(result['errors'], []) + self.assertEqual(self.qty(), 100); self.assertEqual(result['ws']['actions'], 1) + self.assertEqual(result['alerts'][0]['message'], result['alerts'][0]['reason']) + + def test_correction_wait_is_healthy_and_original_trade_continues(self): + self.trade(); self.fills = [] + with patch.object(trade_correction, 'consume', return_value=self.correction_result(waiting=1)): + result = ls.replay_fills() + self.assertTrue(result['ok'], result); self.assertEqual(result['corrections']['waiting'], 1) + self.assertEqual(self.qty(), 100); self.assertEqual(result['errors'], []) + + def test_summary_read_failure_preserves_correction_program_error_and_hold(self): + with patch.object(trade_correction, 'consume', return_value=self.correction_result(held=1, errors=['更正数据库暂不可用'])), \ + patch.object(ds, 'fetch_filled_orders', side_effect=RuntimeError('summary unavailable')): + result = ls.replay_fills() + self.assertFalse(result['ok']); self.assertIn('更正数据库暂不可用', result['errors']) + self.assertTrue(any('summary unavailable' in error for error in result['errors'])) + self.assertEqual(len(result['alerts']), 1) + + def test_daily_summary_distinguishes_hold_wait_and_program_error(self): + from app import scheduler + run = inspect.unwrap(scheduler.daily_settle.run) + cases = [(self.correction_result(held=1), False), (self.correction_result(waiting=1), True), + (self.correction_result(errors=['数据库不可用']), False)] + for correction, expected_ok in cases: + pre = {'ok': correction['ok'], 'actions': 0, 'ws': {'actions':1}, 'corrections': correction, + 'errors': correction['errors'], 'alerts': correction['alerts']} + with patch.object(ls, 'replay_fills', return_value=pre), \ + patch.object(ls, 'daily_settle', return_value={'ok':True,'steps':{},'errors':[]}) as settle, \ + patch.object(executor, 'sweep_windows', return_value={}), \ + patch.object(ls, 'expire_stale_instructions', return_value={}): + result = run() + self.assertEqual(result['ok'], expected_ok, result); settle.assert_called_once() + summary = result['steps']['pre_replay'] + self.assertEqual(summary['corrections']['held'], correction['held']) + self.assertEqual(summary['corrections']['waiting'], correction['waiting']) + self.assertEqual(summary['alerts'], correction['alerts']) + self.assertEqual(result['errors'], correction['errors']) + if __name__ == '__main__': unittest.main() diff --git a/scripts/test_batch51_units.py b/scripts/test_batch51_units.py index d1c24ae..607e100 100644 --- a/scripts/test_batch51_units.py +++ b/scripts/test_batch51_units.py @@ -10,7 +10,7 @@ from datetime import datetime, timedelta from unittest.mock import patch from sqlalchemy import create_engine, text, event from app.db import session -from app.repo import pms_repo, qmt_repo +from app.repo import pms_repo, qmt_repo, execution_repo from app.services import daily_buy_review as dr, proposal_service as ps, dispatcher, execution_conflicts, executor NOW = datetime(2026, 10, 30, 10, 0) @@ -36,6 +36,7 @@ class DailyReviewTests(unittest.TestCase): event.listen(self.engine, 'connect', register_json_functions) self.use(session, 'get_engine', return_value=self.engine) self.use(session, '_check_cooldown') + self.use(execution_repo, 'pending_corrections', return_value=0) self.params = {'PMS_DAILY_BUY_REVIEW': True} self.use(dr.param_store, 'get_bool', side_effect=lambda k, d=False: self.params.get(k, d)) self.orders = [] diff --git a/scripts/test_batch52_units.py b/scripts/test_batch52_units.py index 6b96562..32fb063 100644 --- a/scripts/test_batch52_units.py +++ b/scripts/test_batch52_units.py @@ -12,7 +12,7 @@ from unittest.mock import AsyncMock, patch sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) from sqlalchemy import create_engine, text from app.db import session -from app.repo import qmt_repo, pms_repo +from app.repo import qmt_repo, pms_repo, execution_repo from app.services import executor, dispatcher from app.ws.runner import WsRunner @@ -26,6 +26,8 @@ class ChannelTest(unittest.TestCase): 'status': 'ACCEPTED', 'cancel_state': 'SENT', 'ts_code': '688100.SH'} self.use(qmt_repo, 'get_order', side_effect=lambda _: copy.deepcopy(self.row)) self.use(qmt_repo, 'update_order', side_effect=self.update) + self.use(execution_repo, 'pending_corrections', return_value=0) + self.use(pms_repo, 'get_instruction', return_value={'instruction_id': 'P', 'progress': {}}) def use(self, obj, name, **kw): p = patch.object(obj, name, **kw); m = p.start(); self.addCleanup(p.stop); return m @@ -199,7 +201,7 @@ class SqlTest(unittest.TestCase): class ZBookTest(unittest.TestCase): def test_late_trade_after_parent_cancel_books_once_in_own_strategy(self): from test_wiring import install_fakes - from app.services import rotation_execution as re, rotation_accounting as ra + from app.services import rotation_execution as re, rotation_accounting as ra, trade_lineage from app.repo import rotation_repo fake = install_fakes(prices={'688100.SH': 23.3}) fake.insert_instruction(instruction_id='P', origin_type='proposal', origin_id='X', @@ -208,6 +210,7 @@ class ZBookTest(unittest.TestCase): fake.qmt.put_trade(seq=1, instruction_id='P_D01', ts_code='688100.SH', side='buy', qty=100, price=23.3) act = {'instruction_id': 'P', 'kind': 'BUY', 'ts_code': '688100.SH', 'qty': 100, 'price': 23.3, 'lot_type': 'BASE', 'order_id': 'T1'} with patch.object(re.db, 'transaction', side_effect=nullcontext), \ + patch.object(trade_lineage, 'record'), \ patch.object(rotation_repo, 'account_guard', side_effect=nullcontext), \ patch.object(re.db, 'fetch_one', side_effect=lambda *a, **kw: {'processed': fake.qmt.inbox[1]['processed']}), \ patch.object(pms_repo, 'get_instruction', side_effect=lambda iid: copy.deepcopy(fake.instructions.get(iid))), \ diff --git a/scripts/test_batch53_units.py b/scripts/test_batch53_units.py index dadfd8a..d9ec572 100644 --- a/scripts/test_batch53_units.py +++ b/scripts/test_batch53_units.py @@ -65,6 +65,11 @@ class StrategyOutboxTest(unittest.TestCase): return copy.deepcopy(parent) try: with ExitStack() as stack: + from contextlib import nullcontext + from app.services import execution_orders + from app.repo import execution_repo + stack.enter_context(patch.object(execution_orders, 'account_guard', side_effect=nullcontext)) + stack.enter_context(patch.object(execution_repo, 'pending_corrections', return_value=0)) for obj, name, kw in [ (settings, 'INFRA_STRATEGY_BOOK_SCHEMA', {'new': True}), (qmt_repo, 'next_queued', {'return_value': [row]}), @@ -95,7 +100,8 @@ class StrategyOutboxTest(unittest.TestCase): self.assertEqual(send.call_args.args[:2], ('place_order', dict(BASE, side=side, strategy_type=expected))) self.assertEqual(send.call_args.kwargs, {'corr_id': BASE['instruction_id']}) - self.assertEqual(scopes, ['all']) + self.assertTrue(scopes) + self.assertEqual(set(scopes), {'all'}) # 编码与发送前复核均须按持久父指令归属。 retry.assert_not_called() def test_missing_or_unknown_parent_keeps_queue_without_send(self): diff --git a/scripts/test_batch56_units.py b/scripts/test_batch56_units.py index ffbec3f..b94cd1b 100644 --- a/scripts/test_batch56_units.py +++ b/scripts/test_batch56_units.py @@ -339,6 +339,12 @@ class ServiceBoundaries(unittest.TestCase): class ReservationTests(unittest.TestCase): + def setUp(self): + from app.repo import execution_repo + stub = patch.object(execution_repo, 'pending_corrections', return_value=0) + stub.start() + self.addCleanup(stub.stop) + def release(self, *, filled=0, fresh=True, booked=True): from app.services import rotation_execution as r, execution_orders as e now=dt.datetime(2026,11,3,9,40) diff --git a/scripts/test_batch6_units.py b/scripts/test_batch6_units.py index aac5066..020effb 100644 --- a/scripts/test_batch6_units.py +++ b/scripts/test_batch6_units.py @@ -382,11 +382,13 @@ def run(): for s in ("ACCEPTED", "SUBMITTED", "PARTIAL", ""): assert not wsc.is_final(s), s - @case("只有 trade 需要 worker 入账") + @case("trade及修正由worker入账,普通order_update仅跟踪状态") def _(): - eq(tuple(wsc.NEEDS_LEDGER), ("trade",)) + eq(tuple(wsc.NEEDS_LEDGER), ("trade", "trade_correction")) assert wsc.T_ORDER_UPDATE not in wsc.NEEDS_LEDGER, \ "order_update 只作状态跟踪 —— 拿它入账会和逐笔 trade 重复计数" + assert wsc.needs_ledger('order_update', {'amended': True}) + assert not wsc.needs_ledger('order_update', {'status': 'FILLED'}) print("\n[F2] 持仓快照解析 (§5.7, 对账的事实源)") diff --git a/scripts/test_trade_correction_service.py b/scripts/test_trade_correction_service.py new file mode 100644 index 0000000..c420b57 --- /dev/null +++ b/scripts/test_trade_correction_service.py @@ -0,0 +1,436 @@ +"""成交更正服务:虚构回执、双策略隔离和SQLite事务,禁止真实服务。""" +import unit_test_guard # noqa: F401 +import copy +import datetime as dt +import json +from decimal import Decimal +from pathlib import Path +import sys +import tempfile +import unittest +from unittest.mock import patch +from types import SimpleNamespace + +sys.path.insert(0, str(Path(__file__).resolve().parents[1])) +from sqlalchemy import create_engine, event, text +from app.core import strategy_book as books, ws_codec +from app.db import session as db +from app.repo import pms_repo, qmt_repo, rotation_repo as store +from app.services import trade_correction as service +from config.settings import settings +from test_trade_correction_units import fixture, economic, trade, lot + +CODE = '600001.SH' +NOW = dt.datetime(2026, 11, 5, 12, tzinfo=service.TZ) + + +class ServiceTests(unittest.TestCase): + def setUp(self): + self.tmp = tempfile.TemporaryDirectory(); self.addCleanup(self.tmp.cleanup) + self.engine = create_engine('sqlite:///'+self.tmp.name+'/correction.sqlite'); self.addCleanup(self.engine.dispose) + class Clock(dt.datetime): + @classmethod + def now(cls, tz=None): return NOW if tz else NOW.replace(tzinfo=None) + self.use(service, 'dt', SimpleNamespace(datetime=Clock)) + self.use(settings, 'INFRA_STRATEGY_BOOK_SCHEMA', True) + self.use(db, 'get_engine', lambda *args: self.engine) + self.use(db, '_check_cooldown', lambda *args: None) + @event.listens_for(self.engine, 'connect') + def funcs(conn, _): + def path(value, mode, key): + return int(key.lstrip('$.') in json.loads(value or '{}')) + conn.create_function('JSON_CONTAINS_PATH', 3, path) + @event.listens_for(self.engine, 'before_cursor_execute', retval=True) + def sqlite(conn, cursor, sql, params, ctx, many): + sql = sql.replace(' FOR UPDATE', '') + if 'ON DUPLICATE KEY UPDATE' in sql: + a, _ = sql.split('ON DUPLICATE KEY UPDATE', 1) + self.assertIn('pms_rotation_document', a) + sql = a+'ON CONFLICT(doc_id) DO NOTHING' + def serial(v): + return str(v) if isinstance(v, Decimal) else v.isoformat(' ') if isinstance(v, dt.datetime) else v + if isinstance(params, tuple): params = tuple(serial(v) for v in params) + return sql, params + schema = { + 'pms_qmt_inbox': 'seq INTEGER PRIMARY KEY,msg_id TEXT,msg_type TEXT,corr_id TEXT,dedup_key TEXT,payload_json TEXT,msg_ts INTEGER,received_at TEXT,processed INTEGER,processed_at TEXT,process_note TEXT', + 'pms_qmt_order': 'id INTEGER PRIMARY KEY,instruction_id TEXT,parent_id TEXT,ts_code TEXT,side TEXT,qty INTEGER,limit_price REAL,status TEXT,broker_order_id TEXT,cum_qty INTEGER,cum_avg_price REAL,leaves_qty INTEGER,updated_at TEXT,final_at TEXT', + 'pms_instruction': 'id INTEGER PRIMARY KEY,book_id TEXT,instruction_id TEXT,origin_type TEXT,origin_id TEXT,ts_code TEXT,action TEXT,side TEXT,qty INTEGER,status TEXT,exec_qty INTEGER,exec_avg_price REAL,progress_json TEXT,archived_at TEXT,created_at TEXT,updated_at TEXT', + 'pms_lot': 'id INTEGER PRIMARY KEY,book_id TEXT,ts_code TEXT,instruction_id TEXT,lot_type TEXT,qty INTEGER,closed_qty INTEGER,open_price REAL,close_avg_price REAL,realized_pnl REAL,status TEXT,open_date TEXT,created_at TEXT,updated_at TEXT,archived_at TEXT', + 'pms_position': 'id INTEGER PRIMARY KEY,book_id TEXT,ts_code TEXT,total_qty INTEGER,avail_qty INTEGER,base_qty INTEGER,fill_qty INTEGER,add_qty INTEGER,dca_qty INTEGER,t0_qty INTEGER,avg_cost REAL,realized_t_profit REAL,cushion_pct REAL,cushion_state TEXT,cushion_peak REAL,pct_of_scale REAL,status TEXT,archived_at TEXT,updated_at TEXT', + 'pms_rotation_document': 'doc_id TEXT PRIMARY KEY,kind TEXT,payload_json TEXT,revision INTEGER,updated_at TEXT', + 'pms_strategy': 'id INTEGER PRIMARY KEY,book_id TEXT,ts_code TEXT,strategy_id TEXT,type TEXT,status TEXT,state_json TEXT,params_json TEXT,archived_at TEXT,updated_at TEXT', + 'pms_plan': 'id INTEGER PRIMARY KEY,book_id TEXT,plan_id TEXT,ts_code TEXT,priority INTEGER,status TEXT,filled_qty INTEGER,updated_at TEXT', + } + with self.engine.begin() as conn: + for table, columns in schema.items(): conn.execute(text('CREATE TABLE '+table+'('+columns+')')) + self.use(qmt_repo, 'latest_snapshot', lambda kind: self.snapshot) + self.watermark = 1000 + self.use(qmt_repo, 'get_state', lambda: {'last_seq': self.watermark}) + self.snapshot = {'seq': 1000, 'age_sec': 1, 'received_at': NOW.isoformat(), 'payload': {'kind': 'positions', 'items': [{'ts_code': CODE, 'total_qty': 100, 'avail_qty': 0}]}} + # 费用配置从参数读取的路径本身不参与更正;隔离固定合法费率,禁止真实参数库。 + from app.services import rotation_service + self.use(rotation_service, 'configuration', lambda: {'costs': {'rate': .0002, 'minimum': 5., 'sell_tax': .0005, 'transfer_rate': .00001}}) + self.seed() + + def use(self, obj, name, value): + patcher = patch.object(obj, name, value); patcher.start(); self.addCleanup(patcher.stop) + + def insert(self, table, values): + with self.engine.begin() as conn: + conn.execute(text('INSERT INTO '+table+' ('+','.join(values)+') VALUES ('+','.join(':'+k for k in values)+')'), values) + + def sql(self, sql, params=None): + with self.engine.begin() as conn: return conn.execute(text(sql), params or {}) + + def seed(self, sold=False, book='original'): + with self.engine.begin() as conn: + for table in ('pms_qmt_inbox','pms_qmt_order','pms_instruction','pms_lot','pms_position','pms_rotation_document','pms_strategy','pms_plan'): + conn.execute(text('DELETE FROM '+table)) + data = fixture(sold) + fills = {} + for i, row in enumerate(data['trades'], 1): + payload = copy.deepcopy(row['payload']); payload.pop('parent_id') + pid = row['payload']['parent_id']; iid = payload['instruction_id'] + self.insert('pms_qmt_inbox', dict(seq=row['seq'],msg_id='M'+str(i),msg_type='trade',corr_id=iid,dedup_key=ws_codec.dedup_key('trade',payload),payload_json=json.dumps(payload),msg_ts=1,received_at=row['processed_at'],processed=1,processed_at=row['processed_at'],process_note='原始成交')) + self.insert('pms_qmt_order',dict(id=i,instruction_id=iid,parent_id=pid,ts_code=CODE,side=payload['side'],qty=200,limit_price=20,status='FILLED',broker_order_id=payload['broker_order_id'],cum_qty=payload['qty'],cum_avg_price=float(payload['price']),leaves_qty=0)) + self.insert('pms_instruction',dict(id=i,book_id=book,instruction_id=pid,origin_type='system',origin_id='other',ts_code=CODE,action='OPEN' if payload['side']=='buy' else 'EXIT',side=payload['side'],qty=200,status='CONFIRMED',exec_qty=payload['qty'],exec_avg_price=float(payload['price']),progress_json='{}',archived_at='2026-11-05 11:00:00')) + fills[str(row['seq'])] = dict(book=book,side=payload['side'],amount=float(payload['amount']),fee=0.) + for lot in data['lots']: + lot['book_id']=book + self.insert('pms_lot',lot) + self.insert('pms_position',dict(id=1,book_id=book,ts_code=CODE,total_qty=60 if sold else 100,avail_qty=60 if sold else 0,base_qty=60 if sold else 100,fill_qty=0,add_qty=0,dca_qty=0,t0_qty=0,avg_cost=8.667 if sold else 10.,realized_t_profit=0.,cushion_pct=.2,cushion_state='SOLID',cushion_peak=.3,pct_of_scale=.1,status='HOLDING')) + cash = {'original': 10000., 'micro30': 10000.} + capital = {'cash':cash,'fills':fills,'fees':{'original':0.,'micro30':0.},'units':{'original':20000.,'micro30':10000.},'net_capital':{'original':20000.,'micro30':10000.},'transfers':{}} + self.insert('pms_rotation_document',dict(doc_id='capital:live',kind='capital',payload_json=json.dumps(capital),revision=1)) + self.snapshot['payload']['items'][0]['total_qty']=60 if sold else 100 + self.after = economic(100, '11.000'); self.add_pair('C1', 'BUY', self.after) + + def add_pair(self, cid, no, after, seq=10, before=None, amended=True): + original = service.repo.get_trade(no); p = original['payload'] + before = before or {k:p[k] for k in service.ECONOMIC} + correction = {k:p[k] for k in service.IDENTITY}; correction.update(correction_id=cid,before=before,after=after,reason='券商成交事实更正',correct_time=int(NOW.timestamp()*1000)) + am = {'instruction_id':p['instruction_id'],'broker_order_id':p['broker_order_id'],'correction_id':cid,'amended':amended,'reason':'trade_correction','status':'FILLED','final':True,'leaves_qty':0,'cum_qty':after['qty'],'cum_avg_price':after['price'] if after['qty'] else None} + for offset, kind, payload in ((0,'trade_correction',correction),(1,'order_update',am)): + self.insert('pms_qmt_inbox',dict(seq=seq+offset,msg_id='M'+str(seq+offset),msg_type=kind,corr_id=p['instruction_id'],dedup_key=None,payload_json=json.dumps(payload),msg_ts=1,received_at='2026-11-05T11:00:00+08:00',processed=0)) + return correction, am + + def modify_payload(self, seq, **change): + row=service.repo.get_inbox(seq);payload=row['payload'];payload.update(change) + self.sql('UPDATE pms_qmt_inbox SET payload_json=:p WHERE seq=:s',{'p':json.dumps(payload),'s':seq}) + + def state(self): + result={} + for table in ('pms_qmt_inbox','pms_qmt_order','pms_instruction','pms_lot','pms_position','pms_rotation_document','pms_strategy','pms_plan'): + with self.engine.connect() as conn: result[table]=[dict(x) for x in conn.execute(text('SELECT * FROM '+table+' ORDER BY 1')).mappings()] + return result + + def assert_applied(self): + result=service.consume();self.assertEqual(result['applied'],1,result);self.assertTrue(result['ok'],result) + return result + + def test_buy_price_delta_keeps_original_fees_and_parent_terminal(self): + before=self.state();self.assert_applied();after=self.state() + self.assertEqual(after['pms_qmt_inbox'][0],before['pms_qmt_inbox'][0]) + self.assertEqual([r['processed'] for r in after['pms_qmt_inbox']], [1,1,1]) + pos=pms_repo.get_position(CODE);self.assertEqual(pos['avg_cost'],11.);self.assertEqual(pos['cushion_peak'],0) + parent=pms_repo.get_instruction('P_BUY');self.assertEqual(parent['status'],'CONFIRMED');self.assertIsNotNone(parent['archived_at']) + capital=store.get('capital:live')[0];self.assertEqual(capital['cash']['original'],9900.) + for key in ('fills','fees','units','net_capital','transfers'): self.assertEqual(capital[key],json.loads(before['pms_rotation_document'][0]['payload_json'])[key]) + audit=store.get(service._key('trade-correction:','C1'))[0] + self.assertIsNone(audit['original_fee']);self.assertEqual(audit['position_before']['cushion_peak'],.3) + + def test_chain_uses_effective_before_and_persisted_lot_link(self): + self.assert_applied();self.add_pair('C2','BUY',economic(100,'12'),seq=20,before=self.after) + self.assert_applied();self.assertEqual(pms_repo.get_position(CODE)['avg_cost'],12.) + self.assertEqual(store.get('capital:live')[0]['cash']['original'],9800.) + self.assertEqual(service.repo.get_trade('BUY')['payload']['price'],'10.000') + + def test_duplicate_same_id_is_not_applied_twice(self): + self.assert_applied();cash=store.get('capital:live')[0] + self.add_pair('C1','BUY',self.after,seq=20) + result=service.consume();self.assertEqual(result['applied'],0,result) + self.assertEqual(store.get('capital:live')[0],cash) + self.assertEqual([service.repo.get_inbox(i)['processed'] for i in (20,21)], [2,2]) + + def test_reused_id_changed_content_is_held(self): + self.assert_applied();self.add_pair('C1','BUY',economic(100,'12'),seq=20) + result=service.consume();self.assertEqual(result['held'],1,result) + self.assertEqual(pms_repo.get_position(CODE)['avg_cost'],11.) + + def test_missing_pair_or_unbooked_original_waits_without_blocking_original(self): + for missing in ('pair','original'): + self.seed() + if missing=='pair': self.sql('DELETE FROM pms_qmt_inbox WHERE seq=11') + else: self.sql('UPDATE pms_qmt_inbox SET processed=0 WHERE seq=1') + result=service.consume();self.assertEqual(result['waiting'],1,result) + self.assertEqual(service.repo.get_inbox(10)['processed'],0) + self.assertEqual(service.repo.get_inbox(1)['processed'],0 if missing=='original' else 1) + + def test_before_pair_identity_average_and_original_cash_conflicts_hold(self): + changes=[lambda:self.modify_payload(10,before=economic(100,'9')), + lambda:self.modify_payload(11,broker_order_id='different'), + lambda:self.modify_payload(11,cum_avg_price='12'), + lambda:self.modify_payload(11,amended=False), + lambda:self.modify_payload(10,side='sell'), + lambda:self.sql("UPDATE pms_rotation_document SET payload_json=json_set(payload_json,'$.fills.1.amount',999) WHERE doc_id='capital:live'")] + for change in changes: + self.seed();change();before=self.state();result=service.consume() + self.assertEqual(result['held'],1,result) + after=self.state() + for table in ('pms_lot','pms_position','pms_instruction','pms_qmt_order'):self.assertEqual(after[table],before[table]) + self.assertEqual(service.repo.get_inbox(10)['processed'],3) + + def test_quantity_requires_fresh_later_snapshot_and_pauses_parent_remaining(self): + self.modify_payload(10,after=economic(150,'10'));self.modify_payload(11,cum_qty=150,cum_avg_price=10) + self.snapshot['payload']['items'][0]['total_qty']=150 + self.assert_applied();pos=pms_repo.get_position(CODE) + self.assertEqual((pos['total_qty'],pos['avail_qty']),(150,0)) + parent=pms_repo.get_instruction('P_BUY');self.assertIn('trade_correction_hold',parent['progress']) + self.assertEqual(parent['status'],'CONFIRMED') + self.assertEqual(qmt_repo.get_order('P_BUY_D01')['leaves_qty'],0) + + def test_stale_missing_or_malformed_snapshot_does_not_guess_zero(self): + for snapshot in (None, {'age_sec':61,'received_at':NOW.isoformat(),'payload':{'items':[]}}, + {'age_sec':1,'received_at':NOW.isoformat(),'payload':{}}, + {'age_sec':1,'received_at':'2026-11-05T10:00:00+08:00','payload':{'items':[]}}): + self.seed();self.modify_payload(10,after=economic(0,'0'));self.modify_payload(11,cum_qty=0,cum_avg_price=None) + self.snapshot=snapshot + result=service.consume();expected='held' if snapshot and snapshot.get('payload')=={} else 'waiting' + self.assertEqual(result[expected],1,result) + self.assertEqual(pms_repo.get_position(CODE)['total_qty'],100) + self.snapshot={'seq':1000,'age_sec':1,'received_at':NOW.isoformat(),'payload':{'items':[{'ts_code':CODE,'total_qty':100}]}} + + def test_zero_correction_accepts_explicit_empty_snapshot_and_none_average(self): + self.modify_payload(10,after=economic(0,'0'));self.modify_payload(11,cum_qty=0,cum_avg_price=None) + self.snapshot['payload']['items']=[] + self.assert_applied();self.assertEqual(pms_repo.get_position(CODE)['total_qty'],0) + self.assertEqual(store.get('capital:live')[0]['cash']['original'],11000.) + + def test_after_sell_projection_reallocates_pnl_and_cash(self): + self.seed(sold=True);self.assert_applied() + lot=pms_repo.list_lots(CODE,status=None)[0] + self.assertEqual((lot['qty'],lot['realized_pnl']),(60,40.)) + self.assertAlmostEqual(pms_repo.get_position(CODE)['avg_cost'],10.333) + + def test_other_book_same_code_is_not_modified(self): + self.insert('pms_position',dict(id=2,book_id='micro30',ts_code=CODE,total_qty=50,avail_qty=50,status='HOLDING')) + self.modify_payload(10,after=economic(150,'10'));self.modify_payload(11,cum_qty=150,cum_avg_price=10) + self.snapshot['payload']['items'][0]['total_qty']=200 + self.assert_applied() + with books.scope('micro30'): self.assertEqual(pms_repo.get_position(CODE)['total_qty'],50) + + def test_cash_overdraft_and_reserved_budget_are_held(self): + self.sql("UPDATE pms_rotation_document SET payload_json=json_set(payload_json,'$.cash.original',50) WHERE doc_id='capital:live'") + result=service.consume();self.assertEqual(result['held'],1,result) + self.seed() + self.insert('pms_qmt_order',dict(id=5,instruction_id='WAIT_D01',parent_id='P_BUY',ts_code=CODE,side='buy',qty=500,limit_price=20,status='SUBMITTED',cum_qty=0)) + result=service.consume();self.assertEqual(result['held'],1,result) + + def grid(self, pending=False, type_='GRID'): + self.sql("UPDATE pms_instruction SET origin_id='GRID1',progress_json=:p WHERE instruction_id='P_BUY'",{'p':json.dumps({'leg':'grid_buy:2'})}) + self.insert('pms_strategy',dict(id=1,book_id='original',ts_code=CODE,strategy_id='GRID1',type=type_,status='ACTIVE',state_json=json.dumps({'pending':{'iid':'x'} if pending else None,'filled_levels':{'2':{'qty':100,'price':10}},'invested':1000}),params_json='{}')) + + def test_system_origin_grid_price_sync_and_quantity_pause(self): + self.grid();self.assert_applied();s=pms_repo.get_strategy('GRID1') + self.assertEqual(s['status'],'ACTIVE');self.assertEqual(s['state']['filled_levels']['2']['price'],11) + self.add_pair('C2','BUY',economic(100,'12'),seq=20,before=self.after) + self.assert_applied();s=pms_repo.get_strategy('GRID1') + self.assertEqual(s['state']['trade_correction']['correction_id'],'C2') + self.assertEqual(s['state']['filled_levels']['2']['price'],12) + self.seed();self.grid();self.modify_payload(10,after=economic(150,'10'));self.modify_payload(11,cum_qty=150,cum_avg_price=10) + self.snapshot['payload']['items'][0]['total_qty']=150 + self.assert_applied();s=pms_repo.get_strategy('GRID1') + self.assertEqual(s['status'],'PAUSED');self.assertIn('correction_hold',s['state']) + self.assertEqual(s['state']['filled_levels']['2']['qty'],150) + + def test_pending_or_unknown_strategy_holds_whole_correction(self): + for pending,type_ in ((True,'GRID'),(False,'T0')): + self.seed();self.grid(pending,type_);before=self.state();result=service.consume() + self.assertEqual(result['held'],1,result) + for table in ('pms_lot','pms_position','pms_strategy'):self.assertEqual(self.state()[table],before[table]) + + def test_final_write_failure_rolls_back_all_projection_and_documents(self): + # 账户锁哨兵预先存在,确保比较的不是建锁行副作用。 + with store.account_guard(): pass + before=self.state() + original=qmt_repo.inbox_mark + def fail(seqs,processed=1,note=None): + if processed==qmt_repo.PROC_BOOKED: raise RuntimeError('injected final marker failure') + return original(seqs,processed=processed,note=note) + self.use(qmt_repo,'inbox_mark',fail) + result=service.consume();self.assertTrue(result['errors'],result) + self.assertEqual(self.state(),before) + + def test_soft_zero_write_rolls_back_and_keeps_pending(self): + with store.account_guard(): pass + before=self.state();self.use(pms_repo,'update_position',lambda *a,**k:0) + result=service.consume();self.assertTrue(result['errors'],result);self.assertEqual(self.state(),before) + + def test_plan_progress_changes_without_reviving_done_plan(self): + self.sql("UPDATE pms_instruction SET origin_type='plan',origin_id='PLAN1' WHERE instruction_id='P_BUY'") + self.insert('pms_plan',dict(id=1,book_id='original',plan_id='PLAN1',ts_code=CODE,priority=1,status='DONE',filled_qty=100)) + self.assert_applied();self.assertEqual(pms_repo.list_plans()[0]['status'],'DONE') + + def test_fresh_snapshot_arrival_retries_original_pending_correction(self): + self.modify_payload(10,after=economic(150,'10'));self.modify_payload(11,cum_qty=150,cum_avg_price=10) + self.snapshot['age_sec']=61 + first=service.consume();self.assertEqual(first['waiting'],1,first) + self.assertEqual(service.repo.get_inbox(10)['processed'],0) + self.snapshot['age_sec']=1;self.snapshot['payload']['items'][0]['total_qty']=150 + self.assert_applied() + + def test_terminal_reclassification_preserves_old_terminal_and_final_time(self): + self.sql("UPDATE pms_qmt_order SET final_at='2026-11-05 10:00:00' WHERE id=1") + self.modify_payload(11,status='CANCELLED') + self.assert_applied();child=qmt_repo.get_order('P_BUY_D01') + self.assertEqual(child['status'],'FILLED');self.assertEqual(child['final_at'],'2026-11-05 10:00:00') + self.assertEqual(store.get(service._key('trade-correction:','C1'))[0]['qmt_amended_status'],'CANCELLED') + + def test_terminal_cannot_become_active_or_regain_leaves(self): + self.modify_payload(11,status='PARTIAL',final=False,leaves_qty=100) + result=service.consume();self.assertEqual(result['held'],1,result) + self.assertEqual(qmt_repo.get_order('P_BUY_D01')['status'],'FILLED') + + def test_active_child_can_finish_without_reviving_parent(self): + self.sql("UPDATE pms_qmt_order SET status='PARTIAL',leaves_qty=100 WHERE id=1") + self.modify_payload(11,status='CANCELLED') + self.assert_applied();child=qmt_repo.get_order('P_BUY_D01') + self.assertEqual(child['status'],'CANCELLED');self.assertIsNotNone(child['final_at']) + self.assertEqual(pms_repo.get_instruction('P_BUY')['status'],'CONFIRMED') + + def test_two_decimal_cumulative_average_never_replaces_true_trade_price(self): + self.modify_payload(10,after=economic(100,'10.125'));self.modify_payload(11,cum_avg_price='10.13') + self.assert_applied();self.assertEqual(pms_repo.get_position(CODE)['avg_cost'],10.125) + self.assertEqual(store.get('capital:live')[0]['cash']['original'],9987.5) + + def test_malformed_economics_time_reason_and_nan_are_held_with_raw_audit(self): + for change in ({'before':[1,2]}, {'after':'bad'}, {'reason':''}, {'correct_time':True}, {'correct_time':'12'}, + {'after':{'qty':100,'price':float('nan'),'amount':1000}}): + self.seed();self.modify_payload(10,**change) + result=service.consume();self.assertEqual(result['held'],1,result) + audit=store.get(service._key('trade-correction-hold:', '10'))[0] + self.assertEqual(len(audit['raw_payload_fingerprint']),64) + + def test_unproven_t_profit_and_disabled_book_schema_hold(self): + self.sql('UPDATE pms_position SET realized_t_profit=1 WHERE id=1') + result=service.consume();self.assertEqual(result['held'],1,result) + self.seed();self.use(settings,'INFRA_STRATEGY_BOOK_SCHEMA',False) + result=service.consume();self.assertEqual(result['held'],1,result) + + def test_micro_book_price_correction_does_not_use_original_cash(self): + self.seed(book='micro30');self.assert_applied() + capital=store.get('capital:live')[0] + self.assertEqual(capital['cash'],{'original':10000.,'micro30':9900.}) + + def test_original_raw_principal_must_match_cash_ledger_not_rounded_guess(self): + self.modify_payload(1,qty=1,price='48.285',amount='48.29') + self.sql("UPDATE pms_qmt_order SET cum_qty=1 WHERE id=1") + self.sql("UPDATE pms_instruction SET exec_qty=1 WHERE id=1") + self.sql("UPDATE pms_lot SET qty=1,open_price=48.285 WHERE id=1") + self.sql("UPDATE pms_position SET total_qty=1,base_qty=1,avg_cost=48.285 WHERE id=1") + self.sql("UPDATE pms_rotation_document SET payload_json=json_set(payload_json,'$.fills.1.amount',48.285) WHERE doc_id='capital:live'") + self.modify_payload(10,before=economic(1,'48.285'),after=economic(1,'48.290')) + self.modify_payload(11,cum_qty=1,cum_avg_price='48.290') + result=service.consume();self.assertEqual(result['held'],1,result) + self.assertIn('现金凭据',result['alerts'][0]['reason']) + + def test_old_zeroed_cycle_cannot_change_new_position(self): + self.seed(sold=True) + self.modify_payload(2,qty=100,amount='1200.00') + self.sql('UPDATE pms_instruction SET exec_qty=100 WHERE id=2') + self.sql('UPDATE pms_qmt_order SET cum_qty=100 WHERE id=2') + self.sql("UPDATE pms_lot SET qty=0,closed_qty=100,realized_pnl=200,status='CLOSED' WHERE id=1") + row=trade('NEW',3,'buy',100,'20');raw=copy.deepcopy(row['payload']);raw.pop('parent_id') + self.insert('pms_qmt_inbox',dict(seq=3,msg_id='M3',msg_type='trade',corr_id=raw['instruction_id'],dedup_key=ws_codec.dedup_key('trade',raw),payload_json=json.dumps(raw),processed=1,processed_at=row['processed_at'])) + self.insert('pms_qmt_order',dict(id=3,instruction_id=raw['instruction_id'],parent_id='P_NEW',ts_code=CODE,side='buy',qty=100,limit_price=20,status='FILLED',broker_order_id=raw['broker_order_id'],cum_qty=100,cum_avg_price=20,leaves_qty=0)) + self.insert('pms_instruction',dict(id=3,book_id='original',instruction_id='P_NEW',origin_type='system',origin_id='other',ts_code=CODE,side='buy',qty=100,status='CONFIRMED',exec_qty=100,progress_json='{}')) + self.insert('pms_lot',lot(row,lid=2)) + self.sql('UPDATE pms_position SET total_qty=100,avail_qty=0,avg_cost=20,base_qty=100 WHERE id=1') + before=self.state();result=service.consume();self.assertEqual(result['held'],1,result) + self.assertIn('轮',result['alerts'][0]['reason']) + self.assertEqual(self.state()['pms_lot'],before['pms_lot']) + + def test_non_object_payload_is_held_without_blocking_next_valid_row(self): + self.sql("UPDATE pms_qmt_inbox SET payload_json='[]' WHERE seq=10") + result=service.consume();self.assertEqual(result['held'],1,result) + self.assertEqual(service.repo.get_inbox(10)['processed'],3) + + def test_contiguous_watermark_must_cover_both_correction_messages(self): + for mark in (9,10): + self.watermark=mark;result=service.consume() + self.assertEqual(result['waiting'],1,result) + self.assertEqual(service.repo.get_inbox(10)['processed'],0) + self.assertEqual(pms_repo.get_position(CODE)['avg_cost'],10.) + self.watermark=11;self.assert_applied() + + def test_later_chained_pair_waits_for_earlier_unpaired_correction(self): + amendment=service.repo.get_inbox(11) + self.sql('DELETE FROM pms_qmt_inbox WHERE seq=11') + self.add_pair('C2','BUY',economic(100,'12'),seq=20,before=self.after) + result=service.consume();self.assertEqual(result['waiting'],2,result);self.assertEqual(result['held'],0,result) + self.assertEqual(pms_repo.get_position(CODE)['avg_cost'],10.) + amendment.pop('payload');self.insert('pms_qmt_inbox',amendment) + result=service.consume();self.assertEqual(result['applied'],2,result) + self.assertEqual(pms_repo.get_position(CODE)['avg_cost'],12.) + + def test_decimal_physical_quantity_cannot_be_truncated_into_matching_fact(self): + self.modify_payload(10,after=economic(150,'10'));self.modify_payload(11,cum_qty=150,cum_avg_price=10) + self.snapshot['payload']['items'][0]['total_qty']='150.5' + result=service.consume();self.assertEqual(result['held'],1,result) + self.assertEqual(pms_repo.get_position(CODE)['total_qty'],100) + + def test_sell_price_and_quantity_use_cash_delta_and_actual_available_quantity(self): + for qty, price, expected_cash, expected_qty in ((40,'13',10040.,60),(50,'12',10120.,50)): + self.seed(sold=True);self.sql('DELETE FROM pms_qmt_inbox WHERE seq IN (10,11)') + self.add_pair('SELL-C','SELL',economic(qty,price)) + self.snapshot['payload']['items'][0]['total_qty']=expected_qty + self.assert_applied();pos=pms_repo.get_position(CODE) + self.assertEqual((pos['total_qty'],pos['avail_qty']),(expected_qty,expected_qty)) + self.assertEqual(store.get('capital:live')[0]['cash']['original'],expected_cash) + self.assertEqual(service.repo.get_trade('SELL')['payload']['qty'],40) + + def test_child_cumulative_and_parent_all_children_are_distinct_aggregates(self): + row=trade('SECOND',2,'buy',50,'12',parent='P_BUY') + row['payload']['instruction_id']='P_BUY_D02';row['original_payload']=copy.deepcopy(row['payload']) + raw=copy.deepcopy(row['payload']);raw.pop('parent_id') + self.insert('pms_qmt_inbox',dict(seq=2,msg_id='M2',msg_type='trade',corr_id=raw['instruction_id'],dedup_key=ws_codec.dedup_key('trade',raw),payload_json=json.dumps(raw),processed=1,processed_at=row['processed_at'])) + self.insert('pms_qmt_order',dict(id=2,instruction_id='P_BUY_D02',parent_id='P_BUY',ts_code=CODE,side='buy',qty=50,limit_price=12,status='FILLED',broker_order_id=raw['broker_order_id'],cum_qty=50,cum_avg_price=12,leaves_qty=0)) + self.insert('pms_lot',lot(row,lid=2)) + self.sql('UPDATE pms_instruction SET exec_qty=150 WHERE id=1') + self.sql('UPDATE pms_position SET total_qty=150,base_qty=150,avg_cost=10.667 WHERE id=1') + self.assert_applied() + self.assertEqual(qmt_repo.get_order('P_BUY_D01')['cum_qty'],100) + self.assertEqual(qmt_repo.get_order('P_BUY_D02')['cum_qty'],50) + parent=pms_repo.get_instruction('P_BUY') + self.assertEqual(parent['exec_qty'],150);self.assertAlmostEqual(parent['exec_avg_price'],1700/150) + self.assertEqual(pms_repo.get_position(CODE)['avg_cost'],11.333) + + def test_parent_old_progress_or_new_capacity_conflict_holds(self): + self.sql('UPDATE pms_instruction SET exec_qty=99 WHERE id=1') + result=service.consume();self.assertEqual(result['held'],1,result) + self.seed();self.sql('UPDATE pms_instruction SET qty=110 WHERE id=1') + self.modify_payload(10,after=economic(150,'10'));self.modify_payload(11,cum_qty=150,cum_avg_price=10) + self.snapshot['payload']['items'][0]['total_qty']=150 + result=service.consume();self.assertEqual(result['held'],1,result) + self.assertEqual(pms_repo.get_position(CODE)['total_qty'],100) + + def test_broken_amendment_original_is_held_not_retried_as_database_failure(self): + amendment = service.repo.get_inbox(11)['payload'] + dedup = ws_codec.dedup_key('order_update', amendment) + self.sql('UPDATE pms_qmt_inbox SET payload_json=:body,dedup_key=:dedup WHERE seq=11', + {'body': '{broken original', 'dedup': dedup}) + before = self.state() + result = service.consume() + self.assertEqual(result['held'], 1, result) + self.assertEqual(result['errors'], []) + self.assertIn('配套委托回报原文损坏', result['alerts'][0]['reason']) + self.assertEqual(service.repo.get_inbox(10)['processed'], 3) + for table in ('pms_lot', 'pms_position', 'pms_instruction', 'pms_qmt_order'): + self.assertEqual(self.state()[table], before[table]) + + def test_top_level_read_failure_is_not_a_healthy_empty_batch(self): + self.use(service.repo,'pending',lambda **k:(_ for _ in ()).throw(RuntimeError('offline'))) + result=service.consume();self.assertFalse(result['ok']);self.assertTrue(result['errors']) + + +if __name__=='__main__': unittest.main(verbosity=2) diff --git a/scripts/test_trade_correction_strategy.py b/scripts/test_trade_correction_strategy.py new file mode 100644 index 0000000..c3283c8 --- /dev/null +++ b/scripts/test_trade_correction_strategy.py @@ -0,0 +1,260 @@ +"""策略成交更正版本:真实 SQLite CAS、旧运行快照与提议交错,禁止联网。""" +import unit_test_guard # noqa: F401 +import copy +import datetime as dt +import json +import sys +import tempfile +import unittest +from pathlib import Path +from unittest.mock import patch + +sys.path.insert(0, str(Path(__file__).resolve().parents[1])) +from sqlalchemy import create_engine, event, text +from app.core import strategy_book as books +from app.db import session as db +from app.repo import pms_repo +from app.services import strategy_runner as runner, rotation_service +from config.settings import settings + +CODE = '600001.SH' +DECISION = {'side': 'buy', 'action': 'ADD', 'qty': 100, 'leg': 'grid_buy:1', + 'reason': '测试档位', 'grid_buy_idx': 1, 'grid_buy_price': 10.0} + + +class StrategyCorrectionTests(unittest.TestCase): + def setUp(self): + self.now = dt.datetime(2026, 11, 5, 10) + owner = self + class Clock(dt.datetime): + @classmethod + def now(cls, tz=None): + return owner.now + self.use(runner, 'datetime', Clock) + self.tmp = tempfile.TemporaryDirectory(); self.addCleanup(self.tmp.cleanup) + self.engine = create_engine('sqlite:///'+self.tmp.name+'/strategy.sqlite') + self.addCleanup(self.engine.dispose) + self.statements = [] + @event.listens_for(self.engine, 'before_cursor_execute', retval=True) + def dialect(conn, cursor, sql, params, context, many): + db.assert_single_table(sql) + self.statements.append(sql) + sql = sql.replace(' FOR UPDATE', '') + if 'ON DUPLICATE KEY UPDATE' in sql: + self.assertIn('pms_rotation_document', sql) + sql = sql.split('ON DUPLICATE KEY UPDATE')[0]+'ON CONFLICT DO NOTHING' + return sql, params + for obj, name, value in [ + (db, 'get_engine', lambda *args: self.engine), + (db, '_check_cooldown', lambda *args: None), + (settings, 'INFRA_STRATEGY_BOOK_SCHEMA', True), + (rotation_service, 'original_buy_blocker', lambda: ''), + (runner, '_today', lambda: 20261105), + (runner, '_buypause_codes', lambda: set()), + (runner, '_macro_gate_active', lambda: False), + (runner.param_store, 'get_bool', lambda key, default=False: key == 'PMS_STRATEGY_ENABLED'), + (runner.param_store, 'get_float', lambda key, default=0: default), + (runner.portfolio, 'positions_view', lambda: {'positions': [{'ts_code': CODE, 'total_qty': 100, 'avail_qty': 100}], 'totals': {'scale': 10000}}), + (runner.market, 'day_snapshot', lambda code: {'price': 10}), + ]: + self.use(obj, name, value) + schema = { + 'pms_strategy': 'id INTEGER PRIMARY KEY,book_id TEXT,strategy_id TEXT UNIQUE,ts_code TEXT,type TEXT,status TEXT,autonomy TEXT,state_json TEXT,params_json TEXT,note TEXT,archived_at TEXT,created_at TEXT,updated_at TEXT', + 'pms_instruction': 'id INTEGER PRIMARY KEY,book_id TEXT,instruction_id TEXT UNIQUE,origin_type TEXT,origin_id TEXT,ts_code TEXT,action TEXT,side TEXT,qty INTEGER,limit_price REAL,window_tdays INTEGER,status TEXT,exec_qty INTEGER,exec_avg_price REAL,progress_json TEXT,archived_at TEXT,created_at TEXT,updated_at TEXT', + 'pms_proposal': 'id INTEGER PRIMARY KEY,book_id TEXT,proposal_id TEXT UNIQUE,ts_code TEXT,action TEXT,qty INTEGER,hard_numbers_json TEXT,judge_verdict TEXT,judge_reason TEXT,status TEXT,expire_at TEXT,created_at TEXT,archived_at TEXT', + 'pms_rotation_document': 'doc_id TEXT PRIMARY KEY,kind TEXT,payload_json TEXT,revision INTEGER,updated_at TEXT', + } + with self.engine.begin() as conn: + for table, cols in schema.items(): conn.execute(text('CREATE TABLE '+table+'('+cols+')')) + pms_repo.insert_strategy(strategy_id='GRID', ts_code=CODE, stype='GRID', state={'invested': 1000, 'filled_levels': {'0': {'price': 10, 'qty': 100}}}) + self.original = pms_repo.get_strategy('GRID') + + def use(self, obj, name, value): + p = patch.object(obj, name, value); p.start(); self.addCleanup(p.stop) + + def count(self, table): + with self.engine.connect() as conn: + return conn.execute(text('SELECT COUNT(*) FROM '+table)).scalar_one() + + def correction(self, cid='C1', paused=False): + old = pms_repo.get_strategy('GRID')['state'] + new = copy.deepcopy(old) + new.update(invested=1100, trade_correction={'correction_id': cid}) + if paused: new['correction_hold'] = '数量更正后等待核对' + pms_repo.update_strategy('GRID', state=new, status='PAUSED' if paused else 'ACTIVE', + correction_expected=(old.get('trade_correction') or {}).get('correction_id')) + return new + + def test_legacy_state_and_json_null_epoch_update(self): + self.assertEqual(pms_repo.update_strategy('GRID', state={'pending': None}), 1) + self.assertEqual(pms_repo.update_strategy('GRID', state={'trade_correction': {'correction_id': None}}), 1) + self.assertEqual(pms_repo.update_strategy('GRID', state={'invested': 2}), 1) + + def test_original_snapshot_cannot_overwrite_new_state_or_paused_status(self): + new = self.correction(paused=True) + with self.assertRaisesRegex(RuntimeError, '更正版本'): + pms_repo.update_strategy('GRID', state=self.original['state'], status='ACTIVE', note='旧任务') + fresh = pms_repo.get_strategy('GRID') + self.assertEqual(fresh['state'], new) + self.assertEqual(fresh['status'], 'PAUSED') + self.assertIsNone(fresh['note']) + + def test_chain_compare_and_set_requires_previous_id(self): + self.correction('C1') + self.correction('C2') + with self.assertRaises(RuntimeError): + pms_repo.update_strategy('GRID', state={'trade_correction': {'correction_id': 'C3'}}, correction_expected='C1') + self.assertEqual(runner._correction_id(pms_repo.get_strategy('GRID')), 'C2') + + def test_json_state_cannot_bypass_cas(self): + self.correction() + with self.assertRaises(RuntimeError): pms_repo.update_strategy('GRID', state_json='{}') + self.assertEqual(pms_repo.update_strategy('GRID', state_json=json.dumps({'trade_correction': {'correction_id': 'C1'}, 'pending': None})), 1) + + def test_missing_strategy_state_write_fails_but_ordinary_status_keeps_zero(self): + self.assertEqual(pms_repo.update_strategy('missing', status='PAUSED'), 0) + with self.assertRaises(RuntimeError): pms_repo.update_strategy('missing', state={}) + + def test_version_keyword_is_not_a_column_and_requires_state(self): + with self.assertRaises(ValueError): pms_repo.update_strategy('GRID', status='PAUSED', correction_expected=None) + self.correction() + self.assertFalse(any('SET correction_expected' in sql for sql in self.statements)) + + def test_cross_book_cannot_touch_other_strategy(self): + with books.scope('micro30'): + with self.assertRaises(RuntimeError): pms_repo.update_strategy('GRID', state={}) + self.assertEqual(pms_repo.get_strategy('GRID')['state'], self.original['state']) + + def test_stale_snapshot_rejected_for_instruction_and_proposal(self): + self.correction() + for emit in (runner._emit_instruction, runner._emit_proposal): + with self.assertRaisesRegex(RuntimeError, '更正版本'): emit(self.original, DECISION) + self.assertEqual(self.count('pms_instruction'), 0) + self.assertEqual(self.count('pms_proposal'), 0) + + def test_paused_hold_and_archived_strategy_do_not_emit(self): + for fields in ({'status': 'PAUSED'}, {'state': {'correction_hold': '等待核对'}}): + old = pms_repo.get_strategy('GRID') + pms_repo.update_strategy('GRID', **fields) + with self.assertRaises(RuntimeError): runner._emit_instruction(old, DECISION) + pms_repo.update_strategy('GRID', status='ACTIVE', state={}) + with self.engine.begin() as conn: + conn.execute(text("UPDATE pms_strategy SET archived_at='2026-11-05'")) + with self.assertRaises(RuntimeError): runner._emit_proposal(self.original, DECISION) + self.assertEqual(self.count('pms_instruction'), 0) + self.assertEqual(self.count('pms_proposal'), 0) + + def test_new_instruction_and_proposal_record_exact_epoch_under_lock(self): + self.correction() + fresh = pms_repo.get_strategy('GRID') + insert = pms_repo.insert_instruction + def guarded(**kwargs): + self.assertIsNotNone(db._TRANSACTION.get()) + return insert(**kwargs) + with patch.object(pms_repo, 'insert_instruction', side_effect=guarded): + iid = runner._emit_instruction(fresh, DECISION) + pid = runner._emit_proposal(fresh, DECISION) + self.assertEqual(pms_repo.get_instruction(iid)['progress']['strategy_correction_id'], 'C1') + self.assertEqual(pms_repo.get_proposal(pid)['hard_numbers']['strategy_correction_id'], 'C1') + self.assertTrue(any("doc_id='account-execution' FOR UPDATE" in sql for sql in self.statements)) + + def test_pre_correction_proposal_cannot_be_accepted_after_correction(self): + pid = runner._emit_proposal(self.original, DECISION) + spec = pms_repo.get_proposal(pid)['hard_numbers'] + self.correction() + result = runner.emit_from_spec(spec) + self.assertFalse(result['ok'], result) + self.assertEqual(self.count('pms_instruction'), 0) + self.assertNotIn('pending', pms_repo.get_strategy('GRID')['state']) + + def test_current_proposal_accepts_and_keeps_epoch_in_pending_state(self): + self.correction() + spec = dict(DECISION, strategy_id='GRID', strategy_correction_id='C1') + result = runner.emit_from_spec(spec) + self.assertTrue(result['ok'], result) + fresh = pms_repo.get_strategy('GRID') + self.assertEqual(fresh['state']['pending']['iid'], result['instruction_id']) + self.assertEqual(runner._correction_id(fresh), 'C1') + + def test_legacy_proposal_before_any_correction_still_accepts(self): + result = runner.emit_from_spec(dict(DECISION, strategy_id='GRID')) + self.assertTrue(result['ok'], result) + self.assertIsNone(pms_repo.get_instruction(result['instruction_id'])['progress']['strategy_correction_id']) + + def test_accept_state_write_failure_rolls_back_new_parent(self): + with patch.object(pms_repo, 'update_strategy', side_effect=RuntimeError('模拟状态冲突')): + with self.assertLogs('pms.strategy', level='ERROR'): + result = runner.emit_from_spec(dict(DECISION, strategy_id='GRID')) + self.assertFalse(result['ok'], result) + self.assertEqual(self.count('pms_instruction'), 0) + + def tick(self, *, decision=True, corrected=False, paused=False, autonomy='auto', dry_run=False): + pms_repo.update_strategy('GRID', autonomy=autonomy) + def evaluate(st, pos, day, now, ctx): + if corrected: self.correction(paused=paused) + return copy.deepcopy(DECISION) if decision else None + with patch.dict(runner.EVALUATORS, {'GRID': evaluate}): + return runner.tick(dry_run=dry_run) + + def test_tick_loaded_before_correction_cannot_publish_or_overwrite(self): + for autonomy in ('auto', 'confirm'): + with self.assertLogs('pms.strategy', level='ERROR'): + out = self.tick(corrected=True, paused=autonomy=='confirm', autonomy=autonomy) + self.assertFalse(out['ok'], out) + self.assertEqual(out['fired'], []) + self.assertEqual(out['queued'], []) + self.assertEqual(self.count('pms_instruction'), 0) + self.assertEqual(self.count('pms_proposal'), 0) + self.assertEqual(pms_repo.get_strategy('GRID')['state']['invested'], 1100) + # 下一子例从新的、无更正标记的独立策略状态开始。 + with self.engine.begin() as conn: + conn.execute(text("UPDATE pms_strategy SET state_json=:body,status='ACTIVE'"), {'body':json.dumps(self.original['state'])}) + + def test_idle_tick_old_state_cannot_erase_correction(self): + with self.assertLogs('pms.strategy', level='ERROR'): + out = self.tick(decision=False, corrected=True) + self.assertFalse(out['ok'], out) + self.assertEqual(runner._correction_id(pms_repo.get_strategy('GRID')), 'C1') + + def test_tick_success_persists_pending_and_parent_together(self): + self.correction() + out = self.tick() + self.assertTrue(out['ok'], out) + self.assertEqual(len(out['fired']), 1) + fresh = pms_repo.get_strategy('GRID') + parent = pms_repo.get_instruction(fresh['state']['pending']['iid']) + self.assertEqual(parent['progress']['strategy_correction_id'], 'C1') + self.assertEqual(fresh['state']['invested'], 2100) + + def test_tick_state_failure_rolls_back_parent_and_reports_no_fired(self): + original_update = pms_repo.update_strategy + def fail_state(sid, **fields): + if 'state' in fields: raise RuntimeError('模拟版本冲突') + return original_update(sid, **fields) + with patch.object(pms_repo, 'update_strategy', side_effect=fail_state): + with self.assertLogs('pms.strategy', level='ERROR'): out = self.tick() + self.assertFalse(out['ok'], out) + self.assertEqual(out['fired'], []) + self.assertEqual(self.count('pms_instruction'), 0) + self.assertEqual(pms_repo.get_strategy('GRID')['state'], self.original['state']) + + def test_dry_run_keeps_strategy_and_does_not_lock_or_insert(self): + self.statements.clear() + out = self.tick(dry_run=True) + self.assertTrue(out['ok'], out) + self.assertEqual(pms_repo.get_strategy('GRID')['state'], self.original['state']) + self.assertFalse(any('account-execution' in sql for sql in self.statements)) + self.assertEqual(self.count('pms_instruction'), 0) + + def test_soft_archived_parent_does_not_archive_current_strategy(self): + iid = runner._emit_instruction(self.original, DECISION) + with self.engine.begin() as conn: + conn.execute(text("UPDATE pms_instruction SET status='CONFIRMED',archived_at='2026-11-05' WHERE instruction_id=:iid"), {'iid': iid}) + self.correction() + self.now += dt.timedelta(minutes=1) + result = runner.emit_from_spec(dict(DECISION, strategy_id='GRID', strategy_correction_id='C1', leg='grid_sell:1', side='sell', action='TRIM')) + self.assertTrue(result['ok'], result) + self.assertEqual(self.count('pms_instruction'), 2) + + +if __name__ == '__main__': unittest.main() diff --git a/scripts/test_trade_correction_transport.py b/scripts/test_trade_correction_transport.py new file mode 100644 index 0000000..89c8ab2 --- /dev/null +++ b/scripts/test_trade_correction_transport.py @@ -0,0 +1,368 @@ +"""成交修正收件、幂等及执行隔离回归;仅SQLite内存库和内存业务桩。""" +import unit_test_guard # noqa: F401 +import asyncio +import copy +import hashlib +import json +import sqlite3 +import sys +import threading +import unittest +from contextlib import nullcontext, contextmanager +from datetime import datetime, timedelta +from pathlib import Path +from unittest.mock import patch + +sys.path.insert(0, str(Path(__file__).resolve().parents[1])) +from config.settings import settings +from app.core import ws_codec as wsc +from app.repo import qmt_repo as q, execution_repo as er, rotation_repo as store +from app.services import execution_orders as eo, executor, dispatcher, execution_conflicts, daily_buy_review +from app.services import rotation_execution +from app.ws.runner import WsRunner +from test_wiring import install_fakes + +REAL_PENDING, REAL_CORRECTIONS = er.pending_trades, er.pending_corrections +REAL_INBOX_PENDING = q.inbox_pending +NOW = datetime(2026, 11, 5, 10, 30) +CODE, IID, PARENT = '600000.SH', 'P_D01', 'P' + + +def correction(**extra): + return {'correction_id': 'C1', 'instruction_id': IID, 'broker_order_id': 'B1', + 'trade_no': 'B1#1', 'ts_code': CODE, 'side': 'buy', + 'before': {'qty': 100, 'price': 10, 'amount': 1000}, + 'after': {'qty': 80, 'price': 10, 'amount': 800}, **extra} + + +class TransportTests(unittest.TestCase): + def setUp(self): + self.conn = sqlite3.connect(':memory:', check_same_thread=False) + self.conn.row_factory = sqlite3.Row + self.conn.create_function('JSON_UNQUOTE', 1, lambda value: value) + self.conn.create_function('JSON_CONTAINS_PATH', 3, + lambda value, mode, path: int(path[2:] in json.loads(value))) + self.conn.create_function('JSON_CONTAINS', 3, + lambda value, candidate, path: int(json.loads(value).get(path[2:]) is False) + if candidate == 'false' else None) + self.conn.execute('CREATE TABLE pms_qmt_inbox (seq INTEGER PRIMARY KEY, msg_id TEXT, msg_type TEXT, ' + 'corr_id TEXT, dedup_key TEXT UNIQUE, payload_json TEXT, msg_ts INTEGER, received_at TEXT, ' + 'processed INTEGER, processed_at TEXT, process_note TEXT)') + patch.object(q, 'fetch_one', side_effect=self.one).start() + patch.object(q, 'fetch_all', side_effect=self.all).start() + patch.object(q, 'execute', side_effect=self.execute).start() + patch.object(er.db, 'fetch_one', side_effect=self.one).start() + patch.object(settings, 'INFRA_STRATEGY_BOOK_SCHEMA', False).start() + self.addCleanup(patch.stopall) + self.addCleanup(self.conn.close) + + def one(self, sql, params=None): + row = self.conn.execute(sql, params or {}).fetchone() + return dict(row) if row else None + + def all(self, sql, params=None): + return [dict(row) for row in self.conn.execute(sql, params or {}).fetchall()] + + def execute(self, sql, params=None): + sql = sql.replace('ON DUPLICATE KEY UPDATE seq = seq', 'ON CONFLICT DO NOTHING') + params = {k: v.isoformat() if isinstance(v, datetime) else v for k, v in (params or {}).items()} + return self.conn.execute(sql, params).rowcount + + def put(self, seq, payload=None, kind='trade_correction'): + payload = correction() if payload is None else payload + return q.inbox_put(seq=seq, msg_id=str(seq), msg_type=kind, payload=payload, + msg_ts=1, corr_id=payload.get('instruction_id'), dedup_key=wsc.dedup_key(kind, payload), + processed=0 if wsc.needs_ledger(kind, payload) else 2) + + def row(self, seq): + return self.one('SELECT * FROM pms_qmt_inbox WHERE seq=:s', {'s': seq}) + + def test_message_classification_keeps_plain_false_ordinary_but_routes_malformed(self): + for payload in ({'amended': True}, {'amended': False, 'correction_id': 'C1'}, + {'amended': None}, {'amended': 'false'}, {'amended': 0}, + {'correction_id': None}, {'reason': 'trade_correction'}): + self.assertTrue(wsc.needs_ledger('order_update', payload)) + self.assertFalse(wsc.needs_ledger('order_update', {'amended': False})) + self.assertFalse(wsc.needs_ledger('order_update', {'status': 'FILLED'})) + self.assertTrue(wsc.needs_ledger('trade', {})) + + def test_keys_are_bounded_separate_and_not_original_trade_key(self): + body = correction(correction_id='任意编号'*200, amended=True) + a = wsc.dedup_key('trade_correction', body) + b = wsc.dedup_key('order_update', body) + self.assertLessEqual(len(a), 80) + self.assertLessEqual(len(b), 80) + self.assertNotEqual(a, b) + self.assertEqual(wsc.dedup_key('trade', body), 'trade:B1#1') + + def test_same_id_same_content_archives_new_seq_without_second_work(self): + self.assertEqual(self.put(1), q.PUT_NEW) + self.assertEqual(self.put(2), q.PUT_DUP_KEY) + self.assertEqual(self.row(1)['processed'], 0) + self.assertEqual(self.row(2)['processed'], 2) + self.assertIsNone(self.row(2)['dedup_key']) + self.assertEqual(self.put(1), q.PUT_DUP_SEQ) + + def test_same_id_different_content_is_held_and_original_stays_immutable(self): + self.put(1) + self.assertEqual(self.put(2, correction(after={'qty': 90, 'price': 10, 'amount': 900})), q.PUT_CONFLICT) + self.assertEqual(self.row(2)['processed'], 3) + self.assertIn('不同修正内容', self.row(2)['process_note']) + self.assertEqual(json.loads(self.row(1)['payload_json'])['after']['qty'], 80) + self.assertEqual(REAL_CORRECTIONS(), 2) + + def test_missing_correction_id_and_identity_still_blocks_account_buys(self): + self.put(1, {'amended': True}, 'order_update') + self.assertEqual(self.row(1)['processed'], 3) + self.assertEqual(REAL_CORRECTIONS(), 1) + + def test_amendment_and_correction_are_independent_pending_evidence_in_either_order(self): + self.put(1, correction(amended=True, reason='trade_correction'), 'order_update') + self.put(2) + self.assertEqual(REAL_PENDING([IID]), 2) + self.assertEqual(REAL_CORRECTIONS(ts_code=CODE), 2) + self.assertEqual(REAL_CORRECTIONS(ts_code='600001.SH'), 0) + self.assertEqual(len(q.inbox_for_order(IID)), 2) + self.assertEqual(q.inbox_by_key(wsc.dedup_key('trade_correction', correction()))['seq'], 2) + + def test_correction_queue_cannot_starve_original_trades(self): + self.put(1) + self.put(2, correction(), 'trade') + rows = REAL_INBOX_PENDING(limit=1, msg_type='trade') + self.assertEqual([row['seq'] for row in rows], [2]) + + def test_ordinary_status_is_not_counted_as_pending_correction(self): + self.put(1, {'instruction_id': IID, 'ts_code': CODE, 'status': 'FILLED', 'amended': False}, 'order_update') + self.assertEqual(REAL_CORRECTIONS(), 0) + + def test_sql_classification_matches_non_boolean_and_numbered_amendments(self): + for seq, payload in enumerate(({'amended': None}, {'amended': 0}, {'amended': 'false'}, + {'amended': False, 'correction_id': 'C1'}, {'correction_id': None}), 1): + self.put(seq, {'instruction_id': IID, 'ts_code': CODE, **payload}, 'order_update') + self.assertEqual(REAL_CORRECTIONS(), seq) + + def test_legacy_unknown_correction_is_not_mistaken_for_applied_or_duplicate(self): + self.put(1) + self.execute('UPDATE pms_qmt_inbox SET processed=2, process_note=:note WHERE seq=1', + {'note': 'trade_correction 由通道进程消化, 不入账'}) + self.assertEqual(REAL_CORRECTIONS(), 1) + self.assertEqual(REAL_PENDING([IID]), 1) + self.execute('UPDATE pms_qmt_inbox SET process_note=:note WHERE seq=1', {'note': '重复修正占位'}) + self.assertEqual(REAL_CORRECTIONS(), 0) + + def test_ws_persistence_uses_payload_order_identity_and_pending_state(self): + runner = WsRunner() + with patch.object(runner, '_p', side_effect=lambda key, default=None: default): + asyncio.run(runner._persist_upstream({'seq': 1, 'type': 'order_update', 'msg_id': 'm1', + 'ts': 1, 'corr_id': 'caller-other-id', 'payload': correction(amended=True)})) + self.assertEqual(self.row(1)['processed'], 0) + self.assertEqual(self.row(1)['corr_id'], IID) + + def test_ws_never_applies_amended_or_correction_state_directly(self): + runner = WsRunner() + with patch.object(q, 'update_order', side_effect=AssertionError('修正不得直接改委托')), \ + patch.object(q, 'get_order', side_effect=AssertionError('修正不得进入普通状态分支')): + for kind in ('trade_correction', 'order_update'): + asyncio.run(runner._apply_side_effects(kind, correction(amended=True), {'seq': 1})) + asyncio.run(runner._on_order_update(IID, correction(amended=False))) + + +class ExecutionBoundaryTests(unittest.TestCase): + def setUp(self): + self.fake = install_fakes(prices={CODE: 10}, params={'PMS_DISPATCH_MODE': 'ws', 'PMS_EXEC_REPRICE_ENABLED': False}) + self.fake.qmt.set_channel() + self.fake.insert_instruction(instruction_id=PARENT, origin_type='strategy', origin_id='S', + ts_code=CODE, side='buy', action='OPEN', qty=300, status='DISPATCHED', progress={}) + self.parent = self.fake.instructions[PARENT] + self.parent.update(book_id='original', exec_qty=80) + self.fake.qmt.enqueue_order(instruction_id=IID, parent_id=PARENT, ts_code=CODE, + side='buy', qty=100, limit_price=10, valid_until=1) + self.order = self.fake.qmt.orders[IID] + self.order.update(status='FILLED', cum_qty=80, leaves_qty=0, final_at=NOW-timedelta(minutes=10)) + self.audit = {'instruction_id': IID, 'cum_qty': 80, 'correction_seq': 10, 'received_at': NOW.isoformat()} + patch.object(settings, 'INFRA_STRATEGY_BOOK_SCHEMA', True).start() + patch.object(store, 'get', side_effect=lambda key: (copy.deepcopy(self.audit), 1) + if key == 'order-correction:'+hashlib.sha256(IID.encode()).hexdigest() else (None, 0)).start() + patch.object(eo, 'account_guard', side_effect=nullcontext).start() + self.addCleanup(patch.stopall) + + def pending(self, code=CODE, kind='trade_correction', processed=0): + self.fake.qmt.inbox[11] = {'seq': 11, 'msg_type': kind, 'processed': processed, + 'payload': correction(ts_code=code, amended=True)} + + def test_verified_quantity_reduction_has_one_consistent_effective_quantity(self): + self.assertEqual(eo.effective_filled(self.order), 80) + self.assertEqual(eo.replacement_state(PARENT)['remaining'], 220) + self.assertEqual(executor._parent_available(self.parent, []), 220) + self.assertEqual(daily_buy_review._orders(self.parent)[1], 80) + + def test_unverified_or_mismatched_filled_reduction_is_rejected(self): + for audit in (None, {}, dict(self.audit, cum_qty=79), dict(self.audit, received_at='unknown')): + with patch.object(store, 'get', return_value=(audit, 1)): + with self.assertRaises(ValueError): + eo.effective_filled(self.order) + + def test_quantity_hold_blocks_old_parent_both_sides_and_daily_reapproval(self): + self.parent['progress']['trade_correction_hold'] = {'correction_id': 'C1'} + for side in ('buy', 'sell'): + self.parent['side'] = side + self.assertTrue(eo.correction_blocker(self.parent)) + self.assertEqual(executor._parent_available(self.parent, []), 0) + self.assertFalse(eo.should_replace(self.parent, {'action': 'FIRE', 'limit_price': 11})) + self.assertTrue(execution_conflicts.queued_buy_blocker(dict(self.order, side=side))) + result = daily_buy_review.process(now=NOW) + self.assertEqual(result['approved'], []) + + def test_old_parent_hold_does_not_ban_new_manual_parent_on_same_code(self): + self.parent['progress']['trade_correction_hold'] = True + newer = dict(self.parent, instruction_id='NEW', progress={}) + self.assertEqual(eo.correction_blocker(newer), '') + self.assertEqual(eo.code_settlement_blocker(CODE), '') + + def test_quantity_hold_cannot_be_automatically_approved_by_daily_model(self): + from app.services import proposal_service + self.parent.update(status='WAIT_REVIEW', side='buy') + self.parent['progress'].update(trade_correction_hold=True, + daily_review={'ymd': 20261105, 'status': 'pending'}) + with patch.object(daily_buy_review.param_store, 'get_bool', return_value=True), \ + patch.object(proposal_service, 'scan_and_route', side_effect=AssertionError('修正不调用模型解锁')): + result = daily_buy_review.process(now=NOW) + self.assertEqual(result['approved'], []) + self.assertEqual(result['waiting'], [PARENT]) + + def test_direct_dispatch_hold_remains_effective_when_reprice_switch_off(self): + self.parent['progress']['trade_correction_hold'] = True + with patch.object(dispatcher, '_ws', side_effect=AssertionError('修正后不得再次下单')): + result = dispatcher.dispatch(instruction_id='P_D02', parent_id=PARENT, ts_code=CODE, + side='buy', qty=100, limit_price=10, valid_until=NOW) + self.assertFalse(result['ok']) + self.assertIn('成交数量已修正', result['error']) + + def test_grid_parent_revision_must_match_current_strategy_after_correction(self): + self.parent['progress'].update(stype='GRID', strategy_correction_id=None) + strategy = {'type': 'GRID', 'status': 'ACTIVE', 'state': {}} + with patch.object(eo.pms_repo, 'get_strategy', return_value=strategy): + self.assertEqual(eo.correction_blocker(self.parent), '') + strategy['state']['trade_correction'] = {'correction_id': 'C1'} + self.assertIn('旧版本', eo.correction_blocker(self.parent)) + self.parent['progress']['strategy_correction_id'] = 'C1' + self.assertEqual(eo.correction_blocker(self.parent), '') + strategy['state']['correction_hold'] = True + self.assertIn('正在核实', eo.correction_blocker(self.parent)) + strategy['state']['correction_hold'] = False + strategy['status'] = 'PAUSED' + self.assertTrue(eo.correction_blocker(self.parent)) + + def test_missing_explicit_grid_strategy_fails_closed_without_changing_other_types(self): + self.parent['progress']['stype'] = 'GRID' + with patch.object(eo.pms_repo, 'get_strategy', return_value=None): + self.assertTrue(eo.correction_blocker(self.parent)) + self.parent['progress']['stype'] = 'T0' + self.assertEqual(eo.correction_blocker(self.parent), '') + + def test_pending_correction_blocks_all_new_buys_but_only_affected_sells(self): + self.pending(processed=3) + self.assertTrue(eo.correction_blocker({'side': 'buy', 'ts_code': '600001.SH'})) + self.assertTrue(eo.correction_blocker({'side': 'sell', 'ts_code': CODE})) + self.assertEqual(eo.correction_blocker({'side': 'sell', 'ts_code': '600001.SH'}), '') + self.assertFalse(eo.replacement_state(PARENT)['ready']) + + def test_missing_payload_code_still_blocks_new_sell_parent_by_outbox_identity(self): + self.pending(code=None, processed=3) + self.assertTrue(eo.correction_blocker({'instruction_id': 'NEW', 'side': 'sell', 'ts_code': CODE})) + self.assertEqual(eo.correction_blocker({'instruction_id': 'OTHER', 'side': 'sell', 'ts_code': '600001.SH'}), '') + + def test_final_send_mark_cannot_pass_pending_correction(self): + self.pending() + with patch.object(q, 'mark_send_attempt', side_effect=AssertionError('不能标记可发送')): + self.assertFalse(eo.prepare_send(IID)['marked']) + + def test_ordinary_terminal_repeat_cannot_overwrite_corrected_quantity_or_price(self): + runner = WsRunner() + with patch.object(q, 'update_order', side_effect=AssertionError('旧终态不能覆盖修正')): + asyncio.run(runner._on_order_update(IID, {'status': 'FILLED', 'cum_qty': 100, 'cum_avg_price': 9})) + self.assertEqual(self.order['cum_qty'], 80) + + def test_old_ordinary_update_cannot_overwrite_active_order_correction(self): + self.order.update(status='PARTIAL') + self.audit['amendment_seq'] = 12 + runner = WsRunner() + with patch.object(q, 'update_order', side_effect=AssertionError('乱序旧回报不能覆盖修正')): + asyncio.run(runner._on_order_update(IID, {'status': 'PARTIAL', 'cum_qty': 90}, sequence=11)) + with patch.object(q, 'update_order', return_value=1) as update: + asyncio.run(runner._on_order_update(IID, {'status': 'PARTIAL', 'cum_qty': 90}, sequence=13)) + self.assertEqual(update.call_args.kwargs['cum_qty'], 90) + + def test_legitimate_later_fills_do_not_make_old_correction_cutoff_unreadable(self): + self.order.update(status='FILLED', cum_qty=100) + self.assertEqual(eo.effective_filled(self.order), 100) + self.assertEqual(eo.settlement_cutoff(self.order).replace(tzinfo=None), NOW) + + def test_normal_order_ack_and_reject_are_serialized_with_correction_commit(self): + for kind in ('order_update', 'reject', 'ack'): + with self.subTest(kind=kind): + lock, read, probe = threading.RLock(), threading.Event(), threading.Event() + acquired_without_wait, failures = [], [] + self.order.update(status='PARTIAL', cum_qty=100, cancel_state='SENT') + self.audit = None + @contextmanager + def guarded(): + with lock: + yield + def get_order(_): + old = copy.deepcopy(self.order) + read.set() + if not probe.wait(2): + raise AssertionError('交错测试没有进入竞争点') + return old + def amend(): + try: + if not read.wait(2): + raise AssertionError('普通回报未进入读状态步骤') + acquired = lock.acquire(blocking=False) + acquired_without_wait.append(acquired) + if acquired: + lock.release() + probe.set() + with guarded(): + self.order.update(status='FILLED', cum_qty=80) + self.audit = {'instruction_id': IID, 'cum_qty': 80, + 'correction_seq': 20, 'received_at': NOW.isoformat()} + except BaseException as exc: + failures.append(exc) + probe.set() + def update(_, **fields): + self.assertEqual(acquired_without_wait, [False], '修正不能插入普通回报读写之间') + self.order.update(fields) + return 1 + thread = threading.Thread(target=amend, daemon=True) + with patch.object(eo, 'account_guard', side_effect=guarded), \ + patch.object(q, 'get_order', side_effect=get_order), \ + patch.object(q, 'update_order', side_effect=update): + thread.start() + runner = WsRunner() + payload = {'instruction_id': IID, 'status': 'FILLED', 'cum_qty': 100, + 'code': 'ALREADY_FINAL', 'final': True} + asyncio.run(runner._apply_side_effects(kind, payload, {'seq': 19})) + thread.join(2) + self.assertFalse(thread.is_alive()) + self.assertEqual(failures, []) + self.assertEqual(acquired_without_wait, [False]) + self.assertEqual(self.order['cum_qty'], 80) + + def test_old_snapshot_does_not_release_corrected_reservation(self): + account = {'orders': {IID: 1005.}} + rotation_execution.release_settled_reservations(account, {'received_at': NOW-timedelta(seconds=1)}) + self.assertEqual(account['orders'][IID], 1005.) + rotation_execution.release_settled_reservations(account, {'received_at': NOW+timedelta(seconds=1)}) + self.assertEqual(account['orders'][IID], 805.) + + def test_unpaired_correction_blocks_reservation_release_even_with_new_snapshot(self): + self.pending() + account = {'orders': {IID: 1005.}} + rotation_execution.release_settled_reservations(account, {'received_at': NOW+timedelta(minutes=2)}) + self.assertEqual(account['orders'][IID], 1005.) + + +if __name__ == '__main__': + unittest.main(verbosity=2) diff --git a/scripts/test_trade_correction_units.py b/scripts/test_trade_correction_units.py new file mode 100644 index 0000000..31f8502 --- /dev/null +++ b/scripts/test_trade_correction_units.py @@ -0,0 +1,318 @@ +"""成交更正纯投影专项。只用虚构逐笔与批次,经统一护栏断网执行。""" +import unit_test_guard # noqa: F401 +import sys +from pathlib import Path +sys.path.insert(0, str(Path(__file__).resolve().parents[1])) +from copy import deepcopy +from datetime import datetime, timezone, timedelta +from decimal import Decimal, ROUND_HALF_UP +import unittest + +from app.core.trade_correction import compile_correction + +CODE, TODAY = '600001.SH', '2026-11-05' +TZ = timezone(timedelta(hours=8)) + + +def economic(qty, price): + return {'qty': qty, 'price': str(price), 'amount': format( + (Decimal(str(price)) * qty).quantize(Decimal('.01'), rounding=ROUND_HALF_UP), 'f')} + + +def trade(no, seq, side, qty, price, *, day=TODAY, parent=None, book='original'): + at = datetime.fromisoformat(day + 'T10:00:00+08:00') + timedelta(minutes=seq) + payload = dict(economic(qty, price), trade_no=no, ts_code=CODE, side=side, + instruction_id=(parent or 'P_'+no)+'_D01', parent_id=parent or 'P_'+no, + broker_order_id='BROKER_'+no, trade_time=int(at.timestamp()*1000), fee=None) + return {'seq': seq, 'payload': payload, 'original_payload': deepcopy(payload), + 'processed_at': at.isoformat(), 'book_day': day, 'book_id': book} + + +def lot(row, lid=1, *, qty=None, closed=0, close=None, pnl='0.00', kind='BASE', book='original'): + p = row['payload'] + qty = p['qty'] - closed if qty is None else qty + return {'id': lid, 'book_id': book, 'ts_code': p['ts_code'], 'instruction_id': p['parent_id'], + 'open_date': row['book_day'], 'created_at': row['processed_at'], 'lot_type': kind, + 'qty': qty, 'closed_qty': closed, 'open_price': p['price'], 'close_avg_price': close, + 'realized_pnl': pnl, 'status': 'OPEN' if qty else 'CLOSED'} + + +def fixture(sold=False): + """服务层可复用的最小合法输入;100股买入,或随后40股卖出。""" + buy = trade('BUY', 1, 'buy', 100, '10.000') + if not sold: + return {'lots': [lot(buy)], 'trades': [buy]} + sell = trade('SELL', 2, 'sell', 40, '12.000') + return {'lots': [lot(buy, closed=40, close='12.000', pnl='80.00')], 'trades': [buy, sell]} + + +class ProjectionTests(unittest.TestCase): + def compile(self, data=None, target='BUY', qty=100, price='11.000', **kwargs): + return compile_correction(**(data or fixture()), target_trade_no=target, + after=economic(qty, price), today=kwargs.pop('today', TODAY), **kwargs) + + def test_buy_price_updates_cost_and_cash_without_qty_or_avail(self): + out = self.compile() + self.assertEqual((out['qty_before'], out['qty_after'], out['cash_delta'], out['avail_delta']), + (100, 100, '-100.00', 0)) + self.assertEqual(out['lots'][0]['open_price'], '11.000') + self.assertEqual(Decimal(out['net_cost_after']), Decimal('1100')) + + def test_buy_quantity_grows_current_day_holding_without_sellable(self): + out = self.compile(qty=150, price='10') + self.assertEqual((out['qty_after'], out['avail_delta'], out['cash_delta']), (150, 0, '-500.00')) + + def test_buy_quantity_reduction_refunds_only_principal(self): + out = self.compile(qty=60, price='10') + self.assertEqual((out['qty_after'], out['cash_delta']), (60, '400.00')) + + def test_zero_buy_keeps_closed_lot_and_source_metadata(self): + data = fixture(); old = deepcopy(data) + out = self.compile(data, qty=0, price='0') + self.assertEqual((len(out['lots']), out['lots'][0]['id'], out['lots'][0]['status']), (1, 1, 'CLOSED')) + self.assertEqual((out['lots'][0]['qty'], out['cash_delta']), (0, '1000.00')) + self.assertEqual(out['lots'][0]['created_at'], old['lots'][0]['created_at']) + self.assertEqual(data, old) + + def test_later_sell_is_reallocated_after_buy_price_correction(self): + out = self.compile(fixture(True)) + self.assertEqual((out['lots'][0]['qty'], out['lots'][0]['closed_qty']), (60, 40)) + self.assertEqual(out['lots'][0]['realized_pnl'], '40.00') + self.assertEqual(Decimal(out['net_cost_after']), Decimal('620')) + self.assertEqual(out['allocations'][0]['realized_pnl_delta'], '40.00') + + def test_sell_price_corrects_closed_cost_and_cash(self): + out = self.compile(fixture(True), target='SELL', qty=40, price='13') + self.assertEqual((out['lots'][0]['close_avg_price'], out['lots'][0]['realized_pnl']), ('13.000', '120.00')) + self.assertEqual((out['cash_delta'], out['avail_delta']), ('40.00', 0)) + + def test_sell_quantity_corrects_position_and_sellable(self): + out = self.compile(fixture(True), target='SELL', qty=50, price='12') + self.assertEqual((out['qty_after'], out['avail_delta'], out['cash_delta']), (50, -10, '120.00')) + self.assertEqual(out['lots'][0]['realized_pnl'], '100.00') + + def test_zero_sell_restores_original_lot_without_new_buy(self): + out = self.compile(fixture(True), target='SELL', qty=0, price='0') + self.assertEqual((out['qty_after'], out['avail_delta'], out['cash_delta']), (100, 40, '-480.00')) + self.assertEqual((out['lots'][0]['closed_qty'], out['lots'][0]['close_avg_price']), (0, None)) + + def test_correction_cannot_oversell_at_any_intermediate_step(self): + with self.assertRaises(ValueError): + self.compile(fixture(True), qty=30, price='10') + with self.assertRaises(ValueError): + self.compile(fixture(True), target='SELL', qty=101, price='12') + + def test_allocation_keeps_original_tier_and_add_date_order(self): + rows = [trade('BASE', 1, 'buy', 100, 10, day='2026-11-03'), + trade('T', 2, 'buy', 100, 11, day='2026-11-03'), + trade('OLD', 3, 'buy', 100, 12, day='2026-11-03'), + trade('NEW', 4, 'buy', 100, 13, day='2026-11-04'), + trade('SELL', 5, 'sell', 250, 14)] + lots = [lot(rows[0]), lot(rows[1], 2, closed=100, close='14', pnl='300', kind='T0'), + lot(rows[2], 3, closed=50, close='14', pnl='100', kind='ADD'), + lot(rows[3], 4, closed=100, close='14', pnl='100', kind='ADD')] + out = self.compile({'lots': lots, 'trades': rows}, target='SELL', qty=350, price='14') + self.assertEqual([(a['lot_id'], a['qty']) for a in out['allocations']], [(2, 100), (4, 100), (3, 100), (1, 50)]) + self.assertEqual(out['qty_after'], 50) + + def test_closed_lot_sell_price_can_be_projected_without_reopening(self): + data = fixture(True) + data['trades'][1] = trade('SELL', 2, 'sell', 100, 12) + data['lots'][0] = lot(data['trades'][0], closed=100, close='12', pnl='200') + out = self.compile(data, target='SELL', qty=100, price='11') + self.assertEqual((out['qty_after'], out['lots'][0]['status'], out['lots'][0]['realized_pnl']), (0, 'CLOSED', '100.00')) + + def closed_round_with_new_buy(self): + buy, sell = trade('BUY', 1, 'buy', 100, 10), trade('SELL', 2, 'sell', 100, 12) + new = trade('NEW', 3, 'buy', 100, 10) + return {'lots': [lot(buy, closed=100, close='12', pnl='200'), lot(new, 2)], + 'trades': [buy, sell, new]} + + def test_old_round_price_correction_cannot_change_new_holding_cost(self): + data = self.closed_round_with_new_buy() + for target, qty, price in (('BUY', 100, '11'), ('SELL', 100, '13')): + with self.subTest(target=target), self.assertRaisesRegex(ValueError, '清零并建立新仓'): + self.compile(data, target=target, qty=qty, price=price) + + def test_quantity_correction_cannot_carry_old_inventory_into_new_round(self): + data = self.closed_round_with_new_buy() + for target, qty, price in (('BUY', 150, '10'), ('SELL', 50, '12')): + with self.subTest(target=target), self.assertRaisesRegex(ValueError, '人工核对'): + self.compile(data, target=target, qty=qty, price=price) + + def test_latest_round_correction_keeps_prior_closed_round_unchanged(self): + data = self.closed_round_with_new_buy() + out = self.compile(data, target='NEW') + self.assertEqual(out['lots'][0]['realized_pnl'], '200.00') + self.assertEqual([r['id'] for r in out['changes']], [2]) + self.assertEqual((out['qty_after'], out['cash_delta']), (100, '-100.00')) + + def test_zeroed_old_buy_cannot_be_revived_into_later_new_round(self): + data = fixture(); zero = self.compile(data, qty=0, price='0') + data['lots'] = zero['lots']; data['trades'][0]['payload'].update(zero['trade_after']) + new = trade('NEW', 2, 'buy', 100, 10) + data['lots'].append(lot(new, 2)); data['trades'].append(new) + with self.assertRaisesRegex(ValueError, '清零并建立新仓'): + self.compile(data, qty=50, price='10', lot_links={'BUY': 1, 'NEW': 2}) + + def test_tomorrow_buy_quantity_change_is_sellable(self): + out = self.compile(qty=150, price='10', today='2026-11-06') + self.assertEqual(out['avail_delta'], 50) + + def test_previous_day_sell_delta_still_reduces_today_sellable(self): + out = self.compile(fixture(True), target='SELL', qty=50, price='12', today='2026-11-06') + self.assertEqual(out['avail_delta'], -10) + + def test_cross_day_booked_quantity_needs_verified_t_plus_one(self): + data = fixture(); data['lots'][0]['open_date'] = '2026-11-06' + with self.assertRaises(ValueError): + self.compile(data, qty=150, price='10', today='2026-11-06') + self.assertEqual(self.compile(data, today='2026-11-06')['avail_delta'], 0) + data = fixture(True); data['trades'][1]['book_day'] = '2026-11-06' + with self.assertRaises(ValueError): + self.compile(data, target='SELL', qty=50, price='12', today='2026-11-06') + + def test_missing_or_future_actual_trade_day_is_not_inferred(self): + data = fixture(); data['trades'][0]['original_payload'].pop('trade_time') + with self.assertRaises(ValueError): + self.compile(data) + data['trades'][0]['trade_day'] = TODAY + self.assertEqual(self.compile(data)['qty_after'], 100) + with self.assertRaises(ValueError): + self.compile(today='2026-11-04') + + def test_same_code_other_strategy_is_rejected(self): + data = fixture(); data['lots'][0]['book_id'] = 'micro30' + with self.assertRaises(ValueError): + self.compile(data) + data['trades'][0]['book_id'] = 'micro30' + self.assertEqual(self.compile(data)['lots'][0]['book_id'], 'micro30') + + def test_chain_uses_effective_trade_and_persisted_lineage(self): + data = fixture(True); first = self.compile(data) + data['lots'] = first['lots'] + data['trades'][0]['payload'].update(first['trade_after']) + out = self.compile(data, qty=110, price='11', lot_links=first['lot_links']) + self.assertEqual((out['qty_after'], out['amount_before'], out['cash_delta']), (70, '1100.00', '-110.00')) + self.assertEqual(data['trades'][0]['original_payload']['price'], '10.000') + with self.assertRaises(ValueError): + self.compile(data, qty=110, price='11') + + def test_chain_can_correct_zeroed_trade_without_deleting_its_identity(self): + data = fixture(); first = self.compile(data, qty=0, price='0') + data['lots'] = first['lots'] + data['trades'][0]['payload'].update(first['trade_after']) + out = self.compile(data, qty=50, price='10', lot_links=first['lot_links']) + self.assertEqual((out['qty_before'], out['qty_after'], out['cash_delta']), (0, 50, '-500.00')) + self.assertEqual(out['lots'][0]['id'], 1) + + def test_ambiguous_legacy_lot_needs_explicit_mapping(self): + a = trade('A', 1, 'buy', 100, 10, parent='P') + b = trade('B', 2, 'buy', 100, 10, parent='P'); b['processed_at'] = a['processed_at'] + data = {'lots': [lot(a), lot(b, 2)], 'trades': [a, b]} + with self.assertRaises(ValueError): + self.compile(data, target='A') + out = self.compile(data, target='A', lot_links={'A': 1, 'B': 2}) + self.assertEqual(out['lots'][1]['open_price'], '10.000') + with self.assertRaises(ValueError): + self.compile(data, target='A', lot_links={'A': 1, 'B': 1}) + + def test_explicit_mapping_cannot_cross_parent_or_missing_lot(self): + for links in ({'BUY': 999}, {'OTHER': 1}): + with self.assertRaises(ValueError): + self.compile(lot_links=links) + data = fixture(); data['lots'][0]['instruction_id'] = 'OTHER' + with self.assertRaises(ValueError): + self.compile(data, lot_links={'BUY': 1}) + + def test_recon_external_and_archived_evidence_are_rejected(self): + for patch in ({'lot_type': 'RECON'}, {'lot_type': 'UNKNOWN'}, {'instruction_id': None}, + {'archived_at': '2026-11-05 11:00:00'}): + with self.subTest(patch=patch): + data = fixture(); data['lots'][0].update(patch) + with self.assertRaises(ValueError): self.compile(data) + + def test_unexplained_lot_or_missing_trade_blocks_zero_baseline(self): + data = fixture(); data['lots'].append(dict(data['lots'][0], id=2, instruction_id='EXTERNAL')) + with self.assertRaises(ValueError): self.compile(data) + data = fixture(True); data['trades'] = data['trades'][1:] + with self.assertRaises(ValueError): self.compile(data, target='SELL', qty=40, price='13') + + def test_original_projection_must_match_all_persisted_fields(self): + for patch in ({'qty': 61}, {'closed_qty': 39}, {'open_price': '9'}, + {'close_avg_price': '11.999'}, {'realized_pnl': '79.99'}, {'status': 'CLOSED'}): + with self.subTest(patch=patch): + data = fixture(True); data['lots'][0].update(patch) + with self.assertRaises(ValueError): self.compile(data, lot_links={'BUY': 1}) + + def test_duplicate_trade_number_or_seq_and_identity_change_block(self): + for kind in ('trade_no', 'seq', 'identity'): + data = fixture(True) + if kind == 'seq': data['trades'][1]['seq'] = 1 + elif kind == 'trade_no': + for field in ('payload', 'original_payload'): data['trades'][1][field]['trade_no'] = 'BUY' + else: data['trades'][0]['payload']['side'] = 'sell' + with self.assertRaises(ValueError): self.compile(data) + + def test_other_code_and_beijing_exchange_never_enter_projection(self): + for code in ('600002.SH', '430001.BJ', '600001.SZ'): + data = fixture(); data['lots'][0]['ts_code'] = code + with self.assertRaises(ValueError): self.compile(data) + + def test_invalid_after_values_and_identity_or_fee_changes_block(self): + cases = [dict(qty=-1, price='10', amount='0'), dict(qty=True, price='10', amount='10'), + dict(qty=1.5, price='10', amount='15'), dict(qty=100, price='NaN', amount='0'), + dict(qty=100, price='Infinity', amount='0'), dict(qty=1, price='10.0001', amount='10'), + dict(qty=100, price='11', amount='1099.99'), dict(qty=0, price='0', amount='1'), + dict(qty=100, price='11', amount='1100', side='sell'), + dict(qty=100, price='11', amount='1100', fee='1')] + for value in cases: + with self.subTest(value=value), self.assertRaises(ValueError): + compile_correction(**fixture(), target_trade_no='BUY', after=value, today=TODAY) + + def test_odd_share_and_half_cent_keep_price_precision_and_money_rounding(self): + buy = trade('BUY', 1, 'buy', 1, '48.285') + out = self.compile({'lots': [lot(buy)], 'trades': [buy]}, qty=1, price='48.295') + self.assertEqual((out['lots'][0]['open_price'], out['amount_before'], out['amount_after']), + ('48.295', '48.29', '48.30')) + self.assertEqual(out['cash_delta'], '-0.01') + + def test_realized_pnl_rounds_after_each_sell_not_only_at_end(self): + buy = trade('BUY', 1, 'buy', 3, '10') + a, b = trade('S1', 2, 'sell', 1, '10.005'), trade('S2', 3, 'sell', 1, '10.005') + data = {'lots': [lot(buy, closed=2, close='10.005', pnl='.02')], 'trades': [buy, a, b]} + out = self.compile(data, target='S2', qty=1, price='10.015') + self.assertEqual(out['lots'][0]['realized_pnl'], '0.03') + self.assertEqual([r['realized_pnl_delta'] for r in out['allocations']], ['0.01', '0.02']) + + def test_close_average_uses_previous_stored_three_decimals(self): + buy = trade('BUY', 1, 'buy', 4, 10) + rows = [buy, trade('S1', 2, 'sell', 1, '10.001'), trade('S2', 3, 'sell', 2, '10.002'), + trade('S3', 4, 'sell', 1, '10.000')] + data = {'lots': [lot(buy, closed=4, close='10.002', pnl='0')], 'trades': rows} + out = self.compile(data, target='S3', qty=1, price='10.004') + self.assertEqual(out['lots'][0]['close_avg_price'], '10.003') + + def test_t_profit_rounds_whole_sell_separately_from_each_lot(self): + a, b = trade('A', 1, 'buy', 1, '10.005'), trade('B', 2, 'buy', 1, '10.005') + sell = trade('S', 3, 'sell', 2, '10.010') + data = {'lots': [lot(a, closed=1, close='10.010', pnl='.01', kind='T0'), + lot(b, 2, closed=1, close='10.010', pnl='.01', kind='T0')], 'trades': [a, b, sell]} + out = self.compile(data, target='S', qty=2, price='10.020') + self.assertEqual((out['t_profit_before'], out['t_profit_after'], out['t_profit_delta']), ('0.01', '0.03', '0.02')) + self.assertEqual(sum(Decimal(l['realized_pnl']) for l in out['lots']), Decimal('.04')) + + def test_inputs_are_unchanged_and_unsorted_input_is_deterministic(self): + data = fixture(True); before = deepcopy(data) + one = self.compile(data) + self.assertEqual(data, before) + data['trades'].reverse() + self.assertEqual(one, self.compile(data)) + + def test_bigint_lineage_ids_are_not_limited_to_trade_quantity_range(self): + data = fixture(); data['lots'][0]['id'] = 2**32; data['trades'][0]['seq'] = 2**32 + self.assertEqual(self.compile(data)['lots'][0]['id'], 2**32) + + +if __name__ == '__main__': + unittest.main() diff --git a/scripts/test_trade_lineage_units.py b/scripts/test_trade_lineage_units.py new file mode 100644 index 0000000..b2a0707 --- /dev/null +++ b/scripts/test_trade_lineage_units.py @@ -0,0 +1,170 @@ +"""逐笔批次凭据、原子入账失败回滚与修正仓储的隔离检查。""" +import unit_test_guard +import copy +import json +import sys +import tempfile +import unittest +from pathlib import Path +from unittest.mock import patch +sys.path.insert(0, str(Path(__file__).resolve().parents[1])) +from sqlalchemy import create_engine, event, text +from app.db import session as db +from app.core import strategy_book as books, ws_codec +from app.repo import pms_repo, rotation_repo, trade_correction_repo as repo +from app.services import ledger_service, rotation_execution, trade_lineage +from config.settings import settings + + +class BookingTests(unittest.TestCase): + def setUp(self): + self.tmp = tempfile.TemporaryDirectory(); self.addCleanup(self.tmp.cleanup) + self.engine = create_engine('sqlite:///'+self.tmp.name+'/book.sqlite') + self.addCleanup(self.engine.dispose) + @event.listens_for(self.engine, 'connect') + def connect(connection, _): + connection.create_function('GREATEST', -1, max) + @event.listens_for(self.engine, 'before_cursor_execute', retval=True) + def dialect(conn, cursor, statement, parameters, context, many): + db.assert_single_table(statement) + statement = statement.replace(' FOR UPDATE', '') + if ' ON DUPLICATE KEY UPDATE ' in statement: + statement = statement.split(' ON DUPLICATE KEY UPDATE ')[0]+' ON CONFLICT DO NOTHING' + parameters = parameters[:statement.count('?')] + return statement, parameters + for obj, name, kwargs in [ + (db, 'get_engine', {'return_value': self.engine}), + (db, '_check_cooldown', {}), (settings, 'INFRA_STRATEGY_BOOK_SCHEMA', {'new': True}), + (ledger_service.market, 'get_price', {'return_value': 10.0}), + (ledger_service.param_store, 'get_float', {'side_effect': lambda k, default=0: default})]: + p = patch.object(obj, name, **kwargs); p.start(); self.addCleanup(p.stop) + with self.engine.begin() as c: + statements = [ + 'CREATE TABLE pms_rotation_document(doc_id TEXT PRIMARY KEY,kind TEXT,payload_json TEXT,revision INTEGER,updated_at TEXT)', + 'CREATE TABLE pms_qmt_inbox(seq INTEGER PRIMARY KEY,processed INTEGER,processed_at TEXT,process_note TEXT)', + 'CREATE TABLE pms_instruction(instruction_id TEXT,book_id TEXT,ts_code TEXT,side TEXT,qty INTEGER,exec_qty INTEGER,exec_avg_price REAL,status TEXT,origin_type TEXT,origin_id TEXT,progress_json TEXT,updated_at TEXT)', + 'CREATE TABLE pms_lot(id INTEGER PRIMARY KEY AUTOINCREMENT,book_id TEXT,ts_code TEXT,lot_type TEXT,qty INTEGER,open_price REAL,open_date TEXT,closed_qty INTEGER,close_avg_price REAL,realized_pnl REAL,status TEXT,instruction_id TEXT,note TEXT,created_at TEXT,updated_at TEXT)', + 'CREATE TABLE pms_position(book_id TEXT,ts_code TEXT,status TEXT,total_qty INTEGER DEFAULT 0,avail_qty INTEGER DEFAULT 0,base_qty INTEGER,fill_qty INTEGER,add_qty INTEGER,dca_qty INTEGER,t0_qty INTEGER,avg_cost REAL,realized_t_profit REAL DEFAULT 0,cushion_pct REAL,cushion_peak REAL DEFAULT 0,cushion_state TEXT,pct_of_scale REAL,target_pct REAL,opened_date TEXT,updated_at TEXT,UNIQUE(book_id,ts_code))'] + for sql in statements: c.execute(text(sql)) + c.execute(text("INSERT INTO pms_instruction VALUES('P','original','600000.SH','buy',100,0,NULL,'CANCELLED','system',NULL,'{}','old')")) + c.execute(text('INSERT INTO pms_qmt_inbox(seq,processed) VALUES(1,0)')) + c.execute(text("INSERT INTO pms_rotation_document VALUES('account-execution','funds_reservation','{}',0,'old')")) + account = {'cash': {'original': 10000., 'micro30': 2000.}, + 'units': {'original': 10000., 'micro30': 2000.}, + 'net_capital': {'original': 10000., 'micro30': 2000.}, + 'fees': {'original': 0., 'micro30': 0.}, 'fills': {}, 'transfers': {}} + c.execute(text("INSERT INTO pms_rotation_document VALUES('capital:live','capital',:body,1,'old')"), + {'body': json.dumps(account)}) + self.act = {'instruction_id': 'P', 'child_instruction_id': 'P_D01', 'kind': 'BUY', + 'ts_code': '600000.SH', 'qty': 100, 'price': 10., 'lot_type': 'BASE', 'order_id': 'T1', + 'source_payload': {'trade_no': 'T1', 'instruction_id': 'P_D01', 'qty': 100, 'price': 10., 'fee': None}} + + def state(self): + with books.scope('all'): + return {'positions': db.fetch_all('SELECT * FROM pms_position'), + 'lots': db.fetch_all('SELECT * FROM pms_lot'), + 'parents': db.fetch_all('SELECT * FROM pms_instruction'), + 'inbox': db.fetch_all('SELECT * FROM pms_qmt_inbox'), + 'docs': db.fetch_all('SELECT * FROM pms_rotation_document ORDER BY doc_id')} + + def test_real_booking_retains_exact_lot_and_original_digest(self): + self.assertTrue(rotation_execution.book_trade(self.act, [], 1)) + receipt, _ = rotation_repo.get(trade_lineage.key('T1')) + self.assertEqual(receipt['book_id'], 'original') + self.assertEqual(receipt['instruction_id'], 'P_D01') + self.assertEqual(receipt['lot_id'], self.state()['lots'][0]['id']) + self.assertEqual(receipt['changes'][0]['before'], None) + self.assertEqual(len(receipt['original_payload_digest']), 64) + self.assertEqual(self.act['source_payload']['fee'], None) + self.assertEqual(self.state()['parents'][0]['status'], 'CONFIRMED') + capital, _ = rotation_repo.get('capital:live') + self.assertEqual(capital['cash'], {'original': 9000., 'micro30': 2000.}) + + def test_duplicate_does_not_repeat_cash_lots_or_lineage(self): + rotation_execution.book_trade(self.act, [], 1) + before = self.state() + self.assertFalse(rotation_execution.book_trade(self.act, [], 1)) + self.assertEqual(self.state(), before) + + def test_lineage_write_failure_rolls_back_entire_booking(self): + before = self.state() + with patch.object(trade_lineage.rotation_repo, 'put_once', side_effect=RuntimeError('storage fault')): + with self.assertRaises(RuntimeError): rotation_execution.book_trade(self.act, [], 1) + self.assertEqual(self.state(), before) + + def test_conflicting_existing_lineage_does_not_overwrite_or_book(self): + rotation_repo.put_once(trade_lineage.key('T1'), 'trade_lineage', {'book_id': 'micro30'}) + before = self.state() + with self.assertRaises(ValueError): rotation_execution.book_trade(self.act, [], 1) + self.assertEqual(self.state(), before) + + def test_missing_new_lot_mapping_rolls_back_cash(self): + before = self.state() + with patch.object(trade_lineage, 'lots', return_value=[]): + with self.assertRaises(ValueError): rotation_execution.book_trade(self.act, [], 1) + self.assertEqual(self.state(), before) + + +class EvidenceTests(unittest.TestCase): + def row(self, seq, payload, **fields): + return {'seq': seq, 'payload_json': json.dumps(payload), **fields} + + def test_full_history_limit_is_not_silent_truncation(self): + with patch.object(repo.db, 'fetch_all', return_value=[self.row(1, {'instruction_id': 'I'})]): + with self.assertRaises(ValueError): repo.trades_for_orders(['I'], limit=1) + + def test_trade_identity_is_checked_despite_key_truncation(self): + with patch.object(repo.db, 'fetch_one', return_value=self.row(1, {'trade_no': 'another'}, msg_type='trade')): + with self.assertRaises(ValueError): repo.get_trade('T') + + def test_amendment_conflict_on_other_child_is_not_hidden(self): + first = self.row(1, {'correction_id': 'C', 'instruction_id': 'I'}) + second = self.row(2, {'correction_id': 'C', 'instruction_id': 'wrong'}, processed=3) + with patch.object(repo.db, 'fetch_all', side_effect=[[first], [second]]): + self.assertEqual([r['seq'] for r in repo.amendments('C', 'I')], [1, 2]) + + def test_original_payload_child_beats_historical_envelope_corr_id(self): + rows = [self.row(1, {'instruction_id': 'I'}, corr_id='old-correlation', dedup_key='trade:T'), + self.row(2, {'instruction_id': 'I'}, corr_id='I', dedup_key=None)] + with patch.object(repo.db, 'fetch_all', return_value=rows): + self.assertEqual([r['seq'] for r in repo.trades_for_orders(['I'])], [1]) + + def test_bad_correction_is_returned_with_seq_for_individual_hold(self): + rows = [self.row(1, ['bad']), self.row(2, {'correction_id': 'valid'})] + with patch.object(repo.db, 'fetch_all', return_value=rows): + found = repo.pending() + self.assertEqual(found[0]['seq'], 1) + self.assertIsNone(found[0]['payload']) + self.assertEqual(found[1]['payload']['correction_id'], 'valid') + + def test_unrelated_bad_correction_does_not_poison_valid_pair_lookup(self): + rows = [self.row(1, ['bad'], processed=3), + {'seq': 2, 'payload_json': '{bad', 'processed': 3}, + self.row(3, {'correction_id': 'valid', 'instruction_id': 'I'})] + with patch.object(repo.db, 'fetch_all', return_value=rows): + self.assertEqual([r['seq'] for r in repo.corrections('valid', 'I')], [3]) + + def test_unrelated_bad_amendment_does_not_poison_global_conflict_lookup(self): + first = self.row(3, {'correction_id': 'valid', 'instruction_id': 'I'}) + conflicts = [self.row(1, ['bad'], processed=3), + {'seq': 2, 'payload_json': '{bad', 'processed': 3}] + with patch.object(repo.db, 'fetch_all', side_effect=[[first], conflicts]): + self.assertEqual([r['seq'] for r in repo.amendments('valid', 'I')], [3]) + + def test_bad_payload_with_matching_business_key_remains_conflict_evidence(self): + for kind in ('trade_correction', 'order_update'): + key = ws_codec.dedup_key(kind, {'correction_id': 'C', 'amended': True}) + bad = {'seq': 1, 'payload_json': '{bad', 'processed': 3, 'dedup_key': key} + valid = self.row(2, {'correction_id': 'C', 'instruction_id': 'I'}) + with self.subTest(kind=kind): + if kind == 'trade_correction': + with patch.object(repo.db, 'fetch_all', return_value=[bad, valid]): + found = repo.corrections('C', 'I') + else: + with patch.object(repo.db, 'fetch_all', side_effect=[[valid], [bad]]): + found = repo.amendments('C', 'I') + self.assertEqual([r['seq'] for r in found], [1, 2]) + self.assertIsNone(found[0]['payload']) + + +if __name__ == '__main__': unittest.main() diff --git a/scripts/test_wiring.py b/scripts/test_wiring.py index 82853ef..d8bc0b2 100644 --- a/scripts/test_wiring.py +++ b/scripts/test_wiring.py @@ -489,9 +489,29 @@ class FakeRepo: def update_strategy(self, strategy_id, **fields): s = self.strategies.get(strategy_id) + upd = dict(fields) + unset = object() + expected = upd.pop('correction_expected', unset) + if 'state_json' in upd: + upd['state'] = json.loads(upd.pop('state_json')) + if 'state' in upd: + incoming = upd['state'] + if not isinstance(incoming, dict): + raise ValueError('策略状态必须是对象') + correction = incoming.get('trade_correction') or {} + if not isinstance(correction, dict): + raise ValueError('策略成交更正版本异常') + if expected is unset: + expected = correction.get('correction_id') + if expected is not None and (not isinstance(expected, str) or not expected): + raise ValueError('策略成交更正版本异常') + actual = ((s or {}).get('state', {}).get('trade_correction') or {}).get('correction_id') + if not s or actual != expected: + raise RuntimeError('策略成交更正版本已变化或策略不存在,拒绝覆盖旧状态') + elif expected is not unset: + raise ValueError('成交更正版本比较必须同时保存策略状态') if not s: return 0 - upd = dict(fields) # 与真 repo 同义: params/state 传 dict 落 *_json 列; 其余按白名单更新 touched = False for k in ("params", "state"): @@ -570,6 +590,12 @@ class FakeQmtRepo: def orders_for_parent(self, parent_id): return [dict(o) for o in self.orders.values() if o['parent_id'] == parent_id] + def list_orders(self, *, statuses=None, parent_id=None, ts_code=None, limit=200): + return [dict(o) for o in reversed(list(self.orders.values())) + if (not statuses or o['status'] in statuses) + and (not parent_id or o['parent_id'] == parent_id) + and (not ts_code or o['ts_code'] == ts_code)][:int(limit)] + def orders_for_fill(self, *, order_id, broker_order_id=None): return [r for r in self.orders.values() if r["instruction_id"] == str(order_id) @@ -588,9 +614,9 @@ class FakeQmtRepo: "trade_no": trade_no or f"T-{seq}"}} return self.inbox[int(seq)] - def inbox_pending(self, limit=500): + def inbox_pending(self, limit=500, msg_type=None): rows = [r for r in sorted(self.inbox.values(), key=lambda x: x["seq"]) - if int(r["processed"]) == 0] + if int(r["processed"]) == 0 and (not msg_type or r['msg_type'] == msg_type)] return [dict(r, payload=dict(r["payload"])) for r in rows[:int(limit)]] def inbox_mark(self, seqs, processed=1, note=None): @@ -674,7 +700,7 @@ def install_fakes(prices=None, positions=None, params=None, high5=None, prev_clo fake.consensus_stats = FakeConsensusStatRepo() for name in ("upsert_many", "delete_day_kind", "list_range", "count_by_kind"): setattr(consensus_stat_repo, name, getattr(fake.consensus_stats, name)) - for _n in ("get_state", "inbox_pending_count", "queue_depth", "enqueue_order", + for _n in ("get_state", "inbox_pending_count", "queue_depth", "enqueue_order", "list_orders", "request_cancel", "get_order", "orders_for_parent", "orders_for_fill", "inbox_pending", "inbox_mark", "inbox_orphan_count", "latest_snapshot"): setattr(qmt_repo, _n, getattr(fake.qmt, _n)) @@ -684,9 +710,24 @@ def install_fakes(prices=None, positions=None, params=None, high5=None, prev_clo raise RuntimeError('执行依据缓存没有写入父指令') fake.instructions[iid].setdefault('progress', {})['execution_basis_cache'] = value execution_repo.save_basis = save_basis + from app.core import ws_codec + def pending_correction(row): + return ws_codec.is_correction(row.get('msg_type'), row.get('payload')) and ( + row.get('processed') in (0, 3) or row.get('processed') == 2 and row.get('process_note') == + str(row.get('msg_type'))+' 由通道进程消化, 不入账') execution_repo.pending_trades = lambda ids: sum(1 for row in fake.qmt.inbox.values() - if row.get('msg_type') == 'trade' and row.get('processed') in (0, 3) - and (row.get('payload') or {}).get('instruction_id') in ids) + if (row.get('msg_type') == 'trade' and row.get('processed') in (0, 3) or pending_correction(row)) + and ((row.get('payload') or {}).get('instruction_id') in ids + or row.get('corr_id') in ids)) + def pending_corrections(ts_code=None, child_ids=None): + return sum(1 for row in fake.qmt.inbox.values() + if pending_correction(row) + and (not ts_code and not child_ids or (row.get('payload') or {}).get('ts_code') == ts_code + or (row.get('payload') or {}).get('instruction_id') in (child_ids or []) + or row.get('corr_id') in (child_ids or []))) + execution_repo.pending_corrections = pending_corrections + from app.repo import trade_correction_repo + trade_correction_repo.pending = lambda limit=100: [] fake.params.update(params or {}) for p in (positions or []): base = {"ts_code": p["ts_code"], "status": "HOLDING", "frozen_reason": "NONE",