研究信号独立性与分组权重

This commit is contained in:
baobao 2026-11-11 15:10:55 +08:00
parent 34bc09b8c2
commit 00a7159bc9
24 changed files with 2630 additions and 0 deletions

View File

@ -1,5 +1,9 @@
# 开发节点记录(三系统量化交易)
## 2026-11-11 按独立信息研究信号权重
按用户澄清,不使用未来收益或买卖成功率。只读提取24个已结束交易日的沪深信号,先固定协议,再做来源去重、共同观察相关和时间匹配。资金流强度与资金分布相关0.898,长期对照0.888,应合组。发现QRS354条中337条引用旧日线,实际告警循环未清跨日日线缓存,九组断网边界含真实缓存类复现通过。仅将原公式重算作为单列探索对照,得到三基础组36/32/32点估计,日期及股票重采样均覆盖等权,建议三组各1/3,Bionic组合研判不另增独立预算。未改线上规则或上游服务,根因和待办见[研究接续](docs/信号独立性权重研究接续_2026-11-11.md)。
## 2026-11-11 信号历史覆盖与既有标定核验
14:33至14:38只读188发布系统文件、Redis及上游数据库。确认M14入场、逐股QRS、v3f资金异动与市场实况已有标定;当天QRS120只沪深中66只成功、54只兜底。上游通知明确资金流强度7月21日及以前数值作废,现阈值仍待再校准;资金分布仍为临时值。资金实发流始于10月8日,可联合整理24个已结束交易日期,但不等于完整当时可得回放。Bionic原始广播只剩11月5日起,长期审计另在;实况65日生产与116日回填分开;4只沪深分钟行情抽查240日,不能当一年实发信号。仅文档交付,未改交易代码、参数、账本或服务,详见[核验记录](docs/信号历史覆盖与发布阈值核验_2026-11-11.md)。

View File

@ -0,0 +1,17 @@
# 信号独立性权重研究接续
用户明确权重目标是独立信息,不是涨跌预测或交易成功率。2026年11月11日完成只读数据提取、来源分析、日期核验及断网研究;未改线上代码或参数,未操作188真实PMS。
详细结果、协议和复现入口见[研究报告](../research/signal_independence_20261111/README.md)。研究主窗10月8日至11月10日24个已结束交易日。统计只使用沪深,不读未来收益或成交结果。
主要结论:
- 资金流强度与资金分布共享主力资金原料,53个共同股票日相关0.898、一致率96.23%;7月22日起216个共同股票日相关0.888。应共享预算,不能各计一票。
- 当前实发QRS有严重新鲜度问题:354条中337条数据日过旧。实际代码的日线内存缓存跨日未clear,离线复现通过。净化后只有17条QRS,无法直接标定完整实发权重。
- 对原354个股票日按已有日线、原公式重算,作为明确区分的探索对照,三基础组冗余折扣约36/32/32;所有不确定区间均包含三等分,敏感性中排序改变。建议技术QRS、主力资金、v3f资金异动各1/3作为初始结构预算,不称精确最优。
- Bionic与资金有明显重叠,独立增量尚不可识别。建议组合研判引用已有组预算,不另增加一份独立票;这不等于模型无价值,也不是统计测得模型应为零。不能将示意20%冒充数据定值。
- 缺失组预算保留,不把剩余来源归一为100%。量化BUY、分钟QRS及实况分缺少本窗独立样本或含派生成分,不额外分配新预算。
验证:开发机隔离断网完成统计、原QRS公式重算、按日期1000次与股票500次重采样;九组边界通过。Bionic审计与68条广播时间匹配通过;Redis流ID相对发布者慢约266秒,先后关系不能直接按流ID判断。限定沪深的研究输入已归档到研究目录data,连同代码、协议、聚合及哈希保存,无需再次连接生产服务即可复算。
待处理:QRS跨日缓存及来源新鲜度、已有量化入场QRS输入不可用、Bionic固定95%、恢复与旧退出状态冲突。它们不是本次研究已修复事项。上线前须先解决有效证据和来源身份,不能把旧QRS直接以三分之一预算投入运行;研究结果不自动上线。

View File

@ -1,5 +1,7 @@
# 开发资料入口
以独立信息为目标的权重研究,见[11月11日研究接续](信号独立性权重研究接续_2026-11-11.md)及[完整报告](../research/signal_independence_20261111/README.md)。确认资金两路同源,提出三基础组近似等权、模型共享证据预算;同时发现QRS跨日缓存旧数据,尚未上线修复。
信号实际历史保留、可研究窗口及上游既有标定,见[11月11日覆盖与阈值核验](信号历史覆盖与发布阈值核验_2026-11-11.md)。区分实发、审计与回填;资金流强度7月21日及以前数值失效,不能混入研究。
多源发布公式、分组加权、无信号状态和按证据时间合成,见[11月11日多源择时评估](多源择时加权与时序边界评估_2026-11-11.md)。含量化择时当日四轮输入不可用的只读证据;尚未实施融合或修复上游数据。

View File

@ -0,0 +1,103 @@
# 信号独立性与分组权重研究
研究日期:2026年11月11日。目标是减少同一信息的重复计票,使用现有数据估计来源依赖,不以未来收益、交易成功率或盈亏优化权重。仅沪深;没有线上改动。
## 建议参数及其含义
当前更稳妥的建议是三类基础证据预算各占三分之一。将资金流强度与资金分布合为同一组,Bionic作为组合研判保留,不再额外增加一组独立票数。
| 基础证据组 | 对应现有来源 | 研究建议预算 |
| --- | --- | --- |
| 技术形态 | 此次有资料可评估的逐股QRS | 1/3 |
| 主力资金结构 | 资金流强度、资金分布吸筹/出货,共享预算 | 1/3 |
| 盘中价量资金异动 | v3f资金异动 | 1/3 |
这些是来源预算,不是交易仓位,不是胜率。Bionic额外独立票权暂不分配,是避免重复预算的架构建议,不是统计测得“模型信息量等于零”。模型仍可解释、复核与指出矛盾;若其观点引用上述证据,归回该证据组,不能再让来源包装增加票数。独立新闻、事实或新证据如需增设来源,必须另有来源身份和数据支持,不能靠模型自报置信度获得预算。
本次没有数据支持把整个“量价系统”均赋予同一权重:量化BUY在主要窗口没有实发样本;分钟QRS、放量、日频资金已组合进BUY,不能将BUY与其成分同时作为新来源。上述技术组估计仅对应逐股日频QRS;资金实况分与v3f共同使用价量原料,本次没有独立连续实况轨迹,不给实况分额外预算。
组内不要按消息数量累计。重复、镜像及同证据日重发只算一次;同组出现矛盾需保留分歧,不能只挑更合意的方向。缺数据时保留缺失组的预算,不将剩余组自动扩至100%;新鲜度和覆盖度是是否可用的条件,不能冒充信息独立度。
## 一、样本与预先固定的口径
先写入`protocol.json`,再导出数据和计算。主要窗口为10月8日至11月10日,24个已结束交易日。量化告警另取7月22日起历史检验同源关系。数据来自188上游容器的本地审计、Redis发布流和Bionic审计表,均为只读。实际不调用模型、不访问真实PMS账本、不重放交易。初始缓存位于`/private/tmp/signal-independence-20261111/`;交付时将限定沪深的研究输入与规范化事件归档至本目录`data/`,避免上游TTL到期后丢失复现资料。
| 来源 | 主要窗口事件数 | 股票数 | 有事件日期 | 股票日数 |
| --- | ---: | ---: | ---: | ---: |
| 逐股QRS | 354 | 106 | 24 | 354 |
| 资金流强度 | 722 | 87 | 12 | 163 |
| 资金分布 | 972 | 178 | 24 | 452 |
| 资金异动 | 44094 | 4852 | 24 | 31075 |
| Bionic明确方向 | 199 | 87 | 24 | 139 |
Bionic总审计2641次,其中MAINTAIN 2182次、MAINTAIN_HOLD 260次、转多139次、转弱48次、止盈12次。“维持”没有明确的新方向,不填成中性票。明确方向中189次为预案条件触发、10次为持仓体检。代码显示这些路径会读取当前价量、资金分布及板块QRS,不能因模型是另一套服务就认为其输入独立。
资金流强度在10月26日至11月10日没有发布记录;这是事件缺席,不能仅据此判定“正常中性”,也不能证明采集故障。相关性只用两路均有实际观察的股票日;不把未发布或未覆盖的股票填0。方向一致率不是买卖成功率。
## 二、可以明确确认的重复信息
资金流强度与资金分布来自同一主力资金原料。24日窗口共同股票日53个、11个日期,方向一致率96.23%,方向相关系数0.898,按日期整组重采样95%区间约0.700至1.000。拉长到7月22日起,共同股票日216个,相关系数0.888,结论一致。
30交易分钟最近匹配时,两者146次匹配事件方向全部一致,涉及46个股票日、11个日期;最近匹配可能多对一,不能把146次当成146个独立样本。15和60分钟检查也维持高度一致。这里代码同源与观测结果互相支持,足以支持合组,不能将两者各给一份完整预算。
Bionic与合并后的主力资金组共同股票日39个,方向一致87.18%、相关0.739;样本低于事先设定的50个股票日,不据此给模型精确独立权重。与资金异动共同股票日82个,日内最后方向相关0.441;30分钟匹配44次,涉及37个股票日、16个日期,方向一致95.45%。同一天但不同时间的相关程度明显不同,不能只用日末低相关给模型增权。
这不证明Bionic完全没有增量判断能力。现有资料没有保存完整输入及每条结论的新证据身份,无法把模型独立解释与复述、旧计划延续稳定分开。证据不足的部分保留未识别,不假定其独立度为零或一,也不声称20%已由数据标定。
## 三、质量核验改变了QRS研究边界
354条QRS发布中,337条的数据日不是前一交易日,数据日与发布日的自然日差中位数41天;248条还是同股、同数据日、同方向的重复证据。只有17条使用前一交易日数据。交易日期用600519.SH实际日线核对,不用自然日硬判周末。
字段含义已核实:发布metadata中的trade_date来自QRS实际参与计算的日线最后一行,不是“上次穿越日期”。188当前三个相关文件与本地逐字哈希一致。`KlineDailyBulkMemCache.preload`会跳过已预加载股票,`clear()`要求每日调用;当前告警循环盘前只清空QRS结果,不清这个日线缓存。用当前真实缓存类与内存数据离线复现:第二天preload仍返回旧日线,显式clear后才读到新数据。本次未改上游代码或重启服务。
剔除旧记录后,QRS与主力资金共同股票日仅5个,与资金异动仅7个,与Bionic仅3个;不足以按原实发数据估计完整权重。不能把旧状态不响应当天变化表现出的低相关当成科学的独立性。
因此追加`protocol_v2.json`:只对原354个发布股票日,利用已有历史日线按原生产N=18、M=60公式重算截至前一交易日的原始QRS。没有套用今天阈值,没有补造历史发布,重算值的正负仅用于原始指标依赖对照。该设计是在看到质量问题后追加,属于探索性补充,不是独立样本外实验。
354个股票日全部可重算,其中138个原始指标方向与旧发布方向不同。按旧证据日期复现原值时,295条误差小于1e-6,误差中位数0、95分位0.00506、最大0.19359;剩余差异的历史修订与暖机原因未逐条定位。因此重算对照不能宣称完全还原当时可得输入。
## 四、权重计算结果
将两路主力资金合组后,对共同有观察的股票日计算方向相关。以相同先验预算为起点,使用可解释的冗余折扣:
`u_i = 1 / (1 + Σ(j≠i)|ρ_ij|)`,再令`w_i = u_i / Σu_i`。
这是一条明确约定的去冗余规则,不是数学上唯一的“科学最优权重”。使用绝对值,避免把完全反向的镜像误认为独立;缺失相关不能直接填0。此处仅研究方向,不能据此证明完整连续分数或所有市场状态下统计独立。三个组两两共同样本不同,三组完整共同观察仅44个股票日,存在选择偏差。
| 组合 | 共同股票日 | 相关系数 | 日期重采样95%区间 |
| --- | ---: | ---: | --- |
| 重算QRS与主力资金 | 88 | -0.062 | -0.322至0.130 |
| 重算QRS与资金异动 | 135 | -0.039 | -0.190至0.111 |
| 主力资金与资金异动 | 273 | 0.184 | 0.037至0.323 |
| 基础证据组 | 冗余折扣点估计 | 按日期重采样95%区间 | 按股票重采样95%区间 |
| --- | ---: | --- | --- |
| QRS | 35.92% | 31.89%至37.88% | 32.48%至37.60% |
| 主力资金 | 31.75% | 29.75%至34.19% | 29.86%至34.47% |
| 资金异动 | 32.33% | 30.65%至36.65% | 30.62%至36.00% |
按日期整组重采样1000次、按股票整组500次,固定随机种子20261111;没有把同一天成千上万条消息视为独立样本。区间仅是当前样本及该计算规则下的不确定范围,不包含全部模型选择、历史修订、跨市场阶段风险。
敏感性:日内第一方向给出38.76/30.72/30.52,日内均值37.15/31.67/31.18;前半窗36.46/32.94/30.60,后半窗32.58/30.85/36.57。后半窗QRS共同样本尤其薄,不用于单独定参。主力资金只用覆盖全窗的资金分布代表时为33.44/31.18/35.38。技术与资金异动的排序并不稳定。
所以36/32/32可作为记录在案的探索点估计,三个1/3是更稳妥的初始结构参数。不能声称36显著优于33,也不能把Bionic硬填10%、20%说成数据已经给出的答案。当前结论的确定部分是合并同源、删除镜像票、保留缺失预算;精细差异尚不足以支持长期固定偏置。
若后续上线,先修QRS跨日缓存并建立证据日期与来源身份,原组权重仅在有效状态下贡献;当前不应直接将旧QRS赋予三分之一实际票权。此前确认的Bionic固定95%、旧退出状态阻断恢复等问题仍未实施修复。
## 五、时间与验证
68条Bionic原始广播与审计逐股同结论匹配全部成功。审计按UTC转北京后,与广播时间差中位数0.45秒、最大1.006秒。Redis流ID时间比发布者时间慢约265至267秒,不能直接将流ID当同一时钟推先后因果。本研究匹配使用发布者时间;MTF只给分钟精度。匹配前后事件只是关联性检查,没有用未来事件构造线上策略状态。
开发机在项目隔离配置、断网护栏下完成原始分析、重算、重采样及九组边界验证。验证包含缺失不填0、常量相关不可用、反向镜像冗余、对称数据等权、资金组不重复、午休时钟、沪深过滤和真实缓存类跨日复现。未做收益回测,未声明交易获利改善。
```bash
cd /private/tmp/pms-plan-input-20261111
/private/tmp/pms-audit311/bin/python scripts/unit_test_guard.py research/signal_independence_20261111/test_research.py
/private/tmp/pms-audit311/bin/python scripts/unit_test_guard.py research/signal_independence_20261111/analyze.py
/private/tmp/pms-audit311/bin/python scripts/unit_test_guard.py research/signal_independence_20261111/weights.py
/private/tmp/pms-audit311/bin/python scripts/unit_test_guard.py research/signal_independence_20261111/diagnostics.py
```
复现读取本目录`data/`及本机上游源码,不必再次连接生产服务。`manifest.json`保存原始三个文件哈希,`weights.json`另记日线哈希;`source_hashes.json`保存用到的上游代码哈希及与188的核对结果。上游源码若变动,须恢复所记录版本后复现,不能把新公式结果当作本次结果。当前结果对应特定时点,不是永久健康证明。
方法参考仅用于明确“关联、冗余与独立增量”之间的区别,不把监督学习的未来收益目标引入本研究:[JMLR关于冗余、相关与互补信息的区分](https://www.jmlr.org/beta/papers/v24/21-0482.html)。实际采用的是本文公开的简单依赖折扣,并未声称实现该论文算法。

View File

@ -0,0 +1,191 @@
"""冻结协议后的离线独立性分析;不读取收益,不连接任何外部服务。"""
from __future__ import annotations
import collections
import datetime as dt
import gzip
import hashlib
import itertools
import json
import math
from pathlib import Path
import re
import numpy as np
import pandas as pd
ROOT = Path(__file__).resolve().parent
DATA = ROOT / 'data'
OUT = ROOT
TZ = dt.timezone(dt.timedelta(hours=8))
START, END = '2026-10-08', '2026-11-10'
SOURCES = ['qrs', 'intensity', 'distribution', 'mtf', 'bionic']
LABELS = {'qrs':'逐股QRS','intensity':'资金流强度','distribution':'资金分布','mtf':'资金异动','bionic':'决策系统'}
HS = re.compile(r'(?:(?:60|68)\d{4}\.SH|(?:00|30)\d{4}\.SZ)$')
def code(s):
s = str(s)
if s[:2] in ('SH','SZ'): s=s[2:]+'.'+s[:2]
if not HS.fullmatch(s): raise ValueError('non-HS row')
return s
def load():
rows=[]; raw=[]; broadcasts=[]; provenance=collections.Counter(); all_judgements=[]
manifest={}
for p in [DATA/name for name in ['itd.jsonl.gz','bionic.jsonl.gz','streams.jsonl.gz']]:
manifest[p.name]={'sha256':hashlib.sha256(p.read_bytes()).hexdigest(),'bytes':p.stat().st_size}
records=[json.loads(x) for x in gzip.open(p,'rt')]
manifest[p.name]['rows']=len(records)
for r in records:
if r['kind']=='itd':
c=code(r['ts_code']);day=r['trade_date'];t=dt.datetime.fromtimestamp(int(r['trigger_time'])/1000,TZ)
meta=r.get('metadata') or {};name=r['source'];parts=meta.get('contributing_sources',[]) if name=='multi_source' else [name]
raw.append({'day':day,'source':name,'code':c})
for part in parts:
if part.startswith('daily_qrs_symmetric'):s='qrs';direction=-1 if part.endswith('down') else 1
elif part.startswith('money_flow_'):s='intensity';direction=-1 if '_out_' in part else 1
elif part.startswith('capital_'):s='distribution';direction=-1 if part.endswith('distribution') else 1
elif part=='intraday_buy_emitted':continue
else:raise ValueError('unknown alert source '+part)
value=(meta.get('contributing_values') or {}).get(part,r.get('value'))
rows.append({'day':day,'source':s,'code':c,'t':t,'direction':direction,'value':float(value),'id':r['alert_id']+':'+part,
'parent_source':name,'evidence_date':str(meta.get('trade_date') or '')[:10], 'level':r['level']})
elif r['kind']=='mtf':
c=code(r['ts_code']);received=dt.datetime.fromtimestamp(int(r['stream_id'].split('-')[0])/1000,TZ)
day=received.date().isoformat();t=dt.datetime.fromisoformat(day+'T'+r['calc_time']).replace(tzinfo=TZ)
rows.append({'day':day,'source':'mtf','code':c,'t':t,'direction':1 if r['direction']=='inflow' else -1,
'value':float(r['z_dd']),'id':r['stream_id'],'parent_source':'mtf','evidence_date':day,'level':r.get('mode')})
elif r['kind']=='bionic_audit':
c=code(r['stock_code']);day=dt.datetime.strptime(str(r['audit_date']),'%Y%m%d').date().isoformat()
# 审计库存UTC naive时间;随后用同股同结论广播独立核对偏移。
t=dt.datetime.fromisoformat(r['created_at']).replace(tzinfo=dt.timezone.utc).astimezone(TZ)
if t.date().isoformat()!=day: raise ValueError('audit timezone/date mismatch')
trigger=r['reason'].split('源:',1)[-1].split(' | 判定:',1)[0]
prefix=re.match(r'【盘中([^】]+)】',r['reason']).group(1)
provenance[(prefix,trigger.split('(')[0].strip())]+=1
all_judgements.append({'day':day,'code':c,'t':t,'verdict':r['verdict'],'prefix':prefix,'trigger':trigger})
direction={'REVERSAL_BUY':1,'REVERSAL_SELL':-1,'TAKE_PROFIT':-1}.get(r['verdict'])
if direction is None:continue # 维持不是中性票,不从旧计划反推方向。
rows.append({'day':day,'source':'bionic','code':c,'t':t,'direction':direction,'value':direction,
'id':f'{c}:{t.isoformat()}:{r["verdict"]}','parent_source':prefix,'verdict':r['verdict'],'trigger':trigger,'evidence_date':'','level':''})
elif r['kind']=='bionic_broadcast':
r['code']=code(r['ts_code']);broadcasts.append(r)
df=pd.DataFrame(rows).drop_duplicates(['source','code','id']).sort_values(['t','source','code'])
# 合成告警拆为原成分,同一时刻同股票同成分只保留一条。
df=df.drop_duplicates(['source','code','t','direction'],keep='last')
OUT.joinpath('manifest.json').write_text(json.dumps({'input':manifest,'scope':'沪深','protocol_sha256':hashlib.sha256((ROOT/'protocol.json').read_bytes()).hexdigest()},indent=2,ensure_ascii=False))
return df,pd.DataFrame(raw),broadcasts,pd.DataFrame(all_judgements),provenance
def eligible(x,y,days):
return len(x)>=50 and len(set(days))>=10 and min(sum(x>0),sum(x<0),sum(y>0),sum(y<0))>=10
def corr(x,y):
x=np.asarray(x,dtype=float);y=np.asarray(y,dtype=float)
if len(x)<2 or np.std(x)<1e-12 or np.std(y)<1e-12:return None
return float(np.corrcoef(x,y)[0,1])
def bootstrap_pairs(frame,group,seed=20261111,n=1000):
rng=np.random.default_rng(seed);blocks=[g[['x','y']].to_numpy() for _,g in frame.groupby(group)]
estimates=[]
for _ in range(n):
v=np.concatenate([blocks[i] for i in rng.integers(len(blocks),size=len(blocks))]);r=corr(v[:,0],v[:,1])
if r is not None:estimates.append(r)
return [float(x) for x in np.quantile(estimates,[.025,.5,.975])] if estimates else None
def pair_stats(panel,bootstrap=True):
results=[]
for a,b in itertools.combinations(SOURCES,2):
if a not in panel or b not in panel:continue
p=panel[[a,b]].dropna().reset_index().rename(columns={a:'x',b:'y'})
if p.empty:continue
x,y=p.x.to_numpy(),p.y.to_numpy();r=corr(x,y)
item={'a':a,'b':b,'n':len(p),'dates':p.day.nunique(),'stocks':p.code.nunique(),'corr':r,
'agreement':float(np.mean(np.sign(x)==np.sign(y))),
'a_positive':int(sum(x>0)),'a_negative':int(sum(x<0)), 'b_positive':int(sum(y>0)),'b_negative':int(sum(y<0)),
'eligible':bool(eligible(x,y,p.day))}
if bootstrap and r is not None and len(p)>20:
item['day_ci']=bootstrap_pairs(p,'day');item['stock_ci']=bootstrap_pairs(p,'code',n=500)
results.append(item)
return results
def panels(df,how='last'):
group=df.groupby(['day','code','source'])
if how=='last':return group.direction.last().unstack('source')
if how=='first':return group.direction.first().unstack('source')
if how=='mean':return group.direction.mean().unstack('source')
raise ValueError(how)
def trade_minute(t):
minute=t.hour*60+t.minute+t.second/60
if 570<=minute<=690:return minute-570
if 780<=minute<=900:return minute-660
return None
def event_matching(df):
result=[]
# QRS是日状态;盘中匹配只比较真正按分钟更新的来源。
for a,b in itertools.combinations(['intensity','distribution','mtf','bionic'],2):
for window in [15,30,60]:
matches=[]
by_b={k:g for k,g in df[df.source==b].groupby(['day','code'])}
count=0
for key,g in df[df.source==a].groupby(['day','code']):
q=by_b.get(key)
if q is None:continue
qt=[(trade_minute(r.t),r) for r in q.itertuples()]
qt=[x for x in qt if x[0] is not None]
if not qt:continue
for row in g.itertuples():
now=trade_minute(row.t)
if now is None:continue
count+=1
lag,other=min(((z-now,r) for z,r in qt),key=lambda x:abs(x[0]))
if abs(lag)<=window:matches.append({'day':key[0],'code':key[1],'x':row.direction,'y':other.direction,'lag':lag})
p=pd.DataFrame(matches)
result.append({'a':a,'b':b,'window':window,'eligible_a_events':count,'matched':len(p),
'agreement':float((p.x==p.y).mean()) if len(p) else None,
'corr':corr(p.x,p.y) if len(p) else None,
'median_lag_b_minus_a':float(p.lag.median()) if len(p) else None})
return result
def timezone_check(df,broadcasts):
b=df[df.source=='bionic'];diff=[];clock=[]
for r in broadcasts:
t=dt.datetime.fromtimestamp(int(r['trigger_time'])/1000,TZ)
p=b[(b.code==r['code']) & (b.day==r['trade_date']) & (b.verdict==r['verdict'])]
if len(p):diff.append(float(min(abs((x-t).total_seconds()) for x in p.t)))
rec=dt.datetime.fromtimestamp(int(r['stream_id'].split('-')[0])/1000,TZ)
clock.append((rec-t).total_seconds())
return {'broadcasts':len(broadcasts),'matched_audits':len(diff),
'audit_broadcast_abs_sec_quantiles':np.quantile(diff,[0,.5,.95,1]).tolist() if diff else [],
'redis_minus_publisher_sec_quantiles':np.quantile(clock,[0,.5,.95,1]).tolist() if clock else []}
def main():
df,raw,broadcasts,judgements,provenance=load();primary=df[(df.day>=START)&(df.day<=END)].copy()
summaries=[]
for s,g in primary.groupby('source'):
groups=g.groupby(['day','code']);daily=g.groupby('day').size()
summaries.append({'source':s,'events':len(g),'stocks':g.code.nunique(),'days':g.day.nunique(),'stock_days':len(groups),
'positive_events':int(sum(g.direction>0)),'negative_events':int(sum(g.direction<0)),
'daily_events_mean':float(daily.mean()),'daily_events_median':float(daily.median()),
'direction_changing_stock_days':int((groups.direction.nunique()>1).sum()),
'duplicate_fixed_qrs':len(g)-len(groups) if s=='qrs' else None})
panel=panels(primary)
result={'window':[START,END],'source_summary':summaries,'pairs':pair_stats(panel),
'first_pairs':pair_stats(panels(primary,'first'),False),'mean_pairs':pair_stats(panels(primary,'mean'),False),
'halves':{},'secondary':pair_stats(panels(df[df.source!='bionic']),False),
'event_matches':event_matching(primary),'timezone':timezone_check(primary,broadcasts),
'bionic_trigger_directions':judgements.prefix.value_counts().to_dict(),
'bionic_verdicts':judgements.verdict.value_counts().to_dict(),
'bionic_directional_triggers':primary[primary.source=='bionic'].parent_source.value_counts().to_dict(),
'bionic_trigger_examples': [{'type':k[0],'trigger':k[1],'n':n} for k,n in provenance.most_common(15)]}
days=sorted(primary.day.unique());mid=days[len(days)//2]
for name,d in [('first',primary[primary.day<mid]),('second',primary[primary.day>=mid])]:result['halves'][name]={'start':d.day.min(),'end':d.day.max(),'pairs':pair_stats(panels(d),False)}
# 审计是否在本窗口所有日期均有,不将没有发信号的股票日补0。
result['presence_dates']={s:sorted(g.day.unique().tolist()) for s,g in primary.groupby('source')}
OUT.joinpath('results.json').write_text(json.dumps(result,ensure_ascii=False,indent=2))
# 归一化事件与原始沪深研究输入一并留存,避免上游TTL到期后无法复现。
primary.to_csv(DATA/'normalized.csv.gz',index=False,compression='gzip')
panel.to_csv(DATA/'paired_daily.csv.gz',compression='gzip')
print(json.dumps({'summary':summaries,'pairs':result['pairs'],'timezone':result['timezone'],'bionic':result['bionic_directional_triggers']},ensure_ascii=False,indent=2))
if __name__=='__main__':main()

View File

@ -0,0 +1,114 @@
{
"historical_qrs_reproduction": {
"rows": 354,
"within_1e_6": 295,
"error_quantiles": [
0.0,
0.0,
0.005059570832555149,
0.19359438584218314
],
"note": "原始历史日线可能有修订、请求暖机范围差异;未将重算伪称当时实发。"
},
"event_pairs": [
{
"a": "intensity",
"b": "distribution",
"minutes": 15,
"matched_a_events": 101,
"unique_b_events": 81,
"stock_days": 41,
"dates": 11,
"agree": 1.0,
"note": "最近匹配可能多对一,事件数不作独立样本数"
},
{
"a": "intensity",
"b": "distribution",
"minutes": 30,
"matched_a_events": 146,
"unique_b_events": 93,
"stock_days": 46,
"dates": 11,
"agree": 1.0,
"note": "最近匹配可能多对一,事件数不作独立样本数"
},
{
"a": "intensity",
"b": "distribution",
"minutes": 60,
"matched_a_events": 198,
"unique_b_events": 96,
"stock_days": 49,
"dates": 11,
"agree": 0.9949494949494949,
"note": "最近匹配可能多对一,事件数不作独立样本数"
},
{
"a": "mtf",
"b": "bionic",
"minutes": 15,
"matched_a_events": 31,
"unique_b_events": 31,
"stock_days": 27,
"dates": 15,
"agree": 0.9354838709677419,
"note": "最近匹配可能多对一,事件数不作独立样本数"
},
{
"a": "mtf",
"b": "bionic",
"minutes": 30,
"matched_a_events": 44,
"unique_b_events": 43,
"stock_days": 37,
"dates": 16,
"agree": 0.9545454545454546,
"note": "最近匹配可能多对一,事件数不作独立样本数"
},
{
"a": "mtf",
"b": "bionic",
"minutes": 60,
"matched_a_events": 75,
"unique_b_events": 65,
"stock_days": 57,
"dates": 18,
"agree": 0.8933333333333333,
"note": "最近匹配可能多对一,事件数不作独立样本数"
},
{
"a": "distribution",
"b": "bionic",
"minutes": 15,
"matched_a_events": 15,
"unique_b_events": 12,
"stock_days": 11,
"dates": 10,
"agree": 1.0,
"note": "最近匹配可能多对一,事件数不作独立样本数"
},
{
"a": "distribution",
"b": "bionic",
"minutes": 30,
"matched_a_events": 21,
"unique_b_events": 14,
"stock_days": 12,
"dates": 11,
"agree": 1.0,
"note": "最近匹配可能多对一,事件数不作独立样本数"
},
{
"a": "distribution",
"b": "bionic",
"minutes": 60,
"matched_a_events": 28,
"unique_b_events": 17,
"stock_days": 14,
"dates": 13,
"agree": 1.0,
"note": "最近匹配可能多对一,事件数不作独立样本数"
}
]
}

View File

@ -0,0 +1,40 @@
"""记录原值复现误差、QRS净化后的支持量及同窗事件的有效样本数。"""
import gzip,json
import numpy as np
import pandas as pd
import analyze as a
import weights as w
f=pd.read_csv(a.DATA/'normalized.csv.gz');f.t=pd.to_datetime(f.t)
bars=pd.DataFrame([json.loads(x) for x in gzip.open(a.DATA/'daily.jsonl.gz','rt')]);bars['trade_date']=pd.to_datetime(bars.timestamp).dt.date
bars['code']=bars.symbol.map(a.code)
for col in ['high','low','close']:bars[col]=pd.to_numeric(bars[col])
bycode={c:g for c,g in bars.groupby('code')};calc=w.DailyQRSComputer(None,n_regression=18,m_lookback=60,threshold=.7)
cache={};errs=[]
for r in f[f.source=='qrs'].itertuples():
key=(r.code,r.evidence_date)
if key not in cache:cache[key]=calc._compute_from_dataframe(r.code,bycode[r.code],pd.Timestamp(r.evidence_date).date()).qrs_value
errs.append(abs(cache[key]-r.value))
result={'historical_qrs_reproduction':{'rows':len(errs),'within_1e_6':int(sum(x<1e-6 for x in errs)),
'error_quantiles':np.quantile(errs,[0,.5,.95,1]).tolist(),
'note':'原始历史日线可能有修订、请求暖机范围差异;未将重算伪称当时实发。'},'event_pairs':[]}
for aa,bb in [('intensity','distribution'),('mtf','bionic'),('distribution','bionic')]:
for window in [15,30,60]:
matched=[];seen=set();by_b={k:g for k,g in f[f.source==bb].groupby(['day','code'])}
for key,g in f[f.source==aa].groupby(['day','code']):
q=by_b.get(key)
if q is None:continue
qt=[(a.trade_minute(r.t),r) for r in q.itertuples()];qt=[x for x in qt if x[0] is not None]
if not qt:continue
for row in g.itertuples():
now=a.trade_minute(row.t)
if now is None:continue
lag,other=min(((z-now,r) for z,r in qt),key=lambda x:abs(x[0]))
if abs(lag)<=window:
matched.append((key[0],key[1],row.direction,other.direction,other.id));seen.add(other.id)
p=pd.DataFrame(matched,columns=['day','code','x','y','b_id'])
result['event_pairs'].append({'a':aa,'b':bb,'minutes':window,'matched_a_events':len(p),'unique_b_events':len(seen),
'stock_days':len(p[['day','code']].drop_duplicates()),'dates':p.day.nunique(),
'agree':float((p.x==p.y).mean()),'note':'最近匹配可能多对一,事件数不作独立样本数'})
a.OUT.joinpath('diagnostics.json').write_text(json.dumps(result,ensure_ascii=False,indent=2))
print(json.dumps(result,ensure_ascii=False,indent=2))

View File

@ -0,0 +1,21 @@
"""执行前由开发机拼接已限定沪深的CODES常量;只读日线。"""
import datetime
import gzip
import json
import re
import sys
from sqlalchemy import event,text
from sqlalchemy.engine import Engine
@event.listens_for(Engine,'before_cursor_execute')
def guard(conn,cursor,statement,parameters,context,executemany):
if not statement.lstrip().upper().startswith(('SELECT','SHOW','DESCRIBE')):raise RuntimeError('read-only')
assert all(re.fullmatch(r'(SH(?:60|68)\d{4}|SZ(?:00|30)\d{4})',c) for c in CODES)
from scripts.backfill_v3f_baseline import _get_external_engine
eng=_get_external_engine()
out=gzip.GzipFile(fileobj=sys.stdout.buffer,mode='wb',mtime=0)
with eng.connect() as conn:
for i in range(0,len(CODES),40):
batch=CODES[i:i+40];params={f'c{j}':c for j,c in enumerate(batch)}
q="SELECT symbol,timestamp,high,low,close FROM gp_day_data WHERE symbol IN ("+','.join(':'+k for k in params)+") AND timestamp >= '2026-05-01' AND timestamp < '2026-11-11' ORDER BY symbol,timestamp"
for r in conn.execute(text(q),params):out.write((json.dumps(dict(r._mapping),default=str)+'\n').encode())
out.close();eng.dispose()

View File

@ -0,0 +1,68 @@
"""在上游容器内只读导出;stdout为gzip JSONL,不写上游数据。"""
import datetime as dt
import gzip
import json
import re
import sys
from pathlib import Path
START = '2026-10-08'
END = '2026-11-10'
HS = re.compile(r'(?:(?:60|68)\d{4}\.SH|(?:00|30)\d{4}\.SZ|SH(?:60|68)\d{4}|SZ(?:00|30)\d{4})$')
out = gzip.GzipFile(fileobj=sys.stdout.buffer, mode='wb', mtime=0)
def emit(kind, data):
out.write((json.dumps({'kind': kind, **data}, ensure_ascii=False, default=str)+'\n').encode())
mode = sys.argv[1]
if mode == 'itd':
for p in sorted(Path('/app/data/alert_audit').glob('2026-*.jsonl')):
if not '2026-07-22' <= p.stem <= END: continue
for line in p.open():
r = json.loads(line)
if HS.fullmatch(str(r.get('ts_code', ''))): emit('itd', r)
elif mode == 'bionic':
from sqlalchemy import create_engine, text, event
from sqlalchemy.engine import Engine
@event.listens_for(Engine, 'before_cursor_execute')
def guard(conn, cursor, statement, parameters, context, executemany):
if not statement.lstrip().upper().startswith(('SELECT', 'SHOW', 'DESCRIBE')):
raise RuntimeError('read-only export')
from config.settings import settings
eng = create_engine(settings.SOURCE_DB_EXT_DSN, pool_pre_ping=False)
query = """SELECT stock_code,audit_date,strategy_date,verdict,reason,created_at
FROM strategy_audit_log
WHERE audit_date >= :start AND audit_date <= :end
AND (stock_code LIKE 'SH60%' OR stock_code LIKE 'SH68%' OR stock_code LIKE 'SZ00%' OR stock_code LIKE 'SZ30%')
AND verdict IN ('REVERSAL_BUY','REVERSAL_SELL','TAKE_PROFIT','MAINTAIN_HOLD','MAINTAIN')
AND reason LIKE :reason
ORDER BY audit_date,created_at,stock_code"""
with eng.connect() as c:
for r in c.execute(text(query), {'start':int(START.replace('-','')), 'end':int(END.replace('-','')), 'reason':'【盘中%'}):
emit('bionic_audit', dict(r._mapping))
eng.dispose()
elif mode == 'streams':
import redis
from config.settings import settings
c = redis.Redis(host=settings.SIGNAL_REDIS_HOST, port=settings.SIGNAL_REDIS_PORT,
password=settings.SIGNAL_REDIS_PASSWORD, db=2, decode_responses=True,
socket_timeout=8, socket_connect_timeout=5)
keys = ['mtf:intraday:stream:metrics'] + sorted(c.scan_iter(match='intraday_signals:2026-11-*', count=100))
for key in keys:
if c.type(key) != 'stream': continue
cursor = str(int(dt.datetime.fromisoformat(START).timestamp()*1000))+'-0'
stop = str(int(dt.datetime.fromisoformat('2026-11-11').timestamp()*1000)-1)+'-999999'
while True:
rows = c.xrange(key, min=cursor, max=stop, count=500)
if not rows: break
for ident, raw in rows:
r = {**raw, **json.loads(raw['data'])} if raw.get('data') else raw
code = r.get('ts_code') or r.get('symbol') or r.get('stock_code')
if HS.fullmatch(str(code)):
r.pop('data', None)
emit('mtf' if key.startswith('mtf:') else 'bionic_broadcast', {'stream_key':key,'stream_id':ident, **r})
ms, seq = rows[-1][0].split('-'); cursor = ms+'-'+str(int(seq)+1)
if len(rows) < 500: break
else:
raise ValueError('unsupported export mode')
out.close()

View File

@ -0,0 +1,21 @@
{
"input": {
"bionic.jsonl.gz": {
"sha256": "119b06d0ec011c370104f0cec1f6012faacd2e707d43a7193190242c38480e3e",
"bytes": 173310,
"rows": 2641
},
"itd.jsonl.gz": {
"sha256": "4b5c105a1a1f62506e7fe8a4640d7707f1f388dc45454c962564d7a7d02cf392",
"bytes": 472072,
"rows": 8967
},
"streams.jsonl.gz": {
"sha256": "852161df56ac04d0ae2d3ffbd1909b1470f22c3eaeaf0ee9c944edf7fc346504",
"bytes": 3326817,
"rows": 44162
}
},
"scope": "沪深",
"protocol_sha256": "81b6ad1c50839c709c49bba877a147537de041cbd8648cbed285823840062ba8"
}

View File

@ -0,0 +1,20 @@
{
"version": "1.0",
"objective": "利用现有沪深信号估计来源之间的信息重叠与独立贡献,提出可解释的权重;不使用未来收益、交易成功率或盈亏作目标",
"primary_start": "2026-10-08",
"primary_end": "2026-11-10",
"secondary_start": "2026-07-22",
"secondary_end": "2026-11-10",
"scope": "仅规范化代码为60/68开头.SH及00/30开头.SZ的沪深股票;在导出入口排除北交所",
"inputs": ["量化系统发布审计", "资金异动原始发布流", "Bionic判断审计及可取得的原始广播"],
"exclusions": ["当日未结束样本", "收益和成交数据", "资金流强度2026-07-21及以前的数值", "合成告警与成分重复计票", "同一BUY的跨流镜像", "把无信号或缺覆盖填成中性"],
"source_families": ["逐股QRS", "资金流强度", "资金分布", "资金异动v3f", "Bionic研判"],
"deduplication": "按原始ID先去重;QRS每日固定状态重复发布按股票、证据日、方向合并;资金与模型保留新的观察,但统计使用股票日作为主要单位",
"primary_statistics": ["各源发布覆盖、方向分布、重复率", "共同有观测的股票日方向关联与一致率", "同股事件在交易时间15/30/60分钟内的先后与重合", "Bionic触发来源及镜像关系"],
"weights_method": "先按代码中的原始证据来源分组,组内不因多包装增加预算;在共同可观测样本上以关联度计算冗余折扣。基准等预算,候选独立度u_i=1/(1+sum_j R_ij),归一后仅作为依赖折扣权重。R优先取双向关联的绝对值,相关不足不得设0冒充独立;不以噪声或低覆盖换高权重。结构派生关系单独限制,不能仅凭相关系数消除。",
"support_thresholds": {"min_joint_stock_days": 50, "min_joint_dates": 10, "min_each_direction_per_source": 10},
"uncertainty": "按交易日期整组bootstrap,1000次,固定seed=20261111,报告95%区间;按股票整组复核;前后半窗及15/30/60分钟匹配作敏感性,不选择最漂亮的一组",
"baseline": "来源等权及40/40/20旧示意仅作结构对照,不当已验证参数",
"missingness": "共同有发布的样本只代表条件分布;不声称完整监测覆盖或总体统计独立。无法识别的权重给出范围或保留未分配预算,不伪造点估计",
"deployment": "仅离线研究,不改线上参数、策略或服务"
}

View File

@ -0,0 +1,10 @@
{
"version": "2.0",
"why_added": "首轮质量核对发现QRS发布引用旧日线,不能用该低相关直接给高权重。保留v1原始结果,新增净化和确定性重算对照;该追加设计已见过v1结果,不称独立样本外验证。",
"freshness": "QRS数据日必须为之前最近的实际交易日;用600519.SH日线日期核对,过旧记录从有效实发估计剔除。",
"grouping": "资金流强度与资金分布共用主力资金原料且v1相关约0.90,合成一个主力资金组;同股同日取组内最后有效观察,前值/均值与严格共同观察另作敏感性。",
"reconstruction": "仅为原354个QRS发布股票日,以当前持有的历史日线、生产N=18/M=60公式重算前一交易日原始指标。不是补发信号,不使用今天的逐股阈值替代过去阈值,不伪造历史已可得性。重算值符号只作原始指标依赖的辅助检验,不能当成当时已发布方向。",
"weight_scope": "QRS、主力资金、资金异动三个原始证据组按等先验经绝对相关冗余折扣;Bionic是组合研判层,已复述的证据不能另增一组预算。Bionic独立增量权重如无可识别独立证据和足够匹配样本,不凭空给数。",
"statistical_limits": "主要报告完整实发、有效实发、重算对照的不同支持量;若区间覆盖等权,则选择等权作为易解释基准,不以小数位伪装精度。依赖度不能证明因果、信息质量或预测能力。",
"timing": "Bionic审计按UTC转北京后与68条原始广播核对。Redis消息ID相对发布者约慢266秒,先后分析只使用发布者时间,不用Redis ID推因果;MTF calc_time只有分钟精度,15/30/60分钟匹配只作敏感性。"
}

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,17 @@
{
"alerts_loop.py": {
"path": "/Users/baobao/Documents/work/project/intraday_timing/src/intraday_timing/commands/alerts_loop.py",
"sha256": "a31bac4bff417845f2299f366c9f97c6e00645ef53e85d203dec9cf775fd73f0",
"matches_188_intraday_alerts": true
},
"kline_daily_mem_cache.py": {
"path": "/Users/baobao/Documents/work/project/intraday_timing/src/intraday_timing/adapters/kline_daily_mem_cache.py",
"sha256": "4c7b1f620ad79260aa6ebe210ba8ac46d51d444295db40e463c08c14280ea7f8",
"matches_188_intraday_alerts": true
},
"daily_qrs_computer.py": {
"path": "/Users/baobao/Documents/work/project/intraday_timing/src/intraday_timing/state/daily_qrs_computer.py",
"sha256": "ed9be2ed4804441fde03557648b7e2f2422dbd1e8e74e2187c3760443cc940dc",
"matches_188_intraday_alerts": true
}
}

View File

@ -0,0 +1,60 @@
"""独立性统计与旧缓存行为的离线验证;全部数据为内存样例。"""
import datetime as dt
import math
import sys
from pathlib import Path
import numpy as np
import pandas as pd
import analyze as a
import weights as w
def main():
# 1. 无信号保留NaN,不能在相关样本中伪造0。
x=pd.DataFrame([{'day':'2026-11-09','code':'600000.SH','source':'qrs','direction':1},
{'day':'2026-11-09','code':'600001.SH','source':'mtf','direction':-1}])
p=a.panels(x);assert p[['qrs','mtf']].dropna().empty
# 2. 恒定方向不能得到“零相关、所以独立”的错误结论。
assert a.corr([1,1,1],[1,-1,1]) is None
# 3. 相同或完全反向两个通道都携带冗余,方向相反不增加独立预算。
assert math.isclose(abs(a.corr([1,-1,1,-1],[-1,1,-1,1])),1)
# 4. 正交且信息量对称,权重应等分。
orth=pd.DataFrame({'qrs':[1,1,-1,-1],'fund':[1,-1,1,-1],'mtf':[1,-1,-1,1]})
r=w.dependence_weights(orth);assert all(abs(v-1/3)<1e-12 for v in r['weights'].values())
# 5. 复制资金通道不能令该组多拿预算;分组后只保留观察时间确定的同一状态。
d=pd.DataFrame([{'day':'2026-11-09','code':'600000.SH','source':'intensity','direction':-1},
{'day':'2026-11-09','code':'600000.SH','source':'distribution','direction':-1}])
gp=w.grouped_panel(d);assert list(gp.columns)==['fund'] and gp.iloc[0,0]==-1
# 6. 午休不算经过的交易时间,前后两段相差一分钟。
assert a.trade_minute(dt.datetime(2026,11,9,13,1))-a.trade_minute(dt.datetime(2026,11,9,11,30))==1
# 7. 北交所及格式不明代码不能进入分析。
for c in ['920001.BJ','BJ430001','600000']:
try:a.code(c)
except ValueError:pass
else:raise AssertionError(c)
# 8. 当前生产缓存跨日重复preload不会重新读取;显式clear后才更新。
from intraday_timing.adapters.kline_daily_mem_cache import KlineDailyBulkMemCache
class Conn:
calls=0
day='2026-11-06'
def execute_query(self,*args,**kwargs):
self.calls+=1
return pd.DataFrame([{'symbol':'SH600000','trade_date':self.day,'open':10,'close':10,'high':11,'low':9,'vol':100,'amount':1000,'pre_close':10}])
c=Conn();cache=KlineDailyBulkMemCache(None,c)
cache.preload(['600000.SH'],dt.date(2026,11,1),dt.date(2026,11,9));c.day='2026-11-09'
assert cache.preload(['600000.SH'],dt.date(2026,11,1),dt.date(2026,11,10))==0 and c.calls==1
assert cache.get_kline_daily('600000.SH',dt.date(2026,11,1),dt.date(2026,11,10)).trade_date.max()==dt.date(2026,11,6)
cache.clear();cache.preload(['600000.SH'],dt.date(2026,11,1),dt.date(2026,11,10));assert c.calls==2
assert cache.get_kline_daily('600000.SH',dt.date(2026,11,1),dt.date(2026,11,10)).trade_date.max()==dt.date(2026,11,9)
# 9. 给历史序列追加未来极端行情,不得改变过去时点的QRS。
dates=pd.date_range('2026-01-01',periods=120).date
low=np.linspace(10,20,120)+np.sin(np.arange(120)/3)
high=low+1+np.cos(np.arange(120)/4)*.2
bars=pd.DataFrame({'trade_date':dates,'high':high,'low':low})
calc=w.DailyQRSComputer(None,n_regression=18,m_lookback=60,threshold=.7)
cutoff=dates[90];before=calc._compute_from_dataframe('600000.SH',bars.iloc[:91],cutoff).qrs_value
bars.loc[91:,['high','low']]*=100
after=calc._compute_from_dataframe('600000.SH',bars,cutoff).qrs_value
assert abs(before-after)<1e-12
print('9组离线边界通过,包含真实缓存类跨日复现与QRS未来截断一致性。')
if __name__=='__main__':main()

View File

@ -0,0 +1,562 @@
{
"qrs_quality": {
"events": 354,
"fresh": 17,
"stale": 337,
"age_median": 41.0,
"same_evidence_repeated": 248
},
"reconstruction": {
"rows": 354,
"changed_sign": 138,
"old_value_reproduced": 354,
"old_value_max_abs_error": 0.19359438584218314,
"errors": []
},
"clean_pairs": [
{
"a": "qrs",
"b": "fund",
"n": 5,
"dates": 5,
"stocks": 5,
"corr": -0.24999999999999997,
"agreement": 0.6,
"a_positive": 1,
"a_negative": 4,
"b_positive": 1,
"b_negative": 4,
"eligible": false
},
{
"a": "qrs",
"b": "mtf",
"n": 7,
"dates": 6,
"stocks": 7,
"corr": null,
"agreement": 0.42857142857142855,
"a_positive": 3,
"a_negative": 4,
"b_positive": 7,
"b_negative": 0,
"eligible": false
},
{
"a": "qrs",
"b": "bionic",
"n": 3,
"dates": 3,
"stocks": 3,
"corr": null,
"agreement": 0.6666666666666666,
"a_positive": 2,
"a_negative": 1,
"b_positive": 3,
"b_negative": 0,
"eligible": false
},
{
"a": "fund",
"b": "mtf",
"n": 273,
"dates": 24,
"stocks": 130,
"corr": 0.18367126921523455,
"agreement": 0.6007326007326007,
"a_positive": 78,
"a_negative": 195,
"b_positive": 125,
"b_negative": 148,
"eligible": true
},
{
"a": "fund",
"b": "bionic",
"n": 39,
"dates": 20,
"stocks": 28,
"corr": 0.7385686213637483,
"agreement": 0.8717948717948718,
"a_positive": 22,
"a_negative": 17,
"b_positive": 23,
"b_negative": 16,
"eligible": false
},
{
"a": "mtf",
"b": "bionic",
"n": 82,
"dates": 23,
"stocks": 59,
"corr": 0.4411405309954431,
"agreement": 0.7439024390243902,
"a_positive": 51,
"a_negative": 31,
"b_positive": 66,
"b_negative": 16,
"eligible": true
}
],
"grouped_raw_pairs": [
{
"a": "qrs",
"b": "fund",
"n": 88,
"dates": 21,
"stocks": 55,
"corr": 0.046549465744305435,
"agreement": 0.5454545454545454,
"a_positive": 37,
"a_negative": 51,
"b_positive": 31,
"b_negative": 57,
"eligible": true
},
{
"a": "qrs",
"b": "mtf",
"n": 135,
"dates": 23,
"stocks": 67,
"corr": -0.03688254754085145,
"agreement": 0.48148148148148145,
"a_positive": 66,
"a_negative": 69,
"b_positive": 68,
"b_negative": 67,
"eligible": true
},
{
"a": "qrs",
"b": "bionic",
"n": 27,
"dates": 14,
"stocks": 17,
"corr": -0.2357797812857538,
"agreement": 0.37037037037037035,
"a_positive": 12,
"a_negative": 15,
"b_positive": 19,
"b_negative": 8,
"eligible": false
},
{
"a": "fund",
"b": "mtf",
"n": 273,
"dates": 24,
"stocks": 130,
"corr": 0.18367126921523455,
"agreement": 0.6007326007326007,
"a_positive": 78,
"a_negative": 195,
"b_positive": 125,
"b_negative": 148,
"eligible": true
},
{
"a": "fund",
"b": "bionic",
"n": 39,
"dates": 20,
"stocks": 28,
"corr": 0.7385686213637483,
"agreement": 0.8717948717948718,
"a_positive": 22,
"a_negative": 17,
"b_positive": 23,
"b_negative": 16,
"eligible": false
},
{
"a": "mtf",
"b": "bionic",
"n": 82,
"dates": 23,
"stocks": 59,
"corr": 0.4411405309954431,
"agreement": 0.7439024390243902,
"a_positive": 51,
"a_negative": 31,
"b_positive": 66,
"b_negative": 16,
"eligible": true
}
],
"reconstructed_pairs": [
{
"a": "qrs",
"b": "fund",
"n": 88,
"dates": 21,
"stocks": 55,
"corr": -0.06153759201424826,
"agreement": 0.5227272727272727,
"a_positive": 29,
"a_negative": 59,
"b_positive": 31,
"b_negative": 57,
"eligible": true,
"day_ci": [
-0.3217599860699957,
-0.06536476507361046,
0.1298440332987063
],
"stock_ci": [
-0.29064646227097013,
-0.07666601382777771,
0.16400050425142315
]
},
{
"a": "qrs",
"b": "mtf",
"n": 135,
"dates": 23,
"stocks": 67,
"corr": -0.03896650800208502,
"agreement": 0.48148148148148145,
"a_positive": 32,
"a_negative": 103,
"b_positive": 68,
"b_negative": 67,
"eligible": true,
"day_ci": [
-0.1895798040429639,
-0.046146819853865526,
0.11133587163181648
],
"stock_ci": [
-0.16366816098624398,
-0.04242410499900641,
0.09339577839844783
]
},
{
"a": "qrs",
"b": "bionic",
"n": 27,
"dates": 14,
"stocks": 17,
"corr": 0.2078742949189203,
"agreement": 0.5555555555555556,
"a_positive": 11,
"a_negative": 16,
"b_positive": 19,
"b_negative": 8,
"eligible": false,
"day_ci": [
-0.3119414241066813,
0.20412414523193154,
0.5444634480036133
],
"stock_ci": [
-0.03439982023509114,
0.2297526152623865,
0.5093108126336955
]
},
{
"a": "fund",
"b": "mtf",
"n": 273,
"dates": 24,
"stocks": 130,
"corr": 0.18367126921523455,
"agreement": 0.6007326007326007,
"a_positive": 78,
"a_negative": 195,
"b_positive": 125,
"b_negative": 148,
"eligible": true,
"day_ci": [
0.03684952678543051,
0.18565327904562823,
0.32337939861428566
],
"stock_ci": [
0.07440556454087294,
0.18434597467795827,
0.30328064572466534
]
},
{
"a": "fund",
"b": "bionic",
"n": 39,
"dates": 20,
"stocks": 28,
"corr": 0.7385686213637483,
"agreement": 0.8717948717948718,
"a_positive": 22,
"a_negative": 17,
"b_positive": 23,
"b_negative": 16,
"eligible": false,
"day_ci": [
0.3992071197831581,
0.7458265812824105,
1.0
],
"stock_ci": [
0.4342360156590327,
0.7462896594194197,
0.9483522547960089
]
},
{
"a": "mtf",
"b": "bionic",
"n": 82,
"dates": 23,
"stocks": 59,
"corr": 0.4411405309954431,
"agreement": 0.7439024390243902,
"a_positive": 51,
"a_negative": 31,
"b_positive": 66,
"b_negative": 16,
"eligible": true,
"day_ci": [
0.23808187663035318,
0.4480530170237973,
0.6537413649301814
],
"stock_ci": [
0.1727169648407289,
0.4454950566114418,
0.6315584217263348
]
}
],
"raw_weights": {
"weights": {
"qrs": 0.3612276887094352,
"fund": 0.31812635806831746,
"mtf": 0.32064595322224726
},
"pairs": [
{
"a": "qrs",
"b": "fund",
"n": 88,
"corr": 0.046549465744305435
},
{
"a": "qrs",
"b": "mtf",
"n": 135,
"corr": -0.03688254754085145
},
{
"a": "fund",
"b": "mtf",
"n": 273,
"corr": 0.18367126921523455
}
]
},
"clean_weights_for_diagnostics_only": null,
"reconstructed_weights": {
"weights": {
"qrs": 0.35920866862097045,
"fund": 0.3174653063330021,
"mtf": 0.3233260250460275
},
"pairs": [
{
"a": "qrs",
"b": "fund",
"n": 88,
"corr": -0.06153759201424826
},
{
"a": "qrs",
"b": "mtf",
"n": 135,
"corr": -0.03896650800208502
},
{
"a": "fund",
"b": "mtf",
"n": 273,
"corr": 0.18367126921523455
}
]
},
"day_bootstrap": {
"qrs": [
0.31888104076663437,
0.35055604958364983,
0.3787709812160866
],
"fund": [
0.297511918321756,
0.31875563252606065,
0.34192119962396844
],
"mtf": [
0.30649578874297584,
0.32843807756937865,
0.3664928196780627
]
},
"stock_bootstrap": {
"qrs": [
0.3247551422792676,
0.35176397202561505,
0.3760089641342312
],
"fund": [
0.29863008688752285,
0.31765949234463764,
0.3447372730399579
],
"mtf": [
0.3061582652618046,
0.32736012285104454,
0.36003805707262904
]
},
"sensitivity": {
"first": {
"weights": {
"qrs": 0.38760449566288574,
"fund": 0.30717399714356525,
"mtf": 0.305221507193549
},
"pairs": [
{
"a": "qrs",
"b": "fund",
"n": 88,
"corr": -0.028638223261250706
},
{
"a": "qrs",
"b": "mtf",
"n": 135,
"corr": -0.03724194613619291
},
{
"a": "fund",
"b": "mtf",
"n": 273,
"corr": 0.31633221479075757
}
]
},
"mean": {
"weights": {
"qrs": 0.3714613420062064,
"fund": 0.31671372301602757,
"mtf": 0.31182493497776603
},
"pairs": [
{
"a": "qrs",
"b": "fund",
"n": 88,
"corr": -0.054648287669321144
},
{
"a": "qrs",
"b": "mtf",
"n": 135,
"corr": -0.07542826032972809
},
{
"a": "fund",
"b": "mtf",
"n": 273,
"corr": 0.2707750223532278
}
]
},
"first_half": {
"weights": {
"qrs": 0.3645742557903634,
"fund": 0.329419068256303,
"mtf": 0.30600667595333364
},
"pairs": [
{
"a": "qrs",
"b": "fund",
"n": 73,
"corr": -0.027093815217750113
},
{
"a": "qrs",
"b": "mtf",
"n": 105,
"corr": -0.12461398850182136
},
{
"a": "fund",
"b": "mtf",
"n": 165,
"corr": 0.24752279289653376
}
]
},
"second_half": {
"weights": {
"qrs": 0.3258370606278521,
"fund": 0.30850376769754656,
"mtf": 0.3656591716746013
},
"pairs": [
{
"a": "qrs",
"b": "fund",
"n": 15,
"corr": -0.21821789023599236
},
{
"a": "qrs",
"b": "mtf",
"n": 30,
"corr": -0.014678923792502543
},
{
"a": "fund",
"b": "mtf",
"n": 108,
"corr": 0.0839492662258804
}
]
},
"fund_distribution_only": {
"weights": {
"qrs": 0.33439802248125616,
"fund": 0.3117877231309714,
"mtf": 0.3538142543877724
},
"pairs": [
{
"a": "qrs",
"b": "fund",
"n": 53,
"corr": -0.19633836312465044
},
{
"a": "qrs",
"b": "mtf",
"n": 135,
"corr": -0.03896650800208502
},
{
"a": "fund",
"b": "mtf",
"n": 205,
"corr": 0.1285486526887367
}
]
}
},
"triple_complete_rows": 44,
"data_sha256": "63652a23edbb44f4cc72791232ebb838812aa34538729c5d11dbf38bbc1eb82d"
}

View File

@ -0,0 +1,102 @@
"""v2分组、日期净化与原公式重算;仅离线运行。"""
from __future__ import annotations
import datetime as dt
import gzip
import hashlib
import itertools
import json
from pathlib import Path
import sys
import numpy as np
import pandas as pd
import analyze as a
SOURCE_ROOT=Path('/Users/baobao/Documents/work/project/intraday_timing/src')
sys.path.insert(0,str(SOURCE_ROOT))
from intraday_timing.state.daily_qrs_computer import DailyQRSComputer
GROUPS=['qrs','fund','mtf']
def dependence_weights(panel):
rs=[];R=np.eye(len(GROUPS))
for i,j in itertools.combinations(range(len(GROUPS)),2):
q=panel[[GROUPS[i],GROUPS[j]]].dropna()
r=a.corr(q.iloc[:,0],q.iloc[:,1])
if r is None:return None
R[i,j]=R[j,i]=abs(r)
rs.append({'a':GROUPS[i],'b':GROUPS[j],'n':len(q),'corr':r})
u=1/R.sum(axis=1);weights=u/u.sum()
return {'weights':dict(zip(GROUPS,map(float,weights))), 'pairs':rs}
def grouped_panel(f,how='last'):
z=f.copy();z['source']=z.source.replace({'intensity':'fund','distribution':'fund'})
return a.panels(z,how)
def bootstrap_weights(panel,key='day',n=1000):
q=panel.reset_index();rng=np.random.default_rng(20261111);blocks=[v.drop(columns=key) for _,v in q.groupby(key)]
estimates=[]
for _ in range(n):
sample=pd.concat([blocks[i] for i in rng.integers(len(blocks),size=len(blocks))],ignore_index=True)
r=dependence_weights(sample)
if r:estimates.append([r['weights'][g] for g in GROUPS])
return {g:np.quantile(np.array(estimates)[:,i],[.025,.5,.975]).tolist() for i,g in enumerate(GROUPS)}
def main():
f=pd.read_csv(a.DATA/'normalized.csv.gz');f['t']=pd.to_datetime(f.t)
bars=pd.DataFrame([json.loads(x) for x in gzip.open(a.DATA/'daily.jsonl.gz','rt')])
bars['trade_date']=pd.to_datetime(bars.timestamp).dt.date
bars['code']=bars.symbol.map(a.code)
assert not bars.duplicated(['code','trade_date']).any()
for col in ['high','low','close']:bars[col]=pd.to_numeric(bars[col])
cal=sorted(bars[bars.symbol=='SH600519'].trade_date.unique())
prev={d:max(x for x in cal if x<dt.date.fromisoformat(d)) for d in f.day.unique()}
q=f[f.source=='qrs'].copy()
q['fresh']=q.apply(lambda r:r.evidence_date==prev[r.day].isoformat(),axis=1)
q['age']=(pd.to_datetime(q.day)-pd.to_datetime(q.evidence_date)).dt.days
calc=DailyQRSComputer(None,n_regression=18,m_lookback=60,threshold=.7)
bycode={c:g for c,g in bars.groupby('code')};recon=[];reproduced=[];errors=[]
for row in q.itertuples():
try:
b=bycode[row.code];result=calc._compute_from_dataframe(row.code,b,prev[row.day])
if result.trade_date!=prev[row.day]:raise ValueError('missing prior-day bar')
recon.append({'day':row.day,'code':row.code,'value':result.qrs_value,'direction':float(np.sign(result.qrs_value)),
'old_value':row.value,'old_direction':row.direction,'fresh':row.fresh,'evidence_date':prev[row.day].isoformat()})
try:
old=calc._compute_from_dataframe(row.code,b,dt.date.fromisoformat(row.evidence_date))
reproduced.append(abs(old.qrs_value-row.value))
except Exception as e:errors.append({'code':row.code,'day':row.day,'phase':'old_reproduction','error':type(e).__name__})
except Exception as e:errors.append({'code':row.code,'day':row.day,'phase':'fresh_reconstruction','error':type(e).__name__})
rr=pd.DataFrame(recon)
clean=f[(f.source!='qrs') | f.index.isin(q[q.fresh].index)].copy()
counter=f.copy().set_index(['day','code','source'])
for row in rr.itertuples():
key=(row.day,row.code,'qrs');counter.loc[key,'direction']=row.direction;counter.loc[key,'value']=row.value
counter=counter.reset_index().sort_values(['t','source','code'])
# 未成功重算的QRS不得保留旧值。
success=set(zip(rr.day,rr.code))
counter=counter[(counter.source!='qrs') | counter.apply(lambda r:(r.day,r.code) in success,axis=1)]
a.SOURCES=['qrs','fund','mtf','bionic']
raw_panel=grouped_panel(f);clean_panel=grouped_panel(clean);recon_panel=grouped_panel(counter)
result={'qrs_quality':{'events':len(q),'fresh':int(q.fresh.sum()),'stale':int((~q.fresh).sum()),'age_median':float(q.age.median()),
'same_evidence_repeated':int(q.duplicated(['code','evidence_date','direction']).sum())},
'reconstruction':{'rows':len(rr),'changed_sign':int((rr.old_direction!=rr.direction).sum()),
'old_value_reproduced':len(reproduced),'old_value_max_abs_error':max(reproduced) if reproduced else None,'errors':errors},
'clean_pairs':a.pair_stats(clean_panel,False),'grouped_raw_pairs':a.pair_stats(raw_panel,False),
'reconstructed_pairs':a.pair_stats(recon_panel),
'raw_weights':dependence_weights(raw_panel),'clean_weights_for_diagnostics_only':dependence_weights(clean_panel),
'reconstructed_weights':dependence_weights(recon_panel),
'day_bootstrap':bootstrap_weights(recon_panel),'stock_bootstrap':bootstrap_weights(recon_panel,'code',500),
'sensitivity':{},'triple_complete_rows':len(recon_panel[GROUPS].dropna()),
'data_sha256':hashlib.sha256((a.DATA/'daily.jsonl.gz').read_bytes()).hexdigest()}
for how in ['first','mean']:
result['sensitivity'][how]=dependence_weights(grouped_panel(counter,how))
days=sorted(counter.day.unique());mid=days[len(days)//2]
for name,g in [('first_half',counter[counter.day<mid]),('second_half',counter[counter.day>=mid])]:
result['sensitivity'][name]=dependence_weights(grouped_panel(g))
# 资金组用更稳定全窗的资金分布作唯一代表,查看强度缺发是否改变结论。
result['sensitivity']['fund_distribution_only']=dependence_weights(grouped_panel(counter[counter.source!='intensity']))
a.OUT.joinpath('weights.json').write_text(json.dumps(result,ensure_ascii=False,indent=2))
rr.to_csv(a.DATA/'qrs_reconstruction.csv.gz',index=False,compression='gzip')
print(json.dumps(result,ensure_ascii=False,indent=2))
if __name__=='__main__':main()