研究信号独立性与分组权重
This commit is contained in:
parent
34bc09b8c2
commit
00a7159bc9
|
|
@ -1,5 +1,9 @@
|
|||
# 开发节点记录(三系统量化交易)
|
||||
|
||||
## 2026-11-11 按独立信息研究信号权重
|
||||
|
||||
按用户澄清,不使用未来收益或买卖成功率。只读提取24个已结束交易日的沪深信号,先固定协议,再做来源去重、共同观察相关和时间匹配。资金流强度与资金分布相关0.898,长期对照0.888,应合组。发现QRS354条中337条引用旧日线,实际告警循环未清跨日日线缓存,九组断网边界含真实缓存类复现通过。仅将原公式重算作为单列探索对照,得到三基础组36/32/32点估计,日期及股票重采样均覆盖等权,建议三组各1/3,Bionic组合研判不另增独立预算。未改线上规则或上游服务,根因和待办见[研究接续](docs/信号独立性权重研究接续_2026-11-11.md)。
|
||||
|
||||
## 2026-11-11 信号历史覆盖与既有标定核验
|
||||
|
||||
14:33至14:38只读188发布系统文件、Redis及上游数据库。确认M14入场、逐股QRS、v3f资金异动与市场实况已有标定;当天QRS120只沪深中66只成功、54只兜底。上游通知明确资金流强度7月21日及以前数值作废,现阈值仍待再校准;资金分布仍为临时值。资金实发流始于10月8日,可联合整理24个已结束交易日期,但不等于完整当时可得回放。Bionic原始广播只剩11月5日起,长期审计另在;实况65日生产与116日回填分开;4只沪深分钟行情抽查240日,不能当一年实发信号。仅文档交付,未改交易代码、参数、账本或服务,详见[核验记录](docs/信号历史覆盖与发布阈值核验_2026-11-11.md)。
|
||||
|
|
|
|||
|
|
@ -0,0 +1,17 @@
|
|||
# 信号独立性权重研究接续
|
||||
|
||||
用户明确权重目标是独立信息,不是涨跌预测或交易成功率。2026年11月11日完成只读数据提取、来源分析、日期核验及断网研究;未改线上代码或参数,未操作188真实PMS。
|
||||
|
||||
详细结果、协议和复现入口见[研究报告](../research/signal_independence_20261111/README.md)。研究主窗10月8日至11月10日24个已结束交易日。统计只使用沪深,不读未来收益或成交结果。
|
||||
|
||||
主要结论:
|
||||
|
||||
- 资金流强度与资金分布共享主力资金原料,53个共同股票日相关0.898、一致率96.23%;7月22日起216个共同股票日相关0.888。应共享预算,不能各计一票。
|
||||
- 当前实发QRS有严重新鲜度问题:354条中337条数据日过旧。实际代码的日线内存缓存跨日未clear,离线复现通过。净化后只有17条QRS,无法直接标定完整实发权重。
|
||||
- 对原354个股票日按已有日线、原公式重算,作为明确区分的探索对照,三基础组冗余折扣约36/32/32;所有不确定区间均包含三等分,敏感性中排序改变。建议技术QRS、主力资金、v3f资金异动各1/3作为初始结构预算,不称精确最优。
|
||||
- Bionic与资金有明显重叠,独立增量尚不可识别。建议组合研判引用已有组预算,不另增加一份独立票;这不等于模型无价值,也不是统计测得模型应为零。不能将示意20%冒充数据定值。
|
||||
- 缺失组预算保留,不把剩余来源归一为100%。量化BUY、分钟QRS及实况分缺少本窗独立样本或含派生成分,不额外分配新预算。
|
||||
|
||||
验证:开发机隔离断网完成统计、原QRS公式重算、按日期1000次与股票500次重采样;九组边界通过。Bionic审计与68条广播时间匹配通过;Redis流ID相对发布者慢约266秒,先后关系不能直接按流ID判断。限定沪深的研究输入已归档到研究目录data,连同代码、协议、聚合及哈希保存,无需再次连接生产服务即可复算。
|
||||
|
||||
待处理:QRS跨日缓存及来源新鲜度、已有量化入场QRS输入不可用、Bionic固定95%、恢复与旧退出状态冲突。它们不是本次研究已修复事项。上线前须先解决有效证据和来源身份,不能把旧QRS直接以三分之一预算投入运行;研究结果不自动上线。
|
||||
|
|
@ -1,5 +1,7 @@
|
|||
# 开发资料入口
|
||||
|
||||
以独立信息为目标的权重研究,见[11月11日研究接续](信号独立性权重研究接续_2026-11-11.md)及[完整报告](../research/signal_independence_20261111/README.md)。确认资金两路同源,提出三基础组近似等权、模型共享证据预算;同时发现QRS跨日缓存旧数据,尚未上线修复。
|
||||
|
||||
信号实际历史保留、可研究窗口及上游既有标定,见[11月11日覆盖与阈值核验](信号历史覆盖与发布阈值核验_2026-11-11.md)。区分实发、审计与回填;资金流强度7月21日及以前数值失效,不能混入研究。
|
||||
|
||||
多源发布公式、分组加权、无信号状态和按证据时间合成,见[11月11日多源择时评估](多源择时加权与时序边界评估_2026-11-11.md)。含量化择时当日四轮输入不可用的只读证据;尚未实施融合或修复上游数据。
|
||||
|
|
|
|||
|
|
@ -0,0 +1,103 @@
|
|||
# 信号独立性与分组权重研究
|
||||
|
||||
研究日期:2026年11月11日。目标是减少同一信息的重复计票,使用现有数据估计来源依赖,不以未来收益、交易成功率或盈亏优化权重。仅沪深;没有线上改动。
|
||||
|
||||
## 建议参数及其含义
|
||||
|
||||
当前更稳妥的建议是三类基础证据预算各占三分之一。将资金流强度与资金分布合为同一组,Bionic作为组合研判保留,不再额外增加一组独立票数。
|
||||
|
||||
| 基础证据组 | 对应现有来源 | 研究建议预算 |
|
||||
| --- | --- | --- |
|
||||
| 技术形态 | 此次有资料可评估的逐股QRS | 1/3 |
|
||||
| 主力资金结构 | 资金流强度、资金分布吸筹/出货,共享预算 | 1/3 |
|
||||
| 盘中价量资金异动 | v3f资金异动 | 1/3 |
|
||||
|
||||
这些是来源预算,不是交易仓位,不是胜率。Bionic额外独立票权暂不分配,是避免重复预算的架构建议,不是统计测得“模型信息量等于零”。模型仍可解释、复核与指出矛盾;若其观点引用上述证据,归回该证据组,不能再让来源包装增加票数。独立新闻、事实或新证据如需增设来源,必须另有来源身份和数据支持,不能靠模型自报置信度获得预算。
|
||||
|
||||
本次没有数据支持把整个“量价系统”均赋予同一权重:量化BUY在主要窗口没有实发样本;分钟QRS、放量、日频资金已组合进BUY,不能将BUY与其成分同时作为新来源。上述技术组估计仅对应逐股日频QRS;资金实况分与v3f共同使用价量原料,本次没有独立连续实况轨迹,不给实况分额外预算。
|
||||
|
||||
组内不要按消息数量累计。重复、镜像及同证据日重发只算一次;同组出现矛盾需保留分歧,不能只挑更合意的方向。缺数据时保留缺失组的预算,不将剩余组自动扩至100%;新鲜度和覆盖度是是否可用的条件,不能冒充信息独立度。
|
||||
|
||||
## 一、样本与预先固定的口径
|
||||
|
||||
先写入`protocol.json`,再导出数据和计算。主要窗口为10月8日至11月10日,24个已结束交易日。量化告警另取7月22日起历史检验同源关系。数据来自188上游容器的本地审计、Redis发布流和Bionic审计表,均为只读。实际不调用模型、不访问真实PMS账本、不重放交易。初始缓存位于`/private/tmp/signal-independence-20261111/`;交付时将限定沪深的研究输入与规范化事件归档至本目录`data/`,避免上游TTL到期后丢失复现资料。
|
||||
|
||||
| 来源 | 主要窗口事件数 | 股票数 | 有事件日期 | 股票日数 |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| 逐股QRS | 354 | 106 | 24 | 354 |
|
||||
| 资金流强度 | 722 | 87 | 12 | 163 |
|
||||
| 资金分布 | 972 | 178 | 24 | 452 |
|
||||
| 资金异动 | 44094 | 4852 | 24 | 31075 |
|
||||
| Bionic明确方向 | 199 | 87 | 24 | 139 |
|
||||
|
||||
Bionic总审计2641次,其中MAINTAIN 2182次、MAINTAIN_HOLD 260次、转多139次、转弱48次、止盈12次。“维持”没有明确的新方向,不填成中性票。明确方向中189次为预案条件触发、10次为持仓体检。代码显示这些路径会读取当前价量、资金分布及板块QRS,不能因模型是另一套服务就认为其输入独立。
|
||||
|
||||
资金流强度在10月26日至11月10日没有发布记录;这是事件缺席,不能仅据此判定“正常中性”,也不能证明采集故障。相关性只用两路均有实际观察的股票日;不把未发布或未覆盖的股票填0。方向一致率不是买卖成功率。
|
||||
|
||||
## 二、可以明确确认的重复信息
|
||||
|
||||
资金流强度与资金分布来自同一主力资金原料。24日窗口共同股票日53个、11个日期,方向一致率96.23%,方向相关系数0.898,按日期整组重采样95%区间约0.700至1.000。拉长到7月22日起,共同股票日216个,相关系数0.888,结论一致。
|
||||
|
||||
30交易分钟最近匹配时,两者146次匹配事件方向全部一致,涉及46个股票日、11个日期;最近匹配可能多对一,不能把146次当成146个独立样本。15和60分钟检查也维持高度一致。这里代码同源与观测结果互相支持,足以支持合组,不能将两者各给一份完整预算。
|
||||
|
||||
Bionic与合并后的主力资金组共同股票日39个,方向一致87.18%、相关0.739;样本低于事先设定的50个股票日,不据此给模型精确独立权重。与资金异动共同股票日82个,日内最后方向相关0.441;30分钟匹配44次,涉及37个股票日、16个日期,方向一致95.45%。同一天但不同时间的相关程度明显不同,不能只用日末低相关给模型增权。
|
||||
|
||||
这不证明Bionic完全没有增量判断能力。现有资料没有保存完整输入及每条结论的新证据身份,无法把模型独立解释与复述、旧计划延续稳定分开。证据不足的部分保留未识别,不假定其独立度为零或一,也不声称20%已由数据标定。
|
||||
|
||||
## 三、质量核验改变了QRS研究边界
|
||||
|
||||
354条QRS发布中,337条的数据日不是前一交易日,数据日与发布日的自然日差中位数41天;248条还是同股、同数据日、同方向的重复证据。只有17条使用前一交易日数据。交易日期用600519.SH实际日线核对,不用自然日硬判周末。
|
||||
|
||||
字段含义已核实:发布metadata中的trade_date来自QRS实际参与计算的日线最后一行,不是“上次穿越日期”。188当前三个相关文件与本地逐字哈希一致。`KlineDailyBulkMemCache.preload`会跳过已预加载股票,`clear()`要求每日调用;当前告警循环盘前只清空QRS结果,不清这个日线缓存。用当前真实缓存类与内存数据离线复现:第二天preload仍返回旧日线,显式clear后才读到新数据。本次未改上游代码或重启服务。
|
||||
|
||||
剔除旧记录后,QRS与主力资金共同股票日仅5个,与资金异动仅7个,与Bionic仅3个;不足以按原实发数据估计完整权重。不能把旧状态不响应当天变化表现出的低相关当成科学的独立性。
|
||||
|
||||
因此追加`protocol_v2.json`:只对原354个发布股票日,利用已有历史日线按原生产N=18、M=60公式重算截至前一交易日的原始QRS。没有套用今天阈值,没有补造历史发布,重算值的正负仅用于原始指标依赖对照。该设计是在看到质量问题后追加,属于探索性补充,不是独立样本外实验。
|
||||
|
||||
354个股票日全部可重算,其中138个原始指标方向与旧发布方向不同。按旧证据日期复现原值时,295条误差小于1e-6,误差中位数0、95分位0.00506、最大0.19359;剩余差异的历史修订与暖机原因未逐条定位。因此重算对照不能宣称完全还原当时可得输入。
|
||||
|
||||
## 四、权重计算结果
|
||||
|
||||
将两路主力资金合组后,对共同有观察的股票日计算方向相关。以相同先验预算为起点,使用可解释的冗余折扣:
|
||||
|
||||
`u_i = 1 / (1 + Σ(j≠i)|ρ_ij|)`,再令`w_i = u_i / Σu_i`。
|
||||
|
||||
这是一条明确约定的去冗余规则,不是数学上唯一的“科学最优权重”。使用绝对值,避免把完全反向的镜像误认为独立;缺失相关不能直接填0。此处仅研究方向,不能据此证明完整连续分数或所有市场状态下统计独立。三个组两两共同样本不同,三组完整共同观察仅44个股票日,存在选择偏差。
|
||||
|
||||
| 组合 | 共同股票日 | 相关系数 | 日期重采样95%区间 |
|
||||
| --- | ---: | ---: | --- |
|
||||
| 重算QRS与主力资金 | 88 | -0.062 | -0.322至0.130 |
|
||||
| 重算QRS与资金异动 | 135 | -0.039 | -0.190至0.111 |
|
||||
| 主力资金与资金异动 | 273 | 0.184 | 0.037至0.323 |
|
||||
|
||||
| 基础证据组 | 冗余折扣点估计 | 按日期重采样95%区间 | 按股票重采样95%区间 |
|
||||
| --- | ---: | --- | --- |
|
||||
| QRS | 35.92% | 31.89%至37.88% | 32.48%至37.60% |
|
||||
| 主力资金 | 31.75% | 29.75%至34.19% | 29.86%至34.47% |
|
||||
| 资金异动 | 32.33% | 30.65%至36.65% | 30.62%至36.00% |
|
||||
|
||||
按日期整组重采样1000次、按股票整组500次,固定随机种子20261111;没有把同一天成千上万条消息视为独立样本。区间仅是当前样本及该计算规则下的不确定范围,不包含全部模型选择、历史修订、跨市场阶段风险。
|
||||
|
||||
敏感性:日内第一方向给出38.76/30.72/30.52,日内均值37.15/31.67/31.18;前半窗36.46/32.94/30.60,后半窗32.58/30.85/36.57。后半窗QRS共同样本尤其薄,不用于单独定参。主力资金只用覆盖全窗的资金分布代表时为33.44/31.18/35.38。技术与资金异动的排序并不稳定。
|
||||
|
||||
所以36/32/32可作为记录在案的探索点估计,三个1/3是更稳妥的初始结构参数。不能声称36显著优于33,也不能把Bionic硬填10%、20%说成数据已经给出的答案。当前结论的确定部分是合并同源、删除镜像票、保留缺失预算;精细差异尚不足以支持长期固定偏置。
|
||||
|
||||
若后续上线,先修QRS跨日缓存并建立证据日期与来源身份,原组权重仅在有效状态下贡献;当前不应直接将旧QRS赋予三分之一实际票权。此前确认的Bionic固定95%、旧退出状态阻断恢复等问题仍未实施修复。
|
||||
|
||||
## 五、时间与验证
|
||||
|
||||
68条Bionic原始广播与审计逐股同结论匹配全部成功。审计按UTC转北京后,与广播时间差中位数0.45秒、最大1.006秒。Redis流ID时间比发布者时间慢约265至267秒,不能直接将流ID当同一时钟推先后因果。本研究匹配使用发布者时间;MTF只给分钟精度。匹配前后事件只是关联性检查,没有用未来事件构造线上策略状态。
|
||||
|
||||
开发机在项目隔离配置、断网护栏下完成原始分析、重算、重采样及九组边界验证。验证包含缺失不填0、常量相关不可用、反向镜像冗余、对称数据等权、资金组不重复、午休时钟、沪深过滤和真实缓存类跨日复现。未做收益回测,未声明交易获利改善。
|
||||
|
||||
```bash
|
||||
cd /private/tmp/pms-plan-input-20261111
|
||||
/private/tmp/pms-audit311/bin/python scripts/unit_test_guard.py research/signal_independence_20261111/test_research.py
|
||||
/private/tmp/pms-audit311/bin/python scripts/unit_test_guard.py research/signal_independence_20261111/analyze.py
|
||||
/private/tmp/pms-audit311/bin/python scripts/unit_test_guard.py research/signal_independence_20261111/weights.py
|
||||
/private/tmp/pms-audit311/bin/python scripts/unit_test_guard.py research/signal_independence_20261111/diagnostics.py
|
||||
```
|
||||
|
||||
复现读取本目录`data/`及本机上游源码,不必再次连接生产服务。`manifest.json`保存原始三个文件哈希,`weights.json`另记日线哈希;`source_hashes.json`保存用到的上游代码哈希及与188的核对结果。上游源码若变动,须恢复所记录版本后复现,不能把新公式结果当作本次结果。当前结果对应特定时点,不是永久健康证明。
|
||||
|
||||
方法参考仅用于明确“关联、冗余与独立增量”之间的区别,不把监督学习的未来收益目标引入本研究:[JMLR关于冗余、相关与互补信息的区分](https://www.jmlr.org/beta/papers/v24/21-0482.html)。实际采用的是本文公开的简单依赖折扣,并未声称实现该论文算法。
|
||||
|
|
@ -0,0 +1,191 @@
|
|||
"""冻结协议后的离线独立性分析;不读取收益,不连接任何外部服务。"""
|
||||
from __future__ import annotations
|
||||
import collections
|
||||
import datetime as dt
|
||||
import gzip
|
||||
import hashlib
|
||||
import itertools
|
||||
import json
|
||||
import math
|
||||
from pathlib import Path
|
||||
import re
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
ROOT = Path(__file__).resolve().parent
|
||||
DATA = ROOT / 'data'
|
||||
OUT = ROOT
|
||||
TZ = dt.timezone(dt.timedelta(hours=8))
|
||||
START, END = '2026-10-08', '2026-11-10'
|
||||
SOURCES = ['qrs', 'intensity', 'distribution', 'mtf', 'bionic']
|
||||
LABELS = {'qrs':'逐股QRS','intensity':'资金流强度','distribution':'资金分布','mtf':'资金异动','bionic':'决策系统'}
|
||||
HS = re.compile(r'(?:(?:60|68)\d{4}\.SH|(?:00|30)\d{4}\.SZ)$')
|
||||
|
||||
def code(s):
|
||||
s = str(s)
|
||||
if s[:2] in ('SH','SZ'): s=s[2:]+'.'+s[:2]
|
||||
if not HS.fullmatch(s): raise ValueError('non-HS row')
|
||||
return s
|
||||
|
||||
def load():
|
||||
rows=[]; raw=[]; broadcasts=[]; provenance=collections.Counter(); all_judgements=[]
|
||||
manifest={}
|
||||
for p in [DATA/name for name in ['itd.jsonl.gz','bionic.jsonl.gz','streams.jsonl.gz']]:
|
||||
manifest[p.name]={'sha256':hashlib.sha256(p.read_bytes()).hexdigest(),'bytes':p.stat().st_size}
|
||||
records=[json.loads(x) for x in gzip.open(p,'rt')]
|
||||
manifest[p.name]['rows']=len(records)
|
||||
for r in records:
|
||||
if r['kind']=='itd':
|
||||
c=code(r['ts_code']);day=r['trade_date'];t=dt.datetime.fromtimestamp(int(r['trigger_time'])/1000,TZ)
|
||||
meta=r.get('metadata') or {};name=r['source'];parts=meta.get('contributing_sources',[]) if name=='multi_source' else [name]
|
||||
raw.append({'day':day,'source':name,'code':c})
|
||||
for part in parts:
|
||||
if part.startswith('daily_qrs_symmetric'):s='qrs';direction=-1 if part.endswith('down') else 1
|
||||
elif part.startswith('money_flow_'):s='intensity';direction=-1 if '_out_' in part else 1
|
||||
elif part.startswith('capital_'):s='distribution';direction=-1 if part.endswith('distribution') else 1
|
||||
elif part=='intraday_buy_emitted':continue
|
||||
else:raise ValueError('unknown alert source '+part)
|
||||
value=(meta.get('contributing_values') or {}).get(part,r.get('value'))
|
||||
rows.append({'day':day,'source':s,'code':c,'t':t,'direction':direction,'value':float(value),'id':r['alert_id']+':'+part,
|
||||
'parent_source':name,'evidence_date':str(meta.get('trade_date') or '')[:10], 'level':r['level']})
|
||||
elif r['kind']=='mtf':
|
||||
c=code(r['ts_code']);received=dt.datetime.fromtimestamp(int(r['stream_id'].split('-')[0])/1000,TZ)
|
||||
day=received.date().isoformat();t=dt.datetime.fromisoformat(day+'T'+r['calc_time']).replace(tzinfo=TZ)
|
||||
rows.append({'day':day,'source':'mtf','code':c,'t':t,'direction':1 if r['direction']=='inflow' else -1,
|
||||
'value':float(r['z_dd']),'id':r['stream_id'],'parent_source':'mtf','evidence_date':day,'level':r.get('mode')})
|
||||
elif r['kind']=='bionic_audit':
|
||||
c=code(r['stock_code']);day=dt.datetime.strptime(str(r['audit_date']),'%Y%m%d').date().isoformat()
|
||||
# 审计库存UTC naive时间;随后用同股同结论广播独立核对偏移。
|
||||
t=dt.datetime.fromisoformat(r['created_at']).replace(tzinfo=dt.timezone.utc).astimezone(TZ)
|
||||
if t.date().isoformat()!=day: raise ValueError('audit timezone/date mismatch')
|
||||
trigger=r['reason'].split('源:',1)[-1].split(' | 判定:',1)[0]
|
||||
prefix=re.match(r'【盘中([^】]+)】',r['reason']).group(1)
|
||||
provenance[(prefix,trigger.split('(')[0].strip())]+=1
|
||||
all_judgements.append({'day':day,'code':c,'t':t,'verdict':r['verdict'],'prefix':prefix,'trigger':trigger})
|
||||
direction={'REVERSAL_BUY':1,'REVERSAL_SELL':-1,'TAKE_PROFIT':-1}.get(r['verdict'])
|
||||
if direction is None:continue # 维持不是中性票,不从旧计划反推方向。
|
||||
rows.append({'day':day,'source':'bionic','code':c,'t':t,'direction':direction,'value':direction,
|
||||
'id':f'{c}:{t.isoformat()}:{r["verdict"]}','parent_source':prefix,'verdict':r['verdict'],'trigger':trigger,'evidence_date':'','level':''})
|
||||
elif r['kind']=='bionic_broadcast':
|
||||
r['code']=code(r['ts_code']);broadcasts.append(r)
|
||||
df=pd.DataFrame(rows).drop_duplicates(['source','code','id']).sort_values(['t','source','code'])
|
||||
# 合成告警拆为原成分,同一时刻同股票同成分只保留一条。
|
||||
df=df.drop_duplicates(['source','code','t','direction'],keep='last')
|
||||
OUT.joinpath('manifest.json').write_text(json.dumps({'input':manifest,'scope':'沪深','protocol_sha256':hashlib.sha256((ROOT/'protocol.json').read_bytes()).hexdigest()},indent=2,ensure_ascii=False))
|
||||
return df,pd.DataFrame(raw),broadcasts,pd.DataFrame(all_judgements),provenance
|
||||
|
||||
def eligible(x,y,days):
|
||||
return len(x)>=50 and len(set(days))>=10 and min(sum(x>0),sum(x<0),sum(y>0),sum(y<0))>=10
|
||||
|
||||
def corr(x,y):
|
||||
x=np.asarray(x,dtype=float);y=np.asarray(y,dtype=float)
|
||||
if len(x)<2 or np.std(x)<1e-12 or np.std(y)<1e-12:return None
|
||||
return float(np.corrcoef(x,y)[0,1])
|
||||
|
||||
def bootstrap_pairs(frame,group,seed=20261111,n=1000):
|
||||
rng=np.random.default_rng(seed);blocks=[g[['x','y']].to_numpy() for _,g in frame.groupby(group)]
|
||||
estimates=[]
|
||||
for _ in range(n):
|
||||
v=np.concatenate([blocks[i] for i in rng.integers(len(blocks),size=len(blocks))]);r=corr(v[:,0],v[:,1])
|
||||
if r is not None:estimates.append(r)
|
||||
return [float(x) for x in np.quantile(estimates,[.025,.5,.975])] if estimates else None
|
||||
|
||||
def pair_stats(panel,bootstrap=True):
|
||||
results=[]
|
||||
for a,b in itertools.combinations(SOURCES,2):
|
||||
if a not in panel or b not in panel:continue
|
||||
p=panel[[a,b]].dropna().reset_index().rename(columns={a:'x',b:'y'})
|
||||
if p.empty:continue
|
||||
x,y=p.x.to_numpy(),p.y.to_numpy();r=corr(x,y)
|
||||
item={'a':a,'b':b,'n':len(p),'dates':p.day.nunique(),'stocks':p.code.nunique(),'corr':r,
|
||||
'agreement':float(np.mean(np.sign(x)==np.sign(y))),
|
||||
'a_positive':int(sum(x>0)),'a_negative':int(sum(x<0)), 'b_positive':int(sum(y>0)),'b_negative':int(sum(y<0)),
|
||||
'eligible':bool(eligible(x,y,p.day))}
|
||||
if bootstrap and r is not None and len(p)>20:
|
||||
item['day_ci']=bootstrap_pairs(p,'day');item['stock_ci']=bootstrap_pairs(p,'code',n=500)
|
||||
results.append(item)
|
||||
return results
|
||||
|
||||
def panels(df,how='last'):
|
||||
group=df.groupby(['day','code','source'])
|
||||
if how=='last':return group.direction.last().unstack('source')
|
||||
if how=='first':return group.direction.first().unstack('source')
|
||||
if how=='mean':return group.direction.mean().unstack('source')
|
||||
raise ValueError(how)
|
||||
|
||||
def trade_minute(t):
|
||||
minute=t.hour*60+t.minute+t.second/60
|
||||
if 570<=minute<=690:return minute-570
|
||||
if 780<=minute<=900:return minute-660
|
||||
return None
|
||||
|
||||
def event_matching(df):
|
||||
result=[]
|
||||
# QRS是日状态;盘中匹配只比较真正按分钟更新的来源。
|
||||
for a,b in itertools.combinations(['intensity','distribution','mtf','bionic'],2):
|
||||
for window in [15,30,60]:
|
||||
matches=[]
|
||||
by_b={k:g for k,g in df[df.source==b].groupby(['day','code'])}
|
||||
count=0
|
||||
for key,g in df[df.source==a].groupby(['day','code']):
|
||||
q=by_b.get(key)
|
||||
if q is None:continue
|
||||
qt=[(trade_minute(r.t),r) for r in q.itertuples()]
|
||||
qt=[x for x in qt if x[0] is not None]
|
||||
if not qt:continue
|
||||
for row in g.itertuples():
|
||||
now=trade_minute(row.t)
|
||||
if now is None:continue
|
||||
count+=1
|
||||
lag,other=min(((z-now,r) for z,r in qt),key=lambda x:abs(x[0]))
|
||||
if abs(lag)<=window:matches.append({'day':key[0],'code':key[1],'x':row.direction,'y':other.direction,'lag':lag})
|
||||
p=pd.DataFrame(matches)
|
||||
result.append({'a':a,'b':b,'window':window,'eligible_a_events':count,'matched':len(p),
|
||||
'agreement':float((p.x==p.y).mean()) if len(p) else None,
|
||||
'corr':corr(p.x,p.y) if len(p) else None,
|
||||
'median_lag_b_minus_a':float(p.lag.median()) if len(p) else None})
|
||||
return result
|
||||
|
||||
def timezone_check(df,broadcasts):
|
||||
b=df[df.source=='bionic'];diff=[];clock=[]
|
||||
for r in broadcasts:
|
||||
t=dt.datetime.fromtimestamp(int(r['trigger_time'])/1000,TZ)
|
||||
p=b[(b.code==r['code']) & (b.day==r['trade_date']) & (b.verdict==r['verdict'])]
|
||||
if len(p):diff.append(float(min(abs((x-t).total_seconds()) for x in p.t)))
|
||||
rec=dt.datetime.fromtimestamp(int(r['stream_id'].split('-')[0])/1000,TZ)
|
||||
clock.append((rec-t).total_seconds())
|
||||
return {'broadcasts':len(broadcasts),'matched_audits':len(diff),
|
||||
'audit_broadcast_abs_sec_quantiles':np.quantile(diff,[0,.5,.95,1]).tolist() if diff else [],
|
||||
'redis_minus_publisher_sec_quantiles':np.quantile(clock,[0,.5,.95,1]).tolist() if clock else []}
|
||||
|
||||
def main():
|
||||
df,raw,broadcasts,judgements,provenance=load();primary=df[(df.day>=START)&(df.day<=END)].copy()
|
||||
summaries=[]
|
||||
for s,g in primary.groupby('source'):
|
||||
groups=g.groupby(['day','code']);daily=g.groupby('day').size()
|
||||
summaries.append({'source':s,'events':len(g),'stocks':g.code.nunique(),'days':g.day.nunique(),'stock_days':len(groups),
|
||||
'positive_events':int(sum(g.direction>0)),'negative_events':int(sum(g.direction<0)),
|
||||
'daily_events_mean':float(daily.mean()),'daily_events_median':float(daily.median()),
|
||||
'direction_changing_stock_days':int((groups.direction.nunique()>1).sum()),
|
||||
'duplicate_fixed_qrs':len(g)-len(groups) if s=='qrs' else None})
|
||||
panel=panels(primary)
|
||||
result={'window':[START,END],'source_summary':summaries,'pairs':pair_stats(panel),
|
||||
'first_pairs':pair_stats(panels(primary,'first'),False),'mean_pairs':pair_stats(panels(primary,'mean'),False),
|
||||
'halves':{},'secondary':pair_stats(panels(df[df.source!='bionic']),False),
|
||||
'event_matches':event_matching(primary),'timezone':timezone_check(primary,broadcasts),
|
||||
'bionic_trigger_directions':judgements.prefix.value_counts().to_dict(),
|
||||
'bionic_verdicts':judgements.verdict.value_counts().to_dict(),
|
||||
'bionic_directional_triggers':primary[primary.source=='bionic'].parent_source.value_counts().to_dict(),
|
||||
'bionic_trigger_examples': [{'type':k[0],'trigger':k[1],'n':n} for k,n in provenance.most_common(15)]}
|
||||
days=sorted(primary.day.unique());mid=days[len(days)//2]
|
||||
for name,d in [('first',primary[primary.day<mid]),('second',primary[primary.day>=mid])]:result['halves'][name]={'start':d.day.min(),'end':d.day.max(),'pairs':pair_stats(panels(d),False)}
|
||||
# 审计是否在本窗口所有日期均有,不将没有发信号的股票日补0。
|
||||
result['presence_dates']={s:sorted(g.day.unique().tolist()) for s,g in primary.groupby('source')}
|
||||
OUT.joinpath('results.json').write_text(json.dumps(result,ensure_ascii=False,indent=2))
|
||||
# 归一化事件与原始沪深研究输入一并留存,避免上游TTL到期后无法复现。
|
||||
primary.to_csv(DATA/'normalized.csv.gz',index=False,compression='gzip')
|
||||
panel.to_csv(DATA/'paired_daily.csv.gz',compression='gzip')
|
||||
print(json.dumps({'summary':summaries,'pairs':result['pairs'],'timezone':result['timezone'],'bionic':result['bionic_directional_triggers']},ensure_ascii=False,indent=2))
|
||||
|
||||
if __name__=='__main__':main()
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
|
|
@ -0,0 +1,114 @@
|
|||
{
|
||||
"historical_qrs_reproduction": {
|
||||
"rows": 354,
|
||||
"within_1e_6": 295,
|
||||
"error_quantiles": [
|
||||
0.0,
|
||||
0.0,
|
||||
0.005059570832555149,
|
||||
0.19359438584218314
|
||||
],
|
||||
"note": "原始历史日线可能有修订、请求暖机范围差异;未将重算伪称当时实发。"
|
||||
},
|
||||
"event_pairs": [
|
||||
{
|
||||
"a": "intensity",
|
||||
"b": "distribution",
|
||||
"minutes": 15,
|
||||
"matched_a_events": 101,
|
||||
"unique_b_events": 81,
|
||||
"stock_days": 41,
|
||||
"dates": 11,
|
||||
"agree": 1.0,
|
||||
"note": "最近匹配可能多对一,事件数不作独立样本数"
|
||||
},
|
||||
{
|
||||
"a": "intensity",
|
||||
"b": "distribution",
|
||||
"minutes": 30,
|
||||
"matched_a_events": 146,
|
||||
"unique_b_events": 93,
|
||||
"stock_days": 46,
|
||||
"dates": 11,
|
||||
"agree": 1.0,
|
||||
"note": "最近匹配可能多对一,事件数不作独立样本数"
|
||||
},
|
||||
{
|
||||
"a": "intensity",
|
||||
"b": "distribution",
|
||||
"minutes": 60,
|
||||
"matched_a_events": 198,
|
||||
"unique_b_events": 96,
|
||||
"stock_days": 49,
|
||||
"dates": 11,
|
||||
"agree": 0.9949494949494949,
|
||||
"note": "最近匹配可能多对一,事件数不作独立样本数"
|
||||
},
|
||||
{
|
||||
"a": "mtf",
|
||||
"b": "bionic",
|
||||
"minutes": 15,
|
||||
"matched_a_events": 31,
|
||||
"unique_b_events": 31,
|
||||
"stock_days": 27,
|
||||
"dates": 15,
|
||||
"agree": 0.9354838709677419,
|
||||
"note": "最近匹配可能多对一,事件数不作独立样本数"
|
||||
},
|
||||
{
|
||||
"a": "mtf",
|
||||
"b": "bionic",
|
||||
"minutes": 30,
|
||||
"matched_a_events": 44,
|
||||
"unique_b_events": 43,
|
||||
"stock_days": 37,
|
||||
"dates": 16,
|
||||
"agree": 0.9545454545454546,
|
||||
"note": "最近匹配可能多对一,事件数不作独立样本数"
|
||||
},
|
||||
{
|
||||
"a": "mtf",
|
||||
"b": "bionic",
|
||||
"minutes": 60,
|
||||
"matched_a_events": 75,
|
||||
"unique_b_events": 65,
|
||||
"stock_days": 57,
|
||||
"dates": 18,
|
||||
"agree": 0.8933333333333333,
|
||||
"note": "最近匹配可能多对一,事件数不作独立样本数"
|
||||
},
|
||||
{
|
||||
"a": "distribution",
|
||||
"b": "bionic",
|
||||
"minutes": 15,
|
||||
"matched_a_events": 15,
|
||||
"unique_b_events": 12,
|
||||
"stock_days": 11,
|
||||
"dates": 10,
|
||||
"agree": 1.0,
|
||||
"note": "最近匹配可能多对一,事件数不作独立样本数"
|
||||
},
|
||||
{
|
||||
"a": "distribution",
|
||||
"b": "bionic",
|
||||
"minutes": 30,
|
||||
"matched_a_events": 21,
|
||||
"unique_b_events": 14,
|
||||
"stock_days": 12,
|
||||
"dates": 11,
|
||||
"agree": 1.0,
|
||||
"note": "最近匹配可能多对一,事件数不作独立样本数"
|
||||
},
|
||||
{
|
||||
"a": "distribution",
|
||||
"b": "bionic",
|
||||
"minutes": 60,
|
||||
"matched_a_events": 28,
|
||||
"unique_b_events": 17,
|
||||
"stock_days": 14,
|
||||
"dates": 13,
|
||||
"agree": 1.0,
|
||||
"note": "最近匹配可能多对一,事件数不作独立样本数"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
"""记录原值复现误差、QRS净化后的支持量及同窗事件的有效样本数。"""
|
||||
import gzip,json
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import analyze as a
|
||||
import weights as w
|
||||
|
||||
f=pd.read_csv(a.DATA/'normalized.csv.gz');f.t=pd.to_datetime(f.t)
|
||||
bars=pd.DataFrame([json.loads(x) for x in gzip.open(a.DATA/'daily.jsonl.gz','rt')]);bars['trade_date']=pd.to_datetime(bars.timestamp).dt.date
|
||||
bars['code']=bars.symbol.map(a.code)
|
||||
for col in ['high','low','close']:bars[col]=pd.to_numeric(bars[col])
|
||||
bycode={c:g for c,g in bars.groupby('code')};calc=w.DailyQRSComputer(None,n_regression=18,m_lookback=60,threshold=.7)
|
||||
cache={};errs=[]
|
||||
for r in f[f.source=='qrs'].itertuples():
|
||||
key=(r.code,r.evidence_date)
|
||||
if key not in cache:cache[key]=calc._compute_from_dataframe(r.code,bycode[r.code],pd.Timestamp(r.evidence_date).date()).qrs_value
|
||||
errs.append(abs(cache[key]-r.value))
|
||||
result={'historical_qrs_reproduction':{'rows':len(errs),'within_1e_6':int(sum(x<1e-6 for x in errs)),
|
||||
'error_quantiles':np.quantile(errs,[0,.5,.95,1]).tolist(),
|
||||
'note':'原始历史日线可能有修订、请求暖机范围差异;未将重算伪称当时实发。'},'event_pairs':[]}
|
||||
for aa,bb in [('intensity','distribution'),('mtf','bionic'),('distribution','bionic')]:
|
||||
for window in [15,30,60]:
|
||||
matched=[];seen=set();by_b={k:g for k,g in f[f.source==bb].groupby(['day','code'])}
|
||||
for key,g in f[f.source==aa].groupby(['day','code']):
|
||||
q=by_b.get(key)
|
||||
if q is None:continue
|
||||
qt=[(a.trade_minute(r.t),r) for r in q.itertuples()];qt=[x for x in qt if x[0] is not None]
|
||||
if not qt:continue
|
||||
for row in g.itertuples():
|
||||
now=a.trade_minute(row.t)
|
||||
if now is None:continue
|
||||
lag,other=min(((z-now,r) for z,r in qt),key=lambda x:abs(x[0]))
|
||||
if abs(lag)<=window:
|
||||
matched.append((key[0],key[1],row.direction,other.direction,other.id));seen.add(other.id)
|
||||
p=pd.DataFrame(matched,columns=['day','code','x','y','b_id'])
|
||||
result['event_pairs'].append({'a':aa,'b':bb,'minutes':window,'matched_a_events':len(p),'unique_b_events':len(seen),
|
||||
'stock_days':len(p[['day','code']].drop_duplicates()),'dates':p.day.nunique(),
|
||||
'agree':float((p.x==p.y).mean()),'note':'最近匹配可能多对一,事件数不作独立样本数'})
|
||||
a.OUT.joinpath('diagnostics.json').write_text(json.dumps(result,ensure_ascii=False,indent=2))
|
||||
print(json.dumps(result,ensure_ascii=False,indent=2))
|
||||
|
|
@ -0,0 +1,21 @@
|
|||
"""执行前由开发机拼接已限定沪深的CODES常量;只读日线。"""
|
||||
import datetime
|
||||
import gzip
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from sqlalchemy import event,text
|
||||
from sqlalchemy.engine import Engine
|
||||
@event.listens_for(Engine,'before_cursor_execute')
|
||||
def guard(conn,cursor,statement,parameters,context,executemany):
|
||||
if not statement.lstrip().upper().startswith(('SELECT','SHOW','DESCRIBE')):raise RuntimeError('read-only')
|
||||
assert all(re.fullmatch(r'(SH(?:60|68)\d{4}|SZ(?:00|30)\d{4})',c) for c in CODES)
|
||||
from scripts.backfill_v3f_baseline import _get_external_engine
|
||||
eng=_get_external_engine()
|
||||
out=gzip.GzipFile(fileobj=sys.stdout.buffer,mode='wb',mtime=0)
|
||||
with eng.connect() as conn:
|
||||
for i in range(0,len(CODES),40):
|
||||
batch=CODES[i:i+40];params={f'c{j}':c for j,c in enumerate(batch)}
|
||||
q="SELECT symbol,timestamp,high,low,close FROM gp_day_data WHERE symbol IN ("+','.join(':'+k for k in params)+") AND timestamp >= '2026-05-01' AND timestamp < '2026-11-11' ORDER BY symbol,timestamp"
|
||||
for r in conn.execute(text(q),params):out.write((json.dumps(dict(r._mapping),default=str)+'\n').encode())
|
||||
out.close();eng.dispose()
|
||||
|
|
@ -0,0 +1,68 @@
|
|||
"""在上游容器内只读导出;stdout为gzip JSONL,不写上游数据。"""
|
||||
import datetime as dt
|
||||
import gzip
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
START = '2026-10-08'
|
||||
END = '2026-11-10'
|
||||
HS = re.compile(r'(?:(?:60|68)\d{4}\.SH|(?:00|30)\d{4}\.SZ|SH(?:60|68)\d{4}|SZ(?:00|30)\d{4})$')
|
||||
out = gzip.GzipFile(fileobj=sys.stdout.buffer, mode='wb', mtime=0)
|
||||
|
||||
def emit(kind, data):
|
||||
out.write((json.dumps({'kind': kind, **data}, ensure_ascii=False, default=str)+'\n').encode())
|
||||
|
||||
mode = sys.argv[1]
|
||||
if mode == 'itd':
|
||||
for p in sorted(Path('/app/data/alert_audit').glob('2026-*.jsonl')):
|
||||
if not '2026-07-22' <= p.stem <= END: continue
|
||||
for line in p.open():
|
||||
r = json.loads(line)
|
||||
if HS.fullmatch(str(r.get('ts_code', ''))): emit('itd', r)
|
||||
elif mode == 'bionic':
|
||||
from sqlalchemy import create_engine, text, event
|
||||
from sqlalchemy.engine import Engine
|
||||
@event.listens_for(Engine, 'before_cursor_execute')
|
||||
def guard(conn, cursor, statement, parameters, context, executemany):
|
||||
if not statement.lstrip().upper().startswith(('SELECT', 'SHOW', 'DESCRIBE')):
|
||||
raise RuntimeError('read-only export')
|
||||
from config.settings import settings
|
||||
eng = create_engine(settings.SOURCE_DB_EXT_DSN, pool_pre_ping=False)
|
||||
query = """SELECT stock_code,audit_date,strategy_date,verdict,reason,created_at
|
||||
FROM strategy_audit_log
|
||||
WHERE audit_date >= :start AND audit_date <= :end
|
||||
AND (stock_code LIKE 'SH60%' OR stock_code LIKE 'SH68%' OR stock_code LIKE 'SZ00%' OR stock_code LIKE 'SZ30%')
|
||||
AND verdict IN ('REVERSAL_BUY','REVERSAL_SELL','TAKE_PROFIT','MAINTAIN_HOLD','MAINTAIN')
|
||||
AND reason LIKE :reason
|
||||
ORDER BY audit_date,created_at,stock_code"""
|
||||
with eng.connect() as c:
|
||||
for r in c.execute(text(query), {'start':int(START.replace('-','')), 'end':int(END.replace('-','')), 'reason':'【盘中%'}):
|
||||
emit('bionic_audit', dict(r._mapping))
|
||||
eng.dispose()
|
||||
elif mode == 'streams':
|
||||
import redis
|
||||
from config.settings import settings
|
||||
c = redis.Redis(host=settings.SIGNAL_REDIS_HOST, port=settings.SIGNAL_REDIS_PORT,
|
||||
password=settings.SIGNAL_REDIS_PASSWORD, db=2, decode_responses=True,
|
||||
socket_timeout=8, socket_connect_timeout=5)
|
||||
keys = ['mtf:intraday:stream:metrics'] + sorted(c.scan_iter(match='intraday_signals:2026-11-*', count=100))
|
||||
for key in keys:
|
||||
if c.type(key) != 'stream': continue
|
||||
cursor = str(int(dt.datetime.fromisoformat(START).timestamp()*1000))+'-0'
|
||||
stop = str(int(dt.datetime.fromisoformat('2026-11-11').timestamp()*1000)-1)+'-999999'
|
||||
while True:
|
||||
rows = c.xrange(key, min=cursor, max=stop, count=500)
|
||||
if not rows: break
|
||||
for ident, raw in rows:
|
||||
r = {**raw, **json.loads(raw['data'])} if raw.get('data') else raw
|
||||
code = r.get('ts_code') or r.get('symbol') or r.get('stock_code')
|
||||
if HS.fullmatch(str(code)):
|
||||
r.pop('data', None)
|
||||
emit('mtf' if key.startswith('mtf:') else 'bionic_broadcast', {'stream_key':key,'stream_id':ident, **r})
|
||||
ms, seq = rows[-1][0].split('-'); cursor = ms+'-'+str(int(seq)+1)
|
||||
if len(rows) < 500: break
|
||||
else:
|
||||
raise ValueError('unsupported export mode')
|
||||
out.close()
|
||||
|
|
@ -0,0 +1,21 @@
|
|||
{
|
||||
"input": {
|
||||
"bionic.jsonl.gz": {
|
||||
"sha256": "119b06d0ec011c370104f0cec1f6012faacd2e707d43a7193190242c38480e3e",
|
||||
"bytes": 173310,
|
||||
"rows": 2641
|
||||
},
|
||||
"itd.jsonl.gz": {
|
||||
"sha256": "4b5c105a1a1f62506e7fe8a4640d7707f1f388dc45454c962564d7a7d02cf392",
|
||||
"bytes": 472072,
|
||||
"rows": 8967
|
||||
},
|
||||
"streams.jsonl.gz": {
|
||||
"sha256": "852161df56ac04d0ae2d3ffbd1909b1470f22c3eaeaf0ee9c944edf7fc346504",
|
||||
"bytes": 3326817,
|
||||
"rows": 44162
|
||||
}
|
||||
},
|
||||
"scope": "沪深",
|
||||
"protocol_sha256": "81b6ad1c50839c709c49bba877a147537de041cbd8648cbed285823840062ba8"
|
||||
}
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
{
|
||||
"version": "1.0",
|
||||
"objective": "利用现有沪深信号估计来源之间的信息重叠与独立贡献,提出可解释的权重;不使用未来收益、交易成功率或盈亏作目标",
|
||||
"primary_start": "2026-10-08",
|
||||
"primary_end": "2026-11-10",
|
||||
"secondary_start": "2026-07-22",
|
||||
"secondary_end": "2026-11-10",
|
||||
"scope": "仅规范化代码为60/68开头.SH及00/30开头.SZ的沪深股票;在导出入口排除北交所",
|
||||
"inputs": ["量化系统发布审计", "资金异动原始发布流", "Bionic判断审计及可取得的原始广播"],
|
||||
"exclusions": ["当日未结束样本", "收益和成交数据", "资金流强度2026-07-21及以前的数值", "合成告警与成分重复计票", "同一BUY的跨流镜像", "把无信号或缺覆盖填成中性"],
|
||||
"source_families": ["逐股QRS", "资金流强度", "资金分布", "资金异动v3f", "Bionic研判"],
|
||||
"deduplication": "按原始ID先去重;QRS每日固定状态重复发布按股票、证据日、方向合并;资金与模型保留新的观察,但统计使用股票日作为主要单位",
|
||||
"primary_statistics": ["各源发布覆盖、方向分布、重复率", "共同有观测的股票日方向关联与一致率", "同股事件在交易时间15/30/60分钟内的先后与重合", "Bionic触发来源及镜像关系"],
|
||||
"weights_method": "先按代码中的原始证据来源分组,组内不因多包装增加预算;在共同可观测样本上以关联度计算冗余折扣。基准等预算,候选独立度u_i=1/(1+sum_j R_ij),归一后仅作为依赖折扣权重。R优先取双向关联的绝对值,相关不足不得设0冒充独立;不以噪声或低覆盖换高权重。结构派生关系单独限制,不能仅凭相关系数消除。",
|
||||
"support_thresholds": {"min_joint_stock_days": 50, "min_joint_dates": 10, "min_each_direction_per_source": 10},
|
||||
"uncertainty": "按交易日期整组bootstrap,1000次,固定seed=20261111,报告95%区间;按股票整组复核;前后半窗及15/30/60分钟匹配作敏感性,不选择最漂亮的一组",
|
||||
"baseline": "来源等权及40/40/20旧示意仅作结构对照,不当已验证参数",
|
||||
"missingness": "共同有发布的样本只代表条件分布;不声称完整监测覆盖或总体统计独立。无法识别的权重给出范围或保留未分配预算,不伪造点估计",
|
||||
"deployment": "仅离线研究,不改线上参数、策略或服务"
|
||||
}
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
{
|
||||
"version": "2.0",
|
||||
"why_added": "首轮质量核对发现QRS发布引用旧日线,不能用该低相关直接给高权重。保留v1原始结果,新增净化和确定性重算对照;该追加设计已见过v1结果,不称独立样本外验证。",
|
||||
"freshness": "QRS数据日必须为之前最近的实际交易日;用600519.SH日线日期核对,过旧记录从有效实发估计剔除。",
|
||||
"grouping": "资金流强度与资金分布共用主力资金原料且v1相关约0.90,合成一个主力资金组;同股同日取组内最后有效观察,前值/均值与严格共同观察另作敏感性。",
|
||||
"reconstruction": "仅为原354个QRS发布股票日,以当前持有的历史日线、生产N=18/M=60公式重算前一交易日原始指标。不是补发信号,不使用今天的逐股阈值替代过去阈值,不伪造历史已可得性。重算值符号只作原始指标依赖的辅助检验,不能当成当时已发布方向。",
|
||||
"weight_scope": "QRS、主力资金、资金异动三个原始证据组按等先验经绝对相关冗余折扣;Bionic是组合研判层,已复述的证据不能另增一组预算。Bionic独立增量权重如无可识别独立证据和足够匹配样本,不凭空给数。",
|
||||
"statistical_limits": "主要报告完整实发、有效实发、重算对照的不同支持量;若区间覆盖等权,则选择等权作为易解释基准,不以小数位伪装精度。依赖度不能证明因果、信息质量或预测能力。",
|
||||
"timing": "Bionic审计按UTC转北京后与68条原始广播核对。Redis消息ID相对发布者约慢266秒,先后分析只使用发布者时间,不用Redis ID推因果;MTF calc_time只有分钟精度,15/30/60分钟匹配只作敏感性。"
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
|
|
@ -0,0 +1,17 @@
|
|||
{
|
||||
"alerts_loop.py": {
|
||||
"path": "/Users/baobao/Documents/work/project/intraday_timing/src/intraday_timing/commands/alerts_loop.py",
|
||||
"sha256": "a31bac4bff417845f2299f366c9f97c6e00645ef53e85d203dec9cf775fd73f0",
|
||||
"matches_188_intraday_alerts": true
|
||||
},
|
||||
"kline_daily_mem_cache.py": {
|
||||
"path": "/Users/baobao/Documents/work/project/intraday_timing/src/intraday_timing/adapters/kline_daily_mem_cache.py",
|
||||
"sha256": "4c7b1f620ad79260aa6ebe210ba8ac46d51d444295db40e463c08c14280ea7f8",
|
||||
"matches_188_intraday_alerts": true
|
||||
},
|
||||
"daily_qrs_computer.py": {
|
||||
"path": "/Users/baobao/Documents/work/project/intraday_timing/src/intraday_timing/state/daily_qrs_computer.py",
|
||||
"sha256": "ed9be2ed4804441fde03557648b7e2f2422dbd1e8e74e2187c3760443cc940dc",
|
||||
"matches_188_intraday_alerts": true
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,60 @@
|
|||
"""独立性统计与旧缓存行为的离线验证;全部数据为内存样例。"""
|
||||
import datetime as dt
|
||||
import math
|
||||
import sys
|
||||
from pathlib import Path
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import analyze as a
|
||||
import weights as w
|
||||
|
||||
def main():
|
||||
# 1. 无信号保留NaN,不能在相关样本中伪造0。
|
||||
x=pd.DataFrame([{'day':'2026-11-09','code':'600000.SH','source':'qrs','direction':1},
|
||||
{'day':'2026-11-09','code':'600001.SH','source':'mtf','direction':-1}])
|
||||
p=a.panels(x);assert p[['qrs','mtf']].dropna().empty
|
||||
# 2. 恒定方向不能得到“零相关、所以独立”的错误结论。
|
||||
assert a.corr([1,1,1],[1,-1,1]) is None
|
||||
# 3. 相同或完全反向两个通道都携带冗余,方向相反不增加独立预算。
|
||||
assert math.isclose(abs(a.corr([1,-1,1,-1],[-1,1,-1,1])),1)
|
||||
# 4. 正交且信息量对称,权重应等分。
|
||||
orth=pd.DataFrame({'qrs':[1,1,-1,-1],'fund':[1,-1,1,-1],'mtf':[1,-1,-1,1]})
|
||||
r=w.dependence_weights(orth);assert all(abs(v-1/3)<1e-12 for v in r['weights'].values())
|
||||
# 5. 复制资金通道不能令该组多拿预算;分组后只保留观察时间确定的同一状态。
|
||||
d=pd.DataFrame([{'day':'2026-11-09','code':'600000.SH','source':'intensity','direction':-1},
|
||||
{'day':'2026-11-09','code':'600000.SH','source':'distribution','direction':-1}])
|
||||
gp=w.grouped_panel(d);assert list(gp.columns)==['fund'] and gp.iloc[0,0]==-1
|
||||
# 6. 午休不算经过的交易时间,前后两段相差一分钟。
|
||||
assert a.trade_minute(dt.datetime(2026,11,9,13,1))-a.trade_minute(dt.datetime(2026,11,9,11,30))==1
|
||||
# 7. 北交所及格式不明代码不能进入分析。
|
||||
for c in ['920001.BJ','BJ430001','600000']:
|
||||
try:a.code(c)
|
||||
except ValueError:pass
|
||||
else:raise AssertionError(c)
|
||||
# 8. 当前生产缓存跨日重复preload不会重新读取;显式clear后才更新。
|
||||
from intraday_timing.adapters.kline_daily_mem_cache import KlineDailyBulkMemCache
|
||||
class Conn:
|
||||
calls=0
|
||||
day='2026-11-06'
|
||||
def execute_query(self,*args,**kwargs):
|
||||
self.calls+=1
|
||||
return pd.DataFrame([{'symbol':'SH600000','trade_date':self.day,'open':10,'close':10,'high':11,'low':9,'vol':100,'amount':1000,'pre_close':10}])
|
||||
c=Conn();cache=KlineDailyBulkMemCache(None,c)
|
||||
cache.preload(['600000.SH'],dt.date(2026,11,1),dt.date(2026,11,9));c.day='2026-11-09'
|
||||
assert cache.preload(['600000.SH'],dt.date(2026,11,1),dt.date(2026,11,10))==0 and c.calls==1
|
||||
assert cache.get_kline_daily('600000.SH',dt.date(2026,11,1),dt.date(2026,11,10)).trade_date.max()==dt.date(2026,11,6)
|
||||
cache.clear();cache.preload(['600000.SH'],dt.date(2026,11,1),dt.date(2026,11,10));assert c.calls==2
|
||||
assert cache.get_kline_daily('600000.SH',dt.date(2026,11,1),dt.date(2026,11,10)).trade_date.max()==dt.date(2026,11,9)
|
||||
# 9. 给历史序列追加未来极端行情,不得改变过去时点的QRS。
|
||||
dates=pd.date_range('2026-01-01',periods=120).date
|
||||
low=np.linspace(10,20,120)+np.sin(np.arange(120)/3)
|
||||
high=low+1+np.cos(np.arange(120)/4)*.2
|
||||
bars=pd.DataFrame({'trade_date':dates,'high':high,'low':low})
|
||||
calc=w.DailyQRSComputer(None,n_regression=18,m_lookback=60,threshold=.7)
|
||||
cutoff=dates[90];before=calc._compute_from_dataframe('600000.SH',bars.iloc[:91],cutoff).qrs_value
|
||||
bars.loc[91:,['high','low']]*=100
|
||||
after=calc._compute_from_dataframe('600000.SH',bars,cutoff).qrs_value
|
||||
assert abs(before-after)<1e-12
|
||||
print('9组离线边界通过,包含真实缓存类跨日复现与QRS未来截断一致性。')
|
||||
|
||||
if __name__=='__main__':main()
|
||||
|
|
@ -0,0 +1,562 @@
|
|||
{
|
||||
"qrs_quality": {
|
||||
"events": 354,
|
||||
"fresh": 17,
|
||||
"stale": 337,
|
||||
"age_median": 41.0,
|
||||
"same_evidence_repeated": 248
|
||||
},
|
||||
"reconstruction": {
|
||||
"rows": 354,
|
||||
"changed_sign": 138,
|
||||
"old_value_reproduced": 354,
|
||||
"old_value_max_abs_error": 0.19359438584218314,
|
||||
"errors": []
|
||||
},
|
||||
"clean_pairs": [
|
||||
{
|
||||
"a": "qrs",
|
||||
"b": "fund",
|
||||
"n": 5,
|
||||
"dates": 5,
|
||||
"stocks": 5,
|
||||
"corr": -0.24999999999999997,
|
||||
"agreement": 0.6,
|
||||
"a_positive": 1,
|
||||
"a_negative": 4,
|
||||
"b_positive": 1,
|
||||
"b_negative": 4,
|
||||
"eligible": false
|
||||
},
|
||||
{
|
||||
"a": "qrs",
|
||||
"b": "mtf",
|
||||
"n": 7,
|
||||
"dates": 6,
|
||||
"stocks": 7,
|
||||
"corr": null,
|
||||
"agreement": 0.42857142857142855,
|
||||
"a_positive": 3,
|
||||
"a_negative": 4,
|
||||
"b_positive": 7,
|
||||
"b_negative": 0,
|
||||
"eligible": false
|
||||
},
|
||||
{
|
||||
"a": "qrs",
|
||||
"b": "bionic",
|
||||
"n": 3,
|
||||
"dates": 3,
|
||||
"stocks": 3,
|
||||
"corr": null,
|
||||
"agreement": 0.6666666666666666,
|
||||
"a_positive": 2,
|
||||
"a_negative": 1,
|
||||
"b_positive": 3,
|
||||
"b_negative": 0,
|
||||
"eligible": false
|
||||
},
|
||||
{
|
||||
"a": "fund",
|
||||
"b": "mtf",
|
||||
"n": 273,
|
||||
"dates": 24,
|
||||
"stocks": 130,
|
||||
"corr": 0.18367126921523455,
|
||||
"agreement": 0.6007326007326007,
|
||||
"a_positive": 78,
|
||||
"a_negative": 195,
|
||||
"b_positive": 125,
|
||||
"b_negative": 148,
|
||||
"eligible": true
|
||||
},
|
||||
{
|
||||
"a": "fund",
|
||||
"b": "bionic",
|
||||
"n": 39,
|
||||
"dates": 20,
|
||||
"stocks": 28,
|
||||
"corr": 0.7385686213637483,
|
||||
"agreement": 0.8717948717948718,
|
||||
"a_positive": 22,
|
||||
"a_negative": 17,
|
||||
"b_positive": 23,
|
||||
"b_negative": 16,
|
||||
"eligible": false
|
||||
},
|
||||
{
|
||||
"a": "mtf",
|
||||
"b": "bionic",
|
||||
"n": 82,
|
||||
"dates": 23,
|
||||
"stocks": 59,
|
||||
"corr": 0.4411405309954431,
|
||||
"agreement": 0.7439024390243902,
|
||||
"a_positive": 51,
|
||||
"a_negative": 31,
|
||||
"b_positive": 66,
|
||||
"b_negative": 16,
|
||||
"eligible": true
|
||||
}
|
||||
],
|
||||
"grouped_raw_pairs": [
|
||||
{
|
||||
"a": "qrs",
|
||||
"b": "fund",
|
||||
"n": 88,
|
||||
"dates": 21,
|
||||
"stocks": 55,
|
||||
"corr": 0.046549465744305435,
|
||||
"agreement": 0.5454545454545454,
|
||||
"a_positive": 37,
|
||||
"a_negative": 51,
|
||||
"b_positive": 31,
|
||||
"b_negative": 57,
|
||||
"eligible": true
|
||||
},
|
||||
{
|
||||
"a": "qrs",
|
||||
"b": "mtf",
|
||||
"n": 135,
|
||||
"dates": 23,
|
||||
"stocks": 67,
|
||||
"corr": -0.03688254754085145,
|
||||
"agreement": 0.48148148148148145,
|
||||
"a_positive": 66,
|
||||
"a_negative": 69,
|
||||
"b_positive": 68,
|
||||
"b_negative": 67,
|
||||
"eligible": true
|
||||
},
|
||||
{
|
||||
"a": "qrs",
|
||||
"b": "bionic",
|
||||
"n": 27,
|
||||
"dates": 14,
|
||||
"stocks": 17,
|
||||
"corr": -0.2357797812857538,
|
||||
"agreement": 0.37037037037037035,
|
||||
"a_positive": 12,
|
||||
"a_negative": 15,
|
||||
"b_positive": 19,
|
||||
"b_negative": 8,
|
||||
"eligible": false
|
||||
},
|
||||
{
|
||||
"a": "fund",
|
||||
"b": "mtf",
|
||||
"n": 273,
|
||||
"dates": 24,
|
||||
"stocks": 130,
|
||||
"corr": 0.18367126921523455,
|
||||
"agreement": 0.6007326007326007,
|
||||
"a_positive": 78,
|
||||
"a_negative": 195,
|
||||
"b_positive": 125,
|
||||
"b_negative": 148,
|
||||
"eligible": true
|
||||
},
|
||||
{
|
||||
"a": "fund",
|
||||
"b": "bionic",
|
||||
"n": 39,
|
||||
"dates": 20,
|
||||
"stocks": 28,
|
||||
"corr": 0.7385686213637483,
|
||||
"agreement": 0.8717948717948718,
|
||||
"a_positive": 22,
|
||||
"a_negative": 17,
|
||||
"b_positive": 23,
|
||||
"b_negative": 16,
|
||||
"eligible": false
|
||||
},
|
||||
{
|
||||
"a": "mtf",
|
||||
"b": "bionic",
|
||||
"n": 82,
|
||||
"dates": 23,
|
||||
"stocks": 59,
|
||||
"corr": 0.4411405309954431,
|
||||
"agreement": 0.7439024390243902,
|
||||
"a_positive": 51,
|
||||
"a_negative": 31,
|
||||
"b_positive": 66,
|
||||
"b_negative": 16,
|
||||
"eligible": true
|
||||
}
|
||||
],
|
||||
"reconstructed_pairs": [
|
||||
{
|
||||
"a": "qrs",
|
||||
"b": "fund",
|
||||
"n": 88,
|
||||
"dates": 21,
|
||||
"stocks": 55,
|
||||
"corr": -0.06153759201424826,
|
||||
"agreement": 0.5227272727272727,
|
||||
"a_positive": 29,
|
||||
"a_negative": 59,
|
||||
"b_positive": 31,
|
||||
"b_negative": 57,
|
||||
"eligible": true,
|
||||
"day_ci": [
|
||||
-0.3217599860699957,
|
||||
-0.06536476507361046,
|
||||
0.1298440332987063
|
||||
],
|
||||
"stock_ci": [
|
||||
-0.29064646227097013,
|
||||
-0.07666601382777771,
|
||||
0.16400050425142315
|
||||
]
|
||||
},
|
||||
{
|
||||
"a": "qrs",
|
||||
"b": "mtf",
|
||||
"n": 135,
|
||||
"dates": 23,
|
||||
"stocks": 67,
|
||||
"corr": -0.03896650800208502,
|
||||
"agreement": 0.48148148148148145,
|
||||
"a_positive": 32,
|
||||
"a_negative": 103,
|
||||
"b_positive": 68,
|
||||
"b_negative": 67,
|
||||
"eligible": true,
|
||||
"day_ci": [
|
||||
-0.1895798040429639,
|
||||
-0.046146819853865526,
|
||||
0.11133587163181648
|
||||
],
|
||||
"stock_ci": [
|
||||
-0.16366816098624398,
|
||||
-0.04242410499900641,
|
||||
0.09339577839844783
|
||||
]
|
||||
},
|
||||
{
|
||||
"a": "qrs",
|
||||
"b": "bionic",
|
||||
"n": 27,
|
||||
"dates": 14,
|
||||
"stocks": 17,
|
||||
"corr": 0.2078742949189203,
|
||||
"agreement": 0.5555555555555556,
|
||||
"a_positive": 11,
|
||||
"a_negative": 16,
|
||||
"b_positive": 19,
|
||||
"b_negative": 8,
|
||||
"eligible": false,
|
||||
"day_ci": [
|
||||
-0.3119414241066813,
|
||||
0.20412414523193154,
|
||||
0.5444634480036133
|
||||
],
|
||||
"stock_ci": [
|
||||
-0.03439982023509114,
|
||||
0.2297526152623865,
|
||||
0.5093108126336955
|
||||
]
|
||||
},
|
||||
{
|
||||
"a": "fund",
|
||||
"b": "mtf",
|
||||
"n": 273,
|
||||
"dates": 24,
|
||||
"stocks": 130,
|
||||
"corr": 0.18367126921523455,
|
||||
"agreement": 0.6007326007326007,
|
||||
"a_positive": 78,
|
||||
"a_negative": 195,
|
||||
"b_positive": 125,
|
||||
"b_negative": 148,
|
||||
"eligible": true,
|
||||
"day_ci": [
|
||||
0.03684952678543051,
|
||||
0.18565327904562823,
|
||||
0.32337939861428566
|
||||
],
|
||||
"stock_ci": [
|
||||
0.07440556454087294,
|
||||
0.18434597467795827,
|
||||
0.30328064572466534
|
||||
]
|
||||
},
|
||||
{
|
||||
"a": "fund",
|
||||
"b": "bionic",
|
||||
"n": 39,
|
||||
"dates": 20,
|
||||
"stocks": 28,
|
||||
"corr": 0.7385686213637483,
|
||||
"agreement": 0.8717948717948718,
|
||||
"a_positive": 22,
|
||||
"a_negative": 17,
|
||||
"b_positive": 23,
|
||||
"b_negative": 16,
|
||||
"eligible": false,
|
||||
"day_ci": [
|
||||
0.3992071197831581,
|
||||
0.7458265812824105,
|
||||
1.0
|
||||
],
|
||||
"stock_ci": [
|
||||
0.4342360156590327,
|
||||
0.7462896594194197,
|
||||
0.9483522547960089
|
||||
]
|
||||
},
|
||||
{
|
||||
"a": "mtf",
|
||||
"b": "bionic",
|
||||
"n": 82,
|
||||
"dates": 23,
|
||||
"stocks": 59,
|
||||
"corr": 0.4411405309954431,
|
||||
"agreement": 0.7439024390243902,
|
||||
"a_positive": 51,
|
||||
"a_negative": 31,
|
||||
"b_positive": 66,
|
||||
"b_negative": 16,
|
||||
"eligible": true,
|
||||
"day_ci": [
|
||||
0.23808187663035318,
|
||||
0.4480530170237973,
|
||||
0.6537413649301814
|
||||
],
|
||||
"stock_ci": [
|
||||
0.1727169648407289,
|
||||
0.4454950566114418,
|
||||
0.6315584217263348
|
||||
]
|
||||
}
|
||||
],
|
||||
"raw_weights": {
|
||||
"weights": {
|
||||
"qrs": 0.3612276887094352,
|
||||
"fund": 0.31812635806831746,
|
||||
"mtf": 0.32064595322224726
|
||||
},
|
||||
"pairs": [
|
||||
{
|
||||
"a": "qrs",
|
||||
"b": "fund",
|
||||
"n": 88,
|
||||
"corr": 0.046549465744305435
|
||||
},
|
||||
{
|
||||
"a": "qrs",
|
||||
"b": "mtf",
|
||||
"n": 135,
|
||||
"corr": -0.03688254754085145
|
||||
},
|
||||
{
|
||||
"a": "fund",
|
||||
"b": "mtf",
|
||||
"n": 273,
|
||||
"corr": 0.18367126921523455
|
||||
}
|
||||
]
|
||||
},
|
||||
"clean_weights_for_diagnostics_only": null,
|
||||
"reconstructed_weights": {
|
||||
"weights": {
|
||||
"qrs": 0.35920866862097045,
|
||||
"fund": 0.3174653063330021,
|
||||
"mtf": 0.3233260250460275
|
||||
},
|
||||
"pairs": [
|
||||
{
|
||||
"a": "qrs",
|
||||
"b": "fund",
|
||||
"n": 88,
|
||||
"corr": -0.06153759201424826
|
||||
},
|
||||
{
|
||||
"a": "qrs",
|
||||
"b": "mtf",
|
||||
"n": 135,
|
||||
"corr": -0.03896650800208502
|
||||
},
|
||||
{
|
||||
"a": "fund",
|
||||
"b": "mtf",
|
||||
"n": 273,
|
||||
"corr": 0.18367126921523455
|
||||
}
|
||||
]
|
||||
},
|
||||
"day_bootstrap": {
|
||||
"qrs": [
|
||||
0.31888104076663437,
|
||||
0.35055604958364983,
|
||||
0.3787709812160866
|
||||
],
|
||||
"fund": [
|
||||
0.297511918321756,
|
||||
0.31875563252606065,
|
||||
0.34192119962396844
|
||||
],
|
||||
"mtf": [
|
||||
0.30649578874297584,
|
||||
0.32843807756937865,
|
||||
0.3664928196780627
|
||||
]
|
||||
},
|
||||
"stock_bootstrap": {
|
||||
"qrs": [
|
||||
0.3247551422792676,
|
||||
0.35176397202561505,
|
||||
0.3760089641342312
|
||||
],
|
||||
"fund": [
|
||||
0.29863008688752285,
|
||||
0.31765949234463764,
|
||||
0.3447372730399579
|
||||
],
|
||||
"mtf": [
|
||||
0.3061582652618046,
|
||||
0.32736012285104454,
|
||||
0.36003805707262904
|
||||
]
|
||||
},
|
||||
"sensitivity": {
|
||||
"first": {
|
||||
"weights": {
|
||||
"qrs": 0.38760449566288574,
|
||||
"fund": 0.30717399714356525,
|
||||
"mtf": 0.305221507193549
|
||||
},
|
||||
"pairs": [
|
||||
{
|
||||
"a": "qrs",
|
||||
"b": "fund",
|
||||
"n": 88,
|
||||
"corr": -0.028638223261250706
|
||||
},
|
||||
{
|
||||
"a": "qrs",
|
||||
"b": "mtf",
|
||||
"n": 135,
|
||||
"corr": -0.03724194613619291
|
||||
},
|
||||
{
|
||||
"a": "fund",
|
||||
"b": "mtf",
|
||||
"n": 273,
|
||||
"corr": 0.31633221479075757
|
||||
}
|
||||
]
|
||||
},
|
||||
"mean": {
|
||||
"weights": {
|
||||
"qrs": 0.3714613420062064,
|
||||
"fund": 0.31671372301602757,
|
||||
"mtf": 0.31182493497776603
|
||||
},
|
||||
"pairs": [
|
||||
{
|
||||
"a": "qrs",
|
||||
"b": "fund",
|
||||
"n": 88,
|
||||
"corr": -0.054648287669321144
|
||||
},
|
||||
{
|
||||
"a": "qrs",
|
||||
"b": "mtf",
|
||||
"n": 135,
|
||||
"corr": -0.07542826032972809
|
||||
},
|
||||
{
|
||||
"a": "fund",
|
||||
"b": "mtf",
|
||||
"n": 273,
|
||||
"corr": 0.2707750223532278
|
||||
}
|
||||
]
|
||||
},
|
||||
"first_half": {
|
||||
"weights": {
|
||||
"qrs": 0.3645742557903634,
|
||||
"fund": 0.329419068256303,
|
||||
"mtf": 0.30600667595333364
|
||||
},
|
||||
"pairs": [
|
||||
{
|
||||
"a": "qrs",
|
||||
"b": "fund",
|
||||
"n": 73,
|
||||
"corr": -0.027093815217750113
|
||||
},
|
||||
{
|
||||
"a": "qrs",
|
||||
"b": "mtf",
|
||||
"n": 105,
|
||||
"corr": -0.12461398850182136
|
||||
},
|
||||
{
|
||||
"a": "fund",
|
||||
"b": "mtf",
|
||||
"n": 165,
|
||||
"corr": 0.24752279289653376
|
||||
}
|
||||
]
|
||||
},
|
||||
"second_half": {
|
||||
"weights": {
|
||||
"qrs": 0.3258370606278521,
|
||||
"fund": 0.30850376769754656,
|
||||
"mtf": 0.3656591716746013
|
||||
},
|
||||
"pairs": [
|
||||
{
|
||||
"a": "qrs",
|
||||
"b": "fund",
|
||||
"n": 15,
|
||||
"corr": -0.21821789023599236
|
||||
},
|
||||
{
|
||||
"a": "qrs",
|
||||
"b": "mtf",
|
||||
"n": 30,
|
||||
"corr": -0.014678923792502543
|
||||
},
|
||||
{
|
||||
"a": "fund",
|
||||
"b": "mtf",
|
||||
"n": 108,
|
||||
"corr": 0.0839492662258804
|
||||
}
|
||||
]
|
||||
},
|
||||
"fund_distribution_only": {
|
||||
"weights": {
|
||||
"qrs": 0.33439802248125616,
|
||||
"fund": 0.3117877231309714,
|
||||
"mtf": 0.3538142543877724
|
||||
},
|
||||
"pairs": [
|
||||
{
|
||||
"a": "qrs",
|
||||
"b": "fund",
|
||||
"n": 53,
|
||||
"corr": -0.19633836312465044
|
||||
},
|
||||
{
|
||||
"a": "qrs",
|
||||
"b": "mtf",
|
||||
"n": 135,
|
||||
"corr": -0.03896650800208502
|
||||
},
|
||||
{
|
||||
"a": "fund",
|
||||
"b": "mtf",
|
||||
"n": 205,
|
||||
"corr": 0.1285486526887367
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"triple_complete_rows": 44,
|
||||
"data_sha256": "63652a23edbb44f4cc72791232ebb838812aa34538729c5d11dbf38bbc1eb82d"
|
||||
}
|
||||
|
|
@ -0,0 +1,102 @@
|
|||
"""v2分组、日期净化与原公式重算;仅离线运行。"""
|
||||
from __future__ import annotations
|
||||
import datetime as dt
|
||||
import gzip
|
||||
import hashlib
|
||||
import itertools
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import analyze as a
|
||||
|
||||
SOURCE_ROOT=Path('/Users/baobao/Documents/work/project/intraday_timing/src')
|
||||
sys.path.insert(0,str(SOURCE_ROOT))
|
||||
from intraday_timing.state.daily_qrs_computer import DailyQRSComputer
|
||||
|
||||
GROUPS=['qrs','fund','mtf']
|
||||
|
||||
def dependence_weights(panel):
|
||||
rs=[];R=np.eye(len(GROUPS))
|
||||
for i,j in itertools.combinations(range(len(GROUPS)),2):
|
||||
q=panel[[GROUPS[i],GROUPS[j]]].dropna()
|
||||
r=a.corr(q.iloc[:,0],q.iloc[:,1])
|
||||
if r is None:return None
|
||||
R[i,j]=R[j,i]=abs(r)
|
||||
rs.append({'a':GROUPS[i],'b':GROUPS[j],'n':len(q),'corr':r})
|
||||
u=1/R.sum(axis=1);weights=u/u.sum()
|
||||
return {'weights':dict(zip(GROUPS,map(float,weights))), 'pairs':rs}
|
||||
|
||||
def grouped_panel(f,how='last'):
|
||||
z=f.copy();z['source']=z.source.replace({'intensity':'fund','distribution':'fund'})
|
||||
return a.panels(z,how)
|
||||
|
||||
def bootstrap_weights(panel,key='day',n=1000):
|
||||
q=panel.reset_index();rng=np.random.default_rng(20261111);blocks=[v.drop(columns=key) for _,v in q.groupby(key)]
|
||||
estimates=[]
|
||||
for _ in range(n):
|
||||
sample=pd.concat([blocks[i] for i in rng.integers(len(blocks),size=len(blocks))],ignore_index=True)
|
||||
r=dependence_weights(sample)
|
||||
if r:estimates.append([r['weights'][g] for g in GROUPS])
|
||||
return {g:np.quantile(np.array(estimates)[:,i],[.025,.5,.975]).tolist() for i,g in enumerate(GROUPS)}
|
||||
|
||||
def main():
|
||||
f=pd.read_csv(a.DATA/'normalized.csv.gz');f['t']=pd.to_datetime(f.t)
|
||||
bars=pd.DataFrame([json.loads(x) for x in gzip.open(a.DATA/'daily.jsonl.gz','rt')])
|
||||
bars['trade_date']=pd.to_datetime(bars.timestamp).dt.date
|
||||
bars['code']=bars.symbol.map(a.code)
|
||||
assert not bars.duplicated(['code','trade_date']).any()
|
||||
for col in ['high','low','close']:bars[col]=pd.to_numeric(bars[col])
|
||||
cal=sorted(bars[bars.symbol=='SH600519'].trade_date.unique())
|
||||
prev={d:max(x for x in cal if x<dt.date.fromisoformat(d)) for d in f.day.unique()}
|
||||
q=f[f.source=='qrs'].copy()
|
||||
q['fresh']=q.apply(lambda r:r.evidence_date==prev[r.day].isoformat(),axis=1)
|
||||
q['age']=(pd.to_datetime(q.day)-pd.to_datetime(q.evidence_date)).dt.days
|
||||
calc=DailyQRSComputer(None,n_regression=18,m_lookback=60,threshold=.7)
|
||||
bycode={c:g for c,g in bars.groupby('code')};recon=[];reproduced=[];errors=[]
|
||||
for row in q.itertuples():
|
||||
try:
|
||||
b=bycode[row.code];result=calc._compute_from_dataframe(row.code,b,prev[row.day])
|
||||
if result.trade_date!=prev[row.day]:raise ValueError('missing prior-day bar')
|
||||
recon.append({'day':row.day,'code':row.code,'value':result.qrs_value,'direction':float(np.sign(result.qrs_value)),
|
||||
'old_value':row.value,'old_direction':row.direction,'fresh':row.fresh,'evidence_date':prev[row.day].isoformat()})
|
||||
try:
|
||||
old=calc._compute_from_dataframe(row.code,b,dt.date.fromisoformat(row.evidence_date))
|
||||
reproduced.append(abs(old.qrs_value-row.value))
|
||||
except Exception as e:errors.append({'code':row.code,'day':row.day,'phase':'old_reproduction','error':type(e).__name__})
|
||||
except Exception as e:errors.append({'code':row.code,'day':row.day,'phase':'fresh_reconstruction','error':type(e).__name__})
|
||||
rr=pd.DataFrame(recon)
|
||||
clean=f[(f.source!='qrs') | f.index.isin(q[q.fresh].index)].copy()
|
||||
counter=f.copy().set_index(['day','code','source'])
|
||||
for row in rr.itertuples():
|
||||
key=(row.day,row.code,'qrs');counter.loc[key,'direction']=row.direction;counter.loc[key,'value']=row.value
|
||||
counter=counter.reset_index().sort_values(['t','source','code'])
|
||||
# 未成功重算的QRS不得保留旧值。
|
||||
success=set(zip(rr.day,rr.code))
|
||||
counter=counter[(counter.source!='qrs') | counter.apply(lambda r:(r.day,r.code) in success,axis=1)]
|
||||
a.SOURCES=['qrs','fund','mtf','bionic']
|
||||
raw_panel=grouped_panel(f);clean_panel=grouped_panel(clean);recon_panel=grouped_panel(counter)
|
||||
result={'qrs_quality':{'events':len(q),'fresh':int(q.fresh.sum()),'stale':int((~q.fresh).sum()),'age_median':float(q.age.median()),
|
||||
'same_evidence_repeated':int(q.duplicated(['code','evidence_date','direction']).sum())},
|
||||
'reconstruction':{'rows':len(rr),'changed_sign':int((rr.old_direction!=rr.direction).sum()),
|
||||
'old_value_reproduced':len(reproduced),'old_value_max_abs_error':max(reproduced) if reproduced else None,'errors':errors},
|
||||
'clean_pairs':a.pair_stats(clean_panel,False),'grouped_raw_pairs':a.pair_stats(raw_panel,False),
|
||||
'reconstructed_pairs':a.pair_stats(recon_panel),
|
||||
'raw_weights':dependence_weights(raw_panel),'clean_weights_for_diagnostics_only':dependence_weights(clean_panel),
|
||||
'reconstructed_weights':dependence_weights(recon_panel),
|
||||
'day_bootstrap':bootstrap_weights(recon_panel),'stock_bootstrap':bootstrap_weights(recon_panel,'code',500),
|
||||
'sensitivity':{},'triple_complete_rows':len(recon_panel[GROUPS].dropna()),
|
||||
'data_sha256':hashlib.sha256((a.DATA/'daily.jsonl.gz').read_bytes()).hexdigest()}
|
||||
for how in ['first','mean']:
|
||||
result['sensitivity'][how]=dependence_weights(grouped_panel(counter,how))
|
||||
days=sorted(counter.day.unique());mid=days[len(days)//2]
|
||||
for name,g in [('first_half',counter[counter.day<mid]),('second_half',counter[counter.day>=mid])]:
|
||||
result['sensitivity'][name]=dependence_weights(grouped_panel(g))
|
||||
# 资金组用更稳定全窗的资金分布作唯一代表,查看强度缺发是否改变结论。
|
||||
result['sensitivity']['fund_distribution_only']=dependence_weights(grouped_panel(counter[counter.source!='intensity']))
|
||||
a.OUT.joinpath('weights.json').write_text(json.dumps(result,ensure_ascii=False,indent=2))
|
||||
rr.to_csv(a.DATA/'qrs_reconstruction.csv.gz',index=False,compression='gzip')
|
||||
print(json.dumps(result,ensure_ascii=False,indent=2))
|
||||
|
||||
if __name__=='__main__':main()
|
||||
Loading…
Reference in New Issue