Data Hub
金融市场数据导入
管理行情、成交、事件、资讯和券商数据的导入任务。
导入流水线
从数据采集到质量门禁| 步骤 | 名称 | 职责 | 状态 | 已处理 | 质量 |
|---|---|---|---|---|---|
| 1 | Collect | Pull files, APIs, broker feeds, and manual event data | Ready | 380,221 | 99.2% |
| 2 | Validate | Check schema, required fields, time range, and duplicates | Ready | 379,880 | 98.7% |
| 3 | Normalize | Map symbols, calendars, currencies, and frequency | Ready | 379,640 | 98.4% |
| 4 | Persist | Write normalized market data into target stores | Planned | 0 | - |
| 5 | Quality Gate | Publish import scorecards and exception queues | Planned | 0 | - |
导入异常队列
数据质量问题分拣| ID | 来源 | 标的 | 问题 | 记录数 | 处理方案 | 状态 |
|---|---|---|---|---|---|---|
| #501 | US Daily API | MSFT | Missing close price | 2 | Backfill from secondary provider | Open |
| #502 | ETF CSV | 510300 | Duplicate timestamp | 18 | Merge duplicate bars by latest revision | Review |
| #503 | Crypto Tick Feed | BTCUSDT | Timestamp drift | 126 | Apply exchange clock offset | Open |
| #504 | Manual Event Calendar | GLOBAL | Unmapped event type | 4 | Add event taxonomy mapping | Resolved |
数据源连接
认证、限流、健康状态与负责人| 名称 | 来源 | 认证 | 限流 | 健康 | 负责人 | 检查时间 |
|---|---|---|---|---|---|---|
| US Market API | Api | API Key | 1200 calls/min | Healthy | DataOps | 06-10 19:46 |
| ETF CSV Drop | Csv | Managed folder | Batch every 15 min | Healthy | DataOperator | 06-10 19:38 |
| Crypto Broker Feed | Broker | OAuth client | Streaming | Watch | DataOps | 06-10 19:49 |
| Macro Event Console | Manual | Admin login | Manual approval | Healthy | Research | 06-10 17:50 |
导入任务
统一采集入口| 任务 | 来源 | 市场 | 频率 | 记录数 | 状态 | 负责人 | 最近运行 |
|---|---|---|---|---|---|---|---|
| 美股日线行情同步 | Api | US Stocks | Daily | 128,640 | 成功 | DataOps | 06-10 17:50 |
| A股ETF分钟线导入 | Csv | CN Funds | 1m | 86,400 | 成功 | Quant | 06-10 18:50 |
| 加密货币Tick订阅 | Broker | Crypto | Tick | 3,589,012 | 运行中 | DataOps | 06-10 19:45 |
| 宏观事件日历维护 | Manual | Global | Event | 342 | 待校验 | Research | 06-09 19:50 |