hejianhao
2026-04-02 0940d56748a73525790c350c393a5be960aab62c
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
<!DOCTYPE html><html><head>
      <title>system-architecture-summary</title>
      <meta charset="utf-8">
      <meta name="viewport" content="width=device-width, initial-scale=1.0">
      
      <link rel="stylesheet" href="file:////Users/work/.vscode/extensions/shd101wyy.markdown-preview-enhanced-0.8.20/crossnote/dependencies/katex/katex.min.css">
      
      
      
      
      
      <style>
      code[class*=language-],pre[class*=language-]{color:#333;background:0 0;font-family:Consolas,"Liberation Mono",Menlo,Courier,monospace;text-align:left;white-space:pre;word-spacing:normal;word-break:normal;word-wrap:normal;line-height:1.4;-moz-tab-size:8;-o-tab-size:8;tab-size:8;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none}pre[class*=language-]{padding:.8em;overflow:auto;border-radius:3px;background:#f5f5f5}:not(pre)>code[class*=language-]{padding:.1em;border-radius:.3em;white-space:normal;background:#f5f5f5}.token.blockquote,.token.comment{color:#969896}.token.cdata{color:#183691}.token.doctype,.token.macro.property,.token.punctuation,.token.variable{color:#333}.token.builtin,.token.important,.token.keyword,.token.operator,.token.rule{color:#a71d5d}.token.attr-value,.token.regex,.token.string,.token.url{color:#183691}.token.atrule,.token.boolean,.token.code,.token.command,.token.constant,.token.entity,.token.number,.token.property,.token.symbol{color:#0086b3}.token.prolog,.token.selector,.token.tag{color:#63a35c}.token.attr-name,.token.class,.token.class-name,.token.function,.token.id,.token.namespace,.token.pseudo-class,.token.pseudo-element,.token.url-reference .token.variable{color:#795da3}.token.entity{cursor:help}.token.title,.token.title .token.punctuation{font-weight:700;color:#1d3e81}.token.list{color:#ed6a43}.token.inserted{background-color:#eaffea;color:#55a532}.token.deleted{background-color:#ffecec;color:#bd2c00}.token.bold{font-weight:700}.token.italic{font-style:italic}.language-json .token.property{color:#183691}.language-markup .token.tag .token.punctuation{color:#333}.language-css .token.function,code.language-css{color:#0086b3}.language-yaml .token.atrule{color:#63a35c}code.language-yaml{color:#183691}.language-ruby .token.function{color:#333}.language-markdown .token.url{color:#795da3}.language-makefile .token.symbol{color:#795da3}.language-makefile .token.variable{color:#183691}.language-makefile .token.builtin{color:#0086b3}.language-bash .token.keyword{color:#0086b3}pre[data-line]{position:relative;padding:1em 0 1em 3em}pre[data-line] .line-highlight-wrapper{position:absolute;top:0;left:0;background-color:transparent;display:block;width:100%}pre[data-line] .line-highlight{position:absolute;left:0;right:0;padding:inherit 0;margin-top:1em;background:hsla(24,20%,50%,.08);background:linear-gradient(to right,hsla(24,20%,50%,.1) 70%,hsla(24,20%,50%,0));pointer-events:none;line-height:inherit;white-space:pre}pre[data-line] .line-highlight:before,pre[data-line] .line-highlight[data-end]:after{content:attr(data-start);position:absolute;top:.4em;left:.6em;min-width:1em;padding:0 .5em;background-color:hsla(24,20%,50%,.4);color:#f4f1ef;font:bold 65%/1.5 sans-serif;text-align:center;vertical-align:.3em;border-radius:999px;text-shadow:none;box-shadow:0 1px #fff}pre[data-line] .line-highlight[data-end]:after{content:attr(data-end);top:auto;bottom:.4em}html body{font-family:'Helvetica Neue',Helvetica,'Segoe UI',Arial,freesans,sans-serif;font-size:16px;line-height:1.6;color:#333;background-color:#fff;overflow:initial;box-sizing:border-box;word-wrap:break-word}html body>:first-child{margin-top:0}html body h1,html body h2,html body h3,html body h4,html body h5,html body h6{line-height:1.2;margin-top:1em;margin-bottom:16px;color:#000}html body h1{font-size:2.25em;font-weight:300;padding-bottom:.3em}html body h2{font-size:1.75em;font-weight:400;padding-bottom:.3em}html body h3{font-size:1.5em;font-weight:500}html body h4{font-size:1.25em;font-weight:600}html body h5{font-size:1.1em;font-weight:600}html body h6{font-size:1em;font-weight:600}html body h1,html body h2,html body h3,html body h4,html body h5{font-weight:600}html body h5{font-size:1em}html body h6{color:#5c5c5c}html body strong{color:#000}html body del{color:#5c5c5c}html body a:not([href]){color:inherit;text-decoration:none}html body a{color:#08c;text-decoration:none}html body a:hover{color:#00a3f5;text-decoration:none}html body img{max-width:100%}html body>p{margin-top:0;margin-bottom:16px;word-wrap:break-word}html body>ol,html body>ul{margin-bottom:16px}html body ol,html body ul{padding-left:2em}html body ol.no-list,html body ul.no-list{padding:0;list-style-type:none}html body ol ol,html body ol ul,html body ul ol,html body ul ul{margin-top:0;margin-bottom:0}html body li{margin-bottom:0}html body li.task-list-item{list-style:none}html body li>p{margin-top:0;margin-bottom:0}html body .task-list-item-checkbox{margin:0 .2em .25em -1.8em;vertical-align:middle}html body .task-list-item-checkbox:hover{cursor:pointer}html body blockquote{margin:16px 0;font-size:inherit;padding:0 15px;color:#5c5c5c;background-color:#f0f0f0;border-left:4px solid #d6d6d6}html body blockquote>:first-child{margin-top:0}html body blockquote>:last-child{margin-bottom:0}html body hr{height:4px;margin:32px 0;background-color:#d6d6d6;border:0 none}html body table{margin:10px 0 15px 0;border-collapse:collapse;border-spacing:0;display:block;width:100%;overflow:auto;word-break:normal;word-break:keep-all}html body table th{font-weight:700;color:#000}html body table td,html body table th{border:1px solid #d6d6d6;padding:6px 13px}html body dl{padding:0}html body dl dt{padding:0;margin-top:16px;font-size:1em;font-style:italic;font-weight:700}html body dl dd{padding:0 16px;margin-bottom:16px}html body code{font-family:Menlo,Monaco,Consolas,'Courier New',monospace;font-size:.85em;color:#000;background-color:#f0f0f0;border-radius:3px;padding:.2em 0}html body code::after,html body code::before{letter-spacing:-.2em;content:'\00a0'}html body pre>code{padding:0;margin:0;word-break:normal;white-space:pre;background:0 0;border:0}html body .highlight{margin-bottom:16px}html body .highlight pre,html body pre{padding:1em;overflow:auto;line-height:1.45;border:#d6d6d6;border-radius:3px}html body .highlight pre{margin-bottom:0;word-break:normal}html body pre code,html body pre tt{display:inline;max-width:initial;padding:0;margin:0;overflow:initial;line-height:inherit;word-wrap:normal;background-color:transparent;border:0}html body pre code:after,html body pre code:before,html body pre tt:after,html body pre tt:before{content:normal}html body blockquote,html body dl,html body ol,html body p,html body pre,html body ul{margin-top:0;margin-bottom:16px}html body kbd{color:#000;border:1px solid #d6d6d6;border-bottom:2px solid #c7c7c7;padding:2px 4px;background-color:#f0f0f0;border-radius:3px}@media print{html body{background-color:#fff}html body h1,html body h2,html body h3,html body h4,html body h5,html body h6{color:#000;page-break-after:avoid}html body blockquote{color:#5c5c5c}html body pre{page-break-inside:avoid}html body table{display:table}html body img{display:block;max-width:100%;max-height:100%}html body code,html body pre{word-wrap:break-word;white-space:pre}}.markdown-preview{width:100%;height:100%;box-sizing:border-box}.markdown-preview ul{list-style:disc}.markdown-preview ul ul{list-style:circle}.markdown-preview ul ul ul{list-style:square}.markdown-preview ol{list-style:decimal}.markdown-preview ol ol,.markdown-preview ul ol{list-style-type:lower-roman}.markdown-preview ol ol ol,.markdown-preview ol ul ol,.markdown-preview ul ol ol,.markdown-preview ul ul ol{list-style-type:lower-alpha}.markdown-preview .newpage,.markdown-preview .pagebreak{page-break-before:always}.markdown-preview pre.line-numbers{position:relative;padding-left:3.8em;counter-reset:linenumber}.markdown-preview pre.line-numbers>code{position:relative}.markdown-preview pre.line-numbers .line-numbers-rows{position:absolute;pointer-events:none;top:1em;font-size:100%;left:0;width:3em;letter-spacing:-1px;border-right:1px solid #999;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}.markdown-preview pre.line-numbers .line-numbers-rows>span{pointer-events:none;display:block;counter-increment:linenumber}.markdown-preview pre.line-numbers .line-numbers-rows>span:before{content:counter(linenumber);color:#999;display:block;padding-right:.8em;text-align:right}.markdown-preview .mathjax-exps .MathJax_Display{text-align:center!important}.markdown-preview:not([data-for=preview]) .code-chunk .code-chunk-btn-group{display:none}.markdown-preview:not([data-for=preview]) .code-chunk .status{display:none}.markdown-preview:not([data-for=preview]) .code-chunk .output-div{margin-bottom:16px}.markdown-preview .md-toc{padding:0}.markdown-preview .md-toc .md-toc-link-wrapper .md-toc-link{display:inline;padding:.25rem 0}.markdown-preview .md-toc .md-toc-link-wrapper .md-toc-link div,.markdown-preview .md-toc .md-toc-link-wrapper .md-toc-link p{display:inline}.markdown-preview .md-toc .md-toc-link-wrapper.highlighted .md-toc-link{font-weight:800}.scrollbar-style::-webkit-scrollbar{width:8px}.scrollbar-style::-webkit-scrollbar-track{border-radius:10px;background-color:transparent}.scrollbar-style::-webkit-scrollbar-thumb{border-radius:5px;background-color:rgba(150,150,150,.66);border:4px solid rgba(150,150,150,.66);background-clip:content-box}html body[for=html-export]:not([data-presentation-mode]){position:relative;width:100%;height:100%;top:0;left:0;margin:0;padding:0;overflow:auto}html body[for=html-export]:not([data-presentation-mode]) .markdown-preview{position:relative;top:0;min-height:100vh}@media screen and (min-width:914px){html body[for=html-export]:not([data-presentation-mode]) .markdown-preview{padding:2em calc(50% - 457px + 2em)}}@media screen and (max-width:914px){html body[for=html-export]:not([data-presentation-mode]) .markdown-preview{padding:2em}}@media screen and (max-width:450px){html body[for=html-export]:not([data-presentation-mode]) .markdown-preview{font-size:14px!important;padding:1em}}@media print{html body[for=html-export]:not([data-presentation-mode]) #sidebar-toc-btn{display:none}}html body[for=html-export]:not([data-presentation-mode]) #sidebar-toc-btn{position:fixed;bottom:8px;left:8px;font-size:28px;cursor:pointer;color:inherit;z-index:99;width:32px;text-align:center;opacity:.4}html body[for=html-export]:not([data-presentation-mode])[html-show-sidebar-toc] #sidebar-toc-btn{opacity:1}html body[for=html-export]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc{position:fixed;top:0;left:0;width:300px;height:100%;padding:32px 0 48px 0;font-size:14px;box-shadow:0 0 4px rgba(150,150,150,.33);box-sizing:border-box;overflow:auto;background-color:inherit}html body[for=html-export]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc::-webkit-scrollbar{width:8px}html body[for=html-export]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc::-webkit-scrollbar-track{border-radius:10px;background-color:transparent}html body[for=html-export]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc::-webkit-scrollbar-thumb{border-radius:5px;background-color:rgba(150,150,150,.66);border:4px solid rgba(150,150,150,.66);background-clip:content-box}html body[for=html-export]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc a{text-decoration:none}html body[for=html-export]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc .md-toc{padding:0 16px}html body[for=html-export]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc .md-toc .md-toc-link-wrapper .md-toc-link{display:inline;padding:.25rem 0}html body[for=html-export]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc .md-toc .md-toc-link-wrapper .md-toc-link div,html body[for=html-export]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc .md-toc .md-toc-link-wrapper .md-toc-link p{display:inline}html body[for=html-export]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc .md-toc .md-toc-link-wrapper.highlighted .md-toc-link{font-weight:800}html body[for=html-export]:not([data-presentation-mode])[html-show-sidebar-toc] .markdown-preview{left:300px;width:calc(100% - 300px);padding:2em calc(50% - 457px - 300px / 2);margin:0;box-sizing:border-box}@media screen and (max-width:1274px){html body[for=html-export]:not([data-presentation-mode])[html-show-sidebar-toc] .markdown-preview{padding:2em}}@media screen and (max-width:450px){html body[for=html-export]:not([data-presentation-mode])[html-show-sidebar-toc] .markdown-preview{width:100%}}html body[for=html-export]:not([data-presentation-mode]):not([html-show-sidebar-toc]) .markdown-preview{left:50%;transform:translateX(-50%)}html body[for=html-export]:not([data-presentation-mode]):not([html-show-sidebar-toc]) .md-sidebar-toc{display:none}
/* Please visit the URL below for more information: */
/*   https://shd101wyy.github.io/markdown-preview-enhanced/#/customize-css */
 
      </style>
      <!-- The content below will be included at the end of the <head> element. --><script type="text/javascript">
  document.addEventListener("DOMContentLoaded", function () {
    // your code here
  });
</script></head><body for="html-export">
    
    
      <div class="crossnote markdown-preview  ">
      
<h1 id="深度学习平台迭代方案概述">深度学习平台迭代方案概述 </h1>
<h2 id="1-背景与目标">1. 背景与目标 </h2>
<ul>
<li>前端:Electron + Vue3(Pinia、Vue Router、Vite),目前在 Windows 侧完成项目管理、标注、训练触发,数据多保存在本地 JSON/文件夹。</li>
<li>后端:<code>deep-learning-platform-backend</code> 以 FastAPI 单体运行,SQLite 持久化;训练直接在服务进程中以线程池 + Python 进程执行。</li>
<li>目标:<strong>Windows 端只负责操作与展示,训练/推理/测试在服务器执行;新增 C++ 推理接口;将 JSON 数据迁移至后端数据库;训练/推理结果按阶段顺序输出并可终止。</strong></li>
</ul>
<h2 id="2-现有体系梳理">2. 现有体系梳理 </h2>
<table>
<thead>
<tr>
<th>区域</th>
<th>现状</th>
<th>关键文件/模块</th>
</tr>
</thead>
<tbody>
<tr>
<td>前端</td>
<td>Electron shell + Vue;通过 <code>window.electronAPI</code> 访问本地磁盘;API 调 FastAPI REST;标注区域如 <code>ImageArea.vue</code> 直接读本地图片</td>
<td><code>src/renderer</code> 目录、<code>api/index.js</code>、<code>store/trainingAndEvaluateData.js</code></td>
</tr>
<tr>
<td>后端</td>
<td>FastAPI 应用(<code>app.py</code>)挂载 models/training/labeling/dataset/evaluator 路由;训练接口 <code>src/api/training.py</code> 用线程池调用 <code>src/core/trainer.py</code>;状态写 SQLite</td>
<td><code>src/core</code>, <code>src/api</code>, <code>src/database</code></td>
</tr>
<tr>
<td>数据</td>
<td>项目、方法、模型配置多为 JSON;SQLite 中只有 <code>model_info</code>、<code>model_record</code>、<code>task_records</code></td>
<td><code>config/*.json</code>, <code>src/database/app_database.db</code></td>
</tr>
<tr>
<td>任务控制</td>
<td><code>task_manager.py</code> 为进程内单例,仅能管理当前进程任务;无法穿透多台服务器</td>
<td><code>src/core/task_manager.py</code></td>
</tr>
</tbody>
</table>
<h2 id="3-核心痛点">3. 核心痛点 </h2>
<ol>
<li><strong>计算位置</strong>:训练/推理与前端同机,难以托管服务器 GPU;线程池方式不可扩展。</li>
<li><strong>C++ 推理缺失</strong>:只能用 Python 模型,无法嵌入高性能推理接口。</li>
<li><strong>数据孤岛</strong>:JSON + 本地文件导致 Windows/服务器数据不一致,难以协同。</li>
<li><strong>任务状态与结果</strong>:目前结果一次性写 <code>model_info</code>,前端轮询 <code>/models/latest</code>;不可分阶段推送,也无法可靠终止。</li>
<li><strong>可运维性</strong>:单体 FastAPI + SQLite 缺乏分层与监控,日志/模型/数据与任务耦合。</li>
</ol>
<h2 id="4-目标能力拆解">4. 目标能力拆解 </h2>
<ol>
<li>Windows 前端仅负责交互,所有计算任务调度到服务器。</li>
<li>C++ 推理接口纳入统一调度,兼容 Python 训练。</li>
<li>统一数据库存数据(落地 Python 技术栈,与现有后端一致)。</li>
<li>训练/推理流程按阶段输出结果(顺序展示),支持随时暂停/恢复/终止。</li>
<li>支持扩容(多 GPU/多节点)、任务排队、资源隔离。</li>
</ol>
<h2 id="5-推荐总体架构">5. 推荐总体架构 </h2>
<pre data-role="codeBlock" data-info="" class="language-text"><code>Windows UI (Electron + Vue) ──REST/WebSocket──&gt; API Gateway (FastAPI)
                               │
                               ├─任务编排 (异步队列/调度器: Celery/Ray)
                               │
                               ├─Python 训练 Worker (GPU)
                               ├─C++ 推理服务 (gRPC/pybind11)
                               │
                               └─数据服务 (PostgreSQL/MySQL + 对象存储)
</code></pre><p>要点:</p>
<ul>
<li><strong>API Gateway</strong>:负责认证、项目信息、任务提交、状态查询、消息推送。</li>
<li><strong>任务编排</strong>:将 <code>task_manager</code> 升级为分布式任务状态机(Redis/DB 驱动),支持阶段管理。</li>
<li><strong>计算平面</strong>:Python 训练 Worker 运行 <code>Trainer</code>;C++ 推理可以嵌入或独立服务。</li>
<li><strong>数据平面</strong>:结构化数据进关系型数据库;大文件(数据集、模型、日志)进对象存储/NAS。</li>
<li><strong>事件流</strong>:WebSocket/SSE 将阶段事件回推前端,实现顺序展示与终止控制。</li>
</ul>
<h2 id="6-关键模块设计">6. 关键模块设计 </h2>
<h3 id="61-控制平面api-gateway">6.1 控制平面(API Gateway) </h3>
<ul>
<li>FastAPI 继续作为入口,但拆分出 <code>gateway</code> 层,专注 REST/WebSocket。</li>
<li>新增 <code>/auth</code>, <code>/projects</code>, <code>/tasks</code>, <code>/events</code> API,所有前端行为走远程接口。</li>
<li>WebSocket 渠道向前端推送任务阶段、日志、指标,减少轮询。</li>
</ul>
<h3 id="62-任务编排层">6.2 任务编排层 </h3>
<ul>
<li>引入 Celery/RQ/Ray/Airflow 之一:前端 <code>start training</code> → Gateway → 队列 → Worker。</li>
<li><code>task_manager</code> 改为 Redis/DB 持久化状态,包含 <code>PENDING/READY/RUNNING/PAUSED/STOPPED/FAILED/SUCCEEDED</code> 以及阶段信息。</li>
<li>统一处理暂停/恢复/终止指令:命令写入状态表或 Redis channel,Worker 轮询/订阅后执行。</li>
</ul>
<h3 id="63-计算平面">6.3 计算平面 </h3>
<ol>
<li><strong>Python 训练服务</strong>
<ul>
<li>复用 <code>src/core/trainer.py</code>,封装为 Worker;支持多 GPU(DDP),按任务配置加载数据。</li>
<li>将指标写入 <code>task_stage_events</code>,在每个 epoch/step 检查暂停/停止标志。</li>
</ul>
</li>
<li><strong>C++ 推理服务</strong>
<ul>
<li>定义统一接口:<code>loadModel</code>, <code>warmup</code>, <code>infer</code>, <code>cleanup</code>。</li>
<li>方案 A:pybind11 嵌入 Python Worker,复用相同任务调度。</li>
<li>方案 B:独立 C++ gRPC 服务,Gateway/Worker 通过 RPC 调用。</li>
<li>输出结构化结果(scores、bboxes、heatmaps)并存入数据库/对象存储。</li>
</ul>
</li>
</ol>
<h3 id="64-数据平面">6.4 数据平面 </h3>
<ul>
<li>关系型数据库:模型、任务、项目、标注、拆分、配置、用户等实体统一管理。</li>
<li>对象存储:训练日志、模型权重、推理输出、可视化图像。</li>
<li>提供导入导出工具,将旧 JSON 项目迁移到新库。</li>
</ul>
<h3 id="65-事件与结果顺序">6.5 事件与结果顺序 </h3>
<ul>
<li>设计 <code>task_stage</code>(stage_id, name, order, status, started_at, finished_at)与 <code>task_stage_events</code>(日志、指标、文件引用)。</li>
<li>Worker 在阶段进入/退出时写记录,并通过消息队列推送。</li>
<li>前端按阶段顺序渲染时间线,可显示进行中/成功/失败,每条阶段可单独终止。</li>
</ul>
<h2 id="7-数据库迁移与实现方案python">7. 数据库迁移与实现方案(Python) </h2>
<h3 id="71-技术栈选择">7.1 技术栈选择 </h3>
<ul>
<li><strong>数据库</strong>:PostgreSQL(推荐,支持 JSONB / 并发)或 MySQL 8。</li>
<li><strong>ORM &amp; 迁移</strong>:SQLAlchemy 2.x + Alembic,统一管理 schema。</li>
<li><strong>连接池</strong>:SQLAlchemy AsyncEngine + asyncpg(或 psycopg3),配合 PgBouncer 做长连复用。</li>
<li><strong>部署</strong>:与 FastAPI Gateway 同语言,可直接在现有后端进程中启动 Session 工厂,避免跨语言 RPC。</li>
</ul>
<h3 id="72-数据模型">7.2 数据模型 </h3>
<p>核心实体:Project、Dataset、Split、Annotation、Model、ModelVersion、Task、TaskStage、TaskEvent、User、AccessToken。<br>
示例(伪代码):</p>
<pre data-role="codeBlock" data-info="python" class="language-python python"><code><span class="token keyword keyword-class">class</span> <span class="token class-name">Task</span><span class="token punctuation">(</span>Base<span class="token punctuation">)</span><span class="token punctuation">:</span>
    __tablename__ <span class="token operator">=</span> <span class="token string">"tasks"</span>
    <span class="token builtin">id</span> <span class="token operator">=</span> Column<span class="token punctuation">(</span>UUID<span class="token punctuation">,</span> primary_key<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">)</span>
    project_id <span class="token operator">=</span> Column<span class="token punctuation">(</span>UUID<span class="token punctuation">,</span> ForeignKey<span class="token punctuation">(</span><span class="token string">"projects.id"</span><span class="token punctuation">)</span><span class="token punctuation">)</span>
    <span class="token builtin">type</span> <span class="token operator">=</span> Column<span class="token punctuation">(</span>Enum<span class="token punctuation">(</span>TaskType<span class="token punctuation">)</span><span class="token punctuation">)</span>
    status <span class="token operator">=</span> Column<span class="token punctuation">(</span>Enum<span class="token punctuation">(</span>TaskStatus<span class="token punctuation">)</span><span class="token punctuation">)</span>
    priority <span class="token operator">=</span> Column<span class="token punctuation">(</span>Integer<span class="token punctuation">,</span> default<span class="token operator">=</span><span class="token number">5</span><span class="token punctuation">)</span>
    submitter <span class="token operator">=</span> Column<span class="token punctuation">(</span>UUID<span class="token punctuation">,</span> ForeignKey<span class="token punctuation">(</span><span class="token string">"users.id"</span><span class="token punctuation">)</span><span class="token punctuation">)</span>
    created_at <span class="token operator">=</span> Column<span class="token punctuation">(</span>DateTime<span class="token punctuation">,</span> server_default<span class="token operator">=</span>func<span class="token punctuation">.</span>now<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span>
    updated_at <span class="token operator">=</span> Column<span class="token punctuation">(</span>DateTime<span class="token punctuation">,</span> onupdate<span class="token operator">=</span>func<span class="token punctuation">.</span>now<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span>
</code></pre><p>每个 Stage、Event 表均包含 <code>task_id</code> 外键与 JSONB 字段 <code>metrics</code> / <code>artifacts</code> 以便扩展。</p>
<h3 id="73-演进步骤">7.3 演进步骤 </h3>
<ol>
<li><strong>Schema 初始化</strong>:使用 Alembic 生成迁移脚本,创建全量表结构与索引(task 状态+project_id 复合索引、event 时间索引等)。</li>
<li><strong>数据迁移工具</strong>:编写 <code>python scripts/migrate_from_json.py</code>,流程为
<ul>
<li>扫描 <code>config/*.json</code>, <code>project/**/project.json</code> 等;</li>
<li>解析出项目/拆分/标注/模型文件,映射到 ORM 对象;</li>
<li>批量写入新库,并将原始文件路径写入对象存储后回填 URI。</li>
</ul>
</li>
<li><strong>双写阶段</strong>:前端改造后,同时写入旧 JSON 与新 DB,验证一致性;稳定后切换为数据库唯一事实源。</li>
<li><strong>读写分离与备份</strong>:
<ul>
<li>接入 PgBouncer/ProxySQL,Gateway 与 Worker 走读写分离;</li>
<li>使用 WAL 归档或 pgBackRest 做自动备份与 PITR(Point-in-time Recovery)。</li>
</ul>
</li>
</ol>
<h3 id="74-api服务改动">7.4 API/服务改动 </h3>
<ul>
<li><code>db_manager</code> 升级为 ORM Session 工厂(async with SessionLocal as session)。</li>
<li>所有 <code>/models/*</code>, <code>/training/*</code>, <code>/dataset/*</code> API 改为访问数据库,而非本地 JSON/SQLite。</li>
<li>新增 <code>/admin/migrations</code> 或 CLI 命令,支持查看当前 schema 版本、回滚、重新迁移。</li>
</ul>
<h3 id="75-验证与回滚">7.5 验证与回滚 </h3>
<ul>
<li>数据校验脚本:对比 JSON 与数据库记录数量、哈希、样本抽检。</li>
<li>回滚策略:迁移失败时,删除新库数据/回滚事务,保留旧 JSON;生产环境启用只读窗口。</li>
<li>监控:对 PostgreSQL 连接数、慢查询、复制延迟进行告警,确保高并发场景稳定。</li>
</ul>
<h2 id="8-c-推理接口策略">8. C++ 推理接口策略 </h2>
<ul>
<li><strong>接口契约</strong>:<code>InferenceRequest</code>(模型/版本/输入/设备/批量参数)→ <code>InferenceResult</code>(状态、耗时、输出 URI、可视化路径、错误信息)。</li>
<li><strong>部署形态</strong>:
<ul>
<li>嵌入式:pybind11/ctypes 将 C++ 动态库暴露给 Python Worker,低延迟。</li>
<li>微服务:C++ gRPC 服务与 Python Worker 解耦,利于独立扩缩容。</li>
</ul>
</li>
<li><strong>资源管理</strong>:通过任务编排器向 C++ 服务申请 GPU/CPU 配额;可引入 Kubernetes/NVIDIA DCGM 做 GPU 调度。</li>
</ul>
<h2 id="9-windows-前端改造">9. Windows 前端改造 </h2>
<ol>
<li><code>window.electronAPI</code> 仅保留文件选择/本地预览;项目数据改为调用 <code>/projects</code> API。</li>
<li>标注提交、拆分配置、训练配置等都由后端持久化;本地仅做缓存。</li>
<li>新增 WebSocket 客户端,接收任务阶段事件,驱动 Train/Evaluate 页面。</li>
<li>提供项目导入/导出功能:导入旧 JSON → 上传到服务器 → 服务器写入数据库。</li>
</ol>
<h2 id="10-建议实施步骤">10. 建议实施步骤 </h2>
<ol>
<li><strong>基础设施</strong>:搭建 API Gateway + 新数据库 + 对象存储;完成鉴权与基础 CRUD。</li>
<li><strong>任务队列</strong>:引入队列/调度器,重构 <code>task_manager</code>;Python 训练 Worker 接入。</li>
<li><strong>C++ 推理 MVP</strong>:确定接口方式,完成一个模型的推理链路。</li>
<li><strong>前端重构</strong>:改用远程 API + WebSocket;Train/Evaluate 页面切换到事件驱动。</li>
<li><strong>数据迁移</strong>:编写脚本迁移 JSON/SQLite → 新库;验证一致性。</li>
<li><strong>逐步上线</strong>:先将训练迁移到服务器,再切换推理/评估;最后关闭本地 JSON 写入。</li>
</ol>
<p>此方案在保留现有代码基础上,提供清晰的演进路径,满足集中计算、C++ 推理、统一数据库以及任务可控可视的需求。</p>
 
      </div>
      
      
    
    
    
    
    
    
  
    </body></html>