chore: 初始化仓库
中华文明全图鉴——文物全图系统(PC Web 地图 + NestJS API + 管理后台)。 含三大 IP(文物南迁北归 / 国宝海外回归 / 博物馆手艺人)、AI 文物对话、 文物地图与详情、以及 demo-video-kit 演示视频生成工具。
This commit is contained in:
@@ -0,0 +1,79 @@
|
||||
# 演示视频标准制作方法(视频 + 配音 + 字幕解说)
|
||||
|
||||
一套可复现的流程:用 Playwright 自动操作真实应用录屏,叠加与配音**逐句同步**的字幕解说,再用 DashScope qwen-tts 生成自然中文配音并合成进视频。
|
||||
|
||||
## 成品
|
||||
|
||||
- `e2e/videos/wenwumap-e2e.mp4` — 无声演示(含屏幕字幕)
|
||||
- `e2e/videos/wenwumap-e2e-voiced.mp4` — **带配音成品**(视频 + 字幕 + 中文解说,三者同步)
|
||||
|
||||
## 核心原则:声画字三同步
|
||||
|
||||
解说往往比单步操作时间长。若把语音顺序排布去避免重叠,会越积越晚、滞后于画面。正确做法是 **"画面等解说"**:
|
||||
|
||||
1. 先把每句解说生成语音并测得**时长**;
|
||||
2. 录制时,**每一步至少停留其解说时长**(动作在这段时间内完成,解说同时播完);
|
||||
3. 字幕直接显示**解说词本身**(即"配音字幕"),与语音同时出现;
|
||||
4. 配音按字幕出现的**真实时间点**放置,不做重排。
|
||||
|
||||
这样画面动作、屏幕字幕、配音三者天然对齐。
|
||||
|
||||
## 文件结构
|
||||
|
||||
| 文件 | 职责 |
|
||||
|------|------|
|
||||
| `e2e/narration.mjs` | 解说词词库:`步骤标签 → 完整解说`,及文本归一化 `clean()`。**唯一的文案来源** |
|
||||
| `e2e/gen-voice.mjs` | 预生成语音:逐句调用 qwen-tts(失败回退 macOS `say`),输出 `e2e/voice/*.wav` 与 `clips.json`(文本→{file,dur}) |
|
||||
| `e2e/run.mjs` | Playwright 录屏:注入字幕条、按 `clips.json` 时长让每步停留、导出录像与 `e2e/videos/narration.json`(字幕时间轴) |
|
||||
| `e2e/add-voice.mjs` | 按 `narration.json` 时间点把 `clips.json` 语音用 ffmpeg `adelay`+`amix` 合成进视频;解说超长时冻结末帧补足 |
|
||||
|
||||
## 前置依赖
|
||||
|
||||
- Node ≥ 20、pnpm;应用已在本地运行(`pnpm dev:web` + `pnpm dev:api`)
|
||||
- Playwright + Chromium:`pnpm add -w -D playwright && pnpm exec playwright install chromium`
|
||||
- `ffmpeg` / `ffprobe`(系统安装,需含 libx264)
|
||||
- DashScope 配音:`.env` 中的 `AI_API_KEY`(qwen-tts);离线兜底用 macOS `say`
|
||||
|
||||
## 制作步骤
|
||||
|
||||
```bash
|
||||
# 1) 预生成解说语音(得到每句时长,供录制对齐)
|
||||
node e2e/gen-voice.mjs
|
||||
|
||||
# 2) 录屏:每步停留至解说播完,字幕=解说词,导出 narration.json
|
||||
node e2e/run.mjs
|
||||
|
||||
# 3) webm → mp4(H.264,便于合成与分享)
|
||||
ffmpeg -y -i e2e/videos/wenwumap-e2e.webm \
|
||||
-c:v libx264 -pix_fmt yuv420p -movflags +faststart \
|
||||
e2e/videos/wenwumap-e2e.mp4
|
||||
|
||||
# 4) 合成配音 → wenwumap-e2e-voiced.mp4
|
||||
node e2e/add-voice.mjs
|
||||
```
|
||||
|
||||
## 改解说词 / 加步骤
|
||||
|
||||
1. 在 `narration.mjs` 的 `NARRATION` 里增改 `步骤标签: "解说词"`;
|
||||
2. 在 `run.mjs` 中用相同的 `标签` 调 `safe("标签", async () => { ...操作... })`,字幕与停留自动按解说词处理;
|
||||
3. 重新跑 1→4 即可。
|
||||
|
||||
> 字幕展示与配音文本都来自同一份解说词,无需两处维护。
|
||||
|
||||
## 可配置项
|
||||
|
||||
- 配音音色:`QWEN_TTS_VOICE=Cherry|Serena|Ethan|Chelsie node e2e/gen-voice.mjs`
|
||||
- 离线兜底音色:`TTS_VOICE=Tingting`(macOS 中文语音)
|
||||
- 录制目标地址:`BASE_URL=http://localhost:3000 node e2e/run.mjs`
|
||||
|
||||
## 健壮性设计
|
||||
|
||||
- **限流重试**:qwen-tts 每句最多重试 3 次并间隔限速,失败自动回退系统 TTS,保证流程不中断。
|
||||
- **容错操作**:`run.mjs` 的 `safe()` 单步失败只记录、不中断整段录制。
|
||||
- **时长自适应**:解说总时长超过视频时,`add-voice.mjs` 冻结末帧补足,保证解说完整不被截断。
|
||||
- **沙箱兼容**:图片等外链由浏览器侧加载/回退;服务端只依赖可达的 DashScope。
|
||||
|
||||
## 已知注意点
|
||||
|
||||
- qwen-tts 有突发频率限制,密集重复生成会触发回退(混入系统语音)。`gen-voice.mjs` 已加间隔;如需全部自然语音,单独重试个别句子即可。
|
||||
- 录制时长依赖 AI 流式回答耗时,单次运行略有浮动,但同步逻辑不受影响。
|
||||
@@ -0,0 +1,68 @@
|
||||
// 把预生成的解说语音(e2e/voice/clips.json)按字幕出现的真实时间点(narration.json)合成到视频。
|
||||
// 因为 run.mjs 已让每步停留足够长,时间点本身不重叠,故直接按时间点放置即可保持声画同步。
|
||||
import { execSync } from "child_process";
|
||||
import fs from "fs";
|
||||
import path from "path";
|
||||
import { clean } from "./narration.mjs";
|
||||
|
||||
const DIR = path.resolve("e2e/videos");
|
||||
const VIDEO_IN = path.join(DIR, "wenwumap-e2e.mp4");
|
||||
const VIDEO_OUT = path.join(DIR, "wenwumap-e2e-voiced.mp4");
|
||||
|
||||
const sh = (cmd) => execSync(cmd, { stdio: ["ignore", "pipe", "pipe"] }).toString();
|
||||
const durationOf = (f) =>
|
||||
parseFloat(sh(`ffprobe -v error -show_entries format=duration -of csv=p=0 "${f}"`).trim());
|
||||
|
||||
if (!fs.existsSync(VIDEO_IN)) {
|
||||
console.error("缺少视频:先 node e2e/run.mjs 并转码生成 wenwumap-e2e.mp4");
|
||||
process.exit(1);
|
||||
}
|
||||
const { narration } = JSON.parse(fs.readFileSync(path.join(DIR, "narration.json"), "utf-8"));
|
||||
const clipsMap = JSON.parse(fs.readFileSync(path.resolve("e2e/voice/clips.json"), "utf-8"));
|
||||
|
||||
// 按字幕时间点匹配预生成语音
|
||||
const clips = [];
|
||||
for (const n of narration) {
|
||||
const key = clean(n.text);
|
||||
const c = clipsMap[key];
|
||||
if (!c) {
|
||||
console.warn(`× 缺少语音:${key.slice(0, 24)}…`);
|
||||
continue;
|
||||
}
|
||||
clips.push({ file: path.resolve(c.file), start: n.t, dur: c.dur });
|
||||
console.log(`✓ [${(n.t / 1000).toFixed(1)}s ${c.dur.toFixed(1)}s] ${key.slice(0, 26)}…`);
|
||||
}
|
||||
if (clips.length === 0) {
|
||||
console.error("无可用解说,请先运行 node e2e/gen-voice.mjs");
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// 视频不够长则冻结末帧补足
|
||||
const videoDur = durationOf(VIDEO_IN);
|
||||
const audioEnd = Math.max(...clips.map((c) => c.start / 1000 + c.dur)) + 0.6;
|
||||
const pad = audioEnd - videoDur;
|
||||
|
||||
const inputs = clips.map((c) => `-i "${c.file}"`).join(" ");
|
||||
const audioFilters = clips
|
||||
.map((c, i) => `[${i + 1}:a]adelay=${Math.round(c.start)}|${Math.round(c.start)}[a${i}]`)
|
||||
.join(";");
|
||||
const mixIn = clips.map((_, i) => `[a${i}]`).join("");
|
||||
const amix = `${mixIn}amix=inputs=${clips.length}:normalize=0[a]`;
|
||||
|
||||
let videoMap = "0:v";
|
||||
let vcodec = "-c:v copy";
|
||||
let vfilter = "";
|
||||
if (pad > 0.3) {
|
||||
vfilter = `[0:v]tpad=stop_mode=clone:stop_duration=${pad.toFixed(2)}[v];`;
|
||||
videoMap = '"[v]"';
|
||||
vcodec = "-c:v libx264 -pix_fmt yuv420p";
|
||||
}
|
||||
|
||||
const cmd =
|
||||
`ffmpeg -y -i "${VIDEO_IN}" ${inputs} ` +
|
||||
`-filter_complex "${vfilter}${audioFilters};${amix}" ` +
|
||||
`-map ${videoMap} -map "[a]" ${vcodec} -c:a aac -b:a 160k "${VIDEO_OUT}"`;
|
||||
|
||||
console.log(`视频 ${videoDur.toFixed(1)}s / 解说至 ${audioEnd.toFixed(1)}s${pad > 0.3 ? `(末帧补 ${pad.toFixed(1)}s)` : ""}`);
|
||||
execSync(cmd, { stdio: "inherit" });
|
||||
console.log(`完成:${VIDEO_OUT}`);
|
||||
@@ -0,0 +1,221 @@
|
||||
// WenwuMap 演示视频配置(使用 tools/demo-video-kit)
|
||||
// 运行:node tools/demo-video-kit/src/cli.mjs e2e/demo.config.mjs --out e2e/videos-kit
|
||||
import { NARRATION } from "./narration.mjs";
|
||||
|
||||
const N = (k) => NARRATION[k] || k;
|
||||
|
||||
export default {
|
||||
baseUrl: "http://localhost:3000/map",
|
||||
viewport: { width: 1440, height: 900 },
|
||||
brand: "中华文明全图鉴 · 功能演示",
|
||||
outDir: "e2e/videos-kit",
|
||||
voice: { name: "Cherry", model: "qwen-tts", apiKeyEnv: "AI_API_KEY", fallbackVoice: "Tingting" },
|
||||
|
||||
intro: { narration: N("__intro"), sub: "中华文明全图鉴 · Cultural Heritage Atlas" },
|
||||
outro: { narration: N("__outro") },
|
||||
|
||||
steps: [
|
||||
{
|
||||
label: "搜索",
|
||||
narration: N("搜索“青铜”"),
|
||||
run: async ({ page, pause }) => {
|
||||
const input = page.getByPlaceholder("搜索文物、机构、城市…");
|
||||
await input.click();
|
||||
await input.pressSequentially("青铜", { delay: 120 });
|
||||
await pause(1500);
|
||||
await input.fill("");
|
||||
await pause(500);
|
||||
},
|
||||
},
|
||||
{
|
||||
label: "城市分布翻页",
|
||||
narration: N("城市分布翻页"),
|
||||
run: async ({ page, pause }) => {
|
||||
const next = page.locator("aside").first().getByRole("button", { name: "下一页" });
|
||||
for (let i = 0; i < 2; i++) {
|
||||
await next.click({ timeout: 4000 });
|
||||
await pause(900);
|
||||
}
|
||||
},
|
||||
},
|
||||
{
|
||||
label: "门类筛选:青铜器",
|
||||
narration: N("门类筛选:青铜器"),
|
||||
run: async ({ page, pause }) => {
|
||||
const b = page.getByRole("button", { name: "青铜器" }).first();
|
||||
await b.click({ timeout: 5000 });
|
||||
await pause(1500);
|
||||
await b.click({ timeout: 5000 });
|
||||
await pause(600);
|
||||
},
|
||||
},
|
||||
{
|
||||
label: "年代筛选:唐代",
|
||||
narration: N("年代筛选:唐代"),
|
||||
run: async ({ page, pause }) => {
|
||||
const b = page.getByRole("button", { name: "唐代", exact: true }).first();
|
||||
await b.click({ timeout: 5000 });
|
||||
await pause(1500);
|
||||
await b.click({ timeout: 5000 });
|
||||
await pause(600);
|
||||
},
|
||||
},
|
||||
{
|
||||
label: "左栏收起/展开",
|
||||
narration: N("左栏收起/展开"),
|
||||
run: async ({ page, pause }) => {
|
||||
const t = page.getByTitle(/收起左栏|展开左栏/);
|
||||
await t.click({ timeout: 5000 });
|
||||
await pause(1100);
|
||||
await t.click({ timeout: 5000 });
|
||||
await pause(800);
|
||||
},
|
||||
},
|
||||
{
|
||||
label: "右栏收起/展开",
|
||||
narration: N("右栏收起/展开"),
|
||||
run: async ({ page, pause }) => {
|
||||
const t = page.getByTitle(/收起右栏|展开右栏/);
|
||||
await t.click({ timeout: 5000 });
|
||||
await pause(1100);
|
||||
await t.click({ timeout: 5000 });
|
||||
await pause(800);
|
||||
},
|
||||
},
|
||||
{
|
||||
label: "点击地图标记 · 查看馆藏",
|
||||
narration: N("点击地图标记 · 查看馆藏"),
|
||||
run: async ({ page, pause }) => {
|
||||
const markers = page.locator("div.cursor-pointer.flex-col");
|
||||
await markers.first().waitFor({ state: "attached", timeout: 25000 });
|
||||
await pause(1200);
|
||||
const aside = page.locator("aside").last();
|
||||
const total = Math.min(await markers.count(), 8);
|
||||
for (let i = 0; i < total; i++) {
|
||||
const box = await markers.nth(i).boundingBox();
|
||||
if (!box) continue;
|
||||
await page.mouse.click(box.x + box.width / 2, box.y + box.height - 3);
|
||||
await pause(1300);
|
||||
if (
|
||||
(await aside.getByText("机构藏品").count()) > 0 ||
|
||||
(await page.getByRole("button", { name: /文物信息/ }).count()) > 0
|
||||
)
|
||||
break;
|
||||
}
|
||||
},
|
||||
},
|
||||
{
|
||||
label: "进入文物详情",
|
||||
narration: N("进入文物详情"),
|
||||
run: async ({ page, pause }) => {
|
||||
const aside = page.locator("aside").last();
|
||||
if ((await aside.getByText("机构藏品").count()) > 0) {
|
||||
await aside.locator("button:has(div.truncate)").first().click({ timeout: 4000 });
|
||||
await pause(1600);
|
||||
}
|
||||
await page.getByRole("button", { name: /文物信息/ }).waitFor({ timeout: 8000 });
|
||||
},
|
||||
},
|
||||
{
|
||||
label: "展开同一机构列表",
|
||||
narration: N("展开同一机构列表"),
|
||||
run: async ({ page, pause }) => {
|
||||
await page.getByText(/还有 \d+ 件/).first().click({ timeout: 4000 });
|
||||
await pause(1500);
|
||||
},
|
||||
},
|
||||
{
|
||||
label: "文物信息折叠/展开",
|
||||
narration: N("文物信息折叠/展开"),
|
||||
run: async ({ page, pause }) => {
|
||||
const btn = page.getByRole("button", { name: /文物信息/ });
|
||||
await btn.click({ timeout: 4000 });
|
||||
await pause(1000);
|
||||
await btn.click({ timeout: 4000 });
|
||||
await pause(800);
|
||||
},
|
||||
},
|
||||
{
|
||||
label: "AI 文物对话 · 流式回答与追问建议",
|
||||
narration: N("AI 文物对话 · 流式回答与追问建议"),
|
||||
run: async ({ page, pause }) => {
|
||||
const aside = page.locator("aside").last();
|
||||
await page.getByRole("button", { name: "讲解员" }).first().click({ timeout: 4000 });
|
||||
await pause(800);
|
||||
await page.getByText("它为什么珍贵?").first().click({ timeout: 4000 });
|
||||
await page.locator("div.md-chat").first().waitFor({ timeout: 30000 });
|
||||
await pause(3500);
|
||||
await aside.getByText("下一步 · 你可以问").waitFor({ timeout: 20000 });
|
||||
await aside.locator('button:has-text("?")').first().click({ timeout: 4000 });
|
||||
await page.locator("div.md-chat").nth(1).waitFor({ timeout: 30000 });
|
||||
await pause(3500);
|
||||
},
|
||||
},
|
||||
{
|
||||
label: "图片放大与无极缩放(马踏飞燕)",
|
||||
narration: N("图片放大与无极缩放(马踏飞燕)"),
|
||||
run: async ({ page, pause }) => {
|
||||
const input = page.getByPlaceholder("搜索文物、机构、城市…");
|
||||
await input.fill("马踏飞燕");
|
||||
await page
|
||||
.waitForFunction(() => document.querySelectorAll("div.cursor-pointer.flex-col").length <= 2, null, {
|
||||
timeout: 8000,
|
||||
})
|
||||
.catch(() => {});
|
||||
await pause(900);
|
||||
const markers = page.locator("div.cursor-pointer.flex-col");
|
||||
await markers.first().waitFor({ state: "attached", timeout: 15000 });
|
||||
const box = await markers.first().boundingBox();
|
||||
await page.mouse.click(box.x + box.width / 2, box.y + box.height - 3);
|
||||
await pause(1700);
|
||||
const aside = page.locator("aside").last();
|
||||
if ((await aside.getByText("机构藏品").count()) > 0) {
|
||||
await aside.locator("button:has(div.truncate)").first().click({ timeout: 4000 });
|
||||
await pause(1500);
|
||||
}
|
||||
const cover = page.locator("img.cursor-zoom-in").first();
|
||||
await cover.waitFor({ state: "visible", timeout: 6000 });
|
||||
const cb = await cover.boundingBox();
|
||||
await page.mouse.click(cb.x + cb.width / 2, cb.y + cb.height / 2);
|
||||
await pause(1400);
|
||||
await page.mouse.move(720, 450);
|
||||
for (let i = 0; i < 6; i++) {
|
||||
await page.mouse.wheel(0, -320);
|
||||
await pause(420);
|
||||
}
|
||||
await pause(1200);
|
||||
await page.keyboard.press("Escape");
|
||||
await pause(900);
|
||||
},
|
||||
},
|
||||
{
|
||||
label: "文物南迁北归 · 重走万里守护之路",
|
||||
narration: N("文物南迁北归 · 重走万里守护之路"),
|
||||
run: async ({ page, pause }) => {
|
||||
await page.getByPlaceholder("搜索文物、机构、城市…").fill("");
|
||||
await pause(700);
|
||||
await page.getByRole("button", { name: "文物南迁北归之路" }).click({ timeout: 6000 });
|
||||
await pause(2600);
|
||||
await page.getByRole("button", { name: "播放" }).click({ timeout: 6000 });
|
||||
await pause(12000);
|
||||
await page.getByRole("button", { name: "退出路线" }).click({ timeout: 6000 });
|
||||
await pause(1000);
|
||||
},
|
||||
},
|
||||
{
|
||||
label: "国宝海外回归 · 选择文物,重走回家之路",
|
||||
narration: N("国宝海外回归 · 选择文物,重走回家之路"),
|
||||
run: async ({ page, pause }) => {
|
||||
await page.getByRole("button", { name: "国宝海外回归" }).click({ timeout: 6000 });
|
||||
await page.getByText("国宝海外回归 · 选择文物").first().waitFor({ timeout: 6000 });
|
||||
await pause(1600);
|
||||
await page.locator('button:has-text("现藏")').first().click({ timeout: 6000 });
|
||||
await pause(2600);
|
||||
await page.getByRole("button", { name: "播放" }).click({ timeout: 6000 });
|
||||
await pause(7000);
|
||||
await page.getByRole("button", { name: "退出路线" }).click({ timeout: 6000 });
|
||||
await pause(1000);
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
@@ -0,0 +1,78 @@
|
||||
// 预生成解说语音(DashScope qwen-tts),输出 e2e/voice/*.wav 与 clips.json(文本→{file,dur})。
|
||||
// run.mjs 据此让每步停留足够时长;add-voice.mjs 据此把语音放到字幕出现的时间点。
|
||||
import { execSync } from "child_process";
|
||||
import crypto from "crypto";
|
||||
import fs from "fs";
|
||||
import path from "path";
|
||||
import { NARRATION, clean } from "./narration.mjs";
|
||||
|
||||
const OUT = path.resolve("e2e/voice");
|
||||
const QWEN_VOICE = process.env.QWEN_TTS_VOICE ?? "Cherry";
|
||||
const FALLBACK_VOICE = process.env.TTS_VOICE ?? "Tingting";
|
||||
|
||||
function readApiKey() {
|
||||
const env = fs.readFileSync(path.resolve(".env"), "utf-8");
|
||||
const m = env.match(/^AI_API_KEY=(.*)$/m);
|
||||
return m ? m[1].trim() : "";
|
||||
}
|
||||
const KEY = readApiKey();
|
||||
const sh = (cmd) => execSync(cmd, { stdio: ["ignore", "pipe", "pipe"] }).toString();
|
||||
const durationOf = (f) =>
|
||||
parseFloat(sh(`ffprobe -v error -show_entries format=duration -of csv=p=0 "${f}"`).trim());
|
||||
|
||||
async function qwenTts(text, outFile) {
|
||||
const r = await fetch(
|
||||
"https://dashscope.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation",
|
||||
{
|
||||
method: "POST",
|
||||
headers: { Authorization: `Bearer ${KEY}`, "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ model: "qwen-tts", input: { text, voice: QWEN_VOICE } }),
|
||||
signal: AbortSignal.timeout(40000),
|
||||
}
|
||||
);
|
||||
const j = await r.json();
|
||||
const url = j?.output?.audio?.url;
|
||||
if (!url) throw new Error(JSON.stringify(j).slice(0, 160));
|
||||
const a = await fetch(url, { signal: AbortSignal.timeout(40000) });
|
||||
if (!a.ok) throw new Error(`下载音频失败 ${a.status}`);
|
||||
fs.writeFileSync(outFile, Buffer.from(await a.arrayBuffer()));
|
||||
}
|
||||
|
||||
const main = async () => {
|
||||
fs.rmSync(OUT, { recursive: true, force: true });
|
||||
fs.mkdirSync(OUT, { recursive: true });
|
||||
const texts = Array.from(new Set(Object.values(NARRATION).map(clean))).filter(Boolean);
|
||||
const clips = {};
|
||||
for (const text of texts) {
|
||||
const id = crypto.createHash("md5").update(text).digest("hex").slice(0, 10);
|
||||
const wav = path.join(OUT, `${id}.wav`);
|
||||
let engine = "qwen-tts";
|
||||
let ok = false;
|
||||
for (let attempt = 0; attempt < 5 && !ok; attempt++) {
|
||||
try {
|
||||
await qwenTts(text, wav);
|
||||
ok = true;
|
||||
} catch (e) {
|
||||
if (attempt < 4) await new Promise((r) => setTimeout(r, 2500 * (attempt + 1)));
|
||||
}
|
||||
}
|
||||
if (!ok) {
|
||||
engine = "say";
|
||||
const aiff = path.join(OUT, `${id}.aiff`);
|
||||
sh(`say -v ${FALLBACK_VOICE} -o "${aiff}" "${text.replace(/"/g, "")}"`);
|
||||
sh(`ffmpeg -y -i "${aiff}" "${wav}"`);
|
||||
fs.rmSync(aiff, { force: true });
|
||||
}
|
||||
const dur = durationOf(wav);
|
||||
clips[text] = { file: path.relative(process.cwd(), wav), dur, engine };
|
||||
console.log(`✓ ${dur.toFixed(1)}s [${engine}] ${text.slice(0, 26)}…`);
|
||||
await new Promise((r) => setTimeout(r, 1200)); // 限速,避免触发节流
|
||||
}
|
||||
fs.writeFileSync(path.join(OUT, "clips.json"), JSON.stringify(clips, null, 2));
|
||||
console.log(`完成:${texts.length} 条语音 → e2e/voice/clips.json`);
|
||||
};
|
||||
|
||||
main().catch((e) => {
|
||||
console.error(e);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -0,0 +1,30 @@
|
||||
// 解说词(步骤标签 → 完整解说)。画面字幕用简短标题,配音用这里的完整解说。
|
||||
export const NARRATION = {
|
||||
__intro: "中华文明全图鉴。这是一张可以漫游、可以对话、可以追溯的文物全图,让沉睡的国宝在地图上重新活起来。",
|
||||
打开文物全图地图: "我们在地图上汇聚了海内外数十家机构的珍贵文物,每一个点,都是一段文明的记忆。",
|
||||
"搜索“青铜”": "你可以直接搜索文物、机构或城市,快速定位心之所向。",
|
||||
城市分布翻页: "左侧按城市统计文物分布,分页浏览,纵览华夏各地的文化重镇。",
|
||||
"门类筛选:青铜器": "也可以按门类筛选,比如青铜器,看尽礼乐中国的厚重。",
|
||||
"年代筛选:唐代": "或按年代回望,从商周到明清,触摸不同王朝的气象。",
|
||||
"左栏收起/展开": "左侧筛选栏可以随时收起,把更多空间交给地图。",
|
||||
"右栏收起/展开": "右侧详情栏同样可以收起,也支持自由拖拽宽度。",
|
||||
"点击地图标记 · 查看馆藏": "点击任意标记,就能查看这座城市、这家机构所藏的文物清单。",
|
||||
进入文物详情: "选中一件文物,右侧展开它的详情:门类、年代、级别,以及一句动人的故事钩子。",
|
||||
展开同一机构列表: "你还能一键浏览同一机构的其他珍藏,在文物之间自由穿梭。",
|
||||
"文物信息折叠/展开": "信息区可以折叠,为下面的对话腾出空间。",
|
||||
"AI 文物对话 · 流式回答与追问建议": "最特别的是,你可以直接和文物对话。人工智能以文物的口吻娓娓道来,并贴心地给出下一步追问。",
|
||||
"图片放大与无极缩放(马踏飞燕)": "点击文物图片可以全屏放大,滚轮无极缩放,细看千年工艺的每一处肌理。",
|
||||
"文物南迁北归 · 重走万里守护之路":
|
||||
"这是文物南迁北归之路。抗战烽火中,故宫人将上万箱国宝辗转南迁西运,胜利后又东归北返,行程逾万里而几乎无损,堪称文明守护史上的奇迹。",
|
||||
"国宝海外回归 · 选择文物,重走回家之路":
|
||||
"这是国宝的回家之路。选择一件曾经流失海外的文物,地图便会重现它跨越重洋、终归故土的旅程。",
|
||||
__outro: "留住这些瑰宝,就是守护我们这个世界上唯一没有中断的文明。感谢观看。",
|
||||
};
|
||||
|
||||
// 字幕/配音文本归一化:去掉序号前缀、把分隔点换成停顿
|
||||
export function clean(text) {
|
||||
return String(text)
|
||||
.replace(/^\d+\.\s*/, "")
|
||||
.replace(/·/g, ",")
|
||||
.trim();
|
||||
}
|
||||
+336
@@ -0,0 +1,336 @@
|
||||
import { chromium } from "playwright";
|
||||
import fs from "fs";
|
||||
import path from "path";
|
||||
import { NARRATION, clean } from "./narration.mjs";
|
||||
|
||||
const BASE = process.env.BASE_URL ?? "http://localhost:3000";
|
||||
const OUT_DIR = path.resolve("e2e/videos");
|
||||
fs.mkdirSync(OUT_DIR, { recursive: true });
|
||||
|
||||
const log = (m) => console.log(`▶ ${m}`);
|
||||
const pause = (p, ms = 900) => p.waitForTimeout(ms);
|
||||
|
||||
// 读取预生成语音时长,用于让每步停留足够长以与解说同步
|
||||
const durMap = new Map();
|
||||
try {
|
||||
const clips = JSON.parse(fs.readFileSync(path.resolve("e2e/voice/clips.json"), "utf-8"));
|
||||
for (const [t, v] of Object.entries(clips)) durMap.set(t, v.dur);
|
||||
} catch {
|
||||
/* 未预生成语音则不做停留对齐 */
|
||||
}
|
||||
const dwellMs = (text) => Math.round((durMap.get(clean(text)) ?? 0) * 1000);
|
||||
|
||||
const main = async () => {
|
||||
const browser = await chromium.launch({ headless: true });
|
||||
const context = await browser.newContext({
|
||||
viewport: { width: 1440, height: 900 },
|
||||
recordVideo: { dir: OUT_DIR, size: { width: 1440, height: 900 } },
|
||||
});
|
||||
const page = await context.newPage();
|
||||
page.on("console", (msg) => {
|
||||
if (msg.type() === "error") log(`console.error: ${msg.text().slice(0, 120)}`);
|
||||
});
|
||||
|
||||
// 记录字幕出现的时间轴(相对录像开始),供后期配音对齐
|
||||
const t0 = Date.now();
|
||||
const narration = [];
|
||||
|
||||
// 屏幕字幕:注入一个固定字幕条,录像里可见
|
||||
const say = async (text, sub = "", voice = "") => {
|
||||
const spoken = clean(voice || text);
|
||||
narration.push({ t: Math.max(0, Date.now() - t0), text: spoken });
|
||||
await page
|
||||
.evaluate(
|
||||
({ text, sub }) => {
|
||||
let el = document.getElementById("__cap");
|
||||
if (!el) {
|
||||
el = document.createElement("div");
|
||||
el.id = "__cap";
|
||||
el.style.cssText =
|
||||
"position:fixed;left:50%;bottom:60px;transform:translateX(-50%);z-index:2147483647;" +
|
||||
"max-width:86vw;padding:12px 26px;border-radius:9999px;text-align:center;" +
|
||||
"background:rgba(8,7,5,0.84);border:1px solid rgba(214,170,91,0.5);" +
|
||||
"box-shadow:0 10px 36px rgba(0,0,0,.55);backdrop-filter:blur(8px);" +
|
||||
"font-family:'Noto Serif SC',serif;color:#f2cf83;transition:opacity .2s;pointer-events:none";
|
||||
document.body.appendChild(el);
|
||||
}
|
||||
el.innerHTML =
|
||||
`<div style="font-size:18px;line-height:1.65;letter-spacing:0.5px">${text}</div>` +
|
||||
(sub
|
||||
? `<div style="font-size:11px;color:#c7baa0;letter-spacing:2px;margin-top:4px">${sub}</div>`
|
||||
: "");
|
||||
el.style.opacity = "1";
|
||||
},
|
||||
{ text, sub }
|
||||
)
|
||||
.catch(() => {});
|
||||
};
|
||||
|
||||
let stepNo = 0;
|
||||
/** 执行一步:字幕显示解说词本身(与配音同步)→ 执行 → 停留至解说播完 */
|
||||
const safe = async (label, fn) => {
|
||||
stepNo += 1;
|
||||
const startT = Date.now();
|
||||
const voiceText = NARRATION[label] || label;
|
||||
await say(voiceText, "中华文明全图鉴 · 功能演示", voiceText);
|
||||
try {
|
||||
await fn();
|
||||
log(`ok: ${label}`);
|
||||
} catch (e) {
|
||||
log(`skip: ${label} (${e.message.split("\n")[0]})`);
|
||||
}
|
||||
const need = dwellMs(voiceText) + 500; // 句尾留白
|
||||
const elapsed = Date.now() - startT;
|
||||
if (elapsed < need) await page.waitForTimeout(need - elapsed);
|
||||
};
|
||||
|
||||
log(`打开 ${BASE}/map`);
|
||||
await page.goto(`${BASE}/map`, { waitUntil: "domcontentloaded", timeout: 60000 });
|
||||
await page.waitForTimeout(3000);
|
||||
await say(NARRATION.__intro, "中华文明全图鉴 · Cultural Heritage Atlas", NARRATION.__intro);
|
||||
await page.waitForTimeout(Math.max(2500, dwellMs(NARRATION.__intro) + 500));
|
||||
await safe("打开文物全图地图", async () => {
|
||||
await page.getByText("中华文明全图鉴").first().waitFor({ timeout: 20000 });
|
||||
});
|
||||
await pause(page, 1200);
|
||||
|
||||
// 1. 搜索
|
||||
await safe("搜索“青铜”", async () => {
|
||||
const input = page.getByPlaceholder("搜索文物、机构、城市…");
|
||||
await input.click();
|
||||
await input.pressSequentially("青铜", { delay: 120 });
|
||||
await pause(page, 1500);
|
||||
await input.fill("");
|
||||
await pause(page, 600);
|
||||
});
|
||||
|
||||
// 2. 城市分布翻页(左栏内的下一页按钮)
|
||||
await safe("城市分布翻页", async () => {
|
||||
const aside = page.locator("aside").first();
|
||||
const next = aside.getByRole("button", { name: "下一页" });
|
||||
for (let i = 0; i < 2; i++) {
|
||||
await next.click({ timeout: 4000 });
|
||||
await pause(page, 900);
|
||||
}
|
||||
});
|
||||
|
||||
// 3. 门类筛选
|
||||
await safe("门类筛选:青铜器", async () => {
|
||||
await page.getByRole("button", { name: "青铜器" }).first().click({ timeout: 5000 });
|
||||
await pause(page, 1500);
|
||||
await page.getByRole("button", { name: "青铜器" }).first().click({ timeout: 5000 });
|
||||
await pause(page, 800);
|
||||
});
|
||||
|
||||
// 4. 年代筛选
|
||||
await safe("年代筛选:唐代", async () => {
|
||||
await page.getByRole("button", { name: "唐代", exact: true }).first().click({ timeout: 5000 });
|
||||
await pause(page, 1500);
|
||||
await page.getByRole("button", { name: "唐代", exact: true }).first().click({ timeout: 5000 });
|
||||
await pause(page, 800);
|
||||
});
|
||||
|
||||
// 5. 左栏收起 / 展开
|
||||
await safe("左栏收起/展开", async () => {
|
||||
const leftToggle = page.getByTitle(/收起左栏|展开左栏/);
|
||||
await leftToggle.click({ timeout: 5000 });
|
||||
await pause(page, 1200);
|
||||
await leftToggle.click({ timeout: 5000 });
|
||||
await pause(page, 1000);
|
||||
});
|
||||
|
||||
// 6. 右栏收起 / 展开
|
||||
await safe("右栏收起/展开", async () => {
|
||||
const rightToggle = page.getByTitle(/收起右栏|展开右栏/);
|
||||
await rightToggle.click({ timeout: 5000 });
|
||||
await pause(page, 1200);
|
||||
await rightToggle.click({ timeout: 5000 });
|
||||
await pause(page, 1000);
|
||||
});
|
||||
|
||||
// 7. 点击地图 marker 打开右栏(遍历前几个 marker,直到面板打开)
|
||||
let panelOpened = false;
|
||||
await safe("点击地图标记 · 查看馆藏", async () => {
|
||||
const markers = page.locator("div.cursor-pointer.flex-col");
|
||||
await markers.first().waitFor({ state: "attached", timeout: 25000 });
|
||||
await pause(page, 1500);
|
||||
const aside = page.locator("aside").last();
|
||||
const total = Math.min(await markers.count(), 8);
|
||||
for (let i = 0; i < total; i++) {
|
||||
const box = await markers.nth(i).boundingBox();
|
||||
if (!box) continue;
|
||||
await page.mouse.click(box.x + box.width / 2, box.y + box.height - 3);
|
||||
await pause(page, 1400);
|
||||
const realPanel =
|
||||
(await aside.getByText("机构藏品").count()) > 0 ||
|
||||
(await page.getByRole("button", { name: /文物信息/ }).count()) > 0;
|
||||
if (realPanel) {
|
||||
panelOpened = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (!panelOpened) throw new Error("右栏面板未打开");
|
||||
});
|
||||
|
||||
if (panelOpened) {
|
||||
// 确保进入“文物详情”视图(若是机构藏品列表则先点一件)
|
||||
let detailOpen = false;
|
||||
await safe("进入文物详情", async () => {
|
||||
const aside = page.locator("aside").last();
|
||||
if ((await aside.getByText("机构藏品").count()) > 0) {
|
||||
await aside.locator("button:has(div.truncate)").first().click({ timeout: 4000 });
|
||||
await pause(page, 1800);
|
||||
}
|
||||
await page.getByRole("button", { name: /文物信息/ }).waitFor({ timeout: 8000 });
|
||||
detailOpen = true;
|
||||
});
|
||||
|
||||
if (!detailOpen) {
|
||||
log("详情未打开,跳过详情相关步骤");
|
||||
} else {
|
||||
|
||||
// 8. 同一机构展开
|
||||
await safe("展开同一机构列表", async () => {
|
||||
await page.getByText(/还有 \d+ 件/).first().click({ timeout: 4000 });
|
||||
await pause(page, 1500);
|
||||
});
|
||||
|
||||
// 9. 文物信息折叠/展开
|
||||
await safe("文物信息折叠/展开", async () => {
|
||||
const btn = page.getByRole("button", { name: /文物信息/ });
|
||||
await btn.click({ timeout: 4000 });
|
||||
await pause(page, 1000);
|
||||
await btn.click({ timeout: 4000 });
|
||||
await pause(page, 1000);
|
||||
});
|
||||
|
||||
// 10. 图片放大弹窗改到 马踏飞燕(有 mock 图)的专属环节,见下方
|
||||
|
||||
// 11. AI 角色对话:点起始问题 → 等流式 → 点一个下一步建议
|
||||
await safe("AI 文物对话 · 流式回答与追问建议", async () => {
|
||||
const aside = page.locator("aside").last();
|
||||
await page.getByRole("button", { name: "讲解员" }).first().click({ timeout: 4000 });
|
||||
await pause(page, 800);
|
||||
await page.getByText("它为什么珍贵?").first().click({ timeout: 4000 });
|
||||
// 等待流式回答出现
|
||||
await page.locator("div.md-chat").first().waitFor({ timeout: 30000 });
|
||||
await pause(page, 4000);
|
||||
// 等下一步建议并点击第一个
|
||||
await aside.getByText("下一步 · 你可以问").waitFor({ timeout: 20000 });
|
||||
await pause(page, 1200);
|
||||
await aside.locator('button:has-text("?")').first().click({ timeout: 4000 });
|
||||
await page.locator("div.md-chat").nth(1).waitFor({ timeout: 30000 });
|
||||
await pause(page, 4000);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// 马踏飞燕:搜索 → 选中 → 图片放大弹窗 + 无极缩放(本地真实图片)
|
||||
await safe("图片放大与无极缩放(马踏飞燕)", async () => {
|
||||
// 关闭可能打开的详情,回到可搜索状态
|
||||
const input = page.getByPlaceholder("搜索文物、机构、城市…");
|
||||
await input.click({ timeout: 5000 });
|
||||
await input.fill("马踏飞燕");
|
||||
// 等待筛选生效、marker 数量收敛,避免点到正在被移除的旧 marker
|
||||
await page
|
||||
.waitForFunction(
|
||||
() => document.querySelectorAll("div.cursor-pointer.flex-col").length <= 2,
|
||||
null,
|
||||
{ timeout: 8000 }
|
||||
)
|
||||
.catch(() => {});
|
||||
await pause(page, 1000);
|
||||
// 搜索后通常只剩该点位,点击其 marker
|
||||
const markers = page.locator("div.cursor-pointer.flex-col");
|
||||
await markers.first().waitFor({ state: "attached", timeout: 15000 });
|
||||
await pause(page, 600);
|
||||
const box = await markers.first().boundingBox();
|
||||
await page.mouse.click(box.x + box.width / 2, box.y + box.height - 3);
|
||||
await pause(page, 1800);
|
||||
// 若弹出机构列表则点第一件
|
||||
const aside = page.locator("aside").last();
|
||||
if ((await aside.getByText("机构藏品").count()) > 0) {
|
||||
await aside.locator("button:has(div.truncate)").first().click({ timeout: 4000 });
|
||||
await pause(page, 1600);
|
||||
}
|
||||
// 打开放大弹窗(点击封面图本身,"点击放大"角标是 pointer-events-none)
|
||||
const cover = page.locator("img.cursor-zoom-in").first();
|
||||
await cover.waitFor({ state: "visible", timeout: 6000 });
|
||||
const cb = await cover.boundingBox();
|
||||
await page.mouse.click(cb.x + cb.width / 2, cb.y + cb.height / 2);
|
||||
await pause(page, 1500);
|
||||
// 无极缩放(滚轮放大)
|
||||
await page.mouse.move(720, 450);
|
||||
for (let i = 0; i < 6; i++) {
|
||||
await page.mouse.wheel(0, -320);
|
||||
await pause(page, 450);
|
||||
}
|
||||
await pause(page, 1500);
|
||||
// 拖动平移
|
||||
await page.mouse.move(720, 450);
|
||||
await page.mouse.down();
|
||||
await page.mouse.move(560, 360, { steps: 12 });
|
||||
await page.mouse.up();
|
||||
await pause(page, 1500);
|
||||
await page.keyboard.press("Escape");
|
||||
await pause(page, 1200);
|
||||
});
|
||||
|
||||
// 13. 文物南迁北归:激活路线 + 动画播放
|
||||
await safe("文物南迁北归 · 重走万里守护之路", async () => {
|
||||
const search = page.getByPlaceholder("搜索文物、机构、城市…");
|
||||
await search.fill("");
|
||||
await pause(page, 700);
|
||||
await page.getByRole("button", { name: "文物南迁北归之路" }).click({ timeout: 6000 });
|
||||
await pause(page, 2600); // 等待聚焦全程 + 路线展开
|
||||
await page.getByRole("button", { name: "播放" }).click({ timeout: 6000 });
|
||||
await pause(page, 12000); // 观看逐站推进 + 流动箭头 + 落点脉冲
|
||||
await page.getByRole("button", { name: "退出路线" }).click({ timeout: 6000 });
|
||||
await pause(page, 1200);
|
||||
});
|
||||
|
||||
// 14. 国宝海外回归:选择文物 → 专属回归之路 + 右栏定位
|
||||
await safe("国宝海外回归 · 选择文物,重走回家之路", async () => {
|
||||
await page.getByRole("button", { name: "国宝海外回归" }).click({ timeout: 6000 });
|
||||
await page.getByText("国宝海外回归 · 选择文物").first().waitFor({ timeout: 6000 });
|
||||
await pause(page, 1800);
|
||||
await page.locator('button:has-text("现藏")').first().click({ timeout: 6000 });
|
||||
await pause(page, 2600); // 路线激活 + 右栏定位
|
||||
await page.getByRole("button", { name: "播放" }).click({ timeout: 6000 });
|
||||
await pause(page, 7000);
|
||||
await page.getByRole("button", { name: "退出路线" }).click({ timeout: 6000 });
|
||||
await pause(page, 1200);
|
||||
});
|
||||
|
||||
await pause(page, 1500);
|
||||
await say(NARRATION.__outro, "中华文明全图鉴", NARRATION.__outro);
|
||||
await page.waitForTimeout(Math.max(2500, dwellMs(NARRATION.__outro) + 700));
|
||||
log("结束,保存录像…");
|
||||
|
||||
const video = page.video();
|
||||
await context.close();
|
||||
await browser.close();
|
||||
|
||||
// 录像在 context.close() 后落盘,取目录内最新的 webm
|
||||
await new Promise((r) => setTimeout(r, 500));
|
||||
const webms = fs
|
||||
.readdirSync(OUT_DIR)
|
||||
.filter((f) => f.endsWith(".webm") && f !== "wenwumap-e2e.webm")
|
||||
.map((f) => ({ f, t: fs.statSync(path.join(OUT_DIR, f)).mtimeMs }))
|
||||
.sort((a, b) => b.t - a.t);
|
||||
if (webms.length) {
|
||||
const dest = path.join(OUT_DIR, "wenwumap-e2e.webm");
|
||||
fs.copyFileSync(path.join(OUT_DIR, webms[0].f), dest);
|
||||
log(`录像已保存:${dest}`);
|
||||
} else {
|
||||
log("未找到录像文件");
|
||||
}
|
||||
fs.writeFileSync(path.join(OUT_DIR, "narration.json"), JSON.stringify({ narration }, null, 2));
|
||||
log(`字幕时间轴已保存:${narration.length} 条`);
|
||||
void video;
|
||||
};
|
||||
|
||||
main().catch((e) => {
|
||||
console.error(e);
|
||||
process.exit(1);
|
||||
});
|
||||
Reference in New Issue
Block a user