Initial commit: FunASR Speech Recognition Toolkit

Add complete FunASR codebase including models, runtime, and documentation.

examples/industrial_data_pretraining/fun_asr_nano/docs/finetune.md

@@ -0,0 +1,114 @@
+# Finetune
+
+「[简体中文](fintune_zh.md)」|「English」
+
+## Requirements
+
+```
+pip install funasr>=1.3.0
+```
+
+## Data Prepare
+
+Data examples
+
+```
+head -n1 data/train_example.jsonl | jq
+
+{
+  "messages": [
+    {
+      "role": "system",
+      "content": "You are a helpful assistant."
+    },
+    {
+      "role": "user",
+      "content": "语音转写：<|startofspeech|>!https://modelscope.cn/datasets/FunAudioLLM/funasr-demo/resolve/master/audios/IT0011W0002.wav<|endofspeech|>"
+    },
+    {
+      "role": "assistant",
+      "content": "几点了？"
+    }
+  ],
+  "speech_length": 145,
+  "text_length": 3
+}
+```
+
+Full ref to `data/train_example.jsonl`
+
+Description：
+
+- The content of systemis fixed as `You are a helpful assistant.`
+- The content of userincludes the prompt and the path to the audio file (enclosed between `<|startofspeech|>!`and `<|endofspeech|>`).
+  - The default prompts are `语音转写：`and `Speech transcription: `.
+  - For corresponding languages, prompts can be combined, such as `语音转写成英文：`and `Transcribe speech into Chinese: `.
+  - When the text annotation corresponding to the audio file contains no Arabic numerals or punctuation marks, you can use `语音转写，不进行文本规整：`and `Speech transcription without text normalization: `.
+- The content of assistant corresponds to the text annotation of the audio file.
+- speech_length: The number of fbank frames of the audio file (10ms per frame).
+- text_length: The number of tokens in the annotation text of the audio file (encoded using `Qwen/Qwen3-0.6B`).
+
+- `messages[2]["content"]`: transcription
+- `speech_length`: number of fbank frames of the audio file
+- `text_length`: number of tokens of the transcription (tokenized by `Qwen3-0.6B`)
+
+We provide a data format conversion tool `scp2jsonl.py`, which can convert common speech recognition training data formats such as wav scp and transcription into the ChatML format.
+
+`train_wav.scp`
+
+```
+BAC009S0764W0121 https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/BAC009S0764W0121.wav
+BAC009S0916W0489 https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/BAC009S0916W0489.wav
+```
+
+`train_text.txt`
+
+```
+BAC009S0764W0121 甚至出现交易几乎停滞的情况
+BAC009S0916W0489 湖北一公司以员工名义贷款数十员工负债千万
+```
+
+```
+python tools/scp2jsonl.py \
+  ++scp_file=data/train_wav.scp \
+  ++transcript_file=data/train_text.txt \
+  ++jsonl_file=data/train_example.jsonl
+```
+
+## Finetune
+
+Modify the `audio_encoder_conf.freeze`, `audio_adaptor_conf.freeze`, and `llm_conf.freeze` in `finetune.sh`.
+
+Set the `freeze` parameter of the modules to be fine-tuned to false(by default, only the LLM is fine-tuned).
+
+For more detailed parameters, refer to: [SenseVoice Model Training and Testing](https://github.com/modelscope/FunASR/blob/main/docs/tutorial/README.md#Model%20Training%20and%20Testing)
+
+```
+bash finetune.sh
+```
+
+### Recommended Configuration
+
+- For training data less than 1000 hours, it is recommended to fine-tune the audio_adaptor.
+- For training data less than 5000 hours, it is recommended to fine-tune the audio_encoder and audio_adaptor.
+- For training data greater than 10000 hours, it is recommended to perform full-parameter fine-tuning.
+
+## Model Evaluation
+
+After model fine-tuning is completed, you can decode the model using the decode.py script:
+
+```
+python decode.py \
+  ++model_dir=/path/to/finetuned \
+  ++scp_file=data/val_wav.scp \
+  ++output_file=output.txt
+```
+
+After decoding is completed, text inverse normalization needs to be applied to the annotations and recognition results, and then the WER should be calculated:
+
+```
+python tools/whisper_mix_normalize.py data/val_text.txt data/val_norm.txt
+python tools/whisper_mix_normalize.py output.txt output_norm.txt
+compute-wer data/val_norm.txt output_norm.txt cer.txt
+tail -n8 cer.txt
+```

examples/industrial_data_pretraining/fun_asr_nano/docs/fintune_zh.md

@@ -0,0 +1,114 @@
+# 微调
+
+「简体中文」|「[English](finetune.md)」
+
+## 安装训练环境
+
+```
+pip install funasr>=1.3.0
+```
+
+## 数据准备
+
+数据格式需要包括如下几个字段：
+
+```
+head -n1 data/train_example.jsonl | jq
+
+{
+  "messages": [
+    {
+      "role": "system",
+      "content": "You are a helpful assistant."
+    },
+    {
+      "role": "user",
+      "content": "语音转写：<|startofspeech|>!https://modelscope.cn/datasets/FunAudioLLM/funasr-demo/resolve/master/audios/IT0011W0002.wav<|endofspeech|>"
+    },
+    {
+      "role": "assistant",
+      "content": "几点了？"
+    }
+  ],
+  "speech_length": 145,
+  "text_length": 3
+}
+```
+
+详细可以参考：`data/train_example.jsonl`
+
+数据准备细节介绍：
+
+- system 的 content 固定为 `You are a helpful assistant.`
+- user 的 content 包含了 prompt 和音频文件的路径（位于 `<|startofspeech|>!` 和 `<|endofspeech|>`之间）
+  - prompt 默认为`语音转写：`和`Speech transcription: `
+  - 可以结合对应的语种为`语音转写成英文：`和`Transcribe speech into Chinese: `
+  - 当音频文件对应的文本标注不含阿拉伯数字或者标点符号时，可以使用`语音转写，不进行文本规整：`和 `Speech transcription without text normalization: `
+- assistant 的 content 对应音频文件对应的文本标注
+- speech_length：音频文件的 fbank 帧数（一帧 10ms）
+- text_length：音频文件标注文本的 token 数 (用 `Qwen/Qwen3-0.6B` 编码)
+
+我们提供了数据格式转换工具 `scp2jsonl.py`，可以将常见的语音识别训练数据格式 wav scp 和 transcription 转成 ChatML 格式。
+
+`train_wav.scp`
+
+左边为数据唯一 ID，需与 `train_text.txt` 中的 ID 一一对应 右边为音频文件的路径，格式如下
+
+```
+BAC009S0764W0121 https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/BAC009S0764W0121.wav
+BAC009S0916W0489 https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/BAC009S0916W0489.wav
+```
+
+`train_text.txt`
+
+左边为数据唯一 ID，需与 `train_wav.scp` 中的 ID 一一对应 右边为音频文件标注文本，格式如下：
+
+```
+BAC009S0764W0121 甚至出现交易几乎停滞的情况
+BAC009S0916W0489 湖北一公司以员工名义贷款数十员工负债千万
+```
+
+```
+python tools/scp2jsonl.py \
+  ++scp_file=data/train_wav.scp \
+  ++transcript_file=data/train_text.txt \
+  ++jsonl_file=data/train_example.jsonl
+```
+
+## 启动训练
+
+修改 `finetune.sh` 中的 `audio_encoder_conf.freeze`, `audio_adaptor_conf.freeze` 和 `llm_conf.freeze`。
+
+将需要微调的模块 `freeze` 设置成 `false`（默认只微调 llm）。
+
+更多参数细节参考：[SenseVoice 模型训练与测试](https://github.com/modelscope/FunASR/blob/main/docs/tutorial/README_zh.md#%E6%A8%A1%E5%9E%8B%E8%AE%AD%E7%BB%83%E4%B8%8E%E6%B5%8B%E8%AF%95)
+
+```
+bash finetune.sh
+```
+
+### 推荐配置
+
+- 训练数据少于 1000 小时，建议微调 audio_adaptor
+- 训练数据少于 5000 小时，建议微调 audio_encoder 和 audio_adaptor
+- 训练数据大于 10000 小时，建议全量参数微调
+
+## 模型评测
+
+当模型微调结束后，可以使用 decode.py 脚本对模型进行解码：
+
+```
+python decode.py \
+  ++model_dir=/path/to/finetuned \
+  ++scp_file=data/val_wav.scp \
+  ++output_file=output.txt
+```
+
+解码结束后，需要对标注和识别结果做文本逆归一化，然后计算 WER：
+
+```
+python tools/whisper_mix_normalize.py data/val_text.txt data/val_norm.txt
+python tools/whisper_mix_normalize.py output.txt output_norm.txt
+compute-wer data/val_norm.txt output_norm.txt cer.txt
+tail -n8 cer.txt
+```

examples/industrial_data_pretraining/fun_asr_nano/docs/realtime_demo.md

@@ -0,0 +1,63 @@
+# Fun-ASR-Nano 实时 WebSocket 服务 — 快速上手
+
+> 完整文档请参见：[FunASR vLLM 推理引擎指南](../../../../docs/vllm_guide.md)
+
+## 30 秒启动
+
+```bash
+cd examples/industrial_data_pretraining/fun_asr_nano
+
+# 安装依赖
+pip install -r requirements.txt
+# 按主文档选择并安装与当前 NVIDIA 驱动、CUDA runtime 和 PyTorch wheel 匹配的 vLLM 版本。
+
+# 启动服务
+CUDA_VISIBLE_DEVICES=0 python serve_realtime_ws.py --port 10095 --language 中文
+```
+
+## 客户端
+
+```bash
+# 浏览器
+open client_mic.html
+
+# Python 麦克风
+python client_python.py --server ws://localhost:10095 --mic
+
+# Python 文件
+python client_python.py --server ws://localhost:10095 --file audio.wav
+
+# 自动化测试
+python client_test.py --server ws://localhost:10095 --file audio.wav
+```
+
+## 远程访问
+
+```bash
+ssh -L 10095:localhost:10095 <server>
+# 然后本地打开 client_mic.html
+```
+
+## 功能
+
+- **vLLM 推理引擎**：RTF < 0.08，支持 tensor parallel 多卡加速
+- **流式 VAD 分句**：动态静音阈值，自然断句
+- **说话人分离 (Beta)**：流式 ID 分配 + 最终重聚类
+- **热词定制化**：加载人名、地名等实体词文件
+- **语种指定**：31 种语言 + 中文方言
+- **幻觉检测**：自动检测重复模式并截断
+
+## 文件列表
+
+| 文件 | 说明 |
+|------|------|
+| `serve_realtime_ws.py` | WebSocket 服务端 |
+| `client_mic.html` | 浏览器客户端 |
+| `client_python.py` | Python CLI 客户端 |
+| `client_test.py` | 自动化测试脚本 |
+| `热词列表` | 热词文件示例 |
+| `demo_vllm.py` | 离线 vLLM 推理 demo |
+
+## 详细文档
+
+- [FunASR vLLM 推理引擎指南](../../../../docs/vllm_guide.md) — 完整文档（离线/流式/WebSocket/API/FAQ）