Dev no emo (#123)

* Create emo_gen.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update server.py, fix bugs in func get_text() and infer(). (#52) * Extract get_text() and infer() from webui.py. (#53) * Extract get_text() and infer() from webui.py. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * add emo emb * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * init emo gen * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * init emo * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * init emo * Delete bert/bert-base-japanese-v3 directory * Create .gitkeep * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Create add_punc.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix bug in bert_gen.py (#54) * Update README.md * fix bug in models.py (#56) * 更新 models.py * Fix japanese cleaner (#61) * 初步，睡觉明天继续写（ * 好好好放错分支了，熬夜是大忌 * [pre-commit.ci] pre-commit autoupdate (#55) * [pre-commit.ci] pre-commit autoupdate updates: - [github.com/pre-commit/pre-commit-hooks: v4.4.0 → v4.5.0](https://github.com/pre-commit/pre-commit-hooks/compare/v4.4.0...v4.5.0) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Create tokenizer_config.json * update preprocess_text.py：过滤一个音频匹配多个文本的情况 (#57) * update preprocess_text.py：过滤音频不存在的情况 (#58) * 修复日语cleaner和bert * better * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: Stardust·减 <star_dust_chen@foxmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sora <atri@suzakuintsubaki.com> * Apply Code Formatter Change * Add config.yml for global configuration. (#62) * Add config.yml for global configuration. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bug in webui.py. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Rename config.yml to default_config.yml. Add ./config.yml to gitignore. * Add config.py to parse config.yml * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update webui.py (#65) * Update webui.py: 1. Add auto translation from Chinese to Japanese. 2. Start to use config.py in webui.py to set config instead of using the command line. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Fix (#68) * 加上ー * fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update infer.py and webui.py. Supports loading and inference models of 1.1.1 version. (#66) * Update infer.py and webui.py. Supports loading and inference models of 1.1.1 version. * Update config.json * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Fix bug in translate.py (#69) * Supports loading and inference models of 1.1、1.0.1、1.0 version. (#70) * Supports loading and inference models of 1.1、1.0.1、1.0 version. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Delete useless file in OldVersion --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update japanese.py (#71) Handling JA long pronunciations * 使用配置文件配置bert_gen.py, preprocess_text.py, resample.py (#72) * Update bert_gen.py, preprocess_text.py, resample.py. Support using config.yml in these files. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update bert_gen.py * Update bert_gen.py, fix bug. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Delete bert/bert-base-japanese-v3 directory * Create config.json * Create tokenizer_config.json * Create vocab.txt * Update server.py. 支持多版本多模型 (#76) * Update server.py. 支持多版本多模型 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Dev webui (#77) * 申请pr (#75) * 2023/10/11 update 界面优化 * Update webui.py 翻译英文页面为中文 * Update train_ms.py 单卡训练 * 加入图片 * Update extern_subprocess.py * Update asr_transcript.py * Update asr_transcript.py * Update asr_transcript.py * Update extern_subprocess.py * Update asr_transcript.py * Update asr_transcript.py * Update asr_transcript.py * Update all_process.py * Update extern_subprocess.py * Update all_process.py * Update all_process.py * Update asr_transcript.py * Update extern_subprocess.py * Update webui.py * Create re_matching.py * Update webui.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update all_process.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update all_process.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update all_process.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update asr_transcript.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Pack 'update' functions into a module * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update all_process.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update asr_transcript.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update extern_subprocess.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update all_process.py * Update asr_transcript.py * Update webui.py * Add files via upload * Update extern_subprocess.py * Update all_process.py * Update asr_transcript.py * Update bert_gen.py * Update extern_subprocess.py * Update preprocess_text.py * Update re_matching.py * Update resample.py * Update update_status.py * Update update_status.py * Update webui.py * Update all_process.py * Update preprocess_text.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update train_ms.py --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Stardust·减 <star_dust_chen@foxmail.com> Co-authored-by: innnky <67028263+innnky@users.noreply.github.com> * Delete all_process.py * Delete asr_transcript.py * Delete extern_subprocess.py --------- Co-authored-by: spicysama <122108331+AnyaCoder@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: innnky <67028263+innnky@users.noreply.github.com> * Create config.json * Create preprocessor_config.json * Create vocab.json * Delete emotional/wav2vec2-large-robust-12-ft-emotion-msp-dim/.gitkeep * Update emo_gen.py * Delete add_punc.py * add emotion_clustering.i * Apply Code Formatter Change * Update models.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update preprocess_text.py (#78) * Update preprocess_text.py. 检测重复以及不存在的音频 (#79) * Handle Janpanese long pronunciations (#80) * Handle Janpanese long pronunciations * Update japanese.py * Update japanese.py * Use unified phonemes for Japanese long vowel (#82) * Use an unified phoneme for Japanese long vowel `symbol.py` has not been updated to ensure compatibility with older version models. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * 增加一个按钮，点击后可以按句子切分，添加“|” (#81) * Update re_matching.py * Update webui.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Fix phonemer bug (#83) * Fix phonemer bug * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Fix long vowel handler bug (#84) * Fix long vowel handler bug * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * 加入整合包管理器的特性：长文本合成可以自定义句间段间停顿 (#85) * Update webui.py * Update re_matching.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update train_ms.py * fix' * Update cleaner.py * add en * add en * Update english.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add en * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add en * add en * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add en * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 更新 README.md * 更新 README.md * 更新 README.md * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Change phonemer to pyopenjtalk (#86) * Change phonemer to pyopenjtalk * 修改为openjtalk便于安装 --------- Co-authored-by: Stardust·减 <star_dust_chen@foxmail.com> * 更新 english.py * Fix english_bert_mock.py. (#87) * Add punctuation execptions (#88) * Add punctuation execptions * Ellipses exceptions * remove get bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bug in oldVersion. (#89) * Update requirements.txt * change to large * rollback requirements.txt * Feat: Enable 1.1.1 models using fix-ver infer. (#91) * Feat: Enable 1.1.1 models using fix-ver infer. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Add Japanese accent (high-low) (#90) * Add punctuation execptions * Ellipses exceptions * Add Japanese accent * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Do not replace iteration mark (#92) * Add punctuation execptions * Ellipses exceptions * Add Japanese accent * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Do not replace iteration mark --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Fix: fix import error in oldVersion (#93) * Refactor: reusing model loading in webui.py and server.py. (#94) * Feat: Enable using config.yml in train_ms.py (#96) * 更新 emo_gen.py * Change emo_gen.py (#97) * Fix emo_gen bugs * Add multiprocess * Fix queue (#98) * Fix emo_gen bugs * Add multiprocess * Del var * Fix queue * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Fix training bugs (#99) * Updatge cluster notebook * Fix train * Fix filename * Update infer.py (#100) * Update infer.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Add reference audio (#101) * Add reference audio * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update * Update * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Stardust·减 <star_dust_chen@foxmail.com> * Fix: fix 1.1.1-fix (#102) * Fix infer bug (#103) * Feat: Add server_fastapi.py. (#104) * Feat: Add server_fastapi.py. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix: Update requirements.txt. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Fix: requirements.txt. (#105) * Swith to deberta-v3-large (#106) * Swith to deberta-v3-large * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Feat: Update config.py. (#107) * Feat: Update config.py. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Dev fix (#108) * fix bugs when deploying * fix bugs when deploying * fix bugs when deploying * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Revert "Dev fix (#108)" (#109) This reverts commit 685e18a10498d602b1a9a26079340d11925646f0. * Dev fix (#110) * fix bugs when deploying * fix bugs when deploying * fix bugs when deploying * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix fixed bugs * fix fixed bugs * fix fixed bug 3 * fix fixed bug 4 * fix fixed bug 5 * fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Add emo vec quantizer (#111) Co-authored-by: Stardust·减 <star_dust_chen@foxmail.com> * Clean req and gitignore (#112) * Clean req and gitignore * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Switch to deberta-v2-large-japanese (#113) * Switch to deberta-v2-large-japanese * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Fix emo bugs (#114) * Fix english (#115) * Remove emo (#117) * Don't train codebook * Remove emo * Update * Update * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Merge dev into no-emo (#122) * [pre-commit.ci] pre-commit autoupdate (#95) * [pre-commit.ci] pre-commit autoupdate updates: - [github.com/astral-sh/ruff-pre-commit: v0.0.292 → v0.1.1](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.292...v0.1.1) - [github.com/psf/black: 23.9.1 → 23.10.0](https://github.com/psf/black/compare/23.9.1...23.10.0) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Don't train codebook (#116) * Update requirements.txt * Update english_bert_mock.py * Fix: server_fastapi.py (#118) * Fix: server_fastapi.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Fix: don't print debug logging. (#119) * Fix: don't print debug logging. * Feat: support emo_gen config * Fix config * Apply Code Formatter Change * 更新，修正bug (#121) * Feat: Update infer.py preprocess_text.py server_fastapi.py. * Fix resample.py. Maintain same directory structure in out_dir as in_dir. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update resample.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update server_fastapi.py to no-emo ver * Update config.py, no emo config --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: OedoSoldier <31711261+OedoSoldier@users.noreply.github.com> Co-authored-by: Stardust·减 <star_dust_chen@foxmail.com> Co-authored-by: Stardust-minus <Stardust-minus@users.noreply.github.com> * Update train_ms.py * Update latest version info (#124) --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: jiangyuxiaoxiao <atri@suzakuintsubaki.com> Co-authored-by: AkitoLiu <39857739+Akito-UzukiP@users.noreply.github.com> Co-authored-by: Stardust-minus <Stardust-minus@users.noreply.github.com> Co-authored-by: OedoSoldier <31711261+OedoSoldier@users.noreply.github.com> Co-authored-by: spicysama <122108331+AnyaCoder@users.noreply.github.com> Co-authored-by: innnky <67028263+innnky@users.noreply.github.com> Co-authored-by: YYuX-1145 <138500330+YYuX-1145@users.noreply.github.com>
2023-10-28 10:36:21 +08:00
parent c1ba4c7b3a
commit 82ae8f6928
102 changed files with 175574 additions and 1062 deletions
--- a/config.py
+++ b/config.py
@@ -0,0 +1,228 @@
+"""
+@Desc: 全局配置文件读取
+"""
+import argparse
+import yaml
+from typing import Dict, List
+import os
+import shutil
+import sys
+
+
+class Resample_config:
+    """重采样配置"""
+
+    def __init__(self, in_dir: str, out_dir: str, sampling_rate: int = 44100):
+        self.sampling_rate: int = sampling_rate  # 目标采样率
+        self.in_dir: str = in_dir  # 待处理音频目录路径
+        self.out_dir: str = out_dir  # 重采样输出路径
+
+    @classmethod
+    def from_dict(cls, dataset_path: str, data: Dict[str, any]):
+        """从字典中生成实例"""
+
+        # 不检查路径是否有效，此逻辑在resample.py中处理
+        data["in_dir"] = os.path.join(dataset_path, data["in_dir"])
+        data["out_dir"] = os.path.join(dataset_path, data["out_dir"])
+
+        return cls(**data)
+
+
+class Preprocess_text_config:
+    """数据预处理配置"""
+
+    def __init__(
+        self,
+        transcription_path: str,
+        cleaned_path: str,
+        train_path: str,
+        val_path: str,
+        config_path: str,
+        val_per_spk: int = 5,
+        max_val_total: int = 10000,
+        clean: bool = True,
+    ):
+        self.transcription_path: str = transcription_path  # 原始文本文件路径，文本格式应为{wav_path}|{speaker_name}|{language}|{text}。
+        self.cleaned_path: str = cleaned_path  # 数据清洗后文本路径，可以不填。不填则将在原始文本目录生成
+        self.train_path: str = train_path  # 训练集路径，可以不填。不填则将在原始文本目录生成
+        self.val_path: str = val_path  # 验证集路径，可以不填。不填则将在原始文本目录生成
+        self.config_path: str = config_path  # 配置文件路径
+        self.val_per_spk: int = val_per_spk  # 每个speaker的验证集条数
+        self.max_val_total: int = max_val_total  # 验证集最大条数，多于的会被截断并放到训练集中
+        self.clean: bool = clean  # 是否进行数据清洗
+
+    @classmethod
+    def from_dict(cls, dataset_path: str, data: Dict[str, any]):
+        """从字典中生成实例"""
+
+        data["transcription_path"] = os.path.join(
+            dataset_path, data["transcription_path"]
+        )
+        if data["cleaned_path"] == "" or data["cleaned_path"] is None:
+            data["cleaned_path"] = None
+        else:
+            data["cleaned_path"] = os.path.join(dataset_path, data["cleaned_path"])
+        data["train_path"] = os.path.join(dataset_path, data["train_path"])
+        data["val_path"] = os.path.join(dataset_path, data["val_path"])
+        data["config_path"] = os.path.join(dataset_path, data["config_path"])
+
+        return cls(**data)
+
+
+class Bert_gen_config:
+    """bert_gen 配置"""
+
+    def __init__(
+        self,
+        config_path: str,
+        num_processes: int = 2,
+        device: str = "cuda",
+        use_multi_device: bool = False,
+    ):
+        self.config_path = config_path
+        self.num_processes = num_processes
+        self.device = device
+        self.use_multi_device = use_multi_device
+
+    @classmethod
+    def from_dict(cls, dataset_path: str, data: Dict[str, any]):
+        data["config_path"] = os.path.join(dataset_path, data["config_path"])
+
+        return cls(**data)
+
+
+class Emo_gen_config:
+    """emo_gen 配置"""
+
+    def __init__(
+        self,
+        config_path: str,
+        num_processes: int = 2,
+        device: str = "cuda",
+    ):
+        self.config_path = config_path
+        self.num_processes = num_processes
+        self.device = device
+
+    @classmethod
+    def from_dict(cls, dataset_path: str, data: Dict[str, any]):
+        data["config_path"] = os.path.join(dataset_path, data["config_path"])
+
+        return cls(**data)
+
+
+class Train_ms_config:
+    """训练配置"""
+
+    def __init__(
+        self,
+        config_path: str,
+        env: Dict[str, any],
+        model: str,
+    ):
+        self.env = env  # 需要加载的环境变量
+        self.model = model  # 训练模型存储目录，该路径为相对于dataset_path的路径，而非项目根目录
+        self.config_path = config_path  # 配置文件路径
+
+    @classmethod
+    def from_dict(cls, dataset_path: str, data: Dict[str, any]):
+        # data["model"] = os.path.join(dataset_path, data["model"])
+        data["config_path"] = os.path.join(dataset_path, data["config_path"])
+
+        return cls(**data)
+
+
+class Webui_config:
+    """webui 配置"""
+
+    def __init__(
+        self,
+        device: str,
+        model: str,
+        config_path: str,
+        port: int = 7860,
+        share: bool = False,
+        debug: bool = False,
+    ):
+        self.device: str = device
+        self.model: str = model  # 端口号
+        self.config_path: str = config_path  # 是否公开部署，对外网开放
+        self.port: int = port  # 是否开启debug模式
+        self.share: bool = share  # 模型路径
+        self.debug: bool = debug  # 配置文件路径
+
+    @classmethod
+    def from_dict(cls, dataset_path: str, data: Dict[str, any]):
+        data["config_path"] = os.path.join(dataset_path, data["config_path"])
+        data["model"] = os.path.join(dataset_path, data["model"])
+        return cls(**data)
+
+
+class Server_config:
+    def __init__(
+        self, models: List[Dict[str, any]], port: int = 5000, device: str = "cuda"
+    ):
+        self.models: List[Dict[str, any]] = models  # 需要加载的所有模型的配置
+        self.port: int = port  # 端口号
+        self.device: str = device  # 模型默认使用设备
+
+    @classmethod
+    def from_dict(cls, data: Dict[str, any]):
+        return cls(**data)
+
+
+class Translate_config:
+    """翻译api配置"""
+
+    def __init__(self, app_key: str, secret_key: str):
+        self.app_key = app_key
+        self.secret_key = secret_key
+
+    @classmethod
+    def from_dict(cls, data: Dict[str, any]):
+        return cls(**data)
+
+
+class Config:
+    def __init__(self, config_path: str):
+        if not os.path.isfile(config_path) and os.path.isfile("default_config.yml"):
+            shutil.copy(src="default_config.yml", dst=config_path)
+            print(
+                f"已根据默认配置文件default_config.yml生成配置文件{config_path}。请按该配置文件的说明进行配置后重新运行。"
+            )
+            print("如无特殊需求，请勿修改default_config.yml或备份该文件。")
+            sys.exit(0)
+        with open(file=config_path, mode="r", encoding="utf-8") as file:
+            yaml_config: Dict[str, any] = yaml.safe_load(file.read())
+            dataset_path: str = yaml_config["dataset_path"]
+            self.dataset_path: str = dataset_path
+            self.resample_config: Resample_config = Resample_config.from_dict(
+                dataset_path, yaml_config["resample"]
+            )
+            self.preprocess_text_config: Preprocess_text_config = (
+                Preprocess_text_config.from_dict(
+                    dataset_path, yaml_config["preprocess_text"]
+                )
+            )
+            self.bert_gen_config: Bert_gen_config = Bert_gen_config.from_dict(
+                dataset_path, yaml_config["bert_gen"]
+            )
+            self.train_ms_config: Train_ms_config = Train_ms_config.from_dict(
+                dataset_path, yaml_config["train_ms"]
+            )
+            self.webui_config: Webui_config = Webui_config.from_dict(
+                dataset_path, yaml_config["webui"]
+            )
+            self.server_config: Server_config = Server_config.from_dict(
+                yaml_config["server"]
+            )
+            self.translate_config: Translate_config = Translate_config.from_dict(
+                yaml_config["translate"]
+            )
+
+
+parser = argparse.ArgumentParser()
+# 为避免与以前的config.json起冲突，将其更名如下
+parser.add_argument("-y", "--yml_config", type=str, default="config.yml")
+args, _ = parser.parse_known_args()
+config = Config(args.yml_config)