Dev no emo (#123)

* Create emo_gen.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update server.py, fix bugs in func get_text() and infer(). (#52) * Extract get_text() and infer() from webui.py. (#53) * Extract get_text() and infer() from webui.py. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * add emo emb * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * init emo gen * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * init emo * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * init emo * Delete bert/bert-base-japanese-v3 directory * Create .gitkeep * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Create add_punc.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix bug in bert_gen.py (#54) * Update README.md * fix bug in models.py (#56) * 更新 models.py * Fix japanese cleaner (#61) * 初步，睡觉明天继续写（ * 好好好放错分支了，熬夜是大忌 * [pre-commit.ci] pre-commit autoupdate (#55) * [pre-commit.ci] pre-commit autoupdate updates: - [github.com/pre-commit/pre-commit-hooks: v4.4.0 → v4.5.0](https://github.com/pre-commit/pre-commit-hooks/compare/v4.4.0...v4.5.0) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Create tokenizer_config.json * update preprocess_text.py：过滤一个音频匹配多个文本的情况 (#57) * update preprocess_text.py：过滤音频不存在的情况 (#58) * 修复日语cleaner和bert * better * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: Stardust·减 <star_dust_chen@foxmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sora <atri@suzakuintsubaki.com> * Apply Code Formatter Change * Add config.yml for global configuration. (#62) * Add config.yml for global configuration. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bug in webui.py. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Rename config.yml to default_config.yml. Add ./config.yml to gitignore. * Add config.py to parse config.yml * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update webui.py (#65) * Update webui.py: 1. Add auto translation from Chinese to Japanese. 2. Start to use config.py in webui.py to set config instead of using the command line. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Fix (#68) * 加上ー * fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update infer.py and webui.py. Supports loading and inference models of 1.1.1 version. (#66) * Update infer.py and webui.py. Supports loading and inference models of 1.1.1 version. * Update config.json * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Fix bug in translate.py (#69) * Supports loading and inference models of 1.1、1.0.1、1.0 version. (#70) * Supports loading and inference models of 1.1、1.0.1、1.0 version. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Delete useless file in OldVersion --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update japanese.py (#71) Handling JA long pronunciations * 使用配置文件配置bert_gen.py, preprocess_text.py, resample.py (#72) * Update bert_gen.py, preprocess_text.py, resample.py. Support using config.yml in these files. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update bert_gen.py * Update bert_gen.py, fix bug. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Delete bert/bert-base-japanese-v3 directory * Create config.json * Create tokenizer_config.json * Create vocab.txt * Update server.py. 支持多版本多模型 (#76) * Update server.py. 支持多版本多模型 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Dev webui (#77) * 申请pr (#75) * 2023/10/11 update 界面优化 * Update webui.py 翻译英文页面为中文 * Update train_ms.py 单卡训练 * 加入图片 * Update extern_subprocess.py * Update asr_transcript.py * Update asr_transcript.py * Update asr_transcript.py * Update extern_subprocess.py * Update asr_transcript.py * Update asr_transcript.py * Update asr_transcript.py * Update all_process.py * Update extern_subprocess.py * Update all_process.py * Update all_process.py * Update asr_transcript.py * Update extern_subprocess.py * Update webui.py * Create re_matching.py * Update webui.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update all_process.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update all_process.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update all_process.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update asr_transcript.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Pack 'update' functions into a module * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update all_process.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update asr_transcript.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update extern_subprocess.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update all_process.py * Update asr_transcript.py * Update webui.py * Add files via upload * Update extern_subprocess.py * Update all_process.py * Update asr_transcript.py * Update bert_gen.py * Update extern_subprocess.py * Update preprocess_text.py * Update re_matching.py * Update resample.py * Update update_status.py * Update update_status.py * Update webui.py * Update all_process.py * Update preprocess_text.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update train_ms.py --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Stardust·减 <star_dust_chen@foxmail.com> Co-authored-by: innnky <67028263+innnky@users.noreply.github.com> * Delete all_process.py * Delete asr_transcript.py * Delete extern_subprocess.py --------- Co-authored-by: spicysama <122108331+AnyaCoder@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: innnky <67028263+innnky@users.noreply.github.com> * Create config.json * Create preprocessor_config.json * Create vocab.json * Delete emotional/wav2vec2-large-robust-12-ft-emotion-msp-dim/.gitkeep * Update emo_gen.py * Delete add_punc.py * add emotion_clustering.i * Apply Code Formatter Change * Update models.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update preprocess_text.py (#78) * Update preprocess_text.py. 检测重复以及不存在的音频 (#79) * Handle Janpanese long pronunciations (#80) * Handle Janpanese long pronunciations * Update japanese.py * Update japanese.py * Use unified phonemes for Japanese long vowel (#82) * Use an unified phoneme for Japanese long vowel `symbol.py` has not been updated to ensure compatibility with older version models. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * 增加一个按钮，点击后可以按句子切分，添加“|” (#81) * Update re_matching.py * Update webui.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Fix phonemer bug (#83) * Fix phonemer bug * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Fix long vowel handler bug (#84) * Fix long vowel handler bug * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * 加入整合包管理器的特性：长文本合成可以自定义句间段间停顿 (#85) * Update webui.py * Update re_matching.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update train_ms.py * fix' * Update cleaner.py * add en * add en * Update english.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add en * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add en * add en * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add en * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 更新 README.md * 更新 README.md * 更新 README.md * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Change phonemer to pyopenjtalk (#86) * Change phonemer to pyopenjtalk * 修改为openjtalk便于安装 --------- Co-authored-by: Stardust·减 <star_dust_chen@foxmail.com> * 更新 english.py * Fix english_bert_mock.py. (#87) * Add punctuation execptions (#88) * Add punctuation execptions * Ellipses exceptions * remove get bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bug in oldVersion. (#89) * Update requirements.txt * change to large * rollback requirements.txt * Feat: Enable 1.1.1 models using fix-ver infer. (#91) * Feat: Enable 1.1.1 models using fix-ver infer. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Add Japanese accent (high-low) (#90) * Add punctuation execptions * Ellipses exceptions * Add Japanese accent * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Do not replace iteration mark (#92) * Add punctuation execptions * Ellipses exceptions * Add Japanese accent * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Do not replace iteration mark --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Fix: fix import error in oldVersion (#93) * Refactor: reusing model loading in webui.py and server.py. (#94) * Feat: Enable using config.yml in train_ms.py (#96) * 更新 emo_gen.py * Change emo_gen.py (#97) * Fix emo_gen bugs * Add multiprocess * Fix queue (#98) * Fix emo_gen bugs * Add multiprocess * Del var * Fix queue * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Fix training bugs (#99) * Updatge cluster notebook * Fix train * Fix filename * Update infer.py (#100) * Update infer.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Add reference audio (#101) * Add reference audio * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update * Update * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Stardust·减 <star_dust_chen@foxmail.com> * Fix: fix 1.1.1-fix (#102) * Fix infer bug (#103) * Feat: Add server_fastapi.py. (#104) * Feat: Add server_fastapi.py. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix: Update requirements.txt. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Fix: requirements.txt. (#105) * Swith to deberta-v3-large (#106) * Swith to deberta-v3-large * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Feat: Update config.py. (#107) * Feat: Update config.py. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Dev fix (#108) * fix bugs when deploying * fix bugs when deploying * fix bugs when deploying * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Revert "Dev fix (#108)" (#109) This reverts commit 685e18a10498d602b1a9a26079340d11925646f0. * Dev fix (#110) * fix bugs when deploying * fix bugs when deploying * fix bugs when deploying * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix fixed bugs * fix fixed bugs * fix fixed bug 3 * fix fixed bug 4 * fix fixed bug 5 * fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Add emo vec quantizer (#111) Co-authored-by: Stardust·减 <star_dust_chen@foxmail.com> * Clean req and gitignore (#112) * Clean req and gitignore * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Switch to deberta-v2-large-japanese (#113) * Switch to deberta-v2-large-japanese * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Fix emo bugs (#114) * Fix english (#115) * Remove emo (#117) * Don't train codebook * Remove emo * Update * Update * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Merge dev into no-emo (#122) * [pre-commit.ci] pre-commit autoupdate (#95) * [pre-commit.ci] pre-commit autoupdate updates: - [github.com/astral-sh/ruff-pre-commit: v0.0.292 → v0.1.1](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.292...v0.1.1) - [github.com/psf/black: 23.9.1 → 23.10.0](https://github.com/psf/black/compare/23.9.1...23.10.0) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Don't train codebook (#116) * Update requirements.txt * Update english_bert_mock.py * Fix: server_fastapi.py (#118) * Fix: server_fastapi.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Fix: don't print debug logging. (#119) * Fix: don't print debug logging. * Feat: support emo_gen config * Fix config * Apply Code Formatter Change * 更新，修正bug (#121) * Feat: Update infer.py preprocess_text.py server_fastapi.py. * Fix resample.py. Maintain same directory structure in out_dir as in_dir. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update resample.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update server_fastapi.py to no-emo ver * Update config.py, no emo config --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: OedoSoldier <31711261+OedoSoldier@users.noreply.github.com> Co-authored-by: Stardust·减 <star_dust_chen@foxmail.com> Co-authored-by: Stardust-minus <Stardust-minus@users.noreply.github.com> * Update train_ms.py * Update latest version info (#124) --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: jiangyuxiaoxiao <atri@suzakuintsubaki.com> Co-authored-by: AkitoLiu <39857739+Akito-UzukiP@users.noreply.github.com> Co-authored-by: Stardust-minus <Stardust-minus@users.noreply.github.com> Co-authored-by: OedoSoldier <31711261+OedoSoldier@users.noreply.github.com> Co-authored-by: spicysama <122108331+AnyaCoder@users.noreply.github.com> Co-authored-by: innnky <67028263+innnky@users.noreply.github.com> Co-authored-by: YYuX-1145 <138500330+YYuX-1145@users.noreply.github.com>
2023-10-28 10:36:21 +08:00
parent c1ba4c7b3a
commit 82ae8f6928
102 changed files with 175574 additions and 1062 deletions
--- a/text/chinese_bert.py
+++ b/text/chinese_bert.py
@@ -1,13 +1,14 @@
 import torch
 import sys
 from transformers import AutoTokenizer, AutoModelForMaskedLM
+from config import config

 tokenizer = AutoTokenizer.from_pretrained("./bert/chinese-roberta-wwm-ext-large")

 models = dict()


-def get_bert_feature(text, word2ph, device=None):
+def get_bert_feature(text, word2ph, device=config.bert_gen_config.device):
    if (
        sys.platform == "darwin"
        and torch.backends.mps.is_available()
--- a/text/cleaner.py
+++ b/text/cleaner.py
@@ -1,7 +1,7 @@
-from text import chinese, japanese, cleaned_text_to_sequence
+from text import chinese, japanese, english, cleaned_text_to_sequence


-language_module_map = {"ZH": chinese, "JP": japanese}
+language_module_map = {"ZH": chinese, "JP": japanese, "EN": english}


 def clean_text(text, language):
--- a/text/english.py
+++ b/text/english.py
@@ -171,22 +171,160 @@ def refine_syllables(syllables):
    return phonemes, tones


+import re
+import inflect
+
+_inflect = inflect.engine()
+_comma_number_re = re.compile(r"([0-9][0-9\,]+[0-9])")
+_decimal_number_re = re.compile(r"([0-9]+\.[0-9]+)")
+_pounds_re = re.compile(r"£([0-9\,]*[0-9]+)")
+_dollars_re = re.compile(r"\$([0-9\.\,]*[0-9]+)")
+_ordinal_re = re.compile(r"[0-9]+(st|nd|rd|th)")
+_number_re = re.compile(r"[0-9]+")
+
+# List of (regular expression, replacement) pairs for abbreviations:
+_abbreviations = [
+    (re.compile("\\b%s\\." % x[0], re.IGNORECASE), x[1])
+    for x in [
+        ("mrs", "misess"),
+        ("mr", "mister"),
+        ("dr", "doctor"),
+        ("st", "saint"),
+        ("co", "company"),
+        ("jr", "junior"),
+        ("maj", "major"),
+        ("gen", "general"),
+        ("drs", "doctors"),
+        ("rev", "reverend"),
+        ("lt", "lieutenant"),
+        ("hon", "honorable"),
+        ("sgt", "sergeant"),
+        ("capt", "captain"),
+        ("esq", "esquire"),
+        ("ltd", "limited"),
+        ("col", "colonel"),
+        ("ft", "fort"),
+    ]
+]
+
+
+# List of (ipa, lazy ipa) pairs:
+_lazy_ipa = [
+    (re.compile("%s" % x[0]), x[1])
+    for x in [
+        ("r", "ɹ"),
+        ("æ", "e"),
+        ("ɑ", "a"),
+        ("ɔ", "o"),
+        ("ð", "z"),
+        ("θ", "s"),
+        ("ɛ", "e"),
+        ("ɪ", "i"),
+        ("ʊ", "u"),
+        ("ʒ", "ʥ"),
+        ("ʤ", "ʥ"),
+        ("ˈ", "↓"),
+    ]
+]
+
+# List of (ipa, lazy ipa2) pairs:
+_lazy_ipa2 = [
+    (re.compile("%s" % x[0]), x[1])
+    for x in [
+        ("r", "ɹ"),
+        ("ð", "z"),
+        ("θ", "s"),
+        ("ʒ", "ʑ"),
+        ("ʤ", "dʑ"),
+        ("ˈ", "↓"),
+    ]
+]
+
+# List of (ipa, ipa2) pairs
+_ipa_to_ipa2 = [
+    (re.compile("%s" % x[0]), x[1]) for x in [("r", "ɹ"), ("ʤ", "dʒ"), ("ʧ", "tʃ")]
+]
+
+
+def _expand_dollars(m):
+    match = m.group(1)
+    parts = match.split(".")
+    if len(parts) > 2:
+        return match + " dollars"  # Unexpected format
+    dollars = int(parts[0]) if parts[0] else 0
+    cents = int(parts[1]) if len(parts) > 1 and parts[1] else 0
+    if dollars and cents:
+        dollar_unit = "dollar" if dollars == 1 else "dollars"
+        cent_unit = "cent" if cents == 1 else "cents"
+        return "%s %s, %s %s" % (dollars, dollar_unit, cents, cent_unit)
+    elif dollars:
+        dollar_unit = "dollar" if dollars == 1 else "dollars"
+        return "%s %s" % (dollars, dollar_unit)
+    elif cents:
+        cent_unit = "cent" if cents == 1 else "cents"
+        return "%s %s" % (cents, cent_unit)
+    else:
+        return "zero dollars"
+
+
+def _remove_commas(m):
+    return m.group(1).replace(",", "")
+
+
+def _expand_ordinal(m):
+    return _inflect.number_to_words(m.group(0))
+
+
+def _expand_number(m):
+    num = int(m.group(0))
+    if num > 1000 and num < 3000:
+        if num == 2000:
+            return "two thousand"
+        elif num > 2000 and num < 2010:
+            return "two thousand " + _inflect.number_to_words(num % 100)
+        elif num % 100 == 0:
+            return _inflect.number_to_words(num // 100) + " hundred"
+        else:
+            return _inflect.number_to_words(
+                num, andword="", zero="oh", group=2
+            ).replace(", ", " ")
+    else:
+        return _inflect.number_to_words(num, andword="")
+
+
+def _expand_decimal_point(m):
+    return m.group(1).replace(".", " point ")
+
+
+def normalize_numbers(text):
+    text = re.sub(_comma_number_re, _remove_commas, text)
+    text = re.sub(_pounds_re, r"\1 pounds", text)
+    text = re.sub(_dollars_re, _expand_dollars, text)
+    text = re.sub(_decimal_number_re, _expand_decimal_point, text)
+    text = re.sub(_ordinal_re, _expand_ordinal, text)
+    text = re.sub(_number_re, _expand_number, text)
+    return text
+
+
 def text_normalize(text):
-    # todo: eng text normalize
+    text = normalize_numbers(text)
    return text


 def g2p(text):
    phones = []
    tones = []
+    word2ph = []
    words = re.split(r"([,;.\-\?\!\s+])", text)
-    for w in words:
-        if w.upper() in eng_dict:
-            phns, tns = refine_syllables(eng_dict[w.upper()])
+    words = [word for word in words if word.strip() != ""]
+    for word in words:
+        if word.upper() in eng_dict:
+            phns, tns = refine_syllables(eng_dict[word.upper()])
            phones += phns
            tones += tns
+            word2ph.append(len(phns))
        else:
-            phone_list = list(filter(lambda p: p != " ", _g2p(w)))
+            phone_list = list(filter(lambda p: p != " ", _g2p(word)))
            for ph in phone_list:
                if ph in arpa:
                    ph, tn = refine_ph(ph)
@@ -195,13 +333,23 @@ def g2p(text):
                else:
                    phones.append(ph)
                    tones.append(0)
-    # todo: implement word2ph
-    word2ph = [1 for i in phones]
+            word2ph.append(len(phone_list))

    phones = [post_replace_ph(i) for i in phones]
+
+    phones = ["_"] + phones + ["_"]
+    tones = [0] + tones + [0]
+    word2ph = [1] + word2ph + [1]
+
    return phones, tones, word2ph


+def get_bert_feature(text, word2ph):
+    from text import english_bert_mock
+
+    return english_bert_mock.get_bert_feature(text, word2ph)
+
+
 if __name__ == "__main__":
    # print(get_dict())
    # print(eng_word_to_phoneme("hello"))
--- a/text/english_bert_mock.py
+++ b/text/english_bert_mock.py
@@ -1,5 +1,39 @@
 import torch
+from transformers import DebertaV2Model, DebertaV2Tokenizer
+from config import config
+import sys
+
+tokenizer = DebertaV2Tokenizer.from_pretrained("./bert/deberta-v3-large")
+
+models = dict()


-def get_bert_feature(norm_text, word2ph):
-    return torch.zeros(1024, sum(word2ph))
+def get_bert_feature(text, word2ph, device=config.bert_gen_config.device):
+    if (
+        sys.platform == "darwin"
+        and torch.backends.mps.is_available()
+        and device == "cpu"
+    ):
+        device = "mps"
+    if not device:
+        device = "cuda"
+    if device not in models.keys():
+        models[device] = DebertaV2Model.from_pretrained("./bert/deberta-v3-large").to(
+            device
+        )
+    with torch.no_grad():
+        inputs = tokenizer(text, return_tensors="pt")
+        for i in inputs:
+            inputs[i] = inputs[i].to(device)
+        res = models[device](**inputs, output_hidden_states=True)
+        res = torch.cat(res["hidden_states"][-3:-2], -1)[0].cpu()
+    # assert len(word2ph) == len(text)+2
+    word2phone = word2ph
+    phone_level_feature = []
+    for i in range(len(word2phone)):
+        repeat_feature = res[i].repeat(word2phone[i], 1)
+        phone_level_feature.append(repeat_feature)
+
+    phone_level_feature = torch.cat(phone_level_feature, dim=0)
+
+    return phone_level_feature.T
--- a/text/japanese.py
+++ b/text/japanese.py
@@ -7,367 +7,65 @@ from transformers import AutoTokenizer

 from text import punctuation, symbols

-try:
-    import MeCab
-except ImportError as e:
-    raise ImportError("Japanese requires mecab-python3 and unidic-lite.") from e
 from num2words import num2words

-_CONVRULES = [
-    # Conversion of 2 letters
-    "アァ/ a a",
-    "イィ/ i i",
-    "イェ/ i e",
-    "イャ/ y a",
-    "ウゥ/ u:",
-    "エェ/ e e",
-    "オォ/ o:",
-    "カァ/ k a:",
-    "キィ/ k i:",
-    "クゥ/ k u:",
-    "クャ/ ky a",
-    "クュ/ ky u",
-    "クョ/ ky o",
-    "ケェ/ k e:",
-    "コォ/ k o:",
-    "ガァ/ g a:",
-    "ギィ/ g i:",
-    "グゥ/ g u:",
-    "グャ/ gy a",
-    "グュ/ gy u",
-    "グョ/ gy o",
-    "ゲェ/ g e:",
-    "ゴォ/ g o:",
-    "サァ/ s a:",
-    "シィ/ sh i:",
-    "スゥ/ s u:",
-    "スャ/ sh a",
-    "スュ/ sh u",
-    "スョ/ sh o",
-    "セェ/ s e:",
-    "ソォ/ s o:",
-    "ザァ/ z a:",
-    "ジィ/ j i:",
-    "ズゥ/ z u:",
-    "ズャ/ zy a",
-    "ズュ/ zy u",
-    "ズョ/ zy o",
-    "ゼェ/ z e:",
-    "ゾォ/ z o:",
-    "タァ/ t a:",
-    "チィ/ ch i:",
-    "ツァ/ ts a",
-    "ツィ/ ts i",
-    "ツゥ/ ts u:",
-    "ツャ/ ch a",
-    "ツュ/ ch u",
-    "ツョ/ ch o",
-    "ツェ/ ts e",
-    "ツォ/ ts o",
-    "テェ/ t e:",
-    "トォ/ t o:",
-    "ダァ/ d a:",
-    "ヂィ/ j i:",
-    "ヅゥ/ d u:",
-    "ヅャ/ zy a",
-    "ヅュ/ zy u",
-    "ヅョ/ zy o",
-    "デェ/ d e:",
-    "ドォ/ d o:",
-    "ナァ/ n a:",
-    "ニィ/ n i:",
-    "ヌゥ/ n u:",
-    "ヌャ/ ny a",
-    "ヌュ/ ny u",
-    "ヌョ/ ny o",
-    "ネェ/ n e:",
-    "ノォ/ n o:",
-    "ハァ/ h a:",
-    "ヒィ/ h i:",
-    "フゥ/ f u:",
-    "フャ/ hy a",
-    "フュ/ hy u",
-    "フョ/ hy o",
-    "ヘェ/ h e:",
-    "ホォ/ h o:",
-    "バァ/ b a:",
-    "ビィ/ b i:",
-    "ブゥ/ b u:",
-    "フャ/ hy a",
-    "ブュ/ by u",
-    "フョ/ hy o",
-    "ベェ/ b e:",
-    "ボォ/ b o:",
-    "パァ/ p a:",
-    "ピィ/ p i:",
-    "プゥ/ p u:",
-    "プャ/ py a",
-    "プュ/ py u",
-    "プョ/ py o",
-    "ペェ/ p e:",
-    "ポォ/ p o:",
-    "マァ/ m a:",
-    "ミィ/ m i:",
-    "ムゥ/ m u:",
-    "ムャ/ my a",
-    "ムュ/ my u",
-    "ムョ/ my o",
-    "メェ/ m e:",
-    "モォ/ m o:",
-    "ヤァ/ y a:",
-    "ユゥ/ y u:",
-    "ユャ/ y a:",
-    "ユュ/ y u:",
-    "ユョ/ y o:",
-    "ヨォ/ y o:",
-    "ラァ/ r a:",
-    "リィ/ r i:",
-    "ルゥ/ r u:",
-    "ルャ/ ry a",
-    "ルュ/ ry u",
-    "ルョ/ ry o",
-    "レェ/ r e:",
-    "ロォ/ r o:",
-    "ワァ/ w a:",
-    "ヲォ/ o:",
-    "ディ/ d i",
-    "デェ/ d e:",
-    "デャ/ dy a",
-    "デュ/ dy u",
-    "デョ/ dy o",
-    "ティ/ t i",
-    "テェ/ t e:",
-    "テャ/ ty a",
-    "テュ/ ty u",
-    "テョ/ ty o",
-    "スィ/ s i",
-    "ズァ/ z u a",
-    "ズィ/ z i",
-    "ズゥ/ z u",
-    "ズャ/ zy a",
-    "ズュ/ zy u",
-    "ズョ/ zy o",
-    "ズェ/ z e",
-    "ズォ/ z o",
-    "キャ/ ky a",
-    "キュ/ ky u",
-    "キョ/ ky o",
-    "シャ/ sh a",
-    "シュ/ sh u",
-    "シェ/ sh e",
-    "ショ/ sh o",
-    "チャ/ ch a",
-    "チュ/ ch u",
-    "チェ/ ch e",
-    "チョ/ ch o",
-    "トゥ/ t u",
-    "トャ/ ty a",
-    "トュ/ ty u",
-    "トョ/ ty o",
-    "ドァ/ d o a",
-    "ドゥ/ d u",
-    "ドャ/ dy a",
-    "ドュ/ dy u",
-    "ドョ/ dy o",
-    "ドォ/ d o:",
-    "ニャ/ ny a",
-    "ニュ/ ny u",
-    "ニョ/ ny o",
-    "ヒャ/ hy a",
-    "ヒュ/ hy u",
-    "ヒョ/ hy o",
-    "ミャ/ my a",
-    "ミュ/ my u",
-    "ミョ/ my o",
-    "リャ/ ry a",
-    "リュ/ ry u",
-    "リョ/ ry o",
-    "ギャ/ gy a",
-    "ギュ/ gy u",
-    "ギョ/ gy o",
-    "ヂェ/ j e",
-    "ヂャ/ j a",
-    "ヂュ/ j u",
-    "ヂョ/ j o",
-    "ジェ/ j e",
-    "ジャ/ j a",
-    "ジュ/ j u",
-    "ジョ/ j o",
-    "ビャ/ by a",
-    "ビュ/ by u",
-    "ビョ/ by o",
-    "ピャ/ py a",
-    "ピュ/ py u",
-    "ピョ/ py o",
-    "ウァ/ u a",
-    "ウィ/ w i",
-    "ウェ/ w e",
-    "ウォ/ w o",
-    "ファ/ f a",
-    "フィ/ f i",
-    "フゥ/ f u",
-    "フャ/ hy a",
-    "フュ/ hy u",
-    "フョ/ hy o",
-    "フェ/ f e",
-    "フォ/ f o",
-    "ヴァ/ b a",
-    "ヴィ/ b i",
-    "ヴェ/ b e",
-    "ヴォ/ b o",
-    "ヴュ/ by u",
-    # Conversion of 1 letter
-    "ア/ a",
-    "イ/ i",
-    "ウ/ u",
-    "エ/ e",
-    "オ/ o",
-    "カ/ k a",
-    "キ/ k i",
-    "ク/ k u",
-    "ケ/ k e",
-    "コ/ k o",
-    "サ/ s a",
-    "シ/ sh i",
-    "ス/ s u",
-    "セ/ s e",
-    "ソ/ s o",
-    "タ/ t a",
-    "チ/ ch i",
-    "ツ/ ts u",
-    "テ/ t e",
-    "ト/ t o",
-    "ナ/ n a",
-    "ニ/ n i",
-    "ヌ/ n u",
-    "ネ/ n e",
-    "ノ/ n o",
-    "ハ/ h a",
-    "ヒ/ h i",
-    "フ/ f u",
-    "ヘ/ h e",
-    "ホ/ h o",
-    "マ/ m a",
-    "ミ/ m i",
-    "ム/ m u",
-    "メ/ m e",
-    "モ/ m o",
-    "ラ/ r a",
-    "リ/ r i",
-    "ル/ r u",
-    "レ/ r e",
-    "ロ/ r o",
-    "ガ/ g a",
-    "ギ/ g i",
-    "グ/ g u",
-    "ゲ/ g e",
-    "ゴ/ g o",
-    "ザ/ z a",
-    "ジ/ j i",
-    "ズ/ z u",
-    "ゼ/ z e",
-    "ゾ/ z o",
-    "ダ/ d a",
-    "ヂ/ j i",
-    "ヅ/ z u",
-    "デ/ d e",
-    "ド/ d o",
-    "バ/ b a",
-    "ビ/ b i",
-    "ブ/ b u",
-    "ベ/ b e",
-    "ボ/ b o",
-    "パ/ p a",
-    "ピ/ p i",
-    "プ/ p u",
-    "ペ/ p e",
-    "ポ/ p o",
-    "ヤ/ y a",
-    "ユ/ y u",
-    "ヨ/ y o",
-    "ワ/ w a",
-    "ヰ/ i",
-    "ヱ/ e",
-    "ヲ/ o",
-    "ン/ N",
-    "ッ/ q",
-    "ヴ/ b u",
-    "ー/:",
-    # Try converting broken text
-    "ァ/ a",
-    "ィ/ i",
-    "ゥ/ u",
-    "ェ/ e",
-    "ォ/ o",
-    "ヮ/ w a",
-    "ォ/ o",
-    # Symbols
-    "、/ ,",
-    "。/ .",
-    "！/ !",
-    "？/ ?",
-    "・/ ,",
-]
-
-_COLON_RX = re.compile(":+")
-_REJECT_RX = re.compile("[^ a-zA-Z:,.?]")
-
-
-def _makerulemap():
-    l = [tuple(x.split("/")) for x in _CONVRULES]
-    return tuple({k: v for k, v in l if len(k) == i} for i in (1, 2))
-
-
-_RULEMAP1, _RULEMAP2 = _makerulemap()
+import pyopenjtalk
+import jaconv


 def kata2phoneme(text: str) -> str:
    """Convert katakana text to phonemes."""
    text = text.strip()
+    if text == "ー":
+        return ["ー"]
+    elif text.startswith("ー"):
+        return ["ー"] + kata2phoneme(text[1:])
    res = []
+    prev = None
    while text:
-        if len(text) >= 2:
-            x = _RULEMAP2.get(text[:2])
-            if x is not None:
-                text = text[2:]
-                res += x.split(" ")[1:]
-                continue
-        x = _RULEMAP1.get(text[0])
-        if x is not None:
+        if re.match(_MARKS, text):
+            res.append(text)
            text = text[1:]
-            res += x.split(" ")[1:]
            continue
-        res.append(text[0])
-        text = text[1:]
+        if text.startswith("ー"):
+            if prev:
+                res.append(prev[-1])
+            text = text[1:]
+            continue
+        res += pyopenjtalk.g2p(text).lower().replace("cl", "q").split(" ")
+        break
    # res = _COLON_RX.sub(":", res)
    return res


-_KATAKANA = "".join(chr(ch) for ch in range(ord("ァ"), ord("ン") + 1))
-_HIRAGANA = "".join(chr(ch) for ch in range(ord("ぁ"), ord("ん") + 1))
-_HIRA2KATATRANS = str.maketrans(_HIRAGANA, _KATAKANA)
-
-
 def hira2kata(text: str) -> str:
-    text = text.translate(_HIRA2KATATRANS)
-    return text.replace("う゛", "ヴ")
+    return jaconv.hira2kata(text)


 _SYMBOL_TOKENS = set(list("・、。？！"))
 _NO_YOMI_TOKENS = set(list("「」『』―（）［］[]"))
-_TAGGER = MeCab.Tagger()
+_MARKS = re.compile(
+    r"[^A-Za-z\d\u3005\u3040-\u30ff\u4e00-\u9fff\uff11-\uff19\uff21-\uff3a\uff41-\uff5a\uff66-\uff9d]"
+)


 def text2kata(text: str) -> str:
-    parsed = _TAGGER.parse(text)
-    res = []
-    for line in parsed.split("\n"):
-        if line == "EOS":
-            break
-        parts = line.split("\t")
+    parsed = pyopenjtalk.run_frontend(text)

-        word, yomi = parts[0], parts[1]
+    res = []
+    for parts in parsed:
+        word, yomi = replace_punctuation(parts["orig"]), parts["pron"].replace("’", "")
        if yomi:
+            if re.match(_MARKS, yomi):
+                if len(word) > 1:
+                    word = [replace_punctuation(i) for i in list(word)]
+                    yomi = word
+                    res += yomi
+                    sep += word
+                    continue
+                elif word not in rep_map.keys() and word not in rep_map.values():
+                    word = ","
+                yomi = word
            res.append(yomi)
        else:
            if word in _SYMBOL_TOKENS:
@@ -381,6 +79,66 @@ def text2kata(text: str) -> str:
    return hira2kata("".join(res))


+def text2sep_kata(text: str) -> (list, list):
+    parsed = pyopenjtalk.run_frontend(text)
+
+    res = []
+    sep = []
+    for parts in parsed:
+        word, yomi = replace_punctuation(parts["orig"]), parts["pron"].replace("’", "")
+        if yomi:
+            if re.match(_MARKS, yomi):
+                if len(word) > 1:
+                    word = [replace_punctuation(i) for i in list(word)]
+                    yomi = word
+                    res += yomi
+                    sep += word
+                    continue
+                elif word not in rep_map.keys() and word not in rep_map.values():
+                    word = ","
+                yomi = word
+            res.append(yomi)
+        else:
+            if word in _SYMBOL_TOKENS:
+                res.append(word)
+            elif word in ("っ", "ッ"):
+                res.append("ッ")
+            elif word in _NO_YOMI_TOKENS:
+                pass
+            else:
+                res.append(word)
+        sep.append(word)
+    return sep, [hira2kata(i) for i in res], get_accent(parsed)
+
+
+def get_accent(parsed):
+    labels = pyopenjtalk.make_label(parsed)
+
+    phonemes = []
+    accents = []
+    for n, label in enumerate(labels):
+        phoneme = re.search(r"\-([^\+]*)\+", label).group(1)
+        if phoneme not in ["sil", "pau"]:
+            phonemes.append(phoneme.replace("cl", "q").lower())
+        else:
+            continue
+        a1 = int(re.search(r"/A:(\-?[0-9]+)\+", label).group(1))
+        a2 = int(re.search(r"\+(\d+)\+", label).group(1))
+        if re.search(r"\-([^\+]*)\+", labels[n + 1]).group(1) in ["sil", "pau"]:
+            a2_next = -1
+        else:
+            a2_next = int(re.search(r"\+(\d+)\+", labels[n + 1]).group(1))
+        # Falling
+        if a1 == 0 and a2_next == a2 + 1:
+            accents.append(-1)
+        # Rising
+        elif a2 == 1 and a2_next == 2:
+            accents.append(1)
+        else:
+            accents.append(0)
+    return list(zip(phonemes, accents))
+
+
 _ALPHASYMBOL_YOMI = {
    "#": "シャープ",
    "%": "パーセント",
@@ -503,9 +261,34 @@ rep_map = {
    "！": "!",
    "？": "?",
    "\n": ".",
-    "·": ",",
-    "、": ",",
+    "．": ".",
    "...": "…",
+    "···": "…",
+    "・・・": "…",
+    "·": ",",
+    "・": ",",
+    "、": ",",
+    "$": ".",
+    "“": "'",
+    "”": "'",
+    "‘": "'",
+    "’": "'",
+    "（": "'",
+    "）": "'",
+    "(": "'",
+    ")": "'",
+    "《": "'",
+    "》": "'",
+    "【": "'",
+    "】": "'",
+    "[": "'",
+    "]": "'",
+    "—": "-",
+    "−": "-",
+    "～": "-",
+    "~": "-",
+    "「": "'",
+    "」": "'",
 }


@@ -515,7 +298,7 @@ def replace_punctuation(text):
    replaced_text = pattern.sub(lambda x: rep_map[x.group()], text)

    replaced_text = re.sub(
-        r"[^\u3040-\u309F\u30A0-\u30FF\u4E00-\u9FFF\u3400-\u4DBF"
+        r"[^\u3040-\u309F\u30A0-\u30FF\u4E00-\u9FFF\u3400-\u4DBF\u3005"
        + "".join(punctuation)
        + r"]+",
        "",
@@ -542,44 +325,74 @@ def distribute_phone(n_phone, n_word):
    return phones_per_word


-tokenizer = AutoTokenizer.from_pretrained("./bert/bert-base-japanese-v3")
+def handle_long(sep_phonemes):
+    for i in range(len(sep_phonemes)):
+        if sep_phonemes[i][0] == "ー":
+            sep_phonemes[i][0] = sep_phonemes[i - 1][-1]
+        if "ー" in sep_phonemes[i]:
+            for j in range(len(sep_phonemes[i])):
+                if sep_phonemes[i][j] == "ー":
+                    sep_phonemes[i][j] = sep_phonemes[i][j - 1][-1]
+    return sep_phonemes
+
+
+tokenizer = AutoTokenizer.from_pretrained("./bert/deberta-v2-large-japanese")
+
+
+def align_tones(phones, tones):
+    res = []
+    for pho in phones:
+        temp = [0] * len(pho)
+        for idx, p in enumerate(pho):
+            if len(tones) == 0:
+                break
+            if p == tones[0][0]:
+                temp[idx] = tones[0][1]
+                if idx > 0:
+                    temp[idx] += temp[idx - 1]
+                tones.pop(0)
+        temp = [0] + temp
+        temp = temp[:-1]
+        if -1 in temp:
+            temp = [i + 1 for i in temp]
+        res.append(temp)
+    res = [i for j in res for i in j]
+    assert not any([i < 0 for i in res]) and not any([i > 1 for i in res])
+    return res


 def g2p(norm_text):
-    tokenized = tokenizer.tokenize(norm_text)
-    phs = []
-    ph_groups = []
-    for t in tokenized:
-        if not t.startswith("#"):
-            ph_groups.append([t])
-        else:
-            ph_groups[-1].append(t.replace("#", ""))
+    sep_text, sep_kata, acc = text2sep_kata(norm_text)
+    sep_tokenized = [tokenizer.tokenize(i) for i in sep_text]
+    sep_phonemes = handle_long([kata2phoneme(i) for i in sep_kata])
+    # 异常处理，MeCab不认识的词的话会一路传到这里来，然后炸掉。目前来看只有那些超级稀有的生僻词会出现这种情况
+    for i in sep_phonemes:
+        for j in i:
+            assert j in symbols, (sep_text, sep_kata, sep_phonemes)
+    tones = align_tones(sep_phonemes, acc)
+
    word2ph = []
-    for group in ph_groups:
-        phonemes = kata2phoneme(text2kata("".join(group)))
-        # phonemes = [i for i in phonemes if i in symbols]
-        for i in phonemes:
-            assert i in symbols, (group, norm_text, tokenized)
-        phone_len = len(phonemes)
-        word_len = len(group)
+    for token, phoneme in zip(sep_tokenized, sep_phonemes):
+        phone_len = len(phoneme)
+        word_len = len(token)

        aaa = distribute_phone(phone_len, word_len)
        word2ph += aaa
-
-        phs += phonemes
-    phones = ["_"] + phs + ["_"]
-    tones = [0 for i in phones]
+    phones = ["_"] + [j for i in sep_phonemes for j in i] + ["_"]
+    tones = [0] + tones + [0]
    word2ph = [1] + word2ph + [1]
+    assert len(phones) == len(tones)
    return phones, tones, word2ph


 if __name__ == "__main__":
-    tokenizer = AutoTokenizer.from_pretrained("./bert/bert-base-japanese-v3")
-    text = "hello,こんにちは、世界！……"
+    tokenizer = AutoTokenizer.from_pretrained("./bert/deberta-v2-large-japanese")
+    text = "hello,こんにちは、世界ー！……"
    from text.japanese_bert import get_bert_feature

    text = text_normalize(text)
    print(text)
+
    phones, tones, word2ph = g2p(text)
    bert = get_bert_feature(text, word2ph)

--- a/text/japanese_bert.py
+++ b/text/japanese_bert.py
@@ -1,13 +1,23 @@
 import torch
 from transformers import AutoTokenizer, AutoModelForMaskedLM
 import sys
+from text.japanese import text2sep_kata
+from config import config

-tokenizer = AutoTokenizer.from_pretrained("./bert/bert-base-japanese-v3")
+tokenizer = AutoTokenizer.from_pretrained("./bert/deberta-v2-large-japanese")

 models = dict()


-def get_bert_feature(text, word2ph, device=None):
+def get_bert_feature(text, word2ph, device=config.bert_gen_config.device):
+    sep_text, _, _ = text2sep_kata(text)
+    sep_tokens = [tokenizer.tokenize(t) for t in sep_text]
+    sep_ids = [tokenizer.convert_tokens_to_ids(t) for t in sep_tokens]
+    sep_ids = [2] + [item for sublist in sep_ids for item in sublist] + [3]
+    return get_bert_feature_with_token(sep_ids, word2ph, device)
+
+
+def get_bert_feature_with_token(tokens, word2ph, device=config.bert_gen_config.device):
    if (
        sys.platform == "darwin"
        and torch.backends.mps.is_available()
@@ -18,12 +28,20 @@ def get_bert_feature(text, word2ph, device=None):
        device = "cuda"
    if device not in models.keys():
        models[device] = AutoModelForMaskedLM.from_pretrained(
-            "./bert/bert-base-japanese-v3"
+            "./bert/deberta-v2-large-japanese"
        ).to(device)
    with torch.no_grad():
-        inputs = tokenizer(text, return_tensors="pt")
-        for i in inputs:
-            inputs[i] = inputs[i].to(device)
+        inputs = torch.tensor(tokens).to(device).unsqueeze(0)
+        token_type_ids = torch.zeros_like(inputs).to(device)
+        attention_mask = torch.ones_like(inputs).to(device)
+        inputs = {
+            "input_ids": inputs,
+            "token_type_ids": token_type_ids,
+            "attention_mask": attention_mask,
+        }
+
+        # for i in inputs:
+        #     inputs[i] = inputs[i].to(device)
        res = models[device](**inputs, output_hidden_states=True)
        res = torch.cat(res["hidden_states"][-3:-2], -1)[0].cpu()
    assert inputs["input_ids"].shape[-1] == len(word2ph)
--- a/text/symbols.py
+++ b/text/symbols.py
@@ -117,7 +117,7 @@ ja_symbols = [
    "z",
    "zy",
 ]
-num_ja_tones = 1
+num_ja_tones = 2

 # English
 en_symbols = [