修复部分日文汉字无法被推理/训练 (#148)

* Update japanese.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
南栖
2023-11-04 16:47:46 +08:00
committed by GitHub
parent 8609449b63
commit 3705de9087

View File

@@ -361,7 +361,16 @@ def align_tones(phones, tones):
return res
import pykakasi
kks = pykakasi.kakasi()
def g2p(norm_text):
result = kks.convert(norm_text)
norm_text = ""
for i in result:
norm_text += i["hira"]
sep_text, sep_kata, acc = text2sep_kata(norm_text)
sep_tokenized = [tokenizer.tokenize(i) for i in sep_text]
sep_phonemes = handle_long([kata2phoneme(i) for i in sep_kata])