Commit Graph

114 Commits

Author SHA1 Message Date
litagin02
c81cc7b06f Add experimental speedup option for training (disable log and eval) 2024-02-06 10:21:06 +09:00
litagin02
b0bb147d44 Restore gc and emptycache when training 2024-02-05 16:02:16 +09:00
litagin02
e566ac62f3 Update 2024-02-02 21:11:23 +09:00
litagin02
d461b2e78d Merge pull request #21 from kale4eat/dev-warmup
エポックごとのスケジューリングによるウォームアップ
2024-01-09 15:24:26 +09:00
kale4eat
7d854431f3 Change scheduling by step to by epoch 2024-01-09 14:37:15 +09:00
litagin02
78a25163f3 Merge pull request #19 from kale4eat/dev-warmup
Implement warm-up
2024-01-08 21:53:48 +09:00
kale4eat
a764c1e4af Implement warm-up 2024-01-08 19:19:11 +09:00
litagin02
477b0a8eb0 Feat: give tone for sythesize (WIP for better UX) 2024-01-08 10:48:28 +09:00
litagin02
432689e29f Add preprocess log and adjust log files 2024-01-06 13:07:57 +09:00
RedRayz
bf9a0a12e7 Add --no_progress_bar option 2024-01-06 05:22:16 +09:00
RedRayz
701adfa2a5 Show total progress and don't evaluate when script starts training 2024-01-05 21:32:00 +09:00
litagin02
5b49315009 Fix config bug 2024-01-03 10:46:22 +09:00
litagin02
98ee6f4c69 Refactor: dataset_root and assets_root in configs/paths.yml 2024-01-02 22:51:17 +09:00
litagin02
f2e7c18aa9 Fix: ensure encoding=utf-8 for json 2024-01-01 09:26:36 +09:00
litagin02
4ebd7e21ea Fix colab bug 2024-01-01 00:08:56 +09:00
litagin02
ef4e82defc Refactor and add merge 2023-12-31 14:55:04 +09:00
litagin02
4108478f54 Feat: colab support 2023-12-30 19:51:05 +09:00
litagin02
618368f0b9 Merge branch 'colab' into dev 2023-12-30 18:45:53 +09:00
litagin02
a9652979cc Replace get_stdout() with SAFE_STDOUT 2023-12-30 18:45:04 +09:00
litagin02
6e8bcc2219 Fix colab check 2023-12-30 18:05:40 +09:00
litagin02
5ecf6c66bd Feat: save default style, and colab train support (maybe) 2023-12-30 17:28:07 +09:00
litagin02
35a8bb514f Fix: colab stdout error, so replace with stdout wrapper 2023-12-30 09:58:32 +09:00
litagin02
939cf3ec05 update 2023-12-30 00:46:33 +09:00
litagin02
fc2bde673a Improve WebUI and add loudness normalization 2023-12-29 18:43:14 +09:00
litagin02
5be778284b Feat: slice-and-transcribe 2023-12-27 16:16:22 +09:00
litagin02
58fab45b84 init (not checked bat script yet) 2023-12-27 05:37:46 +09:00
litagin02
c2aabe15b7 Feat: support V220 train (maybe) 2023-12-23 15:18:27 +09:00
litagin02
6c171fcea0 Support V210 train and fix some bugs and improve 2023-12-23 12:59:57 +09:00
Sihan Wang
bd4591aa2c Update train_ms.py 2023-12-20 02:56:32 +08:00
Sihan Wang
2e7c173de2 Update losses.py 2023-12-20 02:50:58 +08:00
Stardust·减
76653b5b6d Dev 2.3. (#242)
* Fix inputs of duration discriminator

* Add LSTM

* Update models.py

* Update tensorboard scalar

* Noise injection for minimizing modality gap

* Update infer.py

* support bf16 run

* del unused_para flag

* support bf16 config

* add grad clip

* fix(logger and grad):add dur grad,fix grad clip

* Update webui_preprocess.py

* Fix English G2P

* fix(bert_gen):add pass

* Pass SDP to DD

* Update webui_preprocess.py

* Update config.json

* Update webui.py

* Update chinese_bert.py

* Upload webui for deploy

* Update webui.py

* torch.save as pt not npy

* Update config.json

* add freeze emo vq

* Update webui_preprocess.py

* Fix tone_sandhi.py

* Comment up grad clip

* Fix in-place addition

* Add SLM discriminator

* Add DDP for WD

* Feat: Style text: make emotions and style similar to the style text by mixing bert (#240) (#241)

* fix:(oldVersion210) Load on demand Emotion model

* feat: update fastapi.py. 添加更多错误日志信息

* Switch pyopenjtalk to pyopenjtalk-prebuilt

* fix: update fastapi.py. 2.2 reference适配

* Update resample.py

* 修复Onnx导出的BUG (#237)

* Add files via upload

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add files via upload

* Add files via upload

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Delete attentions_onnx.py

* Delete models_onnx.py

* Add files via upload

* Add files via upload

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update __init__.py

* Update __init__.py

* Update __init__.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------



* Fix onnx

* Format export

* Feat: style-text and bert mixing (JA only)

* Ensure the same tensor shape

* Update

* update gradio version

* Fix

* Style text for chinese and english (ver 2.2)

* Style text for chinese and english (ver 2.1)

* Style text in FastAPI

* Translate style text desc in chinese

---------

Co-authored-by: litagin02 <139731664+litagin02@users.noreply.github.com>
Co-authored-by: Sora <654163754@qq.com>
Co-authored-by: Sihan Wang <wangsihan1995@gmail.com>
Co-authored-by: Ναρουσέ·μ·γιουμεμί·Χινακάννα <40709280+NaruseMioShirakana@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Remove CLAP

* Revert "Remove CLAP"

This reverts commit 62fd59bc837c580239840a2bc84b15e0663730fc.

Revert

* Remove CLAP

* bf16 audo grad cilp

* Update webui and infer utils

* Update webui.py

* Update webui.py

* Update webui-preprocess.py

* Update webui_preprocess.py

---------

Co-authored-by: Sihan Wang <wangsihan1995@gmail.com>
Co-authored-by: OedoSoldier <31711261+OedoSoldier@users.noreply.github.com>
Co-authored-by: litagin02 <139731664+litagin02@users.noreply.github.com>
Co-authored-by: Sora <654163754@qq.com>
Co-authored-by: Ναρουσέ·μ·γιουμεμί·Χινακάννα <40709280+NaruseMioShirakana@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-12-19 19:12:26 +08:00
Stardust·减
eaefc57d71 Use clap to achieve prompt controlled generation (#223)
* 快速分类音频并把yml格式结果存在训练根目录里 (#190)

* Add files via upload

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Update models.py

* Update webui.py

* Update infer.py

* Create compress_model.py

* 重新提交,更新Gradio推理UI (#193)

* Update webui.py

* Update webui.py

* 更新 train_ms.py

* 更新 models.py

* 更新 models.py

* 更新 models.py

* 更新 train_ms.py

* 更新 train_ms.py

* 更新 models.py

* Update preprocess_text.py

* Update config.json

* Update train_ms.py

* Update webui.py (#206)

* Add files via upload (#209)

* Update train_ms.py

* Update train_ms.py

* Update preprocess_text.py

* Update train_ms.py

* fix (#211)

* Update emotion_clustering.py

* Add files via upload

* Update emotion_clustering.py

* add cluster center save

* Add files via upload

* Update config.py

* Update default_config.yml

* Update config.py

* Update config.py

* Update emotion_clustering.py

* Update emotion_clustering.py

* Update config.py

* Update emotion_clustering.py

* Update emotion_clustering.py

* Update webui.py

* Update emotion_clustering.py

* Update commons.py

* Update emotion_clustering.py

* Update webui.py

* Update webui.py

* Add files via upload

* Update train_ms.py

* Update train_ms.py

* Update train_ms.py

* Update train_ms.py

* Update train_ms.py

* Update webui.py

* Update emotion_clustering.py

* Update emotion_clustering.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix default_config.yml.

* Update infer.py

* feat: support infer 2.1 models

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: support infer 2.1 models 兼容bug修复

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update train_ms.py

* Add CLAP

* Fix data loader

* Fix infer.py

* Fix webui.py

* Add prompt template

* Update clap_gen.py

* Fix wrong environ value

* Add g for dur disc

* Update clap_gen.py

* Fix multilang generation

* Update config.json

* Prompt mode

* Improve slice segments performance

* Add preprocess webui

* Update webui_preprocess.py

* Update webui_preprocess.py

* Update config.py

* Update default_config.yml

* Update config.py

* Update clap_gen.py

* Delete emo_gen.py

* Delete get_emo.py

* Delete emotional/wav2vec2-large-robust-12-ft-emotion-msp-dim directory

* Update README.md

* Update README

* Split val per lang

* Delete emotion_clustering.py

* Update default_config.yml

* Update default_config.yml

* Update config.py

* Update preprocess_text.py

* Update webui_preprocess.py

* Update defalut_config.yml

* Update webui_preprocess.py

* Update preprocess_text.py

* Random augmentation for CLAP

* Update data_utils.py

* Update preprocess_text.py

* Add vq for CLAP features to avoid overfitting

* Random dummy inputs

* Update webui.py

* Update models.py

* Update infer.py

* Apply Code Formatter Change

* Update config.json

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: YYuX-1145 <138500330+YYuX-1145@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Sora <654163754@qq.com>
Co-authored-by: Sihan Wang <wangsihan1995@gmail.com>
Co-authored-by: Stardust-minus <Stardust-minus@users.noreply.github.com>
2023-12-12 19:19:50 +08:00
欸嘿!
649b4fb9c9 Update train_ms.py (#225)
在每次epoch结束后回收显存以避免显存溢出的问题
2023-12-10 21:56:13 +08:00
Isotr0py
dec3fc0737 Auto download update, optimize dataloader num_workers (#195)
* update bert

* auto download emo

* fix typo

* fix typo

* fix bert download

* optimize code format

* remove unsued import

* fix a bug

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-11-26 20:16:25 +08:00
Stardust·减
f65b332f93 Update train_ms.py 2023-11-25 17:30:24 +08:00
Stardust·减
b186499ae9 Dev emo (#171)
* SYNC CHANGE TO EMO BRANCH (#162)

* Update README.md

* 更新 bert_models.json

* fix

* Update data_utils.py

* Update infer.py

* performance improve

* Feat: support auto split in webui (#158)

* Feat: support auto split in webui

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Fix: change /voice api to post (#160)

* Fix: change /voice api to post

* Fix: support /voice api get

* Fix: Add missing torch.cuda.empty_cache() (#161)

---------

Co-authored-by: Sora <atri@suzakuintsubaki.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Artrajz <969242373@qq.com>

* sync  (#163)

* Update README.md

* 更新 bert_models.json

* fix

* Update data_utils.py

* Update infer.py

* performance improve

* Feat: support auto split in webui (#158)

* Feat: support auto split in webui

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Fix: change /voice api to post (#160)

* Fix: change /voice api to post

* Fix: support /voice api get

* Fix: Add missing torch.cuda.empty_cache() (#161)

* del emo

* del emo

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Sora <atri@suzakuintsubaki.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Artrajz <969242373@qq.com>

* Add files via upload

* Update infer.py

* add emo

* add emo

* Update default_config.yml

* Fix slice segments GPU perf (#165)

* Fix slice segments GPU perf

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update commons.py

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Update infer.py

* Update models.py

* Update infer.py

* remove spec cache

* Update data_utils.py

* Update data_utils.py

* Update train_ms.py

* Revert "Fix slice segments GPU perf (#165)" (#169)

This reverts commit 28430fc76bc628297bb59d8f8d25100dbe46ab59.

* Update train_ms.py

* Update train_ms.py

* Update data_utils.py

* Update data_utils.py

* Update train_ms.py

* Update train_ms.py

* Update train_ms.py

* Update train_ms.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update default_config.yml

* Switch to Japanese wwm DeBERTa (#172)

* Switch to Japanese wwm DeBERTa

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Fix wrong ellipsis g2p (#173)

* Switch to Japanese wwm DeBERTa

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix ellipsis g2p

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Add files via upload

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix English phones not aligned with BERT features (#174)

* Fix English phones not aligned with BERT features

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Fix english bert gen (#175)

* Update webui.py

* Update webui.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add NCCL timeout

* Update train_ms.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update train_ms.py

* Update default_config.yml

* Update infer.py

* Update models.py

* Update train_ms.py

* Update infer.py

* Update emo_gen.py

* Feat: Support load and infer 2.0 models (#178)

* Feat: Support load and infer 2.0 models

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* 复用相同逻辑,修正静音添加错误 (#181)

* Refactor: reuse the same part of voice api.

* Fix: server_fastapi.py

* Update train_ms.py

* Update data_utils.py

* Update data_utils.py

* Update train_ms.py

* Update train_ms.py

* Update train_ms.py

* Update train_ms.py

* Update data_utils.py

* Update data_utils.py

* Add files via upload

* Update train_ms.py

* Update train_ms.py

* Update train_ms.py

* Update default_config.yml

* Update utils.py

* Update train_ms.py

* Update utils.py

* Update default_config.yml

* Update data_utils.py

* Update default_config.yml

* Update train_ms.py

* Update train_ms.py

* Update config.py

* Update utils.py

* Update train_ms.py

* Update train_ms.py

* feat: add voice mix and tone mix (#187)

* feat: add voice mix and tone mix

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Stardust·减 <star_dust_chen@foxmail.com>

* Add files via upload

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Sora <atri@suzakuintsubaki.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Artrajz <969242373@qq.com>
Co-authored-by: Leng Yue <lengyue@lengyue.me>
Co-authored-by: OedoSoldier <31711261+OedoSoldier@users.noreply.github.com>
Co-authored-by: 潮幻Mark <sunyunfei201@gmail.com>
2023-11-25 13:08:51 +08:00
Lvjinhong
31de84eb3b 更新并完善分布式训练功能 (#186)
* 修复多机训练问题

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* 更新并完善分布式训练功能

近期融合V2版本代码时发现之前修改的多机功能并不正确,仍会报错,只不过单机多卡情况下local_rank即相当于rank,感知不出
1. 修复train_ms.py中DDP初始化及.cuda绑定到local_rank上
2. 在default_config.yml配置文件中添加env变量 LOCAL_RANK,否则默认情况下会key error
3. 添加run_MnodesAndMgpus.sh,更新分布式相关说明

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-11-16 13:46:15 +08:00
pre-commit-ci[bot]
f15f083523 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2023-11-08 12:26:07 +00:00
Stardust·减
78677dde42 del emo 2023-11-08 20:25:17 +08:00
Stardust·减
33d153505a sync emo branch (#159)
* Update README.md

* 更新 bert_models.json

* fix

* Update data_utils.py

* Update infer.py

* performance improve

* Feat: support auto split in webui (#158)

* Feat: support auto split in webui

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

---------

Co-authored-by: Sora <atri@suzakuintsubaki.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-11-08 17:09:15 +08:00
Stardust·减
8ea241b57a Update loss 2023-11-07 12:58:03 +08:00
Stardust·减
c864828afa add emo 2023-11-07 12:55:48 +08:00
Isotr0py
80c418e807 Add model download to train_ms.py (#150)
* download base model

* fix download

* fix download repo

* set HF default
2023-11-06 06:42:07 +08:00
Lvjinhong
74467f5f73 修复多机训练问题 (#143)
* 修复多机训练问题

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-11-02 10:38:45 +08:00
Sora
88efbed72e Update train_ms.py. Auto choose gloo backend while using Windows platform. (#134) 2023-10-30 20:31:22 +08:00
Stardust·减
82ae8f6928 Dev no emo (#123)
* Create emo_gen.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update server.py, fix bugs in func get_text() and infer(). (#52)

* Extract get_text() and infer() from webui.py. (#53)

* Extract get_text() and infer() from webui.py.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* add emo emb

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* init emo gen

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* init emo

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* init emo

* Delete bert/bert-base-japanese-v3 directory

* Create .gitkeep

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Create add_punc.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix bug in bert_gen.py (#54)

* Update README.md

* fix bug in models.py (#56)

* 更新 models.py

* Fix japanese cleaner (#61)

* 初步,睡觉明天继续写(

* 好好好放错分支了,熬夜是大忌

* [pre-commit.ci] pre-commit autoupdate (#55)

* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/pre-commit/pre-commit-hooks: v4.4.0 → v4.5.0](https://github.com/pre-commit/pre-commit-hooks/compare/v4.4.0...v4.5.0)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Create tokenizer_config.json

* update preprocess_text.py:过滤一个音频匹配多个文本的情况 (#57)

* update preprocess_text.py:过滤音频不存在的情况 (#58)

* 修复日语cleaner和bert

* better

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Stardust·减 <star_dust_chen@foxmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Sora <atri@suzakuintsubaki.com>

* Apply Code Formatter Change

* Add config.yml for global configuration. (#62)

* Add config.yml for global configuration.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix bug in webui.py.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Rename config.yml to default_config.yml. Add ./config.yml to gitignore.

* Add config.py to parse config.yml

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Update webui.py (#65)

* Update webui.py:
1. Add auto translation from Chinese to Japanese.
2. Start to use config.py in webui.py to set config instead of using the command line.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Fix (#68)

* 加上ー

* fix

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Update infer.py and webui.py.  Supports loading and inference models of 1.1.1 version. (#66)

* Update infer.py and webui.py. Supports loading and inference models of 1.1.1 version.

* Update config.json

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Fix bug in translate.py (#69)

* Supports loading and inference models of 1.1、1.0.1、1.0 version. (#70)

* Supports loading and inference models of 1.1、1.0.1、1.0 version.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Delete useless file in OldVersion

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Update japanese.py (#71)

Handling JA long pronunciations

* 使用配置文件配置bert_gen.py, preprocess_text.py, resample.py (#72)

* Update bert_gen.py, preprocess_text.py, resample.py. Support using config.yml in these files.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update bert_gen.py

* Update bert_gen.py, fix bug.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Delete bert/bert-base-japanese-v3 directory

* Create config.json

* Create tokenizer_config.json

* Create vocab.txt

* Update server.py. 支持多版本多模型 (#76)

* Update server.py. 支持多版本多模型

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Dev webui (#77)

* 申请pr (#75)

* 2023/10/11 update

界面优化

* Update webui.py

翻译英文页面为中文

* Update train_ms.py

单卡训练

* 加入图片

* Update extern_subprocess.py

* Update asr_transcript.py

* Update asr_transcript.py

* Update asr_transcript.py

* Update extern_subprocess.py

* Update asr_transcript.py

* Update asr_transcript.py

* Update asr_transcript.py

* Update all_process.py

* Update extern_subprocess.py

* Update all_process.py

* Update all_process.py

* Update asr_transcript.py

* Update extern_subprocess.py

* Update webui.py

* Create re_matching.py

* Update webui.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update all_process.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update all_process.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update all_process.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update asr_transcript.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Pack 'update' functions into a module

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update all_process.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update asr_transcript.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update extern_subprocess.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update all_process.py

* Update asr_transcript.py

* Update webui.py

* Add files via upload

* Update extern_subprocess.py

* Update all_process.py

* Update asr_transcript.py

* Update bert_gen.py

* Update extern_subprocess.py

* Update preprocess_text.py

* Update re_matching.py

* Update resample.py

* Update update_status.py

* Update update_status.py

* Update webui.py

* Update all_process.py

* Update preprocess_text.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update train_ms.py

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Stardust·减 <star_dust_chen@foxmail.com>
Co-authored-by: innnky <67028263+innnky@users.noreply.github.com>

* Delete all_process.py

* Delete asr_transcript.py

* Delete extern_subprocess.py

---------

Co-authored-by: spicysama <122108331+AnyaCoder@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: innnky <67028263+innnky@users.noreply.github.com>

* Create config.json

* Create  preprocessor_config.json

* Create vocab.json

* Delete emotional/wav2vec2-large-robust-12-ft-emotion-msp-dim/.gitkeep

* Update emo_gen.py

* Delete add_punc.py

* add emotion_clustering.i

* Apply Code Formatter Change

* Update models.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update preprocess_text.py (#78)

* Update preprocess_text.py. 检测重复以及不存在的音频 (#79)

* Handle Janpanese long pronunciations (#80)

* Handle Janpanese long pronunciations

* Update japanese.py

* Update japanese.py

* Use unified phonemes for Japanese long vowel (#82)

* Use an unified phoneme for Japanese long vowel

`symbol.py` has not been updated to ensure compatibility with older version models.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* 增加一个按钮,点击后可以按句子切分,添加“|” (#81)

* Update re_matching.py

* Update webui.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Fix phonemer bug (#83)

* Fix phonemer bug

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Fix long vowel handler bug (#84)

* Fix long vowel handler bug

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* 加入整合包管理器的特性:长文本合成可以自定义句间段间停顿 (#85)

* Update webui.py

* Update re_matching.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Update train_ms.py

* fix'

* Update cleaner.py

* add en

* add en

* Update english.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add en

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add en

* add en

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add en

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* 更新 README.md

* 更新 README.md

* 更新 README.md

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Change phonemer to pyopenjtalk (#86)

* Change phonemer to pyopenjtalk

* 修改为openjtalk便于安装

---------

Co-authored-by: Stardust·减 <star_dust_chen@foxmail.com>

* 更新 english.py

* Fix english_bert_mock.py. (#87)

* Add punctuation execptions (#88)

* Add punctuation execptions

* Ellipses exceptions

* remove get bert

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix bug in oldVersion. (#89)

* Update requirements.txt

* change to large

* rollback requirements.txt

* Feat: Enable 1.1.1 models using fix-ver infer. (#91)

* Feat: Enable 1.1.1 models using fix-ver infer.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Add Japanese accent (high-low) (#90)

* Add punctuation execptions

* Ellipses exceptions

* Add Japanese accent

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Do not replace iteration mark (#92)

* Add punctuation execptions

* Ellipses exceptions

* Add Japanese accent

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Do not replace iteration mark

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Fix: fix import error in oldVersion (#93)

* Refactor: reusing model loading in webui.py and server.py. (#94)

* Feat: Enable using config.yml in train_ms.py (#96)

* 更新 emo_gen.py

* Change emo_gen.py (#97)

* Fix emo_gen bugs

* Add multiprocess

* Fix queue (#98)

* Fix emo_gen bugs

* Add multiprocess

* Del var

* Fix queue

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Fix training bugs (#99)

* Updatge cluster notebook

* Fix train

* Fix filename

* Update infer.py (#100)

* Update infer.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Add reference audio (#101)

* Add reference audio

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update

* Update

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Stardust·减 <star_dust_chen@foxmail.com>

* Fix: fix 1.1.1-fix (#102)

* Fix infer bug (#103)

* Feat: Add server_fastapi.py. (#104)

* Feat: Add server_fastapi.py.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix: Update requirements.txt.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Fix: requirements.txt. (#105)

* Swith to deberta-v3-large (#106)

* Swith to deberta-v3-large

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Feat: Update config.py. (#107)

* Feat: Update config.py.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Dev fix (#108)

* fix bugs when deploying

* fix bugs when deploying

* fix bugs when deploying

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Revert "Dev fix (#108)" (#109)

This reverts commit 685e18a10498d602b1a9a26079340d11925646f0.

* Dev fix (#110)

* fix bugs when deploying

* fix bugs when deploying

* fix bugs when deploying

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix fixed bugs

* fix fixed bugs

* fix fixed bug 3

* fix fixed bug 4

* fix fixed bug 5

* fix

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Add emo vec quantizer (#111)

Co-authored-by: Stardust·减 <star_dust_chen@foxmail.com>

* Clean req and gitignore (#112)

* Clean req and gitignore

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Switch to deberta-v2-large-japanese (#113)

* Switch to deberta-v2-large-japanese

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Fix emo bugs (#114)

* Fix english (#115)

* Remove emo (#117)

* Don't train codebook

* Remove emo

* Update

* Update

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Merge dev into no-emo (#122)

* [pre-commit.ci] pre-commit autoupdate (#95)

* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/astral-sh/ruff-pre-commit: v0.0.292 → v0.1.1](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.292...v0.1.1)
- [github.com/psf/black: 23.9.1 → 23.10.0](https://github.com/psf/black/compare/23.9.1...23.10.0)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Don't train codebook (#116)

* Update requirements.txt

* Update english_bert_mock.py

* Fix: server_fastapi.py (#118)

* Fix: server_fastapi.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Fix: don't print debug logging. (#119)

* Fix: don't print debug logging.

* Feat: support emo_gen config

* Fix config

* Apply Code Formatter Change

* 更新,修正bug (#121)

* Feat: Update infer.py preprocess_text.py server_fastapi.py.

* Fix resample.py. Maintain same directory structure in out_dir as in_dir.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update resample.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Update server_fastapi.py to no-emo ver

* Update config.py, no emo config

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: OedoSoldier <31711261+OedoSoldier@users.noreply.github.com>
Co-authored-by: Stardust·减 <star_dust_chen@foxmail.com>
Co-authored-by: Stardust-minus <Stardust-minus@users.noreply.github.com>

* Update train_ms.py

* Update latest version info (#124)

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: jiangyuxiaoxiao <atri@suzakuintsubaki.com>
Co-authored-by: AkitoLiu <39857739+Akito-UzukiP@users.noreply.github.com>
Co-authored-by: Stardust-minus <Stardust-minus@users.noreply.github.com>
Co-authored-by: OedoSoldier <31711261+OedoSoldier@users.noreply.github.com>
Co-authored-by: spicysama <122108331+AnyaCoder@users.noreply.github.com>
Co-authored-by: innnky <67028263+innnky@users.noreply.github.com>
Co-authored-by: YYuX-1145 <138500330+YYuX-1145@users.noreply.github.com>
2023-10-28 10:36:21 +08:00
OedoSoldier
85a467c386 Update train_ms.py (#46)
Fix error when resuming training
2023-10-02 18:10:16 +08:00
Lengyue
f63acb913f Format & polish training and webui 2023-09-12 02:21:48 -04:00
Stardust-minus
53a1c55991 Apply Code Formatter Change 2023-09-08 16:13:34 +00:00
Steve Wang
82409e29ba Merge branch 'dev' into fix-resume 2023-09-08 20:26:15 +08:00