Commit Graph

124 Commits

Author SHA1 Message Date
litagin02
a6a7a30fc6 First check repo-id is valid 2024-02-22 09:42:42 +09:00
litagin02
5b60bf705a Upload all Data/ to hub 2024-02-20 00:52:12 +09:00
litagin02
d26336bf0a Upload to hug as future 2024-02-19 15:42:08 +09:00
litagin02
e0a601737a Error handling for hug upload 2024-02-17 20:26:58 +09:00
litagin02
8b76109b75 Fix hf push 2024-02-17 11:31:35 +09:00
litagin02
ff730476e7 Feat: huggingface hub push 2024-02-17 11:01:16 +09:00
litagin02
50833cda5f Update (maybe ver 2.1) 2024-02-07 16:11:13 +09:00
litagin02
78689f6c2a Try to find best setting 2024-02-07 13:42:20 +09:00
litagin02
b2cde01d77 Fix incompatible setting 2024-02-07 13:34:45 +09:00
litagin02
0d7de644cd Update comment 2024-02-07 10:29:49 +09:00
litagin02
c81cc7b06f Add experimental speedup option for training (disable log and eval) 2024-02-06 10:21:06 +09:00
litagin02
b0bb147d44 Restore gc and emptycache when training 2024-02-05 16:02:16 +09:00
litagin02
e566ac62f3 Update 2024-02-02 21:11:23 +09:00
litagin02
d461b2e78d Merge pull request #21 from kale4eat/dev-warmup
エポックごとのスケジューリングによるウォームアップ
2024-01-09 15:24:26 +09:00
kale4eat
7d854431f3 Change scheduling by step to by epoch 2024-01-09 14:37:15 +09:00
litagin02
78a25163f3 Merge pull request #19 from kale4eat/dev-warmup
Implement warm-up
2024-01-08 21:53:48 +09:00
kale4eat
a764c1e4af Implement warm-up 2024-01-08 19:19:11 +09:00
litagin02
477b0a8eb0 Feat: give tone for sythesize (WIP for better UX) 2024-01-08 10:48:28 +09:00
litagin02
432689e29f Add preprocess log and adjust log files 2024-01-06 13:07:57 +09:00
RedRayz
bf9a0a12e7 Add --no_progress_bar option 2024-01-06 05:22:16 +09:00
RedRayz
701adfa2a5 Show total progress and don't evaluate when script starts training 2024-01-05 21:32:00 +09:00
litagin02
5b49315009 Fix config bug 2024-01-03 10:46:22 +09:00
litagin02
98ee6f4c69 Refactor: dataset_root and assets_root in configs/paths.yml 2024-01-02 22:51:17 +09:00
litagin02
f2e7c18aa9 Fix: ensure encoding=utf-8 for json 2024-01-01 09:26:36 +09:00
litagin02
4ebd7e21ea Fix colab bug 2024-01-01 00:08:56 +09:00
litagin02
ef4e82defc Refactor and add merge 2023-12-31 14:55:04 +09:00
litagin02
4108478f54 Feat: colab support 2023-12-30 19:51:05 +09:00
litagin02
618368f0b9 Merge branch 'colab' into dev 2023-12-30 18:45:53 +09:00
litagin02
a9652979cc Replace get_stdout() with SAFE_STDOUT 2023-12-30 18:45:04 +09:00
litagin02
6e8bcc2219 Fix colab check 2023-12-30 18:05:40 +09:00
litagin02
5ecf6c66bd Feat: save default style, and colab train support (maybe) 2023-12-30 17:28:07 +09:00
litagin02
35a8bb514f Fix: colab stdout error, so replace with stdout wrapper 2023-12-30 09:58:32 +09:00
litagin02
939cf3ec05 update 2023-12-30 00:46:33 +09:00
litagin02
fc2bde673a Improve WebUI and add loudness normalization 2023-12-29 18:43:14 +09:00
litagin02
5be778284b Feat: slice-and-transcribe 2023-12-27 16:16:22 +09:00
litagin02
58fab45b84 init (not checked bat script yet) 2023-12-27 05:37:46 +09:00
litagin02
c2aabe15b7 Feat: support V220 train (maybe) 2023-12-23 15:18:27 +09:00
litagin02
6c171fcea0 Support V210 train and fix some bugs and improve 2023-12-23 12:59:57 +09:00
Sihan Wang
bd4591aa2c Update train_ms.py 2023-12-20 02:56:32 +08:00
Sihan Wang
2e7c173de2 Update losses.py 2023-12-20 02:50:58 +08:00
Stardust·减
76653b5b6d Dev 2.3. (#242)
* Fix inputs of duration discriminator

* Add LSTM

* Update models.py

* Update tensorboard scalar

* Noise injection for minimizing modality gap

* Update infer.py

* support bf16 run

* del unused_para flag

* support bf16 config

* add grad clip

* fix(logger and grad):add dur grad,fix grad clip

* Update webui_preprocess.py

* Fix English G2P

* fix(bert_gen):add pass

* Pass SDP to DD

* Update webui_preprocess.py

* Update config.json

* Update webui.py

* Update chinese_bert.py

* Upload webui for deploy

* Update webui.py

* torch.save as pt not npy

* Update config.json

* add freeze emo vq

* Update webui_preprocess.py

* Fix tone_sandhi.py

* Comment up grad clip

* Fix in-place addition

* Add SLM discriminator

* Add DDP for WD

* Feat: Style text: make emotions and style similar to the style text by mixing bert (#240) (#241)

* fix:(oldVersion210) Load on demand Emotion model

* feat: update fastapi.py. 添加更多错误日志信息

* Switch pyopenjtalk to pyopenjtalk-prebuilt

* fix: update fastapi.py. 2.2 reference适配

* Update resample.py

* 修复Onnx导出的BUG (#237)

* Add files via upload

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add files via upload

* Add files via upload

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Delete attentions_onnx.py

* Delete models_onnx.py

* Add files via upload

* Add files via upload

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update __init__.py

* Update __init__.py

* Update __init__.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------



* Fix onnx

* Format export

* Feat: style-text and bert mixing (JA only)

* Ensure the same tensor shape

* Update

* update gradio version

* Fix

* Style text for chinese and english (ver 2.2)

* Style text for chinese and english (ver 2.1)

* Style text in FastAPI

* Translate style text desc in chinese

---------

Co-authored-by: litagin02 <139731664+litagin02@users.noreply.github.com>
Co-authored-by: Sora <654163754@qq.com>
Co-authored-by: Sihan Wang <wangsihan1995@gmail.com>
Co-authored-by: Ναρουσέ·μ·γιουμεμί·Χινακάννα <40709280+NaruseMioShirakana@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Remove CLAP

* Revert "Remove CLAP"

This reverts commit 62fd59bc837c580239840a2bc84b15e0663730fc.

Revert

* Remove CLAP

* bf16 audo grad cilp

* Update webui and infer utils

* Update webui.py

* Update webui.py

* Update webui-preprocess.py

* Update webui_preprocess.py

---------

Co-authored-by: Sihan Wang <wangsihan1995@gmail.com>
Co-authored-by: OedoSoldier <31711261+OedoSoldier@users.noreply.github.com>
Co-authored-by: litagin02 <139731664+litagin02@users.noreply.github.com>
Co-authored-by: Sora <654163754@qq.com>
Co-authored-by: Ναρουσέ·μ·γιουμεμί·Χινακάννα <40709280+NaruseMioShirakana@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-12-19 19:12:26 +08:00
Stardust·减
eaefc57d71 Use clap to achieve prompt controlled generation (#223)
* 快速分类音频并把yml格式结果存在训练根目录里 (#190)

* Add files via upload

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Update models.py

* Update webui.py

* Update infer.py

* Create compress_model.py

* 重新提交,更新Gradio推理UI (#193)

* Update webui.py

* Update webui.py

* 更新 train_ms.py

* 更新 models.py

* 更新 models.py

* 更新 models.py

* 更新 train_ms.py

* 更新 train_ms.py

* 更新 models.py

* Update preprocess_text.py

* Update config.json

* Update train_ms.py

* Update webui.py (#206)

* Add files via upload (#209)

* Update train_ms.py

* Update train_ms.py

* Update preprocess_text.py

* Update train_ms.py

* fix (#211)

* Update emotion_clustering.py

* Add files via upload

* Update emotion_clustering.py

* add cluster center save

* Add files via upload

* Update config.py

* Update default_config.yml

* Update config.py

* Update config.py

* Update emotion_clustering.py

* Update emotion_clustering.py

* Update config.py

* Update emotion_clustering.py

* Update emotion_clustering.py

* Update webui.py

* Update emotion_clustering.py

* Update commons.py

* Update emotion_clustering.py

* Update webui.py

* Update webui.py

* Add files via upload

* Update train_ms.py

* Update train_ms.py

* Update train_ms.py

* Update train_ms.py

* Update train_ms.py

* Update webui.py

* Update emotion_clustering.py

* Update emotion_clustering.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix default_config.yml.

* Update infer.py

* feat: support infer 2.1 models

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: support infer 2.1 models 兼容bug修复

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update train_ms.py

* Add CLAP

* Fix data loader

* Fix infer.py

* Fix webui.py

* Add prompt template

* Update clap_gen.py

* Fix wrong environ value

* Add g for dur disc

* Update clap_gen.py

* Fix multilang generation

* Update config.json

* Prompt mode

* Improve slice segments performance

* Add preprocess webui

* Update webui_preprocess.py

* Update webui_preprocess.py

* Update config.py

* Update default_config.yml

* Update config.py

* Update clap_gen.py

* Delete emo_gen.py

* Delete get_emo.py

* Delete emotional/wav2vec2-large-robust-12-ft-emotion-msp-dim directory

* Update README.md

* Update README

* Split val per lang

* Delete emotion_clustering.py

* Update default_config.yml

* Update default_config.yml

* Update config.py

* Update preprocess_text.py

* Update webui_preprocess.py

* Update defalut_config.yml

* Update webui_preprocess.py

* Update preprocess_text.py

* Random augmentation for CLAP

* Update data_utils.py

* Update preprocess_text.py

* Add vq for CLAP features to avoid overfitting

* Random dummy inputs

* Update webui.py

* Update models.py

* Update infer.py

* Apply Code Formatter Change

* Update config.json

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: YYuX-1145 <138500330+YYuX-1145@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Sora <654163754@qq.com>
Co-authored-by: Sihan Wang <wangsihan1995@gmail.com>
Co-authored-by: Stardust-minus <Stardust-minus@users.noreply.github.com>
2023-12-12 19:19:50 +08:00
欸嘿!
649b4fb9c9 Update train_ms.py (#225)
在每次epoch结束后回收显存以避免显存溢出的问题
2023-12-10 21:56:13 +08:00
Isotr0py
dec3fc0737 Auto download update, optimize dataloader num_workers (#195)
* update bert

* auto download emo

* fix typo

* fix typo

* fix bert download

* optimize code format

* remove unsued import

* fix a bug

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-11-26 20:16:25 +08:00
Stardust·减
f65b332f93 Update train_ms.py 2023-11-25 17:30:24 +08:00
Stardust·减
b186499ae9 Dev emo (#171)
* SYNC CHANGE TO EMO BRANCH (#162)

* Update README.md

* 更新 bert_models.json

* fix

* Update data_utils.py

* Update infer.py

* performance improve

* Feat: support auto split in webui (#158)

* Feat: support auto split in webui

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Fix: change /voice api to post (#160)

* Fix: change /voice api to post

* Fix: support /voice api get

* Fix: Add missing torch.cuda.empty_cache() (#161)

---------

Co-authored-by: Sora <atri@suzakuintsubaki.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Artrajz <969242373@qq.com>

* sync  (#163)

* Update README.md

* 更新 bert_models.json

* fix

* Update data_utils.py

* Update infer.py

* performance improve

* Feat: support auto split in webui (#158)

* Feat: support auto split in webui

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Fix: change /voice api to post (#160)

* Fix: change /voice api to post

* Fix: support /voice api get

* Fix: Add missing torch.cuda.empty_cache() (#161)

* del emo

* del emo

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Sora <atri@suzakuintsubaki.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Artrajz <969242373@qq.com>

* Add files via upload

* Update infer.py

* add emo

* add emo

* Update default_config.yml

* Fix slice segments GPU perf (#165)

* Fix slice segments GPU perf

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update commons.py

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Update infer.py

* Update models.py

* Update infer.py

* remove spec cache

* Update data_utils.py

* Update data_utils.py

* Update train_ms.py

* Revert "Fix slice segments GPU perf (#165)" (#169)

This reverts commit 28430fc76bc628297bb59d8f8d25100dbe46ab59.

* Update train_ms.py

* Update train_ms.py

* Update data_utils.py

* Update data_utils.py

* Update train_ms.py

* Update train_ms.py

* Update train_ms.py

* Update train_ms.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update default_config.yml

* Switch to Japanese wwm DeBERTa (#172)

* Switch to Japanese wwm DeBERTa

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Fix wrong ellipsis g2p (#173)

* Switch to Japanese wwm DeBERTa

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix ellipsis g2p

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Add files via upload

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix English phones not aligned with BERT features (#174)

* Fix English phones not aligned with BERT features

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Fix english bert gen (#175)

* Update webui.py

* Update webui.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add NCCL timeout

* Update train_ms.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update train_ms.py

* Update default_config.yml

* Update infer.py

* Update models.py

* Update train_ms.py

* Update infer.py

* Update emo_gen.py

* Feat: Support load and infer 2.0 models (#178)

* Feat: Support load and infer 2.0 models

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* 复用相同逻辑,修正静音添加错误 (#181)

* Refactor: reuse the same part of voice api.

* Fix: server_fastapi.py

* Update train_ms.py

* Update data_utils.py

* Update data_utils.py

* Update train_ms.py

* Update train_ms.py

* Update train_ms.py

* Update train_ms.py

* Update data_utils.py

* Update data_utils.py

* Add files via upload

* Update train_ms.py

* Update train_ms.py

* Update train_ms.py

* Update default_config.yml

* Update utils.py

* Update train_ms.py

* Update utils.py

* Update default_config.yml

* Update data_utils.py

* Update default_config.yml

* Update train_ms.py

* Update train_ms.py

* Update config.py

* Update utils.py

* Update train_ms.py

* Update train_ms.py

* feat: add voice mix and tone mix (#187)

* feat: add voice mix and tone mix

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Stardust·减 <star_dust_chen@foxmail.com>

* Add files via upload

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Sora <atri@suzakuintsubaki.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Artrajz <969242373@qq.com>
Co-authored-by: Leng Yue <lengyue@lengyue.me>
Co-authored-by: OedoSoldier <31711261+OedoSoldier@users.noreply.github.com>
Co-authored-by: 潮幻Mark <sunyunfei201@gmail.com>
2023-11-25 13:08:51 +08:00
Lvjinhong
31de84eb3b 更新并完善分布式训练功能 (#186)
* 修复多机训练问题

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* 更新并完善分布式训练功能

近期融合V2版本代码时发现之前修改的多机功能并不正确,仍会报错,只不过单机多卡情况下local_rank即相当于rank,感知不出
1. 修复train_ms.py中DDP初始化及.cuda绑定到local_rank上
2. 在default_config.yml配置文件中添加env变量 LOCAL_RANK,否则默认情况下会key error
3. 添加run_MnodesAndMgpus.sh,更新分布式相关说明

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-11-16 13:46:15 +08:00
pre-commit-ci[bot]
f15f083523 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2023-11-08 12:26:07 +00:00
Stardust·减
78677dde42 del emo 2023-11-08 20:25:17 +08:00
Stardust·减
33d153505a sync emo branch (#159)
* Update README.md

* 更新 bert_models.json

* fix

* Update data_utils.py

* Update infer.py

* performance improve

* Feat: support auto split in webui (#158)

* Feat: support auto split in webui

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

---------

Co-authored-by: Sora <atri@suzakuintsubaki.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-11-08 17:09:15 +08:00