From b55ab668a68cacda2c838165219c5d9d6fbed644 Mon Sep 17 00:00:00 2001 From: litagin02 Date: Sun, 16 Jun 2024 18:03:14 +0900 Subject: [PATCH 01/12] update --- README.md | 2 +- docs/FAQ.md | 4 ++++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 1d558a2..38e7a44 100644 --- a/README.md +++ b/README.md @@ -58,7 +58,7 @@ Pythonライブラリとしてのpipでのインストールや使用例は[libr Windowsを前提としています。 -1. [このzipファイル](https://github.com/litagin02/Style-Bert-VITS2/releases/download/2.5.0/sbv2.zip)を**パスに日本語や空白が含まれない場所に**ダウンロードして展開します。 +1. [このzipファイル](https://github.com/litagin02/Style-Bert-VITS2/releases/download/2.6.0/sbv2.zip)を**パスに日本語や空白が含まれない場所に**ダウンロードして展開します。 - グラボがある方は、`Install-Style-Bert-VITS2.bat`をダブルクリックします。 - グラボがない方は、`Install-Style-Bert-VITS2-CPU.bat`をダブルクリックします。CPU版では学習はできませんが、音声合成とマージは可能です。 2. 待つと自動で必要な環境がインストールされます。 diff --git a/docs/FAQ.md b/docs/FAQ.md index 950affa..63e7ad9 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -5,6 +5,10 @@ Google Colabのノートブックは以前のバージョンのノートブックのコピーを使っていませんか？ Colabノートブックは最新のバージョンに合ったノートブックで動かすことを前提としています。ノートブック記載のバージョンを確認して、[最新のcolabノートブック](http://colab.research.google.com/github/litagin02/Style-Bert-VITS2/blob/master/colab.ipynb)（を必要ならコピーして）から使うようにしてください。 +## `ModuleNotFoundError: No module named '_socket'`と出る + +フォルダ名をインストールした時から変えていませんか？フォルダ名を変えるとパスが変わってしまい、インストール時に指定したパスと異なるためにエラーが出ます。フォルダ名を元に戻してください。 + ## 学習に時間がかかりすぎる デフォルトの100エポックは音声データ量によっては過剰な場合があります。デフォルトでは1000ステップごとにモデルが保存されるはずなので、途中で学習を中断してみて途中のもので試してみてもいいでしょう。 From e56f903450d2855e46b9ea1a560fe2c26da957e5 Mon Sep 17 00:00:00 2001 From: litagin02 Date: Sun, 16 Jun 2024 18:04:21 +0900 Subject: [PATCH 02/12] Fix --- docs/CHANGELOG.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 4b1cd16..2a01618 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -4,9 +4,11 @@ ### 新機能 モデルのマージ時に、今までの `new = (1 - weight) * A + weight * B` の他に、次を追加 + - `new = A + weight * (B - C)`: 差分マージ - `new = a * A + b * B + c * C`: 加重和マージ - `new = A + weight * B`: ヌルモデルのマージ + 差分マージは、例えばBを「Cと同じ話者だけど囁いているモデル」とすると、`B - C`が囁きベクトル的なものだと思えるので、それをAに足すことで、Aの話者が囁いているような音声を生成できるようになります。 また、加重和で`new = A - B`を作って、それをヌルモデルマージで別のモデルに足せば、実質差分マージを実現できます。また謎に`new = -A`や`new = 41 * A`等のモデルも作ることができます。 From 8e73dfbadfd25e6b07f2dbbd3caed4a7e6324b99 Mon Sep 17 00:00:00 2001 From: litagin02 Date: Sun, 16 Jun 2024 18:08:39 +0900 Subject: [PATCH 03/12] Update README --- README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 38e7a44..0d6522b 100644 --- a/README.md +++ b/README.md @@ -92,7 +92,7 @@ python initialize.py # 必要なモデルとデフォルトTTSモデルをダ エディター部分は[別リポジトリ](https://github.com/litagin02/Style-Bert-VITS2-Editor)に分かれています。 -バージョン2.2以前での音声合成WebUIは、`App.bat`をダブルクリックか、`python app.py`するとWebUIが起動します。 +バージョン2.2以前での音声合成WebUIは、`App.bat`をダブルクリックか、`python app.py`するとWebUIが起動します。または`Inference.bat`でも音声合成単独タブが開きます。 音声合成に必要なモデルファイルたちの構造は以下の通りです（手動で配置する必要はありません）。 ``` @@ -123,18 +123,18 @@ model_assets #### データセット作り -- `App.bat`をダブルクリックか`python app.py`したところの「データセット作成」タブから、音声ファイルを適切な長さにスライスし、その後に文字の書き起こしを自動で行えます。 +- `App.bat`をダブルクリックか`python app.py`したところの「データセット作成」タブから、音声ファイルを適切な長さにスライスし、その後に文字の書き起こしを自動で行えます。または`Dataset.bat`をダブルクリックでもその単独タブが開きます。 - 指示に従った後、下の「学習」タブでそのまま学習を行うことができます。 #### 学習WebUI -- `App.bat`をダブルクリックか`python app.py`して開くWebUIの「学習」タブから指示に従ってください。 +- `App.bat`をダブルクリックか`python app.py`して開くWebUIの「学習」タブから指示に従ってください。または`Train.bat`をダブルクリックでもその単独タブが開きます。 ### スタイルの生成 - デフォルトでは、デフォルトスタイル「Neutral」の他、学習フォルダのフォルダ分けに応じたスタイルが生成されます。 - それ以外の方法で手動でスタイルを作成したい人向けです。 -- `App.bat`をダブルクリックか`python app.py`して開くWebUIの「スタイル作成」タブから、音声ファイルを使ってスタイルを生成できます。 +- `App.bat`をダブルクリックか`python app.py`して開くWebUIの「スタイル作成」タブから、音声ファイルを使ってスタイルを生成できます。または`StyleVectors.bat`をダブルクリックでもその単独タブが開きます。 - 学習とは独立しているので、学習中でもできるし、学習が終わっても何度もやりなおせます（前処理は終わらせている必要があります）。 ### API Server @@ -151,8 +151,8 @@ API仕様は起動後に`/docs`にて確認ください。 ### マージ -2つのモデルを、「声質」「声の高さ」「感情表現」「テンポ」の4点で混ぜ合わせて、新しいモデルを作ることが出来ます。 -`App.bat`をダブルクリックか`python app.py`して開くWebUIの「マージ」タブから、2つのモデルを選択してマージすることができます。 +2つのモデルを、「声質」「声の高さ」「感情表現」「テンポ」の4点で混ぜ合わせて、新しいモデルを作ったり、また「あるモデルに、別の2つのモデルの差分を足す」等の操作ができます。 +`App.bat`をダブルクリックか`python app.py`して開くWebUIの「マージ」タブから、2つのモデルを選択してマージすることができます。または`Merge.bat`をダブルクリックでもその単独タブが開きます。 ### 自然性評価 From 801dc99cc43bac8c69ddbce6629806158f79e8ce Mon Sep 17 00:00:00 2001 From: litagin02 Date: Sun, 16 Jun 2024 18:33:39 +0900 Subject: [PATCH 04/12] Update --- docs/CHANGELOG.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 2a01618..b928be2 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -17,8 +17,9 @@ 囁きについて実験的に作ったヌルモデルを[こちら](https://huggingface.co/litagin/sbv2_null_models)に置いています。これをヌルモデルマージで使うことで、任意のモデルを囁きモデルにある程度は変換できます。 -### 改善？ +### 改善 +- スタイルベクトルのマージ部分のUIの改善 - WebUIの`App.bat`の起動が少し重いので、それぞれの機能を分割した`Dataset.bat`, `Inference.bat`, `Merge.bat`, `StyleVectors.bat`, `Train.bat`を追加 (今までの`App.bat`もこれまで通り使えます) ## v2.5.1 (2024-06-14) From 5dd7db10ce7c77a7d04bf3cae646c493e2b55bfa Mon Sep 17 00:00:00 2001 From: litagin02 Date: Sun, 16 Jun 2024 18:45:47 +0900 Subject: [PATCH 05/12] Require gradio>=4.32 due to render decorator --- requirements-colab.txt | 2 +- requirements.txt | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/requirements-colab.txt b/requirements-colab.txt index 160c9e5..565e788 100644 --- a/requirements-colab.txt +++ b/requirements-colab.txt @@ -1,7 +1,7 @@ cmudict cn2an g2p_en -gradio +gradio>=4.32 jieba librosa==0.9.2 loguru diff --git a/requirements.txt b/requirements.txt index 3ae7567..9ab1237 100644 --- a/requirements.txt +++ b/requirements.txt @@ -3,7 +3,7 @@ cn2an faster-whisper==0.10.1 g2p_en GPUtil -gradio +gradio>=4.32 jieba librosa==0.9.2 loguru From 52bb5dc56d819c7f44f7c43c32f83ba9d9bc3c5e Mon Sep 17 00:00:00 2001 From: litagin02 Date: Mon, 17 Jun 2024 06:54:50 +0900 Subject: [PATCH 06/12] Fix: require numpy<2 for numpy major version up --- requirements-colab.txt | 1 + requirements-infer.txt | 1 + requirements.txt | 1 + 3 files changed, 3 insertions(+) diff --git a/requirements-colab.txt b/requirements-colab.txt index 565e788..532edb2 100644 --- a/requirements-colab.txt +++ b/requirements-colab.txt @@ -6,6 +6,7 @@ jieba librosa==0.9.2 loguru num2words +numpy<2 onnxruntime pyannote.audio>=3.1.0 pyloudnorm diff --git a/requirements-infer.txt b/requirements-infer.txt index 6dc1dd4..f5cc577 100644 --- a/requirements-infer.txt +++ b/requirements-infer.txt @@ -8,6 +8,7 @@ jieba # librosa==0.9.2 loguru num2words +numpy<=2 # protobuf==4.25 psutil # punctuators diff --git a/requirements.txt b/requirements.txt index 9ab1237..8704ac3 100644 --- a/requirements.txt +++ b/requirements.txt @@ -8,6 +8,7 @@ jieba librosa==0.9.2 loguru num2words +numpy<2 protobuf==4.25 psutil punctuators From a1fcb433e72d357fdd02e7188b2fb834b641e05d Mon Sep 17 00:00:00 2001 From: litagin02 <139731664+litagin02@users.noreply.github.com> Date: Mon, 17 Jun 2024 21:49:00 +0900 Subject: [PATCH 07/12] Fix numpy<2 --- requirements-infer.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements-infer.txt b/requirements-infer.txt index f5cc577..dda14dc 100644 --- a/requirements-infer.txt +++ b/requirements-infer.txt @@ -8,7 +8,7 @@ jieba # librosa==0.9.2 loguru num2words -numpy<=2 +numpy<2 # protobuf==4.25 psutil # punctuators From 86c2a1b08724643097cef728fb702380e998bc82 Mon Sep 17 00:00:00 2001 From: liruk <34234523+liruk@users.noreply.github.com> Date: Tue, 18 Jun 2024 19:15:20 +0900 Subject: [PATCH 08/12] =?UTF-8?q?=E6=8E=A8=E8=AB=96=E6=99=82=E3=83=8C?= =?UTF-8?q?=E3=83=AB=E3=83=A2=E3=83=87=E3=83=AB=E3=83=9E=E3=83=BC=E3=82=B8?= =?UTF-8?q?(=E3=81=BE=E3=81=A0=E5=8B=95=E3=81=8B=E3=81=AA=E3=81=84?= =?UTF-8?q?=E3=82=88)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- gradio_tabs/inference.py | 122 +++++++++++++++++++++++++++++++++- style_bert_vits2/tts_model.py | 68 ++++++++++++++++++- 2 files changed, 187 insertions(+), 3 deletions(-) diff --git a/gradio_tabs/inference.py b/gradio_tabs/inference.py index 53393be..c7a8d5e 100644 --- a/gradio_tabs/inference.py +++ b/gradio_tabs/inference.py @@ -185,7 +185,12 @@ style_md = f""" - どのくらいに強さがいいかはモデルやスタイルによって異なるようです。 - 音声ファイルを入力する場合は、学習データと似た声音の話者（特に同じ性別）でないとよい効果が出ないかもしれません。 """ +voice_keys = ["dec"] +voice_pitch_keys = ["flow"] +speech_style_keys = ["enc_p"] +tempo_keys = ["sdp", "dp"] +null_models = {}#グローバルに置いてるけどもっと良い置き場所ありそう def make_interactive(): return gr.update(interactive=True, value="音声合成") @@ -228,6 +233,14 @@ def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: ): model_holder.get_model(model_name, model_path) assert model_holder.current_model is not None + #null_model_paths, + #null_voice_weights, + #null_voice_pitch_weights, + #null_speech_style_weights, + #null_tempo_weights + if len(null_models) > 0 and len(null_models["names"].keys()) > 0: + model_holder.get_null_models(null_models) + assert len(model_holder.current_null_models) > 0 wrong_tone_message = "" kata_tone: Optional[list[tuple[str, int]]] = None @@ -282,6 +295,7 @@ def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: speaker_id=speaker_id, pitch_scale=pitch_scale, intonation_scale=intonation_scale, + null_model_params = null_models ) except InvalidToneError as e: logger.error(f"Tone error: {e}") @@ -436,6 +450,113 @@ def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: inputs=[use_assist_text], outputs=[assist_text, assist_text_weight], ) + with gr.Accordion(label="ヌルモデル", open=False): + #null_voice_weights, + #null_voice_pitch_weights, + #null_speech_style_weights, + #null_tempo_weights + with gr.Row(): + style_count = gr.Number(label="作るスタイルの数", value=0, step=1) + with gr.Column(variant="panel"): + @gr.render( + inputs=[ + style_count, + ] + ) + def render_style( + style_count + ): + name_components = {} + path_components = {} + weight_components = {} + pitch_components = {} + style_components = {} + tempo_components = {} + for i in range(style_count): + with gr.Row(): + null_model_name = gr.Dropdown( + label="モデル一覧", + choices=model_names, + key=f"null_model_name_{i}", + value=model_names[initial_id], + interactive=True + ) + null_model_path = gr.Dropdown( + label="モデルファイル", + choices=initial_pth_files, + key=f"null_model_path_{i}", + value=initial_pth_files[0], + interactive=True + ) + null_voice_weights = gr.Slider( + minimum=0, + maximum=1, + value=1, + step=0.1, + key=f"null_voice_weights_{i}", + label="声質", + interactive=True + ) + null_voice_pitch_weights = gr.Slider( + minimum=0, + maximum=1, + value=1, + step=0.1, + key=f"null_voice_pitch_weights_{i}", + label="声の高さ", + interactive=True + ) + null_speech_style_weights = gr.Slider( + minimum=0, + maximum=1, + value=1, + step=0.1, + key=f"null_speech_style_weights_{i}", + label="話し方", + interactive=True + ) + null_tempo_weights = gr.Slider( + minimum=0, + maximum=1, + value=1, + step=0.1, + key=f"null_tempo_weights_{i}", + label="テンポ", + interactive=True + ) + null_model_name.change( + model_holder.update_model_files_for_gradio, + inputs=[null_model_name], + outputs=[null_model_path], + ) + null_model_path.change(make_non_interactive, outputs=[tts_button]) + #もっといい方法ありそう + name_components[str(i)]=(null_model_name) + path_components[str(i)]=(null_model_path) + weight_components[str(i)]=(null_voice_weights) + pitch_components[str(i)]=(null_voice_pitch_weights) + style_components[str(i)]=(null_speech_style_weights) + tempo_components[str(i)]=(null_tempo_weights) + null_models["names"]=(name_components) + null_models["paths"]=(path_components) + null_models["weights"]=(weight_components) + null_models["pitchs"]=(pitch_components) + null_models["styles"]=(style_components) + null_models["tempos"]=(tempo_components) + + add_btn = gr.Button("ヌルモデルを増やす") + del_btn = gr.Button("ヌルモデルを減らす") + add_btn.click( + lambda x: x + 1, + inputs=[style_count], + outputs=[style_count], + ) + del_btn.click( + lambda x: x - 1 if x > 0 else 0, + inputs=[style_count], + outputs=[style_count], + ) + with gr.Column(): with gr.Accordion("スタイルについて詳細", open=False): gr.Markdown(style_md) @@ -524,7 +645,6 @@ def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: return app - if __name__ == "__main__": from config import get_path_config import torch diff --git a/style_bert_vits2/tts_model.py b/style_bert_vits2/tts_model.py index 6df8394..bb38b3a 100644 --- a/style_bert_vits2/tts_model.py +++ b/style_bert_vits2/tts_model.py @@ -61,6 +61,7 @@ class TTSModel: self.model_path: Path = model_path self.device: str = device + self.null_model_params: dict[str, dict[str,Any]] = {} # ハイパーパラメータの Pydantic モデルが直接指定された if isinstance(config_path, HyperParameters): @@ -113,6 +114,36 @@ class TTSModel: device=self.device, hps=self.hyper_parameters, ) + if(len(self.null_model_params.keys())==0): + return + + for index, null_model in enumerate(self.null_model_params["names"].keys()): + null_model_add = get_net_g( + model_path=str(self.null_model_params["paths"][str(index)].value), + version=self.hyper_parameters.version, + device=self.device, + hps=self.hyper_parameters, + ) + #愚直。もっと上手い方法ありそう + params = zip(self.__net_g.dec.parameters(), null_model_add.dec.parameters()) + for v in params: + v[0].data.add(v[1].data,alpha=self.null_model_params["weights"][str(index)].value) + + params = zip(self.__net_g.flow.parameters(), null_model_add.flow.parameters()) + for v in params: + v[0].data.add(v[1].data,alpha=self.null_model_params["pitchs"][str(index)].value) + + params = zip(self.__net_g.enc_p.parameters(), null_model_add.enc_p.parameters()) + for v in params: + v[0].data.add(v[1].data,alpha=self.null_model_params["styles"][str(index)].value) + #テンポはsdpとdp二つあるからとりあえずどっちも足す + params = zip(self.__net_g.sdp.parameters(), null_model_add.sdp.parameters()) + for v in params: + v[0].data.add(v[1].data,alpha=self.null_model_params["tempos"][str(index)].value) + params = zip(self.__net_g.dp.parameters(), null_model_add.dp.parameters()) + for v in params: + v[0].data.add(v[1].data,alpha=self.null_model_params["tempos"][str(index)].value) + def __get_style_vector(self, style_id: int, weight: float = 1.0) -> NDArray[Any]: """ @@ -227,6 +258,7 @@ class TTSModel: given_tone: Optional[list[int]] = None, pitch_scale: float = 1.0, intonation_scale: float = 1.0, + null_model_params: dict[str,dict[str,Any]] = {} ) -> tuple[int, NDArray[Any]]: """ テキストから音声を合成する。 @@ -251,7 +283,7 @@ class TTSModel: given_tone (Optional[list[int]], optional): アクセントのトーンのリスト. Defaults to None. pitch_scale (float, optional): ピッチの高さ (1.0 から変更すると若干音質が低下する). Defaults to 1.0. intonation_scale (float, optional): 抑揚の平均からの変化幅 (1.0 から変更すると若干音質が低下する). Defaults to 1.0. - + null_model_params(dict[str,dict[str,gr.Component],optional):推論時に使用するヌルモデルの名前、重みのdictが入ったdict。 Returns: tuple[int, NDArray[Any]]: サンプリングレートと音声データ (16bit PCM) """ @@ -265,7 +297,10 @@ class TTSModel: reference_audio_path = None if assist_text == "" or not use_assist_text: assist_text = None - + if null_model_params is not {}: + self.null_model_params = null_model_params + else: + self.null_model_params = {} if self.__net_g is None: self.load() assert self.__net_g is not None @@ -372,6 +407,8 @@ class TTSModelHolder: self.device: str = device self.model_files_dict: dict[str, list[Path]] = {} self.current_model: Optional[TTSModel] = None + self.current_null_models: dict[str,TTSModel] = {} + self.null_models_params: dict[str,dict[str,Any]] = {} self.model_names: list[str] = [] self.models_info: list[TTSModelInfo] = [] self.refresh() @@ -384,6 +421,8 @@ class TTSModelHolder: self.model_files_dict = {} self.model_names = [] self.current_model = None + self.current_null_models = {} + self.null_models_params = {} self.models_info = [] model_dirs = [d for d in self.root_dir.iterdir() if d.is_dir()] @@ -445,6 +484,29 @@ class TTSModelHolder: ) return self.current_model + def get_null_models(self, null_model_index:dict[str,dict[str,Any]]) -> dict[str, TTSModel]: + """ + get_modelをヌルモデル用に改変。複数まとめて使うかも知れないのでdictで戻す + + """ + self.current_null_models = {} + self.null_models_params = null_model_index + for index, value in enumerate(null_model_index["names"]): + model_path = Path(null_model_index["paths"][str(index)].value) + model_name = null_model_index["names"][str(index)].value + if model_name not in self.model_files_dict: + raise ValueError(f"Model `{model_name}` is not found") + if model_path not in self.model_files_dict[model_name]: + raise ValueError(f"Model file `{model_path}` is not found") + + if len(self.current_null_models) == 0 or model_path not in self.current_null_models.keys(): + self.current_null_models[model_name] = TTSModel( + model_path=model_path, + config_path=self.root_dir / model_name / "config.json", + style_vec_path=self.root_dir / model_name / "style_vectors.npy", + device=self.device, + ) + return self.current_null_models def get_model_for_gradio(self, model_name: str, model_path_str: str): import gradio as gr @@ -474,6 +536,8 @@ class TTSModelHolder: ) speakers = list(self.current_model.spk2id.keys()) styles = list(self.current_model.style2id.keys()) + #if len(null_models.keys())!=0: + # self.get_null_models(null_models) return ( gr.Dropdown(choices=styles, value=styles[0]), # type: ignore gr.Button(interactive=True, value="音声合成"), From fb99a776ce17ef97a736575647961afbf5d0549e Mon Sep 17 00:00:00 2001 From: liruk <34234523+liruk@users.noreply.github.com> Date: Wed, 19 Jun 2024 18:31:27 +0900 Subject: [PATCH 09/12] =?UTF-8?q?=E6=8E=A8=E8=AB=96=E6=99=82=E3=83=8C?= =?UTF-8?q?=E3=83=AB=E3=83=A2=E3=83=87=E3=83=AB=E3=83=9E=E3=83=BC=E3=82=B8?= =?UTF-8?q?(=E3=81=A7=E3=81=8D=E3=82=8B=E3=81=91=E3=81=A9=E3=83=90?= =?UTF-8?q?=E3=82=B0=E3=83=90=E3=82=B0)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- gradio_tabs/inference.py | 110 +++++++++++++++++++++------------- style_bert_vits2/tts_model.py | 48 +++++---------- 2 files changed, 81 insertions(+), 77 deletions(-) diff --git a/gradio_tabs/inference.py b/gradio_tabs/inference.py index c7a8d5e..38c2af6 100644 --- a/gradio_tabs/inference.py +++ b/gradio_tabs/inference.py @@ -1,6 +1,6 @@ import datetime import json -from typing import Optional +from typing import Optional, Any, Union import gradio as gr @@ -190,8 +190,6 @@ voice_pitch_keys = ["flow"] speech_style_keys = ["enc_p"] tempo_keys = ["sdp", "dp"] -null_models = {}#グローバルに置いてるけどもっと良い置き場所ありそう - def make_interactive(): return gr.update(interactive=True, value="音声合成") @@ -205,7 +203,19 @@ def gr_util(item): return (gr.update(visible=True), gr.Audio(visible=False, value=None)) else: return (gr.update(visible=False), gr.update(visible=True)) - +def change_null_model_row(null_model_index:int, null_model_name:str, null_model_path:str,null_voice_weights:float, + null_voice_pitch_weights:float, null_speech_style_weights:float,null_tempo_weights:float, + null_models:dict[int,dict[str, Any]]): + mid_result={} + mid_result["name"]=null_model_name + mid_result["path"]=null_model_path + mid_result["weight"]=null_tempo_weights + mid_result["pitch"]=null_voice_pitch_weights + mid_result["style"]=null_speech_style_weights + mid_result["tempo"]=null_tempo_weights + null_models[null_model_index] = mid_result + result = null_models + return result def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: def tts_fn( @@ -230,17 +240,14 @@ def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: speaker, pitch_scale, intonation_scale, + null_models:dict[int, dict[str, Union[str, float]]] ): model_holder.get_model(model_name, model_path) assert model_holder.current_model is not None - #null_model_paths, - #null_voice_weights, - #null_voice_pitch_weights, - #null_speech_style_weights, - #null_tempo_weights - if len(null_models) > 0 and len(null_models["names"].keys()) > 0: + + if len(null_models.keys()) > 0: model_holder.get_null_models(null_models) - assert len(model_holder.current_null_models) > 0 + assert len(model_holder.null_models_params.keys()) > 0 wrong_tone_message = "" kata_tone: Optional[list[tuple[str, int]]] = None @@ -337,6 +344,7 @@ def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: with gr.Blocks(theme=GRADIO_THEME) as app: gr.Markdown(initial_md) gr.Markdown(terms_of_use_md) + null_models = gr.State({}) with gr.Accordion(label="使い方", open=False): gr.Markdown(how_to_md) with gr.Row(): @@ -451,29 +459,24 @@ def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: outputs=[assist_text, assist_text_weight], ) with gr.Accordion(label="ヌルモデル", open=False): - #null_voice_weights, - #null_voice_pitch_weights, - #null_speech_style_weights, - #null_tempo_weights with gr.Row(): - style_count = gr.Number(label="作るスタイルの数", value=0, step=1) + null_models_count = gr.Number(label="ヌルモデルの数", value=0, step=1) with gr.Column(variant="panel"): @gr.render( inputs=[ - style_count, + null_models_count, ] ) def render_style( - style_count + null_models_count:int, ): - name_components = {} - path_components = {} - weight_components = {} - pitch_components = {} - style_components = {} - tempo_components = {} - for i in range(style_count): + for i in range(0, null_models_count): with gr.Row(): + null_model_index = gr.Number( + value=i, + key=f"null_model_index_{i}", + visible=False + ) null_model_name = gr.Dropdown( label="モデル一覧", choices=model_names, @@ -530,31 +533,51 @@ def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: outputs=[null_model_path], ) null_model_path.change(make_non_interactive, outputs=[tts_button]) - #もっといい方法ありそう - name_components[str(i)]=(null_model_name) - path_components[str(i)]=(null_model_path) - weight_components[str(i)]=(null_voice_weights) - pitch_components[str(i)]=(null_voice_pitch_weights) - style_components[str(i)]=(null_speech_style_weights) - tempo_components[str(i)]=(null_tempo_weights) - null_models["names"]=(name_components) - null_models["paths"]=(path_components) - null_models["weights"]=(weight_components) - null_models["pitchs"]=(pitch_components) - null_models["styles"]=(style_components) - null_models["tempos"]=(tempo_components) - + #愚直すぎるのでもう少しなんとかしたい + null_model_path.change(change_null_model_row, + inputs=[null_model_index, null_model_name, null_model_path,null_voice_weights, + null_voice_pitch_weights, null_speech_style_weights,null_tempo_weights, + null_models], + outputs=[null_models] + ) + null_voice_weights.change(change_null_model_row, + inputs=[null_model_index, null_model_name, null_model_path,null_voice_weights, + null_voice_pitch_weights, null_speech_style_weights,null_tempo_weights, + null_models], + outputs=[null_models] + ) + null_voice_pitch_weights.change(change_null_model_row, + inputs=[null_model_index, null_model_name, null_model_path,null_voice_weights, + null_voice_pitch_weights, null_speech_style_weights,null_tempo_weights, + null_models], + outputs=[null_models] + ) + null_speech_style_weights.change(change_null_model_row, + inputs=[null_model_index, null_model_name, null_model_path,null_voice_weights, + null_voice_pitch_weights, null_speech_style_weights,null_tempo_weights, + null_models], + outputs=[null_models] + ) + null_tempo_weights.change(change_null_model_row, + inputs=[null_model_index, null_model_name, null_model_path,null_voice_weights, + null_voice_pitch_weights, null_speech_style_weights,null_tempo_weights, + null_models], + outputs=[null_models] + ) + if null_models_count < len(null_models.value.keys()): + for i in range(null_models_count ,len(null_models.value.keys())): + _ = null_models.value.pop(i, None) add_btn = gr.Button("ヌルモデルを増やす") del_btn = gr.Button("ヌルモデルを減らす") add_btn.click( lambda x: x + 1, - inputs=[style_count], - outputs=[style_count], + inputs=[null_models_count], + outputs=[null_models_count], ) del_btn.click( lambda x: x - 1 if x > 0 else 0, - inputs=[style_count], - outputs=[style_count], + inputs=[null_models_count], + outputs=[null_models_count], ) with gr.Column(): @@ -614,6 +637,7 @@ def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: speaker, pitch_scale, intonation_scale, + null_models ], outputs=[text_output, audio_output, tone], ) diff --git a/style_bert_vits2/tts_model.py b/style_bert_vits2/tts_model.py index bb38b3a..f8a387b 100644 --- a/style_bert_vits2/tts_model.py +++ b/style_bert_vits2/tts_model.py @@ -61,7 +61,7 @@ class TTSModel: self.model_path: Path = model_path self.device: str = device - self.null_model_params: dict[str, dict[str,Any]] = {} + self.null_model_params: dict[int, dict[str,Union[float, str]]] = {} # ハイパーパラメータの Pydantic モデルが直接指定された if isinstance(config_path, HyperParameters): @@ -117,32 +117,32 @@ class TTSModel: if(len(self.null_model_params.keys())==0): return - for index, null_model in enumerate(self.null_model_params["names"].keys()): + for index, null_model in enumerate(self.null_model_params.keys()): null_model_add = get_net_g( - model_path=str(self.null_model_params["paths"][str(index)].value), + model_path=str(self.null_model_params[index]["path"]), version=self.hyper_parameters.version, device=self.device, hps=self.hyper_parameters, ) #愚直。もっと上手い方法ありそう + print(str(self.null_model_params[index]["weight"])) params = zip(self.__net_g.dec.parameters(), null_model_add.dec.parameters()) for v in params: - v[0].data.add(v[1].data,alpha=self.null_model_params["weights"][str(index)].value) - + v[0].data.add_(v[1].data,alpha=float(self.null_model_params[index]["weight"])) params = zip(self.__net_g.flow.parameters(), null_model_add.flow.parameters()) for v in params: - v[0].data.add(v[1].data,alpha=self.null_model_params["pitchs"][str(index)].value) + v[0].data.add_(v[1].data,alpha=float(self.null_model_params[index]["pitch"])) params = zip(self.__net_g.enc_p.parameters(), null_model_add.enc_p.parameters()) for v in params: - v[0].data.add(v[1].data,alpha=self.null_model_params["styles"][str(index)].value) + v[0].data.add_(v[1].data,alpha=float(self.null_model_params[index]["style"])) #テンポはsdpとdp二つあるからとりあえずどっちも足す params = zip(self.__net_g.sdp.parameters(), null_model_add.sdp.parameters()) for v in params: - v[0].data.add(v[1].data,alpha=self.null_model_params["tempos"][str(index)].value) + v[0].data.add_(v[1].data,alpha=float(self.null_model_params[index]["tempo"])) params = zip(self.__net_g.dp.parameters(), null_model_add.dp.parameters()) for v in params: - v[0].data.add(v[1].data,alpha=self.null_model_params["tempos"][str(index)].value) + v[0].data.add_(v[1].data,alpha=float(self.null_model_params[index]["tempo"])) def __get_style_vector(self, style_id: int, weight: float = 1.0) -> NDArray[Any]: @@ -258,7 +258,7 @@ class TTSModel: given_tone: Optional[list[int]] = None, pitch_scale: float = 1.0, intonation_scale: float = 1.0, - null_model_params: dict[str,dict[str,Any]] = {} + null_model_params: dict[int,dict[str,Union[str, float]]] = {} ) -> tuple[int, NDArray[Any]]: """ テキストから音声を合成する。 @@ -283,7 +283,7 @@ class TTSModel: given_tone (Optional[list[int]], optional): アクセントのトーンのリスト. Defaults to None. pitch_scale (float, optional): ピッチの高さ (1.0 から変更すると若干音質が低下する). Defaults to 1.0. intonation_scale (float, optional): 抑揚の平均からの変化幅 (1.0 から変更すると若干音質が低下する). Defaults to 1.0. - null_model_params(dict[str,dict[str,gr.Component],optional):推論時に使用するヌルモデルの名前、重みのdictが入ったdict。 + null_model_params(dict[int,dict[str,Union[str,float]],optional):推論時に使用するヌルモデルの名前、適用割合のdictが入ったdict。 Returns: tuple[int, NDArray[Any]]: サンプリングレートと音声データ (16bit PCM) """ @@ -407,8 +407,7 @@ class TTSModelHolder: self.device: str = device self.model_files_dict: dict[str, list[Path]] = {} self.current_model: Optional[TTSModel] = None - self.current_null_models: dict[str,TTSModel] = {} - self.null_models_params: dict[str,dict[str,Any]] = {} + self.null_models_params: dict[int,dict[str,Union[str, float]]] = {} self.model_names: list[str] = [] self.models_info: list[TTSModelInfo] = [] self.refresh() @@ -421,7 +420,6 @@ class TTSModelHolder: self.model_files_dict = {} self.model_names = [] self.current_model = None - self.current_null_models = {} self.null_models_params = {} self.models_info = [] @@ -484,29 +482,13 @@ class TTSModelHolder: ) return self.current_model - def get_null_models(self, null_model_index:dict[str,dict[str,Any]]) -> dict[str, TTSModel]: + def get_null_models(self, null_model_index:dict[int,dict[str,Union[str, float]]]) -> dict[int, dict[str, Union[str, float]]]: """ get_modelをヌルモデル用に改変。複数まとめて使うかも知れないのでdictで戻す """ - self.current_null_models = {} self.null_models_params = null_model_index - for index, value in enumerate(null_model_index["names"]): - model_path = Path(null_model_index["paths"][str(index)].value) - model_name = null_model_index["names"][str(index)].value - if model_name not in self.model_files_dict: - raise ValueError(f"Model `{model_name}` is not found") - if model_path not in self.model_files_dict[model_name]: - raise ValueError(f"Model file `{model_path}` is not found") - - if len(self.current_null_models) == 0 or model_path not in self.current_null_models.keys(): - self.current_null_models[model_name] = TTSModel( - model_path=model_path, - config_path=self.root_dir / model_name / "config.json", - style_vec_path=self.root_dir / model_name / "style_vectors.npy", - device=self.device, - ) - return self.current_null_models + return self.null_models_params def get_model_for_gradio(self, model_name: str, model_path_str: str): import gradio as gr @@ -536,8 +518,6 @@ class TTSModelHolder: ) speakers = list(self.current_model.spk2id.keys()) styles = list(self.current_model.style2id.keys()) - #if len(null_models.keys())!=0: - # self.get_null_models(null_models) return ( gr.Dropdown(choices=styles, value=styles[0]), # type: ignore gr.Button(interactive=True, value="音声合成"), From c957cab1e55b0fb2f946d1b072f5b38485c2e8d3 Mon Sep 17 00:00:00 2001 From: liruk <34234523+liruk@users.noreply.github.com> Date: Thu, 20 Jun 2024 19:42:16 +0900 Subject: [PATCH 10/12] =?UTF-8?q?nullmodel=E6=95=B0=E3=82=92=E5=A2=97?= =?UTF-8?q?=E6=B8=9B=E3=81=95=E3=81=9B=E3=81=9F=E3=81=A8=E3=81=8D=E3=81=AE?= =?UTF-8?q?=E5=95=8F=E9=A1=8C=E3=82=92=E4=B8=80=E9=83=A8=E4=BF=AE=E6=AD=A3?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 1→0にした時の問題は未修正 --- gradio_tabs/inference.py | 32 +++++++++++++++++++++++++++----- 1 file changed, 27 insertions(+), 5 deletions(-) diff --git a/gradio_tabs/inference.py b/gradio_tabs/inference.py index 38c2af6..324b72c 100644 --- a/gradio_tabs/inference.py +++ b/gradio_tabs/inference.py @@ -203,9 +203,12 @@ def gr_util(item): return (gr.update(visible=True), gr.Audio(visible=False, value=None)) else: return (gr.update(visible=False), gr.update(visible=True)) + +null_models_frame = 0 def change_null_model_row(null_model_index:int, null_model_name:str, null_model_path:str,null_voice_weights:float, null_voice_pitch_weights:float, null_speech_style_weights:float,null_tempo_weights:float, null_models:dict[int,dict[str, Any]]): + logger.debug("change_null_model_row:sta"+str(null_models)) mid_result={} mid_result["name"]=null_model_name mid_result["path"]=null_model_path @@ -214,7 +217,12 @@ def change_null_model_row(null_model_index:int, null_model_name:str, null_model_ mid_result["style"]=null_speech_style_weights mid_result["tempo"]=null_tempo_weights null_models[null_model_index] = mid_result + logger.debug("decreasing:"+str(null_models_frame)+":"+str(len(null_models.keys()))) + if null_models_frame < len(null_models.keys()): + for i in range(null_models_frame ,len(null_models.keys())): + _ = null_models.pop(i, None) result = null_models + logger.debug("change_null_model_row:res"+str(null_models)) return result def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: @@ -459,9 +467,9 @@ def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: outputs=[assist_text, assist_text_weight], ) with gr.Accordion(label="ヌルモデル", open=False): - with gr.Row(): + with gr.Row() as null_row: null_models_count = gr.Number(label="ヌルモデルの数", value=0, step=1) - with gr.Column(variant="panel"): + with gr.Column(variant="panel") as null_column: @gr.render( inputs=[ null_models_count, @@ -470,6 +478,8 @@ def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: def render_style( null_models_count:int, ): + global null_models_frame + null_models_frame = null_models_count for i in range(0, null_models_count): with gr.Row(): null_model_index = gr.Number( @@ -484,6 +494,9 @@ def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: value=model_names[initial_id], interactive=True ) + if i in null_models.value: + logger.debug(f"null model parameter exists in index {i}") + null_model_name.value=null_models.value[i]["name"] null_model_path = gr.Dropdown( label="モデルファイル", choices=initial_pth_files, @@ -491,6 +504,9 @@ def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: value=initial_pth_files[0], interactive=True ) + if i in null_models.value: + #null_model_path.choices = #ToDo + null_model_path.value=null_models.value[i]["path"] null_voice_weights = gr.Slider( minimum=0, maximum=1, @@ -500,6 +516,8 @@ def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: label="声質", interactive=True ) + if i in null_models.value: + null_voice_weights.value=null_models.value[i]["weight"] null_voice_pitch_weights = gr.Slider( minimum=0, maximum=1, @@ -509,6 +527,8 @@ def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: label="声の高さ", interactive=True ) + if i in null_models.value: + null_voice_pitch_weights.value=null_models.value[i]["pitch"] null_speech_style_weights = gr.Slider( minimum=0, maximum=1, @@ -518,6 +538,8 @@ def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: label="話し方", interactive=True ) + if i in null_models.value: + null_speech_style_weights.value=null_models.value[i]["style"] null_tempo_weights = gr.Slider( minimum=0, maximum=1, @@ -527,11 +549,14 @@ def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: label="テンポ", interactive=True ) + if i in null_models.value: + null_tempo_weights.value=null_models.value[i]["tempo"] null_model_name.change( model_holder.update_model_files_for_gradio, inputs=[null_model_name], outputs=[null_model_path], ) + #null_model_name.change(model_holder.refresh, outputs=[]) null_model_path.change(make_non_interactive, outputs=[tts_button]) #愚直すぎるのでもう少しなんとかしたい null_model_path.change(change_null_model_row, @@ -564,9 +589,6 @@ def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: null_models], outputs=[null_models] ) - if null_models_count < len(null_models.value.keys()): - for i in range(null_models_count ,len(null_models.value.keys())): - _ = null_models.value.pop(i, None) add_btn = gr.Button("ヌルモデルを増やす") del_btn = gr.Button("ヌルモデルを減らす") add_btn.click( From 7f2a4a8f3e4692825b55c50210c45d99d3a9e202 Mon Sep 17 00:00:00 2001 From: liruk <34234523+liruk@users.noreply.github.com> Date: Fri, 21 Jun 2024 09:07:09 +0900 Subject: [PATCH 11/12] =?UTF-8?q?=E3=82=B3=E3=83=BC=E3=83=89=E6=95=B4?= =?UTF-8?q?=E7=90=86+=E6=9A=AB=E5=AE=9A=E3=81=A7=E3=83=8C=E3=83=AB?= =?UTF-8?q?=E3=83=A2=E3=83=87=E3=83=AB=E4=BD=BF=E7=94=A8=E6=99=82=E3=81=AF?= =?UTF-8?q?=E5=B8=B8=E6=99=82load()=E3=81=99=E3=82=8B=E3=82=88=E3=81=86?= =?UTF-8?q?=E3=81=AB?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit TODO:ヌルモデルの変更を検知できればそれがよさそう。 --- gradio_tabs/inference.py | 10 +++------- style_bert_vits2/tts_model.py | 11 ++--------- 2 files changed, 5 insertions(+), 16 deletions(-) diff --git a/gradio_tabs/inference.py b/gradio_tabs/inference.py index 324b72c..570fce6 100644 --- a/gradio_tabs/inference.py +++ b/gradio_tabs/inference.py @@ -208,7 +208,7 @@ null_models_frame = 0 def change_null_model_row(null_model_index:int, null_model_name:str, null_model_path:str,null_voice_weights:float, null_voice_pitch_weights:float, null_speech_style_weights:float,null_tempo_weights:float, null_models:dict[int,dict[str, Any]]): - logger.debug("change_null_model_row:sta"+str(null_models)) + #logger.debug("change_null_model_row:sta"+str(null_models)) mid_result={} mid_result["name"]=null_model_name mid_result["path"]=null_model_path @@ -217,12 +217,12 @@ def change_null_model_row(null_model_index:int, null_model_name:str, null_model_ mid_result["style"]=null_speech_style_weights mid_result["tempo"]=null_tempo_weights null_models[null_model_index] = mid_result - logger.debug("decreasing:"+str(null_models_frame)+":"+str(len(null_models.keys()))) + #logger.debug("decreasing:"+str(null_models_frame)+":"+str(len(null_models.keys()))) if null_models_frame < len(null_models.keys()): for i in range(null_models_frame ,len(null_models.keys())): _ = null_models.pop(i, None) result = null_models - logger.debug("change_null_model_row:res"+str(null_models)) + #logger.debug("change_null_model_row:res"+str(null_models)) return result def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: @@ -253,10 +253,6 @@ def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: model_holder.get_model(model_name, model_path) assert model_holder.current_model is not None - if len(null_models.keys()) > 0: - model_holder.get_null_models(null_models) - assert len(model_holder.null_models_params.keys()) > 0 - wrong_tone_message = "" kata_tone: Optional[list[tuple[str, int]]] = None if use_tone and kata_tone_json_str != "": diff --git a/style_bert_vits2/tts_model.py b/style_bert_vits2/tts_model.py index f8a387b..4e1cdac 100644 --- a/style_bert_vits2/tts_model.py +++ b/style_bert_vits2/tts_model.py @@ -298,6 +298,8 @@ class TTSModel: if assist_text == "" or not use_assist_text: assist_text = None if null_model_params is not {}: + #ヌルモデルがあるときは常時ロードしなおすけどもっといい手段ありそう + self.__net_g = None self.null_model_params = null_model_params else: self.null_model_params = {} @@ -407,7 +409,6 @@ class TTSModelHolder: self.device: str = device self.model_files_dict: dict[str, list[Path]] = {} self.current_model: Optional[TTSModel] = None - self.null_models_params: dict[int,dict[str,Union[str, float]]] = {} self.model_names: list[str] = [] self.models_info: list[TTSModelInfo] = [] self.refresh() @@ -420,7 +421,6 @@ class TTSModelHolder: self.model_files_dict = {} self.model_names = [] self.current_model = None - self.null_models_params = {} self.models_info = [] model_dirs = [d for d in self.root_dir.iterdir() if d.is_dir()] @@ -482,13 +482,6 @@ class TTSModelHolder: ) return self.current_model - def get_null_models(self, null_model_index:dict[int,dict[str,Union[str, float]]]) -> dict[int, dict[str, Union[str, float]]]: - """ - get_modelをヌルモデル用に改変。複数まとめて使うかも知れないのでdictで戻す - - """ - self.null_models_params = null_model_index - return self.null_models_params def get_model_for_gradio(self, model_name: str, model_path_str: str): import gradio as gr From b645de0750691b7f0dd884a424e4bbb6fac17c7e Mon Sep 17 00:00:00 2001 From: liruk <34234523+liruk@users.noreply.github.com> Date: Fri, 21 Jun 2024 09:19:46 +0900 Subject: [PATCH 12/12] =?UTF-8?q?=E3=83=8C=E3=83=AB=E3=83=A2=E3=83=87?= =?UTF-8?q?=E3=83=AB=E5=A4=89=E6=9B=B4=E6=99=82=E3=81=ABload=E3=81=97?= =?UTF-8?q?=E7=9B=B4=E3=81=99=E7=94=A8=E9=80=94=E3=81=A7=E3=80=81=E6=8E=A8?= =?UTF-8?q?=E8=AB=96=E6=99=82=E3=81=ABforce=5Freload=5Fmodel=E3=82=92?= =?UTF-8?q?=E8=BF=BD=E5=8A=A0?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- gradio_tabs/inference.py | 26 +++++++++++++++----------- style_bert_vits2/tts_model.py | 7 ++++--- 2 files changed, 19 insertions(+), 14 deletions(-) diff --git a/gradio_tabs/inference.py b/gradio_tabs/inference.py index 570fce6..ab8bb1e 100644 --- a/gradio_tabs/inference.py +++ b/gradio_tabs/inference.py @@ -223,7 +223,7 @@ def change_null_model_row(null_model_index:int, null_model_name:str, null_model_ _ = null_models.pop(i, None) result = null_models #logger.debug("change_null_model_row:res"+str(null_models)) - return result + return result, True def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: def tts_fn( @@ -248,7 +248,8 @@ def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: speaker, pitch_scale, intonation_scale, - null_models:dict[int, dict[str, Union[str, float]]] + null_models:dict[int, dict[str, Union[str, float]]], + force_reload_model:bool ): model_holder.get_model(model_name, model_path) assert model_holder.current_model is not None @@ -306,7 +307,8 @@ def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: speaker_id=speaker_id, pitch_scale=pitch_scale, intonation_scale=intonation_scale, - null_model_params = null_models + null_model_params = null_models, + force_reload_model = force_reload_model ) except InvalidToneError as e: logger.error(f"Tone error: {e}") @@ -328,7 +330,7 @@ def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: message = f"Success, time: {duration} seconds." if wrong_tone_message != "": message = wrong_tone_message + "\n" + message - return message, (sr, audio), kata_tone_json_str + return message, (sr, audio), kata_tone_json_str, False model_names = model_holder.model_names if len(model_names) == 0: @@ -349,6 +351,7 @@ def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: gr.Markdown(initial_md) gr.Markdown(terms_of_use_md) null_models = gr.State({}) + force_reload_model = gr.State(False) with gr.Accordion(label="使い方", open=False): gr.Markdown(how_to_md) with gr.Row(): @@ -559,31 +562,31 @@ def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: inputs=[null_model_index, null_model_name, null_model_path,null_voice_weights, null_voice_pitch_weights, null_speech_style_weights,null_tempo_weights, null_models], - outputs=[null_models] + outputs=[null_models,force_reload_model] ) null_voice_weights.change(change_null_model_row, inputs=[null_model_index, null_model_name, null_model_path,null_voice_weights, null_voice_pitch_weights, null_speech_style_weights,null_tempo_weights, null_models], - outputs=[null_models] + outputs=[null_models,force_reload_model] ) null_voice_pitch_weights.change(change_null_model_row, inputs=[null_model_index, null_model_name, null_model_path,null_voice_weights, null_voice_pitch_weights, null_speech_style_weights,null_tempo_weights, null_models], - outputs=[null_models] + outputs=[null_models,force_reload_model] ) null_speech_style_weights.change(change_null_model_row, inputs=[null_model_index, null_model_name, null_model_path,null_voice_weights, null_voice_pitch_weights, null_speech_style_weights,null_tempo_weights, null_models], - outputs=[null_models] + outputs=[null_models,force_reload_model] ) null_tempo_weights.change(change_null_model_row, inputs=[null_model_index, null_model_name, null_model_path,null_voice_weights, null_voice_pitch_weights, null_speech_style_weights,null_tempo_weights, null_models], - outputs=[null_models] + outputs=[null_models,force_reload_model] ) add_btn = gr.Button("ヌルモデルを増やす") del_btn = gr.Button("ヌルモデルを減らす") @@ -655,9 +658,10 @@ def create_inference_app(model_holder: TTSModelHolder) -> gr.Blocks: speaker, pitch_scale, intonation_scale, - null_models + null_models, + force_reload_model ], - outputs=[text_output, audio_output, tone], + outputs=[text_output, audio_output, tone, force_reload_model], ) model_name.change( diff --git a/style_bert_vits2/tts_model.py b/style_bert_vits2/tts_model.py index 4e1cdac..77db3dd 100644 --- a/style_bert_vits2/tts_model.py +++ b/style_bert_vits2/tts_model.py @@ -258,7 +258,8 @@ class TTSModel: given_tone: Optional[list[int]] = None, pitch_scale: float = 1.0, intonation_scale: float = 1.0, - null_model_params: dict[int,dict[str,Union[str, float]]] = {} + null_model_params: dict[int,dict[str,Union[str, float]]] = {}, + force_reload_model:bool = False ) -> tuple[int, NDArray[Any]]: """ テキストから音声を合成する。 @@ -298,11 +299,11 @@ class TTSModel: if assist_text == "" or not use_assist_text: assist_text = None if null_model_params is not {}: - #ヌルモデルがあるときは常時ロードしなおすけどもっといい手段ありそう - self.__net_g = None self.null_model_params = null_model_params else: self.null_model_params = {} + if force_reload_model is True: + self.__net_g = None if self.__net_g is None: self.load() assert self.__net_g is not None