fix
This commit is contained in:
@@ -62,7 +62,7 @@ Optional
|
||||
## 2. Preprocess
|
||||
|
||||
```bash
|
||||
python preprocess_all.py -m <model_name> [--use_jp_extra] [-b <batch_size>] [-e <epochs>] [-s <save_every_steps>] [--num_processes <num_processes>] [--normalize] [--trim] [--val_per_lang <val_per_lang>] [--log_interval <log_interval>] [--freeze_EN_bert] [--freeze_JP_bert] [--freeze_ZH_bert] [--freeze_style] [--freeze_decoder] [--yomi_error <yomi_error>]
|
||||
python preprocess_all.py -m <model_name> [--use_jp_extra] [--matcha_only] [-b <batch_size>] [-e <epochs>] [-s <save_every_steps>] [--num_processes <num_processes>] [--normalize] [--trim] [--val_per_lang <val_per_lang>] [--log_interval <log_interval>] [--freeze_EN_bert] [--freeze_JP_bert] [--freeze_ZH_bert] [--freeze_style] [--freeze_decoder] [--yomi_error <yomi_error>]
|
||||
```
|
||||
|
||||
Required:
|
||||
@@ -81,6 +81,7 @@ Optional:
|
||||
- `--freeze_style`: Freeze style vector.
|
||||
- `--freeze_decoder`: Freeze decoder.
|
||||
- `--use_jp_extra`: Use JP-Extra model.
|
||||
- `--matcha_only`: Enable Matcha-only fine-tuning and Differential Attention V2.
|
||||
- `--val_per_lang`: Validation data per language (default: 0).
|
||||
- `--log_interval`: Log interval (default: 200).
|
||||
- `--yomi_error`: How to handle yomi errors (default: `raise`: raise an error after preprocessing all texts, `skip`: skip the texts with errors, `use`: use the texts with errors by ignoring unknown characters).
|
||||
|
||||
@@ -18,7 +18,38 @@ Relevant settings:
|
||||
- `model.matcha_sigma_min`: minimum flow path noise.
|
||||
- `model.matcha_n_timesteps`: Euler steps at inference; more steps trade speed
|
||||
for refinement quality.
|
||||
- `model.matcha_use_diff_attention`: enables Differential Attention V2 in the
|
||||
U-Net transformer blocks. It uses explicit PyTorch matrix multiplication and
|
||||
softmax, with no FlashAttention, CUDA-only kernel, or extra dependency.
|
||||
|
||||
The Matcha branch has new parameters, so enabling it on an existing checkpoint
|
||||
requires fine-tuning before inference. It is trained on the same random latent
|
||||
segments used by the waveform generator to keep memory use bounded.
|
||||
|
||||
## Matcha-only fine-tuning
|
||||
|
||||
Set `train.matcha_only` to `true` to train only `net_g.matcha`. All legacy
|
||||
generator parameters are frozen, and adversarial, duration-discriminator,
|
||||
WavLM-discriminator, mel, duration, and KL optimization steps are skipped. The
|
||||
existing VITS normalizing-flow latent `z_p` remains the detached CFM target.
|
||||
|
||||
This mode requires `model.use_matcha: true`. Optimizer state from a full-model
|
||||
checkpoint is intentionally not restored because its parameter groups differ;
|
||||
model weights are still loaded. After the Matcha loss has converged, set
|
||||
`train.matcha_only` back to `false` and use a lower learning rate for joint
|
||||
fine-tuning.
|
||||
|
||||
For an already preprocessed dataset (including Google Colab), the mode can be
|
||||
configured without rerunning preprocessing:
|
||||
|
||||
```bash
|
||||
python configure_matcha_ft.py --config Data/MyModel/config.json \
|
||||
--learning-rate 0.0001 --epochs 20 --save-every-steps 100
|
||||
```
|
||||
|
||||
To switch to the second-stage joint fine-tuning:
|
||||
|
||||
```bash
|
||||
python configure_matcha_ft.py --config Data/MyModel/config.json \
|
||||
--joint --learning-rate 0.00002
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user