This commit is contained in:
tuna2134
2026-07-20 21:25:49 +09:00
parent 0c2be00f0a
commit b8ce11605c
17 changed files with 496 additions and 38 deletions

View File

@@ -18,7 +18,38 @@ Relevant settings:
- `model.matcha_sigma_min`: minimum flow path noise.
- `model.matcha_n_timesteps`: Euler steps at inference; more steps trade speed
for refinement quality.
- `model.matcha_use_diff_attention`: enables Differential Attention V2 in the
U-Net transformer blocks. It uses explicit PyTorch matrix multiplication and
softmax, with no FlashAttention, CUDA-only kernel, or extra dependency.
The Matcha branch has new parameters, so enabling it on an existing checkpoint
requires fine-tuning before inference. It is trained on the same random latent
segments used by the waveform generator to keep memory use bounded.
## Matcha-only fine-tuning
Set `train.matcha_only` to `true` to train only `net_g.matcha`. All legacy
generator parameters are frozen, and adversarial, duration-discriminator,
WavLM-discriminator, mel, duration, and KL optimization steps are skipped. The
existing VITS normalizing-flow latent `z_p` remains the detached CFM target.
This mode requires `model.use_matcha: true`. Optimizer state from a full-model
checkpoint is intentionally not restored because its parameter groups differ;
model weights are still loaded. After the Matcha loss has converged, set
`train.matcha_only` back to `false` and use a lower learning rate for joint
fine-tuning.
For an already preprocessed dataset (including Google Colab), the mode can be
configured without rerunning preprocessing:
```bash
python configure_matcha_ft.py --config Data/MyModel/config.json \
--learning-rate 0.0001 --epochs 20 --save-every-steps 100
```
To switch to the second-stage joint fine-tuning:
```bash
python configure_matcha_ft.py --config Data/MyModel/config.json \
--joint --learning-rate 0.00002
```