Improve: Convert ONNX version of BERT models to FP16
I have found that half-precision has little effect on speech synthesis quality and, depending on the environment, can reduce file size and memory usage by half, so I have decided to use FP16.
This commit is contained in:
@@ -82,7 +82,7 @@ def load_model(
|
||||
model_path = Path(
|
||||
hf_hub_download(
|
||||
repo_id=pretrained_model_name_or_path,
|
||||
filename="model.onnx",
|
||||
filename="model_fp16.onnx",
|
||||
cache_dir=cache_dir,
|
||||
revision=revision,
|
||||
)
|
||||
@@ -99,7 +99,7 @@ def load_model(
|
||||
# pretrained_model_name_or_path にファイルパスが指定された場合:
|
||||
# 既にダウンロード済みという前提のもと、モデルへのローカルパスを model_path に格納する
|
||||
else:
|
||||
model_path = Path(pretrained_model_name_or_path).resolve() / "model.onnx"
|
||||
model_path = Path(pretrained_model_name_or_path).resolve() / "model_fp16.onnx"
|
||||
|
||||
start_time = time.time()
|
||||
sess_options = onnxruntime.SessionOptions()
|
||||
|
||||
Reference in New Issue
Block a user