Auto download missing model for bert_gen.py (#146)

* auto download missing model

* support openi

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix wrong delete

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* pass pre-commit

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix repeat login

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
Isotr0py
2023-11-04 04:06:10 +08:00
committed by GitHub
parent 4d6de240a0
commit 8609449b63
9 changed files with 92 additions and 25 deletions

14
bert/bert_models.json Normal file
View File

@@ -0,0 +1,14 @@
{
"deberta-v2-large-japanese": {
"repo_id": "ku-nlp/deberta-v2-large-japanese",
"files": ["spm.model", "pytorch_model.bin"]
},
"chinese-roberta-wwm-ext-large": {
"repo_id": "hfl/chinese-roberta-wwm-ext-large",
"files": ["pytorch_model.bin"]
},
"deberta-v3-large": {
"repo_id": "microsoft/deberta-v3-large",
"files": ["spm.model", "pytorch_model.bin"]
}
}