Commit Graph

850 Commits

Author SHA1 Message Date
Stardust·减
555b1f4bb5 clean code 2023-09-05 19:52:00 +08:00
Stardust·减
ff0c76dccb add logger 2023-09-05 19:50:24 +08:00
Stardust·减
89b6c5d589 Update requirements.txt 2023-09-05 19:48:34 +08:00
Stardust·减
70b6e3bb9a Update bert_gen.py 2023-09-05 19:46:39 +08:00
Stardust·减
9d2ed39365 fix jp 2023-09-05 19:41:52 +08:00
Stardust·减
13c9a12573 fix 2023-09-05 19:41:41 +08:00
Stardust·减
1f080322dc 添加device传参 2023-09-05 19:40:15 +08:00
Stardust·减
3719c5e303 Update chinese_bert.py 2023-09-05 19:38:45 +08:00
Stardust·减
dcd5ed3c43 新增device传参 2023-09-05 19:38:23 +08:00
Stardust·减
eb14f4a0b7 多卡预处理 2023-09-05 19:35:21 +08:00
Stardust·减
d02113267b 多卡预处理 2023-09-05 19:34:28 +08:00
Stardust·减
f945aa6091 Update bert_gen.py 2023-09-05 19:31:35 +08:00
Stardust·减
e86c8a8991 use torchrun 2023-09-05 17:26:39 +08:00
Lengyue
042fb96fd9 Lint bert gen & remove unused vars 2023-09-05 01:59:36 -04:00
Lengyue
d4d0082b05 lint code (no other modify) 2023-09-05 01:21:17 -04:00
Lengyue
eabeb72e27 Optimize preprocess code 2023-09-05 01:08:51 -04:00
Lengyue
55404f92b5 Add reference pre-commit config 2023-09-05 01:08:42 -04:00
Stardust·减
24ba0f26df Update train_ms.py 2023-09-05 11:30:28 +08:00
Stardust·减
ff8469f0f6 decrease num_worker 2023-09-05 11:26:50 +08:00
Stardust·减
2b88603554 Delete bert/bert-base-japanese-v3/.gitkeep 2023-09-05 10:32:09 +08:00
Stardust·减
0000785a53 Update server.py 2023-09-05 10:28:07 +08:00
Stardust·减
96c0a81e71 support mps 2023-09-05 10:24:25 +08:00
Stardust·减
6763e150b7 support ja 2023-09-05 10:23:56 +08:00
Stardust·减
2857d004d8 support jp 2023-09-05 10:19:22 +08:00
Stardust·减
5b5c79dd16 add bert 2023-09-05 10:02:50 +08:00
Stardust·减
b5e7aefeef Update symbols.py 2023-09-05 10:01:13 +08:00
Stardust·减
6d4d123162 add bert 2023-09-05 10:00:54 +08:00
Stardust·减
316e0ba345 add tqdm 2023-09-05 09:57:08 +08:00
Stardust·减
bb4ebb80f7 add bert 2023-09-05 09:54:11 +08:00
Stardust·减
02423d892f add bert 2023-09-05 09:52:56 +08:00
Stardust·减
37bee28b7f ad layers 2023-09-05 09:49:24 +08:00
Stardust·减
1d337caa47 Merge pull request #17 from Stardust-minus/master
Dev pull
2023-09-05 01:11:19 +08:00
Stardust·减
e7f77407d7 Update requirements.txt 2023-09-05 01:08:42 +08:00
Stardust·减
f7db8c997a clean code 2023-09-05 01:05:30 +08:00
Stardust·减
7810be71b0 clean code 2023-09-05 01:05:15 +08:00
Stardust·减
d74fe531e9 Update data_utils.py 2023-09-05 01:04:10 +08:00
Stardust·减
4127f7d00a Update train_ms.py 2023-09-05 01:02:46 +08:00
Stardust·减
f83227dac5 Update train_ms.py 2023-09-05 01:00:47 +08:00
Stardust·减
4a1c09bf31 Update train_ms.py 2023-09-05 00:56:32 +08:00
Stardust·减
82105b1cd7 Create japanese_bert.py 2023-09-05 00:54:00 +08:00
Stardust·减
134bc88822 Update japanese.py 2023-09-05 00:53:27 +08:00
Stardust·减
2d2adb4811 Update __init__.py 2023-09-05 00:52:26 +08:00
Stardust·减
80a50f8768 Create vocab.txt 2023-09-05 00:51:17 +08:00
Stardust·减
bfb39cc114 Create README.md 2023-09-05 00:50:55 +08:00
Stardust·减
972fd2ca1a Create config.json 2023-09-05 00:50:37 +08:00
Stardust·减
c5e8c4c999 Create .gitkeep 2023-09-05 00:50:02 +08:00
Stardust·减
8f9884d6a6 增加resume提示 2023-09-04 22:20:21 +08:00
Stardust·减
36b15ba514 rollback resume 2023-09-04 22:14:46 +08:00
Stardust·减
7761dadcc6 rollback resume 2023-09-04 22:13:57 +08:00
Stardust·减
c6f627f350 修复resume 2023-09-04 22:02:02 +08:00