Commits · jinaai/jina-bert-flash-implementation

Porting v2 models to flash attention (#15)

e55e319
verified

bwang0911

Markus28 commited on Mar 23

feat: updated activation checkpointing (#14)

4ed66c0
verified

Markus28 commited on Mar 21

feat: removed task type embeddings

8e3d0b8

Markus28 commited on Mar 20

fix: remove cleaving (#13)

139b4a5
verified

Markus28 commited on Mar 19

fix: added trust_remote_code to tokenizer

9689b77

Markus28 commited on Mar 19

feat: cleave off layers from encoder (#11)

b641603
verified

Markus28 commited on Mar 19

fix BertForMaskedLM

c0b46cc

Markus28 commited on Mar 14

feat: added separate BertForMaskedLM class

3cb3930

Markus28 commited on Mar 14

feat: added return_dict

59c0808

Markus28 commited on Mar 12

feat: choose flash attention heuristically if not set explicitly

2e2b8d0

Markus28 commited on Mar 6

fix: assert is None for other kwargs too

3f5615c

Markus28 commited on Mar 5

feat: added head_mask

599c64e

Markus28 commited on Mar 5

fix: cast mask to bool

ca5f516

Markus28 commited on Mar 5

fix: move flash components into top-level

5944ec8

Markus28 commited on Mar 5

feat: try to fix import error

4c4562b

Markus28 commited on Mar 5

feat: moved flash attention code into this repository

46df05d

Markus28 commited on Mar 5

feat: added encode method

32458be

Markus28 commited on Mar 1

fix: try to skip initialization of task type embeddings

3b35eab

Markus28 commited on Mar 1

fix: try to skip initialization of task type embeddings

95ca1a8

Markus28 commited on Mar 1

feat: added option for QK normalization

463061d

Markus28 commited on Mar 1

feat: implement task type embeddings (#1)

8adf551
verified

Markus28 commited on Mar 1

feat: added back option not to use flash attention

d4d5621

Markus28 commited on Mar 1

feat: support gradient checkpointing

75d7a16

Markus28 commited on Feb 28

removed unused imports

5e7b835

Markus28 commited on Feb 27

removed init from BertPretrainedModel

44fd417

Markus28 commited on Feb 27

added config_class and base_model_prefix

45b2292

Markus28 commited on Feb 27

Fixed typo

80472cb

Markus28 commited on Feb 27

Fixed typo

6fb6577

Markus28 commited on Feb 27

Try to subclass PretrainedModel

e209593

Markus28 commited on Feb 27

Try to subclass PretrainedModel

2b23340

Markus28 commited on Feb 27

strict=True for debugging

a0c289c

Markus28 commited on Feb 27

try to simplify checkpointing

4c68a4c

Markus28 commited on Feb 27

removed debugging

c2d8dc3

Markus28 commited on Feb 22

debugging

c4185ce

Markus28 commited on Feb 22

debugging

a1e1eff

Markus28 commited on Feb 22

debugging assertion

4d2995d

Markus28 commited on Feb 22

fix: fixed get_input_embeddings method

7e06371

Markus28 commited on Feb 22

feat: added get_input_embeddings method to BertForPreTraining

bb281f0

Markus28 commited on Feb 22

feat: fixed _from_config

18eed80

Markus28 commited on Feb 22

removed from_config

0ce78aa

Markus28 commited on Feb 22

fix: try to get from_config to work

871fd36

Markus28 commited on Feb 22

feat: added from_config, also pass additional kwargs from config to model

4164fd6

Markus28 commited on Feb 22

feat: updated modeling_bert.py to allow MLM-only training

0f43653

Markus28 commited on Feb 22

feat: reverted monkey patch

3160695

Markus28 commited on Feb 22

feat: try to monkey-patch index_first_axis

ed92835

Markus28 commited on Feb 22

feat: try to fix compilation

03d8e7c

Markus28 commited on Feb 22

feat: added debug print

63832b9

Markus28 commited on Feb 21

feat: updated .to() override to handle kwargs

e86d612

Markus28 commited on Feb 21

Revert "feat updated debug pring"

a62c2ab

Markus28 commited on Feb 21

feat updated debug pring

d21ee1b

Markus28 commited on Feb 21

Commit History

Porting v2 models to flash attention (#15) e55e319 verified

feat: updated activation checkpointing (#14) 4ed66c0 verified

feat: removed task type embeddings 8e3d0b8

fix: remove cleaving (#13) 139b4a5 verified

fix: added trust_remote_code to tokenizer 9689b77

feat: cleave off layers from encoder (#11) b641603 verified

fix BertForMaskedLM c0b46cc

feat: added separate BertForMaskedLM class 3cb3930

feat: added return_dict 59c0808

feat: choose flash attention heuristically if not set explicitly 2e2b8d0

fix: assert is None for other kwargs too 3f5615c

feat: added head_mask 599c64e

fix: cast mask to bool ca5f516

fix: move flash components into top-level 5944ec8

feat: try to fix import error 4c4562b

feat: moved flash attention code into this repository 46df05d

feat: added encode method 32458be

fix: try to skip initialization of task type embeddings 3b35eab

fix: try to skip initialization of task type embeddings 95ca1a8

feat: added option for QK normalization 463061d

feat: implement task type embeddings (#1) 8adf551 verified

feat: added back option not to use flash attention d4d5621

feat: support gradient checkpointing 75d7a16

removed unused imports 5e7b835

removed __init__ from BertPretrainedModel 44fd417

added config_class and base_model_prefix 45b2292

Fixed typo 80472cb

Fixed typo 6fb6577

Try to subclass PretrainedModel e209593

Try to subclass PretrainedModel 2b23340

strict=True for debugging a0c289c

try to simplify checkpointing 4c68a4c

removed debugging c2d8dc3

debugging c4185ce

debugging a1e1eff

debugging assertion 4d2995d

fix: fixed get_input_embeddings method 7e06371

feat: added get_input_embeddings method to BertForPreTraining bb281f0

feat: fixed _from_config 18eed80

removed from_config 0ce78aa

fix: try to get from_config to work 871fd36

feat: added from_config, also pass additional kwargs from config to model 4164fd6

feat: updated modeling_bert.py to allow MLM-only training 0f43653

feat: reverted monkey patch 3160695

feat: try to monkey-patch index_first_axis ed92835

feat: try to fix compilation 03d8e7c

feat: added debug print 63832b9

feat: updated .to() override to handle kwargs e86d612

Revert "feat updated debug pring" a62c2ab

feat updated debug pring d21ee1b

Porting v2 models to flash attention (#15)

e55e319
verified

feat: updated activation checkpointing (#14)

4ed66c0
verified

feat: removed task type embeddings

8e3d0b8

fix: remove cleaving (#13)

139b4a5
verified

fix: added trust_remote_code to tokenizer

9689b77

feat: cleave off layers from encoder (#11)

b641603
verified

fix BertForMaskedLM

c0b46cc

feat: added separate BertForMaskedLM class

3cb3930

feat: added return_dict

59c0808

feat: choose flash attention heuristically if not set explicitly

2e2b8d0

fix: assert is None for other kwargs too

3f5615c

feat: added head_mask

599c64e

fix: cast mask to bool

ca5f516

fix: move flash components into top-level

5944ec8

feat: try to fix import error

4c4562b

feat: moved flash attention code into this repository

46df05d

feat: added encode method

32458be

fix: try to skip initialization of task type embeddings

3b35eab

fix: try to skip initialization of task type embeddings

95ca1a8

feat: added option for QK normalization

463061d

feat: implement task type embeddings (#1)

8adf551
verified

feat: added back option not to use flash attention

d4d5621

feat: support gradient checkpointing

75d7a16

removed unused imports

5e7b835

removed init from BertPretrainedModel

44fd417

added config_class and base_model_prefix

45b2292

Fixed typo

80472cb

Fixed typo

6fb6577

Try to subclass PretrainedModel

e209593

Try to subclass PretrainedModel

2b23340

strict=True for debugging

a0c289c

try to simplify checkpointing

4c68a4c

removed debugging

c2d8dc3

debugging

c4185ce

debugging

a1e1eff

debugging assertion

4d2995d

fix: fixed get_input_embeddings method

7e06371

feat: added get_input_embeddings method to BertForPreTraining

bb281f0

feat: fixed _from_config

18eed80

removed from_config

0ce78aa

fix: try to get from_config to work

871fd36

feat: added from_config, also pass additional kwargs from config to model

4164fd6

feat: updated modeling_bert.py to allow MLM-only training

0f43653

feat: reverted monkey patch

3160695

feat: try to monkey-patch index_first_axis

ed92835

feat: try to fix compilation

03d8e7c

feat: added debug print

63832b9

feat: updated .to() override to handle kwargs

e86d612

Revert "feat updated debug pring"

a62c2ab

feat updated debug pring

d21ee1b