| --max_seq_len 8192 --compress_pos_emb 4 --loader exllama_hf | |
| check monkeypatch in ooba .. | |
| for 16384 compress_pos_emb 8 | |
| works on 2 a6000 just fine |
| --max_seq_len 8192 --compress_pos_emb 4 --loader exllama_hf | |
| check monkeypatch in ooba .. | |
| for 16384 compress_pos_emb 8 | |
| works on 2 a6000 just fine |