Hugging Face's logo

Spaces:
hallucinations-leaderboard
/
leaderboard
Running on CPU Upgrade

App Files Community
26
pminervini commited on
Commit
1591f9d
•
1 Parent(s): ecd0860

update

Browse files
Files changed (12) hide show
  1. plots/clustermap_all.pdf +0 -0
  2. plots/clustermap_all.png +2 -2
  3. plots/clustermap_det.pdf +0 -0
  4. plots/clustermap_det.png +2 -2
  5. plots/clustermap_instr.pdf +0 -0
  6. plots/clustermap_instr.png +2 -2
  7. plots/clustermap_qa.pdf +0 -0
  8. plots/clustermap_qa.png +2 -2
  9. plots/clustermap_summ.pdf +0 -0
  10. plots/clustermap_summ.png +2 -2
  11. src/backend/envs.py +1 -0
  12. src/backend/tasks/memo-trap/memo-trap_v2.yaml +20 -0
plots/clustermap_all.pdf CHANGED
Binary files a/plots/clustermap_all.pdf and b/plots/clustermap_all.pdf differ
 
plots/clustermap_all.png CHANGED

Git LFS Details

  • SHA256: 5071d87316e1113bcb05316a14edebc3a88c76a2e59f8f633ed34f9492982265
  • Pointer size: 132 Bytes
  • Size of remote file: 1.55 MB

Git LFS Details

  • SHA256: 5e54f03d44e832da7ece78c2467087677a18d98d7cae3cacb1aac0b480961c95
  • Pointer size: 132 Bytes
  • Size of remote file: 1.61 MB
plots/clustermap_det.pdf CHANGED
Binary files a/plots/clustermap_det.pdf and b/plots/clustermap_det.pdf differ
 
plots/clustermap_det.png CHANGED

Git LFS Details

  • SHA256: 9422ff3b910f71e9f08dfd3a99b463c753abe5ddf5bd45c78a94c29a9cc87737
  • Pointer size: 131 Bytes
  • Size of remote file: 702 kB

Git LFS Details

  • SHA256: d6b1985fe86e22b9f8482fc40d95439d99595a90a7eeb11e610618f8f2342490
  • Pointer size: 131 Bytes
  • Size of remote file: 758 kB
plots/clustermap_instr.pdf CHANGED
Binary files a/plots/clustermap_instr.pdf and b/plots/clustermap_instr.pdf differ
 
plots/clustermap_instr.png CHANGED

Git LFS Details

  • SHA256: 27f691016f3dd11d9556dc4659bcb62be8113a92cd1969706cc771bb05ee2de2
  • Pointer size: 131 Bytes
  • Size of remote file: 580 kB

Git LFS Details

  • SHA256: b015ab8ffc20966e52ce9930ab669db24e6675ad4b1ebcbf25bceeac9b9694aa
  • Pointer size: 131 Bytes
  • Size of remote file: 629 kB
plots/clustermap_qa.pdf CHANGED
Binary files a/plots/clustermap_qa.pdf and b/plots/clustermap_qa.pdf differ
 
plots/clustermap_qa.png CHANGED

Git LFS Details

  • SHA256: 6d539d57bdfe27a7c80b70c93bd32a44bd7e5dc53c491c07452354f9d55c5328
  • Pointer size: 131 Bytes
  • Size of remote file: 697 kB

Git LFS Details

  • SHA256: f30511ccb280429306fb2d2ac50270259adc532919bdf7479a01f7e7143a4876
  • Pointer size: 131 Bytes
  • Size of remote file: 731 kB
plots/clustermap_summ.pdf CHANGED
Binary files a/plots/clustermap_summ.pdf and b/plots/clustermap_summ.pdf differ
 
plots/clustermap_summ.png CHANGED

Git LFS Details

  • SHA256: 56800b49e9322201a89e2572aa9efca66f0f80b9c2a2c334829dc3da93fa4eb4
  • Pointer size: 131 Bytes
  • Size of remote file: 754 kB

Git LFS Details

  • SHA256: 8c9555dc15166f0c768e7b5ab67519357eb0ab39026f316cf4e82a4125b8546d
  • Pointer size: 131 Bytes
  • Size of remote file: 790 kB
src/backend/envs.py CHANGED
@@ -33,6 +33,7 @@ class Tasks(Enum):
33
  task9 = Task("cnndm", "rougeL", "CNN/DM", 2)
34
 
35
  task10 = Task("memo-trap", "acc", "memo-trap", 0)
 
36
 
37
  task11 = Task("nq8", "em", "NQ Open 8", 8)
38
  task12 = Task("tqa8", "em", "TriviaQA 8", 8)
 
33
  task9 = Task("cnndm", "rougeL", "CNN/DM", 2)
34
 
35
  task10 = Task("memo-trap", "acc", "memo-trap", 0)
36
+ task10_2 = Task("memo-trap_v2", "acc", "memo-trap", 0)
37
 
38
  task11 = Task("nq8", "em", "NQ Open 8", 8)
39
  task12 = Task("tqa8", "em", "TriviaQA 8", 8)
src/backend/tasks/memo-trap/memo-trap_v2.yaml ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ task: memo-trap_v2
2
+ dataset_path: pminervini/inverse-scaling
3
+ dataset_name: memo-trap
4
+ output_type: multiple_choice
5
+ training_split: null
6
+ validation_split: data
7
+ test_split: null
8
+ num_fewshot: 0
9
+ doc_to_text: "{{prompt}}"
10
+ doc_to_target: answer_index
11
+ doc_to_choice: "{{classes}}"
12
+ target_delimiter: ""
13
+ should_decontaminate: False
14
+ doc_to_decontamination_query: prompt
15
+ metric_list:
16
+ - metric: acc
17
+ aggregation: mean
18
+ higher_is_better: true
19
+ metadata:
20
+ - version: 0.0