Results Analysis:
Results Analysis:
ARC (25 shots): There is only a minimal difference between the two versions of the model (65.9 vs. 65.8), so there is no significant reduction in performance.
GSM8K (5 shots): The "abliterated" version scores slightly lower (75.2 vs. 76.2), indicating that there is a small drop in mathematical ability.
HellaSwag (10 shots): Here, performance is identical (84.3 vs. 84.3), meaning that orthogonalization did not impact event reasoning.
MMLU (5 hits): Surprisingly, the "abliterated" version scores slightly higher (68.8 vs. 68.4), which may indicate an improvement in academic ability.
TruthfulQA (0-shot): Again, there is a small positive difference for the "abliterated" version (55.0 vs. 54.9), indicating a slight improvement in the ability to provide truthful answers.
Winogrande (5 hits): Again, the "abliterated" version is slightly better (82.6 vs. 82.2), suggesting better common-sense reasoning skills.