mmochtak commited on
Commit
5f0dea5
·
verified ·
1 Parent(s): 948d465

paper link; update model card

Browse files
Files changed (1) hide show
  1. README.md +10 -6
README.md CHANGED
@@ -69,15 +69,19 @@ print(sentence_df)
69
 
70
  ```
71
 
 
 
 
72
  **If you use the model, please cite:**
73
  ```
74
  @article{mochtak_chasing_2024,
75
- title = {Chasing the {Authoritarian} {Specter}: {Detecting} {Authoritarian} {Discourse} with {Large} {Language} {Models}},
76
- volume = {forthcoming},
77
- abstract = {The paper introduces a deep-learning model fine-tuned for detecting authoritarian discourse in political speeches. Set up as a regression problem with weak supervision logic, the model is trained for the task of classification of segments of text for being/not being associated with authoritarian discourse. Rather than trying to define what an authoritarian discourse is, the model builds on the assumption that authoritarian leaders inherently define it. In other words, authoritarian leaders talk like authoritarians. When combined with the discourse defined by democratic leaders, the model learns the instances that are more often associated with authoritarians on the one hand and democrats on the other. The paper discusses several evaluation tests using the model and advocates for its usefulness in a broad range of research problems. It presents a new methodology for studying latent political concepts and positions it as an alternative to more traditional research strategies.},
78
- language = {en},
79
- journal = {European Journal of Political Resarch},
 
80
  author = {Mochtak, Michal},
81
- year = {2024},
82
  }
83
  ```
 
69
 
70
  ```
71
 
72
+ **Known biases and issues**
73
+ This model, like all machine learning models, exhibits biases shaped by its training data and task-specific nuances. Trained primarily on speeches from the UN General Assembly, it has learned discourse patterns unique to that context, which may influence how it classifies leaders along the authoritarian-democratic spectrum. This limitation is compounded by a slight imbalance in the training data, which skews towards authoritarian discourse (mean = 0.430). Although no systematic bias was detected in testing, the model may occasionally lean towards assigning lower values in certain cases. Additionally, the model’s classification may be sensitive to cultural or ideological markers, such as religious phrases commonly used by leaders from majority-Muslim countries, or ideological language like "comrades," which is often associated with authoritarian states. These biases can influence the model’s predictions and may be more apparent with shorter texts or less structured data formats, such as tweets or informal statements. While the model performs best with longer texts, evaluation on any new format, both qualitative and quantitative, is highly recommended to ensure robust performance. Fine-tuning may be necessary to mitigate specific biases and enhance reliability across different applications.
74
+
75
  **If you use the model, please cite:**
76
  ```
77
  @article{mochtak_chasing_2024,
78
+ title = {Chasing the authoritarian spectre: {Detecting} authoritarian discourse with large language models},
79
+ issn = {1475-6765},
80
+ shorttitle = {Chasing the authoritarian spectre},
81
+ url = {https://onlinelibrary.wiley.com/doi/abs/10.1111/1475-6765.12740},
82
+ doi = {10.1111/1475-6765.12740},
83
+ journal = {European Journal of Political Research},
84
  author = {Mochtak, Michal},
85
+ keywords = {authoritarian discourse, deep learning, detecting authoritarianism, model, political discourse},
86
  }
87
  ```