Initial commit

README.md

@@ -0,0 +1,179 @@
+---
+license: cc-by-nc-4.0
+language:
+- en
+metrics:
+- f1
+pipeline_tag: text-classification
+widget:
+- text: "Men shouldn’t be making laws about women’s bodies #abortion #Texas"
+  example_title: "Statement"
+- text: "’Bitter truth’: EU chief pours cold water on idea of Brits keeping EU citizenship after #Brexit HTTPURL via @USER"
+  example_title: "Notification"
+- text: "Opinion: As the draconian (and then some) abortion law takes effect in #Texas, this is not an idle question for millions of Americans. A slippery slope towards more like-minded Republican state legislatures to try to follow suit. #abortion #F24 HTTPURL"
+  example_title: "Reason"
+- text: "@USER Blah blah blah blah blah blah"
+  example_title: "None"
+- text: "republican men and karens make me sick"
+  example_title: "Unlabeled 1"
+- text: "No empire lives forever! Historical fact! GodWins! 🙏💪🇺🇲"
+  example_title: "Unlabeled 2"
+- text: "Further author information regarding registration and visa support letters will be sent to the authors soon. #CIKM2023"
+  example_title: "Unlabeled 3"
+- text: "Ummmmmm"
+  example_title: "Unlabeled 4"
+- text: "whoever says that The Last Jedi is a good movie is lying or trolling everyone"
+  example_title: "Unlabeled 5"
+---
+
+# TACO -- Twitter Arguments from COnversations
+
+Introducing TACO, a baseline classification model built upon `AutoModelForSequenceClassification`, designed to identify tweets belonging to four distinct
+classes: Reason, Statement, Notification, and None. Tailored specifically for argument mining on Twitter, this baseline model is an evolution of the 
+[BERTweet-base](https://huggingface.co/vinai/bertweet-base) architecture, which was originally pre-trained on Twitter data. 
+Through fine-tuning with the [TACO dataset](https://doi.org/10.5281/zenodo.8030026), the baseline model acquires its name and excels in the 
+extraction of *Twitter Arguments from COnversations*.
+
+## Class Semantics
+
+The TACO framework revolves around the two key elements of an argument, as defined by the [Cambridge Dictionary](https://dictionary.cambridge.org).
+It encodes *inference* as *a guess that you make or an opinion that you form based on the information that you have*, and it also leverages the
+definition of *information* as *facts or details about a person, company, product, etc.*.
+
+Taken together, the following classes of tweets can be identified by TACO:
+
+* *Statement*, which refers to unique cases where only the *inference* is presented as *something that someone says or writes officially, or an action
+done to express an opinion* (see ex. 1).
+* *Reason*, which represents a full argument where the *inference* is based on direct *information* mentioned in the tweet, such as a source-reference
+or quotation, and thus reveals the author’s motivation *to try to understand and to make judgments based on practical facts* (see ex. 3).
+* *Notification*, which refers to a tweet that limits itself to providing *information*, such as media channels promoting their latest articles 
+(see ex. 2).
+* *None*, a tweet that provides neither *inference* nor *information* (see ex. 4).
+
+
+In its entirety, TACO can classify the following hierarchy for tweets:
+
+![image](https://www.researchgate.net/profile/Marc-Feger/publication/371595900/figure/fig1/AS:11431281168142295@1686846469455/Hierarchy-of-arguments-with-constituting-elements_W640.jpg)
+
+## Usage
+
+Using this model becomes easy when you have `transformers` installed:
+
+```python
+pip install - U transformers
+```
+
+Then you can use the model to generate tweet classifications like this:
+
+```python
+from transformers import pipeline
+
+pipe = pipeline("text-classification", model="TomatenMarc/TACO")
+prediction = pipe("Huggingface is awesome")
+
+print(prediction)
+```
+
+<a href="https://github.com/TomatenMarc/TACO/blob/main/notebooks/classifier_cv.ipynb">
+    <blockquote style="border-left: 5px solid grey; background-color: #f0f5ff; padding: 10px;">
+        Notice: The tweets need to undergo preprocessing before classification.
+    </blockquote>
+</a>
+
+## Training
+
+The final model underwent training using the entire shuffled ground truth dataset known as TACO, encompassing a total of 1734 tweets.
+This dataset showcases the distribution of topics as: #abortion (25.9%), #brexit (29.0%), #got (11.0%), #lotrrop (12.1%), #squidgame (12.7%), and
+#twittertakeover (9.3%). For training, we utilized [SimpleTransformers](https://simpletransformers.ai).
+
+Additionally, the category and class distribution of the dataset TACO is as follows:
+
+| Argument     | No-Argument      |
+|--------------|------------------|
+| 865 (49.88%) | 869 (50.12%)     |
+
+| Reason       | Statement    | Notification | None         |
+|--------------|--------------|--------------|--------------|
+| 581 (33.50%) | 284 (16.38%) | 500 (28.84%) | 369 (21.28%) |
+
+<p>
+    <blockquote style="border-left: 5px solid grey; background-color: #f0f5ff; padding: 10px;">
+        Notice: Our training involved TACO to forecast class predictions, where the categories (Argument/No-Argument) represent class aggregations 
+based on the inference component.
+    </blockquote>
+<p>
+
+### Dataloader
+
+```
+"data_loader": {
+    "type": "torch.utils.data.dataloader.DataLoader",
+    "args": {
+        "batch_size": 8,
+        "sampler": "torch.utils.data.sampler.RandomSampler"
+    }
+}
+```
+
+Parameters of the fit()-Method:
+
+```
+{
+    "epochs": 5,
+    "max_grad_norm": 1,
+    "optimizer_class": "<class 'torch.optim.adamw.AdamW'>",
+    "optimizer_params": {
+        "lr": 4e-05
+    },
+    "scheduler": "WarmupLinear",
+    "warmup_steps": 66,
+    "weight_decay": 0.06
+}
+```
+
+## Evaluation
+
+We utilized a stratified 10-fold cross-validation approach to present TACO's performance. In doing so, we employed the identical data and parameters
+as outlined in the *Training* section. This involved training on k-1 splits and utilizing the kth split for making predictions.
+
+In total, the TACO classifier performs as follows:
+
+### Classification
+
+|             | Precision | Recall  | F1-Score | Support |
+|-------------|-----------|---------|----------|---------|
+| Reason      | 73.69%    | 75.22%  | 74.45%   | 581     |
+| Statement   | 54.37%    | 59.15%  | 56.66%   | 284     |
+| Notification| 79.02%    | 77.60%  | 78.30%   | 500     |
+| None        | 83.87%    | 77.51%  | 80.56%   | 369     |
+|-------------|-----------|---------|----------|---------|
+| Accuracy    |           |         | 73.76%   | 1734    |
+| Macro Avg   | 72.74%    | 72.37%  | 72.49%   | 1734    |
+| Weighted Avg| 74.23%    | 73.76%  | 73.95%   | 1734    |
+
+### Categorization
+
+|             | Precision | Recall  | F1-Score | Support |
+|-------------|-----------|---------|----------|---------|
+| No-Argument | 86.66%    | 82.97%  | 84.77%   | 869     |
+| Argument    | 83.59%    | 87.17%  | 85.34%   | 865     |
+|-------------|-----------|---------|----------|---------|
+| Accuracy    |           |         | 85.06%   | 1734    |
+| Macro Avg   | 85.13%    | 85.07%  | 85.06%   | 1734    |
+| Weighted Avg| 85.13%    | 85.06%  | 85.06%   | 1734    |
+
+# Environmental Impact
+
+- **Hardware Type:** A100 PCIe 40GB
+- **Hours used:** 10 min
+- **Cloud Provider:** [Google Cloud Platform](https://colab.research.google.com)
+- **Compute Region:** [asia-southeast1](https://cloud.google.com/compute/docs/gpus/gpu-regions-zones?hl=en) (Singapore)
+- **Carbon Emitted:** 0.02kg CO2
+
+## Licensing
+
+[TACO](https://huggingface.co/TomatenMarc/TACO) © 2023 by [Marc Feger](mailto:marc.feger@uni-duesseldorf.de) is licensed under [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/?ref=chooser-v1)
+
+## Contact
+
+If you have any questions, please feel free to reach out to [marc.feger@uni-duesseldorf.de](mailto:marc.feger@uni-duesseldorf.de).

added_tokens.json

@@ -0,0 +1,3 @@
+{
+  "<mask>": 64000
+}

config.json

@@ -0,0 +1,42 @@
+{
+  "_name_or_path": "./model/TACO",
+  "architectures": [
+    "RobertaForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "Reason",
+    "1": "Statement",
+    "2": "Notification",
+    "3": "None"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "None": 3,
+    "Notification": 2,
+    "Reason": 0,
+    "Statement": 1
+  },
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 130,
+  "model_type": "roberta",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "problem_type": "single_label_classification",
+  "tokenizer_class": "BertweetTokenizer",
+  "torch_dtype": "float32",
+  "transformers_version": "4.30.2",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 64001
+}

model_args.json

@@ -0,0 +1 @@
+{"adafactor_beta1": null, "adafactor_clip_threshold": 1.0, "adafactor_decay_rate": -0.8, "adafactor_eps": [1e-30, 0.001], "adafactor_relative_step": true, "adafactor_scale_parameter": true, "adafactor_warmup_init": true, "adam_betas": [0.9, 0.999], "adam_epsilon": 1e-08, "best_model_dir": "outputs/best_model", "cache_dir": "cache_dir/", "config": {}, "cosine_schedule_num_cycles": 0.5, "custom_layer_parameters": [], "custom_parameter_groups": [], "dataloader_num_workers": 0, "do_lower_case": false, "dynamic_quantize": false, "early_stopping_consider_epochs": false, "early_stopping_delta": 0, "early_stopping_metric": "eval_loss", "early_stopping_metric_minimize": true, "early_stopping_patience": 3, "encoding": null, "eval_batch_size": 8, "evaluate_during_training": false, "evaluate_during_training_silent": true, "evaluate_during_training_steps": 2000, "evaluate_during_training_verbose": false, "evaluate_each_epoch": true, "fp16": true, "gradient_accumulation_steps": 1, "learning_rate": 4e-05, "local_rank": -1, "logging_steps": 50, "loss_type": null, "loss_args": {}, "manual_seed": null, "max_grad_norm": 1.0, "max_seq_length": 128, "model_name": "vinai/bertweet-base", "model_type": "bertweet", "multiprocessing_chunksize": -1, "n_gpu": 1, "no_cache": true, "no_save": false, "not_saved_args": [], "num_train_epochs": 5, "optimizer": "AdamW", "output_dir": "./model/TACO", "overwrite_output_dir": true, "polynomial_decay_schedule_lr_end": 1e-07, "polynomial_decay_schedule_power": 1.0, "process_count": 1, "quantized_model": false, "reprocess_input_data": true, "save_best_model": true, "save_eval_checkpoints": false, "save_model_every_epoch": false, "save_optimizer_and_scheduler": true, "save_steps": 2000, "scheduler": "linear_schedule_with_warmup", "silent": false, "skip_special_tokens": true, "tensorboard_dir": null, "thread_count": null, "tokenizer_name": "vinai/bertweet-base", "tokenizer_type": null, "train_batch_size": 8, "train_custom_parameters_only": false, "use_cached_eval_features": false, "use_early_stopping": false, "use_hf_datasets": false, "use_multiprocessing": true, "use_multiprocessing_for_evaluation": true, "wandb_kwargs": {}, "wandb_project": null, "warmup_ratio": 0.06, "warmup_steps": 66, "weight_decay": 0.0, "model_class": "ClassificationModel", "labels_list": [0, 1, 2, 3], "labels_map": {}, "lazy_delimiter": "\t", "lazy_labels_column": 1, "lazy_loading": false, "lazy_loading_start_line": 1, "lazy_text_a_column": null, "lazy_text_b_column": null, "lazy_text_column": 0, "onnx": false, "regression": false, "sliding_window": false, "special_tokens_list": [], "stride": 0.8, "tie_value": 1}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4056bc4e6aada20e6f8caafd283078b609ab06cd6bb1fa1cba8fc28b74d600c1
+size 539682485

special_tokens_map.json

@@ -0,0 +1,9 @@
+{
+  "bos_token": "<s>",
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "mask_token": "<mask>",
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "unk_token": "<unk>"
+}

tokenizer_config.json

@@ -0,0 +1,14 @@
+{
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "<s>",
+  "do_lower_case": false,
+  "eos_token": "</s>",
+  "mask_token": "<mask>",
+  "model_max_length": 128,
+  "normalization": true,
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "tokenizer_class": "BertweetTokenizer",
+  "unk_token": "<unk>"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eb8e68af90bd93faab831a5666ee353336ee96fd6f70a8c18fc486f51acbd20f
+size 3195