Hugging Face's logo

ELCA-SA
/
BPMN_Detection

English
Model card Files Community
BPMN_Detection / README.md
BenjiELCA's picture
BenjiELCA
Update README.md
311dd05 verified
preview code
|
raw
history blame
9.41 kB
metadata
license: mpl-2.0
language:
  - en
metrics:
  - f1
  - accuracy
  - recall
  - precision

license: apache-2.0

BPMN element detection

Model description

This project aims to detect Business Process Model and Notation (BPMN) elements from hand-drawn diagrams using a machine learning model. The model is trained to recognize various BPMN elements such as tasks, events, gateways, and connectors from images of hand-drawn diagrams.

The dataset contains 15 target labels:

  • AGENT

    • pool
    • lane

  • TASK

    • task
    • subProcess

  • TASK_INFO

    • dataObject
    • dataStore

  • PROCESS_INFO

    • background

  • CONDITION

    • exclusiveGateway
    • parallelGateway
    • eventBasedGateway

  • EVENT

    • event
    • messageEvent
    • timerEvent

  • FLOW

    • sequenceFlow
    • dataAssociation
    • messageFlow

Results per type

It achieves the following results on the evaluation set with objects:

  • Labels Precision: 0.97
  • Precision: 0.97
  • Recall: 0.95
  • F1: 0.96

It achieves the following results on the evaluation set with arrows:

  • Labels precision: 0.98
  • Precision: 0.92
  • Recall: 0.93
  • F1: 0.92
  • Keypoints Accuracy: 0.71

Results per class

Class Precision Recall F1
task 0.9518 0.9875 0.9693
exclusiveGateway 0.9548 0.9427 0.9487
event 0.9515 0.9235 0.9373
parallelGateway 0.9333 0.9180 0.9256
messageEvent 0.9291 0.9365 0.9328
pool 0.8797 0.936 0.9070
lane 0.9178 0.67 0.7746
dataObject 0.9333 0.9565 0.9448
dataStore 1.0 0.64 0.7805
eventBasedGateway 0.7273 0.7273 0.7273
timerEvent 0.8571 0.75 0.8
sequenceFlow 0.9292 0.9605 0.9446
dataAssociation 0.8472 0.8095 0.8279
messageFlow 0.8589 0.7910 0.8235

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0176
  • train_batch_size: 4
  • eval_batch_size: 1
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 50

Example of Training results

Epoch Avg Loss Test Loss Classifier Loss Box Reg Loss Objectness Loss RPN Box Reg Loss Precision Recall F1 Score
1 3.9451 2.0591 2.4416 0.5426 0.6502 0.3107 0.2763 0.0393 0.0689
2 2.7259 1.5387 1.6724 0.6697 0.1868 0.1969 0.5754 0.3358 0.4241
3 2.2004 1.1307 1.3860 0.5330 0.1216 0.1598 0.8657 0.6841 0.7643
4 1.8611 1.0110 1.1775 0.4172 0.1099 0.1565 0.7708 0.7790 0.7749
5 1.7461 0.9593 1.1202 0.3820 0.0971 0.1468 0.8542 0.8046 0.8287
6 1.5859 0.8956 0.9986 0.3590 0.0872 0.1412 0.8884 0.8002 0.8420
7 1.5621 0.9073 1.0214 0.3351 0.0776 0.1280 0.9435 0.8034 0.8678
8 1.5194 0.8695 0.9881 0.3261 0.0738 0.1314 0.9048 0.8246 0.8628
9 1.5449 0.9014 1.0105 0.3229 0.0769 0.1346 0.9478 0.8046 0.8704
10 1.5805 0.8134 1.0333 0.3338 0.0703 0.1431 0.8920 0.8920 0.8920
11 1.3838 0.8097 0.8743 0.3065 0.0653 0.1376 0.9634 0.8371 0.8958
12 1.3582 0.7362 0.8751 0.2909 0.0617 0.1306 0.9457 0.8596 0.9006
13 1.3126 0.7149 0.8347 0.2921 0.0593 0.1264 0.9152 0.9295 0.9223
14 1.3532 0.7775 0.9079 0.2783 0.0543 0.1128 0.9639 0.8508 0.9038
15 1.3188 0.6738 0.8986 0.2720 0.0434 0.1048 0.8856 0.9419 0.9129
16 1.2512 0.7478 0.7840 0.2784 0.0621 0.1268 0.9181 0.9101 0.9141
17 1.2909 0.6556 0.8425 0.2778 0.0547 0.1159 0.9012 0.9282 0.9145
18 1.2526 0.7003 0.8442 0.2607 0.0443 0.1034 0.9169 0.9020 0.9094
19 1.1980 0.7136 0.8062 0.2528 0.0361 0.1029 0.9520 0.9157 0.9335
20 1.1821 0.6308 0.7895 0.2517 0.0378 0.1030 0.9023 0.9513 0.9262
21 1.0843 0.6883 0.7168 0.2402 0.0316 0.0957 0.9348 0.9032 0.9187
22 1.1058 0.6192 0.7367 0.2336 0.0374 0.0981 0.9321 0.9513 0.9416
23 1.0699 0.5962 0.7119 0.2340 0.0306 0.0935 0.9353 0.9476 0.9414
24 1.0616 0.6674 0.7031 0.2367 0.0311 0.0908 0.9418 0.9301 0.9359
25 1.0784 0.6158 0.7275 0.2311 0.0295 0.0904 0.9176 0.9320 0.9247
26 1.0618 0.6483 0.7121 0.2283 0.0297 0.0916 0.9411 0.9182 0.9295
27 1.0530 0.5958 0.7139 0.2236 0.0279 0.0876 0.9477 0.9395 0.9436
28 1.0452 0.5964 0.7097 0.2223 0.0283 0.0849 0.9465 0.9494 0.9480
29 1.0966 0.6288 0.7795 0.2176 0.0203 0.0792 0.9558 0.9320 0.9437
30 1.0506 0.5956 0.7312 0.2142 0.0195 0.0856 0.9370 0.9370 0.9370
31 1.0030 0.6099 0.6777 0.2163 0.0204 0.0886 0.9506 0.9251 0.9377
32 0.9748 0.5976 0.6610 0.2098 0.0201 0.0839 0.9527 0.9313 0.9419
33 0.9540 0.5907 0.6402 0.2059 0.0216 0.0863 0.9536 0.9238 0.9385
34 0.9730 0.5809 0.6500 0.2076 0.0281 0.0873 0.9407 0.9413 0.9410
35 0.9894 0.5837 0.6831 0.2066 0.0202 0.0794 0.9451 0.9345 0.9397
36 0.9042 0.5534 0.5873 0.2096 0.0214 0.0860 0.9460 0.9519 0.9490
37 0.9546 0.5562 0.6400 0.2112 0.0216 0.0818 0.9260 0.9457 0.9358
38 0.9806 0.5792 0.6800 0.2031 0.0175 0.0800 0.9476 0.9363 0.9419
39 0.9294 0.5703 0.6247 0.2016 0.0204 0.0826 0.9401 0.9501 0.9450
40 0.9786 0.5880 0.6733 0.2010 0.0268 0.0775 0.9375 0.9170 0.9271
41 1.0026 0.5875 0.7073 0.2033 0.0179 0.0742 0.9476 0.9251 0.9362
42 0.9567 0.5724 0.6677 0.1992 0.0164 0.0734 0.9468 0.9332 0.9400
43 0.8747 0.5709 0.5794 0.1980 0.0159 0.0814 0.9557 0.9432 0.9494
44 1.0310 0.5497 0.7392 0.1956 0.0254 0.0709 0.9589 0.9313 0.9449
45 0.9526 0.5580 0.6598 0.1982 0.0185 0.0762 0.9401 0.9413 0.9407
46 0.8753 0.5548 0.5940 0.1939 0.0176 0.0698 0.9468 0.9438 0.9453
47 0.9328 0.5735 0.6493 0.1953 0.0163 0.0720 0.9534 0.9320 0.9426
48 0.9019 0.5605 0.6071 0.2002 0.0182 0.0765 0.9496 0.9413 0.9455
49 0.8335 0.5637 0.5459 0.1918 0.0175 0.0783 0.9588 0.9307 0.9446
50 0.9043 0.5617 0.6179 0.1933 0.0154 0.0776 0.9597 0.9370 0.9482