AhmedSSabir commited on
Commit
86eab5b
1 Parent(s): 54500ea

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -6
README.md CHANGED
@@ -1,7 +1,7 @@
1
 
2
  # Visual semantic with BERT-CNN
3
 
4
- This model can be used to assign an object-to-caption semantic ness score, which is valuable for
5
  (1) caption diverse re-ranking, and (2) generate soft labels for caption filtering when scraping text-to-captions from the internet.
6
 
7
  To take advantage of the overlapping between the visual context and the caption, and to extract global information from each visual (i.e., object, scene, etc) we use BERT as an embedding layer followed by a shallow CNN (tri-gram kernel) (Kim, 2014).
@@ -9,15 +9,13 @@ To take advantage of the overlapping between the visual context and the caption,
9
  Please refer to [Github](https://github.com/ahmedssabir/Visual-Semantic-Relatedness-Dataset-for-Image-Captioning) for more information.
10
 
11
 
12
-
13
  For datasets that are less than 100K please have look at our [shallow model](https://github.com/ahmedssabir/Semantic-Relatedness-Based-Reranker-for-Text-Spotting)
14
 
15
 
16
-
17
-
18
  The model is trained with a strict filter of 0.4 similarity distance thresholds between the object and its related caption.
19
 
20
- For a quick start please have a look at this [colab](https://colab.research.google.com/drive/1N0JVa6y8FKGLLSpiG7hd_W75UYhHRe2j?usp=sharing)
 
21
 
22
  For the [dataset](https://huggingface.co/datasets/AhmedSSabir/Textual-Image-Caption-Dataset)
23
 
@@ -33,7 +31,6 @@ pip install --upgrade tensorflow_hub==0.7.0
33
  git clone https://github.com/gaphex/bert_experimental/
34
  ```
35
 
36
-
37
  ```python
38
  import tensorflow as tf
39
  import numpy as np
 
1
 
2
  # Visual semantic with BERT-CNN
3
 
4
+ This model can be used to assign an object-to-caption semantic relatedness score, which is valuable for
5
  (1) caption diverse re-ranking, and (2) generate soft labels for caption filtering when scraping text-to-captions from the internet.
6
 
7
  To take advantage of the overlapping between the visual context and the caption, and to extract global information from each visual (i.e., object, scene, etc) we use BERT as an embedding layer followed by a shallow CNN (tri-gram kernel) (Kim, 2014).
 
9
  Please refer to [Github](https://github.com/ahmedssabir/Visual-Semantic-Relatedness-Dataset-for-Image-Captioning) for more information.
10
 
11
 
 
12
  For datasets that are less than 100K please have look at our [shallow model](https://github.com/ahmedssabir/Semantic-Relatedness-Based-Reranker-for-Text-Spotting)
13
 
14
 
 
 
15
  The model is trained with a strict filter of 0.4 similarity distance thresholds between the object and its related caption.
16
 
17
+ For a quick start please have a look at this [demo](https://github.com/ahmedssabir/Textual-Visual-Semantic-Dataset/blob/main/BERT_CNN_Visual_re_ranker_demo.ipynb)
18
+
19
 
20
  For the [dataset](https://huggingface.co/datasets/AhmedSSabir/Textual-Image-Caption-Dataset)
21
 
 
31
  git clone https://github.com/gaphex/bert_experimental/
32
  ```
33
 
 
34
  ```python
35
  import tensorflow as tf
36
  import numpy as np