AhmedSSabir
/

BERT-CNN-Visual-Semantic

Model card Files Files and versions Community

AhmedSSabir commited on Jun 10, 2022

Commit

5043278

•

1 Parent(s): 3c44d8a

Update README.md

Files changed (1) hide show

README.md +1 -2

README.md CHANGED Viewed

@@ -1,7 +1,7 @@
 # Visual semantic with BERT-CNN
-This model can be used to assign an object-to-caption semantic relatedness score, which is valuable for
 (1) caption diverse re-ranking, and (2) generate soft labels for caption filtering when scraping text-to-captions from the internet.
 To take advantage of the overlapping between the visual context and the caption, and to extract global information from each visual (i.e., object, scene, etc) we use BERT  as an embedding layer followed by a shallow CNN (tri-gram kernel) (Kim, 2014).
@@ -333,7 +333,6 @@ class BertLayer(tf.keras.layers.Layer):
 df = pd.read_csv(args.train, sep='\t')
-#labels = df.is_duplicate.values
 labels = df.is_related.values
 texts = []

 # Visual semantic with BERT-CNN
+This model can be used to assign an object-to-caption semantic ness score, which is valuable for
 (1) caption diverse re-ranking, and (2) generate soft labels for caption filtering when scraping text-to-captions from the internet.
 To take advantage of the overlapping between the visual context and the caption, and to extract global information from each visual (i.e., object, scene, etc) we use BERT  as an embedding layer followed by a shallow CNN (tri-gram kernel) (Kim, 2014).
 df = pd.read_csv(args.train, sep='\t')
 labels = df.is_related.values
 texts = []