Evaluation on datasets such as Quora
#3
by
nickmuchi
- opened
Wondering if you have run the above evaluations?
We have not but that should be easy to try. I'd guess this model will do ok on quora but won't be the best for two reasons:
- There is some skew between quora questions (that are full formed sentences) and web queries (that are very short and many grammatical elements are elided)
- More importantly, quora data is based on semantic similarity which is different from web intent similarity. e.g. "shockwave flash player" and "shockwave flash player download" are two different queries according to quora definition but are the same according to our definition of looking for web intents.
See our blog for more discussion on this
https://neeva.com/blog/state-of-the-art-query2query-similarity
Thanks!
nickmuchi
changed discussion status to
closed