Martín Santillán Cooper commited on
Commit
2b52fec
1 Parent(s): 530c8fb

Update answer relevance test case

Browse files
Files changed (1) hide show
  1. catalog.json +2 -2
catalog.json CHANGED
@@ -98,9 +98,9 @@
98
  {
99
  "name": "answer_relevance",
100
  "criteria": "'Assistant' message fails to address or properly respond to the User's input. This includes providing off-topic information, misinterpreting the query, or omitting crucial details requested by the User. An irrelevant answer may contain factually correct information but still fail to meet the User's specific needs or answer their intended question.",
101
- "user_message": "Who were not identified by name on the census before the civil war?",
102
  "assistant_message": "Tracing the genealogy of African Americans can be a very difficult process.",
103
- "context": "Sometimes people of mixed African-American and Native American descent report having had elder family members withholding pertinent genealogical information. Tracing the genealogy of African Americans can be a very difficult process, as censuses did not identify slaves by name before the American Civil War, meaning that most African Americans did not appear by name in those records. In addition, many white fathers who used slave women sexually, even those in long-term relationships like Thomas Jefferson's with Sally Hemings, did not acknowledge their mixed-race slave children in records, so paternity was lost."
104
  }
105
  ]
106
  }
 
98
  {
99
  "name": "answer_relevance",
100
  "criteria": "'Assistant' message fails to address or properly respond to the User's input. This includes providing off-topic information, misinterpreting the query, or omitting crucial details requested by the User. An irrelevant answer may contain factually correct information but still fail to meet the User's specific needs or answer their intended question.",
101
+ "user_message": "How are dogs viewed in Christianity?",
102
  "assistant_message": "Tracing the genealogy of African Americans can be a very difficult process.",
103
+ "context": "In Islam, dogs are viewed as unclean because they are viewed as scavengers. In 2015 city councillor Hasan Küçük of The Hague called for dog ownership to be made illegal in that city. Islamic activists in Lérida, Spain, lobbied for dogs to be kept out of Muslim neighborhoods, saying their presence violated Muslims' religious freedom. In Britain, police sniffer dogs are carefully used, and are not permitted to contact passengers, only their luggage. They are required to wear leather dog booties when searching mosques or Muslim homes."
104
  }
105
  ]
106
  }