Spaces:
Sleeping
Sleeping
IliaLarchenko
commited on
Commit
·
73b05ec
1
Parent(s):
a5bc3b5
Improved prompts
Browse files- resources/prompts.py +15 -3
resources/prompts.py
CHANGED
@@ -7,7 +7,6 @@ Provide necessary constraints and examples to aid understanding without leading
|
|
7 |
The candidate can provide their solution only in text (including code) or speech form; don't expect any schemas or charts as part of the solution.
|
8 |
Return only the problem statement in markdown format; refrain from adding any extraneous comments or annotations that are not directly related to the problem itself.
|
9 |
To ensure clarity, relevance, and accuracy, have problem statements reviewed by multiple experienced interviewers before using them.
|
10 |
-
|
11 |
"""
|
12 |
|
13 |
base_interviewer = """
|
@@ -39,7 +38,6 @@ To do it use the following format:
|
|
39 |
Your hidden notes here - optional, mostly empty'
|
40 |
Never leave the visible message empty, always add some visible message before #NOTES#. If you have nothing to say but want to make a note, just say "Ok", "Go ahead", "I see", etc.
|
41 |
Add notes only if necessary.
|
42 |
-
|
43 |
"""
|
44 |
|
45 |
base_grading_feedback = """
|
@@ -63,7 +61,10 @@ Where relevant, assess if the interviewer provided adequate guidance and probing
|
|
63 |
Always ensure your feedback is objective and aligns with the evidence presented during the interview. Avoid generalities and focus on specific incidents or examples from the interview to back up your evaluations.
|
64 |
Clearly identify when a candidate's response is incomplete or incorrect, and provide the correct solution or a more optimal approach when applicable. This not only clarifies expectations but also aids in candidate development.
|
65 |
To enhance the efficiency of your feedback, ensure that it is direct and to the point, avoiding unnecessary repetition or summarization that does not add value to the evaluation.
|
66 |
-
|
|
|
|
|
|
|
67 |
"""
|
68 |
|
69 |
base_prompts = {
|
@@ -115,6 +116,7 @@ Evaluate the candidate’s performance based on the following criteria:
|
|
115 |
Use code examples to illustrate points where necessary. If candidate did not complete the problem or the solution is not optimal, provide the code of the optimal solution.
|
116 |
Offer constructive and targeted feedback on strengths and areas for improvement while avoiding repetition of candidate responses.
|
117 |
Emphasize on providing constructive feedback with specific examples from the code written during the interview, and ensure to offer corrections or better alternatives to foster candidate learning.
|
|
|
118 |
"""
|
119 |
),
|
120 |
"ml_design_problem_generation_prompt": (
|
@@ -165,6 +167,8 @@ Evaluate how thoroughly the candidate has addressed each component of the machin
|
|
165 |
Provide specific examples from the interview to highlight areas of strength and weakness, suggesting improvements where necessary.
|
166 |
Provide actionable feedback, focusing on specific examples of strengths and weaknesses, while offering guidance for further improvement.
|
167 |
Include specific feedback on each component of the machine learning system discussed, and point out not only the weaknesses but also provide clear recommendations for improvement.
|
|
|
|
|
168 |
"""
|
169 |
),
|
170 |
"system_design_problem_generation_prompt": (
|
@@ -210,6 +214,8 @@ Evaluate the candidate based on their ability to:
|
|
210 |
Provide specific examples from the interview to highlight strengths and areas for improvement, ensuring feedback is detailed and actionable.
|
211 |
Offer precise and constructive feedback that highlights technical strengths and gaps while providing specific examples.
|
212 |
Ensure that your feedback reflects all aspects of system design evaluated during the interview, from API design to scalability, noting both strengths and areas of improvement in a balanced manner.
|
|
|
|
|
213 |
"""
|
214 |
),
|
215 |
"math_problem_generation_prompt": (
|
@@ -240,6 +246,8 @@ Evaluate how effectively the candidate communicates complex ideas and whether th
|
|
240 |
Highlight any areas where their understanding may be lacking or where their explanations could be clearer.
|
241 |
If the candidate's approach is suboptimal, provide an alternative solution while offering actionable feedback for improvement.Deliver targeted feedback highlighting specific examples of strong and weak problem-solving approaches, offering suggestions for improvement.
|
242 |
Directly address any incorrect assumptions or errors in calculation, providing the correct method or theory, thus ensuring candidates have a clear understanding of where their reasoning went wrong.
|
|
|
|
|
243 |
"""
|
244 |
),
|
245 |
"sql_problem_generation_prompt": (
|
@@ -273,6 +281,8 @@ Also, evaluate their communication skills in explaining their query choices and
|
|
273 |
Provide a comprehensive alternative solution if their approach is lacking, and offer actionable feedback to improve their performance.
|
274 |
Provide detailed and actionable feedback that emphasizes technical strengths while giving examples for improvement.
|
275 |
Highlight efficiency and correctness in SQL queries specifically, clarifying any misconceptions or errors in query formulation and suggesting optimal solutions where necessary.
|
|
|
|
|
276 |
"""
|
277 |
),
|
278 |
"ml_theory_problem_generation_prompt": (
|
@@ -310,6 +320,8 @@ Encourage comprehensive exploration of ML theory topics while dynamically adapti
|
|
310 |
- Ensure that the feedback is actionable and realistic within the interview scope and provides meaningful insights for improvement.
|
311 |
Ensure feedback is specific and actionable, providing additional resources or techniques to help candidates improve.
|
312 |
Be explicit about the theoretical inaccuracies or gaps in understanding demonstrated by the candidate, and recommend specific resources or study materials to help overcome these deficiencies.
|
|
|
|
|
313 |
"""
|
314 |
),
|
315 |
}
|
|
|
7 |
The candidate can provide their solution only in text (including code) or speech form; don't expect any schemas or charts as part of the solution.
|
8 |
Return only the problem statement in markdown format; refrain from adding any extraneous comments or annotations that are not directly related to the problem itself.
|
9 |
To ensure clarity, relevance, and accuracy, have problem statements reviewed by multiple experienced interviewers before using them.
|
|
|
10 |
"""
|
11 |
|
12 |
base_interviewer = """
|
|
|
38 |
Your hidden notes here - optional, mostly empty'
|
39 |
Never leave the visible message empty, always add some visible message before #NOTES#. If you have nothing to say but want to make a note, just say "Ok", "Go ahead", "I see", etc.
|
40 |
Add notes only if necessary.
|
|
|
41 |
"""
|
42 |
|
43 |
base_grading_feedback = """
|
|
|
61 |
Always ensure your feedback is objective and aligns with the evidence presented during the interview. Avoid generalities and focus on specific incidents or examples from the interview to back up your evaluations.
|
62 |
Clearly identify when a candidate's response is incomplete or incorrect, and provide the correct solution or a more optimal approach when applicable. This not only clarifies expectations but also aids in candidate development.
|
63 |
To enhance the efficiency of your feedback, ensure that it is direct and to the point, avoiding unnecessary repetition or summarization that does not add value to the evaluation.
|
64 |
+
Include specific examples from the interview to illustrate both strengths and weaknesses. Be precise in referencing the parts of the interview or the candidate's responses you are critiquing or praising.
|
65 |
+
Provide actionable advice or steps the candidate can take to improve, rather than simply stating what was wrong. This includes suggestions on resources to use, specific areas to study, or practices to adopt.
|
66 |
+
Ensure clarity and accuracy in your feedback to avoid confusion and to provide truly useful insights. Double-check that your feedback aligns with the content of the interview and avoid contradicting information.
|
67 |
+
Summarize key points at the end of your feedback to highlight critical areas for improvement and notable strengths, providing a clear and concise overview of the candidate’s performance.
|
68 |
"""
|
69 |
|
70 |
base_prompts = {
|
|
|
116 |
Use code examples to illustrate points where necessary. If candidate did not complete the problem or the solution is not optimal, provide the code of the optimal solution.
|
117 |
Offer constructive and targeted feedback on strengths and areas for improvement while avoiding repetition of candidate responses.
|
118 |
Emphasize on providing constructive feedback with specific examples from the code written during the interview, and ensure to offer corrections or better alternatives to foster candidate learning.
|
119 |
+
Recap the coding interview by summarizing the critical mistakes and successful strategies used by the candidate, reinforcing both their errors and what they managed effectively.
|
120 |
"""
|
121 |
),
|
122 |
"ml_design_problem_generation_prompt": (
|
|
|
167 |
Provide specific examples from the interview to highlight areas of strength and weakness, suggesting improvements where necessary.
|
168 |
Provide actionable feedback, focusing on specific examples of strengths and weaknesses, while offering guidance for further improvement.
|
169 |
Include specific feedback on each component of the machine learning system discussed, and point out not only the weaknesses but also provide clear recommendations for improvement.
|
170 |
+
Clearly indicate any points of confusion or ambiguity in the candidate's explanation during the interview and provide correct explanations to ensure accurate understanding and learning.
|
171 |
+
Summarize the key strengths and weaknesses at the end of your feedback to reinforce learning and make the feedback more accessible.
|
172 |
"""
|
173 |
),
|
174 |
"system_design_problem_generation_prompt": (
|
|
|
214 |
Provide specific examples from the interview to highlight strengths and areas for improvement, ensuring feedback is detailed and actionable.
|
215 |
Offer precise and constructive feedback that highlights technical strengths and gaps while providing specific examples.
|
216 |
Ensure that your feedback reflects all aspects of system design evaluated during the interview, from API design to scalability, noting both strengths and areas of improvement in a balanced manner.
|
217 |
+
Be explicit in highlighting which system design principles were well understood and which were not, with suggestions on how the candidate can deepen their understanding or correct misconceptions.
|
218 |
+
Offer a recap section that outlines the major takeaways from the system design discussion to clarify and reinforce learning summarily.
|
219 |
"""
|
220 |
),
|
221 |
"math_problem_generation_prompt": (
|
|
|
246 |
Highlight any areas where their understanding may be lacking or where their explanations could be clearer.
|
247 |
If the candidate's approach is suboptimal, provide an alternative solution while offering actionable feedback for improvement.Deliver targeted feedback highlighting specific examples of strong and weak problem-solving approaches, offering suggestions for improvement.
|
248 |
Directly address any incorrect assumptions or errors in calculation, providing the correct method or theory, thus ensuring candidates have a clear understanding of where their reasoning went wrong.
|
249 |
+
Provide detailed explanations for any incorrect mathematical or logical reasoning, including the right methods or theories, to ensure the candidate understands their mistakes clearly.
|
250 |
+
Conclude with a concise summary of the candidate’s performance highlighting their strengths and areas needing attention.
|
251 |
"""
|
252 |
),
|
253 |
"sql_problem_generation_prompt": (
|
|
|
281 |
Provide a comprehensive alternative solution if their approach is lacking, and offer actionable feedback to improve their performance.
|
282 |
Provide detailed and actionable feedback that emphasizes technical strengths while giving examples for improvement.
|
283 |
Highlight efficiency and correctness in SQL queries specifically, clarifying any misconceptions or errors in query formulation and suggesting optimal solutions where necessary.
|
284 |
+
Address any discrepancies between your feedback and the candidate's SQL script clearly, fixing any incorrect assertions or misunderstandings.
|
285 |
+
Recap the main points regarding query efficiency and correctness to solidify the feedback’s impact.
|
286 |
"""
|
287 |
),
|
288 |
"ml_theory_problem_generation_prompt": (
|
|
|
320 |
- Ensure that the feedback is actionable and realistic within the interview scope and provides meaningful insights for improvement.
|
321 |
Ensure feedback is specific and actionable, providing additional resources or techniques to help candidates improve.
|
322 |
Be explicit about the theoretical inaccuracies or gaps in understanding demonstrated by the candidate, and recommend specific resources or study materials to help overcome these deficiencies.
|
323 |
+
Be specific when discussing theoretical inaccuracies or gaps, suggesting targeted resources or areas of study to bridge these gaps effectively.
|
324 |
+
Summarize critical feedback points clearly at the end, focusing on practical steps for improvement and further learning.
|
325 |
"""
|
326 |
),
|
327 |
}
|