We describe an approach to automatically distinguish bot-answerable questions, which would mitigate this problem.
f 0.5 is more appropriate here than the standard f score because it places more emphasis on precision.