Recently, Google's AI project Gemini has raised concerns about information accuracy due to new internal regulations. Reports indicate that outsourced personnel responsible for evaluating AI-generated content are now required to score areas in which they lack expertise, particularly on sensitive topics such as healthcare.
These outsourced personnel are from the global technology services company GlobalLogic. Google has asked them to evaluate AI-generated responses, focusing mainly on factors like "authenticity." Previously, outsourced personnel could choose to skip questions for which they had no relevant expertise; for example, they could opt not to evaluate specialized questions related to cardiology. This practice was intended to ensure the accuracy of assessments, allowing only those with relevant backgrounds to conduct effective evaluations.
However, last week, GlobalLogic announced Google's latest requirements, stating that outsourced personnel are no longer permitted to skip questions in specialized fields. Instead, they are now required to score the parts they understand and indicate their lack of knowledge in the relevant areas. This change has sparked widespread concern among the outsourced personnel, who believe this approach could impact Gemini's accuracy on certain complex topics.
For instance, some outsourced personnel mentioned in internal communications that the previous option to skip was meant to enhance scoring accuracy. However, the implementation of the new regulations forces them to evaluate issues where they have no experience, such as rare diseases. Internal emails revealed that the original guideline was: "If you do not have the necessary expertise for this task, please skip." The new rule states: "Do not skip prompts that require expertise." This shift in policy has left the outsourced personnel feeling uneasy.
Under the new regulations, outsourced personnel can only skip evaluation tasks in two situations: one is a complete lack of information, such as missing a full prompt or response; the other is if the content may be harmful, requiring special consent to evaluate. Although these new rules aim to improve Gemini's performance, they may affect its understanding and feedback on complex topics in practice.
Google has not responded to this issue, and the concerns of the outsourced personnel are gradually escalating.
Key Points:
🔍 Outsourced personnel are required to evaluate AI-generated responses in areas where they lack expertise, especially in sensitive fields like healthcare.
🚫 The new regulations eliminate the "skip" option, requiring outsourced personnel to score responses even when lacking professional knowledge.
📉 This policy may impact Gemini's accuracy on complex topics, raising unease and concerns among outsourced personnel.
