Best Practices for Auto QA Configuration

The following table describes the best practices that are based on by Question metrics.

Category	Best Practices
General Configuration Principles	Maintain diversity and realism in metric configurations. Ensure configurations reflect plausible agent responses. Review and update configurations to reflect current use cases regularly.
Sentence Length and Complexity	Limit configurations to concise utterances, typically 1-3 sentences. Avoid configuring very long sentences or extensive text passages.
Handling Specific Identifiers	Company and Brand Names Include when necessary, but be aware they can impact semantic adherence. Implement moderate thresholds (for example, 50-60% match) to accommodate variations Agent Names. Minimize the use of specific agent names in configurations. Use [Agent Name] as a placeholder where the agent references themself or any other agents Product Names and Model IDs. Avoid including specific model identifiers unless necessary. If included, expect precise usage in agent responses. Implement moderate thresholds (for example, 50-60% match) for these configurations.
Threshold Recommendations	General configurations: 60-70% match. Configurations with company/brand names for policy: 70-80% match. Configurations with product specifics: 50-60% match. Note: Adjust these thresholds based on your specific use case and requirements.
Testing and Validation	Validate configurations against real agent responses. Monitor and analyze adherence rates to identify optimization areas for reconfiguration.