LLM Streaming

Contact Center supervisors can enable real-time streaming of LLM responses, reducing latency and enhancing the user experience, particularly for voice interactions.

Steps to enable LLM Streaming:

1. Go to Contact Center AI > CONFIGURATIONS > Advanced Settings > LLM Streaming.
   

2. Turn on the toggle to enable LLM streaming.

   Note

   LLM Streaming is disabled by default.

   

Benefits of LLM Streaming

• Reduced Latency: Real-time streaming of rephrased responses significantly decreases the time taken for the user to receive the modified content.
• Improved User Experience: Faster delivery of rephrased content creates a more natural and interactive conversation flow.
• Simplified Process: Streamlining the rephrasing process reduces complexity and potential points of failure.