GenAI Prompt (BETA)¶

The GenAI Prompt lets bot developers leverage the full potential of LLM and Generative AI models to quickly build their own prompts. Developers can select a specific AI model, tweak its settings, and preview the response for the prompt. The node allows developers to creatively leverage LLMs by defining the prompt using conversation context and the response from the LLMs in defining the subsequent conversation flow.

Node Behavior¶

Runtime¶

You can work with this node like with any other node within Dialog Tasks and can invoke it within multiple tasks. During runtime, the node behaves as follows:

On reaching the GenAI Prompt, the platform parses any variable used in the prompt and constructs the request using the Prompt and the Advanced Settings.
An API call is made to the model with the request.
The response is stored in the context object as part of the dialog context and can be used to define the transitions or any other part of the bot configuration.
The platform exits from the GenAI Prompt node when a successful response is received, or the defined timeout condition is met.

Enable the Node¶

By default, the feature/node is disabled. To enable the feature, see Dynamic Conversations Features.

Add the Node¶

Go to Automation > Dialogs and select the task to which you want to add the GenAI Prompt.
Click Gen AI and then click GenAI Prompt. Alternatively, you can drag and drop the GenAI Prompt node to the required location on the canvas. For more information on adding nodes, see different ways to add a node.
The GenAI Prompt window is displayed with the Component Properties tab selected by default.

Configure the Node¶

Component Properties¶

The settings made within this section affect this node across all instances in all dialog tasks.

Provide Name and Display Name for the node and write your own OpenAI Prompt.
- Prompt: A prompt allows you to define the request to be sent to the LLMs for generating a response. Some of the use cases for prompts include entity or topic extraction, rephrasing, or dynamic content generation. The prompt can have up to 2000 characters, and it can be defined using text, Context, Content, and Environment variables.
- Preview Response: Check the preview of the OpenAI response for your prompt. When you click Preview Response, the Platform parses any variable used in the prompt and constructs OpenAI request using the Prompt and the Advanced Settings. If the response is not relevant, you can tweak the Prompt and the Advanced Settings to make the response better.
Change the model and tweak its settings.

Adjusting the settings allows you to fine-tune the model’s behavior to meet your needs. The default settings work fine for most cases. However, if required, you can tweak the settings and find the right balance for your use case.
- Model: The default model for which the settings are displayed. You can choose another supported mode if it’s configured. If you select a non-default model, it’s used for this node only. If you want to change the default model, you can select the model in the drop-down list and use the Mark Default option shown next to its name.
- Temperature: The setting controls the randomness of the model’s output. A higher temperature, like 0.8 or above, can result in unexpected, creative, and less relevant responses. On the other hand, a lower temperature, like 0.5 or below, makes the output more focused and relevant.
- Max Tokens: It indicates the total number of tokens used in the API call to the model. It affects the cost and the time taken to receive a response. A token can be as short as one character or as long as one word, depending on the text.
Select the maximum wait time to receive a response from the LLM.
- Timeout: Select the maximum wait time from the drop-down list. The timeout range can be any value between 10 to 60, the default being 10.

Instance Properties¶

On the Instance Properties tab, you can configure the instance-specific fields for this GenAI Prompt. These settings are applicable only for this instance and will not affect any other instances of this node.

Custom Tags¶

In this section, you can add Custom Meta Tags to the conversation flow to profile VA-user conversations and derive business-critical insights from usage and execution metrics. You can add tags for the following:

Message: Define custom tags to be added to the current message in the conversation.
User: Define custom tags to be added to the user’s profile information.
Session: Define custom tags to be added to the current conversation session.

For more information on custom tags, see Custom Meta Tags.

Connections Properties¶

Note

If the node is at the bottom in the sequence, then only the connection property is visible.

On the Connections tab, you can set the transition properties to determine the node in the dialog task to execute next. You can write conditional statements based on the values of any Entity or Context Objects in the dialog task, or you can use intents for transitions. See Adding IF-Else Conditions to Node Connections for a detailed setup guide.

Note

These conditions apply only for this instance and will not affect this node when used in any other dialog.

About Responses¶

All the responses collected are stored in context variables. For example, {{context.GenAIPrompt.NodeName.properties}}. You can define transitions using the context variables.

The responses are captured in a specific format, as shown below.

"context":{
"GenAIPrompt": {
    "NodeName": {
      "id": "cmpl-7UbzLTumD9ALpfa1mcpf15dK3RnWM",
      "object": "text_completion",
      "created": 1687530223,
      "model": "text-davinci-003",
      "choices": [
        {
          "text": "\n\nI'm sorry, I'm not able to provide that information. However, I would be happy to direct you to a website that may provide the information you are looking for.",
          "index": 0,
          "logprobs": null,
          "finish_reason": "stop"
        }
      ],
      "usage": {
        "prompt_tokens": 58,
        "completion_tokens": 37,
        "total_tokens": 95
      },
      "1687530221473": [
        {
          "nodeId": "NodeName",
          "startTime": "2023-06-23T14:23:42.904Z",
          "endTime": "2023-06-23T14:23:46.029Z"
        }
      ]
    }
  }
}