StreamedResponse Object Members

The StreamedResponse object in SuiteScript 2.1 provides essential members for managing responses from a Large Language Model (LLM). This object allows developers to access various attributes such as chat history and citations, enabling dynamic interactions with AI-generated content.

What Are the Members of the StreamedResponse Object?

The StreamedResponse object includes several properties, each serving a unique purpose:

Understanding Each Member

• chatHistory: This property provides insights into the ongoing conversation. However, it's important to note that its value isn't accessible until the entire response from the LLM has been generated.
• citations and documents: Both properties are useful for tracking the sources of information that were leveraged to produce the output, enhancing transparency and credibility.
• model: Knowing which model generated the response can help evaluate the response's context and accuracy.
• text: This is the output content provided by the LLM, which developers will typically utilize within their applications.

Handling Errors in StreamedResponse

A common error associated with the StreamedResponse is READ_ONLY, which occurs if there attempt to set any of its properties.

Practical Example

The following code snippet demonstrates how to use the StreamedResponse object in a SuiteScript 2.1 server script:

1const response = llm.generateTextStreamed({
2    prompt: 'Hello World'
3});
4 
5var iter = response.iterator();
6iters.each(function(token) {
7    log.debug('token.value: ' + token.value);
8    log.debug('response.text: ' + response.text);
9    return true;
10});
11 
12const responseChatHistory = response.chatHistory;

This example shows how to generate a streamed response and iterate through tokens while logging valuable information, including the final text produced by the LLM.

Key Considerations

When working with the StreamedResponse object:

• Ensure that you are familiar with the overall structure and capabilities of the llm module in SuiteScript 2.1.
• Understand the timing of when properties like chatHistory and text become available.

Source: This article is based on Oracle's official NetSuite documentation.

Key Takeaways

• The StreamedResponse object comprises several valuable members for handling AI outputs.
• Understanding the individual properties aids in effective LLM response management.
• Careful handling of error conditions can help maintain optimal performance while interacting with the LLM.