Instant Answers Effectiveness by Portal

Report on the overall success of Instant Answers in each portal. The report uses a Portal parameter set and can be filtered by session type (for example, agent or customer). There are daily, weekly and monthly versions of the report which break down the data by portal and interval.

The report may be found under the following path: Reports > AI Reports > {interval}.  The report is based on the Instant Answers Performance model and includes the following columns: 

Portal

The portal in which a search was performed.

Searches

The number of searches during the interval for which an attempt was made to return an Instant Answer.

% Failed

Of the searches performed during the interval, the percentage that failed to return an Instant Answer. This may be because no suitable content could be found, or because an Instant Answer was produced that did not meet the confidence threshold.

% Successful

Of the searches performed during the interval, the percentage where an Instant Answer was successfully returned.

% Success Extracted

Of the searches performed during the interval, the percentage where an Extractive Instant Answer was successfully returned. Extractive answers consist of existing content that has been selected by the AI due to its semantic relevance to the search phrase.

% Success Generated

Of the searches performed during the interval, the percentage where a Generative Instant Answer was successfully returned. A generative answer is a new response which the AI has created by adapting existing sources.

% Positive

Of the times that a user gave feedback to an Instant Answer during the interval, the percentage where they gave positive feedback.

% Negative

Of the times that a user gave feedback to an Instant Answer during the interval, the percentage where they gave negative feedback.

% Rated

Of the times that an Instant Answer was successfully returned during the interval, the proportion of times that a user gave feedback to the Instant Answer. 

% Used

Of the times that an Instant Answer was successfully returned during the interval, the percentage where the user clicked the Add to Reply button. 

Only applicable for agents using the Advisor Desktop – the Add to Reply button adds the Instant Answer content to the reply that the agent is working on. For any other types of user, this will be 0%.

% KB Article Srcs

Of the times that any content item was used as a source for a successful Instant Answer during the interval, the percentage where a Knowledge article was used. This only considers the content of the article itself as any PDF attachments are counted separately.

% KB PDF Srcs

Of the times that any content item was used as a source for a successful Instant Answer during the interval, the percentage where a PDF document was used. This considers PDFs that are attached to Knowledge articles.