Instant Answers Effectiveness by Search Phrase

Report on the search phrases that are used, and the effectiveness of Instant Answers at generating results for each search phrase. The report uses a Portal parameter set and can be filtered by session type (for example, agent or customer). There are daily, weekly and monthly versions of the report which break down the data by search phrase.

The report may be found under the following path: Reports > AI Reports > {interval}. The report is based on the Instant Answers Performance model and includes the following columns: 

Search Phrase

The search phrase that was used.

Searches

The number of searches during the interval for which an attempt was made to return an Instant Answer.

Failed

The number of times during the interval that an attempt to return an Instant Answer failed. This may be because no suitable content could be found, or because an Instant Answer was produced that did not meet the confidence threshold.

Success

The number of times during the interval that an Instant Answer was successfully returned.

Avg Score

The average Answer Score for each search for an Instant Answer during the interval.

Success Extracted

The number of times that an article was used as a source for a successful, extractive Instant Answer during the interval. Extractive answers consist of existing content that has been selected by the AI due to its semantic relevance to the search phrase.

Success Generated

The number of times that an article was used as a source for a successful, generative Instant Answer during the interval. A generative answer is a new response which the AI has created by adapting existing sources.

% Positive

Of the times that a user gave feedback to an Instant Answer during the interval, the percentage where they gave positive feedback.

% Negative

Of the times that a user gave feedback to an Instant Answer during the interval, the percentage where they gave negative feedback.

% Rated

Of the times that an Instant Answer was successfully returned during the interval, the proportion of times that a user gave feedback to the Instant Answer. 

% Used

Of the times that an Instant Answer was successfully returned during the interval, the percentage where the user clicked the Add to Reply button. 

This is only applicable for agents using the Advisor Desktop. the Add to Reply button adds the Instant Answer content to the reply that the agent is working on. For any other types of user, this will be 0%.