Voice AI - Prompt Optimizer

Modified on: Tue, 28 Apr, 2026 at 11:44 AM

Prompt Optimizer helps you test, evaluate, and improve HighLevel Voice AI Agent prompts using real call scenarios before applying changes to your live agent. This article explains how Prompt Optimizer works, how to configure and run test calls, how to review results, and how to apply optimized prompt variations safely.

Note: This feature is currently in Labs and needs to be enabled by Agency for Sub-accounts.

TABLE OF CONTENTS

What is Prompt Optimizer in Voice AI?
Key Benefits of Prompt Optimizer in Voice AI
Good to Know Before Using Prompt Optimizer
How To Use Prompt Optimizer in Voice AI
Frequently Asked Questions
Related Articles

What is Prompt Optimizer in Voice AI?

Prompt Optimizer helps you test and improve HighLevel Voice AI Agents before you rely on them in live conversations. It generates realistic test scenarios, runs real voice calls, reviews the results, and suggests prompt improvements based on what failed. This gives you a faster, more structured way to validate your agent’s prompt, actions, and call flow without manually testing every possible caller interaction.

Prompt Optimizer is a test-and-tune engine for HighLevel Voice AI Agents. It uses the agent’s prompt, language, call direction, and configured actions to create test scenarios that reflect real caller behavior. After testing, Prompt Optimizer scores each call, provides transcripts and recordings, tracks tool usage, and can create optimized prompt variations for review.

Key Benefits of Prompt Optimizer in Voice AI

Automated scenario generation: Creates contextual test scenarios based on your agent’s prompt, language, direction, and configured actions.
Real voice call testing: Runs actual test calls against your Voice AI Agent so you can evaluate real conversation behavior.
Action validation: Tests whether configured actions such as knowledge base lookups, appointments, SMS, emails, and other tools are triggered correctly.
AI scoring and reasoning: Reviews each call against expected outcomes and explains why the result passed or failed.
Transcript and recording review: Lets you read the full call transcript and listen to the test call audio.
Prompt variation testing: Generates optimized prompt variations based on failed calls, then re-tests those variations.
Safer optimization: Tests a cloned version of the agent and does not update the live agent until you review the results and click Use Prompt.
Multi-language testing: Supports scenario generation, calls, and evaluations in the selected language.

Good to Know Before Using Prompt Optimizer

Prompt Optimizer is designed to make agent testing faster and more reliable, but test results should still be reviewed carefully. Understanding how calls, actions, limits, and evaluations work will help you interpret results and avoid unintended activity during testing.

Test calls are real voice calls, not text-only simulations.
Configured actions may fire during testing, including appointments, SMS, emails, knowledge base lookups, and CRM updates.
Each location receives 20 free testing minutes per day.
Individual test calls time out after 5 minutes.
Auto optimization runs up to 5 variations per session.
You can run up to 10 calls at a time.
AI evaluations and prompt behavior are not fully deterministic, so accuracy should be treated as directional.
Evaluations may be strict, so review the transcript and recording alongside the score.
Prompt Optimizer supports the current Voice AI Agent architecture.
The live agent is not modified unless you click Use Prompt.

How To Use Prompt Optimizer in Voice AI

A proper setup helps Prompt Optimizer test the right scenarios and produce useful prompt recommendations. Prompt Optimizer follows a walkthrough-style flow: configure the test plan, run real calls, review the results, and then improvise prompt variations when improvements are needed.

Navigate to AI Agents > Voice AI > Agent List.
Open the Voice AI Agent you want to test.
Click Prompt Optimizer from the agent builder: Prompt Optimizer opens from the Voice AI Agent builder and uses the selected agent’s existing prompt, language, direction, and configured actions to prepare the testing workflow. The live agent is cloned for testing, so the production agent is not changed during the test or optimization process.
Click on Generate Scenarios and review the automatically generated scenarios: Prompt Optimizer can automatically create contextual test scenarios based on the underlying agent configuration. These scenarios help test how the agent responds to realistic caller requests and whether it completes the expected behavior.

Generated scenarios may include:
- Caller persona or caller context
- Caller input, such as the question or request the caller makes
- Expected agent behavior
- Expected tool or action usage
- Scenario priority, such as Critical, High, Medium, or Low
Select the scenarios you want to run: Choose the scenarios that match the workflows you want to validate. For example, you may want to test whether the agent can answer from the knowledge base, verify a caller, book an appointment, send an SMS, send an email, or handle a specific edge case.
Edit or create custom scenarios when needed: Custom scenarios are useful when you need to test a specific workflow, edge case, pronunciation concern, location-based question, required response, or action sequence. Clear expected behaviors help Prompt Optimizer evaluate whether the agent followed the intended call flow.

To create or edit a scenario, configure details such as:
- Scenario name
- Description
- Caller input
- Expected behaviors
- Priority
Example expected behaviors may include verifying the caller’s information, refusing an unsupported request, booking an appointment, sending an SMS, sending an email, or answering from the knowledge base.
Reuse scenarios from a previous run if applicable: Reusing past scenarios helps you retest the same workflows without rebuilding the test plan. This is useful when you make prompt changes and want to compare whether the same set of scenarios performs better after optimization.
Select the language for the test: Prompt Optimizer supports multi-language testing. The selected language is used for scenario generation, calls, and evaluations so you can validate how the agent performs in the language your callers use.
Choose the number of calls per scenario: Select how many calls to run for each scenario. Running multiple calls per scenario can help test consistency because AI behavior may vary slightly between calls. Prompt Optimizer supports 1-10 calls per scenario.
Review usage and the daily testing minutes: Before starting the run, review the usage dashboard to confirm the daily free testing minutes used and remaining. Each location receives 20 free testing minutes per day, and additional usage may apply after the daily free minutes are used.
Review the test contact and pre-run checklist: Prompt Optimizer uses a dedicated test contact for calls. Review the test contact details, such as name, phone, and email, along with any calendar, action, or usage warnings before starting the test.

Because test calls are real calls, configured actions can fire during testing. Actions may include appointment booking, SMS, email, knowledge base retrieval, and CRM updates. Use a test contact and safe test setup whenever possible. For appointment workflows, consider using a test calendar to avoid unintended bookings.
Click Run Calls: Prompt Optimizer starts real voice calls based on the selected scenarios and call count. These are not text-only simulations. The agent answers, speaks with the tester, and executes configured actions end to end.
Monitor call progress: Calls can move through statuses such as Waiting, In Progress, Evaluating, Completed, or Failed. You can leave the screen and return later while calls continue processing in the background.
Review the pass/fail summary: After calls complete, Prompt Optimizer shows how many calls passed, how many failed, and the overall accuracy percentage across the tested scenarios. Use the accuracy score as a directional signal, not the only measure of success.

During and after testing, Prompt Optimizer displays:
- Call status and progress
- Number of scenarios and calls tested
- Passed and failed outcomes
- Overall accuracy percentage
- AI evaluation reasoning
- Tool invocations and tool results
- Full transcript with speaker labels and timestamps
- Call recording playback
Review individual call transcripts and recordings: Open individual call results to read the transcript and listen to the recording. Transcripts include speaker labels and timestamps so you can follow what the tester said, how the agent responded, and where the outcome succeeded or failed.
Review AI scoring and reasoning: Prompt Optimizer evaluates each call against the expected outcomes configured in the scenario. The AI reasoning explains why an outcome passed or failed, making it easier to identify whether the agent missed a step, misunderstood the caller, skipped a required action, or responded with incomplete information.

Evaluations may be strict. A failed result does not always mean the agent completely failed the conversation, but it does identify an area worth reviewing.
Review tool invocations and action results: Tool invocation tracking shows which configured actions were triggered during the test call. This helps confirm whether the agent is using the correct tools at the correct point in the conversation, especially when testing workflows that depend on appointments, messaging, knowledge base answers, or CRM actions.

Prompt Optimizer can help test workflows involving:
- Knowledge base retrieval
- Appointment booking
- Sending SMS messages
- Sending emails
- Caller verification
- CRM-related updates or actions
Review failed calls before changing the prompt: Failed calls should be reviewed before deciding whether the prompt needs to change. A failed result can indicate that the agent missed an expected behavior, completed steps in the wrong order, used an action incorrectly, or did not satisfy the scenario requirements.

When reviewing a failed call, check:
- The AI evaluation reason
- The transcript
- The call recording
- Any tool invocation details
- Whether the expected behavior was written clearly enough
- Whether the agent’s prompt needs more specific instructions
Click Improvise to create one optimized prompt variation: The Improvise option analyzes failed calls, identifies likely root causes, creates a revised prompt variation, and tests that variation. This helps you compare performance before applying anything to the live agent.
Use Auto when you want Prompt Optimizer to test multiple variations: Auto optimization runs up to five prompt variations automatically. It can stop when the target is reached, the maximum number of variations is met, there is no improvement, a timeout occurs, or a billing limit is reached.
Review variation results and the prompt diff: Each variation includes performance details so you can understand what changed and whether the revision improved results.

Each variation may include:
- Accuracy score
- Test results
- Failed and passed calls
- AI reasoning
- Prompt diff
- Variation history
- Best Variation label when applicable
- Use Prompt option when you are ready to apply the selected prompt
The prompt diff helps you review what was removed or added before deciding whether to use the optimized prompt.
Click Use Prompt only when you are ready to update the agent: Prompt Optimizer does not update your production prompt automatically. The live agent remains unchanged until you review the variation and click Use Prompt. After you confirm, the selected optimized prompt is applied to the underlying agent.

Frequently Asked Questions

Q: Does Prompt Optimizer change my live Voice AI Agent automatically?
No. Prompt Optimizer tests a cloned version of the agent. Your live agent is not updated unless you review a variation and click Use Prompt.

Q: What is the difference between Prompt Optimizer and Prompt Evaluator?
Prompt Evaluator reviews the prompt and provides suggestions. Prompt Optimizer runs real test calls, evaluates outcomes, identifies failed behaviors, and generates tested prompt variations.

Q: Are Prompt Optimizer calls real calls?
Yes. Prompt Optimizer runs real voice calls, which allows it to test conversation behavior and configured actions end to end.

Q: Can test calls trigger real actions?
Yes. Actions such as appointment booking, SMS, email, knowledge base retrieval, and CRM updates may run during testing. Use a test contact and safe test setup when possible.

Q: Why did a call fail even though the conversation seemed acceptable?
Evaluations compare the call against the expected behaviors defined in the scenario. A call may fail if it missed a required step, used the wrong order, skipped an action, or did not match the expected outcome closely enough.

Q: Should I rely only on the accuracy score?
No. Use the score as a directional signal, then review transcripts, recordings, tool invocations, and AI reasoning before deciding whether to update the prompt.

Q: How many Auto variations can Prompt Optimizer run?
Auto optimization can run up to 5 variations per session.

Q: Can I test in multiple languages?
Yes. Prompt Optimizer supports scenario generation, calls, and evaluations in the selected language.

Q: Can I reuse scenarios from previous test runs?
Yes. Prompt Optimizer allows you to load past scenarios so you can retest common workflows without rebuilding the test plan.

Q: What happens when I click Use Prompt?
The selected optimized prompt is applied to the underlying agent after you confirm the update.