Skip to main content

Critic

The critic is an additional evaluation pass that reviews the agent's work and predicts how likely the task is to succeed. In Faheem Code, critic results can appear in the conversation timeline as a success-likelihood score with detected issue labels.

Use the critic when you want extra feedback for an Faheem Code agent conversation, or when you want the agent to automatically refine its work after a low critic score.

Prerequisites​

Before enabling the critic:

  1. Configure your active LLM in Settings > LLM.
  2. Prefer the Faheem Code LLM provider when you want the default hosted critic path.
  3. Start a new conversation after saving critic settings. Existing conversations keep the settings they were created with.

Enable the critic​

  1. Open Settings > Verification.
  2. Toggle on Enable Critic.
  3. Configure the Critic API Key field:
    • If Faheem Code is selected as your active LLM provider, leave this field empty. The critic reuses the active provider's Faheem Code Provider LLM Key.
    • If you are not using the Faheem Code LLM provider, paste an Faheem Code Provider LLM Key into Critic API Key, or provide the API key required by your custom critic service.
  4. Save the settings.
  5. Start a new conversation.

Screenshot: Faheem Code Verification settings with Enable Critic, iterative refinement, critic threshold, and Critic API Key guidance.

The Critic API Key and the Faheem Code Provider LLM Key are the same credential when you use the default Faheem Code-hosted critic service. You can find that key in the API Keys tab of Faheem Code Cloud.

The default hosted critic is free today; the key authenticates access to the service.

Enable iterative refinement​

Iterative refinement lets the critic send the agent back to improve its work when the predicted success score is too low.

  1. Open Settings > Verification.
  2. Toggle on Enable Critic.
  3. Toggle on Enable Iterative Refinement.
  4. Optionally switch the settings detail view to Advanced or All.
  5. Adjust:
    • Critic Threshold - the success score required to stop refining. The default is 0.6.
    • Max Refinement Iterations - the maximum number of retry attempts. The default is 3.
  6. Save the settings and start a new conversation.

When refinement is enabled, Faheem Code will let the conversation continue after a low critic score until the score passes the threshold or the maximum iteration count is reached.

View critic results​

When the critic runs, Faheem Code shows the result below the agent message or finish action that was evaluated. The compact view shows the predicted success likelihood score. You can expand the result to inspect detected issue categories and probabilities, such as incomplete changes, missing validation, infrastructure issues, or likely user follow-up patterns.

Screenshot: Faheem Code conversation showing a critic success likelihood score below an agent message.

Troubleshooting​

Critic results do not appear​

  • Confirm Enable Critic is on in Settings > Verification.
  • Start a new conversation after saving the setting.
  • Use the Faheem Code agent path. Third-party ACP agents may not expose critic results.
  • Wait until the agent sends a message or finishes a task. With the default finish_and_message mode, the critic does not run after every tool call.

Authentication errors​

If the critic request fails with an API key or authentication error:

  • If Faheem Code is the active LLM provider, leave Critic API Key empty and confirm the active LLM profile has a saved Faheem Code Provider LLM Key.
  • If another LLM provider is active, enter an Faheem Code Provider LLM Key in Critic API Key.

Conversations become slow​

  • Keep Critic Mode set to finish_and_message unless you need per-action feedback.
  • Disable Enable Iterative Refinement if you only want passive critic scores.
  • Lower Max Refinement Iterations if repeated refinement loops are too costly.