Skip to main content

Groq

Configuration​

When running Faheem Code, you'll need to set the following in the Faheem Code UI through the Settings under the LLM tab:

  • LLM Provider to Groq
  • LLM Model to the model you will be using. Visit here to see the list of models that Groq hosts. If the model is not in the list, enable Advanced options, and enter it in Custom Model (e.g. groq/<model-name> like groq/llama3-70b-8192).
  • API key to your Groq API key. To find or create your Groq API Key, see here.

Using Groq as an OpenAI-compatible endpoint​

The Groq endpoint for chat completion is mostly OpenAI-compatible. Therefore, you can access Groq models as you would access any OpenAI-compatible endpoint. In the Faheem Code UI through the Settings under the LLM tab:

  1. Enable Advanced options
  2. Set the following:
    • Custom Model to the prefix openai/ + the model you will be using (e.g. openai/llama3-70b-8192)
    • Base URL to https://api.groq.com/openai/v1
    • API Key to your Groq API key