LLM Playground
LLM Playground
Starting from version 24.03, Backend.AI introduces the 'LLM Playground' feature, which enables users to conveniently compare and interact with different LLM models all in one location. This allows users to experience the services offered by Backend.AI as well as a variety of large language models (LLMs).
Selecting models
Each card's top left corner features an endpoint selector and a model selector. The endpoint selector enables the search and selection of endpoints established by Backend.AI services. The model selector allows for the choice of a model ID; If the model is unavailable, it defaults to 'Custom'.
Refer to the description below for the necessary inputs to configure custom modal settings:
baseURL: Base URL of the server where the model is located. Make sure to include the version information. For instance, when utilizing the OpenAI API, you should enter https://api.openai.com/v1.
Model ID: Unique identifier of the model. To specify the model you wish to use, you would provide its unique identifier. For instance, for GPT-4, you would input 'gpt-4o'.
Token (optional): An authentication key to access the model service. Tokens can be generated from various services, not just Backend.AI. The format and generation process may vary depending on the service. Always refer to the specific service's guide for details. For instance, when using the service generated by Backend.AI, please refer to the Generating Tokens section for instructions on how to generate tokens.
Add or remove chat cards
To add new chat cards, click the '+' icon located in the top right corner.
To remove a chat session, click on the 'x' icon located in the upper right corner of the chat card. Please be cautious as this will delete all entered content.
Delete chat history
Clicking the 'More' button will reveal the 'Delete Chat History' option. By selecting this, you will erase all chat history associated with the card, although the card session itself will remain active.
Synchronize input
The 'Sync Input' button located at the top right enables the synchronization of input across all chat cards. Enabling 'Sync input' means that pressing 'Enter' or clicking the 'Send' button on any card will submit the input from the card you are currently working on. This functionality is beneficial for comparing the outputs of various models using identical input data.