LlamaIndex integration

Use Modellane models in LlamaIndex through the OpenAILike class, with chat mode and function calling switched on.

LlamaIndex reaches OpenAI-compatible APIs through its OpenAILike class. Set the base URL, the key and two flags, and use the model in queries and agents.

LlamaIndex reaches OpenAI-compatible APIs through its OpenAILike class, which calls the Chat Completions API (POST /v1/chat/completions) when it is set up as a chat model.

Install#

Shell

pip install llama-index-llms-openai-like

Create the model#

Python

import os
from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="lane-1",
    api_base="https://usemodellane.com/v1",
    api_key=os.environ["MODELLANE_API_KEY"],
    # Replace with the context length of the model from Models & Pricing.
    context_window=128000,
    is_chat_model=True,
    is_function_calling_model=True,
)

print(llm.complete("Explain recursion in one sentence."))
ParamValue
modellane-1, or any id from Models & pricing
api_basehttps://usemodellane.com/v1
context_windowThe context length of the model from the models table
is_chat_modelTrue. The default is False, which sends requests to the legacy completions endpoint that Modellane does not serve
is_function_calling_modelTrue for models that list Tool calls, so agents and structured output use tool calls

Use it everywhere#

Set the model as the default for indexes, query engines and agents:

Python

from llama_index.core import Settings

Settings.llm = llm

Verify#

Run the example above. A printed reply means the settings are right; an authentication error points at the key, and model_not_found at the model id.

Images#