Skip to main content

LLMs

info

Conceptual Guide

Large Language Models (LLMs) are a core component of LangChain. LangChain does not serve it's own LLMs, but rather provides a standard interface for interacting with many different LLMs.

type LLM interface {
    Call(ctx context.Context, prompt string, options ...CallOption) (string, error)
    Generate(ctx context.Context, prompts []string, options ...CallOption) ([]*Generation, error)
}

As you can see LLMs expose two main methods. The Call metod takes a prompt and returns a completion and an evetuall error. With the generate method you can take multiple prompts and get a generation for each of the prompts.

// Generation is a single generation from a langchaingo LLM.
type Generation struct {
    // Text is the generated text.
    Text string `json:"text"`
    // Message stores the potentially generated message.
    Message *schema.AIChatMessage `json:"message"`
    // GenerationInfo is the generation info. This can contain vendor-specific information.
    GenerationInfo map[string]any `json:"generation_info"`
}

Beacuse all LLMs also implement the LanguageModel interface, you can also use the GeneratePrompt and GetNumTokens functions on all of the LLMs.

// LanguageModel is the interface all language models must implement.
type LanguageModel interface {
    // Take in a list of prompt values and return an LLMResult.
    GeneratePrompt(ctx context.Context, prompts []schema.PromptValue, options ...CallOption) (LLMResult, error)
    // Get the number of tokens present in the text.
    GetNumTokens(text string) int
}