langchain_core.language_models.llms.LLM

Note

LLM 实现了标准的 Runnable Interface。🏃

Runnable Interface 具有在 runnables 上可用的其他方法，例如 with_types、 with_retry、 assign、 bind、 get_graph 等。

class langchain_core.language_models.llms.LLM[source]

Bases: BaseLLM

Simple interface for implementing a custom LLM.

You should subclass this class and implement the following

  • _call method: Run the LLM on the given prompt and input (used by invoke).

  • _identifying_params property: Return a dictionary of the identifying parameters

    This is critical for caching and tracing purposes. Identifying parameters is a dict that identifies the LLM. It should mostly include a model_name.

Optional: Override the following methods to provide more optimizations

  • _acall: Provide a native async version of the _call method.

    If not provided, will delegate to the synchronous version using run_in_executor. (Used by ainvoke).

  • _stream: Stream the LLM on the given prompt and input.

    stream will use _stream if provided, otherwise it use _call and output will arrive in one chunk.

  • _astream: Override to provide a native async version of the _stream method.

    astream will use _astream if provided, otherwise it will implement a fallback behavior that will use _stream if _stream is implemented, and use _acall if _stream is not implemented.

Please see the following guide for more information on how to implement a custom LLM

https://python.langchain.ac.cn/v0.2/docs/how_to/custom_llm/

param cache: Union[BaseCache, bool, None] = None

Whether to cache the response.

  • If true, will use the global cache.

  • If false, will not use a cache

  • If None, will use the global cache if it’s set, otherwise no cache.

  • If instance of BaseCache, will use the provided cache.

Caching is not currently supported for streaming methods of models.

param callback_manager: Optional[BaseCallbackManager] = None

[DEPRECATED]

param callbacks: Callbacks = None

Callbacks to add to the run trace.

param custom_get_token_ids: Optional[Callable[[str], List[int]]] = None

Optional encoder to use for counting tokens.

param metadata: Optional[Dict[/[/span>str, Any]] = None

Metadata to add to the run trace.

param tags: Optional[List[str]] = None

Tags to add to the run trace.

param verbose: bool [Optional]

Whether to print out response text.

__call__(prompt: str, stop: Optional[List[str]] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → str

Deprecated since version langchain-core==0.1.7: Use invoke instead.

Check Cache and run the LLM on the given prompt and input.

Parameters

  • prompt (str) – The prompt to generate from.

  • stop (Optional[List[str]]) – Stop words to use when generating. Model output is cut off at the first occurrence of any of these substrings.

  • callbacks (Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]) – Callbacks to pass through. Used for executing additional functionality, such as logging or streaming, throughout generation.

  • tags (Optional[List[str]]) – List of tags to associate with the prompt.

  • metadata (Optional[Dict[str, Any]]) – Metadata to associate with the prompt.

  • **kwargs (Any) – Arbitrary additional keyword arguments. These are usually passed to the model provider API call.

Returns

The generated text.

Raises

ValueError – If the prompt is not a string.

Return type

str

async abatch(inputs: List[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Any) → List[str]

Default implementation runs ainvoke in parallel using asyncio.gather.

The default implementation of batch works well for IO bound runnables.

Subclasses should override this method if they can batch more efficiently; e.g., if the underlying Runnable uses an API which supports a batch mode.

Parameters

  • inputs (List[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]]) – A list of inputs to the Runnable.

  • config (Optional[Union[RunnableConfig, List[RunnableConfig]]]) – A config to use when invoking the Runnable. The config supports standard keys like ‘tags’, ‘metadata’ for tracing purposes, ‘max_concurrency’ for controlling how much work to do in parallel, and other keys. Please refer to the RunnableConfig for more details. Defaults to None.

  • return_exceptions (bool) – Whether to return exceptions instead of raising them. Defaults to False.

  • kwargs (Any) – Additional keyword arguments to pass to the Runnable.

Returns

A list of outputs from the Runnable.

Return type

List[str]

async abatch_as_completed(inputs: Sequence[Input], config: Optional[Union[RunnableConfig, Sequence[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Optional[Any]) → AsyncIterator[Tuple[int, Union[Output, Exception]]]

Run ainvoke in parallel on a list of inputs, yielding results as they complete.

Parameters

  • inputs (Sequence[Input]) – A list of inputs to the Runnable.

  • config (Optional[Union[RunnableConfig, Sequence[RunnableConfig]]]) – A config to use when invoking the Runnable. The config supports standard keys like ‘tags’, ‘metadata’ for tracing purposes, ‘max_concurrency’ for controlling how much work to do in parallel, and other keys. Please refer to the RunnableConfig for more details. Defaults to None. Defaults to None.

  • return_exceptions (bool) – Whether to return exceptions instead of raising them. Defaults to False.

  • kwargs (Optional[Any]) – Additional keyword arguments to pass to the Runnable.

Yields

A tuple of the index of the input and the output from the Runnable.

Return type

AsyncIterator[Tuple[int, Union[Output, Exception]]]

async agenerate(prompts: List[str], stop: Optional[List[str]] = None, callbacks: Union[List[BaseCallbackHandler], BaseCallbackManager, None, List[Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]]] = None, *, tags: Optional[Union[List[str], List[List[str]]]] = None, metadata: Optional[Union[Dict[str, Any]], List[Dict[str, Any]]]] = None, run_name: Optional[Union[str, List[str]]] = None, run_id: Optional[Union[UUID, List[Optional[UUID]]]] = None, **kwargs: Any) → LLMResult

Asynchronously pass a sequence of prompts to a model and return generations.

This method should make use of batched calls for models that expose a batched API.

Use this method when you want to

  1. take advantage of batched calls,

  2. need more output from the model than just the top generated value,

  3. are building chains that are agnostic to the underlying language model

    type (e.g., pure text completion models vs chat models).

Parameters

  • prompts (List[str]) – List of string prompts.

  • stop (Optional[List[str]]) – Stop words to use when generating. Model output is cut off at the first occurrence of any of these substrings.

  • callbacks (Union[List[BaseCallbackHandler], BaseCallbackManager, None, List[Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]]]) – Callbacks to pass through. Used for executing additional functionality, such as logging or streaming, throughout generation.

  • tags (可选[Union[List[str], List[List[str]]]]) – 与每个提示关联的标签列表。如果提供，则列表的长度必须与提示列表的长度匹配。

  • metadata (可选[Union[Dict[str, Any], List[Dict[str, Any]]]]) – 与每个提示关联的元数据字典列表。如果提供，则列表的长度必须与提示列表的长度匹配。

  • run_name (可选[Union[str, List[str]]]) – 与每个提示关联的运行名称列表。如果提供，则列表的长度必须与提示列表的长度匹配。

  • run_id (可选[Union[UUID, List[Optional[UUID]]]]) – 与每个提示关联的运行 ID 列表。如果提供，则列表的长度必须与提示列表的长度匹配。

  • **kwargs (Any) – Arbitrary additional keyword arguments. These are usually passed to the model provider API call.

Returns

一个 LLMResult，其中包含每个输入的候选 Generations 列表

prompt 和其他模型提供商特定的输出。

Return type

LLMResult

async agenerate_prompt(prompts: List[PromptValue], stop: Optional[List[str]] = None, callbacks: Union[List[BaseCallbackHandler], BaseCallbackManager, None, List[Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]]] = None, **kwargs: Any) → LLMResult

异步传递一系列提示并返回模型生成结果。

This method should make use of batched calls for models that expose a batched API.

Use this method when you want to

  1. take advantage of batched calls,

  2. need more output from the model than just the top generated value,

  3. are building chains that are agnostic to the underlying language model

    type (e.g., pure text completion models vs chat models).

Parameters

  • prompts (List[PromptValue]) – PromptValue 列表。PromptValue 是一个可以转换为匹配任何语言模型格式的对象（纯文本生成模型的字符串和聊天模型的基础消息）。

  • stop (Optional[List[str]]) – Stop words to use when generating. Model output is cut off at the first occurrence of any of these substrings.

  • callbacks (Union[List[BaseCallbackHandler], BaseCallbackManager, None, List[Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]]]) – Callbacks to pass through. Used for executing additional functionality, such as logging or streaming, throughout generation.

  • **kwargs (Any) – Arbitrary additional keyword arguments. These are usually passed to the model provider API call.

Returns

一个 LLMResult，其中包含每个输入的候选 Generations 列表

prompt 和其他模型提供商特定的输出。

Return type

LLMResult

async ainvoke(input: Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]], config: Optional[RunnableConfig] = None, *, stop: Optional[List[str]] = None, **kwargs: Any) → str

ainvoke 的默认实现，从线程调用 invoke。

即使 Runnable 没有实现 invoke 的原生异步版本，默认实现也允许使用异步代码。

如果子类可以异步运行，则应覆盖此方法。

Parameters

  • input (Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]) –

  • config (Optional[RunnableConfig]) –

  • stop (Optional[List[str]]) –

  • kwargs (Any) –

Return type

str

async apredict(text: str, *, stop: Optional[Sequence[str]] = None, **kwargs: Any) → str

Deprecated since version langchain-core==0.1.7: 使用 ainvoke 代替。

Parameters

  • text (str) –

  • stop (Optional[Sequence[str]]) –

  • kwargs (Any) –

Return type

str

async apredict_messages(messages: List[BaseMessage], *, stop: Optional[Sequence[str]] = None, **kwargs: Any) → BaseMessage

Deprecated since version langchain-core==0.1.7: 使用 ainvoke 代替。

Parameters

  • messages (List[BaseMessage]) –

  • stop (Optional[Sequence[str]]) –

  • kwargs (Any) –

Return type

BaseMessage

as_tool(args_schema: Optional[Type[BaseModel]] = None, *, name: Optional[str] = None, description: Optional[str] = None, arg_types: Optional[Dict[str, Type]] = None) → BaseTool

Beta

此 API 处于 Beta 阶段，未来可能会发生变化。

从 Runnable 创建一个 BaseTool。

as_tool 将从 Runnable 实例化一个具有名称、描述和 args_schema 的 BaseTool。如果可能，模式会从 runnable.get_input_schema 推断。或者（例如，如果 Runnable 接受字典作为输入，但未键入特定的字典键），可以使用 args_schema 直接指定模式。您也可以传递 arg_types 来仅指定必需的参数及其类型。

Parameters

  • args_schema (Optional[Type[BaseModel]]) – 工具的模式。默认为 None。

  • name (Optional[str]) – 工具的名称。默认为 None。

  • description (Optional[str]) – 工具的描述。默认为 None。

  • arg_types (Optional[Dict[str, Type]]) – 参数名称到类型的字典。默认为 None。

Returns

一个 BaseTool 实例。

Return type

BaseTool

Typed dict input

from typing import List
from typing_extensions import TypedDict
from langchain_core.runnables import RunnableLambda

class Args(TypedDict):
    a: int
    b: List[int]

def f(x: Args) -> str:
    return str(x["a"] * max(x["b"]))

runnable = RunnableLambda(f)
as_tool = runnable.as_tool()
as_tool.invoke({"a": 3, "b": [1, 2]})

dict input，通过 args_schema 指定模式

from typing import Any, Dict, List
from langchain_core.pydantic_v1 import BaseModel, Field
from langchain_core.runnables import RunnableLambda

def f(x: Dict[str, Any]) -> str:
    return str(x["a"] * max(x["b"]))

class FSchema(BaseModel):
    """Apply a function to an integer and list of integers."""

    a: int = Field(..., description="Integer")
    b: List[int] = Field(..., description="List of ints")

runnable = RunnableLambda(f)
as_tool = runnable.as_tool(FSchema)
as_tool.invoke({"a": 3, "b": [1, 2]})

dict input，通过 arg_types 指定模式

from typing import Any, Dict, List
from langchain_core.runnables import RunnableLambda

def f(x: Dict[str, Any]) -> str:
    return str(x["a"] * max(x["b"]))

runnable = RunnableLambda(f)
as_tool = runnable.as_tool(arg_types={"a": int, "b": List[int]})
as_tool.invoke({"a": 3, "b": [1, 2]})

String input

from langchain_core.runnables import RunnableLambda

def f(x: str) -> str:
    return x + "a"

def g(x: str) -> str:
    return x + "z"

runnable = RunnableLambda(f) | g
as_tool = runnable.as_tool()
as_tool.invoke("b")

0.2.14 版本新增功能。

async astream(input: Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]], config: Optional[RunnableConfig] = None, *, stop: Optional[List[str]] = None, **kwargs: Any) → AsyncIterator[str]

astream 的默认实现，它调用 ainvoke。如果子类支持流式输出，则应覆盖此方法。

Parameters

  • input (Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]) – Runnable 的输入。

  • config (Optional[RunnableConfig]) – 用于 Runnable 的配置。默认为 None。

  • kwargs (Any) – Additional keyword arguments to pass to the Runnable.

  • stop (Optional[List[str]]) –

Yields

Runnable 的输出。

Return type

AsyncIterator[str]

astream_events(input: Any, config: Optional[RunnableConfig] = None, *, version: Literal['v1', 'v2'], include_names: Optional[Sequence[str]] = None, include_types: Optional[Sequence[str]] = None, include_tags: Optional[Sequence[str]] = None, exclude_names: Optional[Sequence[str]] = None, exclude_types: Optional[Sequence[str]] = None, exclude_tags: Optional[Sequence[str]] = None, **kwargs: Any) → AsyncIterator[Union[StandardStreamEvent, CustomStreamEvent]]

Beta

此 API 处于 Beta 阶段，未来可能会发生变化。

生成事件流。

用于创建 StreamEvents 的迭代器，该迭代器提供关于 Runnable 进度的实时信息，包括来自中间结果的 StreamEvents。

StreamEvent 是一个具有以下模式的字典

  • event: str - 事件名称的格式为：on_[runnable_type]_(start|stream|end)。

    format: on_[runnable_type]_(start|stream|end).

  • name: str - 生成事件的 Runnable 的名称。

  • run_id: str - 与发出事件的 Runnable 的给定执行关联的随机生成的 ID。作为父 Runnable 执行一部分调用的子 Runnable 将被分配其自己唯一的 ID。

    the Runnable that emitted the event. A child Runnable that gets invoked as part of the execution of a parent Runnable is assigned its own unique ID.

  • parent_ids: List[str] - 生成事件的父 runnable 的 ID 列表。根 Runnable 将有一个空列表。父 ID 的顺序是从根到直接父级。仅适用于 API 的 v2 版本。API 的 v1 版本将返回一个空列表。

    generated the event. The root Runnable will have an empty list. The order of the parent IDs is from the root to the immediate parent. Only available for v2 version of the API. The v1 version of the API will return an empty list.

  • tags: Optional[List[str]] - 生成事件的 Runnable 的标签。

    the event.

  • metadata: Optional[Dict[str, Any]] - 生成事件的 Runnable 的元数据。

    that generated the event.

  • data: Dict[str, Any]

下面是一个表格，说明了各种链可能发出的一些事件。为了简洁起见，元数据字段已从表格中省略。链定义已包含在表格之后。

注意 此参考表适用于 V2 版本的模式。

event

name

chunk

input

output

on_chat_model_start

[model name]

{“messages”: [[SystemMessage, HumanMessage]]}

on_chat_model_stream

[model name]

AIMessageChunk(content=”hello”)

on_chat_model_end

[model name]

{“messages”: [[SystemMessage, HumanMessage]]}

AIMessageChunk(content=”hello world”)

on_llm_start

[model name]

{‘input’: ‘hello’}

on_llm_stream

[model name]

‘Hello’

on_llm_end

[model name]

‘Hello human!’

on_chain_start

format_docs

on_chain_stream

format_docs

“hello world!, goodbye world!”

on_chain_end

format_docs

[Document(…)]

“hello world!, goodbye world!”

on_tool_start

some_tool

{“x”: 1, “y”: “2”}

on_tool_end

some_tool

{“x”: 1, “y”: “2”}

on_retriever_start

[retriever name]

{“query”: “hello”}

on_retriever_end

[retriever name]

{“query”: “hello”}

[Document(…), ..]

on_prompt_start

[template_name]

{“question”: “hello”}

on_prompt_end

[template_name]

{“question”: “hello”}

ChatPromptValue(messages: [SystemMessage, …])

除了标准事件之外，用户还可以分派自定义事件（请参阅下面的示例）。

自定义事件将仅在 API 的 v2 版本中显示！

自定义事件具有以下格式

属性

类型

描述

name

str

事件的用户定义名称。

data

Any

与事件关联的数据。这可以是任何内容，但我们建议使其 JSON 可序列化。

以下是与上面显示的标准事件关联的声明

format_docs:

def format_docs(docs: List[Document]) -> str:
    '''Format the docs.'''
    return ", ".join([doc.page_content for doc in docs])

format_docs = RunnableLambda(format_docs)

some_tool:

@tool
def some_tool(x: int, y: str) -> dict:
    '''Some_tool.'''
    return {"x": x, "y": y}

prompt:

template = ChatPromptTemplate.from_messages(
    [("system", "You are Cat Agent 007"), ("human", "{question}")]
).with_config({"run_name": "my_template", "tags": ["my_template"]})

示例

from langchain_core.runnables import RunnableLambda

async def reverse(s: str) -> str:
    return s[::-1]

chain = RunnableLambda(func=reverse)

events = [
    event async for event in chain.astream_events("hello", version="v2")
]

# will produce the following events (run_id, and parent_ids
# has been omitted for brevity):
[
    {
        "data": {"input": "hello"},
        "event": "on_chain_start",
        "metadata": {},
        "name": "reverse",
        "tags": [],
    },
    {
        "data": {"chunk": "olleh"},
        "event": "on_chain_stream",
        "metadata": {},
        "name": "reverse",
        "tags": [],
    },
    {
        "data": {"output": "olleh"},
        "event": "on_chain_end",
        "metadata": {},
        "name": "reverse",
        "tags": [],
    },
]

示例：分派自定义事件

from langchain_core.callbacks.manager import (
    adispatch_custom_event,
)
from langchain_core.runnables import RunnableLambda, RunnableConfig
import asyncio


async def slow_thing(some_input: str, config: RunnableConfig) -> str:
    """Do something that takes a long time."""
    await asyncio.sleep(1) # Placeholder for some slow operation
    await adispatch_custom_event(
        "progress_event",
        {"message": "Finished step 1 of 3"},
        config=config # Must be included for python < 3.10
    )
    await asyncio.sleep(1) # Placeholder for some slow operation
    await adispatch_custom_event(
        "progress_event",
        {"message": "Finished step 2 of 3"},
        config=config # Must be included for python < 3.10
    )
    await asyncio.sleep(1) # Placeholder for some slow operation
    return "Done"

slow_thing = RunnableLambda(slow_thing)

async for event in slow_thing.astream_events("some_input", version="v2"):
    print(event)
Parameters

  • input (Any) – Runnable 的输入。

  • config (Optional[RunnableConfig]) – 用于 Runnable 的配置。

  • version (Literal['v1', 'v2']) – 要使用的模式版本，v2 或 v1。用户应使用 v2。v1 用于向后兼容，将在 0.4.0 中弃用。在 API 稳定之前，不会分配默认值。自定义事件将仅在 v2 中显示。

  • include_names (Optional[Sequence[str]]) – 仅包括来自具有匹配名称的 runnable 的事件。

  • include_types (Optional[Sequence[str]]) – 仅包括来自具有匹配类型的 runnable 的事件。

  • include_tags (Optional[Sequence[str]]) – 仅包括来自具有匹配标签的 runnable 的事件。

  • exclude_names (Optional[Sequence[str]]) – 排除来自具有匹配名称的 runnable 的事件。

  • exclude_types (Optional[Sequence[str]]) – 排除来自具有匹配类型的 runnable 的事件。

  • exclude_tags (Optional[Sequence[str]]) – 排除来自具有匹配标签的 runnable 的事件。

  • kwargs (Any) – 传递给 Runnable 的其他关键字参数。 这些参数将传递给 astream_log，因为 astream_events 的此实现是基于 astream_log 构建的。

Yields

StreamEvents 的异步流。

Raises

NotImplementedError – 如果版本不是 v1 或 v2。

Return type

AsyncIterator[Union[StandardStreamEvent, CustomStreamEvent]]

batch(inputs: List[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Any) → List[str]

默认实现使用线程池执行器并行运行 invoke。

The default implementation of batch works well for IO bound runnables.

Subclasses should override this method if they can batch more efficiently; e.g., if the underlying Runnable uses an API which supports a batch mode.

Parameters
Return type

List[str]

batch_as_completed(inputs: Sequence[Input], config: Optional[Union[RunnableConfig, Sequence[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Optional[Any]) → Iterator[Tuple[int, Union[Output, Exception]]]

并行运行列表中输入的 invoke，并在完成时产生结果。

Parameters

  • inputs (Sequence[Input]) –

  • config (Optional[Union[RunnableConfig, Sequence[RunnableConfig]]]) –

  • return_exceptions (bool) –

  • kwargs (Optional[Any]) –

Return type

Iterator[Tuple[int, Union[Output, Exception]]]

configurable_alternatives(which: ConfigurableField, *, default_key: str = 'default', prefix_keys: bool = False, **kwargs: Union[Runnable[Input, Output], Callable[[], Runnable[Input, Output]]]) → RunnableSerializable[Input, Output]

配置可在运行时设置的 Runnables 的备选项。

Parameters

  • which (ConfigurableField) – 将用于选择备选项的 ConfigurableField 实例。

  • default_key (str) – 如果未选择备选项，则使用的默认键。默认为 “default”。

  • prefix_keys (bool) – 是否使用 ConfigurableField id 作为键的前缀。默认为 False。

  • **kwargs (Union[Runnable[Input, Output], Callable[[], Runnable[Input, Output]]]) – 键到 Runnable 实例或返回 Runnable 实例的可调用对象的字典。

Returns

配置了备选项的新 Runnable。

Return type

RunnableSerializable[Input, Output]

from langchain_anthropic import ChatAnthropic
from langchain_core.runnables.utils import ConfigurableField
from langchain_openai import ChatOpenAI

model = ChatAnthropic(
    model_name="claude-3-sonnet-20240229"
).configurable_alternatives(
    ConfigurableField(id="llm"),
    default_key="anthropic",
    openai=ChatOpenAI()
)

# uses the default model ChatAnthropic
print(model.invoke("which organization created you?").content)

# uses ChatOpenAI
print(
    model.with_config(
        configurable={"llm": "openai"}
    ).invoke("which organization created you?").content
)
configurable_fields(**kwargs: Union[ConfigurableField, ConfigurableFieldSingleOption, ConfigurableFieldMultiOption]) → RunnableSerializable[Input, Output]

在运行时配置特定的 Runnable 字段。

Parameters

**kwargs (Union[ConfigurableField, ConfigurableFieldSingleOption, ConfigurableFieldMultiOption]) – 要配置的 ConfigurableField 实例的字典。

Returns

配置了字段的新 Runnable。

Return type

RunnableSerializable[Input, Output]

from langchain_core.runnables import ConfigurableField
from langchain_openai import ChatOpenAI

model = ChatOpenAI(max_tokens=20).configurable_fields(
    max_tokens=ConfigurableField(
        id="output_token_number",
        name="Max tokens in the output",
        description="The maximum number of tokens in the output",
    )
)

# max_tokens = 20
print(
    "max_tokens_20: ",
    model.invoke("tell me something about chess").content
)

# max_tokens = 200
print("max_tokens_200: ", model.with_config(
    configurable={"output_token_number": 200}
    ).invoke("tell me something about chess").content
)
generate(prompts: List[str], stop: Optional[List[str]] = None, callbacks: Union[List[BaseCallbackHandler], BaseCallbackManager, None, List[Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]]] = None, *, tags: Optional[Union[List[str], List[List[str]]]] = None, metadata: Optional[Union[Dict[str, Any], List[Dict[str, Any]]]] = None, run_name: Optional[Union[str, List[str]]] = None, run_id: Optional[Union[UUID, List[Optional[UUID]]]] = None, **kwargs: Any) → LLMResult

将提示序列传递给模型并返回生成结果。

This method should make use of batched calls for models that expose a batched API.

Use this method when you want to

  1. take advantage of batched calls,

  2. need more output from the model than just the top generated value,

  3. are building chains that are agnostic to the underlying language model

    type (e.g., pure text completion models vs chat models).

Parameters

  • prompts (List[str]) – List of string prompts.

  • stop (Optional[List[str]]) – Stop words to use when generating. Model output is cut off at the first occurrence of any of these substrings.

  • callbacks (Union[List[BaseCallbackHandler], BaseCallbackManager, None, List[Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]]]) – Callbacks to pass through. Used for executing additional functionality, such as logging or streaming, throughout generation.

  • tags (可选[Union[List[str], List[List[str]]]]) – 与每个提示关联的标签列表。如果提供，则列表的长度必须与提示列表的长度匹配。

  • metadata (可选[Union[Dict[str, Any], List[Dict[str, Any]]]]) – 与每个提示关联的元数据字典列表。如果提供，则列表的长度必须与提示列表的长度匹配。

  • run_name (可选[Union[str, List[str]]]) – 与每个提示关联的运行名称列表。如果提供，则列表的长度必须与提示列表的长度匹配。

  • run_id (可选[Union[UUID, List[Optional[UUID]]]]) – 与每个提示关联的运行 ID 列表。如果提供，则列表的长度必须与提示列表的长度匹配。

  • **kwargs (Any) – Arbitrary additional keyword arguments. These are usually passed to the model provider API call.

Returns

一个 LLMResult，其中包含每个输入的候选 Generations 列表

prompt 和其他模型提供商特定的输出。

Return type

LLMResult

generate_prompt(prompts: List[PromptValue], stop: Optional[List[str]] = None, callbacks: Union[List[BaseCallbackHandler], BaseCallbackManager, None, List[Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]]] = None, **kwargs: Any) → LLMResult

将提示序列传递给模型并返回模型生成结果。

This method should make use of batched calls for models that expose a batched API.

Use this method when you want to

  1. take advantage of batched calls,

  2. need more output from the model than just the top generated value,

  3. are building chains that are agnostic to the underlying language model

    type (e.g., pure text completion models vs chat models).

Parameters

  • prompts (List[PromptValue]) – PromptValue 列表。PromptValue 是一个可以转换为匹配任何语言模型格式的对象（纯文本生成模型的字符串和聊天模型的基础消息）。

  • stop (Optional[List[str]]) – Stop words to use when generating. Model output is cut off at the first occurrence of any of these substrings.

  • callbacks (Union[List[BaseCallbackHandler], BaseCallbackManager, None, List[Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]]]) – Callbacks to pass through. Used for executing additional functionality, such as logging or streaming, throughout generation.

  • **kwargs (Any) – Arbitrary additional keyword arguments. These are usually passed to the model provider API call.

Returns

一个 LLMResult，其中包含每个输入的候选 Generations 列表

prompt 和其他模型提供商特定的输出。

Return type

LLMResult

get_num_tokens(text: str) → int

获取文本中存在的 token 数量。

用于检查输入是否适合模型的上下文窗口。

Parameters

text (str) – 要进行 token 化的字符串输入。

Returns

文本中的整数 token 数量。

Return type

int

get_num_tokens_from_messages(messages: List[BaseMessage]) → int

获取消息中的 token 数量。

用于检查输入是否适合模型的上下文窗口。

Parameters

messages (List[BaseMessage]) – 要进行 token 化的消息输入。

Returns

所有消息的 token 数量之和。

Return type

int

get_token_ids(text: str) → List[int]

返回文本中 token 的有序 ID。

Parameters

text (str) – 要进行 token 化的字符串输入。

Returns

与文本中的 token 相对应的 ID 列表，按它们在文本中出现的顺序排列。

在文本中。

Return type

List[int]

invoke(input: Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]], config: Optional[RunnableConfig] = None, *, stop: Optional[List[str]] = None, **kwargs: Any) → str

将单个输入转换为输出。覆盖以实现。

Parameters

  • input (Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]) – Runnable 的输入。

  • config (Optional[RunnableConfig]) – 调用 Runnable 时使用的配置。该配置支持标准键，例如 ‘tags’、‘metadata’ 用于跟踪目的，‘max_concurrency’ 用于控制并行执行的工作量，以及其他键。请参阅 RunnableConfig 以获取更多详细信息。

  • stop (Optional[List[str]]) –

  • kwargs (Any) –

Returns

Runnable 的输出。

Return type

str

predict(text: str, *, stop: Optional[Sequence[str]] = None, **kwargs: Any) → str

Deprecated since version langchain-core==0.1.7: Use invoke instead.

Parameters

  • text (str) –

  • stop (Optional[Sequence[str]]) –

  • kwargs (Any) –

Return type

str

predict_messages(messages: List[BaseMessage], *, stop: Optional[Sequence[str]] = None, **kwargs: Any) → BaseMessage

Deprecated since version langchain-core==0.1.7: Use invoke instead.

Parameters

  • messages (List[BaseMessage]) –

  • stop (Optional[Sequence[str]]) –

  • kwargs (Any) –

Return type

BaseMessage

save(file_path: Union[Path, str]) → None

保存 LLM。

Parameters

file_path (Union[Path, str]) – LLM 要保存到的文件路径。

Raises

ValueError – 如果文件路径不是字符串或 Path 对象。

Return type

None

示例： .. code-block:: python

llm.save(file_path=”path/llm.yaml”)

stream(input: Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]], config: Optional[RunnableConfig] = None, *, stop: Optional[List[str]] = None, **kwargs: Any) → Iterator[str]

stream 的默认实现，它调用 invoke。如果子类支持流式输出，则应覆盖此方法。

Parameters

  • input (Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]) – Runnable 的输入。

  • config (Optional[RunnableConfig]) – 用于 Runnable 的配置。默认为 None。

  • kwargs (Any) – Additional keyword arguments to pass to the Runnable.

  • stop (Optional[List[str]]) –

Yields

Runnable 的输出。

Return type

Iterator[str]

to_json() → Union[SerializedConstructor, SerializedNotImplemented]

将 Runnable 序列化为 JSON 格式。

Returns

Runnable 的 JSON 可序列化表示形式。

Return type

Union[SerializedConstructor, SerializedNotImplemented]

with_structured_output(schema: Union[Dict, Type[BaseModel]], **kwargs: Any) → Runnable[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]], Union[Dict, BaseModel]]

在此类中未实现。

Parameters

  • schema (Union[Dict, Type[BaseModel]]) –

  • kwargs (Any) –

Return type

Runnable[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]], Union[Dict, BaseModel]]

LLM 使用示例