`langchain_experimental.llms.rellm_decoder`.RELLM¶

注意

RELLM 实现了标准的 Runnable 接口。 🏃

Runnable 接口 具有在 runnables 上可用的其他方法，例如 with_types、 with_retry、 assign、 bind、 get_graph 等。

class langchain_experimental.llms.rellm_decoder.RELLM[源代码]¶

基类： HuggingFacePipeline

使用 HuggingFace Pipeline API 封装的 RELLM LLM。

param batch_size: int = 4¶: 当传递多个文档进行生成时使用的批处理大小。

param cache: Union[BaseCache, bool, None] = None¶

是否缓存响应。

如果为 true，将使用全局缓存。
如果为 false，将不使用缓存
如果为 None，如果已设置全局缓存，将使用全局缓存，否则不使用缓存。
如果是 BaseCache 的实例，将使用提供的缓存。

模型的流式方法当前不支持缓存。

param callback_manager: Optional[BaseCallbackManager] = None¶: [已弃用]

param callbacks: Callbacks = None¶: 添加到运行轨迹的回调。

param custom_get_token_ids: Optional[Callable[[str], List[int]]] = None¶: 用于计算 token 的可选编码器。

param max_new_tokens: int = 200¶: 要生成的最大新 token 数。

param metadata: Optional[Dict[str, Any]] = None¶: 添加到运行轨迹的元数据。

param model_id: str = 'gpt2'¶: 要使用的模型名称。

param model_kwargs: Optional[dict] = None¶: 传递给模型的关键字参数。

param pipeline_kwargs: Optional[dict] = None¶: 传递给 pipeline 的关键字参数。

param regex: RegexPattern [必需]¶: 要完成的结构化格式。

param tags: Optional[List[str]] = None¶: 添加到运行轨迹的标签。

param verbose: bool [可选]¶: 是否打印响应文本。

__call__(prompt: str, stop: Optional[List[str]] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → str¶

版本 langchain-core==0.1.7 中已弃用: 请使用 invoke 代替。

检查缓存并在给定提示和输入上运行 LLM。

参数

prompt (str) – 要从中生成的提示。
stop (Optional[List[str]]) – 生成时要使用的停止词。模型输出在第一次出现任何这些子字符串时被截断。
callbacks (Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]) – 要传递的回调。用于在整个生成过程中执行额外的功能，例如日志记录或流式传输。
tags (Optional[List[str]]) – 要与提示关联的标签列表。
metadata (Optional[Dict[str, Any]]) – 要与提示关联的元数据。
**kwargs (Any) – 任意附加关键字参数。这些通常传递给模型提供商 API 调用。

返回值

生成的文本。

Raises

ValueError – 如果提示不是字符串。

返回类型

str

async abatch(inputs: List[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Any) → List[str]¶

默认实现使用 asyncio.gather 并行运行 ainvoke。

batch 的默认实现非常适合 IO 绑定的 runnables。

如果子类可以更有效地进行批处理，则应覆盖此方法；例如，如果底层的 Runnable 使用支持批处理模式的 API。

参数

inputs (List[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]]) – Runnable 的输入列表。
config (Optional[Union[RunnableConfig, List[RunnableConfig]]]) – 调用 Runnable 时要使用的配置。该配置支持标准键，例如用于跟踪目的的“tags”、“metadata”，用于控制并行执行多少工作的“max_concurrency”以及其他键。有关更多详细信息，请参阅 RunnableConfig。默认为 None。
return_exceptions (bool) – 是否返回异常而不是引发异常。默认为 False。
kwargs (Any) – 要传递给 Runnable 的其他关键字参数。

返回值

Runnable 的输出列表。

返回类型

List[str]

async abatch_as_completed(inputs: Sequence[Input], config: Optional[Union[RunnableConfig, Sequence[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Optional[Any]) → AsyncIterator[Tuple[int, Union[Output, Exception]]]¶

在一系列输入上并行运行 ainvoke，并在结果完成时产生结果。

参数

inputs (Sequence[Input]) – Runnable 的输入列表。
config (Optional[Union[RunnableConfig, Sequence[RunnableConfig]]]) – 调用 Runnable 时要使用的配置。该配置支持标准键，例如用于跟踪目的的“tags”、“metadata”，用于控制并行执行多少工作的“max_concurrency”以及其他键。有关更多详细信息，请参阅 RunnableConfig。默认为 None。默认为 None。
return_exceptions (bool) – 是否返回异常而不是引发异常。默认为 False。
kwargs (Optional[Any]) – 要传递给 Runnable 的其他关键字参数。

Yields

输入索引和来自 Runnable 的输出的元组。

返回类型

AsyncIterator[Tuple[int, Union[Output, Exception]]]

async agenerate(prompts: List[str], stop: Optional[List[str]] = None, callbacks: Union[List[BaseCallbackHandler], BaseCallbackManager, None, List">[Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]]] = None, *, tags: Optional[Union[List[str], List">[List">[str]]]] = None, metadata: Optional[Union[Dict[str, Any], List[Dict[str, Any]]]] = None, run_name: Optional[Union[str, List">[str]]] = None, run_id: Optional[Union[UUID, List[Optional[UUID]]]] = None, **kwargs: Any) → LLMResult¶

Asynchronously pass a sequence of prompts to a model and return generations.

This method should make use of batched calls for models that expose a batched API.

Use this method when you want to

take advantage of batched calls,
need more output from the model than just the top generated value,
are building chains that are agnostic to the underlying language model
type (e.g., pure text completion models vs chat models).

参数

prompts (List[str]) – List of string prompts.
stop (Optional[List[str]]) – 生成时要使用的停止词。模型输出在第一次出现任何这些子字符串时被截断。
callbacks (Union[List[BaseCallbackHandler], BaseCallbackManager, None, List[Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]]]) – Callbacks to pass through. Used for executing additional functionality, such as logging or streaming, throughout generation.
tags (Optional[Union[List[str], List[List[str]]]]) – List of tags to associate with each prompt. If provided, the length of the list must match the length of the prompts list.
metadata (Optional[Union[Dict[str, Any], List[Dict[str, Any]]]]) – List of metadata dictionaries to associate with each prompt. If provided, the length of the list must match the length of the prompts list.
run_name (Optional[Union[str, List[str]]]) – List of run names to associate with each prompt. If provided, the length of the list must match the length of the prompts list.
run_id (Optional[Union[UUID, List[Optional[UUID]]]]) – List of run IDs to associate with each prompt. If provided, the length of the list must match the length of the prompts list.
**kwargs (Any) – 任意附加关键字参数。这些通常传递给模型提供商 API 调用。

返回值

An LLMResult, which contains a list of candidate Generations for each input: prompt and additional model provider-specific output.

返回类型

LLMResult

async agenerate_prompt(prompts: List[PromptValue], stop: Optional[List[str]] = None, callbacks: Union[List[BaseCallbackHandler], BaseCallbackManager, None, List[Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]]] = None, **kwargs: Any) → LLMResult¶

Asynchronously pass a sequence of prompts and return model generations.

This method should make use of batched calls for models that expose a batched API.

Use this method when you want to

take advantage of batched calls,
need more output from the model than just the top generated value,
are building chains that are agnostic to the underlying language model
type (e.g., pure text completion models vs chat models).

参数

prompts (List[PromptValue]) – List of PromptValues. A PromptValue is an object that can be converted to match the format of any language model (string for pure text generation models and BaseMessages for chat models).
stop (Optional[List[str]]) – 生成时要使用的停止词。模型输出在第一次出现任何这些子字符串时被截断。
callbacks (Union[List[BaseCallbackHandler], BaseCallbackManager, None, List[Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]]]) – Callbacks to pass through. Used for executing additional functionality, such as logging or streaming, throughout generation.
**kwargs (Any) – 任意附加关键字参数。这些通常传递给模型提供商 API 调用。

返回值

An LLMResult, which contains a list of candidate Generations for each input: prompt and additional model provider-specific output.

返回类型

LLMResult

async ainvoke(input: Union[PromptValue, str, Sequence[Union[BaseMessage, List">[str], Tuple[str, str], str, Dict[str, Any]]]], config: Optional[RunnableConfig] = None, *, stop:> Optional[List[str]] = None, **kwargs:> Any) → str¶

Default implementation of ainvoke, calls invoke from a thread.

The default implementation allows usage of async code even if the Runnable did not implement a native async version of invoke.

Subclasses should override this method if they can run asynchronously.

参数

input (Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]) –
config (Optional[RunnableConfig]) –
stop (Optional[List[str]]) –
kwargs (Any) –

返回类型

str

async apredict(text: str, *, stop: Optional[Sequence[str]] = None, **kwargs:> Any) → str¶

Deprecated since version langchain-core==0.1.7: Use ainvoke instead.

参数

text (str) –
stop (Optional[Sequence[str]]) –
kwargs (Any) –

返回类型

str

async apredict_messages(messages: List[BaseMessage], *, stop: Optional[Sequence[str]] = None, **kwargs:> Any) → BaseMessage¶

Deprecated since version langchain-core==0.1.7: Use ainvoke instead.

参数

messages (List[BaseMessage]) –
stop (Optional[Sequence[str]]) –
kwargs (Any) –

返回类型

BaseMessage

as_tool(args_schema: Optional[Type[BaseModel]] = None, *, name: Optional[str] = None, description: Optional[str] = None, arg_types: Optional[Dict[str, Type]] = None) → BaseTool¶

Beta

This API is in beta and may change in the future.

Create a BaseTool from a Runnable.

as_tool will instantiate a BaseTool with a name, description, and args_schema from a Runnable. Where possible, schemas are inferred from runnable.get_input_schema. Alternatively (e.g., if the Runnable takes a dict as input and the specific dict keys are not typed), the schema can be specified directly with args_schema. You can also pass arg_types to just specify the required arguments and their types.

参数

args_schema (Optional[Type[BaseModel]]) – The schema for the tool. Defaults to None.
name (Optional[str]) – The name of the tool. Defaults to None.
description (Optional[str]) – The description of the tool. Defaults to None.
arg_types (Optional[Dict[str, Type]]) – A dictionary of argument names to types. Defaults to None.

返回值

A BaseTool instance.

返回类型

BaseTool

Typed dict input

from typing import List
from typing_extensions import TypedDict
from langchain_core.runnables import RunnableLambda

class Args(TypedDict):
    a: int
    b: List[int]

def f(x: Args) -> str:
    return str(x["a"] * max(x["b"]))

runnable = RunnableLambda(f)
as_tool = runnable.as_tool()
as_tool.invoke({"a": 3, "b": [1, 2]})

dict input, specifying schema via args_schema

from typing import Any, Dict, List
from langchain_core.pydantic_v1 import BaseModel, Field
from langchain_core.runnables import RunnableLambda

def f(x: Dict[str, Any]) -> str:
    return str(x["a"] * max(x["b"]))

class FSchema(BaseModel):
    """Apply a function to an integer and list of integers."""

    a: int = Field(..., description="Integer")
    b: List[int] = Field(..., description="List of ints")

runnable = RunnableLambda(f)
as_tool = runnable.as_tool(FSchema)
as_tool.invoke({"a": 3, "b": [1, 2]})

dict input, specifying schema via arg_types

from typing import Any, Dict, List
from langchain_core.runnables import RunnableLambda

def f(x: Dict[str, Any]) -> str:
    return str(x["a"] * max(x["b"]))

runnable = RunnableLambda(f)
as_tool = runnable.as_tool(arg_types={"a": int, "b": List[int]})
as_tool.invoke({"a": 3, "b": [1, 2]})

String input

from langchain_core.runnables import RunnableLambda

def f(x: str) -> str:
    return x + "a"

def g(x: str) -> str:
    return x + "z"

runnable = RunnableLambda(f) | g
as_tool = runnable.as_tool()
as_tool.invoke("b")

New in version 0.2.14.

async astream(input: Union[PromptValue, str, Sequence[Union[BaseMessage, List">[str], Tuple[str, str], str, Dict[str, Any]]]], config: Optional[RunnableConfig] = None, *, stop: Optional[List[str]] = None, **kwargs:> Any) → AsyncIterator[str]¶

Default implementation of astream, which calls ainvoke. Subclasses should override this method if they support streaming output.

参数

input (Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]) – The input to the Runnable.
config (Optional[RunnableConfig]) – The config to use for the Runnable. Defaults to None.
kwargs (Any) – 要传递给 Runnable 的其他关键字参数。
stop (Optional[List[str]]) –

Yields

The output of the Runnable.

返回类型

AsyncIterator[str]

astream_events(input: Any, config: Optional[RunnableConfig] = None, *, version: Literal['v1', 'v2'], include_names: Optional[Sequence[str]] = None, include_types: Optional[Sequence[str]] = None, include_tags: Optional[Sequence[str]] = None, exclude_names: Optional[Sequence[str]] = None, exclude_types: Optional[Sequence[str]] = None, exclude_tags: Optional[Sequence[str]] = None, **kwargs: Any) → AsyncIterator[Union[StandardStreamEvent, CustomStreamEvent]]¶

Beta

This API is in beta and may change in the future.

astream_events(input: Any, config: Optional[RunnableConfig] = None, *, version: Literal['v1', 'v2'], include_names: Optional[Sequence[str]] = None, include_types: Optional[Sequence[str]] = None, include_tags: Optional[Sequence[str]] = None, exclude_names: Optional[Sequence[str]] = None, exclude_types: Optional[Sequence[str]] = None, exclude_tags: Optional[Sequence[str]] = None, **kwargs: Any) → AsyncIterator[Union[StandardStreamEvent, CustomStreamEvent]]¶

生成事件流。

用于创建一个迭代器，遍历 StreamEvents，这些事件提供关于 Runnable 进度的实时信息，包括来自中间结果的 StreamEvents。

StreamEvent 是一个具有以下模式的字典

event: str - 事件名称的格式为：
格式：on_[runnable_type]_(start|stream|end)。
name: str - 生成事件的 Runnable 的名称。
run_id: str - 随机生成的 ID，与给定 Runnable 执行的事件相关联。
发出事件的 Runnable 的执行。作为父 Runnable 执行一部分而被调用的子 Runnable 将被分配其自己唯一的 ID。
parent_ids: List[str] - 生成事件的父 runnables 的 ID 列表。
根 Runnable 将具有一个空列表。父 ID 的顺序是从根到直接父级。仅适用于 API 的 v2 版本。API 的 v1 版本将返回一个空列表。
tags: Optional[List[str]] - 生成事件的 Runnable 的标签。
事件。
metadata: Optional[Dict[str, Any]] - Runnable 的元数据
生成事件的 Runnable 的元数据。
data: Dict[str, Any]

下面是一个表格，说明了各种链可能发出的一些事件。为了简洁起见，表格中省略了元数据字段。链定义已包含在表格之后。

注意此参考表适用于 V2 版本的模式。

事件	名称	块	输入	输出
on_chat_model_start	[模型名称]		{“messages”: [[SystemMessage, HumanMessage]]}
on_chat_model_stream	[模型名称]	AIMessageChunk(content=”hello”)
on_chat_model_end	[模型名称]		{“messages”: [[SystemMessage, HumanMessage]]}	AIMessageChunk(content=”hello world”)
on_llm_start	[模型名称]		{‘input’: ‘hello’}
on_llm_stream	[模型名称]	‘Hello’
on_llm_end	[模型名称]		‘Hello human!’
on_chain_start	format_docs
on_chain_stream	format_docs	“hello world!, goodbye world!”
on_chain_end	format_docs		[Document(…)]	“hello world!, goodbye world!”
on_tool_start	some_tool		{“x”: 1, “y”: “2”}
on_tool_end	some_tool			{“x”: 1, “y”: “2”}
on_retriever_start	[retriever name]		{“query”: “hello”}
on_retriever_end	[retriever name]		{“query”: “hello”}	[Document(…), ..]
on_prompt_start	[template_name]		{“question”: “hello”}
on_prompt_end	[template_name]		{“question”: “hello”}	ChatPromptValue(messages: [SystemMessage, …])

除了标准事件之外，用户还可以调度自定义事件（请参见下面的示例）。

自定义事件将仅在 API 的 v2 版本中显示！

自定义事件具有以下格式

属性	类型	描述
名称	str	事件的用户定义名称。
数据	Any	与事件关联的数据。这可以是任何内容，但我们建议使其可 JSON 序列化。

以下是与上面显示的标准事件关联的声明

format_docs:

def format_docs(docs: List[Document]) -> str:
    '''Format the docs.'''
    return ", ".join([doc.page_content for doc in docs])

format_docs = RunnableLambda(format_docs)

some_tool:

@tool
def some_tool(x: int, y: str) -> dict:
    '''Some_tool.'''
    return {"x": x, "y": y}

提示:

template = ChatPromptTemplate.from_messages(
    [("system", "You are Cat Agent 007"), ("human", "{question}")]
).with_config({"run_name": "my_template", "tags": ["my_template"]})

示例

from langchain_core.runnables import RunnableLambda

async def reverse(s: str) -> str:
    return s[::-1]

chain = RunnableLambda(func=reverse)

events = [
    event async for event in chain.astream_events("hello", version="v2")
]

# will produce the following events (run_id, and parent_ids
# has been omitted for brevity):
[
    {
        "data": {"input": "hello"},
        "event": "on_chain_start",
        "metadata": {},
        "name": "reverse",
        "tags": [],
    },
    {
        "data": {"chunk": "olleh"},
        "event": "on_chain_stream",
        "metadata": {},
        "name": "reverse",
        "tags": [],
    },
    {
        "data": {"output": "olleh"},
        "event": "on_chain_end",
        "metadata": {},
        "name": "reverse",
        "tags": [],
    },
]

示例：调度自定义事件

from langchain_core.callbacks.manager import (
    adispatch_custom_event,
)
from langchain_core.runnables import RunnableLambda, RunnableConfig
import asyncio


async def slow_thing(some_input: str, config: RunnableConfig) -> str:
    """Do something that takes a long time."""
    await asyncio.sleep(1) # Placeholder for some slow operation
    await adispatch_custom_event(
        "progress_event",
        {"message": "Finished step 1 of 3"},
        config=config # Must be included for python < 3.10
    )
    await asyncio.sleep(1) # Placeholder for some slow operation
    await adispatch_custom_event(
        "progress_event",
        {"message": "Finished step 2 of 3"},
        config=config # Must be included for python < 3.10
    )
    await asyncio.sleep(1) # Placeholder for some slow operation
    return "Done"

slow_thing = RunnableLambda(slow_thing)

async for event in slow_thing.astream_events("some_input", version="v2"):
    print(event)

参数

input (Any) – Runnable 的输入。
config (Optional[RunnableConfig]) – 用于 Runnable 的配置。
version (Literal['v1', 'v2']) – 要使用的模式版本，可以是 v2 或 v1。用户应使用 v2。v1 用于向后兼容，将在 0.4.0 版本中弃用。在 API 稳定之前，不会分配默认值。自定义事件将仅在 v2 中显示。
include_names (Optional[Sequence[str]]) – 仅包括来自具有匹配名称的 runnables 的事件。
include_types (Optional[Sequence[str]]) – 仅包括来自具有匹配类型的 runnables 的事件。
include_tags (Optional[Sequence[str]]) – 仅包括来自具有匹配标签的 runnables 的事件。
exclude_names (Optional[Sequence[str]]) – 排除来自具有匹配名称的 runnables 的事件。
exclude_types (Optional[Sequence[str]]) – 排除来自具有匹配类型的 runnables 的事件。
exclude_tags (Optional[Sequence[str]]) – 排除来自具有匹配标签的 runnables 的事件。
kwargs (Any) – 传递给 Runnable 的其他关键字参数。这些参数将传递给 astream_log，因为 astream_events 的此实现构建在 astream_log 之上。

Yields

StreamEvents 的异步流。

Raises

NotImplementedError – 如果版本不是 v1 或 v2。

返回类型

AsyncIterator[Union[StandardStreamEvent, CustomStreamEvent]]

batch(inputs: List[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Any) → List[str]¶

默认实现使用线程池执行器并行运行 invoke。

batch 的默认实现非常适合 IO 绑定的 runnables。

如果子类可以更有效地进行批处理，则应覆盖此方法；例如，如果底层的 Runnable 使用支持批处理模式的 API。

参数

inputs (List[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]]) –
config (Optional[Union[RunnableConfig, List[RunnableConfig]]]) –
return_exceptions (bool) –
kwargs (Any) –

返回类型

List[str]

batch_as_completed(inputs: Sequence[Input], config: Optional[Union[RunnableConfig, Sequence[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Optional[Any]) → Iterator[Tuple[int, Union[Output, Exception]]]¶

并行运行列表中输入的 invoke，并在完成时生成结果。

参数

inputs (Sequence[Input]) –
config (Optional[Union[RunnableConfig, Sequence[RunnableConfig]]]) –
return_exceptions (bool) –
kwargs (Optional[Any]) –

返回类型

Iterator[Tuple[int, Union[Output, Exception]]]

configurable_alternatives(which: ConfigurableField, *, default_key: str = 'default', prefix_keys: bool = False, **kwargs: Union[Runnable[Input, Output], Callable[[], Runnable[Input, Output]]]) → RunnableSerializable[Input, Output]¶

配置可在运行时设置的 Runnables 的备选项。

参数

which (ConfigurableField) – 将用于选择备选项的 ConfigurableField 实例。
default_key (str) – 如果未选择备选项，则使用的默认键。默认为“default”。
prefix_keys (bool) – 是否使用 ConfigurableField id 作为键的前缀。默认为 False。
**kwargs (Union[Runnable[Input, Output], Callable[[], Runnable[Input, Output]]]) – 键到 Runnable 实例或返回 Runnable 实例的可调用对象的字典。

返回值

配置了备选项的新 Runnable。

返回类型

RunnableSerializable[Input, Output]

from langchain_anthropic import ChatAnthropic
from langchain_core.runnables.utils import ConfigurableField
from langchain_openai import ChatOpenAI

model = ChatAnthropic(
    model_name="claude-3-sonnet-20240229"
).configurable_alternatives(
    ConfigurableField(id="llm"),
    default_key="anthropic",
    openai=ChatOpenAI()
)

# uses the default model ChatAnthropic
print(model.invoke("which organization created you?").content)

# uses ChatOpenAI
print(
    model.with_config(
        configurable={"llm": "openai"}
    ).invoke("which organization created you?").content
)

configurable_fields(**kwargs: Union[ConfigurableField, ConfigurableFieldSingleOption, ConfigurableFieldMultiOption]) → RunnableSerializable[Input, Output]¶

在运行时配置特定的 Runnable 字段。

参数: **kwargs (Union[ConfigurableField, ConfigurableFieldSingleOption, ConfigurableFieldMultiOption]) – 要配置的 ConfigurableField 实例的字典。
返回值: 配置了字段的新 Runnable。
返回类型: RunnableSerializable[Input, Output]

from langchain_core.runnables import ConfigurableField
from langchain_openai import ChatOpenAI

model = ChatOpenAI(max_tokens=20).configurable_fields(
    max_tokens=ConfigurableField(
        id="output_token_number",
        name="Max tokens in the output",
        description="The maximum number of tokens in the output",
    )
)

# max_tokens = 20
print(
    "max_tokens_20: ",
    model.invoke("tell me something about chess").content
)

# max_tokens = 200
print("max_tokens_200: ", model.with_config(
    configurable={"output_token_number": 200}
    ).invoke("tell me something about chess").content
)

classmethod from_model_id(model_id: str, task: str, backend: str = 'default', device: Optional[int] = -1, device_map: Optional[str] = None, model_kwargs: Optional[dict] = None, pipeline_kwargs: Optional[dict] = None, batch_size: int = 4, **kwargs: Any) → HuggingFacePipeline¶

从 model_id 和 task 构建 pipeline 对象。

参数

model_id (str) –
task (str) –
backend (str) –
device (Optional[int]) –
device_map (Optional[str]) –
model_kwargs (Optional[dict]) –
pipeline_kwargs (Optional[dict]) –
batch_size (int) –
kwargs (Any) –

返回类型

HuggingFacePipeline

generate(prompts: List[str], stop: Optional[List[str]] = None, callbacks: Union[List[BaseCallbackHandler],BaseCallbackManager, None, List[Optional[Union[List[BaseCallbackHandler],BaseCallbackManager]]]] = None, *, tags: Optional[Union[List[str], List[List[str]]]] = None, metadata: Optional[Union[Dict[str, Any], List[Dict[str, Any]]]] = None, run_name: Optional[Union[str, List[str]]] = None, run_id: Optional[Union[UUID, List[Optional[UUID]]]] = None, **kwargs: Any) → LLMResult¶

将提示序列传递给模型并返回生成结果。

This method should make use of batched calls for models that expose a batched API.

Use this method when you want to

take advantage of batched calls,
need more output from the model than just the top generated value,
are building chains that are agnostic to the underlying language model
type (e.g., pure text completion models vs chat models).

参数

prompts (List[str]) – List of string prompts.
stop (Optional[List[str]]) – 生成时要使用的停止词。模型输出在第一次出现任何这些子字符串时被截断。
callbacks (Union[List[BaseCallbackHandler], BaseCallbackManager, None, List[Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]]]) – Callbacks to pass through. Used for executing additional functionality, such as logging or streaming, throughout generation.
tags (Optional[Union[List[str], List[List[str]]]]) – List of tags to associate with each prompt. If provided, the length of the list must match the length of the prompts list.
metadata (Optional[Union[Dict[str, Any], List[Dict[str, Any]]]]) – List of metadata dictionaries to associate with each prompt. If provided, the length of the list must match the length of the prompts list.
run_name (Optional[Union[str, List[str]]]) – List of run names to associate with each prompt. If provided, the length of the list must match the length of the prompts list.
run_id (Optional[Union[UUID, List[Optional[UUID]]]]) – List of run IDs to associate with each prompt. If provided, the length of the list must match the length of the prompts list.
**kwargs (Any) – 任意附加关键字参数。这些通常传递给模型提供商 API 调用。

返回值

An LLMResult, which contains a list of candidate Generations for each input: prompt and additional model provider-specific output.

返回类型

LLMResult

generate_prompt(prompts: List[PromptValue], stop: Optional[List[str]] = None, callbacks: Union[List[BaseCallbackHandler], BaseCallbackManager, None, List[Optional[Union[List">[BaseCallbackHandler], BaseCallbackManager]]]] = None, **kwargs: Any) → LLMResult¶

向模型传递一系列提示，并返回模型生成结果。

This method should make use of batched calls for models that expose a batched API.

Use this method when you want to

take advantage of batched calls,
need more output from the model than just the top generated value,
are building chains that are agnostic to the underlying language model
type (e.g., pure text completion models vs chat models).

参数

prompts (List[PromptValue]) – List of PromptValues. A PromptValue is an object that can be converted to match the format of any language model (string for pure text generation models and BaseMessages for chat models).
stop (Optional[List[str]]) – 生成时要使用的停止词。模型输出在第一次出现任何这些子字符串时被截断。
callbacks (Union[List[BaseCallbackHandler], BaseCallbackManager, None, List[Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]]]) – Callbacks to pass through. Used for executing additional functionality, such as logging or streaming, throughout generation.
**kwargs (Any) – 任意附加关键字参数。这些通常传递给模型提供商 API 调用。

返回值

An LLMResult, which contains a list of candidate Generations for each input: prompt and additional model provider-specific output.

返回类型

LLMResult

get_num_tokens(text: str) → int¶

获取文本中存在的 token 数量。

用于检查输入是否适合模型的上下文窗口。

参数: text (str) – 要进行 token 化的字符串输入。
返回值: 文本中的 token 整数数量。
返回类型: int

get_num_tokens_from_messages(messages: List[BaseMessage]) → int¶

获取消息中的 token 数量。

用于检查输入是否适合模型的上下文窗口。

参数: messages (List[BaseMessage]) – 要进行 token 化的消息输入。
返回值: 消息中 token 数量的总和。
返回类型: int

get_token_ids(text: str) → List[int]¶

返回文本中 token 的有序 ID。

参数

text (str) – 要进行 token 化的字符串输入。

返回值

与文本中的 token 相对应的 ID 列表，按照它们在文本中出现的顺序排列: 在文本中。

返回类型

List[int]

invoke(input: Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]], config: Optional[RunnableConfig] = None, *, stop: Optional[List[str]] = None, **kwargs: Any) → str¶

将单个输入转换为输出。覆盖此方法以进行实现。

参数

input (Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]) – The input to the Runnable.
config (Optional[RunnableConfig]) – 调用 Runnable 时使用的配置。该配置支持标准键，例如用于跟踪目的的 ‘tags’、‘metadata’，用于控制并行执行量的 ‘max_concurrency’ 以及其他键。请参阅 RunnableConfig 以获取更多详细信息。
stop (Optional[List[str]]) –
kwargs (Any) –

返回值

The output of the Runnable.

返回类型

str

predict(text: str, *, stop: Optional[Sequence[str]] = None, **kwargs: Any) → str¶

版本 langchain-core==0.1.7 中已弃用: 请使用 invoke 代替。

参数

text (str) –
stop (Optional[Sequence[str]]) –
kwargs (Any) –

返回类型

str

predict_messages(messages: List[BaseMessage], *, stop: Optional[Sequence[str]] = None, **kwargs: Any) → BaseMessage¶

版本 langchain-core==0.1.7 中已弃用: 请使用 invoke 代替。

参数

messages (List[BaseMessage]) –
stop (Optional[Sequence[str]]) –
kwargs (Any) –

返回类型

BaseMessage

save(file_path: Union[Path, str]) → None¶

保存 LLM。

参数: file_path (Union[Path, str]) – LLM 要保存到的文件路径。
Raises: ValueError – 如果文件路径不是字符串或 Path 对象，则会引发此错误。
返回类型: None

示例: .. code-block:: python

llm.save(file_path=”path/llm.yaml”)

stream(input: Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]], config: Optional[RunnableConfig] = None, *, stop: Optional[List[str]] = None, **kwargs: Any) → Iterator[str]¶

流式传输的默认实现，它调用 invoke。如果子类支持流式输出，则应覆盖此方法。

参数

input (Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]) – The input to the Runnable.
config (Optional[RunnableConfig]) – The config to use for the Runnable. Defaults to None.
kwargs (Any) – 要传递给 Runnable 的其他关键字参数。
stop (Optional[List[str]]) –

Yields

The output of the Runnable.

返回类型

Iterator[str]

to_json() → Union[SerializedConstructor, SerializedNotImplemented]¶

将 Runnable 序列化为 JSON。

返回值: Runnable 的 JSON 可序列化表示形式。
返回类型: Union[SerializedConstructor, SerializedNotImplemented]

with_structured_output(schema: Union[Dict, Type[BaseModel]], **kwargs: Any) → Runnable[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]], Union[Dict, BaseModel]]¶

此类未实现此功能。

参数

schema (Union[Dict, Type[BaseModel]]) –
kwargs (Any) –

返回类型

Runnable[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]], Union[Dict, BaseModel]]

RELLM 使用示例¶

RELLM

langchain_experimental.llms.rellm_decoder.RELLM¶

RELLM 使用示例¶

`langchain_experimental.llms.rellm_decoder`.RELLM¶