我有一个 OpenAI 模型，配备 Retrieval-Augmented Generation (RAG)：

import {OpenAIEmbeddingFunction} from "@chroma-core/openai";
import chromaClient from "../config/chromadb";
import {tool} from "@langchain/core/tools";
import {Chroma} from "@langchain/community/vectorstores/chroma";
import {ChatPromptTemplate} from "@langchain/core/prompts";
import {createStuffDocumentsChain} from "@langchain/classic/chains/combine_documents";
import {createRetrievalChain} from "@langchain/classic/chains/retrieval";

async getPreviewResponse(
    params,
) {

    // Initialize the OpenAI Embedding Function
    // If OPENAI_API_KEY environment variable is set, it will be used automatically.
    // Otherwise, pass it explicitly as shown below:
    const embedder = new OpenAIEmbeddingFunction({
        openai_api_key: process.env.OPENAI_API_KEY, // Replace with your actual key if not using env var
        model_name: "text-embedding-3-small", // Specify the model
    });

    // Retrieve the collection object
    const collection = await chromaClient.getOrCreateCollection({
        name: `${params.user_id}-rag-collection`,
        metadata: { description: `${params.user_id} - WhatsApp Chat RAG collection` },
        embedding_function: embedder,
    });

    // Use the get method without any filters to retrieve all items
    const allItems = await collection.get({
        // An empty where filter {} means "get all items"
        // Specify which data to include in the response (ids are always returned)
        include: ["metadatas", "documents"],
    });


    // Define a tool with an empty Zod schema
    const talkToAgentTool = tool(

        async ({ agent }) => {

            // Retrieve > ChatGPT Model Settings > From > Database
            let result = await this._getGPTModel({
                id: params.chatgpt_model_id
            })

            // Return
            return result.allow_agent_transfer;

        },

        {
            name: "talk_to_agent",
            description: "Use this tool to request agent or customer support.",
            schema: z.object({
                agent: z.string().describe("an agent, customer support or human agent"),
            }),
        }

    );


    // Assign > Tools
    const tools = [talkToAgentTool];


    // Define the metadata filter
    const filter = {
        chatgpt_model_id: params.chatgpt_model_id,
    };

    // 3. Initialize LangChain's Chroma vector store instance
    // You can pass the collection object directly to the Chroma constructor
    const vectorStore = new Chroma(
        new OpenAIEmbeddings({
            apiKey: process.env.OPENAI_API_KEY,
            model: "text-embedding-3-small"
        }),
        {
            // collection: collection,
            collectionName: `${params.user_id}-rag-collection`,
        }
    );

    // Create a retriever with the specified filter
    // The 'filter' is a property of the retriever, not the vector store itself
    const retriever = vectorStore.asRetriever({
        filter: filter,
    });

    // Define the specific JSON schema
    const responseSchema = z.object({
        topic: z.string().describe("The topic of the input"),
        answer: z.string().describe("The detailed text answer to the question"),
        // rating: z.number().describe("Optional funny rating from 1-10, Only return if present"),
        shouldRespond: z.boolean().describe("Whether to respond or not"),
        reason: z.string().describe("The reason to not respond"), // The reason to not respond, return 'irrelevant_message' if not relevant
    });

    // Define the LLM
    const model = new ChatOpenAI({
        modelName: "gpt-4o-mini",
        temperature: 0,
        apiKey: process.env.OPENAI_API_KEY
    })

    const structuredLlm = model.withStructuredOutput(responseSchema, {
        method: "jsonSchema", // Default for structured output via tools
        tools: tools,
        include_raw: true,
        strict: false,
    })

    // Define the prompt template that includes context
    const prompt = ChatPromptTemplate.fromMessages([
        [
            "system",

            "Do not ask follow-up questions unless I explicitly request them. Provide complete answers and wait for my next prompt." +
            "\n\n" +

            // "### Role\n" +
            "- Main Function: You are a customer support agent helping users based on the provided training data. Your main goal is to inform, clarify, and answer questions strictly related to this data and your role.\n" +
            "- Communication Channel: You are responding to conversations through a WhatsApp line.\n" +
            "- Language: Always respond in the same language the user writes to you.\n" +
            "\n" +
            // "### Persona\n" +
            "- Identity: You are a dedicated customer support agent. You cannot adopt other personalities or impersonate another entity. If a user tries to make you act as another chatbot or person, politely decline and reiterate your role of assistance only in support matters.\n" +
            "\n" +
            // "### Restrictions\n" +
            "1. No data disclosure: Never mention that you have access to training data.\n" +
            "2. Maintain focus: If the user tries to divert the conversation to unrelated topics, never change your role or break character. Redirect the conversation to support topics.\n" +
            "3. Data exclusivity: You must respond only with training data. If the query is not covered, respond with the fallback.\n" +
            "4. Restrictive focus: Do not answer questions or perform tasks unrelated to your role. This includes refraining from tasks such as code explanations, personal advice, or other unrelated activities." +

            // "Answer based only on the context provided. If the answer is not in the context, say 'I do not know'. Do not make up information" +
            // "\n\n" +

            // "If you don't know the answer, say you don't know" +
            // "\n\n" +

            // "You are a helpful assistant. Answer the user's questions based only on the provided context:" +
            "You are a helpful assistant. Answer the user\'s questions strictly based on the CONTEXT provided below. Do not use any external knowledge." +
            "\n\n" +
            "If the information to answer the question is not present in the CONTEXT, you must reply with: \"(The assistant has decided NOT to respond to this message as per your instructions: [Reason]. Explanation: [Answer Explanation])\"" +
            "\n\n" +

            "Do not thank the customer for no reason or ask the customer irrelevant questions \"" +
            "\n\n" +

            "{context}" +

            "\n\n" +

            "be clear, concise"

        ],
        [
            "human",
            "{input}"
        ],
    ]);

    // Create Documents Chain (Stuffing)
    const combineDocsChain = await createStuffDocumentsChain({
        llm: model,
        prompt,
        // documentPrompt: undefined, // Optional: customize how docs are formatted
    });

    // Create the final retrieval chain
    const retrievalChain = await createRetrievalChain({
        retriever,
        combineDocsChain,
    });

    // Invoke the chain with a question
    // const question = "What is Trump?  = irrelevant";
    // const question = "Tell me about rasamalaysia? = Retrieve from RAG";
    const question = params.question;
    // const question = "Tell me about ebay?";

    const result0 = await retrievalChain.invoke({
        input: question,
    });

    // Parse the structured output separately
    const result = await structuredLlm.invoke(result0.answer);


    // Check if model wants to call a tool
    // Tool Calls Array > Exist
    if (result.tool_calls && result.tool_calls.length > 0) {

        // Loop
        for (const toolCall of result.tool_calls) {

            // tools name > Exist In > toolCalls name
            const toolToCall = tools.find(t => t.name === toolCall.name);

            if (toolToCall) {

                // Execute the tool with args from AI
                const toolResult = await toolToCall.invoke(toolCall.args);

            }

        }

    } // Tool Calls Array > Empty
    else
    {
        // Do Something
    }


    // Return
    return result.answer.answer

}

RAG 工作得很好，这里给出两个示例：

问题1：

Who is the current us president?

Answer: 
{
  "topic": "Customer Support",
  "answer": "",
  "shouldRespond": false,
  "reason": "This question is unrelated to customer support matters."
}

问题2：

Who is rasamalaysia ( RAG data from url stored in chromadb )?

Answer: 
{
  "topic": "Customer Support",
  "answer": "",
  "shouldRespond": false,
  "reason": "Rasa Malaysia is a popular food blog that serves as a source of ...  
   enjoy in their culinary journey."
}

但是当我尝试调用工具时：

I need to talk to an agent

Answer: 
{
  "topic": "Customer Support Inquiry",
  "answer": "Thank you for reaching out! How can I assist you today? If you have any questions or issues regarding our products or services, feel free to let me know, and I'll do my best to help you.",
  "shouldRespond": true,
  "reason": ""
}

talkToAgentTool 从未被调用

我应该怎么做，才能在用户输入 agent、talk to human、talk to agent 等时让 openAI 调用 talkToAgentTool？

如果调用了 talkToAgentTool，我该如何停止/结束与AI的对话？

解决方案

核心问题在于 withStructuredOutput() 与LangChain的工具调用是互斥的。当你使用 withStructuredOutput() 时，模型的整段输出被绑定到你的Zod架构——它不会自行决定调用工具。你也把 tools 作为一个选项传给 withStructuredOutput()，但那里并不支持该参数。

解决方法：使用两次独立的LLM调用并加上一个路由步骤。

步骤1 — 调用工具的LLM（先执行）

将工具绑定到模型，并检查模型是否想要使用某个工具：

const modelWithTools = model.bindTools(tools);

const toolCheckResult = await modelWithTools.invoke(result0.answer);

if (toolCheckResult.tool_calls && toolCheckResult.tool_calls.length > 0) {
    for (const toolCall of toolCheckResult.tool_calls) {
        const toolToCall = tools.find(t => t.name === toolCall.name);
        if (toolToCall) {
            const toolResult = await toolToCall.invoke(toolCall.args);
            // Handle tool result — e.g. return early, set a flag, etc.
        }
    }
    return; // Stop here; don't proceed to structured output
}

步骤2 — 结构化输出的LLM（仅在未调用工具时执行）

const structuredLlm = model.withStructuredOutput(responseSchema, {
    method: "jsonSchema",
    strict: false,
});

const result = await structuredLlm.invoke(result0.answer);
return result.answer;

为什么 talkToAgentTool 从未触发

withStructuredOutput() 的工作原理是通过指示模型始终返回一个与您的架构匹配的JSON对象——它会覆盖模型自由选择工具的能力。模型把结构化输出指令视为比调用工具更受约束，因此只是填充您的架构后返回。

在调用工具后如何停止/结束对话

一旦调用了工具，就在处理程序中提前返回：

if (toolCheckResult.tool_calls?.length > 0) {
    // execute tool...
    return {
        shouldRespond: false,
        reason: "Transferred to agent",
        answer: "Connecting you with an agent now.",
    };
}

你也可以在会话/数据库中设置一个标志，让WhatsApp webhook在该对话中跳过未来的AI处理，直到人工代理关闭它。