Case Study 3

Correct trajectory is very important and this is not just in overall app flow but also in tool calling where tool calls are dependent on one another.

Tool calling

The module tool_calling.py has 4 tools and they form pairs to test multiple tool calling in a both parallel and sequential modes.

I have tried to muddy the water regarding weather and temperature so that it can stress tool choice.

Independent tools

Parallel or independent tool calling example:

TOOL 1: - get weather (fake api) gets weather for a city.

TOOL 2: - Checks for seating availablity

These two answer the question: 'What is the temperature in Munich and are there seats indoors/outdoors available'

Dependent tools

Sequential or dependent tool calling example:

TOOL 3: - Converts Centigrade to Fahrenheit

TOOL 4: - For a given temperature in F, gives a lable of COLD, MILD, WARM or HOT.

Tools 3 and 4 are sequential in answering the question 'What is 12C in Fahrenheit and give me the label/description of that temperature'.

Of course, this could be just one tool but this is a demo of sequential tool calling and checking correct tools and arguments are used as well as giving the correct outputs.

More tools?

I had extended this by adding a order_food tool that took the label of the weather and then returned a food type based on this.

It worked depending on how the question was asked and other questions added in. This is to be expected as it is a very simple example and does not implement the ReAct or Reflection pattern.

ReAct and Reflection

03_react_agent_eval.py implements the Langchain ReAct template and gives more predictable answers.

I added <debug></debug> to give more data and these were added in the functions rather than around the LLM call, with the temp_desc be the result of the describe_fahrenheit_with_label(temperature: float) tool:

@tool
def order_food(temp_desc: str) -> str:
    """The food to order for a given temperature description. Use this tool if the user wants to order some food or to pick a type of food needed."""
    console.print(f"[cyan bold]<debug>temp_desc: {temp_desc}[/]")
    global DEBUG
    DEBUG += f"<debug>temp_desc: {temp_desc}</debug>"
    if "COLD" in temp_desc:
        food = "~HOT_FOOD"
    elif "MILD" in temp_desc:
        food = "~WARM_FOOD"
    elif "WARM" in temp_desc:
        food = "~COOL_FOOD"
    elif "HOT" in temp_desc:
        food = "~COLD_FOOD"
    else:
        food = "~NONE"

    console.print(f"[cyan bold]<debug>TOOL ORDER_FOOD: {food}[/]")
    return food