Cohere

This notebook covers how to get started with Cohere chat models.

Head to the API reference for detailed documentation of all attributes and methods.

Setup

The integration lives in the langchain-community package. We also need to install the cohere package itself. We can install these with:

pip install -U langchain-community langchain-cohere

We’ll also need to get a Cohere API key and set the COHERE_API_KEY environment variable:

import getpass
import os

os.environ["COHERE_API_KEY"] = getpass.getpass()

It’s also helpful (but not needed) to set up LangSmith for best-in-class observability

# os.environ["LANGCHAIN_TRACING_V2"] = "true"
# os.environ["LANGCHAIN_API_KEY"] = getpass.getpass()

Usage

ChatCohere supports all ChatModel functionality:

from langchain_cohere import ChatCohere
from langchain_core.messages import HumanMessage

chat = ChatCohere(model="command", max_tokens=256, temperature=0.75)

messages = [HumanMessage(content="1"), HumanMessage(content="2 3")]
chat.invoke(messages)

AIMessage(content="4! That's one, two, three, four. Keep adding and we'll reach new heights!", response_metadata={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'token_count': {'prompt_tokens': 73, 'response_tokens': 21, 'total_tokens': 94, 'billed_tokens': 25}})

await chat.ainvoke(messages)

AIMessage(content='4! According to the rules of addition, 1 + 2 equals 3, and 3 + 3 equals 6.', response_metadata={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'token_count': {'prompt_tokens': 73, 'response_tokens': 28, 'total_tokens': 101, 'billed_tokens': 32}})

for chunk in chat.stream(messages):
    print(chunk.content, end="", flush=True)

4! It's a pleasure to be of service in this mathematical game.

chat.batch([messages])

[AIMessage(content='4! According to the rules of addition, 1 + 2 equals 3, and 3 + 3 equals 6.', response_metadata={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'token_count': {'prompt_tokens': 73, 'response_tokens': 28, 'total_tokens': 101, 'billed_tokens': 32}})]

Chaining

You can also easily combine with a prompt template for easy structuring of user input. We can do this using LCEL

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_template("Tell me a joke about {topic}")
chain = prompt | chat

chain.invoke({"topic": "bears"})

AIMessage(content='What do you call a bear with no teeth? A gummy bear!', response_metadata={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'token_count': {'prompt_tokens': 72, 'response_tokens': 14, 'total_tokens': 86, 'billed_tokens': 20}})