Using prompto with Anthropic¶
from prompto.settings import Settings
from prompto.experiment import Experiment
from dotenv import load_dotenv
import os
When using prompto to query models from the Anthropic API, lines in our experiment .jsonl files must have "api": "anthropic" in the prompt dict.
Environment variables¶
For the Anthropic API, there are two environment variables that could be set:
ANTHROPIC_API_KEY: the API key for the Anthropic API
As mentioned in the environment variables docs, there are also model-specific environment variables too which can be utilised. In particular, when you specify a model_name key in a prompt dict, one could also specify a ANTHROPIC_API_KEY_model_name environment variable to indicate the API key used for that particular model (where "model_name" is replaced to whatever the corresponding value of the model_name key is). We will see a concrete example of this later.
To set environment variables, one can simply have these in a .env file which specifies these environment variables as key-value pairs:
ANTHROPIC_API_KEY=<YOUR-ANTHROPIC-KEY>
If you make this file, you can run the following which should return True if it's found one, or False otherwise:
load_dotenv(dotenv_path=".env")
True
Now, we obtain those values. We raise an error if the ANTHROPIC_API_KEY environment variable hasn't been set:
ANTHROPIC_API_KEY = os.environ.get("ANTHROPIC_API_KEY")
if ANTHROPIC_API_KEY is None:
raise ValueError("ANTHROPIC_API_KEY is not set")
If you get any errors or warnings in the above two cells, try to fix your .env file like the example we have above to get these variables set.
Types of prompts¶
With the Anthropic API, the prompt (given via the "prompt" key in the prompt dict) can take several forms:
- a string: a single prompt to obtain a response for
- a list of strings: a sequence of prompts to send to the model
- this is useful in the use case of simulating a conversation with the model by defining the user prompts sequentially
- a list of dictionaries with keys "role" and "content", where "role" is one of "user", "assistant", or "system" and "content" is the message
- this is useful in the case of passing in some conversation history or to pass in a system prompt to the model
We have created an input file in data/input/anthropic-example.jsonl with an example of each of these cases as an illustration.
settings = Settings(data_folder="./data", max_queries=30)
experiment = Experiment(file_name="anthropic-example.jsonl", settings=settings)
We set max_queries to 30 so we send 30 queries a minute (every 2 seconds).
print(settings)
Settings: data_folder=./data, max_queries=30, max_attempts=3, parallel=False Subfolders: input_folder=./data/input, output_folder=./data/output, media_folder=./data/media
len(experiment.experiment_prompts)
5
experiment.experiment_prompts[0]
{'id': 0,
'api': 'anthropic',
'model_name': 'claude-3-haiku-20240307',
'prompt': 'How does technology impact us?',
'parameters': {'temperature': 1, 'max_tokens': 100}}
We can see the prompts that we have in the experiment_prompts attribute:
experiment.experiment_prompts
[{'id': 0,
'api': 'anthropic',
'model_name': 'claude-3-haiku-20240307',
'prompt': 'How does technology impact us?',
'parameters': {'temperature': 1, 'max_tokens': 100}},
{'id': 1,
'api': 'anthropic',
'model_name': 'claude-3-5-sonnet-20240620',
'prompt': 'How does technology impact us?',
'parameters': {'temperature': 1, 'max_tokens': 100}},
{'id': 2,
'api': 'anthropic',
'model_name': 'claude-3-haiku-20240307',
'prompt': ['How does international trade create jobs?',
'I want a joke about that'],
'parameters': {'temperature': 1, 'max_tokens': 100}},
{'id': 3,
'api': 'anthropic',
'model_name': 'claude-3-haiku-20240307',
'prompt': [{'role': 'system',
'content': 'You are a helpful assistant designed to answer questions briefly.'},
{'role': 'user',
'content': 'What efforts are being made to keep the hakka language alive?'}],
'parameters': {'temperature': 1, 'max_tokens': 100}},
{'id': 4,
'api': 'anthropic',
'model_name': 'claude-3-haiku-20240307',
'prompt': [{'role': 'system',
'content': 'You are a helpful assistant designed to answer questions briefly.'},
{'role': 'user', 'content': "Hello, I'm Bob and I'm 6 years old"},
{'role': 'assistant', 'content': 'Hi Bob, how may I assist you?'},
{'role': 'user', 'content': 'How old will I be next year?'}],
'parameters': {'temperature': 1, 'max_tokens': 100}}]
- In the first prompt (
"id": 0), we have a"prompt"key which is a string and specify a"model_name"key to be"claude-3-haiku-20240307". - In the second prompt (
"id": 1), we have a"prompt"key is also a string but we specify a"model_name"key to be"claude-3-5-sonnet-20240620". - In the third prompt (
"id": 2), we have a"prompt"key which is a list of strings. - In the fourth prompt (
"id": 3), we have a"prompt"key which is a list of dictionaries. These dictionaries have a "role" and "content" key. This acts as passing in a system prompt. Here, we just have a system prompt before a user prompt. - In the fifth prompt (
"id": 4), we have a"prompt"key which is a list of dictionaries. These dictionaries have a "role" and "content" key. Here, we have a system prompt and a series of user/assistant interactions before finally having a user prompt. This acts as passing in a system prompt and conversation history.
Note that for each of these prompt dicts, we have "model_name": "claude-3-haiku-20240307", besides "id": 1 where we have "model_name": "claude-3-5-sonnet-20240620".
Running the experiment¶
We now can run the experiment using the async method process which will process the prompts in the input file asynchronously. Note that a new folder named timestamp-anthropic-example (where "timestamp" is replaced with the actual date and time of processing) will be created in the output directory and we will move the input file to the output directory. As the responses come in, they will be written to the output file and there are logs that will be printed to the console as well as being written to a log file in the output directory.
responses, avg_query_processing_time = await experiment.process()
Sending 5 queries at 30 QPM with RI of 2.0s (attempt 1/3): 100%|██████████| 5/5 [00:10<00:00, 2.00s/query] Waiting for responses (attempt 1/3): 100%|██████████| 5/5 [00:01<00:00, 4.10query/s]
We can see that the responses are written to the output file, and we can also see them as the returned object. From running the experiment, we obtain prompt dicts where there is now a "response" key which contains the response(s) from the model.
For the case where the prompt is a list of strings, we see that the response is a list of strings where each string is the response to the corresponding prompt.
responses
[{'id': 0,
'api': 'anthropic',
'model_name': 'claude-3-haiku-20240307',
'prompt': 'How does technology impact us?',
'parameters': {'temperature': 1, 'max_tokens': 100},
'timestamp_sent': '16-07-2024-11-24-26',
'response': "Technology has had a profound impact on our lives in many ways. Here are some of the key ways technology affects us:\n\n1. Communication and connectivity: Technology has greatly enhanced our ability to communicate and stay connected with others, whether it's through email, social media, video calls, messaging apps, etc. This has transformed how we interact and share information.\n\n2. Access to information: The internet and digital technologies give us unprecedented access to information and knowledge on virtually any topic. This has expande"},
{'id': 1,
'api': 'anthropic',
'model_name': 'claude-3-5-sonnet-20240620',
'prompt': 'How does technology impact us?',
'parameters': {'temperature': 1, 'max_tokens': 100},
'timestamp_sent': '16-07-2024-11-24-28',
'response': 'Technology has a profound and multifaceted impact on our lives, affecting various aspects of society, culture, and individual experiences. Here are some key ways technology impacts us:\n\n1. Communication:\n- Instant global connectivity through smartphones and the internet\n- Social media platforms for sharing information and maintaining relationships\n- Video conferencing for remote work and long-distance communication\n\n2. Information Access:\n- Easy access to vast amounts of information through search engines\n- Online educational resources and e'},
{'id': 2,
'api': 'anthropic',
'model_name': 'claude-3-haiku-20240307',
'prompt': ['How does international trade create jobs?',
'I want a joke about that'],
'parameters': {'temperature': 1, 'max_tokens': 100},
'timestamp_sent': '16-07-2024-11-24-30',
'response': ['International trade can create jobs in a few key ways:\n\n1. Exports - When a country exports goods and services, it creates jobs to produce those exports. The more a country can sell abroad, the more jobs are needed domestically to meet that demand.\n\n2. Foreign Investment - When foreign companies invest in a country and establish operations there, they create jobs in the local economy. This brings new employment opportunities.\n\n3. Specialization - Trade allows countries to specialize in producing',
'Okay, here\'s a lighthearted joke about how international trade creates jobs:\n\nA businessman, a factory worker, and an economist were debating how trade impacts employment. The businessman said, "Exporting our products abroad creates so many jobs for us!" \n\nThe factory worker replied, "But all those imported goods are taking our jobs!"\n\nThe economist just chuckled and said, "International trade is a net job creator, you fools. Now let me explain']},
{'id': 3,
'api': 'anthropic',
'model_name': 'claude-3-haiku-20240307',
'prompt': [{'role': 'system',
'content': 'You are a helpful assistant designed to answer questions briefly.'},
{'role': 'user',
'content': 'What efforts are being made to keep the hakka language alive?'}],
'parameters': {'temperature': 1, 'max_tokens': 100},
'timestamp_sent': '16-07-2024-11-24-32',
'response': 'Here are some key efforts to help keep the Hakka language alive:\n\n- Preservation of Hakka dialects and cultural heritage - Organizations like the Hakka Cultural Association work to document and preserve Hakka dialects, literature, music, and other cultural traditions.\n\n- Hakka language education - Some schools and universities in Hakka communities offer Hakka language classes and programs to teach the language to younger generations.\n\n- Promotion of Hakka media an'},
{'id': 4,
'api': 'anthropic',
'model_name': 'claude-3-haiku-20240307',
'prompt': [{'role': 'system',
'content': 'You are a helpful assistant designed to answer questions briefly.'},
{'role': 'user', 'content': "Hello, I'm Bob and I'm 6 years old"},
{'role': 'assistant', 'content': 'Hi Bob, how may I assist you?'},
{'role': 'user', 'content': 'How old will I be next year?'}],
'parameters': {'temperature': 1, 'max_tokens': 100},
'timestamp_sent': '16-07-2024-11-24-34',
'response': "Okay, let's figure this out:\n* You are currently 6 years old\n* Next year, you will be 1 year older\n* So next year, you will be 7 years old."}]
Running the experiment via the command line¶
We can also run the experiment via the command line. The command is as follows (assuming that your working directory is the current directory of this notebook, i.e. examples/anthropic):
prompto_run_experiment --file data/input/anthropic-example.jsonl --max-queries 30