Skip to content
prompto: An open source library for asynchronous querying of LLM endpoints
huggingface_tgi
Initializing search
GitHub
prompto: An open source library for asynchronous querying of LLM endpoints
GitHub
Home
Documentation Guide
Examples
Examples
Examples
Running experiments with prompto
Grouping prompts and specifying rate limits
System Demonstration
System Demonstration
Introduction
Generating the prompts for experiments
Querying different LLM endpoints: prompto vs. synchronous Python for loop
Querying different LLM endpoints: prompto with parallel processing vs. synchronous Python for loop
Querying different models from the same endpoint: prompto vs. synchronous Python for loop
Using prompto for evaluation
Using prompto for evaluation
Running LLM as judge experiment with
Notebook
Rephrasing prompts using
Azure OpenAI
Azure OpenAI
Example
Using prompto with Azure OpenAI
Using prompto for multimodal prompting with Azure OpenAI
OpenAI
OpenAI
Example
Using prompto with OpenAI
Using prompto for multimodal prompting with OpenAI
Anthropic
Anthropic
Example
Using prompto with Anthropic
Using prompto for multimodal prompting with Anthropic
Gemini
Gemini
Example
Using prompto with Gemini
Using prompto for multimodal prompting with Gemini
Vertex AI
Vertex AI
Example
Using prompto with Vertex AI
Using prompto for multimodal prompting with Vertex AI
Ollama
Ollama
Example
Using prompto with Ollama
Using prompto
Using prompto
Setting up an experiment file
Configuring environment variables
Running experiments and the pipeline
prompto commands
Specifying rate limits
Rephrasing prompts
Using prompto for evaluation
Implemented APIs
Implemented APIs
APIs overview
Azure OpenAI
OpenAI
Anthropic
Gemini
Vertex AI
Ollama
Huggingface text-generation-inference
Simple quart API
Contributing to prompto
Contributing to prompto
Adding new API/model
Contribution guide
API Reference
API Reference
src
src
prompto
prompto
apis
apis
anthropic
anthropic
anthropic
anthropic_utils
azure_openai
azure_openai
azure_openai
base
gemini
gemini
gemini
gemini_media
gemini_utils
huggingface_tgi
huggingface_tgi
huggingface_tgi
ollama
ollama
ollama
ollama_utils
openai
openai
openai
openai_utils
quart
quart
quart
quart_api
quart_utils
testing
testing
testing_api
vertexai
vertexai
vertexai
vertexai_utils
experiment
experiment_pipeline
judge
rephrasal
rephrasal_parser
scorer
scripts
scripts
check_experiment
convert_images
create_judge_file
obtain_missing_id_jsonl
run_experiment
run_pipeline
settings
upload_media
utils
tests
tests
apis
apis
anthropic
gemini
ollama
vertexai
core
scripts
Issues
About
License
Table of contents
huggingface_tgi
huggingface_tgi