Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "arms"
version = "0.1.0"
version = "0.0.1"
edition = "2024"

[dependencies]
Expand Down
37 changes: 26 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,30 +2,35 @@

[![Latest Release](https://img.shields.io/github/v/release/inftyai/amrs?include_prereleases)](https://github.com/inftyai/amrs/releases/latest)

The Adaptive Model Routing System (AMRS) is a framework designed to select the best-fit model for exploration and exploitation. (still under development)
The Adaptive Model Routing System (AMRS) is a framework designed to select the best-fit model for exploration and exploitation. Rust core with python bindings. Still under active development 🚧.

Thanks to [async-openai](https://github.com/64bit/async-openai), AMRS builds on top of it to provide adaptive model routing capabilities.
AMRS builds on top of [async-openai](https://github.com/64bit/async-openai) to provide API services for quick setup. Thanks to open source 💙.

## Features

- Flexible routing strategies, including:
- **Random**: Randomly selects a model from the available models.
- **Endpoints Support** (only basic ones because of limited resources):
- Chat Completions
- Responses
- More on the way

- **Flexible Routing Strategies**:
- **Random(default)**: Randomly selects a model from the available models.
- **WRR**: Weighted Round Robin selects models based on predefined weights.
- **UCB1**: Upper Confidence Bound based model selection (coming soon).
- **UCB1**: Upper Confidence Bound for balancing exploration and exploitation (coming soon).
- **Adaptive**: Dynamically selects models based on performance metrics (coming soon).

- Broad provider support:
- OpenAI compatible providers (DeepInfra, OpenRouter, etc.)
- **Various Providers Support**:
- OpenAI compatible providers (OpenAI, DeepInfra, etc.)
- More on the way

## How to use

Here's a simple example with the Weighted Round Robin (WRR) routing mode:
Here's a simple example with the Weighted Round Robin (WRR) routing mode. Before running the code, make sure to set your provider API key in the environment variable by running `export <PROVIDER>_API_KEY="your_openai_api_key"`.
Here we use OpenAI as an example.


```rust
// Before running the code, make sure to set your OpenAI API key in the environment variable:
// export OPENAI_API_KEY="your_openai_api_key"
# Make sure OPENAI_API_KEY is set in your environment variables before running this code.

use tokio::runtime::Runtime;
use arms::client;
Expand Down Expand Up @@ -55,12 +60,22 @@ let mut client = client::Client::new(config);
let request = chat::CreateChatCompletionRequestArgs::default()
.messages([
chat::ChatCompletionRequestSystemMessage::from("You are a helpful assistant.").into(),
chat::ChatCompletionRequestUserMessage::from("How is the weather today?").into(),
chat::ChatCompletionRequestUserMessage::from("Who won the FIFA World Cup in 2025?").into(),
])
.build()
.unwrap();

let result = Runtime::new().unwrap().block_on(client.create_completion(request));
match result {
Ok(response) => {
for choice in response.choices {
println!("Response: {:?}", choice.message.content);
}
}
Err(e) => {
eprintln!("Error: {}", e);
}
}
```

## Contributing
Expand Down
4 changes: 3 additions & 1 deletion src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,9 @@ fn main() {

match result {
Ok(response) => {
println!("Response: {:?}", response);
for choice in response.choices {
println!("Response: {:?}", choice.message.content);
}
}
Err(e) => {
eprintln!("Error: {}", e);
Expand Down
Loading