# Writing a Model Client Plugin A model client plugin adapts Ein's internal request format to a specific LLM API. It implements two traits from `ein_plugin::model_client`: - **`ConstructableModelClientPlugin`** — provides `new(config_json)` called once per session with the plugin's config - **`ModelClientPlugin`** — the core interface: a required `complete` method and an optional `embed` ## The traits ```rust pub trait ConstructableModelClientPlugin: ModelClientPlugin { /// Called once per session with the resolved profile and model parameters, /// serialized as a JSON string. fn new(config_json: &str) -> Self; } pub trait ModelClientPlugin: Send + Sync { /// Receives a serialized CompletionRequest, returns a serialized /// CompletionResponse (or an error string). fn complete(&self, request_json: &str) -> anyhow::Result; /// Receives a serialized EmbeddingRequest, returns a serialized /// EmbeddingResponse. Defaulted to an error, so a plugin only implements /// it if its provider has an embeddings endpoint. fn embed(&self, _request_json: &str) -> anyhow::Result { /* ... */ } } ``` Unlike tool plugins, model client plugins **do** receive config at construction time. The server resolves a profile and model from `~/.ein/eind.json`, overlays their non-conflicting parameters, adds the profile's current secrets, and serializes that construction-only object as `config_json`. ## Key types These are re-exported from `ein_plugin::model_client`: ```rust use ein_plugin::model_client::{ CompletionRequest, // deserialized from request_json CompletionResponse, // serialized and returned from complete() EmbeddingRequest, // deserialized from embed's request_json EmbeddingResponse, // serialized and returned from embed() HttpRequest, // WASM HTTP client openai_embedding_response, // parses an OpenAI-shaped embeddings body }; ``` ### `CompletionRequest` ```rust pub struct CompletionRequest { pub model: String, pub messages: Vec, pub tools: Vec, pub max_tokens: i32, } ``` ### `CompletionResponse` ```rust pub struct CompletionResponse { pub choices: Vec, pub usage: Option, pub error: Option, } ``` If the upstream API returns an error, set `error` rather than returning `Err`. If the whole request fails (network error, auth failure), return `Err` — the server emits an `AgentError` event and preserves the session for retry. ## Embeddings `embed` turns text into vectors, which the server's memory system uses for semantic search over archival passages and past conversations. It is defaulted rather than required, because not every provider offers embeddings — the Anthropic API has none, so `ein_anthropic` leaves the default in place. A session whose model client cannot embed falls back to lexical search rather than failing. ```rust pub struct EmbeddingRequest { pub model: String, pub input: Vec, pub dimensions: Option, } pub struct EmbeddingResponse { pub embeddings: Vec>, pub usage: Option, } ``` `embeddings` must line up with `input` **by position**. If your provider speaks the OpenAI wire format, use `openai_embedding_response` rather than parsing the body yourself: it sorts the response's `data` array by its `index` field before flattening. That array carries an index because its order is not guaranteed, and a response read in arrival order pairs every vector with the wrong text — an error nothing downstream can detect, because each vector is well-formed whatever text produced it. The embedding model is named separately from chat profiles in the server's top-level `embedding` definition, so one plugin can serve both roles without making embedding a per-session choice. ### `HttpRequest` WASM plugins cannot use `reqwest` or `tokio` directly. Use `HttpRequest`, which is backed by `wasi:http/outgoing-handler`: ```rust let resp = HttpRequest::post("https://api.example.com/v1/chat/completions") .bearer_auth(&self.config.api_key) .json(&body)? .send()?; if resp.is_success() { // resp.body is a String let data: MyResponseType = serde_json::from_str(&resp.body)?; } ``` Available methods: `HttpRequest::get`, `post`, `put`, `patch`, `delete`. Builder API: | Method | Description | |--------|-------------| | `.header(key, value)` | Add an arbitrary HTTP header | | `.bearer_auth(token)` | Set `Authorization: Bearer ` | | `.content_type_json()` | Set `Content-Type: application/json` | | `.json(&value)` | Serialize value and set as body | | `.body(string)` | Set raw string body | | `.send()` | Dispatch the request | ## Network access The hostname derived from the resolved profile's `base_url` is automatically allowlisted by the server. Model client plugins do not use the tool plugin policy. A plugin that needs another connection, such as a token refresh endpoint, needs an explicit host policy in the model-client host before it can be used safely. ## Exporting the plugin ```rust #[cfg(target_arch = "wasm32")] ein_plugin::export_model_client!(MyPlugin); ``` --- ## Worked example: a minimal OpenAI-compatible adapter This example connects to any OpenAI-compatible API. Many open-source model servers (Ollama, vLLM, LM Studio, llama.cpp server) expose an OpenAI-compatible chat completions endpoint, so this pattern covers a lot of ground. ```rust use anyhow::anyhow; use ein_plugin::model_client::{ CompletionRequest, CompletionResponse, ConstructableModelClientPlugin, HttpRequest, ModelClientPlugin, }; use serde::Deserialize; #[derive(Deserialize)] struct MyConfig { api_key: String, #[serde(default = "default_base_url")] base_url: String, #[serde(default)] model: String, } fn default_base_url() -> String { "http://localhost:8080/v1".to_string() } pub struct MyModelClient { config: MyConfig, } impl ConstructableModelClientPlugin for MyModelClient { fn new(config_json: &str) -> Self { let config: MyConfig = serde_json::from_str(config_json) .expect("invalid plugin config JSON"); Self { config } } } impl ModelClientPlugin for MyModelClient { fn complete(&self, request_json: &str) -> anyhow::Result { let mut req: CompletionRequest = serde_json::from_str(request_json)?; // Override the model if the config specifies one. if !self.config.model.is_empty() { req.model = self.config.model.clone(); } let url = format!( "{}/chat/completions", self.config.base_url.trim_end_matches('/') ); let resp = HttpRequest::post(url) .bearer_auth(&self.config.api_key) .json(&req)? .send() .map_err(|e| anyhow!("request failed: {e}"))?; if !resp.is_success() { return Err(anyhow!( "API error (HTTP {}): {}", resp.status, resp.body )); } // Validate the response parses before returning. let _: CompletionResponse = serde_json::from_str(&resp.body) .map_err(|e| anyhow!("unexpected response: {e}\nbody: {}", resp.body))?; Ok(resp.body) } } #[cfg(target_arch = "wasm32")] ein_plugin::export_model_client!(MyModelClient); ``` ### Corresponding config ```json { "version": 1, "default_profile": "my-provider", "profiles": { "my-provider": { "display_name": "My provider", "plugin": "my_model_client", "params": { "base_url": "http://localhost:8080/v1" }, "secret_params": { "api_key": "your-key-or-dummy" }, "default_model": "default", "models": { "default": { "display_name": "My custom model", "model": "my-custom-model", "context_window": 32768 } } } } } ``` The filename stem of your `.wasm` file must match the profile's `plugin` value. If you name the file `my_model_client.wasm`, use `"plugin": "my_model_client"`. ## Adapting to non-OpenAI APIs If the upstream API uses a different request/response format, deserialize `CompletionRequest` and translate it into your API's format before sending, then translate the response back into `CompletionResponse` before returning. `CompletionRequest.messages` uses `Role::User`, `Role::Assistant`, `Role::System`, and `Role::Tool` (for tool results). `CompletionRequest.tools` carries `ToolDef` entries that you translate into your API's tool/function spec. The Anthropic and Ollama plugins in the Ein source tree are good references for non-OpenAI translation — see `plugins/ein_anthropic/` and `plugins/ein_ollama/`.