Writing a Model Client Plugin #

A model client plugin adapts Ein's internal request format to a specific LLM API. It implements two traits from ein_plugin::model_client:

• ConstructableModelClientPlugin — provides new(config_json) called once per session with the plugin's config
• ModelClientPlugin — the core interface: a required complete method and an optional embed

The traits #

pub trait ConstructableModelClientPlugin: ModelClientPlugin {
    /// Called once per session with the resolved profile and model parameters,
    /// serialized as a JSON string.
    fn new(config_json: &str) -> Self;
}

pub trait ModelClientPlugin: Send + Sync {
    /// Receives a serialized CompletionRequest, returns a serialized
    /// CompletionResponse (or an error string).
    fn complete(&self, request_json: &str) -> anyhow::Result<String>;

    /// Receives a serialized EmbeddingRequest, returns a serialized
    /// EmbeddingResponse. Defaulted to an error, so a plugin only implements
    /// it if its provider has an embeddings endpoint.
    fn embed(&self, _request_json: &str) -> anyhow::Result<String> { /* ... */ }
}

Unlike tool plugins, model client plugins do receive config at construction time. The server resolves a profile and model from ~/.ein/eind.json, overlays their non-conflicting parameters, adds the profile's current secrets, and serializes that construction-only object as config_json.

Key types #

These are re-exported from ein_plugin::model_client:

use ein_plugin::model_client::{
    CompletionRequest,          // deserialized from request_json
    CompletionResponse,         // serialized and returned from complete()
    EmbeddingRequest,           // deserialized from embed's request_json
    EmbeddingResponse,          // serialized and returned from embed()
    HttpRequest,                // WASM HTTP client
    openai_embedding_response,  // parses an OpenAI-shaped embeddings body
};

CompletionRequest #

pub struct CompletionRequest {
    pub model: String,
    pub messages: Vec<Message>,
    pub tools: Vec<ToolDef>,
    pub max_tokens: i32,
}

CompletionResponse #

pub struct CompletionResponse {
    pub choices: Vec<Choice>,
    pub usage: Option<Usage>,
    pub error: Option<serde_json::Value>,
}

If the upstream API returns an error, set error rather than returning Err. If the whole request fails (network error, auth failure), return Err — the server emits an AgentError event and preserves the session for retry.

Embeddings #

embed turns text into vectors, which the server's memory system uses for semantic search over archival passages and past conversations. It is defaulted rather than required, because not every provider offers embeddings — the Anthropic API has none, so ein_anthropic leaves the default in place. A session whose model client cannot embed falls back to lexical search rather than failing.

pub struct EmbeddingRequest {
    pub model: String,
    pub input: Vec<String>,
    pub dimensions: Option<u32>,
}

pub struct EmbeddingResponse {
    pub embeddings: Vec<Vec<f32>>,
    pub usage: Option<Usage>,
}

embeddings must line up with input by position. If your provider speaks the OpenAI wire format, use openai_embedding_response rather than parsing the body yourself: it sorts the response's data array by its index field before flattening. That array carries an index because its order is not guaranteed, and a response read in arrival order pairs every vector with the wrong text — an error nothing downstream can detect, because each vector is well-formed whatever text produced it.

The embedding model is named separately from chat profiles in the server's top-level embedding definition, so one plugin can serve both roles without making embedding a per-session choice.

HttpRequest #

WASM plugins cannot use reqwest or tokio directly. Use HttpRequest, which is backed by wasi:http/outgoing-handler:

let resp = HttpRequest::post("https://api.example.com/v1/chat/completions")
    .bearer_auth(&self.config.api_key)
    .json(&body)?
    .send()?;

if resp.is_success() {
    // resp.body is a String
    let data: MyResponseType = serde_json::from_str(&resp.body)?;
}

Available methods: HttpRequest::get, post, put, patch, delete.

Builder API:

Method	Description
.header(key, value)	Add an arbitrary HTTP header
.bearer_auth(token)	Set Authorization: Bearer <token>
.content_type_json()	Set Content-Type: application/json
.json(&value)	Serialize value and set as body
.body(string)	Set raw string body
.send()	Dispatch the request

Network access #

The hostname derived from the resolved profile's base_url is automatically allowlisted by the server. Model client plugins do not use the tool plugin policy. A plugin that needs another connection, such as a token refresh endpoint, needs an explicit host policy in the model-client host before it can be used safely.

Exporting the plugin #

#[cfg(target_arch = "wasm32")]
ein_plugin::export_model_client!(MyPlugin);

---

Worked example: a minimal OpenAI-compatible adapter #

This example connects to any OpenAI-compatible API. Many open-source model servers (Ollama, vLLM, LM Studio, llama.cpp server) expose an OpenAI-compatible chat completions endpoint, so this pattern covers a lot of ground.

use anyhow::anyhow;
use ein_plugin::model_client::{
    CompletionRequest, CompletionResponse, ConstructableModelClientPlugin,
    HttpRequest, ModelClientPlugin,
};
use serde::Deserialize;

#[derive(Deserialize)]
struct MyConfig {
    api_key: String,
    #[serde(default = "default_base_url")]
    base_url: String,
    #[serde(default)]
    model: String,
}

fn default_base_url() -> String {
    "http://localhost:8080/v1".to_string()
}

pub struct MyModelClient {
    config: MyConfig,
}

impl ConstructableModelClientPlugin for MyModelClient {
    fn new(config_json: &str) -> Self {
        let config: MyConfig = serde_json::from_str(config_json)
            .expect("invalid plugin config JSON");
        Self { config }
    }
}

impl ModelClientPlugin for MyModelClient {
    fn complete(&self, request_json: &str) -> anyhow::Result<String> {
        let mut req: CompletionRequest = serde_json::from_str(request_json)?;

        // Override the model if the config specifies one.
        if !self.config.model.is_empty() {
            req.model = self.config.model.clone();
        }

        let url = format!(
            "{}/chat/completions",
            self.config.base_url.trim_end_matches('/')
        );

        let resp = HttpRequest::post(url)
            .bearer_auth(&self.config.api_key)
            .json(&req)?
            .send()
            .map_err(|e| anyhow!("request failed: {e}"))?;

        if !resp.is_success() {
            return Err(anyhow!(
                "API error (HTTP {}): {}",
                resp.status,
                resp.body
            ));
        }

        // Validate the response parses before returning.
        let _: CompletionResponse = serde_json::from_str(&resp.body)
            .map_err(|e| anyhow!("unexpected response: {e}\nbody: {}", resp.body))?;

        Ok(resp.body)
    }
}

#[cfg(target_arch = "wasm32")]
ein_plugin::export_model_client!(MyModelClient);

Corresponding config #

{
  "version": 1,
  "default_profile": "my-provider",
  "profiles": {
    "my-provider": {
      "display_name": "My provider",
      "plugin": "my_model_client",
      "params": {
        "base_url": "http://localhost:8080/v1"
      },
      "secret_params": {
        "api_key": "your-key-or-dummy"
      },
      "default_model": "default",
      "models": {
        "default": {
          "display_name": "My custom model",
          "model": "my-custom-model",
          "context_window": 32768
        }
      }
    }
  }
}

The filename stem of your .wasm file must match the profile's plugin value. If you name the file my_model_client.wasm, use "plugin": "my_model_client".

Adapting to non-OpenAI APIs #

If the upstream API uses a different request/response format, deserialize CompletionRequest and translate it into your API's format before sending, then translate the response back into CompletionResponse before returning.

CompletionRequest.messages uses Role::User, Role::Assistant, Role::System, and Role::Tool (for tool results). CompletionRequest.tools carries ToolDef entries that you translate into your API's tool/function spec.

The Anthropic and Ollama plugins in the Ein source tree are good references for non-OpenAI translation — see plugins/ein_anthropic/ and plugins/ein_ollama/.