plugins/openai-compatible/package.json at admin-config-export · sharedcomputer.network/scn-chat · Tangled

ATProto-based AI Assistant Chat interface.

scn-chat plugins openai-compatible package.json

503 B · 20 lines

JSON

at admin-config-export

Blame

· A reply streams at an even pace, a word at a time, or a character at a time for Chinese and Japanese, where before it arrived in whatever bursts the provider sent. The turn's streamText uses the AI SDK's smoothStream with a chunking pattern of our own, since the SDK's word chunking waits for a space and would hold back Chinese and Japanese until a line ended. It waits 10 ms after each piece of text and reasoning alike, which caps a reply at about 100 words a second, so a faster model's record is written once smoothing catches up, within the turn's timeout. OpenAI-compatible models now show reasoning a server leaves in the reply inside <think> tags, such as llama.cpp with --reasoning-format none, as reasoning rather than as part of the answer, through the AI SDK's extractReasoningMiddleware, so that plugin now depends on the ai package. Its new startWithReasoning option is for models such as DeepSeek R1 whose template opens the tag in the prompt, so a reply has only the closing tag. The titles plugin refuses output that stopped at its output limit, such as reasoning that never reached its closing tag. The chat turns, providers, and titles specs describe it, along with the reasoning middleware dropping a reply's last characters when they could start a tag, which is a bug in the AI SDK, and docs/plugins.md says when a provider needs the ai package. Tests cover the smoothing, reasoning in think tags with and without the opening tag, whether streamed or not, and the cut-off title. · 2w ago