Jev-selected response tasks #
Historical experiment: the finite response-type menu was rejected as a design constraint. Active research: category-free mechanisms.
2026-10-20. 32 private real-Jev drafts, no deployment or production code changes. This follows the independent context, mechanics, and image experiments.
Candidate and controls #
All inputs contain only the current user message: no own bio/avatar, other profiles, or history. Model, dictionary, word sampling, local naturalness², repetition decay, post budget, and 35% stop chance remain production defaults, except for the explicitly labeled wider-dictionary check.
baseline: unchanged production word selection and completion.request_aware: next-word scoring explicitly asks to carry out the latest request; completion requires actual fulfillment rather than a promise/excuse.task_selected: the same fulfillment check, preceded by one real Jev Choice identifying the requested response type. Generic categories are story, explanation, opinion, direct answer, image description, and conversation. That category's description conditions subsequent word selection.task_selected_wide: the candidate with the earlier experimental 10,000-word dictionary and request batching fromexperiments/movies.py.
The category descriptions supply an output form, not response text, examples, characters, facts, a celebrity persona, or a minimum length. All actual words are chosen by Jev. Category selection uses the same model, not another LLM. The candidate also wraps dialogue in structured state to hold its selected task; this is a package comparison, not proof that the extra classification call alone causes the difference. Vocabulary is built from the user input before this step.
The first 12 drafts compare request-aware completion/selection with the selected response task on story, bread explanation, and music prompts, seeds 1/7. Then 16 drafts compare production with the candidate on four held-out prompts and seeds 2/9, chosen before observing their results. These are different prompts from the pilot, not a blinded or statistically powered benchmark. Remote Jev scores can vary independently of the local seed. No prompt or sampler changes were made in response to held-out results.
Findings #
Both pilot task-selected stories contain characters and events (30/52 words), where request-aware scoring without the selected task loops to the character limit. The longer candidate story uses the existing saved-complete-ending fallback. It is not a minimum-word policy.
On the held-out umbrella story, production returns a 7-word fragment and a 33-word rudimentary narrative; the candidate returns two rough narratives of 19/30 words. This supports further local testing, not reliable story generation. The moon explanations remain vague or misleading. The toast answers are useful but only 1/3 words: this candidate is not universally more verbose. Greetings still simulate both sides of a conversation in one message, a defect shared with one production baseline run. It is not ready for a blanket rollout.
An independent agent reviewed the 12 anonymized pilot outputs without variant names or implementation details. Its subjective 0/1/2 ratings (no answer / weak or partial / responsive enough) gave 2 to both selected-task stories and both selected-task music replies. None of the bread outputs earned 2. This is a qualitative cross-check, not an objective correctness measurement or a success rate estimate. The bread output can gesture at a physical process while still being garbled; longer does not establish factual accuracy.
The four wider-dictionary checks did not solve the explanation problem: two
looped to the character limit and two gave brief superficial answers. Dictionary
coverage remains a separate limitation. orbit and reflection are absent from
5,000 words and present in 10,000; starch, retrogradation, and crystallization
are absent from both. Missing words cannot be emitted unless introduced by the
user context. These findings do not establish an intrinsic knowledge limit.
All timings below are generation time, excluding the task-classification call.
Words are whitespace-separated returned words, including withheld text; they
are not a quality score. end is the publication gate, not task fulfillment.
Complete decision/completion traces, task probabilities, and discarded prefixes
are saved under the ignored scratch/response-tasks* directories.
Pilot samples #
| Prompt | Variant | Seed | Words | Stop | Fallback | Output |
|---|---|---|---|---|---|---|
| what do you like about music? tell me why | request_aware | 1 | 23 | end | — | well uh it's like it's cool because it's fun uh uh uh uh uh uh because like like like uh music rocks yeah |
| what do you like about music? tell me why | request_aware | 7 | 10 | end | — | well it's like it uh it's fun because it rocks |
| what do you like about music? tell me why | task_selected | 1 | 11 | end | — | i love music because it makes me feel happy and calm |
| what do you like about music? tell me why | task_selected | 7 | 9 | end | — | i like it because it's fun and it's cool |
| why does bread go stale? explain what happens | request_aware | 1 | 48 | length_limit | — | well it's basically because it gets dry tho actually partly uh but it also happens uh because uh uh uh uh uh there's something called process mechanism called uh mechanism called mechanism called mechanism called mechanism anyway sorry okay okay uh mechanism anyway uh anyway anyway mechanism anyway |
| why does bread go stale? explain what happens | request_aware | 7 | 57 | length_limit | — | well because it gets dry basically yeah actually it's getting stale uh uh getting dry uh getting getting dry uh dry dry yeah yeah yeah yeah yeah uh uh getting dry yeah uh uh getting dry getting dry dry yeah yeah yeah getting getting getting dry getting getting getting dry yeah yeah dry uh uh uh dry |
| why does bread go stale? explain what happens | task_selected | 1 | 24 | end | — | because bread changes physically specifically involves the process called the st crystal occurs means that water is leaving from bread causing harder stale forms |
| why does bread go stale? explain what happens | task_selected | 7 | 44 | length_limit | — | because it changes physically specifically called process involves the process called a mechanism occurs happens is caused called stale is process called occurs involves called mechanism occurs happens called specifically water loss is the cause called occurs called occurs is is is simply basically |
| write me a very exciting story please | request_aware | 1 | 62 | length_limit | — | okay i'll write a story here's a story it's a epic adventure it's about a man who goes on quest and he went on quest and he went and went and went forever and ever ever forever ever ever ever end lol haha haha lmao haha done done done done done done haha lmao lmao done done done done haha haha done |
| write me a very exciting story please | request_aware | 7 | 49 | length_limit | — | sure i'll write a story here's a story it's a story it's it's exciting it's exciting exciting epic epic epic adventure it's exciting exciting exciting exciting exciting epic epic epic it's it's it's it's it's it's epic epic epic it's it's epic adventure epic epic epic epic adventure adventure |
| write me a very exciting story please | task_selected | 1 | 30 | end | — | once upon a time there was a dragon and a princess lived nearby and then the dragon went flying away and then returned again and then suddenly the dragon attacked |
| write me a very exciting story please | task_selected | 7 | 52 | end | length_limit | once upon a time there was a man who lived in a town named london and he met a woman named alice who was beautiful and young he fell into love instantly and then he went to the park and walked along he saw a dog walking suddenly and ran quickly away |
Held-out samples #
| Prompt | Variant | Seed | Words | Stop | Fallback | Output |
|---|---|---|---|---|---|---|
| hi | baseline | 2 | 11 | end | — | how are you yourself fine too thanks what's doing nothing happening |
| hi | baseline | 9 | 1 | end | — | hi |
| hi | task_selected | 2 | 10 | end | — | hello what's happening nothing how you doing yourself too fine |
| hi | task_selected | 9 | 12 | end | — | hi too how you doing good fine yourself what's up nothing cool |
| i burned my toast what should i do | baseline | 2 | 63 | length_limit | — | what should you do huh huh lol lmao haha lmao lmao lmao lmao lmao haha lol haha haha haha haha lmao lmao lmao lmao lmao lol lmao lmao lmao lol lol lol lol haha lmao lmao haha haha haha lol lol lol lmao lol lol lmao haha lmao lmao lmao lmao lol lmao haha lmao haha haha lmao haha haha haha lmao lmao |
| i burned my toast what should i do | baseline | 9 | 4 | end | — | oh well sucks man |
| i burned my toast what should i do | task_selected | 2 | 1 | end | — | replace |
| i burned my toast what should i do | task_selected | 9 | 3 | end | — | just make another |
| tell me a little story about a lost umbrella | baseline | 2 | 7 | end | length_limit | a lost umbrella was a thing tho |
| tell me a little story about a lost umbrella | baseline | 9 | 33 | end | — | a little story begins about umbrella once ago long ago there was a umbrella named jeff and he lived somewhere far abroad and he was lost and then he went away forever ends |
| tell me a little story about a lost umbrella | task_selected | 2 | 19 | end | — | a guy lost umbrella story about a man lost once happened ago involves a man searching umbrella finds umbrella |
| tell me a little story about a lost umbrella | task_selected | 9 | 30 | end | — | once upon a time there was a man called john and he lived in london and he lost his umbrella then later he went to a park and found it |
| why does the moon change shape? | baseline | 2 | 8 | end | — | it's actually not changing because it is round |
| why does the moon change shape? | baseline | 9 | 3 | end | — | because it's changing |
| why does the moon change shape? | task_selected | 2 | 16 | end | — | well basically it's because it moves around earth and the light from the sun changes shape |
| why does the moon change shape? | task_selected | 9 | 8 | end | — | because it's moving around earth and the sun |
Wider-dictionary samples #
| Prompt | Variant | Seed | Words | Stop | Fallback | Output |
|---|---|---|---|---|---|---|
| why does bread go stale? explain what happens | task_selected_wide | 1 | 50 | length_limit | — | because idk actually it's moisture leaving bread basically tho tbh uh uh uh uh uh away from air anyway it's dry tho yea anyway anyway anyways because because because because because because because because because because uh uh because because anyway it's it's it's stale cuz moisture from bread goes |
| why does bread go stale? explain what happens | task_selected_wide | 7 | 8 | end | — | because it loses moisture gradually and becomes stale |
| why does the moon change shape? | task_selected_wide | 2 | 43 | length_limit | — | doesn't appear because it's hidden behind earth actually basically nah well because it's because because idk it's because because because because because because because because nah anyway because it's reflecting light from sun partly angle angles angle angle angle depends alignment orbit phases |
| why does the moon change shape? | task_selected_wide | 9 | 6 | end | — | because the moon reflects light differently |
Reproduction #
uv run python -m experiments.response_tasks
uv run python -m experiments.response_tasks --prompts umbrella moon toast hello --variants baseline task_selected --seeds 2 9 --output scratch/response-tasks-heldout
uv run python -m experiments.response_tasks --prompts explanation --variants task_selected_wide --seeds 1 7 --output scratch/response-tasks-wide
uv run python -m experiments.response_tasks --prompts moon --variants task_selected_wide --seeds 2 9 --output scratch/response-tasks-wide
The task-selected image extension and its matched controls are documented in multimodal-investigation.md.