Humaneness Voice 1.1 Alpha

New: reinforcement learning, explained simply

The purple NEW RL players are the two new models. Each is the final Alpha model plus a rank-128 LoRA adapter, not a new model trained from scratch and not a full-parameter fine-tune. The semantic and local Talker attention adapters can change how speech is performed; the base weights, codec and judges remain frozen. The original eight comparisons are retained.

How to read the listening comparison

Compare purple RL against the amber matched control first. They have identical words, directions, seed, reference/name inputs, FP32 arithmetic with TF32 disabled, codec sampling temperature 0.7, top-p 0.95 and top-k 50. The control uses the same frozen base with its adapter disabled. The green original Alpha players are unchanged BF16/temperature-1.0 generations, so comparing only purple versus green mixes a training effect with inference-setting differences. Seeds are matched; different distributions need not produce identical token choices.

All 207 cases and all generated takes are included: there is no best-of-N selection, rejection of bad evaluation clips or reward-based cherry-picking. Learned-name rows remain useful retention tests even though this RL pilot trained reference/no-reference conditions rather than names. Identity scores on name-only rows compare against an audition recording that was not supplied to the model. No-identity rows have no legitimate speaker target and therefore show N/A. Ten assistant prompts contain no performance direction; their VoiceCLAP instruction score is N/A, not zero or a comparison with spoken words.

Higher CE and cosine scores are preferable; lower WER is preferable. VoiceCLAP reference similarity is an additional audio/audio style diagnostic, not the actual style reward. All twelve columns are scored from their 128-kbps mono listening MP3s with the same judges. These are estimated acoustic/performance measures, not a listener study. The same judges were used in RL, so improved judge scores alone are not independent proof of perceived quality. Use the audio players and WER to check for trade-offs.

Why are some similarities negative or missing?

Cosine similarities range from −1 to +1; they are not percentages or probabilities. A speaker cosine of 0.8 does not mean an 80% probability of being the same person. A negative VoiceCLAP instruction score can occur when a generated performance is poorly aligned with this encoder's text representation; it is not a negative emotion intensity. Judge scales depend on their inputs, so compare matched rows and conditions rather than universal thresholds. N/A means no legitimate target exists, not that the model earned zero. The paired table reports changes on the same cases; standard deviation describes variation, not a confidence interval or proof of significance.

Matched RL effect: adapter minus unchanged control

Each difference uses the same case, not unrelated training batches. Mean ± population standard deviation of per-case differences. Positive CE/identity/instruction deltas favor RL; negative WER percentage-point deltas favor RL. Counts differ where no legitimate target exists.

What these results currently show

• Small, reference supplied (121 takes): CE 6.03 → 6.41; word-weighted WER 5.86% → 2.13%; speaker identity 0.411 → 0.414.
• Large, reference supplied (121 takes): CE 5.99 → 6.34; word-weighted WER 4.81% → 2.68%; speaker identity 0.416 → 0.414.

The clearest measured gains are Content Enjoyment and transcript fidelity. Reference-conditioned speaker identity changes are small and mixed, so this pilot does not establish an improved voice-cloning capability. VoiceCLAP instruction match falls in 5 of the six model/conditioning averages; stronger CE is not the same as stronger emotion/style instruction following. Name-only Small WER also worsens slightly. These are real trade-offs, not omitted failures.

Try the RL players if acoustic enjoyment and clear wording are your priority, and compare the requested emotion/style carefully against the matched controls. Do not replace the Alpha base solely because one reward rose. The original model defaults have not been changed. The figures above summarize this demo, not a universal ranking across unseen speakers or languages.

New: SFT6 Small and Large — matched listening comparison

828 / 828 SFT6 predictions available. Green FINAL SFT6 players are Small step 1,372 and Large step 1,661. A separate development-selected column may be an earlier checkpoint; final does not mean best.

Every new player is next to the existing predictions for the same text, performance directions, seed and reference. Choose SFT6 + matched controls in the model filter. SFT6 and amber matched SFT5 controls use FP32/temperature 0.7. Archived green Alpha columns use BF16/temperature 1.0, so those settings differ. References and voice-name-only conditions stay separate.

Open the complete SFT4/SFT5/RL/SFT6 comparison and checkpoint-selection results. Historical RL columns retain their RL-v1 recipe. The new Alpha standard RL-v2 columns below use separate calibrated emotion/style and genuineness rewards.

New: final Alpha standard RL-v2 — 414 / 414 predictions

Small:2,616 attemptedG8 groups,2,256optimizer updates. Large:2,530groups,2,344updates. These are completed fresh rank128 adapters on the Alpha/SFT5 final parent, not the SFT6-based runs. Every take uses the same literal207 prompts, seeds and5–15second reference crops as the matched controls; no best-of-N or cherry-picking. Choose Alpha RL-v2 + matched controls to hear the new adapters directly beside their FP32/temperature0.7 parents. Archived BF16/temperature1.0 Alpha takes use different sampling settings.

Reward with reference:20%Content Enjoyment,20%genuineness,25%source-target emotion matching,25%source-target speaking-style matching,10%Orange identity. Without reference the identity score is absent and the other weights are renormalized. Fixed source-audio intensity targets avoid copying the reference voice’s emotions. Each component is group-normalized before VESPO/GDPO updates. The base and judges stay frozen. Rejected pairs receive no optimizer update. The completed Alpha RL-v2 reward does not contain the newer10%inverse-WER revision used by the repaired SFT6 Large production runs.

All six new RL runs — 1242 / 1242 matched predictions

Choose All RL + matched parents in the model filter to hear every run. In addition to the two Alpha standardRL-v2 adapters, the four new SFT6 columns use the actual final production adapters. Small standard completed2,616 attemptedG8groups /2,324 optimizer updates; Small emotion-focused2,616 /2,384. Large standard2,530 /2,360; Large emotion-focused2,530 /2,378. None of these players uses a failed-smoke adapter. Compare each SFT6 RL column against its green FINAL SFT6 parent:Small1372 or Large1661. HistoricalRL-v1 players are retained and labelled separately.

Small standard reward is20/20/25/25/10 for CE/genuineness/emotion/style/identity. Small emotion-focused is20/0/40/30/10, with the last10%identity being the documented interpretation of the original partial90%recipe. Without reference identity is omitted and weights renormalized. The repaired Large runs add exactly10%inverse-WER reward,1/(1+WER), keeping content gates unchanged:standard reference18/18/22.5/22.5/9/10; emotion-focused18/0/36/27/9/10. For Large no-reference, WER stays10% and the other90% is renormalized without identity. Large uses lower peakLR3.75e-6, Small1e-5, both5%warmup/cosine. All are fresh rank128 adapters with group size8 on frozen bases.

All six production trainings are finished. Pending predictions indicate only evaluation/backfill progress. Every arm uses identical literal text, performance prompt, seed, reference crop and evaluation judges. No best-of-N or seed selection. This remains a developer comparison on known prompts, not an independent held-out benchmark.

Listen immediately: saved training rollouts from all six runs

The following48-player preview uses existing intermediate-policy training generations, not final-checkpoint test predictions. It is explicitly separated from the matched207-case comparison and does not declare a winner. Four shared safe real-speech texts, with/without reference, fixed candidate0. Open the all-six-run preview on its own page.

Reference supplied

Learned speaker name only

Neither reference nor name

Publication status

• Small: COMPLETE; step 6,109/6,109; validation loss 4.204355222838266
• Large: COMPLETE; step 3,055/3,055; validation loss 4.164423295429775

2484 / 2484 predictions published. All predictions and scores are available.

Small weights and code · Large weights and code · Audio, exact prompts and metrics · Previous Small grid.

What the models do and how they work

A semantic transformer plans speech one 80-ms frame at a time. A local Talker predicts 12 residual codec indices inside each frame; MOSS Audio Tokenizer v2 turns those indices into sound. That is 12.5 frames/s × 12 codes/frame = 150 indices/s, not 12 tokens per second. Small has 596.05M semantic parameters, 112.76M active Talker/bridge parameters (708.81M active total), a 1,024-dimensional semantic state and a 2,560-dimensional local state. An additional stored 6.67M score conditioner remains inactive. Large has about 4.13B parameters: 4.02B semantic and 107.52M local Talker/audio parameters, both with 2,560-dimensional states.

This is full-parameter continuation, not LoRA training or a fresh pretraining run. Small starts at Real + Gemini before its Rank-64 adapter; Large starts at the SFT4 Gemini continuation. Both see the same shuffled mix: original Gemini DE/EN once, five-language extension twice, Gaston five times and ElevenLabs six times. This is 390,914 presentations, about 3,285 presentation-hours. Half use distinct same-speaker 5–15-second references, half learned names. Three GENERAL/SCRIPT views and 10% caption dropout preserve flexible prompting; Gaston always keeps French Accent in GENERAL.

Matched comparison and interpretation

The original 70 cases and 35 natural/organic variants preserve the previous exact prompts, transcripts, seeds and 12-second reference excerpts. The earlier 40-emotion, 20-casual and 10-assistant cases are also retained. New Gaston/ElevenLabs scripts are separate. Their complete transcripts were checked against training targets, but this is not a speaker-disjoint benchmark. Some reference recordings come from training material. No best-of-N or take cherry-picking is used. Historical Real+Gemini/Ref15 predictions are reused only where the weights and input match; the rounded merged previous default is regenerated.

Parakeet v3 WER counts transcript errors (lower is better). Orange 128-D timbre cosine estimates reference similarity (higher is better, not verified identity). Frozen Humaneness Ears Medium predicts emotion intensity, genuineness, burst blending, Audiobox Content Enjoyment and Production Quality. These are model-estimated proxies, not human MOS or direct original Audiobox evaluations. Mean ± standard deviation is across clips; WER is word-weighted. Emotion is blank for cases without a single taxonomy target. No-reference cases never secretly receive an audio clip or a name. Compare the same row and conditioning subset; the 14-case checkpoint loss does not establish universal quality. Sentence-level timing alignment and exact burst realization are not claimed here. Gaston and Eleven-prefixed names are new to Alpha: name-only older-model comparisons are therefore out of distribution, and reference-conditioned rows are the fairer cloning comparison. The original Small S10 used at most 2.96-second references in training: its 12-second-reference column is out of its training range. Ref15 is the fairer long-reference control.

Prompting and new voice examples

Only literal spoken words go into Text. GENERAL/SCRIPT directions, round-parenthesis delivery/burst cues and square-bracket durations/pauses go into Instruction. A name is a separate speaker condition, not a spoken word. Gaston has friendly French-accented English/German compliments, each with name-only and reference-conditioned takes. ElevenLabs cases use four voice slots with new anger, fear and elation scenes. Listen to the shared reference to judge voice match. For name-only rows the audition reference below is NOT supplied to the model; for old no-identity rows neither a reference nor a name is supplied.