MODE_IN_COMMUNICATION activates the telephony DSP pipeline which
degrades audio quality and requires setCommunicationDevice hacks for
speaker routing. The Gemini reference implementation doesn't use it.
Echo protection now relies on:
1. VOICE_COMMUNICATION AudioSource (built-in AEC/AGC/NS)
2. AcousticEchoCanceler on AudioRecord session (hardware AEC)
3. Hard ducking (zero mic chunks during model speech)
AudioTrack stays USAGE_ASSISTANT for full-quality speaker output.
This matches the WebRTC.ventures Gemini prototype architecture.
USAGE_VOICE_COMMUNICATION applies telephony DSP (compression, bandwidth
limiting, noise suppression) that degrades audio clarity. Since we now
have hard ducking (zero mic chunks during model speech) + AEC on the
capture side, we don't need aggressive playback-side processing.
USAGE_ASSISTANT gives full-quality audio output while echo protection
comes from ducking + AcousticEchoCanceler on the AudioRecord.
MODE_IN_COMMUNICATION routes to earpiece by default — no sound on speaker.
The deprecated isSpeakerphoneOn doesn't work on Android 12+.
Fix: enterCallMode()/exitCallMode() lifecycle — sets MODE_IN_COMMUNICATION
+ setCommunicationDevice(SPEAKER) on API 31+ (S25 Ultra = Android 15).
Called BEFORE AudioTrack creation so the routing is active when playback
starts. exitCallMode() restores previous audio mode on call end.
The model was talking to itself because the mic picked up speaker output.
Three fixes that together enable Android's hardware echo cancellation:
1. AudioManager.MODE_IN_COMMUNICATION — tells the platform this is a
voice call so the audio routing + AEC pipeline activates
2. USAGE_VOICE_COMMUNICATION on AudioTrack — links playback to the
capture path so AEC knows what to subtract from the mic signal
3. Explicit AcousticEchoCanceler on the AudioRecord session as fallback
Also enables speakerphone mode (isSpeakerphoneOn) since MODE_IN_COMMUNICATION
defaults to earpiece routing. Cleanup restores previous audio mode.
The old ducking logic only skipped chunks below a threshold (RMS < 0.10),
but the S25 Ultra speakers leak enough audio into the mic to exceed that.
Gemini's server-side VAD picks it up as user speech, triggers interrupts,
and the model starts hallucinating a conversation with itself.
Now: when ducking is active (model speaking), NO mic chunks are sent.
The server-side VAD handles the real speech detection after ducking ends.
encodeDefaults=false caused kotlinx.serialization to skip the
type="setup" field since it was a default value. The server's
parseClientMessage needs the type field to dispatch — without it
every setup message was rejected as "malformed_message".
Changed to encodeDefaults=true + explicitNulls=false so type is
always serialized but null optionals are still omitted.
LiveClient.dispose() called client.dispatcher.executorService.shutdown()
which killed the shared KaizenApi.baseClient dispatcher. After one failed
call, ALL subsequent OkHttp requests (including normal API calls) got
"executor rejected". Removed the shutdown — cancel() on the WS is enough.
Debug logging used token.take(10) + "…" (U+2026) directly in the
HTTP header value. OkHttp rejects non-ASCII in header values.
Moved the truncation to the log line, header uses the full token.
Logs WS URL, setup message, every server event (first 200 chars),
onOpen/onClosed/onFailure with codes, and service-level events.
Filter logcat by "LiveClient" or "LiveCallService".
CONNECTING state could hang forever if the server never responded
or kept failing silently. Now: 20s timeout ends the call with a
visible error, and reconnect_failed is treated as terminal.
Error events from the WS server were silently swallowed — the onError
handler was empty and provider_error didn't trigger endCall. Now:
errors display in the overlay subtitle and loadError banner, and
provider_error is treated as terminal (ends the call).
The overlay used a semi-transparent Color.White/Black scrim (alpha 0.85-0.88),
letting the chat content bleed through. Replaced with MeshBackground
(frozen blobs, no animation) for a proper opaque fullscreen call UI.
New pattern categories:
- Tech products: iPhone/Galaxy/Pixel/iPad + version numbers, specs,
release dates, "neuestes Handy", Apple Vision Pro, Meta Quest, etc.
- Movies/series: Staffel/Season/Episode numbers, cast, ratings,
neue Serie/neuer Film, IMDB, Rotten Tomatoes
- Product comparisons: "vs" with model numbers, "lohnt sich"
- Current leaders: Bundeskanzler, CEO, Präsident
These are high-hallucination topics where the AI's training data
is likely outdated — auto-search gives it fresh facts.
8 new tests (tech, movies, comparisons, leaders).
Auto-search:
- detectSearchQuery() pattern-matches common search prompts in DE+EN
(weather, news, prices, scores, current events, etc.)
- send() enables webSearch automatically when detected, no manual
pill needed. Manual pill still works as explicit override.
- 30+ patterns ported from backend lib/chat/detection.ts
Live reasoning preview:
- ReasoningBlock shows first meaningful line as preview label when
collapsed (max 80 chars, ellipsis). Updates live during streaming.
- "Gedankengang" header + preview line stacked vertically.
- Preview hidden when block is expanded (full text visible).
Tests: 4 new tests for detectSearchQuery (DE patterns, EN patterns,
normal prompts not triggered, short input ignored).
Before: sources hidden behind a collapsed "Quellen (3)" toggle,
vertical stack of flat cards, requires click to see anything.
After:
- Always-visible horizontal scrollable chips with numbered colored
circles (indigo/violet/cyan/emerald/amber/red/pink/blue cycling)
- Each chip shows the domain name, tap opens the URL
- Search query shown above sources when available
- "Details anzeigen" expands to full cards with colored left bar,
title, domain, and snippet
- Perplexity/Gemini-inspired compact visual style
Image generation:
- KaizenApi.generateImage() calls POST /api/v1/image-gen
- Bild mode pill in ChatInput triggers image generation instead of chat
- Generated image displayed as attachment in assistant message
- Conversation created + title generated automatically
Streaming isolation (from previous uncommitted work in ChatScreen):
- liveStream StateFlow emits content during streaming
- Only the active MessageRow subscribes via collectAsState()
- Other LazyColumn items stay completely static during streaming
During streaming, content updates now flow through a StateFlow
(liveStream) instead of mutating the SnapshotStateList on every chunk.
Only the single MessageRow with id == streamingMessageId subscribes to
the flow via collectAsState(). All other items in the LazyColumn stay
completely static during streaming.
Before: every chunk mutated messages[idx] → entire LazyColumn evaluated
After: only the live bubble recomposes, list mutation happens once at
stream end
Auto-scroll split into two LaunchedEffects: one for list size changes,
one for liveStream emissions.
Move all business logic and state (~35 variables, ~15 functions) from
the 1540-line ChatScreen @Composable into ChatScreenViewModel.
ChatScreen is now a thin rendering layer (~430 lines) that reads VM
state and wires UI events to VM methods. Android-framework-only concerns
(ActivityResultLaunchers, DrawerState, LazyListState, keyboard insets)
stay in the Composable.
Changes:
- New ChatScreenViewModel.kt: messages, streaming, conversation mgmt,
sidebar actions, file upload, voice recording, TTS, live call, unlock
- ChatScreen.kt: pure UI, delegates all logic to vm.*
- Haptics.kt: add createHaptics(Context) factory for non-Composable use
- SessionViewModel: store nullable for test construction
- MainActivity: instantiates ChatScreenViewModel
Tests: 12 new unit tests for ChatScreenViewModel (state transitions,
newChat reset, mode toggles, edit/delete, password dialog lifecycle).
All 90 tests pass.
Security:
- Fix JSON injection in deleteMessage (raw string → serialized DTO)
- Separate OkHttpClient for streaming (30s timeout on normal requests,
infinite only for chat stream)
- Disable backup (EncryptedSharedPrefs without KeyStore → crash on restore)
- Block cleartext traffic explicitly
- Strip server URLs from log output
Performance:
- Share OkHttp connection pool across KaizenApi + LiveClient
- Eliminate duplicate fetchMe/fetchDefaultModel call on login
- Parallelize prefetchMissing (was sequential per conversation)
- Stream file uploads directly from ContentResolver (no full readBytes)
- Cache assistant message index during streaming (O(1) vs O(n) per chunk)
- Build chat history before adding assistant placeholder (no filter needed)
Bugs:
- Fix send() race condition (isStreaming set before async work)
- Move voice recording file read to IO dispatcher
Turn persistence: onTurnEnd handler creates conversation if needed,
adds user + assistant messages to UI list, caches in Room. Server
already persists via WS-Server — app only needs local state sync.
Model check: CallButton only triggers Live Call when a vertex:*live*
model is in the catalog (liveAvailable). Falls back to PTT otherwise.
Permission: RECORD_AUDIO runtime request via ActivityResultContracts
before starting the call. Denied → toast, no crash.
Title generation: endLiveCall triggers generateTitle + sidebar refresh,
same pattern as text chat.
MediaPlayer.setDataSource(url) with HTTPS URLs can silently fail due to
certificate issues, redirect handling, or listener race conditions (listeners
were set after prepareAsync). Fix: download the WAV file via OkHttp (same
client as all other network calls) to a temp file, set listeners before
prepareAsync, play from local path. Temp file cleaned up on completion/error.
Edit creates a sibling branch (same parentId) instead of deleting messages.
Regenerate creates a new assistant branch. BranchNavigator UI shows chevron
arrows with "1/3" counter on messages with siblings. Tree state (activeRootId,
activeChild) synced to server via PATCH. New ChatTree data structure mirrors
the web frontend's tree.ts. StoredMessage.parentId parsed from API response.
Replace deprecated ClickableText with LinkAnnotation.Url + Text (Compose
native URI handling). Add GET /api/v1/meta call after login and on each
app resume to detect API version skew — shows dismissible amber banner.
ChatInput redesigned: single row (Plus | Text | Tune | Mic | Send).
Pills hidden by default, expandable via tune icon with slide animation
inside the same GlassSurface card. Shape lg (28dp) instead of pill.
ModePillsRow moved from separate floating row into ChatInput as
pillsContent lambda. Bottom dock scrim simplified.
New Chat button (PenLine icon) added to top bar next to TokenBadge.
LazyColumn bottom padding 260→140dp (dock is now smaller).
Text selection: SelectionContainer wraps both user and assistant
messages — long-press to select, copy, share.
Sidebar profile: avatar 30→38dp with radial gradient, rounded lg
shape, icon buttons with subtle backgrounds, bigger text (15sp W400).
Markdown: heading spacing 18→24/20dp, list items better aligned
(8dp gap, bullet muted), blockquote bar 4dp inset + 12dp gap.
Hand-drawn Menu, Plus, Mic, AudioLines, Brain, SlidersHorizontal,
Globe, Image with 1.0px stroke (vs Lucide's 1.5px) to match the
lighter W300 typography. Registered in KaizenIcons — all usages
update automatically, no call-site changes.
Single source of truth for error, success, indigo, link, and dropdown
colors. Replaces ~25 scattered Color(0xFF...) literals across 10 files.
Syntax highlighting and glass-surface tints intentionally left local.
Single button instead of two mutually exclusive ones. Arrow direction
is based on proximity to bottom: near bottom → arrow up (scroll to top),
not near bottom → arrow down (scroll to end). Updates instantly as the
user scrolls. Hidden when already at the target end.
Old logic had an if/else that always showed ArrowUp first (because
!atTop was always true when scrolled down), making ArrowDown unreachable
in most scroll positions.
statusBarsPadding() was on the inner Column (only pushed content down),
but the GlassSurface itself started at a fixed 12dp from the top edge —
on devices with tall notches (S25 Ultra) the sidebar header sat behind
the system clock/icons. Moved statusBarsPadding() to the GlassSurface
modifier so the entire glass panel starts below the status bar.
ReasoningBlock: accent-tinted background + gradient border (matches pill
language), KaizenShapes.md (16dp radius), larger padding, better line
height (20sp), softer icon tint. Replaces flat gray box with solid border.
SourceCards: gradient border instead of solid, KaizenShapes.md, shows
domain instead of full URL in the link line, slightly larger padding.
Both blocks now use the same visual language as the rest of the app —
gradient borders, accent tinting, squircle shapes.
