krypto/kaizen-app
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
kaizen-app/CLAUDE.md
Bruno Deanoz eb633ac9d6 update CLAUDE.md: document Live Call architecture + gotchas 
Added live/ package to structure, replaced placeholder TODO items with
actual implementation docs covering: LiveClient (WS, reconnect, shared
dispatcher gotcha), LiveProtocol (encodeDefaults critical), AudioPipeline
(AEC, ducking, no MODE_IN_COMMUNICATION, buffer tuning), LiveCallService
(timeouts, terminal errors), LiveCallOverlay (opaque background),
model mapping, conversation persistence.
2026-06-24 14:06:39 +02:00

60 KiB
Raw Permalink Blame History

Kaizen App

Architecture map for the native Android client. Source of truth for any agent or human picking up this codebase.

Goal

Make the native Android app (kaizen-app) a valid daily-driver that talks to the Kaizen backend (kaizen). One binary, deployment-agnostic: one uniform Authorization: Bearer <token> contract, same against the official server or any self-host. Design constraint (hard): backend complexity stays invisible to the user.

Repos (absolute paths)

  • Backend (Next.js 16, Drizzle, Postgres): /Users/brunodeanoz/Kaizen/kaizen — git remote origin = git@git.kryptomrx.de:krypto/kaizen.git, branch main. Read its CLAUDE.md first; it is the architecture map.
  • Android app (Kotlin, Jetpack Compose, package dev.kaizen.app): /Users/brunodeanoz/Kaizen/kaizen-app — built/run from Windows Android Studio on a real S25 Ultra; everything else (backend, Postgres, Docker) runs on the dev Mac + dedicated servers.
  • Note: /Users/brunodeanoz/Kaizen itself is NOT a git repo. Only kaizen and kaizen-app are separate git repos.

Platform strategy

  • Android: Kotlin + Jetpack Compose (native, no WebView)
  • iOS (planned): Swift + SwiftUI + SwiftData (native, separate codebase)
  • No KMP — each platform uses its native stack for best feel

The API contract the app uses (source of truth — verify, don't guess)

  • Auth: every request carries Authorization: Bearer kzn_.... Backend resolves via requireActiveUserOrToken (lib/auth/guard.ts).
  • Token acquisition: in-app login (POST {baseUrl}/api/v1/auth/token with { email, password, name }). Consumer "Sign in" pattern — no copy-paste.
  • Chat: POST {baseUrl}/api/v1/chat — { model, messages: [{ role, content }], reasoningEffort?, preferThroughput?, webSearch?, webSearchProvider?: "google"|"enterprise", temperature?, topP?, topK? }. Response is text/plain chunked stream (NOT SSE). Parsed by StreamConsumer.Incremental in net/StreamConsumer.kt.
  • Sentinels (lib/chat/constants.ts): \x00 USAGE, \x01 REASONING, \x02 SOURCES, \x03 TOOL, \x04 QUERY. App currently strips all sentinels and renders only visible content.
  • Models: GET {baseUrl}/api/v1/models → { models: [{ id, name?, provider? }] }.
  • Conversations: GET /api/v1/conversations → { conversations, hasMore }. POST /api/v1/conversations → create. GET /api/v1/conversations/[id] → { messages }. POST /api/v1/conversations/[id]/messages → persist. POST /api/v1/conversations/[id]/title → auto-title.
  • Identity: GET /api/v1/me → { id, email, name, role, defaultModel, ttsVoice }. PATCH /api/v1/me with { ttsVoice } to update preferences.
  • Capability handshake: GET /api/v1/meta → { app, apiVersion, features } (unauth).
  • Upload: POST /api/v1/upload — multipart file → { url, kind, name, mimeType, size, fileId }.
  • Audio gen: POST /api/v1/audio-gen — { prompt, kind: "speech"|"music", voice? } → { url, cost }. Gemini-TTS (speech) or Lyria (music) via Vertex. Audio persisted in Object Storage.

App architecture

Package structure (app/src/main/java/dev/kaizen/app/)

auth/           LoginScreen, BiometricUnlock (KeyStore + CryptoObject), PasswordUnlockDialog
chat/           ChatScreen, ChatViewModel, Sidebar, ChatComponents, ChatModels,
                ModelSheet, Markdown, Background
db/             Room offline cache: KaizenDatabase, Entities, DAOs,
                Repositories, Converters, Mappers
haptics/        Phase-aware haptics
live/           LiveCallService (foreground service), LiveClient (WS),
                AudioPipeline (capture+playback+AEC), LiveProtocol,
                LiveCallOverlay (fullscreen call UI)
net/            KaizenApi (HTTP), SecureStore (encrypted prefs),
                SessionViewModel, ServerConfig, StreamConsumer
settings/       SettingsScreen, SettingsViewModel, SettingsComponents, SecuritySection
ui/
  theme/        Color (P3), Theme (multi-theme), Type, Oklab, Dither
  theme/themes/ AccentScheme, AmberTheme, BlauTheme, MonoTheme, AuroraTheme, ThemeRegistry
  icon/         KaizenIcons (central registry, Lucide base, custom override layer), KaizenCustomIcons (hand-drawn 1.0px stroke)
  shape/        SquircleShape (superellipse), KaizenShapes (token system)
  effect/       GlassSurface (frosted glass), Shadow (multi-layer stack)
  motion/       Motion (easing curves, spring specs, duration tokens)
  sensor/       TiltState (gyro/accel parallax, lifecycle-aware)

Key components

  • KaizenApi (net/KaizenApi.kt) — singleton object, all HTTP calls via OkHttp. Three clients sharing one connection pool: baseClient (30s timeout, also used by LiveClient), streamClient (infinite timeout, chat only), imageClient (same as baseClient). Streaming chat returns Flow<StreamState> via streamClient. uploadStream() accepts InputStream for zero-copy file uploads.
  • StreamConsumer (net/StreamConsumer.kt) — O(chunk-size) incremental parser, each character visited exactly once across the entire stream.
  • SessionViewModel (net/SessionViewModel.kt) — Compose snapshot state for auth session. Holds ServerConfig (baseUrl, token, model, email, speed). Persisted in EncryptedSharedPreferences via SecureStore.
  • ChatViewModel (chat/ChatViewModel.kt) — owns Room database instance + repositories. Created in MainActivity, passed to ChatScreenViewModel.
  • ChatScreenViewModel (chat/ChatScreenViewModel.kt) — owns all chat state and business logic: messages, streaming, conversation management, sidebar actions, file upload, voice recording, TTS, live call, biometric unlock. Extends ViewModel with viewModelScope. Dependencies: Application?, SessionViewModel, ChatViewModel?, SettingsViewModel. Nullable deps for JVM test construction.
  • ChatScreen (chat/ChatScreen.kt) — thin rendering layer (~430 lines). Reads state from ChatScreenViewModel, wires UI events to VM methods. Owns Android-framework-only concerns: ActivityResultLauncher (camera/gallery/file), DrawerState, LazyListState, keyboard insets, LifecycleResumeEffect.

Design System v2 (ui/) — added 2026-06-21

Full spec: DESIGN.md.

Seven pillars (load-bearing — if any regress, the design regresses):

  1. P3 color pipeline — all tokens in DisplayP3 color space (Color.kt). window.colorMode = COLOR_MODE_HDR (API 34+) / COLOR_MODE_WIDE_COLOR_GAMUT (API 26+) in MainActivity. No Color(0xFF…) for brand colors outside Color.kt.
  2. Squircle shapes — iOS-style superellipse with G2 continuity (ui/shape/Squircle.kt). Token system KaizenShapes.{xs,sm,md,lg,xl,pill,circle}. Replaces all RoundedCornerShape.
  3. GlassSurface — frosted-glass composable (gradient tint + specular border + inner highlight + multi-layer shadow). Android has no backdrop-filter; glass effect works via semi-transparent tint over the blob layer. Opacity tiers per component in GlassTiers.
  4. Multi-layer shadows — ShadowStack with 2–4 overlapping layers (KaizenShadows.level0..4). Dark mode multiplies alphas ×1.8. Replaces all Modifier.shadow().
  5. 4-theme system — Amber/Blau/Mono/Aurora. Each theme = KaizenAccentScheme (primary + 4 blob colors). Neutrals are theme-independent. LocalKaizenAccent CompositionLocal. Theme selection wired end-to-end: SettingsViewModel.selectedTheme → MainActivity → KaizenTheme(themeId=...). No server sync (web stores in localStorage, app stores locally). Appearance (Light/Dark/System) also wired end-to-end.
  6. Sensor-reactive motion — rememberTiltState() provides smoothed tilt Offset(-1..1). Uses TYPE_GAME_ROTATION_VECTOR (fused, no drift), SENSOR_DELAY_UI (~15 Hz, sufficient for parallax). Auto-disables on battery saver, reduced motion, background, extreme tilt.
  7. KaizenIcons (ui/icon/KaizenIcons.kt) — centralized icon registry, same architecture as Google's Luminous Symbols. Lucide (1.5px stroke, same library as web frontend) as base, with a custom override layer (KaizenCustomIcons.kt) for the highest-visibility touchpoints. Custom icons use 1.0px stroke to match the lighter W300 typography. Currently custom: Menu, Plus, Mic, AudioLines, SlidersHorizontal, Globe, Image. Every UI component references KaizenIcons.Send, KaizenIcons.Lock, etc. — never a library directly. Zero Material Icon imports remain. com.composables:icons-lucide-android:1.0.0.

Motion tokens (ui/motion/Motion.kt): EaseSpring (overshoot), EaseSmooth (expo-out), EaseEmphasized (M3), SpringSnappy/Default/Gentle. All durations centralised in Durations object.

Migration status: All UI components migrated to the design system. Sidebar, ChatInput, ChatScreen, LoginScreen, ModelSheet, SettingsCard, SettingsScreen use GlassSurface + KaizenShapes + kaizenShadow + KaizenIcons. Only Markdown code-block shapes remain as RoundedCornerShape (intentionally rectangular for code). All icons migrated from Material Icons Rounded to KaizenIcons/Lucide — zero Icons.Rounded.* imports remain.

Theme switching fully wired: All 7 files that previously imported hardcoded Amber/AmberDeep/OnAmber/BlobX now read from LocalKaizenAccent.current. Changing theme in Settings immediately updates: message bubbles, send button, orb, mesh background blobs, sidebar highlights, model sheet, stream blocks, login screen, settings toggles. No hardcoded amber imports remain outside Color.kt and theme definitions.

Floating bar scrims: Top bar has soft transparent gradient scrim. Bottom dock is the ChatInput capsule itself (no separate scrim needed).

Unified top bar: Sidebar trigger (menu icon) and model name merged into a single GlassSurface pill (38dp, KaizenShapes.pill). Menu icon (17dp, muted) → 3dp dot divider → model name (13sp W300, 180dp max with ellipsis). No chevron — pill is self-evidently tappable. Right side: PenLine "New Chat" button → TokenBadge.

ChatInput capsule: Single-row Gemini-style input — Plus | TextField | Tune | Mic | Send/Call. 64dp min height, pill shape (34dp radius), level4 shadow, high opacity (0.92/0.97). Mode pills hidden by default, expandable via tune icon (SlidersHorizontal) with slide animation inside the same GlassSurface card. Replaces the old two-row layout + separate floating pills row.

MeshBackground (chat/Background.kt) — animateBlobs parameter: blob drift animation (22s cycle) only runs on empty state (messages.isEmpty()), frozen at midpoint during chat to save GPU. Blobs remain visible as static background. Blobs are 400-460dp (large, fill entire screen on S25 Ultra), alpha 0.36-0.48 (dark) with light-mode factor 0.78 (was 0.65). Vignette 0.35 alpha.

KaizenOrb (chat/Background.kt) uses rememberTiltState() from ui/sensor/ instead of inline sensor code. Breath animation uses Durations.ORB_BREATH / ORB_BREATH_STREAMING from ui/motion/.

Typography — Inter Variable (res/font/inter_variable.ttf, 859 KB, SIL OFL). Full type scale from DESIGN.md §8: displayLarge (32sp/W600) through labelSmall (11sp/W500). All 9 weight axes registered (W100–W900). Negative tracking at display sizes, positive at label sizes. @OptIn(ExperimentalTextApi::class) for FontVariation.Settings. Visual refresh (2026-06-23): Body text W400→W300 for airier feel (inspired by Gemini's Neural Expressive). Headings scaled up: H1 26sp/W600, H2 22sp/W600, H3 18sp/W500. User bubble text, input field, hero all W300. Labels/pills stay W400-W500 for legibility.

Semantic Colors — KaizenSemanticColors object in Color.kt: error, errorDeep, success, successBright, indigo, linkLight/Dark, dropdownDark/Light. Single source of truth for status/action colors — no Color(0xFF...) for these outside Color.kt. Syntax highlighting colors (Dracula theme) and glass-surface tints (Slate) intentionally stay local.

Oklab (ui/theme/Oklab.kt) — Double-precision matrix math for accurate sRGB↔Oklab round-trips. Used by theme picker's Oklab gradient swatches and generateGradientStops(). Forward LMS matrix corrected to Björn Ottosson reference.

Internationalization (i18n)

  • de (default), en — same as web (messages/de.json, messages/en.json).
  • Strings: res/values/strings.xml (de) + res/values-en/strings.xml (en).
  • All UI strings use stringResource(R.string.xxx) in @Composable functions, or context.getString(R.string.xxx) in non-composable contexts (error handlers, coroutines).
  • Date/time: DateTimeFormatter with Locale.getDefault().
  • Greeting: time-of-day-based (Morgen/Tag/Abend → Morning/Afternoon/Evening), boundaries 5/12/18.
  • ChatModels: Suggestion.labelRes/promptRes and ChatMode.labelRes are resource IDs (Int), not hardcoded strings.
  • ModePillsRow: selected state is Int (resource ID), not String.

Settings persistence + server sync

  • SettingsViewModel takes SecureStore — theme, appearance, username, locale, biometric toggle survive app restart.
  • Theme selection persisted as KaizenThemeId.value string ("amber"/"blue"/"mono"/"aurora").
  • Appearance persisted as AppAppearance.name ("Light"/"Dark"/"System").
  • On login + on each foreground: KaizenApi.fetchMe() → MeResponse(name, email, defaultModel) populates SettingsViewModel name/email.
  • Language selector in Settings: System / Deutsch / English. Stored as locale in SecureStore. Note: currently only stores the preference; runtime locale switching (AppCompatDelegate.setApplicationLocales) needs wiring in MainActivity on next iteration.
  • No server-side theme storage — web uses localStorage, app uses SecureStore. Both are client-local.
  • Security section (SecuritySection.kt): Biometric toggle (enabled/disabled, persisted in SecureStore), signed-in devices (list + remote revoke via GET/DELETE /api/v1/auth/tokens), password change (POST /api/v1/auth/change-password). requestBiometricOrPassword() in ChatScreen respects the biometric toggle — when disabled, always falls back to password dialog.

Sentinel parsing — reasoning, sources, tools (added 2026-06-21)

The /api/v1/chat stream embeds control sections via sentinel code points (from lib/chat/constants.ts):

Sentinel Code What App UI
\x00 USAGE trailing Usage/cost JSON Stripped (display TODO)
\x01 REASONING toggle Model's chain-of-thought ReasoningBlock — collapsible "Gedankengang"
\x02 SOURCES trailing Web search sources JSON SourcesBlock — compact cards with domain
\x03 TOOL leading Tool-call events (JSON+newline) ToolEventsBlock — state machine (thinking/analyzing/start/end/error)
\x04 QUERY trailing Search query string Shown above sources

StreamConsumer.Incremental (net/StreamConsumer.kt) returns StreamState (not String):

  • content — visible text (reasoning stripped, tool events stripped)
  • reasoning — full reasoning text
  • tools — List<ToolEvent> (type, name, args, result, error, elapsed)
  • query — search query string
  • sources — List<SearchSource> (title, url, snippet)

KaizenApi.chat() returns Flow<StreamState>. ChatScreen maps all fields to Message during streaming.

UI files: chat/StreamBlocks.kt — ReasoningBlock, ToolEventsBlock, SourcesBlock, SourceChip.

ToolEventsBlock state machine (matches web frontend LiveToolSteps):

  • type: "thinking" → "✨ Verarbeitet …" (only shown when no other tool activity)
  • type: "analyzing" → "📄 Analysiert {name} …" (file processing)
  • type: "start" → wrench icon + tool name (in progress)
  • type: "end" → ✅ checkmark + tool name + elapsed ms (completed)
  • type: "error" → ❌ error icon + tool name (failed)
  • Empty/thinking-only events are hidden (no generic "Tool" fallback).

App icon

Adaptive icon: translucent glass-lens orb (8 layers — subtle amber tint at 50-60% alpha, light speculars, thin rims). Matches the in-app KaizenOrb aesthetic (glass, not opaque paint). Light mode: neutral off-white background (#F6F7F9). Dark mode: Obsidian gradient (#0F1520). Monochrome layer for Android 13+ themed icons. Splash screen uses orb foreground via windowSplashScreenAnimatedIcon (API 31+). Vector-only — no raster WebPs.

Offline-first cache layer (Room) — added 2026-06-21

Architecture decision document: OFFLINE_CACHE_ADD.md.

Principle: Read-cache, not sync engine. Server is source of truth. No offline-queue, no conflict resolution.

Stack: Room 2.8.4 + KSP 2.2.10-2.0.2. room-ktx merged into room-runtime since Room 2.7 (no separate dep needed). exportSchema = false, fallbackToDestructiveMigration() — it's a cache, not a DB. On schema changes the SQLite is wiped; cache rebuilds on next server fetch.

Schema:

  • conversations — id, title, locked, pinned, updatedAt, cachedAt, titleGenerated
  • messages — id, conversationId (FK CASCADE), role, content, attachments (JSON via TypeConverter), sortOrder

Data flow:

App start    → Room (instant) → UI shows cached data
             → Background: server fetch → Room update → Flow emits → UI updates
             → Background: prefetch messages for all uncached conversations
Chat open    → Room (instant) → Background: server fetch → Room update
Send message → in-memory during streaming → Room flush after stream ends

Background prefetch: After conversation list sync, MessageRepository.prefetchMissing() iterates all unlocked conversations and fetches messages for any not yet cached. This ensures all chats are readable offline, not just previously opened ones. Already-cached conversations are skipped.

Streaming hybrid model: During active streaming, messages are mutated in-memory (60x/s). Room is NOT written during streaming — that would be absurd I/O. After stream completion, user + assistant messages are upserted into Room. This avoids SQLite lock contention while keeping the cache fresh.

Files:

  • db/Entities.kt — ConversationEntity, MessageEntity
  • db/ConversationDao.kt — observeAll() (Flow), replaceAll() (transaction: deleteAll + upsertAll, avoids SQLite 999-variable limit), lastCachedAt(), allUnlockedIds()
  • db/MessageDao.kt — observeByConversation(), getByConversation(), upsertAll(), deleteByConversation(), cachedConversationIds()
  • db/KaizenDatabase.kt — singleton via companion object (double-checked locking), thread-safe
  • db/ConversationRepository.kt — observeAll() (Flow of ConversationSummary), refresh() (server → Room), allUnlockedIds()
  • db/MessageRepository.kt — observeMessages(), getCached(), fetchAndCache(), cacheMessages(), prefetchMissing() (background prefetch for offline)
  • db/Converters.kt — List<Attachment> + List<SearchSource> ↔ JSON string via kotlinx.serialization
  • db/Mappers.kt — ConversationSummary ↔ ConversationEntity, StoredMessage ↔ MessageEntity

AGP 9.x compat: android.disallowKotlinSourceSets=false in gradle.properties (KSP issue #2729).

What's DONE and verified

Backend — all complete, deployed, tested

All backend work for the app is done. pnpm test:run green (774+), tsc + lint clean.

  • api_tokens table + SHA-256 hash storage (never raw). Migration 0024.
  • Bearer auth path (getTokenSession / requireActiveUserOrToken in lib/auth/guard.ts).
  • Token mint endpoint POST /api/v1/auth/token — unauthenticated, Argon2id verify, anti-enumeration, rate-limited.
  • GET /api/v1/me — identity + defaultModel + ttsVoice for native clients.
  • PATCH /api/v1/me — update user preferences (ttsVoice, etc.). Bearer-authed.
  • GET /api/v1/models — Bearer-authed, returns filtered model catalog.
  • /api/v1/conversations — thin re-exports of canonical handlers, Bearer-authed.
  • POST /api/v1/auth/unlock — Bearer-authed, returns unlock token in response body (not cookie) for native clients. Rate-limited 10/5min per user.
  • GET /api/v1/conversations/[id] — accepts X-Unlock-Token header alongside the existing kaizen_unlock cookie for locked conversations.
  • GET /api/v1/meta — capability handshake.
  • App-devices card (components/settings/app-tokens-card.tsx) — see/revoke signed-in devices on /settings/security.
  • POST /api/v1/audio-gen — Gemini-TTS speech generation. Bearer-authed (re-export of /api/audio-gen). { prompt, kind: "speech", voice? } → { url, cost }.
  • tts_voice column on users table (migration 0030). Server-synced voice preference, read via /api/v1/me, written via PATCH /api/v1/me.
  • API versioning — /api/v1/ for outward contract, additive-only within v1. Web-internal endpoints stay unversioned.

App — shipped features (on-device verified on S25 Ultra)

All committed on main. Compile-verified on Mac (./gradlew :app:compileDebugKotlin clean).

Commit Feature
44c17b6 In-app login (email+password → POST /api/v1/auth/token), EncryptedSharedPreferences, real streaming via StreamConsumer.Incremental, ServerConfig persistence
aa0a97b Dither overlay to kill 8-bit sRGB banding on orb + mesh gradients
d2f269f Model + speed picker (top pill → bottom sheet), response-speed segment, favorites, provider badges
ea5e625 Sidebar wired to real conversations, persistence (create/save/load/auto-title), date-grouped history
7399c73 Model sheet redesign with OpenRouter/Vertex badging, offline-first caching
d81c6ff Robust JSON parsing (malformed response handling)
827a2f3 Streaming performance, error handling
423abc0 Specific error diagnostics — FetchResult<T> replaces null-returns; error banner shows exact cause (HTTP status, DNS, SSL, timeout, JSON parsing) instead of generic "Verbindung zum Server fehlgeschlagen"
4616ef2 Markdown rendering with syntax-highlighted code blocks — native Compose renderer: headings, bold/italic, inline code, fenced code blocks with keyword highlighting (15+ languages), ordered/unordered lists, streaming cursor integration
d8dcb25 Attachment support — display, upload, and send files — parse + render message attachments from loaded conversations (images inline via OkHttp + in-memory cache, other kinds as icon chips); + button wired to Android file picker with upload to /api/v1/upload, progress chips, send attachments with chat requests + persist in saved messages. Also: Markdown horizontal rules, blockquotes, links, code block copy button
9759d47 Offline-first cache layer (Room) — sidebar + messages load instantly from SQLite cache, background refresh syncs with server, streaming hybrid model (in-memory during stream, Room flush after), ChatViewModel, repositories, mappers, TypeConverters, unit tests
5804b33 Gradient scrims on floating top/bottom bars — content no longer bleeds through the model pill or mode pills
78f03ad Mode pills cleanup — removed muddy amber tint, solid surface fills, neutral borders
3560665 Mode pills multi-select — modes are now independent toggles, not mutually exclusive
e4c6e7f EmptyHero centered — orb + greeting vertically centered between top/bottom bars
fe5e93c Settings layout fix — option rows stack title above controls instead of cramming side-by-side
9186594 Background prefetch — all conversation messages prefetched for offline access on app start
d468ec8 Theme switching wired — all 7 files migrated from hardcoded Amber to LocalKaizenAccent.current
cad445f i18n cleanup — replaced all remaining hardcoded German strings with stringResource()
5c54b9b ToolEventsBlock rewrite — state machine matching web frontend (thinking/analyzing/start/end/error)
da01b57 Attachment picker menu — three floating GlassSurface pills (camera/gallery/file) with colored round icons, slide animation, dismiss-on-tap-outside. Positioned bottom-left above the + button. Camera uses TakePicturePreview, gallery uses PickMultipleVisualMedia(ImageAndVideo), file uses OpenMultipleDocuments — gallery + files support multi-select
f3d8154 Sidebar redesign — GlassSurface opacity 0.92/0.88, flat conversation rows, 3-dot context menu (rename/delete/pin/lock via PATCH/DELETE /api/v1/conversations), smaller avatar (30dp), logout integrated into user card
f3d8154 Markdown typography overhaul — line-height 23→26sp (×1.625), paragraph spacing 6→12dp, heading margins 18dp/8dp, list spacing doubled. Matches Gemini readability
532cb8c Reasoning + sources persistence — saved to server (SaveMessage.reasoning/sources), loaded back (StoredMessage), cached in Room (MessageEntity + SearchSource TypeConverter). Reasoning rendered above content, sources below
88559bf Biometric unlock for locked conversations — Android KeyStore AES key with setUserAuthenticationRequired(true) + CryptoObject, hardware-enforced. Server unlock token via POST /api/v1/auth/unlock + X-Unlock-Token header. Auto-lock on app background. Password fallback dialog (PasswordUnlockDialog) when biometrics unavailable — server verifies password via Argon2id. All unlock paths (open chat, sidebar toggle) gated behind requestBiometricOrPassword()
30d3f28 Search mode functional — webSearch: true in ChatRequest triggers agentive web search. Sources/query parsed by StreamConsumer, rendered in collapsible SourcesBlock with clickable links (ACTION_VIEW)
f6dc5db Reasoning presets + sampling controls — 8-preset dropdown (Sofort→Maximal) sends reasoningEffort/preferThroughput. Sampling popover with per-parameter toggle + slider for Temperature/Top-P/Top-K
3ca1d2c Token counter — parses usage sentinel from stream (inputTokens/outputTokens), shows ⚡ badge in top bar. Sums tokens from server messages for loaded chats. Context length from model's context_length field
3ca1d2c 3D user bubble — multi-layer gradient, inner highlight, kaizenShadow level1, thicker border
3ca1d2c Scroll buttons — glass arrow at center-right, directional (up OR down), hidden during streaming
f784902 Orb rewrite — transparent glass lens (15-22% alpha tint) instead of opaque painted ball. Background blobs bleed through. Elliptical speculars, double rim, subtle caustics
4ebe812 Theme visibility fix — added missing blob3, increased blob alpha (0.30-0.42), reduced vignette (0.40). Themes now visually distinct
f22838e Message actions — Copy/Delete/Regenerate buttons under each message. Delete via DELETE /api/v1/conversations/[id]/messages
2d57aff Input field redesign (superseded by 34a527c) — original pill shape, higher opacity, Mic/Send toggle
e9e2a66 Stronger haptics — send CLICK 1.0, thinkingStart double-tap, responseStart crescendo, responseEnd firm settle
e9e2a66 Live language switching — LocaleManager (API 33+) or recreate() fallback, immediate effect
15f5781 Biometric bypass fix — closed 3 auth bypass paths in openConversation() + sidebar ToggleLock. Unified requestBiometricOrPassword() gate, PasswordUnlockDialog fallback, server POST /api/v1/auth/unlock now verifies password (Argon2id) when provided
ba7ebaf Security settings — biometric toggle (persisted in SecureStore), signed-in devices list with remote revoke (GET/DELETE /api/v1/auth/tokens), password change form (POST /api/v1/auth/change-password)
e62c525 App icon redesign — 10-layer glass-lens orb (soft halo, 6-stop amber body, cool glass overlay, bounce light, volume shadow, caustic structure, elliptical specular + micro pinpoint glint, gradient rims). Improved backgrounds (light/dark) + monochrome
1929073 Unified top bar — merged hamburger circle + ModelPill into single GlassSurface pill (menu icon → dot divider → model name 13sp Medium, 180dp max ellipsis). No chevron
1362dae Snappy sidebar — optimistic insert (new chat appears instantly with first-line preview), generateTitle returns title directly + Room updateTitle, replaceAll uses upsert+deleteExcept (no empty-list flash)
68721d8 KaizenIcons system — centralized icon registry (ui/icon/KaizenIcons.kt), Lucide 1.5px stroke as base (same as web), custom override layer for brand icons. All 10 source files migrated, zero Material Icon imports remaining
ad11c0e Multi-select file picker — gallery uses PickMultipleVisualMedia(ImageAndVideo), file picker uses OpenMultipleDocuments. Multiple items selectable at once
34a527c ChatInput redesign — two-row layout (text above, action icons below), 32dp pill corners, 16dp margins, level3 shadow, glasier tintAlpha (0.82/0.88). Plus + Mic buttons with subtle circular backgrounds
f703196 Soft bottom dock scrim — transparent gradient (0→25→55% alpha) instead of opaque block. Mesh background bleeds through
313c21d Performance: sensor + blob optimization — SENSOR_DELAY_GAME → SENSOR_DELAY_UI (~15 Hz). Blob drift animation pauses during chat (frozen at midpoint), only animates on empty state
70aa5c5 ChatInput button layout — Mic (voice) + Plus left, Call/Send toggle right. AudioLines icon for call. SendButton: no border, translucent gradient, 38dp
fd0de5c Markdown upgrade — tables (| col | col |), strikethrough, task lists (- [x]), clickable links (ACTION_VIEW), block comments (/* */), HTML comments, template strings, C/C++/Ruby/PHP/Dockerfile/TOML highlighting. All regex pre-compiled
bf94414 CallButton — styled to match SendButton: 38dp circle, subtle accent gradient (16-28% alpha), AudioLines icon in accent color, press-scale animation
141d4de Stop button + auto-logout + locale fix + login shadow migration — red square stop button during streaming (cancels stream, preserves partial content), auto-logout on HTTP 401, EmptyHero date respects system locale, LoginScreen shadows migrated to kaizenShadow
edd8cca Bottom padding fix — LazyColumn bottom contentPadding 152→220dp, fixes messages overlapping mode pills after ChatInput two-row redesign
9e1f30d Voice recording + TTS read-aloud — Mic button wired to MediaRecorder (AAC/M4A), tap-to-record, auto-upload as audio attachment. Volume button on assistant messages → POST /api/v1/audio-gen → Gemini-TTS → MediaPlayer playback. RECORD_AUDIO permission
c4884a3 TTS audio cache in Room — ttsUrl + ttsVoice on MessageEntity (Room v3), skip API call on replay with same voice. Voice change invalidates cache
1645be8 Server-synced TTS voice — reads ttsVoice from GET /api/v1/me, uses it for all TTS calls. Voice set in web settings applies in app automatically
558b1be Sidebar redesign — glasier look (tintAlpha 0.80/0.86), functional search (live title filter), "Titel generieren" (Sparkles, POST .../title with { manual: true }), "Duplizieren" (copies messages with new IDs + parentId chain), titleGenerated on ConversationSummary/Entity (Room v4), accent gradient borders on active items + user card
10ece6d Rounder chat items — KaizenShapes.md (16dp) instead of sm (12dp), 4dp gap, card-style glass borders on inactive items, horizontal padding
497f0a8 Mode pills redesign — extracted shared ModePill composable, thinner gradient borders (0.8dp), compact padding (12×7dp), styled reasoning dropdown (check icon on selected), compact sampling panel (220dp width, 28dp slider height, 16dp custom checkbox)
20a71ad Voice recording auto-send — tap mic → record → tap stop → auto-upload → auto-send → stream response (no manual send). Recording UI: live 32-bar waveform visualization (amplitude from MediaRecorder.getMaxAmplitude() polled every 60ms), centered red stop button
d289466 Live voice waveform — scrolling bar chart reacts to microphone amplitude in real-time, bars use accent color, louder = taller + more opaque
fb6bc98 Web search provider picker — for Vertex models: Google vs Enterprise Web Search dropdown pill, webSearchProvider sent in chat request
1899f2c Voice message display — audio-only user messages render as compact "🎤 Sprachnachricht" pill instead of ugly filename chip
44ff82a Voice message ordering fix — caches user + assistant messages in Room with correct sortOrder after server save (was missing, caused wrong order on reload)
8f72afd Edit and resend user messages — SquarePen icon on user messages, creates branch (old messages preserved), text back in input field
b2df7e9 Modern message action icons — ClipboardCopy (copy), SquarePen (edit), RotateCcw (regenerate), 36dp touch target, 20dp icons
421b29e ReasoningBlock + SourceCards redesign — accent-tinted background + gradient borders, KaizenShapes.md (16dp radius), domain instead of full URL in source cards
75ca8c0 Visual refresh — body text W400→W300 (airier), headings scaled up (H1 26sp, H2 22sp, H3 18sp), animateItem() on messages (350ms fade-in, spring placement), blobs 20% larger + better distribution, higher alpha on OLED, light-mode factor 0.65→0.78, distinct primaryDeep per theme
38678a2 KaizenSemanticColors — extracted ~25 hardcoded Color(0xFF...) into semantic tokens (error, success, indigo, link, dropdown). Single source of truth in Color.kt
0f6e55d NetworkImageCache upgrade — disk cache (cacheDir/img), shared KaizenApi.imageClient (reuses connection pool), 32MB size-based memory limit, Crossfade animation on load
3c6a891 Custom 1.0px icons — KaizenCustomIcons.kt with hand-drawn Menu, Plus, Mic, AudioLines, SlidersHorizontal, Globe, Image. Thinner stroke matches W300 typography
ad98b9f App icon + splash redesign — translucent glass-lens orb (was opaque plastic "Spiegelei"). Neutral backgrounds matching app colors
4958ca8 Single-row ChatInput — Gemini-style capsule: Plus | TextField | Tune | Mic | Send/Call. Mode pills hidden by default, expandable via tune icon inside the same card. New Chat button (PenLine) in top bar
1c9491e Text selection — SelectionContainer on user + assistant messages. Long-press to select, copy, share
1c9491e Markdown spacing — heading spacing 18→24/20dp, list items 8dp gap + muted bullets, blockquote 12dp inset
430d5b6 Flat sidebar user row — removed heavy card (background, border, shadow), replaced with flat row + thin separator
— Branch navigation (tree support) — ChatTree data structure mirrors web frontend's tree. Edit creates a branch (sibling node with same parentId) instead of deleting messages. Regenerate creates a new assistant branch. BranchNavigator UI (chevron arrows + "1/3" counter) on messages with siblings. activeRootId/activeChild synced to server via PATCH. StoredMessage.parentId parsed from API. patchConversation handles JSON object values for activeChild
3b73e15 Security hardening + performance pass — JSON injection fix (deleteMessage), separate stream OkHttpClient (30s timeout on normal requests), disable backup, block cleartext, strip URLs from logs. Shared connection pool (LiveClient reuses KaizenApi.baseClient), eliminate duplicate fetchMe, parallel prefetchMissing, streaming file uploads (no readBytes), O(1) message index during streaming, send() race condition fix, voice recording file read on IO thread
40db869 ChatScreenViewModel extraction — moved ~35 state variables + ~15 business-logic functions from the 1540-line God Composable into ChatScreenViewModel. ChatScreen is now ~430 lines (pure UI). createHaptics() factory for non-Composable use. SessionViewModel.store nullable for JVM test construction. 12 new unit tests

Earlier UI/feel work (Phase 0 prototype → feel-first):

  • Liquid glass styling, floating panels, glassmorphic sidebar
  • KaizenOrb with 3D sphere rendering, streaming animation
  • Mesh gradient background with dithered blobs
  • Phase-aware haptics (click → thinkingStart → responseStart → responseEnd)
  • Tablet/landscape optimization (width capping)
  • Dynamic keyboard handling: WindowInsets.ime offset (NOT imePadding() — that inflates the dock; NOT adjustResize — hides the dock with edge-to-edge). windowSoftInputMode=adjustNothing in manifest.
  • Settings screen with SettingsViewModel

Bugs fixed (2026-06-19/20/21)

  • "Verbindung zum Server fehlgeschlagen" on S25 (06-19) — server running old build without v1 routes. Fix: git pull && docker compose up -d --build.
  • Server 500 on conversation load (06-21) — Drizzle migration 0025 (model_id column) skipped due to rogue watermark entry (created_at = 1781600000000) in drizzle.__drizzle_migrations table, caused by hand-typed fake timestamps. Fix: lowered rogue entry's timestamp, re-ran node scripts/migrate.mjs.
  • Deploy gotcha (migrations): fake timestamps in journal entries 0017–0025 (60-second increments) can cause watermark conflicts. Future migrations via pnpm db:generate use real timestamps and won't hit this.

Bugs fixed (2026-06-21/22 — app session)

  • Room crash on launch — schema hash mismatch after adding reasoning/sources fields to MessageEntity with version still at 1. Fix: bump to version 2, fallbackToDestructiveMigration(true).
  • Keyboard handling — imePadding() on Compose Box/Column caused input field to float to top of screen OR be hidden behind keyboard. Fix: read WindowInsets.ime.getBottom() directly, apply as negative Y offset() on the bottom dock. windowSoftInputMode=adjustNothing in manifest.
  • Locked chats not opening — context as? FragmentActivity silently returned null (ContextWrapper). Fix: walk ContextWrapper chain.
  • Locked chats bypassed biometric auth (06-22) — three paths in openConversation() called unlockAndOpen() without any authentication when biometrics were unavailable (FragmentActivity null, isAvailable() false, Result.NotAvailable). Sidebar ToggleLock also sent PATCH { locked: false } without auth. Fix: unified requestBiometricOrPassword() gates all unlock paths; PasswordUnlockDialog as fallback when biometrics unavailable; server POST /api/v1/auth/unlock now optionally verifies password (Argon2id) when { password } provided.
  • Error banner stuck — loadError never cleared after unlock failure. Fix: auto-clear after 4s, reset on navigation.
  • Reasoning/sources lost — not persisted to server or Room cache. Fix: added fields to SaveMessage, StoredMessage, MessageEntity, Converters, Mappers.
  • Reasoning at bottom — rendered below content like sources. Fix: moved above content in ChatComponents.
  • Themes barely visible — blob3 never rendered, alpha too low (0.22-0.30), vignette too aggressive (0.55). Fix: 4 blobs, alpha 0.30-0.42, vignette 0.40.
  • Mode pills were mock — didn't send anything to backend. Fix: search sends webSearch: true, reasoning sends reasoningEffort/preferThroughput, sampling sends temperature/topP/topK.
  • Token counter always 0 — only updated during streaming, not for loaded chats. Fix: sum inputTokens + outputTokens from server messages.
  • Model pill wrong — showed global default instead of per-chat model. Fix: chatModel state from last assistant message's model field.

Bugs fixed (2026-06-22 — security + UI session)

  • Locked chats bypassed biometric auth — three paths in openConversation() called unlockAndOpen() without authentication when biometrics unavailable. Sidebar ToggleLock sent PATCH { locked: false } without auth. Fix: unified requestBiometricOrPassword() gates all unlock paths; PasswordUnlockDialog fallback; server verifies password via Argon2id.
  • New chats not appearing in sidebar — conversation only showed after stream + save + title + server refresh (seconds of delay). Fix: optimistic Room insert immediately after createConversation() with first-line preview as title.
  • Title generation not working immediately — generateTitle was fire-and-forget, refreshConversations() raced with server-side generation. Fix: generateTitle now returns the title from the response; updateTitle() writes directly to Room.
  • Sidebar flicker on refresh — replaceAll() did DELETE ALL + INSERT, causing Room Flow to emit empty list briefly. Fix: upsert + deleteExcept (no flash).
  • Gallery/file picker single-select only — GetContent() allowed only one item. Fix: gallery → PickMultipleVisualMedia(ImageAndVideo), file → OpenMultipleDocuments(["*/*"]).

Bugs fixed (2026-06-23 — voice + UX session)

  • Messages overlapping mode pills — LazyColumn bottom contentPadding was 152dp but the two-row ChatInput redesign made the dock ~210dp tall. Fix: increased to 220dp.
  • EmptyHero date hardcoded German — Locale.GERMAN → Locale.getDefault(), now respects system/app language.
  • LoginScreen used Modifier.shadow — violated design system pillar 4 (multi-layer shadows). Fix: migrated to kaizenShadow (level1 for fields, level2 for button).
  • Migration 0030 crash — pnpm db:generate bundled the entire org schema (0026-0029) + tts_voice into one migration, causing duplicate CREATE TABLE / ADD COLUMN failures. Fix: reduced 0030 to only ALTER TABLE users ADD COLUMN IF NOT EXISTS tts_voice text.

Bugs fixed + features (2026-06-23 — design + sidebar + voice session)

  • Sidebar too opaque / bland — tintAlpha was 0.88/0.92 (barely any blob bleed-through). Fix: lowered to 0.80/0.86, added gradient borders on active items + user card + search bar.
  • Sidebar search was placeholder-only — just showed "Suchen…" text. Fix: functional BasicTextField with live case-insensitive title filter + X clear button.
  • Missing sidebar features — no "Titel generieren" or "Duplizieren". Fix: added SidebarAction.GenerateTitle (calls POST .../title with { manual: true }, only shown when titleGenerated == false) and SidebarAction.Duplicate (fetches messages, creates new conversation with copied messages + parentId chain).
  • Pills barely visible in dark + light mode — inactive background alpha was too low (0.06/0.04), border was invisible (white-on-light). Fix: raised all alpha values, borders now use visible colors in both modes.
  • Sampling panel too large — overlay covered most of the chat. Fix: width 260→220dp, slider height capped at 28dp, spacing reduced, checkbox 20→16dp.
  • Voice messages shown as ugly file chip — voice_1782170416740.m4a filename displayed. Fix: audio-only user messages render as "🎤 Sprachnachricht" pill.
  • Voice messages wrong order after reload — auto-send code saved to server but not to Room cache. Fix: added Room caching with correct sortOrder.
  • Message spacing too tight — 16dp between messages. Fix: increased to 24dp.
  • No edit/resend for user messages — only copy + delete were available. Fix: SquarePen button removes message + successors, puts text back in input.
  • webSearchProvider not sent by app — Vertex's Google vs Enterprise search toggle was missing. Fix: added webSearchProvider to ChatRequest, dropdown pill in mode pills row (only visible for vertex: models with search active).

Visual refresh + UX overhaul (2026-06-23 — design session)

  • Typography too heavy — W400 body everywhere looked dense. Fix: W300 for body/input/hero, W600/W500 for headings (inspired by Gemini's Neural Expressive thinner Roboto Flex).
  • Headings too small — H1 22sp, H2 19sp, H3 17sp. Fix: H1 26sp, H2 22sp, H3 18sp with proper line-height.
  • Messages pop in without animation — no entry/exit transition. Fix: animateItem() on LazyColumn (350ms fade-in, spring placement, 200ms fade-out).
  • Themes washed out / blobs too small — bottom half of screen had no color, light mode barely visible. Fix: blobs 20% larger (400-460dp), better full-screen distribution, higher alpha (0.48 peak), light factor 0.65→0.78, primaryDeep distinct per theme.
  • Hardcoded colors scattered — ~25 Color(0xFF...) across 10 files. Fix: KaizenSemanticColors object in Color.kt.
  • Image cache had no disk persistence — images re-downloaded every app restart. Fix: file-based disk cache in cacheDir/img, shared KaizenApi.imageClient, 32MB size-based memory limit, Crossfade on load.
  • App icon still old "Spiegelei" — opaque plastic orb didn't match glass-lens in-app orb. Fix: redesigned to translucent glass-lens with neutral backgrounds.
  • Mode pills floating separately above input — two disconnected UI elements, ugly scrim. Fix: pills moved inside ChatInput card, hidden by default, expandable via tune icon.
  • ChatInput was two-row layout — wasted vertical space. Fix: single-row Gemini-style capsule (64dp, 34dp radius, level4 shadow).
  • No "New Chat" button — only accessible via sidebar. Fix: PenLine icon in top bar.
  • Text not selectable — couldn't select/copy text from messages. Fix: SelectionContainer on user + assistant messages.
  • Sidebar profile was heavy card — background, border, shadow. Fix: flat row with thin separator.
  • Custom icons too thick — Lucide 1.5px at 16-19dp looked chunky next to W300 text. Fix: 7 custom ImageVector icons at 1.0px stroke in KaizenCustomIcons.kt.

Bugs fixed (2026-06-23 — locked chat privacy)

  • Locked chat title leaked via search — sidebar search matched against the real item.title of locked chats. Fix: locked chats excluded from search results.
  • "Titel generieren" available on locked chats — generating a title is a form of renaming and shouldn't work on locked chats. Fix: blocked in 3-dot menu (same as rename).
  • "Duplizieren" available on locked chats — created an unlocked copy with all messages, completely bypassing the lock. Fix: blocked in 3-dot menu.

What's NOT done yet (app, priority order)

1. Remaining visual/UX gaps (daily-drive blockers)

All done:

  • Markdown rendering during streaming — all regex patterns pre-compiled as top-level constants, remember(text) memoization. Verified performant
  • Code block copy button — tap-to-copy on fenced code blocks (clipboard + haptic feedback)
  • Horizontal rule / blockquote rendering — --- and > quote parsed and rendered
  • Table rendering — | header | header | with separator detection, styled borders, header background, horizontal scroll for wide tables, inline markdown in cells
  • Link rendering — clickable via LinkAnnotation.Url + LocalUriHandler, opens browser on tap
  • Strikethrough — ~~text~~ rendered with line-through + dimmed alpha
  • Task lists — - [ ] / - [x] with Check/Square icons in accent color
  • Text selection — SelectionContainer on user + assistant messages, long-press to select/copy/share
  • Single-row ChatInput — Gemini-style capsule (64dp, pill shape), mode pills inside card via tune icon
  • New Chat button — PenLine icon in top bar
  • Visual refresh — W300 typography, larger headings, message animations, richer themes/blobs, semantic colors, custom 1.0px icons, glass-lens app icon

2. Conversation management from the app

All done:

  • Conversation rename/delete from the sidebar — 3-dot menu on each chat, PATCH/DELETE /api/v1/conversations/[id]
  • Conversation pin/unpin from the sidebar — PATCH { pinned }, star icon for pinned items
  • In-app unlock for locked conversations — BiometricPrompt with CryptoObject (AES key in TEE/StrongBox), hardware-enforced biometric auth → server unlock token via POST /api/v1/auth/unlock → fetch messages with X-Unlock-Token header. Auto-lock on background. Fallback to direct unlock if biometrics unavailable

3. Security hardening

  • Auto-logout on 401 — if the token is rejected, route to login screen instead of showing error banner
  • Server version check — calls GET /api/v1/meta after login + on each foreground, compares apiVersion against EXPECTED_API_VERSION, shows dismissible amber warning banner on skew
  • Biometric lock (BiometricPrompt + KeyStore/CryptoObject) — hardware-enforced, password fallback dialog when unavailable, biometric toggle in Settings, all unlock paths gated
  • Security settings — biometric toggle, signed-in devices with remote revoke, password change
  • Password change — POST /api/v1/auth/change-password (Argon2id, rate-limited)
  • Device management — GET/DELETE /api/v1/auth/tokens (list + revoke, can't revoke self)
  • JSON injection fix — deleteMessage was using raw string interpolation for JSON body, now uses kotlinx.serialization
  • Separate stream client — main OkHttpClient has 30s readTimeout (was infinite for ALL requests). Streaming-only streamClient keeps readTimeout(0). Prevents normal API calls from hanging forever
  • Disable backup — android:allowBackup="false". EncryptedSharedPreferences without KeyStore on restore → crash
  • Block cleartext traffic — android:usesCleartextTraffic="false" explicit in manifest
  • Strip server URLs from logs — Log.w calls no longer include $baseUrl, prevents leaking instance URLs in release logcat
  • FLAG_SECURE on sensitive screens — see LATER.md §1B
  • Certificate pinning for default server (ask.kryptomrx.de)

4. Features for parity with the web

  • Attachments — image/file upload (multi-select for gallery + files), attachment preview in messages, display in loaded conversations, camera capture, attachment picker menu (camera/gallery/file)
  • Reasoning/thinking display — ReasoningBlock (collapsible), parses \x01 sentinel
  • Web search sources — SourcesBlock (collapsible, clickable links via ACTION_VIEW), parses \x02/\x04 sentinels. webSearch: true in ChatRequest triggers agentive search. Vertex: Google vs Enterprise Web Search picker (webSearchProvider in request body, dropdown pill only for vertex: models)
  • Tool call display — ToolEventsBlock, parses \x03 sentinel, status icons per step
  • Edit + resend user messages — SquarePen button removes message + all successors from UI + server, puts text back in input field for editing
  • Sidebar search — live case-insensitive title filter with clear button
  • Generate title from sidebar — Sparkles icon in 3-dot menu (only when titleGenerated == false), calls POST .../title with { manual: true }
  • Duplicate conversation — Copy icon in sidebar 3-dot menu, creates new conversation with all messages copied (new IDs + parentId chain)
  • Native passkey login (WebAuthn/FIDO2 on Android)
  • Usage/cost display
  • User context / KI-Profil

5. Voice & Call Mode

v1 — Pipeline (Weg A, uses existing backend):

  • Voice recording + auto-send — Android MediaRecorder (AAC/M4A, 44.1kHz/128kbps), tap mic → live 32-bar waveform visualization (amplitude from getMaxAmplitude() polled 60ms, scrolling bars in accent color) → tap stop → auto-upload + auto-send (no manual send needed). Voice-only user messages render as compact "🎤 Sprachnachricht" pill. Room caching with correct sortOrder after server save
  • TTS read-aloud — Volume button on every assistant message. POST /api/v1/audio-gen { prompt, kind: "speech", voice } → Gemini-TTS → MediaPlayer streams the audio URL. Tap again to stop. Accent-colored icon during playback
  • TTS cache — ttsUrl + ttsVoice fields on MessageEntity (Room v3). First play generates + caches; subsequent plays with same voice skip the API call. Voice change auto-invalidates
  • Server-synced voice — tts_voice column on users table (migration 0030). Read from GET /api/v1/me, written via PATCH /api/v1/me. Web's setCallVoice() syncs to server. App reads on login/resume. Default "Kore"
  • Backend: Bearer-auth audio-gen — /api/audio-gen switched to requireActiveUserOrToken(), v1 re-export at /api/v1/audio-gen
  • Call UI — fullscreen LiveCallOverlay (live/LiveCallOverlay.kt): opaque MeshBackground (frozen blobs), KaizenOrb with amplitude-reactive animation, status text, subtitle (transcript), EndCall button. Status machine: IDLE → CONNECTING → LISTENING → SPEAKING
  • In-app voice picker — settings UI to choose from 30 Gemini-TTS voices (synced to server via PATCH /api/v1/me)

The web's call mode (use-call-mode.ts, call-view.tsx) is the reference implementation for PTT. Live Call uses a completely different architecture (WebSocket relay).

v2 — Gemini Live API (Weg B, DONE):

  • Bidirectional WebSocket audio streaming via Vertex AI Gemini Live API (BidiGenerateContent). Proxied through the backend WS relay (ws-server.ts). App connects to wss://{baseUrl}/ws/live, server relays to Gemini.

Live Call Architecture (App-side, live/ package):

  • LiveClient (live/LiveClient.kt) — OkHttp WebSocket to wss://{baseUrl}/ws/live with Bearer auth. Reuses KaizenApi.baseClient connection pool (readTimeout=0, pingInterval=15s). Auto-reconnect with exponential backoff (500ms–8s, max 5 attempts). Speaks the provider-agnostic JSON protocol (LiveProtocol.kt). Critical: dispose() must NOT call client.dispatcher.executorService.shutdown() — the dispatcher is shared with KaizenApi.baseClient, shutting it down kills all HTTP.
  • LiveProtocol (live/LiveProtocol.kt) — JSON serialization via kotlinx.serialization. Critical: encodeDefaults = true required, otherwise the type field (which has a default value like "setup") is omitted and the server rejects the message as malformed. explicitNulls = false to keep null optionals out of the wire format.
  • AudioPipeline (live/AudioPipeline.kt) — Bidirectional audio: Capture (16kHz PCM16 mono, VOICE_COMMUNICATION source for built-in AEC/AGC/NS, 50ms chunks) + Playback (24kHz PCM16 mono, USAGE_ASSISTANT for full quality, LOW_LATENCY mode, MODE_STREAM). Echo prevention: AcousticEchoCanceler on the AudioRecord session (hardware AEC on Samsung) + hard ducking (zero mic chunks while model speaks). No MODE_IN_COMMUNICATION — it activates telephony DSP that degrades audio quality; not needed since AEC + ducking handle echo. Buffer: minBuf * 2 for playback (lower latency).
  • LiveCallService (live/LiveCallService.kt) — Foreground Service (FOREGROUND_SERVICE_TYPE_MICROPHONE). Keeps mic + WS alive during screen rotation / backgrounding. 20s connect timeout (ends call if server doesn't respond). Error events shown in overlay subtitle + loadError banner. Terminal errors: auth_failed, restricted_model, restricted_feature, budget_exceeded, concurrent_limit, provider_error, reconnect_failed.
  • LiveCallOverlay (live/LiveCallOverlay.kt) — Fullscreen Composable with opaque MeshBackground (not transparent scrim — chat content must NOT bleed through).

Model selection: App sends user's current model (e.g. vertex:gemini-3.5-flash) — the backend WS server automatically maps non-live models to the best available live model (gemini-live-2.5-flash-native-audio). No manual live model selection needed.

Conversation persistence: ChatScreenViewModel.startLiveCallInner creates a conversation if needed before starting the service. Turn text + audio persisted server-side by ws-server.ts on each turn_end event. Messages cached in Room for offline access.

6. Image & media generation

  • Image generation — send prompt to /api/image-gen (needs Bearer auth + v1 re-export), display returned image URL inline. Web uses detectMode("image") + dedicated imageModelId picker; app can start simpler (Bild mode pill → image-gen endpoint)
  • Video generation — /api/video-gen (SSE polling, max 4.5 min), display result inline
  • Audio generation — music (Lyria) + speech (Gemini-TTS) via /api/audio-gen, inline <audio> player

7. App distribution

  • Push notifications (FCM)
  • Play Store / F-Droid listing
  • Auto-update mechanism

8. Offline-cache enhancements (future iterations, see OFFLINE_CACHE_ADD.md §11)

  • Stale-check enforcement — use cachedAt to trigger background refresh when older than N minutes
  • Offline-queue for sends — queue messages locally, send when network returns (needs conflict resolution)
  • Incremental sync — only fetch changed conversations (needs server-side since parameter)
  • Media cache — NetworkImageCache with file-based disk cache (cacheDir/img), 32MB size-based memory LRU, shared KaizenApi.imageClient
  • Full-text search — FTS4/FTS5 over cached messages (Room supports this natively)
  • Attachment queries — "show me all chats with images/PDFs" via Room DB

Dependencies

# gradle/libs.versions.toml (key versions)
agp = "9.2.1"
kotlin = "2.2.10"
ksp = "2.2.10-2.0.2"
room = "2.8.4"
composeBom = "2026.02.01"
okhttp = "4.12.0"
kotlinxSerialization = "1.7.3"
securityCrypto = "1.1.0-alpha06"
biometric = "1.4.0-alpha02"
composeIcons = "1.0.0"           # com.composables:icons-lucide-android (Lucide for Compose)

Tests

Unit tests (app/src/test/): JVM-only, no emulator needed.

  • ChatScreenViewModelTest — 12 tests: initial state, newChat reset, send guards (no config, empty input, while streaming), input binding, mode/reasoning/sampling toggles, screen navigation, password dialog lifecycle, edit message, delete message, file removal, attach menu, TTS voice
  • SettingsViewModelTest — state transitions, all 4 themes, appearance modes, name updates with store-less default
  • StreamConsumerTest — batch + incremental parser, sentinel stripping, reasoning/sources/tools/usage parsing, char-by-char edge cases
  • OklabTest — round-trip conversion, interpolation midpoint, gradient generation (all green since Double-precision fix)
  • MappersTest — ConversationSummary ↔ ConversationEntity, StoredMessage ↔ MessageEntity, round-trip integrity, attachment preservation, reasoning round-trip, null reasoning preserved, sources round-trip, empty sources → null, reasoning+sources together
  • ConvertersTest — List<Attachment> + List<SearchSource> ↔ JSON round-trips, malformed JSON returns empty list, unknown fields tolerance via ignoreUnknownKeys
  • ColorPipelineTest — all tokens in DisplayP3, approximate value ranges, no pure black/white in neutrals
  • SquircleShapeTest — KaizenShapes token existence/distinctness, SquircleShape toString
  • ThemeRegistryTest — 4 themes present, forId lookup, fromString parsing + fallback, distinct blob colors
  • ShadowStackTest — layer counts per level, alpha range validation, blur/alpha progression

Constraints & gotchas

  • Local dev Mac has no RustFS (storage on server only); upload/storage flows fail locally with ECONNREFUSED :9000.
  • Live OpenRouter key is encrypted in the DB (api_keys), .env.local is stale.
  • Never hand-write migrations or fake timestamps — use pnpm db:generate. Always git pull before docker build on prod.
  • App philosophy: the real driver is feel, not features; native Compose (no WebView). Backend complexity stays invisible.
  • Deploy gotcha (learned 2026-06-19): git pull BEFORE docker compose up --build, else the image bakes stale code and the app gets 404/401 against endpoints that exist in the repo but aren't in the running build.
  • Headless compile on Mac: JAVA_HOME=/opt/homebrew/opt/openjdk@21/libexec/openjdk.jdk/Contents/Home ./gradlew :app:compileDebugKotlin — JDK 21 via Homebrew, Android SDK at /opt/homebrew/share/android-commandlinetools.
  • AGP 9.x + KSP: requires android.disallowKotlinSourceSets=false in gradle.properties.
  • Keyboard handling: do NOT use imePadding() (inflates the dock composable, input floats to top) or adjustResize (hides input behind keyboard with edge-to-edge). Instead: read WindowInsets.ime.getBottom() and apply as Modifier.offset(y = -bottomPx.toDp()) on the bottom dock. windowSoftInputMode=adjustNothing in manifest.
  • Room schema changes: bump version in @Database annotation AND use fallbackToDestructiveMigration(true). false only covers downgrades, not same-version hash mismatches → crash.
  • BiometricPrompt context: LocalContext.current may be a ContextWrapper, not FragmentActivity. Walk the wrapper chain: while (ctx is ContextWrapper) { if (ctx is FragmentActivity) break; ctx = ctx.baseContext }.
  • Room is a cache: fallbackToDestructiveMigration() means the SQLite DB is wiped on schema changes. No data loss — server is source of truth, cache rebuilds on next fetch.