Compare commits
3 commits
3032dc869f
...
edd8ccaa05
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
edd8ccaa05 | ||
|
|
3c6dab33a0 | ||
|
|
141d4de227 |
6 changed files with 72 additions and 13 deletions
27
CLAUDE.md
27
CLAUDE.md
|
|
@ -259,6 +259,7 @@ All committed on `main`. Compile-verified on Mac (`./gradlew :app:compileDebugKo
|
|||
| `70aa5c5` | **ChatInput button layout** — Mic (voice) + Plus left, Call/Send toggle right. AudioLines icon for call. SendButton: no border, translucent gradient, 38dp |
|
||||
| `fd0de5c` | **Markdown upgrade** — tables (`\| col \| col \|`), ~~strikethrough~~, task lists (`- [x]`), clickable links (ACTION_VIEW), block comments (`/* */`), HTML comments, template strings, C/C++/Ruby/PHP/Dockerfile/TOML highlighting. All regex pre-compiled |
|
||||
| `bf94414` | **CallButton** — styled to match SendButton: 38dp circle, subtle accent gradient (16-28% alpha), AudioLines icon in accent color, press-scale animation |
|
||||
| `141d4de` | **Stop button + auto-logout + locale fix + login shadow migration** — red square stop button during streaming (cancels stream, preserves partial content), auto-logout on HTTP 401, EmptyHero date respects system locale, LoginScreen shadows migrated to kaizenShadow |
|
||||
|
||||
**Earlier UI/feel work (Phase 0 prototype → feel-first):**
|
||||
- Liquid glass styling, floating panels, glassmorphic sidebar
|
||||
|
|
@ -319,7 +320,7 @@ These are what separate "prototype" from "daily-driver":
|
|||
|
||||
### 3. Security hardening
|
||||
|
||||
- [ ] **Auto-logout on 401** — if the token is rejected, route to login screen instead of showing error banner
|
||||
- [x] **Auto-logout on 401** — if the token is rejected, route to login screen instead of showing error banner
|
||||
- [ ] **Server version check** — call `/api/v1/meta` after login to detect server skew and warn cleanly
|
||||
- [x] **Biometric lock** (`BiometricPrompt` + KeyStore/CryptoObject) — hardware-enforced, password fallback dialog when unavailable, biometric toggle in Settings, all unlock paths gated
|
||||
- [x] **Security settings** — biometric toggle, signed-in devices with remote revoke, password change
|
||||
|
|
@ -334,17 +335,35 @@ These are what separate "prototype" from "daily-driver":
|
|||
- [x] **Web search sources** — `SourcesBlock` (collapsible, clickable links via ACTION_VIEW), parses `\x02`/`\x04` sentinels. `webSearch: true` in ChatRequest triggers agentive search
|
||||
- [x] **Tool call display** — `ToolEventsBlock`, parses `\x03` sentinel, status icons per step
|
||||
- [ ] **Native passkey login** (WebAuthn/FIDO2 on Android)
|
||||
- [ ] **WebSocket streaming** (replace chunked HTTP for lower latency on mobile)
|
||||
- [ ] **Usage/cost display**
|
||||
- [ ] **User context / KI-Profil**
|
||||
|
||||
### 5. App distribution
|
||||
### 5. Voice & Call Mode
|
||||
|
||||
**v1 — Pipeline (Weg A, uses existing backend):**
|
||||
- [ ] **Voice recording + send** — Android `MediaRecorder`, upload via `/api/v1/upload`, send as audio attachment via `/api/v1/chat` (Vertex understands audio natively)
|
||||
- [ ] **TTS playback** — POST response text to `/api/v1/audio-gen` (needs Bearer auth, currently web-session-only via `requireActiveUser`), play returned audio URL via `MediaPlayer`/`ExoPlayer`
|
||||
- [ ] **Call UI** — fullscreen overlay, KaizenOrb with amplitude-reactive animation, push-to-talk mic button, status machine (idle → recording → uploading → thinking → speaking), voice picker (30 Gemini-TTS voices)
|
||||
- [ ] **Backend: Bearer-auth audio-gen** — switch `/api/audio-gen` from `requireActiveUser()` to `requireActiveUserOrToken()`, add v1 re-export at `/api/v1/audio-gen`
|
||||
|
||||
The web's call mode (`use-call-mode.ts`, `call-view.tsx`) is the reference implementation. Same pipeline: Record → Upload → Chat (with audio attachment) → Text stream → TTS → Playback. No WebSocket needed.
|
||||
|
||||
**v2 — Gemini Live API (Weg B, target architecture):**
|
||||
- [ ] **Bidirectional WebSocket audio streaming** via Vertex AI Gemini Live API (`BidiGenerateContent`). Available on Vertex AI / enterprise Gemini (GCP), not just the consumer API. True real-time conversation: sub-second latency, server-side VAD, user can interrupt the model mid-response. Requires either proxying through the backend (WebSocket relay) or direct app→Vertex connection with credential delegation. History sync and cost tracking need a separate solution since the conversation bypasses the normal chat persistence pipeline.
|
||||
|
||||
### 6. Image & media generation
|
||||
|
||||
- [ ] **Image generation** — send prompt to `/api/image-gen` (needs Bearer auth + v1 re-export), display returned image URL inline. Web uses `detectMode("image")` + dedicated `imageModelId` picker; app can start simpler (Bild mode pill → image-gen endpoint)
|
||||
- [ ] **Video generation** — `/api/video-gen` (SSE polling, max 4.5 min), display result inline
|
||||
- [ ] **Audio generation** — music (Lyria) + speech (Gemini-TTS) via `/api/audio-gen`, inline `<audio>` player
|
||||
|
||||
### 7. App distribution
|
||||
|
||||
- [ ] **Push notifications** (FCM)
|
||||
- [ ] **Play Store / F-Droid listing**
|
||||
- [ ] **Auto-update mechanism**
|
||||
|
||||
### 6. Offline-cache enhancements (future iterations, see `OFFLINE_CACHE_ADD.md` §11)
|
||||
### 8. Offline-cache enhancements (future iterations, see `OFFLINE_CACHE_ADD.md` §11)
|
||||
|
||||
- [ ] **Stale-check enforcement** — use `cachedAt` to trigger background refresh when older than N minutes
|
||||
- [ ] **Offline-queue for sends** — queue messages locally, send when network returns (needs conflict resolution)
|
||||
|
|
|
|||
|
|
@ -33,7 +33,6 @@ import androidx.compose.runtime.setValue
|
|||
import androidx.compose.ui.Alignment
|
||||
import androidx.compose.ui.Modifier
|
||||
import androidx.compose.ui.draw.clip
|
||||
import androidx.compose.ui.draw.shadow
|
||||
import androidx.compose.ui.graphics.Brush
|
||||
import androidx.compose.ui.graphics.Color
|
||||
import androidx.compose.ui.graphics.SolidColor
|
||||
|
|
@ -49,6 +48,8 @@ import dev.kaizen.app.chat.MeshBackground
|
|||
import dev.kaizen.app.net.DEFAULT_BASE_URL
|
||||
import dev.kaizen.app.net.LoginResult
|
||||
import dev.kaizen.app.net.SessionViewModel
|
||||
import dev.kaizen.app.ui.effect.KaizenShadows
|
||||
import dev.kaizen.app.ui.effect.kaizenShadow
|
||||
import dev.kaizen.app.ui.theme.LocalKaizenAccent
|
||||
import androidx.compose.ui.res.stringResource
|
||||
import dev.kaizen.app.R
|
||||
|
|
@ -181,7 +182,7 @@ private fun GlassField(
|
|||
Modifier
|
||||
.fillMaxWidth()
|
||||
.widthIn(max = 420.dp)
|
||||
.shadow(elevation = 6.dp, shape = KaizenShapes.lg, clip = false)
|
||||
.kaizenShadow(stack = KaizenShadows.level1, cornerRadius = 20.dp)
|
||||
.clip(KaizenShapes.lg)
|
||||
.background(glassBg)
|
||||
.border(1.2.dp, glassBorder, KaizenShapes.lg)
|
||||
|
|
@ -220,7 +221,7 @@ private fun SignInButton(enabled: Boolean, loading: Boolean, onClick: () -> Unit
|
|||
.fillMaxWidth()
|
||||
.widthIn(max = 420.dp)
|
||||
.heightIn(min = 54.dp)
|
||||
.shadow(elevation = 8.dp, shape = KaizenShapes.lg, clip = false)
|
||||
.kaizenShadow(stack = KaizenShadows.level2, cornerRadius = 20.dp)
|
||||
.clip(KaizenShapes.lg)
|
||||
.then(fill)
|
||||
.clickable(enabled = enabled, onClick = onClick)
|
||||
|
|
|
|||
|
|
@ -121,7 +121,7 @@ fun EmptyHero(userName: String, onPick: (String) -> Unit, modifier: Modifier = M
|
|||
val cs = MaterialTheme.colorScheme
|
||||
val now = remember { LocalDateTime.now() }
|
||||
val dateLine = remember {
|
||||
now.format(DateTimeFormatter.ofPattern("EEE, d. MMMM · HH:mm", Locale.GERMAN))
|
||||
now.format(DateTimeFormatter.ofPattern("EEE, d. MMMM · HH:mm", Locale.getDefault()))
|
||||
}
|
||||
|
||||
Column(
|
||||
|
|
@ -525,6 +525,8 @@ fun ChatInput(
|
|||
onValueChange: (String) -> Unit,
|
||||
onSend: () -> Unit,
|
||||
enabled: Boolean,
|
||||
isStreaming: Boolean = false,
|
||||
onStop: () -> Unit = {},
|
||||
pendingFiles: List<PendingFile> = emptyList(),
|
||||
onAddClick: () -> Unit = {},
|
||||
onRemoveFile: (String) -> Unit = {},
|
||||
|
|
@ -590,7 +592,9 @@ fun ChatInput(
|
|||
|
||||
Spacer(Modifier.weight(1f))
|
||||
|
||||
if (hasContent) {
|
||||
if (isStreaming) {
|
||||
StopButton(onClick = onStop)
|
||||
} else if (hasContent) {
|
||||
SendButton(enabled = canSend, onClick = onSend)
|
||||
} else {
|
||||
CallButton(onClick = { /* TODO: call feature */ })
|
||||
|
|
@ -702,6 +706,26 @@ private fun CallButton(onClick: () -> Unit) {
|
|||
}
|
||||
}
|
||||
|
||||
@Composable
|
||||
private fun StopButton(onClick: () -> Unit) {
|
||||
val isDark = isSystemInDarkTheme()
|
||||
val interaction = remember { MutableInteractionSource() }
|
||||
val pressed by interaction.collectIsPressedAsState()
|
||||
val pressScale by animateFloatAsState(if (pressed) 0.88f else 1f, label = "stop")
|
||||
|
||||
Box(
|
||||
Modifier
|
||||
.size(38.dp)
|
||||
.scale(pressScale)
|
||||
.clip(KaizenShapes.circle)
|
||||
.background(Color(0xFFEF4444).copy(alpha = if (isDark) 0.85f else 0.80f))
|
||||
.clickable(interactionSource = interaction, indication = null, onClick = onClick),
|
||||
contentAlignment = Alignment.Center,
|
||||
) {
|
||||
Icon(KaizenIcons.Square, stringResource(R.string.chat_stop), tint = Color.White, modifier = Modifier.size(16.dp))
|
||||
}
|
||||
}
|
||||
|
||||
// ── Attachment rendering ────────────────────────────────────────────────────
|
||||
|
||||
private object NetworkImageCache {
|
||||
|
|
|
|||
|
|
@ -82,6 +82,7 @@ import androidx.activity.result.contract.ActivityResultContracts
|
|||
import androidx.compose.ui.platform.LocalContext
|
||||
import androidx.compose.ui.res.stringResource
|
||||
import androidx.lifecycle.compose.LifecycleResumeEffect
|
||||
import kotlinx.coroutines.Job
|
||||
import kotlinx.coroutines.async
|
||||
import kotlinx.coroutines.launch
|
||||
import java.io.ByteArrayOutputStream
|
||||
|
|
@ -108,6 +109,7 @@ fun ChatScreen(
|
|||
var samplingPrefs by remember { mutableStateOf(SamplingPrefs()) }
|
||||
var usedTokens by remember { mutableStateOf(0) }
|
||||
var chatModel by remember { mutableStateOf<String?>(null) }
|
||||
var streamJob by remember { mutableStateOf<Job?>(null) }
|
||||
|
||||
// Navigation State
|
||||
var currentScreen by remember { mutableStateOf(AppScreen.Chat) }
|
||||
|
|
@ -199,7 +201,10 @@ fun ChatScreen(
|
|||
models = r.data
|
||||
if (r.data.isNotEmpty()) session.store.saveModelsCache(r.data)
|
||||
}
|
||||
is FetchResult.Fail -> errors.add(r.reason)
|
||||
is FetchResult.Fail -> {
|
||||
if ("HTTP 401" in r.reason) { session.logout(); return@LaunchedEffect }
|
||||
errors.add(r.reason)
|
||||
}
|
||||
}
|
||||
KaizenApi.fetchMe(cfg.baseUrl, cfg.token)?.let { me ->
|
||||
me.name?.let { settingsViewModel.updateUserName(it) }
|
||||
|
|
@ -395,7 +400,7 @@ fun ChatScreen(
|
|||
.filter { it.id != assistantId }
|
||||
.map { WireMessage(if (it.role == Role.User) "user" else "assistant", it.content) }
|
||||
|
||||
scope.launch {
|
||||
streamJob = scope.launch {
|
||||
haptics.thinkingStart()
|
||||
var sawContent = false
|
||||
try {
|
||||
|
|
@ -490,7 +495,12 @@ fun ChatScreen(
|
|||
}
|
||||
} catch (e: Exception) {
|
||||
val idx = messages.indexOfLast { it.id == assistantId }
|
||||
if (idx >= 0) {
|
||||
if (e is kotlinx.coroutines.CancellationException) {
|
||||
if (idx >= 0) messages[idx] = messages[idx].copy(streaming = false, thinking = false)
|
||||
} else if (e is ChatHttpException && e.code == 401) {
|
||||
if (idx >= 0) messages[idx] = messages[idx].copy(streaming = false, thinking = false)
|
||||
session.logout()
|
||||
} else if (idx >= 0) {
|
||||
val existing = messages[idx].content
|
||||
val errorSuffix = "\n\n⚠ ${chatErrorText(e, context)}"
|
||||
messages[idx] = messages[idx].copy(
|
||||
|
|
@ -501,6 +511,7 @@ fun ChatScreen(
|
|||
}
|
||||
} finally {
|
||||
isStreaming = false
|
||||
streamJob = null
|
||||
haptics.responseEnd()
|
||||
}
|
||||
}
|
||||
|
|
@ -624,7 +635,7 @@ fun ChatScreen(
|
|||
.then(if (isTablet) Modifier.widthIn(max = 720.dp) else Modifier),
|
||||
contentPadding = PaddingValues(
|
||||
top = 76.dp, // Leaves breathing room for the floating top menu button
|
||||
bottom = 152.dp, // Leaves luxurious room for the floating ModePills + ChatInput + bottom margins
|
||||
bottom = 220.dp, // Clears the floating ModePills + two-row ChatInput + bottom margins
|
||||
start = 16.dp,
|
||||
end = 16.dp
|
||||
),
|
||||
|
|
@ -845,6 +856,8 @@ fun ChatScreen(
|
|||
onValueChange = { input = it },
|
||||
onSend = { send(input) },
|
||||
enabled = !isStreaming,
|
||||
isStreaming = isStreaming,
|
||||
onStop = { streamJob?.cancel() },
|
||||
pendingFiles = pendingFiles,
|
||||
onAddClick = { showAttachMenu = !showAttachMenu },
|
||||
onRemoveFile = { id -> pendingFiles.removeAll { it.id == id } },
|
||||
|
|
|
|||
|
|
@ -18,6 +18,7 @@
|
|||
<string name="chat_remove">Remove</string>
|
||||
<string name="chat_error">Error</string>
|
||||
<string name="chat_open_menu">Open menu</string>
|
||||
<string name="chat_stop">Stop response</string>
|
||||
<string name="attach_camera">Take Photo</string>
|
||||
<string name="attach_gallery">Photo & Video</string>
|
||||
<string name="attach_file">Choose File</string>
|
||||
|
|
|
|||
|
|
@ -20,6 +20,7 @@
|
|||
<string name="chat_remove">Entfernen</string>
|
||||
<string name="chat_error">Fehler</string>
|
||||
<string name="chat_open_menu">Menü öffnen</string>
|
||||
<string name="chat_stop">Antwort stoppen</string>
|
||||
<string name="attach_camera">Foto aufnehmen</string>
|
||||
<string name="attach_gallery">Foto & Video</string>
|
||||
<string name="attach_file">Datei auswählen</string>
|
||||
|
|
|
|||
Loading…
Reference in a new issue