Compare commits
4 commits
edd8ccaa05
...
2f9ada646c
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2f9ada646c | ||
|
|
1645be8a97 | ||
|
|
c4884a3748 | ||
|
|
9e1f30de1f |
12 changed files with 246 additions and 9 deletions
28
CLAUDE.md
28
CLAUDE.md
|
|
@ -26,9 +26,10 @@ Make the **native Android app** (`kaizen-app`) a valid daily-driver that talks t
|
|||
- **Sentinels** (`lib/chat/constants.ts`): `\x00` USAGE, `\x01` REASONING, `\x02` SOURCES, `\x03` TOOL, `\x04` QUERY. App currently strips all sentinels and renders only visible content.
|
||||
- **Models:** `GET {baseUrl}/api/v1/models` → `{ models: [{ id, name?, provider? }] }`.
|
||||
- **Conversations:** `GET /api/v1/conversations` → `{ conversations, hasMore }`. `POST /api/v1/conversations` → create. `GET /api/v1/conversations/[id]` → `{ messages }`. `POST /api/v1/conversations/[id]/messages` → persist. `POST /api/v1/conversations/[id]/title` → auto-title.
|
||||
- **Identity:** `GET /api/v1/me` → `{ id, email, name, role, defaultModel }`.
|
||||
- **Identity:** `GET /api/v1/me` → `{ id, email, name, role, defaultModel, ttsVoice }`. `PATCH /api/v1/me` with `{ ttsVoice }` to update preferences.
|
||||
- **Capability handshake:** `GET /api/v1/meta` → `{ app, apiVersion, features }` (unauth).
|
||||
- **Upload:** `POST /api/v1/upload` — multipart file → `{ url, kind, name, mimeType, size, fileId }`.
|
||||
- **Audio gen:** `POST /api/v1/audio-gen` — `{ prompt, kind: "speech"|"music", voice? }` → `{ url, cost }`. Gemini-TTS (speech) or Lyria (music) via Vertex. Audio persisted in Object Storage.
|
||||
|
||||
## App architecture
|
||||
|
||||
|
|
@ -195,13 +196,16 @@ All backend work for the app is done. `pnpm test:run` green (774+), `tsc` + `lin
|
|||
- **`api_tokens` table** + SHA-256 hash storage (never raw). Migration `0024`.
|
||||
- **Bearer auth path** (`getTokenSession` / `requireActiveUserOrToken` in `lib/auth/guard.ts`).
|
||||
- **Token mint endpoint** `POST /api/v1/auth/token` — unauthenticated, Argon2id verify, anti-enumeration, rate-limited.
|
||||
- **`GET /api/v1/me`** — identity + `defaultModel` for native clients.
|
||||
- **`GET /api/v1/me`** — identity + `defaultModel` + `ttsVoice` for native clients.
|
||||
- **`PATCH /api/v1/me`** — update user preferences (`ttsVoice`, etc.). Bearer-authed.
|
||||
- **`GET /api/v1/models`** — Bearer-authed, returns filtered model catalog.
|
||||
- **`/api/v1/conversations`** — thin re-exports of canonical handlers, Bearer-authed.
|
||||
- **`POST /api/v1/auth/unlock`** — Bearer-authed, returns unlock token in response body (not cookie) for native clients. Rate-limited 10/5min per user.
|
||||
- **`GET /api/v1/conversations/[id]`** — accepts `X-Unlock-Token` header alongside the existing `kaizen_unlock` cookie for locked conversations.
|
||||
- **`GET /api/v1/meta`** — capability handshake.
|
||||
- **App-devices card** (`components/settings/app-tokens-card.tsx`) — see/revoke signed-in devices on `/settings/security`.
|
||||
- **`POST /api/v1/audio-gen`** — Gemini-TTS speech generation. Bearer-authed (re-export of `/api/audio-gen`). `{ prompt, kind: "speech", voice? }` → `{ url, cost }`.
|
||||
- **`tts_voice` column** on `users` table (migration 0030). Server-synced voice preference, read via `/api/v1/me`, written via `PATCH /api/v1/me`.
|
||||
- **API versioning** — `/api/v1/` for outward contract, additive-only within v1. Web-internal endpoints stay unversioned.
|
||||
|
||||
### App — shipped features (on-device verified on S25 Ultra)
|
||||
|
|
@ -260,6 +264,10 @@ All committed on `main`. Compile-verified on Mac (`./gradlew :app:compileDebugKo
|
|||
| `fd0de5c` | **Markdown upgrade** — tables (`\| col \| col \|`), ~~strikethrough~~, task lists (`- [x]`), clickable links (ACTION_VIEW), block comments (`/* */`), HTML comments, template strings, C/C++/Ruby/PHP/Dockerfile/TOML highlighting. All regex pre-compiled |
|
||||
| `bf94414` | **CallButton** — styled to match SendButton: 38dp circle, subtle accent gradient (16-28% alpha), AudioLines icon in accent color, press-scale animation |
|
||||
| `141d4de` | **Stop button + auto-logout + locale fix + login shadow migration** — red square stop button during streaming (cancels stream, preserves partial content), auto-logout on HTTP 401, EmptyHero date respects system locale, LoginScreen shadows migrated to kaizenShadow |
|
||||
| `edd8cca` | **Bottom padding fix** — LazyColumn bottom contentPadding 152→220dp, fixes messages overlapping mode pills after ChatInput two-row redesign |
|
||||
| `9e1f30d` | **Voice recording + TTS read-aloud** — Mic button wired to `MediaRecorder` (AAC/M4A), tap-to-record, auto-upload as audio attachment. Volume button on assistant messages → `POST /api/v1/audio-gen` → Gemini-TTS → `MediaPlayer` playback. `RECORD_AUDIO` permission |
|
||||
| `c4884a3` | **TTS audio cache in Room** — `ttsUrl` + `ttsVoice` on `MessageEntity` (Room v3), skip API call on replay with same voice. Voice change invalidates cache |
|
||||
| `1645be8` | **Server-synced TTS voice** — reads `ttsVoice` from `GET /api/v1/me`, uses it for all TTS calls. Voice set in web settings applies in app automatically |
|
||||
|
||||
**Earlier UI/feel work (Phase 0 prototype → feel-first):**
|
||||
- Liquid glass styling, floating panels, glassmorphic sidebar
|
||||
|
|
@ -298,6 +306,13 @@ All committed on `main`. Compile-verified on Mac (`./gradlew :app:compileDebugKo
|
|||
- **Sidebar flicker on refresh** — `replaceAll()` did `DELETE ALL` + `INSERT`, causing Room Flow to emit empty list briefly. Fix: `upsert` + `deleteExcept` (no flash).
|
||||
- **Gallery/file picker single-select only** — `GetContent()` allowed only one item. Fix: gallery → `PickMultipleVisualMedia(ImageAndVideo)`, file → `OpenMultipleDocuments(["*/*"])`.
|
||||
|
||||
### Bugs fixed (2026-06-23 — voice + UX session)
|
||||
|
||||
- **Messages overlapping mode pills** — LazyColumn bottom `contentPadding` was 152dp but the two-row ChatInput redesign made the dock ~210dp tall. Fix: increased to 220dp.
|
||||
- **EmptyHero date hardcoded German** — `Locale.GERMAN` → `Locale.getDefault()`, now respects system/app language.
|
||||
- **LoginScreen used `Modifier.shadow`** — violated design system pillar 4 (multi-layer shadows). Fix: migrated to `kaizenShadow` (level1 for fields, level2 for button).
|
||||
- **Migration 0030 crash** — `pnpm db:generate` bundled the entire org schema (0026-0029) + `tts_voice` into one migration, causing duplicate CREATE TABLE / ADD COLUMN failures. Fix: reduced 0030 to only `ALTER TABLE users ADD COLUMN IF NOT EXISTS tts_voice text`.
|
||||
|
||||
## What's NOT done yet (app, priority order)
|
||||
|
||||
### 1. Remaining visual/UX gaps (daily-drive blockers)
|
||||
|
|
@ -341,10 +356,13 @@ These are what separate "prototype" from "daily-driver":
|
|||
### 5. Voice & Call Mode
|
||||
|
||||
**v1 — Pipeline (Weg A, uses existing backend):**
|
||||
- [ ] **Voice recording + send** — Android `MediaRecorder`, upload via `/api/v1/upload`, send as audio attachment via `/api/v1/chat` (Vertex understands audio natively)
|
||||
- [ ] **TTS playback** — POST response text to `/api/v1/audio-gen` (needs Bearer auth, currently web-session-only via `requireActiveUser`), play returned audio URL via `MediaPlayer`/`ExoPlayer`
|
||||
- [x] **Voice recording + send** — Android `MediaRecorder` (AAC/M4A, 44.1kHz/128kbps), tap-to-record mic button (red stop-square while recording), auto-upload via `/api/v1/upload`, sent as audio attachment via `/api/v1/chat`. Vertex understands audio natively; OpenRouter passes it as `input_audio`. Persisted in DB as normal attachment
|
||||
- [x] **TTS read-aloud** — Volume button on every assistant message. `POST /api/v1/audio-gen { prompt, kind: "speech", voice }` → Gemini-TTS → `MediaPlayer` streams the audio URL. Tap again to stop. Accent-colored icon during playback
|
||||
- [x] **TTS cache** — `ttsUrl` + `ttsVoice` fields on `MessageEntity` (Room v3). First play generates + caches; subsequent plays with same voice skip the API call. Voice change auto-invalidates
|
||||
- [x] **Server-synced voice** — `tts_voice` column on `users` table (migration 0030). Read from `GET /api/v1/me`, written via `PATCH /api/v1/me`. Web's `setCallVoice()` syncs to server. App reads on login/resume. Default "Kore"
|
||||
- [x] **Backend: Bearer-auth audio-gen** — `/api/audio-gen` switched to `requireActiveUserOrToken()`, v1 re-export at `/api/v1/audio-gen`
|
||||
- [ ] **Call UI** — fullscreen overlay, KaizenOrb with amplitude-reactive animation, push-to-talk mic button, status machine (idle → recording → uploading → thinking → speaking), voice picker (30 Gemini-TTS voices)
|
||||
- [ ] **Backend: Bearer-auth audio-gen** — switch `/api/audio-gen` from `requireActiveUser()` to `requireActiveUserOrToken()`, add v1 re-export at `/api/v1/audio-gen`
|
||||
- [ ] **In-app voice picker** — settings UI to choose from 30 Gemini-TTS voices (synced to server via `PATCH /api/v1/me`)
|
||||
|
||||
The web's call mode (`use-call-mode.ts`, `call-view.tsx`) is the reference implementation. Same pipeline: Record → Upload → Chat (with audio attachment) → Text stream → TTS → Playback. No WebSocket needed.
|
||||
|
||||
|
|
|
|||
|
|
@ -8,6 +8,9 @@
|
|||
<!-- Required to talk to the Kaizen backend (login + chat streaming) -->
|
||||
<uses-permission android:name="android.permission.INTERNET" />
|
||||
|
||||
<!-- Required for voice recording (mic button in chat) -->
|
||||
<uses-permission android:name="android.permission.RECORD_AUDIO" />
|
||||
|
||||
<application
|
||||
android:allowBackup="true"
|
||||
android:dataExtractionRules="@xml/data_extraction_rules"
|
||||
|
|
|
|||
|
|
@ -331,6 +331,8 @@ fun MessageRow(
|
|||
onCopy: (String) -> Unit = {},
|
||||
onDelete: ((String) -> Unit)? = null,
|
||||
onRegenerate: ((String) -> Unit)? = null,
|
||||
onReadAloud: ((String) -> Unit)? = null,
|
||||
isPlayingTts: Boolean = false,
|
||||
) {
|
||||
val cs = MaterialTheme.colorScheme
|
||||
val isDark = isSystemInDarkTheme()
|
||||
|
|
@ -461,6 +463,14 @@ fun MessageRow(
|
|||
copied = false
|
||||
}
|
||||
}
|
||||
onReadAloud?.let { readAloud ->
|
||||
MessageActionButton(
|
||||
icon = KaizenIcons.Volume2,
|
||||
tint = if (isPlayingTts) accent.primary else cs.onSurfaceVariant.copy(alpha = 0.35f),
|
||||
) {
|
||||
haptics.tick(); readAloud(message.content)
|
||||
}
|
||||
}
|
||||
onRegenerate?.let { regen ->
|
||||
MessageActionButton(KaizenIcons.RefreshCw, cs.onSurfaceVariant.copy(alpha = 0.35f)) {
|
||||
haptics.tick(); regen(message.wireId)
|
||||
|
|
@ -527,6 +537,8 @@ fun ChatInput(
|
|||
enabled: Boolean,
|
||||
isStreaming: Boolean = false,
|
||||
onStop: () -> Unit = {},
|
||||
isRecording: Boolean = false,
|
||||
onMicClick: () -> Unit = {},
|
||||
pendingFiles: List<PendingFile> = emptyList(),
|
||||
onAddClick: () -> Unit = {},
|
||||
onRemoveFile: (String) -> Unit = {},
|
||||
|
|
@ -584,10 +596,16 @@ fun ChatInput(
|
|||
}
|
||||
Spacer(Modifier.width(8.dp))
|
||||
Box(
|
||||
Modifier.size(38.dp).clip(KaizenShapes.circle).background(iconBg),
|
||||
Modifier.size(38.dp).clip(KaizenShapes.circle)
|
||||
.background(if (isRecording) Color(0xFFEF4444).copy(alpha = if (isDark) 0.85f else 0.80f) else iconBg)
|
||||
.clickable(onClick = onMicClick),
|
||||
contentAlignment = Alignment.Center,
|
||||
) {
|
||||
Icon(KaizenIcons.Mic, stringResource(R.string.chat_attachment), tint = cs.onSurfaceVariant, modifier = Modifier.size(19.dp))
|
||||
if (isRecording) {
|
||||
Icon(KaizenIcons.Square, stringResource(R.string.chat_recording), tint = Color.White, modifier = Modifier.size(14.dp))
|
||||
} else {
|
||||
Icon(KaizenIcons.Mic, stringResource(R.string.chat_attachment), tint = cs.onSurfaceVariant, modifier = Modifier.size(19.dp))
|
||||
}
|
||||
}
|
||||
|
||||
Spacer(Modifier.weight(1f))
|
||||
|
|
|
|||
|
|
@ -82,10 +82,17 @@ import androidx.activity.result.contract.ActivityResultContracts
|
|||
import androidx.compose.ui.platform.LocalContext
|
||||
import androidx.compose.ui.res.stringResource
|
||||
import androidx.lifecycle.compose.LifecycleResumeEffect
|
||||
import android.Manifest
|
||||
import android.content.pm.PackageManager
|
||||
import android.media.MediaPlayer
|
||||
import android.media.MediaRecorder
|
||||
import android.widget.Toast
|
||||
import androidx.core.content.ContextCompat
|
||||
import kotlinx.coroutines.Job
|
||||
import kotlinx.coroutines.async
|
||||
import kotlinx.coroutines.launch
|
||||
import java.io.ByteArrayOutputStream
|
||||
import java.io.File
|
||||
|
||||
// Screen enumeration for modular state-driven navigation (Zero Technical Debt)
|
||||
enum class AppScreen { Chat, Settings }
|
||||
|
|
@ -111,6 +118,16 @@ fun ChatScreen(
|
|||
var chatModel by remember { mutableStateOf<String?>(null) }
|
||||
var streamJob by remember { mutableStateOf<Job?>(null) }
|
||||
|
||||
// Audio recording state
|
||||
var isRecording by remember { mutableStateOf(false) }
|
||||
val recorderRef = remember { mutableStateOf<MediaRecorder?>(null) }
|
||||
val audioFileRef = remember { mutableStateOf<File?>(null) }
|
||||
|
||||
// TTS playback state
|
||||
var playingTtsForId by remember { mutableStateOf<Long?>(null) }
|
||||
val ttsPlayerRef = remember { mutableStateOf<MediaPlayer?>(null) }
|
||||
var ttsVoice by remember { mutableStateOf("Kore") }
|
||||
|
||||
// Navigation State
|
||||
var currentScreen by remember { mutableStateOf(AppScreen.Chat) }
|
||||
|
||||
|
|
@ -191,6 +208,78 @@ fun ChatScreen(
|
|||
uploadPickedFile("photo_${System.currentTimeMillis()}.jpg", "image/jpeg", stream.toByteArray())
|
||||
}
|
||||
|
||||
val micPermission = rememberLauncherForActivityResult(ActivityResultContracts.RequestPermission()) { granted ->
|
||||
if (granted) startRecording(context, recorderRef, audioFileRef) { isRecording = true }
|
||||
else Toast.makeText(context, context.getString(R.string.chat_mic_permission), Toast.LENGTH_SHORT).show()
|
||||
}
|
||||
|
||||
fun toggleRecording() {
|
||||
if (isRecording) {
|
||||
stopRecording(recorderRef, audioFileRef) { name, mime, bytes ->
|
||||
isRecording = false
|
||||
uploadPickedFile(name, mime, bytes)
|
||||
}
|
||||
} else {
|
||||
if (ContextCompat.checkSelfPermission(context, Manifest.permission.RECORD_AUDIO) == PackageManager.PERMISSION_GRANTED) {
|
||||
startRecording(context, recorderRef, audioFileRef) { isRecording = true }
|
||||
} else {
|
||||
micPermission.launch(Manifest.permission.RECORD_AUDIO)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fun playTtsUrl(url: String, messageId: Long) {
|
||||
try {
|
||||
val player = MediaPlayer().apply {
|
||||
setDataSource(url)
|
||||
prepareAsync()
|
||||
setOnPreparedListener { start() }
|
||||
setOnCompletionListener {
|
||||
release()
|
||||
ttsPlayerRef.value = null
|
||||
playingTtsForId = null
|
||||
}
|
||||
setOnErrorListener { _, _, _ ->
|
||||
release()
|
||||
ttsPlayerRef.value = null
|
||||
playingTtsForId = null
|
||||
true
|
||||
}
|
||||
}
|
||||
ttsPlayerRef.value = player
|
||||
} catch (_: Exception) {
|
||||
playingTtsForId = null
|
||||
}
|
||||
}
|
||||
|
||||
fun readAloud(content: String, messageId: Long, wireId: String) {
|
||||
val cfg = session.config ?: return
|
||||
if (playingTtsForId == messageId) {
|
||||
ttsPlayerRef.value?.release()
|
||||
ttsPlayerRef.value = null
|
||||
playingTtsForId = null
|
||||
return
|
||||
}
|
||||
ttsPlayerRef.value?.release()
|
||||
playingTtsForId = messageId
|
||||
val voice = ttsVoice
|
||||
scope.launch {
|
||||
val cached = chat.messageDao.getTtsUrl(wireId, voice)
|
||||
if (cached != null) {
|
||||
playTtsUrl(cached, messageId)
|
||||
return@launch
|
||||
}
|
||||
val url = KaizenApi.generateSpeech(cfg.baseUrl, cfg.token, content, voice)
|
||||
if (url == null) {
|
||||
playingTtsForId = null
|
||||
Toast.makeText(context, context.getString(R.string.chat_tts_error), Toast.LENGTH_SHORT).show()
|
||||
return@launch
|
||||
}
|
||||
chat.messageDao.updateTts(wireId, url, voice)
|
||||
playTtsUrl(url, messageId)
|
||||
}
|
||||
}
|
||||
|
||||
LaunchedEffect(session.config?.baseUrl, session.config?.token) {
|
||||
val cfg = session.config ?: return@LaunchedEffect
|
||||
loadError = null
|
||||
|
|
@ -209,6 +298,7 @@ fun ChatScreen(
|
|||
KaizenApi.fetchMe(cfg.baseUrl, cfg.token)?.let { me ->
|
||||
me.name?.let { settingsViewModel.updateUserName(it) }
|
||||
me.email?.let { settingsViewModel.updateUserEmail(it) }
|
||||
me.ttsVoice?.let { ttsVoice = it }
|
||||
}
|
||||
chat.conversationRepo.refresh(cfg.baseUrl, cfg.token)
|
||||
if (errors.isNotEmpty()) {
|
||||
|
|
@ -659,6 +749,10 @@ fun ChatScreen(
|
|||
send(userContent)
|
||||
}
|
||||
} else null,
|
||||
onReadAloud = if (message.role == Role.Assistant && !message.streaming && message.content.isNotBlank()) { content ->
|
||||
readAloud(content, message.id, message.wireId)
|
||||
} else null,
|
||||
isPlayingTts = playingTtsForId == message.id,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
|
@ -858,6 +952,8 @@ fun ChatScreen(
|
|||
enabled = !isStreaming,
|
||||
isStreaming = isStreaming,
|
||||
onStop = { streamJob?.cancel() },
|
||||
isRecording = isRecording,
|
||||
onMicClick = { toggleRecording() },
|
||||
pendingFiles = pendingFiles,
|
||||
onAddClick = { showAttachMenu = !showAttachMenu },
|
||||
onRemoveFile = { id -> pendingFiles.removeAll { it.id == id } },
|
||||
|
|
@ -950,3 +1046,53 @@ private fun chatErrorText(e: Throwable, ctx: android.content.Context): String =
|
|||
e is ChatHttpException -> ctx.getString(R.string.error_server, e.code)
|
||||
else -> ctx.getString(R.string.error_no_connection)
|
||||
}
|
||||
|
||||
@Suppress("DEPRECATION")
|
||||
private fun startRecording(
|
||||
context: android.content.Context,
|
||||
recorderRef: androidx.compose.runtime.MutableState<MediaRecorder?>,
|
||||
audioFileRef: androidx.compose.runtime.MutableState<File?>,
|
||||
onStarted: () -> Unit,
|
||||
) {
|
||||
try {
|
||||
val file = File(context.cacheDir, "voice_${System.currentTimeMillis()}.m4a")
|
||||
val recorder = MediaRecorder(context).apply {
|
||||
setAudioSource(MediaRecorder.AudioSource.MIC)
|
||||
setOutputFormat(MediaRecorder.OutputFormat.MPEG_4)
|
||||
setAudioEncoder(MediaRecorder.AudioEncoder.AAC)
|
||||
setAudioSamplingRate(44100)
|
||||
setAudioEncodingBitRate(128000)
|
||||
setOutputFile(file.absolutePath)
|
||||
prepare()
|
||||
start()
|
||||
}
|
||||
recorderRef.value = recorder
|
||||
audioFileRef.value = file
|
||||
onStarted()
|
||||
} catch (_: Exception) {
|
||||
recorderRef.value = null
|
||||
audioFileRef.value = null
|
||||
}
|
||||
}
|
||||
|
||||
private fun stopRecording(
|
||||
recorderRef: androidx.compose.runtime.MutableState<MediaRecorder?>,
|
||||
audioFileRef: androidx.compose.runtime.MutableState<File?>,
|
||||
onRecorded: (String, String, ByteArray) -> Unit,
|
||||
) {
|
||||
val recorder = recorderRef.value ?: return
|
||||
val file = audioFileRef.value ?: return
|
||||
try {
|
||||
recorder.stop()
|
||||
recorder.release()
|
||||
} catch (_: Exception) {
|
||||
recorder.release()
|
||||
}
|
||||
recorderRef.value = null
|
||||
audioFileRef.value = null
|
||||
if (file.exists() && file.length() > 0) {
|
||||
val bytes = file.readBytes()
|
||||
file.delete()
|
||||
onRecorded("voice_${System.currentTimeMillis()}.m4a", "audio/mp4", bytes)
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -8,5 +8,6 @@ import dev.kaizen.app.db.MessageRepository
|
|||
class ChatViewModel(context: Context) {
|
||||
private val db = KaizenDatabase.get(context)
|
||||
val conversationRepo = ConversationRepository(db.conversationDao())
|
||||
val messageRepo = MessageRepository(db.messageDao())
|
||||
val messageDao = db.messageDao()
|
||||
val messageRepo = MessageRepository(messageDao)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -36,4 +36,6 @@ data class MessageEntity(
|
|||
val sortOrder: Int,
|
||||
val reasoning: String? = null,
|
||||
val sources: List<SearchSource> = emptyList(),
|
||||
val ttsUrl: String? = null,
|
||||
val ttsVoice: String? = null,
|
||||
)
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ import androidx.room.TypeConverters
|
|||
|
||||
@Database(
|
||||
entities = [ConversationEntity::class, MessageEntity::class],
|
||||
version = 2,
|
||||
version = 3,
|
||||
exportSchema = false,
|
||||
)
|
||||
@TypeConverters(Converters::class)
|
||||
|
|
|
|||
|
|
@ -21,4 +21,10 @@ interface MessageDao {
|
|||
|
||||
@Query("SELECT DISTINCT conversationId FROM messages")
|
||||
suspend fun cachedConversationIds(): List<String>
|
||||
|
||||
@Query("UPDATE messages SET ttsUrl = :url, ttsVoice = :voice WHERE id = :messageId")
|
||||
suspend fun updateTts(messageId: String, url: String, voice: String)
|
||||
|
||||
@Query("SELECT ttsUrl FROM messages WHERE id = :messageId AND ttsVoice = :voice")
|
||||
suspend fun getTtsUrl(messageId: String, voice: String): String?
|
||||
}
|
||||
|
|
|
|||
|
|
@ -27,6 +27,7 @@ import java.util.concurrent.TimeUnit
|
|||
val name: String? = null,
|
||||
val email: String? = null,
|
||||
val defaultModel: String? = null,
|
||||
val ttsVoice: String? = null,
|
||||
)
|
||||
@Serializable data class WireMessage(val role: String, val content: String)
|
||||
@Serializable private data class ChatRequest(
|
||||
|
|
@ -572,6 +573,35 @@ object KaizenApi {
|
|||
}
|
||||
}
|
||||
|
||||
@Serializable private data class SpeechRequest(val prompt: String, val kind: String = "speech", val voice: String? = null)
|
||||
@Serializable private data class SpeechResponse(val url: String = "", val cost: Double? = null)
|
||||
|
||||
/** POST /api/v1/audio-gen — generate speech from text via Gemini-TTS. Returns the audio URL or null. */
|
||||
suspend fun generateSpeech(baseUrl: String, token: String, text: String, voice: String? = null): String? =
|
||||
withContext(Dispatchers.IO) {
|
||||
val body = json.encodeToString(SpeechRequest(
|
||||
prompt = text.take(2000),
|
||||
voice = voice,
|
||||
)).toRequestBody(jsonMedia)
|
||||
val req = Request.Builder()
|
||||
.url("$baseUrl/api/v1/audio-gen")
|
||||
.post(body)
|
||||
.header("Authorization", "Bearer $token")
|
||||
.build()
|
||||
try {
|
||||
client.newCall(req).execute().use { resp ->
|
||||
if (!resp.isSuccessful) {
|
||||
Log.w(TAG, "generateSpeech: HTTP ${resp.code}")
|
||||
return@use null
|
||||
}
|
||||
json.decodeFromString<SpeechResponse>(resp.body!!.string()).url.takeIf { it.isNotBlank() }
|
||||
}
|
||||
} catch (e: Exception) {
|
||||
Log.w(TAG, "generateSpeech: ${e.javaClass.simpleName}: ${e.message}")
|
||||
null
|
||||
}
|
||||
}
|
||||
|
||||
suspend fun prewarm(baseUrl: String) = withContext(Dispatchers.IO) {
|
||||
try {
|
||||
val req = Request.Builder().url("$baseUrl/api/v1/meta").build()
|
||||
|
|
|
|||
|
|
@ -91,6 +91,9 @@ object KaizenIcons {
|
|||
val ArrowDown: ImageVector get() = Lucide.ArrowDown
|
||||
val ArrowUp: ImageVector get() = Lucide.ArrowUp
|
||||
|
||||
// ── Audio ──────────────────────────────────────────────────────────────
|
||||
val Volume2: ImageVector get() = Lucide.Volume2
|
||||
|
||||
// ── Custom (brand overrides) ────────────────────────────────────────────
|
||||
// Add hand-drawn SVGs here as ImageVector literals when ready.
|
||||
// Example:
|
||||
|
|
|
|||
|
|
@ -19,6 +19,11 @@
|
|||
<string name="chat_error">Error</string>
|
||||
<string name="chat_open_menu">Open menu</string>
|
||||
<string name="chat_stop">Stop response</string>
|
||||
<string name="chat_recording">Recording …</string>
|
||||
<string name="chat_mic_permission">Microphone access required</string>
|
||||
<string name="chat_read_aloud">Read aloud</string>
|
||||
<string name="chat_tts_error">Read aloud failed</string>
|
||||
<string name="chat_tts_unavailable">Read aloud not available (Vertex required)</string>
|
||||
<string name="attach_camera">Take Photo</string>
|
||||
<string name="attach_gallery">Photo & Video</string>
|
||||
<string name="attach_file">Choose File</string>
|
||||
|
|
|
|||
|
|
@ -21,6 +21,11 @@
|
|||
<string name="chat_error">Fehler</string>
|
||||
<string name="chat_open_menu">Menü öffnen</string>
|
||||
<string name="chat_stop">Antwort stoppen</string>
|
||||
<string name="chat_recording">Aufnahme läuft …</string>
|
||||
<string name="chat_mic_permission">Mikrofonzugriff wird benötigt</string>
|
||||
<string name="chat_read_aloud">Vorlesen</string>
|
||||
<string name="chat_tts_error">Vorlesen fehlgeschlagen</string>
|
||||
<string name="chat_tts_unavailable">Vorlesen nicht verfügbar (Vertex benötigt)</string>
|
||||
<string name="attach_camera">Foto aufnehmen</string>
|
||||
<string name="attach_gallery">Foto & Video</string>
|
||||
<string name="attach_file">Datei auswählen</string>
|
||||
|
|
|
|||
Loading…
Reference in a new issue