Compare commits
7 commits
7ba42338c5
...
da3eded81e
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
da3eded81e | ||
|
|
ff66b80966 | ||
|
|
31278f2440 | ||
|
|
65ea3a5656 | ||
|
|
25fbb17d18 | ||
|
|
bb53b24655 | ||
|
|
287cdb4c6b |
16 changed files with 1607 additions and 88 deletions
1
.gitignore
vendored
1
.gitignore
vendored
|
|
@ -13,3 +13,4 @@
|
||||||
.externalNativeBuild
|
.externalNativeBuild
|
||||||
.cxx
|
.cxx
|
||||||
local.properties
|
local.properties
|
||||||
|
bugs/
|
||||||
|
|
|
||||||
|
|
@ -280,7 +280,7 @@ All committed on `main`. Compile-verified on Mac (`./gradlew :app:compileDebugKo
|
||||||
| `fb6bc98` | **Web search provider picker** — for Vertex models: Google vs Enterprise Web Search dropdown pill, `webSearchProvider` sent in chat request |
|
| `fb6bc98` | **Web search provider picker** — for Vertex models: Google vs Enterprise Web Search dropdown pill, `webSearchProvider` sent in chat request |
|
||||||
| `1899f2c` | **Voice message display** — audio-only user messages render as compact "🎤 Sprachnachricht" pill instead of ugly filename chip |
|
| `1899f2c` | **Voice message display** — audio-only user messages render as compact "🎤 Sprachnachricht" pill instead of ugly filename chip |
|
||||||
| `44ff82a` | **Voice message ordering fix** — caches user + assistant messages in Room with correct `sortOrder` after server save (was missing, caused wrong order on reload) |
|
| `44ff82a` | **Voice message ordering fix** — caches user + assistant messages in Room with correct `sortOrder` after server save (was missing, caused wrong order on reload) |
|
||||||
| `8f72afd` | **Edit and resend user messages** — SquarePen icon on user messages, removes message + all after it, puts text back in input field |
|
| `8f72afd` | **Edit and resend user messages** — SquarePen icon on user messages, creates branch (old messages preserved), text back in input field |
|
||||||
| `b2df7e9` | **Modern message action icons** — ClipboardCopy (copy), SquarePen (edit), RotateCcw (regenerate), 36dp touch target, 20dp icons |
|
| `b2df7e9` | **Modern message action icons** — ClipboardCopy (copy), SquarePen (edit), RotateCcw (regenerate), 36dp touch target, 20dp icons |
|
||||||
| `421b29e` | **ReasoningBlock + SourceCards redesign** — accent-tinted background + gradient borders, KaizenShapes.md (16dp radius), domain instead of full URL in source cards |
|
| `421b29e` | **ReasoningBlock + SourceCards redesign** — accent-tinted background + gradient borders, KaizenShapes.md (16dp radius), domain instead of full URL in source cards |
|
||||||
| `75ca8c0` | **Visual refresh** — body text W400→W300 (airier), headings scaled up (H1 26sp, H2 22sp, H3 18sp), `animateItem()` on messages (350ms fade-in, spring placement), blobs 20% larger + better distribution, higher alpha on OLED, light-mode factor 0.65→0.78, distinct `primaryDeep` per theme |
|
| `75ca8c0` | **Visual refresh** — body text W400→W300 (airier), headings scaled up (H1 26sp, H2 22sp, H3 18sp), `animateItem()` on messages (350ms fade-in, spring placement), blobs 20% larger + better distribution, higher alpha on OLED, light-mode factor 0.65→0.78, distinct `primaryDeep` per theme |
|
||||||
|
|
@ -292,6 +292,7 @@ All committed on `main`. Compile-verified on Mac (`./gradlew :app:compileDebugKo
|
||||||
| `1c9491e` | **Text selection** — `SelectionContainer` on user + assistant messages. Long-press to select, copy, share |
|
| `1c9491e` | **Text selection** — `SelectionContainer` on user + assistant messages. Long-press to select, copy, share |
|
||||||
| `1c9491e` | **Markdown spacing** — heading spacing 18→24/20dp, list items 8dp gap + muted bullets, blockquote 12dp inset |
|
| `1c9491e` | **Markdown spacing** — heading spacing 18→24/20dp, list items 8dp gap + muted bullets, blockquote 12dp inset |
|
||||||
| `430d5b6` | **Flat sidebar user row** — removed heavy card (background, border, shadow), replaced with flat row + thin separator |
|
| `430d5b6` | **Flat sidebar user row** — removed heavy card (background, border, shadow), replaced with flat row + thin separator |
|
||||||
|
| — | **Branch navigation (tree support)** — `ChatTree` data structure mirrors web frontend's tree. Edit creates a branch (sibling node with same parentId) instead of deleting messages. Regenerate creates a new assistant branch. `BranchNavigator` UI (chevron arrows + "1/3" counter) on messages with siblings. `activeRootId`/`activeChild` synced to server via PATCH. `StoredMessage.parentId` parsed from API. `patchConversation` handles JSON object values for `activeChild` |
|
||||||
|
|
||||||
**Earlier UI/feel work (Phase 0 prototype → feel-first):**
|
**Earlier UI/feel work (Phase 0 prototype → feel-first):**
|
||||||
- Liquid glass styling, floating panels, glassmorphic sidebar
|
- Liquid glass styling, floating panels, glassmorphic sidebar
|
||||||
|
|
|
||||||
|
|
@ -11,6 +11,10 @@
|
||||||
<!-- Required for voice recording (mic button in chat) -->
|
<!-- Required for voice recording (mic button in chat) -->
|
||||||
<uses-permission android:name="android.permission.RECORD_AUDIO" />
|
<uses-permission android:name="android.permission.RECORD_AUDIO" />
|
||||||
|
|
||||||
|
<!-- Required for Live Call foreground service (mic access while backgrounded) -->
|
||||||
|
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
|
||||||
|
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MICROPHONE" />
|
||||||
|
|
||||||
<application
|
<application
|
||||||
android:allowBackup="true"
|
android:allowBackup="true"
|
||||||
android:dataExtractionRules="@xml/data_extraction_rules"
|
android:dataExtractionRules="@xml/data_extraction_rules"
|
||||||
|
|
@ -32,6 +36,11 @@
|
||||||
<category android:name="android.intent.category.LAUNCHER" />
|
<category android:name="android.intent.category.LAUNCHER" />
|
||||||
</intent-filter>
|
</intent-filter>
|
||||||
</activity>
|
</activity>
|
||||||
|
|
||||||
|
<service
|
||||||
|
android:name=".live.LiveCallService"
|
||||||
|
android:foregroundServiceType="microphone"
|
||||||
|
android:exported="false" />
|
||||||
</application>
|
</application>
|
||||||
|
|
||||||
</manifest>
|
</manifest>
|
||||||
|
|
@ -437,6 +437,51 @@ fun TokenBadge(used: Int, contextLength: Int, modifier: Modifier = Modifier) {
|
||||||
color = cs.onSurfaceVariant,
|
color = cs.onSurfaceVariant,
|
||||||
fontSize = 12.sp,
|
fontSize = 12.sp,
|
||||||
fontWeight = FontWeight.Medium,
|
fontWeight = FontWeight.Medium,
|
||||||
|
maxLines = 1,
|
||||||
|
softWrap = false,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Composable
|
||||||
|
fun BranchNavigator(
|
||||||
|
branchInfo: BranchInfo,
|
||||||
|
onNavigate: (Int) -> Unit,
|
||||||
|
modifier: Modifier = Modifier,
|
||||||
|
) {
|
||||||
|
val cs = MaterialTheme.colorScheme
|
||||||
|
Row(
|
||||||
|
modifier = modifier,
|
||||||
|
verticalAlignment = Alignment.CenterVertically,
|
||||||
|
horizontalArrangement = Arrangement.spacedBy(2.dp),
|
||||||
|
) {
|
||||||
|
Box(
|
||||||
|
Modifier.size(28.dp).clip(KaizenShapes.circle)
|
||||||
|
.clickable(enabled = branchInfo.index > 0) { onNavigate(-1) },
|
||||||
|
contentAlignment = Alignment.Center,
|
||||||
|
) {
|
||||||
|
Icon(
|
||||||
|
KaizenIcons.ChevronLeft, null,
|
||||||
|
tint = cs.onSurfaceVariant.copy(alpha = if (branchInfo.index > 0) 0.65f else 0.20f),
|
||||||
|
modifier = Modifier.size(16.dp),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
Text(
|
||||||
|
"${branchInfo.index + 1}/${branchInfo.total}",
|
||||||
|
color = cs.onSurfaceVariant.copy(alpha = 0.55f),
|
||||||
|
fontSize = 12.sp,
|
||||||
|
fontWeight = FontWeight.Medium,
|
||||||
|
)
|
||||||
|
Box(
|
||||||
|
Modifier.size(28.dp).clip(KaizenShapes.circle)
|
||||||
|
.clickable(enabled = branchInfo.index < branchInfo.total - 1) { onNavigate(1) },
|
||||||
|
contentAlignment = Alignment.Center,
|
||||||
|
) {
|
||||||
|
Icon(
|
||||||
|
KaizenIcons.ChevronRight, null,
|
||||||
|
tint = cs.onSurfaceVariant.copy(alpha = if (branchInfo.index < branchInfo.total - 1) 0.65f else 0.20f),
|
||||||
|
modifier = Modifier.size(16.dp),
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -451,6 +496,8 @@ fun MessageRow(
|
||||||
onEdit: ((String, String) -> Unit)? = null,
|
onEdit: ((String, String) -> Unit)? = null,
|
||||||
onReadAloud: ((String) -> Unit)? = null,
|
onReadAloud: ((String) -> Unit)? = null,
|
||||||
isPlayingTts: Boolean = false,
|
isPlayingTts: Boolean = false,
|
||||||
|
branchInfo: BranchInfo? = null,
|
||||||
|
onNavigateBranch: ((String, Int) -> Unit)? = null,
|
||||||
) {
|
) {
|
||||||
val cs = MaterialTheme.colorScheme
|
val cs = MaterialTheme.colorScheme
|
||||||
val isDark = isSystemInDarkTheme()
|
val isDark = isSystemInDarkTheme()
|
||||||
|
|
@ -535,7 +582,12 @@ fun MessageRow(
|
||||||
Row(
|
Row(
|
||||||
Modifier.padding(top = 6.dp, end = 4.dp),
|
Modifier.padding(top = 6.dp, end = 4.dp),
|
||||||
horizontalArrangement = Arrangement.spacedBy(2.dp),
|
horizontalArrangement = Arrangement.spacedBy(2.dp),
|
||||||
|
verticalAlignment = Alignment.CenterVertically,
|
||||||
) {
|
) {
|
||||||
|
if (branchInfo != null && onNavigateBranch != null) {
|
||||||
|
BranchNavigator(branchInfo, onNavigate = { dir -> onNavigateBranch(message.wireId, dir) })
|
||||||
|
Spacer(Modifier.width(4.dp))
|
||||||
|
}
|
||||||
MessageActionButton(
|
MessageActionButton(
|
||||||
icon = if (copied) KaizenIcons.Check else KaizenIcons.ClipboardCopy,
|
icon = if (copied) KaizenIcons.Check else KaizenIcons.ClipboardCopy,
|
||||||
tint = if (copied) KaizenSemanticColors.success else cs.onSurfaceVariant.copy(alpha = 0.55f),
|
tint = if (copied) KaizenSemanticColors.success else cs.onSurfaceVariant.copy(alpha = 0.55f),
|
||||||
|
|
@ -597,7 +649,12 @@ fun MessageRow(
|
||||||
Row(
|
Row(
|
||||||
Modifier.padding(start = 38.dp, top = 6.dp),
|
Modifier.padding(start = 38.dp, top = 6.dp),
|
||||||
horizontalArrangement = Arrangement.spacedBy(2.dp),
|
horizontalArrangement = Arrangement.spacedBy(2.dp),
|
||||||
|
verticalAlignment = Alignment.CenterVertically,
|
||||||
) {
|
) {
|
||||||
|
if (branchInfo != null && onNavigateBranch != null) {
|
||||||
|
BranchNavigator(branchInfo, onNavigate = { dir -> onNavigateBranch(message.wireId, dir) })
|
||||||
|
Spacer(Modifier.width(4.dp))
|
||||||
|
}
|
||||||
MessageActionButton(
|
MessageActionButton(
|
||||||
icon = if (copied) KaizenIcons.Check else KaizenIcons.ClipboardCopy,
|
icon = if (copied) KaizenIcons.Check else KaizenIcons.ClipboardCopy,
|
||||||
tint = if (copied) KaizenSemanticColors.success else cs.onSurfaceVariant.copy(alpha = 0.55f),
|
tint = if (copied) KaizenSemanticColors.success else cs.onSurfaceVariant.copy(alpha = 0.55f),
|
||||||
|
|
@ -734,6 +791,7 @@ fun ChatInput(
|
||||||
isRecording: Boolean = false,
|
isRecording: Boolean = false,
|
||||||
recordingAmplitude: Float = 0f,
|
recordingAmplitude: Float = 0f,
|
||||||
onMicClick: () -> Unit = {},
|
onMicClick: () -> Unit = {},
|
||||||
|
onCallClick: () -> Unit = {},
|
||||||
pendingFiles: List<PendingFile> = emptyList(),
|
pendingFiles: List<PendingFile> = emptyList(),
|
||||||
onAddClick: () -> Unit = {},
|
onAddClick: () -> Unit = {},
|
||||||
onRemoveFile: (String) -> Unit = {},
|
onRemoveFile: (String) -> Unit = {},
|
||||||
|
|
@ -865,7 +923,7 @@ fun ChatInput(
|
||||||
} else if (hasContent) {
|
} else if (hasContent) {
|
||||||
SendButton(enabled = canSend, onClick = onSend)
|
SendButton(enabled = canSend, onClick = onSend)
|
||||||
} else {
|
} else {
|
||||||
CallButton(onClick = { /* TODO: call feature */ })
|
CallButton(onClick = onCallClick)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -42,6 +42,7 @@ data class Message(
|
||||||
val tools: List<ToolEvent> = emptyList(),
|
val tools: List<ToolEvent> = emptyList(),
|
||||||
val query: String = "",
|
val query: String = "",
|
||||||
val sources: List<SearchSource> = emptyList(),
|
val sources: List<SearchSource> = emptyList(),
|
||||||
|
val branchInfo: BranchInfo? = null,
|
||||||
)
|
)
|
||||||
|
|
||||||
data class Suggestion(val labelRes: Int, val promptRes: Int, val icon: ImageVector)
|
data class Suggestion(val labelRes: Int, val promptRes: Int, val icon: ImageVector)
|
||||||
|
|
|
||||||
|
|
@ -92,11 +92,17 @@ import androidx.core.content.ContextCompat
|
||||||
import androidx.compose.animation.core.Spring
|
import androidx.compose.animation.core.Spring
|
||||||
import androidx.compose.animation.core.spring
|
import androidx.compose.animation.core.spring
|
||||||
import androidx.compose.animation.core.tween
|
import androidx.compose.animation.core.tween
|
||||||
|
import android.content.ComponentName
|
||||||
|
import android.content.ServiceConnection
|
||||||
|
import android.os.IBinder
|
||||||
import kotlinx.coroutines.Job
|
import kotlinx.coroutines.Job
|
||||||
import kotlinx.coroutines.async
|
import kotlinx.coroutines.async
|
||||||
import kotlinx.coroutines.launch
|
import kotlinx.coroutines.launch
|
||||||
import java.io.ByteArrayOutputStream
|
import java.io.ByteArrayOutputStream
|
||||||
import java.io.File
|
import java.io.File
|
||||||
|
import dev.kaizen.app.live.CallStatus
|
||||||
|
import dev.kaizen.app.live.LiveCallOverlay
|
||||||
|
import dev.kaizen.app.live.LiveCallService
|
||||||
|
|
||||||
// Screen enumeration for modular state-driven navigation (Zero Technical Debt)
|
// Screen enumeration for modular state-driven navigation (Zero Technical Debt)
|
||||||
enum class AppScreen { Chat, Settings }
|
enum class AppScreen { Chat, Settings }
|
||||||
|
|
@ -123,12 +129,47 @@ fun ChatScreen(
|
||||||
var chatModel by remember { mutableStateOf<String?>(null) }
|
var chatModel by remember { mutableStateOf<String?>(null) }
|
||||||
var streamJob by remember { mutableStateOf<Job?>(null) }
|
var streamJob by remember { mutableStateOf<Job?>(null) }
|
||||||
|
|
||||||
|
// Tree state for branch navigation
|
||||||
|
var chatTree by remember { mutableStateOf(ChatTree.EMPTY) }
|
||||||
|
val storedMessagesMap = remember { mutableMapOf<String, dev.kaizen.app.net.StoredMessage>() }
|
||||||
|
var editParentId by remember { mutableStateOf<String?>(null) }
|
||||||
|
|
||||||
|
fun rebuildVisibleMessages() {
|
||||||
|
val path = getVisiblePath(chatTree)
|
||||||
|
messages.clear()
|
||||||
|
for (entry in path) {
|
||||||
|
val sm = storedMessagesMap[entry.messageId] ?: continue
|
||||||
|
messages.add(
|
||||||
|
Message(
|
||||||
|
id = nextId++,
|
||||||
|
role = if (sm.role == "assistant") Role.Assistant else Role.User,
|
||||||
|
content = sm.content,
|
||||||
|
wireId = sm.id,
|
||||||
|
attachments = sm.attachments,
|
||||||
|
reasoning = sm.reasoning ?: "",
|
||||||
|
sources = sm.sources ?: emptyList(),
|
||||||
|
branchInfo = entry.branchInfo,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// Audio recording state
|
// Audio recording state
|
||||||
var isRecording by remember { mutableStateOf(false) }
|
var isRecording by remember { mutableStateOf(false) }
|
||||||
val recorderRef = remember { mutableStateOf<MediaRecorder?>(null) }
|
val recorderRef = remember { mutableStateOf<MediaRecorder?>(null) }
|
||||||
val audioFileRef = remember { mutableStateOf<File?>(null) }
|
val audioFileRef = remember { mutableStateOf<File?>(null) }
|
||||||
var recordingAmplitude by remember { mutableStateOf(0f) }
|
var recordingAmplitude by remember { mutableStateOf(0f) }
|
||||||
|
|
||||||
|
// ─── Live Call state (uses context/ttsVoice/currentConversationId declared below) ───
|
||||||
|
var liveService by remember { mutableStateOf<LiveCallService?>(null) }
|
||||||
|
var liveCallActive by remember { mutableStateOf(false) }
|
||||||
|
val liveStatus by liveService?.status?.collectAsState() ?: remember { mutableStateOf(CallStatus.IDLE) }
|
||||||
|
val liveSubtitle by liveService?.subtitle?.collectAsState() ?: remember { mutableStateOf("") }
|
||||||
|
val liveAmplitude by liveService?.amplitude?.collectAsState() ?: remember { mutableStateOf(0f) }
|
||||||
|
val liveUserSpeaking by liveService?.userSpeaking?.collectAsState() ?: remember { mutableStateOf(false) }
|
||||||
|
var liveStartedAt by remember { mutableStateOf(0L) }
|
||||||
|
var liveElapsedMs by remember { mutableStateOf(0L) }
|
||||||
|
|
||||||
LaunchedEffect(isRecording) {
|
LaunchedEffect(isRecording) {
|
||||||
if (!isRecording) { recordingAmplitude = 0f; return@LaunchedEffect }
|
if (!isRecording) { recordingAmplitude = 0f; return@LaunchedEffect }
|
||||||
while (isRecording) {
|
while (isRecording) {
|
||||||
|
|
@ -176,6 +217,144 @@ fun ChatScreen(
|
||||||
val pendingFiles = remember { mutableStateListOf<PendingFile>() }
|
val pendingFiles = remember { mutableStateListOf<PendingFile>() }
|
||||||
val context = LocalContext.current
|
val context = LocalContext.current
|
||||||
|
|
||||||
|
// ─── Live Call service binding + control ──────────────────────────────
|
||||||
|
val liveServiceConnection = remember {
|
||||||
|
object : ServiceConnection {
|
||||||
|
override fun onServiceConnected(name: ComponentName?, binder: IBinder?) {
|
||||||
|
liveService = (binder as? LiveCallService.LocalBinder)?.service
|
||||||
|
}
|
||||||
|
override fun onServiceDisconnected(name: ComponentName?) {
|
||||||
|
liveService = null
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
LaunchedEffect(liveCallActive) {
|
||||||
|
if (liveCallActive) {
|
||||||
|
liveStartedAt = System.currentTimeMillis()
|
||||||
|
while (liveCallActive) {
|
||||||
|
liveElapsedMs = System.currentTimeMillis() - liveStartedAt
|
||||||
|
kotlinx.coroutines.delay(500)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
val liveAvailable = models.any { it.id.startsWith("vertex:") && it.id.contains("live") }
|
||||||
|
|
||||||
|
fun startLiveCallInner() {
|
||||||
|
val cfg = session.config ?: return
|
||||||
|
liveCallActive = true
|
||||||
|
|
||||||
|
val intent = LiveCallService.intent(context)
|
||||||
|
context.startForegroundService(intent)
|
||||||
|
context.bindService(intent, liveServiceConnection, 0)
|
||||||
|
|
||||||
|
scope.launch {
|
||||||
|
var attempts = 0
|
||||||
|
while (liveService == null && attempts < 20) {
|
||||||
|
kotlinx.coroutines.delay(50)
|
||||||
|
attempts++
|
||||||
|
}
|
||||||
|
|
||||||
|
val svc = liveService ?: return@launch
|
||||||
|
|
||||||
|
// Create conversation if needed (same pattern as text chat)
|
||||||
|
val cid = conversationId ?: run {
|
||||||
|
val newId = KaizenApi.createConversation(cfg.baseUrl, cfg.token)
|
||||||
|
if (newId != null) {
|
||||||
|
conversationId = newId
|
||||||
|
chat.conversationRepo.insertOptimistic(newId, "Live Call")
|
||||||
|
}
|
||||||
|
newId
|
||||||
|
}
|
||||||
|
|
||||||
|
svc.onTurnEnd = { turn ->
|
||||||
|
scope.launch {
|
||||||
|
val convId = conversationId ?: return@launch
|
||||||
|
|
||||||
|
// Add messages to local UI
|
||||||
|
if (turn.userText.isNotEmpty()) {
|
||||||
|
val userMsg = Message(nextId++, Role.User, turn.userText, wireId = turn.userMessageId ?: java.util.UUID.randomUUID().toString())
|
||||||
|
messages.add(userMsg)
|
||||||
|
}
|
||||||
|
if (turn.assistantText.isNotEmpty()) {
|
||||||
|
val aMsg = Message(nextId++, Role.Assistant, turn.assistantText, wireId = turn.assistantMessageId ?: java.util.UUID.randomUUID().toString())
|
||||||
|
messages.add(aMsg)
|
||||||
|
chatModel = cfg.model
|
||||||
|
usedTokens = turn.usage.inputTokens + turn.usage.outputTokens
|
||||||
|
}
|
||||||
|
|
||||||
|
// Cache in Room (server already persisted via WS-Server)
|
||||||
|
val msgCount = messages.size
|
||||||
|
val entitiesToCache = mutableListOf<dev.kaizen.app.db.MessageEntity>()
|
||||||
|
if (turn.userMessageId != null && turn.userText.isNotEmpty()) {
|
||||||
|
entitiesToCache.add(dev.kaizen.app.db.MessageEntity(
|
||||||
|
id = turn.userMessageId, conversationId = convId,
|
||||||
|
role = "user", content = turn.userText,
|
||||||
|
attachments = emptyList(), sortOrder = msgCount - 2,
|
||||||
|
))
|
||||||
|
}
|
||||||
|
if (turn.assistantMessageId != null && turn.assistantText.isNotEmpty()) {
|
||||||
|
entitiesToCache.add(dev.kaizen.app.db.MessageEntity(
|
||||||
|
id = turn.assistantMessageId, conversationId = convId,
|
||||||
|
role = "assistant", content = turn.assistantText,
|
||||||
|
attachments = emptyList(), sortOrder = msgCount - 1,
|
||||||
|
))
|
||||||
|
}
|
||||||
|
if (entitiesToCache.isNotEmpty()) {
|
||||||
|
chat.messageRepo.cacheMessages(convId, entitiesToCache)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
svc.onError = { code, message ->
|
||||||
|
scope.launch {
|
||||||
|
val errorText = message ?: code
|
||||||
|
Toast.makeText(context, errorText, Toast.LENGTH_SHORT).show()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
svc.startCall(
|
||||||
|
baseUrl = cfg.baseUrl,
|
||||||
|
token = cfg.token,
|
||||||
|
model = cfg.model,
|
||||||
|
voice = ttsVoice,
|
||||||
|
conversationId = cid,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
val liveCallPermission = rememberLauncherForActivityResult(ActivityResultContracts.RequestPermission()) { granted ->
|
||||||
|
if (granted) startLiveCallInner()
|
||||||
|
else Toast.makeText(context, context.getString(R.string.chat_mic_permission), Toast.LENGTH_SHORT).show()
|
||||||
|
}
|
||||||
|
|
||||||
|
fun startLiveCall() {
|
||||||
|
if (ContextCompat.checkSelfPermission(context, Manifest.permission.RECORD_AUDIO) != PackageManager.PERMISSION_GRANTED) {
|
||||||
|
liveCallPermission.launch(Manifest.permission.RECORD_AUDIO)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
startLiveCallInner()
|
||||||
|
}
|
||||||
|
|
||||||
|
fun endLiveCall() {
|
||||||
|
liveService?.onTurnEnd = null
|
||||||
|
liveService?.onError = null
|
||||||
|
liveService?.endCall()
|
||||||
|
liveCallActive = false
|
||||||
|
try { context.unbindService(liveServiceConnection) } catch (_: Exception) {}
|
||||||
|
liveService = null
|
||||||
|
|
||||||
|
// Generate title for new conversations
|
||||||
|
val cfg = session.config ?: return
|
||||||
|
val cid = conversationId ?: return
|
||||||
|
scope.launch {
|
||||||
|
val title = KaizenApi.generateTitle(cfg.baseUrl, cfg.token, cid)
|
||||||
|
if (title != null) chat.conversationRepo.updateTitle(cid, title)
|
||||||
|
chat.conversationRepo.refresh(cfg.baseUrl, cfg.token)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
var showAttachMenu by remember { mutableStateOf(false) }
|
var showAttachMenu by remember { mutableStateOf(false) }
|
||||||
|
|
||||||
fun uploadPickedFile(name: String, mimeType: String, bytes: ByteArray) {
|
fun uploadPickedFile(name: String, mimeType: String, bytes: ByteArray) {
|
||||||
|
|
@ -309,6 +488,17 @@ fun ChatScreen(
|
||||||
sources = finalSources?.map { dev.kaizen.app.net.SearchSource(it.title, it.url, it.snippet) } ?: emptyList(),
|
sources = finalSources?.map { dev.kaizen.app.net.SearchSource(it.title, it.url, it.snippet) } ?: emptyList(),
|
||||||
),
|
),
|
||||||
))
|
))
|
||||||
|
storedMessagesMap[userMsg.wireId] = dev.kaizen.app.net.StoredMessage(
|
||||||
|
id = userMsg.wireId, role = "user", content = "",
|
||||||
|
parentId = userParentId, attachments = listOf(att),
|
||||||
|
)
|
||||||
|
storedMessagesMap[assistantWireId] = dev.kaizen.app.net.StoredMessage(
|
||||||
|
id = assistantWireId, role = "assistant", content = finalContent,
|
||||||
|
parentId = userMsg.wireId, model = cfg.model,
|
||||||
|
reasoning = finalReasoning,
|
||||||
|
sources = finalSources?.map { dev.kaizen.app.net.SearchSource(it.title, it.url, it.snippet) },
|
||||||
|
)
|
||||||
|
chatTree = buildChatTree(storedMessagesMap.values.toList(), chatTree.activeRootId, chatTree.activeChild)
|
||||||
}
|
}
|
||||||
if (convIdDeferred != null) {
|
if (convIdDeferred != null) {
|
||||||
chat.conversationRepo.insertOptimistic(cid, "(Voice)")
|
chat.conversationRepo.insertOptimistic(cid, "(Voice)")
|
||||||
|
|
@ -334,28 +524,44 @@ fun ChatScreen(
|
||||||
}
|
}
|
||||||
|
|
||||||
fun playTtsUrl(url: String, messageId: Long) {
|
fun playTtsUrl(url: String, messageId: Long) {
|
||||||
|
scope.launch {
|
||||||
try {
|
try {
|
||||||
|
val tmpFile = kotlinx.coroutines.withContext(kotlinx.coroutines.Dispatchers.IO) {
|
||||||
|
val req = okhttp3.Request.Builder().url(url).build()
|
||||||
|
KaizenApi.imageClient.newCall(req).execute().use { resp ->
|
||||||
|
if (!resp.isSuccessful) return@withContext null
|
||||||
|
val bytes = resp.body!!.bytes()
|
||||||
|
java.io.File.createTempFile("tts_", ".wav", context.cacheDir).also { it.writeBytes(bytes) }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (tmpFile == null) {
|
||||||
|
playingTtsForId = null
|
||||||
|
return@launch
|
||||||
|
}
|
||||||
val player = MediaPlayer().apply {
|
val player = MediaPlayer().apply {
|
||||||
setDataSource(url)
|
|
||||||
prepareAsync()
|
|
||||||
setOnPreparedListener { start() }
|
setOnPreparedListener { start() }
|
||||||
setOnCompletionListener {
|
setOnCompletionListener {
|
||||||
release()
|
release()
|
||||||
ttsPlayerRef.value = null
|
ttsPlayerRef.value = null
|
||||||
playingTtsForId = null
|
playingTtsForId = null
|
||||||
|
tmpFile.delete()
|
||||||
}
|
}
|
||||||
setOnErrorListener { _, _, _ ->
|
setOnErrorListener { _, _, _ ->
|
||||||
release()
|
release()
|
||||||
ttsPlayerRef.value = null
|
ttsPlayerRef.value = null
|
||||||
playingTtsForId = null
|
playingTtsForId = null
|
||||||
|
tmpFile.delete()
|
||||||
true
|
true
|
||||||
}
|
}
|
||||||
|
setDataSource(tmpFile.absolutePath)
|
||||||
|
prepareAsync()
|
||||||
}
|
}
|
||||||
ttsPlayerRef.value = player
|
ttsPlayerRef.value = player
|
||||||
} catch (_: Exception) {
|
} catch (_: Exception) {
|
||||||
playingTtsForId = null
|
playingTtsForId = null
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
|
||||||
fun readAloud(content: String, messageId: Long, wireId: String) {
|
fun readAloud(content: String, messageId: Long, wireId: String) {
|
||||||
val cfg = session.config ?: return
|
val cfg = session.config ?: return
|
||||||
|
|
@ -443,6 +649,8 @@ fun ChatScreen(
|
||||||
viewingLockedChat = false
|
viewingLockedChat = false
|
||||||
usedTokens = 0
|
usedTokens = 0
|
||||||
chatModel = null
|
chatModel = null
|
||||||
|
chatTree = ChatTree.EMPTY
|
||||||
|
storedMessagesMap.clear()
|
||||||
}
|
}
|
||||||
|
|
||||||
fun unlockAndOpen(summary: ConversationSummary, password: String? = null) {
|
fun unlockAndOpen(summary: ConversationSummary, password: String? = null) {
|
||||||
|
|
@ -462,22 +670,12 @@ fun ChatScreen(
|
||||||
passwordUnlockTarget = null
|
passwordUnlockTarget = null
|
||||||
passwordUnlockError = null
|
passwordUnlockError = null
|
||||||
passwordUnlockForToggle = false
|
passwordUnlockForToggle = false
|
||||||
val stored = KaizenApi.fetchLockedMessages(cfg.baseUrl, cfg.token, summary.id, unlockToken)
|
val data = KaizenApi.fetchLockedConversationData(cfg.baseUrl, cfg.token, summary.id, unlockToken)
|
||||||
if (stored.isNotEmpty()) {
|
if (data.messages.isNotEmpty()) {
|
||||||
messages.clear()
|
storedMessagesMap.clear()
|
||||||
stored.forEach { m ->
|
data.messages.forEach { storedMessagesMap[it.id] = it }
|
||||||
messages.add(
|
chatTree = buildChatTree(data.messages, data.activeRootId, data.activeChild)
|
||||||
Message(
|
rebuildVisibleMessages()
|
||||||
id = nextId++,
|
|
||||||
role = if (m.role == "assistant") Role.Assistant else Role.User,
|
|
||||||
content = m.content,
|
|
||||||
wireId = m.id,
|
|
||||||
attachments = m.attachments,
|
|
||||||
reasoning = m.reasoning ?: "",
|
|
||||||
sources = m.sources ?: emptyList(),
|
|
||||||
),
|
|
||||||
)
|
|
||||||
}
|
|
||||||
conversationId = summary.id
|
conversationId = summary.id
|
||||||
viewingLockedChat = true
|
viewingLockedChat = true
|
||||||
}
|
}
|
||||||
|
|
@ -546,46 +744,36 @@ fun ChatScreen(
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
conversationId = summary.id
|
conversationId = summary.id
|
||||||
val stored = KaizenApi.fetchMessages(cfg.baseUrl, cfg.token, summary.id)
|
val data = KaizenApi.fetchConversationData(cfg.baseUrl, cfg.token, summary.id)
|
||||||
if (stored.isNotEmpty()) {
|
if (data.messages.isNotEmpty()) {
|
||||||
val entities = stored.mapIndexed { i, m -> m.toEntity(summary.id, i) }
|
val entities = data.messages.mapIndexed { i, m -> m.toEntity(summary.id, i) }
|
||||||
chat.messageRepo.cacheMessages(summary.id, entities)
|
chat.messageRepo.cacheMessages(summary.id, entities)
|
||||||
val lastAssistant = stored.lastOrNull { it.role == "assistant" }
|
val lastAssistant = data.messages.lastOrNull { it.role == "assistant" }
|
||||||
lastAssistant?.model?.let { chatModel = it }
|
lastAssistant?.model?.let { chatModel = it }
|
||||||
usedTokens = stored.sumOf { (it.inputTokens ?: 0) + (it.outputTokens ?: 0) }
|
usedTokens = data.messages.sumOf { (it.inputTokens ?: 0) + (it.outputTokens ?: 0) }
|
||||||
if (stored.size != cachedList.size || stored.zip(cachedList).any { (s, c) -> s.id != c.id || s.content != c.content }) {
|
|
||||||
messages.clear()
|
storedMessagesMap.clear()
|
||||||
stored.forEach { m ->
|
data.messages.forEach { storedMessagesMap[it.id] = it }
|
||||||
messages.add(
|
chatTree = buildChatTree(data.messages, data.activeRootId, data.activeChild)
|
||||||
Message(
|
rebuildVisibleMessages()
|
||||||
id = nextId++,
|
|
||||||
role = if (m.role == "assistant") Role.Assistant else Role.User,
|
|
||||||
content = m.content,
|
|
||||||
wireId = m.id,
|
|
||||||
attachments = m.attachments,
|
|
||||||
reasoning = m.reasoning ?: "",
|
|
||||||
sources = m.sources ?: emptyList(),
|
|
||||||
),
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
fun send(text: String) {
|
fun send(text: String, branchParentId: String? = null, branchAttachments: List<Attachment> = emptyList()) {
|
||||||
val trimmed = text.trim()
|
val trimmed = text.trim()
|
||||||
val uploadedAtts = pendingFiles.mapNotNull { it.uploaded }
|
val uploadedAtts = if (branchAttachments.isNotEmpty()) branchAttachments else pendingFiles.mapNotNull { it.uploaded }
|
||||||
if (trimmed.isEmpty() && uploadedAtts.isEmpty()) return
|
if (trimmed.isEmpty() && uploadedAtts.isEmpty()) return
|
||||||
if (isStreaming) return
|
if (isStreaming) return
|
||||||
val cfg = session.config ?: return
|
val cfg = session.config ?: return
|
||||||
haptics.click()
|
haptics.click()
|
||||||
|
|
||||||
val userParentId = messages.lastOrNull()?.wireId
|
val userParentId = editParentId ?: branchParentId ?: messages.lastOrNull()?.wireId
|
||||||
|
editParentId = null
|
||||||
val userMsg = Message(nextId++, Role.User, trimmed, attachments = uploadedAtts)
|
val userMsg = Message(nextId++, Role.User, trimmed, attachments = uploadedAtts)
|
||||||
messages.add(userMsg)
|
messages.add(userMsg)
|
||||||
input = ""
|
input = ""
|
||||||
pendingFiles.clear()
|
if (branchAttachments.isEmpty()) pendingFiles.clear()
|
||||||
val assistantId = nextId++
|
val assistantId = nextId++
|
||||||
val assistantWireId = java.util.UUID.randomUUID().toString()
|
val assistantWireId = java.util.UUID.randomUUID().toString()
|
||||||
messages.add(Message(assistantId, Role.Assistant, "", streaming = true, thinking = true, wireId = assistantWireId))
|
messages.add(Message(assistantId, Role.Assistant, "", streaming = true, thinking = true, wireId = assistantWireId))
|
||||||
|
|
@ -681,6 +869,24 @@ fun ChatScreen(
|
||||||
sources = finalSources ?: emptyList(),
|
sources = finalSources ?: emptyList(),
|
||||||
),
|
),
|
||||||
))
|
))
|
||||||
|
|
||||||
|
// Update tree with the new nodes
|
||||||
|
storedMessagesMap[userMsg.wireId] = dev.kaizen.app.net.StoredMessage(
|
||||||
|
id = userMsg.wireId, role = "user", content = trimmed,
|
||||||
|
parentId = userParentId, attachments = uploadedAtts,
|
||||||
|
)
|
||||||
|
storedMessagesMap[assistantWireId] = dev.kaizen.app.net.StoredMessage(
|
||||||
|
id = assistantWireId, role = "assistant", content = finalContent,
|
||||||
|
parentId = userMsg.wireId, model = cfg.model,
|
||||||
|
reasoning = finalReasoning,
|
||||||
|
sources = finalSources?.map { dev.kaizen.app.net.SearchSource(it.title, it.url, it.snippet) },
|
||||||
|
)
|
||||||
|
chatTree = buildChatTree(
|
||||||
|
storedMessagesMap.values.toList(),
|
||||||
|
chatTree.activeRootId,
|
||||||
|
chatTree.activeChild,
|
||||||
|
)
|
||||||
|
|
||||||
if (wasNewConversation) {
|
if (wasNewConversation) {
|
||||||
val title = KaizenApi.generateTitle(cfg.baseUrl, cfg.token, convId)
|
val title = KaizenApi.generateTitle(cfg.baseUrl, cfg.token, convId)
|
||||||
if (title != null) {
|
if (title != null) {
|
||||||
|
|
@ -888,24 +1094,22 @@ fun ChatScreen(
|
||||||
onRegenerate = if (message.role == Role.Assistant && !isStreaming) { _ ->
|
onRegenerate = if (message.role == Role.Assistant && !isStreaming) { _ ->
|
||||||
val lastUserIdx = messages.indexOfLast { it.role == Role.User && messages.indexOf(it) < messages.indexOf(message) }
|
val lastUserIdx = messages.indexOfLast { it.role == Role.User && messages.indexOf(it) < messages.indexOf(message) }
|
||||||
if (lastUserIdx >= 0) {
|
if (lastUserIdx >= 0) {
|
||||||
val userContent = messages[lastUserIdx].content
|
val userMsg = messages[lastUserIdx]
|
||||||
messages.removeAt(messages.indexOf(message))
|
val parentId = chatTree.nodes[userMsg.wireId]?.parentId
|
||||||
send(userContent)
|
val msgIdx = messages.indexOf(message)
|
||||||
|
if (msgIdx >= 0) {
|
||||||
|
messages.subList(msgIdx, messages.size).clear()
|
||||||
|
}
|
||||||
|
send(userMsg.content, branchParentId = parentId, branchAttachments = userMsg.attachments)
|
||||||
}
|
}
|
||||||
} else null,
|
} else null,
|
||||||
onEdit = if (message.role == Role.User && !isStreaming) { wireId, content ->
|
onEdit = if (message.role == Role.User && !isStreaming) { wireId, content ->
|
||||||
|
val node = chatTree.nodes[wireId]
|
||||||
|
val parentId = node?.parentId
|
||||||
val idx = messages.indexOfFirst { it.wireId == wireId }
|
val idx = messages.indexOfFirst { it.wireId == wireId }
|
||||||
if (idx >= 0) {
|
if (idx >= 0) {
|
||||||
val cfg = session.config ?: return@MessageRow
|
messages.subList(idx, messages.size).clear()
|
||||||
val toRemove = messages.subList(idx, messages.size).map { it.wireId }
|
editParentId = parentId
|
||||||
messages.removeAll { it.wireId in toRemove }
|
|
||||||
if (conversationId != null) {
|
|
||||||
scope.launch {
|
|
||||||
toRemove.forEach { id ->
|
|
||||||
KaizenApi.deleteMessage(cfg.baseUrl, cfg.token, conversationId!!, id)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
input = content
|
input = content
|
||||||
}
|
}
|
||||||
} else null,
|
} else null,
|
||||||
|
|
@ -913,6 +1117,24 @@ fun ChatScreen(
|
||||||
readAloud(content, message.id, message.wireId)
|
readAloud(content, message.id, message.wireId)
|
||||||
} else null,
|
} else null,
|
||||||
isPlayingTts = playingTtsForId == message.id,
|
isPlayingTts = playingTtsForId == message.id,
|
||||||
|
branchInfo = message.branchInfo,
|
||||||
|
onNavigateBranch = if (!isStreaming) { wireId, direction ->
|
||||||
|
val newTree = treeNavigate(chatTree, wireId, direction)
|
||||||
|
chatTree = newTree
|
||||||
|
rebuildVisibleMessages()
|
||||||
|
if (conversationId != null) {
|
||||||
|
val cfg = session.config ?: return@MessageRow
|
||||||
|
scope.launch {
|
||||||
|
KaizenApi.patchConversation(
|
||||||
|
cfg.baseUrl, cfg.token, conversationId!!,
|
||||||
|
mapOf(
|
||||||
|
"activeRootId" to newTree.activeRootId,
|
||||||
|
"activeChild" to newTree.activeChild,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else null,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -1137,6 +1359,7 @@ fun ChatScreen(
|
||||||
isRecording = isRecording,
|
isRecording = isRecording,
|
||||||
recordingAmplitude = recordingAmplitude,
|
recordingAmplitude = recordingAmplitude,
|
||||||
onMicClick = { toggleRecording() },
|
onMicClick = { toggleRecording() },
|
||||||
|
onCallClick = if (liveAvailable) {{ startLiveCall() }} else {{ /* PTT TODO */ }},
|
||||||
pendingFiles = pendingFiles,
|
pendingFiles = pendingFiles,
|
||||||
onAddClick = { showAttachMenu = !showAttachMenu },
|
onAddClick = { showAttachMenu = !showAttachMenu },
|
||||||
onRemoveFile = { id -> pendingFiles.removeAll { it.id == id } },
|
onRemoveFile = { id -> pendingFiles.removeAll { it.id == id } },
|
||||||
|
|
@ -1234,6 +1457,23 @@ fun ChatScreen(
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ─── Live Call Overlay ────────────────────────────────
|
||||||
|
androidx.compose.animation.AnimatedVisibility(
|
||||||
|
visible = liveCallActive,
|
||||||
|
enter = androidx.compose.animation.fadeIn(tween(250)),
|
||||||
|
exit = androidx.compose.animation.fadeOut(tween(200)),
|
||||||
|
) {
|
||||||
|
LiveCallOverlay(
|
||||||
|
status = liveStatus,
|
||||||
|
subtitle = liveSubtitle,
|
||||||
|
amplitude = liveAmplitude,
|
||||||
|
userSpeaking = liveUserSpeaking,
|
||||||
|
elapsedMs = liveElapsedMs,
|
||||||
|
onEndCall = { endLiveCall() },
|
||||||
|
onInterrupt = { liveService?.interrupt() },
|
||||||
|
)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
106
app/src/main/java/dev/kaizen/app/chat/ChatTree.kt
Normal file
106
app/src/main/java/dev/kaizen/app/chat/ChatTree.kt
Normal file
|
|
@ -0,0 +1,106 @@
|
||||||
|
package dev.kaizen.app.chat
|
||||||
|
|
||||||
|
import dev.kaizen.app.net.StoredMessage
|
||||||
|
|
||||||
|
data class ChatNode(
|
||||||
|
val id: String,
|
||||||
|
val role: String,
|
||||||
|
val parentId: String?,
|
||||||
|
val childIds: List<String> = emptyList(),
|
||||||
|
)
|
||||||
|
|
||||||
|
data class BranchInfo(val index: Int, val total: Int)
|
||||||
|
|
||||||
|
data class VisibleEntry(
|
||||||
|
val messageId: String,
|
||||||
|
val branchInfo: BranchInfo?,
|
||||||
|
)
|
||||||
|
|
||||||
|
data class ChatTree(
|
||||||
|
val nodes: Map<String, ChatNode>,
|
||||||
|
val rootIds: List<String>,
|
||||||
|
val activeRootId: String?,
|
||||||
|
val activeChild: Map<String, String>,
|
||||||
|
) {
|
||||||
|
companion object {
|
||||||
|
val EMPTY = ChatTree(emptyMap(), emptyList(), null, emptyMap())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fun buildChatTree(
|
||||||
|
messages: List<StoredMessage>,
|
||||||
|
savedActiveRootId: String?,
|
||||||
|
savedActiveChild: Map<String, String>,
|
||||||
|
): ChatTree {
|
||||||
|
val nodes = mutableMapOf<String, ChatNode>()
|
||||||
|
val rootIds = mutableListOf<String>()
|
||||||
|
|
||||||
|
for (msg in messages) {
|
||||||
|
nodes[msg.id] = ChatNode(
|
||||||
|
id = msg.id,
|
||||||
|
role = msg.role,
|
||||||
|
parentId = msg.parentId,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
for (msg in messages) {
|
||||||
|
val pid = msg.parentId
|
||||||
|
if (pid != null && nodes.containsKey(pid)) {
|
||||||
|
val parent = nodes[pid]!!
|
||||||
|
nodes[pid] = parent.copy(childIds = parent.childIds + msg.id)
|
||||||
|
} else if (pid == null) {
|
||||||
|
rootIds.add(msg.id)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
val activeRootId = if (savedActiveRootId != null && nodes.containsKey(savedActiveRootId))
|
||||||
|
savedActiveRootId
|
||||||
|
else rootIds.lastOrNull()
|
||||||
|
|
||||||
|
return ChatTree(nodes, rootIds, activeRootId, savedActiveChild)
|
||||||
|
}
|
||||||
|
|
||||||
|
fun getVisiblePath(tree: ChatTree): List<VisibleEntry> {
|
||||||
|
if (tree.rootIds.isEmpty()) return emptyList()
|
||||||
|
val activeRootId = tree.activeRootId ?: tree.rootIds.last()
|
||||||
|
val result = mutableListOf<VisibleEntry>()
|
||||||
|
var currentId: String? = activeRootId
|
||||||
|
|
||||||
|
while (currentId != null) {
|
||||||
|
val node = tree.nodes[currentId] ?: break
|
||||||
|
val siblings = if (node.parentId == null) {
|
||||||
|
tree.rootIds
|
||||||
|
} else {
|
||||||
|
tree.nodes[node.parentId]?.childIds ?: listOf(node.id)
|
||||||
|
}
|
||||||
|
val branchInfo = if (siblings.size > 1) {
|
||||||
|
BranchInfo(index = siblings.indexOf(node.id), total = siblings.size)
|
||||||
|
} else null
|
||||||
|
|
||||||
|
result.add(VisibleEntry(node.id, branchInfo))
|
||||||
|
|
||||||
|
if (node.childIds.isEmpty()) break
|
||||||
|
currentId = tree.activeChild[node.id] ?: node.childIds.last()
|
||||||
|
}
|
||||||
|
|
||||||
|
return result
|
||||||
|
}
|
||||||
|
|
||||||
|
fun treeNavigate(tree: ChatTree, nodeId: String, direction: Int): ChatTree {
|
||||||
|
val node = tree.nodes[nodeId] ?: return tree
|
||||||
|
val siblings = if (node.parentId == null) {
|
||||||
|
tree.rootIds
|
||||||
|
} else {
|
||||||
|
tree.nodes[node.parentId]?.childIds ?: return tree
|
||||||
|
}
|
||||||
|
val idx = siblings.indexOf(nodeId)
|
||||||
|
val newIdx = idx + direction
|
||||||
|
if (newIdx < 0 || newIdx >= siblings.size) return tree
|
||||||
|
val newActiveId = siblings[newIdx]
|
||||||
|
|
||||||
|
return if (node.parentId == null) {
|
||||||
|
tree.copy(activeRootId = newActiveId)
|
||||||
|
} else {
|
||||||
|
tree.copy(activeChild = tree.activeChild + (node.parentId to newActiveId))
|
||||||
|
}
|
||||||
|
}
|
||||||
210
app/src/main/java/dev/kaizen/app/live/AudioPipeline.kt
Normal file
210
app/src/main/java/dev/kaizen/app/live/AudioPipeline.kt
Normal file
|
|
@ -0,0 +1,210 @@
|
||||||
|
package dev.kaizen.app.live
|
||||||
|
|
||||||
|
import android.Manifest
|
||||||
|
import android.content.Context
|
||||||
|
import android.content.pm.PackageManager
|
||||||
|
import android.media.AudioAttributes
|
||||||
|
import android.media.AudioFormat
|
||||||
|
import android.media.AudioRecord
|
||||||
|
import android.media.AudioTrack
|
||||||
|
import android.media.MediaRecorder
|
||||||
|
import android.util.Base64
|
||||||
|
import android.util.Log
|
||||||
|
import androidx.core.content.ContextCompat
|
||||||
|
import kotlinx.coroutines.*
|
||||||
|
import java.nio.ByteBuffer
|
||||||
|
import java.nio.ByteOrder
|
||||||
|
import kotlin.math.abs
|
||||||
|
import kotlin.math.sqrt
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Bidirectional audio pipeline for Live Call.
|
||||||
|
*
|
||||||
|
* Capture: AudioRecord (16kHz PCM16 mono, VOICE_COMMUNICATION for hardware AEC)
|
||||||
|
* → 100ms chunks → Base64 → [onChunk] callback.
|
||||||
|
*
|
||||||
|
* Playback: AudioTrack (24kHz PCM16 mono, LOW_LATENCY, MODE_STREAM)
|
||||||
|
* ← PCM16 chunks from server.
|
||||||
|
*
|
||||||
|
* Mic-ducking: gain reduced to 10% while the KI speaks (prevents echo feedback
|
||||||
|
* loop on devices with weak AEC). Not a digital gain — we skip sending chunks
|
||||||
|
* whose RMS is below the ducking threshold.
|
||||||
|
*/
|
||||||
|
class AudioPipeline(private val context: Context) {
|
||||||
|
|
||||||
|
companion object {
|
||||||
|
private const val TAG = "AudioPipeline"
|
||||||
|
|
||||||
|
private const val CAPTURE_SAMPLE_RATE = 16_000
|
||||||
|
private const val PLAYBACK_SAMPLE_RATE = 24_000
|
||||||
|
private const val CHANNEL_IN = AudioFormat.CHANNEL_IN_MONO
|
||||||
|
private const val CHANNEL_OUT = AudioFormat.CHANNEL_OUT_MONO
|
||||||
|
private const val ENCODING = AudioFormat.ENCODING_PCM_16BIT
|
||||||
|
|
||||||
|
private const val CHUNK_MS = 100
|
||||||
|
private const val CAPTURE_CHUNK_SAMPLES = CAPTURE_SAMPLE_RATE * CHUNK_MS / 1000 // 1600
|
||||||
|
private const val CAPTURE_CHUNK_BYTES = CAPTURE_CHUNK_SAMPLES * 2 // 3200
|
||||||
|
|
||||||
|
private const val DUCKING_GAIN = 0.10f
|
||||||
|
}
|
||||||
|
|
||||||
|
var onChunk: ((base64Pcm: String) -> Unit)? = null
|
||||||
|
var onAmplitude: ((Float) -> Unit)? = null
|
||||||
|
|
||||||
|
private var recorder: AudioRecord? = null
|
||||||
|
private var track: AudioTrack? = null
|
||||||
|
private var captureJob: Job? = null
|
||||||
|
|
||||||
|
@Volatile private var ducking = false
|
||||||
|
@Volatile private var capturing = false
|
||||||
|
|
||||||
|
// ─── Capture ──────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
fun startCapture(scope: CoroutineScope): Boolean {
|
||||||
|
if (ContextCompat.checkSelfPermission(context, Manifest.permission.RECORD_AUDIO) != PackageManager.PERMISSION_GRANTED) {
|
||||||
|
Log.w(TAG, "RECORD_AUDIO permission not granted")
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
val minBuf = AudioRecord.getMinBufferSize(CAPTURE_SAMPLE_RATE, CHANNEL_IN, ENCODING)
|
||||||
|
if (minBuf == AudioRecord.ERROR || minBuf == AudioRecord.ERROR_BAD_VALUE) {
|
||||||
|
Log.e(TAG, "Invalid min buffer size: $minBuf")
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
val rec = AudioRecord(
|
||||||
|
MediaRecorder.AudioSource.VOICE_COMMUNICATION,
|
||||||
|
CAPTURE_SAMPLE_RATE,
|
||||||
|
CHANNEL_IN,
|
||||||
|
ENCODING,
|
||||||
|
maxOf(minBuf * 4, CAPTURE_CHUNK_BYTES * 4),
|
||||||
|
)
|
||||||
|
|
||||||
|
if (rec.state != AudioRecord.STATE_INITIALIZED) {
|
||||||
|
Log.e(TAG, "AudioRecord init failed")
|
||||||
|
rec.release()
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
recorder = rec
|
||||||
|
rec.startRecording()
|
||||||
|
capturing = true
|
||||||
|
|
||||||
|
captureJob = scope.launch(Dispatchers.IO) {
|
||||||
|
val buffer = ShortArray(CAPTURE_CHUNK_SAMPLES)
|
||||||
|
val byteBuffer = ByteBuffer.allocate(CAPTURE_CHUNK_BYTES).order(ByteOrder.LITTLE_ENDIAN)
|
||||||
|
|
||||||
|
while (isActive && capturing) {
|
||||||
|
val read = rec.read(buffer, 0, CAPTURE_CHUNK_SAMPLES)
|
||||||
|
if (read <= 0) continue
|
||||||
|
|
||||||
|
val rms = computeRms(buffer, read)
|
||||||
|
onAmplitude?.invoke(rms)
|
||||||
|
|
||||||
|
if (ducking && rms < DUCKING_GAIN) continue
|
||||||
|
|
||||||
|
byteBuffer.clear()
|
||||||
|
for (i in 0 until read) byteBuffer.putShort(buffer[i])
|
||||||
|
|
||||||
|
val base64 = Base64.encodeToString(byteBuffer.array(), 0, read * 2, Base64.NO_WRAP)
|
||||||
|
onChunk?.invoke(base64)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
fun stopCapture() {
|
||||||
|
capturing = false
|
||||||
|
captureJob?.cancel()
|
||||||
|
captureJob = null
|
||||||
|
try {
|
||||||
|
recorder?.stop()
|
||||||
|
} catch (_: IllegalStateException) {}
|
||||||
|
recorder?.release()
|
||||||
|
recorder = null
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── Playback ─────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
fun initPlayback(): Boolean {
|
||||||
|
val minBuf = AudioTrack.getMinBufferSize(PLAYBACK_SAMPLE_RATE, CHANNEL_OUT, ENCODING)
|
||||||
|
if (minBuf == AudioTrack.ERROR || minBuf == AudioTrack.ERROR_BAD_VALUE) {
|
||||||
|
Log.e(TAG, "Invalid playback buffer size: $minBuf")
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
val t = AudioTrack.Builder()
|
||||||
|
.setAudioAttributes(
|
||||||
|
AudioAttributes.Builder()
|
||||||
|
.setUsage(AudioAttributes.USAGE_ASSISTANT)
|
||||||
|
.setContentType(AudioAttributes.CONTENT_TYPE_SPEECH)
|
||||||
|
.build()
|
||||||
|
)
|
||||||
|
.setAudioFormat(
|
||||||
|
AudioFormat.Builder()
|
||||||
|
.setSampleRate(PLAYBACK_SAMPLE_RATE)
|
||||||
|
.setEncoding(ENCODING)
|
||||||
|
.setChannelMask(CHANNEL_OUT)
|
||||||
|
.build()
|
||||||
|
)
|
||||||
|
.setBufferSizeInBytes(maxOf(minBuf * 4, 24_000 * 2))
|
||||||
|
.setPerformanceMode(AudioTrack.PERFORMANCE_MODE_LOW_LATENCY)
|
||||||
|
.setTransferMode(AudioTrack.MODE_STREAM)
|
||||||
|
.build()
|
||||||
|
|
||||||
|
if (t.state != AudioTrack.STATE_INITIALIZED) {
|
||||||
|
Log.e(TAG, "AudioTrack init failed")
|
||||||
|
t.release()
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
track = t
|
||||||
|
t.play()
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
fun playChunk(base64Pcm: String) {
|
||||||
|
val bytes = Base64.decode(base64Pcm, Base64.NO_WRAP)
|
||||||
|
track?.write(bytes, 0, bytes.size)
|
||||||
|
}
|
||||||
|
|
||||||
|
fun flushPlayback() {
|
||||||
|
track?.pause()
|
||||||
|
track?.flush()
|
||||||
|
track?.play()
|
||||||
|
}
|
||||||
|
|
||||||
|
fun stopPlayback() {
|
||||||
|
try {
|
||||||
|
track?.pause()
|
||||||
|
track?.flush()
|
||||||
|
} catch (_: IllegalStateException) {}
|
||||||
|
track?.release()
|
||||||
|
track = null
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── Ducking ──────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
fun setDucking(enabled: Boolean) {
|
||||||
|
ducking = enabled
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── Cleanup ──────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
fun release() {
|
||||||
|
stopCapture()
|
||||||
|
stopPlayback()
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── Helpers ──────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
private fun computeRms(buffer: ShortArray, count: Int): Float {
|
||||||
|
var sum = 0.0
|
||||||
|
for (i in 0 until count) {
|
||||||
|
val sample = buffer[i].toDouble() / Short.MAX_VALUE
|
||||||
|
sum += sample * sample
|
||||||
|
}
|
||||||
|
return sqrt(sum / count).toFloat()
|
||||||
|
}
|
||||||
|
}
|
||||||
181
app/src/main/java/dev/kaizen/app/live/LiveCallOverlay.kt
Normal file
181
app/src/main/java/dev/kaizen/app/live/LiveCallOverlay.kt
Normal file
|
|
@ -0,0 +1,181 @@
|
||||||
|
package dev.kaizen.app.live
|
||||||
|
|
||||||
|
import androidx.compose.animation.*
|
||||||
|
import androidx.compose.animation.core.*
|
||||||
|
import androidx.compose.foundation.background
|
||||||
|
import androidx.compose.foundation.clickable
|
||||||
|
import androidx.compose.foundation.interaction.MutableInteractionSource
|
||||||
|
import androidx.compose.foundation.interaction.collectIsPressedAsState
|
||||||
|
import androidx.compose.foundation.isSystemInDarkTheme
|
||||||
|
import androidx.compose.foundation.layout.*
|
||||||
|
import androidx.compose.material3.Icon
|
||||||
|
import androidx.compose.material3.MaterialTheme
|
||||||
|
import androidx.compose.material3.Text
|
||||||
|
import androidx.compose.runtime.*
|
||||||
|
import androidx.compose.ui.Alignment
|
||||||
|
import androidx.compose.ui.Modifier
|
||||||
|
import androidx.compose.ui.draw.alpha
|
||||||
|
import androidx.compose.ui.draw.clip
|
||||||
|
import androidx.compose.ui.draw.scale
|
||||||
|
import androidx.compose.ui.graphics.Brush
|
||||||
|
import androidx.compose.ui.graphics.Color
|
||||||
|
import androidx.compose.ui.res.stringResource
|
||||||
|
import androidx.compose.ui.text.font.FontWeight
|
||||||
|
import androidx.compose.ui.text.style.TextAlign
|
||||||
|
import androidx.compose.ui.text.style.TextOverflow
|
||||||
|
import androidx.compose.ui.unit.dp
|
||||||
|
import androidx.compose.ui.unit.sp
|
||||||
|
import dev.kaizen.app.R
|
||||||
|
import dev.kaizen.app.chat.KaizenOrb
|
||||||
|
import dev.kaizen.app.ui.effect.GlassSurface
|
||||||
|
import dev.kaizen.app.ui.effect.KaizenShadows
|
||||||
|
import dev.kaizen.app.ui.icon.KaizenIcons
|
||||||
|
import dev.kaizen.app.ui.shape.KaizenShapes
|
||||||
|
import dev.kaizen.app.ui.theme.LocalKaizenAccent
|
||||||
|
|
||||||
|
@Composable
|
||||||
|
fun LiveCallOverlay(
|
||||||
|
status: CallStatus,
|
||||||
|
subtitle: String,
|
||||||
|
amplitude: Float,
|
||||||
|
userSpeaking: Boolean,
|
||||||
|
elapsedMs: Long,
|
||||||
|
onEndCall: () -> Unit,
|
||||||
|
onInterrupt: () -> Unit,
|
||||||
|
modifier: Modifier = Modifier,
|
||||||
|
) {
|
||||||
|
val isDark = isSystemInDarkTheme()
|
||||||
|
val accent = LocalKaizenAccent.current
|
||||||
|
val cs = MaterialTheme.colorScheme
|
||||||
|
|
||||||
|
Box(
|
||||||
|
modifier
|
||||||
|
.fillMaxSize()
|
||||||
|
.background(
|
||||||
|
if (isDark) Color.Black.copy(alpha = 0.85f)
|
||||||
|
else Color.White.copy(alpha = 0.88f)
|
||||||
|
)
|
||||||
|
.statusBarsPadding()
|
||||||
|
.navigationBarsPadding(),
|
||||||
|
contentAlignment = Alignment.Center,
|
||||||
|
) {
|
||||||
|
Column(
|
||||||
|
horizontalAlignment = Alignment.CenterHorizontally,
|
||||||
|
verticalArrangement = Arrangement.spacedBy(24.dp),
|
||||||
|
modifier = Modifier.fillMaxWidth().padding(horizontal = 32.dp),
|
||||||
|
) {
|
||||||
|
// ─── Duration ─────────────────────────────────────────
|
||||||
|
val elapsed = elapsedMs / 1000
|
||||||
|
val min = elapsed / 60
|
||||||
|
val sec = elapsed % 60
|
||||||
|
Text(
|
||||||
|
String.format("%d:%02d", min, sec),
|
||||||
|
color = cs.onSurface.copy(alpha = 0.5f),
|
||||||
|
fontSize = 14.sp,
|
||||||
|
fontWeight = FontWeight.W400,
|
||||||
|
letterSpacing = 1.sp,
|
||||||
|
)
|
||||||
|
|
||||||
|
// ─── Orb ──────────────────────────────────────────────
|
||||||
|
val isActive = status == CallStatus.LISTENING || status == CallStatus.SPEAKING
|
||||||
|
Box(
|
||||||
|
Modifier
|
||||||
|
.size(180.dp)
|
||||||
|
.then(
|
||||||
|
if (status == CallStatus.SPEAKING) {
|
||||||
|
Modifier.clickable(
|
||||||
|
interactionSource = remember { MutableInteractionSource() },
|
||||||
|
indication = null,
|
||||||
|
onClick = onInterrupt,
|
||||||
|
)
|
||||||
|
} else Modifier
|
||||||
|
),
|
||||||
|
contentAlignment = Alignment.Center,
|
||||||
|
) {
|
||||||
|
KaizenOrb(
|
||||||
|
size = 160.dp,
|
||||||
|
streaming = status == CallStatus.SPEAKING,
|
||||||
|
recording = status == CallStatus.LISTENING && userSpeaking,
|
||||||
|
amplitude = if (isActive) amplitude else null,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── Status text ──────────────────────────────────────
|
||||||
|
val statusText = when (status) {
|
||||||
|
CallStatus.IDLE -> ""
|
||||||
|
CallStatus.CONNECTING -> stringResource(R.string.live_connecting)
|
||||||
|
CallStatus.LISTENING -> if (userSpeaking) stringResource(R.string.live_listening)
|
||||||
|
else stringResource(R.string.live_ready)
|
||||||
|
CallStatus.SPEAKING -> stringResource(R.string.live_speaking)
|
||||||
|
}
|
||||||
|
|
||||||
|
Text(
|
||||||
|
statusText,
|
||||||
|
color = when (status) {
|
||||||
|
CallStatus.SPEAKING -> accent.primary
|
||||||
|
CallStatus.LISTENING -> if (userSpeaking) accent.primary.copy(alpha = 0.8f)
|
||||||
|
else cs.onSurface.copy(alpha = 0.4f)
|
||||||
|
else -> cs.onSurface.copy(alpha = 0.4f)
|
||||||
|
},
|
||||||
|
fontSize = 13.sp,
|
||||||
|
fontWeight = FontWeight.W400,
|
||||||
|
letterSpacing = 0.5.sp,
|
||||||
|
)
|
||||||
|
|
||||||
|
// ─── Subtitle (transcript / KI response) ─────────────
|
||||||
|
AnimatedVisibility(
|
||||||
|
visible = subtitle.isNotEmpty(),
|
||||||
|
enter = fadeIn(tween(200)) + expandVertically(),
|
||||||
|
exit = fadeOut(tween(150)) + shrinkVertically(),
|
||||||
|
) {
|
||||||
|
Text(
|
||||||
|
subtitle,
|
||||||
|
color = cs.onSurface.copy(alpha = 0.75f),
|
||||||
|
fontSize = 15.sp,
|
||||||
|
fontWeight = FontWeight.W300,
|
||||||
|
textAlign = TextAlign.Center,
|
||||||
|
maxLines = 4,
|
||||||
|
overflow = TextOverflow.Ellipsis,
|
||||||
|
lineHeight = 22.sp,
|
||||||
|
modifier = Modifier.fillMaxWidth(),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
Spacer(Modifier.height(32.dp))
|
||||||
|
|
||||||
|
// ─── End Call button ──────────────────────────────────
|
||||||
|
EndCallButton(onClick = onEndCall)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Composable
|
||||||
|
private fun EndCallButton(onClick: () -> Unit) {
|
||||||
|
val interaction = remember { MutableInteractionSource() }
|
||||||
|
val pressed by interaction.collectIsPressedAsState()
|
||||||
|
val pressScale by animateFloatAsState(if (pressed) 0.9f else 1f, label = "end")
|
||||||
|
|
||||||
|
Box(
|
||||||
|
Modifier
|
||||||
|
.size(64.dp)
|
||||||
|
.scale(pressScale)
|
||||||
|
.clip(KaizenShapes.circle)
|
||||||
|
.background(
|
||||||
|
Brush.verticalGradient(
|
||||||
|
listOf(
|
||||||
|
Color(0xFFEF4444),
|
||||||
|
Color(0xFFDC2626),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
.clickable(interactionSource = interaction, indication = null, onClick = onClick),
|
||||||
|
contentAlignment = Alignment.Center,
|
||||||
|
) {
|
||||||
|
Icon(
|
||||||
|
KaizenIcons.PhoneOff,
|
||||||
|
contentDescription = stringResource(R.string.live_end),
|
||||||
|
tint = Color.White,
|
||||||
|
modifier = Modifier.size(28.dp),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
304
app/src/main/java/dev/kaizen/app/live/LiveCallService.kt
Normal file
304
app/src/main/java/dev/kaizen/app/live/LiveCallService.kt
Normal file
|
|
@ -0,0 +1,304 @@
|
||||||
|
package dev.kaizen.app.live
|
||||||
|
|
||||||
|
import android.app.Notification
|
||||||
|
import android.app.NotificationChannel
|
||||||
|
import android.app.NotificationManager
|
||||||
|
import android.app.PendingIntent
|
||||||
|
import android.app.Service
|
||||||
|
import android.content.Context
|
||||||
|
import android.content.Intent
|
||||||
|
import android.content.pm.ServiceInfo
|
||||||
|
import android.os.Binder
|
||||||
|
import android.os.IBinder
|
||||||
|
import android.util.Log
|
||||||
|
import dev.kaizen.app.MainActivity
|
||||||
|
import dev.kaizen.app.R
|
||||||
|
import kotlinx.coroutines.*
|
||||||
|
import kotlinx.coroutines.flow.MutableStateFlow
|
||||||
|
import kotlinx.coroutines.flow.StateFlow
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Foreground Service for Live Call — keeps mic + WS alive when the app is
|
||||||
|
* backgrounded or the screen rotates. Shows a persistent notification with
|
||||||
|
* call duration and an end button.
|
||||||
|
*
|
||||||
|
* Lifecycle: started by ChatScreen → binds → [startCall] → audio flows
|
||||||
|
* → [endCall] or notification "Beenden" → cleanup → stopSelf.
|
||||||
|
*/
|
||||||
|
class LiveCallService : Service() {
|
||||||
|
|
||||||
|
companion object {
|
||||||
|
private const val TAG = "LiveCallService"
|
||||||
|
private const val CHANNEL_ID = "kaizen_live_call"
|
||||||
|
private const val NOTIFICATION_ID = 9001
|
||||||
|
private const val ACTION_END = "dev.kaizen.app.LIVE_CALL_END"
|
||||||
|
|
||||||
|
fun intent(context: Context): Intent = Intent(context, LiveCallService::class.java)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── Public state ─────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
private val _status = MutableStateFlow(CallStatus.IDLE)
|
||||||
|
val status: StateFlow<CallStatus> = _status
|
||||||
|
|
||||||
|
private val _subtitle = MutableStateFlow("")
|
||||||
|
val subtitle: StateFlow<String> = _subtitle
|
||||||
|
|
||||||
|
private val _amplitude = MutableStateFlow(0f)
|
||||||
|
val amplitude: StateFlow<Float> = _amplitude
|
||||||
|
|
||||||
|
private val _userSpeaking = MutableStateFlow(false)
|
||||||
|
val userSpeaking: StateFlow<Boolean> = _userSpeaking
|
||||||
|
|
||||||
|
var onTurnEnd: ((ServerEvent.TurnEnd) -> Unit)? = null
|
||||||
|
var onError: ((String, String?) -> Unit)? = null
|
||||||
|
|
||||||
|
// ─── Internal ─────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
private val scope = CoroutineScope(SupervisorJob() + Dispatchers.Main.immediate)
|
||||||
|
private var client: LiveClient? = null
|
||||||
|
private var audio: AudioPipeline? = null
|
||||||
|
private var durationJob: Job? = null
|
||||||
|
private var startTimeMs = 0L
|
||||||
|
|
||||||
|
// ─── Binder ───────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
inner class LocalBinder : Binder() {
|
||||||
|
val service: LiveCallService get() = this@LiveCallService
|
||||||
|
}
|
||||||
|
|
||||||
|
private val binder = LocalBinder()
|
||||||
|
override fun onBind(intent: Intent?): IBinder = binder
|
||||||
|
|
||||||
|
// ─── Lifecycle ────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
override fun onCreate() {
|
||||||
|
super.onCreate()
|
||||||
|
createNotificationChannel()
|
||||||
|
}
|
||||||
|
|
||||||
|
override fun onStartCommand(intent: Intent?, flags: Int, startId: Int): Int {
|
||||||
|
if (intent?.action == ACTION_END) {
|
||||||
|
endCall()
|
||||||
|
return START_NOT_STICKY
|
||||||
|
}
|
||||||
|
return START_NOT_STICKY
|
||||||
|
}
|
||||||
|
|
||||||
|
override fun onDestroy() {
|
||||||
|
endCall()
|
||||||
|
scope.cancel()
|
||||||
|
super.onDestroy()
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── Start / End ──────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
fun startCall(baseUrl: String, token: String, model: String, voice: String, conversationId: String?) {
|
||||||
|
if (_status.value != CallStatus.IDLE) return
|
||||||
|
|
||||||
|
_status.value = CallStatus.CONNECTING
|
||||||
|
|
||||||
|
startForeground(NOTIFICATION_ID, buildNotification("Verbinde…"), ServiceInfo.FOREGROUND_SERVICE_TYPE_MICROPHONE)
|
||||||
|
|
||||||
|
val pipeline = AudioPipeline(this)
|
||||||
|
audio = pipeline
|
||||||
|
|
||||||
|
val liveClient = LiveClient(baseUrl, token, scope)
|
||||||
|
client = liveClient
|
||||||
|
|
||||||
|
pipeline.onAmplitude = { amp -> _amplitude.value = amp }
|
||||||
|
|
||||||
|
scope.launch {
|
||||||
|
liveClient.events.collect { event -> handleEvent(event, pipeline) }
|
||||||
|
}
|
||||||
|
|
||||||
|
pipeline.onChunk = { base64 -> liveClient.sendAudio(base64) }
|
||||||
|
|
||||||
|
liveClient.connect(model, voice, conversationId)
|
||||||
|
}
|
||||||
|
|
||||||
|
fun endCall() {
|
||||||
|
if (_status.value == CallStatus.IDLE) return
|
||||||
|
|
||||||
|
client?.end()
|
||||||
|
audio?.release()
|
||||||
|
durationJob?.cancel()
|
||||||
|
|
||||||
|
client?.dispose()
|
||||||
|
client = null
|
||||||
|
audio = null
|
||||||
|
durationJob = null
|
||||||
|
|
||||||
|
_status.value = CallStatus.IDLE
|
||||||
|
_subtitle.value = ""
|
||||||
|
_amplitude.value = 0f
|
||||||
|
_userSpeaking.value = false
|
||||||
|
|
||||||
|
stopForeground(STOP_FOREGROUND_REMOVE)
|
||||||
|
stopSelf()
|
||||||
|
}
|
||||||
|
|
||||||
|
fun sendText(text: String) {
|
||||||
|
client?.sendText(text)
|
||||||
|
}
|
||||||
|
|
||||||
|
fun interrupt() {
|
||||||
|
client?.interrupt()
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── Event handling ───────────────────────────────────────────────────
|
||||||
|
|
||||||
|
private fun handleEvent(event: ServerEvent, pipeline: AudioPipeline) {
|
||||||
|
when (event) {
|
||||||
|
is ServerEvent.Ready -> {
|
||||||
|
_status.value = CallStatus.LISTENING
|
||||||
|
|
||||||
|
if (!pipeline.initPlayback()) {
|
||||||
|
Log.e(TAG, "Playback init failed")
|
||||||
|
}
|
||||||
|
if (!pipeline.startCapture(scope)) {
|
||||||
|
Log.e(TAG, "Capture init failed")
|
||||||
|
endCall()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
startTimeMs = System.currentTimeMillis()
|
||||||
|
startDurationTimer()
|
||||||
|
updateNotification("Anruf aktiv")
|
||||||
|
}
|
||||||
|
|
||||||
|
is ServerEvent.Resumed -> {
|
||||||
|
_status.value = CallStatus.LISTENING
|
||||||
|
updateNotification("Anruf fortgesetzt")
|
||||||
|
}
|
||||||
|
|
||||||
|
is ServerEvent.Audio -> {
|
||||||
|
if (_status.value != CallStatus.SPEAKING) {
|
||||||
|
_status.value = CallStatus.SPEAKING
|
||||||
|
pipeline.setDucking(true)
|
||||||
|
}
|
||||||
|
pipeline.playChunk(event.data)
|
||||||
|
}
|
||||||
|
|
||||||
|
is ServerEvent.Text -> {
|
||||||
|
_subtitle.value += event.delta
|
||||||
|
}
|
||||||
|
|
||||||
|
is ServerEvent.Transcript -> {
|
||||||
|
if (event.final) {
|
||||||
|
_subtitle.value = event.text
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
is ServerEvent.Vad -> {
|
||||||
|
_userSpeaking.value = event.speaking
|
||||||
|
if (event.speaking) {
|
||||||
|
_status.value = CallStatus.LISTENING
|
||||||
|
_subtitle.value = ""
|
||||||
|
pipeline.setDucking(false)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
is ServerEvent.Interrupted -> {
|
||||||
|
pipeline.flushPlayback()
|
||||||
|
pipeline.setDucking(false)
|
||||||
|
_status.value = CallStatus.LISTENING
|
||||||
|
}
|
||||||
|
|
||||||
|
is ServerEvent.TurnEnd -> {
|
||||||
|
pipeline.setDucking(false)
|
||||||
|
_status.value = CallStatus.LISTENING
|
||||||
|
onTurnEnd?.invoke(event)
|
||||||
|
}
|
||||||
|
|
||||||
|
is ServerEvent.ToolCall -> {
|
||||||
|
if (event.status == "running") {
|
||||||
|
_subtitle.value = "${event.name}…"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
is ServerEvent.Backpressure -> {
|
||||||
|
// LiveClient handles throttling internally
|
||||||
|
}
|
||||||
|
|
||||||
|
is ServerEvent.GoAway -> {
|
||||||
|
if (event.reason == "timeout") {
|
||||||
|
endCall()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
is ServerEvent.Error -> {
|
||||||
|
onError?.invoke(event.code, event.message)
|
||||||
|
if (event.code in setOf("auth_failed", "restricted_model", "restricted_feature", "budget_exceeded", "concurrent_limit")) {
|
||||||
|
endCall()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── Duration timer ───────────────────────────────────────────────────
|
||||||
|
|
||||||
|
private fun startDurationTimer() {
|
||||||
|
durationJob?.cancel()
|
||||||
|
durationJob = scope.launch {
|
||||||
|
while (isActive) {
|
||||||
|
delay(1000)
|
||||||
|
val elapsed = (System.currentTimeMillis() - startTimeMs) / 1000
|
||||||
|
val min = elapsed / 60
|
||||||
|
val sec = elapsed % 60
|
||||||
|
updateNotification(String.format("Anruf · %d:%02d", min, sec))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── Notification ─────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
private fun createNotificationChannel() {
|
||||||
|
val channel = NotificationChannel(
|
||||||
|
CHANNEL_ID,
|
||||||
|
getString(R.string.app_name),
|
||||||
|
NotificationManager.IMPORTANCE_LOW,
|
||||||
|
).apply {
|
||||||
|
description = "Live Call"
|
||||||
|
setShowBadge(false)
|
||||||
|
}
|
||||||
|
getSystemService(NotificationManager::class.java)?.createNotificationChannel(channel)
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun buildNotification(text: String): Notification {
|
||||||
|
val openIntent = PendingIntent.getActivity(
|
||||||
|
this, 0,
|
||||||
|
Intent(this, MainActivity::class.java).apply {
|
||||||
|
flags = Intent.FLAG_ACTIVITY_SINGLE_TOP
|
||||||
|
},
|
||||||
|
PendingIntent.FLAG_IMMUTABLE,
|
||||||
|
)
|
||||||
|
|
||||||
|
val endIntent = PendingIntent.getService(
|
||||||
|
this, 1,
|
||||||
|
Intent(this, LiveCallService::class.java).apply { action = ACTION_END },
|
||||||
|
PendingIntent.FLAG_IMMUTABLE,
|
||||||
|
)
|
||||||
|
|
||||||
|
return Notification.Builder(this, CHANNEL_ID)
|
||||||
|
.setSmallIcon(R.drawable.ic_launcher_foreground)
|
||||||
|
.setContentTitle(getString(R.string.app_name))
|
||||||
|
.setContentText(text)
|
||||||
|
.setContentIntent(openIntent)
|
||||||
|
.addAction(Notification.Action.Builder(null, "Beenden", endIntent).build())
|
||||||
|
.setOngoing(true)
|
||||||
|
.build()
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun updateNotification(text: String) {
|
||||||
|
getSystemService(NotificationManager::class.java)
|
||||||
|
?.notify(NOTIFICATION_ID, buildNotification(text))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
enum class CallStatus {
|
||||||
|
IDLE,
|
||||||
|
CONNECTING,
|
||||||
|
LISTENING,
|
||||||
|
SPEAKING,
|
||||||
|
}
|
||||||
211
app/src/main/java/dev/kaizen/app/live/LiveClient.kt
Normal file
211
app/src/main/java/dev/kaizen/app/live/LiveClient.kt
Normal file
|
|
@ -0,0 +1,211 @@
|
||||||
|
package dev.kaizen.app.live
|
||||||
|
|
||||||
|
import android.util.Log
|
||||||
|
import kotlinx.coroutines.*
|
||||||
|
import kotlinx.coroutines.flow.MutableSharedFlow
|
||||||
|
import kotlinx.coroutines.flow.SharedFlow
|
||||||
|
import okhttp3.OkHttpClient
|
||||||
|
import okhttp3.Request
|
||||||
|
import okhttp3.Response
|
||||||
|
import okhttp3.WebSocket
|
||||||
|
import okhttp3.WebSocketListener
|
||||||
|
import java.util.concurrent.TimeUnit
|
||||||
|
import java.util.concurrent.atomic.AtomicInteger
|
||||||
|
import java.util.concurrent.atomic.AtomicReference
|
||||||
|
|
||||||
|
/**
|
||||||
|
* WebSocket client for the Kaizen Live Call protocol.
|
||||||
|
*
|
||||||
|
* Connects to `wss://{baseUrl}/ws/live` with Bearer auth, speaks the
|
||||||
|
* provider-agnostic JSON protocol defined in `lib/live/protocol.ts`.
|
||||||
|
* Handles auto-reconnect with exponential backoff + resume.
|
||||||
|
*
|
||||||
|
* Thread-safe: OkHttp dispatches callbacks on its own threads,
|
||||||
|
* all state mutations go through atomic refs or the coroutine scope.
|
||||||
|
*/
|
||||||
|
class LiveClient(
|
||||||
|
private val baseUrl: String,
|
||||||
|
private val token: String,
|
||||||
|
private val scope: CoroutineScope,
|
||||||
|
) {
|
||||||
|
companion object {
|
||||||
|
private const val TAG = "LiveClient"
|
||||||
|
private const val INITIAL_BACKOFF_MS = 500L
|
||||||
|
private const val MAX_BACKOFF_MS = 8_000L
|
||||||
|
private const val MAX_RECONNECT_ATTEMPTS = 5
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── Public event stream ──────────────────────────────────────────────
|
||||||
|
|
||||||
|
private val _events = MutableSharedFlow<ServerEvent>(extraBufferCapacity = 64)
|
||||||
|
val events: SharedFlow<ServerEvent> = _events
|
||||||
|
|
||||||
|
// ─── State ────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
private val ws = AtomicReference<WebSocket?>(null)
|
||||||
|
private val sessionId = AtomicReference<String?>(null)
|
||||||
|
private val lastSeqId = AtomicInteger(0)
|
||||||
|
private val reconnectAttempts = AtomicInteger(0)
|
||||||
|
private var chunkIntervalMs = 100L
|
||||||
|
|
||||||
|
@Volatile private var active = false
|
||||||
|
@Volatile private var pendingSetup: SetupParams? = null
|
||||||
|
|
||||||
|
private data class SetupParams(
|
||||||
|
val model: String,
|
||||||
|
val voice: String,
|
||||||
|
val conversationId: String?,
|
||||||
|
)
|
||||||
|
|
||||||
|
private val client = OkHttpClient.Builder()
|
||||||
|
.readTimeout(0, TimeUnit.MILLISECONDS)
|
||||||
|
.pingInterval(15, TimeUnit.SECONDS)
|
||||||
|
.connectTimeout(10, TimeUnit.SECONDS)
|
||||||
|
.build()
|
||||||
|
|
||||||
|
// ─── Connect ──────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
fun connect(model: String, voice: String, conversationId: String?) {
|
||||||
|
active = true
|
||||||
|
pendingSetup = SetupParams(model, voice, conversationId)
|
||||||
|
reconnectAttempts.set(0)
|
||||||
|
openSocket()
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun openSocket() {
|
||||||
|
if (!active) return
|
||||||
|
|
||||||
|
val wsUrl = baseUrl
|
||||||
|
.replace("https://", "wss://")
|
||||||
|
.replace("http://", "ws://") + "/ws/live"
|
||||||
|
|
||||||
|
val request = Request.Builder()
|
||||||
|
.url(wsUrl)
|
||||||
|
.header("Authorization", "Bearer $token")
|
||||||
|
.build()
|
||||||
|
|
||||||
|
val socket = client.newWebSocket(request, Listener())
|
||||||
|
ws.set(socket)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── Send ─────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
fun sendAudio(base64Pcm: String) {
|
||||||
|
ws.get()?.send(encodeAudio(base64Pcm))
|
||||||
|
}
|
||||||
|
|
||||||
|
fun sendText(text: String) {
|
||||||
|
ws.get()?.send(encodeText(text))
|
||||||
|
}
|
||||||
|
|
||||||
|
fun interrupt() {
|
||||||
|
ws.get()?.send(encodeInterrupt())
|
||||||
|
}
|
||||||
|
|
||||||
|
fun end() {
|
||||||
|
active = false
|
||||||
|
ws.get()?.send(encodeEnd())
|
||||||
|
ws.getAndSet(null)?.close(1000, "end")
|
||||||
|
}
|
||||||
|
|
||||||
|
fun dispose() {
|
||||||
|
active = false
|
||||||
|
ws.getAndSet(null)?.cancel()
|
||||||
|
client.dispatcher.executorService.shutdown()
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── Reconnect ────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
private fun scheduleReconnect() {
|
||||||
|
if (!active) return
|
||||||
|
val attempt = reconnectAttempts.incrementAndGet()
|
||||||
|
if (attempt > MAX_RECONNECT_ATTEMPTS) {
|
||||||
|
Log.w(TAG, "Max reconnect attempts reached")
|
||||||
|
scope.launch { _events.emit(ServerEvent.Error("reconnect_failed", "Max reconnect attempts")) }
|
||||||
|
active = false
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
val backoff = (INITIAL_BACKOFF_MS * (1 shl (attempt - 1))).coerceAtMost(MAX_BACKOFF_MS)
|
||||||
|
Log.d(TAG, "Reconnecting in ${backoff}ms (attempt $attempt)")
|
||||||
|
|
||||||
|
scope.launch {
|
||||||
|
delay(backoff)
|
||||||
|
if (active) openSocket()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── WebSocket Listener ───────────────────────────────────────────────
|
||||||
|
|
||||||
|
private inner class Listener : WebSocketListener() {
|
||||||
|
|
||||||
|
override fun onOpen(webSocket: WebSocket, response: Response) {
|
||||||
|
Log.d(TAG, "WebSocket opened")
|
||||||
|
reconnectAttempts.set(0)
|
||||||
|
|
||||||
|
val sid = sessionId.get()
|
||||||
|
if (sid != null) {
|
||||||
|
webSocket.send(encodeResume(sid, lastSeqId.get()))
|
||||||
|
} else {
|
||||||
|
val setup = pendingSetup ?: return
|
||||||
|
webSocket.send(encodeSetup(setup.model, setup.voice, setup.conversationId))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
override fun onMessage(webSocket: WebSocket, text: String) {
|
||||||
|
val event = parseServerEvent(text) ?: return
|
||||||
|
|
||||||
|
when (event) {
|
||||||
|
is ServerEvent.Ready -> {
|
||||||
|
sessionId.set(event.sessionId)
|
||||||
|
}
|
||||||
|
is ServerEvent.Resumed -> {
|
||||||
|
sessionId.set(event.sessionId)
|
||||||
|
}
|
||||||
|
is ServerEvent.Audio -> {
|
||||||
|
lastSeqId.set(maxOf(lastSeqId.get(), event.seq))
|
||||||
|
}
|
||||||
|
is ServerEvent.Text -> {
|
||||||
|
lastSeqId.set(maxOf(lastSeqId.get(), event.seq))
|
||||||
|
}
|
||||||
|
is ServerEvent.Backpressure -> {
|
||||||
|
chunkIntervalMs = if (event.throttleMs > 0) event.throttleMs.toLong() else 100L
|
||||||
|
}
|
||||||
|
is ServerEvent.GoAway -> {
|
||||||
|
if (event.reason == "shutdown") {
|
||||||
|
scope.launch {
|
||||||
|
delay(500)
|
||||||
|
if (active) openSocket()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
is ServerEvent.Error -> {
|
||||||
|
val terminal = event.code in setOf(
|
||||||
|
"auth_failed", "restricted_model", "restricted_feature",
|
||||||
|
"budget_exceeded", "concurrent_limit",
|
||||||
|
)
|
||||||
|
if (terminal) active = false
|
||||||
|
}
|
||||||
|
else -> {}
|
||||||
|
}
|
||||||
|
|
||||||
|
scope.launch { _events.emit(event) }
|
||||||
|
}
|
||||||
|
|
||||||
|
override fun onClosing(webSocket: WebSocket, code: Int, reason: String) {
|
||||||
|
webSocket.close(code, reason)
|
||||||
|
}
|
||||||
|
|
||||||
|
override fun onClosed(webSocket: WebSocket, code: Int, reason: String) {
|
||||||
|
Log.d(TAG, "WebSocket closed: $code $reason")
|
||||||
|
ws.compareAndSet(webSocket, null)
|
||||||
|
if (active && code != 1000) scheduleReconnect()
|
||||||
|
}
|
||||||
|
|
||||||
|
override fun onFailure(webSocket: WebSocket, t: Throwable, response: Response?) {
|
||||||
|
Log.w(TAG, "WebSocket failure: ${t.message}")
|
||||||
|
ws.compareAndSet(webSocket, null)
|
||||||
|
if (active) scheduleReconnect()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
156
app/src/main/java/dev/kaizen/app/live/LiveProtocol.kt
Normal file
156
app/src/main/java/dev/kaizen/app/live/LiveProtocol.kt
Normal file
|
|
@ -0,0 +1,156 @@
|
||||||
|
package dev.kaizen.app.live
|
||||||
|
|
||||||
|
import kotlinx.serialization.Serializable
|
||||||
|
import kotlinx.serialization.json.Json
|
||||||
|
import kotlinx.serialization.json.JsonObject
|
||||||
|
import kotlinx.serialization.json.boolean
|
||||||
|
import kotlinx.serialization.json.booleanOrNull
|
||||||
|
import kotlinx.serialization.json.double
|
||||||
|
import kotlinx.serialization.json.doubleOrNull
|
||||||
|
import kotlinx.serialization.json.int
|
||||||
|
import kotlinx.serialization.json.intOrNull
|
||||||
|
import kotlinx.serialization.json.jsonObject
|
||||||
|
import kotlinx.serialization.json.jsonPrimitive
|
||||||
|
import kotlinx.serialization.json.long
|
||||||
|
import kotlinx.serialization.json.longOrNull
|
||||||
|
|
||||||
|
private val json = Json { ignoreUnknownKeys = true; encodeDefaults = false }
|
||||||
|
|
||||||
|
// ─── Client → Server ──────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
@Serializable
|
||||||
|
data class SetupMessage(
|
||||||
|
val type: String = "setup",
|
||||||
|
val model: String,
|
||||||
|
val voice: String,
|
||||||
|
val conversationId: String? = null,
|
||||||
|
val tools: Boolean? = null,
|
||||||
|
)
|
||||||
|
|
||||||
|
@Serializable
|
||||||
|
data class ResumeMessage(
|
||||||
|
val type: String = "resume",
|
||||||
|
val sessionId: String,
|
||||||
|
val lastSeqId: Int? = null,
|
||||||
|
)
|
||||||
|
|
||||||
|
@Serializable
|
||||||
|
data class AudioMessage(val type: String = "audio", val data: String)
|
||||||
|
|
||||||
|
@Serializable
|
||||||
|
data class TextMessage(val type: String = "text", val text: String)
|
||||||
|
|
||||||
|
@Serializable
|
||||||
|
data class SimpleMessage(val type: String)
|
||||||
|
|
||||||
|
fun encodeSetup(model: String, voice: String, conversationId: String?, tools: Boolean? = true): String =
|
||||||
|
json.encodeToString(SetupMessage.serializer(), SetupMessage(model = model, voice = voice, conversationId = conversationId, tools = tools))
|
||||||
|
|
||||||
|
fun encodeResume(sessionId: String, lastSeqId: Int?): String =
|
||||||
|
json.encodeToString(ResumeMessage.serializer(), ResumeMessage(sessionId = sessionId, lastSeqId = lastSeqId))
|
||||||
|
|
||||||
|
fun encodeAudio(base64Pcm: String): String =
|
||||||
|
json.encodeToString(AudioMessage.serializer(), AudioMessage(data = base64Pcm))
|
||||||
|
|
||||||
|
fun encodeText(text: String): String =
|
||||||
|
json.encodeToString(TextMessage.serializer(), TextMessage(text = text))
|
||||||
|
|
||||||
|
fun encodeInterrupt(): String = """{"type":"interrupt"}"""
|
||||||
|
fun encodeEnd(): String = """{"type":"end"}"""
|
||||||
|
|
||||||
|
// ─── Server → Client ──────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
data class LiveUsage(
|
||||||
|
val inputTokens: Int = 0,
|
||||||
|
val outputTokens: Int = 0,
|
||||||
|
val cost: Double? = null,
|
||||||
|
val durationMs: Long = 0,
|
||||||
|
)
|
||||||
|
|
||||||
|
sealed interface ServerEvent {
|
||||||
|
data class Ready(val sessionId: String) : ServerEvent
|
||||||
|
data class Resumed(val sessionId: String) : ServerEvent
|
||||||
|
data class Audio(val data: String, val seq: Int) : ServerEvent
|
||||||
|
data class Text(val delta: String, val seq: Int) : ServerEvent
|
||||||
|
data class Transcript(val text: String, val final: Boolean) : ServerEvent
|
||||||
|
data class Vad(val speaking: Boolean) : ServerEvent
|
||||||
|
data object Interrupted : ServerEvent
|
||||||
|
data class TurnEnd(
|
||||||
|
val usage: LiveUsage,
|
||||||
|
val userText: String,
|
||||||
|
val assistantText: String,
|
||||||
|
val userMessageId: String? = null,
|
||||||
|
val assistantMessageId: String? = null,
|
||||||
|
) : ServerEvent
|
||||||
|
data class ToolCall(val name: String, val status: String) : ServerEvent
|
||||||
|
data class Backpressure(val throttleMs: Int) : ServerEvent
|
||||||
|
data class GoAway(val remainingMs: Int, val reason: String) : ServerEvent
|
||||||
|
data class Error(val code: String, val message: String?) : ServerEvent
|
||||||
|
}
|
||||||
|
|
||||||
|
fun parseServerEvent(raw: String): ServerEvent? {
|
||||||
|
val obj: JsonObject
|
||||||
|
try {
|
||||||
|
obj = json.parseToJsonElement(raw).jsonObject
|
||||||
|
} catch (_: Exception) {
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
|
||||||
|
val type = obj["type"]?.jsonPrimitive?.content ?: return null
|
||||||
|
|
||||||
|
return when (type) {
|
||||||
|
"ready" -> ServerEvent.Ready(
|
||||||
|
sessionId = obj["sessionId"]?.jsonPrimitive?.content ?: return null,
|
||||||
|
)
|
||||||
|
"resumed" -> ServerEvent.Resumed(
|
||||||
|
sessionId = obj["sessionId"]?.jsonPrimitive?.content ?: return null,
|
||||||
|
)
|
||||||
|
"audio" -> ServerEvent.Audio(
|
||||||
|
data = obj["data"]?.jsonPrimitive?.content ?: return null,
|
||||||
|
seq = obj["seq"]?.jsonPrimitive?.intOrNull ?: 0,
|
||||||
|
)
|
||||||
|
"text" -> ServerEvent.Text(
|
||||||
|
delta = obj["delta"]?.jsonPrimitive?.content ?: return null,
|
||||||
|
seq = obj["seq"]?.jsonPrimitive?.intOrNull ?: 0,
|
||||||
|
)
|
||||||
|
"transcript" -> ServerEvent.Transcript(
|
||||||
|
text = obj["text"]?.jsonPrimitive?.content ?: "",
|
||||||
|
final = obj["final"]?.jsonPrimitive?.booleanOrNull ?: false,
|
||||||
|
)
|
||||||
|
"vad" -> ServerEvent.Vad(
|
||||||
|
speaking = obj["speaking"]?.jsonPrimitive?.booleanOrNull ?: false,
|
||||||
|
)
|
||||||
|
"interrupted" -> ServerEvent.Interrupted
|
||||||
|
"turn_end" -> {
|
||||||
|
val u = obj["usage"]?.jsonObject
|
||||||
|
ServerEvent.TurnEnd(
|
||||||
|
usage = LiveUsage(
|
||||||
|
inputTokens = u?.get("inputTokens")?.jsonPrimitive?.intOrNull ?: 0,
|
||||||
|
outputTokens = u?.get("outputTokens")?.jsonPrimitive?.intOrNull ?: 0,
|
||||||
|
cost = u?.get("cost")?.jsonPrimitive?.doubleOrNull,
|
||||||
|
durationMs = u?.get("durationMs")?.jsonPrimitive?.longOrNull ?: 0,
|
||||||
|
),
|
||||||
|
userText = obj["userText"]?.jsonPrimitive?.content ?: "",
|
||||||
|
assistantText = obj["assistantText"]?.jsonPrimitive?.content ?: "",
|
||||||
|
userMessageId = obj["userMessageId"]?.jsonPrimitive?.content,
|
||||||
|
assistantMessageId = obj["assistantMessageId"]?.jsonPrimitive?.content,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
"tool_call" -> ServerEvent.ToolCall(
|
||||||
|
name = obj["name"]?.jsonPrimitive?.content ?: "",
|
||||||
|
status = obj["status"]?.jsonPrimitive?.content ?: "",
|
||||||
|
)
|
||||||
|
"backpressure" -> ServerEvent.Backpressure(
|
||||||
|
throttleMs = obj["throttleMs"]?.jsonPrimitive?.intOrNull ?: 0,
|
||||||
|
)
|
||||||
|
"go_away" -> ServerEvent.GoAway(
|
||||||
|
remainingMs = obj["remainingMs"]?.jsonPrimitive?.intOrNull ?: 0,
|
||||||
|
reason = obj["reason"]?.jsonPrimitive?.content ?: "",
|
||||||
|
)
|
||||||
|
"error" -> ServerEvent.Error(
|
||||||
|
code = obj["code"]?.jsonPrimitive?.content ?: "unknown",
|
||||||
|
message = obj["message"]?.jsonPrimitive?.content,
|
||||||
|
)
|
||||||
|
else -> null
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -99,6 +99,7 @@ data class StoredMessage(
|
||||||
val id: String,
|
val id: String,
|
||||||
val role: String,
|
val role: String,
|
||||||
val content: String = "",
|
val content: String = "",
|
||||||
|
val parentId: String? = null,
|
||||||
val attachments: List<Attachment> = emptyList(),
|
val attachments: List<Attachment> = emptyList(),
|
||||||
val reasoning: String? = null,
|
val reasoning: String? = null,
|
||||||
val sources: List<SearchSource>? = null,
|
val sources: List<SearchSource>? = null,
|
||||||
|
|
@ -107,7 +108,17 @@ data class StoredMessage(
|
||||||
val outputTokens: Int? = null,
|
val outputTokens: Int? = null,
|
||||||
)
|
)
|
||||||
|
|
||||||
@Serializable private data class ConversationDetail(val messages: List<StoredMessage> = emptyList())
|
@Serializable private data class ConversationDetail(
|
||||||
|
val messages: List<StoredMessage> = emptyList(),
|
||||||
|
val activeRootId: String? = null,
|
||||||
|
val activeChild: Map<String, String> = emptyMap(),
|
||||||
|
)
|
||||||
|
|
||||||
|
data class ConversationData(
|
||||||
|
val messages: List<StoredMessage> = emptyList(),
|
||||||
|
val activeRootId: String? = null,
|
||||||
|
val activeChild: Map<String, String> = emptyMap(),
|
||||||
|
)
|
||||||
|
|
||||||
/** A message to persist. `parentId` chains the linear history; `kind` is always "chat" for the app. */
|
/** A message to persist. `parentId` chains the linear history; `kind` is always "chat" for the app. */
|
||||||
@Serializable
|
@Serializable
|
||||||
|
|
@ -309,6 +320,9 @@ object KaizenApi {
|
||||||
|
|
||||||
/** GET /api/v1/conversations/[id] — the stored messages, in order (empty on failure/locked). */
|
/** GET /api/v1/conversations/[id] — the stored messages, in order (empty on failure/locked). */
|
||||||
suspend fun fetchMessages(baseUrl: String, token: String, id: String): List<StoredMessage> =
|
suspend fun fetchMessages(baseUrl: String, token: String, id: String): List<StoredMessage> =
|
||||||
|
fetchConversationData(baseUrl, token, id).messages
|
||||||
|
|
||||||
|
suspend fun fetchConversationData(baseUrl: String, token: String, id: String): ConversationData =
|
||||||
withContext(Dispatchers.IO) {
|
withContext(Dispatchers.IO) {
|
||||||
val req = Request.Builder()
|
val req = Request.Builder()
|
||||||
.url("$baseUrl/api/v1/conversations/$id")
|
.url("$baseUrl/api/v1/conversations/$id")
|
||||||
|
|
@ -318,13 +332,14 @@ object KaizenApi {
|
||||||
client.newCall(req).execute().use { resp ->
|
client.newCall(req).execute().use { resp ->
|
||||||
if (!resp.isSuccessful) {
|
if (!resp.isSuccessful) {
|
||||||
Log.w(TAG, "fetchMessages($id): HTTP ${resp.code}")
|
Log.w(TAG, "fetchMessages($id): HTTP ${resp.code}")
|
||||||
return@use emptyList()
|
return@use ConversationData()
|
||||||
}
|
}
|
||||||
json.decodeFromString<ConversationDetail>(resp.body!!.string()).messages
|
val detail = json.decodeFromString<ConversationDetail>(resp.body!!.string())
|
||||||
|
ConversationData(detail.messages, detail.activeRootId, detail.activeChild)
|
||||||
}
|
}
|
||||||
} catch (e: Exception) {
|
} catch (e: Exception) {
|
||||||
Log.w(TAG, "fetchMessages($id): ${e.javaClass.simpleName}: ${e.message}")
|
Log.w(TAG, "fetchMessages($id): ${e.javaClass.simpleName}: ${e.message}")
|
||||||
emptyList()
|
ConversationData()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -348,17 +363,20 @@ object KaizenApi {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/** PATCH /api/v1/conversations/[id] — update title, locked, pinned. */
|
/** PATCH /api/v1/conversations/[id] — update title, locked, pinned, activeRootId, activeChild. */
|
||||||
suspend fun patchConversation(baseUrl: String, token: String, id: String, body: Map<String, Any?>): Boolean =
|
suspend fun patchConversation(baseUrl: String, token: String, id: String, body: Map<String, Any?>): Boolean =
|
||||||
withContext(Dispatchers.IO) {
|
withContext(Dispatchers.IO) {
|
||||||
val payload = json.encodeToString(kotlinx.serialization.json.JsonObject(
|
fun toJsonElement(v: Any?): kotlinx.serialization.json.JsonElement = when (v) {
|
||||||
body.mapValues { (_, v) ->
|
|
||||||
when (v) {
|
|
||||||
is String -> kotlinx.serialization.json.JsonPrimitive(v)
|
is String -> kotlinx.serialization.json.JsonPrimitive(v)
|
||||||
is Boolean -> kotlinx.serialization.json.JsonPrimitive(v)
|
is Boolean -> kotlinx.serialization.json.JsonPrimitive(v)
|
||||||
|
is Map<*, *> -> kotlinx.serialization.json.JsonObject(
|
||||||
|
v.entries.associate { (k, v2) -> k.toString() to toJsonElement(v2) }
|
||||||
|
)
|
||||||
|
null -> kotlinx.serialization.json.JsonNull
|
||||||
else -> kotlinx.serialization.json.JsonNull
|
else -> kotlinx.serialization.json.JsonNull
|
||||||
}
|
}
|
||||||
}
|
val payload = json.encodeToString(kotlinx.serialization.json.JsonObject(
|
||||||
|
body.mapValues { (_, v) -> toJsonElement(v) }
|
||||||
)).toRequestBody(jsonMedia)
|
)).toRequestBody(jsonMedia)
|
||||||
val req = Request.Builder()
|
val req = Request.Builder()
|
||||||
.url("$baseUrl/api/v1/conversations/$id")
|
.url("$baseUrl/api/v1/conversations/$id")
|
||||||
|
|
@ -414,6 +432,9 @@ object KaizenApi {
|
||||||
|
|
||||||
/** GET /api/v1/conversations/[id] with unlock token — fetch messages of a locked conversation. */
|
/** GET /api/v1/conversations/[id] with unlock token — fetch messages of a locked conversation. */
|
||||||
suspend fun fetchLockedMessages(baseUrl: String, token: String, id: String, unlockToken: String): List<StoredMessage> =
|
suspend fun fetchLockedMessages(baseUrl: String, token: String, id: String, unlockToken: String): List<StoredMessage> =
|
||||||
|
fetchLockedConversationData(baseUrl, token, id, unlockToken).messages
|
||||||
|
|
||||||
|
suspend fun fetchLockedConversationData(baseUrl: String, token: String, id: String, unlockToken: String): ConversationData =
|
||||||
withContext(Dispatchers.IO) {
|
withContext(Dispatchers.IO) {
|
||||||
val req = Request.Builder()
|
val req = Request.Builder()
|
||||||
.url("$baseUrl/api/v1/conversations/$id")
|
.url("$baseUrl/api/v1/conversations/$id")
|
||||||
|
|
@ -424,13 +445,14 @@ object KaizenApi {
|
||||||
client.newCall(req).execute().use { resp ->
|
client.newCall(req).execute().use { resp ->
|
||||||
if (!resp.isSuccessful) {
|
if (!resp.isSuccessful) {
|
||||||
Log.w(TAG, "fetchLockedMessages($id): HTTP ${resp.code}")
|
Log.w(TAG, "fetchLockedMessages($id): HTTP ${resp.code}")
|
||||||
return@use emptyList()
|
return@use ConversationData()
|
||||||
}
|
}
|
||||||
json.decodeFromString<ConversationDetail>(resp.body!!.string()).messages
|
val detail = json.decodeFromString<ConversationDetail>(resp.body!!.string())
|
||||||
|
ConversationData(detail.messages, detail.activeRootId, detail.activeChild)
|
||||||
}
|
}
|
||||||
} catch (e: Exception) {
|
} catch (e: Exception) {
|
||||||
Log.w(TAG, "fetchLockedMessages($id): ${e.javaClass.simpleName}: ${e.message}")
|
Log.w(TAG, "fetchLockedMessages($id): ${e.javaClass.simpleName}: ${e.message}")
|
||||||
emptyList()
|
ConversationData()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -22,6 +22,8 @@ object KaizenIcons {
|
||||||
// ── Navigation ──────────────────────────────────────────────────────────
|
// ── Navigation ──────────────────────────────────────────────────────────
|
||||||
val ArrowLeft: ImageVector get() = Lucide.ArrowLeft
|
val ArrowLeft: ImageVector get() = Lucide.ArrowLeft
|
||||||
val ChevronDown: ImageVector get() = Lucide.ChevronDown
|
val ChevronDown: ImageVector get() = Lucide.ChevronDown
|
||||||
|
val ChevronLeft: ImageVector get() = Lucide.ChevronLeft
|
||||||
|
val ChevronRight: ImageVector get() = Lucide.ChevronRight
|
||||||
val ChevronUp: ImageVector get() = Lucide.ChevronUp
|
val ChevronUp: ImageVector get() = Lucide.ChevronUp
|
||||||
val Menu: ImageVector get() = KaizenCustomIcons.Menu
|
val Menu: ImageVector get() = KaizenCustomIcons.Menu
|
||||||
val MoreVertical: ImageVector get() = Lucide.EllipsisVertical
|
val MoreVertical: ImageVector get() = Lucide.EllipsisVertical
|
||||||
|
|
@ -95,8 +97,9 @@ object KaizenIcons {
|
||||||
val ArrowDown: ImageVector get() = Lucide.ArrowDown
|
val ArrowDown: ImageVector get() = Lucide.ArrowDown
|
||||||
val ArrowUp: ImageVector get() = Lucide.ArrowUp
|
val ArrowUp: ImageVector get() = Lucide.ArrowUp
|
||||||
|
|
||||||
// ── Audio ──────────────────────────────────────────────────────────────
|
// ── Audio / Call ────────────────────────────────────────────────────────
|
||||||
val Volume2: ImageVector get() = Lucide.Volume2
|
val Volume2: ImageVector get() = Lucide.Volume2
|
||||||
|
val PhoneOff: ImageVector get() = Lucide.PhoneOff
|
||||||
|
|
||||||
// ── Custom (brand overrides) ────────────────────────────────────────────
|
// ── Custom (brand overrides) ────────────────────────────────────────────
|
||||||
// Add hand-drawn SVGs here as ImageVector literals when ready.
|
// Add hand-drawn SVGs here as ImageVector literals when ready.
|
||||||
|
|
|
||||||
|
|
@ -180,4 +180,12 @@
|
||||||
<string name="settings_language_system">System</string>
|
<string name="settings_language_system">System</string>
|
||||||
<string name="settings_language_de">Deutsch</string>
|
<string name="settings_language_de">Deutsch</string>
|
||||||
<string name="settings_language_en">English</string>
|
<string name="settings_language_en">English</string>
|
||||||
|
|
||||||
|
<!-- Live Call -->
|
||||||
|
<string name="live_connecting">Connecting…</string>
|
||||||
|
<string name="live_listening">Listening…</string>
|
||||||
|
<string name="live_ready">Just start talking</string>
|
||||||
|
<string name="live_speaking">Responding…</string>
|
||||||
|
<string name="live_end">End</string>
|
||||||
|
<string name="live_reconnecting">Reconnecting…</string>
|
||||||
</resources>
|
</resources>
|
||||||
|
|
|
||||||
|
|
@ -182,4 +182,12 @@
|
||||||
<string name="settings_language_system">System</string>
|
<string name="settings_language_system">System</string>
|
||||||
<string name="settings_language_de">Deutsch</string>
|
<string name="settings_language_de">Deutsch</string>
|
||||||
<string name="settings_language_en">English</string>
|
<string name="settings_language_en">English</string>
|
||||||
|
|
||||||
|
<!-- Live Call -->
|
||||||
|
<string name="live_connecting">Verbinde…</string>
|
||||||
|
<string name="live_listening">Hört zu…</string>
|
||||||
|
<string name="live_ready">Sprich einfach los</string>
|
||||||
|
<string name="live_speaking">Antwortet…</string>
|
||||||
|
<string name="live_end">Beenden</string>
|
||||||
|
<string name="live_reconnecting">Verbindung wird wiederhergestellt…</string>
|
||||||
</resources>
|
</resources>
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue