Compare commits
No commits in common. "da3eded81e368c083d12a0b9ebc41dc8381e3278" and "7ba42338c5a8851a4240b79e5b54ab03a09eed2c" have entirely different histories.
da3eded81e
...
7ba42338c5
16 changed files with 88 additions and 1607 deletions
1
.gitignore
vendored
1
.gitignore
vendored
|
|
@ -13,4 +13,3 @@
|
||||||
.externalNativeBuild
|
.externalNativeBuild
|
||||||
.cxx
|
.cxx
|
||||||
local.properties
|
local.properties
|
||||||
bugs/
|
|
||||||
|
|
|
||||||
|
|
@ -280,7 +280,7 @@ All committed on `main`. Compile-verified on Mac (`./gradlew :app:compileDebugKo
|
||||||
| `fb6bc98` | **Web search provider picker** — for Vertex models: Google vs Enterprise Web Search dropdown pill, `webSearchProvider` sent in chat request |
|
| `fb6bc98` | **Web search provider picker** — for Vertex models: Google vs Enterprise Web Search dropdown pill, `webSearchProvider` sent in chat request |
|
||||||
| `1899f2c` | **Voice message display** — audio-only user messages render as compact "🎤 Sprachnachricht" pill instead of ugly filename chip |
|
| `1899f2c` | **Voice message display** — audio-only user messages render as compact "🎤 Sprachnachricht" pill instead of ugly filename chip |
|
||||||
| `44ff82a` | **Voice message ordering fix** — caches user + assistant messages in Room with correct `sortOrder` after server save (was missing, caused wrong order on reload) |
|
| `44ff82a` | **Voice message ordering fix** — caches user + assistant messages in Room with correct `sortOrder` after server save (was missing, caused wrong order on reload) |
|
||||||
| `8f72afd` | **Edit and resend user messages** — SquarePen icon on user messages, creates branch (old messages preserved), text back in input field |
|
| `8f72afd` | **Edit and resend user messages** — SquarePen icon on user messages, removes message + all after it, puts text back in input field |
|
||||||
| `b2df7e9` | **Modern message action icons** — ClipboardCopy (copy), SquarePen (edit), RotateCcw (regenerate), 36dp touch target, 20dp icons |
|
| `b2df7e9` | **Modern message action icons** — ClipboardCopy (copy), SquarePen (edit), RotateCcw (regenerate), 36dp touch target, 20dp icons |
|
||||||
| `421b29e` | **ReasoningBlock + SourceCards redesign** — accent-tinted background + gradient borders, KaizenShapes.md (16dp radius), domain instead of full URL in source cards |
|
| `421b29e` | **ReasoningBlock + SourceCards redesign** — accent-tinted background + gradient borders, KaizenShapes.md (16dp radius), domain instead of full URL in source cards |
|
||||||
| `75ca8c0` | **Visual refresh** — body text W400→W300 (airier), headings scaled up (H1 26sp, H2 22sp, H3 18sp), `animateItem()` on messages (350ms fade-in, spring placement), blobs 20% larger + better distribution, higher alpha on OLED, light-mode factor 0.65→0.78, distinct `primaryDeep` per theme |
|
| `75ca8c0` | **Visual refresh** — body text W400→W300 (airier), headings scaled up (H1 26sp, H2 22sp, H3 18sp), `animateItem()` on messages (350ms fade-in, spring placement), blobs 20% larger + better distribution, higher alpha on OLED, light-mode factor 0.65→0.78, distinct `primaryDeep` per theme |
|
||||||
|
|
@ -292,7 +292,6 @@ All committed on `main`. Compile-verified on Mac (`./gradlew :app:compileDebugKo
|
||||||
| `1c9491e` | **Text selection** — `SelectionContainer` on user + assistant messages. Long-press to select, copy, share |
|
| `1c9491e` | **Text selection** — `SelectionContainer` on user + assistant messages. Long-press to select, copy, share |
|
||||||
| `1c9491e` | **Markdown spacing** — heading spacing 18→24/20dp, list items 8dp gap + muted bullets, blockquote 12dp inset |
|
| `1c9491e` | **Markdown spacing** — heading spacing 18→24/20dp, list items 8dp gap + muted bullets, blockquote 12dp inset |
|
||||||
| `430d5b6` | **Flat sidebar user row** — removed heavy card (background, border, shadow), replaced with flat row + thin separator |
|
| `430d5b6` | **Flat sidebar user row** — removed heavy card (background, border, shadow), replaced with flat row + thin separator |
|
||||||
| — | **Branch navigation (tree support)** — `ChatTree` data structure mirrors web frontend's tree. Edit creates a branch (sibling node with same parentId) instead of deleting messages. Regenerate creates a new assistant branch. `BranchNavigator` UI (chevron arrows + "1/3" counter) on messages with siblings. `activeRootId`/`activeChild` synced to server via PATCH. `StoredMessage.parentId` parsed from API. `patchConversation` handles JSON object values for `activeChild` |
|
|
||||||
|
|
||||||
**Earlier UI/feel work (Phase 0 prototype → feel-first):**
|
**Earlier UI/feel work (Phase 0 prototype → feel-first):**
|
||||||
- Liquid glass styling, floating panels, glassmorphic sidebar
|
- Liquid glass styling, floating panels, glassmorphic sidebar
|
||||||
|
|
|
||||||
|
|
@ -11,10 +11,6 @@
|
||||||
<!-- Required for voice recording (mic button in chat) -->
|
<!-- Required for voice recording (mic button in chat) -->
|
||||||
<uses-permission android:name="android.permission.RECORD_AUDIO" />
|
<uses-permission android:name="android.permission.RECORD_AUDIO" />
|
||||||
|
|
||||||
<!-- Required for Live Call foreground service (mic access while backgrounded) -->
|
|
||||||
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
|
|
||||||
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MICROPHONE" />
|
|
||||||
|
|
||||||
<application
|
<application
|
||||||
android:allowBackup="true"
|
android:allowBackup="true"
|
||||||
android:dataExtractionRules="@xml/data_extraction_rules"
|
android:dataExtractionRules="@xml/data_extraction_rules"
|
||||||
|
|
@ -36,11 +32,6 @@
|
||||||
<category android:name="android.intent.category.LAUNCHER" />
|
<category android:name="android.intent.category.LAUNCHER" />
|
||||||
</intent-filter>
|
</intent-filter>
|
||||||
</activity>
|
</activity>
|
||||||
|
|
||||||
<service
|
|
||||||
android:name=".live.LiveCallService"
|
|
||||||
android:foregroundServiceType="microphone"
|
|
||||||
android:exported="false" />
|
|
||||||
</application>
|
</application>
|
||||||
|
|
||||||
</manifest>
|
</manifest>
|
||||||
|
|
@ -437,51 +437,6 @@ fun TokenBadge(used: Int, contextLength: Int, modifier: Modifier = Modifier) {
|
||||||
color = cs.onSurfaceVariant,
|
color = cs.onSurfaceVariant,
|
||||||
fontSize = 12.sp,
|
fontSize = 12.sp,
|
||||||
fontWeight = FontWeight.Medium,
|
fontWeight = FontWeight.Medium,
|
||||||
maxLines = 1,
|
|
||||||
softWrap = false,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
@Composable
|
|
||||||
fun BranchNavigator(
|
|
||||||
branchInfo: BranchInfo,
|
|
||||||
onNavigate: (Int) -> Unit,
|
|
||||||
modifier: Modifier = Modifier,
|
|
||||||
) {
|
|
||||||
val cs = MaterialTheme.colorScheme
|
|
||||||
Row(
|
|
||||||
modifier = modifier,
|
|
||||||
verticalAlignment = Alignment.CenterVertically,
|
|
||||||
horizontalArrangement = Arrangement.spacedBy(2.dp),
|
|
||||||
) {
|
|
||||||
Box(
|
|
||||||
Modifier.size(28.dp).clip(KaizenShapes.circle)
|
|
||||||
.clickable(enabled = branchInfo.index > 0) { onNavigate(-1) },
|
|
||||||
contentAlignment = Alignment.Center,
|
|
||||||
) {
|
|
||||||
Icon(
|
|
||||||
KaizenIcons.ChevronLeft, null,
|
|
||||||
tint = cs.onSurfaceVariant.copy(alpha = if (branchInfo.index > 0) 0.65f else 0.20f),
|
|
||||||
modifier = Modifier.size(16.dp),
|
|
||||||
)
|
|
||||||
}
|
|
||||||
Text(
|
|
||||||
"${branchInfo.index + 1}/${branchInfo.total}",
|
|
||||||
color = cs.onSurfaceVariant.copy(alpha = 0.55f),
|
|
||||||
fontSize = 12.sp,
|
|
||||||
fontWeight = FontWeight.Medium,
|
|
||||||
)
|
|
||||||
Box(
|
|
||||||
Modifier.size(28.dp).clip(KaizenShapes.circle)
|
|
||||||
.clickable(enabled = branchInfo.index < branchInfo.total - 1) { onNavigate(1) },
|
|
||||||
contentAlignment = Alignment.Center,
|
|
||||||
) {
|
|
||||||
Icon(
|
|
||||||
KaizenIcons.ChevronRight, null,
|
|
||||||
tint = cs.onSurfaceVariant.copy(alpha = if (branchInfo.index < branchInfo.total - 1) 0.65f else 0.20f),
|
|
||||||
modifier = Modifier.size(16.dp),
|
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -496,8 +451,6 @@ fun MessageRow(
|
||||||
onEdit: ((String, String) -> Unit)? = null,
|
onEdit: ((String, String) -> Unit)? = null,
|
||||||
onReadAloud: ((String) -> Unit)? = null,
|
onReadAloud: ((String) -> Unit)? = null,
|
||||||
isPlayingTts: Boolean = false,
|
isPlayingTts: Boolean = false,
|
||||||
branchInfo: BranchInfo? = null,
|
|
||||||
onNavigateBranch: ((String, Int) -> Unit)? = null,
|
|
||||||
) {
|
) {
|
||||||
val cs = MaterialTheme.colorScheme
|
val cs = MaterialTheme.colorScheme
|
||||||
val isDark = isSystemInDarkTheme()
|
val isDark = isSystemInDarkTheme()
|
||||||
|
|
@ -582,12 +535,7 @@ fun MessageRow(
|
||||||
Row(
|
Row(
|
||||||
Modifier.padding(top = 6.dp, end = 4.dp),
|
Modifier.padding(top = 6.dp, end = 4.dp),
|
||||||
horizontalArrangement = Arrangement.spacedBy(2.dp),
|
horizontalArrangement = Arrangement.spacedBy(2.dp),
|
||||||
verticalAlignment = Alignment.CenterVertically,
|
|
||||||
) {
|
) {
|
||||||
if (branchInfo != null && onNavigateBranch != null) {
|
|
||||||
BranchNavigator(branchInfo, onNavigate = { dir -> onNavigateBranch(message.wireId, dir) })
|
|
||||||
Spacer(Modifier.width(4.dp))
|
|
||||||
}
|
|
||||||
MessageActionButton(
|
MessageActionButton(
|
||||||
icon = if (copied) KaizenIcons.Check else KaizenIcons.ClipboardCopy,
|
icon = if (copied) KaizenIcons.Check else KaizenIcons.ClipboardCopy,
|
||||||
tint = if (copied) KaizenSemanticColors.success else cs.onSurfaceVariant.copy(alpha = 0.55f),
|
tint = if (copied) KaizenSemanticColors.success else cs.onSurfaceVariant.copy(alpha = 0.55f),
|
||||||
|
|
@ -649,12 +597,7 @@ fun MessageRow(
|
||||||
Row(
|
Row(
|
||||||
Modifier.padding(start = 38.dp, top = 6.dp),
|
Modifier.padding(start = 38.dp, top = 6.dp),
|
||||||
horizontalArrangement = Arrangement.spacedBy(2.dp),
|
horizontalArrangement = Arrangement.spacedBy(2.dp),
|
||||||
verticalAlignment = Alignment.CenterVertically,
|
|
||||||
) {
|
) {
|
||||||
if (branchInfo != null && onNavigateBranch != null) {
|
|
||||||
BranchNavigator(branchInfo, onNavigate = { dir -> onNavigateBranch(message.wireId, dir) })
|
|
||||||
Spacer(Modifier.width(4.dp))
|
|
||||||
}
|
|
||||||
MessageActionButton(
|
MessageActionButton(
|
||||||
icon = if (copied) KaizenIcons.Check else KaizenIcons.ClipboardCopy,
|
icon = if (copied) KaizenIcons.Check else KaizenIcons.ClipboardCopy,
|
||||||
tint = if (copied) KaizenSemanticColors.success else cs.onSurfaceVariant.copy(alpha = 0.55f),
|
tint = if (copied) KaizenSemanticColors.success else cs.onSurfaceVariant.copy(alpha = 0.55f),
|
||||||
|
|
@ -791,7 +734,6 @@ fun ChatInput(
|
||||||
isRecording: Boolean = false,
|
isRecording: Boolean = false,
|
||||||
recordingAmplitude: Float = 0f,
|
recordingAmplitude: Float = 0f,
|
||||||
onMicClick: () -> Unit = {},
|
onMicClick: () -> Unit = {},
|
||||||
onCallClick: () -> Unit = {},
|
|
||||||
pendingFiles: List<PendingFile> = emptyList(),
|
pendingFiles: List<PendingFile> = emptyList(),
|
||||||
onAddClick: () -> Unit = {},
|
onAddClick: () -> Unit = {},
|
||||||
onRemoveFile: (String) -> Unit = {},
|
onRemoveFile: (String) -> Unit = {},
|
||||||
|
|
@ -923,7 +865,7 @@ fun ChatInput(
|
||||||
} else if (hasContent) {
|
} else if (hasContent) {
|
||||||
SendButton(enabled = canSend, onClick = onSend)
|
SendButton(enabled = canSend, onClick = onSend)
|
||||||
} else {
|
} else {
|
||||||
CallButton(onClick = onCallClick)
|
CallButton(onClick = { /* TODO: call feature */ })
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -42,7 +42,6 @@ data class Message(
|
||||||
val tools: List<ToolEvent> = emptyList(),
|
val tools: List<ToolEvent> = emptyList(),
|
||||||
val query: String = "",
|
val query: String = "",
|
||||||
val sources: List<SearchSource> = emptyList(),
|
val sources: List<SearchSource> = emptyList(),
|
||||||
val branchInfo: BranchInfo? = null,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
data class Suggestion(val labelRes: Int, val promptRes: Int, val icon: ImageVector)
|
data class Suggestion(val labelRes: Int, val promptRes: Int, val icon: ImageVector)
|
||||||
|
|
|
||||||
|
|
@ -92,17 +92,11 @@ import androidx.core.content.ContextCompat
|
||||||
import androidx.compose.animation.core.Spring
|
import androidx.compose.animation.core.Spring
|
||||||
import androidx.compose.animation.core.spring
|
import androidx.compose.animation.core.spring
|
||||||
import androidx.compose.animation.core.tween
|
import androidx.compose.animation.core.tween
|
||||||
import android.content.ComponentName
|
|
||||||
import android.content.ServiceConnection
|
|
||||||
import android.os.IBinder
|
|
||||||
import kotlinx.coroutines.Job
|
import kotlinx.coroutines.Job
|
||||||
import kotlinx.coroutines.async
|
import kotlinx.coroutines.async
|
||||||
import kotlinx.coroutines.launch
|
import kotlinx.coroutines.launch
|
||||||
import java.io.ByteArrayOutputStream
|
import java.io.ByteArrayOutputStream
|
||||||
import java.io.File
|
import java.io.File
|
||||||
import dev.kaizen.app.live.CallStatus
|
|
||||||
import dev.kaizen.app.live.LiveCallOverlay
|
|
||||||
import dev.kaizen.app.live.LiveCallService
|
|
||||||
|
|
||||||
// Screen enumeration for modular state-driven navigation (Zero Technical Debt)
|
// Screen enumeration for modular state-driven navigation (Zero Technical Debt)
|
||||||
enum class AppScreen { Chat, Settings }
|
enum class AppScreen { Chat, Settings }
|
||||||
|
|
@ -129,47 +123,12 @@ fun ChatScreen(
|
||||||
var chatModel by remember { mutableStateOf<String?>(null) }
|
var chatModel by remember { mutableStateOf<String?>(null) }
|
||||||
var streamJob by remember { mutableStateOf<Job?>(null) }
|
var streamJob by remember { mutableStateOf<Job?>(null) }
|
||||||
|
|
||||||
// Tree state for branch navigation
|
|
||||||
var chatTree by remember { mutableStateOf(ChatTree.EMPTY) }
|
|
||||||
val storedMessagesMap = remember { mutableMapOf<String, dev.kaizen.app.net.StoredMessage>() }
|
|
||||||
var editParentId by remember { mutableStateOf<String?>(null) }
|
|
||||||
|
|
||||||
fun rebuildVisibleMessages() {
|
|
||||||
val path = getVisiblePath(chatTree)
|
|
||||||
messages.clear()
|
|
||||||
for (entry in path) {
|
|
||||||
val sm = storedMessagesMap[entry.messageId] ?: continue
|
|
||||||
messages.add(
|
|
||||||
Message(
|
|
||||||
id = nextId++,
|
|
||||||
role = if (sm.role == "assistant") Role.Assistant else Role.User,
|
|
||||||
content = sm.content,
|
|
||||||
wireId = sm.id,
|
|
||||||
attachments = sm.attachments,
|
|
||||||
reasoning = sm.reasoning ?: "",
|
|
||||||
sources = sm.sources ?: emptyList(),
|
|
||||||
branchInfo = entry.branchInfo,
|
|
||||||
),
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Audio recording state
|
// Audio recording state
|
||||||
var isRecording by remember { mutableStateOf(false) }
|
var isRecording by remember { mutableStateOf(false) }
|
||||||
val recorderRef = remember { mutableStateOf<MediaRecorder?>(null) }
|
val recorderRef = remember { mutableStateOf<MediaRecorder?>(null) }
|
||||||
val audioFileRef = remember { mutableStateOf<File?>(null) }
|
val audioFileRef = remember { mutableStateOf<File?>(null) }
|
||||||
var recordingAmplitude by remember { mutableStateOf(0f) }
|
var recordingAmplitude by remember { mutableStateOf(0f) }
|
||||||
|
|
||||||
// ─── Live Call state (uses context/ttsVoice/currentConversationId declared below) ───
|
|
||||||
var liveService by remember { mutableStateOf<LiveCallService?>(null) }
|
|
||||||
var liveCallActive by remember { mutableStateOf(false) }
|
|
||||||
val liveStatus by liveService?.status?.collectAsState() ?: remember { mutableStateOf(CallStatus.IDLE) }
|
|
||||||
val liveSubtitle by liveService?.subtitle?.collectAsState() ?: remember { mutableStateOf("") }
|
|
||||||
val liveAmplitude by liveService?.amplitude?.collectAsState() ?: remember { mutableStateOf(0f) }
|
|
||||||
val liveUserSpeaking by liveService?.userSpeaking?.collectAsState() ?: remember { mutableStateOf(false) }
|
|
||||||
var liveStartedAt by remember { mutableStateOf(0L) }
|
|
||||||
var liveElapsedMs by remember { mutableStateOf(0L) }
|
|
||||||
|
|
||||||
LaunchedEffect(isRecording) {
|
LaunchedEffect(isRecording) {
|
||||||
if (!isRecording) { recordingAmplitude = 0f; return@LaunchedEffect }
|
if (!isRecording) { recordingAmplitude = 0f; return@LaunchedEffect }
|
||||||
while (isRecording) {
|
while (isRecording) {
|
||||||
|
|
@ -217,144 +176,6 @@ fun ChatScreen(
|
||||||
val pendingFiles = remember { mutableStateListOf<PendingFile>() }
|
val pendingFiles = remember { mutableStateListOf<PendingFile>() }
|
||||||
val context = LocalContext.current
|
val context = LocalContext.current
|
||||||
|
|
||||||
// ─── Live Call service binding + control ──────────────────────────────
|
|
||||||
val liveServiceConnection = remember {
|
|
||||||
object : ServiceConnection {
|
|
||||||
override fun onServiceConnected(name: ComponentName?, binder: IBinder?) {
|
|
||||||
liveService = (binder as? LiveCallService.LocalBinder)?.service
|
|
||||||
}
|
|
||||||
override fun onServiceDisconnected(name: ComponentName?) {
|
|
||||||
liveService = null
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
LaunchedEffect(liveCallActive) {
|
|
||||||
if (liveCallActive) {
|
|
||||||
liveStartedAt = System.currentTimeMillis()
|
|
||||||
while (liveCallActive) {
|
|
||||||
liveElapsedMs = System.currentTimeMillis() - liveStartedAt
|
|
||||||
kotlinx.coroutines.delay(500)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
val liveAvailable = models.any { it.id.startsWith("vertex:") && it.id.contains("live") }
|
|
||||||
|
|
||||||
fun startLiveCallInner() {
|
|
||||||
val cfg = session.config ?: return
|
|
||||||
liveCallActive = true
|
|
||||||
|
|
||||||
val intent = LiveCallService.intent(context)
|
|
||||||
context.startForegroundService(intent)
|
|
||||||
context.bindService(intent, liveServiceConnection, 0)
|
|
||||||
|
|
||||||
scope.launch {
|
|
||||||
var attempts = 0
|
|
||||||
while (liveService == null && attempts < 20) {
|
|
||||||
kotlinx.coroutines.delay(50)
|
|
||||||
attempts++
|
|
||||||
}
|
|
||||||
|
|
||||||
val svc = liveService ?: return@launch
|
|
||||||
|
|
||||||
// Create conversation if needed (same pattern as text chat)
|
|
||||||
val cid = conversationId ?: run {
|
|
||||||
val newId = KaizenApi.createConversation(cfg.baseUrl, cfg.token)
|
|
||||||
if (newId != null) {
|
|
||||||
conversationId = newId
|
|
||||||
chat.conversationRepo.insertOptimistic(newId, "Live Call")
|
|
||||||
}
|
|
||||||
newId
|
|
||||||
}
|
|
||||||
|
|
||||||
svc.onTurnEnd = { turn ->
|
|
||||||
scope.launch {
|
|
||||||
val convId = conversationId ?: return@launch
|
|
||||||
|
|
||||||
// Add messages to local UI
|
|
||||||
if (turn.userText.isNotEmpty()) {
|
|
||||||
val userMsg = Message(nextId++, Role.User, turn.userText, wireId = turn.userMessageId ?: java.util.UUID.randomUUID().toString())
|
|
||||||
messages.add(userMsg)
|
|
||||||
}
|
|
||||||
if (turn.assistantText.isNotEmpty()) {
|
|
||||||
val aMsg = Message(nextId++, Role.Assistant, turn.assistantText, wireId = turn.assistantMessageId ?: java.util.UUID.randomUUID().toString())
|
|
||||||
messages.add(aMsg)
|
|
||||||
chatModel = cfg.model
|
|
||||||
usedTokens = turn.usage.inputTokens + turn.usage.outputTokens
|
|
||||||
}
|
|
||||||
|
|
||||||
// Cache in Room (server already persisted via WS-Server)
|
|
||||||
val msgCount = messages.size
|
|
||||||
val entitiesToCache = mutableListOf<dev.kaizen.app.db.MessageEntity>()
|
|
||||||
if (turn.userMessageId != null && turn.userText.isNotEmpty()) {
|
|
||||||
entitiesToCache.add(dev.kaizen.app.db.MessageEntity(
|
|
||||||
id = turn.userMessageId, conversationId = convId,
|
|
||||||
role = "user", content = turn.userText,
|
|
||||||
attachments = emptyList(), sortOrder = msgCount - 2,
|
|
||||||
))
|
|
||||||
}
|
|
||||||
if (turn.assistantMessageId != null && turn.assistantText.isNotEmpty()) {
|
|
||||||
entitiesToCache.add(dev.kaizen.app.db.MessageEntity(
|
|
||||||
id = turn.assistantMessageId, conversationId = convId,
|
|
||||||
role = "assistant", content = turn.assistantText,
|
|
||||||
attachments = emptyList(), sortOrder = msgCount - 1,
|
|
||||||
))
|
|
||||||
}
|
|
||||||
if (entitiesToCache.isNotEmpty()) {
|
|
||||||
chat.messageRepo.cacheMessages(convId, entitiesToCache)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
svc.onError = { code, message ->
|
|
||||||
scope.launch {
|
|
||||||
val errorText = message ?: code
|
|
||||||
Toast.makeText(context, errorText, Toast.LENGTH_SHORT).show()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
svc.startCall(
|
|
||||||
baseUrl = cfg.baseUrl,
|
|
||||||
token = cfg.token,
|
|
||||||
model = cfg.model,
|
|
||||||
voice = ttsVoice,
|
|
||||||
conversationId = cid,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
val liveCallPermission = rememberLauncherForActivityResult(ActivityResultContracts.RequestPermission()) { granted ->
|
|
||||||
if (granted) startLiveCallInner()
|
|
||||||
else Toast.makeText(context, context.getString(R.string.chat_mic_permission), Toast.LENGTH_SHORT).show()
|
|
||||||
}
|
|
||||||
|
|
||||||
fun startLiveCall() {
|
|
||||||
if (ContextCompat.checkSelfPermission(context, Manifest.permission.RECORD_AUDIO) != PackageManager.PERMISSION_GRANTED) {
|
|
||||||
liveCallPermission.launch(Manifest.permission.RECORD_AUDIO)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
startLiveCallInner()
|
|
||||||
}
|
|
||||||
|
|
||||||
fun endLiveCall() {
|
|
||||||
liveService?.onTurnEnd = null
|
|
||||||
liveService?.onError = null
|
|
||||||
liveService?.endCall()
|
|
||||||
liveCallActive = false
|
|
||||||
try { context.unbindService(liveServiceConnection) } catch (_: Exception) {}
|
|
||||||
liveService = null
|
|
||||||
|
|
||||||
// Generate title for new conversations
|
|
||||||
val cfg = session.config ?: return
|
|
||||||
val cid = conversationId ?: return
|
|
||||||
scope.launch {
|
|
||||||
val title = KaizenApi.generateTitle(cfg.baseUrl, cfg.token, cid)
|
|
||||||
if (title != null) chat.conversationRepo.updateTitle(cid, title)
|
|
||||||
chat.conversationRepo.refresh(cfg.baseUrl, cfg.token)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
var showAttachMenu by remember { mutableStateOf(false) }
|
var showAttachMenu by remember { mutableStateOf(false) }
|
||||||
|
|
||||||
fun uploadPickedFile(name: String, mimeType: String, bytes: ByteArray) {
|
fun uploadPickedFile(name: String, mimeType: String, bytes: ByteArray) {
|
||||||
|
|
@ -488,17 +309,6 @@ fun ChatScreen(
|
||||||
sources = finalSources?.map { dev.kaizen.app.net.SearchSource(it.title, it.url, it.snippet) } ?: emptyList(),
|
sources = finalSources?.map { dev.kaizen.app.net.SearchSource(it.title, it.url, it.snippet) } ?: emptyList(),
|
||||||
),
|
),
|
||||||
))
|
))
|
||||||
storedMessagesMap[userMsg.wireId] = dev.kaizen.app.net.StoredMessage(
|
|
||||||
id = userMsg.wireId, role = "user", content = "",
|
|
||||||
parentId = userParentId, attachments = listOf(att),
|
|
||||||
)
|
|
||||||
storedMessagesMap[assistantWireId] = dev.kaizen.app.net.StoredMessage(
|
|
||||||
id = assistantWireId, role = "assistant", content = finalContent,
|
|
||||||
parentId = userMsg.wireId, model = cfg.model,
|
|
||||||
reasoning = finalReasoning,
|
|
||||||
sources = finalSources?.map { dev.kaizen.app.net.SearchSource(it.title, it.url, it.snippet) },
|
|
||||||
)
|
|
||||||
chatTree = buildChatTree(storedMessagesMap.values.toList(), chatTree.activeRootId, chatTree.activeChild)
|
|
||||||
}
|
}
|
||||||
if (convIdDeferred != null) {
|
if (convIdDeferred != null) {
|
||||||
chat.conversationRepo.insertOptimistic(cid, "(Voice)")
|
chat.conversationRepo.insertOptimistic(cid, "(Voice)")
|
||||||
|
|
@ -524,42 +334,26 @@ fun ChatScreen(
|
||||||
}
|
}
|
||||||
|
|
||||||
fun playTtsUrl(url: String, messageId: Long) {
|
fun playTtsUrl(url: String, messageId: Long) {
|
||||||
scope.launch {
|
try {
|
||||||
try {
|
val player = MediaPlayer().apply {
|
||||||
val tmpFile = kotlinx.coroutines.withContext(kotlinx.coroutines.Dispatchers.IO) {
|
setDataSource(url)
|
||||||
val req = okhttp3.Request.Builder().url(url).build()
|
prepareAsync()
|
||||||
KaizenApi.imageClient.newCall(req).execute().use { resp ->
|
setOnPreparedListener { start() }
|
||||||
if (!resp.isSuccessful) return@withContext null
|
setOnCompletionListener {
|
||||||
val bytes = resp.body!!.bytes()
|
release()
|
||||||
java.io.File.createTempFile("tts_", ".wav", context.cacheDir).also { it.writeBytes(bytes) }
|
ttsPlayerRef.value = null
|
||||||
}
|
|
||||||
}
|
|
||||||
if (tmpFile == null) {
|
|
||||||
playingTtsForId = null
|
playingTtsForId = null
|
||||||
return@launch
|
|
||||||
}
|
}
|
||||||
val player = MediaPlayer().apply {
|
setOnErrorListener { _, _, _ ->
|
||||||
setOnPreparedListener { start() }
|
release()
|
||||||
setOnCompletionListener {
|
ttsPlayerRef.value = null
|
||||||
release()
|
playingTtsForId = null
|
||||||
ttsPlayerRef.value = null
|
true
|
||||||
playingTtsForId = null
|
|
||||||
tmpFile.delete()
|
|
||||||
}
|
|
||||||
setOnErrorListener { _, _, _ ->
|
|
||||||
release()
|
|
||||||
ttsPlayerRef.value = null
|
|
||||||
playingTtsForId = null
|
|
||||||
tmpFile.delete()
|
|
||||||
true
|
|
||||||
}
|
|
||||||
setDataSource(tmpFile.absolutePath)
|
|
||||||
prepareAsync()
|
|
||||||
}
|
}
|
||||||
ttsPlayerRef.value = player
|
|
||||||
} catch (_: Exception) {
|
|
||||||
playingTtsForId = null
|
|
||||||
}
|
}
|
||||||
|
ttsPlayerRef.value = player
|
||||||
|
} catch (_: Exception) {
|
||||||
|
playingTtsForId = null
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -649,8 +443,6 @@ fun ChatScreen(
|
||||||
viewingLockedChat = false
|
viewingLockedChat = false
|
||||||
usedTokens = 0
|
usedTokens = 0
|
||||||
chatModel = null
|
chatModel = null
|
||||||
chatTree = ChatTree.EMPTY
|
|
||||||
storedMessagesMap.clear()
|
|
||||||
}
|
}
|
||||||
|
|
||||||
fun unlockAndOpen(summary: ConversationSummary, password: String? = null) {
|
fun unlockAndOpen(summary: ConversationSummary, password: String? = null) {
|
||||||
|
|
@ -670,12 +462,22 @@ fun ChatScreen(
|
||||||
passwordUnlockTarget = null
|
passwordUnlockTarget = null
|
||||||
passwordUnlockError = null
|
passwordUnlockError = null
|
||||||
passwordUnlockForToggle = false
|
passwordUnlockForToggle = false
|
||||||
val data = KaizenApi.fetchLockedConversationData(cfg.baseUrl, cfg.token, summary.id, unlockToken)
|
val stored = KaizenApi.fetchLockedMessages(cfg.baseUrl, cfg.token, summary.id, unlockToken)
|
||||||
if (data.messages.isNotEmpty()) {
|
if (stored.isNotEmpty()) {
|
||||||
storedMessagesMap.clear()
|
messages.clear()
|
||||||
data.messages.forEach { storedMessagesMap[it.id] = it }
|
stored.forEach { m ->
|
||||||
chatTree = buildChatTree(data.messages, data.activeRootId, data.activeChild)
|
messages.add(
|
||||||
rebuildVisibleMessages()
|
Message(
|
||||||
|
id = nextId++,
|
||||||
|
role = if (m.role == "assistant") Role.Assistant else Role.User,
|
||||||
|
content = m.content,
|
||||||
|
wireId = m.id,
|
||||||
|
attachments = m.attachments,
|
||||||
|
reasoning = m.reasoning ?: "",
|
||||||
|
sources = m.sources ?: emptyList(),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
}
|
||||||
conversationId = summary.id
|
conversationId = summary.id
|
||||||
viewingLockedChat = true
|
viewingLockedChat = true
|
||||||
}
|
}
|
||||||
|
|
@ -744,36 +546,46 @@ fun ChatScreen(
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
conversationId = summary.id
|
conversationId = summary.id
|
||||||
val data = KaizenApi.fetchConversationData(cfg.baseUrl, cfg.token, summary.id)
|
val stored = KaizenApi.fetchMessages(cfg.baseUrl, cfg.token, summary.id)
|
||||||
if (data.messages.isNotEmpty()) {
|
if (stored.isNotEmpty()) {
|
||||||
val entities = data.messages.mapIndexed { i, m -> m.toEntity(summary.id, i) }
|
val entities = stored.mapIndexed { i, m -> m.toEntity(summary.id, i) }
|
||||||
chat.messageRepo.cacheMessages(summary.id, entities)
|
chat.messageRepo.cacheMessages(summary.id, entities)
|
||||||
val lastAssistant = data.messages.lastOrNull { it.role == "assistant" }
|
val lastAssistant = stored.lastOrNull { it.role == "assistant" }
|
||||||
lastAssistant?.model?.let { chatModel = it }
|
lastAssistant?.model?.let { chatModel = it }
|
||||||
usedTokens = data.messages.sumOf { (it.inputTokens ?: 0) + (it.outputTokens ?: 0) }
|
usedTokens = stored.sumOf { (it.inputTokens ?: 0) + (it.outputTokens ?: 0) }
|
||||||
|
if (stored.size != cachedList.size || stored.zip(cachedList).any { (s, c) -> s.id != c.id || s.content != c.content }) {
|
||||||
storedMessagesMap.clear()
|
messages.clear()
|
||||||
data.messages.forEach { storedMessagesMap[it.id] = it }
|
stored.forEach { m ->
|
||||||
chatTree = buildChatTree(data.messages, data.activeRootId, data.activeChild)
|
messages.add(
|
||||||
rebuildVisibleMessages()
|
Message(
|
||||||
|
id = nextId++,
|
||||||
|
role = if (m.role == "assistant") Role.Assistant else Role.User,
|
||||||
|
content = m.content,
|
||||||
|
wireId = m.id,
|
||||||
|
attachments = m.attachments,
|
||||||
|
reasoning = m.reasoning ?: "",
|
||||||
|
sources = m.sources ?: emptyList(),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
fun send(text: String, branchParentId: String? = null, branchAttachments: List<Attachment> = emptyList()) {
|
fun send(text: String) {
|
||||||
val trimmed = text.trim()
|
val trimmed = text.trim()
|
||||||
val uploadedAtts = if (branchAttachments.isNotEmpty()) branchAttachments else pendingFiles.mapNotNull { it.uploaded }
|
val uploadedAtts = pendingFiles.mapNotNull { it.uploaded }
|
||||||
if (trimmed.isEmpty() && uploadedAtts.isEmpty()) return
|
if (trimmed.isEmpty() && uploadedAtts.isEmpty()) return
|
||||||
if (isStreaming) return
|
if (isStreaming) return
|
||||||
val cfg = session.config ?: return
|
val cfg = session.config ?: return
|
||||||
haptics.click()
|
haptics.click()
|
||||||
|
|
||||||
val userParentId = editParentId ?: branchParentId ?: messages.lastOrNull()?.wireId
|
val userParentId = messages.lastOrNull()?.wireId
|
||||||
editParentId = null
|
|
||||||
val userMsg = Message(nextId++, Role.User, trimmed, attachments = uploadedAtts)
|
val userMsg = Message(nextId++, Role.User, trimmed, attachments = uploadedAtts)
|
||||||
messages.add(userMsg)
|
messages.add(userMsg)
|
||||||
input = ""
|
input = ""
|
||||||
if (branchAttachments.isEmpty()) pendingFiles.clear()
|
pendingFiles.clear()
|
||||||
val assistantId = nextId++
|
val assistantId = nextId++
|
||||||
val assistantWireId = java.util.UUID.randomUUID().toString()
|
val assistantWireId = java.util.UUID.randomUUID().toString()
|
||||||
messages.add(Message(assistantId, Role.Assistant, "", streaming = true, thinking = true, wireId = assistantWireId))
|
messages.add(Message(assistantId, Role.Assistant, "", streaming = true, thinking = true, wireId = assistantWireId))
|
||||||
|
|
@ -869,24 +681,6 @@ fun ChatScreen(
|
||||||
sources = finalSources ?: emptyList(),
|
sources = finalSources ?: emptyList(),
|
||||||
),
|
),
|
||||||
))
|
))
|
||||||
|
|
||||||
// Update tree with the new nodes
|
|
||||||
storedMessagesMap[userMsg.wireId] = dev.kaizen.app.net.StoredMessage(
|
|
||||||
id = userMsg.wireId, role = "user", content = trimmed,
|
|
||||||
parentId = userParentId, attachments = uploadedAtts,
|
|
||||||
)
|
|
||||||
storedMessagesMap[assistantWireId] = dev.kaizen.app.net.StoredMessage(
|
|
||||||
id = assistantWireId, role = "assistant", content = finalContent,
|
|
||||||
parentId = userMsg.wireId, model = cfg.model,
|
|
||||||
reasoning = finalReasoning,
|
|
||||||
sources = finalSources?.map { dev.kaizen.app.net.SearchSource(it.title, it.url, it.snippet) },
|
|
||||||
)
|
|
||||||
chatTree = buildChatTree(
|
|
||||||
storedMessagesMap.values.toList(),
|
|
||||||
chatTree.activeRootId,
|
|
||||||
chatTree.activeChild,
|
|
||||||
)
|
|
||||||
|
|
||||||
if (wasNewConversation) {
|
if (wasNewConversation) {
|
||||||
val title = KaizenApi.generateTitle(cfg.baseUrl, cfg.token, convId)
|
val title = KaizenApi.generateTitle(cfg.baseUrl, cfg.token, convId)
|
||||||
if (title != null) {
|
if (title != null) {
|
||||||
|
|
@ -1094,22 +888,24 @@ fun ChatScreen(
|
||||||
onRegenerate = if (message.role == Role.Assistant && !isStreaming) { _ ->
|
onRegenerate = if (message.role == Role.Assistant && !isStreaming) { _ ->
|
||||||
val lastUserIdx = messages.indexOfLast { it.role == Role.User && messages.indexOf(it) < messages.indexOf(message) }
|
val lastUserIdx = messages.indexOfLast { it.role == Role.User && messages.indexOf(it) < messages.indexOf(message) }
|
||||||
if (lastUserIdx >= 0) {
|
if (lastUserIdx >= 0) {
|
||||||
val userMsg = messages[lastUserIdx]
|
val userContent = messages[lastUserIdx].content
|
||||||
val parentId = chatTree.nodes[userMsg.wireId]?.parentId
|
messages.removeAt(messages.indexOf(message))
|
||||||
val msgIdx = messages.indexOf(message)
|
send(userContent)
|
||||||
if (msgIdx >= 0) {
|
|
||||||
messages.subList(msgIdx, messages.size).clear()
|
|
||||||
}
|
|
||||||
send(userMsg.content, branchParentId = parentId, branchAttachments = userMsg.attachments)
|
|
||||||
}
|
}
|
||||||
} else null,
|
} else null,
|
||||||
onEdit = if (message.role == Role.User && !isStreaming) { wireId, content ->
|
onEdit = if (message.role == Role.User && !isStreaming) { wireId, content ->
|
||||||
val node = chatTree.nodes[wireId]
|
|
||||||
val parentId = node?.parentId
|
|
||||||
val idx = messages.indexOfFirst { it.wireId == wireId }
|
val idx = messages.indexOfFirst { it.wireId == wireId }
|
||||||
if (idx >= 0) {
|
if (idx >= 0) {
|
||||||
messages.subList(idx, messages.size).clear()
|
val cfg = session.config ?: return@MessageRow
|
||||||
editParentId = parentId
|
val toRemove = messages.subList(idx, messages.size).map { it.wireId }
|
||||||
|
messages.removeAll { it.wireId in toRemove }
|
||||||
|
if (conversationId != null) {
|
||||||
|
scope.launch {
|
||||||
|
toRemove.forEach { id ->
|
||||||
|
KaizenApi.deleteMessage(cfg.baseUrl, cfg.token, conversationId!!, id)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
input = content
|
input = content
|
||||||
}
|
}
|
||||||
} else null,
|
} else null,
|
||||||
|
|
@ -1117,24 +913,6 @@ fun ChatScreen(
|
||||||
readAloud(content, message.id, message.wireId)
|
readAloud(content, message.id, message.wireId)
|
||||||
} else null,
|
} else null,
|
||||||
isPlayingTts = playingTtsForId == message.id,
|
isPlayingTts = playingTtsForId == message.id,
|
||||||
branchInfo = message.branchInfo,
|
|
||||||
onNavigateBranch = if (!isStreaming) { wireId, direction ->
|
|
||||||
val newTree = treeNavigate(chatTree, wireId, direction)
|
|
||||||
chatTree = newTree
|
|
||||||
rebuildVisibleMessages()
|
|
||||||
if (conversationId != null) {
|
|
||||||
val cfg = session.config ?: return@MessageRow
|
|
||||||
scope.launch {
|
|
||||||
KaizenApi.patchConversation(
|
|
||||||
cfg.baseUrl, cfg.token, conversationId!!,
|
|
||||||
mapOf(
|
|
||||||
"activeRootId" to newTree.activeRootId,
|
|
||||||
"activeChild" to newTree.activeChild,
|
|
||||||
),
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
} else null,
|
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -1359,7 +1137,6 @@ fun ChatScreen(
|
||||||
isRecording = isRecording,
|
isRecording = isRecording,
|
||||||
recordingAmplitude = recordingAmplitude,
|
recordingAmplitude = recordingAmplitude,
|
||||||
onMicClick = { toggleRecording() },
|
onMicClick = { toggleRecording() },
|
||||||
onCallClick = if (liveAvailable) {{ startLiveCall() }} else {{ /* PTT TODO */ }},
|
|
||||||
pendingFiles = pendingFiles,
|
pendingFiles = pendingFiles,
|
||||||
onAddClick = { showAttachMenu = !showAttachMenu },
|
onAddClick = { showAttachMenu = !showAttachMenu },
|
||||||
onRemoveFile = { id -> pendingFiles.removeAll { it.id == id } },
|
onRemoveFile = { id -> pendingFiles.removeAll { it.id == id } },
|
||||||
|
|
@ -1457,23 +1234,6 @@ fun ChatScreen(
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// ─── Live Call Overlay ────────────────────────────────
|
|
||||||
androidx.compose.animation.AnimatedVisibility(
|
|
||||||
visible = liveCallActive,
|
|
||||||
enter = androidx.compose.animation.fadeIn(tween(250)),
|
|
||||||
exit = androidx.compose.animation.fadeOut(tween(200)),
|
|
||||||
) {
|
|
||||||
LiveCallOverlay(
|
|
||||||
status = liveStatus,
|
|
||||||
subtitle = liveSubtitle,
|
|
||||||
amplitude = liveAmplitude,
|
|
||||||
userSpeaking = liveUserSpeaking,
|
|
||||||
elapsedMs = liveElapsedMs,
|
|
||||||
onEndCall = { endLiveCall() },
|
|
||||||
onInterrupt = { liveService?.interrupt() },
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -1,106 +0,0 @@
|
||||||
package dev.kaizen.app.chat
|
|
||||||
|
|
||||||
import dev.kaizen.app.net.StoredMessage
|
|
||||||
|
|
||||||
data class ChatNode(
|
|
||||||
val id: String,
|
|
||||||
val role: String,
|
|
||||||
val parentId: String?,
|
|
||||||
val childIds: List<String> = emptyList(),
|
|
||||||
)
|
|
||||||
|
|
||||||
data class BranchInfo(val index: Int, val total: Int)
|
|
||||||
|
|
||||||
data class VisibleEntry(
|
|
||||||
val messageId: String,
|
|
||||||
val branchInfo: BranchInfo?,
|
|
||||||
)
|
|
||||||
|
|
||||||
data class ChatTree(
|
|
||||||
val nodes: Map<String, ChatNode>,
|
|
||||||
val rootIds: List<String>,
|
|
||||||
val activeRootId: String?,
|
|
||||||
val activeChild: Map<String, String>,
|
|
||||||
) {
|
|
||||||
companion object {
|
|
||||||
val EMPTY = ChatTree(emptyMap(), emptyList(), null, emptyMap())
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fun buildChatTree(
|
|
||||||
messages: List<StoredMessage>,
|
|
||||||
savedActiveRootId: String?,
|
|
||||||
savedActiveChild: Map<String, String>,
|
|
||||||
): ChatTree {
|
|
||||||
val nodes = mutableMapOf<String, ChatNode>()
|
|
||||||
val rootIds = mutableListOf<String>()
|
|
||||||
|
|
||||||
for (msg in messages) {
|
|
||||||
nodes[msg.id] = ChatNode(
|
|
||||||
id = msg.id,
|
|
||||||
role = msg.role,
|
|
||||||
parentId = msg.parentId,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
for (msg in messages) {
|
|
||||||
val pid = msg.parentId
|
|
||||||
if (pid != null && nodes.containsKey(pid)) {
|
|
||||||
val parent = nodes[pid]!!
|
|
||||||
nodes[pid] = parent.copy(childIds = parent.childIds + msg.id)
|
|
||||||
} else if (pid == null) {
|
|
||||||
rootIds.add(msg.id)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
val activeRootId = if (savedActiveRootId != null && nodes.containsKey(savedActiveRootId))
|
|
||||||
savedActiveRootId
|
|
||||||
else rootIds.lastOrNull()
|
|
||||||
|
|
||||||
return ChatTree(nodes, rootIds, activeRootId, savedActiveChild)
|
|
||||||
}
|
|
||||||
|
|
||||||
fun getVisiblePath(tree: ChatTree): List<VisibleEntry> {
|
|
||||||
if (tree.rootIds.isEmpty()) return emptyList()
|
|
||||||
val activeRootId = tree.activeRootId ?: tree.rootIds.last()
|
|
||||||
val result = mutableListOf<VisibleEntry>()
|
|
||||||
var currentId: String? = activeRootId
|
|
||||||
|
|
||||||
while (currentId != null) {
|
|
||||||
val node = tree.nodes[currentId] ?: break
|
|
||||||
val siblings = if (node.parentId == null) {
|
|
||||||
tree.rootIds
|
|
||||||
} else {
|
|
||||||
tree.nodes[node.parentId]?.childIds ?: listOf(node.id)
|
|
||||||
}
|
|
||||||
val branchInfo = if (siblings.size > 1) {
|
|
||||||
BranchInfo(index = siblings.indexOf(node.id), total = siblings.size)
|
|
||||||
} else null
|
|
||||||
|
|
||||||
result.add(VisibleEntry(node.id, branchInfo))
|
|
||||||
|
|
||||||
if (node.childIds.isEmpty()) break
|
|
||||||
currentId = tree.activeChild[node.id] ?: node.childIds.last()
|
|
||||||
}
|
|
||||||
|
|
||||||
return result
|
|
||||||
}
|
|
||||||
|
|
||||||
fun treeNavigate(tree: ChatTree, nodeId: String, direction: Int): ChatTree {
|
|
||||||
val node = tree.nodes[nodeId] ?: return tree
|
|
||||||
val siblings = if (node.parentId == null) {
|
|
||||||
tree.rootIds
|
|
||||||
} else {
|
|
||||||
tree.nodes[node.parentId]?.childIds ?: return tree
|
|
||||||
}
|
|
||||||
val idx = siblings.indexOf(nodeId)
|
|
||||||
val newIdx = idx + direction
|
|
||||||
if (newIdx < 0 || newIdx >= siblings.size) return tree
|
|
||||||
val newActiveId = siblings[newIdx]
|
|
||||||
|
|
||||||
return if (node.parentId == null) {
|
|
||||||
tree.copy(activeRootId = newActiveId)
|
|
||||||
} else {
|
|
||||||
tree.copy(activeChild = tree.activeChild + (node.parentId to newActiveId))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
@ -1,210 +0,0 @@
|
||||||
package dev.kaizen.app.live
|
|
||||||
|
|
||||||
import android.Manifest
|
|
||||||
import android.content.Context
|
|
||||||
import android.content.pm.PackageManager
|
|
||||||
import android.media.AudioAttributes
|
|
||||||
import android.media.AudioFormat
|
|
||||||
import android.media.AudioRecord
|
|
||||||
import android.media.AudioTrack
|
|
||||||
import android.media.MediaRecorder
|
|
||||||
import android.util.Base64
|
|
||||||
import android.util.Log
|
|
||||||
import androidx.core.content.ContextCompat
|
|
||||||
import kotlinx.coroutines.*
|
|
||||||
import java.nio.ByteBuffer
|
|
||||||
import java.nio.ByteOrder
|
|
||||||
import kotlin.math.abs
|
|
||||||
import kotlin.math.sqrt
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Bidirectional audio pipeline for Live Call.
|
|
||||||
*
|
|
||||||
* Capture: AudioRecord (16kHz PCM16 mono, VOICE_COMMUNICATION for hardware AEC)
|
|
||||||
* → 100ms chunks → Base64 → [onChunk] callback.
|
|
||||||
*
|
|
||||||
* Playback: AudioTrack (24kHz PCM16 mono, LOW_LATENCY, MODE_STREAM)
|
|
||||||
* ← PCM16 chunks from server.
|
|
||||||
*
|
|
||||||
* Mic-ducking: gain reduced to 10% while the KI speaks (prevents echo feedback
|
|
||||||
* loop on devices with weak AEC). Not a digital gain — we skip sending chunks
|
|
||||||
* whose RMS is below the ducking threshold.
|
|
||||||
*/
|
|
||||||
class AudioPipeline(private val context: Context) {
|
|
||||||
|
|
||||||
companion object {
|
|
||||||
private const val TAG = "AudioPipeline"
|
|
||||||
|
|
||||||
private const val CAPTURE_SAMPLE_RATE = 16_000
|
|
||||||
private const val PLAYBACK_SAMPLE_RATE = 24_000
|
|
||||||
private const val CHANNEL_IN = AudioFormat.CHANNEL_IN_MONO
|
|
||||||
private const val CHANNEL_OUT = AudioFormat.CHANNEL_OUT_MONO
|
|
||||||
private const val ENCODING = AudioFormat.ENCODING_PCM_16BIT
|
|
||||||
|
|
||||||
private const val CHUNK_MS = 100
|
|
||||||
private const val CAPTURE_CHUNK_SAMPLES = CAPTURE_SAMPLE_RATE * CHUNK_MS / 1000 // 1600
|
|
||||||
private const val CAPTURE_CHUNK_BYTES = CAPTURE_CHUNK_SAMPLES * 2 // 3200
|
|
||||||
|
|
||||||
private const val DUCKING_GAIN = 0.10f
|
|
||||||
}
|
|
||||||
|
|
||||||
var onChunk: ((base64Pcm: String) -> Unit)? = null
|
|
||||||
var onAmplitude: ((Float) -> Unit)? = null
|
|
||||||
|
|
||||||
private var recorder: AudioRecord? = null
|
|
||||||
private var track: AudioTrack? = null
|
|
||||||
private var captureJob: Job? = null
|
|
||||||
|
|
||||||
@Volatile private var ducking = false
|
|
||||||
@Volatile private var capturing = false
|
|
||||||
|
|
||||||
// ─── Capture ──────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
fun startCapture(scope: CoroutineScope): Boolean {
|
|
||||||
if (ContextCompat.checkSelfPermission(context, Manifest.permission.RECORD_AUDIO) != PackageManager.PERMISSION_GRANTED) {
|
|
||||||
Log.w(TAG, "RECORD_AUDIO permission not granted")
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
|
|
||||||
val minBuf = AudioRecord.getMinBufferSize(CAPTURE_SAMPLE_RATE, CHANNEL_IN, ENCODING)
|
|
||||||
if (minBuf == AudioRecord.ERROR || minBuf == AudioRecord.ERROR_BAD_VALUE) {
|
|
||||||
Log.e(TAG, "Invalid min buffer size: $minBuf")
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
|
|
||||||
val rec = AudioRecord(
|
|
||||||
MediaRecorder.AudioSource.VOICE_COMMUNICATION,
|
|
||||||
CAPTURE_SAMPLE_RATE,
|
|
||||||
CHANNEL_IN,
|
|
||||||
ENCODING,
|
|
||||||
maxOf(minBuf * 4, CAPTURE_CHUNK_BYTES * 4),
|
|
||||||
)
|
|
||||||
|
|
||||||
if (rec.state != AudioRecord.STATE_INITIALIZED) {
|
|
||||||
Log.e(TAG, "AudioRecord init failed")
|
|
||||||
rec.release()
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
|
|
||||||
recorder = rec
|
|
||||||
rec.startRecording()
|
|
||||||
capturing = true
|
|
||||||
|
|
||||||
captureJob = scope.launch(Dispatchers.IO) {
|
|
||||||
val buffer = ShortArray(CAPTURE_CHUNK_SAMPLES)
|
|
||||||
val byteBuffer = ByteBuffer.allocate(CAPTURE_CHUNK_BYTES).order(ByteOrder.LITTLE_ENDIAN)
|
|
||||||
|
|
||||||
while (isActive && capturing) {
|
|
||||||
val read = rec.read(buffer, 0, CAPTURE_CHUNK_SAMPLES)
|
|
||||||
if (read <= 0) continue
|
|
||||||
|
|
||||||
val rms = computeRms(buffer, read)
|
|
||||||
onAmplitude?.invoke(rms)
|
|
||||||
|
|
||||||
if (ducking && rms < DUCKING_GAIN) continue
|
|
||||||
|
|
||||||
byteBuffer.clear()
|
|
||||||
for (i in 0 until read) byteBuffer.putShort(buffer[i])
|
|
||||||
|
|
||||||
val base64 = Base64.encodeToString(byteBuffer.array(), 0, read * 2, Base64.NO_WRAP)
|
|
||||||
onChunk?.invoke(base64)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
|
|
||||||
fun stopCapture() {
|
|
||||||
capturing = false
|
|
||||||
captureJob?.cancel()
|
|
||||||
captureJob = null
|
|
||||||
try {
|
|
||||||
recorder?.stop()
|
|
||||||
} catch (_: IllegalStateException) {}
|
|
||||||
recorder?.release()
|
|
||||||
recorder = null
|
|
||||||
}
|
|
||||||
|
|
||||||
// ─── Playback ─────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
fun initPlayback(): Boolean {
|
|
||||||
val minBuf = AudioTrack.getMinBufferSize(PLAYBACK_SAMPLE_RATE, CHANNEL_OUT, ENCODING)
|
|
||||||
if (minBuf == AudioTrack.ERROR || minBuf == AudioTrack.ERROR_BAD_VALUE) {
|
|
||||||
Log.e(TAG, "Invalid playback buffer size: $minBuf")
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
|
|
||||||
val t = AudioTrack.Builder()
|
|
||||||
.setAudioAttributes(
|
|
||||||
AudioAttributes.Builder()
|
|
||||||
.setUsage(AudioAttributes.USAGE_ASSISTANT)
|
|
||||||
.setContentType(AudioAttributes.CONTENT_TYPE_SPEECH)
|
|
||||||
.build()
|
|
||||||
)
|
|
||||||
.setAudioFormat(
|
|
||||||
AudioFormat.Builder()
|
|
||||||
.setSampleRate(PLAYBACK_SAMPLE_RATE)
|
|
||||||
.setEncoding(ENCODING)
|
|
||||||
.setChannelMask(CHANNEL_OUT)
|
|
||||||
.build()
|
|
||||||
)
|
|
||||||
.setBufferSizeInBytes(maxOf(minBuf * 4, 24_000 * 2))
|
|
||||||
.setPerformanceMode(AudioTrack.PERFORMANCE_MODE_LOW_LATENCY)
|
|
||||||
.setTransferMode(AudioTrack.MODE_STREAM)
|
|
||||||
.build()
|
|
||||||
|
|
||||||
if (t.state != AudioTrack.STATE_INITIALIZED) {
|
|
||||||
Log.e(TAG, "AudioTrack init failed")
|
|
||||||
t.release()
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
|
|
||||||
track = t
|
|
||||||
t.play()
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
|
|
||||||
fun playChunk(base64Pcm: String) {
|
|
||||||
val bytes = Base64.decode(base64Pcm, Base64.NO_WRAP)
|
|
||||||
track?.write(bytes, 0, bytes.size)
|
|
||||||
}
|
|
||||||
|
|
||||||
fun flushPlayback() {
|
|
||||||
track?.pause()
|
|
||||||
track?.flush()
|
|
||||||
track?.play()
|
|
||||||
}
|
|
||||||
|
|
||||||
fun stopPlayback() {
|
|
||||||
try {
|
|
||||||
track?.pause()
|
|
||||||
track?.flush()
|
|
||||||
} catch (_: IllegalStateException) {}
|
|
||||||
track?.release()
|
|
||||||
track = null
|
|
||||||
}
|
|
||||||
|
|
||||||
// ─── Ducking ──────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
fun setDucking(enabled: Boolean) {
|
|
||||||
ducking = enabled
|
|
||||||
}
|
|
||||||
|
|
||||||
// ─── Cleanup ──────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
fun release() {
|
|
||||||
stopCapture()
|
|
||||||
stopPlayback()
|
|
||||||
}
|
|
||||||
|
|
||||||
// ─── Helpers ──────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
private fun computeRms(buffer: ShortArray, count: Int): Float {
|
|
||||||
var sum = 0.0
|
|
||||||
for (i in 0 until count) {
|
|
||||||
val sample = buffer[i].toDouble() / Short.MAX_VALUE
|
|
||||||
sum += sample * sample
|
|
||||||
}
|
|
||||||
return sqrt(sum / count).toFloat()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
@ -1,181 +0,0 @@
|
||||||
package dev.kaizen.app.live
|
|
||||||
|
|
||||||
import androidx.compose.animation.*
|
|
||||||
import androidx.compose.animation.core.*
|
|
||||||
import androidx.compose.foundation.background
|
|
||||||
import androidx.compose.foundation.clickable
|
|
||||||
import androidx.compose.foundation.interaction.MutableInteractionSource
|
|
||||||
import androidx.compose.foundation.interaction.collectIsPressedAsState
|
|
||||||
import androidx.compose.foundation.isSystemInDarkTheme
|
|
||||||
import androidx.compose.foundation.layout.*
|
|
||||||
import androidx.compose.material3.Icon
|
|
||||||
import androidx.compose.material3.MaterialTheme
|
|
||||||
import androidx.compose.material3.Text
|
|
||||||
import androidx.compose.runtime.*
|
|
||||||
import androidx.compose.ui.Alignment
|
|
||||||
import androidx.compose.ui.Modifier
|
|
||||||
import androidx.compose.ui.draw.alpha
|
|
||||||
import androidx.compose.ui.draw.clip
|
|
||||||
import androidx.compose.ui.draw.scale
|
|
||||||
import androidx.compose.ui.graphics.Brush
|
|
||||||
import androidx.compose.ui.graphics.Color
|
|
||||||
import androidx.compose.ui.res.stringResource
|
|
||||||
import androidx.compose.ui.text.font.FontWeight
|
|
||||||
import androidx.compose.ui.text.style.TextAlign
|
|
||||||
import androidx.compose.ui.text.style.TextOverflow
|
|
||||||
import androidx.compose.ui.unit.dp
|
|
||||||
import androidx.compose.ui.unit.sp
|
|
||||||
import dev.kaizen.app.R
|
|
||||||
import dev.kaizen.app.chat.KaizenOrb
|
|
||||||
import dev.kaizen.app.ui.effect.GlassSurface
|
|
||||||
import dev.kaizen.app.ui.effect.KaizenShadows
|
|
||||||
import dev.kaizen.app.ui.icon.KaizenIcons
|
|
||||||
import dev.kaizen.app.ui.shape.KaizenShapes
|
|
||||||
import dev.kaizen.app.ui.theme.LocalKaizenAccent
|
|
||||||
|
|
||||||
@Composable
|
|
||||||
fun LiveCallOverlay(
|
|
||||||
status: CallStatus,
|
|
||||||
subtitle: String,
|
|
||||||
amplitude: Float,
|
|
||||||
userSpeaking: Boolean,
|
|
||||||
elapsedMs: Long,
|
|
||||||
onEndCall: () -> Unit,
|
|
||||||
onInterrupt: () -> Unit,
|
|
||||||
modifier: Modifier = Modifier,
|
|
||||||
) {
|
|
||||||
val isDark = isSystemInDarkTheme()
|
|
||||||
val accent = LocalKaizenAccent.current
|
|
||||||
val cs = MaterialTheme.colorScheme
|
|
||||||
|
|
||||||
Box(
|
|
||||||
modifier
|
|
||||||
.fillMaxSize()
|
|
||||||
.background(
|
|
||||||
if (isDark) Color.Black.copy(alpha = 0.85f)
|
|
||||||
else Color.White.copy(alpha = 0.88f)
|
|
||||||
)
|
|
||||||
.statusBarsPadding()
|
|
||||||
.navigationBarsPadding(),
|
|
||||||
contentAlignment = Alignment.Center,
|
|
||||||
) {
|
|
||||||
Column(
|
|
||||||
horizontalAlignment = Alignment.CenterHorizontally,
|
|
||||||
verticalArrangement = Arrangement.spacedBy(24.dp),
|
|
||||||
modifier = Modifier.fillMaxWidth().padding(horizontal = 32.dp),
|
|
||||||
) {
|
|
||||||
// ─── Duration ─────────────────────────────────────────
|
|
||||||
val elapsed = elapsedMs / 1000
|
|
||||||
val min = elapsed / 60
|
|
||||||
val sec = elapsed % 60
|
|
||||||
Text(
|
|
||||||
String.format("%d:%02d", min, sec),
|
|
||||||
color = cs.onSurface.copy(alpha = 0.5f),
|
|
||||||
fontSize = 14.sp,
|
|
||||||
fontWeight = FontWeight.W400,
|
|
||||||
letterSpacing = 1.sp,
|
|
||||||
)
|
|
||||||
|
|
||||||
// ─── Orb ──────────────────────────────────────────────
|
|
||||||
val isActive = status == CallStatus.LISTENING || status == CallStatus.SPEAKING
|
|
||||||
Box(
|
|
||||||
Modifier
|
|
||||||
.size(180.dp)
|
|
||||||
.then(
|
|
||||||
if (status == CallStatus.SPEAKING) {
|
|
||||||
Modifier.clickable(
|
|
||||||
interactionSource = remember { MutableInteractionSource() },
|
|
||||||
indication = null,
|
|
||||||
onClick = onInterrupt,
|
|
||||||
)
|
|
||||||
} else Modifier
|
|
||||||
),
|
|
||||||
contentAlignment = Alignment.Center,
|
|
||||||
) {
|
|
||||||
KaizenOrb(
|
|
||||||
size = 160.dp,
|
|
||||||
streaming = status == CallStatus.SPEAKING,
|
|
||||||
recording = status == CallStatus.LISTENING && userSpeaking,
|
|
||||||
amplitude = if (isActive) amplitude else null,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
// ─── Status text ──────────────────────────────────────
|
|
||||||
val statusText = when (status) {
|
|
||||||
CallStatus.IDLE -> ""
|
|
||||||
CallStatus.CONNECTING -> stringResource(R.string.live_connecting)
|
|
||||||
CallStatus.LISTENING -> if (userSpeaking) stringResource(R.string.live_listening)
|
|
||||||
else stringResource(R.string.live_ready)
|
|
||||||
CallStatus.SPEAKING -> stringResource(R.string.live_speaking)
|
|
||||||
}
|
|
||||||
|
|
||||||
Text(
|
|
||||||
statusText,
|
|
||||||
color = when (status) {
|
|
||||||
CallStatus.SPEAKING -> accent.primary
|
|
||||||
CallStatus.LISTENING -> if (userSpeaking) accent.primary.copy(alpha = 0.8f)
|
|
||||||
else cs.onSurface.copy(alpha = 0.4f)
|
|
||||||
else -> cs.onSurface.copy(alpha = 0.4f)
|
|
||||||
},
|
|
||||||
fontSize = 13.sp,
|
|
||||||
fontWeight = FontWeight.W400,
|
|
||||||
letterSpacing = 0.5.sp,
|
|
||||||
)
|
|
||||||
|
|
||||||
// ─── Subtitle (transcript / KI response) ─────────────
|
|
||||||
AnimatedVisibility(
|
|
||||||
visible = subtitle.isNotEmpty(),
|
|
||||||
enter = fadeIn(tween(200)) + expandVertically(),
|
|
||||||
exit = fadeOut(tween(150)) + shrinkVertically(),
|
|
||||||
) {
|
|
||||||
Text(
|
|
||||||
subtitle,
|
|
||||||
color = cs.onSurface.copy(alpha = 0.75f),
|
|
||||||
fontSize = 15.sp,
|
|
||||||
fontWeight = FontWeight.W300,
|
|
||||||
textAlign = TextAlign.Center,
|
|
||||||
maxLines = 4,
|
|
||||||
overflow = TextOverflow.Ellipsis,
|
|
||||||
lineHeight = 22.sp,
|
|
||||||
modifier = Modifier.fillMaxWidth(),
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
Spacer(Modifier.height(32.dp))
|
|
||||||
|
|
||||||
// ─── End Call button ──────────────────────────────────
|
|
||||||
EndCallButton(onClick = onEndCall)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
@Composable
|
|
||||||
private fun EndCallButton(onClick: () -> Unit) {
|
|
||||||
val interaction = remember { MutableInteractionSource() }
|
|
||||||
val pressed by interaction.collectIsPressedAsState()
|
|
||||||
val pressScale by animateFloatAsState(if (pressed) 0.9f else 1f, label = "end")
|
|
||||||
|
|
||||||
Box(
|
|
||||||
Modifier
|
|
||||||
.size(64.dp)
|
|
||||||
.scale(pressScale)
|
|
||||||
.clip(KaizenShapes.circle)
|
|
||||||
.background(
|
|
||||||
Brush.verticalGradient(
|
|
||||||
listOf(
|
|
||||||
Color(0xFFEF4444),
|
|
||||||
Color(0xFFDC2626),
|
|
||||||
)
|
|
||||||
)
|
|
||||||
)
|
|
||||||
.clickable(interactionSource = interaction, indication = null, onClick = onClick),
|
|
||||||
contentAlignment = Alignment.Center,
|
|
||||||
) {
|
|
||||||
Icon(
|
|
||||||
KaizenIcons.PhoneOff,
|
|
||||||
contentDescription = stringResource(R.string.live_end),
|
|
||||||
tint = Color.White,
|
|
||||||
modifier = Modifier.size(28.dp),
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
@ -1,304 +0,0 @@
|
||||||
package dev.kaizen.app.live
|
|
||||||
|
|
||||||
import android.app.Notification
|
|
||||||
import android.app.NotificationChannel
|
|
||||||
import android.app.NotificationManager
|
|
||||||
import android.app.PendingIntent
|
|
||||||
import android.app.Service
|
|
||||||
import android.content.Context
|
|
||||||
import android.content.Intent
|
|
||||||
import android.content.pm.ServiceInfo
|
|
||||||
import android.os.Binder
|
|
||||||
import android.os.IBinder
|
|
||||||
import android.util.Log
|
|
||||||
import dev.kaizen.app.MainActivity
|
|
||||||
import dev.kaizen.app.R
|
|
||||||
import kotlinx.coroutines.*
|
|
||||||
import kotlinx.coroutines.flow.MutableStateFlow
|
|
||||||
import kotlinx.coroutines.flow.StateFlow
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Foreground Service for Live Call — keeps mic + WS alive when the app is
|
|
||||||
* backgrounded or the screen rotates. Shows a persistent notification with
|
|
||||||
* call duration and an end button.
|
|
||||||
*
|
|
||||||
* Lifecycle: started by ChatScreen → binds → [startCall] → audio flows
|
|
||||||
* → [endCall] or notification "Beenden" → cleanup → stopSelf.
|
|
||||||
*/
|
|
||||||
class LiveCallService : Service() {
|
|
||||||
|
|
||||||
companion object {
|
|
||||||
private const val TAG = "LiveCallService"
|
|
||||||
private const val CHANNEL_ID = "kaizen_live_call"
|
|
||||||
private const val NOTIFICATION_ID = 9001
|
|
||||||
private const val ACTION_END = "dev.kaizen.app.LIVE_CALL_END"
|
|
||||||
|
|
||||||
fun intent(context: Context): Intent = Intent(context, LiveCallService::class.java)
|
|
||||||
}
|
|
||||||
|
|
||||||
// ─── Public state ─────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
private val _status = MutableStateFlow(CallStatus.IDLE)
|
|
||||||
val status: StateFlow<CallStatus> = _status
|
|
||||||
|
|
||||||
private val _subtitle = MutableStateFlow("")
|
|
||||||
val subtitle: StateFlow<String> = _subtitle
|
|
||||||
|
|
||||||
private val _amplitude = MutableStateFlow(0f)
|
|
||||||
val amplitude: StateFlow<Float> = _amplitude
|
|
||||||
|
|
||||||
private val _userSpeaking = MutableStateFlow(false)
|
|
||||||
val userSpeaking: StateFlow<Boolean> = _userSpeaking
|
|
||||||
|
|
||||||
var onTurnEnd: ((ServerEvent.TurnEnd) -> Unit)? = null
|
|
||||||
var onError: ((String, String?) -> Unit)? = null
|
|
||||||
|
|
||||||
// ─── Internal ─────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
private val scope = CoroutineScope(SupervisorJob() + Dispatchers.Main.immediate)
|
|
||||||
private var client: LiveClient? = null
|
|
||||||
private var audio: AudioPipeline? = null
|
|
||||||
private var durationJob: Job? = null
|
|
||||||
private var startTimeMs = 0L
|
|
||||||
|
|
||||||
// ─── Binder ───────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
inner class LocalBinder : Binder() {
|
|
||||||
val service: LiveCallService get() = this@LiveCallService
|
|
||||||
}
|
|
||||||
|
|
||||||
private val binder = LocalBinder()
|
|
||||||
override fun onBind(intent: Intent?): IBinder = binder
|
|
||||||
|
|
||||||
// ─── Lifecycle ────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
override fun onCreate() {
|
|
||||||
super.onCreate()
|
|
||||||
createNotificationChannel()
|
|
||||||
}
|
|
||||||
|
|
||||||
override fun onStartCommand(intent: Intent?, flags: Int, startId: Int): Int {
|
|
||||||
if (intent?.action == ACTION_END) {
|
|
||||||
endCall()
|
|
||||||
return START_NOT_STICKY
|
|
||||||
}
|
|
||||||
return START_NOT_STICKY
|
|
||||||
}
|
|
||||||
|
|
||||||
override fun onDestroy() {
|
|
||||||
endCall()
|
|
||||||
scope.cancel()
|
|
||||||
super.onDestroy()
|
|
||||||
}
|
|
||||||
|
|
||||||
// ─── Start / End ──────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
fun startCall(baseUrl: String, token: String, model: String, voice: String, conversationId: String?) {
|
|
||||||
if (_status.value != CallStatus.IDLE) return
|
|
||||||
|
|
||||||
_status.value = CallStatus.CONNECTING
|
|
||||||
|
|
||||||
startForeground(NOTIFICATION_ID, buildNotification("Verbinde…"), ServiceInfo.FOREGROUND_SERVICE_TYPE_MICROPHONE)
|
|
||||||
|
|
||||||
val pipeline = AudioPipeline(this)
|
|
||||||
audio = pipeline
|
|
||||||
|
|
||||||
val liveClient = LiveClient(baseUrl, token, scope)
|
|
||||||
client = liveClient
|
|
||||||
|
|
||||||
pipeline.onAmplitude = { amp -> _amplitude.value = amp }
|
|
||||||
|
|
||||||
scope.launch {
|
|
||||||
liveClient.events.collect { event -> handleEvent(event, pipeline) }
|
|
||||||
}
|
|
||||||
|
|
||||||
pipeline.onChunk = { base64 -> liveClient.sendAudio(base64) }
|
|
||||||
|
|
||||||
liveClient.connect(model, voice, conversationId)
|
|
||||||
}
|
|
||||||
|
|
||||||
fun endCall() {
|
|
||||||
if (_status.value == CallStatus.IDLE) return
|
|
||||||
|
|
||||||
client?.end()
|
|
||||||
audio?.release()
|
|
||||||
durationJob?.cancel()
|
|
||||||
|
|
||||||
client?.dispose()
|
|
||||||
client = null
|
|
||||||
audio = null
|
|
||||||
durationJob = null
|
|
||||||
|
|
||||||
_status.value = CallStatus.IDLE
|
|
||||||
_subtitle.value = ""
|
|
||||||
_amplitude.value = 0f
|
|
||||||
_userSpeaking.value = false
|
|
||||||
|
|
||||||
stopForeground(STOP_FOREGROUND_REMOVE)
|
|
||||||
stopSelf()
|
|
||||||
}
|
|
||||||
|
|
||||||
fun sendText(text: String) {
|
|
||||||
client?.sendText(text)
|
|
||||||
}
|
|
||||||
|
|
||||||
fun interrupt() {
|
|
||||||
client?.interrupt()
|
|
||||||
}
|
|
||||||
|
|
||||||
// ─── Event handling ───────────────────────────────────────────────────
|
|
||||||
|
|
||||||
private fun handleEvent(event: ServerEvent, pipeline: AudioPipeline) {
|
|
||||||
when (event) {
|
|
||||||
is ServerEvent.Ready -> {
|
|
||||||
_status.value = CallStatus.LISTENING
|
|
||||||
|
|
||||||
if (!pipeline.initPlayback()) {
|
|
||||||
Log.e(TAG, "Playback init failed")
|
|
||||||
}
|
|
||||||
if (!pipeline.startCapture(scope)) {
|
|
||||||
Log.e(TAG, "Capture init failed")
|
|
||||||
endCall()
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
startTimeMs = System.currentTimeMillis()
|
|
||||||
startDurationTimer()
|
|
||||||
updateNotification("Anruf aktiv")
|
|
||||||
}
|
|
||||||
|
|
||||||
is ServerEvent.Resumed -> {
|
|
||||||
_status.value = CallStatus.LISTENING
|
|
||||||
updateNotification("Anruf fortgesetzt")
|
|
||||||
}
|
|
||||||
|
|
||||||
is ServerEvent.Audio -> {
|
|
||||||
if (_status.value != CallStatus.SPEAKING) {
|
|
||||||
_status.value = CallStatus.SPEAKING
|
|
||||||
pipeline.setDucking(true)
|
|
||||||
}
|
|
||||||
pipeline.playChunk(event.data)
|
|
||||||
}
|
|
||||||
|
|
||||||
is ServerEvent.Text -> {
|
|
||||||
_subtitle.value += event.delta
|
|
||||||
}
|
|
||||||
|
|
||||||
is ServerEvent.Transcript -> {
|
|
||||||
if (event.final) {
|
|
||||||
_subtitle.value = event.text
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
is ServerEvent.Vad -> {
|
|
||||||
_userSpeaking.value = event.speaking
|
|
||||||
if (event.speaking) {
|
|
||||||
_status.value = CallStatus.LISTENING
|
|
||||||
_subtitle.value = ""
|
|
||||||
pipeline.setDucking(false)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
is ServerEvent.Interrupted -> {
|
|
||||||
pipeline.flushPlayback()
|
|
||||||
pipeline.setDucking(false)
|
|
||||||
_status.value = CallStatus.LISTENING
|
|
||||||
}
|
|
||||||
|
|
||||||
is ServerEvent.TurnEnd -> {
|
|
||||||
pipeline.setDucking(false)
|
|
||||||
_status.value = CallStatus.LISTENING
|
|
||||||
onTurnEnd?.invoke(event)
|
|
||||||
}
|
|
||||||
|
|
||||||
is ServerEvent.ToolCall -> {
|
|
||||||
if (event.status == "running") {
|
|
||||||
_subtitle.value = "${event.name}…"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
is ServerEvent.Backpressure -> {
|
|
||||||
// LiveClient handles throttling internally
|
|
||||||
}
|
|
||||||
|
|
||||||
is ServerEvent.GoAway -> {
|
|
||||||
if (event.reason == "timeout") {
|
|
||||||
endCall()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
is ServerEvent.Error -> {
|
|
||||||
onError?.invoke(event.code, event.message)
|
|
||||||
if (event.code in setOf("auth_failed", "restricted_model", "restricted_feature", "budget_exceeded", "concurrent_limit")) {
|
|
||||||
endCall()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// ─── Duration timer ───────────────────────────────────────────────────
|
|
||||||
|
|
||||||
private fun startDurationTimer() {
|
|
||||||
durationJob?.cancel()
|
|
||||||
durationJob = scope.launch {
|
|
||||||
while (isActive) {
|
|
||||||
delay(1000)
|
|
||||||
val elapsed = (System.currentTimeMillis() - startTimeMs) / 1000
|
|
||||||
val min = elapsed / 60
|
|
||||||
val sec = elapsed % 60
|
|
||||||
updateNotification(String.format("Anruf · %d:%02d", min, sec))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// ─── Notification ─────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
private fun createNotificationChannel() {
|
|
||||||
val channel = NotificationChannel(
|
|
||||||
CHANNEL_ID,
|
|
||||||
getString(R.string.app_name),
|
|
||||||
NotificationManager.IMPORTANCE_LOW,
|
|
||||||
).apply {
|
|
||||||
description = "Live Call"
|
|
||||||
setShowBadge(false)
|
|
||||||
}
|
|
||||||
getSystemService(NotificationManager::class.java)?.createNotificationChannel(channel)
|
|
||||||
}
|
|
||||||
|
|
||||||
private fun buildNotification(text: String): Notification {
|
|
||||||
val openIntent = PendingIntent.getActivity(
|
|
||||||
this, 0,
|
|
||||||
Intent(this, MainActivity::class.java).apply {
|
|
||||||
flags = Intent.FLAG_ACTIVITY_SINGLE_TOP
|
|
||||||
},
|
|
||||||
PendingIntent.FLAG_IMMUTABLE,
|
|
||||||
)
|
|
||||||
|
|
||||||
val endIntent = PendingIntent.getService(
|
|
||||||
this, 1,
|
|
||||||
Intent(this, LiveCallService::class.java).apply { action = ACTION_END },
|
|
||||||
PendingIntent.FLAG_IMMUTABLE,
|
|
||||||
)
|
|
||||||
|
|
||||||
return Notification.Builder(this, CHANNEL_ID)
|
|
||||||
.setSmallIcon(R.drawable.ic_launcher_foreground)
|
|
||||||
.setContentTitle(getString(R.string.app_name))
|
|
||||||
.setContentText(text)
|
|
||||||
.setContentIntent(openIntent)
|
|
||||||
.addAction(Notification.Action.Builder(null, "Beenden", endIntent).build())
|
|
||||||
.setOngoing(true)
|
|
||||||
.build()
|
|
||||||
}
|
|
||||||
|
|
||||||
private fun updateNotification(text: String) {
|
|
||||||
getSystemService(NotificationManager::class.java)
|
|
||||||
?.notify(NOTIFICATION_ID, buildNotification(text))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
enum class CallStatus {
|
|
||||||
IDLE,
|
|
||||||
CONNECTING,
|
|
||||||
LISTENING,
|
|
||||||
SPEAKING,
|
|
||||||
}
|
|
||||||
|
|
@ -1,211 +0,0 @@
|
||||||
package dev.kaizen.app.live
|
|
||||||
|
|
||||||
import android.util.Log
|
|
||||||
import kotlinx.coroutines.*
|
|
||||||
import kotlinx.coroutines.flow.MutableSharedFlow
|
|
||||||
import kotlinx.coroutines.flow.SharedFlow
|
|
||||||
import okhttp3.OkHttpClient
|
|
||||||
import okhttp3.Request
|
|
||||||
import okhttp3.Response
|
|
||||||
import okhttp3.WebSocket
|
|
||||||
import okhttp3.WebSocketListener
|
|
||||||
import java.util.concurrent.TimeUnit
|
|
||||||
import java.util.concurrent.atomic.AtomicInteger
|
|
||||||
import java.util.concurrent.atomic.AtomicReference
|
|
||||||
|
|
||||||
/**
|
|
||||||
* WebSocket client for the Kaizen Live Call protocol.
|
|
||||||
*
|
|
||||||
* Connects to `wss://{baseUrl}/ws/live` with Bearer auth, speaks the
|
|
||||||
* provider-agnostic JSON protocol defined in `lib/live/protocol.ts`.
|
|
||||||
* Handles auto-reconnect with exponential backoff + resume.
|
|
||||||
*
|
|
||||||
* Thread-safe: OkHttp dispatches callbacks on its own threads,
|
|
||||||
* all state mutations go through atomic refs or the coroutine scope.
|
|
||||||
*/
|
|
||||||
class LiveClient(
|
|
||||||
private val baseUrl: String,
|
|
||||||
private val token: String,
|
|
||||||
private val scope: CoroutineScope,
|
|
||||||
) {
|
|
||||||
companion object {
|
|
||||||
private const val TAG = "LiveClient"
|
|
||||||
private const val INITIAL_BACKOFF_MS = 500L
|
|
||||||
private const val MAX_BACKOFF_MS = 8_000L
|
|
||||||
private const val MAX_RECONNECT_ATTEMPTS = 5
|
|
||||||
}
|
|
||||||
|
|
||||||
// ─── Public event stream ──────────────────────────────────────────────
|
|
||||||
|
|
||||||
private val _events = MutableSharedFlow<ServerEvent>(extraBufferCapacity = 64)
|
|
||||||
val events: SharedFlow<ServerEvent> = _events
|
|
||||||
|
|
||||||
// ─── State ────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
private val ws = AtomicReference<WebSocket?>(null)
|
|
||||||
private val sessionId = AtomicReference<String?>(null)
|
|
||||||
private val lastSeqId = AtomicInteger(0)
|
|
||||||
private val reconnectAttempts = AtomicInteger(0)
|
|
||||||
private var chunkIntervalMs = 100L
|
|
||||||
|
|
||||||
@Volatile private var active = false
|
|
||||||
@Volatile private var pendingSetup: SetupParams? = null
|
|
||||||
|
|
||||||
private data class SetupParams(
|
|
||||||
val model: String,
|
|
||||||
val voice: String,
|
|
||||||
val conversationId: String?,
|
|
||||||
)
|
|
||||||
|
|
||||||
private val client = OkHttpClient.Builder()
|
|
||||||
.readTimeout(0, TimeUnit.MILLISECONDS)
|
|
||||||
.pingInterval(15, TimeUnit.SECONDS)
|
|
||||||
.connectTimeout(10, TimeUnit.SECONDS)
|
|
||||||
.build()
|
|
||||||
|
|
||||||
// ─── Connect ──────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
fun connect(model: String, voice: String, conversationId: String?) {
|
|
||||||
active = true
|
|
||||||
pendingSetup = SetupParams(model, voice, conversationId)
|
|
||||||
reconnectAttempts.set(0)
|
|
||||||
openSocket()
|
|
||||||
}
|
|
||||||
|
|
||||||
private fun openSocket() {
|
|
||||||
if (!active) return
|
|
||||||
|
|
||||||
val wsUrl = baseUrl
|
|
||||||
.replace("https://", "wss://")
|
|
||||||
.replace("http://", "ws://") + "/ws/live"
|
|
||||||
|
|
||||||
val request = Request.Builder()
|
|
||||||
.url(wsUrl)
|
|
||||||
.header("Authorization", "Bearer $token")
|
|
||||||
.build()
|
|
||||||
|
|
||||||
val socket = client.newWebSocket(request, Listener())
|
|
||||||
ws.set(socket)
|
|
||||||
}
|
|
||||||
|
|
||||||
// ─── Send ─────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
fun sendAudio(base64Pcm: String) {
|
|
||||||
ws.get()?.send(encodeAudio(base64Pcm))
|
|
||||||
}
|
|
||||||
|
|
||||||
fun sendText(text: String) {
|
|
||||||
ws.get()?.send(encodeText(text))
|
|
||||||
}
|
|
||||||
|
|
||||||
fun interrupt() {
|
|
||||||
ws.get()?.send(encodeInterrupt())
|
|
||||||
}
|
|
||||||
|
|
||||||
fun end() {
|
|
||||||
active = false
|
|
||||||
ws.get()?.send(encodeEnd())
|
|
||||||
ws.getAndSet(null)?.close(1000, "end")
|
|
||||||
}
|
|
||||||
|
|
||||||
fun dispose() {
|
|
||||||
active = false
|
|
||||||
ws.getAndSet(null)?.cancel()
|
|
||||||
client.dispatcher.executorService.shutdown()
|
|
||||||
}
|
|
||||||
|
|
||||||
// ─── Reconnect ────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
private fun scheduleReconnect() {
|
|
||||||
if (!active) return
|
|
||||||
val attempt = reconnectAttempts.incrementAndGet()
|
|
||||||
if (attempt > MAX_RECONNECT_ATTEMPTS) {
|
|
||||||
Log.w(TAG, "Max reconnect attempts reached")
|
|
||||||
scope.launch { _events.emit(ServerEvent.Error("reconnect_failed", "Max reconnect attempts")) }
|
|
||||||
active = false
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
val backoff = (INITIAL_BACKOFF_MS * (1 shl (attempt - 1))).coerceAtMost(MAX_BACKOFF_MS)
|
|
||||||
Log.d(TAG, "Reconnecting in ${backoff}ms (attempt $attempt)")
|
|
||||||
|
|
||||||
scope.launch {
|
|
||||||
delay(backoff)
|
|
||||||
if (active) openSocket()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// ─── WebSocket Listener ───────────────────────────────────────────────
|
|
||||||
|
|
||||||
private inner class Listener : WebSocketListener() {
|
|
||||||
|
|
||||||
override fun onOpen(webSocket: WebSocket, response: Response) {
|
|
||||||
Log.d(TAG, "WebSocket opened")
|
|
||||||
reconnectAttempts.set(0)
|
|
||||||
|
|
||||||
val sid = sessionId.get()
|
|
||||||
if (sid != null) {
|
|
||||||
webSocket.send(encodeResume(sid, lastSeqId.get()))
|
|
||||||
} else {
|
|
||||||
val setup = pendingSetup ?: return
|
|
||||||
webSocket.send(encodeSetup(setup.model, setup.voice, setup.conversationId))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
override fun onMessage(webSocket: WebSocket, text: String) {
|
|
||||||
val event = parseServerEvent(text) ?: return
|
|
||||||
|
|
||||||
when (event) {
|
|
||||||
is ServerEvent.Ready -> {
|
|
||||||
sessionId.set(event.sessionId)
|
|
||||||
}
|
|
||||||
is ServerEvent.Resumed -> {
|
|
||||||
sessionId.set(event.sessionId)
|
|
||||||
}
|
|
||||||
is ServerEvent.Audio -> {
|
|
||||||
lastSeqId.set(maxOf(lastSeqId.get(), event.seq))
|
|
||||||
}
|
|
||||||
is ServerEvent.Text -> {
|
|
||||||
lastSeqId.set(maxOf(lastSeqId.get(), event.seq))
|
|
||||||
}
|
|
||||||
is ServerEvent.Backpressure -> {
|
|
||||||
chunkIntervalMs = if (event.throttleMs > 0) event.throttleMs.toLong() else 100L
|
|
||||||
}
|
|
||||||
is ServerEvent.GoAway -> {
|
|
||||||
if (event.reason == "shutdown") {
|
|
||||||
scope.launch {
|
|
||||||
delay(500)
|
|
||||||
if (active) openSocket()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
is ServerEvent.Error -> {
|
|
||||||
val terminal = event.code in setOf(
|
|
||||||
"auth_failed", "restricted_model", "restricted_feature",
|
|
||||||
"budget_exceeded", "concurrent_limit",
|
|
||||||
)
|
|
||||||
if (terminal) active = false
|
|
||||||
}
|
|
||||||
else -> {}
|
|
||||||
}
|
|
||||||
|
|
||||||
scope.launch { _events.emit(event) }
|
|
||||||
}
|
|
||||||
|
|
||||||
override fun onClosing(webSocket: WebSocket, code: Int, reason: String) {
|
|
||||||
webSocket.close(code, reason)
|
|
||||||
}
|
|
||||||
|
|
||||||
override fun onClosed(webSocket: WebSocket, code: Int, reason: String) {
|
|
||||||
Log.d(TAG, "WebSocket closed: $code $reason")
|
|
||||||
ws.compareAndSet(webSocket, null)
|
|
||||||
if (active && code != 1000) scheduleReconnect()
|
|
||||||
}
|
|
||||||
|
|
||||||
override fun onFailure(webSocket: WebSocket, t: Throwable, response: Response?) {
|
|
||||||
Log.w(TAG, "WebSocket failure: ${t.message}")
|
|
||||||
ws.compareAndSet(webSocket, null)
|
|
||||||
if (active) scheduleReconnect()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
@ -1,156 +0,0 @@
|
||||||
package dev.kaizen.app.live
|
|
||||||
|
|
||||||
import kotlinx.serialization.Serializable
|
|
||||||
import kotlinx.serialization.json.Json
|
|
||||||
import kotlinx.serialization.json.JsonObject
|
|
||||||
import kotlinx.serialization.json.boolean
|
|
||||||
import kotlinx.serialization.json.booleanOrNull
|
|
||||||
import kotlinx.serialization.json.double
|
|
||||||
import kotlinx.serialization.json.doubleOrNull
|
|
||||||
import kotlinx.serialization.json.int
|
|
||||||
import kotlinx.serialization.json.intOrNull
|
|
||||||
import kotlinx.serialization.json.jsonObject
|
|
||||||
import kotlinx.serialization.json.jsonPrimitive
|
|
||||||
import kotlinx.serialization.json.long
|
|
||||||
import kotlinx.serialization.json.longOrNull
|
|
||||||
|
|
||||||
private val json = Json { ignoreUnknownKeys = true; encodeDefaults = false }
|
|
||||||
|
|
||||||
// ─── Client → Server ──────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class SetupMessage(
|
|
||||||
val type: String = "setup",
|
|
||||||
val model: String,
|
|
||||||
val voice: String,
|
|
||||||
val conversationId: String? = null,
|
|
||||||
val tools: Boolean? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class ResumeMessage(
|
|
||||||
val type: String = "resume",
|
|
||||||
val sessionId: String,
|
|
||||||
val lastSeqId: Int? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class AudioMessage(val type: String = "audio", val data: String)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class TextMessage(val type: String = "text", val text: String)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class SimpleMessage(val type: String)
|
|
||||||
|
|
||||||
fun encodeSetup(model: String, voice: String, conversationId: String?, tools: Boolean? = true): String =
|
|
||||||
json.encodeToString(SetupMessage.serializer(), SetupMessage(model = model, voice = voice, conversationId = conversationId, tools = tools))
|
|
||||||
|
|
||||||
fun encodeResume(sessionId: String, lastSeqId: Int?): String =
|
|
||||||
json.encodeToString(ResumeMessage.serializer(), ResumeMessage(sessionId = sessionId, lastSeqId = lastSeqId))
|
|
||||||
|
|
||||||
fun encodeAudio(base64Pcm: String): String =
|
|
||||||
json.encodeToString(AudioMessage.serializer(), AudioMessage(data = base64Pcm))
|
|
||||||
|
|
||||||
fun encodeText(text: String): String =
|
|
||||||
json.encodeToString(TextMessage.serializer(), TextMessage(text = text))
|
|
||||||
|
|
||||||
fun encodeInterrupt(): String = """{"type":"interrupt"}"""
|
|
||||||
fun encodeEnd(): String = """{"type":"end"}"""
|
|
||||||
|
|
||||||
// ─── Server → Client ──────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
data class LiveUsage(
|
|
||||||
val inputTokens: Int = 0,
|
|
||||||
val outputTokens: Int = 0,
|
|
||||||
val cost: Double? = null,
|
|
||||||
val durationMs: Long = 0,
|
|
||||||
)
|
|
||||||
|
|
||||||
sealed interface ServerEvent {
|
|
||||||
data class Ready(val sessionId: String) : ServerEvent
|
|
||||||
data class Resumed(val sessionId: String) : ServerEvent
|
|
||||||
data class Audio(val data: String, val seq: Int) : ServerEvent
|
|
||||||
data class Text(val delta: String, val seq: Int) : ServerEvent
|
|
||||||
data class Transcript(val text: String, val final: Boolean) : ServerEvent
|
|
||||||
data class Vad(val speaking: Boolean) : ServerEvent
|
|
||||||
data object Interrupted : ServerEvent
|
|
||||||
data class TurnEnd(
|
|
||||||
val usage: LiveUsage,
|
|
||||||
val userText: String,
|
|
||||||
val assistantText: String,
|
|
||||||
val userMessageId: String? = null,
|
|
||||||
val assistantMessageId: String? = null,
|
|
||||||
) : ServerEvent
|
|
||||||
data class ToolCall(val name: String, val status: String) : ServerEvent
|
|
||||||
data class Backpressure(val throttleMs: Int) : ServerEvent
|
|
||||||
data class GoAway(val remainingMs: Int, val reason: String) : ServerEvent
|
|
||||||
data class Error(val code: String, val message: String?) : ServerEvent
|
|
||||||
}
|
|
||||||
|
|
||||||
fun parseServerEvent(raw: String): ServerEvent? {
|
|
||||||
val obj: JsonObject
|
|
||||||
try {
|
|
||||||
obj = json.parseToJsonElement(raw).jsonObject
|
|
||||||
} catch (_: Exception) {
|
|
||||||
return null
|
|
||||||
}
|
|
||||||
|
|
||||||
val type = obj["type"]?.jsonPrimitive?.content ?: return null
|
|
||||||
|
|
||||||
return when (type) {
|
|
||||||
"ready" -> ServerEvent.Ready(
|
|
||||||
sessionId = obj["sessionId"]?.jsonPrimitive?.content ?: return null,
|
|
||||||
)
|
|
||||||
"resumed" -> ServerEvent.Resumed(
|
|
||||||
sessionId = obj["sessionId"]?.jsonPrimitive?.content ?: return null,
|
|
||||||
)
|
|
||||||
"audio" -> ServerEvent.Audio(
|
|
||||||
data = obj["data"]?.jsonPrimitive?.content ?: return null,
|
|
||||||
seq = obj["seq"]?.jsonPrimitive?.intOrNull ?: 0,
|
|
||||||
)
|
|
||||||
"text" -> ServerEvent.Text(
|
|
||||||
delta = obj["delta"]?.jsonPrimitive?.content ?: return null,
|
|
||||||
seq = obj["seq"]?.jsonPrimitive?.intOrNull ?: 0,
|
|
||||||
)
|
|
||||||
"transcript" -> ServerEvent.Transcript(
|
|
||||||
text = obj["text"]?.jsonPrimitive?.content ?: "",
|
|
||||||
final = obj["final"]?.jsonPrimitive?.booleanOrNull ?: false,
|
|
||||||
)
|
|
||||||
"vad" -> ServerEvent.Vad(
|
|
||||||
speaking = obj["speaking"]?.jsonPrimitive?.booleanOrNull ?: false,
|
|
||||||
)
|
|
||||||
"interrupted" -> ServerEvent.Interrupted
|
|
||||||
"turn_end" -> {
|
|
||||||
val u = obj["usage"]?.jsonObject
|
|
||||||
ServerEvent.TurnEnd(
|
|
||||||
usage = LiveUsage(
|
|
||||||
inputTokens = u?.get("inputTokens")?.jsonPrimitive?.intOrNull ?: 0,
|
|
||||||
outputTokens = u?.get("outputTokens")?.jsonPrimitive?.intOrNull ?: 0,
|
|
||||||
cost = u?.get("cost")?.jsonPrimitive?.doubleOrNull,
|
|
||||||
durationMs = u?.get("durationMs")?.jsonPrimitive?.longOrNull ?: 0,
|
|
||||||
),
|
|
||||||
userText = obj["userText"]?.jsonPrimitive?.content ?: "",
|
|
||||||
assistantText = obj["assistantText"]?.jsonPrimitive?.content ?: "",
|
|
||||||
userMessageId = obj["userMessageId"]?.jsonPrimitive?.content,
|
|
||||||
assistantMessageId = obj["assistantMessageId"]?.jsonPrimitive?.content,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
"tool_call" -> ServerEvent.ToolCall(
|
|
||||||
name = obj["name"]?.jsonPrimitive?.content ?: "",
|
|
||||||
status = obj["status"]?.jsonPrimitive?.content ?: "",
|
|
||||||
)
|
|
||||||
"backpressure" -> ServerEvent.Backpressure(
|
|
||||||
throttleMs = obj["throttleMs"]?.jsonPrimitive?.intOrNull ?: 0,
|
|
||||||
)
|
|
||||||
"go_away" -> ServerEvent.GoAway(
|
|
||||||
remainingMs = obj["remainingMs"]?.jsonPrimitive?.intOrNull ?: 0,
|
|
||||||
reason = obj["reason"]?.jsonPrimitive?.content ?: "",
|
|
||||||
)
|
|
||||||
"error" -> ServerEvent.Error(
|
|
||||||
code = obj["code"]?.jsonPrimitive?.content ?: "unknown",
|
|
||||||
message = obj["message"]?.jsonPrimitive?.content,
|
|
||||||
)
|
|
||||||
else -> null
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
@ -99,7 +99,6 @@ data class StoredMessage(
|
||||||
val id: String,
|
val id: String,
|
||||||
val role: String,
|
val role: String,
|
||||||
val content: String = "",
|
val content: String = "",
|
||||||
val parentId: String? = null,
|
|
||||||
val attachments: List<Attachment> = emptyList(),
|
val attachments: List<Attachment> = emptyList(),
|
||||||
val reasoning: String? = null,
|
val reasoning: String? = null,
|
||||||
val sources: List<SearchSource>? = null,
|
val sources: List<SearchSource>? = null,
|
||||||
|
|
@ -108,17 +107,7 @@ data class StoredMessage(
|
||||||
val outputTokens: Int? = null,
|
val outputTokens: Int? = null,
|
||||||
)
|
)
|
||||||
|
|
||||||
@Serializable private data class ConversationDetail(
|
@Serializable private data class ConversationDetail(val messages: List<StoredMessage> = emptyList())
|
||||||
val messages: List<StoredMessage> = emptyList(),
|
|
||||||
val activeRootId: String? = null,
|
|
||||||
val activeChild: Map<String, String> = emptyMap(),
|
|
||||||
)
|
|
||||||
|
|
||||||
data class ConversationData(
|
|
||||||
val messages: List<StoredMessage> = emptyList(),
|
|
||||||
val activeRootId: String? = null,
|
|
||||||
val activeChild: Map<String, String> = emptyMap(),
|
|
||||||
)
|
|
||||||
|
|
||||||
/** A message to persist. `parentId` chains the linear history; `kind` is always "chat" for the app. */
|
/** A message to persist. `parentId` chains the linear history; `kind` is always "chat" for the app. */
|
||||||
@Serializable
|
@Serializable
|
||||||
|
|
@ -320,9 +309,6 @@ object KaizenApi {
|
||||||
|
|
||||||
/** GET /api/v1/conversations/[id] — the stored messages, in order (empty on failure/locked). */
|
/** GET /api/v1/conversations/[id] — the stored messages, in order (empty on failure/locked). */
|
||||||
suspend fun fetchMessages(baseUrl: String, token: String, id: String): List<StoredMessage> =
|
suspend fun fetchMessages(baseUrl: String, token: String, id: String): List<StoredMessage> =
|
||||||
fetchConversationData(baseUrl, token, id).messages
|
|
||||||
|
|
||||||
suspend fun fetchConversationData(baseUrl: String, token: String, id: String): ConversationData =
|
|
||||||
withContext(Dispatchers.IO) {
|
withContext(Dispatchers.IO) {
|
||||||
val req = Request.Builder()
|
val req = Request.Builder()
|
||||||
.url("$baseUrl/api/v1/conversations/$id")
|
.url("$baseUrl/api/v1/conversations/$id")
|
||||||
|
|
@ -332,14 +318,13 @@ object KaizenApi {
|
||||||
client.newCall(req).execute().use { resp ->
|
client.newCall(req).execute().use { resp ->
|
||||||
if (!resp.isSuccessful) {
|
if (!resp.isSuccessful) {
|
||||||
Log.w(TAG, "fetchMessages($id): HTTP ${resp.code}")
|
Log.w(TAG, "fetchMessages($id): HTTP ${resp.code}")
|
||||||
return@use ConversationData()
|
return@use emptyList()
|
||||||
}
|
}
|
||||||
val detail = json.decodeFromString<ConversationDetail>(resp.body!!.string())
|
json.decodeFromString<ConversationDetail>(resp.body!!.string()).messages
|
||||||
ConversationData(detail.messages, detail.activeRootId, detail.activeChild)
|
|
||||||
}
|
}
|
||||||
} catch (e: Exception) {
|
} catch (e: Exception) {
|
||||||
Log.w(TAG, "fetchMessages($id): ${e.javaClass.simpleName}: ${e.message}")
|
Log.w(TAG, "fetchMessages($id): ${e.javaClass.simpleName}: ${e.message}")
|
||||||
ConversationData()
|
emptyList()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -363,20 +348,17 @@ object KaizenApi {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/** PATCH /api/v1/conversations/[id] — update title, locked, pinned, activeRootId, activeChild. */
|
/** PATCH /api/v1/conversations/[id] — update title, locked, pinned. */
|
||||||
suspend fun patchConversation(baseUrl: String, token: String, id: String, body: Map<String, Any?>): Boolean =
|
suspend fun patchConversation(baseUrl: String, token: String, id: String, body: Map<String, Any?>): Boolean =
|
||||||
withContext(Dispatchers.IO) {
|
withContext(Dispatchers.IO) {
|
||||||
fun toJsonElement(v: Any?): kotlinx.serialization.json.JsonElement = when (v) {
|
|
||||||
is String -> kotlinx.serialization.json.JsonPrimitive(v)
|
|
||||||
is Boolean -> kotlinx.serialization.json.JsonPrimitive(v)
|
|
||||||
is Map<*, *> -> kotlinx.serialization.json.JsonObject(
|
|
||||||
v.entries.associate { (k, v2) -> k.toString() to toJsonElement(v2) }
|
|
||||||
)
|
|
||||||
null -> kotlinx.serialization.json.JsonNull
|
|
||||||
else -> kotlinx.serialization.json.JsonNull
|
|
||||||
}
|
|
||||||
val payload = json.encodeToString(kotlinx.serialization.json.JsonObject(
|
val payload = json.encodeToString(kotlinx.serialization.json.JsonObject(
|
||||||
body.mapValues { (_, v) -> toJsonElement(v) }
|
body.mapValues { (_, v) ->
|
||||||
|
when (v) {
|
||||||
|
is String -> kotlinx.serialization.json.JsonPrimitive(v)
|
||||||
|
is Boolean -> kotlinx.serialization.json.JsonPrimitive(v)
|
||||||
|
else -> kotlinx.serialization.json.JsonNull
|
||||||
|
}
|
||||||
|
}
|
||||||
)).toRequestBody(jsonMedia)
|
)).toRequestBody(jsonMedia)
|
||||||
val req = Request.Builder()
|
val req = Request.Builder()
|
||||||
.url("$baseUrl/api/v1/conversations/$id")
|
.url("$baseUrl/api/v1/conversations/$id")
|
||||||
|
|
@ -432,9 +414,6 @@ object KaizenApi {
|
||||||
|
|
||||||
/** GET /api/v1/conversations/[id] with unlock token — fetch messages of a locked conversation. */
|
/** GET /api/v1/conversations/[id] with unlock token — fetch messages of a locked conversation. */
|
||||||
suspend fun fetchLockedMessages(baseUrl: String, token: String, id: String, unlockToken: String): List<StoredMessage> =
|
suspend fun fetchLockedMessages(baseUrl: String, token: String, id: String, unlockToken: String): List<StoredMessage> =
|
||||||
fetchLockedConversationData(baseUrl, token, id, unlockToken).messages
|
|
||||||
|
|
||||||
suspend fun fetchLockedConversationData(baseUrl: String, token: String, id: String, unlockToken: String): ConversationData =
|
|
||||||
withContext(Dispatchers.IO) {
|
withContext(Dispatchers.IO) {
|
||||||
val req = Request.Builder()
|
val req = Request.Builder()
|
||||||
.url("$baseUrl/api/v1/conversations/$id")
|
.url("$baseUrl/api/v1/conversations/$id")
|
||||||
|
|
@ -445,14 +424,13 @@ object KaizenApi {
|
||||||
client.newCall(req).execute().use { resp ->
|
client.newCall(req).execute().use { resp ->
|
||||||
if (!resp.isSuccessful) {
|
if (!resp.isSuccessful) {
|
||||||
Log.w(TAG, "fetchLockedMessages($id): HTTP ${resp.code}")
|
Log.w(TAG, "fetchLockedMessages($id): HTTP ${resp.code}")
|
||||||
return@use ConversationData()
|
return@use emptyList()
|
||||||
}
|
}
|
||||||
val detail = json.decodeFromString<ConversationDetail>(resp.body!!.string())
|
json.decodeFromString<ConversationDetail>(resp.body!!.string()).messages
|
||||||
ConversationData(detail.messages, detail.activeRootId, detail.activeChild)
|
|
||||||
}
|
}
|
||||||
} catch (e: Exception) {
|
} catch (e: Exception) {
|
||||||
Log.w(TAG, "fetchLockedMessages($id): ${e.javaClass.simpleName}: ${e.message}")
|
Log.w(TAG, "fetchLockedMessages($id): ${e.javaClass.simpleName}: ${e.message}")
|
||||||
ConversationData()
|
emptyList()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -22,8 +22,6 @@ object KaizenIcons {
|
||||||
// ── Navigation ──────────────────────────────────────────────────────────
|
// ── Navigation ──────────────────────────────────────────────────────────
|
||||||
val ArrowLeft: ImageVector get() = Lucide.ArrowLeft
|
val ArrowLeft: ImageVector get() = Lucide.ArrowLeft
|
||||||
val ChevronDown: ImageVector get() = Lucide.ChevronDown
|
val ChevronDown: ImageVector get() = Lucide.ChevronDown
|
||||||
val ChevronLeft: ImageVector get() = Lucide.ChevronLeft
|
|
||||||
val ChevronRight: ImageVector get() = Lucide.ChevronRight
|
|
||||||
val ChevronUp: ImageVector get() = Lucide.ChevronUp
|
val ChevronUp: ImageVector get() = Lucide.ChevronUp
|
||||||
val Menu: ImageVector get() = KaizenCustomIcons.Menu
|
val Menu: ImageVector get() = KaizenCustomIcons.Menu
|
||||||
val MoreVertical: ImageVector get() = Lucide.EllipsisVertical
|
val MoreVertical: ImageVector get() = Lucide.EllipsisVertical
|
||||||
|
|
@ -97,9 +95,8 @@ object KaizenIcons {
|
||||||
val ArrowDown: ImageVector get() = Lucide.ArrowDown
|
val ArrowDown: ImageVector get() = Lucide.ArrowDown
|
||||||
val ArrowUp: ImageVector get() = Lucide.ArrowUp
|
val ArrowUp: ImageVector get() = Lucide.ArrowUp
|
||||||
|
|
||||||
// ── Audio / Call ────────────────────────────────────────────────────────
|
// ── Audio ──────────────────────────────────────────────────────────────
|
||||||
val Volume2: ImageVector get() = Lucide.Volume2
|
val Volume2: ImageVector get() = Lucide.Volume2
|
||||||
val PhoneOff: ImageVector get() = Lucide.PhoneOff
|
|
||||||
|
|
||||||
// ── Custom (brand overrides) ────────────────────────────────────────────
|
// ── Custom (brand overrides) ────────────────────────────────────────────
|
||||||
// Add hand-drawn SVGs here as ImageVector literals when ready.
|
// Add hand-drawn SVGs here as ImageVector literals when ready.
|
||||||
|
|
|
||||||
|
|
@ -180,12 +180,4 @@
|
||||||
<string name="settings_language_system">System</string>
|
<string name="settings_language_system">System</string>
|
||||||
<string name="settings_language_de">Deutsch</string>
|
<string name="settings_language_de">Deutsch</string>
|
||||||
<string name="settings_language_en">English</string>
|
<string name="settings_language_en">English</string>
|
||||||
|
|
||||||
<!-- Live Call -->
|
|
||||||
<string name="live_connecting">Connecting…</string>
|
|
||||||
<string name="live_listening">Listening…</string>
|
|
||||||
<string name="live_ready">Just start talking</string>
|
|
||||||
<string name="live_speaking">Responding…</string>
|
|
||||||
<string name="live_end">End</string>
|
|
||||||
<string name="live_reconnecting">Reconnecting…</string>
|
|
||||||
</resources>
|
</resources>
|
||||||
|
|
|
||||||
|
|
@ -182,12 +182,4 @@
|
||||||
<string name="settings_language_system">System</string>
|
<string name="settings_language_system">System</string>
|
||||||
<string name="settings_language_de">Deutsch</string>
|
<string name="settings_language_de">Deutsch</string>
|
||||||
<string name="settings_language_en">English</string>
|
<string name="settings_language_en">English</string>
|
||||||
|
|
||||||
<!-- Live Call -->
|
|
||||||
<string name="live_connecting">Verbinde…</string>
|
|
||||||
<string name="live_listening">Hört zu…</string>
|
|
||||||
<string name="live_ready">Sprich einfach los</string>
|
|
||||||
<string name="live_speaking">Antwortet…</string>
|
|
||||||
<string name="live_end">Beenden</string>
|
|
||||||
<string name="live_reconnecting">Verbindung wird wiederhergestellt…</string>
|
|
||||||
</resources>
|
</resources>
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue