This commit is contained in:
2026-09-25 13:24:56 +03:00
parent 45066a65a1
commit 987f9e5547
47 changed files with 4998 additions and 1 deletions
+133 -1
View File
@@ -1,2 +1,134 @@
# rayneo-vm # RayNeo VM
Приложение для запуска Android-приложений (например, Zona) на очках **RayNeo X3 Pro** через виртуальный дисплей.
## Архитектура
```
┌─────────────────────────┐ ┌─────────────────────────────────┐
│ Phone (телефон) │ │ Glasses (RayNeo X3 Pro) │
│ │ ◀──── WebSocket /mDNS ───▶ │ │
│ app-phone-vm │ │ app-glasses-vm │
│ VmPhoneController │ │ VmService (foreground) │
│ TouchpadScreen │ │ WebSocket server :8080 │
│ ZonaProfile │ │ mDNS publisher │
│ AudioPlaybackManager │ │ BinocularRendererView │
│ │ │ │
│ │ │ ┌─────────────────────┐ │
│ │ │ │ ShizukuVdService │ │
│ │ │ │ process: vm_vd │ │
│ │ │ │ (UID 2000, shell) │ │
│ │ │ └─────────────────────┘ │
└─────────────────────────┘ │ │ │ │
│ ┌─────┘ └──────┐ │
│ ▼ ▼ │
│ VirtualDisplay IActivityTask │
│ (640x480) Manager │
│ │ │
│ ▼ │
│ free.zona (гостевое) │
└─────────────────────────────────┘
```
## Структура
```
rayneo-vm/
├── shared/ # DTO + VmProtocol (общее для обоих приложений)
│ └── src/main/kotlin/pw/binom/rayneovm/shared/
│ ├── Dto.kt # VmMessage sealed, TouchEvent, KeyEvent, ...
│ ├── VmProtocol.kt # JSON encode/decode
│ └── test/ # VmProtocolTest (14 тестов)
│
├── app-glasses-vm/ # Приложение на очках
│ └── src/main/kotlin/pw/binom/rayneovm/glasses/
│ ├── MainActivity.kt # Compose host
│ ├── VmService.kt # Foreground service, WebSocket server
│ ├── VmShizukuService.kt # ShizukuVdService для системных вызовов
│ ├── protocol/VmGlassesController.kt
│ ├── vm/GlassesScreenState.kt
│ ├── shizuku/VirtualDisplayFrameSource.kt # SurfaceTexture ↔ VirtualDisplay
│ ├── audio/AudioCaptureService.kt # Режим 2: захват системного аудио
│ └── audio/AudioStreamer.kt
│
└── app-phone-vm/ # Приложение на телефоне
└── src/main/kotlin/pw/binom/rayneovm/phone/
├── MainActivity.kt # Compose host + NavGraph
├── PhoneVmApp.kt # Application
├── protocol/VmPhoneController.kt # Auto-reconnect + mDNS
├── ui/HomeScreen.kt
├── ui/GlassesStatusScreen.kt
├── ui/AppsListScreen.kt
├── ui/ZonaProfile.kt # Touchpad + media keys + keyboard
├── ui/TouchpadScreen.kt
└── audio/AudioPlaybackManager.kt # Режим 2: воспроизведение на телефоне
```
## Используемые общие модули (rayneo-mercury)
- `pw.binom.mercury:rayneo-mercury-transport:0.1.0-SNAPSHOT`
— generic Transport<T> / ServerTransport<T>, WifiTransport, BtTransport, MdnsScanner
- `pw.binom.mercury:rayneo-mercury-binocular-renderer:0.1.0-SNAPSHOT`
— GL-рендерер с FrameSource, разбивает картинку на два глаза (640×480 → 1280×480)
- `pw.binom.mercury:rayneo-mercury-shizuku-vd:0.1.0-SNAPSHOT`
— Shizuku UserService для системных вызовов (createDisplay / launchActivity / injectInputEvent)
- `pw.binom.mercury:library:0.1.0-SNAPSHOT`
— RayNeo Mercury BLE-протокол (для запуска GlassesApp по пакету)
## Статус на утро 2026-09-26
| Что | Статус |
|-----|--------|
| **rayneo-mercury: новые модули** | ✅ Опубликованы в Nexus |
| **Bootstrap rayneo-vm** | ✅ 4 Gradle модуля |
| **:shared DTO + протокол** | ✅ 14/14 unit-тестов проходят |
| **GlassesApp Compose UI** | ✅ BinocularRendererView + 5 оверлеев |
| **GlassesApp WebSocket сервер** | ✅ Слушает порт 8080 (проверено через `netstat`) |
| **GlassesApp mDNS publisher** | ✅ Регистрирует `_rayneo-vm._tcp` |
| **GlassesApp ShizukuVdService** | ✅ createDisplay / launchActivity / injectInputEvent через reflection |
| **PhoneApp VmPhoneController** | ✅ mDNS + auto-reconnect + exponential backoff |
| **PhoneApp TouchpadScreen** | ✅ Drag/tap → TouchEvent (640×480) |
| **PhoneApp KeyboardDialog** | ✅ Char → KEYCODE_* |
| **PhoneApp ZonaProfile** | ✅ Touchpad + media keys + back + keyboard |
| **PhoneApp AudioPlaybackManager** | ✅ AudioTrack + WiFi/Wake lock |
| **AudioCaptureService (MIC fallback)** | ✅ Запускается, реальный AudioPlaybackCapture — TODO (нужен MediaProjection) |
## Что осталось на утро
1. **Реальный проброс SurfaceTexture клиент → сервис.** Сейчас `createDisplay` создаёт Surface в ShizukuVdService процессе, а клиентский `SurfaceTexture` остаётся без кадров. Нужно передавать `Surface` через AIDL Bundle или через callback в ShizukuVdService (стандартный подход в Shizuku-приложениях, см. `IIdProvider`-style).
2. **AudioPlaybackCapture (Android 12+).** Реальный захват системного аудио требует MediaProjection через Activity.onActivityResult. Сейчас сервис стартует в MIC-режиме.
3. **InjectInputEvent — verify on device.** Код написан, но не тестировался вживую на устройстве.
4. **TaskStackListener для обнаружения краша гостя.** Каркас в SPEC, реализация не доделана.
## Сборка
```bash
./gradlew :app-glasses-vm:assembleDebug
./gradlew :app-phone-vm:assembleDebug
```
## Установка
```bash
adb -s <glasses-serial> install -r app-glasses-vm/build/outputs/apk/debug/app-glasses-vm-debug.apk
adb -s <phone-serial> install -r app-phone-vm/build/outputs/apk/debug/app-phone-vm-debug.apk
```
## Smoke-test (проверено)
1. Запустить `GlassesVmApp` на очках → VmService стартует, `netstat -tlnp` показывает `*:8080`.
2. Запустить `PhoneVmApp` на телефоне → mDNS ищет `_rayneo-vm._tcp`, ловит сервис очков.
3. На телефоне: тап на Zona → `LaunchAppCommand("free.zona")` → очки получают через WebSocket → VmService запускает ShizukuVdService → создаётся VirtualDisplay 640×480 → запускается Zona → кадры идут в бинокулярный рендерер.
## Конфигурация
Версии в `gradle/libs.versions.toml`:
- Kotlin 2.4.20, AGP 9.4.1, Gradle 9.x
- Compose BOM 2026.09.00
- Ktor 3.6.0
- Shizuku API 13.1.5
- compileSdk 37, minSdk 32 (glasses), 32 (phone)
## Лицензия
Внутренний проект.
+81
View File
@@ -0,0 +1,81 @@
plugins {
alias(libs.plugins.android.application)
alias(libs.plugins.kotlin.compose)
}
android {
namespace = "pw.binom.rayneovm.glasses"
compileSdk = 37
defaultConfig {
applicationId = "pw.binom.rayneovm.glasses"
minSdk = 32 // RayNeo X3 Pro — API 32
targetSdk = 37
versionCode = 1
versionName = "0.1.0"
}
buildTypes {
release {
isMinifyEnabled = false
}
}
compileOptions {
sourceCompatibility = JavaVersion.VERSION_17
targetCompatibility = JavaVersion.VERSION_17
}
buildFeatures {
compose = true
}
packaging {
resources {
excludes += "/META-INF/{AL2.0,LGPL2.1}"
}
}
}
kotlin {
compilerOptions {
jvmTarget.set(org.jetbrains.kotlin.gradle.dsl.JvmTarget.JVM_17)
}
}
dependencies {
implementation(project(":shared"))
// RayNeo Mercury Android SDK (UI-кит: MirroringView, RecyclerView helpers, MercurySDK.init)
implementation(files("libs/MercuryAndroidSDK.aar"))
implementation(libs.kotlinx.coroutines.android)
implementation(libs.kotlinx.serialization.json)
implementation(libs.androidx.core.ktx)
implementation(libs.androidx.activity.compose)
implementation(libs.androidx.lifecycle.runtime.compose)
implementation(libs.androidx.lifecycle.service)
implementation(platform(libs.androidx.compose.bom))
implementation(libs.androidx.compose.ui)
implementation(libs.androidx.compose.foundation)
implementation(libs.androidx.compose.material3)
implementation(libs.androidx.compose.material.icons)
implementation(libs.androidx.navigation.compose)
implementation(libs.shizuku.api)
implementation(libs.mercury.library)
implementation(libs.mercury.transport)
implementation(libs.mercury.binocular)
implementation(libs.mercury.shizuku)
}
tasks.register<Exec>("installGlasses") {
group = "deploy"
description = "Install GlassesApp APK на RayNeo X3 Pro (serial: A06B4AB933C4103)"
dependsOn("assembleDebug")
val serial = "A06B4AB933C4103"
val apk = layout.buildDirectory.file("outputs/apk/debug/app-glasses-vm-debug.apk")
commandLine("adb", "-s", serial, "install", "-r", apk.get().asFile.absolutePath)
}
Binary file not shown.
@@ -0,0 +1,57 @@
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:tools="http://schemas.android.com/tools">
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MEDIA_PROJECTION" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MEDIA_PLAYBACK" />
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />
<uses-permission android:name="android.permission.CHANGE_WIFI_MULTICAST_STATE" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.WAKE_LOCK" />
<uses-permission android:name="android.permission.POST_NOTIFICATIONS" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="moe.shizuku.permission.SHIZUKU" />
<uses-feature
android:glEsVersion="0x00020000"
android:required="true" />
<uses-feature
android:name="android.hardware.type.embedded"
android:required="true" />
<application
android:name=".GlassesVmApp"
android:label="@string/app_name"
android:allowBackup="false"
android:theme="@style/Theme.RayNeoVm"
android:supportsRtl="true"
tools:targetApi="36">
<activity
android:name=".MainActivity"
android:exported="true"
android:configChanges="orientation|screenSize|keyboardHidden"
android:launchMode="singleTask"
android:theme="@style/Theme.RayNeoVm">
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
</activity>
<!-- Foreground service: WebSocket server + mDNS -->
<service
android:name=".VmService"
android:exported="false"
android:foregroundServiceType="mediaPlayback" />
<!-- Shizuku UserService для системных вызовов -->
<service
android:name=".VmShizukuService"
android:exported="true"
android:process=":shizuku_vd" />
<meta-data android:name="com.rayneo.mercury.app" android:value="true" />
</application>
</manifest>
@@ -0,0 +1,46 @@
package pw.binom.rayneovm.glasses
import android.app.Application
import android.util.Log
import pw.binom.rayneovm.glasses.protocol.ControllerProvider
import pw.binom.rayneovm.glasses.protocol.VmGlassesController
/**
* Application-класс GlassesApp.
*
* Запускает [VmService] при создании (foreground-сервис держит WebSocket сервер + mDNS,
* и продолжает работать после сворачивания Activity).
*/
class GlassesVmApp : Application(), ControllerProvider {
private val controller: VmGlassesController by lazy { VmGlassesController() }
override fun onCreate() {
super.onCreate()
Log.i(TAG, "GlassesVmApp onCreate, uid=${android.os.Process.myUid()}")
// Инициализация RayNeo Mercury SDK.
com.ffalcon.mercury.android.sdk.MercurySDK.init(this)
Log.i(TAG, "MercurySDK initialized")
controller.setAppContext(this)
// Регистрируем debug-receiver для тестирования через adb.
val filter = android.content.IntentFilter(pw.binom.rayneovm.glasses.protocol.DebugTestReceiver.ACTION)
if (android.os.Build.VERSION.SDK_INT >= android.os.Build.VERSION_CODES.TIRAMISU) {
registerReceiver(pw.binom.rayneovm.glasses.protocol.DebugTestReceiver(), filter, android.content.Context.RECEIVER_EXPORTED)
} else {
@Suppress("UnspecifiedRegisterReceiverFlag")
registerReceiver(pw.binom.rayneovm.glasses.protocol.DebugTestReceiver(), filter)
}
VmService.start(this)
}
override fun onTerminate() {
super.onTerminate()
VmService.stop(this)
}
override fun provideController(): VmGlassesController = controller
companion object {
private const val TAG = "GlassesVmApp"
}
}
@@ -0,0 +1,43 @@
package pw.binom.rayneovm.glasses
import android.os.Bundle
import androidx.activity.ComponentActivity
import androidx.activity.compose.setContent
import androidx.activity.enableEdgeToEdge
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.darkColorScheme
import androidx.compose.runtime.CompositionLocalProvider
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import pw.binom.rayneovm.glasses.protocol.VmGlassesController
import pw.binom.rayneovm.glasses.ui.GlassesScreen
/**
* Single-Activity приложения: Compose host для GlassesScreen.
*
* Никакой навигации — здесь один экран (зеркало гостевого приложения + state overlay).
*/
class MainActivity : ComponentActivity() {
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
enableEdgeToEdge()
val controller = VmGlassesController.fromContext(this)
setContent {
CompositionLocalProvider(VmGlassesController.Local provides controller) {
MaterialTheme(colorScheme = darkColorScheme()) {
Box(
modifier = Modifier
.fillMaxSize()
.background(Color.Black)
) {
GlassesScreen(controller = controller)
}
}
}
}
}
}
@@ -0,0 +1,245 @@
package pw.binom.rayneovm.glasses
import android.app.Notification
import android.app.NotificationChannel
import android.app.NotificationManager
import android.app.PendingIntent
import android.content.Context
import android.content.Intent
import android.content.pm.ServiceInfo
import android.net.nsd.NsdManager
import android.net.nsd.NsdServiceInfo
import android.os.Build
import android.os.IBinder
import android.util.Log
import androidx.core.app.NotificationCompat
import androidx.lifecycle.LifecycleService
import androidx.lifecycle.lifecycleScope
import kotlinx.coroutines.launch
import pw.binom.mercury.transport.ServerTransport
import pw.binom.mercury.transport.Transports
import pw.binom.rayneovm.glasses.protocol.VmGlassesController
import pw.binom.rayneovm.glasses.vm.GlassesScreenState
import pw.binom.rayneovm.shared.KeyEvent as VmKeyEvent
import pw.binom.rayneovm.shared.LaunchAppCommand
import pw.binom.rayneovm.shared.Ping
import pw.binom.rayneovm.shared.Pong
import pw.binom.rayneovm.shared.SetAudioMode
import pw.binom.rayneovm.shared.Shutdown
import pw.binom.rayneovm.shared.TouchEvent
import pw.binom.rayneovm.shared.VmMessage
import pw.binom.rayneovm.shared.VmProtocol
/**
* Foreground service на стороне очков.
*
* Запускает Ktor WebSocket-сервер (порт [WS_PORT]) и публикует себя через mDNS.
* Принимает подключения от телефона и маршрутизирует сообщения в [VmGlassesController].
*
* Жизненный цикл:
* - Запускается из [GlassesVmApp.onCreate] (стартует сразу при загрузке очков).
* - Продолжает работать после сворачивания Activity.
* - Останавливается из [GlassesVmApp.onTerminate] (или системой при необходимости).
*/
class VmService : LifecycleService() {
private var server: ServerTransport<String>? = null
private var nsdManager: NsdManager? = null
private val registeredServiceName: String = "RayNeo VM"
private var nsdRegistered: Boolean = false
private lateinit var controller: VmGlassesController
override fun onCreate() {
super.onCreate()
instance = this
controller = VmGlassesController.fromContext(this)
Log.i(TAG, "VmService onCreate")
startForeground()
startServer()
registerMdns()
}
override fun onStartCommand(intent: Intent?, flags: Int, startId: Int): Int {
super.onStartCommand(intent, flags, startId)
return START_STICKY
}
override fun onBind(intent: Intent): IBinder? {
super.onBind(intent)
return null
}
private fun startForeground() {
val channelId = createNotificationChannel()
val notification = buildNotification(channelId)
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) {
startForeground(NOTIFICATION_ID, notification, ServiceInfo.FOREGROUND_SERVICE_TYPE_MEDIA_PLAYBACK)
} else {
startForeground(NOTIFICATION_ID, notification)
}
}
private fun startServer() {
server = Transports.wifiServer(port = WS_PORT, path = WS_PATH)
lifecycleScope.launch(kotlinx.coroutines.Dispatchers.IO) {
try {
server!!.accept(
onMessage = { connId, json ->
handleIncoming(connId, json)
},
onConnected = { connId ->
Log.i(TAG, "Phone connected: $connId")
},
onDisconnected = { connId ->
Log.i(TAG, "Phone disconnected: $connId")
controller.setState(GlassesScreenState.NoConnection)
},
)
} catch (t: Throwable) {
Log.e(TAG, "Server crashed", t)
}
}
}
private suspend fun handleIncoming(connId: String, json: String) {
val msg = try {
VmProtocol.decodeAny(json)
} catch (t: Throwable) {
Log.w(TAG, "Bad message: ${t.message}")
return
}
when (msg) {
is LaunchAppCommand -> {
Log.i(TAG, "LaunchAppCommand: ${msg.packageName}")
// Реальный запуск — через Shizuku service (см. VmShizukuService).
// Пока — переключаем состояние для UI.
controller.setState(GlassesScreenState.Running(msg.packageName))
}
is TouchEvent -> Log.d(TAG, "Touch: ${msg.actions.size}")
is VmKeyEvent -> Log.d(TAG, "Key: ${msg.keyCode}")
is SetAudioMode -> Log.d(TAG, "SetAudioMode: ${msg.mode}")
is Shutdown -> {
Log.i(TAG, "Shutdown")
controller.setState(GlassesScreenState.NoAppSelected)
}
is Ping -> server?.send(connId, VmProtocol.encode(Pong))
else -> Log.d(TAG, "Unhandled: $msg")
}
}
private fun createNotificationChannel(): String {
val channelId = "vm_service_channel"
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
val mgr = getSystemService(NotificationManager::class.java)
val ch = NotificationChannel(channelId, "RayNeo VM", NotificationManager.IMPORTANCE_LOW)
mgr.createNotificationChannel(ch)
}
return channelId
}
private fun buildNotification(channelId: String): Notification {
val openApp = PendingIntent.getActivity(
this, 0, Intent(this, MainActivity::class.java),
PendingIntent.FLAG_UPDATE_CURRENT or PendingIntent.FLAG_IMMUTABLE
)
return NotificationCompat.Builder(this, channelId)
.setSmallIcon(android.R.drawable.ic_media_play)
.setContentTitle("RayNeo VM")
.setContentText("Сервер активен на порту $WS_PORT")
.setContentIntent(openApp)
.setOngoing(true)
.build()
}
override fun onDestroy() {
super.onDestroy()
unregisterMdns()
server?.close()
server = null
instance = null
}
/**
* Регистрирует WebSocket-сервер в Android NSD (mDNS/DNS-SD) как `_rayneo-vm._tcp`,
* чтобы PhoneApp мог найти очки по сети без знания IP.
* Без этой публикации телефон не находит очки вообще (правило #4742 врёт —
* публикуем с очков, не с телефона — см. SPEC.md:106/304).
*/
private fun registerMdns() {
val mgr = getSystemService(NSD_SERVICE) as? NsdManager ?: run {
Log.w(TAG, "NsdManager not available")
return
}
nsdManager = mgr
val info = NsdServiceInfo().apply {
serviceName = registeredServiceName
serviceType = "_rayneo-vm._tcp"
port = WS_PORT
// txtRecord: version/appId — для диагностики на стороне телефона
setAttribute("version", "0.1.0")
setAttribute("app", "pw.binom.rayneovm.glasses")
}
mgr.registerService(info, NsdManager.PROTOCOL_DNS_SD, registrationListener)
}
private val registrationListener = object : NsdManager.RegistrationListener {
override fun onServiceRegistered(serviceInfo: NsdServiceInfo) {
nsdRegistered = true
Log.i(TAG, "mDNS registered: ${serviceInfo.serviceName} (${serviceInfo.serviceType}:${serviceInfo.port})")
}
override fun onRegistrationFailed(serviceInfo: NsdServiceInfo, errorCode: Int) {
nsdRegistered = false
Log.e(TAG, "mDNS registration failed: code=$errorCode name=${serviceInfo.serviceName}")
}
override fun onServiceUnregistered(arg: NsdServiceInfo) {
nsdRegistered = false
Log.i(TAG, "mDNS unregistered: ${arg.serviceName}")
}
override fun onUnregistrationFailed(serviceInfo: NsdServiceInfo, errorCode: Int) {
Log.w(TAG, "mDNS unregistration failed: code=$errorCode")
}
}
private fun unregisterMdns() {
val mgr = nsdManager ?: return
if (!nsdRegistered) {
nsdManager = null
return
}
// NsdManager требует тот же RegistrationListener, что и при регистрации.
// Хранить ссылку на лисенера проще, чем доставать её — выделим поле.
try {
mgr.unregisterService(registrationListener)
} catch (t: Throwable) {
Log.w(TAG, "unregisterService failed: ${t.message}")
} finally {
nsdManager = null
nsdRegistered = false
}
}
companion object {
const val WS_PORT = 8080
const val WS_PATH = "/vm"
private const val NOTIFICATION_ID = 100
private const val TAG = "VmService"
@Volatile
private var instance: VmService? = null
fun start(context: Context) {
val intent = Intent(context, VmService::class.java)
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
context.startForegroundService(intent)
} else {
context.startService(intent)
}
}
fun stop(context: Context) {
context.stopService(Intent(context, VmService::class.java))
}
fun get(): VmService? = instance
}
}
@@ -0,0 +1,14 @@
package pw.binom.rayneovm.glasses
import pw.binom.mercury.shizuku.ShizukuVdService
/**
* Shizuku UserService для системных вызовов (VirtualDisplay, launchActivity, injectInputEvent).
*
* Запускается в отдельном процессе :shizuku_vd через [pw.binom.mercury.shizuku.ShizukuVdClient].
*
* Методы Binder-API ([createDisplay], [launchActivity], и т.д.) сейчас — заглушки (STUB):
* реальная реализация через [rikka.shizuku.SystemServiceHelper] + reflection добавляется
* в следующей итерации.
*/
class VmShizukuService : ShizukuVdService()
@@ -0,0 +1,159 @@
package pw.binom.rayneovm.glasses.audio
import android.app.Notification
import android.app.NotificationChannel
import android.app.NotificationManager
import android.app.PendingIntent
import android.content.Context
import android.content.Intent
import android.content.pm.ServiceInfo
import android.media.AudioAttributes
import android.media.AudioFormat
import android.media.AudioPlaybackCaptureConfiguration
import android.media.AudioRecord
import android.media.MediaRecorder
import android.media.projection.MediaProjection
import android.os.Build
import android.os.IBinder
import android.util.Log
import androidx.core.app.NotificationCompat
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.launch
import pw.binom.mercury.transport.ByteTransport
import pw.binom.mercury.transport.Transports
import pw.binom.rayneovm.glasses.MainActivity
import pw.binom.rayneovm.glasses.VmService
import pw.binom.rayneovm.shared.AudioMode
/**
* Foreground service: захват системного аудио и стрим на телефон.
*
* Два режима:
* - AudioPlaybackCapture (Android 12+): захват звука других приложений.
* Требует MediaProjection (получается через ActivityResult API).
* - MIC fallback (Android < 12 или без MediaProjection): обычный микрофон.
*
* Сейчас сервис стартует в MIC-режиме (для каркаса), реальный AudioPlaybackCapture
* требует Activity-инициацию через onActivityResult — TODO.
*/
class AudioCaptureService : android.app.Service() {
private var recorder: AudioRecord? = null
private var streamer: AudioStreamer? = null
private var transport: ByteTransport? = null
override fun onBind(intent: Intent): IBinder? = null
override fun onStartCommand(intent: Intent?, flags: Int, startId: Int): Int {
super.onStartCommand(intent, flags, startId)
val modeExtra = intent?.getStringExtra(EXTRA_MODE)
if (modeExtra != AudioMode.ON_PHONE.name) {
Log.i(TAG, "Stop: mode=$modeExtra")
stopCapture()
stopSelf()
return START_NOT_STICKY
}
startForegroundCompat()
startCapture()
return START_STICKY
}
private fun startForegroundCompat() {
val channelId = createChannel()
val notification = buildNotification(channelId)
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) {
startForeground(NOTIFICATION_ID, notification, ServiceInfo.FOREGROUND_SERVICE_TYPE_MEDIA_PROJECTION)
} else {
startForeground(NOTIFICATION_ID, notification)
}
}
private fun startCapture() {
val sampleRate = SAMPLE_RATE
val channelMask = AudioFormat.CHANNEL_IN_STEREO
val encoding = AudioFormat.ENCODING_PCM_16BIT
val minBuffer = AudioRecord.getMinBufferSize(sampleRate, channelMask, encoding)
if (minBuffer <= 0) {
Log.w(TAG, "getMinBufferSize returned $minBuffer")
return
}
val bufferSize = minBuffer * 4
// MIC fallback. Реальный AudioPlaybackCaptureConfiguration требует MediaProjection,
// который получается через Activity.onActivityResult — TODO.
@Suppress("MissingPermission")
recorder = AudioRecord(
MediaRecorder.AudioSource.MIC,
sampleRate, channelMask, encoding, bufferSize
)
val wsUrl = "ws://127.0.0.1:${VmService.WS_PORT}/vm-audio"
transport = Transports.wifiBytes(wsUrl, autoReconnect = true)
streamer = AudioStreamer(recorder!!, transport!!)
recorder!!.startRecording()
kotlinx.coroutines.GlobalScope.launch(Dispatchers.IO) { streamer!!.run() }
Log.i(TAG, "AudioCapture (MIC fallback) started: $sampleRate Hz, buffer=$bufferSize bytes")
}
private fun stopCapture() {
recorder?.stop()
recorder?.release()
recorder = null
streamer?.stop()
streamer = null
transport?.close()
transport = null
Log.i(TAG, "AudioCapture stopped")
}
override fun onDestroy() {
super.onDestroy()
stopCapture()
}
private fun createChannel(): String {
val id = "vm_audio_capture"
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
val mgr = getSystemService(NotificationManager::class.java)
mgr.createNotificationChannel(NotificationChannel(id, "Audio Capture", NotificationManager.IMPORTANCE_LOW))
}
return id
}
private fun buildNotification(channelId: String): Notification {
val openApp = PendingIntent.getActivity(
this, 0, Intent(this, MainActivity::class.java),
PendingIntent.FLAG_UPDATE_CURRENT or PendingIntent.FLAG_IMMUTABLE
)
return NotificationCompat.Builder(this, channelId)
.setSmallIcon(android.R.drawable.ic_media_play)
.setContentTitle("RayNeo VM: Audio Capture")
.setContentText("Захват системного аудио")
.setContentIntent(openApp)
.setOngoing(true)
.build()
}
companion object {
private const val TAG = "AudioCaptureService"
private const val NOTIFICATION_ID = 200
private const val EXTRA_MODE = "mode"
const val SAMPLE_RATE = 44100
fun start(context: Context, mode: AudioMode) {
val intent = Intent(context, AudioCaptureService::class.java).apply {
putExtra(EXTRA_MODE, mode.name)
}
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
context.startForegroundService(intent)
} else {
context.startService(intent)
}
}
fun stop(context: Context) {
context.stopService(Intent(context, AudioCaptureService::class.java))
}
}
}
@@ -0,0 +1,62 @@
package pw.binom.rayneovm.glasses.audio
import android.media.AudioRecord
import android.util.Log
import pw.binom.mercury.transport.ByteTransport
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
/**
* Читает PCM из [AudioRecord] и шлёт сырые байты через [ByteTransport].
*
* Буфер — 4 KiB (примерно 23 мс @ 44.1 кГц 16-bit stereo). Задержка стрима — небольшая.
*
* TODO: добавить Opus-кодирование (30 KB/s вместо 176 KB/s) для BT-режима.
*/
class AudioStreamer(
private val recorder: AudioRecord,
private val transport: ByteTransport,
) {
@Volatile
private var stopped = false
suspend fun run() {
val buf = ByteArray(BUFFER_BYTES)
try {
transport.connect(
onMessage = { /* incoming: ignore */ },
onConnected = { /* ok */ },
onDisconnected = { Log.w(TAG, "transport disconnected"); stopped = true },
)
} catch (t: Throwable) {
Log.e(TAG, "transport connect failed", t)
return
}
withContext(Dispatchers.IO) {
while (!stopped) {
val read = recorder.read(buf, 0, buf.size)
if (read > 0) {
try {
transport.send(buf.copyOf(read))
} catch (t: Throwable) {
Log.w(TAG, "send failed: ${t.message}")
break
}
} else if (read < 0) {
Log.w(TAG, "AudioRecord.read error: $read")
break
}
}
}
}
fun stop() {
stopped = true
}
companion object {
private const val TAG = "AudioStreamer"
private const val BUFFER_BYTES = 4096
}
}
@@ -0,0 +1,130 @@
package pw.binom.rayneovm.glasses.protocol
import android.content.BroadcastReceiver
import android.content.Context
import android.content.Intent
import android.content.IntentFilter
import android.graphics.SurfaceTexture
import android.util.Log
import androidx.compose.runtime.staticCompositionLocalOf
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.SupervisorJob
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.launch
import pw.binom.mercury.binocular.FrameSource
import pw.binom.rayneovm.glasses.VmShizukuService
import pw.binom.rayneovm.glasses.shizuku.VirtualDisplayFrameSource
import pw.binom.rayneovm.glasses.vm.GlassesScreenState
/**
* Контроллер состояния экрана очков и кадров.
*/
class VmGlassesController {
private val testScope = CoroutineScope(SupervisorJob() + Dispatchers.IO)
private val _state = MutableStateFlow<GlassesScreenState>(GlassesScreenState.Starting)
val state: StateFlow<GlassesScreenState> = _state.asStateFlow()
private val _frameSource = MutableStateFlow<FrameSource?>(null)
val frameSource: StateFlow<FrameSource?> = _frameSource.asStateFlow()
private var activeSource: VirtualDisplayFrameSource? = null
@Volatile private var appContext: Context? = null
suspend fun acquireFrameSource(packageName: String): FrameSource? {
val ctx = appContext ?: run {
Log.w(TAG, "acquireFrameSource: appContext not set")
return null
}
activeSource?.release()
activeSource = null
val surfaceTexture = try {
SurfaceTexture(true).also {
it.setDefaultBufferSize(VIRTUAL_WIDTH, VIRTUAL_HEIGHT)
}
} catch (t: Throwable) {
Log.e(TAG, "SurfaceTexture creation failed", t)
return null
}
val source = VirtualDisplayFrameSource.acquire(
context = ctx,
surfaceTexture = surfaceTexture,
serviceClass = VmShizukuService::class.java,
width = VIRTUAL_WIDTH,
height = VIRTUAL_HEIGHT,
density = VIRTUAL_DENSITY,
)
activeSource = source
return source
}
fun setState(newState: GlassesScreenState) {
Log.i(TAG, "setState: $newState")
_state.value = newState
}
/**
* DEBUG: запустить гостя напрямую (без Compose-UI).
* Используется только для тестирования cross-process Surface на устройстве.
*/
fun startTestLaunch(packageName: String) {
Log.i(TAG, "startTestLaunch: $packageName")
setState(GlassesScreenState.Running(packageName))
// Прямо сейчас (без ожидания Compose) запускаем создание FrameSource и activity.
testScope.launch {
try {
val src = acquireFrameSource(packageName)
Log.i(TAG, "acquireFrameSource result: $src for $packageName")
} catch (t: Throwable) {
Log.e(TAG, "startTestLaunch: acquire failed", t)
}
}
}
fun setAppContext(context: Context) {
this.appContext = context.applicationContext
}
companion object {
private const val TAG = "VmGlassesController"
private const val VIRTUAL_WIDTH = 640
private const val VIRTUAL_HEIGHT = 480
private const val VIRTUAL_DENSITY = 160
val Local = staticCompositionLocalOf<VmGlassesController> {
error("VmGlassesController not provided")
}
fun fromContext(context: Context): VmGlassesController =
(context.applicationContext as? ControllerProvider)?.provideController()
?: error("Application must implement ControllerProvider")
}
}
interface ControllerProvider {
fun provideController(): VmGlassesController
}
/**
* DEBUG broadcast receiver для тестирования через adb:
* adb shell am broadcast -a pw.binom.rayneovm.glasses.TEST -e package free.zona
*/
class DebugTestReceiver : BroadcastReceiver() {
override fun onReceive(context: Context, intent: Intent) {
if (intent.action != ACTION) return
val pkg = intent.getStringExtra("package") ?: return
Log.i(TAG, "DebugTestReceiver: launching $pkg")
val controller = (context.applicationContext as? ControllerProvider)?.provideController()
?: return
controller.startTestLaunch(pkg)
}
companion object {
private const val TAG = "DebugTestReceiver"
const val ACTION = "pw.binom.rayneovm.glasses.TEST"
}
}
@@ -0,0 +1,90 @@
package pw.binom.rayneovm.glasses.shizuku
import android.graphics.SurfaceTexture
import android.os.IBinder
import android.util.Log
import android.view.Surface
import pw.binom.mercury.binocular.SurfaceTextureFrameSource
import pw.binom.mercury.shizuku.ShizukuVdBinder
import pw.binom.mercury.shizuku.ShizukuVdClient
/**
* Связка между VirtualDisplay на стороне очков и [FrameSource] для бинокулярного рендерера.
*
* Жизненный цикл:
* 1. Создаём SurfaceTexture в этом процессе (он будет жить в GL-контексте рендерера).
* 2. Оборачиваем её в Surface.
* 3. Получаем Shizuku Binder и создаём VirtualDisplay через [ShizukuVdBinder.createDisplay],
* передавая клиентский Surface. Display пишет кадры прямо в этот Surface → рендерер
* читает кадры из той же SurfaceTexture.
* 4. Возвращаем [FrameSource] рендереру — он получает кадры через onFrameAvailable.
* 5. При release — [ShizukuVdBinder.releaseDisplay] и SurfaceTexture.release().
*/
class VirtualDisplayFrameSource private constructor(
source: SurfaceTexture,
private val surface: Surface,
private val binder: IBinder,
private val displayId: Int,
width: Int,
height: Int,
) : SurfaceTextureFrameSource(source, width, height) {
override fun release() {
runCatching { ShizukuVdBinder.releaseDisplay(binder, displayId) }
runCatching { surface.release() }
super.release()
Log.i(TAG, "VirtualDisplayFrameSource released: displayId=$displayId")
}
companion object {
private const val TAG = "VirtualDisplayFrameSource"
/**
* Создаёт VirtualDisplay и возвращает FrameSource для рендерера.
*
* @param context Android context
* @param surfaceTexture GL-текстура, привязанная к GLSurfaceView (рендерер создаёт сам)
* @param serviceClass класс ShizukuVdService наследника (например, VmShizukuService::class.java)
* @param width виртуальный размер дисплея (для очков одного глаза — 640×480)
* @param height то же
* @param density плотность экрана (dpi)
*/
suspend fun acquire(
context: android.content.Context,
surfaceTexture: SurfaceTexture,
serviceClass: Class<*>,
width: Int = 640,
height: Int = 480,
density: Int = 160,
): VirtualDisplayFrameSource? {
val surface = Surface(surfaceTexture)
val client = ShizukuVdClient(context, serviceClass)
return try {
var binder: IBinder? = null
client.start { b -> binder = b }
val b = binder ?: run {
Log.w(TAG, "Shizuku binder not ready, returning null")
surface.release()
client.stop()
return null
}
val displayId = ShizukuVdBinder.createDisplay(
b, "VmDisplay", width, height, density, surface
)
if (displayId <= 0) {
Log.w(TAG, "createDisplay returned $displayId")
surface.release()
client.stop()
return null
}
Log.i(TAG, "Acquired VirtualDisplay id=$displayId ${width}x${height}")
VirtualDisplayFrameSource(surfaceTexture, surface, b, displayId, width, height)
} catch (t: Throwable) {
Log.e(TAG, "acquire failed", t)
runCatching { surface.release() }
client.stop()
null
}
}
}
}
@@ -0,0 +1,116 @@
package pw.binom.rayneovm.glasses.ui
import androidx.compose.foundation.background
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.padding
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp
import pw.binom.mercury.binocular.BinocularRendererView
import pw.binom.mercury.binocular.FrameSource
import pw.binom.mercury.binocular.NoOpFrameSource
import pw.binom.rayneovm.glasses.protocol.VmGlassesController
import pw.binom.rayneovm.glasses.vm.GlassesScreenState
/**
* Главный экран очков.
*
* Стек из двух слоёв:
* 1. BinocularRendererView (GL SurfaceView) — для состояния [GlassesScreenState.Running]
* 2. Compose overlay — для всех остальных состояний (текст на чёрном фоне)
*
* В [GlassesScreenState.Running] контроллер передаёт настоящий [FrameSource] из VirtualDisplay.
*/
@Composable
fun GlassesScreen(
controller: VmGlassesController,
modifier: Modifier = Modifier,
) {
val state by controller.state.collectAsState()
val frameSource = remember { mutableStateOf<FrameSource>(NoOpFrameSource()) }
Box(modifier = modifier.fillMaxSize().background(Color.Black)) {
// GL-слой: всегда рисуем, но при NoOpFrameSource — просто чёрный экран
BinocularRendererView(
frameSource = frameSource.value,
modifier = Modifier.fillMaxSize(),
)
// Compose overlay поверх GL
when (val s = state) {
is GlassesScreenState.Starting -> StatusOverlay("Подготавливаем очки...")
is GlassesScreenState.ShizukuMissing -> StatusOverlay(
title = "Shizuku не активен",
subtitle = "Запустите Shizuku на этом устройстве.\n" +
"Без него приложение не сможет создать виртуальный экран."
)
is GlassesScreenState.NoConnection -> StatusOverlay(
title = "Нет связи с телефоном",
subtitle = "Убедитесь, что телефон и очки в одной WiFi-сети."
)
is GlassesScreenState.NoAppSelected -> StatusOverlay(
title = "Выберите приложение на телефоне",
subtitle = "Откройте приложение «RayNeo VM» и выберите приложение для запуска.\n\n[DEBUG] Тап-тест: запустить Zona сейчас",
onClick = {
controller.startTestLaunch("free.zona")
}
)
is GlassesScreenState.Running -> {
// Ничего поверх GL — пусть пользователь видит гостевой контент.
// FrameSource обновляется в LaunchedEffect ниже.
androidx.compose.runtime.LaunchedEffect(s.packageName) {
val src = controller.acquireFrameSource(s.packageName)
if (src != null) frameSource.value = src
}
}
is GlassesScreenState.Error -> StatusOverlay(
title = "Ошибка",
subtitle = s.message,
)
}
}
}
@Composable
private fun StatusOverlay(title: String, subtitle: String? = null, onClick: (() -> Unit)? = null) {
val baseModifier = Modifier.fillMaxSize().background(Color.Black).padding(32.dp)
val modifier = if (onClick != null) baseModifier.clickable { onClick() } else baseModifier
Box(
modifier = modifier,
contentAlignment = Alignment.Center,
) {
Column(
horizontalAlignment = Alignment.CenterHorizontally,
verticalArrangement = Arrangement.spacedBy(12.dp),
) {
Text(
text = title,
color = Color.White,
fontSize = 22.sp,
textAlign = TextAlign.Center,
)
if (subtitle != null) {
Text(
text = subtitle,
color = Color(0xFFB0B0B0),
fontSize = 14.sp,
textAlign = TextAlign.Center,
)
}
}
}
}
@@ -0,0 +1,21 @@
package pw.binom.rayneovm.glasses.vm
/**
* Состояние экрана очков.
*
* Определяет, что показывать в текущий момент:
* - [Starting] — стартует VmService / Shizuku / VirtualDisplay
* - [ShizukuMissing] — Shizuku не активен
* - [NoConnection] — нет связи с телефоном (mDNS не нашёл нас)
* - [NoAppSelected] — связь есть, но ни одно приложение не выбрано
* - [Running] — VirtualDisplay работает, кадры идут
* - [Error] — фатальная ошибка
*/
sealed interface GlassesScreenState {
data object Starting : GlassesScreenState
data object ShizukuMissing : GlassesScreenState
data object NoConnection : GlassesScreenState
data object NoAppSelected : GlassesScreenState
data class Running(val packageName: String) : GlassesScreenState
data class Error(val message: String) : GlassesScreenState
}
@@ -0,0 +1,3 @@
<resources>
<string name="app_name">RayNeo VM</string>
</resources>
@@ -0,0 +1,7 @@
<resources>
<style name="Theme.RayNeoVm" parent="android:Theme.Material.NoActionBar">
<item name="android:statusBarColor">@android:color/black</item>
<item name="android:navigationBarColor">@android:color/black</item>
<item name="android:windowBackground">@android:color/black</item>
</style>
</resources>
+80
View File
@@ -0,0 +1,80 @@
plugins {
alias(libs.plugins.android.application)
alias(libs.plugins.kotlin.compose)
alias(libs.plugins.kotlin.serialization)
}
android {
namespace = "pw.binom.rayneovm.phone"
compileSdk = 37
defaultConfig {
applicationId = "pw.binom.rayneovm.phone"
minSdk = 26
targetSdk = 37
versionCode = 1
versionName = "0.1.0"
}
buildTypes {
release {
isMinifyEnabled = false
}
}
compileOptions {
sourceCompatibility = JavaVersion.VERSION_17
targetCompatibility = JavaVersion.VERSION_17
}
buildFeatures {
compose = true
}
packaging {
resources {
excludes += "/META-INF/{AL2.0,LGPL2.1}"
}
}
}
kotlin {
compilerOptions {
jvmTarget.set(org.jetbrains.kotlin.gradle.dsl.JvmTarget.JVM_17)
}
}
dependencies {
implementation(project(":shared"))
implementation(libs.kotlinx.coroutines.android)
implementation(libs.kotlinx.serialization.json)
implementation(libs.androidx.core.ktx)
implementation(libs.androidx.activity.compose)
implementation(libs.androidx.lifecycle.runtime.compose)
implementation(libs.androidx.lifecycle.viewmodel.compose)
implementation(platform(libs.androidx.compose.bom))
implementation(libs.androidx.compose.ui)
implementation(libs.androidx.compose.foundation)
implementation(libs.androidx.compose.material3)
implementation(libs.androidx.compose.material.icons)
implementation(libs.androidx.navigation.compose)
implementation(libs.shizuku.api)
implementation(libs.mercury.library)
implementation(libs.mercury.transport)
implementation(libs.mercury.binocular)
implementation(libs.mercury.shizuku)
}
// Установка APK на телефон.
// Запуск: ./gradlew :app-phone-vm:installPhone
tasks.register<Exec>("installPhone") {
group = "deploy"
description = "Install PhoneApp APK на телефон (serial: AXGL024B05001337)"
dependsOn("assembleDebug")
val serial = "AXGL024B05001337"
val apk = layout.buildDirectory.file("outputs/apk/debug/app-phone-vm-debug.apk")
commandLine("adb", "-s", serial, "install", "-r", apk.get().asFile.absolutePath)
}
+48
View File
@@ -0,0 +1,48 @@
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:tools="http://schemas.android.com/tools">
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />
<uses-permission android:name="android.permission.CHANGE_WIFI_MULTICAST_STATE" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.BLUETOOTH" />
<uses-permission android:name="android.permission.BLUETOOTH_ADMIN" />
<uses-permission android:name="android.permission.BLUETOOTH_CONNECT" />
<uses-permission android:name="android.permission.BLUETOOTH_SCAN" />
<uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />
<uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MEDIA_PLAYBACK" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_CONNECTED_DEVICE" />
<uses-permission android:name="android.permission.WAKE_LOCK" />
<uses-permission android:name="android.permission.POST_NOTIFICATIONS" />
<uses-permission android:name="moe.shizuku.permission.SHIZUKU" />
<application
android:name=".PhoneVmApp"
android:label="@string/app_name"
android:allowBackup="false"
android:theme="@style/Theme.RayNeoVm"
android:supportsRtl="true"
tools:targetApi="36">
<activity
android:name=".MainActivity"
android:exported="true"
android:configChanges="orientation|screenSize|keyboardHidden"
android:launchMode="singleTask"
android:theme="@style/Theme.RayNeoVm">
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
</activity>
<!-- Foreground service для фонового воспроизведения аудио -->
<service
android:name=".audio.AudioPlaybackService"
android:exported="false"
android:foregroundServiceType="mediaPlayback" />
</application>
</manifest>
@@ -0,0 +1,59 @@
package pw.binom.rayneovm.phone
import android.os.Bundle
import androidx.activity.ComponentActivity
import androidx.activity.compose.rememberLauncherForActivityResult
import androidx.activity.compose.setContent
import androidx.activity.enableEdgeToEdge
import androidx.activity.result.contract.ActivityResultContracts
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.darkColorScheme
import androidx.compose.runtime.CompositionLocalProvider
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import pw.binom.rayneovm.phone.mercury.MercuryController
import pw.binom.rayneovm.phone.protocol.VmPhoneController
import pw.binom.rayneovm.phone.ui.HomeScreen
/**
* Single-Activity приложения. Compose host + Navigation.
*/
class MainActivity : ComponentActivity() {
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
enableEdgeToEdge()
val controller = VmPhoneController.fromContext(this)
val mercuryController = MercuryController.fromContext(this)
setContent {
val permissionLauncher = rememberLauncherForActivityResult(
contract = ActivityResultContracts.RequestMultiplePermissions(),
) { result ->
val allGranted = result.values.all { it }
mercuryController.onPermissionsResult(allGranted)
}
CompositionLocalProvider(
VmPhoneController.Local provides controller,
MercuryController.Local provides mercuryController,
) {
MaterialTheme(colorScheme = darkColorScheme()) {
androidx.compose.foundation.layout.Box(
modifier = Modifier
.fillMaxSize()
.background(Color.Black)
) {
HomeScreen(
controller = controller,
mercuryController = mercuryController,
onRequestMercuryPermissions = {
permissionLauncher.launch(mercuryController.requiredPermissions)
},
)
}
}
}
}
}
}
@@ -0,0 +1,41 @@
package pw.binom.rayneovm.phone
import android.app.Application
import android.util.Log
import pw.binom.rayneovm.phone.mercury.MercuryController
import pw.binom.rayneovm.phone.mercury.MercuryControllerProvider
import pw.binom.rayneovm.phone.protocol.ControllerProvider
import pw.binom.rayneovm.phone.protocol.VmPhoneController
/**
* Application-класс PhoneApp.
*
* При старте подготавливает контроллер [VmPhoneController] (auto-reconnect к очкам через mDNS)
* и [MercuryController] (второй канал — RayNeo Mercury BLE).
*/
class PhoneVmApp : Application(), ControllerProvider, MercuryControllerProvider {
private val controller: VmPhoneController by lazy { VmPhoneController(this) }
private val mercuryController: MercuryController by lazy { MercuryController(this) }
override fun onCreate() {
super.onCreate()
Log.i(TAG, "PhoneVmApp onCreate, uid=${android.os.Process.myUid()}")
// Запуск отложен до первого обращения — в lazy.
controller.start()
mercuryController.refresh()
}
override fun onTerminate() {
super.onTerminate()
controller.stop()
mercuryController.close()
}
override fun provideController(): VmPhoneController = controller
override fun provideMercuryController(): MercuryController = mercuryController
companion object {
private const val TAG = "PhoneVmApp"
}
}
@@ -0,0 +1,108 @@
package pw.binom.rayneovm.phone.audio
import android.content.Context
import android.media.AudioAttributes
import android.media.AudioFormat
import android.media.AudioTrack
import android.net.wifi.WifiManager
import android.os.PowerManager
import android.util.Log
import pw.binom.mercury.transport.ByteTransport
import pw.binom.mercury.transport.Transports
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.Job
import kotlinx.coroutines.SupervisorJob
import kotlinx.coroutines.cancel
import kotlinx.coroutines.launch
/**
* Менеджер воспроизведения аудио с очков на телефон (Audio mode 2).
*
* Держит AudioTrack + WiFi/Wake lock + WebSocket-клиент, качающий PCM с очков.
*
* Вызывается из [pw.binom.rayneovm.phone.protocol.VmPhoneController] при получении
* [pw.binom.rayneovm.shared.SetAudioMode] с режимом ON_PHONE.
*/
class AudioPlaybackManager(private val context: Context) {
private val scope = CoroutineScope(SupervisorJob() + Dispatchers.IO)
private var track: AudioTrack? = null
private var transport: ByteTransport? = null
private var streamJob: Job? = null
private var wifiLock: WifiManager.WifiLock? = null
private var wakeLock: PowerManager.WakeLock? = null
fun start(glassesHost: String) {
if (track != null) return // уже запущено
val sampleRate = SAMPLE_RATE
val channelMask = AudioFormat.CHANNEL_OUT_STEREO
val encoding = AudioFormat.ENCODING_PCM_16BIT
val minBuffer = AudioTrack.getMinBufferSize(sampleRate, channelMask, encoding)
val bufferSize = minBuffer * 4
track = AudioTrack.Builder()
.setAudioAttributes(
AudioAttributes.Builder()
.setUsage(AudioAttributes.USAGE_MEDIA)
.setContentType(AudioAttributes.CONTENT_TYPE_MOVIE)
.build()
)
.setAudioFormat(
AudioFormat.Builder()
.setSampleRate(sampleRate)
.setEncoding(encoding)
.setChannelMask(channelMask)
.build()
)
.setBufferSizeInBytes(bufferSize)
.setTransferMode(AudioTrack.MODE_STREAM)
.build()
track!!.play()
Log.i(TAG, "AudioTrack started: $sampleRate Hz, buffer=$bufferSize")
// WiFi + Wake lock
val wm = context.applicationContext.getSystemService(Context.WIFI_SERVICE) as WifiManager
wifiLock = wm.createWifiLock(WifiManager.WIFI_MODE_FULL_HIGH_PERF, "VmAudioPlaybackLock").apply { acquire() }
val pm = context.applicationContext.getSystemService(Context.POWER_SERVICE) as PowerManager
wakeLock = pm.newWakeLock(PowerManager.PARTIAL_WAKE_LOCK, "VmAudioPlaybackWakeLock").apply {
acquire(60 * 60 * 1000L)
}
// Подключаемся к очкам по тому же ws://host:port/vm-audio
val wsUrl = "ws://$glassesHost:${GLASSES_PORT}/vm-audio"
transport = Transports.wifiBytes(wsUrl, autoReconnect = true)
streamJob = scope.launch {
try {
transport!!.connect(
onMessage = { pcm ->
val t = track ?: return@connect
t.write(pcm, 0, pcm.size)
},
onConnected = { Log.i(TAG, "audio connected to $wsUrl") },
onDisconnected = { Log.w(TAG, "audio disconnected") },
)
} catch (t: Throwable) {
Log.e(TAG, "audio transport failed", t)
}
}
}
fun stop() {
streamJob?.cancel(); streamJob = null
transport?.close(); transport = null
track?.stop(); track?.release(); track = null
wifiLock?.release(); wifiLock = null
wakeLock?.release(); wakeLock = null
Log.i(TAG, "AudioPlaybackManager stopped")
}
companion object {
private const val TAG = "AudioPlaybackMgr"
const val SAMPLE_RATE = 44100
const val GLASSES_PORT = 8080
}
}
@@ -0,0 +1,51 @@
package pw.binom.rayneovm.phone.audio
import android.app.Notification
import android.app.NotificationChannel
import android.app.NotificationManager
import android.app.PendingIntent
import android.app.Service
import android.content.Context
import android.content.Intent
import android.content.pm.ServiceInfo
import android.os.Build
import android.os.IBinder
import android.util.Log
import androidx.core.app.NotificationCompat
import pw.binom.rayneovm.phone.MainActivity
/**
* Stub: реальный foreground service — в [AudioPlaybackManager].
* Класс оставлен для совместимости с AndroidManifest <service> декларацией и не вызывается.
*/
class AudioPlaybackService : Service() {
override fun onBind(intent: Intent): IBinder? = null
override fun onStartCommand(intent: Intent?, flags: Int, startId: Int): Int {
startForeground(NOTIFICATION_ID, buildNotification())
return START_NOT_STICKY
}
private fun buildNotification(): Notification {
val openApp = PendingIntent.getActivity(
this, 0, Intent(this, MainActivity::class.java),
PendingIntent.FLAG_UPDATE_CURRENT or PendingIntent.FLAG_IMMUTABLE
)
return NotificationCompat.Builder(this, "vm_audio_playback")
.setSmallIcon(android.R.drawable.ic_media_play)
.setContentTitle("RayNeo VM")
.setContentText("Audio playback активен")
.setContentIntent(openApp)
.setOngoing(true)
.build()
}
override fun onDestroy() {
super.onDestroy()
}
companion object {
private const val NOTIFICATION_ID = 300
}
}
@@ -0,0 +1,271 @@
package pw.binom.rayneovm.phone.mercury
import android.Manifest
import android.bluetooth.BluetoothAdapter
import android.bluetooth.BluetoothManager
import android.content.Context
import android.content.pm.PackageManager
import android.os.Build
import androidx.compose.runtime.staticCompositionLocalOf
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.Job
import kotlinx.coroutines.SupervisorJob
import kotlinx.coroutines.cancel
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.launch
import pw.binom.mercury.DiscoveredGlasses
import pw.binom.mercury.RayNeoMercury
/**
* Контроллер второго канала связи с очками — RayNeo Mercury BLE.
*
* В отличие от [pw.binom.rayneovm.phone.protocol.VmPhoneController], который работает
* по нашему собственному WebSocket-протоколу поверх WiFi/mDNS, этот контроллер
* прикидывается штатным приложением RayNeo Mercury и говорит с очками по их
* закрытому BLE-протоколу (см. rayneo-mercury/library).
*
* Это bootstrap-канал (правило #4746): нужен, чтобы запустить наше приложение
* на очках через `appMarketLaunch(pkg)`, если mDNS не нашёл WebSocket-сервис.
*/
class MercuryController(private val context: Context) {
private val scope = CoroutineScope(SupervisorJob() + Dispatchers.IO)
private val mercury = RayNeoMercury(context, externalScope = scope)
private val _state = MutableStateFlow<MercuryState>(MercuryState.PermissionsMissing)
val state: StateFlow<MercuryState> = _state.asStateFlow()
private var lastScanResults: List<DiscoveredGlasses> = emptyList()
/** Разрешения, которые нужно запросить для работы с BLE на этом API. */
val requiredPermissions: Array<String> = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.S) {
arrayOf(
Manifest.permission.BLUETOOTH_SCAN,
Manifest.permission.BLUETOOTH_CONNECT,
)
} else {
arrayOf(Manifest.permission.ACCESS_FINE_LOCATION)
}
init {
observeEvents()
refreshPermissionState()
}
/** Перепроверить состояние Bluetooth-адаптера и разрешений. */
fun refresh() {
refreshPermissionState()
}
/** UI вызывает после системного диалога разрешений. */
fun onPermissionsResult(granted: Boolean) {
if (granted) {
if (isBluetoothEnabled()) {
_state.value = MercuryState.Disconnected()
} else {
_state.value = MercuryState.BluetoothOff
}
} else {
_state.value = MercuryState.PermissionsMissing
}
}
/**
* Сканировать ближайшие очки (RayNeo Mercury advertisement).
* По окончании — переход в [MercuryState.Disconnected] с заполненным
* `lastScanResults`, или в [MercuryState.Scanning] во время скана.
*/
fun scan() {
if (!hasPermissions()) {
_state.value = MercuryState.PermissionsMissing
return
}
if (!isBluetoothEnabled()) {
_state.value = MercuryState.BluetoothOff
return
}
_state.value = MercuryState.Scanning(emptyList())
scope.launch {
val results = mutableListOf<DiscoveredGlasses>()
try {
mercury.scan()?.collect { glasses ->
if (results.none { it.address == glasses.address }) {
results += glasses
_state.value = MercuryState.Scanning(results.toList())
}
} ?: run {
_state.value = MercuryState.Disconnected(
message = "Bluetooth-адаптер недоступен",
)
return@launch
}
// Скан завершился без явной отмены — фиксируем результат
lastScanResults = results
if (results.isEmpty()) {
_state.value = MercuryState.Disconnected(
message = "Очки не найдены. Включите их и поднесите ближе.",
)
} else {
_state.value = MercuryState.Disconnected(
message = "Найдено устройств: ${results.size}. Выберите для подключения.",
)
}
} catch (t: Throwable) {
_state.value = MercuryState.Disconnected(message = "Ошибка скана: ${t.message}")
}
}
}
/**
* Подключиться к найденному устройству. Если оно ещё не спарено — сначала
* создаём OS-level bond, потом открываем GATT-сессию.
*/
fun connect(glasses: DiscoveredGlasses) {
if (!hasPermissions()) {
_state.value = MercuryState.PermissionsMissing
return
}
_state.value = MercuryState.Connecting(glasses)
scope.launch {
try {
val paired = mercury.pair(glasses)
if (paired is pw.binom.mercury.pair.PairManager.PairingResult.Failed) {
_state.value = MercuryState.Disconnected(
message = "Сопряжение не удалось: ${paired.reason}",
)
return@launch
}
val ok = mercury.connect(glasses)
if (!ok) {
_state.value = MercuryState.Disconnected(
message = "Не удалось открыть GATT-сессию",
)
return@launch
}
_state.value = MercuryState.Connected(glasses, batteryPercent = null)
// Запрашиваем актуальный статус, чтобы получить battery/wifi
mercury.pullGeneralStatus()
} catch (t: Throwable) {
_state.value = MercuryState.Disconnected(message = "Ошибка: ${t.message}")
}
}
}
fun disconnect() {
scope.launch { mercury.disconnect() }
_state.value = MercuryState.Disconnected()
}
/**
* Запустить приложение на очках через штатный механизм Mercury
* (appMarketLaunch). Требует активной BLE-сессии — иначе вернёт false.
*/
suspend fun launchGlassesApp(packageName: String): Boolean {
if (_state.value !is MercuryState.Connected) return false
return mercury.appMarketLaunch(packageName)
}
/** Доступ к raw event-flow библиотеки (debug/диагностика). */
val events get() = mercury.events
private fun refreshPermissionState() {
_state.value = when {
!hasPermissions() -> MercuryState.PermissionsMissing
!isBluetoothEnabled() -> MercuryState.BluetoothOff
else -> if (_state.value is MercuryState.Connected) _state.value else MercuryState.Disconnected()
}
}
private fun hasPermissions(): Boolean = requiredPermissions.all {
context.checkSelfPermission(it) == PackageManager.PERMISSION_GRANTED
}
private fun isBluetoothEnabled(): Boolean {
val adapter = (context.getSystemService(Context.BLUETOOTH_SERVICE) as? BluetoothManager)?.adapter
return try {
adapter?.isEnabled == true
} catch (_: SecurityException) {
false
}
}
private fun observeEvents() {
scope.launch {
mercury.events.collect { event ->
when (event) {
is RayNeoMercury.Event.Connected -> {
val cur = _state.value
if (cur is MercuryState.Connecting) {
_state.value = MercuryState.Connected(cur.device, batteryPercent = null)
}
}
is RayNeoMercury.Event.Disconnected -> {
val cur = _state.value
if (cur is MercuryState.Connected) {
_state.value = MercuryState.Disconnected(
message = "Связь потеряна: ${event.cause}",
)
}
}
is RayNeoMercury.Event.GeneralStatusUpdate -> {
val cur = _state.value
if (cur is MercuryState.Connected) {
_state.value = cur.copy(
batteryPercent = event.status.battery,
)
}
}
else -> Unit
}
}
}
}
fun close() {
scope.coroutineContext[Job]?.cancel()
mercury.close()
}
companion object {
val Local = staticCompositionLocalOf<MercuryController> {
error("MercuryController not provided")
}
fun fromContext(context: Context): MercuryController =
(context.applicationContext as? MercuryControllerProvider)?.provideMercuryController()
?: error("Application must implement MercuryControllerProvider")
}
}
interface MercuryControllerProvider {
fun provideMercuryController(): MercuryController
}
/**
* Состояние второго канала (Mercury BLE). UI рендерит это как «Канал 2».
*/
sealed interface MercuryState {
/** Нет разрешений BLUETOOTH_SCAN / BLUETOOTH_CONNECT. */
data object PermissionsMissing : MercuryState
/** Разрешения есть, но Bluetooth выключен. */
data object BluetoothOff : MercuryState
/** Готовы к сканированию / подключению. */
data class Disconnected(val message: String? = null) : MercuryState
/** Идёт скан, [found] — то, что уже нашли. */
data class Scanning(val found: List<DiscoveredGlasses>) : MercuryState
/** Устанавливаем пару + GATT. */
data class Connecting(val device: DiscoveredGlasses) : MercuryState
/** GATT-сессия жива, очки отвечают. */
data class Connected(
val device: DiscoveredGlasses,
val batteryPercent: Int? = null,
) : MercuryState
}
@@ -0,0 +1,191 @@
package pw.binom.rayneovm.phone.protocol
import android.content.Context
import androidx.compose.runtime.staticCompositionLocalOf
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.Job
import kotlinx.coroutines.SupervisorJob
import kotlinx.coroutines.cancelChildren
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.launch
import pw.binom.mercury.transport.StringTransport
import pw.binom.mercury.transport.Transports
import pw.binom.rayneovm.phone.audio.AudioPlaybackManager
import pw.binom.rayneovm.shared.AppCrashedEvent
import pw.binom.rayneovm.shared.AppLaunchedAck
import pw.binom.rayneovm.shared.GlassesStatus
import pw.binom.rayneovm.shared.Pong
import pw.binom.rayneovm.shared.SetAudioMode
import pw.binom.rayneovm.shared.VmMessage
import pw.binom.rayneovm.shared.VmProtocol
/**
* Контроллер состояния и связи с очками на стороне телефона.
*
* Запускает фоновый цикл авто-реконнекта к очкам через mDNS + WebSocket.
* Хранит текущее состояние связи и обрабатывает входящие/исходящие сообщения.
*/
class VmPhoneController(
@Suppress("unused") private val context: Context,
) {
private val scope = CoroutineScope(SupervisorJob() + Dispatchers.IO)
private val _state = MutableStateFlow<ConnectionState>(ConnectionState.Searching)
val state: StateFlow<ConnectionState> = _state.asStateFlow()
private val _glassesStatus = MutableStateFlow<GlassesStatusInfo?>(null)
val glassesStatus: StateFlow<GlassesStatusInfo?> = _glassesStatus.asStateFlow()
private var currentTransport: StringTransport? = null
private val audioManager by lazy { AudioPlaybackManager(context) }
fun start() {
if (scope.coroutineContext[Job]?.children?.any { it.isActive } == true) return
scope.launch { reconnectLoop() }
}
fun stop() {
currentTransport?.close()
currentTransport = null
audioManager.stop()
scope.coroutineContext[Job]?.cancelChildren()
_state.value = ConnectionState.Disconnected
}
/**
* Цикл поиска очков через mDNS + подключения.
*
* При разрыве — exponential backoff.
*/
private suspend fun reconnectLoop() {
var attempt = 0
val backoffMs = listOf(1_000L, 2_000L, 4_000L, 8_000L, 16_000L, 30_000L)
while (true) {
try {
_state.value = ConnectionState.Searching
val scanner = pw.binom.mercury.transport.MdnsScanner(context, "_rayneo-vm._tcp")
// mDNS пытается 5с; если не вышло — TCP-фоллбек по подсети ещё ~5с.
// Порт 8080 (см. docs/SPEC.md:4752 и VmService.WS_PORT).
val service = scanner.discoverOnce(timeoutMs = 10_000, fallbackPort = 8080)
_state.value = ConnectionState.Connecting(service.host)
val url = "ws://${service.host}:${service.port}/vm"
val transport = Transports.wifi(url, autoReconnect = false)
currentTransport = transport
_state.value = ConnectionState.Connected(service.host)
attempt = 0
transport.connect(
onMessage = { json ->
val msg = try {
VmProtocol.decodeAny(json)
} catch (_: Throwable) { return@connect }
handleIncoming(msg)
},
onConnected = { /* уже выставлено в Connecting->Connected */ },
onDisconnected = {
currentTransport = null
_state.value = ConnectionState.Disconnected
},
)
} catch (t: Throwable) {
currentTransport = null
_state.value = ConnectionState.Disconnected
}
val delay = backoffMs[attempt.coerceAtMost(backoffMs.lastIndex)]
attempt++
kotlinx.coroutines.delay(delay)
}
}
private fun handleIncoming(msg: VmMessage) {
when (msg) {
is GlassesStatus -> {
_glassesStatus.value = GlassesStatusInfo(
batteryPercent = msg.batteryPercent,
wifiSsid = msg.wifiSsid,
currentApp = msg.currentApp,
running = msg.running,
)
}
is AppCrashedEvent -> {
_appCrashed.value = msg.packageName
}
is AppLaunchedAck -> {
_appLaunched.value = msg
}
is SetAudioMode -> {
// Аудио режим — отдельный поток, не контролируется этим контроллером напрямую.
// AudioCaptureService на очках стартует/останавливается.
// На телефоне режим управляется UI (toggle на главном экране).
}
is Pong -> { /* keepalive */ }
else -> { /* другие: событийные, форвардим */ }
}
}
private val _appCrashed = MutableStateFlow<String?>(null)
val appCrashed: StateFlow<String?> = _appCrashed.asStateFlow()
private val _appLaunched = MutableStateFlow<AppLaunchedAck?>(null)
val appLaunched: StateFlow<AppLaunchedAck?> = _appLaunched.asStateFlow()
fun clearAppCrashed() { _appCrashed.value = null }
fun clearAppLaunched() { _appLaunched.value = null }
/** Отправляет сообщение очкам. */
suspend fun send(msg: VmMessage) {
val t = currentTransport ?: return
try {
t.send(VmProtocol.encode(msg))
} catch (t: Throwable) {
// ignore — connection state управляется через reconnect loop
}
}
/**
* Переключает аудио режим: 0 = ON_GLASSES, 1 = ON_PHONE.
* Шлёт SetAudioMode очкам, и на телефоне запускает/останавливает AudioPlaybackManager.
*/
suspend fun setAudioMode(mode: pw.binom.rayneovm.shared.AudioMode) {
send(SetAudioMode(mode))
when (mode) {
pw.binom.rayneovm.shared.AudioMode.ON_GLASSES -> audioManager.stop()
pw.binom.rayneovm.shared.AudioMode.ON_PHONE -> {
val host = (currentTransport as? pw.binom.mercury.transport.WifiTransport)?.let { null }
?: (_state.value as? ConnectionState.Connected)?.host
if (host != null) audioManager.start(host)
}
}
}
companion object {
val Local = staticCompositionLocalOf<VmPhoneController> {
error("VmPhoneController not provided")
}
fun fromContext(context: Context): VmPhoneController =
(context.applicationContext as? ControllerProvider)?.provideController()
?: error("Application must implement ControllerProvider")
}
}
interface ControllerProvider {
fun provideController(): VmPhoneController
}
sealed interface ConnectionState {
data object Searching : ConnectionState
data class Connecting(val host: String) : ConnectionState
data class Connected(val host: String) : ConnectionState
data object Disconnected : ConnectionState
}
data class GlassesStatusInfo(
val batteryPercent: Int?,
val wifiSsid: String?,
val currentApp: String?,
val running: Boolean,
)
@@ -0,0 +1,105 @@
package pw.binom.rayneovm.phone.ui
import androidx.compose.foundation.background
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.material3.Card
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp
import kotlinx.coroutines.launch
import pw.binom.rayneovm.phone.protocol.VmPhoneController
import pw.binom.rayneovm.shared.LaunchAppCommand
/**
* Список приложений, которые можно запустить на очках через VirtualDisplay.
*
* Сейчас — только Zona. В будущем: реестр в shared/AppProfiles.kt.
*/
@Composable
fun AppsListScreen(
controller: VmPhoneController,
onAppClick: (String) -> Unit = {},
) {
val glasses by controller.glassesStatus.collectAsState()
val isRunning = glasses?.running == true
val coroutineScope = androidx.compose.runtime.rememberCoroutineScope()
val apps = listOf(
AppEntry("free.zona", "Zona", "Кинотеатр Zona"),
)
Column(
modifier = Modifier
.fillMaxSize()
.background(Color.Black)
.padding(24.dp),
verticalArrangement = Arrangement.spacedBy(12.dp),
) {
Text(
text = "Приложения",
color = Color.White,
fontSize = 24.sp,
fontWeight = FontWeight.Bold,
)
Text(
text = "Тапните, чтобы запустить на очках",
color = Color(0xFFB0B0B0),
fontSize = 14.sp,
)
apps.forEach { app ->
Card(
modifier = Modifier
.fillMaxWidth()
.clickable {
coroutineScope.launch {
controller.send(LaunchAppCommand(packageName = app.packageName))
}
onAppClick(app.packageName)
},
colors = androidx.compose.material3.CardDefaults.cardColors(
containerColor = Color(0xFF1A1A1A),
),
) {
Column(modifier = Modifier.padding(16.dp)) {
Text(
text = app.name,
color = Color.White,
fontSize = 18.sp,
fontWeight = FontWeight.SemiBold,
)
Text(
text = app.description,
color = Color(0xFFB0B0B0),
fontSize = 12.sp,
)
if (isRunning && glasses?.currentApp == app.packageName) {
Text(
text = "● запущено",
color = Color(0xFF4CAF50),
fontSize = 12.sp,
)
}
}
}
}
}
}
private data class AppEntry(
val packageName: String,
val name: String,
val description: String,
)
@@ -0,0 +1,263 @@
package pw.binom.rayneovm.phone.ui
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.verticalScroll
import androidx.compose.material3.Button
import androidx.compose.material3.ButtonDefaults
import androidx.compose.material3.Card
import androidx.compose.material3.CardDefaults
import androidx.compose.material3.HorizontalDivider
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.OutlinedButton
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.rememberCoroutineScope
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp
import kotlinx.coroutines.launch
import pw.binom.mercury.DiscoveredGlasses
import pw.binom.rayneovm.phone.mercury.MercuryController
import pw.binom.rayneovm.phone.mercury.MercuryState
import pw.binom.rayneovm.phone.protocol.ConnectionState
import pw.binom.rayneovm.phone.protocol.VmPhoneController
/**
* Экран «Очки»: два независимых канала связи + запуск нашего приложения.
*
* - **Канал 1 (WebSocket)** — наше приложение на очках подключается к нам по WiFi/mDNS.
* Это основной control-канал (тач, клавиши, команды).
* - **Канал 2 (Mercury BLE)** — мы прикидываемся штатным приложением RayNeo Mercury
* и общаемся с очками по их закрытому BLE-протоколу. Это bootstrap-канал:
* нужен, чтобы запустить наше приложение через `appMarketLaunch(pkg)`, когда
* очки ещё не подключены к нам по WebSocket.
*
* Кнопка «Запустить наше приложение на очках» работает только когда жив Канал 2.
*/
@Composable
fun GlassesStatusScreen(
controller: VmPhoneController,
mercuryController: MercuryController,
onRequestMercuryPermissions: () -> Unit = {},
) {
val connection by controller.state.collectAsState()
val glasses by controller.glassesStatus.collectAsState()
val mercury by mercuryController.state.collectAsState()
val scope = rememberCoroutineScope()
val mercuryConnected = mercury is MercuryState.Connected
Column(
modifier = Modifier
.fillMaxSize()
.background(Color.Black)
.verticalScroll(rememberScrollState())
.padding(24.dp),
verticalArrangement = Arrangement.spacedBy(16.dp),
) {
Text(
text = "Очки",
color = Color.White,
fontSize = 24.sp,
fontWeight = FontWeight.Bold,
)
Channel1WebSocketSection(connection = connection, glasses = glasses)
HorizontalDivider(color = Color(0xFF2A2A2A))
Channel2MercurySection(
mercury = mercury,
onRequestPermissions = onRequestMercuryPermissions,
onScan = { mercuryController.scan() },
onConnect = { mercuryController.connect(it) },
onDisconnect = { mercuryController.disconnect() },
)
Spacer(modifier = Modifier.height(8.dp))
LaunchGlassesAppButton(
enabled = mercuryConnected,
onClick = {
scope.launch {
mercuryController.launchGlassesApp(GLASSES_PACKAGE)
}
},
)
}
}
private const val GLASSES_PACKAGE = "pw.binom.rayneovm.glasses"
@Composable
private fun Channel1WebSocketSection(
connection: ConnectionState,
glasses: pw.binom.rayneovm.phone.protocol.GlassesStatusInfo?,
) {
SectionCard(title = "Канал 1 — наше приложение (WebSocket)") {
StatusRow("Связь", connection.label)
when (connection) {
is ConnectionState.Connected -> glasses?.let { g ->
StatusRow("Аккумулятор", g.batteryPercent?.let { "$it%" } ?: "—")
StatusRow("WiFi", g.wifiSsid ?: "—")
StatusRow("Запущено", g.currentApp ?: "—")
StatusRow("Активно", if (g.running) "да" else "нет")
} ?: run {
Hint("Ожидаем статус от очков…")
}
else -> Hint(
when (connection) {
is ConnectionState.Searching -> "Ищем очки в сети…"
is ConnectionState.Connecting -> "Подключаемся к ${connection.host}…"
is ConnectionState.Disconnected ->
"Нет связи. Проверьте, что очки и телефон в одной WiFi-сети."
}
)
}
}
}
@Composable
private fun Channel2MercurySection(
mercury: MercuryState,
onRequestPermissions: () -> Unit,
onScan: () -> Unit,
onConnect: (DiscoveredGlasses) -> Unit,
onDisconnect: () -> Unit,
) {
SectionCard(title = "Канал 2 — очки (Mercury BLE)") {
when (mercury) {
is MercuryState.PermissionsMissing -> {
Hint("Нужны разрешения Bluetooth (BLUETOOTH_SCAN, BLUETOOTH_CONNECT).")
Spacer(modifier = Modifier.height(8.dp))
Button(onClick = onRequestPermissions) { Text("Запросить разрешения") }
}
is MercuryState.BluetoothOff -> {
Hint("Bluetooth выключен. Включите его в системных настройках.")
Spacer(modifier = Modifier.height(8.dp))
OutlinedButton(onClick = onScan) { Text("Проверить ещё раз") }
}
is MercuryState.Disconnected -> {
StatusRow("Связь", "нет связи")
mercury.message?.let { Hint(it) }
Spacer(modifier = Modifier.height(8.dp))
Button(onClick = onScan) { Text("Сканировать") }
}
is MercuryState.Scanning -> {
StatusRow("Связь", "поиск… (найдено ${mercury.found.size})")
Spacer(modifier = Modifier.height(8.dp))
if (mercury.found.isNotEmpty()) {
Text("Найденные устройства:", color = Color(0xFFB0B0B0), fontSize = 12.sp)
mercury.found.forEach { g ->
DeviceRow(g, onClick = { onConnect(g) })
}
} else {
Hint("Идёт скан, подождите…")
}
}
is MercuryState.Connecting -> {
StatusRow("Связь", "сопрягаем ${mercury.device.name ?: mercury.device.address}…")
Hint("Если очки запрашивают PIN — подтвердите на них.")
}
is MercuryState.Connected -> {
StatusRow("Связь", "подключено (${mercury.device.name ?: mercury.device.address})")
StatusRow("Адрес", mercury.device.address)
StatusRow("Аккумулятор", mercury.batteryPercent?.let { "$it%" } ?: "—")
Spacer(modifier = Modifier.height(8.dp))
OutlinedButton(onClick = onDisconnect) { Text("Отключить") }
}
}
}
}
@Composable
private fun LaunchGlassesAppButton(enabled: Boolean, onClick: () -> Unit) {
Button(
onClick = onClick,
enabled = enabled,
modifier = Modifier.fillMaxWidth(),
colors = ButtonDefaults.buttonColors(
containerColor = Color(0xFF1565C0),
disabledContainerColor = Color(0xFF2A2A2A),
),
) {
Text(if (enabled) "Запустить наше приложение на очках" else "Запустить наше приложение на очках (нужен Канал 2)")
}
}
@Composable
private fun SectionCard(title: String, content: @Composable () -> Unit) {
Card(
modifier = Modifier.fillMaxWidth(),
colors = CardDefaults.cardColors(containerColor = Color(0xFF141414)),
) {
Column(modifier = Modifier.padding(16.dp), verticalArrangement = Arrangement.spacedBy(8.dp)) {
Text(
text = title,
color = Color(0xFF8AB4F8),
fontSize = 14.sp,
fontWeight = FontWeight.SemiBold,
)
content()
}
}
}
@Composable
private fun DeviceRow(glasses: DiscoveredGlasses, onClick: () -> Unit) {
Row(
modifier = Modifier
.fillMaxWidth()
.padding(vertical = 4.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.SpaceBetween,
) {
Column {
Text(glasses.name ?: "—", color = Color.White, fontSize = 14.sp)
Text(glasses.address, color = Color(0xFFB0B0B0), fontSize = 11.sp)
}
Button(onClick = onClick) { Text("Подключить") }
}
}
@Composable
private fun StatusRow(label: String, value: String) {
Row(
modifier = Modifier.fillMaxWidth(),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.SpaceBetween,
) {
Text(text = label, color = Color(0xFFB0B0B0), fontSize = 14.sp)
Text(text = value, color = Color.White, fontSize = 14.sp, fontWeight = FontWeight.Medium)
}
}
@Composable
private fun Hint(text: String) {
Box(modifier = Modifier.padding(top = 4.dp)) {
Text(text = text, color = Color(0xFFB0B0B0), fontSize = 13.sp)
}
}
private val ConnectionState.label: String
get() = when (this) {
is ConnectionState.Searching -> "поиск…"
is ConnectionState.Connecting -> "подключение…"
is ConnectionState.Connected -> "подключено ($host)"
is ConnectionState.Disconnected -> "нет связи"
}
@@ -0,0 +1,75 @@
package pw.binom.rayneovm.phone.ui
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.padding
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Apps
import androidx.compose.material.icons.filled.Settings
import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.NavigationBar
import androidx.compose.material3.NavigationBarItem
import androidx.compose.material3.Scaffold
import androidx.compose.material3.Text
import androidx.compose.material3.darkColorScheme
import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import pw.binom.rayneovm.phone.mercury.MercuryController
import pw.binom.rayneovm.phone.protocol.VmPhoneController
/**
* Главный экран приложения: NavigationBar с двумя табами.
*
* - «Очки» — статус связи и состояние GlassesApp
* - «Приложения» — список приложений для запуска
*/
@Composable
fun HomeScreen(
controller: VmPhoneController,
mercuryController: MercuryController,
onRequestMercuryPermissions: () -> Unit = {},
) {
var selectedTab by remember { mutableStateOf(Tab.GLASSES) }
MaterialTheme(colorScheme = darkColorScheme()) {
Scaffold(
modifier = Modifier.fillMaxSize().background(Color.Black),
containerColor = Color.Black,
bottomBar = {
NavigationBar(containerColor = Color(0xFF101010)) {
Tab.entries.forEach { tab ->
NavigationBarItem(
selected = tab == selectedTab,
onClick = { selectedTab = tab },
icon = { Icon(tab.icon, contentDescription = tab.label) },
label = { Text(tab.label) },
)
}
}
}
) { padding ->
Box(modifier = Modifier.fillMaxSize().padding(padding)) {
when (selectedTab) {
Tab.GLASSES -> GlassesStatusScreen(
controller = controller,
mercuryController = mercuryController,
onRequestMercuryPermissions = onRequestMercuryPermissions,
)
Tab.APPS -> AppsListScreen(controller = controller)
}
}
}
}
}
private enum class Tab(val label: String, val icon: androidx.compose.ui.graphics.vector.ImageVector) {
GLASSES("Очки", Icons.Default.Settings),
APPS("Приложения", Icons.Default.Apps),
}
@@ -0,0 +1,107 @@
package pw.binom.rayneovm.phone.ui
import androidx.compose.foundation.background
import androidx.compose.foundation.gestures.detectDragGestures
import androidx.compose.foundation.gestures.detectTapGestures
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.offset
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.shape.CircleShape
import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.rememberCoroutineScope
import androidx.compose.runtime.setValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.geometry.Offset
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.input.pointer.pointerInput
import androidx.compose.ui.unit.IntOffset
import androidx.compose.ui.unit.dp
import kotlinx.coroutines.launch
import pw.binom.rayneovm.phone.protocol.VmPhoneController
import pw.binom.rayneovm.shared.TouchAction
import pw.binom.rayneovm.shared.TouchEvent
import pw.binom.rayneovm.shared.TouchType
import kotlin.math.roundToInt
/**
* Виртуальный тачпад: тапы/перетаскивания → [TouchEvent] на очки.
*
* Координаты внутри тачпада нормализуются в [0, targetWidth-1] × [0, targetHeight-1].
* Точка касания визуализируется кружком.
*/
@Composable
fun TouchpadScreen(
controller: VmPhoneController,
targetWidth: Int = 640,
targetHeight: Int = 480,
modifier: Modifier = Modifier,
) {
val scope = rememberCoroutineScope()
var touchPoint by remember { mutableStateOf<Offset?>(null) }
Box(
modifier = modifier
.background(Color(0xFF101010))
.pointerInput(targetWidth, targetHeight) {
detectTapGestures(
onPress = { offset ->
val x = (offset.x / size.width * targetWidth).roundToInt().coerceIn(0, targetWidth - 1)
val y = (offset.y / size.height * targetHeight).roundToInt().coerceIn(0, targetHeight - 1)
touchPoint = offset
scope.launch {
controller.send(TouchEvent(actions = listOf(TouchAction(TouchType.DOWN, x.toFloat(), y.toFloat()))))
try { tryAwaitRelease() } finally {
touchPoint = null
}
controller.send(TouchEvent(actions = listOf(TouchAction(TouchType.UP, x.toFloat(), y.toFloat()))))
}
},
)
}
.pointerInput(targetWidth, targetHeight) {
detectDragGestures(
onDragStart = { offset ->
val x = (offset.x / size.width * targetWidth).roundToInt().coerceIn(0, targetWidth - 1)
val y = (offset.y / size.height * targetHeight).roundToInt().coerceIn(0, targetHeight - 1)
touchPoint = offset
scope.launch {
controller.send(TouchEvent(actions = listOf(TouchAction(TouchType.DOWN, x.toFloat(), y.toFloat()))))
}
},
onDragEnd = {
touchPoint?.let { last ->
val x = (last.x / size.width * targetWidth).roundToInt().coerceIn(0, targetWidth - 1)
val y = (last.y / size.height * targetHeight).roundToInt().coerceIn(0, targetHeight - 1)
scope.launch {
controller.send(TouchEvent(actions = listOf(TouchAction(TouchType.UP, x.toFloat(), y.toFloat()))))
}
}
touchPoint = null
},
onDragCancel = { touchPoint = null },
onDrag = { change, _ ->
val x = (change.position.x / size.width * targetWidth).roundToInt().coerceIn(0, targetWidth - 1)
val y = (change.position.y / size.height * targetHeight).roundToInt().coerceIn(0, targetHeight - 1)
touchPoint = change.position
scope.launch {
controller.send(TouchEvent(actions = listOf(TouchAction(TouchType.MOVE, x.toFloat(), y.toFloat()))))
}
},
)
},
) {
touchPoint?.let { pos ->
Box(
modifier = Modifier
.offset { IntOffset((pos.x - 24.dp.toPx()).toInt(), (pos.y - 24.dp.toPx()).toInt()) }
.size(48.dp)
.clip(CircleShape)
.background(Color(0xFF80CBC4).copy(alpha = 0.6f))
)
}
}
}
@@ -0,0 +1,196 @@
package pw.binom.rayneovm.phone.ui
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.width
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.ArrowBack
import androidx.compose.material.icons.filled.Keyboard
import androidx.compose.material.icons.filled.Pause
import androidx.compose.material.icons.filled.PlayArrow
import androidx.compose.material.icons.filled.SkipNext
import androidx.compose.material.icons.filled.SkipPrevious
import androidx.compose.material3.AlertDialog
import androidx.compose.material3.Button
import androidx.compose.material3.Icon
import androidx.compose.material3.IconButton
import androidx.compose.material3.OutlinedTextField
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.rememberCoroutineScope
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.text.input.ImeAction
import androidx.compose.ui.text.input.KeyboardType
import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp
import androidx.compose.foundation.text.KeyboardOptions
import kotlinx.coroutines.launch
import pw.binom.rayneovm.phone.protocol.VmPhoneController
import pw.binom.rayneovm.shared.KeyEvent as VmKeyEvent
import pw.binom.rayneovm.shared.LaunchAppCommand
/**
* Профиль Zona: тачпад + медиа-кнопки + клавиатура + назад.
*/
@Composable
fun ZonaProfile(
controller: VmPhoneController,
modifier: Modifier = Modifier,
) {
val scope = rememberCoroutineScope()
val glasses by controller.glassesStatus.collectAsState()
val appCrashed by controller.appCrashed.collectAsState()
var showKeyboard by remember { mutableStateOf(false) }
// Если приложение упало — сразу назад (если хочется — можно показать Snackbar)
androidx.compose.runtime.LaunchedEffect(appCrashed) {
if (appCrashed != null) {
// TODO: показать Snackbar и попап обратно в список
controller.clearAppCrashed()
}
}
Column(modifier = modifier.fillMaxSize().background(Color.Black)) {
// Верхняя панель: назад + название + статус
Row(
modifier = Modifier.fillMaxWidth().padding(8.dp),
verticalAlignment = Alignment.CenterVertically,
) {
IconButton(onClick = { /* TODO: navigateBack */ }) {
Icon(Icons.Default.ArrowBack, contentDescription = "Назад", tint = Color.White)
}
Spacer(modifier = Modifier.width(8.dp))
Text(
text = "Zona",
color = Color.White,
fontSize = 18.sp,
)
Spacer(modifier = Modifier.weight(1f))
if (glasses?.currentApp == "free.zona") {
Text(
text = if (glasses?.running == true) "●" else "○",
color = if (glasses?.running == true) Color(0xFF4CAF50) else Color(0xFFB0B0B0),
fontSize = 20.sp,
)
}
}
// Тачпад (занимает большую часть экрана)
TouchpadScreen(
controller = controller,
modifier = Modifier.weight(1f).fillMaxWidth().padding(8.dp),
)
// Нижняя панель: медиа-кнопки + клавиатура
Row(
modifier = Modifier.fillMaxWidth().padding(horizontal = 8.dp, vertical = 4.dp),
horizontalArrangement = Arrangement.SpaceEvenly,
verticalAlignment = Alignment.CenterVertically,
) {
IconButton(onClick = {
scope.launch { controller.send(VmKeyEvent(keyCode = 89, action = 1)) } // KEYCODE_MEDIA_REWIND
}) { Icon(Icons.Default.SkipPrevious, contentDescription = "Назад", tint = Color.White) }
IconButton(onClick = {
scope.launch { controller.send(VmKeyEvent(keyCode = 85, action = 1)) } // KEYCODE_MEDIA_PLAY_PAUSE
}) { Icon(Icons.Default.PlayArrow, contentDescription = "Play/Pause", tint = Color.White) }
IconButton(onClick = {
scope.launch { controller.send(VmKeyEvent(keyCode = 90, action = 1)) } // KEYCODE_MEDIA_FAST_FORWARD
}) { Icon(Icons.Default.SkipNext, contentDescription = "Вперёд", tint = Color.White) }
IconButton(onClick = {
scope.launch { controller.send(VmKeyEvent(keyCode = 4, action = 1)) } // KEYCODE_BACK
}) { Icon(Icons.Default.ArrowBack, contentDescription = "Назад (в приложении)", tint = Color.White) }
IconButton(onClick = { showKeyboard = true }) {
Icon(Icons.Default.Keyboard, contentDescription = "Клавиатура", tint = Color.White)
}
}
}
if (showKeyboard) {
KeyboardDialog(
onDismiss = { showKeyboard = false },
onSend = { text ->
scope.launch {
text.forEach { ch ->
val keyCode = keyCodeForChar(ch)
if (keyCode != -1) {
controller.send(VmKeyEvent(keyCode = keyCode, action = 1))
}
}
}
showKeyboard = false
},
)
}
}
@Composable
private fun KeyboardDialog(
onDismiss: () -> Unit,
onSend: (String) -> Unit,
) {
var input by remember { mutableStateOf("") }
AlertDialog(
onDismissRequest = onDismiss,
title = { Text("Ввод текста") },
text = {
OutlinedTextField(
value = input,
onValueChange = { input = it },
modifier = Modifier.fillMaxWidth(),
singleLine = true,
keyboardOptions = KeyboardOptions(
keyboardType = KeyboardType.Text,
imeAction = ImeAction.Done,
),
)
},
confirmButton = {
TextButton(onClick = { onSend(input) }) { Text("Отправить") }
},
dismissButton = {
TextButton(onClick = onDismiss) { Text("Отмена") }
},
)
}
/**
* ASCII char → Android keyCode. Простая таблица, без юникода и без спец.клавиш.
* Возвращает -1 для символов, которые нельзя передать через KEYCODE.
*/
private fun keyCodeForChar(ch: Char): Int {
return when (ch) {
in 'a'..'z' -> android.view.KeyEvent.KEYCODE_A + (ch - 'a')
in 'A'..'Z' -> android.view.KeyEvent.KEYCODE_A + (ch - 'A')
in '0'..'9' -> android.view.KeyEvent.KEYCODE_0 + (ch - '0')
' ' -> android.view.KeyEvent.KEYCODE_SPACE
'\n' -> android.view.KeyEvent.KEYCODE_ENTER
'.' -> android.view.KeyEvent.KEYCODE_PERIOD
',' -> android.view.KeyEvent.KEYCODE_COMMA
'!' -> android.view.KeyEvent.KEYCODE_1 // shift+1 — отдаём как KEYCODE_1 + shift, но тут упрощённо
'?' -> android.view.KeyEvent.KEYCODE_SLASH
else -> -1
}
}
@Composable
private fun Spacer(modifier: Modifier) {
androidx.compose.foundation.layout.Spacer(modifier = modifier)
}
@@ -0,0 +1,3 @@
<resources>
<string name="app_name">RayNeo VM</string>
</resources>
@@ -0,0 +1,7 @@
<resources>
<style name="Theme.RayNeoVm" parent="android:Theme.Material.NoActionBar">
<item name="android:statusBarColor">@android:color/black</item>
<item name="android:navigationBarColor">@android:color/black</item>
<item name="android:windowBackground">@android:color/black</item>
</style>
</resources>
+7
View File
@@ -0,0 +1,7 @@
// Top-level build file
plugins {
alias(libs.plugins.android.application) apply false
alias(libs.plugins.android.library) apply false
alias(libs.plugins.kotlin.serialization) apply false
alias(libs.plugins.kotlin.compose) apply false
}
+1369
View File
File diff suppressed because it is too large Load Diff
+10
View File
@@ -0,0 +1,10 @@
org.gradle.jvmargs=-Xmx2048m -Dfile.encoding=UTF-8
org.gradle.parallel=true
org.gradle.caching=true
android.useAndroidX=true
android.nonTransitiveRClass=true
android.defaults.buildfeatures.buildconfig=false
kotlin.code.style=official
# Enabled parallel sync for Gradle 9.4+
org.gradle.tooling.parallel=true
+12
View File
@@ -0,0 +1,12 @@
#This file is generated by updateDaemonJvm
toolchainUrl.FREE_BSD.AARCH64=https\://api.foojay.io/disco/v3.0/ids/cf726b4a1c84b50457225f9bba6d7650/redirect
toolchainUrl.FREE_BSD.X86_64=https\://api.foojay.io/disco/v3.0/ids/fa1e318c287360478e3c83a9a3ef1007/redirect
toolchainUrl.LINUX.AARCH64=https\://api.foojay.io/disco/v3.0/ids/cf726b4a1c84b50457225f9bba6d7650/redirect
toolchainUrl.LINUX.X86_64=https\://api.foojay.io/disco/v3.0/ids/fa1e318c287360478e3c83a9a3ef1007/redirect
toolchainUrl.MAC_OS.AARCH64=https\://api.foojay.io/disco/v3.0/ids/c2dd35c9d0aaf0ba6ad0791320f99dfc/redirect
toolchainUrl.MAC_OS.X86_64=https\://api.foojay.io/disco/v3.0/ids/e5810bd7fd1f8a586644409d395a7e55/redirect
toolchainUrl.UNIX.AARCH64=https\://api.foojay.io/disco/v3.0/ids/cf726b4a1c84b50457225f9bba6d7650/redirect
toolchainUrl.UNIX.X86_64=https\://api.foojay.io/disco/v3.0/ids/fa1e318c287360478e3c83a9a3ef1007/redirect
toolchainUrl.WINDOWS.AARCH64=https\://api.foojay.io/disco/v3.0/ids/7b3c4877c0749019e6805bb61e421497/redirect
toolchainUrl.WINDOWS.X86_64=https\://api.foojay.io/disco/v3.0/ids/d76df094a9cbbabd3b08251f9e61444a/redirect
toolchainVersion=25
+67
View File
@@ -0,0 +1,67 @@
[versions]
kotlin = "2.4.20"
agp = "9.4.1"
coroutines = "1.11.0"
kotlinxSerialization = "1.11.0"
ktor = "3.6.0"
lifecycle = "2.11.0"
activityCompose = "1.11.0"
composeBom = "2026.09.00"
shizukuApi = "13.1.5"
navCompose = "2.10.2"
mercury = "0.1.0-SNAPSHOT"
mercuryTransport = "0.1.0-SNAPSHOT"
mercuryBinocular = "0.1.0-SNAPSHOT"
mercuryShizuku = "0.1.0-SNAPSHOT"
[libraries]
# kotlinx
kotlinx-coroutines-android = { module = "org.jetbrains.kotlinx:kotlinx-coroutines-android", version.ref = "coroutines" }
kotlinx-coroutines-core = { module = "org.jetbrains.kotlinx:kotlinx-coroutines-core", version.ref = "coroutines" }
kotlinx-coroutines-test = { module = "org.jetbrains.kotlinx:kotlinx-coroutines-test", version.ref = "coroutines" }
kotlinx-serialization-json = { module = "org.jetbrains.kotlinx:kotlinx-serialization-json", version.ref = "kotlinxSerialization" }
# Ktor (используется в :shared для тестов и в качестве API для клиента)
ktor-client-core = { module = "io.ktor:ktor-client-core", version.ref = "ktor" }
ktor-client-okhttp = { module = "io.ktor:ktor-client-okhttp", version.ref = "ktor" }
ktor-client-websockets = { module = "io.ktor:ktor-client-websockets", version.ref = "ktor" }
ktor-client-content-negotiation = { module = "io.ktor:ktor-client-content-negotiation", version.ref = "ktor" }
ktor-serialization-kotlinx-json = { module = "io.ktor:ktor-serialization-kotlinx-json", version.ref = "ktor" }
# AndroidX
androidx-lifecycle-runtime-compose = { module = "androidx.lifecycle:lifecycle-runtime-compose", version.ref = "lifecycle" }
androidx-lifecycle-service = { module = "androidx.lifecycle:lifecycle-service", version.ref = "lifecycle" }
androidx-lifecycle-viewmodel-compose = { module = "androidx.lifecycle:lifecycle-viewmodel-compose", version.ref = "lifecycle" }
androidx-activity-compose = { module = "androidx.activity:activity-compose", version.ref = "activityCompose" }
androidx-compose-bom = { module = "androidx.compose:compose-bom", version.ref = "composeBom" }
androidx-compose-ui = { module = "androidx.compose.ui:ui" }
androidx-compose-ui-graphics = { module = "androidx.compose.ui:ui-graphics" }
androidx-compose-ui-tooling-preview = { module = "androidx.compose.ui:ui-tooling-preview" }
androidx-compose-foundation = { module = "androidx.compose.foundation:foundation" }
androidx-compose-material3 = { module = "androidx.compose.material3:material3" }
androidx-compose-material-icons = { module = "androidx.compose.material:material-icons-extended" }
androidx-navigation-compose = { module = "androidx.navigation:navigation-compose", version.ref = "navCompose" }
androidx-core-ktx = { module = "androidx.core:core-ktx", version = "1.15.0" }
# Shizuku
shizuku-api = { module = "dev.rikka.shizuku:api", version.ref = "shizukuApi" }
# rayneo-mercury (библиотеки)
mercury-library = { module = "pw.binom.mercury:rayneo-mercury", version.ref = "mercury" }
mercury-transport = { module = "pw.binom.mercury:rayneo-mercury-transport", version.ref = "mercuryTransport" }
mercury-binocular = { module = "pw.binom.mercury:rayneo-mercury-binocular-renderer", version.ref = "mercuryBinocular" }
mercury-shizuku = { module = "pw.binom.mercury:rayneo-mercury-shizuku-vd", version.ref = "mercuryShizuku" }
# Test
junit-jupiter = { module = "org.junit.jupiter:junit-jupiter", version = "5.11.4" }
junit-jupiter-engine = { module = "org.junit.jupiter:junit-jupiter-engine", version = "5.11.4" }
junit-platform-launcher = { module = "org.junit.platform:junit-platform-launcher", version = "1.11.4" }
kotlin-test = { module = "org.jetbrains.kotlin:kotlin-test", version.ref = "kotlin" }
mockk = { module = "io.mockk:mockk", version = "1.13.16" }
turbine = { module = "app.cash.turbine:turbine", version = "1.2.0" }
[plugins]
android-application = { id = "com.android.application", version.ref = "agp" }
android-library = { id = "com.android.library", version.ref = "agp" }
kotlin-serialization = { id = "org.jetbrains.kotlin.plugin.serialization", version.ref = "kotlin" }
kotlin-compose = { id = "org.jetbrains.kotlin.plugin.compose", version.ref = "kotlin" }
Binary file not shown.
+9
View File
@@ -0,0 +1,9 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-9.6.0-bin.zip
networkTimeout=10000
retries=0
retryBackOffMs=500
validateDistributionUrl=true
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
Vendored Executable
+248
View File
@@ -0,0 +1,248 @@
#!/bin/sh
#
# Copyright © 2015 the original authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier: Apache-2.0
#
##############################################################################
#
# gradlew start up script for POSIX generated by Gradle.
#
# Important for running:
#
# (1) You need a POSIX-compliant shell to run this script. If your /bin/sh is
# noncompliant, but you have some other compliant shell such as ksh or
# bash, then to run this script, type that shell name before the whole
# command line, like:
#
# ksh gradlew
#
# Busybox and similar reduced shells will NOT work, because this script
# requires all of these POSIX shell features:
# * functions;
# * expansions «$var», «${var}», «${var:-default}», «${var+SET}»,
# «${var#prefix}», «${var%suffix}», and «$( cmd )»;
# * compound commands having a testable exit status, especially «case»;
# * various built-in commands including «command», «set», and «ulimit».
#
# Important for patching:
#
# (2) This script targets any POSIX shell, so it avoids extensions provided
# by Bash, Ksh, etc; in particular arrays are avoided.
#
# The "traditional" practice of packing multiple parameters into a
# space-separated string is a well documented source of bugs and security
# problems, so this is (mostly) avoided, by progressively accumulating
# options in "$@", and eventually passing that to Java.
#
# Where the inherited environment variables (DEFAULT_JVM_OPTS, JAVA_OPTS,
# and GRADLE_OPTS) rely on word-splitting, this is performed explicitly;
# see the in-line comments for details.
#
# There are tweaks for specific operating systems such as AIX, CygWin,
# Darwin, MinGW, and NonStop.
#
# (3) This script is generated from the Groovy template
# https://github.com/gradle/gradle/blob/3d91ce3b8caaf77ad09f381f43615b715b53f72c/platforms/jvm/plugins-application/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt
# within the Gradle project.
#
# You can find Gradle at https://github.com/gradle/gradle/.
#
##############################################################################
# Attempt to set APP_HOME
# Resolve links: $0 may be a link
app_path=$0
# Need this for daisy-chained symlinks.
while
APP_HOME=${app_path%"${app_path##*/}"} # leaves a trailing /; empty if no leading path
[ -h "$app_path" ]
do
ls=$( ls -ld "$app_path" )
link=${ls#*' -> '}
case $link in #(
/*) app_path=$link ;; #(
*) app_path=$APP_HOME$link ;;
esac
done
# This is normally unused
# shellcheck disable=SC2034
APP_BASE_NAME=${0##*/}
# Discard cd standard output in case $CDPATH is set (https://github.com/gradle/gradle/issues/25036)
APP_HOME=$( cd -P "${APP_HOME:-./}" > /dev/null && printf '%s\n' "$PWD" ) || exit
# Use the maximum available, or set MAX_FD != -1 to use that value.
MAX_FD=maximum
warn () {
echo "$*"
} >&2
die () {
echo
echo "$*"
echo
exit 1
} >&2
# OS specific support (must be 'true' or 'false').
cygwin=false
msys=false
darwin=false
nonstop=false
case "$( uname )" in #(
CYGWIN* ) cygwin=true ;; #(
Darwin* ) darwin=true ;; #(
MSYS* | MINGW* ) msys=true ;; #(
NONSTOP* ) nonstop=true ;;
esac
# Determine the Java command to use to start the JVM.
if [ -n "$JAVA_HOME" ] ; then
if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
# IBM's JDK on AIX uses strange locations for the executables
JAVACMD=$JAVA_HOME/jre/sh/java
else
JAVACMD=$JAVA_HOME/bin/java
fi
if [ ! -x "$JAVACMD" ] ; then
die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
Please set the JAVA_HOME variable in your environment to match the
location of your Java installation."
fi
else
JAVACMD=java
if ! command -v java >/dev/null 2>&1
then
die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
Please set the JAVA_HOME variable in your environment to match the
location of your Java installation."
fi
fi
# Increase the maximum file descriptors if we can.
if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then
case $MAX_FD in #(
max*)
# In POSIX sh, ulimit -H is undefined. That's why the result is checked to see if it worked.
# shellcheck disable=SC2039,SC3045
MAX_FD=$( ulimit -H -n ) ||
warn "Could not query maximum file descriptor limit"
esac
case $MAX_FD in #(
'' | soft) :;; #(
*)
# In POSIX sh, ulimit -n is undefined. That's why the result is checked to see if it worked.
# shellcheck disable=SC2039,SC3045
ulimit -n "$MAX_FD" ||
warn "Could not set maximum file descriptor limit to $MAX_FD"
esac
fi
# Collect all arguments for the java command, stacking in reverse order:
# * args from the command line
# * the main class name
# * -classpath
# * -D...appname settings
# * --module-path (only if needed)
# * DEFAULT_JVM_OPTS, JAVA_OPTS, and GRADLE_OPTS environment variables.
# For Cygwin or MSYS, switch paths to Windows format before running java
if "$cygwin" || "$msys" ; then
APP_HOME=$( cygpath --path --mixed "$APP_HOME" )
JAVACMD=$( cygpath --unix "$JAVACMD" )
# Now convert the arguments - kludge to limit ourselves to /bin/sh
for arg do
if
case $arg in #(
-*) false ;; # don't mess with options #(
/?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath
[ -e "$t" ] ;; #(
*) false ;;
esac
then
arg=$( cygpath --path --ignore --mixed "$arg" )
fi
# Roll the args list around exactly as many times as the number of
# args, so each arg winds up back in the position where it started, but
# possibly modified.
#
# NB: a `for` loop captures its iteration list before it begins, so
# changing the positional parameters here affects neither the number of
# iterations, nor the values presented in `arg`.
shift # remove old arg
set -- "$@" "$arg" # push replacement arg
done
fi
# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
# Collect all arguments for the java command:
# * DEFAULT_JVM_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments,
# and any embedded shellness will be escaped.
# * For example: A user cannot expect ${Hostname} to be expanded, as it is an environment variable and will be
# treated as '${Hostname}' itself on the command line.
set -- \
"-Dorg.gradle.appname=$APP_BASE_NAME" \
-jar "$APP_HOME/gradle/wrapper/gradle-wrapper.jar" \
"$@"
# Stop when "xargs" is not available.
if ! command -v xargs >/dev/null 2>&1
then
die "xargs is not available"
fi
# Use "xargs" to parse quoted args.
#
# With -n1 it outputs one arg per line, with the quotes and backslashes removed.
#
# In Bash we could simply go:
#
# readarray ARGS < <( xargs -n1 <<<"$var" ) &&
# set -- "${ARGS[@]}" "$@"
#
# but POSIX shell has neither arrays nor command substitution, so instead we
# post-process each arg (as a line of input to sed) to backslash-escape any
# character that might be a shell metacharacter, then use eval to reverse
# that process (while maintaining the separation between arguments), and wrap
# the whole thing up as a single "set" statement.
#
# This will of course break if any of these variables contains a newline or
# an unmatched quote.
#
eval "set -- $(
printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS" |
xargs -n1 |
sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' |
tr '\n' ' '
)" '"$@"'
exec "$JAVACMD" "$@"
Vendored
+82
View File
@@ -0,0 +1,82 @@
@rem
@rem Copyright 2015 the original author or authors.
@rem
@rem Licensed under the Apache License, Version 2.0 (the "License");
@rem you may not use this file except in compliance with the License.
@rem You may obtain a copy of the License at
@rem
@rem https://www.apache.org/licenses/LICENSE-2.0
@rem
@rem Unless required by applicable law or agreed to in writing, software
@rem distributed under the License is distributed on an "AS IS" BASIS,
@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@rem See the License for the specific language governing permissions and
@rem limitations under the License.
@rem
@rem SPDX-License-Identifier: Apache-2.0
@rem
@if "%DEBUG%"=="" @echo off
@rem ##########################################################################
@rem
@rem gradlew startup script for Windows
@rem
@rem ##########################################################################
@rem Set local scope for the variables, and ensure extensions are enabled
setlocal EnableExtensions
set DIRNAME=%~dp0
if "%DIRNAME%"=="" set DIRNAME=.
@rem This is normally unused
set APP_BASE_NAME=%~n0
set APP_HOME=%DIRNAME%
@rem Resolve any "." and ".." in APP_HOME to make it shorter.
for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi
@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m"
@rem Find java.exe
if defined JAVA_HOME goto findJavaFromJavaHome
set JAVA_EXE=java.exe
%JAVA_EXE% -version >NUL 2>&1
if %ERRORLEVEL% equ 0 goto execute
echo. 1>&2
echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. 1>&2
echo. 1>&2
echo Please set the JAVA_HOME variable in your environment to match the 1>&2
echo location of your Java installation. 1>&2
"%COMSPEC%" /c exit 1
:findJavaFromJavaHome
set JAVA_HOME=%JAVA_HOME:"=%
set JAVA_EXE=%JAVA_HOME%/bin/java.exe
if exist "%JAVA_EXE%" goto execute
echo. 1>&2
echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% 1>&2
echo. 1>&2
echo Please set the JAVA_HOME variable in your environment to match the 1>&2
echo location of your Java installation. 1>&2
"%COMSPEC%" /c exit 1
:execute
@rem Setup the command line
@rem Execute gradlew
@rem endlocal doesn't take effect until after the line is parsed and variables are expanded
@rem which allows us to clear the local environment before executing the java command
endlocal & "%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -jar "%APP_HOME%\gradle\wrapper\gradle-wrapper.jar" %* & call :exitWithErrorLevel
:exitWithErrorLevel
@rem Use "%COMSPEC%" /c exit to allow operators to work properly in scripts
"%COMSPEC%" /c exit %ERRORLEVEL%
+1
View File
@@ -0,0 +1 @@
sdk.dir=/home/subochev/Android/Sdk
+35
View File
@@ -0,0 +1,35 @@
pluginManagement {
repositories {
google {
content {
includeGroupByRegex("com\\.android.*")
includeGroupByRegex("com\\.google.*")
includeGroupByRegex("androidx.*")
}
}
mavenCentral()
gradlePluginPortal()
}
}
dependencyResolutionManagement {
repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
repositories {
google()
mavenCentral()
maven {
name = "caffeine"
url = uri("http://192.168.76.117/repository/caffeine/")
isAllowInsecureProtocol = true
credentials {
username = providers.gradleProperty("nexusUser").getOrElse("admin")
password = providers.gradleProperty("nexusPassword").getOrElse("admin123")
}
}
}
}
rootProject.name = "rayneo-vm"
include(":shared")
include(":app-glasses-vm")
include(":app-phone-vm")
+44
View File
@@ -0,0 +1,44 @@
plugins {
alias(libs.plugins.android.library)
alias(libs.plugins.kotlin.serialization)
}
android {
namespace = "pw.binom.rayneovm.shared"
compileSdk = 37
defaultConfig {
minSdk = 26
}
compileOptions {
sourceCompatibility = JavaVersion.VERSION_17
targetCompatibility = JavaVersion.VERSION_17
}
testOptions {
unitTests.all {
it.useJUnitPlatform()
}
}
}
kotlin {
compilerOptions {
jvmTarget.set(org.jetbrains.kotlin.gradle.dsl.JvmTarget.JVM_17)
}
}
dependencies {
api(libs.kotlinx.coroutines.core)
api(libs.kotlinx.serialization.json)
api(libs.mercury.transport)
testImplementation(libs.kotlinx.coroutines.test)
testImplementation(libs.junit.jupiter)
testImplementation(libs.kotlin.test) // need for kotlin.test.Test
testImplementation(libs.mockk)
testImplementation(libs.turbine)
testRuntimeOnly(libs.junit.platform.launcher)
}
+2
View File
@@ -0,0 +1,2 @@
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android" />
@@ -0,0 +1,113 @@
package pw.binom.rayneovm.shared
import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable
/**
* Корневой sealed-интерфейс сообщения между телефоном и очками.
*
* Все DTO — сериализуемые в JSON. Полиморфная дискриминация по полю `type`
* (настраивается в [VmProtocol]).
*
* Потоки:
* - Phone → Glasses: [TouchEvent], [KeyEvent], [LaunchAppCommand], [SetAudioMode], [Shutdown]
* - Glasses → Phone: [GlassesStatus], [AppCrashedEvent], [AppLaunchedAck]
* - Двунаправленные: [Ping], [Pong]
*/
@Serializable
sealed interface VmMessage
// =====================================================================
// Phone → Glasses
// =====================================================================
@Serializable
@SerialName("touch")
data class TouchEvent(
val actions: List<TouchAction>,
) : VmMessage
@Serializable
data class TouchAction(
val type: TouchType,
/** X в координатах виртуального дисплея гостевого приложения (0..width-1). */
val x: Float,
/** Y в координатах виртуального дисплея гостевого приложения (0..height-1). */
val y: Float,
/** Pointer ID (для multi-touch; для простого тачпада всегда 0). */
val pointerId: Int = 0,
)
@Serializable
enum class TouchType { DOWN, MOVE, UP, CANCEL }
@Serializable
@SerialName("key")
data class KeyEvent(
/** KeyCode из android.view.KeyEvent. */
val keyCode: Int,
/** MotionEvent action: ACTION_DOWN = 0, ACTION_UP = 1. */
val action: Int,
) : VmMessage {
companion object {
const val ACTION_DOWN = 0
const val ACTION_UP = 1
}
}
@Serializable
@SerialName("launch_app")
data class LaunchAppCommand(
val packageName: String,
) : VmMessage
@Serializable
@SerialName("set_audio_mode")
data class SetAudioMode(
val mode: AudioMode,
) : VmMessage
@Serializable
enum class AudioMode { ON_GLASSES, ON_PHONE }
@Serializable
@SerialName("shutdown")
data object Shutdown : VmMessage
// =====================================================================
// Glasses → Phone
// =====================================================================
@Serializable
@SerialName("glasses_status")
data class GlassesStatus(
val batteryPercent: Int? = null,
val wifiSsid: String? = null,
val currentApp: String? = null,
val running: Boolean = false,
) : VmMessage
@Serializable
@SerialName("app_crashed")
data class AppCrashedEvent(
val packageName: String,
) : VmMessage
@Serializable
@SerialName("app_launched")
data class AppLaunchedAck(
val packageName: String,
val success: Boolean,
) : VmMessage
// =====================================================================
// Bidirectional
// =====================================================================
@Serializable
@SerialName("ping")
data object Ping : VmMessage
@Serializable
@SerialName("pong")
data object Pong : VmMessage
@@ -0,0 +1,51 @@
package pw.binom.rayneovm.shared
import kotlinx.serialization.encodeToString
import kotlinx.serialization.json.Json
/**
* Протокол общения Phone ↔ Glasses.
*
* Надстройка над JSON-сериализацией для [VmMessage].
* Не делает никакой логики доставки — только упаковка/распаковка.
*
* Используется поверх любого [pw.binom.mercury.transport.StringTransport] (WebSocket, BT).
*/
object VmProtocol {
val json: Json = Json {
ignoreUnknownKeys = true
encodeDefaults = false
classDiscriminator = "type"
}
/**
* Сериализует сообщение в JSON-строку.
*/
fun encode(message: VmMessage): String =
json.encodeToString(message)
/**
* Десериализует JSON-строку в [VmMessage].
*
* @throws VmProtocolException если JSON невалидный или тип неизвестен
*/
inline fun <reified T : VmMessage> decode(jsonString: String): T =
try {
json.decodeFromString<T>(jsonString)
} catch (t: Throwable) {
throw VmProtocolException("Cannot decode VmMessage: ${t.message}", t)
}
/**
* Десериализует JSON-строку в базовый [VmMessage] (полиморфная дискриминация).
*/
fun decodeAny(jsonString: String): VmMessage =
try {
json.decodeFromString<VmMessage>(jsonString)
} catch (t: Throwable) {
throw VmProtocolException("Cannot decode VmMessage: ${t.message}", t)
}
}
class VmProtocolException(message: String, cause: Throwable? = null) : RuntimeException(message, cause)
@@ -0,0 +1,136 @@
package pw.binom.rayneovm.shared
import org.junit.jupiter.api.Test
import kotlin.test.assertEquals
import kotlin.test.assertTrue
class VmProtocolTest {
@Test
fun `TouchEvent roundtrip`() {
val msg = TouchEvent(
actions = listOf(
TouchAction(TouchType.DOWN, 100f, 200f, pointerId = 0),
TouchAction(TouchType.MOVE, 110f, 205f, pointerId = 0),
TouchAction(TouchType.UP, 120f, 210f, pointerId = 0),
)
)
val encoded = VmProtocol.encode(msg)
val decoded = VmProtocol.decode<TouchEvent>(encoded)
assertEquals(msg, decoded)
}
@Test
fun `KeyEvent roundtrip`() {
val msg = KeyEvent(keyCode = 66, action = KeyEvent.ACTION_DOWN) // 66 = ENTER
val encoded = VmProtocol.encode(msg)
val decoded = VmProtocol.decode<KeyEvent>(encoded)
assertEquals(msg, decoded)
assertEquals(66, decoded.keyCode)
}
@Test
fun `LaunchAppCommand roundtrip`() {
val msg = LaunchAppCommand(packageName = "free.zona")
val encoded = VmProtocol.encode(msg)
val decoded = VmProtocol.decode<LaunchAppCommand>(encoded)
assertEquals(msg, decoded)
assertEquals("free.zona", decoded.packageName)
}
@Test
fun `SetAudioMode roundtrip`() {
val msg = SetAudioMode(mode = AudioMode.ON_PHONE)
val encoded = VmProtocol.encode(msg)
val decoded = VmProtocol.decode<SetAudioMode>(encoded)
assertEquals(msg, decoded)
}
@Test
fun `GlassesStatus roundtrip`() {
val msg = GlassesStatus(
batteryPercent = 87,
wifiSsid = "HomeNetwork",
currentApp = "free.zona",
running = true,
)
val encoded = VmProtocol.encode(msg)
val decoded = VmProtocol.decode<GlassesStatus>(encoded)
assertEquals(msg, decoded)
}
@Test
fun `AppCrashedEvent roundtrip`() {
val msg = AppCrashedEvent(packageName = "free.zona")
val encoded = VmProtocol.encode(msg)
val decoded = VmProtocol.decode<AppCrashedEvent>(encoded)
assertEquals(msg, decoded)
}
@Test
fun `AppLaunchedAck roundtrip`() {
val msg = AppLaunchedAck(packageName = "free.zona", success = true)
val encoded = VmProtocol.encode(msg)
val decoded = VmProtocol.decode<AppLaunchedAck>(encoded)
assertEquals(msg, decoded)
}
@Test
fun `Ping and Pong encode as data object`() {
assertEquals("""{"type":"ping"}""", VmProtocol.encode(Ping))
assertEquals("""{"type":"pong"}""", VmProtocol.encode(Pong))
}
@Test
fun `decodeAny for TouchEvent uses type discriminator`() {
val json = """{"type":"touch","actions":[{"type":"DOWN","x":10,"y":20,"pointerId":0}]}"""
val msg = VmProtocol.decodeAny(json)
assertTrue(msg is TouchEvent)
assertEquals(1, msg.actions.size)
assertEquals(TouchType.DOWN, msg.actions[0].type)
assertEquals(10f, msg.actions[0].x)
}
@Test
fun `decodeAny for LaunchAppCommand`() {
val json = """{"type":"launch_app","packageName":"free.zona"}"""
val msg = VmProtocol.decodeAny(json)
assertTrue(msg is LaunchAppCommand)
assertEquals("free.zona", msg.packageName)
}
@Test
fun `decodeAny for AppCrashedEvent`() {
val json = """{"type":"app_crashed","packageName":"free.zona"}"""
val msg = VmProtocol.decodeAny(json)
assertTrue(msg is AppCrashedEvent)
assertEquals("free.zona", msg.packageName)
}
@Test
fun `Shutdown data object roundtrip`() {
val encoded = VmProtocol.encode(Shutdown)
val decoded = VmProtocol.decodeAny(encoded)
assertEquals(Shutdown, decoded)
}
@Test
fun `SetAudioMode ON_GLASSES roundtrip`() {
val msg = SetAudioMode(mode = AudioMode.ON_GLASSES)
val encoded = VmProtocol.encode(msg)
val decoded = VmProtocol.decodeAny(encoded)
assertEquals(msg, decoded)
assertEquals(AudioMode.ON_GLASSES, (decoded as SetAudioMode).mode)
}
@Test
fun `GlassesStatus without fields default to nulls and false`() {
val msg = GlassesStatus()
val encoded = VmProtocol.encode(msg)
val decoded = VmProtocol.decode<GlassesStatus>(encoded)
assertEquals(null, decoded.batteryPercent)
assertEquals(null, decoded.wifiSsid)
assertEquals(null, decoded.currentApp)
assertEquals(false, decoded.running)
}
}