feat: бинокулярный видеорендер в app-glasses + иммерсивный режим + тест-план

- render/: Grafika-связка (EglCore/WindowSurface/FullFrameRect/Texture2dProgram/GlUtil/Drawable2d/EglSurfaceBase) — перенос из pair-view
- ui/BinocularVideoPlayer: один SimpleExoPlayer → SurfaceTexture → EGL-блит в оба глаза (EglVideoRenderer шарится между LEFT/RIGHT)
- ui/ViewUtils: Binocular/BinocularScreenSide/BinocularScreenSize (Compose, из pair-view)
- ui/GlassesMovieScreen: «нос к носу», calculateVideoSizeFull, FLAG_KEEP_SCREEN_ON
- MovieController: PlayVideo/MediaCommand (PLAY/PAUSE/TOGGLE/SEEK/CLOSE), доступ к ExoPlayer только с main-потока
- HostConnection: PlayVideo→плеер, реальная позиция в PlaybackPosition (withContext Main)
- MainActivity: иммерсивный режим (setDecorFitsSystemWindows(false) + hide systemBars)
- TESTING.md: 10 тест-кейсов human-style; QA-прогон (opencode, 88.19) — 10/10 ПРОЙДЕН
This commit is contained in:
Hermes Agent
2026-08-19 04:15:55 +03:00
parent 298d8e0b69
commit 4458678698
16 changed files with 2148 additions and 7 deletions
+189
View File
@@ -0,0 +1,189 @@
# TESTING.md — view-mate, app-glasses (очки RayNeo X2)
Тест-план для стенда (88.19): AVD `glasses35` (emulator-5554, 1280×480), app-host на :8080
(консоль FIFO `/tmp/host.in`, лог `/tmp/host.log`), http.server :8081 с `test-video.mp4`,
APK `pw.binom.viewmate.glasses` установлен.
Кейсы написаны человеческим языком, без координат. Координаты/точные значения агент
добывает сам (скрины, logcat, uiautomator). «Консоль хоста» = FIFO `/tmp/host.in`
(команды: `play <url>`, `mode <MODE>`, `cmd <CMD> [ms]`, `catalog`, `watch`, `mirror`).
## Общие предусловия
- Стенд поднят: app-host слушает :8080, http.server :8081, AVD загружен
- Приложение очков установлено (последняя сборка) и запущено
- Logcat очищен перед кейсом (`adb logcat -c`)
---
## TC-01 Подключение очков к серверу
**Предусловия:** app-host запущен и слушает :8080.
**Шаги:**
1. Запусти приложение очков (launch `pw.binom.viewmate.glasses/.MainActivity`)
2. Подожди 5–10 секунд
3. Посмотри на экран очков (скрин)
4. Загляни в лог приложения (logcat) и в лог хоста
**Ожидаемо:**
- На экране статус «подключено к телефону» (ConnectionScreen, без системных панелей)
- В logcat: «соединение установлено», «Welcome: режим MOVIE...»
- В `/tmp/host.log` приходят периодические `glasses статус: батарея 80%...`
- Крашей и ошибок нет
---
## TC-02 Воспроизведение тестового видео (play по прямому URL)
**Предусловия:** TC-01 пройден.
**Шаги:**
1. Через консоль хоста пошли: `play http://10.0.2.2:8081/test-video.mp4`
2. Подожди 5–10 секунд
3. Сними скрин экрана очков
4. Проверь logcat и лог хоста
**Ожидаемо:**
- В обеих половинах экрана (левой и правой) — **одинаковое** видеоизображение
(движущийся тестовый паттерн: цветные полосы, сетка)
- В logcat: «получено видео», «SimpleExoPlayer создан», «EglVideoRenderer: GL-стэйт создан»
- Крашей нет; процесс жив
---
## TC-03 Воспроизведение видео из живого Jellyfin
**Предусловия:** TC-01 пройден. Живой стрим-URL (например, фильм «Акира»:
`https://jellyfin.binom.pw/Videos/a3ca9fbf926fe9f60448f591ea945ac8/stream?static=true&MediaSourceId=a3ca9fbf926fe9f60448f591ea945ac8&api_key=59e6380a96a94364ae25d2b7cf9f1359`).
**Шаги:**
1. Через консоль хоста пошли: `play <jellyfin-url>`
2. Подожди 10–15 секунд (стрим с сетевой задержкой)
3. Сними скрин экрана очков
4. Проверь logcat: ошибок загрузки быть не должно (если была ошибка — проверь, что приложение не упало)
**Ожидаемо:**
- На экране кадр фильма (не чёрный экран), одинаковый в обеих половинах
- Приложение живо, крашей нет
- В логе хоста позиция растёт
---
## TC-04 Полноэкранный режим (без системных панелей)
**Предусловия:** приложение запущено (любой экран: статус или кино).
**Шаги:**
1. Посмотри на экран очков (скрин) — и на статус-экране, и во время кино
2. Убедись, что сверху нет системной панели (часы, зарядка, иконки)
3. Убедись, что снизу нет навигационной панели
**Ожидаемо:**
- Системные панели скрыты: приложение занимает весь экран 1280×480
- Даже после паузы/воспроизведения панели не появляются сами
- ⚠️ При первом входе в иммерсивный режим система может показать разовый оверлей
«Viewing full screen / To exit, swipe down... [Got it]» — это НЕ баг приложения,
это системная подсказка. Нажми «Got it» (или свайпни) и проверь, что панели скрыты.
---
## TC-05 Пауза и продолжение (media PAUSE / PLAY)
**Предусловия:** TC-02 пройден (видео играет).
**Шаги:**
1. Через консоль хоста пошли: `cmd PAUSE`
2. Подожди 3–5 секунд
3. Проверь лог хоста (позиция) и сними скрин
4. Через консоль пошли: `cmd PLAY`
5. Подожди 3–5 секунд, снова проверь лог хоста
**Ожидаемо:**
- После PAUSE: в логе хоста `позиция: N мс, пауза` (playing=false), позиция не растёт
- После PLAY: позиция снова растёт (playing=true)
- Кадр на экране не меняется на паузе (стоп-кадр)
---
## TC-06 Перемотка вперёд (media SEEK_FWD)
**Предусловия:** TC-02 пройден.
**Шаги:**
1. Запомни текущую позицию из лога хоста
2. Через консоль пошли: `cmd SEEK_FORWARD 10000`
3. Подожди 3–5 секунд
4. Проверь позицию в логе хоста
**Ожидаемо:**
- Позиция выросла примерно на 10 секунд относительно запомненной
- Видео продолжает играть, крашей нет
---
## TC-07 Закрытие фильма (media CLOSE)
**Предусловия:** TC-02 пройден.
**Шаги:**
1. Через консоль пошли: `cmd CLOSE`
2. Подожди 3–5 секунд
3. Посмотри на экран очков (скрин)
4. Проверь лог хоста (позиция)
**Ожидаемо:**
- Экран вернулся на статус-экран (ConnectionScreen)
- Позиция сброшена (0 мс / нет плеера)
- В logcat: «CLOSE: плеер остановлен и освобождён» (или эквивалент)
- Приложение живо
---
## TC-08 Ошибка загрузки (битый URL) — приложение не падает
**Предусловия:** TC-01 пройден.
**Шаги:**
1. Через консоль хоста пошли: `play http://10.0.2.2:8081/nonexistent.mp4`
2. Подожди 10–15 секунд
3. Проверь logcat и живость процесса
4. Сними скрин
**Ожидаемо:**
- В logcat ошибка загрузки (ExoPlaybackException / 404), НО:
- Приложение живо (pid существует), краша нет
- Экран не «завис» — можно вернуться в рабочее состояние командой `play <валидный url>`
---
## TC-09 Реконнект при обрыве связи
**Предусловия:** TC-01 пройден.
**Шаги:**
1. Останови app-host (убить процесс) — очки потеряют сервер
2. Подожди 10–15 секунд
3. Проверь logcat (сообщение о разрыве/реконнекте)
4. Запусти app-host заново
5. Подожди 10–15 секунд
6. Проверь logcat и экран
**Ожидаемо:**
- При обрыве: в logcat сообщение о потере соединения, приложение живо
- После подъёма хоста: «соединение установлено» снова, Welcome получен
- Повторный `play` работает
---
## TC-10 Статус очков уходит на хост
**Предусловия:** TC-01 пройден, видео играет.
**Шаги:**
1. Подожди 10–15 секунд
2. Загляни в `/tmp/host.log`
**Ожидаемо:**
- В логе периодически (раз в ~5 секунд): `glasses статус: батарея 80%, память 10.5/32.0 ГБ`
- И `glasses позиция: NNN мс, играет/пауза` с реально растущей позицией во время воспроизведения
@@ -17,10 +17,14 @@ class GlassesApp : Application() {
lateinit var hostConnection: HostConnection lateinit var hostConnection: HostConnection
private set private set
lateinit var movieController: MovieController
private set
override fun onCreate() { override fun onCreate() {
super.onCreate() super.onCreate()
instance = this instance = this
hostConnection = HostConnection(GlassesConfig.HOST_URL) movieController = MovieController(applicationContext)
hostConnection = HostConnection(GlassesConfig.HOST_URL, movieController)
hostConnection.start() hostConnection.start()
log("app", "HostConnection стартует: ${GlassesConfig.HOST_URL}") log("app", "HostConnection стартует: ${GlassesConfig.HOST_URL}")
} }
@@ -7,6 +7,7 @@ import kotlinx.coroutines.cancel
import kotlinx.coroutines.delay import kotlinx.coroutines.delay
import kotlinx.coroutines.isActive import kotlinx.coroutines.isActive
import kotlinx.coroutines.launch import kotlinx.coroutines.launch
import kotlinx.coroutines.withContext
import kotlinx.coroutines.flow.MutableStateFlow import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow import kotlinx.coroutines.flow.asStateFlow
@@ -26,10 +27,11 @@ import pw.binom.viewmate.core.protocol.Welcome
/** /**
* Обвязка над [GlassesWsClient] для UI очков: * Обвязка над [GlassesWsClient] для UI очков:
* реконнект, Hello при подключении, периодический статус/позиция, * реконнект, Hello при подключении, периодический статус/позиция,
* обработка входящих HostToGlasses в StateFlow (пока — лог). * обработка входящих HostToGlasses (PlayVideo/MediaCommand выполняются плеером).
*/ */
class HostConnection( class HostConnection(
private val url: String, private val url: String,
private val movieController: MovieController,
) { ) {
private val client = GlassesWsClient(url) private val client = GlassesWsClient(url)
private val scope = CoroutineScope(SupervisorJob() + Dispatchers.IO) private val scope = CoroutineScope(SupervisorJob() + Dispatchers.IO)
@@ -62,9 +64,10 @@ class HostConnection(
scope.launch { scope.launch {
while (isActive) { while (isActive) {
delay(5_000) delay(5_000)
// статус/позиция — заглушки (реальные батарея/память/таймкод — позже) // статус/позиция — заглушки (реальные батарея/память — позже)
client.send(GlassesStatus(batteryPercent = 80, storageUsedGb = 10.5, storageTotalGb = 32.0)) client.send(GlassesStatus(batteryPercent = 80, storageUsedGb = 10.5, storageTotalGb = 32.0))
client.send(PlaybackPosition(positionMs = 0, playing = false)) val (positionMs, playing) = withContext(Dispatchers.Main) { movieController.playbackPosition() }
client.send(PlaybackPosition(positionMs = positionMs, playing = playing))
} }
} }
} }
@@ -78,9 +81,15 @@ class HostConnection(
is SetMode -> addMessage("SetMode: ${msg.mode}") is SetMode -> addMessage("SetMode: ${msg.mode}")
is PlayVideo -> addMessage("получено видео: «${msg.title}» — ${msg.videoUrl}") is PlayVideo -> {
addMessage("получено видео: «${msg.title}» — ${msg.videoUrl}")
movieController.playVideo(msg)
}
is MediaCommandMsg -> addMessage("MediaCommand: ${msg.command} (seekMs=${msg.seekMs})") is MediaCommandMsg -> {
addMessage("MediaCommand: ${msg.command} (seekMs=${msg.seekMs})")
movieController.applyMediaCommand(msg.command, msg.seekMs)
}
is AssistantStateMsg -> addMessage("AssistantState: ${msg.state} (${msg.recordingSeconds}с)") is AssistantStateMsg -> addMessage("AssistantState: ${msg.state} (${msg.recordingSeconds}с)")
@@ -3,6 +3,9 @@ package pw.binom.viewmate.glasses
import android.os.Bundle import android.os.Bundle
import androidx.activity.ComponentActivity import androidx.activity.ComponentActivity
import androidx.activity.compose.setContent import androidx.activity.compose.setContent
import androidx.core.view.WindowCompat
import androidx.core.view.WindowInsetsCompat
import androidx.core.view.WindowInsetsControllerCompat
import androidx.compose.foundation.layout.Arrangement import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.fillMaxSize import androidx.compose.foundation.layout.fillMaxSize
@@ -17,13 +20,40 @@ import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue import androidx.compose.runtime.getValue
import androidx.compose.ui.Modifier import androidx.compose.ui.Modifier
import androidx.compose.ui.unit.dp import androidx.compose.ui.unit.dp
import pw.binom.viewmate.glasses.ui.GlassesMovieScreen
class MainActivity : ComponentActivity() { class MainActivity : ComponentActivity() {
override fun onCreate(savedInstanceState: Bundle?) { override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState) super.onCreate(savedInstanceState)
WindowCompat.setDecorFitsSystemWindows(window, false)
setContent { setContent {
MaterialTheme { MaterialTheme {
ConnectionScreen() val controller = GlassesApp.instance.movieController
val movieActive by controller.movieActive.collectAsState()
val title by controller.title.collectAsState()
val player = controller.player
if (movieActive && player != null) {
GlassesMovieScreen(player = player, title = title)
} else {
ConnectionScreen()
}
}
}
hideSystemBars()
log("ui", "иммерсивный режим включён")
}
override fun onResume() {
super.onResume()
hideSystemBars()
}
private fun hideSystemBars() {
window.decorView.post {
WindowInsetsControllerCompat(window, window.decorView).apply {
hide(WindowInsetsCompat.Type.systemBars())
systemBarsBehavior =
WindowInsetsControllerCompat.BEHAVIOR_SHOW_TRANSIENT_BARS_BY_SWIPE
} }
} }
} }
@@ -0,0 +1,102 @@
package pw.binom.viewmate.glasses
import android.content.Context
import android.os.Handler
import android.os.Looper
import com.google.android.exoplayer2.MediaItem
import com.google.android.exoplayer2.SimpleExoPlayer
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import pw.binom.viewmate.core.MediaCommand
import pw.binom.viewmate.core.protocol.PlayVideo
/**
* Держатель плеера фильма: создаёт/освобождает SimpleExoPlayer,
* выполняет команды из WS-протокола. Сам плеер создаётся только по PlayVideo.
*/
class MovieController(private val context: Context) {
private val _movieActive = MutableStateFlow(false)
val movieActive: StateFlow<Boolean> = _movieActive.asStateFlow()
private val _title = MutableStateFlow<String?>(null)
val title: StateFlow<String?> = _title.asStateFlow()
private var _player: SimpleExoPlayer? = null
val player: SimpleExoPlayer? get() = _player
private val mainHandler = Handler(Looper.getMainLooper())
fun playVideo(msg: PlayVideo) {
mainHandler.post {
val player = _player ?: SimpleExoPlayer.Builder(context).build().also {
_player = it
log("glasses", "SimpleExoPlayer создан")
}
_title.value = msg.title
log("glasses", "PlayVideo: «${msg.title}» — ${msg.videoUrl}")
player.setMediaItem(MediaItem.fromUri(msg.videoUrl))
player.prepare()
player.seekTo(msg.startPositionMs)
player.playWhenReady = true
_movieActive.value = true
}
}
fun applyMediaCommand(command: MediaCommand, seekMs: Long) {
mainHandler.post {
val player = _player ?: run {
log("glasses", "MediaCommand $command проигнорирован: плеер не активен")
return@post
}
when (command) {
MediaCommand.PLAY -> player.play()
MediaCommand.PAUSE -> player.pause()
MediaCommand.TOGGLE -> {
if (player.isPlaying) player.pause() else player.play()
}
MediaCommand.SEEK_FORWARD -> {
val target = player.currentPosition + seekMs
player.seekTo(target)
}
MediaCommand.SEEK_BACKWARD -> {
val target = (player.currentPosition - seekMs).coerceAtLeast(0L)
player.seekTo(target)
}
// Звук — на телефоне, переключение аудио-дорожек очкам не нужно.
MediaCommand.NEXT_AUDIO, MediaCommand.PREV_AUDIO -> {
log("glasses", "MediaCommand $command игнорируется (звук на телефоне)")
}
MediaCommand.CLOSE -> close()
}
}
}
fun close() {
mainHandler.post {
val player = _player
_player = null
if (player != null) {
log("glasses", "CLOSE: плеер остановлен и освобождён")
player.playWhenReady = false
player.stop()
player.release()
}
_movieActive.value = false
_title.value = null
}
}
/** Читать ТОЛЬКО с главного потока. */
fun playbackPosition(): Pair<Long, Boolean> {
val player = _player ?: return 0L to false
return player.currentPosition to player.isPlaying
}
}
@@ -0,0 +1,180 @@
package pw.binom.viewmate.glasses.render
import java.nio.FloatBuffer
class Drawable2d(shape: Prefab) {
companion object {
private const val SIZEOF_FLOAT: Int = 4
/**
* Simple equilateral triangle (1.0 per side). Centered on (0,0).
*/
private val TRIANGLE_COORDS: FloatArray = floatArrayOf(
0.0f, 0.577350269f, // 0 top
-0.5f, -0.288675135f, // 1 bottom left
0.5f, -0.288675135f // 2 bottom right
)
private val TRIANGLE_TEX_COORDS: FloatArray = floatArrayOf(
0.5f, 0.0f, // 0 top center
0.0f, 1.0f, // 1 bottom left
1.0f, 1.0f, // 2 bottom right
)
private val TRIANGLE_BUF: FloatBuffer = GlUtil.createFloatBuffer(TRIANGLE_COORDS)
private val TRIANGLE_TEX_BUF = GlUtil.createFloatBuffer(TRIANGLE_TEX_COORDS)
/**
* Simple square, specified as a triangle strip. The square is centered on (0,0) and has
* a size of 1x1.
*
* Triangles are 0-1-2 and 2-1-3 (counter-clockwise winding).
*/
private val RECTANGLE_COORDS = floatArrayOf(
-0.5f, -0.5f, // 0 bottom left
0.5f, -0.5f, // 1 bottom right
-0.5f, 0.5f, // 2 top left
0.5f, 0.5f, // 3 top right
)
private val RECTANGLE_TEX_COORDS = floatArrayOf(
0.0f, 1.0f, // 0 bottom left
1.0f, 1.0f, // 1 bottom right
0.0f, 0.0f, // 2 top left
1.0f, 0.0f // 3 top right
)
private val RECTANGLE_BUF = GlUtil.createFloatBuffer(RECTANGLE_COORDS)
private val RECTANGLE_TEX_BUF = GlUtil.createFloatBuffer(RECTANGLE_TEX_COORDS)
/**
* A "full" square, extending from -1 to +1 in both dimensions. When the model/view/projection
* matrix is identity, this will exactly cover the viewport.
*
* The texture coordinates are Y-inverted relative to RECTANGLE. (This seems to work out
* right with external textures from SurfaceTexture.)
*/
private val FULL_RECTANGLE_COORDS = floatArrayOf(
-1.0f, -1.0f, // 0 bottom left
1.0f, -1.0f, // 1 bottom right
-1.0f, 1.0f, // 2 top left
1.0f, 1.0f, // 3 top right
)
private val FULL_RECTANGLE_TEX_COORDS = floatArrayOf(
0.0f, 0.0f, // 0 bottom left
1.0f, 0.0f, // 1 bottom right
0.0f, 1.0f, // 2 top left
1.0f, 1.0f // 3 top right
)
private val FULL_RECTANGLE_BUF = GlUtil.createFloatBuffer(FULL_RECTANGLE_COORDS)
private val FULL_RECTANGLE_TEX_BUF = GlUtil.createFloatBuffer(FULL_RECTANGLE_TEX_COORDS)
}
val mVertexArray: FloatBuffer
val mTexCoordArray: FloatBuffer
private var mPrefab: Prefab? = null
val mVertexCount: Int
val mCoordsPerVertex: Int
val mVertexStride: Int
val mTexCoordStride: Int
/**
* Enum values for constructor.
*/
enum class Prefab {
TRIANGLE, RECTANGLE, FULL_RECTANGLE
}
/**
* Prepares a drawable from a "pre-fabricated" shape definition.
*
* Does no EGL/GL operations, so this can be done at any time.
*/
init {
when (shape) {
Prefab.TRIANGLE -> {
mVertexArray = TRIANGLE_BUF
mTexCoordArray = TRIANGLE_TEX_BUF
mCoordsPerVertex = 2
mVertexStride = mCoordsPerVertex * SIZEOF_FLOAT
mVertexCount = TRIANGLE_COORDS.size / mCoordsPerVertex
}
Prefab.RECTANGLE -> {
mVertexArray = RECTANGLE_BUF
mTexCoordArray = RECTANGLE_TEX_BUF
mCoordsPerVertex = 2
mVertexStride = mCoordsPerVertex * SIZEOF_FLOAT
mVertexCount = RECTANGLE_COORDS.size / mCoordsPerVertex
}
Prefab.FULL_RECTANGLE -> {
mVertexArray = FULL_RECTANGLE_BUF
mTexCoordArray = FULL_RECTANGLE_TEX_BUF
mCoordsPerVertex = 2
mVertexStride = mCoordsPerVertex * SIZEOF_FLOAT
mVertexCount = FULL_RECTANGLE_COORDS.size / mCoordsPerVertex
}
else -> throw RuntimeException("Unknown shape $shape")
}
mTexCoordStride = 2 * SIZEOF_FLOAT
mPrefab = shape
}
/**
* Returns the array of vertices.
*
* To avoid allocations, this returns internal state. The caller must not modify it.
*/
fun getVertexArray() = mVertexArray
/**
* Returns the array of texture coordinates.
*
* To avoid allocations, this returns internal state. The caller must not modify it.
*/
fun getTexCoordArray() = mTexCoordArray
/**
* Returns the number of vertices stored in the vertex array.
*/
fun getVertexCount(): Int {
return mVertexCount
}
/**
* Returns the width, in bytes, of the data for each vertex.
*/
fun getVertexStride(): Int {
return mVertexStride
}
/**
* Returns the width, in bytes, of the data for each texture coordinate.
*/
fun getTexCoordStride(): Int {
return mTexCoordStride
}
/**
* Returns the number of position coordinates per vertex. This will be 2 or 3.
*/
fun getCoordsPerVertex(): Int {
return mCoordsPerVertex
}
override fun toString(): String {
return if (mPrefab != null) {
"[Drawable2d: $mPrefab]"
} else {
"[Drawable2d: ...]"
}
}
}
@@ -0,0 +1,348 @@
package pw.binom.viewmate.glasses.render
import android.graphics.SurfaceTexture
import android.opengl.EGL14
import android.opengl.EGLConfig
import android.opengl.EGLContext
import android.opengl.EGLDisplay
import android.opengl.EGLExt
import android.opengl.EGLSurface
import android.view.Surface
import pw.binom.viewmate.glasses.log
class EglCore {
companion object {
const val TAG = "GlUtil"
/**
* Constructor flag: surface must be recordable. This discourages EGL from using a
* pixel format that cannot be converted efficiently to something usable by the video
* encoder.
*/
const val FLAG_RECORDABLE: Int = 0x01
/**
* Constructor flag: ask for GLES3, fall back to GLES2 if not available. Without this
* flag, GLES2 is used.
*/
const val FLAG_TRY_GLES3: Int = 0x02
// Android-specific extension.
const val EGL_RECORDABLE_ANDROID: Int = 0x3142
}
private var mEGLDisplay: EGLDisplay? = EGL14.EGL_NO_DISPLAY
private var mEGLContext: EGLContext = EGL14.EGL_NO_CONTEXT
private var mEGLConfig: EGLConfig? = null
private var mGlVersion = -1
/**
* Prepares EGL display and context.
*
* Equivalent to EglCore(null, 0).
*/
constructor() : this(null, 0) {
}
/**
* Prepares EGL display and context.
*
* @param sharedContext The context to share, or null if sharing is not desired.
* @param flags Configuration bit flags, e.g. FLAG_RECORDABLE.
*/
constructor(sharedContext: EGLContext?, flags: Int) {
var sharedContext: EGLContext? = sharedContext
if (mEGLDisplay !== EGL14.EGL_NO_DISPLAY) {
throw RuntimeException("EGL already set up")
}
if (sharedContext == null) {
sharedContext = EGL14.EGL_NO_CONTEXT
}
mEGLDisplay = EGL14.eglGetDisplay(EGL14.EGL_DEFAULT_DISPLAY)
if (mEGLDisplay === EGL14.EGL_NO_DISPLAY) {
throw RuntimeException("unable to get EGL14 display")
}
val version = IntArray(2)
if (!EGL14.eglInitialize(mEGLDisplay, version, 0, version, 1)) {
mEGLDisplay = null
throw RuntimeException("unable to initialize EGL14")
}
// Try to get a GLES3 context, if requested.
if ((flags and FLAG_TRY_GLES3) != 0) {
val config: EGLConfig? = getConfig(flags, 3)
if (config != null) {
val attrib3_list = intArrayOf(
EGL14.EGL_CONTEXT_CLIENT_VERSION, 3,
EGL14.EGL_NONE
)
val context: EGLContext = EGL14.eglCreateContext(
mEGLDisplay, config, sharedContext,
attrib3_list, 0
)
if (EGL14.eglGetError() == EGL14.EGL_SUCCESS) {
mEGLConfig = config
mEGLContext = context
mGlVersion = 3
}
}
}
if (mEGLContext === EGL14.EGL_NO_CONTEXT) { // GLES 2 only, or GLES 3 attempt failed
val config: EGLConfig = getConfig(flags, 2)
?: throw RuntimeException("Unable to find a suitable EGLConfig")
val attrib2_list = intArrayOf(
EGL14.EGL_CONTEXT_CLIENT_VERSION, 2,
EGL14.EGL_NONE
)
val context: EGLContext = EGL14.eglCreateContext(
mEGLDisplay, config, sharedContext,
attrib2_list, 0
)
checkEglError("eglCreateContext")
mEGLConfig = config
mEGLContext = context
mGlVersion = 2
}
// Confirm with query.
val values = IntArray(1)
EGL14.eglQueryContext(
mEGLDisplay, mEGLContext, EGL14.EGL_CONTEXT_CLIENT_VERSION,
values, 0
)
log(TAG, "EGLContext created, client version " + values[0])
}
/**
* Finds a suitable EGLConfig.
*
* @param flags Bit flags from constructor.
* @param version Must be 2 or 3.
*/
private fun getConfig(flags: Int, version: Int): EGLConfig? {
var renderableType = EGL14.EGL_OPENGL_ES2_BIT
if (version >= 3) {
renderableType = renderableType or EGLExt.EGL_OPENGL_ES3_BIT_KHR
}
// The actual surface is generally RGBA or RGBX, so situationally omitting alpha
// doesn't really help. It can also lead to a huge performance hit on glReadPixels()
// when reading into a GL_RGBA buffer.
val attribList = intArrayOf(
EGL14.EGL_RED_SIZE, 8,
EGL14.EGL_GREEN_SIZE, 8,
EGL14.EGL_BLUE_SIZE, 8,
EGL14.EGL_ALPHA_SIZE, 8, //EGL14.EGL_DEPTH_SIZE, 16,
//EGL14.EGL_STENCIL_SIZE, 8,
EGL14.EGL_RENDERABLE_TYPE, renderableType,
EGL14.EGL_NONE, 0, // placeholder for recordable [@-3]
EGL14.EGL_NONE
)
if ((flags and FLAG_RECORDABLE) != 0) {
attribList[attribList.size - 3] = EGL_RECORDABLE_ANDROID
attribList[attribList.size - 2] = 1
}
val configs: Array<EGLConfig?> = arrayOfNulls<EGLConfig>(1)
val numConfigs = IntArray(1)
if (!EGL14.eglChooseConfig(
mEGLDisplay, attribList, 0, configs, 0, configs.size,
numConfigs, 0
)
) {
log(TAG, "unable to find RGB8888 / $version EGLConfig")
return null
}
return configs[0]
}
/**
* Discards all resources held by this class, notably the EGL context. This must be
* called from the thread where the context was created.
*
* On completion, no context will be current.
*/
fun release() {
if (mEGLDisplay !== EGL14.EGL_NO_DISPLAY) {
// Android is unusual in that it uses a reference-counted EGLDisplay. So for
// every eglInitialize() we need an eglTerminate().
EGL14.eglMakeCurrent(
mEGLDisplay, EGL14.EGL_NO_SURFACE, EGL14.EGL_NO_SURFACE,
EGL14.EGL_NO_CONTEXT
)
EGL14.eglDestroyContext(mEGLDisplay, mEGLContext)
EGL14.eglReleaseThread()
EGL14.eglTerminate(mEGLDisplay)
}
mEGLDisplay = EGL14.EGL_NO_DISPLAY
mEGLContext = EGL14.EGL_NO_CONTEXT
mEGLConfig = null
}
/**
* Destroys the specified surface. Note the EGLSurface won't actually be destroyed if it's
* still current in a context.
*/
fun releaseSurface(eglSurface: EGLSurface?) {
EGL14.eglDestroySurface(mEGLDisplay, eglSurface)
}
fun createWindowSurface(surface: Surface): EGLSurface = createWindowSurface(surface)
fun createWindowSurface(surface: SurfaceTexture): EGLSurface = createWindowSurface(surface)
/**
* Creates an EGL surface associated with a Surface.
*
* If this is destined for MediaCodec, the EGLConfig should have the "recordable" attribute.
*/
fun createWindowSurface(surface: Any): EGLSurface {
if (surface !is Surface && surface !is SurfaceTexture) {
throw RuntimeException("invalid surface: $surface")
}
// Create a window surface, and attach it to the Surface we received.
val surfaceAttribs = intArrayOf(
EGL14.EGL_NONE
)
val eglSurface: EGLSurface? = EGL14.eglCreateWindowSurface(
mEGLDisplay, mEGLConfig, surface,
surfaceAttribs, 0
)
checkEglError("eglCreateWindowSurface")
if (eglSurface == null) {
throw RuntimeException("surface was null")
}
return eglSurface
}
/**
* Creates an EGL surface associated with an offscreen buffer.
*/
fun createOffscreenSurface(width: Int, height: Int): EGLSurface {
val surfaceAttribs = intArrayOf(
EGL14.EGL_WIDTH, width,
EGL14.EGL_HEIGHT, height,
EGL14.EGL_NONE
)
val eglSurface: EGLSurface? = EGL14.eglCreatePbufferSurface(
mEGLDisplay, mEGLConfig,
surfaceAttribs, 0
)
checkEglError("eglCreatePbufferSurface")
if (eglSurface == null) {
throw RuntimeException("surface was null")
}
return eglSurface
}
/**
* Makes our EGL context current, using the supplied surface for both "draw" and "read".
*/
fun makeCurrent(eglSurface: EGLSurface?) {
if (mEGLDisplay === EGL14.EGL_NO_DISPLAY) {
// called makeCurrent() before create?
log(TAG, "NOTE: makeCurrent w/o display")
}
if (!EGL14.eglMakeCurrent(mEGLDisplay, eglSurface, eglSurface, mEGLContext)) {
throw RuntimeException("eglMakeCurrent failed")
}
}
/**
* Makes our EGL context current, using the supplied "draw" and "read" surfaces.
*/
fun makeCurrent(drawSurface: EGLSurface?, readSurface: EGLSurface?) {
if (mEGLDisplay === EGL14.EGL_NO_DISPLAY) {
// called makeCurrent() before create?
log(TAG, "NOTE: makeCurrent w/o display")
}
if (!EGL14.eglMakeCurrent(mEGLDisplay, drawSurface, readSurface, mEGLContext)) {
throw RuntimeException("eglMakeCurrent(draw,read) failed")
}
}
/**
* Makes no context current.
*/
fun makeNothingCurrent() {
if (!EGL14.eglMakeCurrent(
mEGLDisplay, EGL14.EGL_NO_SURFACE, EGL14.EGL_NO_SURFACE,
EGL14.EGL_NO_CONTEXT
)
) {
throw RuntimeException("eglMakeCurrent failed")
}
}
/**
* Calls eglSwapBuffers. Use this to "publish" the current frame.
*
* @return false on failure
*/
fun swapBuffers(eglSurface: EGLSurface?): Boolean {
return EGL14.eglSwapBuffers(mEGLDisplay, eglSurface)
}
/**
* Sends the presentation time stamp to EGL. Time is expressed in nanoseconds.
*/
fun setPresentationTime(eglSurface: EGLSurface?, nsecs: Long) {
EGLExt.eglPresentationTimeANDROID(mEGLDisplay, eglSurface, nsecs)
}
/**
* Returns true if our context and the specified surface are current.
*/
fun isCurrent(eglSurface: EGLSurface): Boolean {
return mEGLContext.equals(EGL14.eglGetCurrentContext()) &&
eglSurface.equals(EGL14.eglGetCurrentSurface(EGL14.EGL_DRAW))
}
/**
* Performs a simple surface query.
*/
fun querySurface(eglSurface: EGLSurface?, what: Int): Int {
val value = IntArray(1)
EGL14.eglQuerySurface(mEGLDisplay, eglSurface, what, value, 0)
return value[0]
}
/**
* Queries a string value.
*/
fun queryString(what: Int): String {
return EGL14.eglQueryString(mEGLDisplay, what)
}
/**
* Returns the GLES version this context is configured for (currently 2 or 3).
*/
fun getGlVersion(): Int {
return mGlVersion
}
/**
* Writes the current display, context, and surface to the log.
*/
fun logCurrent(msg: String) {
val display: EGLDisplay = EGL14.eglGetCurrentDisplay()
val context: EGLContext = EGL14.eglGetCurrentContext()
val surface: EGLSurface = EGL14.eglGetCurrentSurface(EGL14.EGL_DRAW)
log(
TAG, "Current EGL ($msg): display=$display, context=$context, surface=$surface"
)
}
/**
* Checks for EGL errors. Throws an exception if an error has been raised.
*/
private fun checkEglError(msg: String) {
val error: Int
if ((EGL14.eglGetError().also { error = it }) != EGL14.EGL_SUCCESS) {
throw RuntimeException(msg + ": EGL error: 0x" + Integer.toHexString(error))
}
}
}
@@ -0,0 +1,178 @@
package pw.binom.viewmate.glasses.render
import android.graphics.Bitmap
import android.opengl.EGL14
import android.opengl.EGLSurface
import android.opengl.GLES20
import pw.binom.viewmate.glasses.log
import java.io.BufferedOutputStream
import java.io.File
import java.io.FileOutputStream
import java.io.IOException
import java.nio.ByteBuffer
import java.nio.ByteOrder
open class EglSurfaceBase {
companion object {
const val TAG: String = "GlUtil"
}
// EglCore object we're associated with. It may be associated with multiple surfaces.
protected lateinit var mEglCore: EglCore
private var mEGLSurface: EGLSurface = EGL14.EGL_NO_SURFACE
private var mWidth = -1
private var mHeight = -1
protected constructor(eglCore: EglCore) {
mEglCore = eglCore
}
/**
* Creates a window surface.
*
* @param surface May be a Surface or SurfaceTexture.
*/
fun createWindowSurface(surface: Any) {
check(mEGLSurface === EGL14.EGL_NO_SURFACE) { "surface already created" }
mEGLSurface = mEglCore.createWindowSurface(surface)
// Don't cache width/height here, because the size of the underlying surface can change
// out from under us (see e.g. HardwareScalerActivity).
//mWidth = mEglCore.querySurface(mEGLSurface, EGL14.EGL_WIDTH);
//mHeight = mEglCore.querySurface(mEGLSurface, EGL14.EGL_HEIGHT);
}
/**
* Creates an off-screen surface.
*/
fun createOffscreenSurface(width: Int, height: Int) {
check(mEGLSurface === EGL14.EGL_NO_SURFACE) { "surface already created" }
mEGLSurface = mEglCore.createOffscreenSurface(width, height)
mWidth = width
mHeight = height
}
/**
* Returns the surface's width, in pixels.
*
* If this is called on a window surface, and the underlying surface is in the process
* of changing size, we may not see the new size right away (e.g. in the "surfaceChanged"
* callback). The size should match after the next buffer swap.
*/
fun getWidth(): Int {
return if (mWidth < 0) {
mEglCore.querySurface(mEGLSurface, EGL14.EGL_WIDTH)
} else {
mWidth
}
}
/**
* Returns the surface's height, in pixels.
*/
fun getHeight(): Int {
return if (mHeight < 0) {
mEglCore.querySurface(mEGLSurface, EGL14.EGL_HEIGHT)
} else {
mHeight
}
}
/**
* Release the EGL surface.
*/
fun releaseEglSurface() {
mEglCore.releaseSurface(mEGLSurface)
mEGLSurface = EGL14.EGL_NO_SURFACE
mHeight = -1
mWidth = mHeight
}
/**
* Makes our EGL context and surface current.
*/
fun makeCurrent() {
mEglCore.makeCurrent(mEGLSurface)
}
/**
* Makes our EGL context and surface current for drawing, using the supplied surface
* for reading.
*/
fun makeCurrentReadFrom(readSurface: EglSurfaceBase) {
mEglCore.makeCurrent(mEGLSurface, readSurface.mEGLSurface)
}
/**
* Calls eglSwapBuffers. Use this to "publish" the current frame.
*
* @return false on failure
*/
fun swapBuffers(): Boolean {
val result: Boolean = mEglCore.swapBuffers(mEGLSurface)
if (!result) {
log(TAG, "WARNING: swapBuffers() failed")
}
return result
}
/**
* Sends the presentation time stamp to EGL.
*
* @param nsecs Timestamp, in nanoseconds.
*/
fun setPresentationTime(nsecs: Long) {
mEglCore.setPresentationTime(mEGLSurface, nsecs)
}
/**
* Saves the EGL surface to a file.
*
* Expects that this object's EGL surface is current.
*/
@Throws(IOException::class)
fun saveFrame(file: File) {
if (!mEglCore.isCurrent(mEGLSurface)) {
throw RuntimeException("Expected EGL context/surface is not current")
}
// glReadPixels fills in a "direct" ByteBuffer with what is essentially big-endian RGBA
// data (i.e. a byte of red, followed by a byte of green...). While the Bitmap
// constructor that takes an int[] wants little-endian ARGB (blue/red swapped), the
// Bitmap "copy pixels" method wants the same format GL provides.
val filename = file.toString()
val width = getWidth()
val height = getHeight()
val buf = ByteBuffer.allocateDirect(width * height * 4)
buf.order(ByteOrder.LITTLE_ENDIAN)
GLES20.glReadPixels(
0, 0, width, height,
GLES20.GL_RGBA, GLES20.GL_UNSIGNED_BYTE, buf
)
checkGlError("glReadPixels")
buf.rewind()
var bos: BufferedOutputStream? = null
try {
bos = BufferedOutputStream(FileOutputStream(filename))
val bmp = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888)
bmp.copyPixelsFromBuffer(buf)
bmp.compress(Bitmap.CompressFormat.PNG, 90, bos)
bmp.recycle()
} finally {
bos?.close()
}
log(TAG, "Saved $width" + "x" + "$height frame as '$filename'")
}
private fun checkGlError(op: String) {
val error = GLES20.glGetError()
if (error != GLES20.GL_NO_ERROR) {
val msg = op + ": glError 0x" + Integer.toHexString(error)
log(TAG, msg)
throw RuntimeException(msg)
}
}
}
@@ -0,0 +1,70 @@
package pw.binom.viewmate.glasses.render
/**
* @param program The program to use. FullFrameRect takes ownership, and will release
* the program when no longer needed.
*/
class FullFrameRect(program: Texture2dProgram) {
private val mRectDrawable = Drawable2d(Drawable2d.Prefab.FULL_RECTANGLE)
private var mProgram: Texture2dProgram? = program
/**
* Releases resources.
*
* This must be called with the appropriate EGL context current (i.e. the one that was
* current when the constructor was called). If we're about to destroy the EGL context,
* there's no value in having the caller make it current just to do this cleanup, so you
* can pass a flag that will tell this function to skip any EGL-context-specific cleanup.
*/
fun release(doEglCleanup: Boolean) {
if (doEglCleanup) {
mProgram?.release()
}
mProgram = null
}
/**
* Returns the program currently in use.
*/
fun getProgram(): Texture2dProgram? = mProgram
/**
* Changes the program. The previous program will be released.
*
* The appropriate EGL context must be current.
*/
fun changeProgram(program: Texture2dProgram?) {
mProgram?.release()
mProgram = program
}
/**
* Creates a texture object suitable for use with drawFrame().
*/
fun createTextureObject(): Int {
val mProgram = mProgram
checkNotNull(mProgram) { "FrameRect already closed" }
return mProgram.createTextureObject()
}
/**
* Draws a viewport-filling rect, texturing it with the specified texture object.
*/
fun drawFrame(textureId: Int, texMatrix: FloatArray) {
val mProgram = mProgram
checkNotNull(mProgram) { "FrameRect already closed" }
// Use the identity matrix for MVP so our 2x2 FULL_RECTANGLE covers the viewport.
mProgram.draw(
mvpMatrix = GlUtil.IDENTITY_MATRIX,
vertexBuffer = mRectDrawable.mVertexArray,
firstVertex = 0,
vertexCount = mRectDrawable.mVertexCount,
coordsPerVertex = mRectDrawable.mCoordsPerVertex,
vertexStride = mRectDrawable.mVertexStride,
texMatrix = texMatrix,
texBuffer = mRectDrawable.mTexCoordArray,
textureId = textureId,
texStride = mRectDrawable.mTexCoordStride,
)
}
}
@@ -0,0 +1,177 @@
package pw.binom.viewmate.glasses.render
import android.opengl.GLES20
import android.opengl.GLES30
import android.opengl.Matrix
import pw.binom.viewmate.glasses.log
import java.nio.ByteBuffer
import java.nio.ByteOrder
import java.nio.FloatBuffer
object GlUtil {
const val TAG: String = "Grafika"
/** Identity matrix for general use. Don't modify or life will get weird. */
val IDENTITY_MATRIX = FloatArray(16).also {
Matrix.setIdentityM(it, 0)
}
private const val SIZEOF_FLOAT: Int = 4
/**
* Creates a new program from the supplied vertex and fragment shaders.
*
* @return A handle to the program, or 0 on failure.
*/
fun createProgram(vertexSource: String?, fragmentSource: String?): Int {
val vertexShader = loadShader(GLES20.GL_VERTEX_SHADER, vertexSource)
if (vertexShader == 0) {
error("Can't create Vertex Shader")
return 0
}
val pixelShader = loadShader(GLES20.GL_FRAGMENT_SHADER, fragmentSource)
if (pixelShader == 0) {
error("Can't create Pixel Shader")
return 0
}
var program = GLES20.glCreateProgram()
checkGlError("glCreateProgram")
if (program == 0) {
log(TAG, "Could not create program")
}
GLES20.glAttachShader(program, vertexShader)
checkGlError("glAttachShader")
GLES20.glAttachShader(program, pixelShader)
checkGlError("glAttachShader")
GLES20.glLinkProgram(program)
val linkStatus = IntArray(1)
GLES20.glGetProgramiv(program, GLES20.GL_LINK_STATUS, linkStatus, 0)
if (linkStatus[0] != GLES20.GL_TRUE) {
log(TAG, "Could not link program: ")
log(TAG, GLES20.glGetProgramInfoLog(program))
GLES20.glDeleteProgram(program)
program = 0
}
return program
}
/**
* Compiles the provided shader source.
*
* @return A handle to the shader, or 0 on failure.
*/
fun loadShader(shaderType: Int, source: String?): Int {
var shader = GLES20.glCreateShader(shaderType)
checkGlError("glCreateShader type=$shaderType")
GLES20.glShaderSource(shader, source)
GLES20.glCompileShader(shader)
val compiled = IntArray(1)
GLES20.glGetShaderiv(shader, GLES20.GL_COMPILE_STATUS, compiled, 0)
if (compiled[0] == 0) {
log(TAG, "Could not compile shader $shaderType:")
log(TAG, " " + GLES20.glGetShaderInfoLog(shader))
GLES20.glDeleteShader(shader)
shader = 0
}
return shader
}
/**
* Checks to see if a GLES error has been raised.
*/
fun checkGlError(op: String) {
val error = GLES20.glGetError()
if (error != GLES20.GL_NO_ERROR) {
val msg = op + ": glError 0x" + Integer.toHexString(error)
log(TAG, msg)
throw RuntimeException(msg)
}
}
/**
* Checks to see if the location we obtained is valid. GLES returns -1 if a label
* could not be found, but does not set the GL error.
*
* Throws a RuntimeException if the location is invalid.
*/
fun checkLocation(location: Int, label: String) {
if (location < 0) {
throw RuntimeException("Unable to locate '$label' in program")
}
}
/**
* Creates a texture from raw data.
*
* @param data Image data, in a "direct" ByteBuffer.
* @param width Texture width, in pixels (not bytes).
* @param height Texture height, in pixels.
* @param format Image data format (use constant appropriate for glTexImage2D(), e.g. GL_RGBA).
* @return Handle to texture.
*/
fun createImageTexture(data: ByteBuffer?, width: Int, height: Int, format: Int): Int {
val textureHandles = IntArray(1)
GLES20.glGenTextures(1, textureHandles, 0)
val textureHandle = textureHandles[0]
GlUtil.checkGlError("glGenTextures")
// Bind the texture handle to the 2D texture target.
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textureHandle)
// Configure min/mag filtering, i.e. what scaling method do we use if what we're rendering
// is smaller or larger than the source image.
GLES20.glTexParameteri(
GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER,
GLES20.GL_LINEAR
)
GLES20.glTexParameteri(
GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER,
GLES20.GL_LINEAR
)
GlUtil.checkGlError("loadImageTexture")
// Load the data from the buffer into the texture handle.
GLES20.glTexImage2D(
GLES20.GL_TEXTURE_2D, /*level*/0, format,
width, height, /*border*/0, format, GLES20.GL_UNSIGNED_BYTE, data
)
GlUtil.checkGlError("loadImageTexture")
return textureHandle
}
/**
* Allocates a direct float buffer, and populates it with the float array data.
*/
fun createFloatBuffer(coords: FloatArray): FloatBuffer {
// Allocate a direct ByteBuffer, using 4 bytes per float, and copy coords into it.
val bb = ByteBuffer.allocateDirect(coords.size * SIZEOF_FLOAT)
bb.order(ByteOrder.nativeOrder())
val fb = bb.asFloatBuffer()
fb.put(coords)
fb.position(0)
return fb
}
/**
* Writes GL version info to the log.
*/
fun logVersionInfo() {
log(TAG, "vendor : " + GLES20.glGetString(GLES20.GL_VENDOR))
log(TAG, "renderer: " + GLES20.glGetString(GLES20.GL_RENDERER))
log(TAG, "version : " + GLES20.glGetString(GLES20.GL_VERSION))
if (false) {
val values = IntArray(1)
GLES30.glGetIntegerv(GLES30.GL_MAJOR_VERSION, values, 0)
val majorVersion = values[0]
GLES30.glGetIntegerv(GLES30.GL_MINOR_VERSION, values, 0)
val minorVersion = values[0]
if (GLES30.glGetError() == GLES30.GL_NO_ERROR) {
log(TAG, "iversion: $majorVersion.$minorVersion")
}
}
}
}
@@ -0,0 +1,311 @@
package pw.binom.viewmate.glasses.render
import android.opengl.GLES11Ext
import android.opengl.GLES20
import pw.binom.viewmate.glasses.log
import pw.binom.viewmate.glasses.render.GlUtil.checkGlError
import pw.binom.viewmate.glasses.render.GlUtil.checkLocation
import pw.binom.viewmate.glasses.render.GlUtil.createProgram
import java.nio.FloatBuffer
class Texture2dProgram(private val mProgramType: ProgramType) {
companion object {
private const val TAG: String = GlUtil.TAG
// Simple vertex shader, used for all programs.
private const val VERTEX_SHADER = "uniform mat4 uMVPMatrix;\n" +
"uniform mat4 uTexMatrix;\n" +
"attribute vec4 aPosition;\n" +
"attribute vec4 aTextureCoord;\n" +
"varying vec2 vTextureCoord;\n" +
"void main() {\n" +
" gl_Position = uMVPMatrix * aPosition;\n" +
" vTextureCoord = (uTexMatrix * aTextureCoord).xy;\n" +
"}\n"
// Simple fragment shader for use with "normal" 2D textures.
private const val FRAGMENT_SHADER_2D = "precision mediump float;\n" +
"varying vec2 vTextureCoord;\n" +
"uniform sampler2D sTexture;\n" +
"void main() {\n" +
" gl_FragColor = texture2D(sTexture, vTextureCoord);\n" +
"}\n"
// Simple fragment shader for use with external 2D textures (e.g. what we get from
// SurfaceTexture).
private const val FRAGMENT_SHADER_EXT = "#extension GL_OES_EGL_image_external : require\n" +
"precision mediump float;\n" +
"varying vec2 vTextureCoord;\n" +
"uniform samplerExternalOES sTexture;\n" +
"void main() {\n" +
" gl_FragColor = texture2D(sTexture, vTextureCoord);\n" +
"}\n"
// Fragment shader that converts color to black & white with a simple transformation.
private val FRAGMENT_SHADER_EXT_BW = "#extension GL_OES_EGL_image_external : require\n" +
"precision mediump float;\n" +
"varying vec2 vTextureCoord;\n" +
"uniform samplerExternalOES sTexture;\n" +
"void main() {\n" +
" vec4 tc = texture2D(sTexture, vTextureCoord);\n" +
" float color = tc.r * 0.3 + tc.g * 0.59 + tc.b * 0.11;\n" +
" gl_FragColor = vec4(color, color, color, 1.0);\n" +
"}\n"
// Fragment shader with a convolution filter. The upper-left half will be drawn normally,
// the lower-right half will have the filter applied, and a thin red line will be drawn
// at the border.
const val KERNEL_SIZE = 9
private val FRAGMENT_SHADER_EXT_FILT = "#extension GL_OES_EGL_image_external : require\n" +
"#define KERNEL_SIZE " + KERNEL_SIZE + "\n" +
"precision highp float;\n" +
"varying vec2 vTextureCoord;\n" +
"uniform samplerExternalOES sTexture;\n" +
"uniform float uKernel[KERNEL_SIZE];\n" +
"uniform vec2 uTexOffset[KERNEL_SIZE];\n" +
"uniform float uColorAdjust;\n" +
"void main() {\n" +
" int i = 0;\n" +
" vec4 sum = vec4(0.0);\n" +
" if (vTextureCoord.x < vTextureCoord.y - 0.005) {\n" +
" for (i = 0; i < KERNEL_SIZE; i++) {\n" +
" vec4 texc = texture2D(sTexture, vTextureCoord + uTexOffset[i]);\n" +
" sum += texc * uKernel[i];\n" +
" }\n" +
" sum += uColorAdjust;\n" +
" } else if (vTextureCoord.x > vTextureCoord.y + 0.005) {\n" +
" sum = texture2D(sTexture, vTextureCoord);\n" +
" } else {\n" +
" sum.r = 1.0;\n" +
" }\n" +
" gl_FragColor = sum;\n" +
"}\n"
}
enum class ProgramType {
TEXTURE_2D, TEXTURE_EXT, TEXTURE_EXT_BW, TEXTURE_EXT_FILT
}
// Handles to the GL program and various components of it.
private var mProgramHandle = 0
private val muMVPMatrixLoc: Int
private val muTexMatrixLoc: Int
private var muKernelLoc = 0
private val muTexOffsetLoc: Int
private val muColorAdjustLoc: Int
private val maPositionLoc: Int
private val maTextureCoordLoc: Int
private val mTextureTarget: Int
private val mKernel = FloatArray(KERNEL_SIZE)
private var mTexOffset: FloatArray? = null
private var mColorAdjust = 0f
/**
* Prepares the program in the current EGL context.
*/
init {
when (mProgramType) {
ProgramType.TEXTURE_2D -> {
mTextureTarget = GLES20.GL_TEXTURE_2D
mProgramHandle = createProgram(VERTEX_SHADER, FRAGMENT_SHADER_2D)
}
ProgramType.TEXTURE_EXT -> {
mTextureTarget = GLES11Ext.GL_TEXTURE_EXTERNAL_OES
mProgramHandle = createProgram(VERTEX_SHADER, FRAGMENT_SHADER_EXT)
}
ProgramType.TEXTURE_EXT_BW -> {
mTextureTarget = GLES11Ext.GL_TEXTURE_EXTERNAL_OES
mProgramHandle = createProgram(VERTEX_SHADER, FRAGMENT_SHADER_EXT_BW)
}
ProgramType.TEXTURE_EXT_FILT -> {
mTextureTarget = GLES11Ext.GL_TEXTURE_EXTERNAL_OES
mProgramHandle = createProgram(VERTEX_SHADER, FRAGMENT_SHADER_EXT_FILT)
}
else -> throw RuntimeException("Unhandled type " + mProgramType)
}
if (mProgramHandle == 0) {
throw RuntimeException("Unable to create program")
}
log(TAG, "Created program $mProgramHandle ($mProgramType)")
// get locations of attributes and uniforms
maPositionLoc = GLES20.glGetAttribLocation(mProgramHandle, "aPosition")
checkLocation(maPositionLoc, "aPosition")
maTextureCoordLoc = GLES20.glGetAttribLocation(mProgramHandle, "aTextureCoord")
checkLocation(maTextureCoordLoc, "aTextureCoord")
muMVPMatrixLoc = GLES20.glGetUniformLocation(mProgramHandle, "uMVPMatrix")
checkLocation(muMVPMatrixLoc, "uMVPMatrix")
muTexMatrixLoc = GLES20.glGetUniformLocation(mProgramHandle, "uTexMatrix")
checkLocation(muTexMatrixLoc, "uTexMatrix")
muKernelLoc = GLES20.glGetUniformLocation(mProgramHandle, "uKernel")
if (muKernelLoc < 0) {
// no kernel in this one
muKernelLoc = -1
muTexOffsetLoc = -1
muColorAdjustLoc = -1
} else {
// has kernel, must also have tex offset and color adj
muTexOffsetLoc = GLES20.glGetUniformLocation(mProgramHandle, "uTexOffset")
checkLocation(muTexOffsetLoc, "uTexOffset")
muColorAdjustLoc = GLES20.glGetUniformLocation(mProgramHandle, "uColorAdjust")
checkLocation(muColorAdjustLoc, "uColorAdjust")
// initialize default values
setKernel(floatArrayOf(0f, 0f, 0f, 0f, 1f, 0f, 0f, 0f, 0f), 0f)
setTexSize(256, 256)
}
}
/**
* Releases the program.
*
* The appropriate EGL context must be current (i.e. the one that was used to create
* the program).
*/
fun release() {
log(TAG, "deleting program $mProgramHandle")
GLES20.glDeleteProgram(mProgramHandle)
mProgramHandle = -1
}
/**
* Returns the program type.
*/
fun getProgramType() = mProgramType
/**
* Creates a texture object suitable for use with this program.
*
* On exit, the texture will be bound.
*/
fun createTextureObject(): Int {
val textures = IntArray(1)
GLES20.glGenTextures(1, textures, 0)
checkGlError("glGenTextures")
val texId = textures[0]
GLES20.glBindTexture(mTextureTarget, texId)
checkGlError("glBindTexture $texId")
GLES20.glTexParameterf(
GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MIN_FILTER,
GLES20.GL_NEAREST.toFloat()
)
GLES20.glTexParameterf(
GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MAG_FILTER,
GLES20.GL_LINEAR.toFloat()
)
GLES20.glTexParameteri(
GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_S,
GLES20.GL_CLAMP_TO_EDGE
)
GLES20.glTexParameteri(
GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_T,
GLES20.GL_CLAMP_TO_EDGE
)
checkGlError("glTexParameter")
return texId
}
/**
* Configures the convolution filter values.
*
* @param values Normalized filter values; must be KERNEL_SIZE elements.
*/
fun setKernel(values: FloatArray, colorAdj: Float) {
require(values.size == KERNEL_SIZE) {
"Kernel size is " + values.size +
" vs. " + KERNEL_SIZE
}
System.arraycopy(values, 0, mKernel, 0, KERNEL_SIZE)
mColorAdjust = colorAdj
}
/**
* Sets the size of the texture. This is used to find adjacent texels when filtering.
*/
fun setTexSize(width: Int, height: Int) {
val rw = 1.0f / width
val rh = 1.0f / height
// Don't need to create a new array here, but it's syntactically convenient.
mTexOffset = floatArrayOf(
-rw, -rh, 0f, -rh, rw, -rh,
-rw, 0f, 0f, 0f, rw, 0f,
-rw, rh, 0f, rh, rw, rh
)
}
/**
* Issues the draw call. Does the full setup on every call.
*/
fun draw(
mvpMatrix: FloatArray?, vertexBuffer: FloatBuffer, firstVertex: Int,
vertexCount: Int, coordsPerVertex: Int, vertexStride: Int,
texMatrix: FloatArray?, texBuffer: FloatBuffer, textureId: Int, texStride: Int
) {
checkGlError("draw start")
// Select the program.
GLES20.glUseProgram(mProgramHandle)
checkGlError("glUseProgram")
// Set the texture.
GLES20.glActiveTexture(GLES20.GL_TEXTURE0)
GLES20.glBindTexture(mTextureTarget, textureId)
// Copy the model / view / projection matrix over.
GLES20.glUniformMatrix4fv(muMVPMatrixLoc, 1, false, mvpMatrix, 0)
checkGlError("glUniformMatrix4fv")
// Copy the texture transformation matrix over.
GLES20.glUniformMatrix4fv(muTexMatrixLoc, 1, false, texMatrix, 0)
checkGlError("glUniformMatrix4fv")
// Enable the "aPosition" vertex attribute.
GLES20.glEnableVertexAttribArray(maPositionLoc)
checkGlError("glEnableVertexAttribArray")
// Connect vertexBuffer to "aPosition".
GLES20.glVertexAttribPointer(
maPositionLoc, coordsPerVertex,
GLES20.GL_FLOAT, false, vertexStride, vertexBuffer
)
checkGlError("glVertexAttribPointer")
// Enable the "aTextureCoord" vertex attribute.
GLES20.glEnableVertexAttribArray(maTextureCoordLoc)
checkGlError("glEnableVertexAttribArray")
// Connect texBuffer to "aTextureCoord".
GLES20.glVertexAttribPointer(
maTextureCoordLoc, 2,
GLES20.GL_FLOAT, false, texStride, texBuffer
)
checkGlError("glVertexAttribPointer")
// Populate the convolution kernel, if present.
if (muKernelLoc >= 0) {
GLES20.glUniform1fv(muKernelLoc, KERNEL_SIZE, mKernel, 0)
GLES20.glUniform2fv(muTexOffsetLoc, KERNEL_SIZE, mTexOffset, 0)
GLES20.glUniform1f(muColorAdjustLoc, mColorAdjust)
}
// Draw the rect.
GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, firstVertex, vertexCount)
checkGlError("glDrawArrays")
// Done -- disable vertex array, texture, and program.
GLES20.glDisableVertexAttribArray(maPositionLoc)
GLES20.glDisableVertexAttribArray(maTextureCoordLoc)
GLES20.glBindTexture(mTextureTarget, 0)
GLES20.glUseProgram(0)
}
}
@@ -0,0 +1,67 @@
package pw.binom.viewmate.glasses.render
import android.graphics.SurfaceTexture
import android.view.Surface
class WindowSurface : EglSurfaceBase {
private var mSurface: Surface? = null
private var mReleaseSurface = false
/**
* Associates an EGL surface with the native window surface.
*
* Set releaseSurface to true if you want the Surface to be released when release() is
* called. This is convenient, but can interfere with framework classes that expect to
* manage the Surface themselves (e.g. if you release a SurfaceView's Surface, the
* surfaceDestroyed() callback won't fire).
*/
constructor(eglCore: EglCore, surface: Surface, releaseSurface: Boolean) : super(eglCore) {
createWindowSurface(surface)
mSurface = surface
mReleaseSurface = releaseSurface
}
/**
* Associates an EGL surface with the SurfaceTexture.
*/
constructor(eglCore: EglCore, surfaceTexture: SurfaceTexture) : super(eglCore) {
createWindowSurface(surfaceTexture)
}
/**
* Releases any resources associated with the EGL surface (and, if configured to do so,
* with the Surface as well).
*
* Does not require that the surface's EGL context be current.
*/
fun release() {
releaseEglSurface()
val mSurface = mSurface
if (mSurface != null) {
if (mReleaseSurface) {
mSurface.release()
}
this.mSurface = null
}
}
/**
* Recreate the EGLSurface, using the new EglBase. The caller should have already
* freed the old EGLSurface with releaseEglSurface().
*
* This is useful when we want to update the EGLSurface associated with a Surface.
* For example, if we want to share with a different EGLContext, which can only
* be done by tearing down and recreating the context. (That's handled by the caller;
* this just creates a new EGLSurface for the Surface we were handed earlier.)
*
* If the previous EGLSurface isn't fully destroyed, e.g. it's still current on a
* context somewhere, the create call will fail with complaints from the Surface
* about already being connected.
*/
fun recreate(newEglCore: EglCore) {
val mSurface = mSurface
checkNotNull(mSurface) { "not yet implemented for SurfaceTexture" }
mEglCore = newEglCore // switch to new context
createWindowSurface(mSurface) // create new surface
}
}
@@ -0,0 +1,181 @@
package pw.binom.viewmate.glasses.ui
import android.graphics.SurfaceTexture
import android.opengl.GLES20
import android.view.Surface
import android.view.SurfaceHolder
import android.view.SurfaceView
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.runtime.Composable
import androidx.compose.runtime.DisposableEffect
import androidx.compose.runtime.remember
import androidx.compose.ui.Modifier
import androidx.compose.ui.viewinterop.AndroidView
import com.google.android.exoplayer2.SimpleExoPlayer
import pw.binom.viewmate.glasses.log
import pw.binom.viewmate.glasses.render.EglCore
import pw.binom.viewmate.glasses.render.FullFrameRect
import pw.binom.viewmate.glasses.render.Texture2dProgram
import pw.binom.viewmate.glasses.render.WindowSurface
/**
* Бинокулярный видеорендер: один [SimpleExoPlayer] → SurfaceTexture → EGL-блит
* в несколько выводов. Внутри [Binocular] content вызывается ДВАЖДЫ (левый/правый
* глаз), поэтому [EglVideoRenderer] создаётся ОДИН на экран и шарится между
* двумя экземплярами [BinocularVideoPlayer] — иначе второй setVideoSurface
* перезаписал бы первый.
*/
@Composable
fun BinocularVideoPlayer(
player: SimpleExoPlayer,
renderer: EglVideoRenderer,
modifier: Modifier = Modifier,
) {
Box(modifier = modifier) {
// Два SurfaceView на экземпляр (левый/правый глаз). Все они
// регистрируются в общем рендерере и получают один и тот же кадр.
AndroidView(
modifier = Modifier.fillMaxSize(),
factory = { context ->
SurfaceView(context).also { sv ->
sv.holder.addCallback(renderer.createCallback())
}
}
)
AndroidView(
modifier = Modifier.fillMaxSize(),
factory = { context ->
SurfaceView(context).also { sv ->
sv.holder.addCallback(renderer.createCallback())
}
}
)
}
}
/**
* Общее GL-состояние: один EglCore + FullFrameRect + SurfaceTexture для плеера,
* кадр дублируется во все готовые выводы (окна поверхностей).
*
* Всё создаётся на главном потоке (SurfaceHolder.Callback и OnFrameAvailableListener),
* поэтому защита от гонок — null-проверки: кадр может прийти между созданиями выводов.
*/
class EglVideoRenderer(private val player: SimpleExoPlayer) {
private var eglCore: EglCore? = null
private var fullFrameBlit: FullFrameRect? = null
private var textureId = 0
private var videoSurfaceTexture: SurfaceTexture? = null
private val transformMatrix = FloatArray(16)
private var playerSurface: Surface? = null
private val outputs = mutableListOf<Output>()
private class Output(
val holder: SurfaceHolder,
val surface: WindowSurface,
) {
var width = 0
var height = 0
}
private var released = false
private val frameAvailableListener = SurfaceTexture.OnFrameAvailableListener { _ ->
if (released) return@OnFrameAvailableListener
val st = videoSurfaceTexture
val blit = fullFrameBlit
if (st == null || blit == null) return@OnFrameAvailableListener
st.updateTexImage()
st.getTransformMatrix(transformMatrix)
outputs.forEach { output ->
if (output.width > 0 && output.height > 0) {
drawFrame(blit, output.surface, output.width, output.height)
}
}
}
/** Колбэк для очередного SurfaceView (каждая половина экрана регистрирует свой вывод). */
fun createCallback(): SurfaceHolder.Callback = object : SurfaceHolder.Callback {
override fun surfaceCreated(holder: SurfaceHolder) = attach(holder)
override fun surfaceChanged(
holder: SurfaceHolder, format: Int, width: Int, height: Int
) {
outputs.firstOrNull { it.holder === holder }?.let {
it.width = width
it.height = height
}
}
override fun surfaceDestroyed(holder: SurfaceHolder) {
val index = outputs.indexOfFirst { it.holder === holder }
if (index >= 0) {
val output = outputs.removeAt(index)
output.surface.release()
log("glasses", "EglVideoRenderer: вывод удалён")
}
}
}
private fun attach(holder: SurfaceHolder) {
if (released) return
val egl = eglCore ?: EglCore().also { eglCore = it }
val ws = WindowSurface(egl, holder.surface, false)
outputs.add(Output(holder, ws))
if (fullFrameBlit == null) {
// Первый вывод: создаём общий GL-стэйт, текущий контекст — его.
ws.makeCurrent()
fullFrameBlit = FullFrameRect(Texture2dProgram(Texture2dProgram.ProgramType.TEXTURE_EXT))
textureId = fullFrameBlit!!.createTextureObject()
videoSurfaceTexture = SurfaceTexture(textureId).also {
it.setOnFrameAvailableListener(frameAvailableListener)
}
playerSurface = Surface(videoSurfaceTexture).also {
player.setVideoSurface(it)
}
log("glasses", "EglVideoRenderer: GL-стэйт создан, плеер привязан к SurfaceTexture")
}
}
private fun drawFrame(blit: FullFrameRect, windowSurface: WindowSurface, viewWidth: Int, viewHeight: Int) {
windowSurface.makeCurrent()
GLES20.glViewport(0, 0, viewWidth, viewHeight)
blit.drawFrame(textureId, transformMatrix)
windowSurface.swapBuffers()
}
fun release() {
if (released) return
released = true
log("glasses", "EglVideoRenderer: release")
// Отвязать плеер (плеер мог быть уже освобождён — не падаем).
val ps = playerSurface
if (ps != null) {
try {
player.setVideoSurface(null)
} catch (e: Throwable) {
log("glasses", "setVideoSurface(null) пропущен: ${e.message}")
}
ps.release()
playerSurface = null
}
videoSurfaceTexture?.release()
videoSurfaceTexture = null
outputs.forEach { it.surface.release() }
outputs.clear()
fullFrameBlit?.release(false)
fullFrameBlit = null
eglCore?.release()
eglCore = null
}
}
@@ -0,0 +1,124 @@
package pw.binom.viewmate.glasses.ui
import android.app.Activity
import android.view.WindowManager
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.width
import androidx.compose.runtime.Composable
import androidx.compose.runtime.DisposableEffect
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableIntStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.platform.LocalContext
import com.google.android.exoplayer2.Player
import com.google.android.exoplayer2.SimpleExoPlayer
import com.google.android.exoplayer2.video.VideoSize
import pw.binom.viewmate.glasses.log
/**
* Экран кино: один плеер, вывод — дважды (левый/правый глаз) через [Binocular].
* «Дубль» на эмуляторе 1280×480 — ожидаемое поведение.
*/
@Composable
fun GlassesMovieScreen(
player: SimpleExoPlayer,
title: String?,
) {
log("glasses", "GlassesMovieScreen: ${title ?: "без названия"}")
val context = LocalContext.current
val window = (context as Activity).window
DisposableEffect(window) {
window.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
onDispose {
window.clearFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
}
}
var videoWidth by remember { mutableIntStateOf(640) }
var videoHeight by remember { mutableIntStateOf(480) }
// Один рендерер на экран: оба экземпляра BinocularVideoPlayer (левый/правый
// глаз из Binocular) регистрируют в нём свои SurfaceView.
val renderer = remember { EglVideoRenderer(player) }
DisposableEffect(player) {
val listener = object : Player.Listener {
override fun onVideoSizeChanged(videoSize: VideoSize) {
if (videoSize.width > 0 && videoSize.height > 0) {
log("glasses", "videoSize: ${videoSize.width}x${videoSize.height}")
videoWidth = videoSize.width
videoHeight = videoSize.height
}
}
}
player.addListener(listener)
onDispose {
player.removeListener(listener)
renderer.release()
}
}
Binocular(scale = true) {
val screenWidthPx = BinocularScreenSize.current.width
val screenHeightPx = BinocularScreenSize.current.height
val (playerWidth, playerHeight) = calculateVideoSizeFull(
videoWidth = videoWidth,
videoHeight = videoHeight,
maxWidth = screenWidthPx, // padding = 0
maxHeight = screenHeightPx - 25,
)
// «Нос к носу»: окно каждого глаза прижато к центру экрана.
val horizontalArrangement = when (BinocularScreenSide.current) {
Side.LEFT -> Arrangement.End
Side.RIGHT -> Arrangement.Start
}
Row(
modifier = Modifier
.background(Color.Black)
.width((screenWidthPx).px),
horizontalArrangement = horizontalArrangement,
) {
BinocularVideoPlayer(
player = player,
renderer = renderer,
modifier = Modifier.size(playerWidth.px, playerHeight.px),
)
}
}
}
/** Fit с сохранением пропорций: вариант по ширине/по высоте, берём помещающийся. */
fun calculateVideoSizeFull(
videoWidth: Int, videoHeight: Int,
maxWidth: Int, maxHeight: Int,
paddingStart: Int = 0, paddingEnd: Int = 0,
paddingTop: Int = 0, paddingBottom: Int = 0
): Pair<Int, Int> {
val availableWidth = maxWidth - (paddingStart + paddingEnd)
val availableHeight = maxHeight - (paddingTop + paddingBottom)
val aspectRatio = videoWidth.toFloat() / videoHeight.toFloat()
// Вариант: ширина по максимальной ширине
val widthByWidth = availableWidth
val heightByWidth = (widthByWidth / aspectRatio).toInt()
// Вариант: высота по максимальной высоте
val heightByHeight = availableHeight
val widthByHeight = (heightByHeight * aspectRatio).toInt()
// Выбираем вариант, который помещается целиком
return if (heightByWidth <= availableHeight) {
Pair(widthByWidth, heightByWidth)
} else {
Pair(widthByHeight, heightByHeight)
}
}
@@ -0,0 +1,10 @@
package pw.binom.viewmate.glasses.ui
import androidx.compose.runtime.Composable
import androidx.compose.ui.platform.LocalDensity
import androidx.compose.ui.unit.Dp
/** Перевод пикселей (например, из расчёта размера видео) в Dp текущей плотности. */
@get:Composable
val Int.px: Dp
get() = with(LocalDensity.current) { this@px.toDp() }
@@ -0,0 +1,161 @@
package pw.binom.viewmate.glasses.ui
import android.app.Activity
import androidx.compose.foundation.focusable
import androidx.compose.foundation.layout.BoxWithConstraints
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.runtime.Composable
import androidx.compose.runtime.CompositionLocalProvider
import androidx.compose.runtime.ProvidableCompositionLocal
import androidx.compose.runtime.remember
import androidx.compose.runtime.staticCompositionLocalOf
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.TransformOrigin
import androidx.compose.ui.graphics.graphicsLayer
import androidx.compose.ui.layout.Layout
import androidx.compose.ui.platform.LocalContext
import kotlin.reflect.KProperty
enum class Side {
LEFT, RIGHT,
}
val BinocularScreenSide = staticCompositionLocalOf<Side> {
error("Rayneo Layout not active")
}
@Composable
fun <T> ProvidableCompositionLocal<T>.getValue(thisRef: Any?, property: KProperty<*>): T = current
data class ScreenSize(
val width: Int,
val height: Int,
)
val BinocularScreenSize = staticCompositionLocalOf<ScreenSize> {
error("Rayneo Layout not active")
}
@Composable
fun BinocularSize(scale: Boolean, func: @Composable () -> Unit) {
val context = LocalContext.current
val screenWidthPx =
remember { (context as Activity).windowManager.currentWindowMetrics.bounds.width() }
val screenHeightPx =
remember { (context as Activity).windowManager.currentWindowMetrics.bounds.height() }
if (scale) {
CompositionLocalProvider(
BinocularScreenSize provides ScreenSize(
width = screenWidthPx,
height = screenHeightPx,
)
) {
func()
}
} else {
CompositionLocalProvider(
BinocularScreenSize provides ScreenSize(
width = screenWidthPx / 2,
height = screenHeightPx,
)
) {
func()
}
}
}
@Composable
@Suppress("UnusedBoxWithConstraintsScope")
fun Binocular(
modifier: Modifier = Modifier,
eachModifier: Modifier = Modifier,
leftModifier: Modifier = Modifier,
rightModifier: Modifier = Modifier,
leftRotation: Float = 0f,
rightRotation: Float = 0f,
scale: Boolean,
content: @Composable () -> Unit,
) {
BinocularSize(scale) {
if (scale) {
BoxWithConstraints {
val parentWidth = constraints.maxWidth
PositionedBox(
x = 0, y = 0, modifier = modifier
.graphicsLayer {
scaleX = 0.5f
transformOrigin = TransformOrigin(0f, 0.5f)
}
.graphicsLayer {
transformOrigin = TransformOrigin.Center
rotationZ = leftRotation
}
) {
CompositionLocalProvider(
BinocularScreenSide provides Side.LEFT
) {
content()
}
}
PositionedBox(
x = (parentWidth),
y = 0,
modifier = modifier
.graphicsLayer {
scaleX = 0.5f
transformOrigin = TransformOrigin(0f, 0.5f)
}
.graphicsLayer {
transformOrigin = TransformOrigin.Center
rotationZ = rightRotation
}
) {
CompositionLocalProvider(
BinocularScreenSide provides Side.RIGHT
) {
content()
}
}
}
} else {
Row(modifier = modifier.focusable()) {
CompositionLocalProvider(
BinocularScreenSide provides Side.LEFT
) {
Column(
modifier = eachModifier
.weight(1f, true)
.then(leftModifier)
) { content() }
}
// Box(modifier = Modifier.width(1.dp).fillMaxHeight().background(color= Color.Green)) { Text("") }
CompositionLocalProvider(
BinocularScreenSide provides Side.RIGHT
) {
Column(
modifier = eachModifier
.weight(1f, true)
.then(rightModifier)
) { content() }
}
}
}
}
}
@Composable
fun PositionedBox(
x: Int,
y: Int,
modifier: Modifier = Modifier,
content: @Composable () -> Unit,
) {
Layout(content = content, modifier = modifier) { measurables, constraints ->
val placeable = measurables.first().measure(constraints)
layout(constraints.maxWidth, constraints.maxHeight) {
placeable.placeRelative(x, y)
}
}
}