Skip to content

feat(mobile): model roles, subagent timeline, composer chrome, and immersive shell - #3284

Merged
wgqqqqq merged 8 commits into
GCWing:mainfrom
YodonTan:feat/mobile-ux
Oct 8, 2026
Merged

wgqqqqq merged 8 commits into
GCWing:mainfrom
YodonTan:feat/mobile-ux

Conversation

@YodonTan

@YodonTan YodonTan commented Oct 7, 2026

Copy link
Copy Markdown

Summary

HarmonyOS remote-controller UX work on top of latest main. All changes are confined to src/apps/mobile/harmonyos/ (152 files; no cross-directory or protocol changes).

Five themes, each independently testable:

  1. Model roles in the model list — the session model picker now groups by primary/fast role (matching the host's default_models semantics, fast falls back to primary when unset) instead of a flat list of model ids.
  2. Subagent process visibility — subagent entries now attach to their owning Task card (host marks the sub-session id on the Task tool entry; the client previously dropped all of them), orphan branches get a fallback group, and task cards refresh live instead of freezing on an empty snapshot.
  3. File visibility diagnostics — preview surfaces + tests for the file projection chain (verified: the reported "os error 2" was a stale link to a deleted file, not a client defect).
  4. Composer draft chrome — the collapsed input ellipsizes overflow instead of scrolling sideways, and the supplemental microphone only mounts in the expanded action row (the collapsed draft keeps the space).
  5. Immersive shell — the window now runs full-screen with a transparent navigation bar: the transcript and scroll viewports run to the screen edge, scroll content gets tail insets so scrolled-to-end rows clear the gesture indicator, and fixed controls inset themselves. Cover sheet, gallery, sidebar list, tools panel, editor tab, and every bottom sheet were measured on-device.

Verification

  • node --test src/apps/mobile/harmonyos/tools/tests/*.test.cjs — 265 pass / 1 fail (workspace-editor.test.cjs, pre-existing on main, reproduced on untouched files).
  • pnpm run harmony:architecture — satisfied.
  • On-device evidence (emulator, sanitized layout dumps + screenshots) committed under src/apps/mobile/harmonyos/artifacts/{model-role-selector,subagent-visibility,file-visibility,composer-overflow,immersive-bottom}/.
  • Contract tests are reverse-verified: removing any key line (full-screen switch, tail inset, viewport extension, mic render-site deletion) fails the matching case (10/10 mutations caught).

Cross-client status

  • The supplemental-mic slot is a per-client presentation decision (not in the shared Kotlin policy or the design-system composer_bar anatomy). iOS (ComposerBar.swift) and Android (ComposerBar.kt) still carry older layouts; follow-up commits will align them to this reference implementation.
  • Window insets are a local device fact; no remote command, protocol, or persisted shape changed.

Not verified

  • Fold/half-fold (FolderStack) and the real wide breakpoint: the 418vp emulator cannot reach them; covered by contract tests and noted in src/apps/mobile/harmonyos/AGENTS.md.

Tant added 8 commits October 7, 2026 14:20
Show the semantic `primary` / `fast` selectors as first-class rows in the
composer model list, with the resolved model metadata and a role badge on each
model that serves them.

- ConversationModelPresentationPolicy resolves a semantic selector through the
  catalog, including the fast-to-primary fallback, and projects the role rows.
- RemoteUiState keeps a semantic selector while the host can still resolve it.
- ComposerBar renders the role rows and reports the selected payload.
- Adds the `model-role-selector*` preview scenarios, the `emulator` build target
  (x86_64 abiFilters) and the `chat.model*` strings.

EntryAbility.ets and the MobileDesignGallery scenario registry mix all three
batches on shared lines; the EntryAbility hunks go whole into this commit
(largest share), while the gallery hunks are split per batch.
A running Task's subagent steps stayed invisible in the transcript.

- ChatMessageStructurePolicy keeps the owner Task tool item as a branch owner
  even though the host marks it as a subagent record, and keeps a marked record
  that arrives before its Task in its own branch instead of dropping it.
- ChatMessageBubble re-resolves the branch from the observed message on every
  render, so the ForEach item builder no longer freezes the empty branch a Task
  has before its subagent starts.
- ToolStatusPresentationPolicy treats a blank tool name as absent before
  normalizing it.
- Adds the `subagent-process*` preview scenarios and the
  `subagent-process-visibility` test with its evidence under
  `artifacts/subagent-visibility/`.
Add a native preview that mounts the real `ChatTimeline` and
`ChatMessageBubble` over the production reducer and routes a card tap through
the real `FilePreviewController` into `FilePreviewSurface`, with only the
transport faked.

The `file-visibility` / `file-visibility-dark` scenarios cover markdown links
and bare `computer://` references, the relative-link extension rule, the
per-message card budget, the image renderer versus the unsupported phase, and
the three content paths that draw cards. Evidence is captured under
`artifacts/file-visibility/`.
…al mic

The collapsed input field used to scroll its overflow sideways, so a long
draft ran under the reader's finger with no indication of what was cut off.
It now limits its one-line viewport with TextOverflow.Ellipsis and
MaxLinesMode.CLIP, marking the cut with an ellipsis instead of scrolling
horizontally; the expanded field keeps its scrollable multi-line viewport.

The supplemental microphone slot is asked for by the expanded action row
alone. The collapsed field is one line tall and the draft being composed has
the stronger claim on that room, so the render site inside the field is
deleted rather than gated: a collapsed draft shows the primary action only,
while an expanded one keeps the second way into dictation beside Send.

The preview fixtures cover the collapsed, empty, expanded, dark, and wide
states, report the slot as mounted instead of the bare policy answer, and the
composer-overflow-expanded scenario starts on a multiline draft so the
expanded state is captured without a tap.
… navigation bar

The chat page could not draw into the navigation-bar strip: the window was laid
out non-immersive, so every page's box stopped above it, and the only page that
switched the window to full-screen was the compact welcome page, which switched
it back on the way out. The transcript therefore ended above the bar and the
bottom fade covered only the part above it, leaving a band of bare page colour
under the composer.

Run the shell immersive for every page and keep the navigation bar transparent so
the page's own bottom edge shows through. Immersive layout also moves each page's
origin to the top of the window, so pages that draw chrome against an edge now
reserve the insets themselves through one shared binding
(WindowInsetsBinding, backed by WindowSystemBarService.observeInsets) that
follows fold, rotation and the soft keyboard:

- The chat page keeps the bottom edge as background: the bottom layer carries the
  bottom inset, so its gradient fills the strip; the composer keeps the distance
  from the screen edge it had while the page area ended above the bar; and the
  transcript's measured inset grows with the layer, so the last message scrolls
  above the fade. The header band reserves the status bar the same way, which
  keeps the header pixel-identical to the non-immersive build.
- Every other surface keeps the strip as content: the sidebar footer, the
  mini-app gallery, the settings sheet and the welcome dock pad their bottom
  controls by the strip or by the design's own spacing, whichever is larger, so
  their fill still reaches the screen edge while their rows and scrolling
  viewports stop above the navigation bar. The compact account cover is already
  inset by the framework (enableSafeArea: true).

Measured on the 1256x2760 emulator (status bar 136, navigation indicator 98): the
chat page root, transcript and conversation-bottom-fade all end at 2760 instead
of 2662, the composer stays at [271,2444][986,2598], and no other surface has a
content node past 2662.

Adds the immersive-bottom preview scenarios, the focused contract test and the
on-device evidence in artifacts/immersive-bottom/, and refreshes
artifacts/composer-overflow/ for the taller page.
…n bar

The previous round kept every non-chat surface out of the navigation bar by
padding the container that owned the scrolling viewport, so the settings sheet
and the mini-app gallery stopped above the bar and left a band of bare page
colour under the gesture indicator. Align them with the chat page instead:
the viewport runs to the screen edge and the content ends in a tail spacer.

WindowInsetsBinding gains tailSpacing(designSpacing) — the strip plus the
design's own breathing — which is the number a scrolling surface needs at the
end of its content, where bottomPadding's max() would leave the last row
touching the bar:

- The settings sheet drops its root bottom padding and ends its scrolling
  column in tailSpacing(34).
- The mini-app gallery drops its container's bottom padding and ends the grid
  with a full-width, untouchable spacer row (columnStart/columnEnd span).
- The sidebar is a scrolling session list with a floating footer, so it gets
  the chat page's shape: the list runs to the panel's own bottom edge and ends
  in the footer's clearance plus the strip, while the footer and its fade keep
  the strip clear themselves. The offset is measured from the panel's own
  edge, so the wide master pane's floating card keeps its footer clear too.
- The workspace tools sheet (a full-height bindSheet) runs its file list to
  the screen edge with a tail spacer, and its fixed bottom rows — the terminal
  key row and the upload controls — keep the strip clear.
- The workspace picker confirm button and the connect sheet status strip,
  both fixed controls at a full-height sheet's edge, become strip-aware.
- The file preview scrollers keep their tails strip-aware.

Measured on the 1256x2760 emulator (indicator 98 px, bar top 2662): the
settings sheet scroll and the gallery grid end at 2760 with rows inside the
strip while moving and their last rows at 2697/2667 and 2509/2437 above the
bar; the sidebar list shows session rows at 2739 in the strip and rests its
last row at 2321; the tools file list shows report-13 clipped at 2760 and
rests report-18 at 2526; the connect status strip ends at 2662. The chat page
is unchanged: the fade still ends at 2760 and the composer stays at
[271,2444][986,2598].

The immersive-bottom preview gains scrollable fixture content (an account
with desktops, a device directory with workspaces and sessions, a file
directory, a legacy pairing link) and -tools, -connect and -settings-dark
scenarios, and the contract test now asserts the viewport-plus-tail shape and
catches each spacer's removal.
The immersive-bottom pass left two classes of surface untreated: the workspace
tools editor tab, and the sheets whose controls end at the screen edge without
scrolling under the bar.

- The editor tab is a WebView whose scrolling and padding live inside the web
  renderer, so the ArkUI tail spacer cannot reach its content. The card is
  fixed content instead, like the terminal key row: a strip-aware bottom
  margin stops its edge at the bar top while the sheet's fill still reaches
  the screen edge. Pushing a vp tail through the JS bridge was rejected: the
  web side cannot measure the strip, so the padding would rest on an
  unverifiable vp-to-CSS-px assumption instead of the device fact the binding
  reads.
- The file action form (FIT_CONTENT) and the session action sheet (300vp) are
  fixed controls at the edge: their bottom paddings become strip-aware, with
  the action surface taking the value as a param so only the bottom-sheet
  presentation grows — a popover keeps the plain design spacing.
- The session details and remote view settings sheets are scrolling surfaces:
  their columns end in tailSpacing(24), the same contract the settings sheet
  keeps.
- The welcome dock reads the strip through the shared binding now —
  bottomPadding(G.welcomeDockBottom) — instead of a hard-coded clearance that
  only cleared the bar on the reference device.

The session action sheet was unopenable in between: a node carries at most one
bindSheet, and RemoteSessionList chained the action sheet and the details sheet
on its root, which left the first sheet dead. The action sheet now binds on
its own zero-height carrier row — the pattern the sidebar's section already
uses for its pair — and that is also what made its bottom edge auditable on
device at all.

Three preview scenarios join the evidence pass: -tools-editor seeds the
editor tab, -tools-file-action opens the file action form, and -view-settings
mounts the remote view settings sheet through a real bindSheet with the
production compact placement.

Measured on the 1256x2760 emulator (indicator 98 px, bar top 2662): the
editor card ends at 2659 and does not move while its web content scrolls
inside it; the file action form's input ends at 2662, where the fixed 24vp
padding left it at 2662+14; the view settings sheet shows filter rows inside
the strip while scrolling (2667-2720) and rests its last control at 2512,
where the fixed 24vp tail left it at 2662+14; the session action sheet's last
row sits at 2351 and the details sheet's content ends at 2137 — both above the
bar with fixture content that fits their sheets, so their strip-aware offsets
are hardening for content that grows to fill them; and the welcome dock's
entries keep their exact bounds (explore 2568, sign-in 2603), identical to
the capture taken before the dock moved onto the binding.

The subagent-process test's host reference now points at the durable record
contract — is_subagent_item / subagent_session_id on the item data in
services-core's session types — instead of a coordinator.rs line number that
had already drifted, and the reverse-verification set gains the editor card
and connect status strip mutations alongside the existing ten.
@wgqqqqq
wgqqqqq merged commit 864d893 into GCWing:main Oct 8, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants