fix(downloader): rank universal macOS builds and x86-64 microarchitecture levels

skills/dload-fetch-tool/references/registry-entry.md

@@ -22,14 +22,14 @@ Filters the release's asset list. dload then runs OS/arch detection on every mat
 - match all OS/arch variants of the tool;
 - leave out sibling tools, checksums, signatures, source archives.
 
-Build variants stay in the pattern too. Among the assets for the host, dload prefers the host libc (`musl` builds on Alpine and Android, the others elsewhere), then the name with the fewest extra tokens — so `bun-linux-x64.zip` wins over `-baseline`, `-profile` and `-debug` twins — then archives. Android builds are dropped on Linux whenever the entry has a `binary` (without one, every other platform is only ranked lower), and so are OS packages (`.deb`, `.rpm`, `.apk`, `.snap`, `.flatpak`, `.msi`, `.msix`, `.msixbundle`, `.appx`, `.appxbundle`, `.nupkg`, `.dmg`, `.pkg`) and archives dload has no extractor for (known archive formats other than `.zip`, `.tar.gz`, `.tgz`, `.tar`, `.gz`, `.phar` — such as `.tar.xz`, `.tar.bz2`, `.zst`, `.pkg.tar.zst`, `.7z`, `.rar`) everywhere: dload cannot unpack them, so a release that has only such assets for the host gives way to an older one among the 10 newest releases that satisfy `version` (pin `version` to go further back). On ARM macOS and Windows, x86-64 builds are a fallback: Rosetta 2 and the Windows emulation run them.
+Build variants stay in the pattern too. Among the assets for the host, dload prefers the lowest x86-64 microarchitecture level (`amd64` or `amd64_v1` over `amd64_v2`, `_v3`, `_v4`, also after `x86_64` or `x86-64`: the CPU is not checked, so a higher level is only a fallback, still picked when it is the only build for the host), then the host libc (`musl` builds on Alpine and Android, the others elsewhere), then the name with the fewest extra tokens — so `bun-linux-x64.zip` wins over `-baseline`, `-profile` and `-debug` twins — then archives. Android builds are dropped on Linux whenever the entry has a `binary` (without one, every other platform is only ranked lower), and so are OS packages (`.deb`, `.rpm`, `.apk`, `.snap`, `.flatpak`, `.msi`, `.msix`, `.msixbundle`, `.appx`, `.appxbundle`, `.nupkg`, `.dmg`, `.pkg`) and archives dload has no extractor for (known archive formats other than `.zip`, `.tar.gz`, `.tgz`, `.tar`, `.gz`, `.phar` — such as `.tar.xz`, `.tar.bz2`, `.zst`, `.pkg.tar.zst`, `.7z`, `.rar`) everywhere: dload cannot unpack them, so a release that has only such assets for the host gives way to an older one among the 10 newest releases that satisfy `version` (pin `version` to go further back). On ARM macOS and Windows, x86-64 builds are a fallback: Rosetta 2 and the Windows emulation run them. On macOS, a universal build — `universal`, `universal2` or `all` right next to `darwin` or `macos`, like `darwin-universal` or `macOS_all` — counts as native on both Intel and Apple Silicon, so it is kept by a strict selection and ranks with the host-arch build; elsewhere it has no arch.
 
 Tokens the OS/arch matchers recognise (case-insensitive, bounded by `_` or a word boundary):
 
 | Kind | Tokens |
 |---|---|
 | OS   | `windows`, `linux`, `darwin`, `macos`, `alpine`, `bsd`, `freebsd`, `win32`, `win64`, `android` (wins over `linux`) |
-| Arch | `amd64`, `arm64`, `aarch64`, `x86_64`, `x64`, `win64` |
+| Arch | `amd64`, `arm64`, `aarch64`, `x86_64`, `x86-64`, `x64`, `win64` |
 | Libc | `musl`, `musleabi*`, `alpine`; `gnu`, `gnueabi*`, `glibc`; none means glibc |
 
 | Situation | Pattern |

skills/dload-fetch-tool/references/troubleshooting.md

@@ -38,18 +38,18 @@ curl -s "https://api.github.com/repos/<owner>/<repo>/releases/tags/<tag>" \
 After filtering, dload picks the host's variant with:
 
 - OS: `/(?:\b|_)(windows|linux|darwin|macos|alpine|bsd|freebsd|win32|win64)(?:\b|_)/i`; an `android` token makes it Android, a bare `musl` makes it Linux
-- Arch: `/(?:\b|_)(amd64|arm64|aarch64|x86_64|x64|win64)(?:\b|_)/i`
+- Arch: `/(?:\b|_)(amd64|arm64|aarch64|x86[_-]64|x64|win64)(?:\b|_)/i`
 
 A candidate that matches neither is discarded. Common offenders:
 
-- Non-standard tokens: `linux64`, `osx`, `darwin_universal`.
+- Non-standard tokens: `linux64`, `osx`. A universal macOS build (`darwin-universal`, `macOS_all`) has no arch token but is kept on macOS: `universal`, `universal2` or `all` next to `darwin`/`macos` counts as the host arch.
 - No arch separator: `tool-linux64.tar.gz` — the OS matches, no arch token is extractable.
 
 Compare with the host: `php -r "echo PHP_OS_FAMILY, ' / ', php_uname('m'), PHP_EOL;"`.
 
 With a `binary`, OS packages (`.deb`, `.msi`… — the list is in [`registry-entry.md`](registry-entry.md#asset-pattern)) and archives of a format dload cannot unpack (known archive formats other than `.zip`, `.tar.gz`, `.tgz`, `.tar`, `.gz`, `.phar`, such as `.tar.xz`, `.7z`, `.zst`) are discarded too, and a release offering only such assets is skipped for an older one among the 10 newest releases that satisfy `version`, with a warning naming both versions. When their names carry the host OS and arch, the report says "only OS packages match" or "only archives of an unsupported format match" and lists them under "OS packages are skipped because a binary is expected" or "Archives of an unsupported format are skipped because a binary is expected"; one without an OS token (`tool.dmg`) is discarded by the OS check above instead. When no older release fits, pin a `version` that ships a supported archive or a raw binary for the host — pinning also reaches past the 10 newest.
 
-Several candidates left: dload tries them best first — host libc, then the fewest extra name tokens, then archives, then the release order. Run with `-vvv` to see the rank of every asset (`Asset ... ranked: os=0 arch=0 libc=0 extras=1 archive=0`); a wrong pick usually means the wanted build carries an extra token the others lack.
+Several candidates left: dload tries them best first — the lowest x86-64 microarchitecture level (`amd64_v3` after `amd64` or `amd64_v1`), then host libc, then the fewest extra name tokens, then archives, then the release order. Run with `-vvv` to see the rank of every asset (`Asset ... ranked: os=0 arch=0 level=0 libc=0 extras=1 archive=0`); a wrong pick usually means the wanted build carries an extra token the others lack.
 
 ## Stage 4 — binary extraction
 

src/Module/Common/Architecture.php

@@ -44,15 +44,15 @@ public static function fromGlobals(): self
     public static function tryFromString(string $arch): ?self
     {
         return match ($arch) {
-            'AMD64', 'amd64', 'x64', 'x86_64', 'win64' => self::X86_64,
+            'AMD64', 'amd64', 'x64', 'x86_64', 'x86-64', 'win64' => self::X86_64,
             'arm64', 'aarch64' => self::ARM_64,
             default => null,
         };
     }
 
     public static function tryFromBuildName(string $name): ?self
     {
-        if (\preg_match('/(?:\b|_)(amd64|arm64|aarch64|x86_64|x64|win64)(?:\b|_)/i', $name, $matches) !== 1) {
+        if (\preg_match('/(?:\b|_)(amd64|arm64|aarch64|x86[_-]64|x64|win64)(?:\b|_)/i', $name, $matches) !== 1) {
             return null;
         }
 

src/Module/Downloader/Internal/AssetSelection/AssetName.php

@@ -25,35 +25,75 @@ final class AssetName
     private const PLATFORM = '/(?<![a-z0-9])(?:x86[_-]64|amd64|arm64|aarch64|x64|win64|win32|windows|linux|darwin|macos|apple|alpine|android|bsd|freebsd|unknown|pc|musl(?:eabi(?:hf)?)?|gnu(?:eabi(?:hf)?)?|glibc)(?![a-z0-9])/';
 
     /**
+     * macOS build for every Mac architecture, like `darwin-universal` or `macOS_all`.
+     *
+     * `universal` and `all` count only next to the macOS token: elsewhere they are ordinary words.
+     */
+    private const UNIVERSAL = '/(?<![a-z0-9])(?:(?:darwin|macos)[-_.](?:universal2?|all)|(?:universal2?|all)[-_.](?:darwin|macos))(?![a-z0-9])/';
+
+    /**
+     * x86-64 microarchitecture level right after the architecture, like `amd64_v3` or `x86-64-v2`,
+     * but not a version there, like `amd64-v2.1.0`.
+     */
+    private const LEVEL = '/(?<![a-z0-9])(?:amd64|x86[_-]64|x64)[-_]v([1-4])(?![a-z0-9]|\.\d)/';
+
+    /**
+     * @param bool $universal Whether the asset is a macOS build for every Mac architecture.
+     * @param int<1, 4> $level x86-64 microarchitecture level the build requires; 1 when the name has none.
      * @param list<non-empty-string> $extras Name tokens that are not a platform, a version or an extension,
      *        like `profile` or `debug`. The tool name is one of them too.
      */
     private function __construct(
         public readonly ?Libc $libc,
+        public readonly bool $universal,
+        public readonly int $level,
         public readonly array $extras,
     ) {}
 
     public static function fromString(string $name): self
     {
+        $lower = \strtolower($name);
+
         return new self(
             libc: Libc::tryFromBuildName($name),
-            extras: self::extras($name),
+            universal: \preg_match(self::UNIVERSAL, $lower) === 1,
+            level: self::level($lower),
+            extras: self::extras($lower),
         );
     }
 
+    /**
+     * @return int<1, 4>
+     */
+    private static function level(string $name): int
+    {
+        \preg_match(self::LEVEL, $name, $matches);
+
+        return match ($matches[1] ?? null) {
+            '2' => 2,
+            '3' => 3,
+            '4' => 4,
+            default => 1,
+        };
+    }
+
     /**
      * @return list<non-empty-string>
      */
     private static function extras(string $name): array
     {
-        $name = \strtolower($name);
         do {
             $name = (string) \preg_replace(self::EXTENSION, '', $name, count: $count);
         } while ($count > 0);
 
         /** @var list<non-empty-string>|false $tokens */
-        $tokens = \preg_split('/[-_.\s]+/', (string) \preg_replace(self::PLATFORM, '', $name), flags: \PREG_SPLIT_NO_EMPTY);
+        $tokens = \preg_split(
+            '/[-_.\s]+/',
+            (string) \preg_replace([self::UNIVERSAL, self::PLATFORM], '', $name),
+            flags: \PREG_SPLIT_NO_EMPTY,
+        );
 
+        // The version filter also drops a microarchitecture level like `v3`, which has its own rank
         return \array_values(\array_filter(
             $tokens === false ? [] : $tokens,
             static fn(string $token): bool => \preg_match('/^v?\d+$/', $token) !== 1,

src/Module/Downloader/Internal/AssetSelection/AssetSelector.php

@@ -15,6 +15,7 @@
 use Internal\DLoad\Module\Downloader\Internal\AssetSelection\Rule\ExtrasRule;
 use Internal\DLoad\Module\Downloader\Internal\AssetSelection\Rule\FormatRule;
 use Internal\DLoad\Module\Downloader\Internal\AssetSelection\Rule\LibcRule;
+use Internal\DLoad\Module\Downloader\Internal\AssetSelection\Rule\MicroarchitectureLevelRule;
 use Internal\DLoad\Module\Downloader\Internal\AssetSelection\Rule\NamePatternRule;
 use Internal\DLoad\Module\Downloader\Internal\AssetSelection\Rule\OperatingSystemRule;
 use Internal\DLoad\Module\Downloader\Internal\AssetSelection\Rule\PackageRule;
@@ -56,6 +57,8 @@ public function __construct(
             new CompanionRule(),
             new OperatingSystemRule($operatingSystem),
             new ArchitectureRule($architecture, $operatingSystem),
+            // Right after the architecture: a level the CPU may lack outweighs the libc and the variant
+            new MicroarchitectureLevelRule(),
             // After the platform rules, so the report names only packages and archives built for the host. One whose
             // name carries no OS token, like `tool.dmg` or `tool.7z`, is gone by then and stays out of the report.
             new PackageRule(),

src/Module/Downloader/Internal/AssetSelection/Rule/ArchitectureRule.php

@@ -13,7 +13,8 @@
 /**
  * Prefers assets built for the host architecture, then the ones the host emulates.
  *
- * A strict selection removes the assets the host cannot run; otherwise they are ranked last.
+ * A universal macOS build counts as built for the host architecture on macOS. A strict selection
+ * removes the assets the host cannot run; otherwise they are ranked last.
  *
  * @internal
  * @psalm-internal Internal\DLoad\Module\Downloader
@@ -34,22 +35,24 @@ public function __construct(
     public function select(Selection $selection, callable $next): Selection
     {
         $selection->strict and $selection = $selection->remove(
-            fn(Candidate $candidate): bool => $this->rank($candidate->asset->getArchitecture()) === null,
+            fn(Candidate $candidate): bool => $this->rank($candidate) === null,
         );
 
         return $next($selection->rank(
             self::KEY,
-            fn(Candidate $candidate): int => $this->rank($candidate->asset->getArchitecture()) ?? 2,
+            fn(Candidate $candidate): int => $this->rank($candidate) ?? 2,
         ));
     }
 
     /**
      * @return int<0, 1>|null Null when the host cannot run the asset.
      */
-    private function rank(?Architecture $arch): ?int
+    private function rank(Candidate $candidate): ?int
     {
+        $arch = $candidate->asset->getArchitecture();
         return match (true) {
             $arch === $this->architecture => 0,
+            $candidate->name->universal && $this->operatingSystem === OperatingSystem::Darwin => 0,
             // Rosetta 2 on macOS and the built-in emulation of Windows run x86-64 builds on ARM
             $arch === Architecture::X86_64
                 && $this->architecture === Architecture::ARM_64

src/Module/Downloader/Internal/AssetSelection/Rule/MicroarchitectureLevelRule.php

@@ -0,0 +1,26 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Internal\DLoad\Module\Downloader\Internal\AssetSelection\Rule;
+
+use Internal\DLoad\Module\Downloader\Internal\AssetSelection\AssetRule;
+use Internal\DLoad\Module\Downloader\Internal\AssetSelection\Candidate;
+use Internal\DLoad\Module\Downloader\Internal\AssetSelection\Selection;
+
+/**
+ * Prefers the lowest x86-64 microarchitecture level, like `amd64` or `amd64_v1` over `amd64_v3`.
+ *
+ * The host CPU features are not checked, so a level above v1 may not run: such a build is ranked
+ * lower rather than removed, and is still picked when the release has nothing else for the host.
+ *
+ * @internal
+ * @psalm-internal Internal\DLoad\Module\Downloader
+ */
+final class MicroarchitectureLevelRule implements AssetRule
+{
+    public function select(Selection $selection, callable $next): Selection
+    {
+        return $next($selection->rank('level', static fn(Candidate $candidate): int => $candidate->name->level - 1));
+    }
+}

tests/Unit/Module/Common/ArchitectureTest.php

@@ -21,6 +21,8 @@ public static function provideBuildNames(): iterable
         yield ['roadrunner-2024.1.5-linux-amd64.deb', Architecture::X86_64];
         yield ['protoc-27.3-win64.zip', Architecture::X86_64];
         yield ['temporal-test-server_1.33.0_macOS_arm64.tar.gz', Architecture::ARM_64];
+        yield ['tool_1.0.0_linux_amd64_v3.tar.gz', Architecture::X86_64];
+        yield ['tool-linux-x86-64-v3.tar.gz', Architecture::X86_64];
     }
 
     #[DataProvider('provideBuildNames')]

tests/Unit/Module/Downloader/Internal/AssetSelection/AssetNameTest.php

@@ -25,6 +25,33 @@ public static function provideExtras(): iterable
         yield 'checksum' => ['tool-linux-amd64.tar.gz.sha256sum', ['tool']];
         yield 'unknown extension' => ['deno-x86_64-unknown-linux-gnu.from-2.9.6.bsdiff', ['deno', 'from', 'bsdiff']];
         yield 'no platform' => ['tool.phar', ['tool']];
+        yield 'universal macOS build' => ['hugo_0.139.0_darwin-universal.tar.gz', ['hugo']];
+        yield 'all macOS architectures' => ['tool_1.0.0_macOS_all.tar.gz', ['tool']];
+        yield 'universal not next to macOS' => ['universal-ctags-linux-x64.tar.gz', ['universal', 'ctags']];
+        yield 'microarchitecture level' => ['tool_1.0.0_linux_amd64_v3.tar.gz', ['tool']];
+    }
+
+    public static function provideUniversal(): iterable
+    {
+        yield 'darwin-universal' => ['hugo_0.139.0_darwin-universal.tar.gz', true];
+        yield 'universal before macOS' => ['tigerbeetle-universal-macos.zip', true];
+        yield 'macOS_all' => ['tool_1.0.0_macOS_all.tar.gz', true];
+        yield 'universal2' => ['tool-macos-universal2.zip', true];
+        yield 'single architecture' => ['tool-darwin-arm64.tar.gz', false];
+        yield 'all without macOS' => ['tool-all-linux-amd64.tar.gz', false];
+        yield 'universal without macOS' => ['universal-ctags-linux-x64.tar.gz', false];
+        yield 'all inside a word' => ['install-darwin-allinone.zip', false];
+    }
+
+    public static function provideLevels(): iterable
+    {
+        yield 'no level' => ['tool_linux_amd64.tar.gz', 1];
+        yield 'v1' => ['tool_linux_amd64_v1.tar.gz', 1];
+        yield 'v2' => ['tool_linux_amd64_v2.tar.gz', 2];
+        yield 'v3 after x86_64' => ['tool-x86_64_v3-linux.tar.gz', 3];
+        yield 'v4 after x86-64' => ['tool-linux-x86-64-v4.tar.gz', 4];
+        yield 'version after the arch' => ['tool-linux-amd64-v2.1.0.tar.gz', 1];
+        yield 'arm' => ['tool-android-arm64-v8a.zip', 1];
     }
 
     /**
@@ -43,4 +70,18 @@ public function libcIsReadFromTheName(): void
         Assert::same(AssetName::fromString('bun-linux-x64-musl.zip')->libc, Libc::Musl);
         Assert::null(AssetName::fromString('bun-linux-x64.zip')->libc);
     }
+
+    #[DataProvider('provideUniversal')]
+    #[Test]
+    public function universalMacOsBuildsAreReadFromTheName(string $name, bool $expected): void
+    {
+        Assert::same(AssetName::fromString($name)->universal, $expected);
+    }
+
+    #[DataProvider('provideLevels')]
+    #[Test]
+    public function theMicroarchitectureLevelIsReadFromTheName(string $name, int $expected): void
+    {
+        Assert::same(AssetName::fromString($name)->level, $expected);
+    }
 }