Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
root = true

[*]
charset = utf-8
end_of_line = lf
insert_final_newline = true

[*.py]
indent_style = space
indent_size = 4

[*.{json,yml,yaml,md,html}]
indent_style = space
indent_size = 2
3 changes: 3 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Every tracked text file is stored with LF; keep working trees the same on
# Windows and macOS.
* text=auto eol=lf
6 changes: 4 additions & 2 deletions .github/workflows/deploy-pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,12 @@ concurrency:
jobs:
build:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0 # history.json replays every data commit
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
- uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
with:
python-version: "3.12"
- name: Build static index
Expand All @@ -33,6 +34,7 @@ jobs:
deploy:
needs: build
runs-on: ubuntu-latest
timeout-minutes: 10
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
Expand Down
12 changes: 10 additions & 2 deletions .github/workflows/validate-data.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,20 @@ on:
push:
branches: [develop, main]

permissions:
contents: read

concurrency:
group: validate-data-${{ github.ref }}
cancel-in-progress: true

jobs:
validate:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
with:
python-version: "3.12"
- name: Validate ES catalog
Expand Down
31 changes: 25 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
[![validate-data](https://github.com/GetTechAPI/cpu-engineering-samples/actions/workflows/validate-data.yml/badge.svg)](https://github.com/GetTechAPI/cpu-engineering-samples/actions/workflows/validate-data.yml)
 Data: **CC-BY-SA 4.0** · Code: **MIT**

Retail / production CPUs live in [**TechAPI**](https://github.com/GetTechAPI/TechAPI). Qualification samples (QS) are also out of scope. This repository is a **prototype** holding area for publicly documented ES parts so they never mix into the official catalog.
Retail / production CPUs live in [**TechAPI**](https://github.com/GetTechAPI/TechAPI) under `data/cpu/`; this repository holds engineering samples only and publishes its own `summary.json`, which TechAPI's site reads for the record count. Qualification samples (QS) are also out of scope. This repository is a **prototype** holding area for publicly documented ES parts so they never mix into the official catalog.

## Scope

Expand All @@ -20,11 +20,21 @@ ES processors are pre-production parts. They are not for sale, not warranted, an
## Layout

```text
data/cpu/<manufacturer>/<year>/<segment>/<slug>.json
app/validate.py # stdlib validator; rejects QS/retail
site/ # static index (GitHub Pages)
data/cpu/<manufacturer>/<year>/<segment>/<slug>.json # one ES record per file
app/validate.py # stdlib validator; rejects QS/retail
app/ingest/classify.py # offline es / qs / retail / unknown classifier (python -m app.ingest)
app/ingest/from_table.py # writes ES records from a local JSON/JSONL table
site/build.py # writes site/catalog.json, summary.json and history.json
site/index.html # static index (GitHub Pages)
tests/ # pytest: validator, classifier, table ingest
.github/workflows/
validate-data.yml # validator + tests on PRs and pushes to develop/main
final-check.yml # label-triggered check with one TechEngineBot comment
deploy-pages.yml # builds site/ and deploys Pages from main
```

The three JSON files under `site/` are build output and are not committed.

Slugs identify the **sample** (`intel-qxlb`, `amd-100-000000665-21-n`), not the retail name. `retail_equivalent` is an optional TechAPI CPU slug string, not a foreign key.

## Self-check
Expand All @@ -44,7 +54,16 @@ Intake is **classify-then-write**. A candidate is `es`, `qs`, `retail`, or `unkn
python -m app.ingest path/to/candidate.json
```

No network. Identifier rules: Intel S-spec → retail, Q-spec + early stepping / low clocks / “Intel Confidential” → es, Q-spec labeled QS or mature stepping with retail clocks → qs. AMD `100-00000…` / Eng Sample → es, historical `Z…` OPN → qs.
It prints the classification as JSON and writes nothing. No network. Identifier rules: Intel S-spec → retail, Q-spec + early stepping / low clocks / “Intel Confidential” → es, Q-spec labeled QS or mature stepping with retail clocks → qs. AMD `100-00000…` / Eng Sample → es, historical `Z…` OPN → qs.

To turn a local table into records, run the table ingest:

```bash
python -m app.ingest.from_table path/to/table.json --dry-run
python -m app.ingest.from_table path/to/table.json
```

The table is a JSON array or JSONL, one object per row. Each row is classified; `es` rows are written to `data/cpu/<manufacturer>/<year>/<segment>/<slug>.json` (`year` comes from the row's `year` or `first_seen_date`, `segment` defaults to `desktop`). Rows that classify as `qs`, `retail` or `unknown`, and slugs that already exist, are skipped and listed in the printed summary. `--dry-run` prints the same summary without writing. The exit code is 1 when a row has no usable slug. Run `python -m app.validate` afterwards.

Crawlers come later. This classifier is the gate.

Expand All @@ -64,7 +83,7 @@ A maintainer adds the `final-check` label to an important PR. The `final-check`

## Contributing

Open a PR against `develop` with a new/updated JSON file. `sample_class` must be `es`. Include `qspec` (Intel) and/or `opn` (AMD) and at least one public `source_url`.
Open a PR against `develop` with a new/updated JSON file. `sample_class` must be `es`. Include `qspec` (Intel) and/or `opn` (AMD) and at least one public URL in `source_urls`.

## License

Expand Down
3 changes: 2 additions & 1 deletion app/ingest/classify.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
"""Offline ES / QS / retail classifier.

No network. A candidate must be ``es`` before the ingest pipeline may write JSON.
QS and retail belong in TechAPI or nowhere; ``unknown`` goes to a review issue.
QS and retail belong in TechAPI or nowhere; ``unknown`` is reported and left for
a maintainer to review by hand.
"""

from __future__ import annotations
Expand Down
3 changes: 3 additions & 0 deletions app/ingest/from_table.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
"""Write ES JSON from a local table (JSON/JSONL). No network.

Run as ``python -m app.ingest.from_table TABLE [--dry-run]``.

Rows that classify as qs/retail/unknown are skipped. Existing slugs are skipped.
Skipped rows are only listed in the printed summary; nothing else is written.
"""

from __future__ import annotations
Expand Down
Loading