Merge pull request #134 from GetTechAPI/chore/repo-cleanup

chore: drop the TechAPI submodule loop and unused files; document the layout

Author: Seungpyo1007

.editorconfig

@@ -0,0 +1,18 @@
+root = true
+
+[*]
+charset = utf-8
+end_of_line = lf
+insert_final_newline = true
+trim_trailing_whitespace = true
+
+[*.py]
+indent_style = space
+indent_size = 4
+
+[*.{yml,yaml,json,astro,md,mjs}]
+indent_style = space
+indent_size = 2
+
+[*.md]
+trim_trailing_whitespace = false

.gitattributes

@@ -0,0 +1,3 @@
+# Every tracked text file is stored with LF; keep working trees the same on
+# Windows and macOS.
+* text=auto eol=lf

.github/CODEOWNERS

@@ -1,6 +1,4 @@
 # Default owners for everything in the repo.
-# TODO: replace with the maintainer's GitHub handle or org team before Phase 1.
 * @GetTechAPI/maintainers
 
-/data/      @GetTechAPI/maintainers
 /app/       @GetTechAPI/maintainers

.github/workflows/final-check.yml

@@ -4,7 +4,7 @@ name: final-check
 # runs what test.yml runs (ruff, mypy, the seed-data validator, pytest with
 # coverage) on the PR, and TechEngineBot leaves ONE comment with the verdict;
 # adding the label again edits that comment. This is the only place the
-# TechEngineBot account is used in this repo, and it replaces Copilot reviews.
+# TechEngineBot account is used by a workflow in this repo.
 #
 # The check job runs PR code, so it gets no secrets and a read-only token.
 # Only the comment job sees TECHENGINEBOT_TOKEN, which must be a fine-grained

.github/workflows/ingest.yml

@@ -188,8 +188,9 @@ jobs:
           PY
 
       # The token is read-only, so the commit leaves the run as a patch. A
-      # maintainer opens the PR from their own account with TechAPI's
-      # .github/scripts/open-collect-pr.sh <run-id>, so the PR's checks run.
+      # maintainer runs TechAPI's .github/scripts/open-collect-pr.sh <run-id>:
+      # their account pushes the branch and TechEngineBot opens the PR, so the
+      # PR's checks run.
       - name: Export the commit as collect.patch
         if: steps.commit.outputs.changed == 'true'
         env:

.github/workflows/refresh.yml

@@ -206,8 +206,9 @@ jobs:
           PY
 
       # The token is read-only, so the commit leaves the run as a patch. A
-      # maintainer opens the PR from their own account with TechAPI's
-      # .github/scripts/open-collect-pr.sh <run-id>, so the PR's checks run.
+      # maintainer runs TechAPI's .github/scripts/open-collect-pr.sh <run-id>:
+      # their account pushes the branch and TechEngineBot opens the PR, so the
+      # PR's checks run.
       - name: Export the commit as collect.patch
         if: steps.commit.outputs.changed == 'true'
         env:

.github/workflows/verify-promote.yml

@@ -221,8 +221,9 @@ jobs:
           PY
 
       # The token is read-only, so the commit leaves the run as a patch. A
-      # maintainer opens the PR from their own account with TechAPI's
-      # .github/scripts/open-collect-pr.sh <run-id>, so the PR's checks run.
+      # maintainer runs TechAPI's .github/scripts/open-collect-pr.sh <run-id>:
+      # their account pushes the branch and TechEngineBot opens the PR, so the
+      # PR's checks run.
       - name: Export the commit as collect.patch
         if: steps.commit.outputs.changed == 'true'
         env:

.gitignore

@@ -19,15 +19,16 @@ env/
 *.sqlite
 *.sqlite3
 
+# A TechAPI checkout placed inside this repo (CI does this and sets
+# TECHAPI_DATA_DIR); the dataset is never committed here.
+/TechAPI/
+
 # Generated static dump (build artifact; published by refresh-data workflow)
 /dump/
 
 # Local-only handoff notes (private; never committed to the public repo)
 *.local.md
 
-# Local tooling/editor config (kept out of the public repo)
-.claude/
-
 # Note: data/_staging/ (raw collected candidate pool) is intentionally tracked —
 # comprehensive data collection is a purpose of this repo.
 

.gitmodules

@@ -7,7 +7,9 @@
 
 TechEngine owns everything *around* the data: schema validation, the FastAPI
 read API, the static JSON dump generator, the engine's own landing site,
-automated coverage checks, and the weekly ingestion/refresh crawlers.
+coverage checks, the ingestion/refresh collectors and the verification tiers.
+Nothing here runs on a schedule; collectors run only when a maintainer
+dispatches them.
 
 The dataset and the public-facing playground site live in
 [TechAPI](https://github.com/GetTechAPI/TechAPI) so each can be versioned,
@@ -17,60 +19,89 @@ engine's own landing: what TechEngine is, what it runs, and links out to docs.
 ## Layout
 
 ```text
-app/
-  validate.py          # schema/range/uniqueness checks
-  seed.py              # data/ -> SQLModel database
-  dump.py              # API -> static JSON tree
-  main.py              # FastAPI entrypoint
+app/                   # the engine (the one installable Python package)
+  main.py              # FastAPI entry point; mounts every router under /v1
+  routers/             # /v1 routes, one module per collection (list below)
   models/              # SQLModel tables
-  routers/             # /v1/{brands,socs,smartphones,gpus,cpus}
-  schemas/             # Pydantic response models
-  services/            # scoring (algorithm_version-tagged)
-  coverage/            # upstream-vs-curated diff + Markdown report
-  ingest/              # draft new records from upstream pages
-tests/                 # unit + integration
-site/                  # Astro engine landing (deploys to Pages)
-docs/                  # SPEC / DATA_PIPELINE / DEVELOPMENT
-TechAPI/               # submodule -> GetTechAPI/TechAPI (clickable @ <sha> link)
-.github/workflows/
-  test.yml             # lint + type-check + tests
-  validate-data.yml    # read-only data validation (workflow_call or manual)
-  deploy-pages.yml     # build & deploy engine site + dump
-  coverage-report.yml  # manual: gap report to the job summary + artifact
-  ingest.yml           # workflow_call: draft new SKUs -> collect.patch
-  refresh.yml          # workflow_call: re-scrape benchmarks -> collect.patch
-  verify-promote.yml   # workflow_call: Tier 1-3 checks + verified promotion patch
-  final-check.yml      # label-triggered checks + one TechEngineBot comment
+  schemas/             # Pydantic response models + serializers
+  services/scoring/    # 0-100 scores, weights from config/scoring.yaml
+  data_root.py         # finds the TechAPI dataset (see below)
+  categories.py        # the 13 dataset categories and their collection names
+  validate.py          # schema/range/uniqueness checks on the dataset
+  seed.py              # dataset -> SQL database
+  dump.py              # API -> static JSON tree (./dump)
+  coverage/            # upstream-vs-curated gap report
+  ingest/              # collectors: draft new records, refresh benchmarks
+  verify/              # trust tiers 0-3, verified promotion, backfill tools
+config/scoring.yaml    # scoring weights and reference scales
+integrity_check.py     # whole-dataset anomaly scan (the --strict gate)
+tests/                 # unit/, integration/, verify/
+site/                  # Astro landing page, deployed to GitHub Pages
+docs/                  # SPEC, DATA_PIPELINE, DEVELOPMENT
+.github/workflows/     # CI; every workflow is listed below
+Dockerfile, docker-compose.yml   # stale, do not build (see Docker)
 ```
 
-## How The Two Repos Connect
+Routes under `/v1`: `health`, `version`, `brands`, `socs`, `smartphones`,
+`tablets`, `watches`, `pdas`, `gpus`, `cpus`, `laptops`, `monitors`,
+`software`, `websites`, `device-catalog`. Dataset folders are singular
+(`cpu/`), routes are plural (`/v1/cpus`); `app/categories.py` holds the mapping.
+
+**How the engine finds the dataset.** This repo holds no data. Every entry
+point asks `app/data_root.py`, which returns the first of: the
+`TECHAPI_DATA_DIR` environment variable; `../data` (this repo checked out as
+TechAPI's `TechEngine` submodule); `../TechAPI/data` (the two repos cloned side
+by side). CI checks TechAPI out into `./TechAPI`, which is git-ignored, and
+sets `TECHAPI_DATA_DIR` to its `data/` folder.
+
+Workflows:
+
+```text
+test.yml             # lint + type-check + tests
+validate-data.yml    # read-only data validation (workflow_call or manual)
+deploy-pages.yml     # on push to main: build site/ + static dump -> Pages
+coverage-report.yml  # manual: gap report to the job summary + artifact
+ingest.yml           # workflow_call: draft new SKUs -> collect.patch
+refresh.yml          # workflow_call: re-scrape benchmarks -> collect.patch
+verify-promote.yml   # workflow_call: Tier 1-3 checks + verified promotion patch
+final-check.yml      # label-triggered checks + one TechEngineBot comment
+```
 
-Both repos live in the **GetTechAPI** org and each includes the other as a git
-**submodule** (a clickable `@ <sha>` pin). Nothing syncs them automatically.
+## How The Two Repos Connect
 
-No workflow here runs on a schedule or uses a personal access token (ADR-017).
-The one comment any workflow posts is TechEngineBot's final check, and only when
-a maintainer adds the `final-check` label to a PR (`final-check.yml`).
+Both repos live in the **GetTechAPI** org. TechAPI pins this repo as its
+`TechEngine` git **submodule** (a clickable `@ <sha>` pin); this repo does not
+pin TechAPI back, so there is no submodule loop. Nothing syncs them
+automatically: the pin moves only when a TechAPI commit moves it.
+
+No workflow here runs on a schedule or uses a personal access token (ADR-017),
+and no workflow opens or approves a PR. TechEngineBot opens PRs by hand from a
+maintainer's machine, with its gh login kept apart from the maintainer's own;
+the maintainer's account still commits, pushes and merges. The one comment any
+workflow posts is TechEngineBot's final check, and only when a maintainer adds
+the `final-check` label to a PR (`final-check.yml`).
 Collection runs only when a maintainer dispatches TechAPI's `collect.yml`, which
 calls `ingest.yml`, `refresh.yml` or `verify-promote.yml` with a read-only
 token. Each run commits locally, gates the result with `app.validate` and
 `integrity_check.py --strict`, and uploads the commit as `collect.patch`. A
-maintainer opens the PR from their own account with TechAPI's
-`.github/scripts/open-collect-pr.sh`; no workflow pushes or opens a PR. The
+maintainer then runs TechAPI's `.github/scripts/open-collect-pr.sh` by hand:
+their account pushes the run's `collect/*` branch and TechEngineBot opens the
+PR. No workflow pushes or opens a PR. The
 engine that runs is the TechEngine commit TechAPI pins as its submodule.
 Crawlers identify themselves with a `TechEngine-*` User-Agent.
 
 Every Python entry point reads data from a **TechAPI checkout**. The location
 can be overridden via `TECHAPI_DATA_DIR`; by default TechEngine supports both
-layouts: `TechAPI/TechEngine` as a submodule and `TechAPI` beside `TechEngine`.
+layouts: `TechAPI/TechEngine` as TechAPI's submodule and `TechAPI` beside
+`TechEngine` (see Layout).
 
 ## Quickstart
 
 ```bash
 git clone https://github.com/GetTechAPI/TechAPI.git ../TechAPI   # data source
 pip install -e ".[dev]"
 python -m app.validate          # check data integrity
-python -m app.seed              # data/ -> ./techapi.db (SQLite)
+python -m app.seed              # dataset -> ./techapi.db (SQLite)
 uvicorn app.main:app --reload   # serve; curl localhost:8000/v1/cpus/ryzen-9-9950x3d
 python -m app.dump              # generate ./dump/v1/... static tree
 ```
@@ -82,13 +113,13 @@ seed records may omit them or use `null` when no source value is known, and API
 detail responses and static dumps preserve unknown values as `null`.
 Smartphones continue to require numeric values for both fields.
 
-### Docker Compose (Postgres)
-
-```bash
-docker compose up --build
-```
+### Docker
 
-Spins up Postgres 16, seeds from the mounted TechAPI checkout, serves on `:8000`.
+`Dockerfile` and `docker-compose.yml` predate the split from TechAPI and do
+not work as they are: the Dockerfile copies `data/` and `scripts/`, which no
+longer exist here, so the image does not build, and the compose file mounts no
+TechAPI checkout for `app.seed` to read. No workflow builds or uses them. Use
+the Quickstart above; for Postgres, point `DATABASE_URL` at your own instance.
 
 ## Roadmap