Protocols / NAICS — Industry Classification

6 stepsduckdb >=1.2

The reproducible definition of how the source.naics Dataset is ingested from the U.S. Census 2022 NAICS descriptions workbook. Fetch the .xlsx, read it directly with DuckDB's excel extension (strip the trilateral 'T' marker, derive level, null blanks), materialise, export a reproducible Parquet ordered by code, then describe and gate it.

producessource.naicsNAICS — Industry Classification

Gated against a finetype schema contract

This Protocol’s output is checked against a self-derived finetype schema (schema.finetype.json) by its finetype_validate@1 step (validate). The gate runs in check-only mode: it types every column and verifies the observable envelope — type, length bounds, numeric ranges, closed enum domains and semantic labels. A pass-through on today’s run; a fail-closed drift tripwire for future ones. It writes nothing, so the built Parquet stays byte-identical.

The flow

Every step runs in order. op: steps are typed, versioned operators; sql: steps are DuckDB models; a command step is opaque by design.

01fetch_naicsop: http_fetch@1
Fetch the Census 2022 NAICS descriptions workbook (.xlsx). Carries the content-freshness contract; the 2022 vintage is effectively frozen.
producesbuild/naics_desc.xlsx
with
```
url: https://www.census.gov/naics/2022NAICS/2022_NAICS_Descriptions.xlsx
out: build/naics_desc.xlsx
headers:
  User-Agent: Meridian Protocol (open-analytics; [email protected])
```
02loadsql: models/load.sql
Read the .xlsx directly (DuckDB excel extension, read_xlsx); strip the trilateral 'T' marker, derive the level, null blank descriptions.
readsbuild/naics_desc.xlsx
model
```
models/load.sql
```
03packagesql: models/package.sql
Materialise the terminal table (adds the search corpus).
producesnaics_out
model
```
models/package.sql
```
04export_naicsop: parquet_export@1
Write the terminal Parquet. order_by code (the primary key) is a total order → reproducible bytes.
readsnaics_outproducesbuild/naics.parquet
with
```
input: naics_out
dest: build/naics.parquet
compression: zstd
order_by: code
```
05describeop: datapackage_describe@1
Emit datapackage.json — finetype types every column; descriptor.overrides.json overlays the curated identity, licenses, sources, per-field prose and primaryKey. Hard-fails on drift.
readsbuild/naics.parquetdescriptor.overrides.jsonproducesdatapackage.json
with
```
parquet: build/naics.parquet
overrides: descriptor.overrides.json
out: datapackage.json
```
06validatefinetype gateop: finetype_validate@1
Gate the built Parquet against the self-derived finetype contract. CHECK-ONLY: checks it against schema.finetype.json — a pass-through today, a fail-closed drift tripwire for future runs. Writes nothing; the Parquet stays byte-identical.
readsbuild/naics.parquetschema.finetype.json
with
```
parquet: build/naics.parquet
schema: schema.finetype.json
```