parquet-shape-profiler
description (verbatim frontmatter)
Compute a structured, typed data-shape profile (DatasetShapeProfile: row/column counts, file size, per-column dtype + kind + null rate, numeric percentiles [min/p25/median/p75/max/mean] or categorical distinct-count + top-values) for any parquet dataset, as validated Pydantic JSON — `uv run shape-profiler <path.parquet>` (add `--out profile.json` to write to a file). Every number in the output is a REAL computed statistic, never a placeholder — the same "capture is source-shaped, never invented" discipline the sibling corpus-cleaning-pipeline skill applies to FilterReason. This skill owns COMPUTING the numbers; the sibling plugin cwc-design's `data-shape-svg` skill owns rendering the resulting JSON as SVG markup — a deliberate cross-plugin split (profiler computes, design draws) first proven end-to-end on the xAI take-home's real 100MB sample and rendered live on agenttables.com. Use when the user wants to profile a parquet file's shape/structure, needs real column statistics before designing filters or visualizations, or asks "what does this data look like". Do NOT use to render the visualization itself (data-shape-svg's territory), to clean/filter the data (corpus-cleaning-pipeline), or for Coworkers board/session telemetry (session-telemetry-erd).
summary
Compute a structured, typed data-shape profile (DatasetShapeProfile: row/column counts, file size, per-column dtype + kind + null rate, numeric percentiles [min/p25/median/p75/max/mean] or categorical distinct-count + top-values) for any parquet dataset, as validated Pydantic JSON — `uv run shape...
source
plugins/cwc-data-engineering/skills/parquet-shape-profiler/SKILL.md
provenance
created 2026-07-10 18:13:13 · JSON