Map data·5 min read

What Is a Shapefile? It Is a Set of Files, Not One

A shapefile is the most common vector format in GIS and it is not a file. It is three or four files that share a name, and the one you forgot to send is usually the one that decides where the map lands.

5 November 20265 min read

Nothing called a shapefile exists

The name comes from a 1990s Esri specification and describes a family, not an object. What you download is a folder of files sharing one basename: counties.shp, counties.shx, counties.dbf, counties.prj. Rename any one of them and the set stops working. Zip the folder and you have something you can send; zip the .shp alone and you have a file that draws nothing.

Two of those are structural (.shp, .shx), two are what make the map usable (.dbf, .prj), and the rest are conveniences that nobody notices until a name arrives as Muñoz.

The header is 100 bytes, and the first four are 9994

Every .shp opens with a fixed 100-byte header. The first four bytes are a big-endian integer 9994 (hex 0x0000270A) — the format's only magic number — and bytes 24 to 27 hold the file length. That length is counted in 16-bit words, not bytes, which is why a reader written from memory reports a file half its real size.

import struct
from pathlib import Path

head = Path("cb_2025_us_cbsa_5m.shp").open("rb").read(100)

magic,  = struct.unpack(">i", head[0:4])     # 9994, always
length, = struct.unpack(">i", head[24:28])   # in 16-bit words
kind,   = struct.unpack("<i", head[32:36])   # 5 = polygon
bbox    = struct.unpack("<4d", head[36:68])  # xmin, ymin, xmax, ymax

print(magic, length * 2, kind, bbox)
# 9994 1555036 5 (-160.244953, 17.878309, -65.560725, 65.440492)

The header is the one part of the format where the byte order splits: the file-level integers are big-endian, and everything after byte 100 — shape types, bounding boxes, every coordinate — is little-endian. Read the bounding box with the wrong one and you get four numbers that are not coordinates at all.

Those four numbers are also a free sanity check. 1,555,036 is exactly the size of that file on disk, so the length field is honest; shape type 5 says every record is a polygon; the box runs from Hawaii out to Puerto Rico and up to northern Alaska, which is what a national file should say. If your own file's declared length does not match its size, the download was truncated — no reader will fix that.

The .shx is why the file opens at all

The index file has no content of its own. It is a 100-byte header followed by one 8-byte entry per feature — the record's offset and its length, both in 16-bit words — so its size is arithmetic rather than a coincidence.

shx = Path("cb_2025_us_cbsa_5m.shx")
print(shx.stat().st_size)                    # 7580
print((shx.stat().st_size - 100) // 8)       # 935 features

That is the whole index: 100 + 8 × 935 = 7,580 bytes. A reader uses it to jump straight to the thousandth record instead of walking the previous nine hundred and ninety-nine. Delete the .shx and some software will rebuild it, some will warn, and some will simply say the file is not a shapefile — the extension is .shp but the format is the set.

The .prj is the file people lose

A .prj is a single line of well-known text, no trailing newline:

GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",SPHEROID["GRS_1980",6378137,298.257222101]],PRIMEM["Greenwich",0],UNIT["Degree",0.017453292519943295]]

That is the entire file for the 2025 national boundary set — 165 bytes, one line, no trailing newline.

It says which coordinate system the numbers are in, and nothing else in the set records that. Without it, a reader has to guess, and the usual guess is longitude and latitude on WGS84. For the Census national files that guess is close enough to be harmless: they are NAD83, which differs from WGS84 by a meter or two across the continental United States — invisible on a slide, not invisible if you are measuring a right of way.

The guess stops being harmless when the file is projected. A layer in State Plane feet carries coordinates in the millions; read as degrees, those numbers are off the planet, and most tools either refuse the file or drop every shape at a single point in the Gulf of Guinea. If a colleague sends you a map that "opens but is empty", the .prj is the first thing to ask for.

What the .dbf will not do

The attribute table is a dBase III file, a format from 1984 with the limits still in place:

• Field names stop at 10 characters. A column written as MEDIAN_HOUSEHOLD_INCOME arrives truncated, and each exporter truncates it its own way.
• 255 fields, maximum. A wide join has to be split across files.
• There is no NULL. An empty number is a blank field or a zero, and the format cannot tell you which. A county with no measurement and a county measured at zero occupy the same cell.
• Numbers are text with a declared width. Precision is lost when the file is written, not when it is read: a value stored in a field with two decimals is already rounded in the bytes you have.
• A date is YYYYMMDD and nothing more. No time of day, no zone.
• Text encoding is declared outside the file, in the .cpg. When it is missing, the reader guesses and an accent becomes two wrong characters.

None of this is a reason to avoid the format. It is a reason to check the table before you trust the numbers in it.

Text that looks like a number

GEOID is stored as a five-character text field. Autauga County, Alabama is 01001; let a tool infer the type on the way in and it becomes 1001, which then matches nothing in any other table. The shapefile has the type right — text — and the importer throws it away.

This is the spreadsheet join failure one layer down, and the fix is the same: read codes as strings, and check the join for rows that came back empty rather than trusting the row count. What the numbers on a map mean covers the codes themselves; the failure here is in the reader.

Why a slide deck is built from one of these

The geometry is the .shp and the labels are the .dbf. A deck that you can edit is those two joined and converted: one PowerPoint shape per polygon, the name from the .dbf attached to it, no import step and no .prj to lose. That is the only reason the conversion is worth doing — a picture of a map is one object; the county map is every county as its own shape, and the state map is the same idea one level up.

If you would rather shade the shapes you already have than start from a boundary file, the choropleth walkthrough uses the same shapes and stops at the fill.

Questions