Shapefile shape format (.shp)

The main file (.shp) contains the geometry data. Geometry of a given feature is stored as a set of vector coordinates.^[1]: 5 The binary file consists of a single fixed-length header followed by one or more variable-length records. Each of the variable-length records includes a record-header component and a record-contents component. A detailed description of the file format is given in the ESRI Shapefile Technical Description.^[1] This format should not be confused with the AutoCAD shape font source format, which shares the .shp extension.

The 2D axis ordering of coordinate data assumes a Cartesian coordinate system, using the order (X Y) or (Easting Northing). This axis order is consistent for Geographic coordinate systems, where the order is similarly (longitude latitude). Geometries may also support 3- or 4-dimensional Z and M coordinates, for elevation and measure, respectively. A Z-dimension stores the elevation of each coordinate in 3D space, which can be used for analysis or for visualisation of geometries using 3D computer graphics. The user-defined M dimension can be used for one of many functions, such as storing linear referencing measures or relative time of a feature in 4D space.

The main file header is fixed at 100 bytes in length and contains 17 fields; nine 4-byte (32-bit signed integer or int32) integer fields followed by eight 8-byte (double) signed floating point fields:

The file then contains any number of variable-length records. Each record is prefixed with a record header of 8 bytes:

Following the record header is the actual record:

The variable-length record contents depend on the shape type, which must be either the shape type given in the file header or Null. The following are the possible shape types:

Shapefile shape index format (.shx)

The index contains positional index of the feature geometry and the same 100-byte header as the .shp file, followed by any number of 8-byte fixed-length records which consist of the following two fields:

Using this index, it is possible to seek backwards in the shapefile by, first, seeking backwards in the shape index (which is possible because it uses fixed-length records), then reading the record offset, and using that offset to seek to the correct position in the .shp file. It is also possible to seek forwards an arbitrary number of records using the same method.

It is possible to generate the complete index file given a lone .shp file. However, since a shapefile is supposed to always contain an index, doing so counts as repairing a corrupt file.^[2]

Shapefile attribute format (.dbf)

This file stores the attributes for each shape; it uses the dBase IV format. The format is public knowledge, and has been implemented in many dBase clones known as xBase. The open-source shapefile C library, for example, calls its format "xBase" even though it's plain dBase IV.^[3]

The names and values of attributes are not standardized, and will be different depending on the source of the shapefile.

Shapefile spatial index format (.sbn)

This is a binary spatial index file, which is used only by Esri software. The format is not documented by Esri. However it has been reverse-engineered and documented by the open source community. The 100-byte header is similar to the one in .shp.^[4] It is not currently implemented by other vendors. The .sbn file is not strictly necessary, since the .shp file contains all of the information necessary to successfully parse the spatial data.

Topology and the shapefile format

The shapefile format does not have the ability to store topological information. The ESRI ArcInfo coverages and personal/file/enterprise geodatabases do have the ability to store feature topology.

Spatial representation

The edges of a polyline or polygon are composed of points. The spacing of the points implicitly determines the scale at which the feature is useful visually. Exceeding that scale results in jagged representation. Additional points would be required to achieve smooth shapes at greater scales. For features better represented by smooth curves, the polygon representation requires much more data storage than, for example, splines, which can capture smoothly varying shapes efficiently. None of the shapefile format types supports splines.

Data storage

The size of both .shp and .dbf component files cannot exceed 2 GB (or 2³¹ bytes) — around 70 million point features at best.^[5] The maximum number of feature for other geometry types varies depending on the number of vertices used.

The attribute database format for the .dbf component file is based on an older dBase standard. This database format inherently has a number of limitations:^[5]

• While the current dBase standard, and GDAL/OGR (the main open source software library for reading and writing shapefile format datasets) support null values, ESRI software represents these values as zeros — a very serious issue for analyzing quantitative data, as it may skew representation and statistics if null quantities are represented as zero
• Poor support for Unicode field names or field storage
• Maximum length of field names is 10 characters
• Maximum number of fields is 255
• Supported field types are: floating point (13 character storage), integer (4 or 9 character storage), date (no time storage; 8 character storage), and text (maximum 254 character storage)
• Floating point numbers may contain rounding errors since they are stored as text

Mixing shape types

Because the shape type precedes each geometry record, a shapefile is technically capable of storing a mixture of different shape types. However, the specification states, "All the non-Null shapes in a shapefile are required to be of the same shape type." Therefore, this ability to mix shape types must be limited to interspersing null shapes with the single shape type declared in the file's header. A shapefile must not contain both polyline and polygon data, for example, the descriptions for a well (point), a river (polyline), and a lake (polygon) would be stored in three separate datasets.