Ptex File Format Specification Version 1.3r1

Revision history:

1.0	9/29/06	Initial Release
1.1	2/27/07	"Edit face data" amended to included precomputed constant value
1.2	1/5/09	Added: subface flag, const neighborhood flag, u/v border modes, separately stored large metadata items. Clarified triangle texture storage.
1.3	10/20/09	Corrected size of "keysize" and "datatype" in meta data spec to match implementation. Revised large metadata spec (not previously implemented). Added "editdatapos" to extended header. Added "compatibility barrier" for pre-1.3 file readers. Added minor version to header.
1.3r1	6/2/10	Corrected details of Constant Data block to match implementation.

General Notes:

A "face" is one face of a polymesh. The polymesh has a base type of either quad or triangle. For quad meshes, ptex stores rectangular textures. For triangle meshes, ptex stores triangular textures.
Non-quads embedded in a quad mesh and non-tris embedded in a tri-mesh are subdivided once to generate quad/tri subfaces and textures are stored per subface.
Every face (or subface) has a unique index, or "faceid", which is implied by the ordering in the "Face Info" block. The faceids are numbered starting from zero. Subfaces are ordered in following the edge order within the parent face.
A "channel" is a scalar data value stored in the file (e.g. one component of a color). There can be any number of channels in the file, but all channels must have the same data type. Channels are stored separately for better compression.
A "level" is a set of data across all the faces at a particular resolution. The first level is the full-res set of data where the resolution of each face is specified in the "Face Info" block. Subsequent levels are power-of-two reductions (i.e. "mip maps").
One channel may optionally be indicated as the alpha channel. When an alpha channel is present, the data values in the first level are expected to be stored "unmultiplied" (as opposed to being premultiplied with the alpha). This choice was made to preserve the original data in the file for subsequent editing as premultiplying can result in data loss. The tradeoff is that the multiplication must now be handled during filtering. Reduction levels are stored premultiplied.
Compression, where indicated, uses the "deflate" algorithm as implemented in zlib's compress and uncompress functions. Differencing (where each value is replaced by the difference between the value and the preceeding value) is used where indicated to achieve better compression.
All data is stored in "little endian" format.
Face data is stored in v-major order with the first sample corresponding to u=v=0. This is considered the bottom-left corner with u pointing right and v pointing up.
Triangle texture data is stored in a specialized layout (documented elsewhere). The texels are packed into a square texture for storage. The number of texels is the same as for quad textures (n-squared texels where n is a power-of-two) though only isotropic textures are supported (i.e. ures = vres). Storage of the square-packed triangle textures is identical to quad textures including tiling, compression, etc.
No padding or data alignment within the file is assumed beyond what is explicitly specified, though an attempt is made to keep data aligned within structures. However, data blocks are not generally aligned on file offsets due to the variable length nature of the zip compression.

General File Structure

The file is composed of a number of data blocks stored in the order listed in the table below:

Header
description of file contents (fixed length, will never grow)

Extended Header
additional header data, may grow in future

Face Info
information about each face

Constant Data pre-filtered, constant valued data for each face

Level Info
information about each level

Level Data
data for each level

Meta Data miscellaneous small data stored as single block

Compatibility Barrier
barrier preventing pre-1.3 file readers from reading new data sections

Large Meta Data
separately stored large meta data items

(future expansion)

Edit Data
incremental edits posted to the file

Header

magic	char[4]	magic identifier = string "Ptex"
version	uint32	major version number (=1)
meshtype	uint32	0=triangle, 1=quad
datatype	uint32	0=uint8, 1=uint16, 2=float16, 3=float32
alphachan	uint32	index of alpha channel -1 (0xffffffff) indicates no alpha channel
nchannels	uint16	number of data channels
nlevels	uint16	number of levels
nfaces	uint32	number of faces
extheader size	uint32	size of Extended Header block
faceinfo size	uint32	size of Face Info block
constdata size	uint32	size of Constant Data block
levelinfo size	uint32	size of Level Info block
minor version	uint32	minor version number
leveldata size	uint64	total size of all Level Data blocks
metadata zipsize	uint32	on-disk size of Meta Data block (zipped)
metadata memsize	uint32	in-memory size of Meta Data block (unzipped)

Extended Header

ubordermode	uint32	0=clamp (default), 1=black, 2=periodic
vbordermode	uint32	0=clamp (default), 1=black, 2=periodic
largemetadata header zipsize	uint32	on-disk size of Large Meta Data Header (zipped)
largemetadata header memsize	uint32	in-memory size of Large Meta Data Header (unzipped)
largemetadata size	uint64	total on-disk size of Large Meta Data blocks (not including header)
editdata size	uint64	size of edit data blocks
editdata pos	uint64	absolute file position of first edit data block

Face Info

Information about the resolution and topological adjacency of each face.

ures	uint8	log2(number of samples in u direction)
vres	uint8	log2(number of samples in v direction)
adjedges	uint8	ids of edges on adjacent faces (0 .. 3) x 4 faces (see notes)
flags	uint8	bit0=constant - all pixels in face have same value bit1=has edits (not stored in file, for api only) bit2=constant neighborhood - all neighboring faces have same value as this face bit3=subface - face is a subface
adjfaces	uint32[4]	faceids of adjacent faces (0 .. nfaces) -1 (0xffffffff) indicates a border (no adjacent face)

(times number of faces)

Notes:

The entire block is zip compressed and the size is indicated in the header.
Constant faces (indicated by flag bit0) are only stored in the Constant Data section and not in the Level Data section.
The "adjedge" values are stored as 2-bit values in little-endian order (i.e the first entry is stored in the lowest 2 bits, etc.).
For triangle meshes, the adjface array has only 3 entries but 4 entries are still allocated to simplify the implementation (the 4th entry in each case is 0).
For a face adjacent to a subface, there will always be two subfaces sharing the edge with the face. The face's adjface id must point to the first subface as encountered in a counter-clockwise traversal of the face.

Constant Data

The constant data block contains a single value per channel for each face. For non-constant faces, these values represent the average of the values for each face.

The per-channel data values are interleaved for each face, and all the faces are packed together in faceid order.
The entire block is zip compressed as a single unit (with no differencing).
For textures with alpha, the values are stored un-multiplied. This is done to exactly preserve the original data values for constant-valued faces.
To determine the average value of a non-constant face, the data are pre-multiplied with alpha, averaged, and then divided by alpha before being stored.

Level Info

The data are grouped into levels with the first level representing the full-res data for all of the faces, the next level representing a power-of-two reduction of the first (half the res in both u and v), and subsequent levels representing half the previous level. The Level Info block lists the number of faces in the level and size of each level's data.

leveldatasize	uint64	size of level's data block (including level data header)
levelheadersize	uint32	compressed size of level data header
nfaces	uint32	number of faces in level

(times number of levels)

Notes:

The first level stores faces in faceid order. Only non-constant faces are stored, though all the faces are included in the level data header.
Reduction levels exclude constant faces and store non-constant faces in largest-to-smallest order (according to the smaller dimension of each face). Ordering among faces with the same smaller-dimension size follows the original face id order. The purpose of this ordering is to allow each level to remain fully populated as the smallest faces may be omitted from the level.
The number of faces in reduction levels may be less than the total number of faces in the file:

The level data only includes non-constant faces (as indicated by the "constant" flag in the face data header).
As face resolutions are reduced, small faces may be omitted from further reduction levels.

Level Data

Each Level Data block (one per level) consists of a header containing a list of face sizes followed by a number of face data blocks of the given sizes. Non-constant face data is packed (with channels separated) and then optionally differenced and compressed.

Level data header (compressed):

facesize	uint32 (bits 0..29)	sizes of data block for each face
encoding	bits 30..31 of facesize	0 = constant valued (1 sample per channel) 1 = zip compressed 2 = zip compressed w/ differencing 3 = tiled

(times number of faces in level)

For tiled faces, the face data begins with the following pre-header:

tileures	uint8	log2(number of samples per tile in u direction)
tilevres	uint8	log2(number of samples per tile in v direction)
tileheadersize	uint32	compressed size of tile header

and is followed by a compressed header listing the tile sizes and encodings:

tilesizes	uint32 (bits 0..29)	sizes of data block for each tile
encoding	bits 30..31 of tilesize	0 = constant valued (1 sample per channel) 1 = zip compressed 2 = zip compressed w/ differencing

(times number of tiles in face)

Each tile is then stored separately (with tiles in v-major order) in the same manner as the untiled face data. Note: the number of tiles for a face is determined by comparing the tile_ures and tile_vres with the ures and vres of the face.

Meta Data

Miscellaneous application defined data. Possibilities include channel names, timestamp, software settings, 3d vertex data, etc. The meta data is a stream of individual data blocks.

Meta data block

keysize	uint8	length of key string (including null)
key	uint8[keysize]	ascii character string (null-terminated)
datatype	uint8	0=ascii, 1=int8, 2=int16, 3=int32, 4=float, 5=double
datasize	uint32	size of data block (in bytes)
data	uint8[datasize]	data block

The entire stream of data blocks is compressed as a whole. The compressed and uncompressed sizes are both stored in the header.
Ascii characters strings (both the key and "ascii" data values) are stored null-terminated. The size includes the null character.

Compatibility Barrier

File versions prior to 1.3 lacked the "editdata pos" extended header attribute. Early versions of the file reader would seek to the end of the known data blocks and assume any additional data blocks were edit blocks. The compatibility barrier appears to old reader code as an empty edit block and prevents the old reader from misinterpreting newer data sections. The compatibility barrier is always written.

barrier

uint64

must be zero

Large Meta Data

The large meta data section includes meta data blocks that are stored separately for individual access (presumably because the items are large, though there's no real limit). The large meta data section consists of a header followed by a series of individually compressed blocks.

Large meta data header (compressed)

keysize	uint8	length of key string (including null)
key	uint8[keysize]	ascii character string (null-terminated)
datatype	uint8	0=ascii, 1=int8, 2=int16, 3=int32, 4=float, 5=double
datasize	uint32	size of data block (in bytes)
zipsize	uint32	size of data block on disk

(times number of large meta data blocks)

The individual compressed data blocks follow.

Edit Data

The edit data block includes incremental data edits that are pending to be applied to the file. The format is designed to allow appending edit blocks to a file without otherwise affecting the file.

Edit data block

edittype	uint8	1=edit face data, 2=edit meta data
editsize	uint32	size of editdata block
editdata	(variable)	based on edit type

Edit face data

faceid	uint32	index of face
faceinfo	FaceInfo block	(as described in FaceInfo section)
facesize & encoding	uint32	(as described in Level Data section)
constval	uint8[pixelsize]	constant value (average pixel value, as described in Constant Data section)
facedata	uint8[facesize]	encoded data block

Edit meta data

metadatazipsize	uint32	compressed size of meta data
metadatamemsize	uint32	uncompressed size of meta data
metadata	uint8[metazipsize]	one or more data blocks (as described in Meta Data section)

Header	description of file contents (fixed length, will never grow)
Extended Header	additional header data, may grow in future
Face Info	information about each face
Constant Data	pre-filtered, constant valued data for each face
Level Info	information about each level
Level Data	data for each level
Meta Data	miscellaneous small data stored as single block
Compatibility Barrier	barrier preventing pre-1.3 file readers from reading new data sections
Large Meta Data	separately stored large meta data items
(future expansion)
Edit Data	incremental edits posted to the file