Assets manifest

From The Ghar Station Wiki
Revision as of 07:41, 21 June 2017 by Imathrowback (Talk | contribs)

Jump to: navigation, search

RIFT assets consist of 2 main parts, the manifest, and the asset files.

RIFT uses a FNV1 hash to hash all filenames. This hash is used in many places when referring to files.


Manifest

assets.manifest - 32 bit

assets64.manifest - 64 bit

The main identified use of the manifest is to map asset hashes with filename hashes.

However, the manifest contains much more data than that, but it is currently unknown what else it contains.

It is likely that some the other data pertains to the streaming download functionality of RIFT where assets can be streamed while playing.

Binary Format

File should be interpreted as LITTLE ENDIAN.

Header

Bytes/Offset Content Comment
4 bytes TWAM File signature
2 bytes 0x0200 - integer Major version
2 bytes 0x0100 - integer Minor version
4 bytes integer 256 Block Table offset
4 bytes integer 256 Block Table size
48 bytes varies 3 Data Table Reference as per below, each 16 bytes long. The second table is the Entry Table as per below.


Data Table Reference
Bytes/Offset Content Comment
4 bytes integer offset from start of file
4 bytes integer size of the entire table in bytes
4 bytes integer count of entries in the table
4 bytes integer stride - size of each entry in the table

Note that the table itself may consist of more data than the header indicates. For example if you multiply the stride by the count and it is less than the size of the table in bytes, then extra data is appended.

Tables

256 Block Table

This table consists of 256 bytes. It is not yet known what these are but it one possibility is some kind of RC4 substitution table.

Table 0 - PAK listing
Bytes/Offset Content Comment
4 bytes integer Offset of the PAK file string this entry relates to
4 bytes integer Size of the uncompressed PAK file when the compression type is zero.
4 bytes integer Size of the compressed PAK file when the compression type is not zero.
1 byte Enum Compression type: 0 - no compression. 1 - ZLIB. 2 - LZMA2
20 bytes SHA1 hash of the file when compression type is 0
20 bytes SHA1 hash of the file when compression type is not zero


After the normal table data, there are "count" number of null terminated strings that appear to be references to "pak" files.

eg:

assets/audio/audio_0.pak

Table 1 - Entry Table

FNV hash and asset names. No extra data

Bytes/Offset Content Comment
8 bytes varies ID of the entry - corresponds to the first 8 bytes of a 16byte SHA hash of the contents of the data the entry relates to.
4 bytes integer The FNV1 hash of the filename
4 bytes integer Offset in the PAK file this asset is at
4 bytes integer compressed size of the asset in the PAK file in bytes
4 bytes integer size of the asset in bytes
2 bytes short Index of which PAK file the asset comes from. See Table_0
5 bytes short Unknown
1 byte Enum Language this asset represents. 0 - universal. 1 - english. 3 - german. 5 - french, 6 - russian
20 bytes Sometimes is the SHA1 of the asset but otherwise unknown hash.
Table 2 - Unknown

This table contains a smaller table up to the number of entries in the main table definition.

Bytes/Offset Content Comment
4 bytes integer/unsigned? unknown
4 bytes integer count of items
4 bytes integer offset from the start of the file.

Each entry offset points to a collection of integers up to count.

Asset files

Asset files are in the assets directory of the RIFT installation and are labelled like so:

 assets.001
 assets.002
 assets64.001
 etc

The assets with 64 indicate the file should be preferred if using the 64bit manifest.

Asset files are packed files, sometimes compressed.

Header

Bytes/Offset Content Comment
4 bytes TWAD File signature
4 bytes integer version
4 bytes integer size of header block
4 bytes integer maximum number of entries in the file
4 bytes integer index of the last entry. Note that this is NOT always the last entry in the file. The "index" of the entry will indicate the total number of entries in the file.

Entry Table

Directly after the header is the entry table.

Note that an entry in the table CAN be null entry. When this occurs all of the data in the entry is null/zero and should be skipped when processing.

Bytes/Offset Content Comment
8 bytes ID of the entry - corresponds to the first 8 bytes of a 20byte SHA hash of the contents of the data the entry relates to.
4 bytes offset in bytes in the file the entry is at
4 bytes size in bytes of the entry
4 bytes unsure. possibly amount of data streamed so far if asset is being streamed.
2 bytes Index to the NEXT entry in the file. (subtract 1 off it)
2 bytes Entry flag. If non-zero, indicates the entry is compressed. 1 - ZLIB. 2 - LZMA2
20 bytes SHA hash of the entry