Difference between revisions of "Assets manifest"
Imathrowback (Talk | contribs) (→Table 1 - Entry Table: added missing value) |
Imathrowback (Talk | contribs) (→Table 1 - Entry Table: resolve an unknown field) |
||
| Line 165: | Line 165: | ||
|Index of which PAK file the asset comes from. See Table_0 | |Index of which PAK file the asset comes from. See Table_0 | ||
|- | |- | ||
| − | | | + | |2 bytes |
|short | |short | ||
| + | |Bitfield: bit 1 = Unsure, appears to always be set when an asset is NOT localized. bit 4 - texture/geometry/vfx/model files | ||
| + | |- | ||
| + | |2 bytes | ||
| + | |short | ||
| + | |Bitfield: bit 0 = Core asset, should be downloaded before play. bit 2 - if set indicates a 64bit asset | ||
| + | |- | ||
| + | |1 byte | ||
| + | |byte | ||
|Unknown | |Unknown | ||
|- | |- | ||
Latest revision as of 21:35, 24 June 2018
RIFT assets consist of 2 main parts, the manifest, and the asset files.
RIFT uses a FNV1 hash to hash all filenames. This hash is used in many places when referring to files.
Contents
Manifest
assets.manifest - 32 bit
assets64.manifest - 64 bit
The main identified use of the manifest is to map asset hashes with filename hashes.
However, the manifest contains much more data than that, but it is currently unknown what else it contains.
It is likely that some the other data pertains to the streaming download functionality of RIFT where assets can be streamed while playing.
Binary Format
File should be interpreted as LITTLE ENDIAN.
NOTE: As of the February 23rd, 2018 PTS patch, the Major Version in the manifest has changed from 0x0200 to 0x0300, and the minor version has changed from 0x0100 to 0x0000. The major change is the addition of 2 bytes to the end of each entry in table 1
Header
| Bytes/Offset | Content | Comment |
|---|---|---|
| 4 bytes | TWAM | File signature |
| 2 bytes | 0x0200 - integer | Major version |
| 2 bytes | 0x0100 - integer | Minor version |
| 4 bytes | integer | 256 Block Table offset |
| 4 bytes | integer | 256 Block Table size |
| 48 bytes | varies | 3 Data Table Reference as per below, each 16 bytes long. The second table is the Entry Table as per below. |
Data Table Reference
| Bytes/Offset | Content | Comment |
|---|---|---|
| 4 bytes | integer | offset from start of file |
| 4 bytes | integer | size of the entire table in bytes |
| 4 bytes | integer | count of entries in the table |
| 4 bytes | integer | stride - size of each entry in the table |
Note that the table itself may consist of more data than the header indicates. For example if you multiply the stride by the count and it is less than the size of the table in bytes, then extra data is appended.
Tables
256 Block Table
This table consists of 256 bytes. It is not yet known what these are but it one possibility is some kind of RC4 substitution table.
Table 0 - PAK listing
| Bytes/Offset | Content | Comment |
|---|---|---|
| 4 bytes | integer | Offset of the PAK file string this entry relates to |
| 4 bytes | integer | Size of the uncompressed PAK file when the compression type is zero. |
| 4 bytes | integer | Size of the compressed PAK file when the compression type is not zero. |
| 1 byte | Enum | Compression type: 0 - no compression. 1 - ZLIB. 2 - LZMA2 |
| 20 bytes | SHA1 hash of the file when compression type is 0 | |
| 20 bytes | SHA1 hash of the file when compression type is not zero |
After the normal table data, there are "count" number of null terminated strings that appear to be references to "pak" files.
eg:
assets/audio/audio_0.pak
Table 1 - Entry Table
FNV hash and asset names. No extra data
| Bytes/Offset | Content | Comment |
|---|---|---|
| 8 bytes | varies | ID of the entry - corresponds to the first 8 bytes of a 16byte SHA hash of the contents of the data the entry relates to. |
| 4 bytes | integer | The FNV1 hash of the filename |
| 4 bytes | integer | Offset in the PAK file this asset is at |
| 4 bytes | integer | compressed size of the asset in the PAK file in bytes |
| 4 bytes | integer | size of the asset in bytes |
| 2 bytes | short | Index of which PAK file the asset comes from. See Table_0 |
| 2 bytes | short | Bitfield: bit 1 = Unsure, appears to always be set when an asset is NOT localized. bit 4 - texture/geometry/vfx/model files |
| 2 bytes | short | Bitfield: bit 0 = Core asset, should be downloaded before play. bit 2 - if set indicates a 64bit asset |
| 1 byte | byte | Unknown |
| 1 byte | Enum | Language this asset represents. 0 - universal. 1 - english. 3 - german. 5 - french, 6 - russian |
| 20 bytes | Sometimes is the SHA1 of the asset but otherwise unknown hash. | |
| 4 bytes | int? | Unknown |
| 2 bytes | short | Length in bytes of the unhashed filename for the entry (Asset manifest version 3.0+, added February 23, 2018 to PTS client) |
Table 2 - Unknown
This table contains a smaller table up to the number of entries in the main table definition.
| Bytes/Offset | Content | Comment |
|---|---|---|
| 4 bytes | integer/unsigned? | unknown |
| 4 bytes | integer | count of items |
| 4 bytes | integer | offset from the start of the file. |
Each entry offset points to a collection of integers up to count.
Asset files
Asset files are in the assets directory of the RIFT installation and are labelled like so:
assets.001 assets.002 assets64.001 etc
The assets with 64 indicate the file should be preferred if using the 64bit manifest.
Asset files are packed files, sometimes compressed.
Header
| Bytes/Offset | Content | Comment |
|---|---|---|
| 4 bytes | TWAD | File signature |
| 4 bytes | integer | version |
| 4 bytes | integer | size of header block |
| 4 bytes | integer | maximum number of entries in the file |
| 4 bytes | integer | index of the first in a linked list of entries in the file. Follow them using the next index field. |
Entry Table
Directly after the header is the entry table.
Note that an entry in the table CAN be null entry. When this occurs all of the data in the entry is null/zero and should be skipped when processing.
| Bytes/Offset | Content | Comment |
|---|---|---|
| 8 bytes | ID of the entry - corresponds to the first 8 bytes of a 20byte SHA hash of the contents of the data the entry relates to. | |
| 4 bytes | offset in bytes in the file the entry is at | |
| 4 bytes | size in bytes of the entry | |
| 4 bytes | unsure. possibly amount of data streamed so far if asset is being streamed. | |
| 2 bytes | Index to the NEXT entry in the file. (subtract 1 off it) | |
| 2 bytes | Entry flag. If non-zero, indicates the entry is compressed. 1 - ZLIB. 2 - LZMA2 | |
| 20 bytes | SHA hash of the entry |
