I have developed a new WIi ISO disc format:
WIA = Wii ISO Archive
This article describes the background and some aspects of the implementation.
Classic File formats
A PLAIN ISO file stores the content of the DVD byte by byte. It has a usual size of 4.7 GB. Most WBFS and ISO tools are able to detect unused areas (=holes) and fill them with a zeros. This method is called scrubbing. There are 4 classic formats:
You see, that WDF, CISO and WBFS are similar small, because they do almost the same with different containers. WDF is the smallest format, but not playable by USB loaders. I set the WDF size to 100%.
Compressing with BZIP2 and RAR
ISO images can be compressed with with tools like bzip2 or rar. But this is very ineffective because most of the data is encrypted and encrypted data looks like random data and compressors can't compress random data. See this table:
WDF is a little better as BIZP2 or RAR, but don't need as much time as the compressors. This is the main reason that I had developed WDF. WDF manage the holes and allow random access and is easy expandable.
Decrypted images
3-4 month ago I made test with decrypted images. If encrypting a decrypted images the result is the original ISO image. The advantage of decrypted images are that data don't looks like random data and that make decrypted images compressible:
Since that time I know that decrypting and compressing is the best way to keep Wii discs small. Decrypted images contain all the hash data which takes 3% of the partition data. Hash data looks also like random data and is not compressible. But the hash values are reproducible from the war data. So the best is to throw it away.
WIA = Wii ISO Archive
With all this knowledge I have designed a new Wii disc image format: WIA = Wii ISO Archive. My main interest is to find out, what is possible. I don't know If I ever convert my few games into WIA because the handling is much slower than WDF and I have enough space on my drives. The WIT tools already support this new format, long term tests are running and a first beta will be released in the next days.
A WIA archive allows random access reading and is implemented in the file layer of the WIT tools. The WIA files can be read and written like the other supported formats (ISO,CISO,WDF,WBFS). Only the direct editing and patching is not possible. Converting a WIA into a PLAIN ISO restores all non scrubbed data including wrong hash values.
I support 2 compression formats: Uncompressed and compressed with bzip2 (default). My decision for bzip2 was easy: Easy to use lib for all systems and good compression ratio for small (2MB) chunks.
Here is the size statistic:
The WIA compression is really good and an additional compression with BZIP2 or RAR has no positive effect. The smallest images are made by creating uncompressed WIA images and then compressing it with an external tool. You see also that WIA/NOCOMPRESS is better than WDF/DECRYPT because of throwing away unneeded hash values.
Summary
Last not least all formats in one table:
Other images
Animal crossing is an extreme example. The image is reducible to 23%. Other games are not so good for compression. The next table shows the WIA rates of some other games. And remember: 100% is the WDF size of a scrubbed image.
WIA = Wii ISO Archive
This article describes the background and some aspects of the implementation.
Classic File formats
A PLAIN ISO file stores the content of the DVD byte by byte. It has a usual size of 4.7 GB. Most WBFS and ISO tools are able to detect unused areas (=holes) and fill them with a zeros. This method is called scrubbing. There are 4 classic formats:
- PLAIN ISO (1:1 copy)
- CISO (Compressed ISO, ignore unused blocks)
- WBFS (Multi ISO container, ignore unused blocks)
- WDF (Wii Disc File, mange holes directly)
Code:
-----------------------------------------------
size percent creation time format
-----------------------------------------------
352695380 100.00% 8.939 sec WDF
358645760 101.68% 14.946 sec CISO
360448000 102.19% 8.842 sec WBFS
4699979776 1332% 8.798 sec PLAIN ISO
-----------------------------------------------You see, that WDF, CISO and WBFS are similar small, because they do almost the same with different containers. WDF is the smallest format, but not playable by USB loaders. I set the WDF size to 100%.
Compressing with BZIP2 and RAR
ISO images can be compressed with with tools like bzip2 or rar. But this is very ineffective because most of the data is encrypted and encrypted data looks like random data and compressors can't compress random data. See this table:
Code:
------------------------------------------------------------
size percent creation time format
------------------------------------------------------------
352695380 100.00% 8.939 sec WDF
353345496 100.18% 6:24.597 m:s WDF + RAR
354147564 100.41% 6:04.515 m:s PLAIN ISO + BZIP2
354162098 100.41% 3:19.670 m:s WDF + BZIP2
355472608 100.78% 12:52.996 m:s PLAIN ISO + RAR
4699979776 1332% 8.798 sec PLAIN ISO
------------------------------------------------------------WDF is a little better as BIZP2 or RAR, but don't need as much time as the compressors. This is the main reason that I had developed WDF. WDF manage the holes and allow random access and is easy expandable.
Decrypted images
3-4 month ago I made test with decrypted images. If encrypting a decrypted images the result is the original ISO image. The advantage of decrypted images are that data don't looks like random data and that make decrypted images compressible:
Code:
------------------------------------------------------------
size percent creation time format
------------------------------------------------------------
84717686 24.02% 2:40.267 m:s WDF/DECRYPT + RAR
133009483 37.71% 2:40.254 m:s WDF/DECRYPT + BZIP2
330201884 93.62% 25.208 sec WDF/DECRYPT
352695380 100.00% 8.939 sec WDF
353345496 100.18% 6:24.597 m:s WDF + RAR
354162098 100.41% 3:19.670 m:s WDF + BZIP2
------------------------------------------------------------WIA = Wii ISO Archive
With all this knowledge I have designed a new Wii disc image format: WIA = Wii ISO Archive. My main interest is to find out, what is possible. I don't know If I ever convert my few games into WIA because the handling is much slower than WDF and I have enough space on my drives. The WIT tools already support this new format, long term tests are running and a first beta will be released in the next days.
A WIA archive allows random access reading and is implemented in the file layer of the WIT tools. The WIA files can be read and written like the other supported formats (ISO,CISO,WDF,WBFS). Only the direct editing and patching is not possible. Converting a WIA into a PLAIN ISO restores all non scrubbed data including wrong hash values.
I support 2 compression formats: Uncompressed and compressed with bzip2 (default). My decision for bzip2 was easy: Easy to use lib for all systems and good compression ratio for small (2MB) chunks.
Here is the size statistic:
Code:
------------------------------------------------------------
size percent creation time format
------------------------------------------------------------
79942145 22.66% 2:41.197 m:s WIA/NOCOMPRESS + RAR
84717686 24.02% 2:40.267 m:s WDF/DECRYPT + RAR
127436075 36.13% 2:36.284 m:s WIA/NOCOMPRESS + BZIP2
127603521 36.17% 4:43.668 m:s WIA + RAR
127696363 36.20% 2:31.820 m:s WIA
128007670 36.29% 3:40.140 m:s WIA + BZIP2
133009483 37.71% 2:40.254 m:s WDF/DECRYPT + BZIP2
312976812 88.73% 27.094 sec WIA/NOCOMPRESS
330201884 93.62% 25.208 sec WDF/DECRYPT
352695380 100.00% 8.939 sec WDF
353345496 100.18% 6:24.597 m:s WDF + RAR
354162098 100.41% 3:19.670 m:s WDF + BZIP2
------------------------------------------------------------Summary
Last not least all formats in one table:
Code:
Summary of RUUP01, Animal Crossing: Let's Go to the City:
------------------------------------------------------------
size percent creation time format
------------------------------------------------------------
79942145 22.66% 2:41.197 m:s WIA/NOCOMPRESS + RAR
84717686 24.02% 2:40.267 m:s WDF/DECRYPT + RAR
127436075 36.13% 2:36.284 m:s WIA/NOCOMPRESS + BZIP2
127603521 36.17% 4:43.668 m:s WIA + RAR
127696363 36.20% 2:31.820 m:s WIA
128007670 36.29% 3:40.140 m:s WIA + BZIP2
133009483 37.71% 2:40.254 m:s WDF/DECRYPT + BZIP2
312976812 88.73% 27.094 sec WIA/NOCOMPRESS
330201884 93.62% 25.208 sec WDF/DECRYPT
352695380 100.00% 8.939 sec WDF
353345496 100.18% 6:24.597 m:s WDF + RAR
354147564 100.41% 6:04.515 m:s PLAIN ISO + BZIP2
354162098 100.41% 3:19.670 m:s WDF + BZIP2
355472608 100.78% 12:52.996 m:s PLAIN ISO + RAR
358645760 101.68% 14.946 sec CISO
360448000 102.19% 8.842 sec WBFS
4699979776 1332% 8.798 sec PLAIN ISO
------------------------------------------------------------Other images
Animal crossing is an extreme example. The image is reducible to 23%. Other games are not so good for compression. The next table shows the WIA rates of some other games. And remember: 100% is the WDF size of a scrubbed image.
Code:
8054276 31.98% 24.213 sec WIA RTYP01, Wii Chess
127696363 36.20% 3:00.483 m:s WIA RUUP01, Animal Crossing
531722793 80.26% 5:25.284 m:s WIA RSPP01, Wii Sports
153673062 86.74% 1:35.435 m:s WIA RHAP01, Wii Play
1010766389 87.80% 9:36.026 m:s WIA RZDP01, Twilight Princess
2567636439 92.60% 25:50.572 m:s WIA RMCP01, Mario Kart Wii
