File Filters

1. Introduction

Some commands (for example »wit FILES«, »wit EXTRACT«, »wit DIFF« and »wit MIX«) will operate on file level within a Wii disc image. The options --files, --rm-files, --zero-files and --ignore-files and also the »wit MIX« qualifier 'ignore' accept pattern rules to select files for the specific operation.

This article describes the syntax of these rules.

2. Rule lists and syntax

For each option or qualifier a rule list is created. A rule list is expected with 1 or more rules as parameter. Rules are separated by a semicolon (';'). Each rule is appended to the end of the rule list. If an option or qualifier is used multiple times, the new rules are appended to the end of the rule list; rules are never removed or overwritten.

Rule Syntax:

	rule_list  := rule [ ';' rule_list ]
	rule       := std_rule | skip_rule | macro_rule
	std_rule   := sign pattern
	skip_rule  := number sign pattern
	macro_rule := ':' name
	sign       := '+' | '-'
	pattern    := [char]...
	name       := 'a'..'z' ['a'..'z']...
	number     := '1'..'9' ['0'..'9']...

There are 3 kinds of rules: standard rules, skip rules and macro rules. The first character of a rule decide the kind of the rule:

Rules beginning with a plus or minus sign are standard rules.
Rules beginning with one or more digits are skip rules.
Rules beginning with a colon are macro rules.

2.1 Rule files

If a parameter begins with an at sign ('@') the characters behind are a filename. The file is opened and every non empty line is interpreted as exact one rule and appended to the end of the rule list. LF and CR+LF are accepted as end of line marker.

3. Pattern

'pattern' is a file pattern including wildcards. The design is like rsync.

The rule patterns are compared with the source path inside the disc image. A source path looks like 'DIRECTORY/FILE.EXT' without the partition prefix (example: sys/boot.bin).

The '/' at the beginning of the pattern is handled like a '^' in regular expressions. If the pattern starts with a '/' then it is matched against the beginning of the source path. Otherwise it is matched against the end of the source path. To force matching against the end use a '$' as last character in the pattern.

3.1 Special characters

chars	description
/	The slash has only a special meaning as very first charcter of a pattern: Match the pattern against the beginning of the source path.
$	The dollar sign has only a special meaning as very last charcter of a pattern: Match the pattern against the end of the source path. This is the default if no slash ('/') is at the beginning or the end of the pattern.
*	Match any number (zero or more) of characters but not the slash ('/').
**	Match any number (zero or more) of characters including the slash ('/').
#	Match any number (one or more) of digits ('0'..'9').
?	Match exact 1 character but not a slash ('/').
SPACE	A SPACE (ASCII 32) match a SPACE and any control character (ASCII 1..32).
[..]	Match 1 character of the list within the brackets. The list is analyzed in the following way: If the first character is a '^' then characters not in the list are matched. If the first character (behind optional '^') is a '+' or a '*' then it is matched against the maximum possible characters. A '+' means that at least one character must match. The list contains single characters 'c' or ranges 'x-y' to allow this characters. To use a ']' it must be the first character. To use a '-' it must be the first or last character.
{..}	This is a comma separated list of alternatives. It matches if at least one alternative match. '{..}' can be used recursively.
\c	Any character preceded by back slash ('\') looses its special meaning and is handled as normal character.

3.2 Examples

pattern	description
.wad	Pattern matches all files ending with '.wad'.
*/	Pattern matches all directories but no true files.
dir/	Pattern matches all directories which name ends with 'dir'.
dir/+	Pattern matches all files in directories which name ends with 'dir'.
dir/**	Pattern matches all files and directories in (sub-)directories which name ends with 'dir'.
/dir/	Pattern matches all files in (sub-)directories of a directory with name 'dir'.
/dir/ /dir/* /dir/**	Pattern matches all files in the first level directory 'dir' including sub directories.
/dir/*$	Pattern matches all files in in the first level directory 'dir' excluding sub directories.
/dir/*.wad	Pattern matches all '.wad' files in the first level directory 'dir' excluding sub directories. It matches also all files in directories named .wad.
/dir/**.wad$	Pattern matches all '*.wad' files in in the first level directory 'dir' including sub directories.

4. Processing the rule list

Each parameter defines a list of rules (see above). Each file is checked against every rule on the list until a pattern matches the file path. If the rule match the file path the rule prefix decide what to do. A plus ('+') means include the file and a minus ('-') means exclude the file.

If the sign ('+' or '-') is preceded by a decimal number the rule is a skip rule : If the sign is '+' and the pattern does match then the NUMBER of the following rules are skipped. If the sign is '-' and the pattern does not match then the NUMBER of the following rules are skipped. It acts like an IF..ENDIF clause.

If no rule of the complete rule set match the path, the last entered rule in the list decide what to do (opposite handling): If the last entered rule is a plus rule than the file is excluded. If the last entered rule is a minus rule than the file is included.

A empty list means: Include all files.

5. Rule macros

Rule macros are predefined rules to make live easier. Some of them exists as positive (without prefix 'no') and as negative (with prefix 'no') rule.

pattern	description
:negate	This special macro don't add a rule. If this macro is set once or more, all results are negated: The meaning of plus and minus rules, but not of skip rules, is swapped.
:base :nobase	Allow or deny files of the base directory (ticket.bin, tmd.bin, cert.bin, h3.bin).
:disc :nodisc	Allow or deny files from the 'disc/' directory. (disc/header.bin, disc/region.bin)
:sys :nosys	Allow or deny files from the 'sys/' directory. (sys/boot.bin, sys/bi2.bin, sys/apploader.img, sys/main.dol, sys/fst.bin)
:files :nofiles	Allow or deny files from the 'files/' directory. This are the real files of the ISO image.
:wit :wwt	Allow files that are used by wit and wwt to compose an ISO image.
:compose	Allow files that are needed by wit and wwt to compose an ISO image. This is like ':wit' expect that the optional files (/disc/*, tmd.bin and ticket.bin) are not included.
:sneek	Allow files that SNEEK needs.

6. Rule set examples

rule set	description
+	Include all files.
-	Exclude all files.
+.wad	Include all '*.wad' files.
-.wad	Exclude all '*.wad' files.
+.wad;-d	Exclude all 'd' files but not '.wad'.
+.h3.bin;:sneek	Suppress for SNEEK unneeded files, but add 'h3.bin'.