[Files] Find
Q100012: [Files] Find
Locates files based on one or more combinations of the supplied criteria.
| Parameter | In/Out | Description | Other Info |
|---|---|---|---|
| Source folder | In | A folder to search for eligible or matching files. | |
| Filename pattern | In | The base filename pattern to look for eligible files, which accepts OS based wildcards for filenames such as the ? and * characters. Can also use the vertical bar character \| to specify multiple patterns.Examples: *.pngFinds only PNG image files. *.jpg\|*.jpegFinds only JPG and JPEG image files. |
|
| Include RegEx | In | Optional filter to only look for filenames matching the specified RegEx. Example: ^.+\d\d\d\d(0[1-9]\|1[0-2])(0[1-9]\|[12][0-9]\|3[01])\.txt$Matches any TXT files that end with a date formatted as YYYYMMDD in the filename. (Case is always ignored.) |
|
| Exclude RegEx | In | Optional filter to ignore filenames matching the specified RegEx. Examples: \.(bat\|exe)$Ignores any file ending with .bat or .exe ^(readme).*\.txt$Ignores any .txt file that starts with 'readme' in the filename (Case is always ignored.) |
|
| Include subfolders? | In | Option to look within subfolders for eligible files. | Default: False |
| Age threshold type | In | Choices: None, Older, NewerIf filtering by files above or below a certain age, select the theshold option Above or Below. (Age is determined by the file's last modified date.) |
Default: None |
| Age threshold value | In | If filtering by files above or below a certain age, enter the integer age threshold value here. (Age is determined by the file's last modified date.) |
|
| Age threshold units | In | Choices: Minutes, Hours, Days, Weeks, MonthsIf filtering by files above or below a certain age, enter the units (Minutes, Hours, ...) for the age threshold value. (Age is determined by the file's last modified date.) |
Default: Days |
| Include files with attribute(s) | In | Choices: Archive, ReadOnly, Hidden, SystemInclude only files with the selected attribute(s). If you choose more than one, the file will be included only if ALL the selected attributes are present. READ-ONLY - File should not be altered, and will affect any attempt to edit or delete it.ARCHIVE - File has been modified since the last time it was backed up.HIDDEN - File is hidden. Commands like "dir" and apps like File Explorer do not show hidden files by default.SYSTEM - File is a critical or important system file that is necessary for the operating system (OS) to work properly and are also normally hidden from view by default. |
Multiple selections allowed |
| Exclude files with attribute(s) | In | Choices: Archive, ReadOnly, Hidden, SystemExclude any files with the selected attribute(s). If you choose more than one, the file will be excluded if ANY of the selected attributes are present. READ-ONLY - File should not be altered, and will affect any attempt to edit or delete it.ARCHIVE - File has been modified since the last time it was backed up.HIDDEN - File is hidden. Commands like "dir" and apps like File Explorer do not show hidden files by default.SYSTEM - File is a critical or important system file that is necessary for the operating system (OS) to work properly and are also normally hidden from view by default. |
Multiple selections allowed |
| Minimum # of files required | In | Sometimes you might want to stop your job and prevent it from going to the next step if it doesn't find enough files, or any files at all. If so, set the number here. For instance, if you want to stop if no files are found, set this to 1, meaning that it must find at least 1 file to continue. If you set this to 12, at least 12 files must be found before continuing to the next step.If you set this value, you must also set the next one below. |
|
| Action if less than minimum files found | In | Choices: ContinueJob, StopJobWithFailure, StopJobWithSuccessIf you entered a minimum number N above, this parameter determines the action you would like to take if the number of files found is less than N. |
|
| Maximum # of files to return | In | Sometimes you might want to limit the number of files returned. Your next step might only be able to accept up to 20 files at a time, so in that case you can enter 20 here and this step will never return more than 20 files.If you set this value, you must also set the MaxFilesOrder parameter. PRO TIP: If you use this parameter to process N files at a time in batches, one of the subsequent steps should either delete or move the files when you are done processing them, so they do not get selected again on the same job. |
|
| Maximum files selection method | In | Choices: Newest, Oldest, Largest, SmallestIf you entered a maximum number N above, this parameter determines which N files are chosen: the N newest, the N oldest, the N largest, or the N smallest. This doesn't affect the order of output files -- they will always be sorted alphabetically by folder. |
|
| Log output level | In | Choices: Minimal, VerboseThe level of execution detail you want to appear in the log. |
Default: Minimal |
| Matched file list | Out | List of file paths that matched. | |
| Files found | Out | The number of files found by the module. |
(Italics = required parameters)
Notes
The module [Files] Find is built specifically to provide you with a powerful file selection and filtering tool. This module is the key to making many other modules do some really useful work. It allows you to combine a variety of filtering features in a nearly unlimited fashion so that you have many options for working with the specific files you need, regardless of how they might be stored on your local machine or network.
Starting off with a simple example, the most basic option would be to just specify a path in the File Source parameter. By doing that, you would get a list of all the files contained in that path. By adding a basic pattern to the Filename Pattern parameter, you can limit it o more specific files. An example to find all the image files stored in either PNG or JPEG format, you might use a pattern like *.png|*.jpg|*.jpeg for this parameter. Notice we can specify multiple patterns by separating then with a vertical bar character. In our example here, we include the file extensions .jpg and .jpeg for the JPEG images, as some applications may use one or the other. This way we get both.
Filename patterns allow very basic filtering of input files, and this is the fastest way to get a set of files from a file source that might contain a lot of other files. There is a much more powerful option for filtering your files based on the filename, but this option can be much slower than the basic file patterns for large amounts of files. This option is the Include RegEx and the Exclude RegEx parameters. These allow the definition of pretty complex filename selection, but it can be challenging if you are not already familiar with how Regular Expressions work. RegEx is the abbreviated name for Regular Expressions. When you add a value for the Include RegEx parameter, the RegEx pattern matching is applied to all the files found in the file source. Then you can combine this with the Exclude RegEx parameter to narrow the focus even further.
The ability to mix and match these filters and options provides a very powerful way of getting just the list of files you need. If we combine the two RegEx options, we get a powerful method for file selection. If you used the value below for the Include RegEx parameter:
^.+\d\d\d\d(0[1-9]\|1[0-2])(0[1-9]\|[12][0-9]\|3[01])\.txt$
And now combine it with the value below for the Exclude RegEx parameter:
^(readme).*\.txt$
You wind up with a filter that gets a list of all .txt files that end with what might look like a date value in the filename such as Chain-Sales-20230325.txt, but will not include a file like ReadMe-20230325.txt.
The option to Include Subfolders will allow you to specify if you just want the find module to look for files directly in the file source you specified. If you enable this option, then it will also search through all the subfolders located within the file source. This option is usually used in combination with other modules that can recognize a hierarchy of files, such as the [Files] Copy/Move module.
The age options allow filtering of the file selection by the age of the file. Thus you can select only files that are older than a certain age. Or you can select age based on newness.