Read from file

Read from file

A simple way to activate InterFormNG2 processing is to monitor a directory for input files.

This is done using the workflow input type:

Read from file


InterFormNG2 monitors folders by checking them at regular intervals (every 0.5 seconds) and processing any files found inside.

  

1. Input directory

  • The directory that is monitored for new files.

  • If multiple workflows monitor the same directory, then each workflow must have a unique Filename filter.

Alert
If you want to monitor an IFS folder from a Windows installation, see: Monitor an IFS folder from a Windows PC.

2. Filename filter

  • The filter selects files based on patterns.

  • * → matches any string

  • ? → matches any single character

AlertIf multiple workflows monitor the same directory, the filename filter must be unique.
Otherwise, multiple workflows might try to process the same file.
Case sensitivity example
Filters are case-sensitive. For example, the filter:
*.xml
will only select files with the .xml extension (lowercase).
To select both .xml and .XML files, use:
*.xml,*.XML
(Separate values using a comma.)
If the selection is not unique, you will see the error:
"Parameter(s) not unique"
(Visible if you scroll down, and the Create option will be disabled.)

  

3. Input type

Defines what type of input files are expected in the input directory.

4. Sample XML file

  • If you are going to do conditions and/or Xpath expressions, then it is recommended to refer to an optional, sample XML file

  • If you click the magnifying glass, then you can select the sample XML file from the Library documents, that you want to use as a reference.

5. Sort by

Here you can select between the values below. This states which input files to select first, if there are more input files found in the input directory than the limit stated in the Max files per poll field below.


  

  • If you, for example, want to ensure that older input files are processed first, then you should consider selecting: Oldest first

  • However, there is a limitation to this because sorting based on the creation timestamp depends on how precise the timestamp is.

    • If you use this method to monitor a folder in the IFS on an IBM i, the timestamp may not be accurate enough if the files are created less than a second apart.

​
Alternative processing methods
If the sequence in which the files are processed is important, then you should consider an alternative method:

  • Use a REST webservice to process files in the correct order.

  • For the Spool2XML module, consider sorting by: File name (Alphanumeric A-Z) since the file name contains a detailed timestamp.

  • A better approach could be to call a program based on the demo source XML_NG2CMD: Found in source file APISRC in the IFORMNG2 library to use the NG2CMD command to call a workflow for processing the XML file.

​
Ensuring files are processed in a specific order
If the output files must be processed in a specific order, then:

  • Disable the multithreaded option (see below).

    Alert
    The downside is that performance will not be optimal without multi-threading.

  • Alternatively, use different folders:

    • Fast-processing folders: For cases where sequence is not critical.

    • Slower folders: Processed without multithreading but in sequence.

6. Max files per poll

Each time InterFormNG2 looks into the monitored folder, it will start processing a number of files found. This is configured here.

  • You should set a realistic number of files that can be processed within each poll.

  • Since InterFormNG2 polls every 0.5 seconds, the recommended value should be a maximum of 10.

  • Default value: 5

  • If the workflow processes very few but very large files, it may be a good idea to reduce this to 1.


Execution Order & Performance Considerations

  • The files in the monitored folder are executed in sequence – one at a time (not in parallel).

  • Other folders, however, are processed in parallel.


Setting a realistic number is important to prevent overload of InterFormNG2.
For example, if you place: 100,000 files in the same folder and then start InterFormNG2, it will take a long time to sort the files—possibly longer than the polling interval.
This is why it is essential to configure a realistic number of files per poll.

File Processing List
InterFormNG2 retrieves a list of files to process next, and with this, you can define how long this list is.

7. Keep processed files

The default behaviour is, that InterFormNG2 will keep the input files forever, but you can change that via this parameter.

Via a drop down list you can select between these values:
Forever
1 day
2 days
3 days
7 days
30 days

8. Multithreaded

  • Enabled → Multiple input files from the same folder are processed in parallel.

  • Disabled (default) → Files are processed one at a time.

Advantages of Multithreading

  • Increases performance.

  • Multiple threads can process more files faster.

Disadvantages of Multithreading

  • Output files may be out of order (if sequence matters, disable this).

  • Uses more RAM than single-threading.

Notes
Other performance settings can be found under Performance settings of the System Settings.

9. On error workflow

Here is an example of how to move files to an error folder, if an error occurred.


    • Related Articles

    • Read from Windows share

      You can setup InterFormNG2 to read input files from a folder found via Windows share (Samba network/UNC). For that you need to create a workflow with the input type, Read from Windows share. This can e.g. be useful, if you want to monitor a folder on ...

    • Read from FTP

      This makes it possible for a workflow to connect to an FTP server and poll a folder on the remote file system. It works in a similar fashion to "read from file" except that the files are transferred over FTP. Also, since this requires a connection to ...

    • Create PDF file

      The Create PDF file workflow component is found under Basic and create document. This component accepts an XML file as input. The component creates a PDF file and stores it directly in the file system. The output payload is the same as the input ...

    • Read completed jobs in database

      Reading completed jobs When a job has completed processing, the following columns are updated: Column Type Value Description processed Integer 1 Indicates processing status of the job. 1 = completed processedDate datetime processed time The time at ...

    • Transform a spooled file into XML

      InterFormNG2 can help you to transform a spooled file into XML. In order to do that you first need to load the spooled file in the library in the version 2 format. Next you need to setup a transformation of the spooled file in the designer. This will ...