Read from file

Read from file

A simple way to activate InterFormNG2 processing is to monitor a directory for input files. This is done with the workflow input type: Read from file.

 

InterFormNG2 monitors folders by looking into the monitored folders with regular intervals (every 0.5 seconds) and then starts processing the files inside.

 

NG2ReadFromFile0001

 

 

The fields are:

 

Input directory

The directory, which is monitored for new files. Please note, that if multiple workflows are monitoring the same directory, then each workflow need to have different Filename filters.

 

If you want to monitor a folder in the IFS of an iSeries machine from a Windows installation, then you should read the section: Monitor an IFS folder from a Windows PC.

 

Filename filter

The filter to select files. * means any string and ? mean any single character. It is important to note, that if multiple workflows are monitoring the same directory, then the filename filter must be unique, so that any input file only fits the conditions of the filename filter of a single workflow.

 

Please note, that the filter is case sensitive, so the filter *.xml will only select files with the extension .xml. If you want a workflow, that reacts to files that has either the extension .xml or .XML, then you can use this as the filter:

 

*.xml,*.XML

 

So as you can see you can separate values for the filter with a comma.

 

If the selection is not unique you will see the error message "Parameter(s) not unique" (if you scroll down), and the create option is disabled:

 

NG2Workflow0004

 

Input type

What type of input files are expected in the input directory.

 

Sample XML file

If you are going to do conditions and/or Xpath expressions, then it is recommended to refer to an optional, sample XML file. If you click the magnifying glass, then you can select the sample XML file from the Library documents, that you want to use as a reference.

 

Sort by

Here you can select between the values below. This states which input files to select first, if there are more input files found in the input directory, than the limit states in Max files per poll field below.

NG2ReadFromFile0002

If you e.g. want to make sure, that the old input files are processed first, then you should consider the value, Oldest first. There is however a limitation to this as the sorting on the creation timestamp depends on how exact this timestamp is. If you e.g. use this to monitor a folder in the IFS on an IBM i, then this is not exact enough, if there is less than a second between the input files.

 

If the sequence in which the files are processed is important, then you should use another way to process the files e.g. via a REST webservice.

For use with the Spool2XML module you could consider to sort by the file name (Alphanumeric A-Z) instead as the file name contains a detailed timestamp or better: Call a program based on the demo source XML_NG2CMD (in source file APISRC in the IFORMNG2 library) to use the NG2CMD command to call a workflow for processing the XML file.

 

If the output files must be processed in a specific order, then you also need to disable the multithreaded option as described below. The downside with that is, that the performance is not optimal without multi-threading. Alternatively you can consider to use different folders: Some folders, where processing must be fast but perhaps not in a specific sequence and other slower folders, that are processed slower (without multithreading), but in sequence.

 

 

Max files per poll

Each time InterFormNG2 looks into the monitored folder, it will start processing a number of files found. This is setup here. You should here set a number of files, that should be possible to process within each poll. As InterFormNG2 is polling every 0.5 seconds the recommended value should be as low as max. 10. The default value is 5. If the workflow is processing very few, but very large files it can be a good idea to reduce this to 1.

 

The files in the folder are executed in sequence - one at a time -  not in parallel (Other folders are however processed in parallel). It is however a good idea to set a realistic number here in order to prevent overload of InterFormNG2. If you e.g. place 100.000 files in the same folder and then start InterFormNG2, then it will take a long time for InterFormNG2 to sort the files - perhaps even longer than the polling interval, so that is why you need to setup a realistic number.

 

InterFormNG2 will retrieve a list of the files to process next, and with this you can state how long this list is.

 

 

Keep processed files

The default behavior is, that InterFormNG2 will keep the input files forever, but you can change that via this parameter. Via a drop down list you can select between these values:

Forever

1 day

2 days

3 days

7 days

30 days

 

Multithreaded

The option, Multithreaded is an option, that improves the performance of InterFormNG2. This is default disabled. If enabled any input files found in the selected input folder will be processed in parallel by multiple threads. Without multi-threading only one file from the input folder will be processed at a time. This means e.g. that if a large file is found in the folder (that takes long to process), then that will cause the other files (in the same folder) to wait until this file has been fully processed.

The advantage with this is, that multiple threads can process more and faster.

The disadvantages are:

1.The output files might not be generated in the same sequence as the input files has arrived, so if the sequence of the output files is important, then you should not enable this.

2.Multi-threading will use a bit more RAM compared to single threading.

Other performance related settings are found under Performance settings of the system settings.

 

 

On error workflow

Another workflow to be called, if there is an error during processing of this workflow. This can be overwritten with another workflow via the error handler.You can also consider a general error report email setup for the whole tenant. There are hints for usage in the section for the error handler.

 

Here is an example of how to move files to an error folder, if an error occurred.

    • Related Articles

    • Read from Windows share

      You can setup InterFormNG2 to read input files from a folder found via Windows share (Samba network/UNC). For that you need to create a workflow with the input type, Read from Windows share. This can e.g. be useful, if you want to monitor a folder on ...
    • Read from FTP

      This makes it possible for a workflow to connect to an FTP server and poll a folder on the remote file system. It works in a similar fashion to "read from file" except that the files are transferred over FTP. Also, since this requires a connection to ...
    • Create PDF file

      The Create PDF file workflow component is found under Basic and create document. This component accepts an XML file as input. The component creates a PDF file and stores it directly in the file system. The output payload is the same as the input ...
    • Read completed jobs in database

      Reading completed jobs When a job has completed processing, the following columns are updated: Column Type Value Description processed Integer 1 Indicates processing status of the job. 1 = completed processedDate datetime processed time The time at ...
    • Transform a spooled file into XML

      InterFormNG2 can help you to transform a spooled file into XML. In order to do that you first need to load the spooled file in the library in the version 2 format. Next you need to setup a transformation of the spooled file in the designer. This will ...