To split database records when querying from a database, use the Batch Count setting on the Database Operation component.
Batch Count --- Sets the number of records to appear in each document. The default value is 0 (no batching). This option is used most often in conjunction with a link element but is also recommended when processing very large result sets. For example, if your query returned 100,000 records, specify a batch count of 5000 or 10000 (depending on the size of each record). This breaks up the data for efficient processing through process steps and especially maps.
For more information, see Database operation.
Note that Database Profile documents cannot be split using the Data Process - Split Document shape.
7 people found this helpful