The "loop batches" operator splits an example set into batches for the inner operators to work on. The output is simply a copy of the full input example set. The results of the inner operators are not passed to the output because the idea is for these to process the batches perhaps by writing to a file or to a database.
When writing a large example set to a database, machine resource limits can prevent this from working so batching is a good way to proceed.
An example is provided. This takes an example set with 100 examples and uses the "loop batches" operator to write each batch to a file. A macro is used to make the file names unique.
Search this blog
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment