Wednesday, January 15, 2020

batching items with batch.tryAdd(item)

Process items in batches with bool batch.tryAdd(item).

This is a bit clunky as it mixes the batch population and sending.
process(sender, items):
    batch = new
    for item in items:
        if !batch.tryAdd(item):
            if batch.any():
                sender.send(batch)
            batch = new
            if !batch.tryAdd(item):
                // poison message
                throw
    if batch.any():
        sender.send(batch)
Better as batch population is separated out.
process(sender, items):
    i = 0
    while i < items.count():
        batch = new
        while i < items.count() && batch.tryAdd(items[i]):
            i++
        if !batch.any():
            // poison message
            throw
        sender.send(batch)

No comments:

Post a Comment