Class AbstractConcurrentProcessor<T extends Context>
- java.lang.Object
-
- com.univocity.parsers.common.processor.core.AbstractConcurrentProcessor<T>
-
- All Implemented Interfaces:
Processor<T>
- Direct Known Subclasses:
ConcurrentRowProcessor
public abstract class AbstractConcurrentProcessor<T extends Context> extends java.lang.Object implements Processor<T>
AProcessor
implementation to perform row processing tasks in parallel. TheConcurrentRowProcessor
wraps anotherProcessor
, and collects rows read from the input. The actual row processing is performed in by wrappedProcessor
in a separate thread. Note: by default theContext
object passed on to the wrappedProcessor
will not reflect the state of the parser at the time the row as generated, but the current state of the parser instead. You can enable thecontextCopyingEnabled
flag to generate copies of theContext
at the time each row was generated.- See Also:
AbstractParser
,Processor
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description private static class
AbstractConcurrentProcessor.Node<T>
-
Field Summary
Fields Modifier and Type Field Description private boolean
contextCopyingEnabled
private T
currentContext
private boolean
ended
private java.util.concurrent.ExecutorService
executor
private long
input
private AbstractConcurrentProcessor.Node<T>
inputQueue
private int
limit
private java.lang.Object
lock
private long
output
private AbstractConcurrentProcessor.Node<T>
outputQueue
private java.util.concurrent.Future<java.lang.Void>
process
private Processor
processor
private long
rowCount
-
Constructor Summary
Constructors Constructor Description AbstractConcurrentProcessor(Processor<T> processor)
Creates a non-blockingAbstractConcurrentProcessor
, to perform processing of rows parsed from the input in a separate thread.AbstractConcurrentProcessor(Processor<T> processor, int limit)
Creates a blockingConcurrentProcessor
, to perform processing of rows parsed from the input in a separate thread.
-
Method Summary
All Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description protected abstract T
copyContext(T context)
protected long
getRowCount()
private T
grabContext(T context)
boolean
isContextCopyingEnabled()
void
processEnded(T context)
This method will by invoked by the parser once, after the parsing process stopped and all resources were closed.void
processStarted(T context)
This method will by invoked by the parser once, when it is ready to start processing the input.void
rowProcessed(java.lang.String[] row, T context)
Invoked by the parser after all values of a valid record have been processed.void
setContextCopyingEnabled(boolean contextCopyingEnabled)
private void
startProcess()
protected abstract T
wrapContext(T context)
-
-
-
Field Detail
-
processor
private final Processor processor
-
ended
private boolean ended
-
executor
private final java.util.concurrent.ExecutorService executor
-
rowCount
private volatile long rowCount
-
process
private java.util.concurrent.Future<java.lang.Void> process
-
inputQueue
private AbstractConcurrentProcessor.Node<T extends Context> inputQueue
-
outputQueue
private volatile AbstractConcurrentProcessor.Node<T extends Context> outputQueue
-
limit
private final int limit
-
input
private volatile long input
-
output
private volatile long output
-
lock
private final java.lang.Object lock
-
contextCopyingEnabled
private boolean contextCopyingEnabled
-
-
Constructor Detail
-
AbstractConcurrentProcessor
public AbstractConcurrentProcessor(Processor<T> processor)
Creates a non-blockingAbstractConcurrentProcessor
, to perform processing of rows parsed from the input in a separate thread.- Parameters:
processor
- a regularProcessor
implementation which will be executed in a separate thread.
-
AbstractConcurrentProcessor
public AbstractConcurrentProcessor(Processor<T> processor, int limit)
Creates a blockingConcurrentProcessor
, to perform processing of rows parsed from the input in a separate thread.- Parameters:
processor
- a regularProcessor
implementation which will be executed in a separate thread.limit
- the limit of rows to be kept in memory before blocking the input parsing process.
-
-
Method Detail
-
isContextCopyingEnabled
public boolean isContextCopyingEnabled()
Indicates whether this processor should persist theContext
object that is sent to the wrappedProcessor
given in the constructor of this class, so all methods ofContext
reflect the parser state at the time each row was parsed. Defaults tofalse
- Returns:
- a flag indicating whether the parsing context must be persisted along with the parsed row so its methods reflect the state of the parser at the time the record was produced.
-
setContextCopyingEnabled
public void setContextCopyingEnabled(boolean contextCopyingEnabled)
Configures this processor to persist theContext
object that is sent to the wrappedProcessor
given in the constructor of this class, so all methods ofContext
reflect the parser state at the time each row was parsed. Defaults tofalse
- Parameters:
contextCopyingEnabled
- a flag indicating whether the parsing context must be persisted along with the parsed row so its methods reflect the state of the parser at the time the record was produced.
-
processStarted
public final void processStarted(T context)
Description copied from interface:Processor
This method will by invoked by the parser once, when it is ready to start processing the input.- Specified by:
processStarted
in interfaceProcessor<T extends Context>
- Parameters:
context
- A contextual object with information and controls over the current state of the parsing process
-
startProcess
private void startProcess()
-
rowProcessed
public final void rowProcessed(java.lang.String[] row, T context)
Description copied from interface:Processor
Invoked by the parser after all values of a valid record have been processed.- Specified by:
rowProcessed
in interfaceProcessor<T extends Context>
- Parameters:
row
- the data extracted by the parser for an individual record. Note that:- it will never by null.
- it will never be empty unless explicitly configured using
CommonSettings.setSkipEmptyLines(boolean)
- it won't contain lines identified by the parser as comments. To disable comment processing set
Format.setComment(char)
to '\0'
context
- A contextual object with information and controls over the current state of the parsing process
-
processEnded
public final void processEnded(T context)
Description copied from interface:Processor
This method will by invoked by the parser once, after the parsing process stopped and all resources were closed.It will always be called by the parser: in case of errors, if the end of the input us reached, or if the user stopped the process manually using
Context.stop()
.- Specified by:
processEnded
in interfaceProcessor<T extends Context>
- Parameters:
context
- A contextual object with information and controls over the state of the parsing process
-
getRowCount
protected final long getRowCount()
-
-