com.hof.mi.etl.step

Class AbstractETLRowStep

java.lang.Object

com.hof.mi.etl.step.AbstractETLStep

com.hof.mi.etl.step.AbstractETLRowStep

All Implemented Interfaces:

ETLStep, java.io.Serializable

Direct Known Subclasses:

AbstractETLInlineRowStep

public abstract class AbstractETLRowStep
extends AbstractETLStep

This is a partial implementation AbstractETLStep. Step implementations which extend this should process data row by row.

See Also:

Serialized Form

Field Summary

Fields inherited from class com.hof.mi.etl.step.AbstractETLStep

commonDeps, dataPackets, dependencies

Constructor Summary

Constructors

Constructor and Description
AbstractETLRowStep()
AbstractETLRowStep(com.hof.mi.etl.ETLDependencies dependencies, com.hof.util.dependency.CommonDependencies commonDeps)

Method Summary

Methods

Modifier and Type	Method and Description
protected void	cleanup() All cleanup and releasing resources to happen here.
void	endRows(java.lang.String inFlowUuid) Set the step's status and signal endRows() in subsequent steps.
protected void	postRow(ETLStepResults processedData) Perform any post-processing here..
protected void	preEndRows(java.lang.String inFlowUuid) Any pre-processing before finishing this step.
protected java.lang.Object[]	preRow(java.lang.Object[] data, java.util.List<ETLStepMetadataFieldBean> metadata) Perform any pre-processing here.
protected void	processDependentRow(java.lang.Object[] data, java.util.List<ETLStepMetadataFieldBean> metadata, java.lang.String inFlowUuid) Processing logic for any dependent input steps goes here.
protected ETLStepResults	processRow(java.lang.Object[] data, java.util.List<ETLStepMetadataFieldBean> metadata) This method initiates the core processing of this step and maps data from the input structure to the output structure.
protected abstract boolean	processWireData(java.util.List<ETLStepMetadataFieldBean> fields) Process data on Wires.
void	row(java.lang.Object[] data, java.lang.String inFlowUuid) Process a row of data received from the specified input flow.

Methods inherited from class com.hof.mi.etl.step.AbstractETLStep

addDefaultMetadataField, addError, addGeneratedField, addInputStep, addNewGeneratedField, addOutputStep, addStepOption, beginInternalTransmission, checkErrorThreshold, clearAllWires, disable, emitData, emitData, endInternalTransmission, endInternalTransmission, excludeDefaultField, flowFinished, generatePanelCollection, getContext, getDefaultInternalOptions, getDefaultMetadataFields, getDefaultMetadataFieldsMap, getDefaultMetadataFieldUuids, getDefaultMetadataFieldUuidsList, getDefaultMetadataMap, getDefaultToInputFieldMap, getErrorFlowUuid, getErrorStep, getEtlDataTypeUtil, getETLException, getETLStepBean, getFile, getFile, getFileName, getFileName, getFirstInputFlow, getFirstOutputFlow, getFreshDataPacket, getInputFlowForDefaultMetadataGroup, getInputFlowUuids, getInputStep, getInputSteps, getInputToDefaultFieldMap, getInvalidConfigETLException, getMaxInputSteps, getMaxOutputSteps, getMetadataFields, getMinInputSteps, getMinOutputSteps, getOffImage, getOffImageMIMEType, getOffImagePath, getOnImage, getOnImageMIMEType, getOnImagePath, getOutputFlowUuids, getOutputFlowUuidsExceededRowLimit, getOutputStep, getOutputSteps, getPanelCollection, getRowLimit, getStatus, getStepBooleanValue, getStepClassification, getStepIntegerValue, getStepOption, getStepOptions, getStepOptionValueJSONArray, getStepOptionValueJSONObject, getStepSubCategory, getStringUtil, getText, getText, getUnhandledETLException, getUuid, getValidatedStepOptions, getWireForField, hasPendingDependency, includeDefaultField, isEnabled, isGeneratedField, isMainInputFlow, isReplacementField, onCopy, onDelete, parseData, removeDefaultMetadataField, replaceDefaultField, restoreReplacedField, setContext, setErrorStep, setETLStepBean, setMetadataFields, setRowLimit, setStatus, setStepOptions, setupDefaultFields, setupGeneratedFields, setupOptions, shouldOutputField, stepCanRun, throwInvalidConfigETLException, throwUnhandledETLException, updateFieldSortOrder, waitForDependencies

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.hof.mi.etl.step.ETLStep

getAPIVersion, getDefaultDescription, getDefaultName, getStepCategory, validate

Constructor Detail

AbstractETLRowStep

public AbstractETLRowStep()

AbstractETLRowStep

public AbstractETLRowStep(com.hof.mi.etl.ETLDependencies dependencies,
                  com.hof.util.dependency.CommonDependencies commonDeps)

Method Detail

row

public void row(java.lang.Object[] data,
       java.lang.String inFlowUuid)
         throws ETLException

Description copied from interface: ETLStep

Process a row of data received from the specified input flow. This will be called from the input step and is used to process data row by row.

Parameters:

data - - A row of data

inFlowUuid - - UUID of the input flow which sent the data.

Throws:

ETLException

preRow

protected java.lang.Object[] preRow(java.lang.Object[] data,
                        java.util.List<ETLStepMetadataFieldBean> metadata)
                             throws ETLException

Perform any pre-processing here. This implementation returns the input data as-is. Override to implement in a subclass.

Parameters:

data - - A row of data in an Object[]

metadata - - ETLStepMetadataFieldBeans representing the structure of a row of data

Returns:

Object[] having preprocessed data

Throws:

ETLException

processRow

protected ETLStepResults processRow(java.lang.Object[] data,
                        java.util.List<ETLStepMetadataFieldBean> metadata)
                             throws ETLException,
                                    java.lang.InterruptedException

This method initiates the core processing of this step and maps data from the input structure to the output structure. The row of data may be processed into multiple outputs. Processed data is returned as an instance of ETLStepResults.
The default implementation does the following:

• Put the row of data on corresponding Wires.
• Call processWireData(List) which contains only the transformation logic.
• Finally, read transformed data from wires, construct output row(s) and put them into ETLStepResult objects.

Parameters:

data - - A row of data in an Object[]

metadata - - ETLStepMetadataFieldBeans representing the structure of a row of data

Returns:

Processed data mapped to output structure(s), in an instance of ETLStepResults

Throws:

ETLException

java.lang.InterruptedException

postRow

protected void postRow(ETLStepResults processedData)
                throws ETLException

Perform any post-processing here.. This implementation does nothing. Override to implement in a subclass.

Parameters:

processedData - - Processed row of data for post-processing.

Throws:

ETLException

processDependentRow

protected void processDependentRow(java.lang.Object[] data,
                       java.util.List<ETLStepMetadataFieldBean> metadata,
                       java.lang.String inFlowUuid)

Processing logic for any dependent input steps goes here. This implementation does nothing. Override if the step handles multiple inputs.

endRows

public void endRows(java.lang.String inFlowUuid)
             throws ETLException

Set the step's status and signal endRows() in subsequent steps.

Parameters:

inFlowUuid - - UUID of the input flow which finished sending.

Throws:

ETLException

preEndRows

protected void preEndRows(java.lang.String inFlowUuid)
                   throws ETLException

Any pre-processing before finishing this step. This implementation does nothing. Override to implement in a subclass.

Throws:

ETLException

cleanup

protected void cleanup()

All cleanup and releasing resources to happen here. This method is always called before a step finishes. The method in AbstractETLRowStep does nothing, but a step may override it if it needs to.

processWireData

protected abstract boolean processWireData(java.util.List<ETLStepMetadataFieldBean> fields)
                                    throws ETLException,
                                           java.lang.InterruptedException

Process data on Wires. When the framework calls this method, data from each column of the current row is on its own Wire. Each Wire is mapped to a metadata field and may be accessed using AbstractETLStep.getWireForField(String). Data may be retrieved from the Wire, processed, and put back on the same Wire or a different one.

Errors must be wrapped in a ETLExceptions and thrown so that the framework can show them to the user. Do not catch and swallow exceptions.

Parameters:

fields - - Metadata Fields for a row of data to be processed.

Returns:

boolean - if the row should be processed, return true, otherwise return false

Throws:

ETLException

java.lang.InterruptedException