com.hof.mi.etl.step

Interface ETLStep

All Superinterfaces:

java.io.Serializable

All Known Subinterfaces:

ETLInlineStep

All Known Implementing Classes:

AbstractETLCachedStep, AbstractETLInlineRowStep, AbstractETLRowStep, AbstractETLStep, AbstractGroupedDataInlineStep

public interface ETLStep
extends java.io.Serializable

All ETL Transformation Steps must implement this interface. It has methods for managing the step's inputs, outputs and metadata.

Method Summary

Methods

Modifier and Type	Method and Description
void	addDefaultMetadataField(ETLStepMetadataFieldBean field) Add a Default Metadata Field for this step.
void	addInputStep(java.lang.String inFlowUuid, ETLStep inputStep) Register an input step.
void	addOutputStep(java.lang.String outFlowUuid, ETLStep outputStep) Register an output step.
void	addStepOption(java.lang.String optionKey, java.lang.String optionValue) Add a config option for this step.
void	disable(boolean disable) Disable the step
void	endRows(java.lang.String inFlowUuid) This method is used to signal end-of-input.
ETLStepAPIVersion	getAPIVersion() Get the Version of the Yellowfin Step API.
ETLRunContext	getContext() Get information about the ETL Runner running this step.
java.lang.String	getDefaultDescription() Get the step's Default Description.
java.util.List<ETLStepMetadataFieldBean>	getDefaultMetadataFields() Get Default Metadata of this step.
java.lang.String	getDefaultName() Get the step's Default Name.
java.lang.String	getErrorFlowUuid() FlowUUID connecting this step to an error step.
ETLStep	getErrorStep() The error step.
ETLStepBean	getETLStepBean() Get the ETLStepBean backing this step.
java.util.Set<java.lang.String>	getInputFlowUuids() Get all input flowUUIDs.
ETLStep	getInputStep(java.lang.String inFlowUuid) Get the input step connected to this step by the specified input flowUUID.
java.lang.Integer	getMaxInputSteps() Maximum number of inputs this step supports
java.lang.Integer	getMaxOutputSteps() Maximum number of outputs this step supports
java.util.List<ETLStepMetadataFieldBean>	getMetadataFields(java.lang.String outFlowUuid) Get Output Metadata Fields for a specified output flowUUID.
java.lang.Integer	getMinInputSteps() Minimum number of inputs required for this step to work
java.lang.Integer	getMinOutputSteps() Minimum number of outputs required for this step to work
java.lang.String	getOffImage() Return a base 64 off image
java.lang.String	getOffImageMIMEType() Return the file type for a base 64 image off
java.lang.String	getOnImage() Return a base 64 on image
java.lang.String	getOnImageMIMEType() Return the file type for a base 64 image on
java.util.Set<java.lang.String>	getOutputFlowUuids() Get all output flowUUIDs.
ETLStep	getOutputStep(java.lang.String outFlowUuid) Get the output step connected to this step by the specified output flowUUID.
ParameterPanelCollection	getPanelCollection() Get the ParameterPanelCollection object which represents the step's configuration UI elements.
long	getRowLimit() The maximum number of rows which will be output by this step.
ETLStepStatus	getStatus() Get the status of the step as an instance of ETLStepStatus.
ETLStepCategory	getStepCategory() Return the step's category, which should be one defined in ETLStepCategory
ETLStepClassification	getStepClassification() Returns a specific classification for an ETLStep
java.lang.String	getStepOption(java.lang.String optionKey) Get config option value for this step, corresponding to the specified option key.
java.util.Map<java.lang.String,java.lang.String>	getStepOptions() Get all config options for this step as a map of Option Key and Option Value.
java.lang.String	getStepSubCategory() Return the step's sub category, used to determine a grouping of the step in UI.
java.lang.String	getUuid() Get the step's UUID
boolean	isEnabled() Is this step enabled?
java.util.Map<java.lang.String,java.lang.String>	onCopy(java.util.Map<java.lang.String,java.lang.String> uuidMap) This method is called while the step is being copied and is intended to be used as a way for steps to copy their external dependencies.
void	onDelete() This method is called just before the step is deleted and is intended to be used as a way for steps to cleanup external dependencies.
void	parseData(java.util.Map<java.lang.String,java.lang.Object> data) Parse the data of panel collection from the front end to the correct format that should be saved into the database and return the updated step options
void	removeDefaultMetadataField(java.lang.String fieldUuid) Remove a Default Metadata Field from this step.
void	row(java.lang.Object[] data, java.lang.String inFlowUuid) Process a row of data received from the specified input flow.
void	setContext(ETLRunContext context) Set the step's ETLRunContext.
void	setErrorStep(java.lang.String errorFlowUuid, ETLStep errorStep) Register the error step.
void	setETLStepBean(ETLStepBean stepBean) Set the ETLStepBean backing this step.
void	setMetadataFields(java.lang.String outFlowUuid, ETLFlowType flowType, java.util.List<ETLStepMetadataFieldBean> metadataFields) Set Output Metadata Fields for a specified output flowUUID.
void	setRowLimit(long errorThreshold) Set the step's rowLimit.
void	setStatus(ETLStepStatus status) Set the status object for this step.
void	setStepOptions(java.util.Map<java.lang.String,java.lang.String> options) Reset the step options with the provided values.
void	setupDefaultFields() Setup Default Metadata Fields for this step.
void	setupOptions() Set up step options.
java.util.Collection<ETLException>	validate() Perform validations on the ETLStep and return errors as a collection of ETLException

Method Detail

getUuid

java.lang.String getUuid()

Get the step's UUID

getAPIVersion

ETLStepAPIVersion getAPIVersion()

Get the Version of the Yellowfin Step API. This is used to determine if the Step is compatible with the current API. Step writers must return a value from enum ETLStepAPIVersion. This is usually the highest version, unless the writer is coding to a different API version for compatibility.

getDefaultName

java.lang.String getDefaultName()

Get the step's Default Name. Use getETLStepBean() and use its getStepName() method to get user-defined name

getDefaultDescription

java.lang.String getDefaultDescription()

Get the step's Default Description. Use getETLStepBean() and use its getDescription() method to get user-defined name

getStepCategory

ETLStepCategory getStepCategory()

Return the step's category, which should be one defined in ETLStepCategory

getStepSubCategory

java.lang.String getStepSubCategory()

Return the step's sub category, used to determine a grouping of the step in UI. There are some standard categories provided in ETLStepSubCategory which can be used, or a custom category can be used. If this value is a localisation text key, the category will be localised in UI.

getStepClassification

ETLStepClassification getStepClassification()

Returns a specific classification for an ETLStep

getETLStepBean

ETLStepBean getETLStepBean()

Get the ETLStepBean backing this step.

setETLStepBean

void setETLStepBean(ETLStepBean stepBean)

Set the ETLStepBean backing this step.

getOutputFlowUuids

java.util.Set<java.lang.String> getOutputFlowUuids()

Get all output flowUUIDs.

getOutputStep

ETLStep getOutputStep(java.lang.String outFlowUuid)

Get the output step connected to this step by the specified output flowUUID.

addOutputStep

void addOutputStep(java.lang.String outFlowUuid,
                 ETLStep outputStep)

Register an output step.

Parameters:

outFlowUuid - - FlowUUID connecting this step to the output step

outputStep - - The output step

getInputFlowUuids

java.util.Set<java.lang.String> getInputFlowUuids()

Get all input flowUUIDs.

getInputStep

ETLStep getInputStep(java.lang.String inFlowUuid)

Get the input step connected to this step by the specified input flowUUID.

addInputStep

void addInputStep(java.lang.String inFlowUuid,
                ETLStep inputStep)

Register an input step.

Parameters:

inFlowUuid - - FlowUUID connecting this step to the input step

inputStep - - The input step

getErrorFlowUuid

java.lang.String getErrorFlowUuid()

FlowUUID connecting this step to an error step.

See Also:

getErrorStep()

getErrorStep

ETLStep getErrorStep()

The error step. Errors which can be handled are sent to this step.

See Also:

getErrorFlowUuid()

setErrorStep

void setErrorStep(java.lang.String errorFlowUuid,
                ETLStep errorStep)

Register the error step.

Parameters:

errorFlowUuid - - FlowUUID connecting this step to the error step

errorStep - - The error step

See Also:

getErrorStep(), getErrorFlowUuid()

getDefaultMetadataFields

java.util.List<ETLStepMetadataFieldBean> getDefaultMetadataFields()

Get Default Metadata of this step. Default Metadata is the collection of all fields from all inputs to the step, and all fields generated internally.

addDefaultMetadataField

void addDefaultMetadataField(ETLStepMetadataFieldBean field)

Add a Default Metadata Field for this step. It inserts it in the correct position, as per the field's sortOrder field. If the sort order is null, negative or larger than the current highest sort order, the field is added at the end.

See Also:

getDefaultMetadataFields()

removeDefaultMetadataField

void removeDefaultMetadataField(java.lang.String fieldUuid)

Remove a Default Metadata Field from this step.

See Also:

getDefaultMetadataFields()

setupDefaultFields

void setupDefaultFields()
                        throws ETLException

Setup Default Metadata Fields for this step.

Throws:

ETLException

See Also:

getDefaultMetadataFields()

getMetadataFields

java.util.List<ETLStepMetadataFieldBean> getMetadataFields(java.lang.String outFlowUuid)

Get Output Metadata Fields for a specified output flowUUID. Output Metadata is a subset of the Default Metadata. This is the structure of data which will be sent to the output step, along the output flow.

See Also:

getDefaultMetadataFields()

setMetadataFields

void setMetadataFields(java.lang.String outFlowUuid,
                     ETLFlowType flowType,
                     java.util.List<ETLStepMetadataFieldBean> metadataFields)

Set Output Metadata Fields for a specified output flowUUID. The flow type specified in ETLFlowType must be specified as it determines which fields to filter out. Passed-in fields which should not be output will be removed.

See Also:

getMetadataFields(String)

getMinInputSteps

java.lang.Integer getMinInputSteps()

Minimum number of inputs required for this step to work

getMaxInputSteps

java.lang.Integer getMaxInputSteps()

Maximum number of inputs this step supports

getMinOutputSteps

java.lang.Integer getMinOutputSteps()

Minimum number of outputs required for this step to work

getMaxOutputSteps

java.lang.Integer getMaxOutputSteps()

Maximum number of outputs this step supports

getStatus

ETLStepStatus getStatus()

Get the status of the step as an instance of ETLStepStatus.

setStatus

void setStatus(ETLStepStatus status)

Set the status object for this step.

getRowLimit

long getRowLimit()

The maximum number of rows which will be output by this step.

setRowLimit

void setRowLimit(long errorThreshold)

Set the step's rowLimit.

See Also:

getRowLimit()

getContext

ETLRunContext getContext()

Get information about the ETL Runner running this step.

setContext

void setContext(ETLRunContext context)

Set the step's ETLRunContext.

row

void row(java.lang.Object[] data,
       java.lang.String inFlowUuid)
         throws ETLException

Process a row of data received from the specified input flow. This will be called from the input step and is used to process data row by row.

Parameters:

data - - A row of data

inFlowUuid - - UUID of the input flow which sent the data.

Throws:

ETLException

endRows

void endRows(java.lang.String inFlowUuid)
             throws ETLException

This method is used to signal end-of-input. It will be called from the input step of the specified flowUUID.

Parameters:

inFlowUuid - - UUID of the input flow which finished sending.

Throws:

ETLException

validate

java.util.Collection<ETLException> validate()

Perform validations on the ETLStep and return errors as a collection of ETLException

getPanelCollection

ParameterPanelCollection getPanelCollection()

Get the ParameterPanelCollection object which represents the step's configuration UI elements.

getStepOptions

java.util.Map<java.lang.String,java.lang.String> getStepOptions()

Get all config options for this step as a map of Option Key and Option Value. One cannot add/remove options by manipulating the returned map.

See Also:

addStepOption(String, String), setStepOptions(Map)

getStepOption

java.lang.String getStepOption(java.lang.String optionKey)

Get config option value for this step, corresponding to the specified option key.

addStepOption

void addStepOption(java.lang.String optionKey,
                 java.lang.String optionValue)

Add a config option for this step.

setStepOptions

void setStepOptions(java.util.Map<java.lang.String,java.lang.String> options)

Reset the step options with the provided values. Protected options, which have been setup automatically by the ETL framework (e.g OPERATING_FIELD for Inline Steps) should not be modified or removed when resetting the options list.

setupOptions

void setupOptions()

Set up step options. It creates "internal" step options such as those identifying generated fields. It also validates and fixes up step options. After this method runs, the step will have valid options for its present configuration.

getOnImage

java.lang.String getOnImage()

Return a base 64 on image

getOffImage

java.lang.String getOffImage()

Return a base 64 off image

getOnImageMIMEType

java.lang.String getOnImageMIMEType()

Return the file type for a base 64 image on

getOffImageMIMEType

java.lang.String getOffImageMIMEType()

Return the file type for a base 64 image off

parseData

void parseData(java.util.Map<java.lang.String,java.lang.Object> data)

Parse the data of panel collection from the front end to the correct format that should be saved into the database and return the updated step options

onCopy

java.util.Map<java.lang.String,java.lang.String> onCopy(java.util.Map<java.lang.String,java.lang.String> uuidMap)

This method is called while the step is being copied and is intended to be used as a way for steps to copy their external dependencies. The step can also choose to return a map of options which need to be updated - this is used for steps which have complex options which are related to the copied entities. Any option keys which are returned with a null value will be deleted. NOTE: When this method is called, the entities which are setup on the step will be the old entities, not the newly copied ones.

Parameters:

uuidMap - mapping of oldUUID to newUUID for any UUID's inside the step including stepUUID, fieldUUIDs, flowUUIDs, metadataUUIDs, groupUUIDs, etc. which have been copied.

Returns:

map of optionKey -> optionValue for any options which should be updated after copying.

onDelete

void onDelete()

This method is called just before the step is deleted and is intended to be used as a way for steps to cleanup external dependencies.

isEnabled

boolean isEnabled()

Is this step enabled?

disable

void disable(boolean disable)

Disable the step