deduper / org.bradfordmiller.deduper.consumers / DeduperDataConsumer

DeduperDataConsumer

class DeduperDataConsumer : BaseConsumer<Map<String, Any>, TargetPersistor>

Consumer for processing and persisting target data, IE "deduped" data

Constructors

<init>

Consumer for processing and persisting target data, IE "deduped" data

DeduperDataConsumer(targetPersistor: TargetPersistor, dataQueue: BlockingQueue<MutableList<Map<String, Any>>>, controlQueue: ArrayBlockingQueue<DedupeReport>, deleteIfExists: Boolean, sourceDataSource: DataSource, sqlStatement: String)

Properties

sourceDataSource
  • the data source used by the publisher. This is needed to build a persistent store which mirrors the published data
val sourceDataSource: DataSource

sqlStatement
  • SQL statement used by the publisher
val sqlStatement: String

Functions

createTarget

create/prep target persistence - can be database table or flat file

fun createTarget(deleteIfExists: Boolean, persistor: TargetPersistor): Unit

getDeduperReportCount

gets the deduped count from dedupeReport

fun getDeduperReportCount(dedupeReport: DedupeReport): Long