BryteFlow TruData is data reconciliation application for the data migrated onto Cloud Data Lake or Cloud Data Warehouse. It does near real time reconciliation for the configured objects, replicated on any Cloud DataWarehouse i.e. AWS Redshift or Snowflake via BryteFlow’s Ingest application.
Data reconciliation using TruData is typically to verify target data against the original source data, to ensure that the migration application BryteFlow Ingest has transferred the data correctly.
It uses process information and mathematical methods to do a complete count and checksum validation of source and target data, at a particular point in time.
It performs a thorough and accurate verification on the correctness of the migrated data. It is also easy to
implement and provides near real-time stats for the reconciled data and the ongoing reconciliation process. It has flexibility to break down really huge tables to a very granular level and provide the comparison.
Table size doesn’t matter anymore. Every bit can be reconciled.
The User Interface of TruData is very informative and user friendly. Its graphical representation of the completeness results is very informative for the user, summarises everything in one place.
New tables setup is just a few clicks away. Monitor progress and results instantly.
The BryteFlow TruData dashboard provides complete summary of the reconciliations that are currently setup within the BryteFlow TruData application.
It shows summary for :
In this page you will see the brief summary of all the tables configured for reconciliation. It list the tables along with the table name, status, no.of Slice, last run timing and percent complete details.
Clicking on a row results in a detail summary being displayed as shown below :
Daily run summary, a graph showing daily accuracy status of the table for each day, over a span of a rolling 30 day window
Slice Detail, shows detailed statistics for the most recent reconciliation for each slice of the table that has been processed
Slice Min, minimum value of the processed Slice (this equates to >= 1 in first row)
Slice Max, maximum value of the processed Slice (this equates to < 5 in first row)
Status, reconciliation status for the Slice
Src. Records, number of records in source database for the Slice
Dest. Records, number of records in destination database for the Slice
Src. Checksum, checksum value for the Slice at Source
Dest. Checksum, checksum value for the Slice at Destination
Last Checked on, date and time of the last reconciliation run that happened for the Slice
This is a graph showing the daily accuracy status of the selected table over a rolling 30 day window :
This section can be used to do simple configuration of the TruData application which covers :
Location of CDC installation : The path of BryteFlow Ingest software
Web port : Port no. for the TruData web application to be hosted
No. of Source threads: Configured parallelism at source. No. of threads performing the reconciliation checks at source database.
No. of Destination threads: Configured parallelism at destination. No. of threads performing the reconciliation checks at destination database
Estimated Latency: Estimated latency of Ingest application. The schedule at which Ingest instance is scheduled to perform the load.
Default Schedule : This is the default schedule of the reconciliation process for all the tables selected. When no schedule is configured, default schedule is set to 24hrs.
Product ID : Product ID of the TruData installation
Licence Key : Licence key for the product id of the installation
This section provides ease of access to configure the metadata details of BryteFlow TruData.
It includes the configuration of :
Database Host: Host name of the database where the results of TruData Application needs to be pushed to.
Database port: Port no. of the database host
Database Name: Database name of the database host
User ID: Login User ID for the database
Password: Password for the login User id
In this section, the Users get to configure the table settings individually.
They get to,