User Guide

About TruData

BryteFlow TruData is data reconciliation application for the data migrated onto Cloud Data Lake or Cloud Data Warehouse. It does near real time reconciliation for the configured objects, replicated on any Cloud DataWarehouse i.e. AWS Redshift or Snowflake via BryteFlow’s Ingest application.

Data reconciliation using TruData is typically to verify target data against the original source data, to ensure that the migration application BryteFlow Ingest has transferred the data correctly.

It uses process information and mathematical methods to do a complete count and checksum validation of source and target data, at a particular point in time.

It performs a thorough and accurate verification on the correctness of the migrated data. It is also easy to
implement and provides near real-time stats for the reconciled data and the ongoing reconciliation process. It has flexibility to break down really huge tables to a very granular level and provide the comparison.

Table size doesn’t matter anymore. Every bit can be reconciled.

Overview of User Interface

The User Interface of TruData is very informative and user friendly. Its graphical representation of the completeness results is very informative for the user, summarises everything in one place.

New tables setup is just a few clicks away. Monitor progress and results instantly.

Dashboard

The BryteFlow TruData dashboard provides complete summary of the reconciliations that are currently setup within the BryteFlow TruData application.

It shows summary for :

  • Last Complete Run Status, the result of the last completed reconciliation in a graphical representation using colour legends
  • Tables, the number of tables configured for reconciliation
  • Records, the total number of records across all the tables configured for reconciliation
  • Src. Run Time, date and time the reconciliation process ran against the source database
  • Src. Duration, total time taken at source database to perform reconciliation of configured tables
  • Dest. Duration, total time taken at destination database to perform reconciliation of configured tables
  • Current Run Status, shows percent complete status of the ongoing reconciliation in a graphical representation using colour legends
  • Daily run summary, a bar chart showing the overall reconciliation of all configured tables for each day using colour legends

Table Summary

In this page you will see the brief summary of all the tables configured for reconciliation. It list the tables along with the table name, status, no.of Slice, last run timing and percent complete details.

 

 

Clicking on a row results in a detail summary being displayed as shown below :

 

Daily run summary, a graph showing daily accuracy status of the table for each day, over a span of a rolling 30 day window

Slice Detail, shows detailed statistics for the most recent reconciliation for each slice of the table that has been processed

 

Slice Min, minimum value of the processed Slice (this equates to >= 1 in first row)

Slice Max, maximum value of the processed Slice (this equates to < 5 in first row)

Status,  reconciliation status for the Slice

Src. Records, number of records in source database for the Slice

Dest. Records, number of records in destination database for the Slice

Src. Checksum, checksum value for the Slice at Source

Dest. Checksum, checksum value for the Slice at Destination

Last Checked on, date and time of the last reconciliation run that happened for the Slice

Daily Run Summary

This is a graph showing the daily accuracy status of the selected table over a rolling 30 day window :

 

Config Options

This section can be used to do simple configuration of the TruData application which covers :

Location of CDC installation : The path of BryteFlow Ingest software

Web port : Port no. for the TruData web application to be hosted

No. of Source threads: Configured parallelism at source. No. of threads performing the reconciliation checks at source database.

No. of Destination threads: Configured parallelism at destination. No. of threads performing the reconciliation checks at destination database

Estimated Latency: Estimated latency of Ingest application. The schedule at which Ingest instance is scheduled to perform the load.

Default Schedule : This is the default schedule of the reconciliation process for all the tables selected. When no schedule is configured, default schedule is set to 24hrs.

Product ID :  Product ID of the TruData installation

Licence Key : Licence key for the product id of the installation

 

Config Metadata

This section provides ease of access to configure the metadata details of BryteFlow TruData.

It includes the configuration of :

Database Host: Host name of the database where the results of TruData Application needs to be pushed to.

Database port: Port no. of the database host

                Database Name: Database name of the database host

User ID: Login User ID for the database

Password: Password for the login User id

 

 

Table Settings

In this section, the Users get to configure the table settings individually.

They get to,

  • choose the columns to be reconciled
  • to define the Slice columns for each table
  • to specify the schedule for each table, this is in case when the tables need to be validated at different times
  • to apply filter criteria for each table, if needed.

 

 

Suggest Edit