User Guide

About BryteFlow Control Room #

BryteFlow Data Dashboard is a companion product to BryteFlow Ingest and Bryteflow Blend, it can seamlessly collect the status of Blend and Ingest instances.

Prerequisite #

The following are minimum requirements for BryteFlow Control Room

  • A physical or virtual machine running Windows server-level operating system.
  • Java version 7 or higher is required to be installed on the machine.
  • In an AWS environment, Connectivity and permissions to access AWS Cloudwatch Log.
  • BryteFlow Control Room on AWS, uses AWS access key and secret key to read AWS Cloudwatch logs.
  • Port 8091 needs to be available on the system for the application to bind to.
  • Open access to Port 8091 through any firewalls for interface access.
  • Google Chrome

Installation #

The application can be placed in any folder, however Bryte recommends installing to the following location:

c:\bryteflow\controlroom\

Step 1 – Application Files

Unzip the application archive to your desired folder, for example

c:\bryteflow\controlroom\

Step 2 – Run the Application

  1. In order to install the service open windows command prompt with ‘Run as Administrator’
  2. Change the directory to the path of ‘controlroom’
  3. Execute ‘Install-BryteFlow controlroom Service.bat’
  4. ‘BryteFlow Controlroom’ service is now installed on the server.
  5. Go to Windows ‘services’ and start the service for Ingest.
  6. Launch the user interface from google chrome browser

Step 3 – Access the User Interface

BryteFlow Control Room utilizes a browser-based user interface. You can access this, once the application is running, on:

http://localhost:8091/

If you are accessing remotely then use the appropriate domain and port 8091.

Configuration #

The configuration is divided into two sections i.e. Control Room Configuration and Instance Settings.

Credentials #

When setting it up in an AWS environment,  from the drop-down either select AWS credentials or AWS IAM access.


In AWS Region type in the region where the AWS Cloudwatch Logs are configured.
The region is to required to be in the standard format e.g. us-west-2.

If AWS credential is selected, you need to enter the AWS access key and AWS secret key.  For AWS IAM access no other inputs are required.

Instance Settings #

In Instance Settings, we can add an Ingest or Blend instance to be monitored to the Control Room.

Add Instance

Add new Ingest or Blend Instances in Control Room

  1. Go to ‘Instance Settings’
  2. Click on ‘Add instance’
  3. Chose appropriate ‘Instance Type’ from the drop down list
  4. When setting it up for Azure environment, please select instance type as ‘Ingest Instance (Old)’
  5. Provide a relevant name in the ‘Name’ section
  6. Provide Ingest URL in the ‘URL’
  7. Specify Latency limits for Amber and Red zones.
  8. Hit ‘Save’. The Ingest instance will appear under ‘Ingest Instance’ tab and Blend instance will appear under ‘Blend Instance’ tab.
  9. To add more instances, repeat steps 1-8.


To add a new instance click on Add Instance Button located in the top left corner.
In the center pane, select Ingest or Blend from the drop-down. If your Ingest version is lower than Ingest 3.10 then select Ingest (Old).
Enter a suitable name for the Instance for easy identification. Enter the CW Log Group and CW Log Stream  for the  Ingest or Blend instance.
If your Ingest version is lower than 3.10 then you need to select Ingest (Old) from the dropdown. Specify the URI of the Ingest instance which you want to monitor. e.g. http://192.168.10.51:8081
Ingest prior to version 3.10 did not support the AWS Cloudwatch log for monitoring.

There are two latency thresholds set here. For latency values less than the first threshold as show as green on the dashboard, between the first and second thresholds as yellow and over the second threshold as red.
Set the two latency thresholds (yellow and red).

Ingest Instances #

The Ingest Instances tab lists all Ingest instances that have been added from the Instance Settings tab.
The Ingest instances are listed in a tabular format. Under the Ingest Instance column the name is divided into two parts the first part is taken from the Name field of Ingest Instance tab, the second part specified in parentheses is the name given within the Ingest instance.

The host column specifies the Hostname or the IP address of the server where the Ingest instance is running.
The version column displays the current version and build number of Ingest and the Status column shows the current status of the Ingest instance. The status can have three states – OK shown as green, Idle/Halted shown as yellow, and Error/ Offline shown as red.
the extract column shows the current status of the extract and can have a value of Running or Idle. The Load column shows the current status of the load and can have values Running or Idle.

More information about tables can be viewed by clicking on the instance name.

Ingest Instance Details #

The Ingest Instance Details page provides more information about the tables in that Ingest Instance.
For each table, it shows the number of times the table has been loaded in the last 24 hours, the number of records loaded in that period and the last time the table was updated.

Blend Instances #

The Blend Instances tab, lists all the Blend instances that have been added to the Data Dashboard.
The Blend instances are listed in a tabular format. Under the Blend Instance column the name is divided into two parts – the first part is taken from the Name field of Blend Instance tab, the second part specified in parentheses is the name given within the Blend instance.
The host column specifies the Hostname or the IP address of the server where the Blend instance is running.
The version column displays the current version and build number of Blend and the Status column shows the current status of the Blend instance. The status can have two states – OK shown as green and Offline shown as red.
The Last Job column displays the date and time when the last job was executed successfully.

Blend Instance Details #

The Blend Instance Details page provides more information about the jobs on the Blend instance.
The jobs are listed in a tabular format. The name of the job is displayed along with the number of times the job was executed in the last 24 hours, number of seconds the job ran on the EMR instance in the same period and the last time the job was run.

Dashboard #

 

The dashboard displays the number of Ingest and Blend instances configured in the Data Dashboard along with the number of tables in Ingest and Jobs and Blend.
On the right side, the total number of records process, and in case of Blend total number of jobs executed in the last 24 hours are displayed. The worst-case latency for Ingest instance along with the instance identified by its name is displayed. The time since the last successful extract is also displayed.
The lower part of the page is a graphical representation of the latency of each Ingest instance. The color indicates the latency with respect to the two defined thresholds for each instance. Hovering over the box will display the latency for that time period along with its value compared to the two thresholds

Suggest Edit