Broadcasting Data Aggregation and Monitoring System

Equipped with predictive analytics functionality

Prologue: The DevOps rescue journey begins

The client to help

Specialized IT services provider

The system solution to recover

Collects and monitors data on transcoding and online broadcasting

The DevOps hero team

DevOps engineers covering quality assurance, system design maintenance, and infrastructure support

Chapter 1: The data aggregation system is compromised

A long time ago, the client solution made use of a DevOps system to aggregate data on transcoding processes and online broadcasts. However, it proved vulnerable to cyberattacks and was eventually hacked. Determined to enhance their security, the company intended to replace it with an advanced data monitoring and aggregation solution that would respond to their data security and resistance system requirements.

Chapter 2: Oxagile’s DevOps heroes raced to fortify system security

Once upon a time, the DevOps heroes received an urgent call for help from a distressed client.

Determined to restore their trust with clients, the DevOps hero embarked on a mission to revamp the system and make it more reliable and secure. The team devised a brilliant strategy:

First, they decided to ensure that all traffic would exclusively flow through the internal infrastructure and deploy the entire system within a closed data center.
Second, the DevOps team tapped into the power of open-source products that operated on a push model for Edge location. By utilizing these tools, they were able to enhance the system’s performance and efficiency, guaranteeing a seamless user experience for the client and their customers.
Finally, the DevOps heroes understood the importance of fortifying the system’s defenses. So, they added certification and secured all ports, leaving no room for potential breaches.

Chapter 3: Getting started with predictive analytics in the data monitoring system

The team began with designing an analytical monitoring system that seamlessly aggregated data from various sources into several local data centers. This allowed for centralized and efficient management of critical metrics such as bitrate, memory usage, CPU performance, state, and device status.

But the team didn’t stop there. They recognized the power of analytics in driving proactive decision-making. To achieve this, they integrated an advanced analytics system capable of analyzing current broadcasting metrics to foresee potential issues. For example, they could predict transcoding errors that might occur in a month based on early indicators such as an increasing error rate.

The reasons behind choosing VictoriaMetrics as the time series database

After carefully considering the options, we decided to go with VictoriaMetrics instead of Prometheus in Grafana. This decision was based on the following factors:

High performance and scalability

For both data ingestion and data querying on vertical and horizontal levels

Easy data backups

To restore the data when real-time tracking may occasionally be unavailable

Strict security measures

Proven by security certifications for Database Software Development and Software-Based Monitoring Services

The new DevOps hero, the new value due to QA practices

“The introduction of DevOps QA to our project brought immense value by implementing an automated testing system for our infrastructure. Though we had considered involving a QA engineer, we soon realized that we required the deep domain knowledge and low-level understanding that our DevOps team possessed. This marked a significant milestone as during the deployment of updates, unit tests were activated to ensure all essential databases were functioning, system components responded correctly, and metrics were running seamlessly.”

— Project Manager, Oxagile

Happy ending in numbers

7,000+

SD/HD-quality channels are monitored to ensure seamless transcoding processes

X10

Reduction in the monthly amount of big data stored within a system

X10

Decrease in regular issue requests from end users in real time

Epilogue: The synergy of project streams accompanied the value delivery

Proactive monitoring approach
Ensuring addressing issues in about 2 minutes
GStreamer-based framework audit
Making the transcoding and live streaming operations more efficient