On Monday, Kyligence announced the general availability of the Kyligence Zen metrics platform. Already a leading OLAP (online analytical processing) engine to its name, with the new product, Kyligence is expanding its use cases to support more IT personas working with data metrics.
We met with Kai Liu, Director of Field Engineering at Kyligence, to learn about the new product, how it fits into the rest of the platform, and what makes it different from other metrics solutions.
A Mark of Distinction
Kyligence is a data intelligence company founded in 2016 by the creators of Apache Kylin, an open-source project with over 1500 free, and 100 commercial users worldwide. Kyligence Zen is powered by this same analytics engine.
Mr. Liu emphasized that Kyligence Zen metrics platform is markedly different from the metrics layers or metric stores that we have seen so far.
“We call it the metrics platform because we are not a single layer. We are a combination of their data processing, data modeling and metrics layer. That’s why we call it a metrics platform.”
Mr. Liu outlined the three core capabilities of Kyligence Zen – an SQL query engine, data modelling and the new metrics layer that pulls all data metrics into a unified catalog system. Kylin, the OLAP engine, fuels this solution with its ability to perform data pre-calculation.
“We have the capability to define data models by joining the tables together allowing the users to define dimensions, metrics, calculations, and so on.”
A Deterrent to Fast Decision-Making
So what made Kyligence add this new metrics layer? A common problem within businesses is the lack of consistent definitions of key metrics.
Between multiple business units and teams, there are numerous data stacks and analytics tools. Too many tools and data in scattered locations create silos, leading to misaligned definitions of metrics and eventually, a chaotic operations.
An All-in-One Metrics Catalogue
With a unified, all-in-one metrics catalogue, these divergent definitions can be aligned into shared and consistent ones across the organization that all IT personas can consume.
“The target is to abstract data modeling more for the business users so that they don’t have to understand all of the details of data modeling, like how the tables are joined together for example, leaving them to just focus on their business definition of those metrics,” says Mr. Liu.
The unified metrics catalog is designed to “create, maintain and manage all of their metrics together into a single layer,” for better quality data, and elimination of silos. With it, Mr. Liu says, administrators can define, compute, share, organize and analyze metrics all in one place.
This may sound similar to what OLAP does, but he reminded that OLAP leverages multiple data tools, and the data silo problem still persists with it.
The platform works with all types of metrics – “basic metrics directly from the data sources, derived metrics like metrics on metrics, several metrics put together, for example the year over year, or any other complicated calculations, to define new metrics.”
Leveraging the capabilities of the OLAP engine, it also offers smart caching with which precalculated data is cached for easy fetching.
A Magic Language
Who do this benefit most? Data engineers and business users, says Mr. Liu. Kyligence Zen uses ZenML, a user-friendly YAML-based descriptive language that allows data engineers to maintain and define metrics, dimensions and underlying relational datasets like codes. It is how “data engineers team reduce their efforts maintaining their daily works.”
The system generates metrics based on users’ definitions without engineers needing to write any code. Business users can consume technical data easily as ZenML converts it into business metrics.
“Also with this, we can define very restricted workflows for the business users. They can submit their own changes in this config file, and for the IT team, they can do the review and approval and then promote it to the production environment. That’s how they can maintain thousands of metrics without operating in the UI or writing any code.”
An OLAP Engine under the Hood and Open APIs
The AI-augmented OLAP engine that sits underneath performs all of the metrics calculation providing time-saving automation. The engine automates the whole of the pipeline including data appropriation, data processing and data consuming so that performance does not have to be manually tuned. “The system will guarantee performance automatically,” says Liu.
Kyligence Zen has open APIs that allows users to connect consistent metrics to any third-party business intelligence or SaaS tools that business users prefer. Mr. Liu says, “We provide these open APIs because we maintain everything on a single layer. Consumers are not limited to only business intelligence tools, but also other applications that they can use through these open APIs.”
In Conclusion
By introducing a common data language across organization, Kyligence Zen helps IT personas like data engineers to develop and manage metrics at low effort, and business analysts to yield faster and higher-quality decisions. The metrics catalogue provides a single source of truth where key business metrics are consistent across all teams and tools. The muscle behind it, the OLAP engine, unlocks the full power of automation saving businesses time, effort and cost in their data analysis initiatives.
For more information on the Kyligence Zen intelligent metrics platform, head over to their website. For more stories like this one, keep reading here at Gestalt IT.
