Ubiquitous (Augmented) Analytics

Analytics today is a necessity that has re-defined traditional ways to make decisions by enabling making inferences from massive amount of relational as well as non-relational data sources.

Augmented analytics is a next-generation data and analytics paradigm that uses ML to automate data preparation, insight discovery and insight sharing for a broad range of business users, operational workers and citizen data scientists. Refer Gartner conference details – https://www.gartner.com/en/conferences/apac/data-analytics-australia/why-attend/event-resources/research-augmented-analytics

PS – Related to Augmentation, a well-known technology is AR – Augmented Reality – which is about computer generated graphics on top of actual reality. It’s an interactive experience of a real-world environment where the objects that reside in the real-world are “augmented” by computer-generated perceptual information using ML/DL. It has helped us redefine many things in Computer Vision world.

As per Gartner – “Augmented Analytics – is the Future of Data and Analytics. It’s an approach that automates insights using ML and natural-language generation, marks the next wave of disruption in the data and analytics market. Both small startups and large vendors now offer augmented analytics capabilities that could disrupt business intelligence (BI) and analytics, data science, data integration and embedded analytic application vendors. Data and analytics leaders must therefore review their investments.”

Apart from pervasiveness of AI/ML to achieve such Analytics, it also matters how fast (read Real-time) data is being analyzed. Is it being analyzed over days/hours OR near real-time? Now if we combine ability to get such Analytics done, then we need access to such Analytics capabilities instantly with right set of ML tools to automate generating insights. If you think about it, what we are then saying is – we need such Analytics capability to be Ubiquitous. It means – we need such capability both at the Edge & Cloud side which we need to leverage as apt to meet both our goals of near real-time as well as pervasiveness of AI/ML to achieve Augmented Analytics. This is needed since based on problem being solved, apt AI/ML support can be at the Edge and/or at the Cloud also.

We have seen in previous blogs as to how Edge side Analytics can get us cost advantages as well as near real-time effectiveness. However, it’s possible that customer may or may not have adequate infrastructure (read CPU/GPU capacity) OR manpower to develop such ML logic to meet his/her near term Analytics goal. Cloud based Analytics comes handy in such regard to meet near-term/immediate goals of such customer(s).

One good candidate for such Augmented Analytics capability in Cloud is Google Cloud Platform’s BigQuery (BQ). It’s a fast, highly scalable, serverless, cost-effective, and fully managed cloud data warehouse for analytics, with built-in ML (https://cloud.google.com/bigquery/). It’s designed to make customer’s data analysts productive at an unmatched price-performance. Because there is no infrastructure to manage, one can focus on analyzing data to find meaningful insights using familiar SQL without the need for a DBA. It’s about analyzing all the data by creating a logical data warehouse over managed, columnar storage, as well as data from object storage and spreadsheets. One can build and operationalize ML solutions with simple SQL. BQ is free for up to 1 TB of data analyzed each month and 10 GB of data stored. On top of this, BigQuery ML enables data scientists and data analysts to build and operationalize ML models on planet-scale structured or semi-structured data, directly inside BigQuery, using simple SQL — in a fraction of the time.

In order to make BQ really effective for Augmented Analytics – what’s needed is reliable, cost effective, near real-time ingestion capability from Edge (read as on-premise) side which needs to handle existing (heterogeneous) data sources that customer is hosting. Let’s take specific example here – A Windows customer having MSSQL database deployed for traditional OLTP workload. Fairly common scenario. Now for such customer to get continuous Analytics would mean simplified way to ingest SQL data near real-time to BQ to start getting it analyzed.

Question is – would existing framework allow such effective data ingestion to achieve Augmented Analytics? Also would it enable Ubiquitous Analytics that can be Augmented – be it on Edge or Cloud native side?

Intelligent IOs,
IOPhysics Systems