Generative Machine Learning: is it the next big revolution?
As AI tends to take over the world, the revolution is deepening at a fast pace. This pace is probably higher than what was expected. The reason for this high pace is the opportunity - to operationalize, automate, create, and save costs. Everyone wants to do AI today - however, only some are doing it right. There exists different grades of tasks which can be automated starting with very unstructured tasks such as creating new content, implementing chatbots to answer user's queries, deploying agents to perform monotonous tasks of humans. However, there could also be more sophisticated tasks that AI could do - such as creating another AI platform or agent, or to perform predictive analytics.
Introduction to GenML
I had an idea - what if AI could help us write classical machine learning codes and provide insights to us which are interpretable by a business person. Wouldn't it be cool? The costs that go into building ML models which are finally not productionized is very high across the organizations. Often, business stakeholders want to understand correlations, important features, explainability, or they want to forecast. Data science teams are expensive - often charging by the hour to build such models and a lot of times, there are several issues encountered including:
- communication issues between data scientists and business stakeholders
- Business stakeholders finding it difficult to interpret the outcomes of the ML models
- Ultimately, the models are deemed "not so useful" because the business users are not able fine tune it themselves
I am not saying these problems are not navigable, often there are business analysts, project managers or product managers to communicate between business teams and data scientists. However, that leads to high costs, and a very formalized setup for problems which are most often not that complicated. Sometimes, all it needs is a much simpler solution.
In this era of enabling the business users more and more, GenBI has taken over the world really fast. With ThoughtSpot, PowerBI, to some extent Tableau, and many other platforms providing the users flexibility to perform data exploration and reporting tasks on the fly.
I believe soon there will be a new era - to let citizen data scientists create predictive and prescriptive models themselves, with limited support from data science teams.
Only when a model is good enough and useful enough to be operationalized, MLOps could come into play there and use the hyperparameters, fine tuning steps, and feature engineering that the GenML model has finalized. I do genuinely believe that this is the next best revolution. This idea is not completely new - there already are many players such as h2o.ai and julius.ai which have similar vision.
Driverless AI is one of the products of H2o.ai which helps in automating ML processes without requiring much expertise in machine learning.
Julius.AI: Another product which I really liked is Julius.AI. To explain how it really works, I did a very quick exercise on one of the datasets from Kaggle - crop recommendation dataset. Without saying much in words, let me show you how I did data analysis and applied a simple classification ML on this dataset. See below snippets.
Conclusion
This small exercise was just an example on one of the available tools to explain my ideology. GenML has real potential across industries and will be one of the areas which organizations will tend to tap into once they scale into the GenBI. GenBI is being tried and tested, and a lot of POCs have been conducted successfully. However, only a few organizations have scaled these POCs into enterprise solutions with a homogenous strategy.
GenML will be a much harder area to really scale. Data exploration is more tied to retrospective analysis. However, Machine Learning has a lot to do with predictions and prescriptions - hence this area is very well connected with "trust". Business decisions are often taken through the inference of these algorithms so the ideology needs to be that we need to train GenML to produce "highly reliable outcomes". In addition, the business users also need to be trained on ML concepts to an extent that they can make sound decisions. There also would need to be "Data Science in the loop" so they can help business users whenever required, and also to enable a communication stream between the techies and business. The successful strategy would include containing the areas which GenML can perform (for ex: forecasting and regression analysis to start with), and slowly scale to more complex areas such as deep learning.
I am very excited about Generative Machine Learning.
Sources:
