GenAI: where to tap so we don't fail?
Why is this area so hazy? Predictive analytics & self serve has been in the market for sometime and with this new area, it is becoming not just exciting but also blurry. With so much pressure from senior leadership to do GenAI proof of concepts, some of the concepts are being rushed from both ends: development as well as adoption. As of today, in my experience, the only mantra to make a GenAI product successful is to create something "that does not fail". Cost and effort goes into building this solution which is not just driven by data scientists, but also technology teams to operationalize this expensive concept. Before building this, we should just ask ourselves - "what are we generating really?" Is it something that has not been generated before in my organization? Who else is doing it? Can we learn from their success and failures? GenAI solutions built in silos might fail because of these reasons.
Timing is the key
Whilst rushing to adoption leads to failures, it is also important to test out things and deal with success and failures. This is because nothing in technology completely fails. Even if it fails, its rarely because of its design. Often, its because of low business adoption. This is the hardest bit, and this is what we need to avoid really. Currently, this relatively new area is being carefully monitored across organizations but some day not too long from now, there will come a time when GenAI will mature. At that time, the one who will be successful will be the one who has failed.. failed with proof of concepts, ideations, strategy and the one who has learnt. Not that failure is necessary, but failure will increase the chances of success in the future in this case. Hence, timing is key! Don't rush but don't delay as well. If not building a solution, build the right team to be ready for the future. Also, GenAI teams should not just comprise of data scientists or MLOps engineers, it would also need strategists who can strategize the development of products and models according to unmet needs. In the end, all components working really well with each other will make it successful. Otherwise, it might just be a fully working model of a good POC which never gets adopted really.
Why some orgs are quick wins v/s some are not?
Bain published a study wherein across different organizations within healthcare, wherein, the maturity of GenAI capabilities is evaluated across different functions. Below chart summarizes the results.
For instance, IT organization has most use cases in pilot or POC phase, whereas competitive intelligence, R&D and trial design have higher proportion of use cases in ideation phase. Given that the complexity of the use cases across different organizations vary, and compliance kicks in from different angles, ideation increases. Hence, if your organization is difficult to navigate, don't worry. Keep ideating, researching and innovating.
Even within functions, sentiments of the different users and adopters is not homogenous in terms of the confidence on GenAI.
There was a research by Slalom to understand from users - the degree by which GenAI would improve the BI function and the results are shown below.
Based on above table, it is evident that data engineers, business users and head of BI unit are quite optimistic about the GenAI use cases, whereas developers and consultants are not so much.
You see - on one hand, GenAI solves so much across the unstructured use cases such as summarizing a document, finding patterns, calculating complex statistics from a document, but the right methodology to adopt GenAI becomes more complex simply because unstructured use cases are difficult to solve and there are so many options and methods available. On the other hand, there are low hanging fruits within an organization which can solve structured use cases such as GenBI solution in a relatively simpler manner.
Human in the loop
In today's technology ecosystem, there are hundreds of solutions which offer transformation of the org through GenAI. Even tableau has come up with so many features including co-pilot, which can write complex formulas and calculated fields based on simple human input, Einstein Discovery, which applies complex predictive algorithms to predict significant features impacting a business case, Explain Data to explain the story behind certain data points in a visual. This is just an example. There are bunch of GenBI solutions which are quite transformative including GoodData, Pyramid Analytics, LogiAnalytics, ThoughtSpot Sage, etc.
There are several strategies put forth by consulting firms on the adoption of Generative AI solutions and selecting the technologies. Some of the most prominent ones which are available across different frameworks include:
- Choosing performance over novelty
- Agility
- Choosing a versatile solution
- The one that is great with conversational aspects, and many more
Despite many strategies, the one that is most significant is "human in the loop". In the end, we are solving problems for humans. Therefore, the most important factor which should always be considered is the stakeholders for whom we are solving the problems for. Strikingly, organizations fail to consider this simple factor which leads to POC failures.
Consider this:
A robust GenAI solution which is unique and powerful has a high chance of failing if the stakeholders for whom its been developed are not in the loop.
At the same time:
A naive GenAI solution, which does consider the problems of stakeholders, and tries to fill the gaps for those, might become highly adopted.
Its that simple - I wont give you any complex framework behind it (there are hundreds by the way).
ChatGPT did not just succeed because it is powerful and has been trained on billions of parameters, it succeeded because it was solving the problems of everyone, which was to get better search results in shorter duration of time. So, I believe this race of reaching there first is a part of a bubble, and fastest solutions might not be sustainable in the long run.
The technology world today is so dynamic and agile that if any solution is forced upon people, it will not work. It will not reach the adoption which it is supposed to reach.
Simple - keep humans in the loop before, during and after creating the GenAI solutions.
Sometimes FOMO is good
I have observed a lot that there is a rush among organizations to develop GenAI solutions. Is this rush good? I don't know. However, its not bad either. As I said, testing out solutions which work for your organization might be a great idea. Some of them could work, and some could fail. However, if it is done in a right way, there are high chances, that even if your strategy fails, that failure would lead to something very fruitful in terms of creating experience. For GenAI, which is a relatively naive area still for most organizations, failures will help refine your strategy so that you can hit the nail at the right place in the long run.
Although this visual above generalizes and the success of GenAI depends on so many other factors, still if we control for all the other factors, the probability of GenAI success reduces if the organizations do not try custom POCs to solve for deep problems at the right time. If organizations take a bottom up approach later on, success might come but it would be like fitting the problems into the solutions, instead of the other way around, and it would not be the best approach.
Is Predictive AI sufficient?
Predictive AI has been there for a while now. It has been mastered and re-mastered. Supervised and unsupervised algorithms have solved unimaginable problems for business organizations, and I see no reasons for them to go out of fashion. Ideology is simple - if you want to reach point A to point B, and predictive AI is helping you reach there with reduced costs and optimal strategies, why would you try more complex neural networks ridden, a computational heavy technique such as GenAI transformers to solve the same problem.
I see no point in forcing yourself into the heavy solutions, if not needed. Even predictive algorithms have advanced so much that algorithms including Convolutional Neural Networks, Recurring Neural networks are capable to ingesting hundreds of features and understanding very complex patterns without overfitting. Even then, if something simpler like a naive bayes algorithm might also give you similar results (if not the same), I would still go with the latter. Why waste money, resources and efforts unnecessarily?
I strongly feel that prior to any GenAI model adoption, there should be thorough strategy project to assess its potential impact on the business problem. If the impact is not significant, ignore it (just like hypothesis testing).
Summary
Generative AI models are highly complex models which have the capability to solve immense problems. However, they are not trained in a fundamentally different way than the other ML models. At first, data is gathered, cleaned and curated for model ingestion. Then, the model is trained on the data. Post that, the model is fine tuned for specific use cases, and the model is evaluated.
Similarly, even the fundamental principle of "adopting" GenAI in any organization should not be different. Problems statements need to be found and strategic assessment needs to be done to ensure that the projects are successful.
A POC which is not assessed structurally with stakeholders might never even see the light of the day. My strategy is simple (no framework strategy) - Don't let your GenAI strategy fail, its that simple.
