Deep learning AI systems are “experts within a black box.” They produce a decision for an input situation. The logic for generating the decision is not revealed. One cannot determine why it made that decision. It cannot explain why a text output is the best speech recognition for a given audio input or why a given diagnosis represents the most probable cause for inputs of symptoms and medical history.
When we go to a human expert – a doctor, a tax consultant, or a financial advisor, we expect them to provide a rationale for the consultation advice or decision, which gives us confidence and trust in the advice offered. A deep learning AI system can provide us with the best advice and decisions, but does not, and cannot, provide the rationale for the decision. Deep learning technique that is used in AI systems cannot reveal the rationale used for decision-making. Just as we cannot understand the reasons behind perceptions and emotions of humans making decisions, we have to accept the opaqueness of AI systems, so long as we are happy with their decisions. Maybe a new technique will emerge to extract the rationale for an AI decision. Alternative methods of machine learning are being researched, where the rationale of decision is transparent?—?e.g. Bayesian method where you start with a hypothesis and every additional data input is used to tune that hypothesis.
Even if ML/AI decision-making remains a black box, there are significant advantages of leveraging AI for decision making. AI systems do not get distracted or tired, are generally more available, and they will get better and cheaper over time. AI systems can be exactly replicated and massively networked to work collaboratively with other AI systems. In contrast, the knowledge base and expertise of human experts has to develop individually and cannot be automatically networked. We believe that human experts and AI systems will work collaboratively, each contributing with their unique skills.
Data is the critical resource for machine learning. Patterns for learning and for taking decisions are contained in the data. Machine learning systems can produce valuable output if the data is relevant, clean, up-to-date, and reliable. In the new economy “data is the new oil” since AI converts data into business value.