- Mayank Mishra
- Dhiraj Madan
- et al.
- 2022
- IJCAI 2022
Foundation Models for Conversation
Overview
Grounded Dialog Response Generation
One of the key problem that deep learning based systems based on pre-trained large language models suffer is the problem of hallucination. They could provide factually incorrect or inconsistent information. To overcome this problem, we are exploring ways in which we ground the dialog response generation on trustworthy enterprise content. In addition to exploring ways in which we could use decision trees or flow charts as well as structured data for grounding dialog responses, we are also investigating use of large language models, where we could use approved unstructured documents for response generation.
A major concern when it comes to using LLMs as conversational agents is ensuring generated responses are faithful to the contents in the documents fed in the prompt. We are exploring new paradigms of using LLMs, where we first predict an evidence necessary to generate a response and then check if the generated response is faithful to the evidence as shown in the figure below:
Foundation model for digital interaction/structured/unstructured
Most of the pain points for customers in digital interaction space revolves around two main buckets :
1. Expensive to get New insights
Need significant effort to answer a new question / use case.
i. Current models struggle on unseen use cases.
ii. There are use cases that current models cannot even do (example: There is no way to communicate what actually happened with the user from digital interaction data in the form of human-readable natural language).
2. Efficiency
Need large amount of training data readily available to create a new model, Expensive to maintain lots of models Effort in incorporating a new model.
We hypothesize both the problems can be handled if the tons of data exhaust generated by digital interaction is used to its full potential. To do that, we are exploring ways to build foundational models for digital interaction data by solving the necessary research challenges around modelling interaction data, identifying right pre-training objectives, ensuring cost effectiveness and efficiency to name a few. It is a very new research direction in the world of digital interactions with an ambitious goal of supporting multiple down stream tasks, often new unseen task with the same foundational model with minimal tuning. Examples of downstream tasks may include : predicting a user’s next action, analyzing a user groups activity footprint to match a product with its target user group or initiate a personalized dialog flow with a customer.
Publications
- Dinesh Raghu
- Nikhil Gupta
- et al.
- 2021
- ACL-IJCNLP 2021
- Dinesh Raghu
- Shantanu Agarwal
- et al.
- 2021
- EMNLP 2021
- Song Feng
- Siva Sankalp Patel
- et al.
- 2021
- EMNLP 2021
- Bishwesh Mohapatra
- Gaurav Pandey
- et al.
- 2021
- EMNLP 2021
- Dinesh Raghu
- Atishya Jain
- et al.
- 2021
- ACL-IJCNLP 2021
- Gaurav Pandey
- Dinesh Raghu
- et al.
- 2020
- AAAI 2020
- Dhiraj Madan
- Dinesh Raghu
- et al.
- 2020
- ECAI 2020