AI Horizons Network

IIT Bombay

Robust Multimodal Interaction

Goal

Develop a robust, interactive AI system for multimodal question answering and joint human-machine problem solving related to text, images, video and audio.

Technologies

Representation adaptation, domain generalization, transfer learning Weakly supervised, unsupervised, and zero-shot learning Deep learning models for interpretability and explainability Fusion of text, images, video and audio into decision-making Complex, multi-modal question answering.

Research team

Soumen Chakrabarti and Karthik Sankaranarayanan (IBM), Pushpak Bhattacharyya, Sunita Sarawagi, Ganesh Ramakrishnan, Preethi Jyothi, Samarth Bharadwaj, Anupama Ray, Shreya Khare, Tejas Dhamecha, Vishwajeet Kumar.

Bold text denotes principal investigator

Learn more

University website

Press release

University of São Paulo - C4AI

Advancing artificial intelligence in Brazil

Goal

To produce advanced research in Artificial Intelligence in Brazil, disseminate and debate its main results, train students and professionals, and transfer the technology to society.

Technologies

NLP of Portuguese language, knowledge-enhanced machine learning, graph-oriented machine learning, and causal multi-criteria decision-making applied to ocean data, food production chains, and stroke characterization and rehabilitation. Also, studying the impact of AI in work in emerging countries and public policies to foster and regulate AI.

Research team

Fabio Cozman (USP Director), Claudio Pinhanez (IBM PI), Fernando Osório, Glauco Arbix, Marcelo Finger, Thiago A. S. Pardo, Eduardo Tannuri, José Krieger, Zhao Liang, Antonio Saraiva, Alexandre Delbem, João Paulo Veiga

Learn more

Center for Artificial Intelligence

IBM Research Blog

University of Illinois Urbana-Champaign

Center for Cognitive Computing Systems Research (C3SR)

Goal

Developing the stack of applications, accelerators, software, hardware and networking needed to support larger and more complex cognitive workloads.

Technologies

Cognitive application builder, creative experiential learning advisor, cognitive workload acceleration, new system architectures for cognitive systems

Research team

Wen-mei Hwu (UIUC), Jinjun Xiong (IBM)Mohamed El-Hadedy, Haizi Yu, Carl Pearson, Xinheng Liu, Xiaofan Zhang, Raymond Yeh, Abdul Dakkak, Cheng Li, Nguyen Mac, Ashutosh Khar, David Min, Tarek Sakakini , Hongyu Gong, Xiou Ge, Omer Anjum, Ketan Hemant H Date

Bold text denotes principal investigator

Learn more

University website

Press release

Rensselaer Polytechnic Institute

Artificial Intelligence Research Collaboration (AIRC)

Dedicated to advancing the science of artificial intelligence and enabling the use of AI and machine learning in research investigations, innovations, and applications of join interest to both Rensselaer and IBM

Research team

James Hendler (RPI), Pin-Yu Chen (IBM)

Learn more

Artificial Intelligence Research Collaboration

Rensselaer Polytechnic Institute

Cognitive and Immersive Systems Lab (CISL)

Goal

Building environments that explore and advance natural, collaborative problem-solving among groups of people and machines, with the goal of improving how people work together to make decisions.

Technologies

Multimodal interaction and dialog, Spatial context, Spherical microphone array, Group dynamics understanding, Situations reasoning and planning, Collaborative decision making, Narrative/Analogy generation, Data visualization and sonification, Multimodal storytelling.

Research team

Selmer Bringsjord (RPI), Jonas Braasch (RPI), Hui Su (IBM), Kevin Blissett, Zev Battad, Samuel Chabot, Gyanendra Sharma, Kang Wang, Matt Peveler, Robert (Boning) Dong, Devavrat Jivani, Atriya Sen, Rui Zhao, Wennan Zhu, Robert Rouhani, Ryan Weaver, Anitra Chowdhury, Rahul Divekar, Corey Robinson, Rose Clare Pisacano, Anna Stephenson, Huang Zou, Yueqing (Eve) Dai, Guangyuan Li, Lilit Balagyozyan, Hongyang (Lion) Lin, Shuyue (Chelsea) Zheng, Yihao Zhu, Haoming Li, Lisheng Ren, Ziyi Song

Bold text denotes principal investigator

Learn more

University website

Publications

University of Stuttgart

Knowledge-Language Interaction Project

Goal

We are generally interested in the relationship between "unstructured" information sources (primarily natural language text, but also figures) and "structured' information sources (tables, databases, knowledge graphs). How can knowledge be extracted from unstructured information sources and appropriately represented to be acted upon by computers? How can knowledge be turned into text and figures describing it? How can knowledge from different sources be merged?

Technologies

Deep learning, low-shot machine learning techniques, word embeddings, language models, structural analysis (parsing) of natural language sentences and document structure, natural language generation, data augmentation, human-in-the-loop techniques, joint modeling of images and text, knowledge graphs, entity and relation extraction, question answering.

Research team

University of Stuttgart: Jonas Kuhn, Sebastian Pado, Ngoc Thang Vu, Lukas Fromme, Sean Papay, Maximilian Schmidt, Simon Tannert, Moritz Völkel

IBM Research Zürich: Anika Schumann, Antonio Foncubierta Rodriguez, Cristiano Malossi, Jasmina Bogojeska, Peter Staar, Teodoro Laino

Bold text denotes principal investigator

Learn more

Press release

Institute for Natural Language Processing

Rensselaer Polytechnic Institute

Health Empowerment by Analytics, Learning, and Semantics (HEALS)

Goal

Advancing the understanding of chronic condition prevention in the pre-disease stage by focusing on the identification of actionable personal-risk determinants via a coupling of the knowledge of lifestyles risks and data-driven health causal and correlational factors.

Technologies

Multi-model semantics graph integration, NLP of health forum, Personal health data wrangling, Automatic behavior phenotypes identification and analysis, Watson powered conversation agents, Engagement Apps.

Research team

Jim Hendler (RPI), Ching-Hua Chen (IBM), Henry Chang (IBM), Yu Chen, Sabbir Rashid, Jonathan Harris, Dylan Elliott, Oshani Seneviratne, Ryan Sherman

Bold text denotes principal investigator

Learn more

Press release

UC San Diego

Artificial Intelligence for Healthy Living (AIHL) Center

Goal

To develop and evaluate AI technology solutions that enable older adults to live independently longer and have a higher quality of life; to develop machine learning methods and software tools, and generate novel findings, that implicate the human microbiome in health and disease.

Technologies

Behavior prediction models and prevention innovation from ML by using datasets of batteries, video, audio, accelerometer, and other wearables; Human gut bacteria-disease association knowledge base built by Natural Language Processing, ML methods for analysis of microbiome datasets and phenotype prediction.

Research team

Ruoyi Zhou (IBM), Ho-Cheol Kim (IBM), Dilip Jeste (UCSD), Rob Knight (UCSD), (IBM): Peri Tarr, Yasu Yamada, Laxmi Parida, Niina Haiminen, Yannis Katsis, Peter Fay, Ban Kawas, Anna Paola Carrieri; (UCSD): Colin Depp, Danielle Glorioso, Camille Nebeker, A’verria Martin, Elizabeth Twamley, Sandrine Miller-Montgomery, Austin Swafford, Chun-Han Hsu, Siavash Mir Arabbaygi

Bold text denotes principal investigator

Learn more

Press release

UCSD Center for Healthy Aging

UCSD Center for Microbiome Innovation (CMI)

Université de Montréal

Montreal Institute for Learning Algorithms (MILA)

Goal

Developing next-generation deep learning algorithms and techniques to help computers improve their understanding and interpretation of language, speech, and vision.

Technologies

Deep learning, generative models, adversarial learning, reinforcement learning, natural language processing, dialogue, speech recognition, hardware acceleration of inference with deep models.

Publications

Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generation
Proc. AAAI, 2017
I. V. Serban, T. Klinger, G. Tesauro, K. Talamadupula, B. Zhou, Y. Bengio, and A. Courville

A Structured Self-Attentive Sentence Embedding
Proc. International Conference on Learning Representations
I. V. Serban, T. Klinger, G. Tesauro, K. Talamadupula, B. Zhou, Y. Bengio, and A. Courville

Pointing the Unknown Words
Proc. ACL, 2016
Ç. Gülçehre, S. Ahn, R. Nallapati, B. Zhou, and Y. Bengio

Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond
Proc. SIGNLL Conference on Computational Natural Language Learning (CoNLL), 2016
R. Nallapati, B. Zhou, C. dos Santos, Ç. Gülçehre, and B. Xiang

Invariant Representations for Noisy Speech Recognition
NIPS Workshop on End-to-End Speech Recognition, 2016
D. Serdyuk, K. Audhkhasi, P. Brakel, B. Ramabhadran, S. Thomas, and Y. Bengio

Oracle performance for visual captioning
NIPS Workshop on End-to-End Speech Recognition, 2016
L. Yao, N. Ballas, K. Cho, J. R. Smith, and Y. Bengio

Empirical performance upper bounds for image and video captioning
International Conference on Learning Representations (ICLR), 2016
L. Yao, N. Ballas, K. Cho, J. R. Smith, and Y. Bengio

Research team

Yoshua Bengio (Montreal), Brian Kingsbury (IBM), Anirudh Goyal , Jose Sotelo, Philemon Brakel , Sungjin Ahn , Dzmitry Bahdanau , Xavier Bouthillier , Junyoung Chung , Akram Erraqabi , Kyle Kastner , César Laurent , Ahmed Touati , Ying Zhang, Aristide Baratin , Tom Bosc , Tong Che, Laurent Dinh , Taesup Kim, Samuel Laférière , Dmitriy Serdyuk, Olexa Bilaniuk, Matthieu Courbariaux, Rosemary Ke

Bold text denotes principal investigator

Learn more

Blog

University website

University of Massachusetts at Amherst

Information Extraction and Synthesis Lab (IESL)

Goal

Combine symbolic and neural representation and reasoning techniques to develop Deep Reasoning: systems that understand comprehensively, know deeply, reason with purpose, learn continuously, and interact naturally.

Technologies

Knowledge Representation and Reasoning platforms and corpora; probabilistic reasoning about Knowledge Base changes; Common-sense representation of entities and types; Neural implementations of multi-step common-sense reasoning for question answering.

Research team

Andrew McCallum (UMass), Michael Witbrock (IBM), Tian Gao, Trapit Bansal, Jeff Flanigan, Cristina Cornelio, Nicholas Monath, Pat Verga, Pavan Kapanipathi, Maria Chang, Rajarshi Das, Xiang Li, Enara Vijil, Haw-Shiuan Chang, Mohit Yadav, Sheshera Shashidhar

Bold text denotes principal investigator

Learn more

University website

IIT Delhi

Neuro-Symbolic Information Systems

Goal

Develop a Generalized Comprehension Engine which moves us along the artificial general intelligence task of enabling a computer to use and ingest background knowledge from disparate domains to drive comprehensive reading, reasoning, question answering and multi-step dialog.

Technologies

Reasoning, dialog and Q&A, explainability, neuro-symbolic models

Research team

Srikanta Bedathur (IITD PI), Mausam, Parag Singla, Maya Ramanath, Niladri Chatterjee, Sayan Ranu, Amitabha Bagchi, Dinesh Garg, Hima Karanam, Koyel Mukherjee, Sumit Bhatia, Shajith Mohamed, Sachindra Joshi, L V Subramaniam (IBM PI)

Bold text denotes principal investigator

Learn more

University website

Press release

Massachusetts Institute of Technology

Laboratory for Brain-inspired Multi-Media Machine Comprehension (BM3C)

Goal

Developing machines that can emulate the human ability to understand inputs from multiple video streams and predict potential future events in real-time.

Technologies

Short activities recognition (in 3 seconds videos), visual Q&A, generative models for simulated worlds.

Research team

Josh Tenenbaum, Josh McDermott, and Danny Gutfreund (IBM), Andrei Barbu, Pouya Bashivan, Maddie Cusimano, Yen-Ling Kuo, Jonathan Malmaud, Mathew Monfort, Candace Ross, Guy Ben-Yossef, Sue Felshin, David Mayo, James Traer

Bold text denotes principal investigator

Learn more

Moments in Time Dataset: A large-scale dataset for recognizing and understanding action in videos

@ IBM Research

Press release

University of Michigan

Sapphire Project

Goal

Developing technology to allow people to interact more naturally and effectively with computers through text or speech dialogues.

Technologies

Discourse disambiguation, structured disentanglement, NL queries over relational & knowledge bases, emotion detection, dialog action & next utterance prediction, multiparty conversation dynamic management, response generation.

Research team

Satinder Singh (Michigan), David Nahamoo (IBM), Lazaros Polymenakos (IBM), Jonathan Kummerfeld, Mason Hill, Stephanie O'Keefe, Janarthanan Rajendran, Lajanugen Logeswaren, Rui Zhang, Sai Gouravajhala, Falk Pollok, Charlie Welch, Zakaria Aldeneh, Harmanpreet Kaur, Lucy Jiang , Janarthanan Rajendran , Matthew Skach , Ram Kannan, Vignesh Athreya, Fei Li

Bold text denotes principal investigator

Learn more

University website

University of Maryland at Baltimore County

Accelerated Cognitive Cybersecurity Lab (ACCL)

Goal

Applying cognitive computing to cybersecurity via analytics and machine learning, while also exploring specialized computer power optimized for these new intensive computing workloads.

Technologies

Advanced Named Entity Recognition (NER), Relation Extraction (RE), Event Extraction (EE) and text summarization tools to support effective text analysis for cybersecurity

Research team

Anupam Joshi (UMBC), JR Rao (IBM), Sudip Mittal, Ankur Padia, Arpita Roy, Sandeep Narayanan, Priyanka Ranade, Ashwinikumar Ganesan, Askhay Peshavve, Pooja Parmeswarappa, Kaushik Veluswamy

Bold text denotes principal investigator

Learn more

University website

Press release