This project aims to develop a prototype for a dialogue system that personalizes recipe instructions for a dish to a user. Cooking recipes are an instructional text that many people interact with frequently, yet they pose unique challenges for automatic understanding and use in conversational systems. For automatic understanding based on language processing, recipe texts pose specific challenges such as imperative mood (“stir the batter”); implicit arguments (“beat ∅”); and complex anaphoric expressions that relate to intermediate products. For a conversational agent, recipes for the same dish often differ in which cooking actions they describe explicitly and how; it is challenging yet necessary to properly explain steps while taking into account recipe difficulty and the user’s cooking proficiency. A conversational recipe assistant would enhance user interaction with recipes for many users, assisting in recipe selection, instructing steps to the user while cooking, and addressing questions throughout.

Part I of the project will address the linguistic challenges posed by recipe text through semantic parsing. Representing recipes as graphs is a common choice for this task to capture dependencies between ingredients, tools, and actions. We will adapt an existing state-of-the-art semantic parsing pipeline to parse recipes for the same dish and then align actions between recipes for a more detailed dish-level representation. Part II of the project will investigate dialogue management to facilitate sequence expansion of recipe instructions at varying levels of detail for an adaptive conversational agent. The cooking domain is a broad knowledge space, and being able to address questions asked during cooking instruction is not trivial. A good understanding of the recipe itself is required, as is formulating the answer properly to a particular user.

Researchers:

Students: