Differences

This shows you the differences between two versions of the page.

--- 2024-25:fall:f24-listing [2025/01/13 16:46] – [Evaluating Planning Domain Validation Tools] lesperan
+++ 2024-25:fall:f24-listing [2025/01/13 16:48] (current) – lesperan
@@ Line 502: / Line 502: @@
+==== Evaluating Planning Domain Validation Tools ====
+**[added 2024-09-18]**
-==== Exploring the Use of Large Language Models to Generate Control Knowledge for HTN Planning ====
-**[added 2025-01-13]**
 **Course:**  { EECS4080 }
@@ Line 517: / Line 513: @@
 **Project Description:**
-Hierarchical Task Network (HTN) planning is a popular approach to automated planning with domain-specific search control knowledge.  In HTN planning, the objective is to find a sequence of actions to perform an abstract goal task, and the plan is generated by decomposing the root goal task into simpler tasks and actions using a handcrafted set of “refinement” methods.  However, developing such a set of methods and the associated subtasks requires knowledge of the problem domain and normally necessitates the involvement of a knowledge engineer and/or domain experts.
+In this project, the student will evaluate software tools (such as Val, FastDownward, and the "unquestionable parser for PDDL 3.1") that are used to validate planning domains and planning problems specified in the Planning Domain Description Language (PDDL), focussing on the STRIPS fragment.  This is part of a larger project to use Large Language Models to generate abstract planning domain models to support more efficient planning and provide explanations at an abstract level by supressing uninteresting details from domain models.  The validation tool selected will become part of a neuro-symbolic systems to help users develop such abstract planning models.
-Given that Large Language Models (LLMs) can display broad world knowledge, it seems reasonable that they should be able to help generate methods and tasks for a given HTN planning domain and root goal task.  That is, given the domain’s atomic tasks/operators, the root goal task, and the initial state specification, the LLM should be able to fill in the methods and abstract tasks (possibly with some human help), in order to generate a complete HTN planning problem, which could then be given to an HTN planner to have it generate a plan to accomplish the goal task.  The LLM-generated methods and abstract tasks should embody good “control knowledge” for accomplishing the goal task in the domain, similar to what a human engineer could produce.  Let’s call this the HTN control knowledge generation problem.
-In this project, after learning the basics of HTN planning, the student will first adapt some existing HTN planning problems to be used as benchmarks in evaluating the ability of LLMs to solve the HTN control knowledge generation problem, essentially by deleting some or all of their methods and abstract tasks.  The student will also engineer prompts to try get the LLM to produce good quality solutions to the HTN control knowledge generation problem.  The performance of the LLM will then be evaluated in some experiments using the benchmark problems.   As part of this, the student will feed the LLM-generated HTN planning problems to an HTN planner and see if it can generate plans to solve them, providing one measure of correctness.
 **Required skills or prerequisites:**
@@ Line 540: / Line 531: @@
-==== Evaluating Planning Domain Validation Tools ====
+==== Exploring the Use of Large Language Models to Generate Control Knowledge for HTN Planning ====
-**[added 2024-09-18]**
+**[added 2025-01-13]**
 **Course:**  { EECS4080 }
@@ Line 551: / Line 542: @@
 **Project Description:**
-In this project, the student will evaluate software tools (such as Val, FastDownward, and the "unquestionable parser for PDDL 3.1") that are used to validate planning domains and planning problems specified in the Planning Domain Description Language (PDDL), focussing on the STRIPS fragment.  This is part of a larger project to use Large Language Models to generate abstract planning domain models to support more efficient planning and provide explanations at an abstract level by supressing uninteresting details from domain models.  The validation tool selected will become part of a neuro-symbolic systems to help users develop such abstract planning models.
+Hierarchical Task Network (HTN) planning is a popular approach to automated planning with domain-specific search control knowledge.  In HTN planning, the objective is to find a sequence of actions to perform an abstract goal task, and the plan is generated by decomposing the root goal task into simpler tasks and actions using a handcrafted set of “refinement” methods.  However, developing such a set of methods and the associated subtasks requires knowledge of the problem domain and normally necessitates the involvement of a knowledge engineer and/or domain experts.
+Given that Large Language Models (LLMs) can display broad world knowledge, it seems reasonable that they should be able to help generate methods and tasks for a given HTN planning domain and root goal task.  That is, given the domain’s atomic tasks/operators, the root goal task, and the initial state specification, the LLM should be able to fill in the methods and abstract tasks (possibly with some human help), in order to generate a complete HTN planning problem, which could then be given to an HTN planner to have it generate a plan to accomplish the goal task.  The LLM-generated methods and abstract tasks should embody good “control knowledge” for accomplishing the goal task in the domain, similar to what a human engineer could produce.  Let’s call this the HTN control knowledge generation problem.
+In this project, after learning the basics of HTN planning, the student will first adapt some existing HTN planning problems to be used as benchmarks in evaluating the ability of LLMs to solve the HTN control knowledge generation problem, essentially by deleting some or all of their methods and abstract tasks.  The student will also engineer prompts to try get the LLM to produce good quality solutions to the HTN control knowledge generation problem.  The performance of the LLM will then be evaluated in some experiments using the benchmark problems.   As part of this, the student will feed the LLM-generated HTN planning problems to an HTN planner and see if it can generate plans to solve them, providing one measure of correctness.
 **Required skills or prerequisites:**
@@ Line 565: / Line 561: @@
 ----
 ==== TEMPLATE ENTRY 10  - PUT PROJECT TITLE HERE====