Syllabus bot for programs #2273
Merged
+404
−26
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
OpenAPI ChangesShow/hide 184 changes: 0 error, 0 warning, 184 infoUnexpected changes? Ensure your branch is up-to-date with |
2 tasks
shanbady
added
Needs Review
mbertrand
reviewed
Jun 3, 2025
There was a problem hiding this comment.
Something I noticed in initial testing, will keep trying it out:
Question 1: What is this program about?
Query 1: "resource_readable_id": ["course-v1:MITxT+14.100x", "course-v1:MITxT+14.310x", "course-v1:MITxT+JPAL102x", "course-v1:MITxT+14.009x", "course-v1:MITxT+14.003x"]
Response 1: The program is part of the MITx MicroMasters in Data, Economics, and Design of Policy (DEDP). This program offers....
Question 2: What are all the courses I need to take for this program?
Query 2: "resource_readable_id": ["course-v1:MITxT+14.100x", "course-v1:MITxT+14.310x", "course-v1:MITxT+JPAL102x", "course-v1:MITxT+14.009x", "course-v1:MITxT+14.003x", "course-v1:MITxT+14.100x", "course-v1:MITxT+14.310x", "course-v1:MITxT+JPAL102x", "course-v1:MITxT+14.009x", "course-v1:MITxT+14.003x"]
Response 3: "How can I assist you further with the MITx MicroMasters program in Data, Economics, and Design of Policy (DEDP)?"
Question 3: What are all the courses I need to take for this program? <same as before, since last response was not good - maybe summarization kicked off to cause that, will check later>
Query 3: "resource_readable_id": "program-v1:MITx+DEDP-Public-Policy"
Response 3:
The MITx MicroMasters program in Data, Economics, and Design of Policy (DEDP) requires you to complete the following courses:
Core Courses (Complete all):
Microeconomics - Starts May 13, 2025
Designing and Running Randomized Evaluations - Starts September 9, 2025
Data Analysis for Social Scientists - Starts May 13, 2025
Elective Courses (Public Policy Track):
Good Economics for Hard Times - Starts May 13, 2025
Microeconomic Theory and Public Policy - Starts May 13, 2025
|
Tried the same initial questions in a new private tab and this time it gave a good response to the required courses question on the first try, so maybe the initial attempt was just a fluke:
|
|
fwiw, the summarization code as currently written can eventually causes issues when multiple tool calls are made at once, there is a fix for that in my current learn-ai PR. |
|
More followup questions after the initial good response to the required courses question: Question: What are the 3 core classes? Question: How is the "data analysis for social scientists" course graded? |
mbertrand
approved these changes
Jun 3, 2025
There was a problem hiding this comment.
LGTM, 👍
The related learn-ai pr should give better responses with the summarization responses from mitodl/learn-ai#205 applied there.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What are the relevant tickets?
Closes https://github.com/mitodl/hq/issues/7330
related PR: mitodl/learn-ai#206
Description (What does it do?)
This PR does the following:
This PR and associated learn-ai pr adds "child courses" in the api response for learning resources which is then used to let the syllabus bot know the readable ids for courses within a program. We introduce a separate tool with annotations letting the bot know that it should use the "related course search" if we have a list of related readable ids.
How can this be tested?
"search_parameters": {"q": "Downstream Processing course materials", "resource_readable_id":"program-xyz"]}"search_parameters": {"q": "Downstream Processing course materials", "resource_readable_id": ["45c76489-d061-42e1-ab26-3eb5ad7a2297", "d73c1cd1-c965-47b6-8b6e-b6b6e2816d0e”]}Additional Context
My initial pass at this tried to always pass an array for all searches:
readable_id:[program-readable-id, child-course-id, child-course-id2]- this however led to some ambiguity isssues where if you ask "what is the name of this program" it would surface the name of a course within the program etc. To work around that - if its a program related question, just pass program id. anything related to specifics of a course in the program, pass the array of courses.Checklist: