You are asked to develop a web-based decision aid that predicts the probability of
a COVID-19 diagnosis from patient features. To do so, you will transform
patient-level data available on this page into a knowledgebase. You then apply the DEMI algorithm
to the knowledgebase to predict probability of COVID-19. Finally,
you
build an HTML interface that runs in a web browser on your computer or
online. You may use AI to help
write and revise code for this assignment. You are responsible for understanding the code,
checking its calculations, and documenting its limitations. AI
coders make mistakes. You have to identify and remove these errors.
Section 1: Understand the Data and Define the Prediction Task
Review the COVID-19 course resource, including its data, knowledgebase structure, DEMI procedure, and example decision aid.
Define the outcome your decision aid will predict. Explain how a COVID-19 diagnosis is represented in the data.
Identify the patient features available before the diagnosis is known. Exclude information that would reveal the outcome or become
available only afterward.
Review data dictionary identifying each variable, its meaning, its possible values, and how missing information is
represented.
Here are some questions asked by students about this project:
How should I determine whether a variable could reveal the
COVID-19 diagnosis? You do not need to determine the
likelihood of COVID-19 given various pieces of information. DEMI
algorithm will do so.
What fields and calculations must be included in every
knowledge base entry? Please include symptoms, medical
history, and home test results.
What test cases or expected outputs should I use to verify
the algorithm? You should develop the test cases as
combinations of symptoms, medical history and home tests and notice
the change in likelihood of COVID-19 predicted by the system already
on the web and your system.
What is needed for the final report? See later
sections here.
Section 2: Construct the Knowledgebase
A patient-level database contains individual patients’ features and outcomes. A knowledgebase summarizes relationships between variables
across patients. The knowledgebase should contain the information required by the course’s DEMI procedure without retaining individual patient
records. Aggregation reduces disclosure of patient information but does not automatically guarantee de-identification.
Design a knowledgebase structure that stores relationships between pairs of variables, including relationships among patient
features and between features and the diagnosis.
Specify each knowledgebase field, its meaning, and how it will be calculated.
Provide to an AI system the patient level data, the structure of
the knowledgebase, the calculation requirements to transform patient
level information into the knowledgebase, and an
example of the transformation. Ask the AI to write code that converts
the patient level data into the knowledgebase.
Review the code and execute it on the patient-level database.
Independently calculate several knowledgebase entries from the patient-level data. Include relationships involving the outcome
and relationships between patient features.
Compare your calculations with the generated knowledgebase. Correct discrepancies.
AI systems make mistakes.
Section 3: Apply the DEMI Algorithm
Examine the DEMI procedure. Identify the knowledgebase entries it
uses. Predict for a synthetic patient from the knowledgebase the
probability of COVID.
Ask an AI system to
write code that applies DEMI to your knowledgebase and predicts the probability of a COVID-19 diagnosis.
Review the code against the decision aid provide on this page.
Make sure that the implementation handles absent features, unknown features, and relationships unavailable in the knowledgebase.
Follow the course decision aid where it specifies these rules; document any additional assumptions.
Section 4: Build the HTML Decision Aid
Create a single HTML file containing the interface, embedded knowledgebase, and DEMI calculation code. You may include CSS and
JavaScript within the HTML file. All predictions must run in the browser without a server-side program, external AI service, or external
calculation library.
Ask an AI system to design an HTML interface that collects the patient features used by your model,
reports the probability of COVID and explains its findings.
Allow users to distinguish an absent symptom from an unknown or unanswered symptom.
Include instructions, a prediction button, and a reset button.
Display the predicted probability of a COVID-19 diagnosis with a clear explanation of its meaning.
Include a visible statement that the tool is an educational demonstration and is not intended for clinical diagnosis.
Ensure that the published HTML contains no individual patient records. Describe whether any user-entered information is stored or
transmitted.
Section 5: Compare Your Decision Aid with the Available Aid
Open the COVID-19 decision aid available through the course
Compare the course example COVID clinician with your implementation. Examine:
Patient features and input choices.
Knowledgebase content and structure.
DEMI calculations.
Handling of absent, unknown, or missing information.
Presentation and explanation of predictions.
Input validation, reset behavior, and accessibility.
Educational notices and stated limitations.
External files, services, or other dependencies.
Report features present in the course example but missing from yours, and features present in yours but missing from the course example.
Distinguish differences in appearance from differences that could change predictions.
Enter 10 patient profiles in both tools and compare their predictions. Investigate discrepancies.
Section 6: Publish (optional)
You can choose to publish your decision aid to the web site. The
publication of a decision aid can help your career but is not required for
this assignment.
Upload the HTML file to a web host
Open the published link in a separate browser session and confirm that it loads.
Repeat representative prediction tests on the published version.
Check the published page on a computer and a phone-sized screen.
Confirm that the educational notice is visible and that no patient-level development or test data have been published.
Email the URL to all students in the course through Canvas
Final Submission
Submit one report organized, including:
The knowledgebase and code used to construct it.
The verified DEMI prediction code.
The complete HTML file and optional working web link.
Details of testing, including calculation checks, prediction results, and browser test results.
The one-on-one comparison with the course decision aid.
An appendix containing the AI prompts and relevant responses.
A brief statement explaining how AI assisted your work, what you independently verified, and what limitations remain.
Your work will be assessed on the correctness of the knowledgebase, the
DEMI predictions, the quality of verification and evaluation, the
functionality of the HTML interface, and the completeness of your comparison and documentation.
