Overview
- The tool is currently geared toward NIH grants but should work with other proposal types (e.g., NSF—just select any NIH mechanism).
- Some modules, such as Cost Share and Foreign Components, are primarily designed to assist grant administrators by flagging institutional names, people, or services mentioned.
- None of the results are prescriptive; all are intended as sanity checks.
- The Citation Analysis and Strawman modules are meant to help faculty strengthen their narratives.
- Feedback from proposals outside the biomedical domain is especially welcome.
Accessing the Tool
- Link: https://devils-advocate.oit.duke.edu/
- Duke network or VPN required.
- This pilot focuses on usability testing.
Step-by-Step Instructions
- Log in to view your Research Portfolio.
- Create a new project
- Enter the PAR or mechanism (R01, R21, R35, etc.); it auto-fills.
- Returned to your research portfolio view
- You will be returned to your research portfolio view and see your new project card.
- You will be returned to your research portfolio view and see your new project card.
- Click on the project card to open project view.
- Upload your files
- In the upper right corner of the project card will be a blue "manage files" button. This is where you will upload your narrative, abstract, specific aims, research plan and literature cited as individual PDFs. At a minimum, a research plan and literature cited is required to run the scans. If you prefer you can upload the specific aims, narrative and abstract in a later session.
- Run the scans
- Once uploaded, you can close the modal window and the available scan options will be displayed for scanning.
- The pre-selected blue scans are the compliance scans. Suggest you run these first, and then return and run the citation analysis and strawman separately (can take up to 30 minutes to process and can time out by OpenAI).
- Review results
- Flags appear as sanity checks — not prescriptive.
- Examples: missed IRB mention, DOGE-prohibited term, or foreign collaborator flag.
- Once you click run, it will redirect you to the portfolio view so you can watch the progress.
- Next, run the strawman and citation analysis. These are agentic searches to improve rigor and check every claim/citation in the proposal.
- Download results
- Once the scans are complete, you can click on the card to see the results.
- Use “Download All” to get SPS-ready zip files.
- Versioning and semantic “Diff” show meaningful changes between iterations.
- Screenshot placeholder
Notes and Known Issues
- Strawman and Citation modules are computationally heavy and may time out.
- Results are for feedback, not compliance certification.
- Expect bugs during pilot phase.
Feedback and Support
- Email: duke-grant-ai-help@duke.edu (goes to developers and project team).
- Your feedback and ideas for improvement are strongly encouraged.