Explore how AI can simplify proposal prep and strengthen research at Duke.

Welcome to the Devil's Advocate Pilot

Devil’s Advocate is Duke’s new AI-powered grant review assistant — designed to help researchers and grant administrators identify potential compliance risks and strengthen proposal narratives before submission.

We’re inviting faculty and grant managers from across Duke to test the tool and share feedback. Whether you’re working on an NIH, NSF, or other sponsor proposal, your participation will help us refine how AI can support Duke’s research community. The current version is optimized for NIH-style formats but can also be used with other proposal types. 

Designed to complement the expertise of Duke’s research administrators and reviewers, Devil's Advocate automates routine checks, freeing time for deeper advising, collaboration and scientific refinement —  improving overall proposal quality.

About the Devil's Advocate Pilot

Think of it as an AI “sanity check,” not a reviewer. It helps catch small but important details that can delay or weaken a proposal:

  • Human Subjects – “Did you mention that survey in Aim 3 needs IRB approval?”
  • Animal Research – “That mouse study in your prelim data may require IACUC.”
  • Foreign Components – “Your collaborator at Max Planck might trigger this requirement.”
  • Cost Share – “This phrasing could be read as a cost-sharing commitment.”
  • Citation Cross-Check – “Citation 47 doesn’t seem to support this claim.”
  • Red Team / Strawman Review – “Your power calculation or experimental design could raise reviewer questions.”
  • DOGE-Prohibited Terms – “This language may not align with current federal priorities.”

You decide what matters — Devil’s Advocate just flags what might be worth a second look.

How it Fits Into Your Workflow

  • Minimal disruption. Maximum insight.
  • Upload your proposal PDFs (abstract, narrative, specific aims, research plan, literature cited).
  • Select which modules to run — skip anything that doesn’t apply.
  • Your grant administrator automatically has access to the latest versions.
  • Download everything in a clean ZIP with systematic file names, ready for SPS upload.
  • Use the semantic diff viewer to see meaningful changes between document versions — no more Final_v3_really_final.pdf
Who Can Join

The Devil’s Advocate Pilot is open to Duke faculty, researchers, and grant administrators preparing proposals for any sponsor — NIH, NSF, DOE, foundations, or others.

The current version is optimized for NIH-style formats but can also be used with other proposal types (simply select any NIH mechanism when prompted).

We especially welcome participation from:

  • Faculty who want to test how AI can strengthen proposal clarity and compliance.
  • Grant administrators who manage multiple submissions and want to streamline final reviews.
  • Research staff and trainees preparing fellowship or career development applications.

Your participation helps shape how Duke applies AI to real-world research workflows.

Why Participate

Your participation helps Duke:

  • Identify edge cases and refine AI accuracy
  • Streamline compliance reviews and reduce back-and-forth edits
  • Save time during final submission
  • Strengthen proposal clarity and competitiveness

Even if you don’t act on any suggestions, your testing provides valuable insight that improves the tool for everyone at Duke.

How to Participate
  1. Visit: https://devils-advocate.oit.duke.edu/ (Duke network or VPN required)
  2. Log in with your Duke NetID to access your research portfolio.
  3. Create a new project by entering the PAR or mechanism (e.g., R01, R21, R35).
  4. Upload proposal files – abstract, narrative, specific aims, research plan, and literature cited (PDFs).
    1. A research plan and literature cited are required to run scans.
  5. Select the scans you’d like to run. Start with the compliance modules, then try the citation analysis and strawman modules for deeper narrative feedback.
  6. Review results – flags appear as “sanity checks,” not prescriptions.
  7. Download all files in a clean, SPS-ready ZIP with clear versioning and naming conventions.

Your feedback will help refine the modules and inform how we expand Devil’s Advocate across disciplines and sponsor types.

Tips
  • Use real proposals for the most meaningful feedback.
  • Devil’s Advocate is an AI assistant, not a reviewer — treat its flags as advisory, not final.
  • The Strawman and Citation Analysis modules can take longer to process; run them separately for best results.
  • The tool works across disciplines — feedback from non-biomedical proposals is especially valuable.
Share Your Feedback

We want to hear how Devil’s Advocate performs across different research domains.
Was the feedback useful? Were there false positives or missing checks?

Share your input in one of these ways:

  • In-app feedback – flag results that seem off or particularly helpful.
  • Email us – send feedback or screenshots directly to duke-grant-ai-help@duke.edu.
  • Follow-up discussions – select pilot participants may be invited for short feedback sessions.

Your real-world use and insights directly influence how Duke improves and expands this tool.

Data & Security

Devil’s Advocate operates entirely within Duke’s secure computing environment.
All proposal documents remain on Duke-managed servers and are never transmitted to external AI providers.

Access is limited to authorized users (typically the Principal Investigator and their grant administrator) through Duke Grouper permissions.
Uploaded files and results are used only for pilot testing and system improvement — not for compliance certification or external sharing.

Our goal is to ensure that AI-assisted grant review at Duke remains confidential, compliant, and secure, in full alignment with Duke’s data governance and research integrity standards.

Support 

For questions or feedback beyond the in-chat feature, contact the Devil's Advocate team at  duke-grant-ai-help@duke.edu