Duke's AI Tools: A year in review

At the second annual ‘AI in Education at Duke’ summit late last month, attendees heard from a variety of professors, technologists, and leaders sharing how they have engaged with AI over the past year.  Organized by the Center for Teaching and Learning and its partners, engagement included everything from the development of custom AI tools, AI integrations with their courses, and identifying critical use cases, and workflows that changed how they teach and operate. 

What we didn't hear much of this year was questions about access.  Nobody raised infrastructure as a problem or spent time advocating for resources that are needed to do this work. For a community that was focused on that shortcoming just a year ago, the absence of that discussion was a win.

Since the first summit in spring 2025, Duke OIT launched a set of tools developed internally that we hoped would give the Duke community a flexible, self-controlled way to engage with AI: DukeGPT, MyGPT Builder, and the AI Gateway. In combination with Microsoft Copilot and ChatGPT Edu, these tools comprised what we called the Duke AI Suite. 

The landscape has changed dramatically over the past year, our tools have evolved in both their execution and the needs they are fulfilling, and we've learned a lot. Here's where we've been, what we built, and where things stand heading into Fall 2026.

Why we built it

When the Duke AI Suite launched, the landscape was genuinely uncertain, so we prioritized building tools internally, long before our conversations with OpenAI. We started off with our own gateway: a Duke-hosted API layer that lets anyone at Duke access AI models through a unified, managed interface.  We wanted to move fast, experiment openly, and keep institutional control over how models were accessed and billed.

During the summer of 2025, the bet was: Let’s build something now, learn from real usage, and be in a better position to make decisions about which commercial and homegrown tools to invest in and how.

We characterized the work we were doing as just being able to get to the 'starting line' of AI at Duke.  We wanted to prioritize expediency over perfection, and use what we built as a way to start to ask the important questions about why and how AI will impact the service, learning, and research missions of Duke.  But before we could do any of that, we needed to level the playing field and provide access.

What we delivered

There are three Duke-developed AI tools in the AI Suite:

DukeGPT 
DukeGPT is a browser-based chat interface, similar to ChatGPT, that any Duke community member can use to interact with AI models. It's currently available at no direct cost to users, and it lets people compare results across different AI models in a single interface.

MyGPT Builder 
MyGPT Builder lets faculty, staff, and researchers create their own custom AI workspaces. These workspaces can be shared with students or teams, configured for specific purposes, and connected to document sets. Over 750 workspaces have been created in the last year.

The AI Gateway
The AI Gateway is the engine underneath it all — a flexible API layer that allows developers, researchers, and power users to connect AI models directly to the software they're building. Rather than managing separate agreements and integrations with every AI vendor, groups at Duke can route their usage through a single, monitored, cost-managed interface.

What we learned

There are some major takeaways from our year in review. 

The Gateway is the story. 
The most important thing we learned is that the API gateway underneath DukeGPT is where most of the value lives. It's the layer that lets Duke groups build on trusted AI without managing their own vendor relationships.

This became especially clear when we looked at where usage was actually coming from. More than half of all traffic through the gateway isn't coming from people browsing DukeGPT — it's coming from teams and developers who have connected the gateway directly to research pipelines, software projects, and tools they've built. One research computing group alone has used the gateway to manage AI access across four different projects, giving team members individualized access while maintaining central budget visibility. That kind of institutional control isn't easily available through most commercial platforms.

Coding with AI is here.
When we launched, the primary use case we had in mind was conversational AI — asking questions, drafting text, exploring ideas. Within a year, the landscape shifted. AI-assisted coding tools are now some of the most actively-used capabilities on the platform, with daily usage growing steadily. Teams doing software development, data analysis, and research computing are connecting AI coding tools directly to the gateway and building them into their workflows.

Embeddings are unsung infrastructure.
Nearly half of all gateway traffic comes from embedding requests, which may seem surprising at first. Embeddings are the technology that allows AI to work with documents and data—searching through large collections of files, supporting retrieval-augmented systems, and helping tools understand and reason over uploaded content. What this shows is that a significant portion of Duke's community is not just using AI for general questions; they are using it to work with their own documents, information, and data.

How the original vision continues to evolve

We originally imagined the gateway would power an ecosystem of Duke-connected tools including custom integrations, campus data sources, purpose-built applications. Some of that came to pass, but we also quickly found that the community's first instinct was to use AI through whatever interface they already trusted (often commercial platforms). Rather than fight that tendency, we focused on complementing it. The AI Suite fills the gaps where commercial platforms don't reach, particularly for access to open-source models and for use cases that benefit from institutional cost management.

We recently retired our on-premise AI model hosting. We had run our own server to host open-source models, but real-world usage showed very few people actually needed it in that form, and we redeployed those resources into Duke's research computing pool. 

At this time, we will direct open-source models through cloud platforms while we re-evaluate where on-prem offerings fit into our model.

Heading into the next academic year, we're planning a few changes:

• More open-source models. Through a partnership with Microsoft Azure, a public cloud computing platform, we're adding several open-source models to the Gateway. For users who want to experiment with non-commercial AI or find budget-friendly alternatives to the major frontier models, the Duke AI Suite will be the place to do that.
• Better tools for teaching and research groups. We're designing class and lab management features such as the ability for an instructor or group lead to provision AI access for an entire team, set per-user limits, and monitor usage — without every individual needing their own fund code or vendor account. This is especially useful for courses, research labs, and training programs.
• Expanded model coverage. Anthropic models are coming to the Azure platform, which will make the AI Gateway a single access point for multiple frontier model families in one place

Over the course of this past year, the context around AI has shifted, and the tools we built turned out to be valuable for many unexpected reasons. We'll continue to respond to the needs of the community, building out a durable infrastructure that is Duke-managed, cost-controlled, and institutionally-accountable for trusted AI access.

Learn more about how you can use Duke's AI tools with an AI consultation. 

Photo credit: Blyth Tyrone, Center for Teaching and Learning.