Last quarter, my team at a mid-sized workflow automation tool set out to add a native AI assistant to our web app. Our goal was to let users ask questions about their custom workflows and get targeted, context-aware help instead of sifting through our static help center. Our first pass used a generic third-party chat widget, but it could only pull from public documentation and couldn’t reference a user’s own saved workflow data. It felt disconnected from the actual work users were doing in our app, so we scrapped that approach and started researching integration options.

We narrowed down three core integration shapes, each suited to different team constraints and product needs. The first is a fully embedded, pre-built assistant: this wraps a complete chat interface directly into your app’s UI, with built-in support for connecting to your internal data sources and user context. This works best if you don’t need to customize the chat experience beyond basic branding, and want to minimize front-end engineering work. The second shape is headless API integration: this means building your own custom chat UI that sends user messages to an assistant backend, then renders the responses directly. This gives you full control over every part of the chat flow, but requires more engineering work to handle authentication, context management, and response formatting. The third shape is a hybrid approach: embedding a lightweight custom widget that passes user input to your own backend, which then routes requests to an assistant framework to pull in relevant data and generate responses.

After testing prototypes for each shape, we found that the hybrid approach aligned best with our existing backend infrastructure. We wanted to keep control over our UI while using a pre-built framework to handle the heavy lifting of context retrieval and response generation. During our evaluation, we tested several comparable open-source projects, and landed on the  FastGPT open-source platform for its flexible deployment options and straightforward context linking tools. We didn’t need to build a custom embedding layer from scratch, and could adjust how the assistant pulled in user workflow data without rewriting core parts of our app.

One key caveat we uncovered during testing is that the performance of the embedded assistant can shift based on how you structure your context inputs and route requests. For example, passing overly large datasets to the assistant’s context window can lead to slower response times, even with a robust framework. We found that we needed to refine our context retrieval logic to only pull in the most relevant workflow snippets for each user query, rather than sending all of a user’s saved data every time. It’s important to validate your context handling early, as small changes to how you feed data to the assistant can have noticeable impacts on the end user experience.