CopilotKit: Build, Deploy, and Operate AI Copilots with Ease

Lately, in the generative AI era, almost every technology company is finding ways to integrate AI into their software products.

These are also called “copilots” and they let users ask questions in natural language and provide responses specific to the user’s context (the details in their dashboard, the information provided in the query, etc.)

To most software engineers, this appears like a simple thing to do. They think all we have to do is invoke an API, so what could go wrong?

But it is only when they get hands-on that they realize the hidden challenges:

• What is the best way to design and manage prompts?

• How do we consistently maintain an interaction between the LLM and the app state?

• What and how to gather user-specific context from the application?

• When using chatbots, how to manage a conversational state?

• How to test the end-to-end integration workflow?

• And many many more.

Today, I want to tell you about CopilotKit, an open-source copilot platform that lets us integrate AI into ANY application:

1. With very little code.

2. And without caring about the typical integration challenges.

Let’s begin!

A high-level overview of CopilotKit

As mentioned above, CopilotKit is an open-source framework that lets us integrate AI into our applications (currently, it works with React apps) by providing a robust and seamless infrastructure for such integrations.

It does this by providing functionalities that facilitate various tasks that we may need to perform after integrating AI, like:

• Collecting data from the app which will be sent to the model.

• Managing third-party interactions that, say, return an output, which is then sent to the model.

• Performing actions to display the response received from the model.

• Making updates to the application’s backend/frontend if needed, etc.

Additionally, CopilotKit also provides various UI components that can be directly displayed in our application, like:

• CopilotChat: A chat interface that inherently maintains a conversational state.

• CopilotSidebar: Similar to a CopilotChat but displayed as a sidebar instead.

• CopilotTextarea: A text field that provides auto-completions. Just replace the typical <textarea/> tag in the React app with <CopilotTextarea/>.

Isn’t that cool?

CopilotKit Presentation Creator Demo

In this section, let me give you a text-to-powerpoint creator application built using CopilotKit.

Before proceeding further, make sure you have done the following:

• Installed React: Instructions.

• Generated an OpenAI API key here.

• Generated a TAVILY API key here. This is a third-party tool for researching the topic. The output will then be sent to OpenAI to generate content for the slide.

Begin by cloning this repository: Presentation demo.

Navigate to the cloned repository and install the packages:

Now, create a “.env.local” file in the root directory of the project and specify the two API keys obtained earlier:

Done!

Now run the app as follows:

Open http://localhost:3000 in your browser to see the app.

We have a CopilotSidebar here.

Let’s enter this prompt: “Create a slide on the benefits of AI in healthcare.”

We get the desired slide within a few seconds:

On the backend, here’s what CopiloKit did:

• It took the prompt and sent it to TAVILY to research the topic.

• The response was then sent to OpenAI for creating the slide content.

• The output from OpenAI LLM (slide title, pointers, speaker notes, and image) was then placed in the desired places by CopiloKit. It did this using the update functionalities I mentioned earlier.

Isn’t that cool?

Departing note

AI copilots are becoming increasingly popular lately. However, they are really hard for engineers to build from scratch.

In my experience, CopilotKit is possibly the easiest way to integrate AI capabilities into any application while avoiding all the hassles.

CopilotKit also has native support for tools like LangChain, with which we can define a sequence of language processing tasks seamlessly.

Moreover, it also provides functionalities to perform Retrieval Augmented Generation (RAG) from any data source.

Of course, one thing to note is that it currently supports React apps only.

However, the founders of CopilotKit told me that they are actively working on expanding to other languages, Python being one of the most prominent ones, which is expected pretty soon.

I love CopilotKit’s mission of supporting software engineers, data scientists, and machine learning engineers in building AI copilots easily, while CopilotKit takes care of all core functionalities and challenges.

They are solving a big problem, and I’m quite eager to see how they continue!

Do star CopilotKit repo to support them: CopilotKit GitHub.

🙌 Also, a big thanks to the CopilotKit team, who very kindly partnered with me on today’s newsletter and let me share my thoughts openly.

👉 Over to you: What problems will you use CopilotKit for?

Thanks for reading!