5 Lessons-Learned from Creating a CustomGPT

We’ve been working on a CustomGPT for AnswerStage.  Here are five lessons we learned the hard way so you don’t have to:

1) Problems with Paywalls: Currently, Custom GPTs are intended for Plus members or other paying accounts. Free accounts are limited to about 4,000 characters per session, which is enough for a few steps or a short dialog, but not enough for a more involved process like what we’ve created for AnswerStage. Our process involves selecting questions, branding, and talking points for your video. Unfortunately, ChatGPT doesn’t provide a way to track usage, so running out of your allocation feels like hitting a brick wall. You receive a notice saying, “try again in 4 hours,” which makes for a frustrating user experience.

2) Browser vs. ChatGPT App Behavior: We encountered issues with creating and managing accounts and APIs through the different paths of the browser and the ChatGPT app. The behavior within a browser is different from that within the app, which caused complications.

3) Hosted Assistant as a Solution: To address both issues above, we’re creating an Assistant hosted on our domain. The Assistant provides a similar experience to the GPT but interprets instructions slightly differently. It’s a good option for users without paying accounts to complete a session. However, developing an Assistant is probably more work up front but may provide more flexibility in the long run. The disadvantage is that currently you can’t feature Assistants in the ChatGPT store.

4) “Speaking GPT” and Its Quirks: Crafting precise instructions for GPT is an evolving art. Despite our efforts to make instructions specific, GPT sometimes interprets them in unexpected ways or hallucinates. For instance, it created its own names for our video themes like “Modern,” “Elegant,” and “Vibrant.” While interesting, these names don’t match our designs. We’ve adjusted our instructions to eliminate this behavior in most sessions, but it might reappear. Fortunately, these quirks don’t affect the final results.

5) QA Testing Challenges: Quality assurance testing on Custom GPTs can be tricky because you need to log out of accounts with each session. While there are tools for this in ChatGPT’s browser version, it’s harder to manage within the app.

We’re sure we’ll learn more as we progress through the upcoming beta phase and official release.

By the way, if any of you want an early look, send email to service@answerstage.com. We’d love to get your feedback.

If you’re wondering — “what’s a Custom GPT?” This is from OpenAI:

Custom GPTs are a new way for anyone to create a tailored version of ChatGPT to be more helpful for their business, in their daily life, at specific tasks, or at home—and then share that creation with others.