Jeremiah Marble
1y ago
AI Product Leader 🦊 | Author http://Model47.com | Co-Founder #PrimaDonaStudios & #TowardsArcadia | Speaker | Writer🚢
I made a promise to myself that I'd try to publish every single day for at least 100 days, at least one thing about #RAI. Today I've been heads down on doing actual work, but keeping my promise again today: here is #RAI 14/100.
As you know, I'm using OpenAI's Prompt Playground for my experiments. This gives me additional flexibility beyond "just" the ChatGPT interface, without making me spend time (yet) on standing up my own chat interface UX, for example through writing a web app and hosting it on Azure.
Eventually, that's exactly what I'll do - but there is a lot of good learning I can do before I take on the additional overhead of dealing with Azure quotas and instances and, you know, writing the code. That is a task for later.
But I'm making progress today. In this post, let me go a bit deeper into what I'm doing.
With the Playground, first I select the underlying LLM, which is going to give me a response to my "user" prompts. I'm currently using GPT-4-mini for cost savings, but of course for prod I'd evaluate several models to see what was best for what I was trying to do.
But - remember - I want to test MY "Boxes & Foxes" chatbot, which aims to help non-traditional founders start successful businesses. Just returning GPT-4-mini output wouldn't teach me a bunch I couldn't get from ChatGPT.
One level beyond which model I select are the modifications I make to the "plain vanilla" its responses. These are things like temperature (which I started at 0.5), top-p sampling (which I started at 0.9), and max tokens (which I started at 1000-1500).
I'll want to experiment also with these settings, to make sure that I'm optimizing my chatbot for my own specific purposes.
The next level I'm testing is a high-level Retrieval-Augmented Generation, or RAG. To do this, I enabled Code Interpreter, and provided it with a text file of "Model 47" as my grounding document.
I have LOTS of learnings & thoughts, but TBH I'm heads down in actually doing this stuff today, so I'll seek to tease them out later.
Onwards, friends! Hope that you're having a great Wednesday.