Prompt Versioning

Prompt versioning is a crucial feature for managing the evolution of your prompts and iterating on your agents. It allows you to track changes, revert to previous states, and deploy specific versions with confidence. This document outlines how to use prompt versioning within the platform website UI and integrate it with the SDK.

Overview of Prompt Versioning

The platform provides a robust system to manage different iterations of your prompts, similar to how code version control systems like Git manage code. Each saved change to a prompt creates a new "commit," which is a snapshot of the prompt at a specific point in time. These commits can be tagged for easy identification and deployment.

The versioning system allows you to:

• Track Changes: Every modification to a prompt is recorded as a commit with a unique ID
• Tag Versions: Attach meaningful labels like "production", "staging", or "experiment-A" to specific commits
• Revert Changes: Easily switch back to any previous version of your prompt
• Deploy Dynamically: Retrieve specific versions programmatically in your applications

Creating and Managing Prompt Versions

You can create and manage prompts through the platform website UI or the SDK. This section will outline the steps to take when using the UI. For details on prompt management via the SDK, see the Prompt SDK Reference.

Creating a New Prompt

To create a new prompt, navigate to the "Prompts" page under your project and click the "New Prompt" button. Make your desired changes in the prompt editor, add any desired tags, and then click the "Commit" button on the bottom right of the page to create your first commit for the prompt.

Creating a New Version

To create a new prompt version:

1. On the left, select the commit whose content you would like to base your new commit on.
2. Click the "New Commit" button on the left to initiate a new commit.
3. Make your changes in the prompt editor, add any desired tags, and then click the "Commit" button on the bottom right of the page to create your new commit.

Managing Past Commits

You can view past commits for a prompt by clicking on the cards on the left side of the page. You'll notice that past commits are read only. To modify them, create a new commit as described in the previous section. However, you can still add and remove tags from past commits.

As mentioned at the top of this page, please keep in mind that if you try to add a tag to a commit when that tag is already attached to a different commit for the same prompt, the tag will be moved from the original commit to your new commit.

Using Your Prompts in Code

The SDK allows you to programmatically retrieve your prompts and their versions. See the Prompt SDK Reference for detailed API documentation.

Basic Usage

To retrieve a prompt and its versions, you can use the Prompt.get method. There are three ways to use this method:

1. Retrieve the latest version of a prompt

from judgeval.prompt import Prompt

prompt = Prompt.get(
  project_name="default_project",
  name="Movie Recommendation Prompt"
)

2. Retrieve a specific version of a prompt by commit ID

from judgeval.prompt import Prompt

prompt = Prompt.get(
  project_name="default_project",
  name="Movie Recommendation Prompt",
  commit_id="aee4ae6bfa320b87890120ce3"
)

3. Retrieve a specific version of a prompt by tag

from judgeval.prompt import Prompt

prompt = Prompt.get(
  project_name="default_project",
  name="Movie Recommendation Prompt",
  tag="production"
)

After retrieving a prompt, you can use the compile method to compile the prompt with the given variables.

from judgeval.prompt import Prompt

# Assume "Movie Recommendation Prompt" is a prompt whose latest commit has the following content:
# "Given this genre: {{genre}} and this decade: {{decade}}, can you recommend a movie of that genre that was released in that decade?"

prompt = Prompt.get(
  project_name="default_project",
  name="Movie Recommendation Prompt",
  tag="production"
)

response = prompt.compile(genre="horror", decade="90s")
print(response) # "Given this genre: horror and this decade: 90s, can you recommend a movie of that genre that was released in that decade?"

Example Workflow

from openai import OpenAI
from judgeval.prompt import Prompt

# Retrieve and use a tagged version
class MovieRecommendationAgent:
    def __init__(self):
        self.client = OpenAI()

    def get_recommendation(self, user_genre: str, user_decade: str):
        prompt = Prompt.get(
          project_name="default_project",
          name="Movie Recommendation Prompt",
          tag="production"
        )
        response = self.client.chat.completions.create(
            model="gpt-4", 
            messages=[
              {
                "role": "user",
                "content": prompt.compile(genre=user_genre, decade=user_decade)
              }
            ]
        )
        return f"Response: {response.choices[0].message.content}"

if __name__ == "__main__":
    agent = MovieRecommendationAgent()
    agent.get_recommendation("horror", "90s")