Open AI recently released a [longgg prompt engineering guide](https://platform.openai.com/docs/guides/prompt-engineering) for getting the best possible outputs from their `gpt-x` (predominantly `gpt-4`) models. It definitely does feel like prompt engineering is fundamentally dependent on the ability of a user to provide as much detail as possible to the LLM and guide it to the right answer. Similar to how you would instruct a graduate with no clue about the job they are expected to do. And more importantly before you read ahead, recognise that an LLM does not rarely bullshit like people. Instead, it is a complete bullshitter that often gets things right - Refer [[Generative AI for Everyone - Course Notes]]. So, don't be surprised it you see it spewing absolute garbage at times! Below are the ChatGPT summarised and formatted notes about the prompt engineering guide for ChatGPT created by Open AI 😝 --- ## **Strategy: Write Clear Instructions** To get more relevant and accurate responses, it's crucial to provide detailed, specific queries. - **Tactic 1: Include Details in Your Query** Enhance the relevance of answers by incorporating important details and context. Example: Instead of asking "How do I add numbers in Excel?", ask "How do I add up a row of dollar amounts in Excel? I want this to happen automatically for every row, with the totals displayed in a column labeled 'Total'." - **Tactic 2: Ask the Model to Adopt a Persona** Use the system message to define a specific persona for the model, influencing how it replies. Example: "When I ask for help in writing, respond with a document that includes at least one joke or playful comment in every paragraph." - **Tactic 3: Use Delimiters to Clearly Indicate Distinct Parts of the Input** Employ delimiters like triple quotes or XML tags to separate different sections of text. Example: "To summarize a text, first enclose it in triple quotes, then write a haiku based on its content." - **Tactic 4: Specify the Steps Required to Complete a Task** Break tasks into a clear sequence of steps for easier comprehension and execution. Example: "Step 1: Summarize the provided text in triple quotes in one sentence. Step 2: Translate this summary into Spanish." - **Tactic 5: Provide Examples** Demonstrating with examples can guide the model, especially when a specific response style is needed. Example: "Teach me about patience. The river that carves the deepest valley flows from a modest spring..." - **Tactic 6: Specify the Desired Length of the Output** Direct the model to produce outputs of a specific target length, like word count or number of paragraphs. Example: "Summarize the provided text in about 50 words." ## **Strategy: Provide Reference Text** Leverage provided texts to compose more accurate and relevant answers. - **Tactic 1: Instruct the Model to Answer Using a Reference Text** Direct the model to use specific documents for answering questions, enhancing accuracy. Example: "Use the articles provided in triple quotes to answer questions. If the answer is not in the articles, state 'I could not find an answer.'" - **Tactic 2: Instruct the Model to Answer with Citations from a Reference Text** Require the model to cite passages from given documents when formulating answers. Example: "Answer the question using only the document provided and cite the relevant passages used for the answer." - **Tactic 3: Summarize Long Documents Piecewise and Construct a Full Summary Recursively** Tackle summarizing lengthy documents by breaking them down into sections and summarizing those. Example: "Summarize each section of a long document separately, then create an overall summary by combining these individual summaries." ## **Strategy: Split Complex Tasks into Simpler Subtasks** Decompose complex tasks into smaller, more manageable parts for effective problem-solving. - **Tactic 1: Use Intent Classification to Identify the Most Relevant Instructions** Categorize queries into specific areas to provide more targeted instructions. Example: "Classify customer service queries into categories like Billing or Technical Support, and provide specific instructions for each." - **Tactic 2: Summarize Long Documents Piecewise and Construct a Full Summary Recursively** Approach summarization of extensive documents by summarizing parts and then combining these. Example: "For a lengthy report, summarize each chapter individually, then create an overarching summary from these." - **Tactic 3: For Dialogue Applications That Require Very Long Conversations, Summarize or Filter Previous Dialogue** In extended dialogues, summarize or filter earlier parts of the conversation for clarity and brevity. Example: "In a lengthy chat session, periodically summarize the conversation so far to maintain clarity and focus." ## **Strategy: Give Models Time to 'Think'** Encourage in-depth analysis and reasoning before the model concludes. - **Tactic 1: Instruct the Model to Work Out Its Own Solution Before Rushing to a Conclusion** Prompt the model to independently solve problems before evaluating presented solutions. Example: "First, work out your solution to a math problem, then compare it to the student's solution to see if it's correct." - **Tactic 2: Use Inner Monologue or a Sequence of Queries to Hide the Model's Reasoning Process** Allow the model to internally process a problem, revealing only the final guidance or conclusion. Example: "When evaluating a student's answer, first reason through it internally, then provide just the final hint without revealing the answer." - **Tactic 3: Ask the Model if It Missed Anything on Previous Passes** Encourage the model to reassess its responses and verify if it overlooked any details. Example: "After listing excerpts from a source, prompt the model to check for any relevant excerpts it might have missed." ## **Strategy: Use External Tools** Utilize external sources and tools for more informed and up-to-date responses. - **Tactic 1: Use Embeddings-Based Search to Implement Efficient Knowledge Retrieval** Apply text embeddings for efficient retrieval of information relevant to the query. Example: "Use embeddings to find the most relevant sections of a text corpus for a given query." - **Tactic 2: Use Code Execution to Perform More Accurate Calculations or Call External APIs** Use code execution for precise calculations or to make use of external APIs. Example: "To find the roots of a polynomial, write and execute the necessary code." - **Tactic 3: Give the Model Access to Specific Functions** Enable models to generate arguments for external function calls based on provided schemas. Example: "Use the Chat Completions API to pass descriptions of functions for the model to execute." ## **Strategy: Test Changes Systematically** Systematically evaluate and optimize system designs through testing and analysis. - **Tactic 1: Evaluate Model Outputs with Reference to Gold-Standard Answers** Compare model answers to known correct answers to assess accuracy. Example: "Check if the model's response includes specific facts related to a question." - **Tactic 2: Use Model-Based Evals to Evaluate Outputs with a Range of Acceptable Answers** Employ model-based evaluations for questions where multiple outputs might be considered high-quality. Example: "Use a model to evaluate answers to questions with long or subjective answers." --- ## Further Reads 1. [[Generative AI for Everyone - Course Notes]] - I summarise Andrew Ng's course on the same 2. [How to write expert prompts for ChatGPT, GPT-4 and other language models](https://towardsdatascience.com/how-to-write-expert-prompts-for-chatgpt-gpt-4-and-other-language-models-23133dc85550#b564) - An in-depth guide published in Towards Data Science | Medium