I don't do a lot of #gen-ai in my day-2-day even though I work as a data consultant. But, I happen to work with a lot of super smart colleagues who have been doing #gen-ai day in and day out for the last 6 months or so. My brush with this discipline has just been interacting with Chat GPT and attending a hackathon where I spent 24 hours trying to copy-paste code and learn on the fly[^1] Of late, I have noticed it has gotten harder for me to have conversations with them over drinks 🍺 . The main reason of course, is my lack of #gen-ai knowledge 🥲 So, last weekend I completed [Andrew Ng's Generative AI for Everyone course](https://www.deeplearning.ai/courses/generative-ai-for-everyone/) on [deeplearning.ai](https://www.deeplearning.ai/). Highly recommend this to anyone looking to get a bird's eye view of the subject. I went cheeky and passed on my rough, running notes from the course to Chat GPT and asked for it to be summarised as a formalised structure. Here is what I got! ## Prompt This was the initial prompt[^2] I provided. I had to give subtle instructions later on during the iteration for making final copy. ``` Heya, I am currently listening to an online course about Generative AI. I have the habit of taking running notes while listening to lectures. However, I often find it hard to structure these notes at a later stage. Are you able to help take my notes and convert them into well structured notes which I can use at a later stage? Feel free to add in your own thoughts into my thoughts to augment and enhance my notes. Also, if my notes are inaccurate in areas, do correct it. When doing so, please let me know about it. Make sure that you are not adding too many extra bits in terms of re-organisation. the structure i write my notes in is quite linear and should be maintained ``` ## Result If you are pressed for time, hopefully these notes help you gain an expedited understanding of #gen-ai . ### Introduction to Large Language Models (LLMs) 1. **Fundamentals of LLMs**: - Built using supervised learning to predict the next word, a central aspect of LLM technology. - Developed by training AI on large datasets. 2. **Role as a Thought Partner**: - LLMs can engage in effective back-and-forth discussions, acting as interactive thought partners. ### LLMs in Comparison to Web Search - **Web Search**: - Offers higher accuracy but may lead to information overload. - **LLMs**: - Provides more focused information but can be less accurate. - The choice depends on the balance between the need for accuracy and information breadth. ### Applications of LLMs 1. **Web-Based Applications**: - Examples include conversational AI like ChatGPT. 2. **Software Integration**: - Uses include functionalities like email routing. ### Generative AI Applications 1. **Writing**: - Generating content based on prompts, aiding in brainstorming and creating generic drafts. 2. **Reading**: - Proofreading, critically analyzing text, summarizing articles, and other textual analyses. 3. **Chatting**: - Functions like ChatGPT, maintaining conversation history, and providing customized responses. ### Limitations of LLMs - **Skill Comparison**: - Unable to perform tasks beyond the capability of a new college graduate with detailed instructions. - **Knowledge Cutoff**: - Cannot process information not included in their training data. - **Context Limitation**: - Effectiveness is limited by the combined length of input and output (context length). - **Structured Data**: - Less effective with structured or tabular data. - **Potential Issues**: - May produce hallucinations, biases, or toxic speech. ### LLM Prompting Tips 1. **Specificity in Prompts**: - Ensure detailed and specific prompts for adequate context. 2. **Guiding the Model**: - Direct the model to process and think through responses. 3. **Experimentation and Iteration**: - Employ an agile approach in prompt engineering. ### Using Generative AI in Software Applications - **Simplifying Software Development**: - Lowers barriers to entry, aligning with the "low code" movement. - **Assisting in Code Generation**: - Facilitates creation of real code applications without extensive coding knowledge. ### Lifecycle of a Generative AI Project 1. **Scope Project**: Define objectives and boundaries. 2. **Build/Improve System**: Develop and refine the AI model. 3. **Internal Evaluation**: Assess the system's performance. 4. **Deploy & Monitor**: Implement the system and track its effectiveness. ### Costing an LLM - **Pricing Considerations**: - Check pricing per token and estimate the number of tokens needed. - **Cost vs. Revenue**: - Utilizing an LLM can be cost-effective, but like any product, it might be expensive if not generating revenue. ### Improving Performance of an LLM 1. **Prompting**: Enhance how prompts are structured. 2. **RAG (Retrieval Augmented Generation)**: - Incorporate external knowledge for better responses. - Steps include searching relevant documents, updating prompts with this information, and generating answers from these enriched prompts. 3. **Fine-Tuning**: - Aimed at creating smaller, more efficient models suitable for low-cost, low-latency applications, including mobile and laptop use. 4. **Pre-Training**: - Undertaken for highly specialized, larger models, considered as a last resort. ### Automation vs Augmentation - **Task Automation vs. Job Automation**: - Generative AI automates tasks, not necessarily entire jobs. - **Job Task Analysis**: - Identifying automatable tasks within a job to enhance efficiency. - **Augmenting Professional Capabilities**: - Knowledge workers stand to benefit significantly from Generative AI. #### Footnotes [^1]: Once again, has become clear that this is not an approach that works for me ;) [^2]: Prompt engineering seems like a discipline on it own. For my daily interactions with Chat GPT, providing elaborate explanations has seemed to work well.