Dolma is a research project developed by the [[Allen Institute for AI]] (AI2). It aims to build an advanced conversational AI system that can engage in open-ended, multi-turn conversations with users. Dolma leverages state-of-the-art deep learning techniques and large-scale pretraining to create a language model capable of generating coherent and contextually relevant responses. Also see [[whisper]] by [[OpenAI]]. The primary focus of Dolma is to improve the conversational abilities of AI systems, including aspects like coherence, contextual understanding, and response generation. The researchers at AI2 train Dolma using reinforcement learning from human feedback (RLHF), which involves collecting data where human AI trainers have conversations with each other while being periodically ranked on the quality of their responses. The training process for Dolma involves two main steps: pretraining and fine-tuning. During pretraining, Dolma learns from a large corpus of publicly available text data to acquire a broad understanding of language. Fine-tuning follows pretraining and involves training Dolma on dialogue data generated by human trainers. The AI2 team has also introduced safety mitigations in Dolma to ensure responsible use. This includes suppressing outputs that may be harmful or inappropriate. Additionally, they are actively working on addressing biases and reducing offensive or misleading outputs. Dolma is part of the larger suite of projects at AI2 aimed at advancing artificial intelligence research while emphasizing ethical considerations. By pushing the boundaries of conversational AI capabilities, projects like Dolma contribute to improving natural language understanding and generation systems for various applications such as customer service chatbots, virtual assistants, and more.