A task vector specifies a direction in the weight space of a pre-trained model, such that movement in that direction improves performance on the task. Task vectors are built by subtracting the weights of a pre-trained model from the weights of the same model after fine-tuning on a task(see [[@ilharcoEditingModelsTask2023|paper]]). Task vectors can be modified and combined together through arithmetic operations such as negation and addition, and the behavior of the resulting model is steered accordingly. Negating a task vector decreases performance on the target task, with little change in model behavior on control tasks. Moreover, adding task vectors together can improve performance on multiple tasks at once. Finally, when tasks are linked by an analogy relationship of the form “A is to B as C is to D”, combining task vectors from three of the tasks can improve performance on the fourth, even when no data from the fourth task is used for training. Overall, experiments with several models, modalities and tasks show that task arithmetic is a simple, efficient and effective way of editing models. ## Answer from OpenAI's ChatGPT 3.5 In the literature of task arithmetic in AI/ML research, a task vector refers to a mathematical representation of a specific machine learning task. It is used to describe the input and output requirements of a given task, allowing for the mapping of different tasks onto a common space. Task vectors are often represented as high-dimensional vectors, where each dimension corresponds to a particular aspect or requirement of the task. These dimensions can include variables such as input data type, output data type, data size, complexity level, and other relevant attributes. By representing tasks as vectors, researchers can compare and analyze different tasks based on their similarities and differences. This approach enables the study of relationships between tasks and facilitates techniques like task interpolation or extrapolation. Task vectors are particularly useful in the context of meta-learning or few-shot learning frameworks. By embedding tasks into a shared vector space, algorithms can learn from multiple related tasks and generalize their knowledge to new, unseen tasks more effectively. Overall, task vectors provide a formal representation that allows for systematic exploration and analysis of different machine learning tasks. They enable researchers to quantify and compare various aspects of tasks, ultimately contributing to advancements in AI/ML research. # References [[@ilharcoEditingModelsTask2023|Editing Models with Task Arithmetic]]