--- publish: true --- [Elena Razlogova](http://elenarazlogova.org/) History Department, Concordia University, Montreal ![[zach drawing.png]] Notetaking is an essential aspect of research and writing. Historians in particular take notes on a grand scale: they process hundreds or thousands of primary sources when writing a book. You may start with index cards and a corkboard. But once you get into a major historical project you may be overwhelmed with sources and connections, and a corkboard will not be enough. Digital tools become crucial at that stage. I chose Zotero and Obsidian for digital notetaking because they are free at the entry level, and their paid features are not expensive. Both apps have large and active user communities that support their development so they are not likely to disappear. Finally, their data is portable: Zotero can export all its data as plain-text files, and Obsidian data is basically a folder (called a "vault" in the app) of text files written in [markdown](https://en.wikipedia.org/wiki/Markdown), a simple markup language that creates formatted text in a plain-text file. Obsidian features such the local Graph (a map of connections between notes) and Dataview (a plugin that structures the text in my notes so they can be queried like a database) help analyze the information I collect. I use Zotero to import source metadata and annotate PDFs, and Obsidian to work with notes. A popular way to do that among Obsidian users is to create one "literature note" per source with all annotations. But, as the above chart (based on a research process slide by historian [Zachary Schrag](https://zacharyschrag.com/)) shows, different annotations from the same source actually might belong in different places in your draft. In order to make this work, I split annotations into several research notes that will be later sorted into groups for writing. I put them in the top folder in my Obsidian vault, "01 notes" and mark them up for further analysis. My research notes look like this: ![[research note example.png]] This note includes a title that summarizes the note (it doubles as the file name for the note), source information, text from the source, tags, Dataview fields I set up with data added by me, including my comments, and links to other notes (clickable text in blue reflected in the graph on the top left). Links between notes are a basic feature of Obsidian. They are created by encasing the title of the note in double square brackets: `[[note title here]]`. The "live preview" feature in Obsidian hides this formatting code for easier reading. Below I explain what you need to do to get to this point. ### What You Need to Install [Zotero 6 and Zotero Browser Connector](https://www.zotero.org/download/) [Better Bibtex for Zotero](https://retorque.re/zotero-better-bibtex/) - Zotero plugin [Zotero-Obsidian Citations](https://github.com/daeh/zotero-obsidian-citations) - Zotero plugin [Obsidian](https://obsidian.md/) [Zotero Integration](https://github.com/mgmeyers/obsidian-zotero-integration) (formerly Zotero Desktop Connector) - Obsidian plugin. __If your [Obsidian installer version](https://github.com/mgmeyers/obsidian-zotero-integration/blob/main/README.md#help-the-plugin-doesnt-load) is before v0.13.24 you will need to reinstall from [obsidian.md website](https://obsidian.md/) (rather than updating from within the app).__ [Dataview](https://github.com/blacksmithgu/obsidian-dataview) - Obsidian plugin [Zotero's built-in PDF reader](https://www.zotero.org/support/pdf_reader), new in Zotero 6, adds annotations in a way that keeps them separate from the PDF. This setup requires you to keep your PDFs stored within the Zotero file system rather than as linked files residing elsewhere on your hard drive. I prefer this setup because it allows me to sync my files and annotations with the iOS Zotero app. I can share clean PDFs with my students, or I can export a PDF with annotations if desired. If you prefer to use linked PDF files, or to keep annotations embedded in PDFs and work in an external PDF editor you will need a different set of plugins--check out [this workflow](https://medium.com/@alexandraphelan/an-academic-workflow-zotero-obsidian-56bf918d51ab) by Alexandra Phelan. I am skipping the installation of Zotero and its plugins here, as well as the basics of importing sources into Zotero, because these instructions are available widely on the web (for example, see relevant sections [here](https://github.com/erazlogo/obsidian-history-notes) or [here](https://daily.jstor.org/how-to-use-zotero-and-scrivener-for-research-driven-writing/)) and many universities offer Zotero tutorials. I move on directly to the new (as of 2022) feature in Zotero 6: PDF annotations. ### Zotero Annotations I turn as many analog and electronic sources into OCRed PDFs as I can: articles and chapters in scholarly databases, scanned ILL materials, photographed archival sources, and converted EPUB files, DjVu files, and webpages. I read PDFs in Zotero’s built-in PDF reader, make annotations--highlights and comments. Finally, in the right column I create a note that includes all annotations: ![[add item note from annotations.png]] I added a custom export template for notes (see code below) in Config Editor in Preferences (in the File menu in Zotero). The template relies on highlighting in different colors. This step is not required, but it is useful for separating different kinds of information (important highlights from casual highlights, or quotes from references, for example). ![[zotero preferences.png]] ![[zotero config.png]] This template converts the highlight colors into directions for me, formatted as tasks (to create a task in Obsidian, you just type `- [ ] ` on a new line): '#ff6666' (red) = create research note '#5fb236' (green) = enter reference into Zotero '#2ea8e5' (blue) = ignore - already processed as a research note or reference 'else' (in my case, yellow) = a regular highlight ``` {{if color == '#ff6666'}} <p>- [ ] Create research note</p> <p>{{highlight}} {{citation}}</p> {{if comment}}<blockquote>{{comment}}</blockquote>{{endif}} <p>---</p> {{elseif color == '#5fb236'}} <p>- [ ] Enter reference</p> <p>{{highlight}}</p> {{if comment}}<blockquote>{{comment}}</blockquote>{{endif}} <p>---</p> {{elseif color == '#2ea8e5'}} {{else}} <p>{{highlight}} {{citation}}</p> {{if comment}}<blockquote>{{comment}}</blockquote>{{endif}} <p>---</p> {{endif}} ``` ### Zotero Integration Setup The basics of installing Obsidian are also available on the web; see, for example, [my guidelines](https://github.com/erazlogo/obsidian-history-notes). Here I am focusing on how I import annotations with Zotero Integration ([requires Obsidian installer version v0.13.24 or later](https://github.com/mgmeyers/obsidian-zotero-integration/blob/main/README.md#help-the-plugin-doesnt-load)). First, install Zotero Integration in the Community Plugins section. Turn "Safe mode" off. Then click on "Browse" and search for the plugin. ![[community plugins.png]] Set up Zotero Integration preferences: Install PDF utility Select database: Zotero Note Import location: The folder where you will save annotations Add Citation Format: This setting is optional at this stage but will be necessary for creating formatted footnote citations in Obsidian. Set up the Import template, saved as an .md (markdown) file in a separate folder in your vault. This file has to be specified in Zotero Integration preferences (mine is "meta/templates/research note.md"). Specify your research notes output path - mine is "01 notes" folder and Bibtex citekey (created by BetterBibtex in Zotero) as the file name. (The folder naming strategy of starting with "01" is recommended to set up for correct sorting in case you have more than nine folders down the road) ![[zotero integration settings.png]] My template for a research note separates authors by type so I can use Dataview to search by letter recipient and interviewer. It also imports Zotero tags for type of source and project name. And it includes a template for data entry, including fields for for page number, for my comments, and for the dates of an event described in the note. (If you cut and paste this template from Safari into an Obsidian note, make sure to delete the accents before and after "---"). ``` --- type: "{{itemType}}"{% for type, creators in creators | groupby("creatorType") -%}{% if loop.first %} {% endif %}{{type | replace("interviewee", "author") | replace("director", "author") | replace("presenter", "author") | replace("podcaster", "author") | replace("programmer", "author") | replace("cartographer", "author") | replace("inventor", "author") | replace("sponsor", "author") | replace("performer", "author") | replace("artist", "author")}}: "{%- for creator in creators -%}{%- if creator.name %}{{creator.name}}{%- else %}{{creator.lastName}}, {{creator.firstName}}{%- endif %}{% if not loop.last %}; {% endif %}{% endfor %}"{% if not loop.last %} {% endif %}{%- endfor %}{% if title %} title: "{{title}}"{% endif %}{% if publicationTitle %} publication: "{{publicationTitle}}"{% endif %}{% if date %} date: {{date | format("YYYY-MM-DD")}}{% endif %}{% if archive %} archive: "{{archive}}"{% endif %}{% if archiveLocation %} archive-location: "{{archiveLocation}}"{% endif %} citekey: {{citekey}} --- {{bibliography}} [online]({{uri}}) [local]({{desktopURI}}) {%- for attachment in attachments | filterby("path", "endswith", ".pdf") %} [pdf](file://{{attachment.path | replace(" ", "%20")}}){% if loop.last %} {% endif %}{%- endfor -%} {%- for tag in tags %}{% if tag.tag == "secondary" %} #source/secondary{% endif %}{% if tag.tag == "primary" %} #source/primary{% endif %}{% if "-project" in tag.tag %} #project/{{tag.tag | replace("-project", "")}}{% endif %}{%- endfor %} ### Index start-date:: end-date:: page-no:: ### Connections comment:: ### Note {% if markdownNotes %} {{markdownNotes}}{% endif %} ``` You can import all tags attached to the Zotero item. To do that, including the following code in the template (omit \`\`\`): ``` {% if tags.length > 0 -%} Tags: {% for t in tags -%}#{{t.tag | lower | replace(" ", "-")}}{% if not loop.last %}, {% endif %}{% endfor %} {%- endif %} ``` I only import two tags: type of source (primary or secondary), and project name, to separate notes relating to different research projects. Zotero allows for spaces in tags, but Obsidian does not, so any tags you want to add to Obsidian should have dashes or underscores instead of spaces. My template reformats Zotero tags into nested tags ( #source/secondary and #project/film-translation) to more easily manage them (Zotero does not allow for nested tags). In Zotero, I add "-project" in the end of any tag that denotes a research project, to find and transfer them to Obsidian (Zotero tag "film-translation-project" becomes Obsidian tag #project/film-translation). To remove my code for tags, delete the following portion of the template: ``` {%- for tag in tags %}{% if tag.tag == "secondary" %} #source/secondary{% endif %}{% if tag.tag == "primary" %} #source/primary{% endif %}{% if "-project" in tag.tag %} #project/{{tag.tag | replace("-project", "")}}{% endif %}{%- endfor %} ``` ### Importing Annotations Once you set up this system, import is easy. I set up a shortcut (Cmd-R) to call up the template, but you can also use the command palette. The command palette icon appears by default in the left-hand bar. To turn it off and on, go to Core Plugins in Settings. To call it up with a shortcut, type Cmd-P on Mac (Ctrl-P on PC). ![[open command palette.png]] Choose "Create research note." ![[select zotero integration from palette.png]] Then type keywords to find the source: ![[zotero selector.png]] And Zotero Integration will add the note that already has basic tags for type of source and project. You will need to create a descriptive title instead of the default title based on the BibTex citekey. ![[imported note.png]] Everything below the "Note" heading is your note made from annotations. If I did have any other notes attached to this source in Zotero they would import as well. To keep things organized, I don't keep notes of any kind in Zotero. After I import annotations, I delete "note from annotations." This way my notes are only in one place. I also change color for all annotations I added as research notes to blue (which means "skip on export"). This way, if I add any more annotations to the PDF I can repeat the process to export only my new annotations. ### Processing imported annotations The imported data I get falls into four categories: 1. Zotero Item metadata without annotations: If I am working with an analog source (an archival source I am not allowed to photograph, for example) and need to enter notes by hand, I just make sure there are no notes attached to the Zotero item. Then the import includes only source metadata and a template for entering the note. 2. Annotations relate to the same point or topic: If I have a short source that only requires one research note, I just highlight everything in yellow. Then after import I only have to rename the note in a way that describes its content. One note is all I need. 3. Annotations relate to several points or topics: If the annotations will have to be put into different parts of my draft (see Zachary Schrag's chart above), I will need several research notes. I highlight each annotation in red so I can track my own work as I process the data further. 4. Annotations are citations or bibliography entries: anything in footnotes or bibliography that belongs in my Zotero database. I highlight these in green. I need to process them as well: add them to Zotero. To make sure I don't forget any highlights slated for processing, I created a Dataview task list (first, you have to install the Dataview plugin via Community Plugins in Settings): ^bce1c4 <pre><code> ``` dataview TASK FROM "01 notes" WHERE text="Create research note" OR text="Enter reference" ``` </code></pre> Dataview can search for fields, links, and tasks. This task list is in a separate markdown file in my "dataview" folder. It displays all unprocessed notes and references from imported annotations. ^0895c8 ![[process research notes.png]] To get to the annotation, I just click on the icon next to it to get to the section of the note that contains the task. (You can't get to the exact task; if you have several tasks in one note you will see the last task in the section.) To create additional research notes, I create a new blank note. Then I drag-and-drop (or copy-paste) the top part of the existing note with the template included (everything up to the "Note" heading) into the new note. ![[drag metadata into new note.png]] My template puts source metadata on the top into the "YAML" space, denoted by three dashes on top and bottom. This data can go anywhere in a note but I like to hide it because I only need it to create Dataview searches and to connect from Zotero back to Obsidian. YAML allows it to be hidden, either with an arrow in the "editing" view, or completely in the "reading" view. I prefer to see the formatted bibliographic entry instead. ![[hiding yaml space.gif]] Then I drag and drop whatever annotations need to go into the separate note and delete them from the original file. ![[drag annotation into new note.png]] I add a descriptive title to the note and my own metadata: tags, page number, events, people, or other entities (films in my case), links to other related notes, my own comments, and, if necessary, the dates of the main event the note describes. The text ending with a double column `::` is a field name (in YAML you only need one `:`); everything to the right of `::` is field data. I will show later how to aggregate these fields into a table. The local graph displays the links I create: ![[adding links and graph.png]] ### Connecting Zotero Items to Research Notes Each imported annotation comes with the link back to the annotation in the PDF: ![[obsidian to zotero.gif]] Even if you entered your notes manually and don't have a pdf attached, you'll still be able to get to the Zotero item from the "local" link after your bibliographic reference: ![[from obsidian to zotero item.gif]] What's left to do now is to connect from Zotero back to Obsidian notes with the [Zotero-Obsidian Citations](https://github.com/daeh/zotero-obsidian-citations) plugin. This plugin requires BetterBibTex citekeys included in the YAML section of your notes. It also requires that all your notes linked to Zotero items reside in one folder ("01 notes" in my case). You may also want to keep only notes with a valid Zotero BibTex citekey in this folder, otherwise you'll get error notices from the plugin every time you sync. To configure the plugin, open settings: ![[open citations plugin settings.png]] You need to change the following: set the folder containing notes; choose the “Custom File Filter” radio button and change the custom file filter to `.+\.mdFrom Zotero Annotations to Obsidian Research Notes - Doing History with Zotero and Obsidian - Obsidian Publish; select "Match notes based on BetterBibTex citekey" and set yaml keyword to "citekey". ![[citation plugin first screen.png]] On the next screen, select the name of the tag that you will use to mark Zotero items that have related research notes. If you have several Obsidian vaults, you need to select the Obsidian vault name that will open the notes. ![[citation plugin second screen.png]] I assign a color (blue) to the tag used by the plugin to distinguish the items with reference notes in a list view. ![[assign tag color.gif]] I also create a "saved search" (a collection based on search criteria) by this tag that contains all my items that have research notes associated with them. Now you just need to run the plugin: ![[run citation plugin.png]] Now you can connect from Zotero back to the research note: ![[zotero to obsidian.gif]] ### Browsing Research Notes with Dataview Structuring my notes in this way allows me to keep track of my research. To do that, I created a separate note that contains two Dataview tables. The first table counts all research notes I have: <pre><code>```dataview TABLE WITHOUT ID length(rows) AS "Total Research Notes" FROM "01 notes" GROUP by true ```</code></pre> You can modify this count with a "WHERE" query (see below) to find out how many notes you have on a particular subject or by a particular author. The second table displays all basic information about my notes. This example query filters for all notes that include my comments and limits search results to 100 notes. <pre><code>```dataview TABLE without ID file.link as "Note", comment as "Comments", start-date AS "Event Start Date", end-date AS "Event End Date", author as "Author", recipient as "Recipient", title as "Title", publication as "Publication", date as "Date", page-no as "Pages", archive as "Archive", archive-location as "Loc. in Archive" FROM "01 notes" WHERE comment SORT file.mtime DESC LIMIT 100 ```</code></pre> The results look like this: ![[browse research notes.png]] Another example: this query contains all letters from Alex Viany to Jay Leyda sorted by date of letter, descending: <pre><code>```dataview TABLE without ID file.link as "Note", comment as "Comments", start-date AS "Event Start Date", end-date AS "Event End Date", author as "Author", recipient as "Recipient", title as "Title", publication as "Publication", date as "Date", page-no as "Pages", archive as "Archive", archive-location as "Loc. in Archive" FROM #project WHERE contains(author,"Viany") and contains(recipient,"Leyda") SORT date DESC ```</code></pre> The result looks like this (note that date of letter is different from event dates in one case): ![[dataview quiery example letters.png]] ### From Source to Research Note at a Glance Here is my research process in one chart: ![[zotero-obsidian workflow.png]] Enjoy!