If you are having trouble with the sign in process, please <ContactLink>contact the SSH Open Marketplace team</ContactLink>.

If you are having trouble with the sign up process, please <ContactLink>contact the SSH Open Marketplace team</ContactLink>.

Subject

Social Sciences & Humanities Open Marketplace

Successfully submitted!

Before starting, it's important to collect key administrative and descriptive data about your collection. Some numbers will not only be very helpful for estimating the workload, but also to shape the workflow in good balance to your capabilities and resources. It is good advice to be as precise as possible.

Apart from numbers, it’s really important to get a deep understanding of what the data set is all about and to gain a deeper understanding of the material. It may be worthwhile spending some days reading and exploring the collection as a whole, before taking long-term decisions. 

Whether or not you’re a field expert yourself, discussions with staff members and even colleagues from other disciplines may make sense in order to avoid biased decisions and “blindness”. And if you’re solo sailing, this will be even more important...

So try answering the following questions as precise as possible:
- Apart from developing the scheme itself: What will help to make the overall workload as transparent as possible?
- How many documents (or better: individual pages, or countable entities) have to be text-tagged?
- What are collection-wide traits of the material?
- How homo- or heterogeneous is my collection? On what grounds?
- Does it make sense to develop one tagging scheme for the entire collection?
- Are there any similar content element types across the entire collection, such as personal names, locations, time and date entries? What are these?

The development and deployment of any text tagging (or annotation / content analysis / data mining scheme) has many dimensions. So, depending on the nature of the material in the text corpus, it bears risks of overdoing, over-interpreting and traps of lengthy discussions about peripheral aspects and detail, that may distract from a focused and analytical approach.

Since time available from you and your staff is the one core limitation in most projects - especially when including manual data entry - it is absolutely crucial to bind pre-defined resources to a transparent and goal-oriented process. When aligned with the desired or externally imposed time frame for the deployment of the scheme (or parts of it as intermediary steps) and reasonable workload-calculations however, the path to the desired outcome becomes a lot more manageable and the scale of the workflow can be adjusted accordingly.

Try answering the following questions as precise as possible:
- How many net working hours (per staff, per week) are effectively available to edit documents and work on tagging on average? Count in: initial learning curves, times on sick leave, vacation time, public holidays, other obligations at work, etc.
- What is the first desired point in time, when the tagging work, or a specific part of it should be finished / is there a deadline to be respected?
- Can tagging work be split up between staff members? How?

How to do this?
- Ask for a reliable self-declaration from your staff about a mid-term possible commitment (weekly hours) or - if you are solo or responsible for the distribution of work - calculate the total available hours yourself.
- Collect feedback on obstacles

Separate from the actual text tagging work, the documentation of progress is crucial.

In order to make the documentation helpful and an acceptable effort for everyone involved, it is important to calibrate its granularity: It must not take up too much of the actual working time but it must still be granular enough to bear any reasonable additional value. Whether you set up every entity as a github issue and track it down or you develop a table with a line for every document is entirely up to you, but some sort of progress monitoring must be done. Stick to one functioning system that can display meaningful numbers of the progress at any given point in time.
Ideally, progress is measured in terms of comparable entities with clearly defined  and meaningful statuses, such as "to do", "in progress" "done" or counting hours of work still to be done. Any Project Management tool will contain a basic function like that. If no separate tool is applicable, a Google Docs Spreadsheet will do the job, and if the documentation is rigorously done by all members, it may suit your needs perfectly.

If you implement some sort of review process, with the idea of re-assessing work previously done, you may want to include a mechanism, where staff members change their role in a rotational system.
In order to make this feasible, it is crucial that all members of staff are working on a shared understanding of the tagging scheme, otherwise, different attitudes and interpretations may end up working "against" each other, annihilating previous investments of time and decisions.

At least, review steps must be reflected in the documentation process, i.e. with additional steps "review", or "tagging complete". 

How to do this?
Walk through the entire process at least once and describe every status indicator of the pipeline as precisely as possible. Use only such status indicators, that inform about a relevant difference. 

1. "to do" document is 
- fully transcribed (and checked)
- no tags added yet
2. "in progress" (and “checked out” by a staff member)
- tagging according to tagging scheme is ongoing
- special cases are documented
3. "tagging complete"
- the entire tagging scheme has been applied to the whole document
- open questions are documented
4. "reviewed"
- the tagging choices have been double checked
- open questions ideally solved

You may also want to break down the workload into smaller chunks that can be assigned to different staff members.


The tagging scheme addresses questions that regard the collection as a whole, so it works across the boundaries of individual documents and makes them comparable by clustering information and counting individual occurences. It is therefore important to choose tagging categories that make sense for more or less the entire material of the collection. The paradox here is that you need to define tagging categories that are generic but not too broad, general, yet specific to your material and research needs.

Secondly, text tagging is “positive” in the sense that it is bound to actual written text in the document and it adds a second (and only one) layer of information to a specific string in the text.

Now, re-assess Step No. 2 and think about the collection as a whole:
What clusters of information across the collection are of interest?

In order to generate categories for text tagging, there could be two major initial categories, namely:
1. Place Names / Person Names / Time Information, that is "normalizable" categories, that are 
a) relatively independent from the domain and 
b) have potential references in a controlled vocabulary Wikidata / GND / GEONAMES / AAT / etc.)  

2. "All the rest.", say: More domain-related categories that are a) context-sensitive and b) shaped towards specific research questions

I suggest you define “low-hanging fruits” first: Information from category 1. and then add from category 2. Remember, that every new category adds significantly to the workload. 

Helpful questions may be:
- What does the material itself generically include?
- What are core elements that should be analyzable / countable / comparable later?
- What are domain-specific questions that may become a part within a tagging scheme?

For every tagging category, write a concise definition, and include typical positive and negative examples from your data.

Examples:

Tag: “Natural Disaster” 
Definition: A catastrophic event based on natural forces
Examples: “when the thunderstorm hit the neighbourhood, a flood developed”

Double check, whether the tagging scheme is not too large, since every new tag category multiplies the workload. Double check, whether or not there are clearly defined boundaries between individual tags.


Once the decision is made for the scheme elements, transfer them to your text tagging environment.
Within Transkribus, this is easily done in the "tags" section of the interface.

It may be useful to deploy the scheme in reasonable parts and start with those categories that are most promising, tied closely to the most important goals.
An iterative approach also lets you estimate more clearly how much of the workload in relation to your resources and given time frame will be possible and it may make much more sense to finish the work in one tagging category throughout the entire collection before starting the next.

One clear suggestion is to start with issues, that are easy to identify.

Johannes Alisch

The workflow describes the strategic development and deployment of a text tagging scheme for transcribed historical prints and / or handwritten resources. It aims primarily at larger collections that consist of relatively homogenous resource types. That means they are similar compared to each other regarding their historic context, structure, extent, language, etc.

If you’ve done transcriptions using Transkribus, it may be the ideal platform to implement your tagging scheme, but the approach is transferable.

Key objectives of the workflow are:
- manage quality and consistency for basic text analysis in the light of limited resources of academic research projects
- make large collections more manageable
- iteratively build up a comprehensible and consistent dataset
- adapt to the specific and generic characteristics of the source material on one side and 
- adapt early to key research questions and analytical dimensions on the other side
- provide the foundation for further data enrichment and higher levels of analysis such as normalized collection dictionaries, NER, Geo-Referencing, etc.
- create a model tagging scheme adapted to your needs for further scalability
- create ground truth data set for domain-specific AI text tagging

Please mind:
1. The workflow was developed in the context of a specific research project at the University of Basel, please visit ["VOCAL POWER"](https://dg.philhist.unibas.ch/de/bereiche/fruehe-neuzeit/snf-professur-macht-der-stimme/) for more information.
2. For the Open Marketplace it was shaped to address challenges that will very likely arise in similar projects. However: We strongly suggest thoroughly discussing all the steps with key stakeholders and project staff in order to adapt the parameters according to your own needs on the ground.
3. Work on data projects is prone to get out of hand easily, so plan your steps carefully, test and iterate small portions early on and finish one step at a time.


Transkribus Text Tagging

Media

Related items(2)

How to use Github Issues in the Digital Humanities project communication between humanists and developers?

Transkribus Resource Center

Workflow steps(5)

1 Know Your Data

2 Know Your Resources

3 Know Your Progress

4 Develop Tagging Scheme

5 Deploy Tagging Scheme

Media

Related items(2)

WorkflowHow to use Github Issues in the Digital Humanities project communication between humanists and developers?

Training materialTranskribus Resource Center

Workflow steps(5)

How to use Github Issues in the Digital Humanities project communication between humanists and developers?

Transkribus Resource Center