If you are having trouble with the sign in process, please <ContactLink>contact the SSH Open Marketplace team</ContactLink>.

If you are having trouble with the sign up process, please <ContactLink>contact the SSH Open Marketplace team</ContactLink>.

Subject

Social Sciences & Humanities Open Marketplace

Successfully submitted!

#### Why use ATR?
* To **speed up text acquisition** &rarr; Identify the tools and ATR models available for a **ready-to-use** solution to apply to your corpus directly.
* **Impossibility of transcribing the corpus by hand** &rarr; Either opt for a **ready-to-use** solution or **fine-tune** an ATR model by collecting or creating training data to form your own template.  

_Please note that building an ATR model from scratch is **expensive** (costs of data creation and gathering), **time-consuming**, and **energy-intensive**. In addition, processing complex layouts remains a major challenge._ 


#### Your objective(s)

|                           | Precision | Accuracy | Model               |
| :------------------------ | :-------: | :------: | :------------------ |
| **Quantitative Analysis** | Low       | ~85%     | Generic             |
| **Text Repository**       | Medium    | ~90%     | Specific or Generic |
| **Scientific Editing**    | High      | ~95%     | Specific            |

If the quality of ATR predictions is not sufficient to reach your research objectives, the quality of the generated text can be improved by introducing a **human proofreading** phase, creating **more training data**, or implementing **post-treatment checks**.

_The larger the project, the easier to find resources and expertise._  

##### Individual vs. Collective
* My project is an **individual** project &rarr; Keep it simple and try to look for as many **ready-to-use solutions** as possible.
* My project is a **collective** project &rarr; Use the **expertise and resources** (infrastructures, models, data) **of your partners** to build your project on solid ground.

##### Not Funded vs. Funded
* My project is **not funded** &rarr; Keep it simple and try to look for as many **ready-to-use solutions** as possible.
* My project is **funded** &rarr; The project can grow depending on your budget and project objectives. You can **train your own team or call experts in**, **hire engineer(s) to create or optimize your pipeline**, **finance the creation of training data**, or even **buy/pay for access to compute servers** in order to train new generic models. 

##### Team Training
* My team **cannot be trained** in ATR techniques &rarr; Keep it simple and try to look for as many **ready-to-use solutions** as possible.
* My tean **can be trained** in ATR techniques &rarr; You can **expand** your use of ATR, **fine-tune** or **create your own** models, in order to adapt the ATR techniques as much as possible to your research objectives.

As ATR facilitates the acquisition of texts, it should be the **first step** in your project. It is preferable to **anticipate** having to dedicate a considerable amount of time for this part of the project workflow since it **determines the quality of the textual data**. Set aside **at least 6 months** in order to **prepare the training data**, **correct the transcriptions**, and anticipate segmentation problems. Do not forget to **include checkpoints** to verify data quality at **each step of the workflow**. This is likely to impact the text output in your corpus.

#### Choosing your tool
* **[eScriptorium](https://escriptorium.inria.fr/)** ([contact](https://cremmacall.sciencescall.org)) is an ATR web application which allows you to annotate and transcribe your documents. It is not a totally ready-to-use solution since you need to **either train or download a model**. However, your are completely free to train your model through your chosen parameters with _Kraken_.
* **[OCR-D](https://ocr-d.de/en)** is an open source software that covers a full OCR workflow as long as you provide digitized images. It is optimized for old prints (16th-18th centuries).
* **[Transkribus](https://www.transkribus.org/)** is a ready-to-use solution, even though it is not totally open. It requires that you stay in the _Transkribus_ environment, which means that you cannot run your training on the command line and choose your training parameters.
* Other: Please note that the less common your solution is, the fewer ready-to-use resources will be available to you.  

_Once you have chosen a tool, you should stay in the same environment for compatibility reasons. Data and models are not totally compatible between different tools._  

#### Training data
You can find available data and models in the **[HTR-United](https://htr-united.github.io/)** catalog, on **GitHub** or **Zenodo**. With _Transkribus_, models are accessible via the interface. When you collect data or reuse a model, make sure that the transcription guidelines used are appropriate for your project.  
* If your transcriptions are to be aligned, you will need to spend time creating data for segmentation and text alignment, for example.
* Training data in an XML format (**ALTO** or **PAGE**) are ready to use. You can gather data from different projects, but be careful about compatibility.  

#### Is there an appropriate generic model?
* YES &rarr; You can use the model directly or fine-tune it on your corpus. Most of the time, 5 images with aligned text are enough to achieve results of around 95% accuracy if the generic model is appropriate. 
* NO &rarr; You have to create a model from scratch by reusing and/or create new data. For a simple document, 20 images with aligned text may be enough.  

#### Transcription rules and outputs
* Your pipeline should include the **type of text output** you want, which will **determine your transcription rules**. These rules are extremely important to ensure the homogeneity of your data. See for example the [CATMuS](https://catmus-guidelines.github.io/) transcription guidelines.
* Your research objectives are key in designing your pipeline and making sure that you have integrated all the necessary textual features.
* **Text post-processing protocols** (_word segmentation, lemmatisation, named entity recognition, etc._) are determined by the type of text output you want, either raw (for quantitative analysis) or pre-edited text.
* It is crucial to decide what you can share and where you will do so in order to design your pipeline and data management plan.  
_You do not have to keep everything_.

Images of text sources are the basis of many Humanities scholars' academic work. Finding, creating and/or collecting images of textual material is often the **first step of the research process**. Users may digitize archival materials themselves, but digitization is **often carried out by the archival administration or an external provider**.

The **pre-processing** step is **optional**, and sometimes not necessary. Image optimization includes processes like **adjusting** brightness and contrast, cropping, splitting, rotating, and dewarping.

Once the images are ready and uploaded onto a transcription platform (e.g. eScriptorium), the **software recognizes the different zones on the page** like text, non-textual content, or no content at all. This process is called **segmentation**. It can itself be broken down into two different steps:
1. Recognition of text zones;
2. Recognition of lines within the text zones.  

The layout analysis step is essential for defining the layout of the text and **locating the titles, subtitles, margins, or annotations** that accompany the main body of text.

Once the zones have been segmented, the model recognizes the **text within the zones**. The machine does not actually read the text, but rather **predicts the probability of a combination of letters or for a specific word to appear**. A model is trained and improved based on **manually curated** pre-existing training data, also known as **ground truth** data.  

The model can either be created from scratch, or be a fine-tuning of an existing model:
* Building a model from scratch demands a substantial volume of training data and involves significant investment in terms of time, computing resources, and technical expertise.
* Fine-tuning a model involves incorporating your own data into a generic ATR model before customizing it to suit the specific corpus.

Once the model has produced the transcription, you need to **check its performance**. Several types of errors can occur in ATR transcriptions: addition, deletion, or replacement of characters, for example.  
Scores to evaluate the performance of a model include:
* **Word Error Rate** (WER) is the sum of the three types of transcription errors divided by the total number of words in the ground truth.
* **Character Error Rate** (CER) compares the total number of characters in the test set (including spaces) with the total number of additions, deletions, and substitutions of characters required to obtain the ground truth results.  

These scores do not have the same significance depending on the diversity of the training corpus. The greater the diversity in the model's training corpus, the more data it requires to achieve higher accuracy. However, with a diverse training corpus, the model is more likely to successfully adapt to an unknown document.

Once the transcription has been predicted by the machine and corrected by a human, the final stage is to **export the results** from the transcription platform. There are several possible formats for this, such as raw text or XML formats, depending on what the transcription is needed for.  

**Sharing your data will help the community to grow and build more generic models while attracting greater visibility for your project**. You may need to share images in order to share training data, so make sure that you have the rights to do so. You can link your repositories to [HTR-United](https://htr-united.github.io/) for greater visibility.

Ariane Pinche

Pauline Spychala

*Work-in-progress: this workflow is not finalised yet.*


**Automatic Text Recognition** (ATR) uses Artificial Intelligence (AI), in particular **machine learning** (ML), to **extract text from a scanned image**. It encompasses two main techniques: **Optical Character Recognition** (OCR), extracting text from printed documents, and **Handwritten Text Recognition** (HTR), exracting text from manusripts.  
This workflow presents the main steps of an ATR workflow and how to integrate it in your research project.  

Automatic Text Recognition Roadmap

Media

Workflow steps(9)

1 Needs and Objective(s)

2 Resources

3 Integrating ATR in your workflow

4 Image Acquisition

5 Image Pre-Processing

6 Layout Analysis

7 Text Recognition and Model Training

8 Quality Assurance and Metrics

9 Endformat and Reusability