

In short, one can provide a document image (and possibly additional data) to [LayoutLMv2Processor],
and it will create the inputs expected by the model. Internally, the processor first uses
[LayoutLMv2ImageProcessor] to apply OCR on the image to get a list of words and normalized
bounding boxes, as well to resize the image to a given size in order to get the image input. The words and
normalized bounding boxes are then provided to [LayoutLMv2Tokenizer] or
[LayoutLMv2TokenizerFast], which converts them to token-level input_ids,
attention_mask, token_type_ids, bbox. Optionally, one can provide word labels to the processor,
which are turned into token-level labels.
[LayoutLMv2Processor] uses PyTesseract, a Python
wrapper around Google's Tesseract OCR engine, under the hood. Note that you can still use your own OCR engine of
choice, and provide the words and normalized boxes yourself. This requires initializing
[LayoutLMv2ImageProcessor] with apply_ocr set to False.
In total, there are 5 use cases that are supported by the processor. Below, we list them all. Note that each of these
use cases work for both batched and non-batched inputs (we illustrate them for non-batched inputs).
Use case 1: document image classification (training, inference) + token classification (inference), apply_ocr =
True
This is the simplest case, in which the processor (actually the image processor) will perform OCR on the image to get
the words and normalized bounding boxes.
thon
from transformers import LayoutLMv2Processor
from PIL import Image
processor = LayoutLMv2Processor.from_pretrained("microsoft/layoutlmv2-base-uncased")
image = Image.open(
    "name_of_your_document - can be a png, jpg, etc. of your documents (PDFs must be converted to images)."
).convert("RGB")
encoding = processor(
    image, return_tensors="pt"
)  # you can also add all tokenizer parameters here such as padding, truncation
print(encoding.keys())
dict_keys(['input_ids', 'token_type_ids', 'attention_mask', 'bbox', 'image'])