PDF document DOM

[igalM]

I've been getting some weird results.
Some documents are being viewed as a DOM tree of spans, where each span is a whole line in the PDF,
but some documents are being viewed as a span for each letter or two.
Why is this happening? And is there a way to distinguish when will it happen?

[stephanrauh]

What you've seen is the text layer above the PDF document. The document itself is rendered as a canvas per page below the text layer.

The text layer, in turn, is invisible. It's only there to allow users to select text and to highlight the find results. When you make it visible - as this demo does - you'll see that the text layer is slightly off. It's not an exact representation of the PDF file. Instead, it's a best-effort approximation based on a lot of heuristics.

In earlier versions, almost every word was an individual <span>, and frequently words even required two <span> tags. This way, nobody noticed that the text was a bit too small or too large, because each new <span> comes with a new position. Some time ago, the Mozilla team managed to improve the algorithm considerable. But still, it's not perfect, and it doesn't match every document. There are still documents requiring a large number of <span> tags, especially if they're using an exotic font.

[igalM]

I see, thank you for taking the time to answer me.
Do you have an idea if we are able to diffrentiate between kinds of documents? Those who would appear with spans for every word, and those who are rendered much better?

[stephanrauh]

@calixteman You've contributed quite a few commits to the text layer of pdf.js. Maybe you can answer the question? Is there a way to predict whether there's one <span> per line or many <span> tags, each covering a word or even a few letters?

[TLucasIND]

@stephanrauh
Is it somehow possible, to tap the search-function of the underlying pdf.js?
My goal is to modify the DOM of the TextLayer as to permanently highlight the occurences of words in a words list.
I already tried your suggestion on https://pdfviewer.net/extended-pdf-viewer/find using the (updateFindMatchesCount) event but I assume that it is one layer too high.

My own approach of simply searching the DOM myself is working kind of, but it's too slow and some words aren't found.

Example: In the PDF is the Text "The quick brown fox jumps over a lazy dog".
Rendering it in ngx-extended-pdf-viewer results in a DOM as follows (notice the stripped word "brown"):

<span role="presentation">The</span>
<span role="presentation"> </span>
<span role="presentation">quick</span>
<span role="presentation"> </span>
<span role="presentation">br</span>
<span role="presentation">own</span>
...

There is no single element for the word "brown", so I cannot find it.

But when searching for "brown" in the extended-pdf-viewer the DOM changes in a way that "brown" appears in a single span and is highlighted. This formatting is of course lost again, when exiting the search mode.

Would it be possible to use the npm package of pdf.js and use their own find function to alter the DOM after utilizing the search/find function?

[calixteman]

When we refactored the text stuff our main goal was to fix almost all the bugs around copy/paste and search without degrading the text selection itself.
The main difficulty was to make the distinction between a normal white space (" ") and usual kerning.
There are several reasons to create a new chunk (span):

• the font or its size is changing;
• a char isn't really "following" the previous one (exponent, line break, ...);
• too much space between two consecutive chars ;
• too much negative space: it helps to improve the text selection, for example if you have the "abcde" which has the same width as "abc", then when you select "abcde" on the canvas it will result to only select "abc" in the text layer.

We have of course some heuristics to guess when a space is really a space and they're of course imperfect but they helped us to really fix a ton of bugs and the price to pay is sometimes to have too much spans.

As far as I can tell, TeX/LaTeX tends to produce a little more spans than a basic pdf producer mainly because of/thanks to Knuth algorithm and kerning adjustment to make nice line breaking.

If you can share the pdfs with a lot of spans, you can file a bug in pdf.js repo and I'll have a look on them and I'll see (when I'll have some time) if there is something we can do to improve the situation.

[TLucasIND]

I have a related question here. For the documents I'm working with, I get the spans too. Most of the time the spans are placed in a sense that I can utilize them. But sometimes there are spans inbetween single words, which breaks them up. Hence, these words won't match when manually searching for them (iterating the "presentation"-elements).

But I realized that using the built-in-search-function of the pdf-viewer these words are found and will be marked. Did you use a special technique (i.e. concatenating the elements) here?

[calixteman]

When searching the text is just extract from the pdf:
https://github.com/mozilla/pdf.js/blob/3256761ead52c495cb4e5ebdbaa0ebebee65538e/web/pdf_find_controller.js#L726

and the highlight is done thanks to:
https://github.com/mozilla/pdf.js/blob/75b8647a32e57c5f3bac246a6d01e83212b22976/web/text_highlighter.js#L31

[igalM]

I have resolved my issue by extracting the page content and searching the occurances of the words I need without the built-in-search-function, and then highlighting them myself.

[calixteman]

FYI, the search stuff in pdf.js tries to be at parity with Firefox search, and we try to handle correctly , spaces, eol, ...
https://github.com/mozilla/pdf.js/blob/3256761ead52c495cb4e5ebdbaa0ebebee65538e/web/pdf_find_controller.js#L35

So if you think that a feature is missing in search stuff, you can file a bug.