Is the pytest+doctest configuration ALLOW_UNICODE worth keeping?

[ajnelson-nist]

I was looking through doctests, tried modifying one, fed the modified file into python3 -m doctest, and got many more errors than I was expecting. Here's the excerpted shell transcript:

$ python3 -m doctest rdflib/term.py
[SNIP]
**********************************************************************
File "/tmp/rdflib/rdflib/term.py", line 702, in term.Literal.normalize
Failed example:
    Literal("a", datatype=XSD.integer, normalize=False)
Expected:
    rdflib.term.Literal(u'a', datatype=rdflib.term.URIRef(u'http://www.w3.org/2001/XMLSchema#integer'))
Got:
    rdflib.term.Literal('a', datatype=rdflib.term.URIRef('http://www.w3.org/2001/XMLSchema#integer'))
**********************************************************************
9 items had failures:
   2 of  21 in term.Literal
   3 of   4 in term.Literal.__abs__
   2 of  10 in term.Literal.__add__
   3 of   4 in term.Literal.__invert__
   4 of   5 in term.Literal.__neg__
   4 of   5 in term.Literal.__pos__
  11 of  12 in term.Literal._literal_n3
  11 of  13 in term.Literal.n3
   2 of   3 in term.Literal.normalize
***Test Failed*** 42 failures.

The issue is that the docstrings' code snippets contain the Python 2 style of unicode string spelling, u"some string". This hasn't been flagged as a test failure because (and here I'm guessing) it seems the docstrings are tested with pytest, and pytest is configured today to use ALLOW_UNICODE to bypass this issue.

As part of moving on from Python 2, should ALLOW_UNICODE be removed, and the doc strings' tests updated?

FWIW, my experience note: 41 of the 42 issues in rdflib/term.py were mechanically addressable by just replacing u' with '. However, the last issue that python3 -m doctest showed appeared to be a behavior change in the < operator for Literals, and it's not clear to me whether the behavior change was something else that "Python 2 mode" was letting slip by.

I don't know how much work this would be.

[aucampia]

As part of moving on from Python 2, should ALLOW_UNICODE be removed, and the doc strings' tests updated?

It seems like it should, yes.

[ajnelson-nist]

Thank you!