Simon Cross
2010-09-05 13:58:03 UTC
Hi Jeroen (and others)
It looks like the sprint was fairly successful and we have a patch
that updates Genshi to be Python 3 compatible (without losing Python 2
compatibility). The patch is available at either of:
* bitbucket -- http://bitbucket.org/hodgestar/genshi-py3k
* CTPUG wiki --
http://ctpug.org.za/attachment/wiki/Meeting20100904/genshi-py3k.diff
Bitbucket has the full commit history. The sprint occurred in two
stages. First we worked on simply porting to Python 3 without
maintaining compatibility with Python 2. This work was done in the
2to3 bitbucket branch and took about seven hours. In the second stage
we created a single code-base that supports Python 2 and Python 3 (via
2to3 and some helpers). This was done in the default branch and took
about eight hours.
We've tested the result under Python 2.4, 2.5, 2.6, 3.1 and 3.2 and on
Linux (Ubuntu Lucid and Maverick) and Mac OS X.
API Change
^^^^^^^^^^^
We made one important API change. Previously the default encoding used
by Genshi in a number of functions was UTF-8. This is a little awkward
in Python 3 since strings are unicode (and it's generally bytes that
have to be explicitly asked for). We opted to change the default
encoding to None (i.e. use a unicode string). Although this breaks
compatibility, we think it eases the migration towards Python 3 (and
it made porting easier).
Usage
^^^^^^
The patch should apply cleanly against trunk. After applying one can
build, test and install under Python 2 using:
$ python setup.py build
$ python setup.py test
$ python setup.py install
and under Python 3 using:
$ python3 setup.py build
$ python3 setup.py test
$ python3 setup.py install
Running setup.py under Python 3 requires Distribute (which runs under
Python 3 and provides hooks for running 2to3).
Patch Tricks
^^^^^^^^^^^
- "if not isinstance(s, unicode):" is converted by 2to3 while "if
isinstance(s, str)" is not.
- "u'foo'.encode('utf-8')" creates a byte string in UTF-8 both before
and after 2to3 while "'foo'" does not.
Patch Details
^^^^^^^^^^^^
.hgignore
README
- Hg ignore file and note about the patch. No need to commit.
setup.py
MANIFEST.in
- changes to make setup.py importable by both Python 2 and Python 3
(without 2to3).
- include tests in build so that "python3 setup.py test" works.
- Distribute runs the tests on the build rather than on the source
since the source is not modified by 2to3.
- add Distribute 2to3 options when run with Python 3.
doc/common/doctools.py
- change to an svn:external to make prints python3 compatible.
- doctools is import by setup.py so this needs to be importable by
python3 without running 2to3 on it.
examples_to_py3k.sh
- script to run 2to3 on the python files in the examples folder.
fixes/fix_unicode_in_strings.py
- We wrote a custom 2to3 fixer to fix unicode output strings inside
doctests and other string constants.
genshi/_speedups.c
- ported using #ifdefs for Python 3.
genshi/compat.py
- new home for cross-python-version compatibility functions
genshi/core.py
genshi/tests/core.py
- change default encoding from UTF-8 to None (i.e. unicode strings)
- stringrepr no longer used inside __repr__ in Python 3 repr()
returns a string (i.e. unicode)
genshi/input.py
genshi/tests/input.py
- change default encoding for HTML() to None.
- track changes to expat parser in Python 3 (mostly it accepts bytes
instead of strings).
genshi/output.py
genshi/tests/output.py
- change default encoding for encode() to None.
genshi/util.py
- move cross-Python-version compatibility fixes into genshi.compat
genshi/filters/html.py
- minor changes to track encoding=None API change
genshi/filters/tests/html.py
- renamed to test_html.py. In Python 3 there is a top-level module
named html and having this overridden when
running "python3 genshi/filters/tests/__init__.py" or similar was
incredibly frustrating.
- left genshi/filters/html.py with its own name since
genshi/filters/ doesn't typically end up in sys.path. :)
genshi/filters/i18n.py
genshi/filters/tests/i18n.py
- ugettext and friends are gone in Python 3 (and only gettext and
friends exist and they now handle unicode).
- Some \ line continuations inside doctests confused 2to3 so we removed them.
- Testing picked up a problem (already present in trunk) where
Translator.__call__ could end up defining gettext
as an endlessly recursive function. Noted with a TODO.
genshi/filters/transform.py
genshi/filters/tests/transform.py
- minor changes to track encoding=None API change
genshi/template/astutil.py
- AST for raise has changed in Python 3.
- Python 3 adds AST nodes for individual arguments and Bytes.
genshi/template/base.py
- distinguish between bytes and unicode in Python 3 compatible way.
genshi/template/directives.py
genshi/template/tests/directives.py
- slightly odd syntax changes to make the 2to3 .next() fixer pick
up *stream.next().
- minor test fix for change in behaviour of division (/) in Python 3.
genshi/template/eval.py
genshi/template/tests/eval.py
- use genshi.compat functions for dealing with code objects.
- replace doctests that reply on exception names with uglier but
more compatible try:.. except:.. doctest
- handle filename preferences of Python 2 and 3 (2 prefers bytes, 3
prefers unicode).
- ifilter is gone from itertools in Python 3 so use repeat for tests instead.
A more minor change is that the ASTTransformer in Genshi coerces byte
strings to unicode (in genshi/templates/eval.py -- visit_Str). I'm not
sure I fully understand the rationale for this in Python 2 but in
Python 3 it's definitely wrong. We've retained the old behaviour in
Python 2 and left bytes as they are in Python 3.
genshi/template/loader.py
genshi/template/tests/loader.py
- add 'b' to file modes to ensure it's loaded as bytes in Python 3.
genshi/template/tests/markup.py
- use BytesIO compatibility function in test
genshi/template/plugin.py
genshi/template/tests/plugin.py
- track encoding=None change.
genshi/template/text.py
- use not isinstance(s, unicode) instead of isinstance(s, str)
Schiavo
Simon
It looks like the sprint was fairly successful and we have a patch
that updates Genshi to be Python 3 compatible (without losing Python 2
compatibility). The patch is available at either of:
* bitbucket -- http://bitbucket.org/hodgestar/genshi-py3k
* CTPUG wiki --
http://ctpug.org.za/attachment/wiki/Meeting20100904/genshi-py3k.diff
Bitbucket has the full commit history. The sprint occurred in two
stages. First we worked on simply porting to Python 3 without
maintaining compatibility with Python 2. This work was done in the
2to3 bitbucket branch and took about seven hours. In the second stage
we created a single code-base that supports Python 2 and Python 3 (via
2to3 and some helpers). This was done in the default branch and took
about eight hours.
We've tested the result under Python 2.4, 2.5, 2.6, 3.1 and 3.2 and on
Linux (Ubuntu Lucid and Maverick) and Mac OS X.
API Change
^^^^^^^^^^^
We made one important API change. Previously the default encoding used
by Genshi in a number of functions was UTF-8. This is a little awkward
in Python 3 since strings are unicode (and it's generally bytes that
have to be explicitly asked for). We opted to change the default
encoding to None (i.e. use a unicode string). Although this breaks
compatibility, we think it eases the migration towards Python 3 (and
it made porting easier).
Usage
^^^^^^
The patch should apply cleanly against trunk. After applying one can
build, test and install under Python 2 using:
$ python setup.py build
$ python setup.py test
$ python setup.py install
and under Python 3 using:
$ python3 setup.py build
$ python3 setup.py test
$ python3 setup.py install
Running setup.py under Python 3 requires Distribute (which runs under
Python 3 and provides hooks for running 2to3).
Patch Tricks
^^^^^^^^^^^
- "if not isinstance(s, unicode):" is converted by 2to3 while "if
isinstance(s, str)" is not.
- "u'foo'.encode('utf-8')" creates a byte string in UTF-8 both before
and after 2to3 while "'foo'" does not.
Patch Details
^^^^^^^^^^^^
.hgignore
README
- Hg ignore file and note about the patch. No need to commit.
setup.py
MANIFEST.in
- changes to make setup.py importable by both Python 2 and Python 3
(without 2to3).
- include tests in build so that "python3 setup.py test" works.
- Distribute runs the tests on the build rather than on the source
since the source is not modified by 2to3.
- add Distribute 2to3 options when run with Python 3.
doc/common/doctools.py
- change to an svn:external to make prints python3 compatible.
- doctools is import by setup.py so this needs to be importable by
python3 without running 2to3 on it.
examples_to_py3k.sh
- script to run 2to3 on the python files in the examples folder.
fixes/fix_unicode_in_strings.py
- We wrote a custom 2to3 fixer to fix unicode output strings inside
doctests and other string constants.
genshi/_speedups.c
- ported using #ifdefs for Python 3.
genshi/compat.py
- new home for cross-python-version compatibility functions
genshi/core.py
genshi/tests/core.py
- change default encoding from UTF-8 to None (i.e. unicode strings)
- stringrepr no longer used inside __repr__ in Python 3 repr()
returns a string (i.e. unicode)
genshi/input.py
genshi/tests/input.py
- change default encoding for HTML() to None.
- track changes to expat parser in Python 3 (mostly it accepts bytes
instead of strings).
genshi/output.py
genshi/tests/output.py
- change default encoding for encode() to None.
genshi/util.py
- move cross-Python-version compatibility fixes into genshi.compat
genshi/filters/html.py
- minor changes to track encoding=None API change
genshi/filters/tests/html.py
- renamed to test_html.py. In Python 3 there is a top-level module
named html and having this overridden when
running "python3 genshi/filters/tests/__init__.py" or similar was
incredibly frustrating.
- left genshi/filters/html.py with its own name since
genshi/filters/ doesn't typically end up in sys.path. :)
genshi/filters/i18n.py
genshi/filters/tests/i18n.py
- ugettext and friends are gone in Python 3 (and only gettext and
friends exist and they now handle unicode).
- Some \ line continuations inside doctests confused 2to3 so we removed them.
- Testing picked up a problem (already present in trunk) where
Translator.__call__ could end up defining gettext
as an endlessly recursive function. Noted with a TODO.
genshi/filters/transform.py
genshi/filters/tests/transform.py
- minor changes to track encoding=None API change
genshi/template/astutil.py
- AST for raise has changed in Python 3.
- Python 3 adds AST nodes for individual arguments and Bytes.
genshi/template/base.py
- distinguish between bytes and unicode in Python 3 compatible way.
genshi/template/directives.py
genshi/template/tests/directives.py
- slightly odd syntax changes to make the 2to3 .next() fixer pick
up *stream.next().
- minor test fix for change in behaviour of division (/) in Python 3.
genshi/template/eval.py
genshi/template/tests/eval.py
- use genshi.compat functions for dealing with code objects.
- replace doctests that reply on exception names with uglier but
more compatible try:.. except:.. doctest
- handle filename preferences of Python 2 and 3 (2 prefers bytes, 3
prefers unicode).
- ifilter is gone from itertools in Python 3 so use repeat for tests instead.
A more minor change is that the ASTTransformer in Genshi coerces byte
strings to unicode (in genshi/templates/eval.py -- visit_Str). I'm not
sure I fully understand the rationale for this in Python 2 but in
Python 3 it's definitely wrong. We've retained the old behaviour in
Python 2 and left bytes as they are in Python 3.
genshi/template/loader.py
genshi/template/tests/loader.py
- add 'b' to file modes to ensure it's loaded as bytes in Python 3.
genshi/template/tests/markup.py
- use BytesIO compatibility function in test
genshi/template/plugin.py
genshi/template/tests/plugin.py
- track encoding=None change.
genshi/template/text.py
- use not isinstance(s, unicode) instead of isinstance(s, str)
Schiavo
Simon
--
You received this message because you are subscribed to the Google Groups "Genshi" group.
To post to this group, send email to ***@googlegroups.com.
To unsubscribe from this group, send email to genshi+***@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/genshi?hl=en.
You received this message because you are subscribed to the Google Groups "Genshi" group.
To post to this group, send email to ***@googlegroups.com.
To unsubscribe from this group, send email to genshi+***@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/genshi?hl=en.