[Twisted-Python] Deprecating subproject packaging

[HawkOwl]

Hi,

I think that Twisted should deprecate subproject packaging -- they aren't uploaded to pip, don't support Python 3, I don't think we have docs for using them anywhere, and when included in downstream they usually only end up in confusion as various imports fail but some succeed due to only some subprojects being installed. Eventuall removing them means we have less code (and only one setup.py + setup3.py), less packages, and simplifies things.

It does mean that you can't *just* get Twisted Web, but Twisted Core is so big (internet, trial, amp, python, logger, cred, protocols -- even positioning is in there) that the size difference is marginal. For example, uncompressed Twisted (without the docs) is 14MB, while Core alone is 7MB. Assuming that you want to use Twisted Web, you'll need Twisted-Web and Twisted-Names (because not using Twisted Names in web.http.Request is deprecated), which is ~9MB. The difference in tarball size is even less -- Twisted without docs being 2.2MB and Twisted Core + Web + Names being 1.4MB.

Plus, I am not sure that there is "subprojects" really, anymore -- it's all just Twisted.

So, I think we should:

- Deprecate the subproject setup.pys starting with 15.2 and stop packaging subprojects separately as of 16.0 (15.2 in May, 15.3 in July, 15.4 in September, 15.5 in November, 16.0 in January 2016 -- so ~4 releases + 1 major release notice).
- Remove the support code for subprojects in twisted.python._release + the release scripts in the 16.0 release.
- Migrate Twisted's source dist to being done by "python setup.py sdist" instead of manually assembling the tarball. This means we can generate the requisite PKG-INFO metadata automatically. This should be done for the main tarball (which is uploaded to PyPI) in maybe 15.2, but not the subproject tarballs (Twisted Core + rest) which will remain the same until they are removed in 16.0.
- Replace manually uploading these distributions to PyPI and use twine to automate it for 15.2, when Twisted's tarball has the needed PKG-INFO (from being generated by sdist). Twine doesn't support uploading .msis, but as we now have Python wheels on PyPI, I think we can do without uploading MSIs, and just have them on the website.

Keeping the projects topfiles as they are I think is good -- although I am not sure if having a NEWS in each subproject is useful. Maybe we should just have the one NEWS file once 16.0 lands, since the other ones wouldn't be packaged, and are therefore less useful.

As for downstream:
- Apple includes all of Twisted, so I don't think this will affect them.
- Ubuntu and Debian package the subprojects separately. I guess they would have to make dummy packages that depend on the monolithic python-twisted instead.
- Fedora (so Red Hat too?) have obsoleted the subprojects and point them at python-twisted instead, so they don't package the subprojects.
- FreeBSD packages the Twisted subprojects separately, so they would have to do the same as Debian to migrate.

Overall, I don't think this adversely affects downstream in the long term -- it does what Red Hat did as of Fedora 21, and it'll end up being less packages overall.

I think this is a good idea and am willing to put the work in to implement this. Does anyone object to this? Is there a use case for keeping the subprojects packagable that I don't know about?

- Hawkie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 455 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://twistedmatrix.com/pipermail/twisted-python/attachments/20150422/4db276c1/attachment.pgp>