Priorities for Pyrus alpha / beta releases

This will serve a a place for us to figure out what to do for each alpha / beta releases. We will aim for X week interval between each release.

This will be moved into a proper tool when we have sketched out enough of this

• Use the new naming standard, this involves fixing up the file/class names http://wiki.php.net/pear/rfc/pear2_class_naming
  • Interfaces: IChannel => ChannelInterface
  • Proper case: Remotecategories => RemoteCategories?
  • Package/ICreator to CreatorInterface
  • PackageFile/IValidator => ValidatorInterface
• Finalize the vendor namespace, pear or PEAR, and make the change.
• Get all the deps released and moved to namespaces etc
• Finalize move to github
• Remove all @link or at least correct them to point to somewhere more meaningful (svn.pear.php.net doesn't really exist per se anyway). We could move this to a later stage if we intend on using pyrus.php.net as a link
• Fix up the coverage generator. Deal with dead code, make sure all maths are correct and such. Currently a lot of the work is done but the coloring per files is either being wrong or the math to do the coverage percentage is miles off, needs investigation.
• Investigate if Task/{dir}/rw.php is cruft, and if so, remove it.
  • removed
• Fix up REST/ protocol files
  • These are no longer used with the change to the Remote* channel interaction. I'm unsure if they should be added back since all the logic is in the specific files.
  • How does the installer know if it suppose to load 1.0 1.1 or 1.3 handling? I can't see (I might have to look closer) any logic which deals with that
  • it's directly in the Remote* files. There's a lot of if (isset($this->parent->protocols->rest['REST1.3'])) … for example.
    • That seems like a very bad design when a system is supporting multiple different versions of a standard - I think we need to rethink that. Sure the current approach solves some code duplication in a way but it makes the whole thing potentially end up like spagetti code :S
• Files still have a mention of the PHP license even if they also link to New BSD.
  • Are those files even needed? OSGuess is the only one used. The Task/{dir}/rw.php files appear to be cruft that needs to be removed.
  • There are more than just those as far as I can see. Pyrus/Validate.php is a prime example
  • We are missing the LICENSE file - Copy from PEAR and fix up the authors and s/PEAR/Pyrus/ I reckon
• We use fully qualified namespace class names everywhere even if we are in the namespace. Example is if we are in namespace pear2/Pyrus we still do new \pear2\Pyrus\Channel\classname, same for Logger - Oppose to just calling it like with unqualified or qualified names - This would make the code much more readable and uses the namespaces properly IMHO.
• ? Readability is enhanced by removing the redundant information. That's a big benefit of namespaces and referencing them without using fully qualified names, improved readability.
• Code understanding/comprehension can be helped by using FQN when the classes are not within the current namespace. IE, lessening the amount of imported namespaces/use statements. But, this also affects re-factoring, as FQN's are harder to work around by simple use changes. ^^^ IMHO

• Bi-directional registry upgrade using the pearregistryversion as the reference point. Simple class setup which has up/down and each migration has it's own class extending from the base class. It can do any XML/SQLite/Whatever fiddling we need.
• Look at merging together into a base class the various element access codes (re discussion with Greg)
• Figure out if the XMLWriter/Parser needs to support NS or not, at the moment none of our tests touch it and it doesn't seem to be used. Figure out use cases and write tests if we are keeping it.
• Use the new Exception standard http://wiki.php.net/pear/rfc/pear2_exception_policy
• Remove all the usage of pear2\Exception
• Fix up some of the file structures (Do not merge ChannelFile and ChannelRegistries into Channel please!)
• Comb the code and fix up CS problem and add more spacing around code blocks to make things more readable
• Look at replacing all serialzie() calls with json_encode/decode - REST cache and in SQLite3 Tasks... This will be the first migration script (it should delete the REST cache)

• Make sure everything in PEAR 1.8 and 1.9 are ported over to Pyrus (good bunch are listed here already)

upgrade and install work the same effectively, running pear install foo on an package that exists would upgrade and pear upgrade bar on a package that does not exist will install it. (see work done in 1.8)

• Support https properly (see work done in 1.8)
• Support 302 redirects properly (see work done in 1.8)
  • backport the redirect support added in http_request2?
• Make sure even if stable is preferred that if I have forced package-beta then if I upgrade everything I still pick up on beta release of said package (see work done in 1.8)
• channel specific upgrades (e.g. I only want to upgrade everything in pear.phpunit.de) (see work done in 1.8)
• Kill off upgrade-all infavor of using upgrade with no param (This was done in PEAR 1.8 and is in line with what other management softwares use anyway)
• cvsdiff / cvstag - Needs to be implemented along with git. svn, bazar and hbase most likley - Talk to Kore about his VCWrapper for that. to keep with svntag/svndiff/cvstag/svndiff in PEAR

• Add REST 1.4 to Pyrus as a consumer and SCS as a generator http://pastie.org/527988.txt This could be extended into REST 2.0 (complete overhaul, ditch old behaviors) or we could invest more time in making 1.4 even more efficient for everyone (1.4 only goes so far) -- using scfrontend as a frontend will really indicate which areas of the api are slow, and what areas we need to improve on. For example, adding releasedate to the latestpackages listing so we can automatically generate an rss feed of sorts. The biggest point from the installer perspective is to have less HTTP traffic, with 1.3 and below we are doing a lot of requests when doing an upgrade all or even just a normal upgrade. The difference will be bigger single files vs smaller many files.

• Look into what CLI values we can merge together and instead of switches. People are overwhelmed when they do pear help (having pyrus and pyrus_developer already helps tho). Drop list-all or similar things, make it way more simple and less confusing for the end users (see all the bogus bug reports in v1)
  • Figure out if the whole set / get switches for config are actually a good idea, they seem too ambigious to me.
• Fix cli error messages so they're user friendly - exceptions don't help
• Look over all the files and see if we have any dead files like we had Task/{dir}/rw.php that should have been removed already
• See if Console_Table or similar can be in PEAR2 - Make Pyrus use it possible ? (less duplication of code, less Pyrus code to maintain)
• Look at files over 1000 lines and see if they can be split up / refactored
• Simplify the HTTP code so REST and the Downloader use same base class to retrieve things with (less duplication et all) This might be more geared at PEAR and the old Pyrus code, needs investigation
  • I think this is already done..? We use configurable http driver for all http interaction, heavily used in testing with the mock Internet driver
  • Okey lets call this a verification task then, to make sure that it has been concolidated properly

• In PEAR doing an upgrade-all with many channels (10 is an avg number for people) is very slow and expensive operation, esp. if they are fairly out of date package wise. we need to look into how to make this both more efficient in contacting the channels and processing the responses we get back.
• Profile various common use cases (install, upgrade, uninstall) - This one should be repeated after the security work is done to see how much if affected speed.
• Investigate if internal structures are being as inefficient as PEARv1 had

• Add a new REST standard with API version in the release list, with package hash for each release in the list as well so we can programmatically verify nothing was changed (another level of safety).
  • output the hash of a package at install-time so that users can compare, and put the release hash on pear2.php.net package page
• Put whether the package is a plugin in the REST API.
• Use sha1 or sha256/512 for hash verifications, phars have internal hashes, but this needs to be prominently displayed to the end user through the cli, published in the rest api, and displayed on the web pages as well (Fedora / yum do this) People need to be able to run their own CLI sha1 or whichever tools that they have on their system, against what we are providing them to verify that this is valid. Internal phar hashes are more for internal consistency - So we'd have a 2 fold hash system really.

• Work on making the CLI frontend more consistent
• Make sure all the help documents are up to date and useful (pyrus help <command>)
• Make sure proxies work properly (neglected at large in v1)

• Finish documenting the API for programmers
• Design specs required for the auto create package.xml for more complex cases (Perhaps we still do need a PFM like thing for complex cases)
• This is already there, just need better documentation, see the generate-pear2 command for all the proper filesystem layout, then the make and package commands.. even customizing the package.xml and dependencies is available

• Marketing
• Write proper promotional and marketing material to establish Pyrus as the replacement of pear installer, make it an attractive choice for apps for their plugin system
• Setup a separate site for pyrus ? still be a part of pear but we need to have some gap between the installer and the repo to be honest pyrus.php.net
• Try to setup an effort to have more release schedule like approach ala Firefox and Gnome to keep the ball rolling. This is just not for us, but also for people to see we are active and alive, incentive for them to us pyrus as a plugin system.

• Write example apps that utilize the API for a plugin system and talk to Sean and get it into Hibari (iirc it is called)
• See AutoloadPackage for a dangerous example - http://github.com/saltybeagle/AutoloadPackage/blob/master/AutoloadPackage.php
• Try to keep Pyrus modular, anything that could be useful to other people should be in their own package (e.g. channel handling comes to mind). Talk to various people, figure out how they may want to use this.

• Get coverage to 80% or higher (Pyrus and Pyrus_Developer, potentially SCS as well)
• Make a PEAR to Pyrus migration guide in terms of using the commands (config-set -> set and such)

• Only require explicit packages requested for installation by a plugin to be plugins, allow plugin dependencies to be non-plugins.
• re-factor remote-list to always display with 80-char column width
• add ability to temporarily set config values with pyrus -d doc_dir=blah install
• add --offline as top-level option, pyrus -o install LocalPackage.tgz
• add --nodeps option to install/upgrade command. This is mostly implemented, but needs to be done by skipping pre-download dep resolution
• Add a -q quiet option (opposite of verbose) to hide all the extraneous info, make it easy to use `pyrus -q get php_dir`
• Finish unit testing core, begin unit testing developer package creation
• Add phpunit command that uses PHPUnit to run tests
  • Do we really want to? Isn't it better if people just use phpunit CLI for that? If there are any real advantages then I'm up for considering it (PEAR1 does support it so maybe we do have to carry it over but I think we should only do it if it brings any benefits to people)
• We need to start thinking like the SVN people - no inherit need to make the installer support the older REST service forever people just have to upgrade their installers to use certain channels, it's not a biggy, works for svn so it can work for us.
• Pre Install Script
  • Ability to auto run (if certain security things are passed, we will have to decide on that)
• Post Install Script
  • Ability to auto run (if certain security things are passed, we will have to decide on that)
• update channel info auto (security implications)
• Channels
  • dep files should also be in json (or only json ?) - This will allow not just PHP to consume the channel information. Handy for github for example where ruby would be used to generate, or potentially could, better integration into their system.
  • support .htpasswd for logins to a channel (private channels)
  • A pyrus publish command, see if it can be done with static channels, would be a nice addition for people, needs extra work on channel side and work in client, both should probably be additions to the installer / channel server
• Security
  • Look at checklist in http://www.cs.arizona.edu/people/justin/packagemanagersecurity/
  • Sign metadata in repos
  • Expire metadata in repos
  • auto sign packages on release (default? Needs a PGP key tho if we go for that)
  • [Greg Note: phar uses OpenSSL for signing, PGP is not needed, and it would require clients to have the openssl ext installed]
  • check signature on install / upgrade
  • don't allow md5 and sha1 for hashes in package.xml but support md5 for older packages
• On frontend, prompts and -y for automatic like pear is now, using pear install foo could auto imply -y (think yum)
  • prompt support is already there add -y auto-accept option
• uninstall should follow deps ala yum - uninstall the dep chain without compromising the whole tree
• Perhaps look into using headers more to our advantage, that will tho somewhat defeat the purpose of a fully static no vhost / .htaccess channels concept (perhaps that's fine)
• Proper mirror handling (has security implications that are in the section above)
• See if we can improve the bundle concept, very popular amongst a lot of pear haters it seems :)
• Keep memory footprint low, very low - CPU to minimum - the ususal performance thing but be more serious about it than in v1. E.g. no serialized objects inside another and another and so on. Makes PEAR hard to debug and makes it SLOW
• Look into how we can extend pyrus to make it to the same thing as the MS web installer thing-o-matic, could we even go as far as writing something that understands their config file and install their packages with Pyrus (extension to pyrus tho, not built in ?)

<radagast> I suppose we don't have anything which determines which pear packages are [potentially] being used by an app? I don't suppose pfm does this? :D <helgi> no <helgi> till wrote something like that <radagast> I'll check his blog so <radagast> thanks for the pointer <helgi> something that basically parsed all the includes or new Foo calls <radagast> yeah...in theory it seems simple enough <bbieber> PHP_Depend for PEAR packages.. hmm <radagast> bbieber: auto create “wrapper packages” * bbieber 's mind is blown away. <bbieber> radagast: I'd like one for Pyrus that scans then builds the whole distributable registry. <radagast> it'd be one way of luring people from distributing old versions of packages in their own projects <bbieber> I like the way you think <radagast> and then they wonder why things break and is bugg

Windows ENV variables:

• http://vlaurie.com/computers2/Articles/environment.htm
• http://vlaurie.com/computers2/Articles/environment-variables-windows-vista-7.htm
• http://en.kioskea.net/contents/systemes/variables-environnement.php3

• eventually, a web frontend inside Pyrus so we can run its web installer by putting pyrus.phar in htdocs
• eventually, a gtk frontend inside Pyrus so we can run it straight from php-gtk2
• The ability to freeze / hold a package, while frozen, one can not upgrade it (potentially not remove it either).http://pear.php.net/bugs/10570
• Ability to migrate one package from X channel to Y (PHPUnit) http://pear.php.net/bugs/4208 this one does impose some security implication we have to solve
• [Greg Note: ^^ This is already implemented]
• Look at AMO, yum, apt-get, canory and friends (PackageKit ?)
• rpm / deb generations out of the box (external packages like now ?) - potentially others, as plugins perhaps to have less pollution

investigate maven integration ?

• Obsolete OS/Guess if possible (that entitles writing a new solution) (highly unlikely and has been already talked about as far as I can see)