Differences

This shows you the differences between two versions of the page.

--- vcs:cvs2svnconversion [2008/10/09 04:43] – checkout results gwynne
+++ vcs:cvs2svnconversion [2019/08/26 16:54] (current) – gwynne
@@ Line 1: / Line 1: @@
+====== Complete ======
+The SVN migration was completed in July 2009. This document has been retained for historical purposes.
 ====== CVS to SVN Migration Path ======
@@ Line 32: / Line 37: @@
 I have the CVS repository stored in a directory called "realroot" (why not?) and stored the temporary data files on a separate mount point because of drive space issues. My initial commandline, based purely on the documentation, before any kind of testing was this:
 <code bash>
-./cvs2svn --svnrepos=./svnroot --fs-type=fsfs --dry-run --no-cross-branch-commits --username=svnconvert --cvs-revnums --use-cvs --tempdir=/Volumes/External/private/tmp/cvs2svn-tmp ./realroot
+./cvs2svn --svnrepos=./svnroot --fs-type=fsfs \
+--dry-run --no-cross-branch-commits \
+--username=svnconvert \
+--cvs-revnums --use-cvs \
+--tempdir=/Volumes/External/private/tmp/cvs2svn-tmp ./realroot
 </code>
@@ Line 99: / Line 108: @@
 </code>
-===== Pass 4 =====
+===== Passes 4-8 =====
 Pass 4 worked on the first try:
+Pass 5 was also a clean sweep:
+Pass 6 was just a beautiful thing, went by without a hitch. Not that I expected any of these "sort" passes to be a big deal, but you never know...
+Pass 7 worked without problems too, though it was a much slower pass than the last three. I finally started to feel like I was making progress. Pass 7 done, means pass 8 is up! Halfway there!
+Pass 8 took something along the lines of an hour to run, but it finally finished without errors... I hope the other phases aren't similarly insane with their timing. I'm considering taking the --quiet flag back out, and I think I will for pass 9. It's nice to know //something's// happening; I checked my top -u output twice during pass 8 to make sure it hadn't frozen up.
 <code>
 ----- pass 4 (FilterSymbolsPass) -----
 Filtering out excluded symbols and summarizing items...
 Done
-</code>
-When pass 4 succeeded, my commandline was:
-<code bash>
-./cvs2svn --svnrepos=./svnroot --fs-type=fsfs --dry-run --no-cross-branch-commits --username=svnconvert --cvs-revnums --use-internal-co --quiet --retain-conflicting-attic-files --encoding=ascii --encoding=utf_8 --encoding=utf_16 --encoding=shift_jis --encoding=mac_roman --encoding=latin_1 --encoding=euc_jp --exclude=php4/CREDITS --tempdir=/Volumes/External/private/tmp/cvs2svn-tmp ./realroot
-</code>
-===== Pass 5 =====
-Pass 5 was also a clean sweep:
-<code>
 ---- pass 5 (SortRevisionSummaryPass) -----
 Sorting CVS revision summaries...
@@ Line 122: / Line 129: @@
 </code>
-When pass 5 succeeded, my commandline was:
+When passes 4-8 succeeded, my commandline was:
-<code bash>
-./cvs2svn --svnrepos=./svnroot --fs-type=fsfs --dry-run --no-cross-branch-commits --username=svnconvert --cvs-revnums --use-internal-co --quiet --retain-conflicting-attic-files --encoding=ascii --encoding=utf_8 --encoding=utf_16 --encoding=shift_jis --encoding=mac_roman --encoding=latin_1 --encoding=euc_jp --exclude=php4/CREDITS --tempdir=/Volumes/External/private/tmp/cvs2svn-tmp ./realroot
-</code>
-===== Pass 6 =====
-Pass 6 was just a beautiful thing, went by without a hitch. Not that I expected any of these "sort" passes to be a big deal, but you never know...
-When pass 6 succeeded, my commandline was:
-<code bash>
-./cvs2svn --svnrepos=./svnroot --fs-type=fsfs --dry-run --no-cross-branch-commits --username=svnconvert --cvs-revnums --use-internal-co --quiet --retain-conflicting-attic-files --encoding=ascii --encoding=utf_8 --encoding=utf_16 --encoding=shift_jis --encoding=mac_roman --encoding=latin_1 --encoding=euc_jp --exclude=php4/CREDITS --tempdir=/Volumes/External/private/tmp/cvs2svn-tmp ./realroot
-</code>
-===== Pass 7 =====
-Pass 7 worked without problems too, though it was a much slower pass than the last three. I finally started to feel like I was making progress. Pass 7 done, means pass 8 is up! Halfway there!
-When pass 7 succeeded, my commandline was:
-<code bash>
-./cvs2svn --svnrepos=./svnroot --fs-type=fsfs --dry-run --no-cross-branch-commits --username=svnconvert --cvs-revnums --use-internal-co --quiet --retain-conflicting-attic-files --encoding=ascii --encoding=utf_8 --encoding=utf_16 --encoding=shift_jis --encoding=mac_roman --encoding=latin_1 --encoding=euc_jp --exclude=php4/CREDITS --tempdir=/Volumes/External/private/tmp/cvs2svn-tmp ./realroot
-</code>
-===== Pass 8 =====
-Pass 8 took something along the lines of an hour to run, but it finally finished without errors... I hope the other phases aren't similarly insane with their timing. I'm considering taking the --quiet flag back out, and I think I will for pass 9. It's nice to know //something's// happening; I checked my top -u output twice during pass 8 to make sure it hadn't frozen up.
-When pass 8 succeeded, my commandline was:
 <code bash>
 ./cvs2svn --svnrepos=./svnroot --fs-type=fsfs --dry-run --no-cross-branch-commits --username=svnconvert --cvs-revnums --use-internal-co --quiet --retain-conflicting-attic-files --encoding=ascii --encoding=utf_8 --encoding=utf_16 --encoding=shift_jis --encoding=mac_roman --encoding=latin_1 --encoding=euc_jp --exclude=php4/CREDITS --tempdir=/Volumes/External/private/tmp/cvs2svn-tmp ./realroot
@@ Line 196: / Line 176: @@
 </code>
-===== Pass 10 =====
+===== Passes 10-16 =====
+: Pass 10 was what my Warcraft friends would call "easysauce":
+: Well, pass 11 was faster than 8 and 9, if not as fast as 10...
+: Another one bites the dust!
+: Pass 13 certainly was interesting. Generating all the SVN commits...
+: Nice short easy one.
+: 15 down, 1 to go!
+: Pop the champagne cork!
-Pass 10 was what my Warcraft friends would call "easysauce":
 <code>
 ----- pass 10 (BreakSymbolChangesetCyclesPass) -----
@@ Line 204: / Line 197: @@
 Done
 Time for pass10 (BreakSymbolChangesetCyclesPass): 181.9 seconds.
-</code>
-When pass 10 succeeded, my commandline was:
-<code bash>
-./cvs2svn --svnrepos=./svnroot --fs-type=fsfs --dry-run --no-cross-branch-commits --username=svnconvert --cvs-revnums --use-internal-co --retain-conflicting-attic-files --encoding=ascii --encoding=utf_8 --encoding=utf_16 --encoding=shift_jis --encoding=mac_roman --encoding=latin_1 --encoding=euc_jp --exclude=php4/CREDITS --tempdir=/Volumes/External/private/tmp/cvs2svn-tmp ./realroot
-</code>
-===== Pass 11 =====
-Well, pass 11 was faster than 8 and 9, if not as fast as 10...
-<code>
 ----- pass 11 (BreakAllChangesetCyclesPass) -----
 Breaking CVSSymbol dependency loops...
 Done
 Time for pass11 (BreakAllChangesetCyclesPass): 1039 seconds.
-</code>
-When pass 11 succeeded, my commandline was:
-<code bash>
-./cvs2svn --svnrepos=./svnroot --fs-type=fsfs --dry-run --no-cross-branch-commits --username=svnconvert --cvs-revnums --use-internal-co --retain-conflicting-attic-files --encoding=ascii --encoding=utf_8 --encoding=utf_16 --encoding=shift_jis --encoding=mac_roman --encoding=latin_1 --encoding=euc_jp --exclude=php4/CREDITS --tempdir=/Volumes/External/private/tmp/cvs2svn-tmp ./realroot
-</code>
-===== Pass 12 =====
-Another one bites the dust!
-<code>
 ----- pass 12 (TopologicalSortPass) -----
 Generating CVSRevisions in commit order...
 Done
 Time for pass12 (TopologicalSortPass): 255.5 seconds.
-</code>
-When pass 12 succeeded, my commandline was:
-<code bash>
-./cvs2svn --svnrepos=./svnroot --fs-type=fsfs --dry-run --no-cross-branch-commits --username=svnconvert --cvs-revnums --use-internal-co --retain-conflicting-attic-files --encoding=ascii --encoding=utf_8 --encoding=utf_16 --encoding=shift_jis --encoding=mac_roman --encoding=latin_1 --encoding=euc_jp --exclude=php4/CREDITS --tempdir=/Volumes/External/private/tmp/cvs2svn-tmp ./realroot
-</code>
-===== Pass 13 =====
-Pass 13 certainly was interesting. Generating all the SVN commits...
-<code>
 ...
 Creating Subversion r192256 (commit)
 Done
 Time for pass13 (CreateRevsPass): 512.2 seconds.
-</code>
-So 192,256 commits in the CVS repository, including tagging and branching. That's a pretty impressive total over 12 years, don't you think?
-When pass 13 succeeded, my commandline was:
-<code bash>
-./cvs2svn --svnrepos=./svnroot --fs-type=fsfs --dry-run --no-cross-branch-commits --username=svnconvert --cvs-revnums --use-internal-co --retain-conflicting-attic-files --encoding=ascii --encoding=utf_8 --encoding=utf_16 --encoding=shift_jis --encoding=mac_roman --encoding=latin_1 --encoding=euc_jp --exclude=php4/CREDITS --tempdir=/Volumes/External/private/tmp/cvs2svn-tmp ./realroot
-</code>
-===== Pass 14 =====
-Nice short easy one.
-<code>
 ----- pass 14 (SortSymbolsPass) -----
 Sorting symbolic name source revisions...
 Done
 Time for pass14 (SortSymbolsPass): 9.787 seconds.
-</code>
-When pass 14 succeeded, my commandline was:
-<code bash>
-./cvs2svn --svnrepos=./svnroot --fs-type=fsfs --dry-run --no-cross-branch-commits --username=svnconvert --cvs-revnums --use-internal-co --retain-conflicting-attic-files --encoding=ascii --encoding=utf_8 --encoding=utf_16 --encoding=shift_jis --encoding=mac_roman --encoding=latin_1 --encoding=euc_jp --exclude=php4/CREDITS --tempdir=/Volumes/External/private/tmp/cvs2svn-tmp ./realroot
-</code>
-===== Pass 15 =====
-down, 1 to go!
-<code>
 ----- pass 15 (IndexSymbolsPass) -----
 Determining offsets for all symbolic names...
 Done.
 Time for pass15 (IndexSymbolsPass): 6.344 seconds.
-</code>
+----- pass 16 (OutputPass) -----
-When pass 15 succeeded, my commandline was:
-<code bash>
-./cvs2svn --svnrepos=./svnroot --fs-type=fsfs --dry-run --no-cross-branch-commits --username=svnconvert --cvs-revnums --use-internal-co --retain-conflicting-attic-files --encoding=ascii --encoding=utf_8 --encoding=utf_16 --encoding=shift_jis --encoding=mac_roman --encoding=latin_1 --encoding=euc_jp --exclude=php4/CREDITS --tempdir=/Volumes/External/private/tmp/cvs2svn-tmp ./realroot
-</code>
-===== Pass 16 =====
-Pop the champagne cork!
-<code>
 Starting Subversion r192256 / 192256
 Done.
@@ Line 463: / Line 389: @@
 </code>
-Whew. That was going to make for a //very// long options file where it was very easy to make copypasta errors. I needed to add a little Python code. How does one do foreach (array(//blah blah blah//) as $item) { /* etc */ } in Python? So I went to my wife Sarah, who **does** know Python.
+Whew. That was going to make for a //very// long options file where it was very easy to make copypasta errors. I needed to add a little Python code. How does one do foreach (array(//blah blah blah//) as $item) { /* etc */ } in Python? So I went to a close friend who **does** know Python.
 We came up with this rather handy little bit of code:
@@ Line 951: / Line 877: @@
 $ sudo -u svn \
 svn rm -m "[SVN CONVERSION] Removing .svn directories that break SVN checkout." \
+  $SVNROOT/pear/Selenium/branches/shin/.svn \
+  $SVNROOT/pear/Selenium/branches/shin/tests/.svn \
+  $SVNROOT/pear/Selenium/branches/shin/tests/events/.svn \
+  $SVNROOT/pear/Selenium/branches/shin/tests/html/.svn \
+  $SVNROOT/pear/Selenium/branches/shin/docs/.svn \
+  $SVNROOT/pear/Selenium/branches/shin/examples/.svn \
   $SVNROOT/pear/Selenium/tags/start/tests/.svn \
   $SVNROOT/pear/Selenium/tags/start/tests/events/.svn \
@@ Line 962: / Line 894: @@
   $SVNROOT/pear/Testing_Selenium/branches/shin/tests/html/.svn \
   $SVNROOT/pear/Testing_Selenium/branches/shin/docs/.svn \
-  $SVNROOT/pear/Testing_Selenium/branches/shin/examples/.svn
+  $SVNROOT/pear/Testing_Selenium/branches/shin/examples/.svn \
+  $SVNROOT/pear/Testing_Selenium/tags/start/.svn \
+  $SVNROOT/pear/Testing_Selenium/tags/start/tests/.svn \
+  $SVNROOT/pear/Testing_Selenium/tags/start/tests/events/.svn \
+  $SVNROOT/pear/Testing_Selenium/tags/start/tests/html/.svn \
+  $SVNROOT/pear/Testing_Selenium/tags/start/docs/.svn \
+  $SVNROOT/pear/Testing_Selenium/tags/start/examples/.svn
 Committed revision 279477.
 $
 </code>
+===== Meta-SVN! =====
+About this time I realized that a lot of things related to SVN would require version control //before// the repository was ready for use! Things like all the various scripts involved in the conversion itself, all the authorization data, the commit hooks, all the fun stuff. Putting these things into CVS would result in a bit of recursive failure. Putting them into the SVN repository I'd set up would interfere with the conversion, and besides, this was metadata, stuff that belongs in an equivelant to CVSROOT. Solution: A second SVN repository under much more restricted authorization control. I put in a request for a metasvn.php.net domain name and set up cvs.php.net's Apache to serve it from a separate repository. Then Wez and a couple others convinced me that was a stupid idea. There wasn't //really// any reason this stuff couldn't go into CVS, other than my ornery resistance to the older and less useful system.
+It was about this time that I had to study Git for another project and began to wonder if maybe it wasn't better than SVN, but I'm just not into the idea of learning an entirely new system and forcing everyone else to do the same. SVN maps 90% onto CVS commands... Git maps more like 40%. SVN is a good midway step to true distributed VCS, and there are plenty of Git/SVN interface tools.
+So I set up a CVS module called SVNROOT/, got karma to it, and checked in my options file along with the checkout script above. Almost immediately I got an interesting question:
+"Didn't we decide to use PHP instead of Python?"
+Yes, we did. And yes, the options file is written in Python. Unfortunately, the way cvs2svn is set up makes this necessary; it includes the options file similarly to a PHP include directive.
+===== Reorganization ====
+Next step: Decide on a repository structure. Ooops... lots of differing opinions on that.
+Well, this was getting complicated. It was time to step back and automate some of the process. So I popped open a new PHP file and came up with automation for the svn create, cvs2svn, and svn rm commands already discussed. Then I went back and added some nice command-line-y-ness to it using PEAR's Console_CommandLine (a VERY nice package, kudos to its author(s)!). The script can be viewed at [[http://cvs.php.net/viewvc.cgi/SVNROOT/run-conversion.php?view=log]].
+That done, I looked back at the reorganization mess. It looked like there would in fact be a few separate repositories for things like PEAR and GTK. I needed advice on this one, so I went to the mailing list. They wanted to know, "why separate repositories?" Well, it's a matter of maitenance, really. GTK, PEAR, Zend, they all have their own little quirks in the hook scripts and really it's just simpler and more elegant for them to have their own workspaces to play in rather than all this endless special-casing in the hooks and ACLs.
+So I rewrote the conversion script completely to support this premise, and contacted various people to find out what to do with the "miscellaneous" modules scattered all over the place. Turned out most of them either belonged alongside php-src or were just plain defunct! The choice was made not to convert defunct modules, since there is a plan to leave the CVS repository available in some form.
+===== Hook scripts =====
+At a glance it might seem that would be the end of it. But unfortunately, no. There are a lot of administrative tasks done by scripts in CVSROOT, all of which need to be ported to SVN equivelants. I decided it would be astute to make a list of what needed to be ported before actually getting into it! To do that, I grabbed a copy of CVSROOT itself and had a looksee. It turned out the following things needed conversion:
+  * Access Control Lists - replaced by the SVN authz database
+  * commitinfo.pl - I couldn't quite figure out what this was for. It seemed to write the name of the committed directory to a file. A little more investigation showed it to be part of the loginfo.pl automation
+  * cvswrappers - Replaced by SVN's autoprops
+  * loginfo.pl - Sends the e-mails to various mailing lists when commits happen
+  * modules - Replaced by svn:externals and restructuring
+  * readers - Replaced by SVN's authz database
 ===== Available for the curious =====
@@ Line 974: / Line 944: @@
 </code>
-This will check out all the projects in the repository; it's suggested to specify a particular module like <nowiki><http://svn.php.net/php-src/trunk></nowiki>. Don't forget about svn ls!
+This will check out all the projects in the repository; it's suggested to specify a particular module like [[http://svn.php.net/php-src/trunk]]. Don't forget about svn ls!

Differences

Page Tools