[cmview-devel] First steps to remove Sphoxel data from cmview repos

Tue Apr 5 04:50:34 EDT 2011

Fantastic! thanks a lot!

It seems to work as if nothing happened, that's amazing. My existing 
working copies are all fine and didn't even notice it (as the revisions 
really didn't change, no new revisions added or anything). I also tried 
checking out revisions around the removed ones (revs 684 to 718) and all 
worked. The removed revisions are simply marked as "no author" and with 
a comment "empty revision for padding".

As for the preclean repo in /svnroot/cmview-preclean we could get rid of 
it straight away as long as we have a dump of it somewhere else (you say 
we do have one in tallyn don't we?). Dallas0 could very well do with 5GB 
more! If we do eventually find something not working we can reload the 
dump somewhere.

Anyway I'll store yet another copy of the dumps here at PSI. So that we 
have even more data redundancy across borders ;)

Jose

On 04/04/2011 07:42 PM, Henning Stehr wrote:
> Hi Jose,
>
> I finished the clean-up of cmview now. It's basically all done except
> for some more testing.
> Here is what I did (for future reference):
>
> # -- on talyn --
>
> # create a mirror of the repository
> svnadmin create /project/StruPPi/svnsync/cmview
> svnadmin setuuid /project/StruPPi/svnsync/cmview
> 11196e73-9f2b-0410-a4ce-b943a9933111
> svnsync init file:///project/StruPPi/svnsync/cmview
> svn://bioinformatics.org/svnroot/cmview
> svnsync sync file:///project/StruPPi/svnsync/cmview
>
> # filter out unwanted data
> svnadmin dump /project/StruPPi/svnsync/cmview>  cmview-svndump-r794
> svndumpfilter exclude trunk/src/resources/sphoxelBG<
> cmview-svndump-r794>  cmview-svndump-r794-cleaned
>
> # -- on dallas0 --
>
> # replacing the live version
> mv /home/svn/cmview /home/svn/cmview-preclean
> svnadmin create /home/svn/cmview
> svnadmin load /home/svn/cmview<  cmview-svndump-r794-cleaned
>
> # -- end procedure --
>
> The new repository takes now 18MB instead of 4.9GB. Hooray!
>
> It seems that the config files in dallas0:/home/svn/cmview-preclean
> are the default ones so I didn't copy any of them over. I tested that
> I can check out from svn://bioinformatics.org/svnroot/cmview. I can
> also update my old working copy without relocating.  Eventually, we
> can get rid of dallas0:/home/svn/cmview-preclean which is the old
> version. In any case, we still have backup dump files in
> talyn:/project/StruPPi/svnsync.
>
> Cheers,
> Henning
>
>
> On Mon, Apr 4, 2011 at 1:19 PM, Henning Stehr<stehr at molgen.mpg.de>  wrote:
>> Hi Jose,
>>
>> I will test it locally a bit first. Then I can send you the (cleaned
>> up) dumpfile.
>> Maybe you can import it to the actual repository since you own the files.
>>
>> I was thinking that we can just mv /svnroot/cmview /svnroot/cmview-backup
>> so we can go back to the old one in case of problems. Then we can
>> svnadmin create /svnroot/cmview
>> and load from the dumpfile. The only thing is, the config files and
>> stuff will have to be copied over from
>> cmview-backup then.
>>
>> I'll let you know when I'm ready.
>>
>> Cheers,
>> Henning
>>
>> On Mon, Apr 4, 2011 at 12:47 PM, Jose Duarte<jose.duarte at psi.ch>  wrote:
>>> Great, sounds like a good plan!
>>>
>>> Let me know if you do it, I can test it for a while to make sure everything
>>> is working before we replace the repository.
>>>
>>> Jose
>>>
>>>
>>> On 04/04/2011 12:27 PM, Henning Stehr wrote:
>>>> Hi Jose,
>>>>
>>>> I started to have a look at how to remove the monster file(s) from the
>>>> svn history.
>>>>
>>>> In principle, the procedure using 'svndumpfilter' is described here:
>>>>
>>>> http://svnbook.red-bean.com/en/1.5/svn.reposadmin.maint.html#svn.reposadmin.maint.filtering
>>>>
>>>> The command 'svndumpfilter' takes a pathname to exclude from the history.
>>>>   From inspecting the history with 'svn log', it seems that it should be
>>>> sufficient to exclude the path:
>>>>
>>>> trunk/src/resources/sphoxelBG
>>>>
>>>> So, the relevant commands should be:
>>>>
>>>> svnadmin dump REPOS_PATH>    dump-file
>>>> svndumpfilter exclude trunk/src/resources/sphoxelBG<    dump-file>
>>>>   new-dump-file
>>>> svnadmin load REPOS_PATH<    new-dump-file
>>>>
>>>> I'll double check and give it a try later. I'm posting this to keep a
>>>> record and to let you know what's going on.
>>>>
>>>> Cheers,
>>>> Henning
>>>>
>>>> P.S. Here is the list of revisions which will be affected (if I didn't
>>>> overlook anything):
>>>>
>>>> r718 | cvehlow | 2010-11-29 12:17:39 +0100 (Mon, 29 Nov 2010) | 1 line
>>>>     D /trunk/src/resources/sphoxelBG
>>>> r688 | vehlow | 2010-07-09 13:04:54 +0200 (Fri, 09 Jul 2010) | 1 line
>>>>     M /trunk/src/cmview/gmbp/ContactPane.java
>>>>     A /trunk/src/resources/sphoxelBG/SphoxelBGs.zip
>>>> r687 | vehlow | 2010-07-09 12:34:24 +0200 (Fri, 09 Jul 2010) | 1 line
>>>>     D /trunk/src/resources/sphoxelBG/SphoxelBGs.zip
>>>>     D /trunk/src/resources/sphoxelBG/allSphoxelBGs.zip
>>>> r686 | vehlow | 2010-07-09 12:18:42 +0200 (Fri, 09 Jul 2010) | 1 line
>>>>     A /trunk/src/resources/sphoxelBG/allSphoxelBGs.zip
>>>> r684 | vehlow | 2010-07-09 09:41:41 +0200 (Fri, 09 Jul 2010) | 1 line
>>>>     M /trunk/src/resources/sphoxelBG/SphoxelBGs.zip
>>>> r638 | vehlow | 2010-05-06 13:20:15 +0200 (Thu, 06 May 2010) | 1 line
>>>>     A /trunk/src/resources/sphoxelBG
>>>>     A /trunk/src/resources/sphoxelBG/SphoxelBGs.zip
>>>> r635 | vehlow | 2010-05-04 11:40:51 +0200 (Tue, 04 May 2010) | 1 line
>>>>     D /trunk/src/resources/sphoxelBG
>>>> r634 | vehlow | 2010-05-04 10:59:11 +0200 (Tue, 04 May 2010) | 1 line
>>>>     A /trunk/src/resources/sphoxelBG/SphoxelBGs.zip
>>>> r632 | vehlow | 2010-05-03 12:37:36 +0200 (Mon, 03 May 2010) | 2 lines
>>>>     A /trunk/src/resources/sphoxelBG/sphoxelBG_A-a_A-a_rLR.csv
>>>>     ...
>>>>     A /trunk/src/resources/sphoxelBG/sphoxelBG_Y-s_Y-a_rSR.csv
>>>> r629 | vehlow | 2010-04-28 16:43:50 +0200 (Wed, 28 Apr 2010) | 1 line
>>>>     M /trunk/src/cmview/gmbp/ContactPane.java
>>>>     M /trunk/src/cmview/gmbp/ContactStatusBar.java
>>>>     M /trunk/src/cmview/gmbp/ContactView.java
>>>>     M /trunk/src/cmview/gmbp/JRangeSlider.java
>>>>     A /trunk/src/resources/sphoxelBG
>>>>
>>>> _______________________________________________
>>>> cmview-devel mailing list
>>>> cmview-devel at bioinformatics.org
>>>> http://www.bioinformatics.org/mailman/listinfo/cmview-devel
>>>
>>> _______________________________________________
>>> cmview-devel mailing list
>>> cmview-devel at bioinformatics.org
>>> http://www.bioinformatics.org/mailman/listinfo/cmview-devel
>>>
> _______________________________________________
> cmview-devel mailing list
> cmview-devel at bioinformatics.org
> http://www.bioinformatics.org/mailman/listinfo/cmview-devel