[cmview-devel] First steps to remove Sphoxel data from cmview repos

Mon Apr 4 13:42:52 EDT 2011

Hi Jose,

I finished the clean-up of cmview now. It's basically all done except
for some more testing.
Here is what I did (for future reference):

# -- on talyn --

# create a mirror of the repository
svnadmin create /project/StruPPi/svnsync/cmview
svnadmin setuuid /project/StruPPi/svnsync/cmview
11196e73-9f2b-0410-a4ce-b943a9933111
svnsync init file:///project/StruPPi/svnsync/cmview
svn://bioinformatics.org/svnroot/cmview
svnsync sync file:///project/StruPPi/svnsync/cmview

# filter out unwanted data
svnadmin dump /project/StruPPi/svnsync/cmview > cmview-svndump-r794
svndumpfilter exclude trunk/src/resources/sphoxelBG <
cmview-svndump-r794 > cmview-svndump-r794-cleaned

# -- on dallas0 --

# replacing the live version
mv /home/svn/cmview /home/svn/cmview-preclean
svnadmin create /home/svn/cmview
svnadmin load /home/svn/cmview < cmview-svndump-r794-cleaned

# -- end procedure --

The new repository takes now 18MB instead of 4.9GB. Hooray!

It seems that the config files in dallas0:/home/svn/cmview-preclean
are the default ones so I didn't copy any of them over. I tested that
I can check out from svn://bioinformatics.org/svnroot/cmview. I can
also update my old working copy without relocating.  Eventually, we
can get rid of dallas0:/home/svn/cmview-preclean which is the old
version. In any case, we still have backup dump files in
talyn:/project/StruPPi/svnsync.

Cheers,
Henning

On Mon, Apr 4, 2011 at 1:19 PM, Henning Stehr <stehr at molgen.mpg.de> wrote:
> Hi Jose,
>
> I will test it locally a bit first. Then I can send you the (cleaned
> up) dumpfile.
> Maybe you can import it to the actual repository since you own the files.
>
> I was thinking that we can just mv /svnroot/cmview /svnroot/cmview-backup
> so we can go back to the old one in case of problems. Then we can
> svnadmin create /svnroot/cmview
> and load from the dumpfile. The only thing is, the config files and
> stuff will have to be copied over from
> cmview-backup then.
>
> I'll let you know when I'm ready.
>
> Cheers,
> Henning
>
> On Mon, Apr 4, 2011 at 12:47 PM, Jose Duarte <jose.duarte at psi.ch> wrote:
>> Great, sounds like a good plan!
>>
>> Let me know if you do it, I can test it for a while to make sure everything
>> is working before we replace the repository.
>>
>> Jose
>>
>>
>> On 04/04/2011 12:27 PM, Henning Stehr wrote:
>>>
>>> Hi Jose,
>>>
>>> I started to have a look at how to remove the monster file(s) from the
>>> svn history.
>>>
>>> In principle, the procedure using 'svndumpfilter' is described here:
>>>
>>> http://svnbook.red-bean.com/en/1.5/svn.reposadmin.maint.html#svn.reposadmin.maint.filtering
>>>
>>> The command 'svndumpfilter' takes a pathname to exclude from the history.
>>>  From inspecting the history with 'svn log', it seems that it should be
>>> sufficient to exclude the path:
>>>
>>> trunk/src/resources/sphoxelBG
>>>
>>> So, the relevant commands should be:
>>>
>>> svnadmin dump REPOS_PATH>  dump-file
>>> svndumpfilter exclude trunk/src/resources/sphoxelBG<  dump-file>
>>>  new-dump-file
>>> svnadmin load REPOS_PATH<  new-dump-file
>>>
>>> I'll double check and give it a try later. I'm posting this to keep a
>>> record and to let you know what's going on.
>>>
>>> Cheers,
>>> Henning
>>>
>>> P.S. Here is the list of revisions which will be affected (if I didn't
>>> overlook anything):
>>>
>>> r718 | cvehlow | 2010-11-29 12:17:39 +0100 (Mon, 29 Nov 2010) | 1 line
>>>    D /trunk/src/resources/sphoxelBG
>>> r688 | vehlow | 2010-07-09 13:04:54 +0200 (Fri, 09 Jul 2010) | 1 line
>>>    M /trunk/src/cmview/gmbp/ContactPane.java
>>>    A /trunk/src/resources/sphoxelBG/SphoxelBGs.zip
>>> r687 | vehlow | 2010-07-09 12:34:24 +0200 (Fri, 09 Jul 2010) | 1 line
>>>    D /trunk/src/resources/sphoxelBG/SphoxelBGs.zip
>>>    D /trunk/src/resources/sphoxelBG/allSphoxelBGs.zip
>>> r686 | vehlow | 2010-07-09 12:18:42 +0200 (Fri, 09 Jul 2010) | 1 line
>>>    A /trunk/src/resources/sphoxelBG/allSphoxelBGs.zip
>>> r684 | vehlow | 2010-07-09 09:41:41 +0200 (Fri, 09 Jul 2010) | 1 line
>>>    M /trunk/src/resources/sphoxelBG/SphoxelBGs.zip
>>> r638 | vehlow | 2010-05-06 13:20:15 +0200 (Thu, 06 May 2010) | 1 line
>>>    A /trunk/src/resources/sphoxelBG
>>>    A /trunk/src/resources/sphoxelBG/SphoxelBGs.zip
>>> r635 | vehlow | 2010-05-04 11:40:51 +0200 (Tue, 04 May 2010) | 1 line
>>>    D /trunk/src/resources/sphoxelBG
>>> r634 | vehlow | 2010-05-04 10:59:11 +0200 (Tue, 04 May 2010) | 1 line
>>>    A /trunk/src/resources/sphoxelBG/SphoxelBGs.zip
>>> r632 | vehlow | 2010-05-03 12:37:36 +0200 (Mon, 03 May 2010) | 2 lines
>>>    A /trunk/src/resources/sphoxelBG/sphoxelBG_A-a_A-a_rLR.csv
>>>    ...
>>>    A /trunk/src/resources/sphoxelBG/sphoxelBG_Y-s_Y-a_rSR.csv
>>> r629 | vehlow | 2010-04-28 16:43:50 +0200 (Wed, 28 Apr 2010) | 1 line
>>>    M /trunk/src/cmview/gmbp/ContactPane.java
>>>    M /trunk/src/cmview/gmbp/ContactStatusBar.java
>>>    M /trunk/src/cmview/gmbp/ContactView.java
>>>    M /trunk/src/cmview/gmbp/JRangeSlider.java
>>>    A /trunk/src/resources/sphoxelBG
>>>
>>> _______________________________________________
>>> cmview-devel mailing list
>>> cmview-devel at bioinformatics.org
>>> http://www.bioinformatics.org/mailman/listinfo/cmview-devel
>>
>>
>> _______________________________________________
>> cmview-devel mailing list
>> cmview-devel at bioinformatics.org
>> http://www.bioinformatics.org/mailman/listinfo/cmview-devel
>>
>