Reduce Subversion pristine copy: Porovnání verzí

Z ZděchovNET
Skočit na navigaci Skočit na vyhledávání
Bez shrnutí editace
Bez shrnutí editace
Řádek 1: Řádek 1:
Subversion VCS stores copy of each file in pristine directory (formerly called textbase) located in hidden .svn directory in working copy root directory. Storing copy for every file means that working copy takes twice as much size as stored files. This may not be problem for small repository but if you want to store for example 30GB of photos in repository and want to checkout working copy on machine with 60 GB drive then you have a problem.
Subversion VCS stores copy of each file in pristine directory (formerly called textbase) located in hidden .svn directory in working copy root directory. Storing copy for every file means that working copy takes twice as much size as user stored files alone. This may not be problem for small repository but if you want to store for example 30 GB of photos, music, video or other big binary files in repository and want to checkout working copy on machine with just 60 GB or smaller drive then you have a problem. Also during checkout twice as content size is written to disk which lead to higher disk load.


Also this slows down every checkout as two copies of downloaded files needed to be created on filesystem instead one.
Also this slows down every checkout as two copies of downloaded files needed to be created on filesystem instead one.
Řádek 14: Řádek 14:
==File deduplication of working copy==
==File deduplication of working copy==


It could possibly work on filesystem level with existing filesystems or as virtual filesystem. But file deduplication on system level need more CPU power and more memory. There are also some copy-on-write file systems which could be usable if subversion client would make copy of file by using system service function for copying.
It could possibly work on file system level with existing file systems or as virtual file system. But file deduplication on system level need more CPU power and more memory. There are also some copy-on-write file systems which could be usable if subversion client would make copy of file by using system service function for copying. Such file system is for example [https://btrfs.wiki.kernel.org/index.php/Main_Page Btrfs]


One example could be [http://scord.sourceforge.net/ Scord] which solves this for versions <= 1.6 on Linux platform.
One example could be [http://scord.sourceforge.net/ Scord] which solves this for versions <= 1.6 on Linux platform.

Theoretically file system with deduplication support could be used. This could be realized under Linux as file mounted as loop device. [https://en.wikipedia.org/wiki/ZFS#Deduplication ZFS] file system support deduplication but it is not part of Linux kernel.


==Make pristine files zero size by helper program==
==Make pristine files zero size by helper program==


It maybe possible to temporary make pristine file empty and update their content manually by invoking some helper program. Such program could check svn status and download correct content of modified files from repository to be able to commit them. Set zero size to other pristine files. The program should be able to work with svn sqlite database to read file name to pristine name relation. Also it should be multi-platform.
It maybe possible to temporary make pristine file empty and update their content manually by invoking some helper program. Such program could check svn status and download correct content of modified files from repository to be able to commit them. Set zero size to other pristine files. The program should be able to work with SVN sqlite database to read file name to pristine name relation. Also it should be multi-platform.


<source lang="bash">
<source lang="bash">
Řádek 34: Řádek 36:
=Alternative VCS=
=Alternative VCS=


Unfortunately there not many other VCS systems which can download just single copy of files in checkout.
Unfortunately there are not many other VCS systems which can download just single copy of files during checkout.
* CVS - old predecessor of Subversion
* [http://www.nongnu.org/cvs/ CVS] - old predecessor of Subversion
* Bazzar - can work in client-server mode and checkout just single copy of files. Not developed anymore.
* [http://bazaar.canonical.com Bazaar] - can work in client-server mode and checkout just single copy of files. Not developed anymore.


=External links=
=External links=

Verze z 25. 10. 2015, 17:29

Subversion VCS stores copy of each file in pristine directory (formerly called textbase) located in hidden .svn directory in working copy root directory. Storing copy for every file means that working copy takes twice as much size as user stored files alone. This may not be problem for small repository but if you want to store for example 30 GB of photos, music, video or other big binary files in repository and want to checkout working copy on machine with just 60 GB or smaller drive then you have a problem. Also during checkout twice as content size is written to disk which lead to higher disk load.

Also this slows down every checkout as two copies of downloaded files needed to be created on filesystem instead one.

Possible methods of reducing pristine size

Subversion client built-in mechanism

The feature could be implemented directly to the client in some later 1.9+ version.

There is already opened Issue 525 named "allow working copies without text-base/". But the authors want to stay with original subversion principle that storage is cheap and network is slow so they want to keep copy of files.

File deduplication of working copy

It could possibly work on file system level with existing file systems or as virtual file system. But file deduplication on system level need more CPU power and more memory. There are also some copy-on-write file systems which could be usable if subversion client would make copy of file by using system service function for copying. Such file system is for example Btrfs

One example could be Scord which solves this for versions <= 1.6 on Linux platform.

Theoretically file system with deduplication support could be used. This could be realized under Linux as file mounted as loop device. ZFS file system support deduplication but it is not part of Linux kernel.

Make pristine files zero size by helper program

It maybe possible to temporary make pristine file empty and update their content manually by invoking some helper program. Such program could check svn status and download correct content of modified files from repository to be able to commit them. Set zero size to other pristine files. The program should be able to work with SVN sqlite database to read file name to pristine name relation. Also it should be multi-platform.

#!/bin/bash

for F in $(find ./.svn/pristine -type f -size +0);
do
cat /dev/null >$F
done

If pristine files are set to empty then all svn operation could be used except these related to file content as svn diff or svn commit of modified files. To be able also work with file diff, pristine file content should be restored from repository somehow.

Alternative VCS

Unfortunately there are not many other VCS systems which can download just single copy of files during checkout.

  • CVS - old predecessor of Subversion
  • Bazaar - can work in client-server mode and checkout just single copy of files. Not developed anymore.

External links