Pacho Ramos

One more way to get full Portage tree in less than 300MB

Friday, January 15, 2010 12:13:51 PM

In some places like forums, there are a lot of different ways to shrink portage tree. The way I finally chose was to simply create a separate partition using ReiserFS and, of course, not using "notail" option with it ;-)

This is how does its fstab entry look:
/dev/sda6 /usr/portage reiserfs defaults,noatime,nolog 0 0

And this is df -h output for it:
/dev/sda6 353M 292M 62M 83% /usr/portage

It includes full portage tree+sunrise+suka overlays

Best regards

Rectificando"Vi doparse en muchas ocasiones a Valverde"

Comments

Anonymous # Friday, January 15, 2010 1:32:09 PM

Anonymous writes: Pfft... I give you Btrfs with compression! # df -hl /usr/portage/tree/ Filesystem Size Used Avail Use% Mounted on /dev/sda6 495M 217M 278M 44% /usr/portage/tree # layman -l * suka [Subversion] (svn://overlays.gentoo.org/dev/suka ) * sunrise [Subversion] (svn://overlays.gentoo.org/proj/sunrise/reviewed/ ) # grep portage /etc/fstab /dev/sda6 /usr/portage/tree btrfs noatime,compress,nodatasum 0 0

Anonymous # Friday, January 15, 2010 4:46:16 PM

Miguel Tavares writes: With some tuned ext4 partition: /dev/hda5 1.8G 287M 1.4G 17% /usr/portage The tune was done during creation. I think it was: mkfs.ext4 -b 1024 -i 1024 -L PORTAGE -I 128 -O dir_index,extent,uninit_bg /dev/hda5

Anonymous # Friday, January 15, 2010 6:46:09 PM

Anonymous writes: You could save even more space by dropping ChangeLogs: PORTAGE_RSYNC_EXTRA_OPTS="--exclude ChangeLog --delete-excluded" Depends on how often you actually use them, of course ;)

Anonymous # Friday, January 15, 2010 8:29:54 PM

Phlogi writes: I'm using portage from a squashfs file since years :) /dev/loop0 45M 45M 0 100% /usr/portage I'm using a script to sync it (aufs on ramdisk [very fast]) need a link? =)

Anonymous # Friday, January 15, 2010 8:39:32 PM

code writes: @Phlogi I am using an aufs+ramdisk to sync the squashfs file too, you can post your script so i can try to improve mine? My script: http://pastebin.com/f1f56bb1

Anonymous # Friday, January 15, 2010 8:44:38 PM

Phlogi writes: http://paste.pocoo.org/show/165828/ Here you go... maybe we should create a project for this? I got that from forums http://forums.gentoo.org/viewtopic-t-401647-start-75-postdays-0-postorder-asc-highlight-.html works fine for me

Anonymous # Saturday, January 16, 2010 11:52:39 AM

mv writes: Probably one of the most flexible (at least highly customizable) initscripts for squashfs + aufs2|aufs|unionfs|unionfs-fuse|funionfs (support for unionfs-fuse is brand new) can be found on http://www.mathematik.uni-wuerzburg.de/~vaeth/gentoo/index.html The corresponding discussion in the forum (originally based on a different script) is http://forums.gentoo.org/viewtopic-t-465367.html

Pacho Ramospacho # Saturday, January 16, 2010 1:07:23 PM

Thanks a lot for your suggestions:
1. I am using this since some months, that time, I also checked how much time did a cold (I mean just after rebooting) "emerge -pvuDN world" take with ReiserFS, Ext-{2,3,4}, XFS and JFS (with kernel 2.6.29 or 30 I think)

Finally, reiserfs was the faster (for me at least) followed by ext2, but maybe things changed with ext4 since then (that time I also had some corruptions problems with ext4 on / that caused me to go back to reiserfs).

How fast does emerge perform with your different setups?

2. About squashfs, I read about it but finally I was too lazy to try it since simply having a new partition and mounting it from fstab seemed easier for me, but probably I will have to try it also if I have enough time ;-)

3. About btrfs, I haven't tried it yet since it is "experimental" :-/, but probably I will look to it in the future

4. About changelogs, it's an interesting hint but, in my case, I read them a lot ;-) (a way for compressing them with bzip2, for example, would be much better for me)

Best regards

Anonymous # Saturday, January 16, 2010 5:19:37 PM

mv writes: > squashfs: ... having a new partition [...] seemed easier for me If you have fuse support enabled in your kernel, it essentially only comes down to emerging squashfs-tools and unionfs-fuse, calling mksquashfs /usr/portage /usr/portage.sqfs and configuring the initscripts; e.g. with the script I mentioned, you need (after copying the script to /etc/init.d) something like ln -s squash_dir /etc/init.d/squash_portage and for playing around I would recommend in /etc/conf.d/squash_portage: DIRECTORY=/usr/portage DIR_CHANGE=/usr/portage.changes DIR_SQUASH=/usr/portage.readonly ORDER=unionfs-fuse (later you might want to add THRESHOLD=40000). Much easier than setting up a whole new partition... However, a main advantage of squashfs + aufs is the speed gain which may be less when you use funionfs-fuse instead of aufs; therefore, you might think about patching your kernel with aufs2 support anyway (do not forget to fix/remove ORDER if you do).

Anonymous # Saturday, January 16, 2010 9:55:57 PM

Anónimo writes: Note that besides compression, Btrfs also packs small files

Anonymous # Sunday, January 17, 2010 1:41:57 PM

js writes: I followed the short description of mv and used his script with squashfs and unionfs-fuse on my AMD64 platform (had to add the amd64 keyword to unionfs-fuse) here my results: portage-tree with Changelog: ~45MB without Changelog: ~32MB

Anonymous # Tuesday, January 19, 2010 6:36:29 AM

Lars writes: I use /usr/portage tree in a loop with reiserfs for years. Most the time I get fresh portage snapshots and unpack this to the loop device. My experience is if reiserfs is near full ~>92% it will slow down dramatically. JM2C Lars