Tag Archives: Linux - Page 3

Using rsync to download large files over ssh with resume option

In the last months I have been doing this a lot so it might be best to put it up here so I don’t have to google it every time. There is a lot of cool features to rsync. The one we will concentrate on here is the –partial option that lets us resume a broken download. Have to start over with the download of a large file can be a real pain

Basic syntax

rsync --partial --progress --rsh=ssh user@host:remote_file local_file

I also use the –progress option to output the progress of the download

Example:

rsync --partial --progress --rsh=ssh niklas@niklasottosson.com:.vimrc .

This will download the file ‘.vimrc’ from my home directory on niklasottosson.com and into the folder I’m currently in. If the download is interrupted rsync will keep the partial downloaded file and resume from it when the downloaded is started again

Tested on OSX 10.7.4 and rsync version 3.0.3

Open multiple files in individual tabs in vim using script

I have always wanted one program to open to access all important (for me) files in a system. This can for instance be http.conf, passwd, groups and so on. Using tabs in vim, the -p option (from version 7 and above) and a small script can make vim into a nice little admin console:

I call my script ‘configs’ and it looks like this (put it in your PATH for easy access):

#!/bin/bash
sudo vim -p /etc/hosts /private/etc/php.ini /etc/httpd.conf

Too move between tabs in command mode:

gt - next tab
gT - previous tab

Too move between tabs in insert mode (and also in command mode):

CTRL + PgUp - next tab
CTRL + PgDown - previous tab  

Remove files in batches of x files

Every now and then I try to do things on a large amount of files – this often ends in a command error about “Too many files” or something similar. One solution to this is to send batches of information to the command instead of sending all at once.

Here is an example on how to delete files in batches of 1000 files:

ls | xargs -n 1000 rm -v

The trick here is the -n 1000 that will send the result from the ls command in batches of a 1000 results per time until the ls list is empty to rm. Neat ah? 🙂

Tested on RHEL 3