Using SCP with file compression

The command-line tool SCP can be effectively used to move files and perform file compression in the transfer. However, there’s different ways to achieve this, with different results.

Here are a few axamples with different results:

To copy & compress in a single line

gzip -c test_arch.arc | ssh user@new_serv "cat > /home/oracle/backup/backup.tgz"
# or
ssh new_serv "cat /tmp/backup.sql | gzip -c1" | gunzip -c > backup.sql

Best Gzip compression & transfer

time gzip --fast -c /home/oracle/oradata/SID/FILE01.dbf | ssh -oCompression=no  oracle@new_serv  "gunzip -c > /home/oracle/FILE01.dat"

SCP with online transfer compression

This solution achieves network compression only.

scp -C new_serv:/tmp/backup.sql /path/to/backup.sql

scp -C /home/oracle/oradata/SID/file01.dbf   \
       /home/oracle/oradata/SID/file02.dbf   \
       /home/oracle/oradata/SID/file03.dbf   oracle@new_serv:/path/to/.

Written on April 20th, 2014 by Samy Gejzenblozen

Tags:

#mysql #linux

Social networks

You may also enjoy:

Last rowcount in MySQL

#mysql

Suppose you want to count the number of lines returned by the last SQL statement issued. For **select** statements you can use the *FOUND_ROWS* construct: ```sql SELECT SQL_CALC_FOUND_ROWS something FROM your_table WHERE whatever; SELECT FOUND_ROWS(); ``` This will return the number of rows in the last **select** query (or if the first query ... Read more

20 Sep 2016 - less than 1 minute read

MySQL punch user creation DDL

#mysql

You can get the DDL necessary to duplicate an existing user with the following system command: ``` MYSQL_CONN="-uroot -ppassword" mysql ${MYSQL_CONN} --skip-column-names -A -e "SELECT CONCAT('SHOW GRANTS FOR ''',user,'''@''',host,''';') FROM mysql.user WHERE user'' " | mysql ${MYSQL_CONN} --skip-column-names -A | sed 's/$/;/g' > MySQLUserGrants... Read more

20 Jul 2016 - less than 1 minute read

InnoDB table rebuild

#mysql

### PROBLEM On a **MySQL 5.6** database server, the *ibdata1* file includes 5 InnoDB tables in the mysql schema. ```sql mysql> select table_name from information_schema.tables -> where table_schema='mysql' and engine='InnoDB'; +----------------------+ | table_name | +----------------------+ | innodb_index_stats | | innodb_table... Read more

20 Jun 2016 - 1 minute read

Convert FLAC to mp3 like a hacker

#system #linux

So you need to convert a bonch of FLAC files to mp3 and your best search engine only recommands you online file converters that nobody trusts ore require a paid subscription to work. Don't worry, *there's always a way*, and even better, a true ***hacker style*** way to achieve this! This procedure works for MacOS through Brew, but you should b... Read more

20 Feb 2019 - less than 1 minute read

Blockchain and IoT

#blockchain #linux

Here's a small script to open a JavaScript interface to interact with an IoT device. This interface can then be easily incorporated in a larger Blockchain project with real tangible everyday life objects. I'll cover the Blockchain aspect of this project in another article, focusing on *Hyperledger*. The IoT device I'll be using for this project... Read more

20 Aug 2017 - 2 minute read

Image compression from the command line

#linux

Here's a small script to compress JPEG images in a folder. Useful to save a few kilobytes of bandwith when serving images from your website. I might improve this script in the future to include more compression tools and/or file extensions. ```bash #!/bin/bash for f in *.jpg *.JPG *.jpeg *.JPEG do echo -en "Converting ${f} ... " kb_or... Read more

20 Jul 2017 - less than 1 minute read