How to count total number of lines of found files. Bash - "while read LINE do" and grep problems. String - Extract filename and extension in bash. GNU parallel. GNU parallel is a command-line driven utility for Linux or other Unix-like operating systems which allows the user to execute shell scripts in parallel. GNU parallel is free software, written by Ole Tange in Perl. It is available under the terms of GPLv3.[2] Usage[edit] The most common usage is to replace the shell loop, for example for x in `cat list` ; do do_something "$x" done | process_output to the form of cat list | parallel do_something | process_output where the file list contains arguments for do_something and where process_output may be empty.
Scripts using parallel are often easier to read than scripts using pexec. The program parallel features also grouping of standard output and standard error so the output of the parallel running jobs do not run together;retaining the order of output to remain the same order as input;dealing nicely with file names containing special characters such as space, single quote, double quote, ampersand, and UTF-8 encoded characters; Examples[edit] find . Amazon S3 tools: s3cmd : command line S3 client. S3cmd : Command Line S3 Client and Backup for Linux and Mac Amazon S3 is a reasonably priced data storage service. Ideal for off-site file backups, file archiving, web hosting and other data storage needs. It is generally more reliable than your regular web hosting for storing your files and images. Check out about Amazon S3 to find out more. S3cmd is a free command line tool and client for uploading, retrieving and managing data in Amazon S3 and other cloud storage service providers that use the S3 protocol, such as Google Cloud Storage or DreamHost DreamObjects.
It is best suited for power users who are familiar with command line programs. It is also ideal for batch scripts and automated backup to S3, triggered from cron, etc. S3cmd is written in Python. S3cmd System Requirements: S3cmd requires Python 2.6 or newer. S3Express : Command Line S3 Client and S3 Backup for Windows S3Express is a commercial S3 command line tool for Windows. Www.catonmat.net/download/bash-redirections-cheat-sheet.pdf. Linux: Show the number of CPU cores and sockets on your system (Brian Smith's AIX / UNIX / Linux / Open Source blog) Finding the number of CPU cores on a Linux server can be challenging. The way /proc/cpuinfo displays information makes it hard to distinguish between real CPU cores and logical hyperthreading CPU's.
For example, consider if I run the following command on my Linux laptop: $ cat /proc/cpuinfo | grep processorprocessor : 0processor : 1processor : 2processor : 3 At first glance it appears that my laptop has 4 CPU cores in it. However, it really has 2 CPU cores with hyperthreading enabled. The hyperthreading presents 2 logical CPU's to the operating system for each actual core so it effectively doubles the number of logical CPU's your system will see in /proc/cpuinfo. The fields under /proc/cpuinfo that you need to compare to find the number of cores are "physical id" and "cored id"
. $ cat /proc/cpuinfo | grep "physical id" | sort | uniq | wc -l1 In this example the computer only has 1 physical socket being used. $ cat /proc/cpuinfo | grep "physical id" | sort | uniq | wc -l1.