Unix Shortcuts

find . -name "*.java" -type f

find all the files within a director and its sub-directory ended with .java

 

rm *~

delete all the files ended with ~

grep setup BotConfigTODO > log

(grep the lines including setup keyword and save it in a file called log)

ps aux | grep 'less Dockerfile'| awk '{print $2}' | head -1 | xargs kill -9  

(Kill the first greped process)

docker exec -i cassandra /bin/bash -c "cat -- > InsertStackState.txt" < InsertStackState.txt

docker exec -it cassandra bash

find ./* -iname "pom.xml"

zless dash-container.log.2015-10-13_10.gz

zgrep 19363011 dash-container.log.2015-10-13_10.gz --color

docker stop service_name (restart - if service dint failed already)

u need to be in that node

if it already failed - fleetctl start service_name

head -5 science.txt

tail -5 science.txt

grep -ivc ‘keyword science’ science.txt

-v display those lines that do NOT match

-n precede each matching line with the line number

-c print only the total count of matched lines

netstat -a | grep LISTEN

Check the top 10 memory/cpu eaters

ps aux --sort=-%mem | awk 'NR<=10{print $0}'

Check the default heap size of jvm

java -XX:+PrintFlagsFinal -version | grep HeapSize

Check the access and modification time of a file

stat filename.txt 

Check the file size and sort it

du -sh * | sort -h 会把占用空间最大的文件列在最后 放在开头的话 du -sh * | sort -rh

Get the count in the file and get the sum of them

zgrep "Broadcasting EachesPickedNot" debug-dash-container.log | sed -e "s/.*quantity=([^,]*).*/1/g" | paste -sd+ | bc

paste -sd+ concatenate the number and separate them using "+"

bc caculate the number

Run commands in the background

  1. command & ：command running in the background will be stopped if you close the terminal/session

  2. nohup command & ： command will still run in the background even if you close the terminal/session

Get unique string from lines in a file

grep -o 'sysToteId.*' IMS.txt | sort -u | cut -f1 -d',' | uniq | less -S | wc -l

zgrep "Broadcasting MoveBinsToInventoryManagementNotification.*MIXED_PRODUCT_PURGING" dash-container.log.2016-05-14_*.gz | grep -o 'sysToteId.*' | sort -u | cut -f1 -d',' | uniq | less -S | wc -l

Or

zgrep "Broadcasting MoveBinsToInventoryManagementNotification.*MIXED_PRODUCT_PURGING" dash-container.log.2016-05-14_*.gz | grep -o 'sysToteId.*,' | grep -o ^[^,]* | uniq | less -S

Check whether disk is full

#!/bin/bash

disk_usage=$(df -h | grep "sda5" | awk '{split($5,p,"%"); print p[1]}')

if [ "$disk_usage" -gt 90 ]; then

   echo -e "Disk is full, usage is e[1;31m$disk_usage%e[0m"

   echo "Disk is full, usage is $disk_usage%" | mailx -r "chi.ronchy.zhang@gmail.com" -s "SUBJECT" "chi.zhang@ocado.com"

Fi

Onsite Version

checkDiskUsage.sh is

#!/bin/bash

export PROFILE=andoverambientCR1

disk_usage=$(df -h | grep "vg-root" | awk '{split($5,p,"%"); print p[1]}')

if [ "$disk_usage" -gt 85 ]; then

   echo -e "$PROFILE Disk is full, usage is e[1;31m$disk_usage%e[0m"

   echo "$PROFILE Disk is full, usage is $disk_usage%" | mailx -s "$PROFILE Disk is full, usage is $disk_usage%" "dash_container@ocado.com"

Fi

1. Upload the script to the ambient box

2. crontab -e

3. add

0 * * * * /app/checkDiskUsage.sh > /dev/null 2>&1

Improved Version



#!/bin/bash

export PROFILE=andoverchillCR1

root_disk_usage=$(df -h | grep "vg-root" | awk '{split($5,p,"%"); print p[1] }')

data_disk_usage=$(df -h | grep "vg-data" | awk '{split($5,p,"%"); print p[1] }')

email_message="$PROFILE Disk is full, root partition usage is $root_disk_usage% and data partition usage is $data_disk_usage%"

if [ "$data_disk_usage" -gt 85 ] || [ "$root_disk_usage" -gt 85 ]; then

   echo "$email_message" | mailx -s "$email_message" dash_container@ocado.com,dash_controller@ocado.com

fi

sed

http://www.grymoire.com/Unix/Sed.html#TOC

s Substitute command

A simple example is changing "day" in the "old" file to "night" in the "new" file:

sed s/day/night/ <old >new

Or another way (for UNIX beginners),

sed s/day/night/ old >new

The character after the s is the delimiter. It is conventionally a slash, because this is what ed, more, and vi use. It can be anything you want

The escaped parentheses (that is, parentheses with backslashes before them) remember a substring of the characters matched by the regular expression. You can use this to exclude part of the characters matched by the regular expression. The "1" is the first remembered pattern, and the "2" is the second remembered pattern. Sed has up to nine remembered patterns.

echo abcd123 | sed 's/([a-z]*).*/1/'

This will output "abcd" and delete the numbers.

If you want it to make changes for every word, add a "g" after the last delimiter and use the work-around:

sed 's/[^ ][^ ]*/(&)/g' <old >new

With no flags, the first matched substitution is changed. With the "g" option, all matches are changed. If you want to modify a particular pattern that is not the first one on the line, you could use "(" and ")" to mark each pattern, and use "1" to put the first pattern back unchanged. This next example keeps the first word on the line but deletes the second:

sed 's/([a-zA-Z]*) ([a-zA-Z]*) /1 /' <old >new

Yuck. There is an easier way to do this. You can add a number after the substitution command to indicate you only want to match that particular pattern. Example:

sed 's/[a-zA-Z]* //2' <old >new

You can combine a number with the g (global) flag. For instance, if you want to leave the first word alone, but change the second, third, etc. to be DELETED instead, use /2g:

sed 's/[a-zA-Z]* /DELETED /2g' <old >new

There is one more flag that can follow the third delimiter. With it, you can specify a file that will receive the modified data. An example is the following, which will write all lines that start with an even number, followed by a space, to the file even:

sed -n 's/^[0-9]*[02468] /&/w even' <file

This flag makes the pattern match case insensitive. This will match abc, aBc, ABC, AbC, etc.:

sed -n '/abc/I p' <old >new

p is the print command

If you need to make two changes, and you didn't want to read the manual, you could pipe together multiple sed commands:

sed 's/BEGIN/begin/' <old | sed 's/END/end/' >new

This used two processes instead of one. A sed guru never uses two processes when one can do.

One method of combining multiple commands is to use a -e before each command:

sed -e 's/a/A/' -e 's/b/B/' <old >new

If you have many commands and they won't fit neatly on one line, you can break up the line using a backslash:

sed -e 's/a/A/g'
   -e 's/e/E/g'
   -e 's/i/I/g'
   -e 's/o/O/g'
   -e 's/u/U/g'  <old >new