Cheat sheet on Linux for users

Home

1 What Linux am I running?

You can issue uname -a to see what version of the Linux kernel you are running To see what distribution depends on the distribution, so chicken and egg issue Best to try each of them. uname -a or also uname -or

1.1 hostnamectl

This is my favourite way, and seems to be available on many different distros hostnamectl You can also change your name with hostnamectl set-hostname

1.2 If availabe try:

cat /etc/os-release

1.3 Redhat, CentOS, Fedora

cat /etc/redhat-release
cat /etc/centos-release
cat /etc/fedora-release

1.4 Debian

cat /etc/debian_version

1.5 Ubuntu

cat /etc/lsb-release

1.6 Gentoo

cat /etc/gentoo_release

2 Customizing your shell

2.1 .bashrc vs .bash_profile

From mitre.org, ~/.bash_profile and ~/.bashrc are shell scripts that contain shell commands. These files are executed in a user's context when a new shell opens or when a user logs in so that their environment is set correctly. ~/.bash_profile is executed for login shells and ~/.bashrc is executed for interactive non-login shells.

This means that when a user logs in (via username and password) to the console (either locally or remotely via something like SSH), the ~/.bash_profile script is executed before the initial command prompt is returned to the user.

Subsequently, every time a new shell is opened, the ~/.bashrc script is executed. This allows users more fine-grained control over when they want certain commands executed. These shell scripts are meant to be written to by the local user to configure their own environment.

The macOS Terminal.app is a little different in that it runs a login shell by default each time a new terminal window is opened, thus calling ~/.bash_profile each time instead of ~/.bashrc.

2.1.1 .bashrc runs each time a new terminal is opened

2.1.2 .bash_profile runs only when a user logs in

2.2 .bashrc

You can edit this in your root directory and add after which you save the changes, exit, then run source .bashrc to re-read and initialize the new .bashrc

2.3 .bashrc aliases

alias [name[='value']]

No spaces allowed next the = sign. If value has spaces, it must be enclosed in "" marks.

alias[name[=value]]

alias lst='ls -lart'
alias font22='setfont LatGrkCyr-12x22.psfu.gz' 
alias font16='setfont LatGrkCyr-8x16.psfu.gz' 
alias ip4='nmcli -f ip4 device show'
alias ip4='nmcli -f ip4 device show'
alias skip='sed "/^$/d ; /^ *#/d"'   # Strip blanks and comments from files.
  • ls -S is sort by size, but I would not make that an alias.

2.3.1 Bypass the alias with a \

If ls is set to an alias like ls -la \ls will actually get you an ls output, and NOT the ls -la output. You are basically temporarily telling bash to ignore the alias setting.

2.3.2 single or double quotation marks

The choice of single or double quotation marks is significant in the alias syntax when the alias includes variables.

If you enclose value within double quotation marks, any variables that appear in value are expanded when the alias is created.

If you enclose value within single quotation marks, variables are not expanded until the alias is used.

The following example illustrates the difference.

echo $PWD
/home/max
alias dirA="echo Working directory is $PWD"
alias dirA
alias dirA='echo Working directory is /home/max'

alias dirB='echo Working directory is $PWD'
alias dirB
alias dirB='echo Working directory is $PWD'

2.3.3 unalias

To remove the alias for this session.

2.3.4 aliases that accept parameters

You will probably have the need at some point to have an alias accept one or more parameters. You cannot. Aliases do not accept any parameters from the command line.

The fix is the create a bash function as descibed in Bash functions.

For example, to pass one or more arguments to the function, just call the function with the arguments separated by spaces. For example:

mult 55 2.54 on the command line will multiply the two numbers.

While creating/writing the function, the arguments are assigned to variables $1, $2, $3, $4 etc…

So my mult function would look like this:

mult()
{
  echo "$1" * "$2"
}

A good approach is to create a script called functions.sh ( in the ~/bin directory for example). In this script, you add all your personal function definitions, and every time you add a function you add it to this file,

Finally you just have to add the source ~/bin/functions.sh line to your .bashrc file. This way you will be able to call them from the command line, your .bashrc will stay clean, and you will have a specific place for your personal functions.

2.4 Customizing the prompt

export PS1="Monty > "   # only display Monty >
export PS1="\u "        # only display the username
export PS1="\u@\h "      # display the username and host
export PS1="\u@\H "      # display the username and FQDN
export PS1="\u@\h\\$ "   # display the username@host $
export PS1="\u@\h\ \s \\$ "         # display the username@host >bash $
export PS1="\u@\h\ \s\v \\$ "         # display the username@host >bash4.4 $
export PS1="\w \\$"      # display only the current path and $
export PS1="\W"         # will remove the first / in the path displayed
export PS1="[\!] $"     # show the history line number i.e. [15] $

The export command sets the value of the TERM variable to xterm-color. In addition, xterm-color is a terminal type that supports colorization. We can also interpret the other parts of the script:

PS1: defines the shell prompt \u: represents the username \h: represents the hostname \w: represents the current working directory \[\e[31m\]: changes the text to red \[\e[0m\]: sets the color to default

2.4.1 Adding colour to the prompt

see LSCOLORS in this doc and LSCOLORS.org file. This section is on colouring the prompt itself. I believe this applies to the ls command.?

Good info available here howtogeek.com of which I have extracted some paragraphs:

You must include the entire color code information between the \[ and \] characters. Inside the tag, you must begin with either \033[ or \e[ to indicate to Bash that this is color information.

Both \033[ and \e[ do the same thing. \e[ is shorter so might be more convenient to use, but we’ll use \033[ here as it matches what’s used by default.

You must indicate the end of a color tag with m\

So, every color tag will look like: \[\033[COLORm\] where COLOR is:

  • 30: Black
  • 31: Red
  • 32: Green
  • 33: Yellow
  • 34: Blue
  • 35: Purple
  • 36: Cyan
  • 37: White

So:

\[\033[30m\] : Black
\[\033[31m\] : Red
\[\033[32m\] : Green
\[\033[33m\] : Yellow
\[\033[34m\] : Blue
\[\033[35m\] : Purple
\[\033[36m\] : Cyan
\[\033[37m\] : White

Specify color code 00 to clear the color information. i.e. \[\033[00m\]

Bash also allows you to change the color of foreground text, add attributes like “bold” or “underline” to the text, and set a background color.

Attributes are added before the color number So, every attribute color tag will look like:

  • \[\033[ATTRIBUTE;COLORm\]

Where ATTRIBUTE is:

  • 0 Normal text ( the default )
  • 1 Bold or light text
  • 2 Dim text
  • 3
  • 4 Underlined text
  • 5 Blinking text
  • 7 Reversed text
  • 8 Hidden text.

Background colours cannot have attributes, but are:

  • 40 Black
  • 41 Red
  • 42 Green
  • 43 Yellow
  • 44 Blue
  • 45 Purple
  • 46 Cyan
  • 47 White

So just changing the background colour to purple would be: \[\033[44m\]

You can specify both foreground and background color tags. For example, 42 represents a green background and 31 represents red text. So, to make the default prompt become red text on a green background, you’d use:

  • PS1="\[\033[42m\]\[\033[31m\]\u@\h:\w\$ "

My current favourites for bash: export PS1="\[\033[01;31m\]\u@\h \w\[\033[01;32m\]$ \[\033[00m\]" or flip it as :

Yellow: 33 For example, since purple text is color code 32, you’d use \[\033[32m\] for purple text.

My current favourites for zsh: export PS1="\[\033[01;33m\]\$PWD\[\`if [[ \$? = "0" ]]; then echo '\e[32m[\!]\e[0m'; else echo '\e[31m[\!]\e[0m' ; fi\`:\n\$ " which will colour the history count, [\!] , red if your previous command returned an error.

  1. Colour attributes

    You can also specify an attribute for the text. This attribute must be added before the color number, separated by a semicolon ; Text with these attributes will look different in different terminal emulators.

    Here are the values for text attributes:

    Normal Text: 0
    Bold or Light Text: 1 (It depends on the terminal emulator.)
    Dim Text: 2
    Underlined Text: 4
    Blinking Text: 5 (This does not work in most terminal emulators.)
    Reversed Text: 7 (This inverts the foreground and background colors, so
    you’ll see black text on a white background if the current text is white
    text on a black background.)
    
    Hidden Text: 8
    

    You don’t actually need to include the normal text attribute. That’s the default, anyway.

    For example, since red text is code 31 and bold text is code 1, you’d use \[\033[1;31m\] for bold red text.

    You can also specify a background color, but you can’t add an attribute to a background color.

  2. Background colours

    Here are the values for background colors:

    Black background: 40
    Blue background: 44
    Cyan background: 46
    Green background: 42
    Purple background: 45
    Red background: 41
    White background: 47
    Yellow background: 43
    

    For example, since a blue background is code 44, \[\033[44m\] would specify a blue background.

  3. Specifying both

    You can specify both foreground and background color tags. For example,

    42 represents a green background and 31 represents red text. So, to make
    

    the default prompt become red text on a green background, you’d use:

    PS1="\[\033[42m\]\[\033[31m\]\u@\h:\w\$ "
    

    We just specify a single background color and then a single foreground text color here, which begins at the start of the prompt and is applied to all text in the prompt. However, you can specify as many color tags as you want in the variable to color different sections of your prompt however you like.

  4. add 00 to reset after the prompt

    The background and foreground text colors keep going past the prompt unless you specify color code 00 or more precisely \[\033[00m\] to clear the color information. You can also use this tag within the variable to reset formatting back to default somewhere in your prompt. For example, the following line would end all coloring before the \$ character.

    PS1="\[\033[42m\]\[\033[31m\]\u@\h:\w\\[\033[00m\]\$ "
    

2.5 .cshrc

You can edit this in your root directory and add after which you save the changes, exit, then run source .cshrc to re-read and initialize the new .cshrc

2.6 Precedence on Colors

There are multiple ways to set colors: LSCOLORS env variable LSCOLORS env variable

In zsh using:

PROMPT="%B%{$fg[yellow]%}%d%(?:%{$fgbold[green]%}[%h]:%{$fgbold[red]%}[%h]) $%{$resetcolor"

In bash shell using ANSI escape codes \e or \033

2.7 focus follows mouse

This is controlled by gnome settings. Per session settings are: gsettings set org.gnome.desktop.wm.preferences focus-mode 'mouse'

So save this permanently? Apparently the gsettings above IS permanently saved; so no need to do anything else.

3 BASH commands

3.1 uptime

Two very useful commands related to uptime are last and who -b So together they are:

  • uptime shows time since last reboot. Also uptime -s
  • last will show you all the users that have logged in and when as well as when and how many times the system was rebooted. Can also use last reboot
  • last -x will also show you reboots and other system level changes.
  • who -b just shows the last time the system was rebooted.

3.2 $ Kill -l

This lists the kill signals used by the system. May be slightly different from distro to distro, or linux to mac OSX, BSD etc.

3.3 killing a stuck tty session

from another tty, issue the command who Note the tty session, e.g. pts/0 (psuedo terminal ssh 0 )

ps -ef | grep pts to see the associated process

kill processid # of the tty process.

To kill all processess that contain "brave" killall brave

3.4 tree

other tools similar to tree are broot

3.5 cut

cut -d" " -f 1 # means use a space as the delimiter and return field 1. cut -d| -f 3-4 # means use a "|" as the delimiter and return fields 3 & 4. cut -f 2,4-6 # means use a tab as the delimiter and return field 2, 4 to 6 cut -c 2 # means use a tab as the delimiter and return field 2.

3.6 rename

rename files can rename files, replacing all occurances of strng1 with strng2. for example renamin all occurances of "ciscospark" with "webexapi" you could

  • rename 's/ciscospark/webexapi/' *.py should work on linux, but not macosx
  • sed for macosx should do.
for f in ./*.*; do
       cp $f ciscospark-$f;   # this did not work, but is a naive way to try.
done


for file in ACDC*.xxx; do
    mv "$file" "${file//ACDC/AC-DC}"
done

for file in webex-*.py; do
    mv "$file" "${file//webex/ciscospark}"
done

# including subdirectories
find . -type f -name webex\*.py -print0 | while read -r -d '' file; do
    mv "$file" "${file//webex/ciscospark}"
done

find . -type f -name "ACDC*" -print0 | while read -d $'\0' f
do
   new=`echo "$f" | sed -e "s/webex/ciscospark/"`
   mv "$f" "$new"
done

4 Logical Operators, &&, ||

  • x && y ==> =y only if x true i.e x had to complete successfully before you do y
  • x || y ==> y only if x false i.e only do y if x failed

These can often let you avoid using cubmersome if statements in your bash scripts.

Below are details for each type. Remember that $? is a shell variable that holds the error condition from the last executed command. 0 if it worked, 1 if it failed. I've seen 0 if it worked and 127 if not, so be careful and check.

4.1 and operator && only execute the 2nd part if the first PASSED

ansible@c8host ~[1174] $
true && echo "ok it worked"
ok it worked

ansible@c8host ~[1175] $
false && echo "ok it worked"

<nothing was echoed here, but that is hard to show, hence this explanation>
ansible@c8host ~[1176] $

That was taken from the MIT missing lecture series on youtube

This is the AND operator &&:

The second command will ONLY run if the first command succeeded. So, here is a better example, good for handling ugly errors:

cat ".bashrc" ;  echo "this is my profile"
cat ".bashrc" && echo "this is my profile"
  • The first choice above will fail with an ugly error if there is no file called .bashrc.
  • The second choice will print "this is my profile" only if there really was a profile. Much better.

A common use of this is to set the bash prompt different whenever the last command returned an error:

\[\033[30m\] : Black
\[\033[31m\] : Red
\[\033[32m\] : Green
\[\033[33m\] : Yellow
\[\033[34m\] : Blue
\[\033[35m\] : Purple
\[\033[36m\] : Cyan
\[\033[37m\] : White
 # loops:  export PS1='$(if [ $? -eq 0 ]; then printf "\033[01;32m""\xE2\x9C\x93"; else printf "\033[01;31m""\xE2\x9C\x95"; fi) \[\e[00;32m\]\u@\h\[\e[00;30m\]:\[\e[01;33m\]\w\[\e[01;37m\]\$ '

  export PS1='... \e[01;31m$(code=${?##0};echo ${code:+[error: ${code}]})\e[00m'

 # on Linux
  export PS1="\$([ \$? == 0 ] && echo ✅ || echo ⚠️ ) \[\033[01;36m\]\u\[\033[01;32m\]@\h\w[\!] \[\033[01;31m\]$\[\033[00m\]\n"
  export PS1="\[\033[01;36m\]\u\[\033[01;32m\]@\h\w[\!] \[\033[01;31m\]$\[\033[00m\]\$([ \$? == 0 ] && echo ✅ || echo ⚠️ )\n"
  export PS1="\[\033[01;36m\]\u\[\033[01;34m\]@\h\w[\!]\$([ \$? == 0 ] && echo ✅ || echo ⚠️ ) \[\033[01;31m\]$\[\033[00m\]\n"
  export PS1="\[\033[01;36m\]\u\[\033[01;34m\]@\h\w[\!]\$([ \$? == 0 ] && echo ✅ || echo '\[\033[01;31m\]⚠️ ') \[\033[01;31m\]$\[\033[00m\]\n"
  export PS1="\[\033[01;36m\]\u\[\033[01;34m\]@\h\w[\!]\$([ \$? == 0 ] && echo ✅ || echo '\[\033[01;31m\]⚠️ ') \[\033[01;31m\]$\[\033[00m\]\n"
  export PS1="\[\033[01;36m\]\u\[\033[01;34m\]@\h\w[\!]\$([ \$? == 0 ] && echo ✅ || echo '\[\033[01;31m\]⚠️ ') \[\033[01;31m\]$\[\033[00m\]\n"
  export PS1="\[\033[01;36m\]\u\[\033[01;34m\]@\h\w[\!]\$([ \$? == 0 ] && echo \[\033[01;36m\] ✅ || echo ⚠️ ) \[\033[01;31m\]$\[\033[00m\]\n"
  export PS1="\[\033[01;36m\]\u\[\033[01;33m\]@\h\w[\!]\$([ \$? == 0 ] && echo '\[\033[01;32m\]✅' || echo '\[\033[01;31m\]⚠️ ') \[\033[01;31m\]$\[\033[00m\]\n"
  export PS1="\[\033[01;36m\]\u\[\033[01;33m\]@\h\w[\!]\$([ \$? == 0 ] && echo '\[\033[01;32m\]✅' || echo '\[\033[01;31m\]⚠️ ') \[\033[01;31m\]$\[\033[00m\]\n"
  export PS1="\[\033[01;36m\]\u\[\033[01;33m\]@\h\w\$([ \$? == 0 ] && echo '\[\033[01;32m\][\!] ✅' || echo '\[\033[01;31m\][\!] ⚠️ ')\[\033[00m\]\n"
  export PS1="\[\033[01;36m\]\u\[\033[01;33m\]@\h\w\$([ \$? == 0 ] && echo '\[\033[01;32m\][\!] ✅' || echo '\[\033[01;31m\][\!] ⚠️ ')\[\033[00m\]\n"
  export PS1="\[\033[01;36m\]\u\[\033[01;33m\]@\h\w\$([ \$? == 0 ] && echo '\[\033[01;32m\][\!] ✅' || echo '\[\033[01;31m\][\!] ⚠️ ')\[\033[00m\]\n"i
  #               cyan       user   yellow     @host path                          green                           red                   reset colour
  zintis@vm1~[202] ✅

  # on zintis.net
  export PS1="\[\033[01;34m\]\u@\h.net\[\033[01;33m\] \$PWD\[\`if [[ \$? = "0" ]]; then echo '\e[32m[\!]\e[0m'; else echo '\e[31m[\!]\e[0m' ; fi\`:\n\$ "
  # on macbook zsh
  export PS1="\[\033[01;33m\]\$PWD\[\`if [[ \$? = "0" ]]; then echo '\e[32m[\!]\e[0m'; else echo '\e[31m[\!]\e[0m' ; fi\`:\n\$ "

4.1.1 Summarizing x && y ==> y only if x true

4.2 or operator || only execute the 2nd part if the first FAILED

The second choice will print the message ONLY if the first command failed. If the first command succeeded, the second command will NOT run.

The above still might have an ugly error, which you can supress by redirecting to null, stderr is &2, i.e. &2 > /dev/null

ansible@c8host ~[1172] $
false || echo "logical OR says false was false"
logical OR says false was false

ansible@c8host ~[1173] $
true || echo "logical OR says false was false"

ansible@c8host ~[1174] $

In bash shell scripts you can do 'cd .. || exit' as a safe way to run a command and if that fails, simply exit the script altogether.

4.2.1 Summarizing x || y ==> y only if x false

4.3 always do the second no matter what, … just concatentate two commands

echo "first" ; echo "second will always print too" The first command completes, before the second command runs.

4.3.1 Summarizing x ; y ==> do x, do y

4.4 &

This is the same as the shell &, i.e. "put this in the background" sleep 5 ; echo "all done." sleep 5 & echo "all done."

The first command will print "all done" only after waiting 5 seconds. the second commnd will print "all done" right away, but the script will still take 5 more seconds to finish (less the time it took to echo all done)

4.5 Example:

  1. Inellegant
    #!/bin/sh
    if [ "$EDITOR"= "" ]; then
        EDITOR=vim
    fi
    echo $EDITOR
    

    slightly better, but identical is if [ -z "$EDITOR" ]; then

  2. Ellegant
    #!/bin/sh
    [ -Z "$EDITOR" ] && $EDITOR=vim 
    echo $EDITOR
    

    This reads: "if $EDITOR is equal to null, then assign vim to $EDITOR"

  3. -Z "$EDITOR" asks does $EDITOR = null??

4.6 Good code fragment example

add) asktype && askinfo && tryconnect && finalize || delete ;;

This will run each of these commands in order, and continue as long as each of them runs successfully. As soon as one fails, all the others will fail, until you get to the || which will run if any of the previous failed. If they all succeeded, then the delete will NOT run.

Slick and effective.

5 Job Control

Any command or script you run can be controlled from the command line with a few commands, and understanding Linux signals. First to realize is that every linux process will have a process ID, or PID. That lets you uniquely identify each process on your system. PIDs are listed on all commands related to running jobs, such as top or htop and jobs

5.1 Interactive job control (bash)

Depending on which shell you are running, it will contain commands for interactive job control. For example, in bash, when a user runs an interactive command or program, you can temporarily suspend the program with C-z / ^z followed by the command bg or background.

When you run ^z on a running process, then issue the command jobs you will see that the state of that job is suspended. after running bg the state will have changed to running.

5.1.1 &

You can start a job while simultaneously putting it in the background with the & sign after the job. For example: tail -f /var/log/messages > temp-messages &

5.1.2 ^y

C-y is similar to C-z in that it suspends a process, but C-y delays suspending the job until the job itself is waiting for keyboard input. C-z does not wait but the trade-off is that any pending output will be dropped.

5.1.3 identifying jobs

See man bash for a description on how to refer to jobs, particularly the use of:

  • %jobname refers to a stopped job jobname
  • $?obnam refers to a stopped job whose name contains obnam
  • %n refers to a stopped job numbered n (not to be confused with PID
  • %% current job (also %+ is used)
  • %- previous job

The job number is NOT the PID, rather it is a sequential number given to the jobs started in this interactive bash session starting from [1] etc. A user can kill with a PID; kill 555, or kill with a job number; kill %2

fg for "foreground" brings a job back to the foreground (interactive session). You can fg based on a PID or based on the job number (more common) A short cut for fg is to simply name the jobname or jobnumber. For example

  • fg %2
  • %2

Are synonymous and bring job [2] to the foreground.

5.2 Linux Signals

Linux signals are always at your fingertips if you remember man signal

       
No. Name Default Action Description
1 SIGHUP terminate process terminal line hangup
2 SIGINT terminate process interrupt program
3 SIGQUIT create core image quit program
4 SIGILL create core image illegal instruction
5 SIGTRAP create core image trace trap
6 SIGABRT create core image abort program (formerly SIGIOT)
7 SIGEMT create core image emulate instruction executed
8 SIGFPE create core image floating-point exception
9 SIGKILL terminate process kill program
10 SIGBUS create core image bus error
11 SIGSEGV create core image segmentation violation
12 SIGSYS create core image non-existent system call invoked
13 SIGPIPE terminate process write on a pipe with no reader
14 SIGALRM terminate process real-time timer / alarm expired
15 SIGTERM terminate process software termination signal
16 SIGURG discard signal urgent condition present on socket
17 SIGSTOP stop process stop (cannot be caught or ignored)
18 SIGTSTP stop process stop signal generated from keyboard
19 SIGCONT discard signal continue after stop (what bg does)
20 SIGCHLD discard signal child status has changed
21 SIGTTIN stop process background read attempted
      from control terminal
22 SIGTTOU stop process background write attempted to
      control terminal
23 SIGIO discard signal I/O is possible on a descriptor
      (see fcntl(2))
24 SIGXCPU terminate process cpu time limit exceeded
      (see setrlimit(2))
25 SIGXFSZ terminate process file size limit exceeded
      (see setrlimit(2))
26 SIGVTALRM terminate process virtual time alarm
      (see setitimer(2))
27 SIGPROF terminate process profiling timer alarm
      (see setitimer(2))
28 SIGWINCH discard signal Window size change
29 SIGINFO discard signal status request from keyboard
30 SIGUSR1 terminate process User defined signal 1
31 SIGUSR2 terminate process User defined signal 2

common

Linux also has keyboard shortcuts as well as kill commands for the most common signals, i.e. kill -TERM 555 to terminate PID 555 See man kill for a listing of these signal names, and more information.

No. Name Usage name Key kill alt kill
        command command
1 SIGHUP HUP (hangup) C-d kill -1 kill -HUP %n
2 SIGINT INT (interrupt) C-c kill -2 kill -INT %n
3 SIGQUIT QUIT C-\ kill -3 kill -QUIT %n
         
6 SIGABRT ABORT   kill -6 kill -ABRT %n
         
9 SIGKILL KILL   kill -9 kill -KILL %n
         
14 SIGALRM ALRM (alarm)   kill -15 kill -ALRM %n
15 SIGTERM TERM (terminate)   kill -15 kill -TERM %n
17 SIGSTOP STOP (suspend)) C-z kill -17 kill STOP %n
         
19 SIGCONT CONT (continue) bg bg kill -CONT %n

kill -s TERM %2 and kill -15 %2 and kill -TERM %2 are all equivalent, and all terminate job 2. And since the default signal is SIGTERM these all are also equivalent to simply kill %2 And since the default job number is %1, if you want to kill the

Check out this simple python program that you can use to get familiar with signals. The program will not stop for most signals, but SIGKILL will always override, so that is how you get out of the infinite loop below:

  #!/usr/bin/env python
import signal, time

def handler(signum, time):
    print(f"\n   I got a SIGINT, but will keep going")

signal.signal(signal.SIGINT, handler)

# infite loop
print(" Press ^C i.e. SIGKILL to exit this program")

i = 0
while Ture:
    time.sleep(0.1)
    print(f"\r{i}", end="")
    i += 1

SIGKILL should be used with caution when that process has spawned child processes, because SIGKILL will terminate the process and leave the child processes orphaned, and keep running. Will get weird behaviour.

logging out of a session will send a SIGHUP.

5.2.1 nohup

You can start a job and prefix it with nohup. This will tell the shell that this job should ignore the SIGHUP command. Useful if you are running a job on a remote server and want to logout, but keep the job running. Otherwise a logout action will hang up all jobs you have started.

nohup tail -f /var/log/messages > ~/messages_for_myself
logout

6 Users and shell setup

To change a users default shell, see cat /etc/shells Or, as a user, to confirm the default shell setting use: echo $SHELL

Technically that just echos the environment variable $SHELL, but you could actually be running a shell interactively that is NOT what is reflected in $SHELL. see below.

You could also run printenv SHELL but that already supposes some csh is being run.

echo $0 will report -bash if you are running bash or tcsh. That is because $0 is the name of the runnin process. This will NOT work for csh.

If you are running fish (by running /usr/local/bin/fish from the command line) then echo $SHELL will show you /bin/tcsh (because I had set that as the default shell, which obviously is not the shell I am running currently!!. Watch that and don't get caught by this

6.0.1 Check what shell you are running

There are several ways. It is good to know several as there are differences between the shells you are running, and one method may work for one and not the other.

  1. echo $0 Most shells store the current shell in the variable $0
  2. echo $SHELL Another common variable set to the shell you are running
  3. echo $$ Most shells set the PID of the currently running process to the special variable $$ You can follow up with ps -p PID
  4. ps -p $$ does the above in one command
  5. echo $BASH within a bash shell
  6. echo $VERSION within a tcsh shell
  7. if [ -z "$BASH" ]; then echo "Run script $0 with bash please"; exit; fi Actually $0 is set to the current running process. From within a shell that is obviously the shell itself, but $0 would be the script name from within a running bash script etc. (-z object) means if object exists return true.

6.1 Look of the process (shell) having PID of the current shell instance

The pid of the current instance of shell is echo "$$" Then, find the process of that pid with ps -p <PID>

Or putting it in a single command: ps -p "$$"

Notice that $$ is NOT the pid in fish. For fish you need to use $fish_pid.

6.2 chsh -s /usr/bin/bash

This will persistently change the default shell for you. Logout and back in to see the effects.

6.3 Mac OSX (Darwin, 19.6.0)

When I had set my terminal default shell to be /bin/bash, then I got this error when opening a new terminal window:

The default interactive shell is now zsh.
To update your account to use zsh, please run `chsh -s /bin/zsh`.
For more details, please visit https://support.apple.com/kb/HT208050.

Here is me on Darwin, changing my default shell to bash.

$ chsh -s /bin/bash
Changing shell for zintis.
Password for zintis: 
chsh: no changes made

No changes were made because it was already /bin/bash.

6.4 usermod

usermod is a utility for modifying a user's account details that are store in /etc/passwd.

One could just sudo vi /etc/passwd and not make ANY mistakes, but usermod is a safer option.

  • usermod --shell /usr/bin/tcsh zintis

Then confirm by grep zintis /etc/passwd

6.5 chsh

does the same thing usermod does, but only for changing a users shell.

  • chsh --shell /usr/bin/tcsh zintis

6.5.1 chsh not found

It could be that you have not installed chsh on CentOS. To remedy that, first find out which dnf utility provides chsh using this command:

  • dnf provides chsh

The output of which will give you this:

Last metadata expiration check: 2:35:04 ago on Thu 22 Apr 2021 10:20:16 AM EDT.
util-linux-user-2.32.1-24.el8.x86_64 : libuser based util-linux utilities
Repo        : @System
Matched from:
Filename    : /usr/bin/chsh

util-linux-user-2.32.1-24.el8.x86_64 : libuser based util-linux utilities
Repo        : baseos
Matched from:
Filename    : /usr/bin/chsh

Then, install it with: dnf install util-linux-user After that, I found chsh to be in /usr/bin/chsh and all was good.

6.6 adduser (symlink to useradd, so just use useradd)

useradd zintis
passwd zintis
usermod -aG wheel zintis

useradd zintis -u 1989  # specify the uid for zintis to be 1989

useradd -c "Zintis Perkons" zintis


visudo 
# add zintis below ROOT user privileges section.  Make it identical
# add into line: Defaults    env_reset,timestamp_timeout=120
Defaults    env_reset,timestamp_timeout=120

## Allow root to run any commands anywhere
root    ALL=(ALL)       ALL
zintis  ALL=(ALL)       ALL

6.7 userdel

6.8 groups

From linuxconfig.org Each time a new user is created, automatically a new group with the same name of the user is created too, and the user becomes its sole member. This is what is called a primary or private group.

Each user has its own primary group, named after himself, with no other members. This setup makes possible to change the default umask value: traditionally it was 022 (this means 644 permissions for files and 755 for directories), now it is usually set to to 002 (664 permissions for files and 775 for directories). (mask is subtracted from 666 for files and 777 for directories. … -1 is the same as 0)

Since, by default, each file or directory created by a user is created with that user's primary group, this setup, while preserving security (a user can still modify only its own files), simplifies resources sharing and collaboration between users which are members of the same group when the setgid bit is used, by allowing write permissions for the group.

6.8.1 usermod to add a user to an existing group

In general we can add a group, remove a group, change the home directory…

  • -G is same as –groups
  • -a is append

So, usermod -a -G teamleads zintis where you need the -a to append.

  • sudo usermod -a -G existing-groupname username
  • sudo usermod -a -G wheel zintis
  • sudo usermod -a -G wheel,operators,engineering zintis
  • > this won't work. You have to add the groups one at a time as so:
  • sudo usermod -a -G wheel zintis
  • sudo usermod -a -G operators zintis
  • sudo usermod -a -G engineering zintis

groups zintis # to show all the groups that have zintis as a member

  1. usermod -a (append) requires -G (supplementary groups)

    this will append the additional groups to a user. For example sudo usermod -a -G community kulmanis Compare that to:

  2. usermod -g group sets the primary group for that user

    Notice that you cannot delete an undesired group if that group has a user with that group as a primary user. If that happens, reset that user's group to the proper group (typically its own group) for example. =sudo usermod -g braun braun Then you can groupdel undesiredgroup.

6.8.2 Adding a user to an existing group by editing /etc/group & /etc/gshadow

Use the command similar to visudo, called vigr vigr -s is for modifying the /etc/gshadow file. (will have to w! to override the read-only aspect of this file. That is normal.)

Then change the lines of the groups you want to add the user as so: group-name:group-passwd:group-id:users where users can be a comma separated list of users.

  1. Alternatively use grpconv

    grpconv will keep the /etc/group and /etc/gshadow files in sync. It will create a new /etc/gshadow file from the /etc/group file, clobbering the existing one.

    sudo grpconv

    Then you can check / verify that the files are consistent by:

    sudo grpck

    … and see no output. This is good.

6.8.3 display the group ownership of a files

  • getent group <group-id, or group-name>
  • getent group <group-id, or group-name>
  • getent group <group-id, or group-name>
  • getent group <group-id, or group-name>
  • getent group # will show all of them
  • awk -F':' '/gawlergrp/{print $4}' /etc/group

6.9 some homegrown way of changing group ownership

6.9.1 change ALL files in a folder to a given group:

  • find . -type f -exec chown <owner>.<group> {} \;

6.9.2 change ALL directories in a folder to a given group

  • find . -type d -exec chown <owner>.<group> {} \;

6.10 The RIGHT way is to use chgrp recursive function, -R

chown -R owner:group * .[^.]*

6.11 groupadd

This adds a new group to /etc/group

6.12 Ensuring new files in a directory have the proper group

chgrp super kaapvall/ sarmation/
chmod g+s kaapvall/ sarmation/

This forces new files created inside the directories kaapvall and sarmation to have the group ownership "super" instead of the user's group.

6.13 filesystem ACLs

To set the default group permissions, you will have to use ACLs. Set a "default" ACL:

  • setfacl -m "default:group::rwx" /var/www

To also adjust subdirectories: find /var/www -type d -exec setfacl -m d:g::rwx {} +

Note: The file system must have ACL support enabled. Sometimes it is on by default; on ext3 or ext4 you might get "Operation not supported", in which case it must be enabled manually:

For a currently mounted filesystem: mount -o remount,acl /

Permanently – one of the methods below:

  • at fstab level: edit /etc/fstab to have acl in the options field
  • at filesystem level: tune2fs -o acl /dev/diskname

7 setuid / setgid / sticky

Just like normal permissions, the special bits can be assigned with the chmod command, using the numeric or the ugo/rwx format. In the former case the setuid, setgid, and sticky bits are represented respectively by a value of 4, 2 and 1.

So for example if we want to set the setgid bit on a directory we would execute: chmod 2775 test With this command we set the setgid bit on the directory, (identified by the first of the four numbers), and gave full privileges on it to it's owner and to the user that are members of the group the directory belongs to, plus read and execute permission for all the other users (remember the execute bit on a directory means that a user is able to cd into it or use ls to list its content).

The other way we can set the special permissions bits is to use the ugo/rwx syntax: chmod g+s test. To apply the setuid bit to a file, we would have run: $ chmod u+s file While to apply the sticky bit: $ chmod o+t test

  setuid setgid sticky
  chmod u+s chmod g+s chmod o+t
  4 2 1
Operating Run as Run as member  
on files owner of group IGNORED on Linux
       
Operating IGNORED on all created files allow anyone to create,
on directories Linux systems set to group of but only the owner and
    directory. root to delete the file
  "file giveaway"   The file is "Sticky"
  not allowed Group consistency  

7.1 On directories

  1. Setuid on directories (ignored)

    Ignored on most Unix systems including Linux. Setuid and setgid were meant for completely different purposes. Setuid is for causing an executable file to run with its owner's uid or gid for setgid, rather that the uid of the user calling the executable. Setuid has no function on directories. Does it make sense for a user to create a file in a directory only to have it owned by someone else? What happens to the creator? Will they still have access to the file? "File giveaways" are not allowed.

7.1.1 setgid on directories

All files created in this directory will have GID set to the /same/ GID as the parent directory, and NOT the usual GID of the user creating the file. It ensures group owner consistency on every file in a directory.

7.1.2 sticky bit on directories

All users allowed to create files in this directory. No other users can delete these files. Only owner and root of the files can delete files.

7.2 On files

7.2.1 setuid on files

On executable files, will assume the rights and privileges of the owner of the file, not the user running/executing

7.2.2 setgid on files

On executable files, will assume the rights and privileges of the group of the file, not the user running the executable.

  1. sticky on files (ignored)

    ? doesn't make sense, hence it is IGNORED on Linux systems. Other *nixes use it for special purposes, but they are not standard uses.

7.2.3 Reviewing:

sticky bit applies to directories, to make any files created in that directory "sticky" meaning, no one else can delete them, other than root and the file owner.

setgid bit applies to directories, and makes any file or sub-directory created in that directory have a group-owner set to match the group-owner of the directory itself (the one that has the sticky bit)

setuid bit applies to program files and makes those executable files run with the same permissions as the owner of the exectuable file (and not the user running the executable as usually happens)

For directories where you want a master user have access over and above the normal access that user would have, you use the setgid on the directory, and make both the master user and the directory in a group that matches. i.e. directory has chgrp on it for a group that the master user is a member of.

Press Shift + Tab to navigate to chat history.

8 umask

from man pages: "The user file-creation mask is set to mode. If mode begins with a digit, it is interpreted as an octal number; otherwise it is interpreted as a symbolic mode mask similar to that accepted by chmod(1). If mode is omitted, the current value of the mask is printed. The -S option causes the mask to be printed in symbolic form; the default output is an octal number. If the -p option is supplied, and mode is omitted, the output is in a form that may be reused as input.

The return status is 0 if the mode was successfully changed or if no mode argument was supplied, and false otherwise."

The final permissions of a file are determined from TWO SOURCES

  1. Proccess permissions
  2. umask permissions

    Linux umask bits define what is forbidden. SAMBA create bit masks define what is allowed.

8.1 Process permissions

When a process creates a file, the file has certain default permissions, for example, -rw-rw-r–. These initial permissions are partially defined by the file mode creation mask, also called file permission mask or umask. Every process has its own umask, for example, bash has umask 0022 by default. Process umask can be changed.

When a file or directory is created, the permissions are set by the process that created that file or directory. For example, when using the touch command to create a new file, the default permissions set by that process for files is 0666. If the UMASK were set to 0000, then the file permissions would be 0666.

From that I gather that the touch proccess has default permissions of 0666

8.2 umask subtractions from 666 or 777

The umask is the value that is subtracted from 666 (rw-rw-rw-) permissions when creating new files, or from 777 (rwxrwxrwx) when creating new directories.

For example, if the default umask is 002, new files will be created with the 664 (rw-rw-r–) permissions, and new directories with the 775 (rwxrwxr-x) permissions.

If umask is 0026 new files will have chmod 640 new dirs will have chomd 751

8.3 umask bits

A umask consists of bits corresponding to standard file permissions. For example, for umask 0137, the digits mean that:

0 = no meaning, it is always 0 (umask does not affect special bits) 1 = for owner permissions, the execute bit is set 3 = for group permissions, the execute and write bits are set 7 = for others permissions, the execute, write, and read bits are set Umasks can be represented in binary, octal, or symbolic notation. For example, the octal representation 0137 equals symbolic representation u=rw-,g=r–,o=—.

Symbolic notation specification is the reverse of the octal notation specification: it shows the allowed permissions, not the prohibited permissions.

8.3.1 How umask works

How umask works Umask prohibits permissions from being set for a file:

Put another way: Linux umask bits define what is forbidden. SAMBA create bit masks define what is allowed.

When a bit is set in umask, it is unset in the file. When a bit is not set in umask, it can be set in the file, depending on other factors. The following figure shows how umask 0137 affects creating a new file.

The mask is just that, a mask to mask off what bits will be OFF So umask of 137 applied to a new file will be like running chmod 640.

All ones:

    owner group others
4 read # # #
2 write # # #
1 execute # # #
7   7 7 7

umask 137:

    owner group others
4 read     #
2 write   # #
1 execute # # #
7   1 3 7

Resulting permissions:

    owner group others
4 read # #  
2 write #    
1 execute      
7   6 4 0

Easiest way to approach this is to say, the umask adn mask have to add up to 7

8.4 umask excercise, easily reproduced:

umask by itself will display what the current umask is set to. umask <octalno.> will set it to the octalno value.

umask 0000 touch one two three ls -l * umask 0002 touch four fix six ls -l *

The above will show you that: UMASK did not allow the others component to have write permission. The way that the UMASK works is it defines what is allowed or permissible. With all zeros, it’s saying that read, write and execute permissions are permissible, so it’s not restricting anything and the permissions will be set by the process. Yet in our second example with a UMASK set to 0002, this is restricting the access for others to not have write permission.

Octal digit This allows resulting file as if
in umask   permission chmod was
0 read, write & execute rwx 7
1 read and write rw- 6
2 read and execute r-x 5
3 read only r-- 4
4 write and execute -wx 3
5 write only -w- 2
6 execute only –x 1
7 no permissions --- 0

8.5 Summarizing umasks

UMASK did not allow the others component to have write permission. The way that the UMASK works is it defines what is allowed or permissible.

With a umask set to all zeros, it’s saying that read, write and execute permissions are all permissible,

it’s not restricting anything and the permissions will be set by the process.

With a UMASK set to 0002, it's saying to remove write permission.

Put another way: Linux umask bits define what is forbidden. SAMBA create bit masks define what is allowed.

9 chmod g+s on directories if you want files created in directory to give group rights

9.1 Ensuring new files in a directory have the proper group

chgrp super kaapvall/ sarmation/ # makes group 'super' for these 2 directories chmod g+s kaapvall/ sarmation/ # makes new files have group 'super' in these 2

This forces new files created inside the directories kaapvall and sarmation to have the group ownership "super" instead of the user's group.

10 Permissions for samba users

SAMBA IGNORES UMASK SETTINGS IN THE LINUX ENVIRONMENT! Really!!!

Instead it provides its own equivalent parameter. But here'w the gotcha:

It is the I N V E R S E of the Linux UMASK


So instead of 000 it woud be 777.

Here are the default options:

create mask = 0744 force create mode = 000 create directory mask = 0755 force directory mode = 000

“The create mask and create directory mask options consist of octal permission
sets for files and folders, respectively, that combine with the permissions
requested by the user using a logical AND. Any bit not set in the mask will be
removed from the final permission set. The force create mode and force
directory mode options also specify octal permission sets, but are combined
with the permission set using a logical OR (after applying any creation
masks). Therefore, any permission bit that is set in the force mode is
included in the final permissions.”

— Gerald Carter, Jay Ts, and Robert Eckstein. Using Samba, 3rd Edition
Octal digit This allows resulting file SAMBA
in umask   permission create masks
0 read, write & execute rwx 7
1 read and write rw- 6
2 read and execute r-x 5
3 read only r-- 4
4 write and execute -wx 3
5 write only -w- 2
6 execute only –x 1
7 no permissions --- 0

Put another way: Linux umask bits define what is forbidden. SAMBA create bit masks define what is allowed.

10.1 SAMBA and Linux UMASK

This works quite differently than the Linux environment UMASK. Instead of defining what is restricted, it defines what is allowed. With the create mask and create directory mask options, a setting of 0777 would allow all bits to be set, thus leaving the permissions to be set by the requested user. Furthermore, the force create mode and force directory mode would force a particular bit to be set, even if it wasn’t requested by the user.

As an example, consider the following option settings as you would define them for a share in smb.conf:

create mask = 0770 force create mode = 060 create directory mask = 0770 force directory mode = 070

This would allow both files and directories to have read, write and execute permissions for user and group, but not others. At the same time, it forces the group permissions to be read/write for files and read/write/execute for directories.

10.2 –no-same-owner

When extracting a tar archive from another vm or Linux host, you may have different owners defined on one machine to the next. In order to over-ride the original tarred owner of files, use the --no-same-owner option.

This will make the onwer of the new un-tarred files yourself. Otherwise you will see some user id number as the owner for these files, that may not even be a valid user on your system, or even worse, make an unsuspecting innocent valid user on your system the owner of these files, which of course they should not be.

10.3 Related SELinux context

First of all, selinux (Secure Enterprise Linux) config files are in: /etc/selinux/config where you can change SELinux from enforcing to not.

  1. Change from enforcing to not enforcing (permissive) SELinux or disabled
    # This file controls the state of SELinux on the system.
    # SELINUX= can take one of these three values:
    #     enforcing - SELinux security policy is enforced.
    #     permissive - SELinux prints warnings instead of enforcing.
    #     disabled - No SELinux policy is loaded.
    SELINUX=permissive
    

    (Related SELinux contexts. See below on SELinux Contexts) For more info on samba, see the samba.org file If user connects anonymously he gets r/o, if he logs in and is a member of assigned group he gets r/w.

    I have group named 'admin' set as primary group to users with write privileges, everyone else gets read only rights.

    I force user to nobody, so different people working on same files don't interfere with each other.

    I set chmod 2755 on shared directory, so it inherits created directories with the same group 'admin'

    $ chmod -R 2755 /home/shares/test Checking if all is good:

    $ stat home/shares/test Access: (2755/drwxr-sr-x) Uid: (65534 nobody) Gid: ( 1001/ admin) Relevant part of /etc/samba/smb.conf:

    [test] comment = test path = /home/shares/test force user = nobody read only = No create mask = 0664 force create mode = 0664 directory mask = 02775 force directory mode = 02775 This post put me on right track, but testparm revealed 4 incorrect directives, so I'm sharing fixed config here. In samba, the less directives you specify the better it works.

10.4 Additional online disuccsions to be reviewed.

I had the same problem, but everything like mask directives did not work for me (Samba 4.3.11):

create mask = 0664 force create mode = 0664 directory mask = 02775 force directory mode = 02775 The only option that worked was under the [global] or share section:

inherit permissions = yes Just change all folder and file permissions to your need, so future folders and files will inherit the same permissions.

3

There is a very similar problem when connecting from other Unix / Linux / OSX / MacOS devices: all of the settings are ignored unless you specify

[global]

unix extensions = no And connect with smb://<serverhost> instead of cifs://<serverhost>.

another: [html] comment = admin access path = /var/www/html browsable = yes guest ok = no writable = yes valid users = @admin create mask = 664 force create mode = 664 security mask = 664 force security mode = 664 directory mask = 2775 force directory mode = 2775 directory security mask = 2775 force directory security mode = 2775


SOLVED (seems to be)

The problem was in the obey PAM restrictions parameter. By default it is turned off, and I could not remember why I turned it on. The SAMBA config was partially taken from older installation may be I had reasons to be obeyant there :-)

When it is ON, then SAMBA-created files are under UMASK restrictions. I don't know, if it is correctable via login defaults (what is the user?), but umask command gives me "0022" which means "u+a g-w o-w".

Hope, it will help to someone with similar problem.

shareimprove this answerfollow


/etc/samba/smb.conf file. For my Music share, I added the following options:

create mask = 664 force create mode = 664 security mask = 664 force security mode = 664 directory mask = 2775 force directory mode = 2775 directory security mask = 2775 force directory security mode = 2775





11 Setting Unix umasks

First off what is my umask? run : umask (or umask -S for symbolic form)

setting a umask is as simple as umask 137

11.1 Environment variables

11.1.1 bash: PS1="my prompt >\n"

  • export PS1="my prompt >\n"
  • echo $PS1
  • env
  1. printenv

    this will print out the user environment variables

  2. set

    will print out the user environment variables AND system environment settings

  3. env

    will print out the user environment variables (is this synonymous with printenv)?

11.1.2 tcsh: setenv prompt='my prompt >"

setenv prompt='my prompt >" printenv prompt

11.1.3 fish set PYENVVERSION 3.7.5

set -x key value set -x PYENVVERSION 3.7.5 set -e PYENVVERSION 3.7.5 set PYENVVERSION 3.7.5

printenv env echo $PYENVVERSION

set -g PYENVVERSION 3.7.5 set -x PYENVVERSION 3.7.5 set -xg PYENVVERSION 3.7.5

what is the difference? see man set but in a nutshell:

-x flag is orthogonal ot the -g, -l, -u flags -x simply sets the export attribute on the var -g,-l,-u set the scope of the var

You can have a global, unexported var and inside a function do set -lx var value to create a locally scoped instance that is exported. When control returns from the function the global scope var pops back into existence and won't be exported.

An exported variable is visible to commands run by your fish shell.

Note that env is an external command that shows the exported variables it inherited. So, it should only show the variables created with set -gx

11.2 fish scripting

Start with the very simple: for i in (ls) echo $i end

Note how that is different from bash that would use backtics for subcommands `ls`

12 export and variable scope

12.1 shell variables

Shell variables are created when you first assign a value to them:

MYVAR = 314159
echo "MYVAR is set to $MYVAR"

You can have a global, unexported var such as MYVAR in the above example, and then inside a function do set -lx MYVAR =value to create a locally scoped instance, that is different from 314159. When control returns from the function the global scope var pops back into existence and won't be exported.

Variables that are NOT exported are called shell variables.

12.2 Environment variables

An exported variable is visible to commands and child processes run by that shell. All shells opened beneath this shell will still have this variable set to the same value. This is what is known as an environment variable, because it holds for the entire environment below it.

A full example of a shell variable:

$ MYVAR=314159
$ echo "MYVAR is set to $MYVAR"
MYVAR is set to 314159    
$ bash
$ echo "MYVAR is set to $MYVAR"
MYVAR is set to
$

Now as an environment variable:

$ export MYVAR = 314159
$ echo "MYVAR is set to $MYVAR"
MYVAR is set to 314159    
$ bash
$ echo "MYVAR is set to $MYVAR"
MYVAR is set to 314159
$

** Useful environment variables In no particular order:

HISTTIMEFORMAT="%Y-%m-%d %T"
# used by logs and other processes.. year month day and time

12.2.1 printenv

You can printenv variablename to see what and if that variable is set, or just printenv with no arguments to see all the environment variables.

12.3 Special shell variables

There are several special shell variables set by most shells (not all). They are:

In bash:

  1. ${$} the current running process ID, i.e. PID
  2. ${n} positional parameters Where ${1} is the first parameter, $2 is 2nd parm etc. Where the n is a number, the curly braces are options.
  3. $0 Special case of the above, but results in the current running job/command
  4. ${@} all parameters All of them. Can say $@
  5. ${*} all parameters All of them. Can say $* But output has the IFS between parameters For example myscript 40 myfile.txt results in $* as 40;myfile.txt when IFS is ";"
  6. ${#} how many parameters exist
  7. ${?} the error code of the last command
  8. ${!} the PID of the last background job/command
  9. ${_} the last parameter

In most cases the curly braces are optional, so you can use:

  1. $$
  2. $3
  3. $0
  4. $@
  5. $*
  6. $#
  7. $?
  8. $!
  9. $_

Not to be confused with the command line history commands that all begin with the 'bang' character, like !! and !$ and !* and !-3 as in:

  • !! previous command, often used with sudo !!
  • !$ previous command's last argument, often used with ls *.*~; rm !$
  • M-. does the same as !$ i.e. the previous command's last argument
  • !-2 two commands ago
  • !-n n commands ago
  • !545 the 545th command, often used after history | grep somestring

13 crontab

See crontab.org file for more details

*    *    *   *    *    <command to be executed>
-    -    -   -    -   
|    |    |   |    |  
|    |    |   |    +------day of the week (0 - 6)  (Sunday=0
|    |    |   |
|    |    |   +-------month (1 - 12)
|    |    | 
|    |    +------ day of month ( 1 - 31 )
|    |   
|    +------hour (0 -23)
|
+----+ min (0 - 59 )

14 xargs

xargs takes a list and passes the list as arguments to another program.

xargs -options arguments

By default the arguments is "echo". But argueents is usually a command.

tail -3 ~/bash_history | xargs
or
ls *.org | xargs wc -l

printf "file1\nfile2\nfile3" | xargs touch
even better:
printf "1\n2\n3\n4" | xargs -I_ touch file_.txt

for i in {10..30} ; do echo $i ; done | xargs  -I_ touch deletme-file-_.txt

-i tells xargs we will replace some text for each argument

Note: in the above example the for i loop is valid in /bin/bash and /bin/zsh It is NOT a valid syntax in /bin/tcsh

14.0.1 fix this example

Let's say your command is "dnf uninstall" And the list of apps you want to uninstall came from a grep: dnf list --installed | grep microsoft | xargs | dnf uninstall will issue the command dnf uninstall arg (one at a time where arg is the list of apps tht include microsoft) you could test this first with dnf info in place of dnf uninstall.

14.1 a better chgrp example

Let's say you want to change all python programs, (ending in .py) in the current directory, to be executable, and belong to the zintis group.

ls *.py | xargs chmod 740 
ls *.py | xargs chgrp zintis

And you are done!

Notice that you don't actually put the file as an argument at the end of the chmod or chgrp command. If that makes it confusing for you, the following two command are identical in action.

ls *.py | xargs -I{} chmod 740 {}
ls *.py | xargs -I{} chgrp zintis {}

One final point, these xargs examples actually generates just one chmod or chgrp command, ef chmod 740 file1.py file2.ph file3.py …. filen.py

If you want to issue the command itself once for each file to get chmod 740 file1.py chmod 740 file2.py . . chmod 740 filen.py

Then you must also include -n 1, so ls *.py | xargs -n 1 chmod 740

15 Candidate Packages from EPEL

EPEL or 'Extra Packages for Enterprise Linux" from the EPEL Wiki

Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special Interest Group that creates, maintains, and manages a high quality set of additional packages for Enterprise Linux, including, but not limited to, Red Hat Enterprise Linux (RHEL), CentOS and Scientific Linux (SL), Oracle Linux (OL).

EPEL packages are usually based on their Fedora counterparts and will never conflict with or replace packages in the base Enterprise Linux distributions. EPEL uses much of the same infrastructure as Fedora, including buildsystem, bugzilla instance, updates manager, mirror manager and more.

While there are far fewer packages in EPEL than in Fedora (~4,000 vs ~30,000) that is still way too many packages for my liking given that I won't be using most of them. However, you can list the included packages from the download directories in fedoraproject.org

From these directories you can choose the individual packages (.rpm) that you want to install, and because these are vetted, they are most likely going to be compatible with the version of RHEL, or Fedora, or CentOS that you are running. Just look at the right download page that matches the CentOS that you are running.

17 Manual pages

man man

Man pages are divided in sections, mostly in section 1 To see specific sections enter the section # followed by the man page, for example

man ls man 5 man.conf

Where man man defaults to section 1, i.e. man 1 man is the equivalent But man 5 man.conf will open the manual pages for man.conf under section 5

man -k network # show which man pages talk about "network"

17.1 MAC OSX

On Darwin (Mac OSX) the sections are The standard sections of the manual include:

1 User Commands

2 System Calls

3 C Library Functions

4 Devices and Special Files

5 File Formats and Conventions

6 Games et. Al.

7 Miscellanea

8 System Administration tools and Deamons

17.2 CentOS

On CentOS the sections are The standard sections of the manual include:

1 User Commands

18 YUM a.k.a. DNF

See YUM a.k.a. DNF in my yum.org file. But in a nutshell these are both package managers for RedHat, CentOS and some others much like RPM is for RedHat, and BREW is for MAC OSX.

19 sudo

On CentOS there does not seem to be the wheel group. But I could still add an individual user to the sudoers file the following steps:

su > this switches to superuse (need to enter password)

19.1 sudoers file and visudo

visudo > this command opens up vi with the sudoers file add the user after the line for root: i.e.:

## Allow root to run any commands anywhere
root    ALL=(ALL)       ALL
zintis  ALL=(ALL)       ALL

19.2 sudo timeouts

By default the sudo timeout is fairly short (well depending where you are I guess). It extend the timeout to say, 25 minutes you can do the following:

sudo visudo edit the line: Defaults env_reset to add: Defaults env_reset, timestamp_timeout=30 Defaults env_reset,timestamp_timeout=30 not sure if a space comes after the comma…

Or, just add a line by itself as in: Defaults timestamp_timeout=30

You can see the value by sudo grep timeout /etc/sudoers

20 hostname

On CentOS the hostname is stored in /etc/hostname and the command to change it is: you guessed it, hostname.

20.1 hostname

hostname # to display it hostname -s to hostname -f # display the FQDN

To permanently change it, use hostnamectl set-hostname your-new-hostname

This worked, and changed it in /etc/hostname

20.2

There is also sudo vi /etc/sysconfig/network

With lines like: HOSTNAME=c8host.zintis.net

And /etc/hosts where you can add 192.168.122.76 c8host.zintis.net c8host

20.3 nmcli general hostname

Can also set the hostname to "arthurdent" by: nmcli general hostname arthur_dent

21 domain name

/etc/sysconfig has many configurations that are similar to the old init.d or . It is the new approach to init.d with more flexibility. Among the sysconfig directory is the netowkr config that on a new CentOS has only one comment line: # Created by anaconda

But, domain suffix, i.e. senecacollege.ca, is configured in the dns configs, so, /etc/resolv.conf

nameserver # DNS server IP (can have three of these) domain # Domain Name of local host search # Which Domain to search

But with CentOS 8 this file is generated by the NetworkManager. That means that /etc/sysconfig/network-scripts/ifcfg-* changes will overwrite /etc/resolv.conf /etc/sysconfig/network-scripts/ifcfg-* changes will overwrite /etc/resolv.conf

If you want to prevent that from happening (and going back to old-school) you have to remove DNS1, DNS2, etc lines from /etc/sysconfig/network-scirpts/ifcfg-* files. Then NetworkManager will stop overwritting your /etc/resolv.conf file.

The search line is optional. If missing, then name resolution for host1 will be limited to the host1 plus the domain, i.e. host1.acme.com but if search has acme.ca then the resolution will be host1.acme.com followed by host1.acme.ca

21.1 Using NetworkManager

If you want to go with the flow and want to config DNS, you can add entries to /etc/sysconfig/network-scripts/ifcfg-eth0 like these two lines: DNS1=208.67.222.222 DNS2=208.67.220.220

21.2 /etc/sysconfig/network

Can straight edit this file too (needs SU priviledges) add HOSTNAME=myserver.cisco.com

22 Restarting the network

22.1 As of CentOS7 and CentOS8

The easiest is nmcli network off; nmcli network on

22.2 Deprecated init.d method

/etc/init.d/network restart In general Centos does not use service, but other Linux distributions still do. They would run sudo service postfix restart in place of sudo systemctl restart postfix Just for those times you are not on a CentOS, or RHEL, or Fedora or Ubuntuu system. Like Cisco switches for example.

This is the traditional approach, which has been replaced by systemctl When looking in the directory /etc/init.d the CentOS developers were good enough to leave you a note: as follows:

You are looking for the traditional init scripts in /etc/rc.d/init.d, and they are gone?

Here's an explanation on what's going on:

You are running a systemd-based OS where traditional init scripts have been replaced by native systemd services files. Service files provide very similar functionality to init scripts. To make use of service files simply invoke systemctl, which will output a list of all currently running services (and other units). Use systemctl list-unit-files to get a listing of all known unit files, including stopped, disabled and masked ones. Use systemctl start foobar.service and systemctl stop foobar.service to start or stop a service, respectively. For further details, please refer to systemctl(1).

Note that traditional init scripts continue to function on a systemd system. An init script /etc/rc.d/init.d/foobar is implicitly mapped into a service unit foobar.service during system initialization.

Thank you!

Further reading: man:systemctl(1) man:systemd(1) http://0pointer.de/blog/projects/systemd-for-admins-3.html https://www.freedesktop.org/wiki/Software/systemd/Incompatibilities

22.3 Network intefaces (ip command)

ip link help ip link show ip addr show ip -4 address show (use tab completion, and double-tab to see options)

23 Enabling nested KVM in CentOS8

Edit /etc/modprobe.d/kvm.conf uncomment the line options kvm_intel nested=1

As of Jan 12, I have not done this next step. Still as of <2022-03-07 Mon> my c8host still has this line commented out. Why? See kvm-nested.conf

Then ADD a new file called kvm-nested.conf with the following content:

options kvm-intel nested=1
options kvm-intel enable_shadow_vmcs=1
options kvm-intel enable_apicv=1
options kvm-intel ept=1

Then reboot.

23.1 reboot

shutdown -h now # actually halts the server, then you manually start it shutdown -r now # reboots reboot init 6

reboot and shutdown are both symbolic links to systemctl.

24 Switching between console and gui

C-M-f1 will boot into a tty (console) C-M-f7 will switch back to the graphics interface.

25 Notes on installing KVM on CentOS 8

These notes are from linuxconfig.org

26 Fix for my VMWare fusion CentOS8

I was not seeing any output on "grep -E '(vmx|svm)' /proc/cpuinfo" so I suspected that the fusion settings on my CentOS8 host did not support hardware virtualization.

I was right CentOS-host-settings.png

26.1 Result:

I was now getting the responses I was looking for. For example:


zintis@c8host ~$ modprobe -a kvm_intel
zintis@c8host ~$ cat /sys/module/kvm
kvm/       kvm_intel/ 
zintis@c8host ~$ cat /sys/module/kvm_intel/parameters/nested
Y
zintis@c8host ~$ 

26.2 SystemV used "run levels" (old school)

To make this stick: edit /etc/inittab and look for a line that begin with id:5. Replace the 5 in that line by 3. You can find a brief description of runlevels here, but shortly:

- Runlevel 0 and 6: halt and reboot the machine, respectively.
- Runlevel 1: No services running, only root can login.
- Runlevel 2: Users can login but no networking.
- Runlevel 3: Networking and text-mode.
- Runlevel 4: unused.
- Runlevel 5: GUI.

27 History

echo $HISTSIZE
echo $HISTFILESIZE

export HISTSIZE=200
echo $HISTSIZE

To clear a line from history, (for instance if you inadvertently entered a clear text password into the wrong window) use the -d command.

history -d 467

will delete line 467 from history.

28 Navigating and finding files

See also examples (assuming sudo is used on all) in the find.org file.

28.1 find older files

You can use the -mtime +5 option of the find command to find files that are older than 5 days

Other find time attributes:

-atime n  access time
-mtime n  modified time

n can be an exact time, i.e. 5 would be 5 days ago (5 X 24 hrs) but +5 is 5 days or older ago.

An example to move *.jpg files older than 30 days is:

find /path/to/files/ -type f -name '*.jpg' -mtime +30 -exec mv {} /path/to/archive/ \;

28.2 find large files taking up space

df and du of course, but here's how you can easily find the largest 20 files

  1. du -a /dir | sort -n -r | head -20 du -a . | sort -n -r | head

There are a few alternatives to du the most popular one being ncdu. -ncdu is written in c and users ncurses library, hence the name ncdu. To install ncdu you simply brew install du or sudo dnf install ncdu or apt install ncdu

But others are dust, written in rust programming language, diskus and others.

28.2.1 du usage

Common usages are:

  • du -sh . human readable and summarize total . is the default this dir
  • du -sh /etc /var as many directories as you want in one command
  • du -shc /etc /var the c totals the usage from all the directories listed
  • du -shc ~/* is very useful. It combines total of all sub-directories in ~
  • du -h --max-depth 2 human readable and summarize to 2 directories deep
  • du -shc * will be probably the most common options you use

While we are talking about du make sure you check out df command.

28.2.2 df

disk free

Common usages are:

  • df -h human readable
  • df -hT show type column along with usual Filesystem, Size, Used, Avaiable…
  • df -hT -x tmpfs same output but ignore lines that have tmpfs
  • df -hTx tmpfs exact same thing
  • =df -

Combine with watch for times when you want to monitor disk usage, or when you mount a drive and want to see what changes.

  • watch df -hTx tmpfs every 2 seconds.

28.3 find files or directories using globbing.

  • find . -name src -type d
  • find . -path '**/test/*.py' -type f

grep -R to recursively look for the string in all files and all subdirectories.

28.3.1 find and delete small image files

This one is useful, as mail and web pages often have many many small image files that are typically not needed after a few months, and they just take up inodes. So delete the like this:

  • find * -type f -name "*.jpeg" -size -1k -delete

But be careful where you use that!! Use caution.

28.3.2 rg "import requests" -t py ~/bin

28.3.3 fd is a simpler version of find

28.4 locate

Much like spotlight on mac, it indexes files on your system once a day.

28.4.1 locate (is a pre-indexed dbase of files) that make it fast.

locate whereismyfile.txt

28.4.2 Manually building locate index database

sudo updatedb The first time this is run it will take a couple of minutes, but after that it is very quick. But you don't have to every day, as updatedb ir run automatically every day.

28.5 which

Searches your path for executable files.

28.6 whatis

Gives you a brief man summary for a command

28.7 apropo (man -k)

28.7.1 rg "import request" -t

rg -u –files-without-match "^#\!" -t sh recursively grep, -u including hidden files, –files-without-match i.e. files that are missing somethin, ^#\! a shebang at the beginning of line that are of type sh (i.e. end in .sh) So this looks for .sh files that are missing the shebang line.

28.7.2 ripgrep

28.7.3 grep with –stats gives you interesting data.

28.7.4 fzf (pipe a grep into fzf) fzf does fuzzy matching too.

29 list of miscellaneous commands used by Seneca students

grep, wc, pwd, ls, more, less, file, wget, chmod, vi, vim, hostname, uname, ps, lblk, yum, dnf, rpm, nmcli, ip id, whoami, su, sudo, visudo, grep, selinux (see selinux.org) sestatus, netstat -nr, route, mkdir,

29.1 notes

"route" is an alias for "netstat -nr"

30 Setting console font

30.1 First test interactively using setfont

Easiest is to interactively test the fonts using setfont. The available console fonts are located in the different directories for different distros:

/usr/share/consolefonts (Debian/etc.), /usr/lib/kbd/consolefonts (Fedora, RHEL, CentOS), /usr/share/kbd/consolefonts (openSUSE)…

setfont UniCyr8x14.psf.gz setfont sun12x22.psfu.gz setfont LatGrkCyr-8x16.psfu.gz setfont LatGrkCyr-12x22.psfu.gz

30.2 Make it the default font

30.2.1 console-setup

Possibly edit /etc/default/console-setup and put the following untested lines CHARMAP="UTF-8" CODESET="Lat7" FONTFACE="Terminus" FONTSIZE="28x14"

This example sets the Terminus Bold font at 32 points, and restricts the width to 80 columns.

ACTIVECONSOLES="/dev/tty[1-6]" CHARMAP="UTF-8" CODESET="guess" FONTFACE="TerminusBold" FONTSIZE="16x32" SCREENWIDTH="80"

The FONTFACE and FONTSIZE values come from the font’s filename, TerminusBold32x16.psf.gz.

Yes, you have to know to reverse the order for FONTSIZE. Computers are so much fun. Run setupcon to apply the new configuration. You can see the whole character set for your active font with showconsolefont. Refer to man console-setup for complete options.

30.2.2 vconsole.conf

freedesktop.org

Or, edit /etc/vconsole.conf with FONT="ter-v32n"

Or, edit /etc/fonts/fonts.conf and add: <!– Fallback fonts preference order –> <alias> <family>sans-serif</family> <prefer> <family>Tahoma</family> <family>Arial</family> </prefer> </alias> <alias> <family>serif</family> <prefer> <family>Times New Roman</family> </prefer> </alias> <alias> <family>monospace</family> <prefer> <family>Courier New</family> </prefer> </alias>

<match target="font"> <edit name="antialias" mode="assign"> <bool>false</bool> </edit> <edit name="hinting" mode="assign"> <bool>true</bool> </edit> <edit name="autohint" mode="assign"> <bool>false</bool> </edit> <edit name="hintstyle" mode="assign"> <const>hintfull</const> </edit> </match>

Then updated font cache with fc-cache. Now desired fonts available system wide.

Or,

30.2.3 Systemd

Systemd is different from console-setup, and you don’t need to install anything, except maybe some extra font packages. All you do is edit /etc/vconsole.conf and then reboot. On my Fedora and openSUSE systems I had to install some extra Terminus packages to get the larger sizes as the installed fonts only went up to 16 points, and I wanted 32. This is the contents of /etc/vconsole.conf on both systems:

KEYMAP="us" FONT="ter-v32b"

31 Adding colour to the console

See LSCOLOUR.org file, but basically in your .bashrc include these two lines:

export CLICOLOR=1
export LSCOLORS=ExFxCxDxBxegedabagaced
# or is it
export LS_COLORS='*.txt=01;32:*.sh=31:*.org=01;35:'

I have also seen $LS_COLORS env variable, so check which one works for you.

32 gnome desktop manager

32.1 gnome auto start apps

Enter the app called "tweaks" then add the app you want started automatically, "Startup Applications" You may have to install "tweaks", if so use dnf install gnome-tweaks

32.2 gnome user avatars

within the gnome screen, upper right, click zintis/account settings Then click on avatar and choose a file or a picture. Easy-peasy.

33 kde desktop manager

My gnome was having trouble allowing a login. Even with correct passwords I would catch a glimpse of my desktop, including my auto-start terminal window, but then it would kick me right back to GUI login screen.

I clicked the settings gears on my login screen and tried all options to no avail. So, I installed kde. And now the options gears show a new option cslled "plasma" (which is kde plasma). That works. To se it as a default desktop manager, you can edit ~/.dmrc or also see /usr/share/xsessions

33.1 ~/.dmrc

[Desktop]
Session=kde-plasma

You might haver to add a kde.desktop file if not already in the xsessions directory mentioned above

33.2 ~/.dmrc may be obsolete.

If so, try to specify a default session for a user in :

  • /var/lib/AccountsService/users/*username* file.
    [User]
    Language=
    XSessions=plasma
    

    One user suggested this:

    From dir /usr/share/xsessions move all *.desktop file to another
    directory(unnecessary_env, for example) and leave only one you need(in my
    case - xfce.desktop):
    
    $ ls /usr/share/xsessions
    xfce.desktop  unnecessary_env/
    
    After logout/reboot XFCE will be loaded by default
    
    Note! You won't be able to choose between GUI
    
    
    

33.3 systemd to boot into a GUI (could be obsolete)

exec "usr/bin/startkde" >> ~/.xinitrc startx use systemctl instead.

34 ssh setup

The ssh application is called openssh and openssh-server. Install these both (using yum or dnf). But systemctl sees these both as sshd. i.e. the service is called sshd. systemctl start|stop sshd (or enable|disable)

If not found, have to sudo apt install or sudo dnf install sshd

34.1 sshdconfig

In /etc/ssh/sshd_config

PermitRootLogin without-password is deprecated alias: use prohibit password PermitRootLogin prohibit-password in combination with: PubkeyAuthentication yes

This results in root being able to login, but ONLY with a public key authentication. i.e. NOT with username password.

Interesting that the lab specifically asked to set PermitRootLogin to yes the line before asking to permit root login ONLY using key authentication. —there might be more going on. See man sshd_config that states that prohibit-password means password and keyboard-interactive authentication are disabled for root.

have to restart sshd for these changes to take effect so: sudo systemctl restart sshd (or for older systems service sshd restart

34.2 Additional hardening best practices with sshdconfig

in /etc/ssh there is the file sshd_config (not to be confused with the client config file ssh_config

34.2.1 change port away from 22

this needs work, as for me it id not work making it 727 and then from my client running ssh zinux -p 727 as it gave me an error "conneciton refused"

34.2.2 AddressFamily inet

that is if you want to only support ipv4. otherwise leave it as "any"

34.2.3 PermitRootLogin no

This one is a no-brainer. Do it! Of course you have to create a normal user first, and make sure that user can sudo (visudo needs to be set) and that normal user needs to be part of the wheel group.

34.2.4 PasswordAuthentication no

After this one, you will be locked out unless you have already copied your ssh public key to this server and confirmed that ssh is working using your key.

Note: if you have done this, you won't be able to add the public keys from new hosts, as that process needs to prompt for a password the first time you send the ssh key. So, simply do this:

  1. change this back to yes
  2. restart sshd
  3. from the new host run your ssh-copy-id -i ~/.ssh/id_rsa.pub zinux (at this point the ssh-copy command should prompt for a password.) -i option specifies to look for the identity file in the following file.
  4. confirm that you can ssh without password to zinux
  5. change this to no again.
  6. restart sshd

34.2.5 if 22 does not work, follow the steps to tell SELinux to allow port 727

semanage port -a -t ssh_port_t -p tcp 727 followed by semanage port -l to list all the port policies.

If your server uses port 727, do you have to also specifiy the port 727 when you run the ssh-copy-id command? I think you do. Check the man page on ssh-copy-id command.

34.3 Auto ssh login by pre-sharing your own hosts public key:

You can think of it as sending your password ahead of time to the server you want to connect to. But rather than pre-sending your password, which would obviously be a huge security risk, you send just your public key. The public key will be used by the server to confirm that you are you, as only you, i.e. the possessor of the matching private key, could have sent a session key that was encrypted with the private key. That session key is decrypted using your pre-shared public key, and if it is a match, then the server knows that you are you.

34.3.1 1st generate a public/private key pair:

ssh-keygen -t rsa

That creates two files in ~/.ssh

  • ~/.ssh/id_rsal.pub
  • ~/ssh/id_rsa

The public one is what you share, and that is idrsa1.pub

  • that is very important NOT to share you private key. to ANYONE
  1. Where has the public and private key files gone??

    Also note that you won't see these files using sudo, or even su You have to login to root as root, so su - Then, and only then will you see the idrsa and idrsa.pub files.

    One can SSH to a Linux machine without using a password! This can really speed up your administrative tasks everywhere that Linux servers are used.

    You first must share the public key from the root user in your host machine with the root user of your vm guest machines (vm1, vm2 and vm3)

    Do that by copying the contents of your ~/.ssh/idrsa.pub from your host machine and append to ~/.ssh/authorizedkeys on each of your Virtual Machines. In your case, you will issue the following command once for every vm IPADDR, (or hostname if DNS is setup).

34.4 ssh-copy-id

34.4.1 2nd ssh-copy-id the public key to the remote server root account.

ssh-copy-id -i ~/.ssh/id_rsa.pub root@IPADDR_for_vm ssh-copy-id -i ~/.ssh/id_rsa.pub root@IPADDR_for_vm ssh-copy-id -p 1028 -i ~/.ssh/id_rsa.pub root@IPADDR_for_vm # if not 22

Note: Press ENTER for all prompted information including the password (although this may seen counter-intuitive!) -except of course the login to the remote ssh host the first time. -duh!

This will enter the NULL character for password, so you won't have to enter a password when you ssh using the keys.

ssh-copy-id options:

  • -i look for identity in the following file
  • -p use the ssh port specified rather than the default port ~22
  • -n do a dry run and just tell me the keys that you would copy

Running the ssh-copy-id command will copy the key you specify (make sure it is your public key) and store it in the remote host ~/.ssh/authorized

ssh-copy-id -i id_rsa.pub root@192.168.111.11
ssh root@192.168.111.11

ssh-copy-id -i id_rsa.pub root@192.168.111.12
ssh root@192.168.111.12

ssh-copy-id -i id_rsa.pub root@192.168.111.13
ssh root@192.168.111.13


ssh-copy-id   -i id_rsa.pub  -p 1122 zintis@192.168.11.66

# if your permission was denied by the remote host, make sure you restart
# ssh on the remote host with password login enabled first (it is a chicken
# and egg situtation).  After the public key is copied, you can turn off
# password login again...  See ssh.org

35 ssh for a regular user

35.1 Generate public/private key pair for user

As user:

  • ssh-keygen -t rsa
  • ssh-keygen -t rsa -b 2048
  • ssh-keygen -t rsa -b 4096
  • ssh-keygen -t dsa
  • ssh-keygen -t ecdsa -b 521
  • ssh-keygen -t ed25519

I recommend using rsa. It is the most widely accepted key type.

35.2 Check the fingerprints of your public key: (and private key)

  • ssh-keygen -l -f <filename> # i.e. in my case ~/.ssh/idrsa.pub
  • ssh-keygen -lv -f <filename>

The -l option is to list fingerprint.

35.3 List all fingerprints (random art format) of all known hosts

  • ssh-keygen -lv -f ~/.ssh/known_hosts
  • ssh-keygen -lv -f ~/.ssh/id_rsa.pub
  • ssh-keygen -lv -f ~/.ssh/id_rsa

36 ssh trick to make your life easier

You can ssh into a remote linux host without using a password if that remote host has a copy of your public key. Then that remote host can use your public key to confirm that you are who you say you are by challenging you an encryption messages, encrypted with your public key.

No one but the holder of the secret key will be able to un-encrypt this challenge and respond back correctly, so the remote host knows it is you.

Lots happens in a hand-shake that is hidden from the users view. All a user has to do is follow these steps:

  1. generate the public key - private key pair
  2. copy the public key into remote host, specifically append it to ~/.ssh/authorized_keys

    The actual copy could be done using:

    • ssh-copy-id
    • ssh-copy-id

    See the man ssh-copy-id, but essentially it is a custom command for doing exactly this one thing, copying public keys to remote hosts to allow password-less login to those remote hosts.

  3. use ssh to the remote host. BINGO

37 Openssh compatibility with older ssh servers/routers

If you get an error such as:

The fix lies with the fact that openssh by default disables older, less secure algorithms in their ssh configs. This can be overridden as clearly detailed on openssh.com: http://www.openssh.com/legacy.html

As did this site: support.tibco.com

It is a common problem while the encrypted channel is being set up. If your system and the remote system don't share at least one cipher, there is no cipher to agree on and no encrypted channel is possible. Usually SSH servers will offer a small handful of different ciphers in order to cater to different clients. You can fix it manually or more permanently:

37.1 Manual override ssh key exchange

So I had success with command-line options to override the supported handshake

  • sudo ssh -oKexAlgorithms=+diffie-hellman-group-exchange-sha1 ansible@r0 (of course I had previously set r0 in the /etc/hosts file on the Alpine server on CML

37.2 Semi-permanent fix

You could also fix it more permanently (for yourself only) by adding this to your ~/.ssh/config file:

Host 10.255.252.1
  Ciphers 3des-cbc
  KexAlgorithms +diffie-hellman-group-exchange-sha1

If you usually login to the same user on a particular host you can also add the userid like this:

Host 192.168.111.33
  Ciphers 3des-cbc
  KexAlgorithms +diffie-hellman-group-exchange-sha1
  User ansible

37.3 Yet another learning moment

The above did not work when I added that to /etc/ssh/sshconfig That is because this config was ONLY offering support for the 3des-cbc cipher. The error message was clear so that the fix was easy: Unable to negotiate with 192.168.111.38 port 22: no matching cipher found. Their offer: aes128-ctr,aes192-ctr,aes256-ctr

So I changed the above host specific sections to this:

Host r0
  Ciphers 3des-cbc,aes128-ctr,aes192-ctr,aes256-ctr
  KexAlgorithms +diffie-hellman-group-exchange-sha1
  User ansible

Host r1
  Ciphers 3des-cbc,aes128-ctr,aes192-ctr,aes256-ctr
  KexAlgorithms +diffie-hellman-group-exchange-sha1
  User ansible

Host r2
  Ciphers 3des-cbc,aes128-ctr,aes192-ctr,aes256-ctr
  KexAlgorithms +diffie-hellman-group-exchange-sha1
  User ansible

Host r3
  Ciphers 3des-cbc,aes128-ctr,aes192-ctr,aes256-ctr
  KexAlgorithms +diffie-hellman-group-exchange-sha1
  User ansible

Host r4
  Ciphers 3des-cbc,aes128-ctr,aes192-ctr,aes256-ctr
  KexAlgorithms +diffie-hellman-group-exchange-sha1
  User ansible

Host r5
  Ciphers 3des-cbc,aes128-ctr,aes192-ctr,aes256-ctr
  KexAlgorithms +diffie-hellman-group-exchange-sha1
  User ansible

Next I should clean that up under Host * section.

38 scp

Copy file from a remote host to local host SCP example: $ scp username@fromhost:file.txt /local/directory

scp zintis@zintis.net:/home/zintis/fix-selinux-wordpress.log  wp.log

Copy file from local host to a remote host SCP example:

# cd to where the file is located, assuming file is "file.txt"
scp        file.txt username@to_host:/remote/directory/
scp        file.txt username@to_host:/remote/directory/ 2>scp-errors.txt
scp -P 741 file.txt username@to_host:/remote/directory/
 # if ssh port is 741, not 22

39 File Descriptors

File descriptors are simply positive integers that represent your open files. In unix, everything is a file, and you may have many open files, including where a process sends its output and error messages. These are called standard file descriptors as discussed next.

40 Standard File Descriptors

There are two standard output file descriptors

  • 0 is stdin
  • 1 is stdout
  • 2 is stderr
  • & all outputs

By default both are the terminal where the command was entered.

41 Redirecting Output, stdout, stderr, stdin

Redirecting is done with the > operator, and the < operator, for stdout and stdin respectively. So, before a command is executed, its input, output, errors can be all redirected from the normal stdin, stdout and stderr. Redirectors are processed by the shell in the order that they appear, left to right.

41.0.1 stdout to a file > or 1>

ls -l > long-list-output.txt # redirects stdout to a file. Since 1 is the file descriptor for stdout as well, leaving the file descriptor out is just a shortcut to 1> So these two are identical:

ls -lart >  long-list-output.txt
ls -lart 1> long-list-output.txt

41.0.2 stderr to a file, 2>

grep da * 2> grep-output-errors.txt # redirects errors, 2 to a file

41.0.3 both stdout and stderr to the same file, 2>&1

When running a script from a cron job, i.e NOT from the standard terminal you can redirect &1 and &2 to some other location, typically a file, and often the same file.

  • batchjob.py > ~/batchoutput 2>&1

To break this down:

  • the standard output is redirected, > to the file ~/batchoutput
  • which also is automatically assigned the standard file descriptor 1.
  • By adding the 2>&1 we are also telling bash to also redirect stderr to the same place i.e. redirect 2 to what &1 is, and in this case file ~/batchoutput
  • The &1 is needed when the file redirect is the target. When the file redirect is the source, you do not need the &. &1 is ~/batchouput.

41.0.4 stderr and stdout to the bit bucket &> /dev/null

rm -f $(find /home/zintis -name \*.junk ) &> /dev/null

This will redirect all outputs to /dev/null. You may want to check what it was doing by redirecting both to a file with &> all-outputs.txt

41.0.5 stderr and stdout to a file

rm -f $(find /home/zintis -name \*.junk ) &> /home/zintis/junk-removal.log

41.1 Common errors

2>1 it would simply rediredt standard errors to a file called '1'

job &2>&1 it would run "job" in the background, & There does not have to be a space between the first & and the 2, so that simply takes "2" which is stderr and redirects it to, in this case &1, which is stdout.

42 CentOS/AlmaLinux os-release file

/etc/os-release

Careful making changes here, but it should be something like this:

$ cat /etc/os-release     
NAME=Fedora
VERSION="29 (Workstation Edition)"
ID=fedora
VERSION_ID=29
PLATFORM_ID="platform:f29"
PRETTY_NAME="Fedora 29 (Workstation Edition)"
ANSI_COLOR="0;34"
CPE_NAME="cpe:/o:fedoraproject:fedora:29"
HOME_URL="https://fedoraproject.org/"
SUPPORT_URL="https://fedoraproject.org/wiki/Communicating_and_getting_help"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=rawhide
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=rawhide
PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"
VARIANT="Workstation Edition"
VARIANT_ID=workstation

$ cat /etc/os-release 
NAME="AlmaLinux"
VERSION="8.7 (Stone Smilodon)"
ID="almalinux"
ID_LIKE="rhel centos fedora"
VERSION_ID="8.7"
PLATFORM_ID="platform:el8"
PRETTY_NAME="AlmaLinux 8.7 (Stone Smilodon)"
ANSI_COLOR="0;34"
LOGO="fedora-logo-icon"
CPE_NAME="cpe:/o:almalinux:almalinux:8::baseos"
HOME_URL="https://almalinux.org/"
DOCUMENTATION_URL="https://wiki.almalinux.org/"
BUG_REPORT_URL="https://bugs.almalinux.org/"

ALMALINUX_MANTISBT_PROJECT="AlmaLinux-8"
ALMALINUX_MANTISBT_PROJECT_VERSION="8.7"
REDHAT_SUPPORT_PRODUCT="AlmaLinux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.7"

If your system is not Fedora 29 but with a similar problem, try to replace os-release with a normal one copied from another normally working Fedora of the same version.

43 Cool, low-footprint gadgets

43.1 figlet

figlet zintis

 ______       _   _     
|__  (_)_ __ | |_(_)___ 
  / /| | '_ \| __| / __|_
 / /_| | | | | |_| \__ \
/____|_|_| |_|\__|_|___/

figlet `date +%A\ %b%e` figlet `date +%A\ %b%e` figlet `date +%A\ %b%e`

43.2 command line web browsers

You can set the environment variable BROWSER=lynx and or BROWSER=w3m to open web pages in a command line browser. After that, you can run web searches such as googler or … to run command line web searches.

43.3 googler

Open source Google search from the command line. Make sure your BROWSER env variable is set to a command line browser such as lynx or w3m to get the results on the command line too. See ddgr if you are restricting google for your personal reasons.

  • googler ? will give you the quick instructions.

To install, use dnf install googler as an example.

You can set the BROWSER variable directly on the same command that you run googler, or even set an alias goog='BROWSER=w3m googler -j'

43.4 ddgr

Open source Duck-Duck-Go, ddg search from the command line. Just like with googler, make sure your BROWSER env variable is set and expoerted to a command line browser such as lynx or w3m to get the results on the command line too.

  • googler ? will give you the quick instructions.

To install, use dnf install googler as an example.

You can set the BROWSER variable directly on the same command that you run googler, or even set an alias goog='BROWSER=w3m googler -j'

43.5 surfraw

The oldest command line search tool, available on http://surfraw.org

You can tell surfraw to search a specific search service like google or duck duck go. Surfraw has many, that are listed in elvi from the surfraw.org home page or direclty here: https://gitlab.com/surfraw/Surfraw/-/wikis/current-elvi

BROSWER=w3m surfraw google "figure eight knot"
BROSWER=w3m surfraw duckduckgo "figure eight knot"

44 MacOSX commands (differences)

In MacOSX terminal, there are a few different commands. i.e.

  • shell vairables are different
  • usermod is missing, in its place is dscl see man page.
  • sysadminctl -h for adding users and changing shells etc.
  • shell as a command does not exist on Mac OSX. Instead you change the shell using chsh /bin/bash or chsh /bin/zsh
  • echo $0 to see what shell you are currently running
  • echo $SHELL to see what shell you are currently running (both work)

45 zsh

See link to zsh.org

46 fzf Fuzzy Find

fzf is simply a unix filter. Unix filters take some list as STDIN input, filter this list, then produces output to STDOUT. grep for example is also a unix filter, for example history | grep systemctl will output to STDOUT the history output that is filtered for lines containing "systemctl".

What is special about fzf is that the filter process is interactive where you have to opportunity to interactively define what the filter will select, and when completed with hammer sends the results to STDOUT. So the above example could be history | fzf and the the user starts typing "systemctl" and when the appropriate line is selected, hammer will send only that line to STDOUT

That is it in its simplest form. A unix filter.

Instead of just using printenv, try piping it into fzf with printenv | fzf Then once displayed, just start typing and fzf will search for those strings in the output of printenv Very useful! The search is case insensitive, i.e. fuzzy If you want to restrict the search to consecutive characters, case insensitive searches, start the search with tic, i.e. 'home

Putting it all together: printenv | fzf ... then 'path will search for all path and PATH strings in your printenv.

46.1 installing

I installed it on my mac with:

  • brew install fzf or
  • brew --cask install fzf

Note, that this installs fzf as well as the related fzf-tmux. More on that later.

46.2 fzf environment variables

There is no .config file for fzf. All preferences are stored in environment variables.

   FD_OPTIONS="--follow --exclude .git --exclude node_modules --hidden"  
   # no need to export FD_OPTIONS as they are only used in this file itself
   export FZF_DEFAULT_OPTS="--height 50% -1 --reverse --multi --inline-info --preview='[[ \$(file --mime {}) =~ --binary ]] && echo {} is a binary file || (bat --style=numbers --color=always {} || cat {}) 2> /dev/null | head -300 --prview-window='right:hidden:wrap' --bind='f3: execute(bat --style=numbers {} || less -f {}),f2:toggle-preview,ctrl-d:half-page-down,ctrl-u:half-page-up,ctrl-a:select-all+accept,ctrl-y:execute-silent(echo {+} | pbcopy)'"
   export FZF_DEFAULT_OPTS='--no-height --color=bg+:#eeee00,gutter:-1,pointer:#fd3b00,info:#dada00,hl:#11eded,h1+#22dfff'
   export FZF_DEFAULT_COMMAND='find * -type f -not -path "*/\.*"'
   export FZF_DEFAULT_COMMAND="fd --type f --type l $FD_OPTIONS "
   export FZF_DEFAULT_COMMAND="git ls-files --cached --others --exclude-standard | fd --type f --type l $FD_OPTIONS "
   export FZF_DEFAULT_COMMAND="git ls-files --cached --others --exclude-standard | fd --type f --type l $FD_OPTIONS "
   #The above will use git-ls-files inside a git repo, otherwise will use fd

   # --color=bg+:#eeee00,gutter:-1,pointer:#fd3b3c,info:#dada05'


   # C-t searches files and directories (works nicely)
   export FZF_CTRL_T_COMMAND="fd $FD_OPTIONS"
   export FZF_CTRL_T_OPTS="--preview 'cat --color=always --line-range :60 {}' "
#  export FZF_CTRL_T_OPTS="--preview 'cat --color=always --line-range :60 {}' "
#  export FZF_CTRL_T_OPTS="--preview 'less {}' "

#  M-c  = cd.  Searches only directories, and then cd's into them (works nicely)
#  export FZF_ALT_C_COMMAND='find * -type f -not -path "*/\.*"'
   export FZF_ALT_C_COMMAND="fd --type d $FD_OPTIONS "
   export FZF_ALT_C_OPTS="--preview 'tree -C {} | head -60'"

   export BAT_PAGER="less -R"


Other colour settings can be set just when you execute fzf as follows:

export FZF_DEFAULT_OPTS="--color=fg:#ffffff,bg:#252c31,hl:#c678dd,fg+:#ffffff,bg+:#4b5263,hl+:#d858fe,info:#98c379,prompt:#61afef,pointer:#be5046,marker:#e5c07b,spinner:#61afef,header:#61afef"
fzf --color=fg:#ffffff,bg:#252c31,hl:#c678dd,fg+:#ffffff,bg+:#4b5263,hl+:#d858fe,info:#98c379,prompt:#61afef,pointer:#be5046,marker:#e5c07b,spinner:#61afef,header:#61afef
env fzf --color=fg:#ff0000,bg:#ffff00
export FZF_DEFAULT_OPTS="--color=fg:#ffffff,bg:#252c31,hl:#c678dd,fg+:#ffffff,bg+:#4b5263,hl+:#d858fe,info:#98c379,prompt:#61afef,pointer:#be5046,marker:#e5c07b,spinner:#61afef,header:#61afef"


If I'm not mistaken, terminal.app doesn't support 24-bit colors (https://github.com/termstandard/colors#not-supporting-truecolor),

  • fzf –color=fg:#ff0000,bg:#ffff00

will not work as expected. But it supports 256 ANSI colors fine, you can get identical colors with

  • fzf –color=fg:196,bg:226

I was getting:

invalid color specification: h1+#22dfff

And I found my ~/.config/zsh/zsh-exports file had this line:

If I removed the characters: ,hl:#11eded,h1+#22dfff' fzf would work but with colour codes not translated to proper colours but left as

46.3 Fzf key bindings

When installed, fzf sets up these three default key bindings. They can be changed using environment variables.

  • C-r searches history
  • C-t searches for both files and directories
  • M-c searches for directories and switches (cd) into them

These fzf defaults did not work for me as they conflicted with other key bindings I had already set. Maybe I will return to this later.

46.4 Use fzf with bat for preview

You can use the brew install bat to preview files found with fd

In fact you can change your default fzf command from find to fd by setting the environment variable, FZF_DEFAULT_COMMAND TO BE 'fd

46.5 Using fzf

46.6 fzf options

Then tried these fzf options to learn what they would do:

  • echo 'red\ngreen\nblue'
  • echo 'red\ngreen\nblue' | fzf # use the up and down arrow keys and hammer
  • echo 'red\ngreen\nblue' | fzf --multi # now add TAB key on several lines
  • echo 'red\ngreen\nblue' | fzf --multi --cycle # now keep pressing up/down
  • -e restrict finds to exact matches only (like forcing a ')

46.7 interacting with fzf

While interacting with fzf, you can be quite flexible in what fzf will eventually filter for you. Here's the most useful:

46.7.1 hammer to select a single line

With your cursor on a line, hammer will select that line, and end fzf with that line sent to STDOUT.

46.7.2 <TAB> to select a line (–multi)

~TAB is actually is a toggle, so unselects as well. You need to have run fzf with the --multi or -m option, to select multiple lines. All selected lines will be sent to STDOUT when you hammer.

46.7.3 regexp supported in fzf

By default all regexp are supported by fzf, so if you search

  • home$ fzf will find all lines or items ending with "home".
  • ^home will find all lines beginning with "home" or "hxoxmxe" etc.
  • !home will find all lines NOT CONTAINING "home"
  • home$ | .config$ will find all files ending with "home" or ".config"
  • 'home will find exact matches of "home".
  • !.mp3$ items not ending in .mp3

46.7.4 glob patterns do NOT function

This is because fzf is already running glob pattern searches, so it does not make sense to use * character. i.e. *.txt will not work.

46.7.5 multiple strings separated by space

If you search for home png$ it will find all lines that have home and also end in png It is a logical OR not an AND, so all lines matching either string will be displayed.

46.7.6 the order fzf displays matches

As you type strings into fzf, exact matches will appear 1st, with more fuzzy matches appearing next. The matched strings (or substrings) will be coloured, according to your chosed colour scheme set in env. varialbes.

46.7.7 ' to restrict to exact matches

Prefixing your string with a tic, will force fzf to only show exact matches.

46.8 using fzf to search for files

With no parameters, fzf defaults to using a list of all files in the current directory, including all subdirectories.

So, vim -R | fzf will let you search what file you want to edit.

46.9 piping fzf STDOUT into xargs

By piping the output of fzf into xargs, you can take action on your selected lines very easily.

fzf -m | xargs ls -la
fzf -m | xargs chmod 740

this xargs works with fzf, but does NOT work with –multi ??? Find out why.

46.10 treating fzf stdout as arguments requires use of the back-tic

emacs `fzf`
vim `fzf`

46.11 alternatively to using back-tics is ** <tab>

Use of the ** as arguments to a command is also very convenient, and most likely will be more useful than using back-tics as described above.

Usage examples would be:

  • vim **<tab> # should only present files that I can edit
  • chmod 740 **<tab> # should only present files that I can chmod
  • ssh **<tab> # only presents hosts in my .sshhosts file.
  • git add C-t # only finds files that are not part of .gitignore. test this.

this was not working for me…. need to figure out why. vim kind of worked, and allowed me to fuzzy find a file, but hammer just echoed that line out, and had dropped the "vim". hmmmm …..

also git add C-t while it did present all files for me to fuzzy find, it did NOT restrict those fuzzy files found to ones that needed to be git added. AFAIK, this is broken. See this youtube video on this feature. Maybe the dude has a .zshrc file or git page that has his settings for me to compare with mine, to see why mine is not working.

** is also command sensitive, and will present find options that make sense with the command it is used with. For example, cd only makes sense if the parameter after is a directory, so fzf will only present directories.

This too was not working for me. I had the zsh tell me "too many parameters".

46.12 piping find into fzf

You can use the tradtional find command and pipe it into fzf for a much more powerfull find:

# find everthing: *
# type is a file: -type f
# no hidden files: -not -path '*/\.*'

find * -type f -not -path '*/\.*' | fzf

You will see that the find show a counter of the thousands of files found so far. Just start typing a string, and very quickly you will show all files names that contain that string.

46.13 Default fzf find files

Finding files is so common that the default fzf (no parameters) is actually equivalent to find * -type f | fzf.

However, you can choose to make this an alias, or even better set up your default FZF command to use the find or perhaps fd to just what you like to see. See the section fzf environment variables above.

46.14 fzf and functions

You can create functions and run them in your .bashprofile, or just source them from a file, say fzf-functions Note that this itself is not a bash script, as you can easily source them when you need them during an interactive bash shell session.

# fd - cd to selected directory
fd() {
  local dir
  dir=$(find ${1:-.} -path '*/\.*' -prune \
                  -o -type d -print 2> /dev/null | fzf +m) &&
  cd "$dir"
}
# fh - search in your command history and execute selected command
fh() {
  eval $( ([ -n "$ZSH_NAME" ] && fc -l 1 || history) | fzf +s --tac | sed 's/ *[0-9]* *//')
}

46.15 C-t and M-c and C-r

46.15.1 C-t

When fzf shows you the filtered results, hitting hammer will simply type that line into your command line. By itself that does nothing.

So what you would normally do is type vim -R C-t then when you have found the file you want, hitting hammer will be like hitting vim -R file hammer

or also mv C-t subdirectory will move whatever you found into that subdir.

My M-c did not work as it did in this youtube video. I will investigate later.

46.15.2 M-c cd into fuzzy searched sub-directory

  • M-c cd into searched for directories

46.15.3 C-r fuzzy search history

  • C-r searches history

46.15.4 C-t fuzzy search files and directory

  • C-t searches for both files and directories

A common use case is running a command on it, so chmod 740 C-t hammer You will see the resulting list on your command line so you will have to hammer again. Of course this allows you to still edit the command or list to fit your needs.

  • chmod 740 C-t
  • chmod 740 C-t
  • chmod 740 C-t
  • rm C-t can select multiple files to remove, and hammer, hammer to completed
  • open C-t then .png$ and hammer to open an image file. easy peasy.

If you have your preview set up properly, like:

  • export FZF_CTRL_T_OPTS="--preview 'bat --color=always --line-range :60 {}'"

Then you can even check what files you will be selecting, right in fsf.

46.16 Limit directory depth

This is not done in fzf, but in the find or fd commands.

For instance, usually I'll have my options set like this:

FD_OPTIONS="--follow --exclude .git --exclude node_modules --hidden"  

export FZF_DEFAULT_COMMAND="fd --type f --type l $FD_OPTIONS "

export FZF_CTRL_T_COMMAND="fd $FD_OPTIONS"
export FZF_CTRL_T_OPTS="--preview 'cat --color=always --line-range :60 {}' "

46.16.1 fd –max-depth 1

I can change this by adding -d 1 to limit to current directory. a.k.a. --max-depth and update the environment variables.

46.16.2 find -maxdepth 1

I can change find also with -maxdepth 1

Or, I can do that temporarily like this:

printenv | fzf <fzf to find the environment variable I want to change>

then change it, for this zsh session ONLY. export FZF_CTRL_T_COMMAND="fd --follow --exclude .git --exclude node_modules --hidden -d 1"

This worked, but maybe I can make it more streamlined…

46.17 cd vs M-c

You can use M-c as shown above, but you can also simply use the cd command like this:

  • cd $(find . -type d -print | fzf)

46.18 Home