Cheat sheet on Linux for users
1 What Linux am I running?
You can issue uname -a to see what version of the Linux kernel you are running
To see what distribution depends on the distribution, so chicken and egg issue
Best to try each of them. uname -a
or also uname -or
1.1 hostnamectl
This is my favourite way, and seems to be available on many different distros
hostnamectl
You can also change your name with hostnamectl set-hostname
1.2 If availabe try:
cat /etc/os-release
1.3 Redhat, CentOS, Fedora
cat /etc/redhat-release cat /etc/centos-release cat /etc/fedora-release
1.4 Debian
cat /etc/debian_version
1.5 Ubuntu
cat /etc/lsb-release
1.6 Gentoo
cat /etc/gentoo_release
2 Customizing your shell
2.1 .bashrc
vs .bash_profile
From mitre.org,
~/.bash_profile
and ~/.bashrc
are shell scripts that contain shell commands.
These files are executed in a user's context when a new shell opens or when
a user logs in so that their environment is set correctly. ~/.bash_profile
is executed for login shells
and ~/.bashrc
is executed for interactive
non-login shells.
This means that when a user logs in (via username and password) to the console
(either locally or remotely via something like SSH), the ~/.bash_profile
script is executed before the initial command prompt is returned to the user.
Subsequently, every time a new shell is opened
, the ~/.bashrc
script is
executed. This allows users more fine-grained control over when they want
certain commands executed. These shell scripts are meant to be written to by
the local user to configure their own environment.
The macOS Terminal.app
is a little different in that it runs a login shell
by
default each time a new terminal window is opened, thus calling
~/.bash_profile
each time instead of ~/.bashrc
.
2.1.1 .bashrc
runs each time a new terminal is opened
2.1.2 .bash_profile
runs only when a user logs in
2.2 .bashrc
You can edit this in your root directory and add after which you save the
changes, exit, then run
source .bashrc
to re-read and initialize the new .bashrc
2.3 .bashrc
aliases
alias [name[='value']]
No spaces allowed next the = sign. If value has spaces, it must be enclosed in "" marks.
alias[name[=value]] alias lst='ls -lart' alias font22='setfont LatGrkCyr-12x22.psfu.gz' alias font16='setfont LatGrkCyr-8x16.psfu.gz' alias ip4='nmcli -f ip4 device show' alias ip4='nmcli -f ip4 device show' alias skip='sed "/^$/d ; /^ *#/d"' # Strip blanks and comments from files.
ls -S
is sort by size, but I would not make that an alias.
2.3.1 Bypass the alias with a \
If ls
is set to an alias like ls -la
\ls
will actually get you an ls output
, and NOT the ls -la
output. You are
basically temporarily telling bash to ignore the alias setting.
2.3.2 single or double quotation marks
The choice of single
or double quotation marks
is significant in the alias
syntax
when the alias includes variables.
If you enclose value within double quotation marks, any variables that
appear in value are expanded when the alias is created.
If you enclose value within single quotation marks, variables are not
expanded until the alias is used
.
The following example illustrates the difference.
echo $PWD /home/max alias dirA="echo Working directory is $PWD" alias dirA alias dirA='echo Working directory is /home/max' alias dirB='echo Working directory is $PWD' alias dirB alias dirB='echo Working directory is $PWD'
2.3.3 unalias
To remove the alias for this session.
2.3.4 aliases that accept parameters
You will probably have the need at some point to have an alias accept one or more parameters. You cannot. Aliases do not accept any parameters from the command line.
The fix is the create a bash function
as descibed in Bash functions.
For example, to pass one or more arguments to the function, just call the
function
with the arguments separated by spaces. For example:
mult 55 2.54
on the command line will multiply the two numbers.
While creating/writing the function, the arguments are assigned to variables $1, $2, $3, $4 etc…
So my mult
function would look like this:
mult() { echo "$1" * "$2" }
A good approach is to create a script called functions.sh
( in the ~/bin
directory for example). In this script, you add all your personal function
definitions, and every time you add a function you add it to this file,
Finally you just have to add the source ~/bin/functions.sh
line to your .bashrc
file. This way you will be able to call them from the command line, your .bashrc
will stay clean, and you will have a specific place for your personal functions.
2.4 Customizing the prompt
export PS1="Monty > " # only display Monty > export PS1="\u " # only display the username export PS1="\u@\h " # display the username and host export PS1="\u@\H " # display the username and FQDN export PS1="\u@\h\\$ " # display the username@host $ export PS1="\u@\h\ \s \\$ " # display the username@host >bash $ export PS1="\u@\h\ \s\v \\$ " # display the username@host >bash4.4 $ export PS1="\w \\$" # display only the current path and $ export PS1="\W" # will remove the first / in the path displayed export PS1="[\!] $" # show the history line number i.e. [15] $
The export command sets the value of the TERM variable to xterm-color. In addition, xterm-color is a terminal type that supports colorization. We can also interpret the other parts of the script:
PS1:
defines the shell prompt
\u
: represents the username
\h
: represents the hostname
\w
: represents the current working directory
\[\e[31m\]
: changes the text to red
\[\e[0m\]
: sets the color to default
2.4.1 Adding colour to the prompt
see LSCOLORS in this doc and LSCOLORS.org file. This section is on colouring the prompt itself. I believe this applies to the ls command.?
Good info available here howtogeek.com of which I have extracted some paragraphs:
You must include the entire color code
information between the \[
and \]
characters. Inside the tag, you must begin with either \033[
or \e[
to indicate to Bash that this is color information
.
Both \033[
and \e[
do the same thing. \e[
is shorter so might be more
convenient to use, but we’ll use \033[
here as it matches what’s used by
default
.
You must indicate the end of a color tag
with m\
So, every color tag will look like: \[\033[COLORm\]
where COLOR is:
- 30: Black
- 31: Red
- 32: Green
- 33: Yellow
- 34: Blue
- 35: Purple
- 36: Cyan
- 37: White
So:
\[\033[30m\] : Black \[\033[31m\] : Red \[\033[32m\] : Green \[\033[33m\] : Yellow \[\033[34m\] : Blue \[\033[35m\] : Purple \[\033[36m\] : Cyan \[\033[37m\] : White
Specify color code 00
to clear the color
information. i.e. \[\033[00m\]
Bash also allows you to change the color of foreground text, add attributes like “bold” or “underline” to the text, and set a background color.
Attributes are added before the color number So, every attribute color tag will look like:
\[\033[ATTRIBUTE;COLORm\]
Where ATTRIBUTE is:
0
Normal text ( the default )1
Bold or light text2
Dim text3
4
Underlined text5
Blinking text7
Reversed text8
Hidden text.
Background colours cannot have attributes, but are:
40
Black41
Red42
Green43
Yellow44
Blue45
Purple46
Cyan47
White
So just changing the background colour to purple would be: \[\033[44m\]
You can specify both foreground and background color tags. For example, 42 represents a green background and 31 represents red text. So, to make the default prompt become red text on a green background, you’d use:
PS1="\[\033[42m\]\[\033[31m\]\u@\h:\w\$ "
My current favourites for bash:
export PS1="\[\033[01;31m\]\u@\h \w\[\033[01;32m\]$ \[\033[00m\]"
or flip it as :
Yellow: 33
For example, since purple text
is color code 32
, you’d use \[\033[32m\]
for purple text.
My current favourites for zsh:
export PS1="\[\033[01;33m\]\$PWD\[\`if [[ \$? = "0" ]]; then echo '\e[32m[\!]\e[0m'; else echo '\e[31m[\!]\e[0m' ; fi\`:\n\$ "
which will colour the history count, [\!] , red if your previous command
returned an error.
- Colour attributes
You can also specify an attribute for the text. This attribute must be added before the color number, separated by a semicolon
;
Text with these attributes will look different in different terminal emulators.Here are the values for text attributes:
Normal Text: 0 Bold or Light Text: 1 (It depends on the terminal emulator.) Dim Text: 2 Underlined Text: 4 Blinking Text: 5 (This does not work in most terminal emulators.) Reversed Text: 7 (This inverts the foreground and background colors, so you’ll see black text on a white background if the current text is white text on a black background.) Hidden Text: 8
You don’t actually need to include the normal text attribute. That’s the default, anyway.
For example, since red text is code 31 and bold text is code 1, you’d use
\[\033[1;31m\]
for bold red text.You can also specify a background color, but you can’t add an attribute to a background color.
- Background colours
Here are the values for background colors:
Black background: 40 Blue background: 44 Cyan background: 46 Green background: 42 Purple background: 45 Red background: 41 White background: 47 Yellow background: 43
For example, since a
blue background
is code44
,\[\033[44m\]
would specify a blue background. - Specifying both
You can specify both foreground and background color tags. For example,
42 represents a green background and 31 represents red text. So, to make
the default prompt become red text on a green background, you’d use:
PS1="\[\033[42m\]\[\033[31m\]\u@\h:\w\$ "
We just specify a single background color and then a single foreground text color here, which begins at the start of the prompt and is applied to all text in the prompt. However, you can specify as many color tags as you want in the variable to color different sections of your prompt however you like.
- add 00 to reset after the prompt
The background and foreground text colors keep going past the prompt unless you specify color code
00
or more precisely\[\033[00m\]
to clear the color information. You can also use this tag within the variable toreset formatting back to default
somewhere in your prompt. For example, the following line wouldend all
coloring
before the \$ character.PS1="\[\033[42m\]\[\033[31m\]\u@\h:\w\\[\033[00m\]\$ "
2.5 .cshrc
You can edit this in your root directory and add after which you save the
changes, exit, then run
source .cshrc
to re-read and initialize the new .cshrc
2.6 Precedence on Colors
There are multiple ways to set colors: LSCOLORS env variable LSCOLORS env variable
In zsh using:
PROMPT="%B%{$fg[yellow]%}%d%(?:%{$fgbold[green]%}[%h]:%{$fgbold[red]%}[%h]) $%{$resetcolor"
In bash shell using ANSI escape codes \e
or \033
2.7 focus follows mouse
This is controlled by gnome settings. Per session settings are:
gsettings set org.gnome.desktop.wm.preferences focus-mode 'mouse'
So save this permanently? Apparently the gsettings above IS permanently saved; so no need to do anything else.
3 BASH commands
3.1 uptime
Two very useful commands related to uptime are last
and who -b
So together
they are:
uptime
shows time since last reboot. Alsouptime -s
last
will show you all the users that have logged in and when as well as when and how many times the system was rebooted. Can also uselast reboot
last -x
will also show you reboots and other system level changes.who -b
just shows the last time the system was rebooted.
3.2 $ Kill -l
This lists the kill signals used by the system. May be slightly different from distro to distro, or linux to mac OSX, BSD etc.
3.3 killing a stuck tty session
from another tty, issue the command who
Note the tty session, e.g. pts/0 (psuedo terminal ssh 0 )
ps -ef | grep pts
to see the associated process
kill processid
# of the tty process.
To kill all processess that contain "brave" killall brave
3.4 tree
other tools similar to tree are broot
3.5 cut
cut -d" " -f 1
# means use a space
as the delimiter and return field 1.
cut -d| -f 3-4
# means use a "|
" as the delimiter and return fields 3 & 4.
cut -f 2,4-6
# means use a tab
as the delimiter and return field 2, 4 to 6
cut -c 2
# means use a tab
as the delimiter and return field 2.
3.6 rename
rename files
can rename files, replacing all occurances of strng1 with strng2.
for example renamin all occurances of "ciscospark"
with "webexapi"
you could
rename 's/ciscospark/webexapi/' *.py
should work on linux, but not macosxsed
for macosx should do.
for f in ./*.*; do cp $f ciscospark-$f; # this did not work, but is a naive way to try. done for file in ACDC*.xxx; do mv "$file" "${file//ACDC/AC-DC}" done for file in webex-*.py; do mv "$file" "${file//webex/ciscospark}" done # including subdirectories find . -type f -name webex\*.py -print0 | while read -r -d '' file; do mv "$file" "${file//webex/ciscospark}" done find . -type f -name "ACDC*" -print0 | while read -d $'\0' f do new=`echo "$f" | sed -e "s/webex/ciscospark/"` mv "$f" "$new" done
4 Logical Operators, &&, ||
x && y
==> =y only if x true
i.e x had to complete successfully before you do yx || y
==> y only if x false
i.e only do y if x failed
These can often let you avoid using cubmersome if statements in your bash scripts.
Below are details for each type. Remember that $?
is a shell variable that
holds the error condition from the last executed command. 0 if it worked, 1 if
it failed. I've seen 0 if it worked and 127 if not, so be careful and check.
4.1 and operator && only execute the 2nd part if the first PASSED
ansible@c8host ~[1174] $ true && echo "ok it worked" ok it worked ansible@c8host ~[1175] $ false && echo "ok it worked" <nothing was echoed here, but that is hard to show, hence this explanation> ansible@c8host ~[1176] $
That was taken from the MIT missing lecture series on youtube
This is the AND
operator &&
:
The second
command will ONLY run if the first
command succeeded
.
So, here is a better example, good for handling ugly errors:
cat ".bashrc" ; echo "this is my profile" cat ".bashrc" && echo "this is my profile"
- The first choice above will fail with an ugly error if there is no file called .bashrc.
- The second choice will print "this is my profile" only if there really was a profile. Much better.
A common use of this is to set the bash prompt different whenever the last command returned an error:
\[\033[30m\] : Black \[\033[31m\] : Red \[\033[32m\] : Green \[\033[33m\] : Yellow \[\033[34m\] : Blue \[\033[35m\] : Purple \[\033[36m\] : Cyan \[\033[37m\] : White # loops: export PS1='$(if [ $? -eq 0 ]; then printf "\033[01;32m""\xE2\x9C\x93"; else printf "\033[01;31m""\xE2\x9C\x95"; fi) \[\e[00;32m\]\u@\h\[\e[00;30m\]:\[\e[01;33m\]\w\[\e[01;37m\]\$ ' export PS1='... \e[01;31m$(code=${?##0};echo ${code:+[error: ${code}]})\e[00m' # on Linux export PS1="\$([ \$? == 0 ] && echo ✅ || echo ⚠️ ) \[\033[01;36m\]\u\[\033[01;32m\]@\h\w[\!] \[\033[01;31m\]$\[\033[00m\]\n" export PS1="\[\033[01;36m\]\u\[\033[01;32m\]@\h\w[\!] \[\033[01;31m\]$\[\033[00m\]\$([ \$? == 0 ] && echo ✅ || echo ⚠️ )\n" export PS1="\[\033[01;36m\]\u\[\033[01;34m\]@\h\w[\!]\$([ \$? == 0 ] && echo ✅ || echo ⚠️ ) \[\033[01;31m\]$\[\033[00m\]\n" export PS1="\[\033[01;36m\]\u\[\033[01;34m\]@\h\w[\!]\$([ \$? == 0 ] && echo ✅ || echo '\[\033[01;31m\]⚠️ ') \[\033[01;31m\]$\[\033[00m\]\n" export PS1="\[\033[01;36m\]\u\[\033[01;34m\]@\h\w[\!]\$([ \$? == 0 ] && echo ✅ || echo '\[\033[01;31m\]⚠️ ') \[\033[01;31m\]$\[\033[00m\]\n" export PS1="\[\033[01;36m\]\u\[\033[01;34m\]@\h\w[\!]\$([ \$? == 0 ] && echo ✅ || echo '\[\033[01;31m\]⚠️ ') \[\033[01;31m\]$\[\033[00m\]\n" export PS1="\[\033[01;36m\]\u\[\033[01;34m\]@\h\w[\!]\$([ \$? == 0 ] && echo \[\033[01;36m\] ✅ || echo ⚠️ ) \[\033[01;31m\]$\[\033[00m\]\n" export PS1="\[\033[01;36m\]\u\[\033[01;33m\]@\h\w[\!]\$([ \$? == 0 ] && echo '\[\033[01;32m\]✅' || echo '\[\033[01;31m\]⚠️ ') \[\033[01;31m\]$\[\033[00m\]\n" export PS1="\[\033[01;36m\]\u\[\033[01;33m\]@\h\w[\!]\$([ \$? == 0 ] && echo '\[\033[01;32m\]✅' || echo '\[\033[01;31m\]⚠️ ') \[\033[01;31m\]$\[\033[00m\]\n" export PS1="\[\033[01;36m\]\u\[\033[01;33m\]@\h\w\$([ \$? == 0 ] && echo '\[\033[01;32m\][\!] ✅' || echo '\[\033[01;31m\][\!] ⚠️ ')\[\033[00m\]\n" export PS1="\[\033[01;36m\]\u\[\033[01;33m\]@\h\w\$([ \$? == 0 ] && echo '\[\033[01;32m\][\!] ✅' || echo '\[\033[01;31m\][\!] ⚠️ ')\[\033[00m\]\n" export PS1="\[\033[01;36m\]\u\[\033[01;33m\]@\h\w\$([ \$? == 0 ] && echo '\[\033[01;32m\][\!] ✅' || echo '\[\033[01;31m\][\!] ⚠️ ')\[\033[00m\]\n"i # cyan user yellow @host path green red reset colour zintis@vm1~[202] ✅ # on zintis.net export PS1="\[\033[01;34m\]\u@\h.net\[\033[01;33m\] \$PWD\[\`if [[ \$? = "0" ]]; then echo '\e[32m[\!]\e[0m'; else echo '\e[31m[\!]\e[0m' ; fi\`:\n\$ " # on macbook zsh export PS1="\[\033[01;33m\]\$PWD\[\`if [[ \$? = "0" ]]; then echo '\e[32m[\!]\e[0m'; else echo '\e[31m[\!]\e[0m' ; fi\`:\n\$ "
4.1.1 Summarizing x && y ==> y only if x true
4.2 or operator || only execute the 2nd part if the first FAILED
The second choice will print the message ONLY if the first command failed. If the first command succeeded, the second command will NOT run.
The above still might have an ugly error, which you can supress by redirecting
to null, stderr
is &2
, i.e. &2 > /dev/null
ansible@c8host ~[1172] $ false || echo "logical OR says false was false" logical OR says false was false ansible@c8host ~[1173] $ true || echo "logical OR says false was false" ansible@c8host ~[1174] $
In bash shell scripts you can do 'cd .. || exit'
as a safe way to run
a command and if that fails, simply exit the script altogether.
4.2.1 Summarizing x || y ==> y only if x false
4.3 always do the second no matter what, … just concatentate two commands
echo "first" ; echo "second will always print too" The first command completes, before the second command runs.
4.3.1 Summarizing x ; y ==> do x, do y
4.4 &
This is the same as the shell &, i.e. "put this in the background" sleep 5 ; echo "all done." sleep 5 & echo "all done."
The first command will print "all done" only after waiting 5 seconds. the second commnd will print "all done" right away, but the script will still take 5 more seconds to finish (less the time it took to echo all done)
4.5 Example:
- Inellegant
#!/bin/sh if [ "$EDITOR"= "" ]; then EDITOR=vim fi echo $EDITOR
slightly better, but identical is if [ -z "$EDITOR" ]; then
- Ellegant
#!/bin/sh [ -Z "$EDITOR" ] && $EDITOR=vim echo $EDITOR
This reads: "if $EDITOR is equal to null, then assign vim to $EDITOR"
- -Z "$EDITOR" asks does $EDITOR = null??
4.6 Good code fragment example
add) asktype && askinfo && tryconnect && finalize || delete ;;
This will run each of these commands in order, and continue as long as each of them runs successfully. As soon as one fails, all the others will fail, until you get to the || which will run if any of the previous failed. If they all succeeded, then the delete will NOT run.
Slick and effective.
5 Job Control
Any command or script you run can be controlled from the command line with a few
commands, and understanding Linux signals. First to realize is that every
linux process will have a process ID, or PID
. That lets you uniquely identify
each process on your system. PIDs are listed on all commands related to running
jobs, such as top
or htop
and jobs
5.1 Interactive job control (bash)
Depending on which shell you are running, it will contain commands for
interactive job control. For example, in bash, when a user runs an interactive
command or program, you can temporarily
suspend
the program with C-z
/ ^z
followed by the command bg
or background.
When you run ^z
on a running process, then issue the command jobs
you will
see that the state of that job is suspended
. after running bg
the state
will have changed to running
.
5.1.1 &
You can start a job while simultaneously putting it in the background with the
&
sign after the job. For example: tail -f /var/log/messages > temp-messages &
5.1.2 ^y
C-y
is similar to C-z
in that it suspends a process, but C-y
delays suspending
the job until the job itself is waiting for keyboard input. C-z
does not wait
but the trade-off is that any pending output will be dropped.
5.1.3 identifying jobs
See man bash
for a description on how to refer to jobs, particularly the use
of:
%jobname
refers to a stopped jobjobname
$?obnam
refers to a stopped job whose namecontains obnam
%n
refers to a stopped job numberedn
(not to be confused withPID
%%
current job (also%+
is used)%-
previous job
The job number is NOT the PID, rather it is a sequential number given to the
jobs started in this interactive bash session starting from [1] etc. A user
can kill
with a PID; kill 555
, or kill
with a job number; kill %2
fg
for "foreground" brings a job back to the foreground (interactive session).
You can fg
based on a PID
or based on the job number
(more common) A short
cut for fg
is to simply name the jobname or jobnumber. For example
fg %2
%2
Are synonymous and bring job [2] to the foreground.
5.2 Linux Signals
Linux signals are always at your fingertips if you remember man signal
No. | Name | Default Action | Description |
---|---|---|---|
1 | SIGHUP | terminate process | terminal line hangup |
2 | SIGINT | terminate process | interrupt program |
3 | SIGQUIT | create core image | quit program |
4 | SIGILL | create core image | illegal instruction |
5 | SIGTRAP | create core image | trace trap |
6 | SIGABRT | create core image | abort program (formerly SIGIOT) |
7 | SIGEMT | create core image | emulate instruction executed |
8 | SIGFPE | create core image | floating-point exception |
9 | SIGKILL | terminate process | kill program |
10 | SIGBUS | create core image | bus error |
11 | SIGSEGV | create core image | segmentation violation |
12 | SIGSYS | create core image | non-existent system call invoked |
13 | SIGPIPE | terminate process | write on a pipe with no reader |
14 | SIGALRM | terminate process | real-time timer / alarm expired |
15 | SIGTERM | terminate process | software termination signal |
16 | SIGURG | discard signal | urgent condition present on socket |
17 | SIGSTOP |
stop process | stop (cannot be caught or ignored) |
18 | SIGTSTP | stop process | stop signal generated from keyboard |
19 | SIGCONT | discard signal | continue after stop (what bg does) |
20 | SIGCHLD | discard signal | child status has changed |
21 | SIGTTIN | stop process | background read attempted |
from control terminal | |||
22 | SIGTTOU | stop process | background write attempted to |
control terminal | |||
23 | SIGIO | discard signal | I/O is possible on a descriptor |
(see fcntl(2)) | |||
24 | SIGXCPU | terminate process | cpu time limit exceeded |
(see setrlimit(2)) | |||
25 | SIGXFSZ | terminate process | file size limit exceeded |
(see setrlimit(2)) | |||
26 | SIGVTALRM | terminate process | virtual time alarm |
(see setitimer(2)) | |||
27 | SIGPROF | terminate process | profiling timer alarm |
(see setitimer(2)) | |||
28 | SIGWINCH | discard signal | Window size change |
29 | SIGINFO | discard signal | status request from keyboard |
30 | SIGUSR1 | terminate process | User defined signal 1 |
31 | SIGUSR2 | terminate process | User defined signal 2 |
common
Linux also has keyboard shortcuts as well as kill commands for the most common
signals, i.e. kill -TERM 555
to terminate PID 555
See man kill
for a listing
of these signal names, and more information.
No. | Name | Usage name | Key | kill | alt kill |
---|---|---|---|---|---|
command | command | ||||
1 | SIGHUP | HUP (hangup) | C-d | kill -1 |
kill -HUP %n |
2 | SIGINT | INT (interrupt) | C-c | kill -2 |
kill -INT %n |
3 | SIGQUIT | QUIT | C-\ | kill -3 |
kill -QUIT %n |
… | |||||
6 | SIGABRT | ABORT | kill -6 |
kill -ABRT %n |
|
… | |||||
9 | SIGKILL | KILL | kill -9 |
kill -KILL %n |
|
… | |||||
14 | SIGALRM | ALRM (alarm) | kill -15 |
kill -ALRM %n |
|
15 | SIGTERM | TERM (terminate) | kill -15 |
kill -TERM %n |
|
17 | SIGSTOP | STOP (suspend)) | C-z | kill -17 |
kill STOP %n |
… | |||||
19 | SIGCONT | CONT (continue) | bg | bg |
kill -CONT %n |
kill -s TERM %2
and kill -15 %2
and kill -TERM %2
are all equivalent, and all
terminate job 2. And since the default signal is SIGTERM these all are also
equivalent to simply kill %2
And since the default job number is %1, if you
want to kill the
Check out this simple python program that you can use to get familiar with
signals. The program will not stop for most signals, but SIGKILL
will
always override, so that is how you get out of the infinite loop below:
#!/usr/bin/env python import signal, time def handler(signum, time): print(f"\n I got a SIGINT, but will keep going") signal.signal(signal.SIGINT, handler) # infite loop print(" Press ^C i.e. SIGKILL to exit this program") i = 0 while Ture: time.sleep(0.1) print(f"\r{i}", end="") i += 1
SIGKILL should be used with caution when that process has spawned child processes, because SIGKILL will terminate the process and leave the child processes orphaned, and keep running. Will get weird behaviour.
logging out of a session will send a SIGHUP.
5.2.1 nohup
You can start a job and prefix it with nohup
. This will tell the shell that
this job should ignore the SIGHUP
command. Useful if you are running a job
on a remote server and want to logout, but keep the job running. Otherwise
a logout action will hang up all jobs you have started.
nohup tail -f /var/log/messages > ~/messages_for_myself logout
6 Users and shell setup
To change a users default shell, see cat /etc/shells
Or, as a user, to confirm the default shell setting use:
echo $SHELL
Technically that just echos the environment variable $SHELL, but
you could actually be running a shell interactively that is NOT
what is reflected in $SHELL
. see below.
You could also run printenv SHELL
but that already supposes some csh is
being run.
echo $0
will report -bash if you are running bash or tcsh. That is
because $0 is the name of the runnin process. This will NOT work for csh.
If you are running fish (by running /usr/local/bin/fish from the command line) then echo $SHELL will show you /bin/tcsh (because I had set that as the default shell, which obviously is not the shell I am running currently!!. Watch that and don't get caught by this
6.0.1 Check what shell you are running
There are several ways. It is good to know several as there are differences between the shells you are running, and one method may work for one and not the other.
echo $0
Most shells store the current shell in the variable$0
echo $SHELL
Another common variable set to the shell you are runningecho $$
Most shells set the PID of the currently running process to the special variable$$
You can follow up withps -p PID
ps -p $$
does the above in one commandecho $BASH
within a bash shellecho $VERSION
within a tcsh shellif [ -z "$BASH" ]; then echo "Run script $0 with bash please"; exit; fi
Actually$0
is set to the current running process. From within a shell that is obviously the shell itself, but$0
would be the script name from within a running bash script etc. (-z object) means if object exists return true.
6.1 Look of the process (shell) having PID of the current shell instance
The pid of the current instance of shell is echo "$$"
Then, find the process of that pid with ps -p <PID>
Or putting it in a single command: ps -p "$$"
Notice that $$
is NOT the pid in fish. For fish you need to use $fish_pid
.
6.2 chsh -s /usr/bin/bash
This will persistently change the default shell for you. Logout and back in to see the effects.
6.3 Mac OSX (Darwin, 19.6.0)
When I had set my terminal default shell to be /bin/bash, then I got this error when opening a new terminal window:
The default interactive shell is now zsh. To update your account to use zsh, please run `chsh -s /bin/zsh`. For more details, please visit https://support.apple.com/kb/HT208050.
Here is me on Darwin, changing my default shell to bash.
$ chsh -s /bin/bash Changing shell for zintis. Password for zintis: chsh: no changes made
No changes were made because it was already /bin/bash.
6.4 usermod
usermod
is a utility for modifying a user's account details that are store
in /etc/passwd
.
One could just sudo vi /etc/passwd
and not make ANY mistakes, but usermod
is a safer option.
usermod --shell /usr/bin/tcsh zintis
Then confirm by grep zintis /etc/passwd
6.5 chsh
does the same thing usermod does, but only for changing a users shell.
chsh --shell /usr/bin/tcsh zintis
6.5.1 chsh not found
It could be that you have not installed chsh on CentOS. To remedy that, first
find out which dnf utility provides chsh
using this command:
- dnf provides chsh
The output of which will give you this:
Last metadata expiration check: 2:35:04 ago on Thu 22 Apr 2021 10:20:16 AM EDT. util-linux-user-2.32.1-24.el8.x86_64 : libuser based util-linux utilities Repo : @System Matched from: Filename : /usr/bin/chsh util-linux-user-2.32.1-24.el8.x86_64 : libuser based util-linux utilities Repo : baseos Matched from: Filename : /usr/bin/chsh
Then, install it with: dnf install util-linux-user
After that, I found chsh to be in /usr/bin/chsh
and all was good.
6.6 adduser (symlink to useradd, so just use useradd)
useradd zintis passwd zintis usermod -aG wheel zintis useradd zintis -u 1989 # specify the uid for zintis to be 1989 useradd -c "Zintis Perkons" zintis visudo # add zintis below ROOT user privileges section. Make it identical # add into line: Defaults env_reset,timestamp_timeout=120 Defaults env_reset,timestamp_timeout=120 ## Allow root to run any commands anywhere root ALL=(ALL) ALL zintis ALL=(ALL) ALL
6.7 userdel
6.8 groups
From linuxconfig.org Each time a new user is created, automatically a new group with the same name of the user is created too, and the user becomes its sole member. This is what is called a primary or private group.
Each user has its own primary group, named after himself, with no other
members. This setup makes possible to change the default umask
value:
traditionally it was 022
(this means 644 permissions for files and 755 for
directories), now it is usually set to to 002 (664 permissions for files and
775 for directories). (mask is subtracted from 666 for files and 777 for
directories. … -1 is the same as 0)
Since, by default, each file or directory created by a user is created with that user's primary group, this setup, while preserving security (a user can still modify only its own files), simplifies resources sharing and collaboration between users which are members of the same group when the setgid bit is used, by allowing write permissions for the group.
6.8.1 usermod to add a user to an existing group
In general we can add a group, remove a group, change the home directory…
- -G is same as –groups
- -a is append
So, usermod -a -G teamleads zintis
where you need the -a to append.
sudo usermod -a -G existing-groupname username
sudo usermod -a -G wheel zintis
sudo usermod -a -G wheel,operators,engineering zintis
- > this won't work. You have to add the groups one at a time as so:
sudo usermod -a -G wheel zintis
sudo usermod -a -G operators zintis
sudo usermod -a -G engineering zintis
groups zintis
# to show all the groups that have zintis as a member
- usermod -a (append) requires -G (supplementary groups)
this will append the additional groups to a user. For example
sudo usermod -a -G community kulmanis
Compare that to: - usermod -g group sets the primary group for that user
Notice that you cannot delete an undesired group if that group has a user with that group as a primary user. If that happens, reset that user's group to the proper group (typically its own group) for example. =sudo usermod -g braun braun Then you can groupdel undesiredgroup.
6.8.2 Adding a user to an existing group by editing /etc/group & /etc/gshadow
Use the command similar to visudo, called vigr vigr -s is for modifying the /etc/gshadow file. (will have to w! to override the read-only aspect of this file. That is normal.)
Then change the lines of the groups you want to add the user as so:
group-name:group-passwd:group-id:users
where users can be a comma
separated list of users.
- Alternatively use grpconv
grpconv will keep the /etc/group and /etc/gshadow files in sync. It will create a new /etc/gshadow file from the /etc/group file, clobbering the existing one.
sudo grpconv
Then you can check / verify that the files are consistent by:
sudo grpck
… and see no output. This is good.
6.8.3 display the group ownership of a files
getent group <group-id, or group-name>
getent group <group-id, or group-name>
getent group <group-id, or group-name>
getent group <group-id, or group-name>
getent group
# will show all of themawk -F':' '/gawlergrp/{print $4}' /etc/group
6.9 some homegrown way of changing group ownership
6.9.1 change ALL files in a folder to a given group:
find . -type f -exec chown <owner>.<group> {} \;
6.9.2 change ALL directories in a folder to a given group
find . -type d -exec chown <owner>.<group> {} \;
6.10 The RIGHT way is to use chgrp recursive function, -R
chown -R owner:group * .[^.]*
6.11 groupadd
This adds a new group to /etc/group
6.12 Ensuring new files in a directory have the proper group
chgrp super kaapvall/ sarmation/ chmod g+s kaapvall/ sarmation/
This forces new files created inside the directories kaapvall and sarmation to have the group ownership "super" instead of the user's group.
6.13 filesystem ACLs
To set the default group permissions, you will have to use ACLs. Set a "default" ACL:
setfacl -m "default:group::rwx" /var/www
To also adjust subdirectories:
find /var/www -type d -exec setfacl -m d:g::rwx {} +
Note: The file system must have ACL support enabled. Sometimes it is on by
default; on ext3
or ext4
you might get "Operation not supported"
, in which
case it must be enabled manually:
For a currently mounted filesystem: mount -o remount,acl /
Permanently – one of the methods below:
- at fstab level: edit
/etc/fstab
to have acl in the options field - at filesystem level:
tune2fs -o acl /dev/diskname
7 setuid / setgid / sticky
Just like normal permissions, the special bits can be assigned with the chmod
command, using the numeric
or the ugo/rwx
format. In the former case the
setuid
, setgid
, and sticky
bits are represented respectively by a value of 4
,
2
and 1
.
So for example if we want to set the setgid bit on a directory we would
execute: chmod 2775 test
With this command we set the setgid
bit on the
directory, (identified by the first of the four numbers), and gave full
privileges
on it to it's owner
and to the user that are members of the group
the directory belongs to, plus read and execute permission for all the other
users (remember the execute bit on a directory means that a user is able to cd
into it or use ls to list its content).
The other way we can set the special permissions bits is to use the ugo/rwx
syntax: chmod g+s test
. To apply the setuid bit
to a file, we would have run:
$ chmod u+s file
While to apply the sticky bit: $ chmod o+t test
setuid | setgid | sticky | |
---|---|---|---|
chmod u+s |
chmod g+s |
chmod o+t |
|
4 | 2 | 1 | |
Operating | Run as | Run as member | |
on files | owner | of group | IGNORED on Linux |
Operating | IGNORED on | all created files | allow anyone to create, |
on directories | Linux systems | set to group of | but only the owner and |
directory. | root to delete the file | ||
"file giveaway" | The file is "Sticky" | ||
not allowed | Group consistency |
7.1 On directories
- Setuid on directories (ignored)
Ignored
on most Unix systems including Linux. Setuid and setgid were meant for completely different purposes.Setuid
is for causing an executable file torun with its owner's uid
or gid for setgid, rather that the uid of the user calling the executable. Setuid has no function on directories. Does it make sense for a user to create a file in a directory only to have it owned by someone else? What happens to the creator? Will they still have access to the file? "File giveaways" are not allowed.
7.1.1 setgid on directories
All files created in this directory will have GID set to the /same/ GID
as
the parent directory
, and NOT the usual GID of the user creating the file.
It ensures group owner consistency on every file in a directory.
7.1.2 sticky bit on directories
All users allowed to create files in this directory. No other users can
delete these files. Only owner and root of the files can delete files.
7.2 On files
7.2.1 setuid on files
On executable files
, will assume the rights and privileges of the owner of
the file, not the user running/executing
7.2.2 setgid on files
On executable files
, will assume the rights and privileges of the group of
the file, not the user running the executable
.
7.2.3 Reviewing:
sticky bit
applies to directories
, to make any files created in that
directory "sticky" meaning, no one else can delete them
, other than root and
the file owner.
setgid bit
applies to directories
, and makes any file or sub-directory
created in that directory have a group-owner set
to match the group-owner of
the directory itself
(the one that has the sticky bit)
setuid bit
applies to program files
and makes those executable files
run with the same permissions as the owner of the exectuable file (and not
the user running the executable as usually happens)
For directories where you want a master user have access over and
above the normal access that user would have, you use the setgid
on the
directory
, and make both the master user and the directory in a group that
matches. i.e. directory has chgrp
on it for a group that the master user is
a member of.
Press Shift + Tab to navigate to chat history.
8 umask
from man pages:
"The user file-creation mask is set to mode. If mode begins with a digit, it
is interpreted as an octal number
; otherwise it is interpreted as a
symbolic
mode mask similar to that accepted by chmod(1). If mode is
omitted, the current value
of the mask is printed. The -S
option causes
the mask to be printed in symbolic form; the default output is an octal
number. If the -p
option is supplied, and mode is omitted, the output is
in a form that may be reused as input.
The return status is 0 if the mode was successfully changed or if no mode argument was supplied, and false otherwise."
The final permissions of a file are determined from TWO SOURCES
- Proccess permissions
- umask permissions
Linux umask bits define what is forbidden. SAMBA create bit masks define what is allowed.
8.1 Process permissions
When a process creates a file, the file has certain default permissions, for example, -rw-rw-r–. These initial permissions are partially defined by the file mode creation mask, also called file permission mask or umask. Every process has its own umask, for example, bash has umask 0022 by default. Process umask can be changed.
When a file or directory is created, the permissions are set by the process that created that file or directory. For example, when using the touch command to create a new file, the default permissions set by that process for files is 0666. If the UMASK were set to 0000, then the file permissions would be 0666.
From that I gather that the touch proccess has default permissions of 0666
8.2 umask subtractions from 666 or 777
The umask is the value that is subtracted from 666 (rw-rw-rw-) permissions when creating new files, or from 777 (rwxrwxrwx) when creating new directories.
For example, if the default umask is 002, new files will be created with the 664 (rw-rw-r–) permissions, and new directories with the 775 (rwxrwxr-x) permissions.
If umask is 0026 new files will have chmod 640 new dirs will have chomd 751
8.3 umask bits
A umask consists of bits corresponding to standard file permissions. For example, for umask 0137, the digits mean that:
0 = no meaning, it is always 0 (umask does not affect special bits) 1 = for owner permissions, the execute bit is set 3 = for group permissions, the execute and write bits are set 7 = for others permissions, the execute, write, and read bits are set Umasks can be represented in binary, octal, or symbolic notation. For example, the octal representation 0137 equals symbolic representation u=rw-,g=r–,o=—.
Symbolic notation specification is the reverse of the octal notation specification: it shows the allowed permissions, not the prohibited permissions.
8.3.1 How umask works
How umask works Umask prohibits permissions from being set for a file:
Put another way: Linux umask bits define what is forbidden. SAMBA create bit masks define what is allowed.
When a bit is set in umask, it is unset in the file. When a bit is not set in umask, it can be set in the file, depending on other factors. The following figure shows how umask 0137 affects creating a new file.
The mask is just that, a mask to mask off what bits will be OFF So umask of 137 applied to a new file will be like running chmod 640.
All ones:
owner | group | others | ||
---|---|---|---|---|
4 | read | # | # | # |
2 | write | # | # | # |
1 | execute | # | # | # |
7 | 7 | 7 | 7 |
umask 137:
owner | group | others | ||
---|---|---|---|---|
4 | read | # | ||
2 | write | # | # | |
1 | execute | # | # | # |
7 | 1 | 3 | 7 |
Resulting permissions:
owner | group | others | ||
---|---|---|---|---|
4 | read | # | # | |
2 | write | # | ||
1 | execute | |||
7 | 6 | 4 | 0 |
Easiest way to approach this is to say, the umask adn mask have to add up to 7
8.4 umask excercise, easily reproduced:
umask by itself will display what the current umask is set to. umask <octalno.> will set it to the octalno value.
umask 0000 touch one two three ls -l * umask 0002 touch four fix six ls -l *
The above will show you that: UMASK did not allow the others component to have write permission. The way that the UMASK works is it defines what is allowed or permissible. With all zeros, it’s saying that read, write and execute permissions are permissible, so it’s not restricting anything and the permissions will be set by the process. Yet in our second example with a UMASK set to 0002, this is restricting the access for others to not have write permission.
Octal digit | This allows | resulting file | as if |
---|---|---|---|
in umask | permission | chmod was | |
0 | read, write & execute | rwx | 7 |
1 | read and write | rw- | 6 |
2 | read and execute | r-x | 5 |
3 | read only | r-- | 4 |
4 | write and execute | -wx | 3 |
5 | write only | -w- | 2 |
6 | execute only | –x | 1 |
7 | no permissions | --- | 0 |
8.5 Summarizing umasks
UMASK did not allow the others component to have write permission. The way that the UMASK works is it defines what is allowed or permissible.
With a umask set to all zeros, it’s saying that read, write and execute permissions are all permissible,
it’s not restricting anything and the permissions will be set by the process.
With a UMASK set to 0002, it's saying to remove write permission.
Put another way: Linux umask bits define what is forbidden. SAMBA create bit masks define what is allowed.
9 chmod g+s on directories if you want files created in directory to give group rights
9.1 Ensuring new files in a directory have the proper group
chgrp super kaapvall/ sarmation/ # makes group 'super' for these 2 directories chmod g+s kaapvall/ sarmation/ # makes new files have group 'super' in these 2
This forces new files created inside the directories kaapvall and sarmation to have the group ownership "super" instead of the user's group.
10 Permissions for samba users
SAMBA IGNORES UMASK SETTINGS IN THE LINUX ENVIRONMENT! Really!!!
Instead it provides its own equivalent parameter. But here'w the gotcha:
It is the I N V E R S E of the Linux UMASK
So instead of 000 it woud be 777.
Here are the default options:
create mask = 0744 force create mode = 000 create directory mask = 0755 force directory mode = 000
“The create mask and create directory mask options consist of octal permission sets for files and folders, respectively, that combine with the permissions requested by the user using a logical AND. Any bit not set in the mask will be removed from the final permission set. The force create mode and force directory mode options also specify octal permission sets, but are combined with the permission set using a logical OR (after applying any creation masks). Therefore, any permission bit that is set in the force mode is included in the final permissions.” — Gerald Carter, Jay Ts, and Robert Eckstein. Using Samba, 3rd Edition
Octal digit | This allows | resulting file | SAMBA |
---|---|---|---|
in umask | permission | create masks | |
0 | read, write & execute | rwx | 7 |
1 | read and write | rw- | 6 |
2 | read and execute | r-x | 5 |
3 | read only | r-- | 4 |
4 | write and execute | -wx | 3 |
5 | write only | -w- | 2 |
6 | execute only | –x | 1 |
7 | no permissions | --- | 0 |
Put another way: Linux umask bits define what is forbidden. SAMBA create bit masks define what is allowed.
10.1 SAMBA and Linux UMASK
This works quite differently than the Linux environment UMASK. Instead of defining what is restricted, it defines what is allowed. With the create mask and create directory mask options, a setting of 0777 would allow all bits to be set, thus leaving the permissions to be set by the requested user. Furthermore, the force create mode and force directory mode would force a particular bit to be set, even if it wasn’t requested by the user.
As an example, consider the following option settings as you would define them for a share in smb.conf:
create mask = 0770 force create mode = 060 create directory mask = 0770 force directory mode = 070
This would allow both files and directories to have read, write and execute permissions for user and group, but not others. At the same time, it forces the group permissions to be read/write for files and read/write/execute for directories.
10.2 –no-same-owner
When extracting a tar archive from another vm or Linux host, you may have
different owners defined on one machine to the next. In order to over-ride
the original tarred owner of files, use the --no-same-owner
option.
This will make the onwer of the new un-tarred files yourself. Otherwise you will see some user id number as the owner for these files, that may not even be a valid user on your system, or even worse, make an unsuspecting innocent valid user on your system the owner of these files, which of course they should not be.
10.3 Related SELinux context
First of all, selinux (Secure Enterprise Linux) config files are in:
/etc/selinux/config
where you can change SELinux from enforcing to not.
- Change from enforcing to not enforcing (permissive) SELinux or disabled
# This file controls the state of SELinux on the system. # SELINUX= can take one of these three values: # enforcing - SELinux security policy is enforced. # permissive - SELinux prints warnings instead of enforcing. # disabled - No SELinux policy is loaded. SELINUX=permissive
(Related SELinux contexts. See below on SELinux Contexts) For more info on samba, see the samba.org file If user connects anonymously he gets r/o, if he logs in and is a member of assigned group he gets r/w.
I have group named 'admin' set as primary group to users with write privileges, everyone else gets read only rights.
I force user to nobody, so different people working on same files don't interfere with each other.
I set chmod 2755 on shared directory, so it inherits created directories with the same group 'admin'
$ chmod -R 2755 /home/shares/test Checking if all is good:
$ stat home/shares/test Access: (2755/drwxr-sr-x) Uid: (65534 nobody) Gid: ( 1001/ admin) Relevant part of /etc/samba/smb.conf:
[test] comment = test path = /home/shares/test force user = nobody read only = No create mask = 0664 force create mode = 0664 directory mask = 02775 force directory mode = 02775 This post put me on right track, but testparm revealed 4 incorrect directives, so I'm sharing fixed config here. In samba, the less directives you specify the better it works.
10.4 Additional online disuccsions to be reviewed.
I had the same problem, but everything like mask directives did not work for me (Samba 4.3.11):
create mask = 0664 force create mode = 0664 directory mask = 02775 force directory mode = 02775 The only option that worked was under the [global] or share section:
inherit permissions = yes Just change all folder and file permissions to your need, so future folders and files will inherit the same permissions.
3
There is a very similar problem when connecting from other Unix / Linux / OSX / MacOS devices: all of the settings are ignored unless you specify
[global]
unix extensions = no And connect with smb://<serverhost> instead of cifs://<serverhost>.
another: [html] comment = admin access path = /var/www/html browsable = yes guest ok = no writable = yes valid users = @admin create mask = 664 force create mode = 664 security mask = 664 force security mode = 664 directory mask = 2775 force directory mode = 2775 directory security mask = 2775 force directory security mode = 2775
SOLVED (seems to be)
The problem was in the obey PAM restrictions parameter. By default it is turned off, and I could not remember why I turned it on. The SAMBA config was partially taken from older installation may be I had reasons to be obeyant there :-)
When it is ON, then SAMBA-created files are under UMASK restrictions. I don't know, if it is correctable via login defaults (what is the user?), but umask command gives me "0022" which means "u+a g-w o-w".
Hope, it will help to someone with similar problem.
shareimprove this answerfollow
/etc/samba/smb.conf file. For my Music share, I added the following options:
create mask = 664 force create mode = 664 security mask = 664 force security mode = 664 directory mask = 2775 force directory mode = 2775 directory security mask = 2775 force directory security mode = 2775
11 Setting Unix umasks
First off what is my umask? run : umask (or umask -S for symbolic form)
setting a umask is as simple as umask 137
11.1 Environment variables
11.1.1 bash: PS1="my prompt >\n"
export PS1=
"my prompt >\n"echo $PS1
env
11.1.2 tcsh: setenv prompt='my prompt >"
setenv prompt='my prompt >" printenv prompt
11.1.3 fish set PYENVVERSION 3.7.5
set -x key value set -x PYENVVERSION 3.7.5 set -e PYENVVERSION 3.7.5 set PYENVVERSION 3.7.5
printenv env echo $PYENVVERSION
set -g PYENVVERSION 3.7.5 set -x PYENVVERSION 3.7.5 set -xg PYENVVERSION 3.7.5
what is the difference? see man set but in a nutshell:
-x flag is orthogonal ot the -g, -l, -u flags -x simply sets the export attribute on the var -g,-l,-u set the scope of the var
You can have a global
, unexported
var
and inside a function do set -lx var
value
to create a locally scoped instance that is exported
. When control
returns from the function the global scope var pops back into existence
and won't be exported.
An exported variable
is visible to commands run by your fish shell.
Note that env
is an external command that shows the exported variables it
inherited. So, it should only show the variables created with set -gx
11.2 fish scripting
Start with the very simple: for i in (ls) echo $i end
Note how that is different from bash that would use backtics for subcommands `ls`
12 export and variable scope
12.1 shell variables
Shell variables are created when you first assign a value to them:
MYVAR = 314159 echo "MYVAR is set to $MYVAR"
You can have a global
, unexported
var
such as MYVAR
in the above example, and
then inside a function do set -lx MYVAR =value
to create a locally scoped
instance
, that is different from 314159. When control returns from the
function the global scope var pops back into existence and won't be exported.
Variables that are NOT exported are called shell variables
.
12.2 Environment variables
An exported variable
is visible to commands and child processes run by that
shell. All shells opened beneath this shell will still have this variable
set to the same value. This is what is known as an environment variable
,
because it holds for the entire environment below it.
A full example of a shell variable
:
$ MYVAR=314159 $ echo "MYVAR is set to $MYVAR" MYVAR is set to 314159 $ bash $ echo "MYVAR is set to $MYVAR" MYVAR is set to $
Now as an environment variable
:
$ export MYVAR = 314159 $ echo "MYVAR is set to $MYVAR" MYVAR is set to 314159 $ bash $ echo "MYVAR is set to $MYVAR" MYVAR is set to 314159 $
** Useful environment variables In no particular order:
HISTTIMEFORMAT="%Y-%m-%d %T" # used by logs and other processes.. year month day and time
12.2.1 printenv
You can printenv variablename
to see what and if that variable is set, or just
printenv
with no arguments to see all the environment variables.
12.3 Special shell variables
There are several special shell variables
set by most shells (not all). They are:
In bash:
${$}
the current running process ID, i.e.PID
${n} positional parameters
Where${1}
is the first parameter,$2
is 2nd parm etc. Where the n is a number, the curly braces are options.$0
Special case of the above, but results in the current running job/command${@} all parameters
All of them. Can say$@
${*} all parameters
All of them. Can say$*
But output has the IFS between parameters For examplemyscript 40 myfile.txt
results in$*
as40;myfile.txt
when IFS is ";"${#}
how many parameters exist${?}
the error code of the last command${!}
the PID of the last background job/command${_}
the last parameter
In most cases the curly braces are optional, so you can use:
$$
$3
$0
$@
$*
$#
$?
$!
$_
Not to be confused with the command line history commands that all begin with
the 'bang' character, like !!
and !$
and !*
and !-3
as in:
!!
previous command, often used withsudo !!
!$
previous command's last argument, often used withls *.*~; rm !$
M-.
does the same as!$
i.e. the previous command's last argument!-2
two commands ago!-n
n commands ago!545
the 545th command, often used afterhistory | grep somestring
13 crontab
See crontab.org file for more details
* * * * * <command to be executed> - - - - - | | | | | | | | | +------day of the week (0 - 6) (Sunday=0 | | | | | | | +-------month (1 - 12) | | | | | +------ day of month ( 1 - 31 ) | | | +------hour (0 -23) | +----+ min (0 - 59 )
14 xargs
xargs takes a list and passes the list as arguments to another program.
xargs -options arguments
By default the arguments is "echo". But argueents is usually a command.
tail -3 ~/bash_history | xargs or ls *.org | xargs wc -l printf "file1\nfile2\nfile3" | xargs touch even better: printf "1\n2\n3\n4" | xargs -I_ touch file_.txt for i in {10..30} ; do echo $i ; done | xargs -I_ touch deletme-file-_.txt
-i tells xargs we will replace some text for each argument
Note: in the above example the for i loop is valid in /bin/bash and /bin/zsh It is NOT a valid syntax in /bin/tcsh
14.0.1 fix this example
Let's say your command is "dnf uninstall"
And the list of apps you want to uninstall came from a grep:
dnf list --installed | grep microsoft | xargs | dnf uninstall
will issue the command dnf uninstall arg (one at a time where arg is
the list of apps tht include microsoft) you could test this first with
dnf info
in place of dnf uninstall.
14.1 a better chgrp example
Let's say you want to change all python programs, (ending in .py) in the current directory, to be executable, and belong to the zintis group.
ls *.py | xargs chmod 740 ls *.py | xargs chgrp zintis
And you are done!
Notice that you don't actually put the file as an argument at the end of the chmod or chgrp command. If that makes it confusing for you, the following two command are identical in action.
ls *.py | xargs -I{} chmod 740 {} ls *.py | xargs -I{} chgrp zintis {}
One final point, these xargs examples actually generates just one chmod or chgrp command, ef chmod 740 file1.py file2.ph file3.py …. filen.py
If you want to issue the command itself once for each file to get chmod 740 file1.py chmod 740 file2.py . . chmod 740 filen.py
Then you must also include -n 1, so ls *.py | xargs -n 1 chmod 740
15 Candidate Packages from EPEL
EPEL or 'Extra Packages for Enterprise Linux" from the EPEL Wiki
Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special Interest Group that creates, maintains, and manages a high quality set of additional packages for Enterprise Linux, including, but not limited to, Red Hat Enterprise Linux (RHEL), CentOS and Scientific Linux (SL), Oracle Linux (OL).
EPEL packages are usually based on their Fedora counterparts and will never conflict with or replace packages in the base Enterprise Linux distributions. EPEL uses much of the same infrastructure as Fedora, including buildsystem, bugzilla instance, updates manager, mirror manager and more.
While there are far fewer packages in EPEL than in Fedora (~4,000 vs ~30,000) that is still way too many packages for my liking given that I won't be using most of them. However, you can list the included packages from the download directories in fedoraproject.org
From these directories you can choose the individual packages (.rpm) that you want to install, and because these are vetted, they are most likely going to be compatible with the version of RHEL, or Fedora, or CentOS that you are running. Just look at the right download page that matches the CentOS that you are running.
16 Verify CentOS ISO
17 Manual pages
man man
Man pages are divided in sections, mostly in section 1 To see specific sections enter the section # followed by the man page, for example
man ls man 5 man.conf
Where man man defaults to section 1, i.e. man 1 man is the equivalent But man 5 man.conf will open the manual pages for man.conf under section 5
man -k network # show which man pages talk about "network"
17.1 MAC OSX
On Darwin (Mac OSX) the sections are The standard sections of the manual include:
1 User Commands
2 System Calls
3 C Library Functions
4 Devices and Special Files
5 File Formats and Conventions
6 Games et. Al.
7 Miscellanea
8 System Administration tools and Deamons
17.2 CentOS
On CentOS the sections are The standard sections of the manual include:
1 User Commands
18 YUM a.k.a. DNF
See YUM a.k.a. DNF in my yum.org file. But in a nutshell these are both package managers for RedHat, CentOS and some others much like RPM is for RedHat, and BREW is for MAC OSX.
19 sudo
On CentOS there does not seem to be the wheel group. But I could still add an individual user to the sudoers file the following steps:
su > this switches to superuse (need to enter password)
19.1 sudoers file and visudo
visudo
> this command opens up vi with the sudoers file
add the user after the line for root: i.e.:
## Allow root to run any commands anywhere root ALL=(ALL) ALL zintis ALL=(ALL) ALL
19.2 sudo timeouts
By default the sudo timeout is fairly short (well depending where you are I guess). It extend the timeout to say, 25 minutes you can do the following:
sudo visudo
edit the line:
Defaults env_reset
to add:
Defaults env_reset, timestamp_timeout=30
Defaults env_reset,timestamp_timeout=30
not sure if a space comes after the comma…
Or, just add a line by itself as in:
Defaults timestamp_timeout=30
You can see the value by sudo grep timeout /etc/sudoers
20 hostname
On CentOS the hostname is stored in /etc/hostname and the command to change it is: you guessed it, hostname.
20.1 hostname
hostname # to display it hostname -s to hostname -f # display the FQDN
To permanently change it, use
hostnamectl set-hostname your-new-hostname
This worked, and changed it in /etc/hostname
20.2
There is also sudo vi /etc/sysconfig/network
With lines like: HOSTNAME=c8host.zintis.net
And /etc/hosts where you can add 192.168.122.76 c8host.zintis.net c8host
20.3 nmcli general hostname
Can also set the hostname to "arthurdent" by:
nmcli general hostname arthur_dent
21 domain name
/etc/sysconfig
has many configurations that are similar to the old init.d
or . It is the new approach to init.d with more flexibility. Among the
sysconfig directory is the netowkr config that on a new CentOS has only
one comment line: # Created by anaconda
But, domain suffix, i.e. senecacollege.ca, is configured in the dns configs, so, /etc/resolv.conf
nameserver # DNS server IP (can have three of these) domain # Domain Name of local host search # Which Domain to search
But with CentOS 8 this file is generated by the NetworkManager. That means that /etc/sysconfig/network-scripts/ifcfg-* changes will overwrite /etc/resolv.conf /etc/sysconfig/network-scripts/ifcfg-* changes will overwrite /etc/resolv.conf
If you want to prevent that from happening (and going back to old-school) you have to remove DNS1, DNS2, etc lines from /etc/sysconfig/network-scirpts/ifcfg-* files. Then NetworkManager will stop overwritting your /etc/resolv.conf file.
The search line is optional. If missing, then name resolution for host1 will be limited to the host1 plus the domain, i.e. host1.acme.com but if search has acme.ca then the resolution will be host1.acme.com followed by host1.acme.ca
21.1 Using NetworkManager
If you want to go with the flow and want to config DNS, you can add entries to /etc/sysconfig/network-scripts/ifcfg-eth0 like these two lines: DNS1=208.67.222.222 DNS2=208.67.220.220
21.2 /etc/sysconfig/network
Can straight edit this file too (needs SU priviledges) add HOSTNAME=myserver.cisco.com
22 Restarting the network
22.1 As of CentOS7 and CentOS8
The easiest is nmcli network off; nmcli network on
22.2 Deprecated init.d method
/etc/init.d/network restart
In general Centos does not use service, but other Linux distributions
still do. They would run sudo service postfix restart
in place of
sudo systemctl restart postfix
Just for those times you are not
on a CentOS, or RHEL, or Fedora or Ubuntuu system. Like Cisco switches
for example.
This is the traditional approach, which has been replaced by systemctl When looking in the directory /etc/init.d the CentOS developers were good enough to leave you a note: as follows:
You are looking for the traditional init scripts in /etc/rc.d/init.d, and they are gone?
Here's an explanation on what's going on:
You are running a systemd-based OS where traditional init scripts have
been replaced by native systemd services files. Service files provide
very similar functionality to init scripts. To make use of service
files simply invoke systemctl
, which will output a list of all
currently running services (and other units).
Use systemctl list-unit-files
to get a listing of all known unit files,
including stopped, disabled and masked ones.
Use systemctl start foobar.service
and systemctl stop foobar.service
to start or stop a service, respectively. For further details, please refer
to systemctl(1).
Note that traditional init scripts continue to function on a systemd
system. An init script /etc/rc.d/init.d/foobar
is implicitly mapped
into a service unit foobar.service during system initialization.
Thank you!
Further reading: man:systemctl(1) man:systemd(1) http://0pointer.de/blog/projects/systemd-for-admins-3.html https://www.freedesktop.org/wiki/Software/systemd/Incompatibilities
22.3 Network intefaces (ip command)
ip link help ip link show ip addr show ip -4 address show (use tab completion, and double-tab to see options)
23 Enabling nested KVM in CentOS8
Edit /etc/modprobe.d/kvm.conf
uncomment the line options kvm_intel nested=1
As of Jan 12, I have not done this next step. Still as of kvm-nested.conf
Then ADD a new file
called kvm-nested.conf
with the following content:
options kvm-intel nested=1 options kvm-intel enable_shadow_vmcs=1 options kvm-intel enable_apicv=1 options kvm-intel ept=1
Then reboot.
23.1 reboot
shutdown -h now # actually halts the server, then you manually start it shutdown -r now # reboots reboot init 6
reboot and shutdown are both symbolic links to systemctl.
24 Switching between console and gui
C-M-f1
will boot into a tty (console)
C-M-f7
will switch back to the graphics interface.
25 Notes on installing KVM on CentOS 8
These notes are from linuxconfig.org
26 Fix for my VMWare fusion CentOS8
I was not seeing any output on "grep -E '(vmx|svm)' /proc/cpuinfo" so I suspected that the fusion settings on my CentOS8 host did not support hardware virtualization.
I was right
26.1 Result:
I was now getting the responses I was looking for. For example:
zintis@c8host ~$ modprobe -a kvm_intel zintis@c8host ~$ cat /sys/module/kvm kvm/ kvm_intel/ zintis@c8host ~$ cat /sys/module/kvm_intel/parameters/nested Y zintis@c8host ~$
26.2 SystemV used "run levels" (old school)
To make this stick: edit /etc/inittab
and look for a line that begin with id:5.
Replace the 5 in that line by 3. You can find a brief description of runlevels here,
but shortly:
- Runlevel 0 and 6: halt and reboot the machine, respectively. - Runlevel 1: No services running, only root can login. - Runlevel 2: Users can login but no networking. - Runlevel 3: Networking and text-mode. - Runlevel 4: unused. - Runlevel 5: GUI.
27 History
echo $HISTSIZE echo $HISTFILESIZE export HISTSIZE=200 echo $HISTSIZE
To clear a line from history, (for instance if you inadvertently entered a clear text password into the wrong window) use the -d command.
history -d 467
will delete line 467 from history.
28 Navigating and finding files
See also examples (assuming sudo is used on all) in the find.org file.
28.1 find older files
You can use the -mtime +5
option of the find command to find files
that are older than 5 days
Other find time attributes:
-atime n access time -mtime n modified time
n can be an exact time, i.e. 5 would be 5 days ago (5 X 24 hrs) but +5 is 5 days or older ago.
An example to move *.jpg files older than 30 days is:
find /path/to/files/ -type f -name '*.jpg' -mtime +30 -exec mv {} /path/to/archive/ \;
28.2 find large files taking up space
df
and du
of course, but here's how you can easily find the largest 20 files
- du -a /dir | sort -n -r | head -20
du -a . | sort -n -r | head
There are a few alternatives to du
the most popular one being ncdu
. -ncdu
is
written in c
and users ncurses
library, hence the name ncdu
. To install
ncdu
you simply brew install du
or sudo dnf install ncdu
or apt install ncdu
But others are dust
, written in rust
programming language, diskus
and others.
28.2.1 du usage
Common usages are:
du -sh .
human readable and summarize total . is the default this dirdu -sh /etc /var
as many directories as you want in one commanddu -shc /etc /var
thec
totals the usage from all the directories listeddu -shc ~/*
is very useful. It combines total of all sub-directories in ~du -h --max-depth 2
human readable and summarize to 2 directories deepdu -shc *
will be probably the most common options you use
While we are talking about du
make sure you check out df command.
28.2.2 df
disk free
Common usages are:
df -h
human readabledf -hT
showtype
column along with usual Filesystem, Size, Used, Avaiable…df -hT -x tmpfs
same output butignore
lines that havetmpfs
df -hTx tmpfs
exact same thing- =df -
Combine with watch
for times when you want to monitor disk usage, or when you
mount a drive and want to see what changes.
watch df -hTx tmpfs
every 2 seconds.
28.3 find files or directories using globbing.
find . -name src -type d
find . -path '**/test/*.py' -type f
grep -R
to recursively look for the string in all files and all subdirectories.
28.3.1 find and delete small image files
This one is useful, as mail and web pages often have many many small image files that are typically not needed after a few months, and they just take up inodes. So delete the like this:
find * -type f -name "*.jpeg" -size -1k -delete
But be careful where you use that!! Use caution.
28.3.2 rg "import requests" -t py ~/bin
28.3.3 fd is a simpler version of find
28.4 locate
Much like spotlight on mac, it indexes files on your system once a day.
28.4.1 locate (is a pre-indexed dbase of files) that make it fast.
locate whereismyfile.txt
28.4.2 Manually building locate index database
sudo updatedb
The first time this is run it will take a couple of minutes, but after that
it is very quick. But you don't have to every day, as updatedb ir run
automatically every day.
28.5 which
Searches your path for executable files.
28.6 whatis
Gives you a brief man summary for a command
28.7 apropo (man -k)
28.7.1 rg "import request" -t
rg -u –files-without-match "^#\!" -t sh recursively grep, -u including hidden files, –files-without-match i.e. files that are missing somethin, ^#\! a shebang at the beginning of line that are of type sh (i.e. end in .sh) So this looks for .sh files that are missing the shebang line.
28.7.2 ripgrep
28.7.3 grep with –stats gives you interesting data.
28.7.4 fzf (pipe a grep into fzf) fzf does fuzzy matching too.
29 list of miscellaneous commands used by Seneca students
grep, wc, pwd, ls, more, less, file, wget, chmod, vi, vim, hostname, uname, ps, lblk, yum, dnf, rpm, nmcli, ip id, whoami, su, sudo, visudo, grep, selinux (see selinux.org) sestatus, netstat -nr, route, mkdir,
29.1 notes
"route" is an alias for "netstat -nr"
30 Setting console font
30.1 First test interactively using setfont
Easiest is to interactively test the fonts using setfont. The available console fonts are located in the different directories for different distros:
/usr/share/consolefonts (Debian/etc.), /usr/lib/kbd/consolefonts (Fedora, RHEL, CentOS), /usr/share/kbd/consolefonts (openSUSE)…
setfont UniCyr8x14.psf.gz setfont sun12x22.psfu.gz setfont LatGrkCyr-8x16.psfu.gz setfont LatGrkCyr-12x22.psfu.gz
30.2 Make it the default font
30.2.1 console-setup
Possibly edit /etc/default/console-setup and put the following untested lines CHARMAP="UTF-8" CODESET="Lat7" FONTFACE="Terminus" FONTSIZE="28x14"
This example sets the Terminus Bold font at 32 points, and restricts the width to 80 columns.
ACTIVECONSOLES="/dev/tty[1-6]" CHARMAP="UTF-8" CODESET="guess" FONTFACE="TerminusBold" FONTSIZE="16x32" SCREENWIDTH="80"
The FONTFACE and FONTSIZE values come from the font’s filename, TerminusBold32x16.psf.gz.
Yes, you have to know to reverse the order for FONTSIZE. Computers are so much fun. Run setupcon to apply the new configuration. You can see the whole character set for your active font with showconsolefont. Refer to man console-setup for complete options.
30.2.2 vconsole.conf
Or, edit /etc/vconsole.conf with FONT="ter-v32n"
Or, edit /etc/fonts/fonts.conf and add: <!– Fallback fonts preference order –> <alias> <family>sans-serif</family> <prefer> <family>Tahoma</family> <family>Arial</family> </prefer> </alias> <alias> <family>serif</family> <prefer> <family>Times New Roman</family> </prefer> </alias> <alias> <family>monospace</family> <prefer> <family>Courier New</family> </prefer> </alias>
<match target="font"> <edit name="antialias" mode="assign"> <bool>false</bool> </edit> <edit name="hinting" mode="assign"> <bool>true</bool> </edit> <edit name="autohint" mode="assign"> <bool>false</bool> </edit> <edit name="hintstyle" mode="assign"> <const>hintfull</const> </edit> </match>
Then updated font cache with fc-cache. Now desired fonts available system wide.
Or,
30.2.3 Systemd
Systemd is different from console-setup, and you don’t need to install anything, except maybe some extra font packages. All you do is edit /etc/vconsole.conf and then reboot. On my Fedora and openSUSE systems I had to install some extra Terminus packages to get the larger sizes as the installed fonts only went up to 16 points, and I wanted 32. This is the contents of /etc/vconsole.conf on both systems:
KEYMAP="us" FONT="ter-v32b"
31 Adding colour to the console
See LSCOLOUR.org file, but basically in your .bashrc include these two lines:
export CLICOLOR=1 export LSCOLORS=ExFxCxDxBxegedabagaced # or is it export LS_COLORS='*.txt=01;32:*.sh=31:*.org=01;35:'
I have also seen $LS_COLORS
env variable, so check which one works for you.
32 gnome desktop manager
32.1 gnome auto start apps
Enter the app called "tweaks" then add the app you want started
automatically, "Startup Applications" You may have to install
"tweaks", if so use dnf install gnome-tweaks
32.2 gnome user avatars
within the gnome screen, upper right, click zintis/account settings Then click on avatar and choose a file or a picture. Easy-peasy.
33 kde desktop manager
My gnome was having trouble allowing a login. Even with correct passwords I would catch a glimpse of my desktop, including my auto-start terminal window, but then it would kick me right back to GUI login screen.
I clicked the settings gears on my login screen and tried all options to
no avail. So, I installed kde. And now the options gears show a new option
cslled "plasma" (which is kde plasma). That works. To se it as a default
desktop manager, you can edit ~/.dmrc
or also see /usr/share/xsessions
33.1 ~/.dmrc
[Desktop] Session=kde-plasma
You might haver to add a kde.desktop file if not already in the xsessions directory mentioned above
33.2 ~/.dmrc
may be obsolete.
If so, try to specify a default session for a user in :
/var/lib/AccountsService/users/*username*
file.[User] Language= XSessions=plasma
One user suggested this:
From dir /usr/share/xsessions move all *.desktop file to another directory(unnecessary_env, for example) and leave only one you need(in my case - xfce.desktop): $ ls /usr/share/xsessions xfce.desktop unnecessary_env/ After logout/reboot XFCE will be loaded by default Note! You won't be able to choose between GUI
33.3 systemd to boot into a GUI (could be obsolete)
exec "usr/bin/startkde" >> ~/.xinitrc startx use systemctl instead.
34 ssh setup
The ssh application is called openssh and openssh-server. Install these both
(using yum or dnf).
But systemctl sees these both as sshd
. i.e. the service is called sshd
.
systemctl start|stop sshd
(or enable|disable
)
If not found, have to sudo apt install or sudo dnf install sshd
34.1 sshdconfig
In /etc/ssh/sshd_config
PermitRootLogin without-password
is deprecated alias: use prohibit password
PermitRootLogin prohibit-password
in combination with:
PubkeyAuthentication yes
This results in root being able to login, but ONLY with a public key authentication. i.e. NOT with username password.
Interesting that the lab specifically asked to set PermitRootLogin to yes
the line before asking to permit root login ONLY using key authentication.
—there might be more going on. See man sshd_config
that states that
prohibit-password means password and keyboard-interactive authentication are
disabled for root.
have to restart sshd for these changes to take effect so: sudo systemctl restart sshd (or for older systems service sshd restart
34.2 Additional hardening best practices with sshdconfig
in /etc/ssh
there is the file sshd_config
(not to be confused with the client
config file ssh_config
34.2.1 change port away from 22
this needs work, as for me it id not work making it 727 and then from my
client running ssh zinux -p 727
as it gave me an error "conneciton refused"
34.2.2 AddressFamily inet
that is if you want to only support ipv4. otherwise leave it as "any"
34.2.3 PermitRootLogin no
This one is a no-brainer. Do it! Of course you have to create a normal
user first, and make sure that user can sudo (visudo needs to be set
) and
that normal user needs to be part of the wheel group
.
34.2.4 PasswordAuthentication no
After this one, you will be locked out unless you have already copied your ssh public key to this server and confirmed that ssh is working using your key.
Note: if you have done this, you won't be able to add the public keys from new hosts, as that process needs to prompt for a password the first time you send the ssh key. So, simply do this:
- change this back to
yes
- restart sshd
- from the new host run your
ssh-copy-id -i ~/.ssh/id_rsa.pub zinux
(at this point the ssh-copy command should prompt for a password.)-i
option specifies to look for theidentity
file in the following file. - confirm that you can ssh without password to zinux
- change this to
no
again. - restart sshd
34.2.5 if 22 does not work, follow the steps to tell SELinux to allow port 727
semanage port -a -t ssh_port_t -p tcp 727
followed by semanage port -l
to list all the port policies.
If your server uses port 727
, do you have to also specifiy the port 727
when you run the ssh-copy-id
command? I think you do. Check the man
page on ssh-copy-id
command.
34.3 Auto ssh login by pre-sharing your own hosts public key:
You can think of it as sending your password ahead of time to the server
you want to connect to. But rather than pre-sending your password, which
would obviously be a huge security risk, you send just your public key.
The public key will be used by the server to confirm that you are you, as
only you, i.e. the possessor of the matching private key, could have sent
a session key that was encrypted with the private key.
That session key
is decrypted using your pre-shared public key, and if it is a match, then
the server knows that you are you.
34.3.1 1st generate a public/private key pair:
ssh-keygen -t rsa
That creates two files in ~/.ssh
~/.ssh/id_rsal.pub
~/ssh/id_rsa
The public one is what you share, and that is idrsa1.pub
- that is very important NOT to share you private key. to ANYONE
- Where has the public and private key files gone??
Also note that you won't see these files using sudo, or even su You have to login to root as root, so su - Then, and only then will you see the idrsa and idrsa.pub files.
One can SSH to a Linux machine without using a password! This can really speed up your administrative tasks everywhere that Linux servers are used.
You first must share the public key from the root user in your host machine with the root user of your vm guest machines (vm1, vm2 and vm3)
Do that by copying the contents of your ~/.ssh/idrsa.pub from your host machine and append to ~/.ssh/authorizedkeys on each of your Virtual Machines. In your case, you will issue the following command once for every vm IPADDR, (or hostname if DNS is setup).
34.4 ssh-copy-id
34.4.1 2nd ssh-copy-id the public key to the remote server root account.
ssh-copy-id -i ~/.ssh/id_rsa.pub root@IPADDR_for_vm
ssh-copy-id -i ~/.ssh/id_rsa.pub root@IPADDR_for_vm
ssh-copy-id -p 1028 -i ~/.ssh/id_rsa.pub root@IPADDR_for_vm
# if not 22
Note: Press ENTER
for all prompted information including the password
(although this may seen counter-intuitive!) -except of course the login
to the remote ssh host the first time. -duh!
This will enter the NULL character for password, so you won't have to enter a password when you ssh using the keys.
ssh-copy-id options:
-i
look foridentity
in the following file-p
use the sshport
specified rather than the default port ~22-n
do adry run
and just tell me the keys that you would copy
Running the ssh-copy-id
command will copy the key you specify (make sure it
is your public key) and store it in the remote host ~/.ssh/authorized
ssh-copy-id -i id_rsa.pub root@192.168.111.11 ssh root@192.168.111.11 ssh-copy-id -i id_rsa.pub root@192.168.111.12 ssh root@192.168.111.12 ssh-copy-id -i id_rsa.pub root@192.168.111.13 ssh root@192.168.111.13 ssh-copy-id -i id_rsa.pub -p 1122 zintis@192.168.11.66 # if your permission was denied by the remote host, make sure you restart # ssh on the remote host with password login enabled first (it is a chicken # and egg situtation). After the public key is copied, you can turn off # password login again... See ssh.org
35 ssh for a regular user
35.1 Generate public/private key pair for user
As user:
ssh-keygen -t rsa
ssh-keygen -t rsa -b 2048
ssh-keygen -t rsa -b 4096
ssh-keygen -t dsa
ssh-keygen -t ecdsa -b 521
ssh-keygen -t ed25519
I recommend using rsa. It is the most widely accepted key type.
35.2 Check the fingerprints of your public key: (and private key)
ssh-keygen -l -f <filename>
# i.e. in my case ~/.ssh/idrsa.pubssh-keygen -lv -f <filename>
The -l option is to list fingerprint.
35.3 List all fingerprints (random art format) of all known hosts
ssh-keygen -lv -f ~/.ssh/known_hosts
ssh-keygen -lv -f ~/.ssh/id_rsa.pub
ssh-keygen -lv -f ~/.ssh/id_rsa
36 ssh trick to make your life easier
You can ssh into a remote linux host without using a password if that remote host has a copy of your public key. Then that remote host can use your public key to confirm that you are who you say you are by challenging you an encryption messages, encrypted with your public key.
No one but the holder of the secret key will be able to un-encrypt this challenge and respond back correctly, so the remote host knows it is you.
Lots happens in a hand-shake that is hidden from the users view. All a user has to do is follow these steps:
- generate the
public key - private key pair
- copy the public key into remote host, specifically append it to
~/.ssh/authorized_keys
The actual copy could be done using:
ssh-copy-id
ssh-copy-id
See the
man ssh-copy-id,
but essentially it is a custom command for doing exactly this one thing, copying public keys to remote hosts to allowpassword-less login
to those remote hosts. - use ssh to the remote host. BINGO
37 Openssh compatibility with older ssh servers/routers
If you get an error such as:
The fix lies with the fact that openssh by default disables older, less secure algorithms in their ssh configs. This can be overridden as clearly detailed on openssh.com: http://www.openssh.com/legacy.html
As did this site: support.tibco.com
It is a common problem while the encrypted channel is being set up. If your system and the remote system don't share at least one cipher, there is no cipher to agree on and no encrypted channel is possible. Usually SSH servers will offer a small handful of different ciphers in order to cater to different clients. You can fix it manually or more permanently:
37.1 Manual override ssh key exchange
So I had success with command-line options to override the supported handshake
sudo ssh -oKexAlgorithms=+diffie-hellman-group-exchange-sha1 ansible@r0
(of course I had previously set r0 in the /etc/hosts file on the Alpine server on CML
37.2 Semi-permanent fix
You could also fix it more permanently (for yourself only) by adding this to your ~/.ssh/config file:
Host 10.255.252.1 Ciphers 3des-cbc KexAlgorithms +diffie-hellman-group-exchange-sha1
If you usually login to the same user on a particular host you can also add the userid like this:
Host 192.168.111.33 Ciphers 3des-cbc KexAlgorithms +diffie-hellman-group-exchange-sha1 User ansible
37.3 Yet another learning moment
The above did not work when I added that to /etc/ssh/sshconfig That is because
this config was ONLY offering support for the 3des-cbc
cipher. The error
message was clear so that the fix was easy:
Unable to negotiate with 192.168.111.38 port 22:
no matching cipher found. Their offer: aes128-ctr,aes192-ctr,aes256-ctr
So I changed the above host specific sections to this:
Host r0 Ciphers 3des-cbc,aes128-ctr,aes192-ctr,aes256-ctr KexAlgorithms +diffie-hellman-group-exchange-sha1 User ansible Host r1 Ciphers 3des-cbc,aes128-ctr,aes192-ctr,aes256-ctr KexAlgorithms +diffie-hellman-group-exchange-sha1 User ansible Host r2 Ciphers 3des-cbc,aes128-ctr,aes192-ctr,aes256-ctr KexAlgorithms +diffie-hellman-group-exchange-sha1 User ansible Host r3 Ciphers 3des-cbc,aes128-ctr,aes192-ctr,aes256-ctr KexAlgorithms +diffie-hellman-group-exchange-sha1 User ansible Host r4 Ciphers 3des-cbc,aes128-ctr,aes192-ctr,aes256-ctr KexAlgorithms +diffie-hellman-group-exchange-sha1 User ansible Host r5 Ciphers 3des-cbc,aes128-ctr,aes192-ctr,aes256-ctr KexAlgorithms +diffie-hellman-group-exchange-sha1 User ansible
Next I should clean that up under Host * section.
38 scp
Copy file from a remote host to local host SCP example: $ scp username@fromhost:file.txt /local/directory
scp zintis@zintis.net:/home/zintis/fix-selinux-wordpress.log wp.log
Copy file from local host to a remote host SCP example:
# cd to where the file is located, assuming file is "file.txt" scp file.txt username@to_host:/remote/directory/ scp file.txt username@to_host:/remote/directory/ 2>scp-errors.txt scp -P 741 file.txt username@to_host:/remote/directory/ # if ssh port is 741, not 22
39 File Descriptors
File descriptors are simply positive integers that represent your open files.
In unix, everything is a file, and you may have many open files, including where
a process sends its output and error messages. These are called standard file
descriptors
as discussed next.
40 Standard File Descriptors
There are two standard output file descriptors
0
isstdin
1
isstdout
2
isstderr
&
all outputs
By default both are the terminal
where the command was entered.
41 Redirecting Output, stdout, stderr, stdin
Redirecting is done with the >
operator, and the <
operator, for stdout
and
stdin
respectively. So, before a command is executed, its input, output, errors
can be all redirected from the normal stdin
, stdout
and stderr
. Redirectors are
processed by the shell in the order that they appear, left to right.
41.0.1 stdout to a file > or 1>
ls -l > long-list-output.txt
# redirects stdout to a file. Since 1
is the
file descriptor for stdout
as well, leaving the file descriptor out is just a
shortcut to 1>
So these two are identical:
ls -lart > long-list-output.txt ls -lart 1> long-list-output.txt
41.0.2 stderr to a file, 2>
grep da * 2> grep-output-errors.txt
# redirects errors, 2
to a file
41.0.3 both stdout and stderr to the same file, 2>&1
When running a script from a cron job, i.e NOT from the standard terminal
you can redirect
&1
and &2
to some other location, typically a file, and often
the same file.
batchjob.py > ~/batchoutput 2>&1
To break this down:
- the standard output is redirected,
>
to the file~/batchoutput
- which also is automatically assigned the standard file descriptor
1
. - By adding the
2>&1
we are also telling bash to also redirectstderr
to the same place i.e.redirect 2 to what &1
is, and in this case file~/batchoutput
- The
&1
is needed when the fileredirect is the target
. When the file redirect is the source, you do not need the&
.&1
is~/batchouput
.
41.0.4 stderr and stdout to the bit bucket &> /dev/null
rm -f $(find /home/zintis -name \*.junk ) &> /dev/null
This will redirect all outputs
to /dev/null
. You may want to check
what it was doing by redirecting both to a file with &> all-outputs.txt
41.0.5 stderr and stdout to a file
rm -f $(find /home/zintis -name \*.junk ) &> /home/zintis/junk-removal.log
41.1 Common errors
2>1
it would simply rediredt standard errors to a file called '1'
job &2>&1
it would run "job
" in the background, &
There does not have to be
a space between the first &
and the 2
, so that simply takes "2"
which is
stderr
and redirects it to, in this case &1
, which is stdout.
42 CentOS/AlmaLinux os-release file
/etc/os-release
Careful making changes here, but it should be something like this:
$ cat /etc/os-release NAME=Fedora VERSION="29 (Workstation Edition)" ID=fedora VERSION_ID=29 PLATFORM_ID="platform:f29" PRETTY_NAME="Fedora 29 (Workstation Edition)" ANSI_COLOR="0;34" CPE_NAME="cpe:/o:fedoraproject:fedora:29" HOME_URL="https://fedoraproject.org/" SUPPORT_URL="https://fedoraproject.org/wiki/Communicating_and_getting_help" BUG_REPORT_URL="https://bugzilla.redhat.com/" REDHAT_BUGZILLA_PRODUCT="Fedora" REDHAT_BUGZILLA_PRODUCT_VERSION=rawhide REDHAT_SUPPORT_PRODUCT="Fedora" REDHAT_SUPPORT_PRODUCT_VERSION=rawhide PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy" VARIANT="Workstation Edition" VARIANT_ID=workstation $ cat /etc/os-release NAME="AlmaLinux" VERSION="8.7 (Stone Smilodon)" ID="almalinux" ID_LIKE="rhel centos fedora" VERSION_ID="8.7" PLATFORM_ID="platform:el8" PRETTY_NAME="AlmaLinux 8.7 (Stone Smilodon)" ANSI_COLOR="0;34" LOGO="fedora-logo-icon" CPE_NAME="cpe:/o:almalinux:almalinux:8::baseos" HOME_URL="https://almalinux.org/" DOCUMENTATION_URL="https://wiki.almalinux.org/" BUG_REPORT_URL="https://bugs.almalinux.org/" ALMALINUX_MANTISBT_PROJECT="AlmaLinux-8" ALMALINUX_MANTISBT_PROJECT_VERSION="8.7" REDHAT_SUPPORT_PRODUCT="AlmaLinux" REDHAT_SUPPORT_PRODUCT_VERSION="8.7"
If your system is not Fedora 29 but with a similar problem, try to replace os-release with a normal one copied from another normally working Fedora of the same version.
43 Cool, low-footprint gadgets
43.1 figlet
figlet zintis
______ _ _ |__ (_)_ __ | |_(_)___ / /| | '_ \| __| / __|_ / /_| | | | | |_| \__ \ /____|_|_| |_|\__|_|___/
figlet `date +%A\ %b%e` figlet `date +%A\ %b%e` figlet `date +%A\ %b%e`
43.2 command line web browsers
You can set the environment variable BROWSER=lynx
and or BROWSER=w3m
to open
web pages in a command line browser. After that, you can run web searches
such as googler or … to run command line web searches.
43.3 googler
Open source Google search from the command line. Make sure your BROWSER
env
variable is set to a command line browser such as lynx
or w3m
to get the
results on the command line too. See ddgr if you are restricting google for
your personal reasons.
- googler ? will give you the quick instructions.
To install, use dnf install googler
as an example.
You can set the BROWSER variable directly on the same command that you run
googler, or even set an alias goog='BROWSER=w3m googler -j
'
43.4 ddgr
Open source Duck-Duck-Go, ddg search from the command line. Just like with
googler, make sure your BROWSER
env variable is set and expoerted to a command
line browser such as lynx
or w3m
to get the results on the command line too.
- googler ? will give you the quick instructions.
To install, use dnf install googler
as an example.
You can set the BROWSER variable directly on the same command that you run
googler, or even set an alias goog='BROWSER=w3m googler -j
'
43.5 surfraw
The oldest command line search tool, available on http://surfraw.org
You can tell surfraw to search a specific search service like google or duck duck go. Surfraw has many, that are listed in elvi from the surfraw.org home page or direclty here: https://gitlab.com/surfraw/Surfraw/-/wikis/current-elvi
BROSWER=w3m surfraw google "figure eight knot" BROSWER=w3m surfraw duckduckgo "figure eight knot"
44 MacOSX commands (differences)
In MacOSX terminal, there are a few different commands. i.e.
- shell vairables are different
usermod
is missing, in its place isdscl
see man page.sysadminctl -h
for adding users and changing shells etc.shell
as a command does not exist on Mac OSX. Instead you change the shell usingchsh /bin/bash
orchsh /bin/zsh
echo $0
to see what shell you are currently runningecho $SHELL
to see what shell you are currently running (both work)
45 zsh
See link to zsh.org
46 fzf Fuzzy Find
fzf
is simply a unix filter
. Unix filters take some list as STDIN
input,
filter this list, then produces output to STDOUT
. grep for example is also
a unix filter, for example history | grep systemctl
will output to STDOUT
the history output that is filtered for lines containing "systemctl".
What is special about fzf
is that the filter process is interactive where you
have to opportunity to interactively define what the filter will select, and
when completed with hammer
sends the results to STDOUT
. So the above example
could be history | fzf
and the the user starts typing "systemctl" and when
the appropriate line is selected, hammer
will send only that line to STDOUT
That is it in its simplest form. A unix filter.
Instead of just using printenv
, try piping it into fzf
with printenv | fzf
Then once displayed, just start typing
and fzf will search for those strings
in the output of printenv
Very useful! The search is case insensitive,
i.e. fuzzy If you want to restrict the search to consecutive characters
,
case insensitive searches, start the search with tic, i.e. 'home
Putting it all together: printenv | fzf ... then 'path
will search for all
path and PATH strings in your printenv.
46.1 installing
I installed it on my mac with:
brew install fzf
orbrew --cask install fzf
Note, that this installs fzf
as well as the related fzf-tmux
. More on that
later.
46.2 fzf environment variables
There is no .config
file for fzf. All preferences are stored in environment
variables.
FD_OPTIONS="--follow --exclude .git --exclude node_modules --hidden" # no need to export FD_OPTIONS as they are only used in this file itself export FZF_DEFAULT_OPTS="--height 50% -1 --reverse --multi --inline-info --preview='[[ \$(file --mime {}) =~ --binary ]] && echo {} is a binary file || (bat --style=numbers --color=always {} || cat {}) 2> /dev/null | head -300 --prview-window='right:hidden:wrap' --bind='f3: execute(bat --style=numbers {} || less -f {}),f2:toggle-preview,ctrl-d:half-page-down,ctrl-u:half-page-up,ctrl-a:select-all+accept,ctrl-y:execute-silent(echo {+} | pbcopy)'" export FZF_DEFAULT_OPTS='--no-height --color=bg+:#eeee00,gutter:-1,pointer:#fd3b00,info:#dada00,hl:#11eded,h1+#22dfff' export FZF_DEFAULT_COMMAND='find * -type f -not -path "*/\.*"' export FZF_DEFAULT_COMMAND="fd --type f --type l $FD_OPTIONS " export FZF_DEFAULT_COMMAND="git ls-files --cached --others --exclude-standard | fd --type f --type l $FD_OPTIONS " export FZF_DEFAULT_COMMAND="git ls-files --cached --others --exclude-standard | fd --type f --type l $FD_OPTIONS " #The above will use git-ls-files inside a git repo, otherwise will use fd # --color=bg+:#eeee00,gutter:-1,pointer:#fd3b3c,info:#dada05' # C-t searches files and directories (works nicely) export FZF_CTRL_T_COMMAND="fd $FD_OPTIONS" export FZF_CTRL_T_OPTS="--preview 'cat --color=always --line-range :60 {}' " # export FZF_CTRL_T_OPTS="--preview 'cat --color=always --line-range :60 {}' " # export FZF_CTRL_T_OPTS="--preview 'less {}' " # M-c = cd. Searches only directories, and then cd's into them (works nicely) # export FZF_ALT_C_COMMAND='find * -type f -not -path "*/\.*"' export FZF_ALT_C_COMMAND="fd --type d $FD_OPTIONS " export FZF_ALT_C_OPTS="--preview 'tree -C {} | head -60'" export BAT_PAGER="less -R"
Other colour settings can be set just when you execute fzf as follows:
export FZF_DEFAULT_OPTS="--color=fg:#ffffff,bg:#252c31,hl:#c678dd,fg+:#ffffff,bg+:#4b5263,hl+:#d858fe,info:#98c379,prompt:#61afef,pointer:#be5046,marker:#e5c07b,spinner:#61afef,header:#61afef" fzf --color=fg:#ffffff,bg:#252c31,hl:#c678dd,fg+:#ffffff,bg+:#4b5263,hl+:#d858fe,info:#98c379,prompt:#61afef,pointer:#be5046,marker:#e5c07b,spinner:#61afef,header:#61afef env fzf --color=fg:#ff0000,bg:#ffff00 export FZF_DEFAULT_OPTS="--color=fg:#ffffff,bg:#252c31,hl:#c678dd,fg+:#ffffff,bg+:#4b5263,hl+:#d858fe,info:#98c379,prompt:#61afef,pointer:#be5046,marker:#e5c07b,spinner:#61afef,header:#61afef"
If I'm not mistaken, terminal.app doesn't support 24-bit colors (https://github.com/termstandard/colors#not-supporting-truecolor),
- fzf –color=fg:#ff0000,bg:#ffff00
will not work as expected. But it supports 256 ANSI colors fine, you can get identical colors with
- fzf –color=fg:196,bg:226
I was getting:
invalid color specification: h1+#22dfff
And I found my ~/.config/zsh/zsh-exports
file had this line:
- export FZFDEFAULTOPTS='–no-height –color=bg+:#eeee00,gutter:-1,pointer:#fd3b00,#dada00,hl:#11eded,h1+#22dfff'
If I removed the characters: ,hl:#11eded,h1+#22dfff'
fzf would work but with colour codes not
translated to proper colours but left as
46.3 Fzf key bindings
When installed, fzf sets up these three default key bindings
. They can be
changed
using environment variables
.
C-r
searches historyC-t
searches for both files and directoriesM-c
searches for directories andswitches
(cd) into them
These fzf defaults did not work for me as they conflicted with other key bindings I had already set. Maybe I will return to this later.
46.4 Use fzf with bat for preview
You can use the brew install bat
to preview files found with fd
In fact you can change your default fzf command from find
to fd
by setting
the environment variable, FZF_DEFAULT_COMMAND
TO BE 'fd
46.5 Using fzf
46.6 fzf options
Then tried these fzf options
to learn what they would do:
echo 'red\ngreen\nblue'
echo 'red\ngreen\nblue' | fzf
# use the up and down arrow keys and hammerecho 'red\ngreen\nblue' | fzf --multi
# now add TAB key on several linesecho 'red\ngreen\nblue' | fzf --multi --cycle
# now keep pressing up/down-e
restrict finds to exact matches only (like forcing a ')
46.7 interacting with fzf
While interacting with fzf, you can be quite flexible in what fzf will eventually filter for you. Here's the most useful:
46.7.1 hammer to select a single line
With your cursor on a line, hammer will select that line
, and end fzf
with
that line sent to STDOUT
.
46.7.2 <TAB> to select a line (–multi)
~TAB
is actually is a toggle
, so unselects as well. You need to have run
fzf with the --multi
or -m
option, to select multiple lines. All
selected lines will be sent to STDOUT
when you hammer.
46.7.3 regexp supported in fzf
By default all regexp
are supported by fzf, so if you search
home$
fzf will findall lines or items ending with
"home
".^home
will findall lines beginning
with "home
" or "hxoxmxe
" etc.!home
will findall lines NOT CONTAINING
"home
"home$ | .config$
will find all files ending with "home
" or ".config
"'home
will findexact matches
of "home
".!.mp3$
itemsnot ending
in.mp3
46.7.4 glob patterns do NOT function
This is because fzf is already running glob pattern searches, so it does not make sense to use * character. i.e. *.txt will not work.
46.7.5 multiple strings separated by space
If you search for home png$
it will find all lines that have home
and
also end in png
It is a logical OR not an AND, so all lines matching
either string
will be displayed.
46.7.6 the order fzf displays matches
As you type strings into fzf, exact matches
will appear 1st
, with more
fuzzy matches appearing next
. The matched strings (or substrings) will
be coloured, according to your chosed colour scheme set in env. varialbes.
46.7.7 ' to restrict to exact matches
Prefixing
your string with a tic
, will force fzf to only show exact matches
.
46.8 using fzf to search for files
With no parameters
, fzf defaults
to using a list of all files
in the
current directory
, including all subdirectories.
So, vim -R | fzf
will let you search what file you want to edit.
46.9 piping fzf STDOUT into xargs
By piping the output
of fzf into xargs
, you can take action
on your selected
lines
very easily.
fzf -m | xargs ls -la fzf -m | xargs chmod 740
this xargs works with fzf, but does NOT work with –multi ??? Find out why.
46.10 treating fzf stdout as arguments requires use of the back-tic
emacs `fzf` vim `fzf`
46.11 alternatively to using back-tics is ** <tab>
Use of the ** as arguments to a command is also very convenient, and most likely will be more useful than using back-tics as described above.
Usage examples would be:
vim **<tab>
# should only present files that I can editchmod 740 **<tab>
# should only present files that I can chmodssh **<tab>
# only presents hosts in my .sshhosts file.git add C-t
# only finds files that are not part of .gitignore. test this.
this was not working for me…. need to figure out why. vim kind of worked, and allowed me to fuzzy find a file, but hammer just echoed that line out, and had dropped the "vim". hmmmm …..
also git add C-t
while it did present all files for me to fuzzy find, it did
NOT restrict those fuzzy files found to ones that needed to be git added
.
AFAIK, this is broken. See this youtube video on this feature. Maybe the
dude has a .zshrc file or git page that has his settings for me to compare
with mine, to see why mine is not working.
**
is also command sensitive, and will present find options that make sense
with the command it is used with. For example, cd
only makes sense if the
parameter after is a directory
, so fzf will only present directories.
This too was not working for me. I had the zsh tell me "too many parameters".
46.12 piping find into fzf
You can use the tradtional find
command and pipe it into fzf
for a much
more powerfull find:
# find everthing: * # type is a file: -type f # no hidden files: -not -path '*/\.*' find * -type f -not -path '*/\.*' | fzf
You will see that the find show a counter of the thousands of files found so
far. Just start typing a string
, and very quickly you will show all files
names that contain that string
.
46.13 Default fzf find files
Finding files is so common that the default fzf (no parameters) is actually
equivalent to find * -type f | fzf
.
However, you can choose to make this an alias, or even better set up your
default FZF command to use the find
or perhaps fd
to just what you like to
see. See the section fzf environment variables above.
46.14 fzf and functions
You can create functions and run them in your .bashprofile, or just
source them from a file, say fzf-functions
Note that this itself is not a
bash script, as you can easily source them when you need them during an
interactive bash shell session.
# fd - cd to selected directory fd() { local dir dir=$(find ${1:-.} -path '*/\.*' -prune \ -o -type d -print 2> /dev/null | fzf +m) && cd "$dir" } # fh - search in your command history and execute selected command fh() { eval $( ([ -n "$ZSH_NAME" ] && fc -l 1 || history) | fzf +s --tac | sed 's/ *[0-9]* *//') }
46.15 C-t and M-c and C-r
46.15.1 C-t
When fzf shows you the filtered results, hitting hammer
will simply type
that line into your command line. By itself that does nothing.
So what you would normally do is type vim -R C-t
then when you have found
the file
you want, hitting hammer
will be like hitting vim -R file hammer
or also mv C-t
subdirectory will move whatever you found into that subdir.
My M-c did not work as it did in this youtube video. I will investigate later.
46.15.2 M-c cd into fuzzy searched sub-directory
M-c
cd into searched for directories
46.15.3 C-r fuzzy search history
C-r
searches history
46.15.4 C-t fuzzy search files and directory
C-t
searches for both files and directories
A common use case is running a command on it, so chmod 740 C-t
hammer
You will see the resulting list on your command line so you will have
to hammer
again. Of course this allows you to still edit the command
or list to fit your needs.
chmod 740 C-t
chmod 740 C-t
chmod 740 C-t
rm C-t
can select multiple files to remove, and hammer, hammer to completedopen C-t
then.png$
andhammer
to open an image file. easy peasy.
If you have your preview set up properly, like:
export FZF_CTRL_T_OPTS="--preview 'bat --color=always --line-range :60 {}'"
Then you can even check what files you will be selecting, right in fsf.
46.16 Limit directory depth
This is not done in fzf
, but in the find
or fd
commands.
For instance, usually I'll have my options set like this:
FD_OPTIONS="--follow --exclude .git --exclude node_modules --hidden" export FZF_DEFAULT_COMMAND="fd --type f --type l $FD_OPTIONS " export FZF_CTRL_T_COMMAND="fd $FD_OPTIONS" export FZF_CTRL_T_OPTS="--preview 'cat --color=always --line-range :60 {}' "
46.16.1 fd –max-depth 1
I can change this by adding -d 1
to limit to current directory. a.k.a.
--max-depth
and update the environment variables.
46.16.2 find -maxdepth 1
I can change find also with -maxdepth 1
Or, I can do that temporarily like this:
printenv | fzf <fzf to find the environment variable I want to change>
then change it, for this zsh session ONLY.
export FZF_CTRL_T_COMMAND="fd --follow --exclude .git --exclude node_modules --hidden -d 1"
This worked, but maybe I can make it more streamlined…
46.17 cd vs M-c
You can use M-c
as shown above, but you can also simply use the cd command
like this:
cd $(find . -type d -print | fzf)