Saturday, May 11, 2019
Wednesday, May 8, 2019
Controlling Access to Files with Linux file system permissions
Linux file system permissions
Access to files by users are controlled by file permission. The Linux file permissions system is simple but flexible, which makes it easy to understand and apply, yet able to handle most normal permission cases easily.
There are also just three categories of permissions which apply: read, write and execute. These permissions affect access to files and directories as follow:
r(read) - Content of the file can be read and content of the directory (file names) can be listed.
w(write) - Content of the file can be changed and any file in the directory may be created or deleted.
x(exec) - File can be executed as commands and Content of the directory can be accessed (dependent on the permissions of he files in the directory).
Changing file/directory permissions
The command used to change permissions from the command line is chmod, short for "change mode" (permissions are also called the mode of a file). The chmod command takes a permission instruction followed by a list of files or directories to change. The permission instruction can be issued either symbolically (the symbolic method) or (the numeric method).
Symbolic method keywords:
chmod WhoWhatWhich file|directory
Who is u,g,o,a (for user,group,other,all)
What is +,-,= (for add, remove, set exactly)
Which is r,w,x (for read, write, executable)
Numeric method:
chmod ### file | directory
Each digit represents an access level: user, group, other.
# is sum of r=4 ,w=2, and x=1.
Using the numeric method, permissions are represented by a three-digit (or four, when setting advance permissions) octal number. A single octal digit can represent the numbers 0-7, exactly the number of possibilities for a three-bit number.
Examples
Remove read and write permission for group and other on file1:
[student@desktop ~]$ chmod go-rw file1
Add execute permission for everyone on file2
[student@desktop ~]$ chmod a+x file2
Set read,write,and execute permission for user,read, and execute for group, and no permission for other on sampledir.
[student@desktop ~]$ chmod 750 sampledir
Changing file/directory user or group ownership
File ownership can be changed with the chown command. For example, to grant ownership of the file foofile to student, the following command could be used:
[root@desktop ~]$ chown student foofile
chown can be used with the -R option to recursively change the ownership of an entire directory tree. The following command would grant ownership of foodir and all files and subdirectories within it to student:
[root@desktop ~]$ chown -R student foodir
The chown command can also be used to change group ownership of a file by preceding the group name with a colon (:). For example, the following command will change the group foodir to admins:
[root@desktop ~]$ chown :admins foodir
The chown command can also be used to change both owner and group at the same time by using the syntax owner:group. For example, to change the ownership of foodir to visitor and the group to guests, use:
[root@desktop ~]$ chown visitor:guests foodir
Only root can change the ownership of a file. Group ownership, however, can be set by root or files owner. root can grant ownership to any group, while non-root users can grant ownership only to groups they belong to.
Special permisions
The setuid (or setgid) permission on an executable file means that the command will run as the user (or group) of the file, not as the user that ran the command. One example is the passwd command:
[student@desktop ~]$ ls -l /usr/bin/passwd
-rwsr-xr-x. 1 root root 35504 jul 16 2010 /usr/bin/passwd
The sticky bit for a directory sets a special restriction on deletion of files: Only the owner of the file (and root) can delete files within the directory. An example is /tmp:
[student@desktop ~]$ ls -ld /tmp
Effect of special permissions on files and directories
u+s (suid) - File executes as the user that owns the file, not user that ran the file. No effect on directories.
g+s (sgid) - File execute as the group that owns the file. and Files newly created in the directory have their group owner set to match the group owner of the directory.
o+t (sticky) - No effect on files. and Users with write on the directory can only remove files that they own; they cannot remove or force saves to files owned by others users.
Setting special permission
symbolically; setuid = u+s; setgid = g+s; sticky = o+t
Numerically (Fourth preceding digit): setuid = 4; setgid=3;
sticky = 1
Examples;
Add the setgid bit on directory
[root@desktop ~]$ chmod g+s directory
set the setgid bit, and read/write/execute for user and group on directory:
[root@desktop ~]$ chmod 2770 directory
Access to files by users are controlled by file permission. The Linux file permissions system is simple but flexible, which makes it easy to understand and apply, yet able to handle most normal permission cases easily.
There are also just three categories of permissions which apply: read, write and execute. These permissions affect access to files and directories as follow:
r(read) - Content of the file can be read and content of the directory (file names) can be listed.
w(write) - Content of the file can be changed and any file in the directory may be created or deleted.
x(exec) - File can be executed as commands and Content of the directory can be accessed (dependent on the permissions of he files in the directory).
Changing file/directory permissions
The command used to change permissions from the command line is chmod, short for "change mode" (permissions are also called the mode of a file). The chmod command takes a permission instruction followed by a list of files or directories to change. The permission instruction can be issued either symbolically (the symbolic method) or (the numeric method).
Symbolic method keywords:
chmod WhoWhatWhich file|directory
Who is u,g,o,a (for user,group,other,all)
What is +,-,= (for add, remove, set exactly)
Which is r,w,x (for read, write, executable)
Numeric method:
chmod ### file | directory
Each digit represents an access level: user, group, other.
# is sum of r=4 ,w=2, and x=1.
Using the numeric method, permissions are represented by a three-digit (or four, when setting advance permissions) octal number. A single octal digit can represent the numbers 0-7, exactly the number of possibilities for a three-bit number.
Examples
Remove read and write permission for group and other on file1:
[student@desktop ~]$ chmod go-rw file1
Add execute permission for everyone on file2
[student@desktop ~]$ chmod a+x file2
Set read,write,and execute permission for user,read, and execute for group, and no permission for other on sampledir.
[student@desktop ~]$ chmod 750 sampledir
Changing file/directory user or group ownership
File ownership can be changed with the chown command. For example, to grant ownership of the file foofile to student, the following command could be used:
[root@desktop ~]$ chown student foofile
chown can be used with the -R option to recursively change the ownership of an entire directory tree. The following command would grant ownership of foodir and all files and subdirectories within it to student:
[root@desktop ~]$ chown -R student foodir
The chown command can also be used to change group ownership of a file by preceding the group name with a colon (:). For example, the following command will change the group foodir to admins:
[root@desktop ~]$ chown :admins foodir
The chown command can also be used to change both owner and group at the same time by using the syntax owner:group. For example, to change the ownership of foodir to visitor and the group to guests, use:
[root@desktop ~]$ chown visitor:guests foodir
Only root can change the ownership of a file. Group ownership, however, can be set by root or files owner. root can grant ownership to any group, while non-root users can grant ownership only to groups they belong to.
Special permisions
The setuid (or setgid) permission on an executable file means that the command will run as the user (or group) of the file, not as the user that ran the command. One example is the passwd command:
[student@desktop ~]$ ls -l /usr/bin/passwd
-rwsr-xr-x. 1 root root 35504 jul 16 2010 /usr/bin/passwd
The sticky bit for a directory sets a special restriction on deletion of files: Only the owner of the file (and root) can delete files within the directory. An example is /tmp:
[student@desktop ~]$ ls -ld /tmp
Effect of special permissions on files and directories
u+s (suid) - File executes as the user that owns the file, not user that ran the file. No effect on directories.
g+s (sgid) - File execute as the group that owns the file. and Files newly created in the directory have their group owner set to match the group owner of the directory.
o+t (sticky) - No effect on files. and Users with write on the directory can only remove files that they own; they cannot remove or force saves to files owned by others users.
Setting special permission
symbolically; setuid = u+s; setgid = g+s; sticky = o+t
Numerically (Fourth preceding digit): setuid = 4; setgid=3;
sticky = 1
Examples;
Add the setgid bit on directory
[root@desktop ~]$ chmod g+s directory
set the setgid bit, and read/write/execute for user and group on directory:
[root@desktop ~]$ chmod 2770 directory
Managing Local Linux users and groups
What is a user?
Every process on the system runs as a particular user. Every file is owned by a particular user. Access to files and directories are restricted by user. The user associate with a running process determine the files and directories accessible to that process.
id - The id command is used to show information about the current logged-in user. Basic information about another user can also be requested by passing in the username of that user as the first argument to the id command.
[student@desktop ~]$ id
uid=1000(student) gid=1000(student) groups=1000(student), 10(wheel)
To view the user associated with a file or directory, use the ls -l command. The third column shows the username:
[student@desktop ~]$ ls -l /tmp
drwx-----. 2 gdm gdm
drwx-----. 2 student student
-rw-r--r-. 1 root root
To view process information, use the ps command. The default is to show only processes in the current shell. Add the a option to view all processes with a terminal. To view the user associated with a process, include the u option. The first column show the username:
[student@desktop ~]$ ps au
User PID %CPU %MEM
root
root
student
The output of the previous commands displays users by name but internally the operating system tracks users by a UID number. The mapping of names to number is defined in databases of account information. By default, system use a simple "flat file". the /etc/passwd file to strore information about local users. The format of /etc/passwd follows (seven colon-separated fields):
username: It is a mapping of a UID to a name for the benefit of human users.
password: password is where, historically, passwords were kept in an encrypted format. Today, they are stored in a separate file called /etc/shadow.
UID: It is a user ID, a number that identifies the user at the most fundamental level.
GID: It is user's primary group ID number. Groups will be discussed in a moment.
GECOS: GECOS field is arbitrary text, which usually includes the user's real name.
/home/dir - It is the location of the user's personal data and configuration files.
shell is a program that runs as the user logs in. For a regular user, this is normally the program that provide the user's command line prompt.
What is a group?
Like users, groups have a name and a number (GID). Local groups are defined in /etc/groups.
Primary groups
Every user has exactly one primary group
For local users, the primary group is defined by the GID number of the group listed in the third field of /etc/passwd
Normally, the primary group owns new files created by the user.
Normally the primary group of newly created user is a newly created group with the same name as the user. The user is the only member of this User Private Group (UPG)
Supplementary groups
User may be a member of zero or more supplementary groups.
The user that are supplementary members of local groups are listed in the last field of the groups entry in /etc/group. For local groups, user membership is determined by a comma-separated list of users found in the last field of the groups entry in /etc/group:
groupname:password:GID:list,of,users,in,this,group
Supplementary group membership is used to help ensure that users have access permissions to files and other resources on the system.
Gaining Superuser Access
The root user
Most operating systems have some sort of superuser, a user that has all power over the system. This user in Red Hat Enterprise Linux is the root user. This user has the power to override normal privileges on the file system and is used to manage and administer the system. In order to perform tasks such as installing or removing software and to manage system files and directories, a user must escalate privileges to the root users.
The root account on Linux is roughly equivalent to the local Administrator account on Windows. In Linux, most system administrators log into an unprivileged user account and use various tools to temporarily gain root privileges.
Switching users with su
The su command allows a user to switch to a different user account. If a username is not specified, the root account is implied. When invokes as a regular user, a prompt will display asking for the password of the account you are switching to; when invoked as root, there is no need to enter the account password.
su [-] <username>
[student@desktop ~]$ su -
Password: redhat
The command su username starts a non-login shell, while the command su - username starts a login shell. The main distinction is su - sets up the shell environment as if this were a clean login as that user, while su just starts a shell as that user with the current environment setting
Running commands as root with sudo
The sudo command allows a user to be permitted to run a command as root, or as another user, based on setting in the /etc/sudoers file. Unlike other tools such as su, sudo requires users to enter their own password for authentication, not the password of the account they are trying to access. This allows an administrator to hand out fine-grained permissions to users to delegate system administration task, without having to hand out the root password.
For example when sudo has been configured to allow the user student to run the command usermod as root, student could run the following command to lock a user account
[student@server~]$ sudo usermod -L username
[sudo] password for student: password
One additional benefit to using sudo is that all commands executed using sudo are logged by default to /var/log/secure
Managing local users
Useradd creates users
useradd username sets reasonable defaults for all fields in /etc/passwd when run without options. The useradd command does not set any valid password by default, and the user cannot log in until a password is set.
useradd --help will display the basic option that can be user to override the defaults.
usermod modifies existing users
usermod --help will display the basic options that can be used to modify an account.
userdel delete users
userdel username removes the user from /etc/passwd, but leaves the home directory intact by default/
userdel -r username removes the user and the users home directory.
E.g.
[root@server ~]# useradd prince
[root@server ~]# ls -l /home
[root@server ~]# userdel prince
[root@server ~]# ls -l /home
[root@server ~]# useradd bob
[root@server ~]# ls -l /home
UID ranges
Specific UID numbers and ranges of numbers are used for specific purpose by Red Hat Enterprise Linux.
UID 0 is always assigned to the superuser account, root.
UID 1-200 is a range of "system users" assigned statically to system process by Red Hat.
UID 201-999 is a range of "system user" user by system processes that do not own files on the file system. They are typically assigned dynamically from the available pool when the software that needs them is installed. programs run as these "Unprivileged" system users in order to limit their access to just the resources they need to function.
UID 1000+ is the range available for assignment to regular users.
Managing supplementary groups
Groupadd creates groups
groupadd groupname without options uses the next available GID from the range specified in the /etc/login.defs file
The -g GID option is used to specify a specific GID.
[student@server ~]$ sudo groupadd -g 5000 ateam
Always GID greater than 1000+ to avoid collision with system groups
The -r option will create a system group usig a GID from the range of valid system GID numbers listed in the /etc/login.defs file.
[student@server ~]$ sudo groupadd -r appusers
groupmod modifies existing groups
The groupmod command is used to change a group name to a GID mapping. The -n option is used to specify a new name.
[student@server ~]$ sudo groupmod -n javaapp appusers
The -g option is used to specify a new GID.
[student@server ~]$ sudo groupmod -g 6000 ateam
groupdel deletes a group
The groupdel command will remove a group
[student@server ~]$ sudo groupdel javaapp
usermod alters group membership
The membership of a group is controlled with user management. Change a user's primary group with usermod -g groupname
[student@server ~]$ sudo usermod -g student student
Add a user to a supplementary group with usermode -aG groupname username.
[student@server ~]$ sudo usermod -aG Wheel elvis
Every process on the system runs as a particular user. Every file is owned by a particular user. Access to files and directories are restricted by user. The user associate with a running process determine the files and directories accessible to that process.
id - The id command is used to show information about the current logged-in user. Basic information about another user can also be requested by passing in the username of that user as the first argument to the id command.
[student@desktop ~]$ id
uid=1000(student) gid=1000(student) groups=1000(student), 10(wheel)
To view the user associated with a file or directory, use the ls -l command. The third column shows the username:
[student@desktop ~]$ ls -l /tmp
drwx-----. 2 gdm gdm
drwx-----. 2 student student
-rw-r--r-. 1 root root
To view process information, use the ps command. The default is to show only processes in the current shell. Add the a option to view all processes with a terminal. To view the user associated with a process, include the u option. The first column show the username:
[student@desktop ~]$ ps au
User PID %CPU %MEM
root
root
student
The output of the previous commands displays users by name but internally the operating system tracks users by a UID number. The mapping of names to number is defined in databases of account information. By default, system use a simple "flat file". the /etc/passwd file to strore information about local users. The format of /etc/passwd follows (seven colon-separated fields):
username: It is a mapping of a UID to a name for the benefit of human users.
password: password is where, historically, passwords were kept in an encrypted format. Today, they are stored in a separate file called /etc/shadow.
UID: It is a user ID, a number that identifies the user at the most fundamental level.
GID: It is user's primary group ID number. Groups will be discussed in a moment.
GECOS: GECOS field is arbitrary text, which usually includes the user's real name.
/home/dir - It is the location of the user's personal data and configuration files.
shell is a program that runs as the user logs in. For a regular user, this is normally the program that provide the user's command line prompt.
What is a group?
Like users, groups have a name and a number (GID). Local groups are defined in /etc/groups.
Primary groups
Every user has exactly one primary group
For local users, the primary group is defined by the GID number of the group listed in the third field of /etc/passwd
Normally, the primary group owns new files created by the user.
Normally the primary group of newly created user is a newly created group with the same name as the user. The user is the only member of this User Private Group (UPG)
Supplementary groups
User may be a member of zero or more supplementary groups.
The user that are supplementary members of local groups are listed in the last field of the groups entry in /etc/group. For local groups, user membership is determined by a comma-separated list of users found in the last field of the groups entry in /etc/group:
groupname:password:GID:list,of,users,in,this,group
Supplementary group membership is used to help ensure that users have access permissions to files and other resources on the system.
Gaining Superuser Access
The root user
Most operating systems have some sort of superuser, a user that has all power over the system. This user in Red Hat Enterprise Linux is the root user. This user has the power to override normal privileges on the file system and is used to manage and administer the system. In order to perform tasks such as installing or removing software and to manage system files and directories, a user must escalate privileges to the root users.
The root account on Linux is roughly equivalent to the local Administrator account on Windows. In Linux, most system administrators log into an unprivileged user account and use various tools to temporarily gain root privileges.
Switching users with su
The su command allows a user to switch to a different user account. If a username is not specified, the root account is implied. When invokes as a regular user, a prompt will display asking for the password of the account you are switching to; when invoked as root, there is no need to enter the account password.
su [-] <username>
[student@desktop ~]$ su -
Password: redhat
The command su username starts a non-login shell, while the command su - username starts a login shell. The main distinction is su - sets up the shell environment as if this were a clean login as that user, while su just starts a shell as that user with the current environment setting
Running commands as root with sudo
The sudo command allows a user to be permitted to run a command as root, or as another user, based on setting in the /etc/sudoers file. Unlike other tools such as su, sudo requires users to enter their own password for authentication, not the password of the account they are trying to access. This allows an administrator to hand out fine-grained permissions to users to delegate system administration task, without having to hand out the root password.
For example when sudo has been configured to allow the user student to run the command usermod as root, student could run the following command to lock a user account
[student@server~]$ sudo usermod -L username
[sudo] password for student: password
One additional benefit to using sudo is that all commands executed using sudo are logged by default to /var/log/secure
Managing local users
Useradd creates users
useradd username sets reasonable defaults for all fields in /etc/passwd when run without options. The useradd command does not set any valid password by default, and the user cannot log in until a password is set.
useradd --help will display the basic option that can be user to override the defaults.
usermod modifies existing users
usermod --help will display the basic options that can be used to modify an account.
userdel delete users
userdel username removes the user from /etc/passwd, but leaves the home directory intact by default/
userdel -r username removes the user and the users home directory.
E.g.
[root@server ~]# useradd prince
[root@server ~]# ls -l /home
[root@server ~]# userdel prince
[root@server ~]# ls -l /home
[root@server ~]# useradd bob
[root@server ~]# ls -l /home
UID ranges
Specific UID numbers and ranges of numbers are used for specific purpose by Red Hat Enterprise Linux.
UID 0 is always assigned to the superuser account, root.
UID 1-200 is a range of "system users" assigned statically to system process by Red Hat.
UID 201-999 is a range of "system user" user by system processes that do not own files on the file system. They are typically assigned dynamically from the available pool when the software that needs them is installed. programs run as these "Unprivileged" system users in order to limit their access to just the resources they need to function.
UID 1000+ is the range available for assignment to regular users.
Managing supplementary groups
Groupadd creates groups
groupadd groupname without options uses the next available GID from the range specified in the /etc/login.defs file
The -g GID option is used to specify a specific GID.
[student@server ~]$ sudo groupadd -g 5000 ateam
Always GID greater than 1000+ to avoid collision with system groups
The -r option will create a system group usig a GID from the range of valid system GID numbers listed in the /etc/login.defs file.
[student@server ~]$ sudo groupadd -r appusers
groupmod modifies existing groups
The groupmod command is used to change a group name to a GID mapping. The -n option is used to specify a new name.
[student@server ~]$ sudo groupmod -n javaapp appusers
The -g option is used to specify a new GID.
[student@server ~]$ sudo groupmod -g 6000 ateam
groupdel deletes a group
The groupdel command will remove a group
[student@server ~]$ sudo groupdel javaapp
usermod alters group membership
The membership of a group is controlled with user management. Change a user's primary group with usermod -g groupname
[student@server ~]$ sudo usermod -g student student
Add a user to a supplementary group with usermode -aG groupname username.
[student@server ~]$ sudo usermod -aG Wheel elvis
Tuesday, May 7, 2019
Creating Viewing and Editing Text File
Standard input, standard output and standard error -
A process structure is constructed with numbered channels (file description) to manage open files. Process connect to files to reach data content or devices these files represent. Processes are created with default connections for channels 0,1, and 2 known as standard input, standard output and standard error. Processes use channel 3 and above to connect to other files.
channels (File Descriptors)
Table
Output Redirection Operators
>file - redirect stdout to a file(1)
>>file - redirect stdout to a file, append to current file content(2)
2>file - redirect stderr to a file(1)
2>/dev/null - discard stderr error message by redirecting to /dev/null
&>file - combine stdout and stderr to one file (1)
>>file 2>&1 - combine stdout and stderr, append to current file content (2)(3)
Note: -
(1) overwrite existing file, create file if new.
(2) Append existing file, create file if new.
(3) The order of direction is important to avoid unexpected command behavior.
2>&1 sends stderr to the same place as stdout. For this work, stdout needs to be directed first, before adding stderr to stdout. Although &>> is an alternate way to append both stdout and stderr to a file, 2>&1 is the method needed to send both stdout and stderr through a pipe.
Example for output redirection
Save a timestamp for later reference
[student@desktop ~]$ date > /tmp/saved-timestamp
Copy the last 100 lines from a log file to another file.
[student@desktop ~]$ tail -n 100 /var/log/dmseg > /tmp/last-100-boot-message
Concatenate four lines into one.
[student@desktop ~]$ cat file1 file2 file3 file4 >tmp/all-four-in-one
List the home directory's hidden and regular file names into a file.
[student@desktop ~]$ ls -a >tmp/my-file-name
Append output to an existing file
[student@desktop ~]$ echo "new line of information" >> /tmp/many-lines-of-information
[student@desktop ~]$ diff previous-file current-file >> /tmp/tracking-changes-made
In the next examples, errors are generated since normal users are denied access to system directories. Redirect error to a file while viewing normal command output on the terminal.
[student@desktop ~]$ find /etc -name passwd 2> tmp/errors
Save process output and error message to separate files.
[student@desktop ~]$ find /etc -name passwd > /tmp/output 2> /tmp/errors
Ignore and discard error message.
[student@desktop ~]$ find /etc -name passwd > /tmp/output 2>dev/null
Store output and generated errors together
[student@desktop ~]$ find /etc -name passwd &> /tmp/save-both
Append output and generated errors to an existing file.
[student@desktop ~]$ find /etc -name passwd >> /tmp/save-both 2>&1
Constructing pipelines
Redirection controls channel output to or from files while piping sends channel output to another process
Examples for process pipeline redirection
Paginate a command's long ouput
[student@desktop ~]$ ls -l /usr/bin | less
Count the number of lines in an output or listing.
[student@desktop ~]$ ls | wc -l > /tmp/how-many-files
Grab the first lines, last lines, or selected lines of command output.
[student@desktop ~]$ ls -t | head -n 10 > /tmp/ten-last-changed-files
Example for using the tee command for piping
The Tee command displays or redirects the intermediate result normally suppressed due to piping, In the first example, the ls listing is viewed on a terminal while simultaneously being stored in a file.
[student@desktop ~]$ ls -l | tee /tmp/saved-output
Determine the terminal device for the current window. Send the results as mail and view the same results in his window.
[student@desktop ~]$ tty
/dev/pts/0
[student@desktop ~]$ ls -l | tee /dev/pts/0 | mail -s subject
student@desktop1.example.com
Editing files with Vim
Vim is improved version of the vi editor distributed with Linux and Unix systems. Vim is highly configurable and effective for practiced users, including such features as split screen editing, color formatting and highlighting for editing text.
An i keystroke enters insert mode, where all text typed becomes file content. Pressing Esc return to command mode.
A v keystroke enters visual mode, where multiple characters may be selected for text manipulation. use V for multi-line and Ctrl-v for block selection. The same keystroke used to enter visual mode (v, V or Ctrl-v) is used to exit.
The : keystrokes begins extended command mode for tasks like writing the file to save it, and quitting the Vim editor
The instructor will demonstrate a typical file editing session using only basic Vim keystrokes.
1. Open a file with vim filename.
2. Repeat this text entry cycle, as many time as the task requires:
Use arrow key to position the cursor.
Press i to enter insert mode.
Enter Text
Press Esc to return to command mode.
If necessary, press u to undo mistaken edits on the current file.
3. Repeat this text deletion cycle, as many times as the task requires:
Use arrow keys to position the cursor.
Press X to delete a selection of text
If necessary, use u to undo mistaken edits on the current file.
4. To save or exit, choose one of the following to write or discard file edits:
Enter :w to write(save) the file and remain in command mode for ore editing
Enter :wq to write the file and quit Vim.
Enter :q! to quit Vim, but discard all file changes since the last write.
The instructor will demonstrate "yank and put" using visual mode.
1. Open a file with vim filename.
2. Repeat this text entry cycle, as many time as the task requires:
Use arrow key to position the cursor to the first character
Press v to enter visual mode.
Use arrow keys to position the cursor to the last character.
Press y to yank(copy) the selection.
Use arrow key to position the cursor at the insert location.
Press p to put (paste) the selection.
3. Repeat this text deletion cycle, as many times as the task requires:
Use arrow keys to position the cursor.
Press X to delete a selection of text
If necessary, use u to undo mistaken edits on the current file.
4. To save or exit, choose one of the following to write or discard file edits:
Enter :w to write(save) the file and remain in command mode for ore editing
Enter :wq to write the file and quit Vim.
Enter :q! to quit Vim, but discard all file changes since the last write.
A process structure is constructed with numbered channels (file description) to manage open files. Process connect to files to reach data content or devices these files represent. Processes are created with default connections for channels 0,1, and 2 known as standard input, standard output and standard error. Processes use channel 3 and above to connect to other files.
channels (File Descriptors)
Table
Output Redirection Operators
>file - redirect stdout to a file(1)
>>file - redirect stdout to a file, append to current file content(2)
2>file - redirect stderr to a file(1)
2>/dev/null - discard stderr error message by redirecting to /dev/null
&>file - combine stdout and stderr to one file (1)
>>file 2>&1 - combine stdout and stderr, append to current file content (2)(3)
Note: -
(1) overwrite existing file, create file if new.
(2) Append existing file, create file if new.
(3) The order of direction is important to avoid unexpected command behavior.
2>&1 sends stderr to the same place as stdout. For this work, stdout needs to be directed first, before adding stderr to stdout. Although &>> is an alternate way to append both stdout and stderr to a file, 2>&1 is the method needed to send both stdout and stderr through a pipe.
Example for output redirection
Save a timestamp for later reference
[student@desktop ~]$ date > /tmp/saved-timestamp
Copy the last 100 lines from a log file to another file.
[student@desktop ~]$ tail -n 100 /var/log/dmseg > /tmp/last-100-boot-message
Concatenate four lines into one.
[student@desktop ~]$ cat file1 file2 file3 file4 >tmp/all-four-in-one
List the home directory's hidden and regular file names into a file.
[student@desktop ~]$ ls -a >tmp/my-file-name
Append output to an existing file
[student@desktop ~]$ echo "new line of information" >> /tmp/many-lines-of-information
[student@desktop ~]$ diff previous-file current-file >> /tmp/tracking-changes-made
In the next examples, errors are generated since normal users are denied access to system directories. Redirect error to a file while viewing normal command output on the terminal.
[student@desktop ~]$ find /etc -name passwd 2> tmp/errors
Save process output and error message to separate files.
[student@desktop ~]$ find /etc -name passwd > /tmp/output 2> /tmp/errors
Ignore and discard error message.
[student@desktop ~]$ find /etc -name passwd > /tmp/output 2>dev/null
Store output and generated errors together
[student@desktop ~]$ find /etc -name passwd &> /tmp/save-both
Append output and generated errors to an existing file.
[student@desktop ~]$ find /etc -name passwd >> /tmp/save-both 2>&1
Constructing pipelines
Redirection controls channel output to or from files while piping sends channel output to another process
Examples for process pipeline redirection
Paginate a command's long ouput
[student@desktop ~]$ ls -l /usr/bin | less
Count the number of lines in an output or listing.
[student@desktop ~]$ ls | wc -l > /tmp/how-many-files
Grab the first lines, last lines, or selected lines of command output.
[student@desktop ~]$ ls -t | head -n 10 > /tmp/ten-last-changed-files
Example for using the tee command for piping
The Tee command displays or redirects the intermediate result normally suppressed due to piping, In the first example, the ls listing is viewed on a terminal while simultaneously being stored in a file.
[student@desktop ~]$ ls -l | tee /tmp/saved-output
Determine the terminal device for the current window. Send the results as mail and view the same results in his window.
[student@desktop ~]$ tty
/dev/pts/0
[student@desktop ~]$ ls -l | tee /dev/pts/0 | mail -s subject
student@desktop1.example.com
Editing files with Vim
Vim is improved version of the vi editor distributed with Linux and Unix systems. Vim is highly configurable and effective for practiced users, including such features as split screen editing, color formatting and highlighting for editing text.
An i keystroke enters insert mode, where all text typed becomes file content. Pressing Esc return to command mode.
A v keystroke enters visual mode, where multiple characters may be selected for text manipulation. use V for multi-line and Ctrl-v for block selection. The same keystroke used to enter visual mode (v, V or Ctrl-v) is used to exit.
The : keystrokes begins extended command mode for tasks like writing the file to save it, and quitting the Vim editor
The instructor will demonstrate a typical file editing session using only basic Vim keystrokes.
1. Open a file with vim filename.
2. Repeat this text entry cycle, as many time as the task requires:
Use arrow key to position the cursor.
Press i to enter insert mode.
Enter Text
Press Esc to return to command mode.
If necessary, press u to undo mistaken edits on the current file.
3. Repeat this text deletion cycle, as many times as the task requires:
Use arrow keys to position the cursor.
Press X to delete a selection of text
If necessary, use u to undo mistaken edits on the current file.
4. To save or exit, choose one of the following to write or discard file edits:
Enter :w to write(save) the file and remain in command mode for ore editing
Enter :wq to write the file and quit Vim.
Enter :q! to quit Vim, but discard all file changes since the last write.
The instructor will demonstrate "yank and put" using visual mode.
1. Open a file with vim filename.
2. Repeat this text entry cycle, as many time as the task requires:
Use arrow key to position the cursor to the first character
Press v to enter visual mode.
Use arrow keys to position the cursor to the last character.
Press y to yank(copy) the selection.
Use arrow key to position the cursor at the insert location.
Press p to put (paste) the selection.
3. Repeat this text deletion cycle, as many times as the task requires:
Use arrow keys to position the cursor.
Press X to delete a selection of text
If necessary, use u to undo mistaken edits on the current file.
4. To save or exit, choose one of the following to write or discard file edits:
Enter :w to write(save) the file and remain in command mode for ore editing
Enter :wq to write the file and quit Vim.
Enter :q! to quit Vim, but discard all file changes since the last write.
Monday, May 6, 2019
Managing files from the command line
File system hierarchy
/ - This is the system's root directory.
/root - This is the root account's home directory
/home - User home directories are located under this directory.
/etc - This directory contains static, persistent system configuration data.
/var - This directory contains dynamic configuration data, such as FTP and websites.
/usr/bin - Regular user commands and utilities are located here.
/usr/sbin - System administration binaries, for root user, are here.
/tmp - Temporary files are stored here.
/run - Contains dynamic, non-persistent application runtime data.
/usr - Contains installed software programs and libraries.
Locating Files and Directories
ls -l ~ - List the current user's home directory (long format) in simplest syntax, when it is not the current location.
cd - Return to the current user's home directory.
pwd - Determine the absolute path name of the current location.
cd - -Return to the most previous working directory.
cd ../.. - Move up two levels from the current location.
ls -al - List the current location (long format) with hidden files.
cd /bin - Move to the binaries location, from any current location.
cd.. - Move up to the parent of the current location.
cd bin - Move to the binaries location, from the root directory.
Path Name Expansion
b* - Only filename beginning with "b"
*b - Only filenames ending in "b"
*b* - Only filenames containing a "b"
[!b*] - Only filenames where first character is not "b"
???* - Only filenames at least 3 characters in length
*[[:digit:]]* - Only filenames that contain a number
[[]:upper:]* - Only filenames that begin with an upper-case letter
Navigating paths
pwd - The pwd command display the full path name of the current location.
E.g. [student@desktop ~]$ pwd
/home/student
ls - The ls command lists directory contents for the specified directory or, if no directory is given, for the current directory.
E.g. [student@desktop ~]$ ls
Desktop Documents Download Music Pictures
Public Templates Videos
cd - use the cd command to change directories.With a working directory of /home/student, relative path syntax is shortest to read the Videos subdirectory. The Documents subdirectory is then reached using absolute path syntax.
E.g.
[student@desktop ~]$ cd Videos
[student@desktop Videos ~]$ pwd
/home/students/Videos
[student@desktop Videos ~]$ cd /home/student/Documents
[student@desktop Documents ~]$ pwd
/home/student/Documents
[student@desktop Documents ~]$ cd
[student@desktop ~]$ pwd
/home/student
touch - The touch command normally updates a file's timestamp to the current date and time without otherwise modifying it. This is useful for creating empty files, which can be used for practice, since "touching' a file name that does not exist caused the file to be created. Using touch, practice files are created in the Documents and Videos subdirectories.
E.g.
[student@desktop ~]$ touch Videos/blockbuster1.ogg
[student@desktop ~]$ touch Videos/blockbuster2.ogg
[student@desktop ~]$ touch Documents/thesis_chapter1.odf
[student@desktop ~]$ touch Documents/thesis_chapter2.odf
ls - The ls command has multiple options for displaying attributes on file. The most command and useful are -l (long listing format), -a (all files, includes hidden files), and -R(recursive, to include the contents of all subdirectories).
E.g.
[student@desktop ~]$ ls -l
[student@desktop ~]$ ls -a
[student@desktop ~]$ ls - R
cd -
E.g.
[student@desktop ~]$ cd Videos
[student@desktop Videos ~]$ pwd
/home/student/Videos
[student@desktop Videos ~]$ cd /home/student/Documents
[student@desktop Documents ~]$ pwd
/home/student/Documents
[student@desktop Documents ~]$ cd -
[student@desktop Videos ~]$ pwd
/home/student/Videos
[student@desktop Videos ~]$ cd -
[student@desktop Documents ~]$ pwd
/home/student/Documents
cd.. -
E.g.
[student@desktop Videos ~]$ pwd
/home/student/Videos
/ - This is the system's root directory.
/root - This is the root account's home directory
/home - User home directories are located under this directory.
/etc - This directory contains static, persistent system configuration data.
/var - This directory contains dynamic configuration data, such as FTP and websites.
/usr/bin - Regular user commands and utilities are located here.
/usr/sbin - System administration binaries, for root user, are here.
/tmp - Temporary files are stored here.
/run - Contains dynamic, non-persistent application runtime data.
/usr - Contains installed software programs and libraries.
Locating Files and Directories
ls -l ~ - List the current user's home directory (long format) in simplest syntax, when it is not the current location.
cd - Return to the current user's home directory.
pwd - Determine the absolute path name of the current location.
cd - -Return to the most previous working directory.
cd ../.. - Move up two levels from the current location.
ls -al - List the current location (long format) with hidden files.
cd /bin - Move to the binaries location, from any current location.
cd.. - Move up to the parent of the current location.
cd bin - Move to the binaries location, from the root directory.
Path Name Expansion
b* - Only filename beginning with "b"
*b - Only filenames ending in "b"
*b* - Only filenames containing a "b"
[!b*] - Only filenames where first character is not "b"
???* - Only filenames at least 3 characters in length
*[[:digit:]]* - Only filenames that contain a number
[[]:upper:]* - Only filenames that begin with an upper-case letter
Navigating paths
pwd - The pwd command display the full path name of the current location.
E.g. [student@desktop ~]$ pwd
/home/student
ls - The ls command lists directory contents for the specified directory or, if no directory is given, for the current directory.
E.g. [student@desktop ~]$ ls
Desktop Documents Download Music Pictures
Public Templates Videos
cd - use the cd command to change directories.With a working directory of /home/student, relative path syntax is shortest to read the Videos subdirectory. The Documents subdirectory is then reached using absolute path syntax.
E.g.
[student@desktop ~]$ cd Videos
[student@desktop Videos ~]$ pwd
/home/students/Videos
[student@desktop Videos ~]$ cd /home/student/Documents
[student@desktop Documents ~]$ pwd
/home/student/Documents
[student@desktop Documents ~]$ cd
[student@desktop ~]$ pwd
/home/student
touch - The touch command normally updates a file's timestamp to the current date and time without otherwise modifying it. This is useful for creating empty files, which can be used for practice, since "touching' a file name that does not exist caused the file to be created. Using touch, practice files are created in the Documents and Videos subdirectories.
E.g.
[student@desktop ~]$ touch Videos/blockbuster1.ogg
[student@desktop ~]$ touch Videos/blockbuster2.ogg
[student@desktop ~]$ touch Documents/thesis_chapter1.odf
[student@desktop ~]$ touch Documents/thesis_chapter2.odf
ls - The ls command has multiple options for displaying attributes on file. The most command and useful are -l (long listing format), -a (all files, includes hidden files), and -R(recursive, to include the contents of all subdirectories).
E.g.
[student@desktop ~]$ ls -l
[student@desktop ~]$ ls -a
[student@desktop ~]$ ls - R
cd -
E.g.
[student@desktop ~]$ cd Videos
[student@desktop Videos ~]$ pwd
/home/student/Videos
[student@desktop Videos ~]$ cd /home/student/Documents
[student@desktop Documents ~]$ pwd
/home/student/Documents
[student@desktop Documents ~]$ cd -
[student@desktop Videos ~]$ pwd
/home/student/Videos
[student@desktop Videos ~]$ cd -
[student@desktop Documents ~]$ pwd
/home/student/Documents
cd.. -
E.g.
[student@desktop Videos ~]$ pwd
/home/student/Videos
[student@desktop Videos ~]$ cd .
[student@desktop Videos ~]$ pwd
/home/student/Videos
[student@desktop Videos ~]$ cd . .
/home
[student@desktop home ~]$ cd . .
[student@desktop / ]$ pwd
/
[student@desktop / ]$ cd
[student@desktop ~]$ pwd
/home/student
Create Directory
mkdir - The mkdir command creates one or more directories or subdirectories, generating error if the file name already exists or when attempting to create a directory in parent directory that doesn't exit. The -p parent option creates missing parent directories for the requested destination. be cautious when using mkdir -p, since accidental spelling mistake create unintended directories without generating error message.
E.g.
[student@desktop ~]$ mkdir Video/Watched
mkdir: cannot create directory Video/Watched : No such file or directory.
The mkdir failed because Videos was misspelled. "Video"
does not exist as a location in which to create the Watched subdirectory. If -p were used, the user would not have received an error message and now have two directories, Video and Videos.
[student@desktop ~]$ mkdir Videos/Watched
[student@desktop ~]$ cd Documents
[student@desktop ~]$ mkdir ProjectX ProjectY
[student@desktop ~]$ mkdir -p Thesis/Chapter1 Thesis/Chapter2 Thesis/Chapter3
[student@desktop ~]$ cd
[student@desktop ~]$ ls - R Videos Documents
The last mkdir create three ChapterN subdirectories with one command. The -p parent option created the missing parent directory Thesis.
copy file cp - The cp command copies one or more files to become new, independent file. In any destination, new file names must be unique. If the new file name is not unique, the copy command will overwrite the existing file.
E.g.
[student@desktop ~]$ cd Videos
[student@desktop ~]$ cp blockbuster1.ogg blokbuster3.ogg
[student@desktop ~]$ ls -l
Move files mv - The mv command renames file in the same directory, or relocates files to a new directory.
E.g.
[student@desktop Videos ~]$ cd .. /Documents
[student@desktop ~]$ ls - l
[student@desktop Documents ~]$ mv thesis_chapter2.odf thesis_chapter2_reviewed.odf
[student@desktop Documents ~]$ mv thesis_chapter1.odf Thesis/Chapter1
[student@desktop Documents ~]$ ls -lR
The first mv command is example of remaining a file. The second causes the file to be relocated to another directory.
Remove files and directories rm - Default syntax for rm deletes files, but not directories. Deleting a directory, and potentially many subdirectories and files below it, requires the -r recursive option. There is no command-line undelete feature, nor a trash bin from which to restore.
E.g.
[student@desktop Documents ~]$ pwd
/home/student/Documents
[student@desktop Documents ~]$ rm thesis_chapter2_reviewed.odf
[student@desktop Documents ~]$ rm Thesis/Chapter1
rm cannot remove 'Thesis/Chapter1' :Is a directory
[student@desktop Documents ~]$ rm -r Thesis/Chapter1
[student@desktop Documents~]$ ls -l Thesis
[student@desktop Documents ~]$ ls -ri Thesis
press 'y' to all queries
Using -i will interactively prompt for each deletion. This is essentially the opposite -f which will force the deletion without prompting the user.
The rmdir command deletes directories only if empty. Remove directories cannot be undeleted
E.g.
[student@desktop Documents ~]$ pwd
/home/student/Documents
[student@desktop Documents ~]$ rmdir ProjectY
[student@desktop Documents ~]$ rmdir ProjectX
rmdir failed to remove 'ProjectX' : Directory not empty
[student@desktop Documents ~]$ rm -r ProjectX
[student@desktop Documents ~]$ ls -lR
The rmdir command failed to delete non-empty ProjectX, but rm - r succeeded.
Pattern Matching
A sample set of files is useful to demonstrate expansion.
E.g.
[student@desktop ~]$ mkdir glob; cd glob
[student@desktop glob ~]$ touch alfa bravo charlie delta echo able baker cast dog easy
[student@desktop glob ~]$ ls
First, simple pattern matching using * and ?.
[student@desktop glob ~]$ ls a*
able alfa
[student@desktop glob ~]$ ls *a*
able alfa baker bravo cast charlie delta easy
[student@desktop glob ~]$ ls [ac]*
able alfa cast charlie
[student@desktop glob ~]$ ls ????
able alfa cast easy echo
[student@desktop glob ~]$ ls ?????
baker bravo delta
Tilde expansion - The tilde character (~), when followed by a slash delimiter, matches the current user's home directory.
When followed by a string of characters up to a slash,it will be interpreted as a username. If one matches. If no username matches, then an actual tilde followed by the string of characters will be returned.
E.g.
[student@desktop glob ~]$ ls -/glob
able alfa baker bravo cast charlie delta dog easy echo
[student@desktop glob ~]$ echo -/glob
/home/student/glob
Brace expansion - Brace expansion is used to generate discretionary string of characters.
E.g.
[student@desktop glob ~]$ echo {Sunday, Monday, Tuesday, Wednesday}.log
Sunday.log Monday.log Tuesday.log Wednesday.log
[student@desktop glob ~]$ echo file{1..3}.txt
[student@desktop glob ~]$ echo file{a..c}.txt
[student@desktop glob ~]$ echo file{a,b}{1,2}.txt
[student@desktop glob ~]$ echo file{a{1,2}b,c}.txt
Command substitution - Command substitution allows the output of a command to replace the command itself.
The $(command) from can nest multiple command expansion inside each other
E.g.
[student@desktop glob ~]$ echo Today is 'date +%A'
Today is Tuesday
[student@desktop glob ~]$ echo The time is $(date +%M) minutes past $(date +%1%p).
The time is 26 minutes past 11AM.
[student@desktop ~]$ pwd
/home/student
[student@desktop ~]$ cd . .
[student@desktop home ~]$ pwd/home
[student@desktop home ~]$ cd . .
[student@desktop / ]$ pwd
/
[student@desktop / ]$ cd
[student@desktop ~]$ pwd
/home/student
Create Directory
mkdir - The mkdir command creates one or more directories or subdirectories, generating error if the file name already exists or when attempting to create a directory in parent directory that doesn't exit. The -p parent option creates missing parent directories for the requested destination. be cautious when using mkdir -p, since accidental spelling mistake create unintended directories without generating error message.
E.g.
[student@desktop ~]$ mkdir Video/Watched
mkdir: cannot create directory Video/Watched : No such file or directory.
The mkdir failed because Videos was misspelled. "Video"
does not exist as a location in which to create the Watched subdirectory. If -p were used, the user would not have received an error message and now have two directories, Video and Videos.
[student@desktop ~]$ mkdir Videos/Watched
[student@desktop ~]$ cd Documents
[student@desktop ~]$ mkdir ProjectX ProjectY
[student@desktop ~]$ mkdir -p Thesis/Chapter1 Thesis/Chapter2 Thesis/Chapter3
[student@desktop ~]$ cd
[student@desktop ~]$ ls - R Videos Documents
The last mkdir create three ChapterN subdirectories with one command. The -p parent option created the missing parent directory Thesis.
copy file cp - The cp command copies one or more files to become new, independent file. In any destination, new file names must be unique. If the new file name is not unique, the copy command will overwrite the existing file.
E.g.
[student@desktop ~]$ cd Videos
[student@desktop ~]$ cp blockbuster1.ogg blokbuster3.ogg
[student@desktop ~]$ ls -l
Move files mv - The mv command renames file in the same directory, or relocates files to a new directory.
E.g.
[student@desktop Videos ~]$ cd .. /Documents
[student@desktop ~]$ ls - l
[student@desktop Documents ~]$ mv thesis_chapter2.odf thesis_chapter2_reviewed.odf
[student@desktop Documents ~]$ mv thesis_chapter1.odf Thesis/Chapter1
[student@desktop Documents ~]$ ls -lR
The first mv command is example of remaining a file. The second causes the file to be relocated to another directory.
Remove files and directories rm - Default syntax for rm deletes files, but not directories. Deleting a directory, and potentially many subdirectories and files below it, requires the -r recursive option. There is no command-line undelete feature, nor a trash bin from which to restore.
E.g.
[student@desktop Documents ~]$ pwd
/home/student/Documents
[student@desktop Documents ~]$ rm thesis_chapter2_reviewed.odf
[student@desktop Documents ~]$ rm Thesis/Chapter1
rm cannot remove 'Thesis/Chapter1' :Is a directory
[student@desktop Documents ~]$ rm -r Thesis/Chapter1
[student@desktop Documents~]$ ls -l Thesis
[student@desktop Documents ~]$ ls -ri Thesis
press 'y' to all queries
Using -i will interactively prompt for each deletion. This is essentially the opposite -f which will force the deletion without prompting the user.
The rmdir command deletes directories only if empty. Remove directories cannot be undeleted
E.g.
[student@desktop Documents ~]$ pwd
/home/student/Documents
[student@desktop Documents ~]$ rmdir ProjectY
[student@desktop Documents ~]$ rmdir ProjectX
rmdir failed to remove 'ProjectX' : Directory not empty
[student@desktop Documents ~]$ rm -r ProjectX
[student@desktop Documents ~]$ ls -lR
The rmdir command failed to delete non-empty ProjectX, but rm - r succeeded.
Pattern Matching
A sample set of files is useful to demonstrate expansion.
E.g.
[student@desktop ~]$ mkdir glob; cd glob
[student@desktop glob ~]$ touch alfa bravo charlie delta echo able baker cast dog easy
[student@desktop glob ~]$ ls
First, simple pattern matching using * and ?.
[student@desktop glob ~]$ ls a*
able alfa
[student@desktop glob ~]$ ls *a*
able alfa baker bravo cast charlie delta easy
[student@desktop glob ~]$ ls [ac]*
able alfa cast charlie
[student@desktop glob ~]$ ls ????
able alfa cast easy echo
[student@desktop glob ~]$ ls ?????
baker bravo delta
Tilde expansion - The tilde character (~), when followed by a slash delimiter, matches the current user's home directory.
When followed by a string of characters up to a slash,it will be interpreted as a username. If one matches. If no username matches, then an actual tilde followed by the string of characters will be returned.
E.g.
[student@desktop glob ~]$ ls -/glob
able alfa baker bravo cast charlie delta dog easy echo
[student@desktop glob ~]$ echo -/glob
/home/student/glob
Brace expansion - Brace expansion is used to generate discretionary string of characters.
E.g.
[student@desktop glob ~]$ echo {Sunday, Monday, Tuesday, Wednesday}.log
Sunday.log Monday.log Tuesday.log Wednesday.log
[student@desktop glob ~]$ echo file{1..3}.txt
[student@desktop glob ~]$ echo file{a..c}.txt
[student@desktop glob ~]$ echo file{a,b}{1,2}.txt
[student@desktop glob ~]$ echo file{a{1,2}b,c}.txt
Command substitution - Command substitution allows the output of a command to replace the command itself.
The $(command) from can nest multiple command expansion inside each other
E.g.
[student@desktop glob ~]$ echo Today is 'date +%A'
Today is Tuesday
[student@desktop glob ~]$ echo The time is $(date +%M) minutes past $(date +%1%p).
The time is 26 minutes past 11AM.
Subscribe to:
Posts (Atom)