This Blog is to share our knowledge and expertise on Linux System Administration and VMware Administration

Monday, December 25, 2017

Screen command in Linux

Monday, December 25, 2017 0

Screen command in Linux


NAME
       screen - screen manager with VT100/ANSI terminal emulation

Screen  is  a full-screen window manager that multiplexes a physical terminal between several processes. When screen is called, it creates a single window with a shell in it (or the specified command) and then gets out of your way so that you can  use the  program  as  you  normally  would.  Then,  at any time, you can create new (full-screen) windows with other programs in them (including more shells), kill existing windows, view a list of windows, turn output logging on and off, copy-and-paste text between windows, view  the  scrollback history,  switch  between  windows in whatever manner you wish, etc. All windows run their programs completely independent of each other. Programs continue to run when their window is currently not visible and even when the whole screen session is detached from the user’s  terminal.   When  a program  terminates,  screen  (per default) kills the window that contained it.  If this window was in the foreground, the display switches to the previous window; if none are left, screen exits.

#screen -h 2000 -t SERVER_UPDATE -S SERVER_UPDATE

Here
  -h  num - Specifies the history scrollback buffer to be num lines high.
  -t  name - sets the title for the default shell or specified program. 
  -S  sessionname  -  When  creating  a  new  session,  this  option can be used to specify a meaningful name for the session. This name identifies the session for
            "screen -list" and "screen -r" actions. It substitutes the default [tty.host] suffix.

Prints a list of screen session
#screen -ls
There is a screen on:
        27137.pts-1.testserver       (Attached)
1 Socket in /var/run/screen/S-root.

Detach the screen session
#screen -D 27137.pts-1.testserver
[27137.pts-1.testserver power detached.]

Attach the screen session
#screen -r 27137.pts-1.testserver

Please check man screen page for more options.

Sunday, December 24, 2017

Configuring logrotate for yum in RHEL7

Sunday, December 24, 2017 0

Configuring logrotate for yum in RHEL7


Every time you use yum to install and/or update packages, it logs to /var/log/yum.log. 

I do recommend keeping your complete yum history as it doesn't grow a lot, unless you reinstall packages a lot.

For a rich interface to your yum history, I suggest you use yum history.

By default, your yum log file is rotated yearly, and even then, it only rotates if the size of your log file exceeds 30 KB, and your logs are only kept for 4 years. 

virtual servers have the potential to stay "alive" beyond these 3-4 years.

How to do it…
Modify /etc/logrotate.d/yum to the following:

/var/log/yum.log {
    missingok
    notifempty
    size 30k
    rotate 100
    yearly
    create 0600 root root
}

How it works…
This configuration will only rotate the yum log when it exceeds 30 KB in size on a yearly basis, and it will keep 100 rotated logs, which is basically log files for 100 years!

Friday, December 22, 2017

Install KVM in RHEL7

Friday, December 22, 2017 0

Install KVM in RHEL7

By default, a RHEL 7 system doesn't come with a KVM or libvirt preinstalled. This can be installed in three ways:

Through the graphical setup during the system's setupVia a kickstart installationThrough a manual installation from the command line


To install a KVM, you will require at least 6 GB of free disk space, 2 GB of RAM, and an additional core or thread per guest.

Check whether your CPU supports a virtualization flag (such as SVM or VMX). Some hardware vendors disable this in the BIOS, so you may want to check your BIOS as well. Run the following command:

# grep -E 'svm|vmx' /proc/cpuinfo
flags    : ... svm ...
Check whether the hardware virtualization modules (such as kvm_intel and kvm) are loaded in the kernel using the following command:

# lsmod | grep kvm
kvm_intel             155648  0
kvm                      495616  1 kvm_intel

Manual installation
This way of installing a KVM is generally done once the base system is installed by some other means. 

Install the software needed to provide an environment to host virtualized guests with the following command:
# yum -y install qemu-kvm qemu-img libvirt

The installation of these packages will include quite a lot of dependencies.

Install additional utilities required to configure libvirt and install virtual machines by running this command:
# yum -y install virt-install libvirt-python python-virthost libvirt-client

By default, the libvirt daemon is marked to autostart on each boot. Check whether it is enabled by executing the following command:
# systemctl status libvirtd

libvirtd.service - Virtualization daemon
   Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled)
   Active: inactive
   Docs: man:libvirtd(8)
    http://libvirt.org

If for some reason this is not the case, mark it for autostart by executing the following:
# systemctl enable libvirtd
To manually stop/start/restart the libvirt daemon, this is what you'll need to execute:
# systemctl stop libvirtd
# systemctl start libvirtd
# systemctl restart libvirtd

Kickstart installation
Installing a KVM during kickstart offers you an easy way to automate the installation of KVM instances. 

Add the following package groups to your kickstarted file in the %packages section:
@virtualization-hypervisor
@virtualization-client
@virtualization-platform
@virtualization-tools
Start the installation of your host with this kickstart file.

Graphical setup during the system's setup
This is probably the least common way of installing a KVM. The only time I used this was during the course of writing this recipe. Here's how you can do this:

Boot from the RHEL 7 Installation media.
Complete all steps besides the Software selection step.
Go to Software Selection to complete the KVM software selection.
Select the Virtualization host radio button in Base Environment, and check the Virtualization Platform checkbox in Add-Ons for Selected Environment:
Finalize the installation.
On the Installation Summary screen, complete any other steps and click on Begin Installation.

Wednesday, December 20, 2017

ssh_exchange_identification: Connection closed by remote host - Password less authentication setup

Wednesday, December 20, 2017 0

ssh_exchange_identification: Connection closed by remote host - Password less authentication setup 


Situation:
While setup passwordless authentication from testserver2 (192.181.166.55) to testserver1 (192.181.130.55)  server, getting  error "ssh_exchange_identification: Connection closed by remote host"

[kanachim@testserver2 .ssh]$ ssh -vv testserver1
OpenSSH_5.3p1-hpn13v7, OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to 192.181.166.55 [192.181.166.55] port 22.
debug1: Connection established.
debug2: key_type_from_name: unknown key type '-----BEGIN'
debug2: key_type_from_name: unknown key type '-----END'
debug1: identity file /home/kanachim/.ssh/id_rsa type 1
debug1: identity file /home/kanachim/.ssh/id_dsa type -1
ssh_exchange_identification: Connection closed by remote host

Solution:
Check /etc/hosts.deny on server testserver1

root@host testserver1# grep sshd /etc/hosts.deny 
# DenyHosts: Mon Dec 18 22:10:38 2017 | sshd: 192.181.166.55
sshd: 192.181.166.55

Remove the sshd entry from hosts.deny on testserver1

Login to testserver2 Switch to user kanachim & create new key

-bash-3.2$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/kanachim/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/kanachim/.ssh/id_rsa.
Your public key has been saved in/home/kanachim/.ssh/id_rsa.pub.
The key fingerprint is:
37:6a:69:c1:a4:01:4b:c5:34:be:32:42:90:0b:20:a0 kanachim@192.181.166.55
The key's randomart image is:
+--[ RSA 2048]----+
|B.  o++          |
|=. . +..         |
|E.. . o .        |
|..     *         |
|  . o o S o      |
|   . o   = .     |
|        =        |
|       o         |
|                 |
+-----------------+
-bash-3.2$
-bash-3.2$ ssh-copy-id kanachim@192.181.130.55
Password:
Now try logging into the machine, with "ssh 'kanachim@192.181.130.55'", and check in:
  .ssh/authorized_keys
to make sure we haven't added extra keys that you weren't expecting.

Now login from testserver2 to testserver1

-bash-3.2$ ssh 192.181.130.55
Last login: Thu Feb 27 00:30:38 2014 from 10.217.230.145
-bash-3.2$ hostname
testserver1

Tuesday, December 19, 2017

Recover RPM DB from a corrupted RPM database in RHEL 7

Tuesday, December 19, 2017 0

Rebuilding corrupted rpm database in RHEL7

Situation:
Although everything is done to ensure that your RPM databases are intact, your RPM database may become corrupt and unuseable. This happens mainly if the filesystem on which the rpm db resides is suddenly inaccessible (full, read-only, reboot, or so on).

Solution:
1.Start by creating a backup of your corrupt rpm db, as follows:
[root@nsk ~]# tar zcvf rpm-db.tar.gz /var/lib/rpm/*
tar: Removing leading `/' from member names
/var/lib/rpm/Basenames
/var/lib/rpm/Conflictname
/var/lib/rpm/__db.001
/var/lib/rpm/__db.002
/var/lib/rpm/__db.003
/var/lib/rpm/Dirnames
/var/lib/rpm/Group
/var/lib/rpm/Installtid
/var/lib/rpm/Name
/var/lib/rpm/Obsoletename
/var/lib/rpm/Packages
/var/lib/rpm/Providename
/var/lib/rpm/Requirename
/var/lib/rpm/Sha1header
/var/lib/rpm/Sigmd5
/var/lib/rpm/Triggername

2.Remove stale lock files if they exist through the following command:
[root@nsk ~]# rm -f /var/lib/rpm/__db*

3.Now, verify the integrity of the Packages database via the following:
[root@nsk ~]# /usr/lib/rpm/rpmdb_verify /var/lib/rpm/Packages; echo $?
BDB5105 Verification of /var/lib/rpm/Packages succeeded.
0

If it prints 0, proceed to next step.

4. Rename the Packages file (don't delete it, we'll need it!), as follows:
[root@nsk ~]# mv /var/lib/rpm/Packages  /var/lib/rpm/Packages.org

5. Now, dump the Packages db from the original Packages db by executing the following command:
[root@nsk ~]# cd /var/lib/rpm/
 [root@nsk rpm]# /usr/lib/rpm/rpmdb_dump Packages.org | /usr/lib/rpm/rpmdb_load Packages
rpmdb_load: BDB1540 configured environment flags incompatible with existing environment

6.Verify the integrity of the newly created Packages database. Run the following:
[root@nsk rpm]#  /usr/lib/rpm/rpmdb_verify /var/lib/rpm/Packages; echo $?
BDB5105 Verification of /var/lib/rpm/Packages succeeded.
0

If the exit code is not 0, you will need to restore the database from backup.

7. Rebuild the rpm indexes, as follows:
[root@nsk ~]# rpm -vv --rebuilddb
[root@nsk rpm]# rpm -vv --rebuilddb
D: rebuilding database /var/lib/rpm into /var/lib/rpmrebuilddb.1312
D: opening  db environment /var/lib/rpm private:0x401
D: opening  db index       /var/lib/rpm/Packages 0x400 mode=0x0
D: locked   db index       /var/lib/rpm/Packages
D: opening  db environment /var/lib/rpmrebuilddb.1312 private:0x401
D: opening  db index       /var/lib/rpmrebuilddb.1312/Packages (none) mode=0x42
D: opening  db index       /var/lib/rpmrebuilddb.1312/Packages 0x1 mode=0x42
D: disabling fsync on database
....
...
D: adding "5f7fd424d0773a4202731bff4901d449699b0929" to Sha1header index.
D: closed   db index       /var/lib/rpm/Packages
D: closed   db environment /var/lib/rpm
D: closed   db index       /var/lib/rpmrebuilddb.1312/Sha1header
D: closed   db index       /var/lib/rpmrebuilddb.1312/Sigmd5
D: closed   db index       /var/lib/rpmrebuilddb.1312/Installtid
D: closed   db index       /var/lib/rpmrebuilddb.1312/Dirnames
D: closed   db index       /var/lib/rpmrebuilddb.1312/Triggername
D: closed   db index       /var/lib/rpmrebuilddb.1312/Obsoletename
D: closed   db index       /var/lib/rpmrebuilddb.1312/Conflictname
D: closed   db index       /var/lib/rpmrebuilddb.1312/Providename
D: closed   db index       /var/lib/rpmrebuilddb.1312/Requirename
D: closed   db index       /var/lib/rpmrebuilddb.1312/Group
D: closed   db index       /var/lib/rpmrebuilddb.1312/Basenames
D: closed   db index       /var/lib/rpmrebuilddb.1312/Name
D: closed   db index       /var/lib/rpmrebuilddb.1312/Packages
D: closed   db environment /var/lib/rpmrebuilddb.1312

8. Use the following command to check the rpm db with yum for any other issues (this may take a long time):
[root@nsk rpm]# yum check
Loaded plugins: fastestmirror
....
...

9. Restore the SELinux context of the rpm database through the following command:
[root@nsk rpm]# restorecon -R -v /var/lib/rpm

Monday, December 18, 2017

How to reboot the Xen Virtual Machine when was at hung state.

Monday, December 18, 2017 0

How to reboot the Xen Virtual Machine when was at hung state.

If console is not working from the host(Dom0), the console was opened in order to use sysrq magic packages
#xm console xenvm006

From other terminal on the node, run the below  comamnd one by one.
#xm sysrq xenvm006  h
#xm sysrq xenvm006  m    #should show the amount of memory been used
#xm sysrq xenvm006  t      #should show the current tasks

the output is  "console is opened and was not good"  also   "vm was with out resources"
Check the xentop, if  a high cpu usage is shown like 200%

Then reboot the virtual machine.

Open one more terminal & run the below command.

#xm destroy  xenvm006

if the virtual machine is running under cluster,  first disable the vm from cluster to avoid failover.

#clusvcadm -d vm:xenvm006

Then destroy the virtual machine.

Because as the vm is hung state, clusvcadm -d wont do a clean shutdown, so we need to destroy it manually.

#xm create xenvm006  (will start the vm)
#clusvcadm -e vm:xenvm006   (with cluster)

Sunday, December 17, 2017

Understanding the LVM Configuration Files in Linux

Sunday, December 17, 2017 0

Understanding the LVM Configuration Files in Linux

The following files are part of LVM configuration:

/etc/lvm/lvm.conf
    Central configuration file read by the tools.

etc/lvm/lvm_hosttag.conf
    For each host tag, an extra configuration file is read if it exists: lvm_hosttag.conf. If that file defines new tags,
    then further configuration files will be appended to the list of tiles to read in.
In addition to the LVM configuration files, a system running LVM includes the following files that affect LVM system setup:

/etc/lvm/cache
    Device name filter cache file (configurable).

/etc/lvm/backup/
    Directory for automatic volume group metadata backups (configurable).

/etc/lvm/archive/
    Directory for automatic volume group metadata archives (configurable with regard to directory path and archive history depth).

/var/lock/lvm/
    In single-host configuration, lock files to prevent parallel tool runs from corrupting the metadata; in a cluster, cluster-wide DLM is used.

How to monitor the permission change and ownership change of a particular directory or file in Linux Server

Sunday, December 17, 2017 0

Monitoring the permission change and ownership change of a particular directory or file in Linux Server

1. Use the audit package to accomplish this task.2. Ensure the auditd service is running, and set to start on boot chkconfig auditd on3. Set a watch on the required file to be monitored by using the auditctl command:

# auditctl -w /etc/passwd -p war -k monitor-passwd

here:
    auditctl   :the command used to add entries to the audit database.
    -w            :Insert a watch for the file system object at path, i.e. /etc/shadow.
    -p             :Set permissions filter for a file system watch. r=read, w=write, x=execute, a=attribute change.
    -k             : Set a filter key on an audit rule. The filter key is an arbitrary string of text that can be up to 31 bytes long. 

It can uniquely identify the audit records produced by a rule.

Note that you must add the rule to /etc/audit/audit.rules on RHEL5 or RHEL6  in order for them to persist after reboot.
    You can place the watch rule in the /etc/audit/audit.rules file to set permanently:
-w /etc/passwd -p a -k monitor-passwd
       Check auditd service and if it is stopped, start it.

# service auditd status
# chkconfig --list auditd
# chkconfig auditd on

In this example, a watch is placed on the /etc/passwd file for any syscalls which perform a write, read, or attribute change
    (-p war). This is logged with the key monitor-passwd. This key can be used to search through the audit logs to find these actions, 
 using the ausearch command:

# ausearch -ts today -k monitor-passwd
----
time->Sat May  3 07:32:20 2009
type=PATH msg=audit(117045140.872:34): item=0 name="/etc/passwd" inode=1308742 dev=fd:00 mode=0100644 ouid=0 ogid=0 rdev=00:00 obj=system_u:object_r:etc_t:s0
type=CWD msg=audit(117045140.872:34): cwd="/root"
type=SYSCALL msg=audit(117045140.872:34): arch=40000003 syscall=226 success=yes exit=0 a0=867c4b8 a1=458bcc4f a2=8686800 a3=1c
items=1 ppid=3544 pid=3558 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 comm="vim" exe="/usr/bin/vim"
subj=root:system_r:unconfined_t:s0-s0:c0.c1023 key="monitor-passwd"

    From this trace, it can be seen that the file /etc/resolve.conf was edited using the /usr/bin/vim command. The user that ran the  command was running with the root:system_r:unconfined_t:s0-s0:c0.c1023 SELinux context. And, the timestamp can be converted into readable form.

# date -d @117045140
Sat May  3 05:32:20 CST 2009

    You can search for an event based on the given key string
# ausearch -k monitor-passwd
    For a clearly view, you can generate report base on audit rule keys
# aureport -k