This Blog is to share our knowledge and expertise on Linux System Administration and VMware Administration

Sunday, December 9, 2018

How to setup and configure Network Teaming on RHEL 7 server

Sunday, December 09, 2018 0
Here we are using nmcli to configure the network teaming. 

nmtui : Text user Interface
nmcli : Command line Interface
Teaming type : loadbalance
Interface         : team0, eth0, eth2

teamd package should be installed on server.

We can see the previous network configuration setup by using nmcli command.

root@nsk# nmcli con show
NAME  UUID                                                            TYPE      DEVICE
eth0     5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03     ethernet    eth0
eth1     8761e75a-0ba3-4f73-80e4-66df47023ed1  ethernet    eth1

Teaming configuration

Execute the below command on server to create team0 teaming interface.

root@nsk# nmcli con add type team con-name team0 ifname team0 config '{ "runner": {"name": "loadbalance"}}'
Connection 'team0' (e07596d2-59b6-406a-a742-07d5ff96901b) successfully added.

root@nsk# nmcli con show
NAME   UUID                                                              TYPE      DEVICE
eth0     5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03        ethernet    eth0
eth1     8761e75a-0ba3-4f73-80e4-66df47023ed1     ethernet    eth1
team0  e07596d2-59b6-406a-a742-07d5ff96901b     team         team0

Now, /etc/sysconfig/network-scripts/team0 contains below lines

root@nsk# cat ifcfg-team0
TEAM_CONFIG="{\"runner\": {\"name\": \"loadbalance\", \"tx_hash\": [\"eth\", \"ipv4\", \"ipv6\"]}}"
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=dhcp
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=team0
UUID=e07596d2-59b6-406a-a742-07d5ff96901b
DEVICE=team0
ONBOOT=yes
DEVICETYPE=Team

Now add IPv4 address on team0 interface.

root@nsk# nmcli con mod team0 ipv4.addresses 172.27.0.169/28
root@nsk# nmcli con mod team0 ipv4.gateway 172.27.0.161
root@nsk# nmcli con mod team0 ipv4.method manual

Add eth0 interface to the teaming interface.

root@nsk# nmcli con add type team-slave con-name team0-slave0 ifname eth0 master team0
Connection 'team0-slave0' (5fa33d1b-3f25-4fda-b10d-a184e4260cbe) successfully added.

Add eth2 interface to the teaming interface.

root@nsk# nmcli con add type team-slave con-name team0-slave1 ifname eth2 master team0
Connection 'team0-slave1' (bbdb784b-2a9c-4fe3-9a19-11d64a9748ff) successfully added.

root@nsk# ls -al | grep -i team0

ifcfg-team0  
ifcfg-team0-slave0  
ifcfg-team0-slave1

Now bring up the team0 interface.

root@nsk# nmcli connection up team0
Connection successfully activated (master waiting for slaves) (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/11)

root@nsk# teamdctl team0 state
setup:
  runner: loadbalance

root@nsk# /etc/init.d/network restart
Restarting network (via systemctl):                        [  OK  ]

root@nsk# ifconfig | egrep -i "eth0|eth2|team0"

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether b0:83:fe:d8:44:02  txqueuelen 1000  (Ethernet)
        RX packets 2930895882  bytes 2216873481505 (2.0 TiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1694125755  bytes 312320131645 (290.8 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 56

eth2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether b0:83:fe:d8:44:02  txqueuelen 1000  (Ethernet)
        RX packets 2705861  bytes 301266106 (287.3 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 25  bytes 4159 (4.0 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 59

team0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.27.0.169  netmask 255.255.255.240  broadcast 172.27.0.175
        inet6 fe80::bbe8:90aa:4108:e728  prefixlen 64  scopeid 0x20<link>
        ether b0:83:fe:d8:44:02  txqueuelen 1000  (Ethernet)
        RX packets 35  bytes 2586 (2.5 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 28  bytes 4451 (4.3 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Check the configuration

root@nsk# nmcli con reload
root@nsk# nmcli con show
NAME                 UUID                                                             TYPE      DEVICE
eth1                    8761e75a-0ba3-4f73-80e4-66df47023ed1   ethernet    eth1
team0                 e07596d2-59b6-406a-a742-07d5ff96901b   team         team0
team0-slave0     5fa33d1b-3f25-4fda-b10d-a184e4260cbe     ethernet    eth0
team0-slave1     bbdb784b-2a9c-4fe3-9a19-11d64a9748ff     ethernet    eth2

Check the team0 ports status

root@nsk# teamnl team0 ports
 4: eth2: up 1000Mbit FD
 2: eth0: up 1000Mbit FD

We can use teamdctl command to check team0 configuration status.

root@nsk# teamdctl team0 state
setup:
  runner: loadbalance
ports:
  eth0
    link watches:
      link summary: up
      instance[link_watch_0]:
        name: ethtool
        link: up
        down count: 0
  eth2
    link watches:
      link summary: up
      instance[link_watch_0]:
        name: ethtool
        link: up
        down count: 0
root@nsk#

Note : If we need Activebackup setup, just replace the loadbalance with activebackup while creating team0 interface.

Thursday, November 8, 2018

Extend the cluster file system by extending the existing netapp storage lun in RHEL server

Thursday, November 08, 2018 0
If storage team extended the existing LUN instead of creating new LUN, below steps need to follow.

Run multipath -ll command and search the device info which is mapped to the LUN.

root@nsk# multipath -ll | grep -A 6 -i 3600a09634224747a367d4b55357c4f87
3600a09634224747a367d4b55357c4f87 dm-6 NETAPP,LUN C-Mode
size=400G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=50 status=active
  |- 2:0:0:1 sdg 8:96  active ready running
  `- 1:0:0:1 sdc 8:32  active ready running
3600a09803830436a345d4b51506c4f43 dm-2 NETAPP,LUN C-Mode
size=110G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='0' wp=rw

As per above output, sdg and sdc is the devices.  Now rescan the LUN.

root@nsk# echo "1" > /sys/block/sdg/device/rescan
root@nsk# echo "1" > /sys/block/sdc/device/rescan 

Reload multipathd service 

root@nsk# /etc/init.d/multipathd reload

Resize the PV (provide complete path like below)

root@nsk# pvresize /dev/mapper/3600a09634224747a367d4b55357c4f87  

Extend the LV

root@nsk# lvextend -L +199.95g /dev/mapper/oracle_vg-oracledata

Monday, October 15, 2018

How to generate sosreport in different directory instead of default directory

Monday, October 15, 2018 0
Sosreport is a tool that collects configuration details, system information, diagnostic information and system logs from Linux server. Sosreport  generates  an archive report from the running system. The archive may be stored locally or centrally for recording or tracking purposes.

Normally sosreport will be saved under /tmp (RHEL6) /var/tmp (RHEL7) directory on linux server. Some time we need to save the sosreport to some other temporary directory because of space issue. 

Below steps are used to generate sosreport in different directory.

RHEL 5.6 and earlier

We need to set the temporary path environment before running sosreport.

[root@nsk ~]# env TMP=/path/to/target/directory sosreport

RHEL 5.7 and above

We have to use option --tmp-dir with sosreport to redirect the sosreport path 

[root@nsk ~]# sosreport --tmp-dir /path/to/target/directory

Sosreport may have hung because of a specific plugin. We can list all available sosreport plugin modules by below command

[root@nsk ~]# sosreport -l

Identify which plugin is hanging on by below command

[root@nsk ~]# sosreport -vvvv 

If a plugin times out or last plugin does not finish, rerun the sosreport and skip that plugin

[root@nsk ~]# sosreport -v -n plugin_name --tmp-dir /path/to/target/directory

Friday, October 5, 2018

Digging audit log with ausearch tool in RHEL 7

Friday, October 05, 2018 0
ausearch is a tool to search audit daemon logs based upon the events based on different search criteria.

 Audit system stores log entries in the /var/log/audit/audit.log file; if log rotation is enabled, rotated audit.log files are stored in the same directory.

Each event consists of three records, which share the same time stamp. Each record consists of several name=value pairs separated by a white space or a comma. 

type=SYSCALL msg=audit(1510471123.129:36): arch=c000003e syscall=175 success=yes exit=0 a0=1901a20 a1=1c5d a2=41a2d8 a3=18fe400 items=0 ppid=7 24 pid=725 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="modprobe" exe="/usr/bin/kmo d" subj=system_u:system_r:insmod_t:s0 key=(null)
type=PROCTITLE msg=audit(1510471123.129:36): proctitle=2F7362696E2F6D6F6470726F6265002D71002D2D0069707461626C655F6D616E676C65
type=NETFILTER_CFG msg=audit(1510471123.276:37): table=nat family=2 entries=0

Here, ausearch utility allows us to search Audit log files for specific events. Also it is used to read the audit log timestamp to user readable timestamp by using -i (--interpret) option.

Example:

1.Search for hostname with given hostname specific.

[root@nsk ~]# ausearch -i -hn nsk.testingserver.internal
----
type=USER_AUTH msg=audit(10/04/2018 12:32:02.610:75) : pid=1617 uid=root auid=unset ses=unset subj=system_u:system_r:xdm_t:s0-s0:c0.c1023 msg='op=PAM:authentication grantors=pam_permit acct=gdm exe=/usr/libexec/gdm-session-worker hostname=nsk.testingserver.internal addr=? terminal=/dev/tty1 res=success'
----

2. Search only the first event that matches the search pattern.

[root@nsk ~]# ausearch -i -hn nsk.testingserver.internal --just-one
----
type=USER_AUTH msg=audit(10/04/2018 12:32:02.610:75) : pid=1617 uid=root auid=unset ses=unset subj=system_u:system_r:xdm_t:s0-s0:c0.c1023 msg='op=PAM:authentication grantors=pam_permit acct=gdm exe=/usr/libexec/gdm-session-worker hostname=nsk.testingserver.internal addr=? terminal=/dev/tty1 res=success'
[root@nsk ~]#

3. Search for all logged actions performed by user nsk, using the user's login ID (auid). 

[root@nsk log]# ausearch -ua 1001 -i
----
type=LOGIN msg=audit(12/04/2017 11:39:18.052:156) : pid=1433 uid=root subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 old-auid=unset auid=nsk tty=(none) old-ses=4294967295 ses=2 res=yes
----
type=USER_ROLE_CHANGE msg=audit(12/04/2017 11:39:18.991:157) : pid=1433 uid=root auid=nsk ses=2 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='pam: default-context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 selected-context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 exe=/usr/sbin/sshd hostname=nsk addr=fe80::f24d:eb49:d219:6b0 terminal=ssh res=success'

4. Search for an event matching the given Process ID.

[root@nsk ~]# ausearch -i -p 2162
----
type=CRYPTO_SESSION msg=audit(10/05/2018 13:07:54.052:118) : pid=2162 uid=root auid=unset ses=unset subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=start direction=from-server cipher=aes256-ctr ksize=256 mac=hmac-sha2-256 pfs=curve25519-sha256@libssh.org spid=2163 suid=sshd rport=50935 laddr=10.0.2.15 lport=22  exe=/usr/sbin/sshd hostname=? addr=10.0.2.2 terminal=? res=success'
----

5. Search the /var/log/audit/audit.log file for successful login attempts

[root@nsk log]# ausearch --message USER_LOGIN --success yes --interpret
----
type=USER_LOGIN msg=audit(11/12/2017 13:04:08.279:110) : pid=662 uid=root auid=root ses=2 subj=system_u:system_r:local_login_t:s0-s0:c0.c1023
msg='op=login id=root exe=/usr/bin/login hostname=nsk addr=? terminal=tty1 res=success'
----
type=USER_LOGIN msg=audit(11/12/2017 13:05:46.297:107) : pid=1242 uid=root auid=root ses=1 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c
0.c1023 msg='op=login id=root exe=/usr/sbin/sshd hostname=10.0.2.2 addr=10.0.2.2 terminal=/dev/pts/0 res=success'
----

6. Search for all account and group activities.

[root@nsk log]# ausearch -m ADD_USER -m DEL_USER -m ADD_GROUP -m DEL_GROUP -i
----
type=ADD_GROUP msg=audit(11/15/2017 22:05:29.396:169) : pid=15764 uid=root auid=root ses=2 subj=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 msg='op=add-group id=dovecot exe=/usr/sbin/groupadd hostname=? addr=? terminal=? res=success'
----
type=ADD_USER msg=audit(11/15/2017 22:05:29.431:171) : pid=15769 uid=root auid=root ses=2 subj=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 msg='op=add-user id=dovecot exe=/usr/sbin/useradd hostname=? addr=? terminal=? res=success'

7. Search for all failed PROCTITLE from yesterday up until now

[root@nsk log]# ausearch --start yesterday --end now -m PROCTITLE -sv no -i
----
type=PROCTITLE msg=audit(10/03/2018 18:00:02.623:71) : proctitle=/usr/sbin/httpd -DFOREGROUND
type=SYSCALL msg=audit(10/03/2018 18:00:02.623:71) : arch=x86_64 syscall=open success=no exit=EACCES(Permission denied) a0=0x560fb5acc008 a1=O_RDONLY a2=0x1b6 a3=0x24 items=0 ppid=1 pid=1145 auid=unset uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=(none) ses=unset comm=httpd exe=/usr/sbin/httpd subj=system_u:system_r:httpd_t:s0 key=(null)
type=AVC msg=audit(10/03/2018 18:00:02.623:71) : avc:  denied  { read } for  pid=1145 comm=httpd name=php-mapi.cfg dev="dm-0" ino=19498435 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:zarafa_etc_t:s0 tclass=file

Please refer man ausearch for more option.

Thursday, October 4, 2018

How to read/convert Audit log timestamp in RHEL7

Thursday, October 04, 2018 0
By default, audit log would be like below.

type=SYSCALL msg=audit(1510471123.129:36): arch=c000003e syscall=175 success=yes exit=0 a0=1901a20 a1=1c5d a2=41a2d8 a3=18fe400 items=0 ppid=7 24 pid=725 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="modprobe" exe="/usr/bin/kmo d" subj=system_u:system_r:insmod_t:s0 key=(null)
type=PROCTITLE msg=audit(1510471123.129:36): proctitle=2F7362696E2F6D6F6470726F6265002D71002D2D0069707461626C655F6D616E676C65 type=NETFILTER_CFG msg=audit(1510471123.276:37): table=nat family=2 entries=0

Below commands are used to convert the auditlog timestamp to user readable timestamp.

1. AUSEARCH : 

ausearch utility allows us to search Audit log files for specific events. Also it is used to read the audit log epoch timestamp to user readable timestamp.By default, ausearch searches the /var/log/audit/audit.log file. We can specify a different file using the ausearch options -if file_name command

[root@nsk log]# ausearch -i | grep -i CONFIG
type=CONFIG_CHANGE msg=audit(11/12/2017 12:48:40.357:5) : audit_backlog_limit=8192 old=64 auid=unset ses=unset subj=system_u:system_r:unconfined_service_t:s0 res=yes
type=CONFIG_CHANGE msg=audit(11/12/2017 12:48:40.357:6) : audit_failure=1 old=1 auid=unset ses=unset subj=system_u:system_r:unconfined_service_t:s0 res=yes

-i, --interpret : Interpret numeric entities into text

2. DATE :  

A) Date command with %s is used to convert normal time to epoch time.
%s : seconds since 1970-01-01 00:00:00 UTC

Display the current time in the given FORMAT, or set the system date
[root@nsk audit]# date +%s
1538570770
[root@nsk audit]# date +%s
1538570773

B) Date command with -d@ is used to convert epoch time to normal time.
[root@nsk audit]# date -d@1538570776
Wed Oct  3 18:16:16 IST 2018
[root@nsk audit]#

C) If we want to convert the specific system time to epoch time.
[root@nsk audit]# date --date="Wed Oct  3 18:16:16 IST 2018" +%s
1538570776

3. PERL : Perl was originally a language optimized for scanning arbitrary text files, extracting information from those text files, and printing reports based on that information

[root@nsk audit]# perl -pe 's/(\d+)/localtime/e' audit.log  | more
type=DAEMON_START msg=audit(Sun Nov Wed Oct  3 20:22:26 2018 12:48:40 2017.186:6974): op=start ver=2.7.6 format=raw kernel=3.10.0-693.el7.x86_64 auid=4294967295 pid=606 uid=0 ses=4294967295 subj=system_u:system_r:auditd_t:s0 res=success
type=CONFIG_CHANGE msg=audit(Sun Nov Wed Oct  3 20:22:26 2018 12:48:40 2017.357:5): audit_backlog_limit=8192 old=64 auid=4294967295 ses=429496 7295 subj=system_u:system_r:unconfined_service_t:s0 res=1

Here,
-p : Print out the pattern space
-e : command (This command allows one to pipe input from a shell command into pattern space)
localtime         : it is perl function, it is having 9 elements.
-d : Match 1 or more repetition of digits

Wednesday, October 3, 2018

How to rebuild the GRUB configuration file in RHEL7

Wednesday, October 03, 2018 0
Rebuilding the GRUB configuration file in RHEL7

Situation : Whenever there is inconsistency in the GRUB file, empty grub configuration or of the grub file got removed accidentally, follow the below steps to fix the grub issue.







Solution:

STEP 1 : List the partitions by using “ ls “ command
grub > ls
(proc) (hd0) (hd0,msdos3) (hd0,msdos2) (hd0,msdos1) (fd0)

STEP 2: Findout the root partitions by using command ls < partition name >. Run ls command to all above listed available  partitions.










STEP 3: Set the root partition. In our case its present in (hd0,msdos1)
grub> set root=(hd0,msdos1)
grub>

STEP 4: Type Linux and specify the kernel or Press tab for listing the available kernel. Once choosing it, we have to specify the root volume (In our example its /dev/sda1).






grub> linux /vmlinuz-3.10.0-229.el7.x86_64 ro root=/dev/sda1

STEP 5: Specify the initramfs image press tab for listing it. Choose the initramfs image with the same kernel version.







grub> initrd /initramfs-3.10.0-229.el7.x86_64.img

Execute 'boot' command to boot the server with above configuration.

Once the server is up, use grub2-mkconfig command to rebuild the grub.conf file. 

[root@nsk /]grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.10.0-229.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-229.el7.x86_64.img
...
.
done

Finally verify the grub file under /boot/grub2/
[root@nsk /]# cd /boot/grub2/
[root@nsk grub2]# ls
device.map  fonts  grub.cfg  grubenv  i386-pc  locale

Reboot the server and verify one more time.

Friday, September 14, 2018

python: SELinux is preventing httpd from connectto access on the unix_stream_socket /var/lib/mysql/mysql.sock - Mariadb running on RHEL 7

Friday, September 14, 2018 0
SELinux is preventing the httpd access by default.

Situation
While connecting mysql database from web, getting this error.
 python: SELinux is preventing httpd from connectto access on the unix_stream_socket /var/lib/mysql/mysql.sock

Solution
Run below command to allow httpd in Selinux.

[root@nsk ~]#  ausearch -c 'httpd' --raw | audit2allow -M my-httpd
******************** IMPORTANT ***********************
To make this policy package active, execute:

semodule -i my-httpd.pp

[root@nsk ~]# semodule -i my-httpd.pp

Hope it helps.

Friday, September 7, 2018

ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' - Mariadb running on RHEL7

Friday, September 07, 2018 0
Situation : 
                  I have tried to login as root to MariaDB Database in Linux Server, Getting below error.

[root@nsk ~]# mysql -u root -p
Enter password:
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock'

Solution: 
                We need to recover the root password of mariadb. Follow the below steps to achieve root password recovery.

STEP 1 : Stop the mariadb service
[root@nsk ~]# systemctl stop mariadb.service
[root@nsk ~]#

STEP 2 : Run mysql in safe mode
[root@nsk ~]# mysqld_safe --skip-grant-tables &
[1] 8051
[root@nsk ~]# 180907 13:23:49 mysqld_safe Logging to '/var/log/mariadb/mariadb.log'.
180907 13:23:49 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql

STEP 3 : Login as root and select mysql db. Then reset root password.
[root@nsk ~]# mysql -u root -p
Enter password:
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 1
Server version: 5.5.60-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> show databases;
+-------------------------------+
| Database                      |
+-------------------------------+
| information_schema     |
| company                       |
| mydb                             |
| mysql                            |
| performance_schema   |
| test                                |
+-------------------------------+
6 rows in set (0.00 sec)

MariaDB [(none)]> use mysql;
Database changed
MariaDB [mysql]> update user set password=PASSWORD("NewPassword") where User='root';
Query OK, 0 rows affected (0.00 sec)
Rows matched: 3  Changed: 0  Warnings: 0

MariaDB [mysql]>
MariaDB [mysql]> flush privileges;
Query OK, 0 rows affected (0.00 sec)

MariaDB [mysql]> quit
Bye

STEP 4 : Now stop and start the mariadb service and test the new password.

[root@nsk ~]# systemctl stop mariadb.service
[root@nsk ~]#
[root@nsk ~]# systemctl start mariadb.service
[root@nsk ~]#
[root@nsk ~]# mysql -u root -p
Enter password:
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 5
Server version: 5.5.60-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]>

Hope it helps.

Tuesday, September 4, 2018

Error: Cannot retrieve metalink for repository: epel. Please verify its path and try again - SOLVED

Tuesday, September 04, 2018 0
Situation : I have installed epel-release-latest-6.noarch.rpm on CentOS release 6.10 (Final) and trying to install 
python-pip, but getting the below error.

[root@puppetlabs tmp]# rpm -ivh epel-release-latest-6.noarch.rpm
warning: epel-release-latest-6.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 0608b895: NOKEY
Preparing...                ########################################### [100%]
   1:epel-release         ###########################################  [100%]
[root@puppetlabs tmp]#
[root@puppetlabs tmp]# yum install python-pip
Loaded plugins: fastestmirror
Setting up Install Process
Loading mirror speeds from cached hostfile
Error: Cannot retrieve metalink for repository: epel. Please verify its path and try again

Solution:

Edit both /etc/yum.repos.d/epel.repo and /etc/yum.repos.d/epel-testing.repo files, commenting all entries 
starting with mirrorlist=  and uncomment all the entries starting with baseurl= 

epel.repo base url is changed from http://download.fedoraproject.org/pub/epel/6/  to http://del-repos.extreme-ix.org/epel/6/x86_64/repodata/ update the same.
epel-testing.repo base url is changed from http://del-repos.extreme-ix.org/epel/testing/6/ to http://del-repos.extreme-ix.org/epel/testing/6 update the same.

Now run yum repolist command.

[root@puppetlabs yum.repos.d]# yum repolist
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: centos.mirror.far.fi
 * extras: centos.mirror.far.fi
 * updates: centos.mirror.far.fi
repo id                 repo name                                                                        status
base                    CentOS-6 - Base                                                               6,713
epel                     Extra Packages for Enterprise Linux 6 - x86_64           12,522
extras                  CentOS-6 - Extras                                                                 31
updates               CentOS-6 - Updates                                                             114
repolist: 19,380

Hope it helps.

Monday, September 3, 2018

How to reduce (lvreduce) the Logical Volume in Linux Server

Monday, September 03, 2018 0
How to reduce (lvreduce) the Logical Volume in Linux Server.

Situation

Here, /app1 is 100GB filesystem. We need to reduce it to 70GB 

[root@testserver ~]# df -hP
Filesystem                                                 Size  Used Avail Use% Mounted on
/dev/mapper/vg_main-lv_root                   31G  2.0G    28G   7%    /
tmpfs                                                        3.7G     0      3.7G   0%   /dev/shm
/dev/xvdb1                                              477M   93M  355M  21%  /boot
/dev/mapper/vg_DPFERT-lv_app1           99G   11G    84G  11%   /app1


[root@testserver ~]# vgs
  VG                #PV #LV #SN Attr      VSize    VFree
  vg_DPFERT    1   1      0    wz--n-  100.00g    0
  vg_main           1   2      0    wz--n-    31.50g   0

[root@testserver ~]# fdisk -l /dev/xvdc

Disk /dev/xvdc: 107.4 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

    Device Boot      Start         End      Blocks   Id  System
/dev/xvdc1               1       13054   104856254+  8e  Linux LVM

STEP 1 : First Unmount the LV

STEP 2 : Run e2fsck command to check the file system

[root@testserver ~]# e2fsck -f /dev/vg_DPFERT/lv_app1
e2fsck 1.43-WIP (20-Jun-2013)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/vg_DPFERT/lv_app1: 57971/6553600 files (0.2% non-contiguous), 3073907/26213376 blocks


STEP 3 : Run resize2fs command for resizing the file system.

[root@testserver ~]# resize2fs /dev/vg_DPFERT/lv_app1 70G 
resize2fs 1.43-WIP (20-Jun-2013)
Resizing the filesystem on /dev/vg_DPFERT/lv_app1 to 18350080 (4k) blocks.
The filesystem on /dev/vg_DPFERT/lv_app1 is now 18350080 blocks long.

STEP 4 : Run lvreduce command to resuce the lvsize.

[root@testserver ~]# lvreduce -L 70G /dev/vg_DPFERT/lv_app1
  WARNING: Reducing active logical volume to 70.00 GiB
  THIS MAY DESTROY YOUR DATA (filesystem etc.)
Do you really want to reduce lv_app1? [y/n]: y
  Size of logical volume vg_DPFERT/lv_app1 changed from 100.00 GiB (25599 extents) to 70.00 GiB (17920 extents).
  Logical volume lv_app1 successfully resized
[root@testserver ~]#

STEP 5 : Mount the LV
[root@testserver ~]# lvs
  LV          VG                  Attr          LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  lv_app1  vg_DPFERT  -wi-a-----  70.00g
  lv_root    vg_main       -wi-ao----  31.22g
  lv_swap  vg_main       -wi-ao---- 288.00m

[root@testserver ~]# vgs
  VG                  #PV #LV #SN Attr       VSize      VFree
  vg_DPFERT    1      1      0    wz--n-  100.00g   30.00g
  vg_main           1      2     0     wz--n-  31.50g     0
  
[root@testserver home]# df -hP
Filesystem                                          Size  Used Avail Use% Mounted on
/dev/mapper/vg_main-lv_root             31G   2.0G   28G   7%   /
tmpfs                                                  3.7G     0      3.7G   0%  /dev/shm
/dev/xvdb1                                        477M   93M  355M  21% /boot
/dev/mapper/vg_DPFERT-lv_app1     69G   11G   56G   16%  /app1

Hope it helps.

Export proxy variable in Linux by using command line

Monday, September 03, 2018 0
Here Export Command is used to set Proxy Environment Variables in Linux Servers.

Syntax:

export "http_proxy=http://username:password@proxy.example.com:port_no/"
export "https_proxy=https://username:password@proxy.example.com:port_no/"

Note : If password contains Reserved charecters, need to replace with respective percent-encoding.

Reserved characters and Percent-encoding
! - %21
# - %23
$ - %24
& - %26
' - %27
( - %28
) - %29
* - %2A
+ - %2B
, - %2C
/ - %2F
: - %3A
; - %3B
= - %3D
? - %3F
@ - %40
[ - %5B
] - %5D

Check the proxy server settings by running env command.

[root@puppetlabs ~]# env | grep -i proxy

If we want to remove proxy variables. Run unset command.

[root@puppetlabs ~]# unset http_proxy
[root@puppetlabs ~]# unset https_proxy

Tuesday, July 10, 2018

How to upgrade Ansible by using PIP?

Tuesday, July 10, 2018 0
What is PIP

PIP is a package management system used to install and manage software packages written in Python. If you do not have PIP installed, we can download and install it from this page: https://pypi.org/project/pip/

Download the required ansible tar.gz package from below URL.

https://releases.ansible.com/ansible/

Here i already have ansible 2.0 running on this server. I need to upgrade it to 2.2.

[root@ansibleserver nskselvan]# pip install ansible-2.2.0.0.tar.gz
DEPRECATION: Python 2.6 is no longer supported by the Python core team, please upgrade your Python. A future version of pip will drop support for Python 2.6
Processing ./ansible-2.2.0.0.tar.gz
Requirement already satisfied: paramiko in /usr/local/lib/python2.6/site-packages/paramiko-1.15.2-py2.6.egg (from ansible==2.2.0.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python2.6/site-packages/Jinja2-2.8-py2.6.egg (from ansible==2.2.0.0)
Requirement already satisfied: PyYAML in /usr/local/lib/python2.6/site-packages (from ansible==2.2.0.0)
Requirement already satisfied: setuptools in /usr/local/lib/python2.6/site-packages/setuptools-18.1-py2.6.egg (from ansible==2.2.0.0)
Requirement already satisfied: pycrypto>=2.6 in /usr/local/lib/python2.6/site-packages (from ansible==2.2.0.0)
Requirement already satisfied: ecdsa>=0.11 in /usr/local/lib/python2.6/site-packages/ecdsa-0.13-py2.6.egg (from paramiko->ansible==2.2.0.0)
Requirement already satisfied: MarkupSafe in /usr/local/lib/python2.6/site-packages/MarkupSafe-0.23-py2.6-linux-x86_64.egg (from jinja2->ansible==2.2.0.0)
Installing collected packages: ansible
  Found existing installation: ansible 2.0.0.1
    Uninstalling ansible-2.0.0.1:
      Successfully uninstalled ansible-2.0.0.1
  Running setup.py install for ansible ... done
Successfully installed ansible-2.2.0.0
[root@ansibleserver nskselvan]#

[root@ansibleserver nskselvan]# ansible --version
ansible 2.2.0.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
[root@ansibleserver nskselvan]#

Note : For safer side, please take the backup of all necessary files.

Tuesday, June 26, 2018

Solved : Sendmail stat=Service unavailable

Tuesday, June 26, 2018 0
Situation : Maillog says Service unavailable

Jun 24 06:43:54 testserver sendmail[24622]: w5O4hsFN024620: to=<testuser@remotetest.com> ctladdr=<oratest@testserver.mail.com> (20001/20001), delay=00:00:00, xdelay=00:00:00, mailer=relay, pri=150623, relay=relayserver.com. [10.0.34.1], dsn=5.0.0, stat=Service unavailable

Solution:




Go to /etc/mail path 

 
Step1 : generics-domains file should be below format.

[root@testserver mail]# cat generics-domains
testserver
testserver.mail.com
mail.com
testserver.mail.com

Step2 : genericstable file should be below format

[root@testserver mail]# cat /etc/mail/genericstable


@testserver.mail.com auth_mail_id@relaydomain.com
@mail.com auth_mail_id@relaydomain.com

Map the genericstable by running below command.


# makemap hash /etc/mail/genericstable < /etc/mail/genericstable

Here,
auth_mail_id@relaydomain.com    : Change to your relay server authentication mail id
@testserver.mail.com                      : Your smtp server name
testuser@remotetest.com                : Where you want to send mails
oratest@testserver.mail.com           : smtp server mail id

Now mail will be accepted by relay server.

Thursday, June 14, 2018

Difference between NFSv2, NFSv3 and NFS4 and advantage of NFSv4

Thursday, June 14, 2018 0

Difference between NFSv2, NFSv3 and NFS4 and advantage of NFSv4


Network File System (NFS), allows remote hosts to mount file systems over a network and interact with those file systems as though they are mounted locally. This enables system administrators to consolidate resources on to centralized servers on the network.

NFSv2:
    1. NFS version 2 (NFSv2) is older and widely supported.  NFSv2 is not supported on RHEL7
    2. It can use both TCP and UDP protocol over an IP network(port 2049). But it use UDP running over an IP network to provide a stateless network connection between the client and server.
    3. UDP is stateless, if the server goes down unexpectedly, UDP clients continue to saturate the network with requests for the server. when a frame is lost with UDP, the entire RPC request must be re transmitted

NFSv3:
    1. NFS version 3 (NFSv3) supports safe asynchronous writes and is more robust at error handling than NFSv2; it also supports 64-bit file sizes and offsets, allowing clients to access more than 2Gb of file data.
    2. It can use both TCP and UDP protocol over an IP network(port 2049). But it use UDP running over an IP network to provide a stateless network connection between the client and server.
    3. UDP is stateless, if the server goes down unexpectedly, UDP clients continue to saturate the network with requests for the server. when a frame is lost with UDP, the entire RPC request must be re transmitted.

NFSv4:
    1. NFS version 4 (NFSv4) works through firewalls and on the Internet, no longer requires an rpcbind service, supports ACLs, and utilizes stateful operations.
    2. RHEL 6 supports NFSv2, NFSv3, and NFSv4 clients. When mounting a file system via NFS, RHEL uses NFSv4 by default, if the server supports it.
    3. It use TCP protocol. With TCP, only the lost frame needs to be resent. For these reasons, TCP is the preferred protocol when connecting to an NFS server.

Advantage of NFSv4:
    1. The mounting and locking protocols have been incorporated into the NFSv4 protocol
    2. The server also listens on the well-known TCP port 2049. As such, NFSv4 does not need to interact with rpcbind, lockd, and rpc.statd daemons. The rpc.mountd daemon is required on the NFS server to set up the exports.

Wednesday, April 25, 2018

How to free the disk space from deleted files in which PIDs are still running?

Wednesday, April 25, 2018 0
Sometimes we have removed large size of logfiles or files from linux server as part of housekeeping. But it will not release the disk space while running lsof command

[root@testserver ~]# lsof / | grep -i deleted
sh          3716       root    4w   REG  249,0  2269000 3342481 /var/log/cellos/mcelogd-mon.log (deleted)
sh          3716       root    5w   REG  249,0  2333681 3342525 /var/log/cellos/mcelogd-mon.trc (deleted)
tgtd        4144       root    4w   REG  249,0 26684869 3342480 /var/log/cellos/qd.log (deleted)
tgtd        4144       root    5w   REG  249,0 27453138 3342483 /var/log/cellos/qd.trc (deleted)
tgtd        4145       root    4w   REG  249,0 26684869 3342480 /var/log/cellos/qd.log (deleted)
tgtd        4145       root    5w   REG  249,0 27453138 3342483 /var/log/cellos/qd.trc (deleted)
iscsiuio    4592       root    4w   REG  249,0 26684869 3342480 /var/log/cellos/qd.log (deleted)
iscsiuio    4592       root    5w   REG  249,0 27453138 3342483 /var/log/cellos/qd.trc (deleted)

Normally we need to stop the logging service (syslog) before deleting the log files or stop the service which is related to the files.
Stopping  log collection on running server is not good practice.

Then how to solve the issue?

Free the Disk Space

Find the PID of deleted process, go to /proc then nullify it.

[root@testserver ~]# cd /proc/3716/fd
[root@testserver fd]# ls -al
total 0
dr-x------ 2 root root  0 Apr  9 17:08 .
dr-xr-xr-x 9 root root  0 Apr  9 17:08 ..
lrwx------ 1 root root 64 Apr  9 17:08 0 -> /dev/null
l-wx------ 1 root root 64 Apr  9 17:08 1 -> /var/log/exadatatmp/__imglog_stdout__.tFvWrjjEZTPOJWm3racIEPF5W9YQqMI3
lrwx------ 1 root root 64 Apr  9 17:08 10 -> /dev/null
lrwx------ 1 root root 64 Apr  9 17:08 11 -> /dev/null
l-wx------ 1 root root 64 Apr  9 17:08 2 -> /var/log/exadatatmp/__imglog_stderr__.tFvWrjjEZTPOJWm3racIEPF5W9YQqMI3
lr-x------ 1 root root 64 Apr  9 17:08 255 -> pipe:[21686]
l-wx------ 1 root root 64 Apr  9 17:08 4 -> /var/log/cellos/mcelogd-mon.log (deleted)
l-wx------ 1 root root 64 Apr  9 17:08 5 -> /var/log/cellos/mcelogd-mon.trc (deleted)

The process ID 3716 running on 4w (write mode).

[root@testserver fd]# ls -al 4
l-wx------ 1 root root 64 Apr  9 17:08 4 -> /var/log/cellos/mcelogd-mon.log (deleted)

We can release the space by nullify this file.

[root@testserver ~]# >/proc/3716/fd/4
[root@testserver ~]# lsof / | grep -i deleted | grep -i 3716
sh          3716       root    5w   REG  249,0        0 3342525 /var/log/cellos/mcelogd-mon.trc (deleted)

Alternate way, we can compress the file or we can kill the PID if not needed.

Thursday, April 19, 2018

Facter command in Linux

Thursday, April 19, 2018 0
Facter command is used to Collect and display facts about the current server. The library behind Facter is easy to expand, making Facter an easy way to collect information about a system information such as hardware details, network settings, virtualization type and kernel/OS information from within the shell or within Ruby. If no facts are specifically asked for, then all facts will be displayed.

If we want to use facter command, we need to install facter rpm.

Display all facts

[root@testserver manifests]# facter
architecture => x86_64
augeasversion => 1.1.0
bios_release_date => 01/06/2018
bios_vendor => Xen
bios_version => 4.4.4OVM
blockdevice_xvda_size => 53687091200
blockdevices => xvda
domain => puppet.test.com
facterversion => 2.4.4
filesystems => ext4,iso9660
fqdn => testserver.puppet.test.com
gid => root
hardwareisa => x86_64
....
....
uptime_days => 62
uptime_hours => 1491
uptime_seconds => 5368909
uuid => 0004FB00-0006-0000-7071-48CB71E70F8A
virtual => xenhvm

Display a single fact

[root@testserver ~]#  facter interfaces
eth0,eth1,eth2,lo

Facts format as JSON

[root@testserver ~]# facter --json architecture kernel  uptime timezone puppetversion bios_vendor
{
  "timezone": "CAT",
  "uptime": "62 days",
  "architecture": "x86_64",
  "kernel": "Linux",
  "bios_vendor": "Xen",
  "puppetversion": "3.8.1"

Facts format as YAML

[root@testserver  ~]# facter --yaml architecture kernel  uptime timezone puppetversion bios_vendor
---
bios_vendor: Xen
kernel: Linux
uptime: 62 days
timezone: CAT
architecture: x86_64
puppetversion: 3.8.1

Facter will produce output by below format.
a. JSON
b. YAML
c. Plaintext

Thursday, April 12, 2018

How to change the default login shell by command line in linux server?

Thursday, April 12, 2018 0
By using chsh command we can change the default login shell to other shell.

By using -l option we can list the available login shell on linux server.

[root@testserver ~]# chsh -l
/bin/sh
/bin/bash
/sbin/nologin
/bin/dash
/bin/tcsh
/bin/csh
/bin/ksh

Then which user login shell need to change, we can change by below command. (If a shell is not given on the command line, chsh prompts for one)

[root@testserver ~]# chsh hygmtng7
Changing shell for hygmtng7.
New shell [/bin/ksh]: /bin/bash
Shell changed.

Before
[root@testserver ~]# cat /etc/passwd | grep -i hygmtng7
hygmtng7:x:20009:20009::/gmtng7/hyperion:/bin/ksh

After
[root@testserver ~]# cat /etc/passwd | grep -i hygmtng7
hygmtng7:x:20009:20009::/gmtng7/hyperion:/bin/bash