LINUX AND VMWARE STUFF

This Blog is to share our knowledge and expertise on Linux System Administration and VMware Administration

Monday, October 15, 2018

How to generate sosreport in different directory instead of default directory

Monday, October 15, 2018 0
Sosreport is a tool that collects configuration details, system information, diagnostic information and system logs from Linux server. Sosreport  generates  an archive report from the running system. The archive may be stored locally or centrally for recording or tracking purposes.

 Normally sosreport will be saved under /tmp (RHEL6) /var/tmp (RHEL7) directory on linux server. Some time we need to save the sosreport to some other temporary directory because of space issue. 

 Below steps are used to generate sosreport in different directory.

 RHEL 5.6 and earlier

 We need to set the temporary path environment before running sosreport.

 [root@nsk ~]# env TMP=/path/to/target/directory sosreport

 RHEL 5.7 and above

 We have to use option --tmp-dir with sosreport to redirect the sosreport path 

 [root@nsk ~]# sosreport --tmp-dir /path/to/target/directory

 Sosreport may have hung because of a specific plugin. We can list all available sosreport plugin modules by below command

 [root@nsk ~]# sosreport -l

 Identify which plugin is hanging on by below command

 [root@nsk ~]# sosreport -vvvv 

 If a plugin times out or last plugin does not finish, rerun the sosreport and skip that plugin

 [root@nsk ~]# sosreport -v -n plugin_name --tmp-dir /path/to/target/directory
Tags # Linux General Continue Reading
By v�o l�c No comments:
Labels:

Friday, October 5, 2018

Digging audit log with ausearch tool in RHEL 7

Friday, October 05, 2018 0
ausearch is a tool to search audit daemon logs based upon the events based on different search criteria.

  Audit system stores log entries in the /var/log/audit/audit.log file; if log rotation is enabled, rotated audit.log files are stored in the same directory.

 Each event consists of three records, which share the same time stamp. Each record consists of several name=value pairs separated by a white space or a comma. 

 type=SYSCALL msg=audit(1510471123.129:36): arch=c000003e syscall=175 success=yes exit=0 a0=1901a20 a1=1c5d a2=41a2d8 a3=18fe400 items=0 ppid=7 24 pid=725 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="modprobe" exe="/usr/bin/kmo d" subj=system_u:system_r:insmod_t:s0 key=(null)
type=PROCTITLE msg=audit(1510471123.129:36): proctitle=2F7362696E2F6D6F6470726F6265002D71002D2D0069707461626C655F6D616E676C65
type=NETFILTER_CFG msg=audit(1510471123.276:37): table=nat family=2 entries=0

 Here, ausearch utility allows us to search Audit log files for specific events. Also it is used to read the audit log timestamp to user readable timestamp by using -i (--interpret) option.

 Example:

 1.Search for hostname with given hostname specific.

 [root@nsk ~]# ausearch -i -hn nsk.testingserver.internal
----
type=USER_AUTH msg=audit(10/04/2018 12:32:02.610:75) : pid=1617 uid=root auid=unset ses=unset subj=system_u:system_r:xdm_t:s0-s0:c0.c1023 msg='op=PAM:authentication grantors=pam_permit acct=gdm exe=/usr/libexec/gdm-session-worker hostname=nsk.testingserver.internal addr=? terminal=/dev/tty1 res=success'
----

 2. Search only the first event that matches the search pattern.

 [root@nsk ~]# ausearch -i -hn nsk.testingserver.internal --just-one
----
type=USER_AUTH msg=audit(10/04/2018 12:32:02.610:75) : pid=1617 uid=root auid=unset ses=unset subj=system_u:system_r:xdm_t:s0-s0:c0.c1023 msg='op=PAM:authentication grantors=pam_permit acct=gdm exe=/usr/libexec/gdm-session-worker hostname=nsk.testingserver.internal addr=? terminal=/dev/tty1 res=success'
[root@nsk ~]#

 3. Search for all logged actions performed by user nsk, using the user's login ID (auid). 

 [root@nsk log]# ausearch -ua 1001 -i
----
type=LOGIN msg=audit(12/04/2017 11:39:18.052:156) : pid=1433 uid=root subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 old-auid=unset auid=nsk tty=(none) old-ses=4294967295 ses=2 res=yes
----
type=USER_ROLE_CHANGE msg=audit(12/04/2017 11:39:18.991:157) : pid=1433 uid=root auid=nsk ses=2 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='pam: default-context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 selected-context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 exe=/usr/sbin/sshd hostname=nsk addr=fe80::f24d:eb49:d219:6b0 terminal=ssh res=success'

 4. Search for an event matching the given Process ID.

 [root@nsk ~]# ausearch -i -p 2162
----
type=CRYPTO_SESSION msg=audit(10/05/2018 13:07:54.052:118) : pid=2162 uid=root auid=unset ses=unset subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=start direction=from-server cipher=aes256-ctr ksize=256 mac=hmac-sha2-256 pfs=curve25519-sha256@libssh.org spid=2163 suid=sshd rport=50935 laddr=10.0.2.15 lport=22  exe=/usr/sbin/sshd hostname=? addr=10.0.2.2 terminal=? res=success'
----

 5. Search the /var/log/audit/audit.log file for successful login attempts

 [root@nsk log]# ausearch --message USER_LOGIN --success yes --interpret
----
type=USER_LOGIN msg=audit(11/12/2017 13:04:08.279:110) : pid=662 uid=root auid=root ses=2 subj=system_u:system_r:local_login_t:s0-s0:c0.c1023
msg='op=login id=root exe=/usr/bin/login hostname=nsk addr=? terminal=tty1 res=success'
----
type=USER_LOGIN msg=audit(11/12/2017 13:05:46.297:107) : pid=1242 uid=root auid=root ses=1 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c
0.c1023 msg='op=login id=root exe=/usr/sbin/sshd hostname=10.0.2.2 addr=10.0.2.2 terminal=/dev/pts/0 res=success'
----

 6. Search for all account and group activities.

 [root@nsk log]# ausearch -m ADD_USER -m DEL_USER -m ADD_GROUP -m DEL_GROUP -i
----
type=ADD_GROUP msg=audit(11/15/2017 22:05:29.396:169) : pid=15764 uid=root auid=root ses=2 subj=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 msg='op=add-group id=dovecot exe=/usr/sbin/groupadd hostname=? addr=? terminal=? res=success'
----
type=ADD_USER msg=audit(11/15/2017 22:05:29.431:171) : pid=15769 uid=root auid=root ses=2 subj=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 msg='op=add-user id=dovecot exe=/usr/sbin/useradd hostname=? addr=? terminal=? res=success'

 7. Search for all failed PROCTITLE from yesterday up until now

 [root@nsk log]# ausearch --start yesterday --end now -m PROCTITLE -sv no -i
----
type=PROCTITLE msg=audit(10/03/2018 18:00:02.623:71) : proctitle=/usr/sbin/httpd -DFOREGROUND
type=SYSCALL msg=audit(10/03/2018 18:00:02.623:71) : arch=x86_64 syscall=open success=no exit=EACCES(Permission denied) a0=0x560fb5acc008 a1=O_RDONLY a2=0x1b6 a3=0x24 items=0 ppid=1 pid=1145 auid=unset uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=(none) ses=unset comm=httpd exe=/usr/sbin/httpd subj=system_u:system_r:httpd_t:s0 key=(null)
type=AVC msg=audit(10/03/2018 18:00:02.623:71) : avc:  denied  { read } for  pid=1145 comm=httpd name=php-mapi.cfg dev="dm-0" ino=19498435 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:zarafa_etc_t:s0 tclass=file

 Please refer man ausearch for more option.
Tags # Linux General Continue Reading
By v�o l�c No comments:
Labels:

Thursday, October 4, 2018

How to read/convert Audit log timestamp in RHEL7

Thursday, October 04, 2018 0
By default, audit log would be like below.

 type=SYSCALL msg=audit(1510471123.129:36): arch=c000003e syscall=175 success=yes exit=0 a0=1901a20 a1=1c5d a2=41a2d8 a3=18fe400 items=0 ppid=7 24 pid=725 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="modprobe" exe="/usr/bin/kmo d" subj=system_u:system_r:insmod_t:s0 key=(null)
type=PROCTITLE msg=audit(1510471123.129:36): proctitle=2F7362696E2F6D6F6470726F6265002D71002D2D0069707461626C655F6D616E676C65 type=NETFILTER_CFG msg=audit(1510471123.276:37): table=nat family=2 entries=0

 Below commands are used to convert the auditlog timestamp to user readable timestamp.

 1. AUSEARCH : 

 ausearch utility allows us to search Audit log files for specific events. Also it is used to read the audit log epoch timestamp to user readable timestamp.By default, ausearch searches the /var/log/audit/audit.log file. We can specify a different file using the ausearch options -if file_name command

 [root@nsk log]# ausearch -i | grep -i CONFIG
type=CONFIG_CHANGE msg=audit(11/12/2017 12:48:40.357:5) : audit_backlog_limit=8192 old=64 auid=unset ses=unset subj=system_u:system_r:unconfined_service_t:s0 res=yes
type=CONFIG_CHANGE msg=audit(11/12/2017 12:48:40.357:6) : audit_failure=1 old=1 auid=unset ses=unset subj=system_u:system_r:unconfined_service_t:s0 res=yes

 -i, --interpret : Interpret numeric entities into text

 2. DATE :  

 A) Date command with %s is used to convert normal time to epoch time.
%s : seconds since 1970-01-01 00:00:00 UTC

 Display the current time in the given FORMAT, or set the system date
[root@nsk audit]# date +%s
1538570770
[root@nsk audit]# date +%s
1538570773

 B) Date command with -d@ is used to convert epoch time to normal time.
[root@nsk audit]# date -d@1538570776
Wed Oct  3 18:16:16 IST 2018
[root@nsk audit]#

 C) If we want to convert the specific system time to epoch time.
[root@nsk audit]# date --date="Wed Oct  3 18:16:16 IST 2018" +%s
1538570776

 3. PERL : Perl was originally a language optimized for scanning arbitrary text files, extracting information from those text files, and printing reports based on that information

 [root@nsk audit]# perl -pe 's/(\d+)/localtime/e' audit.log  | more
type=DAEMON_START msg=audit(Sun Nov Wed Oct  3 20:22:26 2018 12:48:40 2017.186:6974): op=start ver=2.7.6 format=raw kernel=3.10.0-693.el7.x86_64 auid=4294967295 pid=606 uid=0 ses=4294967295 subj=system_u:system_r:auditd_t:s0 res=success
type=CONFIG_CHANGE msg=audit(Sun Nov Wed Oct  3 20:22:26 2018 12:48:40 2017.357:5): audit_backlog_limit=8192 old=64 auid=4294967295 ses=429496 7295 subj=system_u:system_r:unconfined_service_t:s0 res=1

 Here,
-p : Print out the pattern space
-e : command (This command allows one to pipe input from a shell command into pattern space)
localtime         : it is perl function, it is having 9 elements.
-d : Match 1 or more repetition of digits
Tags # Linux General Continue Reading
By v�o l�c No comments:
Labels:

Wednesday, October 3, 2018

How to rebuild the GRUB configuration file in RHEL7

Wednesday, October 03, 2018 0
Rebuilding the GRUB configuration file in RHEL7

 Situation : Whenever there is inconsistency in the GRUB file, empty grub configuration or of the grub file got removed accidentally, follow the below steps to fix the grub issue.





Solution:

 STEP 1 : List the partitions by using “ ls “ command
grub > ls
(proc) (hd0) (hd0,msdos3) (hd0,msdos2) (hd0,msdos1) (fd0)

 STEP 2: Findout the root partitions by using command ls < partition name >. Run ls command to all above listed available  partitions.








STEP 3: Set the root partition. In our case its present in (hd0,msdos1)
grub> set root=(hd0,msdos1)
grub>

STEP 4: Type Linux and specify the kernel or Press tab for listing the available kernel. Once choosing it, we have to specify the root volume (In our example its /dev/sda1).






grub> linux /vmlinuz-3.10.0-229.el7.x86_64 ro root=/dev/sda1

STEP 5: Specify the initramfs image press tab for listing it. Choose the initramfs image with the same kernel version.







grub> initrd /initramfs-3.10.0-229.el7.x86_64.img

Execute 'boot' command to boot the server with above configuration.

Once the server is up, use grub2-mkconfig command to rebuild the grub.conf file. 

[root@nsk /]grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.10.0-229.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-229.el7.x86_64.img
...
.
done

Finally verify the grub file under /boot/grub2/
[root@nsk /]# cd /boot/grub2/
[root@nsk grub2]# ls
device.map  fonts  grub.cfg  grubenv  i386-pc  locale

Reboot the server and verify one more time.
Tags # Linux Troubleshooting Continue Reading
By v�o l�c No comments:
Labels: