Translated from 日本語 • View original Hello. This is Komiya. Recently, there have been many requests for "recording execution commands" in security-related work. I think there are several ways to do this, but this time I would like to put the time in bash_history and combine it into a single log.
What I found convenient after doing this is that the log can be resolved with grep when reporting work time during maintenance.
Building with jenkins has recently been a continuous integration. At first, there was a psychological barrier, but when I tried it, I thought it was good because it was quite easy and there was no mistake in the procedure. I had to build it several times in response to vulnerabilities, but it was convenient to have a jenkins job.
CentOS 6.5. Jenkins' job is as follows: It is simply the execution of a shell with parameters.
#!/bin/bash
topdir="${HOME}/rpmbuild"
rpmdir="${topdir}/RPMS/x86_64"
if [ -f ${HOME}/.rpmmacros ]; then
echo "%_topdir ${topdir}" > "${HOME}/.rpmmacros"
echo "%_signature gpg" >> "${HOME}/.rpmmacros"
echo "%_gpg_name D279xxxx" >> "${HOME}/.rpmmacros"
fi
if [ -d ${HOME}/rpmbuild ]; then
mv ${HOME}/rpmbuild{,.`date +%Y%m%d.%H%M`}
mkdir -p ${HOME}/rpmbuild/SRPM
fi
case "${TARGET}" in
*.src.rpm) rpm -Uvh "${TARGET}"
cp -p ${topdir}/SPECS/bash.spec{,.org}
sed -i 's/make "CFLAGS=$CFLAGS -fwrapv" "CPPFLAGS=-D_GNU_SOURCE -DRECYCLES_PIDS `getconf LFS_CFLAGS`"/make "CFLAGS=$CFLAGS -fwrapv" "CPPFLAGS=-D_GNU_SOURCE -DRECYCLES_PIDS ` getconf LFS_CFLAGS` -DSYSLOG_HISTORY"/g' ${topdir}/SPECS/bash.spec
sed -i 's/Release: 29%{?dist}/Release: 29%{?dist}_isao_5/g' ${topdir}/SPECS/bash.spec
sed -i '108s/^$/\n/g' ${topdir}/SPECS/bash.spec
sed -i '109s/^$/Patch145: bash-syslog_facirity.patch\n/g' ${topdir}/SPECS/bash.spec
sed -i '181s/^$/%patch145 -p1\n/g' ${topdir}/SPECS/bash.spec
cd ${HOME}/rpmbuild/SOURCES/
tar xzf bash-4.1.tar.gz
cp -rp bash-4.1{,.org}
sed -i 's/# define SYSLOG_FACILITY LOG_USER/# define SYSLOG_FACILITY LOG_LOCAL6/g' ${HOME}/rpmbuild/SOURCES/bash-4.1/config-top.h
sed -i 's! (SYSLOG_FACILITY| SYSLOG_LEVEL, "HISTORY: PID=%d UID=%d %s", getpid(), current_user.uid, line)! (SYSLOG_FACILITY| SYSLOG_LEVEL, "HISTORY: PID=%d PPID=%d SID=%d User=%s UID=%d CMD=%s", getpid(), getppid(), getsid(getpid()), current_user.user_name, current_user.uid, line)!g' ${HOME}/rpmbuild/ SOURCES/bash-4.1/bashhist.c
sed -i 's! (SYSLOG_FACILITY| SYSLOG_LEVEL, "HISTORY (TRUNCATED): PID=%d UID=%d %s", getpid(), current_user.uid, trunc)! (SYSLOG_FACILITY| SYSLOG_LEVEL, "HISTORY (TRUNCATED): PID=%d PPID=%d SID=%d User=%s UID=%d CMD=%s", getpid(), getppid(), getsid(getpid()), current_user.user_name, current_user.uid, trunc)!g' ${HOME}/ rpmbuild/SOURCES/bash-4.1/bashhist.c
diff -crN bash-4.1.org bash-4.1 > ${HOME}/rpmbuild/SOURCES/bash-syslog_facirity.patch
rpmbuild -ba ${topdir}/SPECS/bash.spec
;;
*) echo 'environment variable TARGET must be set.'; exit 1;;
esac
The parameters I chose to TARGET are as follows: http://vault.centos.org/6.5/updates/Source/SPackages/bash-4.1.2-15.el6_5.2.src.rpm
upgraded. Jobs are separate for each version. http://vault.centos.org/6.6/os/Source/SPackages/bash-4.1.2-29.el6.src.rpm
'${HOME}' is '/var/lib/jenkins'.
The flow is to put the source RPM as a drop, unzip the source archive, sed it, make a patch, sed the spec file so that the created patch can be applied, and build it. I found out that the option to patch the entire directory in diff was -crN. The patch is as follows.
# cat bash-syslog_facirity.patch
diff -crN bash-4.1.org/bashhist.c bash-4.1/bashhist.c
*** bash-4.1.org/bashhist.c 2009-08-15 04:33:02.000000000 +0900
--- bash-4.1/bashhist.c 2014-12-16 19:13:45.272586470 +0900
***************
*** 705,716 ****
char trunc[SYSLOG_MAXLEN];
if (strlen(line) < SYSLOG_MAXLEN)
! syslog (SYSLOG_FACILITY|SYSLOG_LEVEL, "HISTORY: PID=%d UID=%d %s", getpid(), current_user.uid, line);
else
{
strncpy (trunc, line, SYSLOG_MAXLEN);
trunc[SYSLOG_MAXLEN - 1] = '';
! syslog (SYSLOG_FACILITY|SYSLOG_LEVEL, "HISTORY (TRUNCATED): PID=%d UID=%d %s", getpid(), current_user.uid, trunc);
}
}
#endif
--- 705,716 ----
char trunc[SYSLOG_MAXLEN];
if (strlen(line) < SYSLOG_MAXLEN)
! syslog (SYSLOG_FACILITY|SYSLOG_LEVEL, "HISTORY: PID=%d PPID=%d SID=%d User=%s UID=%d CMD=%s", getpid(), getppid(), getsid(getpid()), current_user.user_name, current_user.uid, line);
else
{
strncpy (trunc, line, SYSLOG_MAXLEN);
trunc[SYSLOG_MAXLEN - 1] = '';
! syslog (SYSLOG_FACILITY|SYSLOG_LEVEL, "HISTORY (TRUNCATED): PID=%d PPID=%d SID=%d User=%s UID=%d CMD=%s", getpid(), getppid(), getsid(getpid()), current_user.user_name, current_user.uid, trunc); }
}
#endif
diff -crN bash-4.1.org/config-top.h bash-4.1/config-top.h
*** bash-4.1.org/config-top.h 2009-12-23 05:29:39.000000000 +0900
--- bash-4.1/config-top.h 2014-12-16 19:07:53.607579557 +0900
***************
*** 103,109 ****
bash_add_history() to be sent to syslog(). */
/* #define SYSLOG_HISTORY */
#if defined (SYSLOG_HISTORY)
! # define SYSLOG_FACILITY LOG_USER
# define SYSLOG_LEVEL LOG_INFO
#endif
--- 103,109 ----
bash_add_history() to be sent to syslog(). */
/* #define SYSLOG_HISTORY */
#if defined (SYSLOG_HISTORY)
! # define SYSLOG_FACILITY LOG_LOCAL6
# define SYSLOG_LEVEL LOG_INFO
#endif
The log format has been changed to include the parent process ID and the SU PID with reference to this URL. (I thought it would be easier to audit it)
The facility has been changed to local6. Suppose you do the following in 'rsyslog.conf':
*.info; mail.none; authpriv.none; cron.none; local6.none /var/log/messages
# bash_history log
local6.* /var/log/bash_history
I think it is necessary to remove bash from the update target of yum separately. (If 'yum-updatesd' is valid, use 'exclude=bash*' in 'yum.conf' later, etc.) If you don't do it later, it will not be updated to custom bash when you run the recipe, so be careful. When I entered it with Chef(yum), I had to put 'options "--disablerepo=base,updates"'. This is a countermeasure that does not update packages in other repositories. In addition, if you do not put a recipe in the run_list order when flowing a recipe in Chef, it will be judged that there is no repository and will not be installed or upgraded.
I use a plugin called 'RPMSign' to sign the gpg and scp it into a custom yum repository. Other plugins include 'SSH plugin'.
Set the gpg key and passphrase in the system settings, and in the job you choose the key and write the full path of the package to sign the command line options (wildcards are acceptable).
The actual way the logs are produced is as follows.
Aug 21 11:11:54 localhost -bash: HISTORY: PID=18837 PPID=18836 SID=18837 User=komiyay UID=501 CMD=sudo su -
Aug 21 11:44:55 localhost bash: HISTORY: PID=19298 PPID=18864 SID=18837 User=root UID=0 CMD=knife solo data bag edit ssl nginx
I created a separate job to create a difference between SCP and Yum repository metadata, and I wanted to save the trouble of deleting it if the package fails to build and gets a slightly strange name.
bash_history is at risk of being tampered with, so measures such as forwarding it to another server in real time are still necessary, and I think there are methods such as rsyslog, syslog-ng, and td-agent, but they will be omitted because they will be super long.
Jenkins does a lot of things for you automatically, and it's convenient because there are no unspoken steps. I learned a lot. I was told about jenkins by Mr. Hiragata, who is an in-house developer. Thank you very much. Thanks to you, the psychological barrier has been reduced.
As an aside, if you want to use something other than bash and you can't cover it with an operation that only uses bash, I think you can add 'psacct' (RHEL system) or 'pacct' (Debian system) and add an option such as '--user' in 'lastcomm' to check and then check the history of each user. Maybe. I googled a little and saw someone who had the script command automatically launched in /etc/profile. It seems convenient to keep track of what happened, but I personally thought it was possible to do it with the package. By default, hisotry does not display the time, so it seems that there are different ways to display the time in a shell other than bash, and when I googled it, it was as shown in the link below. Put the date in the history of tcsh - Allow you to see the command execution time in the history of Toolbox zsh - Qiita If you are using anything other than the above, please check it at your own risk. By the way, this time bash is modified and sent to syslog, so it is not necessary, but if you want to put the time in each history file without doing that, it seems that you can set the shell variable "HISTTIMEFORMAT" after 3.0. Also, sh and csh can't edit the history, so I don't think it's possible to set the time. Also, is audited necessary to achieve the purpose of making tampering with history files detectable? I have never used auditd properly because it seems to take a lot of time to decide on the rules properly and think about proper operation, or because the readability of the log is not good.
Reference: the syslog recorded history history - C - C Program Develop [[Tech Blog vol.2] Process Accounting with PSACCT Package | Tech Blog | Managed Hosting Deenet] (http://www.denet.ad.jp/technology/2013/06/psacct-psacct-1-centos-yum-rootvaraccountpacct-history.html) How to keep a history of command execution on Linux/UNIX - drk7jp history Allow the date and time to be displayed in the results of the command - takami_hiroki's diary
Thank you for reading.