詳細検索

Writing Recipes for Existing Procedures in Chef 4 (DB Server)

Avatar
by komi
6 min read

Writing Recipes for Existing Procedures in Chef 4 (DB Server)
Translated from 日本語 • View original

Thank you. This is Komiya.

Continuing from the previous article, I will write about dbserver recipes.
The only thing I have to do is test with munin, zabbx and serverspec. It's a little patience already.

Since it is a fixed version installation like the web, external recipes are basically not used.

  • Install and launch MySQL and enable auto-start

[shell]# cd /root/chef-repo/site-cookbooks/mysqld/recipes

mkdir -p /root/chef-repo/site-cookbooks/mysqld/files/default/usr/local/src/rpms

mkdir -p .. /templates/default/etc/init.d

mkdir -p .. /files/default/etc/init.d

mkdir -p .. /files/default/root

mkdir -p /root/chef-repo/site-cookbooks/mysqld/files/default/etc/logrotate.d

mkdir -p /root/chef-repo/site-cookbooks/mysqld/files/default/opt/{backup,bin}

scp -Cp xxx-db02:/opt/bin/mysql-back.sh /root/chef-repo/site-cookbooks/mysqld/files/default/opt/bin/

cp -p /etc/logrotate.d/mysqld /root/chef-repo/site-cookbooks/mysqld/files/default/etc/logrotate.d/

scp -Cp xxx-db01:/etc/init.d/mysqld .. /files/default/etc/init.d/

scp -Cp xxx-db02:/etc/my.cnf .. /templates/default/etc/

cp -p /root/.path_to_file .. /files/default/root/

cd /opt/src/rpms/

tar czf /root/chef-repo/site-cookbooks/mysqld/files/default/usr/local/src/rpms/db-rpm.tar.gz ./db-rpm/

tar czf /root/chef-repo/site-cookbooks/mysqld/files/default/usr/local/src/rpms/db-tools-rpm.tar.gz ./db-tools-rpm/

tar tzf /root/chef-repo/site-cookbooks/mysqld/files/default/usr/local/src/rpms/db-rpm.tar.gz

vi .. /templates/default/etc/my.cnf

diff /etc/my.cnf .. /templates/default/etc/my.cnf

53c53

< server-id = 100

server-id = <%= node['mysqld']['server_id'] %>

vi .. /.. /.. /nodes/10.0.0.241.json

Write the server_id attribute to the ↓node (server_id needs to be changed for each node to be replicated, { "mysqld" : { "server_id" : 103 }, "run_list":[ "role[dbserver]" ] }

cd /root/chef-repo/site-cookbooks/mysqld/recipes

vi mysqld-server.rb

package "perl-DBI" do not_if "rpm -qa|grep perl-DBI" action :install end

package "perl-TermReadKey" do not_if "rpm -qa|grep perl-TermReadKey" action :install end

cookbook_file "/tmp/db-rpm.tar.gz" do source "usr/local/src/rpms/db-rpm.tar.gz" mode 0644 end

cookbook_file "/tmp/db-tools-rpm.tar.gz" do source "usr/local/src/rpms/db-tools-rpm.tar.gz" mode 0644 end

filename = "db-rpm" filename2 = "db-tools-rpm" script "install_mysqld" do not_if 'rpm -qa|grep mysql-server' interpreter "bash" user "root" code <-EOL cd="" mp="" tar="" xzf="" mp/#{filename}.tar.gz="" rpm="" -i="" mp/#{filename}/_rpm="" rpm="" -i="" mp/#{filename}/5.5.27/mysql_rpm="" tar="" xzf="" mp/#{filename2}.tar.gz="" rpm="" -i="" mp/#{filename2}/m*rpm="" eol="" end="" cookbook_file="" "/etc/init.d/mysqld"="" do="" source="" "etc/init.d/mysqld"="" mode="" 0755="" end="" template="" '/etc/my.cnf'="" do="" owner="" 'root'="" group="" 'root'="" source="" 'etc/my.cnf'="" end="" service="" "mysqld"="" do="" supports="" :status=""> true, :restart => true, :reload => :true action [ :enable, :start ] end

cookbook_file "/etc/logrotate.d/mysqld" do source "etc/logrotate.d/mysqld" owner 'root' group 'root' mode 0644 end

directory '/opt/bin/' do owner 'root' group 'root' mode '0755' action :create end

directory '/opt/backup/' do owner 'root' group 'root' mode '0755' action :create end

cookbook_file "/opt/bin/mysql-back.sh" do source "opt/bin/mysql-back.sh" owner 'root' group 'root' mode 0755 end

#cron "mysql-backup" do

minute "31"

hour "4"

day "*"

month "*"

weekday"*"

command "/opt/bin/mysql-back.sh"

action :create

#end[/shell]

・Create a user for mysql
I decided not to use opscode recipes because they are difficult for beginners to read and debug errors.
Reference:
Install MySQL in ChefHandle MySQL passwords in ChefGet started todayManage MySQL users and databases in Chef Chef
[shell]# vi site-cookbooks/mysqld/recipes/mysql-users.rb #include_recipe "openssl" #include_recipe 'database::mysql'

#mysql_connection_info = {:host => "localhost",

:username => 'root',

#:password => node['mysql']['server_root_password']}

:password => ''}

#mysql_database "xxx_db" do

connection mysql_connection_info

action :create

#end

xxxadm_data = Chef::EncryptedDataBagItem.load("mysqlusers","xxx_admin") root_data = Chef::EncryptedDataBagItem.load("mysqlusers","root") repl_data = Chef:: EncryptedDataBagItem.load("mysqlusers","repl") myuser_o = xxxadm_data["user"] mypass_o =xxxadm_data["pass"] myuser_ro = root_data["user"] mypass_ro =root_data[" pass"] myuser_re = repl_data["user"] mypass_re =repl_data["pass"] #mysql_database_user "#{user}" do

connection mysql_connection_info

password "#{password}"

database_name "*"

host "[#{host}, %, localhost]"

privileges [:all]

action [:create, :grant]

#end # mysqlconn = "mysql -u root" script "create_msql_xxxadm" do not_if "ls /root/.path_to_file" #not_if "#{mysqlconn} -p #{mypass_ro} -e 'select count(_) from mysql.user where user=\ 'repl\';'" interpreter "bash" user "root" code

<-EOL #{mysqlconn} < EOF grant all privileges on *._ to #{myuser_o}@'%' identified by "#{mypass_o}"; grant all privileges on . to #{myuser_o}@'10.0.0.%' identified by "#{mypass_o}"; grant all privileges on . to #{myuser_o}@'localhost' identified by "#{mypass_o}"; grant all privileges on . to #{myuser_ro}@'10.0.0.%' identified by "#{mypass_ro}"; grant all privileges on . to #{myuser_ro}@'localhost' identified by "#{mypass_ro}"; grant replication slave, replication client on . to #{myuser_re}@'10.0.0.%' identified by "#{mypass_re}"; grant replication slave, replication client on . to #{myuser_re}@'localhost' identified by "#{mypass_re}"; drop database test; delete from mysql.user where password=''; flush privileges; EOF EOL end

script "create_pfile" do not_if 'ls /root/.path_to_file' interpreter "bash" user "root" code <-EOL echo="" "#{mypass_ro}"=""> /root/.path_to_file chown 400 /root/.path_to_file EOL end[/shell] *An error will occur if EOF is not at the beginning of the line.

・Replication construction recipe (note that it has not been tested)

I haven't been able to test it yet, but I thought about how to turn the actual procedure into a recipe, and it turned out to be as follows.
 I can only think of doing it in bash.
 It seems like it will take time to go through third-party recipes and decipher them completely
 It's very dangerous to use something you don't understand, so I'll go with it for now.
  The stg server is backed up by stg db, so I don't use it.
  Bring the latest dump file taken with db02 or something like remote_file and put it in.
  Since it may be used for migration, it is data taken with master-data=2, so
  Extract the GHANGE MASTER TO syntax in the dump file with zgrep or something like that and put it in a variable.
  I passed a variable in mysql -e or something like that, let it see the master, and enable it to be read-only.
  I think it will feel like replication starts.
  For the time being, in the direction of making a replication_setup.sh and having it executed.
  If you put in the data you have taken in all, the entire user will enter the mysql schema.
   Maybe I didn't have to encrypt it with databags. (Now)

Writing a Shell to Set Up Replication
[shell]# vi replication_setup.sh ------------------------------- #!/bin/bash

BACKUP_SRV='10.0.0.222'
BACKUP_PATH='/opt/backup'
BACKUP_FL='mysqldump_3306_&quot;`LANG=C;date +%y%m%d`&quot;.sql'
MSQL_USER=root
MSQL_PASS=`cat /path_to_file`
MSQL_CONN=&quot;mysql -u ${MSQL_USER} -p${MSQL_PASS}&quot;
MSQL_CMD_RO='set global read_only=1;'
#MAIL_TO='hoge@isao.co.jp'
MAIL_TO='hoge@isao.net'
MYHOST=`uname -n`
DATETIME=`date +%Y%m%d_%H:%M:%D`

if [ -s /tmp/${BACKUP_FL}.gz ]; then
 :
else
 echo &quot;replication setup faild&quot;|mail -s `uname -n`_`date +%Y%m%d_%H:%M:%S` ${MAIL_TO}
 exit 0
fi

gunzip /tmp/${BACKUP_FL}.gz
${MSQL_CONN} &lt; /tmp/${BACKUP_FL}
CHANGE_MASTER=`cat /tmp/${BACKUP_FL}|head -30|grep 'CHANGE MASTER TO'|sed -e 's/^-- //g'`

${MSQL_CONN} -e &quot;stop slave; &quot;
${MSQL_CONN} -e &quot;${CHANGE_MASTER}&quot;
${MSQL_CONN} -e &quot;${MSQL_CMD_RO}&quot;
${MSQL_CONN} -e &quot;start slave; &quot;

echo `${MSQL_CONN} -e &quot;show slave status\G&quot;`|mail -s `uname -n`_`date +%Y%m%d_%H:%M:%S` ${MAIL_TO}

exit 0
-------------------------------

chmod +x replication_setup.sh

mkdir /root/chef-repo/site-cookbooks/mysqld/files/default/tmp

mv replication_setup.sh /root/chef-repo/site-cookbooks/mysqld/files/default/tmp/[/shell]

<br>For the time being, I'm going to pick it up from STG every day.<br>```

\[shell\]# mkdir -p /root/chef-repo/site-cookbooks/mysqld/files/default/opt/{backup,bin}

# vi /opt/bin/get\_dbbkup.sh

#!/bin/bash

CHEF_BKUP_PATH=/root/chef-repo/site-cookbooks/mysqld/files/default/opt/backup BACKUP_SRV='10.0.0.222' BACKUP_PATH='/opt/backup' BACKUP_FL='mysqldump_3306_"LANG=C;date +%y%m%d".sql' scp -Cp root@${BACKUP_SRV}:${BACKUP_PATH}/${BACKUP_FL}.gz ${CHEF_BKUP_PATH}

exit 0


# chmod +x /opt/bin/get\_dbbkup.sh

# crontab -e

# crontab -l

get db backup data for chef slave setup.

0 6 * * * /opt/bin/get_dbbkup.sh > /dev/null 2>&1 --------------[/shell]
```

Place the replication stop detection script in files
[shell]# scp -p xxx-db02:/opt/bin/rep_fail_mail.sh /root/chef-repo/site-cookbooks/mysqld/files/default/opt/bin/[/shell]
Write a recipe for replication setup
[shell]# cd /root/chef-repo/site-cookbooks/mysqld/recipes

vi replication.rb

----------------------------
dumpfile = mysqldump_3306_&quot;`LANG=C;date +%y%m%d`&quot;.sql
cookbook_file &quot;/tmp/#{dumpfile}.gz&quot; do
  source &quot;opt/backup/#{dumpfile}.gz&quot;
  mode 0644
end

script &quot;setup_replication&quot; do
  interpreter &quot;bash&quot;
  user        &quot;root&quot;
  code &lt;&lt;-EOL
    /tmp/replication_setup.sh 
  EOL
end

cookbook_file &quot;/opt/bin/rep_fail_mail.sh&quot; do
  source &quot;opt/bin/rep_fail_mail.sh&quot;
  mode 0755
end

cron &quot;rep_fail_mail&quot; do
  minute &quot;0-59/15&quot;
  hour &quot;*&quot;
  day &quot;*&quot;
  month &quot;*&quot;
  weekday&quot;*&quot;
  command &quot;/opt/bin/rep_fail_mail.sh&quot;
  action :create
end
----------------------------[/shell]

・About HA construction settings

There are three main types used in the company: Heartbeat linkage, MHA, and VIP manual switching.
So, this time it will be VIP manual switching (the unfortunate type with the longest downtime and the most likely to cause data loss).
I don't particularly feel the need to attach VIP to the DB after the third one, but for when both are broken.
Just in case, I will prepare a recipe that only transfers the VIP grant script. (Not in rolls)

[shell]# scp -Cp xxx-db02:/opt/bin/eni_vip_up.sh /root/chef-repo/site-cookbooks/mysqld/files/default/opt/bin/

# vi vip-attach.rb

cookbook_file "/opt/bin/eni_vip_up.sh" do source "opt/bin/eni_vip_up.sh" mode 0755 end

#script "eni_vip_up" do

interpreter "bash"

user "root"

code <-EOL

/opt/bin/eni_vip_up.sh

EOL

#end -----------------------------[/shell] ※間違ってロールに入れてしまったときのために転送したスクリプトの実行はコメントインしておく

クックブックのシンタックステスト
[shell]# knife cookbook test mysqld[/shell] FATAL:エラーがでなければOKと思われる

今回はここまでです。長々読んでいただいてありがとうございました。
Next time it will be a recipe for munin and zabbix.</-EOL></-EOL>

Related Articles