Translated from 日本語 • View original Hello. This is Komiya.
Thanks to you, I had the opportunity to customize it, so I would like to quote some of them and introduce them.
Basic things, etc.,
I think you should also take a look at the link below and "Introductory chef-solo" and its droplets.
Building a Nico Nico Server with Chef Solo and Knife Solo (2) - Introduction - :d wango Engineer Bromaga
chef-solo - All the knowledge to read and run Chef - Qiita
Chef Techniques for DevOps // Speaker Deck
Also, I am very much looking forward to hearing that an introductory book on chef practice will be published around GW by the person from Engine Yard (a company that has been using it for about 8 years before chef came out).
This is an explanation of Chef's node, role, enviroment, attribute, data_bags.
I think this article will provide a basic explanation and simple information for customization.
The only environment that actually used the recipe described below is CentOS 6.4, and unfortunately, we cannot guarantee that it will work in a different environment.
It seems that general-purpose (complex) recipes that guarantee operation between different operating systems can be used in cookbooks full of ☆ in the opscode community.
(I think it is not suitable for the demand to add as little extraneous things as possible or to make recipes for existing environments or procedures.) ）
The following is a description of the directory directly under the chef repository.
cookbooks: third-party cookbook storage
Data_bags :d ata_bags data storage
environments : Preferences file storage
nodes: Nodes (host/server) configuration file storage
roles: Role configuration file storage
site-cookbooks: A place to store your own cookbooks (cookbooks include recipes, handouts, etc.)
Each term is briefly explained below.
・ohai
This command retrieves the information of the host that applies the recipe that is included with chef.
It is possible to define an attribute based on the value obtained by ohai.
For example, if you want to change the setting value according to the OS installed memory and the number of CPU cores, or the hostname or
It is useful to use it when you want to put unique information such as IP addresses in a configuration file.
・node
nodes directory, and place a file called hostname.json or ipaddress.json directly under the
Define the server-specific attribure and the role to be assigned.
・role
A recipe that combines servers with the same role, such as DB and web, into a role and applies them to a group of servers with the same role.
and parameters together.
・enviroment
Define the attribures that need to be changed for each environment, such as the test environment, production environment, and development environment.
(It seems that it is possible to fix the cookbook version in Chef-Server, but not with Chef-Solo)
・attribute
This is the parameter you want to change for each node, role, and enviroment.
Define variables such as <%= %> in the file you call from the template resource.
This is used when you want to change the value of the configuration file for each individual role.
You can also define a default value in attributes/default.rb in each cookbook, and
It is also possible to set it with variables from the template resource. You can also set the value of ohai as it is.
・data_bags
Cookbooks contain global data that you don't want to include.
This includes user information.
It's not data that changes per role like attribute, so I write it in only one place.
It seems to be a good place to put the data you want to load.
・How to apply the recipe
If you want to apply a recipe for something to a host, you can specify the recipe after the host in the knife command, or
You need to specify the recipe or role in the run_list of node or role.
How to give it as an argument to a knife command directly without setting a role or node:
knife solo cook -o ＜cookbook＞::＜recipe＞【,＜cookbook＞::＜recipe＞】
*This method will not be able to manage which recipes were applied to which host in that repository.
I can't recommend it, but I think it may be used if you don't want to affect the environment that is already running.
Example of specifying it to run_list in node:
[shell]vi
Example of specifying a role in run_list [shell]$ vi roles/API-ext.json ---------------- { "name":"API-ext", "chef_type": "role", "json_class":"Chef::Role", "default_attributes":{ "base_ setting": { "swappiness": "0", "tcp_tw_reuse": "0", "tcp_tw_recycle": "0", "tcp_fin_timeout": "10", "tcp_max_syn_backlog": "8192", "somaxconn": "8192" } }, "override_ attributes":{}, "description":"API's role", "run_list": [ "recipe[roles::API-ext]" ] } ---------------- $ cat site-cookbooks/roles/recipes/API-ext.rb ---------------- include_recipe "base_setting::common-pkgs" include_recipe "base_setting::sysctl" include_recipe "base_setting::ntpd" include_recipe "base_setting::rps_cpus" include_ recipe "login-users::create_user" include_recipe "login-users::key_copy" include_recipe "login-users::api_user" include_recipe "java::java" include_recipe "nginx::nginx " include_recipe "nginx::nginx_virtual" include_recipe "play::play" include_recipe "mysqld::mysql-client" include_recipe "git::git" include_recipe "Flydata::Flydata" include_recipe "munin::munin-node" include_recipe "munin::munin-node-API-PVP-robi" include_recipe "zabbix::zabbix-agent" ----------------[/shell] ※ In the above section, the recipe is not listed in the role file, but the recipe required by the API-ext.rb of the roles cookbook is included.
*In the run_list of the JSON file of node and role, it is possible to write recipe and role with comma separation.
[[Japanese Translation] Beginner Chef Anti-Pattern by Julian Dunn #opschef_ja] (http://www.creationline.com/lab/3080)
Now, it is recommended to manage roles in a cookbook.
role, node settings are done properly, you can specify the hostname without specifying a cookbook or recipe with -o, and
Run Knife Solo Cook and the recipe will be applied as you set it.
The order in which recipes are applied is basically specified, but the restart of services specified in notifies is
After all recipes have been applied (converged). Please take a look at the following for easy understanding.
The misconception that Chef recipes are executed from top to bottom
・About the test method
As I may add later, you can check the syntax of the cookbook with the following command:
[shell]knife cookbook test
・Usage examples and implementation procedures for attribute and data_bags
We will list examples, but please also refer to the link below.
Understanding the [[chef] attribute | IT Infrastructure Miscellaneous Journal] (http://tech.blog.piyo.org/2012/05/23/chef-attribute%E3%81%AE%E7%90%86%E8%A7%A3/)
・Use ohai data to set up according to the number of OS memory and CPU cores.
When using ohai, you only need to define the template resource in the recipe and define the variables in the configuration file.
You don't need to define a role or node attribute.
If you type the ohai command directly, you can see what kind of value you can get.
[shell]$ ohai ipaddress [ "192.168.1.133" ][/shell]
Set 75% of OS onboard memory to innodb_buffer_pool_size [shell]$ vi site-cookbooks/mysqld/recipes/mysqld-server.rb template '/etc/my.cnf' do owner 'root' group 'root' source 'etc/ my.cnf.erb' notifies :restart, 'service[mysql]' end $ vi site-cookbooks/mysqld/templates/default/etc/my.cnf.erb innodb_buffer_pool_size = <%= ("#{node[:memory][: total]}"[/\d+/].to_f * 1024 * 0.75 ).to_i %>[/shell]
Set a value equivalent to the number of CPU cores installed in the OS to worker_processes. [shell]$ vi site-cookbooks/nginx/recipes/nginx.rb template "nginx.conf" do path "/etc/nginx/nginx.conf" source "etc/nginx/nginx.conf.erb" owner "root" group "root" mode 0644 notifies :reload, 'service[nginx]' end $ vi site-cookbooks/nginx/templates/default/etc/nginx/nginx.conf.erb worker_processes <%= node[:cpu][:total] %>;[/shell]
・Set hostname and IP address using ohai data [shell]$ vi site-cookbooks/zabbix/recipes/zabbix-agent.rb template '/etc/zabbix/zabbix_agentd.conf' do owner 'root' group 'root' source 'etc/ zabbix/zabbix_agentd.conf' end
$ vi site-cookbooks/zabbix/templates/default/etc/zabbix/zabbix_agentd.conf Hostname=<%= node['hostname'] %> $ cat site-cookbooks/mysqld/templates/default/etc/ my.cnf.erb|grep ipaddress server-id = <%= node[:ipaddress].split("."). last %> report-host = <%= node[:ipaddress] %>[/shell]
Since different values can be automatically set for each node,
I think it is very convenient because you don't have to prepare in advance or modify it after logging in and reload it one by one.
- Call different attributes for each role in the template resource and reflect them in the configuration file.
The recipe for sysctl in the base_setting cookbook does that.
[shell]$ cat site-cookbooks/base_setting/recipes/sysctl.rb template "/etc/sysctl.conf" do owner 'root' group 'root' source 'etc/sysctl.conf.erb' mode 0644 end $ cat site-cookbooks/base_setting/templates/default/etc/sysctl.conf.erb vm.swappiness = <%= node['base_setting']['swappiness'] %> net.ipv4.tcp_tw_reuse = <%= node['base_setting']['tcp_tw_reuse'] %> net.ipv4.tcp_tw_recycle = <%= node['base_setting']['tcp_tw_recycle'] %> net.ipv4.tcp_fin_timeout = <%= node['base_setting']['tcp_fin_timeout'] %> net.ipv4.tcp_max_syn_backlog = <%= node['base_setting']['tcp_max_syn_backlog'] %> net.core.somaxconn = <%= node['base_setting']['somaxconn'] %> $ cat roles/db.json omitted "base_setting": { "swappiness": "0", "tcp_tw_reuse": "1", #バックエンドのdbはtcp接続をリサイクルする "tcp_tw_ recycle": "1", #バックエンドのdbはtcp接続をリサイクルする "tcp_fin_timeout": "10", "tcp_max_syn_backlog": "8192", "somaxconn": "8192", "ntpserver1": "ntp.nict.jp" } omitted $ cat roles/API.json omitted "base_setting": { "swappiness": "0", "tcp_tw_reuse": "0", #フロントエンドのAPIはtcp接続をリサイクルしない "tcp_tw_recycle": "0", #フロントエンドのAPIはtcp接続をリサイクルしない "tcp_fin_timeout": "10", "tcp_ max_syn_backlog": "8192", "somaxconn": "8192", "ntpserver1": "ntp.nict.jp" } abbreviation[/shell] *In terms of the meaning of changing for each role, the source IP of the wifi connection from the smartphone is the same.
If multiple terminals access at the same time, if you recycle the TCP connection, one packet will be
Since it is discarded, it must be recycled on the backend that has a connection from a different IP
It is to enable it and reduce the load.
In the case of MHA Manager, introduce RPM for Manager, otherwise do not introduce it, etc.
[shell]$ cat site-cookbooks/mysqld/recipes/mysql-mha.rb if node[:mysqld][:mha] == 'manager' %W{perl-Config-Tiny perl-Log-Dispatch perl-Parallel-ForkManager perl-Params-Validate perl-Time-HiRes}.each do |pkg| package pkg do not_if "rpm -qa|grep #{pkg}" action :install end end cookbook_file "/usr/local/src/#{mhamanager}" do not_if "ls /usr/local/src/#{mhamanager}" source "usr/ local/src/#{mhamanager}" end omitted end $ cat roles/mhamanager.json omitted "default_attributes":{ "mysqld": { "mha":"manager" }, omitted[/shell]
- Call a different attribute for each node with if to execute the resource
This includes switching the contents of the configuration file and whether to register the cron.
The slave that is supposed to switch to the DB master does not enable read_only in the configuration file,
The second slave that does not switch enables read_only
[shell]$ cat site-cookbooks/mysqld/recipes/mysqld-server.rb omitted if node[:mysqld][:mha_nomaster] == 'true' template '/etc/my.cnf' do owner 'root' group 'root' source 'etc/ my.cnf_ro.erb' notifies :restart, 'service[mysql]' end else template '/etc/my.cnf' do owner 'root' group 'root' source 'etc/my.cnf.erb' notifies :restart, 'service[mysql]' end end omitted $ cat nodes/db3.hoge.jp.json { "mysqld" : { "master" : "false", "only_mysql" : "true", "only_innodb":"true", "mha_nomaster" : "true" }, omitted[/shell]
cron resource to register and run only in slave
[shell]omitted if node[:mysqld][:master] == 'false' cron "mysql_replication_check" do not_if "crontab -l|grep rep_fail_mail.sh" minute "0-59/15" hour "*" day "*" month "*" weekday "*" command "/opt/bin/rep_fail_mail.sh > /dev/null 2>&1" action :create end end omitted $ cat nodes/db1.hoge.jp.json { "mysqld" : { "master" : "true", omitted $ cat nodes/ db2.hoge.jp.json { "mysqld" : { "master" : "false", omitted[/shell]
- Execute resources by calling a different configuration file with if for each enviroment.
It is used to distribute the nginx configuration file of the API. (I'm changing the directory path of source) [shell]$ cat site-cookbooks/nginx/recipes/nginx_virtual.rb if node[:environment] == 'huka' template "/etc/nginx/conf.d/virtual.conf" do source "etc/ nginx/conf.d/huka/virtual.conf.erb" owner "root" group "root" mode 0644 notifies :reload, 'service[nginx]' end elsif node[:environment] == 'prd' template "/etc/nginx/conf.d/ virtual.conf" do source "etc/nginx/conf.d/prd/virtual.conf.erb" owner "root" group "root" mode 0644 notifies :reload, 'service[nginx]' end elsif node[:environment] == 'stg' template "/etc/nginx/conf.d/virtual.conf" do source "etc/nginx/conf.d/stg/virtual.conf.erb" owner "root" group "root" mode 0644 notifies :reload, 'service[nginx]' end end[/shell] It seems that it is common to change the value of attributes such as URLs in the contents with the same configuration file, but this is because I don't know the parameters in advance.
Reference: Chef Solo's Environments - naoya's Hatena Diary
Load version data from data_bags and define the package name
I use it most of my cookbooks.
[shell]$ cat site-cookbooks/nginx/recipes/nginx.rb version = data_bag_item('pkg_versions','nginx')['version'] nginx_pkg="nginx-#{version}.ngx.x86_64.rpm" omitted
$ knife solo data bag create pkg_versions nginx $ knife solo data bag edit pkg_versions nginx $ knife solo data bag show pkg_versions nginx id: nginx version: 1.4.4-1. el6[/shell]
・Load user data from data_bags and create a user
login_users cookbooks create_user recipes [shell]$ cat site-cookbooks/login-users/recipes/create_user.rb data_ids = data_bag('users') data_ids.each do |id| u = data_bag_item('users', id) name = u['username'] user u['username'] do not_if "id #{name}" home u['home'] shell u['shell'] uid u['uid'] gid u['gid'] password u[' password'] end end[/shell]
A recipe used by loading and using data encrypted with an encryption key in the mysqld-server recipe in the mysqld cookbook.
[shell]$ cat site-cookbooks/mysqld/recipes/mysqld-server.rb omitted ## load data_bag data. root = Chef::EncryptedDataBagItem.load("mysqlusers","root") repl = Chef::EncryptedDataBagItem.load("mysqlusers","repl") suuser = root["user"] supass = root["pass"] repluser = repl["user"] replpass = repl["pass"] ### secure installation,etc. mysqlconn = "/usr/bin/mysql -u root" if #{version} == '5.6' package 'expect' do :install not_if "rpm -qa|grep expect" end template "/tmp/setpass.sh" do only_if 'ls /root/.mysql_ secret' source "setpass.sh" end bash "set_password" do only_if "ls /root/.mysql_secret" code <<-EOC chmod +x /tmp/setpass.sh && /tmp/setpass.sh && rm -f /tmp/setpass.sh EOC end end bash "secure_installation" do ignore_failure true only_if "#{mysqlconn} -e 'show databases;'" code <<-EOC #{mysqlconn} << EOF grant all on *.* to #{suuser}@'%' identified by "#{supass}"; grant all on *.* to #{suuser}@'localhost' identified by "#{supass}"; grant all on *.* to #{suuser}@'::1' identified by "#{supass}"; grant all on *.* to #{suuser}@'127.0.0.1' identified by "#{supass}"; grant replication slave,replication client on *.* to #{repluser}@'%' identified by "#{replpass}"; drop database test; delete from mysql.user where password=''; flush privileges; EOF EOC end pfile="/root/.my.cnf" bash "create-pfile" do not_if "ls #{pfile}" code <<-EOC echo '[mysqladmin]' >> #{pfile} echo "user= #{suuser}" >> #{ pfile} echo "password = #{supass}" >> #{pfile} echo "" >> #{pfile} echo '[mysql]' >> #{pfile} echo "user= #{suuser}" >> #{pfile} echo " password = #{supass}" >> #{pfile} chmod 400 #{pfile} echo "#{supass}" >> /root/.mysql_pwd chmod 400 /root/.mysql_pwd EOC end omitted[/shell] 5. Reset the initial password of the 6th system with expect in bash, connect to mysql, and grant
Unnecessary items are deleted and password files are generated by other scripts.
I used mysql to create with chef - Qiita as a reference.
*For the encryption key, you need to generate data_bag_key and set data_bag_path in .chef/knife.rb directly under the chef repository.
Generate data_bag_key
[shell]$ openssl rand -base64 512 |tr -d '\r\n' > ~/.chef/encrypted_data_bag_secret $ chmod 400 ~/.chef/encrypted_data_bag_secret $ vi .chef/knife.rb[/shell]
- Load array data from data_bags and reflect it in multiple rows
This is loading data_setting data from the template resource of the NTPD recipe in the base_bags cookbook. [shell]$ cat site-cookbooks/base_setting/recipes/ntpd.rb ntpservers = data_bag_item('base_setting','ntpserver')['ntpservers'] template "/etc/ntp.conf" do path "/etc/ ntp.conf" source "etc/ntp.conf.erb" owner "root" group "root" mode 0644 notifies :restart, 'service[ntpd]' variables({ :ntpservers => ntpservers }) end
$ cat site-cookbooks/base_setting/templates/default/etc/ntp.conf.erb driftfile /var/lib/ntp/drift #server -4 <%= node['base_setting']['ntpserver1'] %> iburst <% for ntpserver in @ntpservers %> server -4 <%= ntpserver["server"] %> iburst <% end %>
Definition of data_bags $ knife solo data bag create base_setting ntpserver $ knife solo data bag edit base_setting ntpserver { "id": "ntpserver", "ntpservers": [ { "server" : "ntp.nict.jp" }, { " server" : "ntp.nict.jp" }, { "server" : "ntp.nict.jp" } ] } [/shell] The reason why all three are ntp.nict.jp is because dig is suitable for multiple servers in DNS round-robin.
For more information on how to retrieve array data from data_bags, please see the following URL for an easy-to-understand explanation.
The whole story of using databags (1)
Also, the following may be helpful for customization.
Customizing Your Environment with Chef Recipes : Developer Center
I'm sorry if you feel that the important point is a pointer.
Thank you for reading the above for a long time.