Translated from 日本語 • View original Hello. This is Komiya. I need to explain data_bags internally, so I will write it here as well.
The following is how to use data_bags these days.
１． Load OS user information for system applications that are not managed by LDAP and add them in recipes 2. Store SSL certificates, load them from recipes, and set them in templates 3. Load ntp server information and set them in templates 4. Manage AWS credential information and embed it in various scripts 5. Manage account information such as passwords such as MySQL and LDAP 6. Management of information such as vhost and upstream in nginx and apache that does not fit into roles and environments 7. Management of information in Ganglia's head_node and cluster_name
0. Preliminary explanation
*As a prerequisite, in the case of chef-solo, you need a gem called "knife-solo_data_bag", so you can add it. (If it's not a gem list, put it in gem install or bundler)
How to store and check
If you do not want to encrypt it, edit it in any editor, such as vi, and create OK chef-repository/data_bags/arbitrary directory name/arbitrary filename.json The contents need to follow the rules of json, so CHK the syntax with json_verify.
cat data_bags/dirname/filename.json |json_verify and see JSON is valid
When encrypting, you first need to set the environment variable of the EDITOR to ~/.bashrc, and you need to create and set a key for encryption. (*The key is a different image for each project and needs to be written in .gitignore.) If you strictly prohibit leakage and remove it, the data will not be visible. ）
export EDITOR=vi
vi .chef/knife.rb
------ uncommented---------------------
encrypted_data_bag_secret "data_bag_key"
-----------------------------------------
*If you uncomment, even if you do not specify the key, it will always be encrypted if you create with the knife command, so you need to create the data file you want to handle without encryption in vi or comment in before creating it.
If you need a key, make it (*The relative path of .chef/knife.rb means directly below the chef repository)
openssl rand -base64 512 > data_bag_key
knife solo data bag create dirname filename # Create (*existing ones will be overwritten)
knife solo data bag edit dirname filename # Edit files in the editor
Knife Solo Data Bag show dirname filename # JSON contents
Knife Solo Data Bag List # directory list
・Things to be careful about→ bash and execute resources should not be used as much as possible. template resources. When you sed with bash, the special characters you put in the variable from data_bags are expanded and unintended values are included.
１． Load OS user information for system applications that are not managed by LDAP and add them as recipes
This is not a big deal, so excuse me with the pointer Write a recipe for an existing procedure in Chef 2 (user created) | DEVLAB patterns that are not encrypted. (Password is a pre-encrypted string) Individual users are managed by LDAP, and chef handles LDAP client configuration file distribution, etc.
２． Load from SSL certificate storage and recipe and set in template
I am doing it with Apache and NGINX, but I will describe the case of NGINX.
As for how to store SSL certificates, paste the line break code converted to a string as follows. Since it is a certificate and a key, of course it is encrypted and loaded.
cat cert.pem|perl -pe 's/n/n/g'
knife solo data bag create ssl jenkins
knife solo data bag edit ssl jenkins
knife solo data bag show ssl jenkins
{
"id": "jenkins",
"crt": "-----BEGIN CERTIFICATE-----nMIIDd~omitted~",
"key": "~omitted~",
"passkey": "~omitted~",
"passphrase": "hogehoge"
}
How to load on the recipe side (some excerpts)
# cat site-cookbooks/nginx/recipes/nginx_ssl_client.rb
---
# I put the data_type (≒role) name and environment name in the variables to make it the name of the data file
#role = ("#{node.roles}"[/[w_-]+/])
environment = ("#{node.environment}"[/w+/])
data_type = node['nginx']['data_type']
sslclientconf_type = node['nginx']['sslclientconf_type']
datafile = "#{data_type}_#{environment}"
# Define the name of the key, etc. as a variable (this is loaded from attributes such as role)
sslkeyfile = node['nginx']['sslkey1']
sslcrtfile = node['nginx']['sslcrt1']
clientcrtfile = node['nginx']['clientcrt']
clientcrlfile = node['nginx']['clientcrl']
sslkeydir = '/etc/nginx'
# Loading data_bags encrypted data and defining it as a variable
ssl = Chef::EncryptedDataBagItem.load("ssl", "#{datafile}")
sslcrtdata = ssl["crt"]
sslkeydata = ssl["key"]
clientcrtdata = ssl["clientcrt"]
#clientcrldata = ssl["clientcrl"]
# This is a recipe that branches off with data_type an if.
if "#{data_type}" == ""
#~Omitted~
else
# You can maintain idempotency by using template resources
template "#{sslkeydir}/#{sslcrtfile}" do
notifies :restart, 'service[nginx]'
source "sslcrtfile.erb"
variables({
:sslcrtdata => sslcrtdata
})
end
template "#{sslkeydir}/#{sslkeyfile}" do
notifies :restart, 'service[nginx]'
source "sslkeyfile.erb"
variables({
:sslkeydata => sslkeydata
})
end
template "#{sslkeydir}/#{clientcrtfile}" do
notifies :restart, 'service[nginx]'
source "clientcrtfile.erb"
variables({
:clientcrtdata => clientcrtdata
})
end
---
What happens to the template file is
# cat site-cookbooks/nginx/templates/default/sslcrtfile.erb
<%= @sslcrtdata %>
# cat site-cookbooks/nginx/templates/default/sslkeyfile.erb
<%= @sslkeydata %>
# cat site-cookbooks/nginx/templates/default/clientcrtfile.erb
<%= @clientcrtdata %>
You can load from data_bags where variables is @ and set the value in the variable.
3. Loading information on the NTP server and setting it in the template
5. Manage account information such as passwords such as mysql and ldap
Excuse me for using a pointer too. Chef-Solo's Recipe Customization Record | DEVLAB for your reference.
4. Manage AWS credentials and embed them in various scripts
Of course, I will encrypt it. In the case of SSL certificates, the loading method and embedding method are not much different, so I will omit it.
Create, edit, and check as follows.
# knife solo data bag create awskeys deploy
# knife solo data bag edit awskeys deploy
# knife solo data bag show awskeys deploy
{
"id": "deploy",
"aws_access_key_id": "Axxxxxxxxxxxxxxxxxx",
"aws_secret_access_key": "oxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"region": "us-xxxx-2"
}
For example, the method of loading it with this is as follows. (I pass variables to the script without installing it)
cookbook_file "/usr/local/sbin/renamehost" do
source "renamehost"
owner 'root'
group 'root'
mode 0750
end
ohai "reload" do
plugin "hostname"
end
# Get the corresponding AWS account from your environment
env_rank = data_bag_item("master_data", "env_rank")
rank_awsaccount = data_bag_item("master_data", "rank_awsaccount")
environment = node.chef_environment
rank = env_rank[environment]
awsaccount = rank_awsaccount[rank]
# Get the corresponding credentials for your account
awscredential = Chef::EncryptedDataBagItem.load("awskeys", "deploy_#{awsaccount}")
aws_access_key_id = awscredential["aws_access_key_id"]
aws_secret_access_key = awscredential["aws_secret_access_key"]
aws_region = awscredential["region"]
role = ("#{node.roles}"[/[\w_-]+/])
bash "exec-renamehost" do
not_if { node[:hostname] =~ /^#{role}/ }
notifies :reload, "ohai[reload]", :immediately
notifies :restart, "service[rsyslog]", :immediately
Code <-EOC
/usr/local/sbin/renamehost `echo #{aws_access_key_id} #{aws_secret_access_key} #{aws_region}`
EOC
end
include_recipe "base_setting::hosts"
6. Manage information such as vhost and upstream in nginx and apache that does not fit into roles and environments
The recipe is as follows
ngxdata = data_bag_item('nginx',"#{datafile}")
servername = ngxdata['servername1']
upstream = ngxdata['upstream1']
upstreamport = ngxdata['upstreamport']
template "/etc/nginx/conf.d/ssl.conf" do
source "#{role}/ssl.conf.erb"
path '/etc/nginx/conf.d/ssl.conf'
backup 5
owner 'root'
group 'root'
mode '0644'
action :create
notifies :restart, 'service[nginx]'
variables({
:servername1 => servername,
:upstream1 => upstream,
:upstreamport => upstreamport
})
end
The json file is not particularly encrypted (it's hard to see in git)
# cat data_bags/nginx/nginx-hoge_test1.json
----
{
"id": "nginx-hoge_test1",
"servername1": "exsample.hoge.net",
"upstream1": "internal-ELB-int-hoge-test1-xxxxxxx.us-xxxx-2.elb.amazonaws.com",
"upstreamport": "80"
}
----
**There are many roles and environments, and if you can't fit into one or the other, you have no choice but to put it in the data_bags.
7. Manage Ganglia Head_node and Cluster_name Information
This is not much different from 6, so I will omit it
Reference:
The whole story of using databags (1) - Yohei's daily devotion [chef] How to use data bag | IT Infrastructure Miscellaneous Notebooks A record of using DataBags to encrypt JSON data that you do not want to publish in Chef - Goodbye Internet For your reference. Thank you for reading. **