Translated from 日本語 • View original Hello. This is Komiya.
How it happened
If you are in a hurry, you can skip it. It was about half a year ago that Chef-Solo was deprecated (deprecated) and was known by the developer to use chef-zero (localmode). At that time, I tried to compare various things to see if I could do anything, but after a while, I finally decided to verify it. It seems that there are cases where people do not want to leave technical debt even if they cost money, if it is assumed that it will last for a long time rather than being replaced in a few years. Personally, to be honest, it's a delicate matter whether or not to migrate at a cost, and I don't think it's a story that SOLO will disappear anytime soon. In the world, cookpad has come out with itamae, Ansible is popular, etc. I heard that Ansible seems to be fatal because if you try to have a function equivalent to not_if, you won't be able to run dry, but I haven't tried it because there is no cost to rewrite it in YAML in the first place. I was also curious about itamae, but it seemed that there would be a hassle in reconciling the role and environment to attributes. The number of roles and environments is deliberately deepmerged and initialized with priority for array attributes on a decent scale, and I feel that the modification is quite heavy. Also, the lack of a data_bags function is also very troublesome. It seems that I was stuck in the chef's sweep to move to other places. Also, the community version of Chef-Server sometimes had no HA function.
Realistically, it would be localmode, so I tried to verify it back and forth as usual, so it's a memorandum.
There is a lot of information about chef, but because there are only Mohawk people (people with too high a technical level), it is difficult to understand the important point because beginners can't imagine where they are confused, and in the end, they often look at the help carefully or google it and try it out. There are many ways to use it, and each environment is different, so if you mix it in various ways, it may be easy to go crazy and get a little confused.
I will add it later because I saw a nice chart that is easy to understand. Which Chef is right for you? 〜 Recommended configuration confirmation chart #getchef - Creationline Co., Ltd.
I think that ease of understanding is very important for dissemination and development. I hope that an article like this will contribute to it a little.
References
Well, I've looked at a lot of things, so let's introduce them from the reference information. I will separate the patterns. I think it's easier to understand if you google the mechanism and functions of Chef-Zero separately, but In-memory Chef-Server and local modeThere is a difference, and what I want to do is the local mode. If you want to use it as an in-memory Chef-Server, you need to start Chef-Zero and register the data of cookbook, role, environment, and data_bags with the knife command. Since it is in-memory, the registered data will disappear when you stop Chef-Zero. So considering the time, I thought it would be used for primary use or when I want to see information that can be obtained by chef-shell in anticipation of migrating to Chef-Server. It may be convenient to say that it is convenient to see the registered JSON information when accessed from the browser, but I think it was at the level of Git. When using it as a local mode, Chef-Zero starts internally only when the command is executed, and it seems to fall when the recipe is applied, and it seems that there is no extra hassle because it looks at the Chef-repo path written in knife.rb without any extra effort.
・Type that wants to run Chef-Zero as an in-memory Chef-Server I tried running a simple Chef Server (chef-zero) on Amazon Linux | Developers.IO Installing chef-zero on CentOS - clavier's diary The whole story of building chef-zero and working with knife-xenserver - Yohei no Riri-shin XP Ruby - knife zero bootstrap can't install chef remotely - Qiita Let's use the lightweight and simple chef server "chef-zero" #opschef_ja « CREATIONLINE, INC. #Docker Use Chef Zero as a lightweight in-memory Chef Server to manage Docker containers from the host OS with Chef #Chef #GetChef_ja - Qiita
・Type that wants to run Chef-Zero in local mode Chef-Zero & Local Mode I created Knife-Zero to try to operate a remote server only in Chef's local mode. You can also get Node configuration information. - Chef Super Introduction for Qiita Server Administrators (2): Create/Run/Delete Cookbooks with Knife-Zero > Get Cookbooks with git clone commands (1/2) - @IT chef_localmode_tutorial/knife.rb at master · higanworks/ chef_localmode_tutorial Configuring Tool Let's Use Chef - ConoHa VPS (Virtual Private Server) Chef Super Introduction for Server Administrators (2) - Supplement to @IT, Cookbook Directory - Qiita Steps to Install Chef on Amazon EC2 Using Knife-Zero bootstrap - Qiita Migrating from Chef-Solo to Chef-Client Local Mode #opschef_ja #getchef_ja « CREATIONLINE, INC. Oreno Knife-Zero Notes - Orenoblog
・Other Chef Practical Techniques for Implementing DevOps Get Started with Chef # 1 – Overview, Installing Chef Server on EC2, and Configuring Workstation | Developers.IO #GetChef_ja - Twitter Search I created a plugin called "knife-helper" because the knife command was too cumbersome #getchef_ja - Miscellaneous notes I created a plugin called "knife-helper" because the knife command was too cumbersome, so I created a tutorial (with knife-zero, knife-ec2) - Qiita knife.rb — Tried Chef Docs GNU parallel | Ore no homepage
Also, take a closer look at the "chef usage guide". It seems like a dictionary without a tutorial, but recently I finally thought I was lucky♪ to be able to read man in Japanese.
Installing Chef-Zero, Knife-Zero
The OS is CentOS 6.5. For the time being, I felt like OHAI would get angry if the Ruby version was low, so I put 2.1.5 in RBENV and made it global. (However, there is a little DL warning.) There is also a chef-dk, but I was worried about the path to enter, so I put it in with a gem for the time being.
bundler is more dependent on you, so don't do the following: $ sudo -i gem install chef-zero json --no-ri --no-rdoc $ sudo -i gem install knife-zero --no-ri --no-rdoc
Root on the switch user before bundle
$ sudo su -
# cd /path_to_chef-repo/
# vi Gemfile
source 'https://rubygems.org'
gem 'berkshelf', '~>3.0.0'
# berkshelf require hashie/hash_extensions file.
gem 'hashie', '>=2.0.2', '<3.0.0' gem 'chef' #追加 gem 'chef-zero' #追加 gem 'knife-zero' #追加 group :test do gem 'foodcritic', '~>3.0.0'
gem 'test-kitchen', '~>1.3.0'
gem 'mixlib-shellout', '~>1.3.0'
gem 'kitchen-ec2', '~>0.8.0'
gem 'serverspec'
end
# bundle install
Successfully installed net-ssh-2.9.3.beta1
Successfully installed net-scp-1.2.1
~Omitted~
Successfully installed mixlib-authentication-1.3.0
Successfully installed chef-12.2.1 *Chef12 is included
Successfully installed knife-zero-1.4.0
34 gems installed
List
# bundle show
chef-client can also be used as follows: sudo curl -L https://www.opscode.com/chef/install.sh | bash -x When I do this, Workstation tries to put chef-12.2.1-1.el6.x86_64.rpm. (As of April 2015) If you put knife-zero in, chef-12 will enter on its own, and for the time being, it seems that it will still move. However, if you put it on the remote side with knife zero bootstrap, chef (11.12.4) and chef-zero (2.0.2) will be included. Recipes can be applied.
help can list what appears to be '$ knife help list'. If you say 'knife zero bootstrap --help', you should also see help for knife zero. If you don't see it, it's probably not going well.
Applying Recipes to Localhost in Chef-Client's Local Mode
If you want to use an IAM role and a cloud-init to apply a recipe to the localhost the moment you start from an AMI, you probably don't need to set up chef-zero, and it would be nice to git clone the Chef-Repo, get the key, pass the parameters, and execute the following.
$ sudo chef-client -z -r role[web] -E sandbox --validation_key /etc/chef/validation.pem
When I put -z, chef-zero seemed to run only at that time, and if chef-zero was already started and the ports were in conflict, the error would end. The details of the options appear to be in the command --help. option is written in client.rb, so you don't have to specify it. After dropping the chef repository with git clone or something, it seems OK to do something like this. If you don't add sudo, the error ended when the permission was required. Although it is necessary to set knife.rb, client.rb, and varidation-key. (I didn't need to renovate the chef repository!) yes！！ In fact, even if chef-zero is not included, chef-client seems to be able to run with a local mode option and apply recipes. (In INFO, the chef-zero server appears as NotFound, and internally, it only starts when the command is executed, and there is no such thing as it crashes after it is finished, but the recipe flows.)
chef-repo/.chef/knife.rb looks like this: If you put localmode true, it seemed to work in local mode without -z.
chef_repo = File.join(File.dirname(__FILE__), "..")
current_dir = File.dirname(__FILE__)
log_level :info
log_location STDOUT
#node_name `uname -n`
client_key "#{current_dir}/dummy.pem"
validation_client_name 'chef-validator'
validation_key "#{current_dir}/dummy.pem"
#validation_key "/etc/chef/validation.pem"
cookbook_path ["#{chef_repo}/cookbooks", "#{chef_repo}/site-cookbooks"]
node_path "#{chef_repo}/nodes"
role_path "#{chef_repo}/roles"
environment_path "#{chef_repo}/environments"
data_bag_path "#{chef_repo}/data_bags"
encrypted_data_bag_secret "path_to_databagkey"
knife[:berkshelf_path] = "#{chef_repo}/cookbooks"
#knife[:ssh_user] = ""
#knife[:ssh_password] = ""
knife[:ssh_port] = 22
knife[:editor] = "vim"
localmode true
chef_server_url 'http://127.0.0.1:8889'
syntax_check_cache_path "#{current_dir}/syntax_check_cache"
#puts Chef::Config.inspect
I have come to understand that the chef configuration file is ruby-like. (I just gave it to node_name with a backquote, and knife zero bootstrap works, but chef-client seemed to stop working. I prepared the key in the following steps. chef-zero seems to require authentication, but if it doesn't seem to require a formal key, the knife cannot be executed and an error occurs.
$ ssh-keygen -t rsa -N "" -f ~/chef-repo/.chef/dummy.pem
/etc/chef/client.rb looks like this: The parameters that can be written are listed in the "Chef Usage Guide".
log_level :info
log_location STDOUT
chef_server_url "https://127.0.0.1:8889" ## Chef-Zero's API endpoint
#validation_client_name 'chef-validator'
client_fork true
client_registration_retries 5
#node_name `uname -n`
#puts Chef::Config.inspect
If you want to apply a recipe from Workstation to a remote client
After twists and turns, it worked out with KniFe Zero as follows.
url=https://www.opscode.com/chef/install.sh
newip=xxx.xxx.xxx.xxx
role=web
env=sandbox
user=hoge
dpas=`cat /pashto_sshpassword_file`
$ knife zero bootstrap --node-name $newip --bootstrap-url $url -i .chef/dummy.pem -x $user --sudo \
-E $env -r role\[rolename\] -z -s $(cat ./databag_key) -P $dpas $newip
bootstrap, but it was done with one command to apply the recipe. When I logged in, /etc/chef/client.rb was automatically created and the key for data_bags was copied and placed. -S was not needed after the next time. (There is no repository anywhere, and I thought that if rsync was loaded, one of the processes running multiple on the workstation would fall.) I think I saw an article that verified that Sawanobori could go up to 2000 nodes, but I remember thinking that the machine specs were not made. ）
$ ls /etc/chef
client.pem client.rb encrypted_data_bag_secret first-boot.json validation.pem
$ sudo cat /etc/chef/client.rb
log_location STDOUT
chef_server_url "http://localhost:8889"
validation_client_name "chef-validator"
node_name "10.xxx.xxx.xxx"
So I was happy. I want to authenticate the key for a while, but I don't understand that it becomes a password authentication! I think, but I haven't done anything about it yet. It may be quite easy to get stuck. I wonder if it's because I'm not used to port forwarding. I don't really know. I can write in plain text on knife.rb, so I guess it's like being careful not to commit to it. #knife[:ssh_user] = "" #knife[:ssh_password] = ""
There are two more subcommands for knife zero, client and diagnose. The client seems to be used to apply the recipe after the second time, and the diagnosis seems to go to see the settings.
・client worked when I did the following. -o is an example of applying a recipe partially.
$ knife zero chef_client -c ~/chef-repo/.chef/knife.rb 'name:10.xxx.xxx.xxx' \
-x $user -i .chef/dummy.pem --sudo -E $env -o base::openssl -a ipaddress 10.xxx.xxx.xxx --remote-chef-zero-port 8889 -z
・Diagnosis is like a list of currently recognized values as follows. If you check when the command doesn't work, you can see if there are any missing settings or options.
$ knife zero diagnose -c ~/Chef-Repo/.chef/knife.rb -z
Chef::Config
====================
---
:local_mode: true
:knife_zero: true
:verbosity:
:config_file: "/path-to-config/.chef/knife.rb"
:log_level: :error
:log_location: !ruby/object:IO {}
:node_name: |
~Omitted~
"Knife Zero Bootstrap ≒ Knife Solo Prepare+Cook", "Knife Zero Chef_client ≒ Knife SSH+Search+Cook", etc. I've been holding on to Knife Zero so far, but it seems that Knife Solo also plans to support local mode. (If you don't want to manage the node configuration because the node configuration changes quite dynamically in the BlueGreen deployment-like swapping operation, knife zero seems to be too much for you.) ） Chef-Solo, Chef-Client LocalMode, Knife-Solo, Knife-Zero and us. — sawanoboly.net add local_mode to solo.rb · Issue #353 · matschaffer/knife-solo (transfer the entire related directory, remotely change it to a LocalMode equivalent executable format, etc.)
If knife-zero is troublesome, I think it would be okay to do the same processing as knife-solo yourself in local mode. In other words, I think it would be a good idea to transfer the chef-repo and hit the chef-client over the ssh. (If you want to sudo, you can add -t to ssh)
About Parallel Processing
I didn't feel parallelism, probably because the CPU was one core, but I was able to move it by using the following GNU command called parallel. The standard output felt like it came one by one after completion. The output was not mixed for each host.
$ parallel -j 3 knife zero bootstrap -c ~/Chef-Repo/.chef/knife.rb -z -E $env -r role\[role-name\] \
-i .chef/dummy.pem -s $(cat ~/path_to_databag-key) --sudo --no-host-key-verify ::: $host1 $host2
Parallel's man seemed to be able to eat from the file. It may be easier than looking at knife help and being bothered by errors. However, there is no rpm, so you have to make or make your own rpm and put it in.
I tried using the Search function
・Example of searching for data_bags
$ knife show data_bags/base/* --local-mode
$ knife search nginx 'id:nginx_status' --local-mode
$ knife search nginx 'servername:"proxy01"' --local-mode
$ knife search nginx 'servername:"*.net"' --local-mode --id-only
$ knife search nginx 'servername:*.net AND certificatename:xxx-wildcard' --local-mode
If you add '--id-only', it will only be id. In addition to AND, it seems that OR and NOT can also be used. It seems that the next special character has to be backslashed (from around Chef Usage Guide P242) '+ - && || ! () {} [] ^ " ~ * ? : `
I would be happy if I could run the results of the search in parallel, but I thought I would have to manage the node properly.
Well, it's a little long, so I'll excuse you here. Thank you for watching.