Hi, this is AINOYA(@ainoya) from AP Solution Group. I developed a simple tool named Walter with ATL which helps to execute Build Pipeline in CI and CD.
My motivation of the development: the Build Pipe lines setting depended heavily on the Jenkins plugin.
If we perform CI with Jenkins, we make a set of Build Pipelines connected with several jobs in general. Jenkins has a plugin convenient for forming Build Pipelines, which operate executive control in fail and concurrency control easily.
When we operate the CI with those plugins in real, however, some points almost came out perfectly;
Although, with Docker and Ansible, we can manage the server environment as we aggregate source code, we have to depend on Jenkins when we build the setting with Jenkins.
Overview of Walter
The tool Walter allows you to do the following things;
The Walter activity is simple, and it executes jobs in order along with executive rule which you write in your original settings file. The way of writing is similar to the one in Travis-ci setting file (Travis.yml), but in Walter you can easily write concurrency executive processing.
In Walter, you write the executive order and processing context of Build Pipeline under Pipeline:, and you write the details of each job in each property defined in Stage.
The Stage where you write the setting on each job has the following properties;
Some sample setting files (pipeline.yml) of Walter are demonstrated below.
# step 1
- stage_name: ビルドパイプラインの処理開始をHipChatに通知
command: echo "build pipeline is started"
# step 2
- stage_name: ansibleでサーバのプロビジョニング
command: ansible-playbook -i hosts site.yml
- stage_name: アプリケーションのテスト実行(rspec)
command: bundle exec rake test
# step 3
- stage_name: Capistranoでアプリケーションをサーバにデプロイ
command: bundle exec cap production deploy
# step 4
- stage_name: Serverspecでサーバ状態をテスト
command: bundle exec rake spec
# step 5
- stage_name: ビルドパイプラインの処理終了をHipChatに通知
command: echo "build pipeline is finished"
We can illustrate the steps of the Build Pipeline defined in Pipeline.yml above. As you see in the illustration, the Stage defined in the top level of Pipeline is executed in order after it waits for each job to finish. On the other hand, the job written in run_after is executed in parallel.
We give you other examples such as providing an application with building a Docker cantina. Please refer to them.
You can execute Walter by downloading the Walter-built binary files distributed in GitHub. Let’s execute Walter in the mac environment.
- Obtain the binary files which is built specifically for Mac.
- Obtain the Sample Build Pipeline setting file.
- Execute Walter and select the Build Pipeline defined file for execution in –c option. If you do not make any selection, the default value is ./pipeline.yml.
./walter -c pipeine_sample.yml
Was the stage executed in the order you selected in Pipeline? Make sure that the behavior has changed if you changed the Stage writing and executing order. If you find any bugs, please report to issues.
Microservices with walter
In some architecture composed of several services/servers, there are some restrictions in the deployed order if there is a relationship depending on each service.
It often happens in the case that the monolithic service becomes separated and to be operated as several services. In this case too, when you apply the way to use Walter explained above, you can easily write the Build Pipeline of the architecture composed of several services/servers. You can also apply to the cases where you deploy the two kinds of services which should be ready first and then you can deploy the rest of the 4 services which depend on the first two.
Wait_for writing and parallel executing option makes it possible to provision several servers with Ansible only. I feel personally it is easier to be handled if you execute Ansible with appropriate granularity. I aim to have a rich visibility of the whole composition through appropriately dividing the executing units provisioned by Ansible, joining them by Walter, and getting control of them.
Current and future development of the schedule
I wrote the Walter tool so that it solved the inconveniences to the ways of composing the Build Pipeline as-is. I am going to make its quality better mainly of the error processing and the retry mechanism, although the code quality is still a prototype. I will give a feedback through the improvements on production in company, of course. To make it more convenient, I will approach the variety of functionalities as follows;
Please leave me a comment about how you feel when using the code in GitHub opened to the public.