(Français) Le printemps amène le soleil et les bourgeons, et nous une nouvelle version de Rudder (2.10) !
Sorry, this entry is only available in Français.
Sorry, this entry is only available in Français.
Happy New Year!
At the foot of the Christmas tree was a new version of Rudder, 2.9 “Liberty ship”! (for the curious: http://en.wikipedia.org/wiki/Liberty_ship)
To begin 2014, our latest version, 2.9, further extends usability and adaptability of the previous version, 2.8. There are relatively few new features in this version though it will improve the way you use Rudder.
In “Liberty ship”, the main additions and new features are:
Last month Infoworld published an article titled “Review: Puppet vs. Chef vs. Ansible vs. Salt” written by Paul Venezia, that prompted many people in the configuration management community to get in touch with us to ask why CFEngine was not included. Our answer was simple: “No, we have no clear reason as to why this has occurred and it is bizarre not to have included CFEngine in the article.”
The article seeks to present the reader with a choice of configuration management technologies according to the following needs:
“In some cases, we may be talking about large installations that exist only to support very large applications or large installations that support myriad smaller services. In either case, the ability to wave a wand and cause them all to bend to the will of the admin cannot be discounted. It’s the only way to manage these large and growing infrastructures.”
Certainly, the scope indicated suggests CFEngine should be on the shortlist!
While the article is titled and written in the style of a review we would seek to remind readers that any review of such technology that dedicates 5 to 7 paragraphs per technology is hardly exhaustive. Indeed, we need a decent testbed to deploy all these tools to in order to back up assertions and claims with hard data for this to be considered a factual review. Instead, at best, it can be considered a primer to the options available.
We thought we’d try and help out here and share some content that could easily be inserted into the existing article at Infoworld should the editors feel inclined to provide a more complete picture to their readership…
CFEngine also started out as an open source project, in 1993. Both the technology and the author, Mark Burgess, receive recognition for this on-going contribution and have been called the Grand/Godfather of Configuration Management. The latest version of CFEngine continues to build on scientific research conducted by Mark Burgess. The concept of promise theory, first proposed by Mark in 2004, and the facility to exploit this has been available since version 3 of CFEngine, released in 2009.
Isn’t CFEngine just a by-product of scientifc research? Yes and no. Yes, CFEngine continues to receive plenty of interest from academia. However, in 2008, CFEngine AS was founded to support the ever-growing commercial interest as enterprise had not missed the opportunity to benefit from such learning. Today, JP Morgan, LinkedIn, IBM and many other organisations publicly acknowledge the value they get from using CFEngine for configuration management. CFEngine is by far the most widely used technology in production and is a core sysadmin skillset, even for today’s cloud architects!
You can find CFEngine deployed on many different platforms: AIX support came in 2005, we now use agents across Android-based deployments and of course Microsoft’s platform is supported as well, albeit only the enterprise version of CFEngine and that requires purchase of a license. Either way, if you are running a mixed environment then CFEngine is a very interesting proposition for you.
Not one single tool uses the same approach, all tools require the understanding and use of their specific vocabulary and syntax. CFEngine is no exception and with that there is the inevitable learning curve. Where some of the other tools (Puppet and Chef) have opted to leverage the Ruby ecosystem and require the dependencies and resources that go along with such a decision, CFEngine’s unequaled strength in this regard is its lightweight approach. Written in C, with a very small footprint and far fewer dependencies, CFEngine continues to execute significantly faster and with greater reliability than any other tool thus far.
CFEngine AS offers an enterprise version with a Web UI, for a fee. There is also Rudder, from Normation that is free (100% open source, free software, with enterprise-grade support available). In terms of what one can get done using a Web UI and as we provide an enterprise Web UI in the form of Rudder we’ll talk about what you can do with CFEngine using Rudder.
With Rudder you have a complete web interface with which to administer and interact with your configuration management technology (CFEngine).
You can configure and manage nodes and their configuration. You are able to automate common system administration tasks (installation, configuration), enforce configuration over time (configuring once is good, ensuring that configuration is valid and automatically fixing it is better), establish and maintain an inventory of all managed nodes, and report on the compliance of configurations (desired vs actual state) by node or group(s).
Rudder’s workflow combines templates known as ‘Techniques’, from which new instances are created as ‘Directives’. Together, with Nodes that are grouped together according to user-specified patterns, Configuration Rules are generated that are responsible for applying the Directives to the associated Group(s). All of this is done via the web interface. Separation of user roles is also possible allowing for the implementation of change control throughout each and every stage of interaction with Rudder.
“Is my infrastructure compliant?” Rudder reports on the state of all configuration items under management providing a clear signal as to whether you have converged on your desired configuration state. Rudder will show you when configurations have failed to apply and display pertinent information to help with further investigations to resolve these configuration incidents.
In terms of what comes out-of-the-box, Rudder is loaded with predefined ‘techniques’ that cover most common configuration management needs and are easily customized and extended via the GUI for specific and complex use cases. In the latest version, Rudder also offers the ability to manage isolated groups of nodes via Rudder relay servers.
So, we hope Infoworld will appreciate the support and update their article in light of these arguments. Feel free to tweet to Paul and Infoworld to request they correct the omission and include CFEngine – you know it makes sense!
We have just released the latest version of Rudder, v2.8 ‘Patrol Boat’. In this post we will share with you what has been fixed, improved and the new features we’ve added in this release.
This release cycle was shorter than usual, just under a month in fact, though it did not stop us working just as hard to make it another exciting release for our community. We continued to make improvements to Rudder’s user experience (Rudder application configuration is now possible via the web interface) and introduce more new features (relay servers)!
In keeping with our nautical theme for the 2.x series of Rudder, we named this version ‘Patrol Boat’ based on the improvements in workflow and security. The main changes and new features are:
In previous versions of Rudder, in order to modify an application setting you had to do two things: locate and modify the parameter in the configuration file /opt/rudder/etc/rudder-web.properties and then restart the web application server from the command line: /etc/init.d/jetty restart
In version 2.8, we have begun to make available certain settings of the configuration file to enable management of these settings directly from the web interface, without having to restart the web application. The configurable settings are:
Rudder now enables the deployment of relay servers, so what are relay servers?
A server relay is a form of proxy between a Rudder server and a group of nodes (these nodes could be on another subnet that is inacessible to the Rudder server). The Rudder relay server is a server-managed node, that relays the promises, generated by the Rudder server, to a group of sub-nodes and relays the reports generated from these nodes to the Rudder server.
The short and simple procedure for using Rudder with relay servers is documented here: http://www.rudder-project.org/rudder-doc-2.8/rudder-doc.html#relay-servers. To get started using relay servers just follow the instructions and you’ll be up and running in minutes!
In 2.7, we added an API to manage the majority of the elements in Rudder (Rules, Directives, Groups and Nodes). Yet even with workflow valiation activated there was no way in 2.7 to validate the changes via the API.
This functionality has now been added in version 2.8. Each Change Request can now be managed via the API. It is now possible to request a list of all change requests according to various filters, read the details of individual change requests and thereafter accept or refuse these requests including the ability to modify their information, such as in the case of automating bulk-reassignment of change requests for example.
With this latest addition of Rudder it is now possible to use Rudder headless, i.e. independently of the web interface. Deployment of changes can also be automated: for example deployment of changes can be launched every day at 23:00…
For more information and several examples please see our REST API documentation.
Rudder is now configured to use HTTP/S by default. Thus, all communication with the Rudder server is encrypted (including the REST API) ensuring security communications are enabled and active for all interactions with infrastructure configuration services under Rudder’s management.
We have made several changes in the way Directives are displayed in the web interface:
We are especially grateful to our community for the following two contributions:
All Rudder agents are now using CFEngine 3.5.2 (previously 3.4.4), allowing us to benefit from all the new functionality offered in the latest version of CFEngine (https://cfengine.com/docs/3.5/whats-new.html) as well as all the latest bug fixes!
With the addition of the latest version of CFEngine it was necessary to update our techniques library as CFEngine 3.5 is more strict about parsing of promises.
Before upgrading to our latest version, users must ensure they have already upgraded their server to one of the following versions of Rudder: 2.4.11, 2.6.8 and 2.7.5. The full upgrade procedure is documented here: http://www.rudder-project.org/rudder-doc-2.8/rudder-doc.html#_upgrade_rudder
In order to perform certain tasks (reporting, date of the last report, …) we run queries on the entire content of all reports in the database even though we are only interested in a subset of that data.
We now store the execution event logs of Rudder agents in a separate table, this allows us to stop losing time by performing queries on tables containing massive amounts of data and instead run queries in a more optimized fashion, bringing significant performance increases.
The change in performance for displaying a page that lists nodes and the various reporting pages are quite impressive, we’ve seen at least a 30% improvement already.
When migrating to version 2.8, an update script is launched. This script takes charge of the database modifications in order to migrate each node’s execution data to a separate table and build new indexes. The script is expected to take some time to run so don’t worry if that is the case when running the upgrade on your deployment of Rudder.
We have introduced several small changes which will make your use of Rudder a more pleasant experience.
That’s it for Rudder 2.8 Patrol boat! In order to upgrade to this version it is necessary to use Techniques compatible with CFEngine 3.5. To this end, you must have already upgraded your version of Rudder server to one of the latest versions in the following branches: 2.4, 2.6 and 2.7 (starting from 2.4.11, 2.6.8 et 2.7.5). The complete procedure is available and documented here.
As stated earlier, this release is not yet declared “stable“. In all cases, when we release a new version of Rudder it has been thoroughly tested, and we consider the release enterprise-ready for deployment. To be declared “stable” we prefer to wait until a version has been available and running in production for several months. As such, we expect version 2.7 of Rudder to be declared stable very soon.
We are deeply grateful for the continued support and enthusiasm of our Rudder community. We welcome your feedback, ideas and bug reports. We are listening to you! Please do not hesitate to contact us via IRC (#rudder on Freenode) or the Rudder community mailing lists (https://www.rudder-project.org/site/community/mailing-lists/), we will be delighted to welcome you and respond to any questions or feedback you might wish to share.
The changelog for Rudder v2.8 ‘Patrol Boat’ is here.
The documentation for Rudder v2.8 ‘Patrol Boat’ is available online (or directly via your Rudder server!)
During Devops Days Paris 2013 earlier this year (18/19 April 2013) I was surprised to discover that very few people had heard of Vagrant. Luckily; there was an open space dedicated to Vagrant (and its usage) where 15-20 people attended but only 5 among them knew about Vagrant (including my colleague, François Armand and myself). This tools deserves to be better known, so the aim of this post is to introduce you to Vagrant and share one of several ways we regularly use Vagrant at Normation.
We began using this tool extensively during July 2012. At that time Rudder 2.4.0~beta3 was about to be released. We needed to test installation as much as possible, thus we needed to start from a fresh, clean environment for each test. This required the creation of several VMs (a server and some nodes) followed by installing and configuring Rudder (server or agent) on each of them.. Doing this manually took a lot of time, was laborious and could be prone to error. All the time spent on installation was wasted every time we destroyed those VMs after a test. Now all of this is automated with Vagrant and we no longer lose time configuring virtual machines.
Vagrant is an open source virtual environment building tool created in January 2010 by Mitchell Hashimoto, and developed on his spare time. As Vagrant became more and lore popular worldwide, in November 2012 Mitchell founded Hashicorp, a company dedicated to Vagrant. With 3 years of development, the huge community and the support of Hashicorp, Vagrant has become a proven and reliable software.
Vagrant’s approach is very simple:
Once you have configured your template (Vagrantfile) you can then re-use it and thus automate the entire installation process!
Vagrant uses a single configuration file: the Vagrantfile. In this file you describe how your virtual machines will be created and configured. Everything you need to know will be centralized in that file. How to configure a Vagrantfile is clearly documented on the website http://docs.vagrantup.com/v2/.
Vagrant will use three things to create the VM:
So there are a huge variety of possibilities and it can be adapted to all of your needs and abilities.
Vagrant is very simple to install and use, you just need to choose your provider (virtualbox is the easiest one, other providers are also available).
Creating your first VM is very easy, just run the following in a dedicated directory:
$ vagrant box add base http://files.vagrantup.com/lucid32.box $ vagrant init $ vagrant up
Vagrant offers you several commands to manage your environment :
… and a lot more!
So, to use vagrant the only things you need to know are how to declare a VM (very easy and straight forward with the Vagrantfile), and how to configure it – depending on your needs!
Vagrant gives you automation and I do think that automation is one of the best things ever invented. Automation helps you get back the time spent doing things manually so that you can make even better use of your time on other tasks. With Vagrant you do the preparation only once, then (almost, if we take into account Murphy’s Law) everything works as expected. At Normation, we use it in three areas: our development environment, testing environment and embedded demo. I’ll explain in details how we manage to do this with vagrant, and all advantages we gain by using it in a follow-up blog post.
Vagrant is not the only tool for virtual environment automation and provisioning, there is Veewee (creation of base boxes, developped with love by Patrick Debois) and Cobbler (my colleague, Mathieu Cerda, gave a talk about Cobbler at LOADays 2013, here is a link to his slides). They are very good and I encourage you to take a look at them! Easy automation is now a reality, it would be a shame not to take advantage of this!