I'm currently a puppet master for thousands of servers, for which thousands of lines of puppet DSL code have been written, and keep being written every day.
The Challenge
One of the immediate goals was to make those thousands of lines code to comply with the current
Puppetlabs' style guide.
For this purpose I wrote a library for applying arbitrary transformations to a puppet DSL code source.
It was very interesting because I learned a lot about how puppet is built while looking for an economic way to do these transformations.
Object orientation and code reuse
Honestly, I considered a combination of sed, awk and perl the first time I tackled this challenge.
puppet-lint for example uses regular expressions to tokenize the code and then warns you about violations to the style guide.
I didn't want to reinvent the wheel so I thought of tapping on puppet's lexical analyser to do the job for me.
It wasn't as easy to extend it as I expected it to be for an object oriented piece of software. My approach required some changes to the analyser in order to reuse it that were "impossible" due to some internal design decisions or shortcuts.
I ended up exploiting Ruby's dynamic typing by monkey patching Puppet::Parser::Lexer and finally get the BLANK and RETURN tokens I needed.
The design
The next challenge was to design the library, but hopefully it just presented to me: a production line (stream of tokens) where specialized workers (transformation algorithms) were waiting for the token they know how to work on and transform the set of tokens that matches the pattern they were taught to recognize.
Testing
The final challenge was to test the library output for correctness. The usefulness of puppet-cleaner depended on this. What use is a code cleaner if it modifies the behaviour of the original code?
Then I dug up a serialization method from puppet's source code that would help me
prove that two different text files were actually the same puppet DSL code, and put it in puppet-diff and puppet-inspect.
The result
It was fun indeed. Lots of new stuff.
The goal was achieved: thousands of puppet DSL code were made to comply in a matter of minutes (I still had to be very cautious.)
Today I'm
uploading the product of this effort to github for anyone to use. It's faster that puppet-lint although not as complete, but it actually fixes your code, instead of just warning you about it.
Finally, here's an input and output example.
input:
/*
multiline comment
trailing white space here ->
*/
class someclass($version = "5", $platform = "rhel6")
{
if ! ($version in ["4", "5"] or !($platform in ["rhel5", "rhel6"])) {
fail("Version $version on $platform is not supported")
}
else {
notice("Defining class for version \"$version\" on '$platform'")
package {
[
"package-1",
"package-2",
]:
}
sysctl2::conf {"${something}":
settings => [
"set key1 'value1'",
"set key2 'value2'",
]
}
file {
"/file/a": # test
source => 'puppet:///someclass/file/a',
owner => 'root',
ensure => ensure_a,
group=> "${group}";
"/file/b":
source => 'puppet:///someclass/file/b',
mode=>755,
ensure => ensure_b,
owner => 'root',
group=> "${group}";
}
file {
"/file/c":
source => 'puppet:///someclass/file/c',
owner => 'root',
ensure => ensure_c,
group=> "${group}";
"/file/d":
source => 'puppet:///someclass/file/d',
mode=>"755",
owner => 'root',
group=> "${group}"
}
file {
"/file/e":
source => 'puppet:///someclass/file/c',
owner => 'root',
alias => ["$hostname"],
group=> "${group}",
ensure => 'absent';
"/file/f":
ensure => 'ensure_f';
}
user {
"jdoe":
managed_home => "true",
}
}
}
output:
#
# multiline comment
# trailing white space here ->
class someclass($version = '5', $platform = 'rhel6')
{
if ! ($version in ['4', '5'] or !($platform in ['rhel5', 'rhel6'])) {
fail("Version ${version} on ${platform} is not supported")
}
else {
notice("Defining class for version \"${version}\" on '${platform}'")
package {
[
'package-1',
'package-2',
]:
}
sysctl2::conf {$something:
settings => [
'set key1 \'value1\'',
'set key2 \'value2\'',
]
}
file {
'/file/a':
ensure => link, # test
source => 'puppet:///someclass/file/a',
owner => 'root',
target => ensure_a,
group => $group;
'/file/b':
ensure => link,
source => 'puppet:///someclass/file/b',
mode => '0755',
target => ensure_b,
owner => 'root',
group => $group;
}
file {
'/file/c':
ensure => link,
source => 'puppet:///someclass/file/c',
owner => 'root',
target => ensure_c,
group => $group;
'/file/d':
source => 'puppet:///someclass/file/d',
mode => '0755',
owner => 'root',
group => $group
}
file {
'/file/e':
ensure => 'absent',
source => 'puppet:///someclass/file/c',
owner => 'root',
alias => [$hostname],
group => $group;
'/file/f':
ensure => link,
target => 'ensure_f';
}
user {
'jdoe':
managed_home => true,
}
}
}
