Spacewalk cluster with self-signed certs.

Spacewalk has a lot of options. There’s a lot of good docs out there by the default docs are somewhat hard to follow. This post is intended to be a framework to create a self-signed CA (or use a third-party CA and skip the self-signed portion), and then apply the root CA cert to the entire cluster simplifying configuration. If you go self-signed, you’re definitely going to want to create the CA certs.

For the purpose of this post, we’re making a master and slave configuration with proxies.

Wewt. We have our signing key. Time to sign some certs. For this post we’re going to have 7 servers: – the spacewalk master server. – These are the spacewalk portal/slave servers. These will pull packages and channels from the master server. – These are the Spacewalk proxies. Each of these sits in front of the Spacewalk slave and caches packages.

A quick note on how this configuration works. The master server pulls in packages and assigns them to channels (repos.) The slave servers sync content with the master. Errata is assigned to the slaves and tied to channels, and packages in those channels. You can run servers directly against the slaves or master, but it can be tipped over under heavy load. The proxies use Squid to cache packages offloading much of the work but otherwise are just relays.

On relays, there’s two basic forms of package management. OSAD, and RHN. OSAD uses a HTTP keepalive sessions from each client to the SW server to allow for pushes. It’s very handy, but it can be a pain to maintain with a large number of clients. RHN is the other method. Each server will check into the proxy/slave/master every 60 minutes by default and see if there are commands queued.

Back to the build. We’re going to make some client certs, then sign them.

Now we have ssl certs for each of the servers in this cluster. Copy them over to each server. The structure of the directories on each server will need to be this:

This imports the cert into the Spacewalk application. Lets create the RPM for the CA, and each server:

Substitute proxy.c.slave for each of the hostnames listed previously. There will be 7 total here. Validate that each cert matches:

If there are any problems, recreate, or resign the certs.

Now that we have the actual SSL certs, lets start applying them. Spacewalk has tools that creates RPM’s for the CA (/root/RHN-ORG-TRUSTED-SSL-CERT) and also the certs.

There’s an additional step on the slaves, and the master:


On the proxies:

There’s some basic management needed next and good guides out there for that stage of things. At a minimum, you need to go into the slave servers and add the proxies, grant them access to any wanted channels and make sure that they have a provisioning entitlement.

On each client to connect to these servers:

This took a lot of piecing things together. It’s not really complex and there’s some good guides out there but it’s hard to get everything working from end to end.

Media de-duplication script.

I enjoy using Plex Media Server. To the point where I have backed up all of my media, tossed the cases, and put everything into folders. A big plug for Plex and MakeMKV.

There are filename formats that you should use for TV shows:

That’s nice, but once in a while you might replace content (Blu-Ray instead of DVD quality, etc.) It can be a pain to re-rip all that content. If you use automated tools to pull content and sort it out and move it places, sometimes you’ll wind up with multiple versions of the same file, possibly with different filenames, formats, etc.

That was the case for me, and I had changed filename formats due to the conversion to Plex from XBMC. That’s the biggest reason for this (simple) script:

While most of the filename was different what wasn’t is the sxxexx format (s01e01, s01e02, etc.) With some regex, that’s enough to compare and remove duplicates. Note, this will prompt for every match. Even if you’ve removed the first instance.


Auto tar find results.

You’re a standard sysadmin. You get an alert that your filesystem is filling up (usually on /var.) You don’t need to spend a lot of time cleaning it up so like any good sysadmin you write a script. Here’s a couple of quick ones.
First, grab files a, but exclude set b. (Maybe you want to archive log_files.may.txt, but not log_files.june.txt.tgz.)
A little simpler. Grab anything that matches the wildcard and compress it.
Assuming the following:


You want to keep secure-20150315 but compress everything else.

What does this do?

Find files in /var/log with the name of secure-20150315 and exclude them from processing. Then (-o) search for anything not excluded with the name secure-2015*. The print0, and xargs options are really the neat thing here. Normally the results of find will be a hash. This iterates through each file individually instead. {} is the variable so {}.tgz will create a file called ‘secure-20150301.tgz’ containing all content in 20150301.tgz.

This is super useful if you’re trying to condense directories. All you have to change? type f to instead be type d.


SSL vuln script

This script helps determine what processes are open, the port they’re listening to, and the process name. This is intended for testing for the Poodle/CVE-2014-3566 SSLv3 vulnerability discovered recently.


var=($(netstat -alntpu|grep LISTEN | awk '{print $4}'|sed 's/.*:\(.*\)/\1/g'))
sorted=($(sort <<<"${var[*]}"| sort -k 1n|uniq ))

sslcheck () {


  if [[ "$1" =~ "::" ]]; then

  ret=$(echo Q | openssl s_client -connect "$ip:$port" -ssl3 2> /dev/null) 2> /dev/null
  servicename=($(netstat -tulpn |grep "$2" |grep "$1" | awk '{print $7}'| sed 's/.*\/\(.*\)/\1/g'))

  if [[ -z "$servicename" ]]; then
  elif [[ "$servicename" == "-" ]]; then

  if echo "${ret}" | grep -q 'Protocol.*SSLv3'; then
    if echo "${ret}" | grep -q 'Cipher.*0000'; then
      echo "$servicename,$ip,$port,SSL 3.0 disabled on port"
      echo "$servicename,$ip,$port,SSL 3.0 enabled"
    if [[ -n "$port" ]]; then
      echo "$servicename,$ip,$port,SSL disabled or other error"

scan_all_listening_ips () {
  listenarray=($(netstat -tulpn |grep "$1" | awk '{print $4'}| sed 's/\(.*\):.*/\1/g'))
  sortedlistenarray=($(sort <<<"${listenarray[*]}"| sort -k 1n|uniq ))
  for ip in "${sortedlistenarray[@]}"; do
      if [[ "$ip" =~ "::" ]]; then

    sslcheck $ip $1

for i in "${sorted[@]}"; do
  scan_all_listening_ips $i

unset IFS
exit 0</code></pre>

Ovirt, hosted engine, and cert problems.

I recently ran into a weird problem with Ovirt and the new hosted-engine. I created a new setup with 3.4 and am currently running 3.4.3. For whatever reason, the ca file was not copied over and ovirt-image-uploader isn’t designed to catch those sorts of things. That led to some really ugly, useless errors:

ERROR: ‘NoneType’ object is not iterable

What this is actually indicating (in this case) is that the cert file is missing. A quick copy:

rsync hosted_engine:/etc/pki/ovirt-engine/ca.pem /etc/pki/ovirt-engine/ -avrthP

then an attempt to upload again:

engine-iso-uploader -v -i iso_domain upload centos.iso –insecure

Progress! It begins copying. Though silently. Without a status message to indicate it’s not just hanging. If in doubt, use a ls -alh /path/to/iso/domain. The file will retain the filename you assigned, and be pre-pended with a dot.

A big thank you to Keith for the patch which actually gave a valid error message:

VNC – automatic console sharing upon XFCE boot

There’s a lot of ways to share a screen in Linux. Messing with GDM, various windows managers and the like can be painful. Here’s one way to have XFCE automatically share the screen on boot – pre graphic login.


Just enter your password here and you’re set!

Installing Rocket on CentOS 7

Rocket is CoreOS’ answer to Docker. Their projects goal is enhanced security, a little more freedom, and possibly just to capture the value spot

So, a couple of quick things. First, they have a module up on Github. This is NOT considered production ready. This is heavily alpha/beta. They are making good progress on it though. Second, are you sure you want to run this on CentOS? They have management built into Docker that is more supported.

If you know what you’re doing, or you’re sure that you want this method (you know what you’re doing) forge ahead.

Final note before we start, either you need to roll with a kernel much newer than 2.6 on EL5/6 platforms – and you will not be using EL 5/6 at that point – or you need to outright use EL7. There’s not a ton of difference between RHEL, CentOS, Oracle Linux, Scientific Linux or any flavor that is a binary equivalent of RHEL.

Onto the code!

It will run for a bit and compile some things. Once that’s done:

rkt version

rkt version 0.3.2+git appc version 0.3.0+git

If you get something similar, you’ve got it working! I’ll update the blog this weekend with the next few stages of things, and a basic deploy.