Media de-duplication script.

I enjoy using Plex Media Server. To the point where I have backed up all of my media, tossed the cases, and put everything into folders. A big plug for Plex and MakeMKV.

There are filename formats that you should use for TV shows:

That’s nice, but once in a while you might replace content (Blu-Ray instead of DVD quality, etc.) It can be a pain to re-rip all that content. If you use automated tools to pull content and sort it out and move it places, sometimes you’ll wind up with multiple versions of the same file, possibly with different filenames, formats, etc.

That was the case for me, and I had changed filename formats due to the conversion to Plex from XBMC. That’s the biggest reason for this (simple) script:

While most of the filename was different what wasn’t is the sxxexx format (s01e01, s01e02, etc.) With some regex, that’s enough to compare and remove duplicates. Note, this will prompt for every match. Even if you’ve removed the first instance.


Auto tar find results.

You’re a standard sysadmin. You get an alert that your filesystem is filling up (usually on /var.) You don’t need to spend a lot of time cleaning it up so like any good sysadmin you write a script. Here’s a couple of quick ones.
First, grab files a, but exclude set b. (Maybe you want to archive log_files.may.txt, but not log_files.june.txt.tgz.)
A little simpler. Grab anything that matches the wildcard and compress it.
Assuming the following:


You want to keep secure-20150315 but compress everything else.

What does this do?

Find files in /var/log with the name of secure-20150315 and exclude them from processing. Then (-o) search for anything not excluded with the name secure-2015*. The print0, and xargs options are really the neat thing here. Normally the results of find will be a hash. This iterates through each file individually instead. {} is the variable so {}.tgz will create a file called ‘secure-20150301.tgz’ containing all content in 20150301.tgz.

This is super useful if you’re trying to condense directories. All you have to change? type f to instead be type d.


Bash mass change permissions using find

A quick one-liner to recursively change all files in a directory to a set permission:

You can substitute chown if you want to set ownership:

This comes in handy when you want to give a group the ability to navigate through directories while not just blindly giving write/execute permissions.

If you want to avoid doing recursion, use –max-depth=x where x is the number of subdirectories to navigate:

This will only modify files in the path directory.

Putty Reverse forwarding command line options.

Port 80 is the port that you’re connecting to on the remote server. Port 12000 is the port you’re connecting to locally. Say this is a HTTP connection. The path to connecting is simply pulling up localhost:12000 in any web browser.

Bash renaming utility

Ubuntu/Debian packages the rename utility with perl and regex. For distros that do not, this one-liner is handy:

As with anything Linux, there are many ways to accomplish this.

Replacing Google Fiber’s Network Box

Google Fiber is amazing. Their network box does a pretty decent job but it lacks a lot of features for more advanced users. Their router does a good job with active connections and considering the bandwidth in use for the average person it’s fine. If, however, you are wishing to do more with it (DMZ, bridging, better port forwarding, or even just wanting to use your own router) it’s not currently possible with their modem.

There’s some projects in play (pfsense, etc) that allow you to connect your own hardware. Google doesn’t discourage your replacing their network box and even gives you some basic information on how to proceed:

The takeaway there is that you need to vlan a port, and set QoS bits on egress traffic. Once that is done, you can hook any Linux machine directly up to the fiber jack.

I’m using eth3 for WAN in this example. Replace the WAN port on your machine with whatever yours is.

You need to VLAN the interface that is connected to the fiber jack. If this is a single machine, this is easy. Typically eth0.

On EL systems:

Now create the VLAN:

Create the VLAN device:

Finally add the route:

The route here is necessary for TV services. If you just have internet… you won’t need to do it.

This gets all of your changes active. You should now have a new address on eth3.2.

Finally, set the QoS/CoS bit on egress traffic. Until this is done you will max out at roughly 10mb/s upload speed (I was getting 500mb/s down even here.)

Special thanks to my friends Josh Bergland and John Narron who helped me with some packet diving to get everything working!