Teaching A Homely Mutt New Tricks

January 19, 2015 | posted in: nerdliness

About two years ago I read Steve Losh's excellent The Homely Mutt. It's a fantastic how-to on setting up Mutt as your email client. Mostly to see if I could, I set aside a day and managed to get it all working. Having scratched my technology itch I set Mutt aside and continuted on with Mail.app.

About a year ago my employer switched to Office365 for email. While I find the webmail interface to be clunky and difficult to use, getting my mail through Mail.app worked so I wasn't too bothered by the switch. However when I upgraded to Yosemite in October I started having problems with my Office365 mail. The 10.10.1 release did improve things some, but it's still a pain to use.

This past weekend I decided to resurrect my Mutt setup, and see if I could make it work -- not just for one account, but for six accounts. The Homely Mutt (THM) posting describes how to setup Mutt to work with a single GMail account. I have 3 domain accounts, two Gmail accounts, and the afore mentioned Office365 (Exchange) account.

My dotfiles repository had the Mutt configuration I had created in 2013, and I had managed to get that working with multiple accounts. Not having touched it in nearly two years I decided to start over from scratch.

Mutt for Gmail

I reread THM and started in from the top. This allowed me to setup one of my two Gmail accounts. I picked the one with the least amount of mail to reduce the time Offline imap would need to complete the initial download of my mail.

Offlineimap for Multiple Computers

I have three computers that I use on a daily or near daily basis and I want to have my mail available on all three. So I wanted to modify the original offlineimap.py script to work using the current user account, and not a hard coded account as shown in THM. Digging around in Mr. Losh's dotfiles I discoverd his solution was to simply use whoami in place of the hard coded user account name. The resulting script looks like this:

#!/usr/bin/python

import re, subprocess

def get_keychain_pass(account=None, server=None):
    params = {
        'security': '/usr/bin/security',
        'command': 'find-internet-password',
        'account': account,
        'server': server,
        'keychain': '/Users/`whoami`/Library/Keychains/login.keychain',
    }

    command = "sudo -u `whoami` %(security)s -v %(command)s -g -a %(account)s -s %(server)s %(keychain)s" %params
    output = subprocess.check_output(command, shell=True, stderr=subprocess.STDOUT)
    outtext = [l for l in output.splitlines()
                     if l.startswith('password: ')][0]

    return re.match(r'password: "(.*)"', outtext).group(1)

Mutt with sidebar-patch

On OS X the easiest way to install Mutt is using Homebrew. Unfortunately the formula for Mutt no longer contains the sidebar patch. Having a sidebar where your accounts and their mailboxes are displayed is, in my opinion, essential. A short Google search lead me to how to apply this mutt sidebar patch on StackOverflow. You just edit the Mutt formula and insert the missing option.

$ brew edit mutt

And then, scroll down to a section of commands that all start with "option", sort of like:

option "with-debug", "Build with debug option enabled"
option "with-trash-patch", "Apply trash folder patch"
option "with-s-lang", "Build against slang instead of ncurses"
option "with-ignore-thread-patch", "Apply ignore-thread patch"
option "with-pgp-verbose-mime-patch", "Apply PGP verbose mime patch"
option "with-confirm-attachment-patch", "Apply confirm attachment patch"

Add this line to the bottom of the options:

option "with-sidebar-patch", "Apply sidebar patch"

Scroll down further to the section with all the patches, e.g.

patch do
  url "http://patch-tracker.debian.org/patch/series/dl/mutt/1.5.21-6.2+deb7u1/features/trash-folder"
  sha1 "6c8ce66021d89a063e67975a3730215c20cf2859"
end if build.with? "trash-patch"

And add this block:

patch do
  url "https://raw.github.com/nedos/mutt-sidebar-patch/7ba0d8db829fe54c4940a7471ac2ebc2283ecb15/mutt-sidebar.patch"
  sha1 "1e151d4ff3ce83d635cf794acf0c781e1b748ff1"
end if build.with? "sidebar-patch"

Exit the editor and run

$ brew install mutt --with-sidebbar-patch

Multiple Accounts in Mutt

Much of the configuration described in THM still applies for multiple mail accounts. Below I've described where there are differences.

Offlineimap

You'll need to add the account identifier for each account to the accounts = line. And you'll need to create new [Account ...], [Repository ...-Local], and [Repository ...-Remote] sections for the new email account as well. I've included a shortened version of my .offlineimaprc file showing two accounts. The complete version is a bit long.

# vim: ft=rc:

[general]
# ui = ttyui
ui = blinkenlights
#ui = quiet
accounts = mark,root
pythonfile=~/.mutt/offlineimap.py
fsync = False

[Account mark]
localrepository = mark-Local
remoterepository = mark-Remote

[Repository mark-Local]
type = Maildir
localfolders = ~/.mail/mark

[Repository mark-Remote]
maxconnections = 3
type = IMAP
remoteuser = mark_zanshin
remotehost = mail.example.com
remoteport = 993
ssl = yes
remotepasseval = get_keychain_pass(account="mark_zanshin", server="mail.webfaction.com")
realdelete = no

[Account root]
localrepository = root-Local
remoterepository = root-Remote

[Repository root-Local]
type = Maildir
localfolders = ~/.mail/root

[Repository root-Remote]
maxconnections = 3
type = IMAP
remoteuser = root_zanshin
remotehost = mail.example.com
remoteport = 993
ssl = yes
remotepasseval = get_keychain_pass(account="root_zanshin", server="mail.webfaction.com")
realdelete = no

You'll also need to visit Keychain Access and create new password entries for incoming (imap) and outoging (smtp) messages. This process is unchanged from THM, you just need to do it for each new account. With all the accounts setup in .offlineimaprc and new Keychain entries I was able to kick off the rather lenghty process of downloading my mail.

.muttrc Changes

Some of the changes to the .muttrc file happen in that file and others I farmed out to a new directory called accounts.

The .muttrc changes are fairly straight forward. You need to edit the list of mailboxes you want to have appear in the sidebar. I've added what amounts to a comment or tag before each new set of mailbaoxes to make identifying which account is which easier.

# Mailboxes to show in the sidebar.
mailboxes "+-- mark ---------------" \
          +mark/INBOX \
          +mark/archive \
          +mark/Drafts \
          +mark/Sent\ Messages \
          "+-- root ---------------" \
          +root/INBOX \
          +root/archive \
          +root/Drafts \
          +root/Sent\ Messages \
          "+-- MarkNichols ----------" \
          +marknichols/INBOX \
          +marknichols/archive \
          +marknichols/drafts \
          +marknichols/sent \
          "+-- CodeProle ----------" \
          +codeprole/INBOX \
          +codeprole/archive \
          +codeprole/drafts \
          +codeprole/sent 
          "+-- mhn (ksu) ----------" \
          +mhn/INBOX \
          +mhn/Archive \
          +mhn/Drafts \
          +mhn/Sent\ Items \
          +mhn/github \
          +mhn/bugzilla \
          +mhn/ome-dt-l \
          +mhn/Junk\ Email \
          +mhn/Deleted\ Items \\
          "+-- Chef ---------------" \
          +chef/INBOX \
          +chef/archive \
          +chef/Drafts \
          +chef/Sent\ Messages \

Using the folder-hook feature of Mutt I was able to setup account specifc settings, each in their own file. When an account is selected in the sidebar and one of its mailboxes opened the folder hook sources a short chunk of muttrc code to set the account specific values.

Under the .mutt directory I created a new directory called accounts. In accounts I put a file for each email account. Here's an example file:

# Account Settings ------------------------------

set from      = "mark@zanshin.net"
set sendmail  = "/usr/local/bin/msmtp -a mark"

# Default inbox.
set spoolfile = "+mark/INBOX"

# Other special folders.
set mbox      = "+mark/archive"
set postponed = "+mark/drafts"
set record    = "+mark/Sent Messages"

color status yellow default

The folder-hook lines in the .muttrc file look like this:

# Set account specific options on folder change
folder-hook mark/*        source ~/.mutt/accounts/mark
folder-hook root/*        source ~/.mutt/accounts/root
folder-hook chef/*        source ~/.mutt/accounts/chef
folder-hook marknichols/* source ~/.mutt/accounts/marknichols
folder-hook codeprole/*   source ~/.mutt/accounts/codeprole
folder-hook mhn/*         source ~/.mutt/accounts/mhn

.msmtprc

The final piece of the puzzle is sending email. Edit the .msmtprc file and add the necessary information for each of the additional accounts. Again, here is a shortened version of my .msmtprc file:

account mark
host smtp.webfaction.com
port 587
protocol smtp
auth on
from mark@zanshin.net
user mark_zanshin
tls on
tls_trust_file ~/.mutt/Equifax_Secure_CA.cert

account root
host smtp.webfaction.com
port 587
protocol smtp
auth on
from root@zanshin.net
user root_zanshin
tls on
tls_trust_file ~/.mutt/Equifax_Secure_CA.cert

Conclusion and Caveats

You can view my complete Mutt setup on Github.

A word of caution applies. Unlike configuring tmux or Vim, making a mistake with Offlineimap or Mutt could potential wipe out some or all of your mail and cause the end of Western civilization. Proceed at your own risk.

A Guide for Setting Up Weechat and Bitlbee

January 10, 2015 | posted in: nerdliness

Introduction

With polished chat and IRC clients like Adium and Textual readily available why would you want to use a terminal-based, largely text-only IRC client for instant messaging? For me the reason was to have the same chat regardless of which computer I was using at the moment. Not the same chat application, the same chat. By using Weechat and Bitlbee in conjunction with tmux I'm able to create a singleton instance of my chat, complete with logs and all my chat providers. This posting is a guide to how I accomplished this setup.

The Setup

At my employment I have a desktop with a static IP address. By installing both Weechat and Bitlbee on that machine, and launching Weechat inside a tmux session, I can secure shell to my desktop from any of my other computers and attach to the chat tmux session. With one single instance of weechat+bitlbee installed this way I've centralized my instant message and IRC experience. One set of logs, one set of configuration files, one of everything.

The steps that follow were all done on OS X. Your experience will certainly vary on other operating systems.

Install bitlbee

$ brew install bitlbee

Install weechat

$ brew install weechat --with-lua --with-perl --with-python --with-ruby --HEAD --with-aspell

Add Lua, Perl, Python, and Ruby flags to support scripts (plug ins) in those languages. The --HEAD flag installs the latest available version of weechat. --with-aspell provides spell check support. To use this you first need to install aspell (using brew or method of your choice) along with any languages you wish. See the weechat documentation.

Start bitlbee

$ bitlbee -F -u <your-user-id> 

The -F flag starts bitlbee as a daemon. And the -u flag followed by your user id starts it as you and not as root. It is recommended to not run bitlbee as root.

Start weechat

$ weechat

Configure weechat

There are almost a thousand settings in weechat, so configuration option combinations are nearly limitless.

/set *
...
980 options (matching with "*")

Getting help

The best way to get help for any weechat configuration option is to use the /help command.

/help <command>

You can also see "settable" options using the /set command.

/set *

or use the * wildcard

/set *nick*

bitlbee also has excellent built-in help, accessed by using the help (no leading slash) command in the &bitlbee control channel.

Also, be sure to periodically save your settings or they won't be persisted when you quit weechat.

/save

You should also save your bitlbee work periodically.

save

Basic weechat setup and configuration

These are the configuration settings I've made.

Saner time format:

/set weechat.look.buffer_time_format “%H:%M:%S”

Use a tilde (~) between panes within weechat:

/set weechat.look.separator_horizontal “~”

Position the weechat nicklist bar on the left:

/set weechat.bar.nicklist.position left

And limit the nicklist bar width to 15 characters:

/set weechat.bar.nicklist.size 15

Mute the time color:

/set weechat.color.chat_time gray

Set the background color for the status bar (at the bottom):

/set weechat.bar.status.color_bg black

Turn off highlighting of your own nick, it gets annoying otherwise.

/set weechat.color.chat_nick_self *!lightgreen

Limit the width of the buffers side bar:

/set weechat.bar.buffers.size 20

Status bar settings

Don't hide the status bar:

/set weechat.bar.status.hidden off

Mute the status bar background:

/set weechat.bar.status.color_bg darkgray

Configure the contents of the status bar:

/set weechat.bar.status.items “buffer_number+:+buffer_name+{buffer_nicklist_count}+buffer_filter,completion,scroll”

Allow for multi-line input:

/set weechat.bar.input.size 0
/set weechat.bar.input.size_max 3

Miscellaneous weechat settings

Hide join/leave messages in IRC (After enabling freenode)

/set irc.look.smart_filter on
/filter add irc_smart * irc_smart_filter *

Use array of colors for participants in channels or chat rooms.

/set weechat.color.chat_nick_colors red,green,brown,blue,magenta,cyan,white,lightred,lightgreen,yellow,lightblue,lightmagenta,lightcyan

Colorize nick list away status. First check every x minutes, second limits to nicklists of a size or smaller (less processing)

/set irc.server_default.away_check 5
/set irc.server_default.away_check_max_nicks 25

One of my favorites, substitue the UTF-8 hookrightarrow character for the nick when the same person has multiple back-to-back messages.

/set weechat.look.prefix_same_nick ↪

Scripts (plugins)

weechat has a large library of scripts (think plugins). Here are the 4 that I use.

  • buffers.pl - Sidebar with list of buffers
  • highmon.pl - Highlight monitor
  • iset.pl - Interactive Set for configuration options
  • autosort.pl - Automatically or manually keep buffers sorted and grouped by server

buffers

buffers.pl provides a nice sidebar presentation of all the open buffers (channels or individual chats) you have open. There are a number of settings for buffers (64!); here are the ones I use:

/set buffers.color.number white
/set buffers.look.hotlist_counter on
/set buffers.name_size_max 20 
/set buffers.color.current_bg  default
/set buffers.color.current_fg = blue
/set buffers.color.hotlist_message_bg default
/set buffers.color.hotlist_message_fg yellow

These are fairly self explanatory. Show the number for each buffer, and indicate how many new messages are in a buffer. Limit the length of buffer names to 20 characters. Make the name of the current buffer blue against a default background -- my terminals are black, and make any buffer with new activity yellow against the default background.

highmon

highmon.pl creates a highligh monitor allowing you to see mentions of your nick, or any other key word you choose. Since my given name is Mark I actually have my highlight monitor set to ignore uses of mark, e.g., bookmark, to prevent highlight notification I don't want. Like buffers, highmon has a number settings. I used Pascal Poitras's Weechat Hightlight to get my highligh monitor setup.

iset

iset.pl creates an interactive set interface for configuration options. It has a search feature that makes find obscure options a bit easier.

autosort

autosort.py by default groups your channels by server. I'm using this script to mimic GUI chat clients in grouping chatrooms and individual chats under their owning server. More on this in the Advanced Topics section at the end of this article.

Create a sidebar with list of all people from all servers

These steps will create a sidebar containing a list of all people in all your networks: IRC, Jabber, et cetera. It makes use of the weechat /bar command. Read the /help bar output to learn more.

Create the bar and populate it

/bar add bitlist root right 15 on "@irc.Bitlbee.&bitlbee:buffer_nicklist"

Configure the bar. I no longer remember where I found these settings. Your mileage may vary.

/set weechat.bar.bitlist.color_bg = default
/set weechat.bar.bitlist.color_delim = default
/set weechat.bar.bitlist.color_fg = 37
/set weechat.bar.bitlist.conditions = ""
/set weechat.bar.bitlist.filling_left_right = vertical
/set weechat.bar.bitlist.filling_top_bottom = columns_vertical
/set weechat.bar.bitlist.hidden = off
/set weechat.bar.bitlist.items = "@irc.Bitlbee.&bitlbee:buffer_nicklist"  (default: "")
/set weechat.bar.bitlist.position = right
/set weechat.bar.bitlist.priority = 0
/set weechat.bar.bitlist.separator = on
/set weechat.bar.bitlist.size = 15
/set weechat.bar.bitlist.size_max = 0
/set weechat.bar.bitlist.type = root

The result of this will be a column on the right side of the terminal showing the ids of all the people connected to your IRC and chat servers. People online will be indicated by a plus sign, e.g., +somebuddy.

Configure Weechat for IRC

Out of the box weechat is an IRC client so by default it connects you to freenode using generic connection settings. You can customize your connection like this.

Set your IRC nicks

/set irc.server.freenode.nicks “your-irc-nick”

This can be a comma delimited list.

Set your IRC username

/set irc.server.freenode.username “your-irc-username”

Set your real name or pseudonym

/set irc.server.freenode.realname “Your Name”

Configure things to automatically connect

/set irc.server.freenode.autoconnect on

Add the list of channels you'd like to always join

/set irc.server.freenode.autojoin “#channel1,#channel2,#channel3,#weechat”

Change your IRC passphrase

If you ever need or want to change your IRC account passphrase, here's how.

/set irc.server.freenode.command /msg nickserv identify NEWPASSHERE

Joining/parting channels and being away

To join a channel use the /join command:

/join #channel

or use the short form

/j #channel

Or to leave a channel use the /part command:

/part [quit message]

You can mark yourself as away:

/away <away message>

You can also mark yourself as away for -all connected servers

/away -all <away message>

And you can return from being away

/away [-all]

Configure bitlbee in weechat

bitlbee is a gateway to servers that aren't IRC servers. Jabber, Twitter, Yahoo! and others are possible. The setup described below is for a Google Talk and Jabber account.

Connect to bitlbee server

bitlbee runs on port 6667

/connect localhost 6667

The connection will use a default user name based on your IRC name. Since you haven't created or signed into a nick yet the id used will likely be your operating system account.

Autoconnect Bitlbee

If you'd like to have weechat automatically connect to bitlbee, create a server in weechat for the connection. Here the tag im for instant message is used as the server name.

/server add im localhost -autoconnect

Register a password for bitlbee

Change to the control channel called &bitlbee and register a password for the account. This creates an XML file on the host system named for the user to store connections and chat room information.

register <password>

Create a "instant message" server

To create a server called im for "instant message" that automatically connects when you start weechat, and further identifies you to bitlbee using the previously registered password, do this:

/server add im localhost/6667 -autoconnect
/set irc.server.im.command "/msg &bitlbee identify YOUR_PASS"

Add accounts

While in the &bitlbee control channel (denoted by the &) add a Jabber account:

account add jabber you@server.tld <password>

Or add a Google Talk account.

account add jabber first.last@gmail.com <password>

Use the ac list (ac is short for account) to see your newly created accounts.

ac list

If you want to see the full name of an buddies on GTalk and not just their account short name, you can do this:

ac gtalk set nick_format %full_name

Conclusion

I've been using weechat+bitlbee as my primary chat interface for the better part of a year now. I'm still learning the ins and outs of how to tweak this setup. The most recent change has been to have individual bitlbee connections for each service rather than having all services merged into a single &bitlbee control channel. Combined with some buffers.pl settings changes the result is quite nice.

Advanced Topics

Multiple bitlbee users

If you want to separate individual and group chats by service (MSN, Yahoo!, Jabber, GTalk, et cetera) you need to create separate bitlbee users and register a password for each.

In a &bitlbee channel use /nick to change your identity and then use register to set a password for that identity.

/nick gtalk
register <password>

Next create a new weechat server and connect to it.

/server add gtalk  localhost/6667 -autoconnect
/connect gtalk

Move to the &bitlbee control channel belonging to gtalk and switch to to new nick and identify yourself.

/nick gtalk
identify <password>

Now you can add your Google Talk account as described above. Repeat these steps for Jabber or MSN or what have you.

Adjusting the buffer list

These settings changes will make use of the multiple bitlbee (and IRC for that matter) connections.

/set irc.look.server_buffer independent
/set buffers.look.indenting on
/set buffers.look.show_number off

By default the server buffers are all merged together (merge_with_core), using independent splits them apart. Turning indenting on causes the channels belonging to a service to be indented. And turning buffering numbering off cleans up the look.

Here's an image of the final result.

weechat

New Year

January 01, 2015 | posted in: site

The start of a new year always inspires people to start new ventures, set new goals, and make resolutions. Quite without prior planning or thought I've added two new sub-domains to my site today.

Books

I read. I read a lot. I frequently am reading more than one book at a time. Books is where I plan to track all the books I read or listen to this year.

Health

New year beginnings seem to attract health resolutions. I'm not making a resolution. What I am doing is keeping score; tracking my exercise and eating habits. I want to have a more active lifestyle overall and to increase my general fitness. Toward that end I bought a treadmill desk for my office where I hope to walk at least a couple, three, hours every work day. At home we also have a treadmill, and my wife and I take walks most days around the block. By cataloging my health metrics for a year I hope to see patterns, good and not so good, and develop better habits. You can follow along on the new health sub-domain.

Changing My tmux Command Prefix to Tic

December 27, 2014 | posted in: nerdliness

By default the tmux Command Prefix is Control-B (<C-b>). As this isn't the easiest key combination on the standard US layout QWERTY keyboard most tutorials suggest remapping it to Control-A (<C-a>). I went one step further and remapped my Caps Lock key to be a control key (I wasn't using it anyway). Now any time I need to communicate to the tmux server I can <C-a> plus a command key.

The tmux commands I issue frequently are:

<C-a> c - create a new pane inside the current window
<C-a> , - rename the current pane
<C-a> # - switch to the pane numbered #
<C-a> d - detach from the current session
<C-a> s - synchronization toggle, syncs all panes or turns sync off
<C-a> o - maximizes current pane by minimizing all others
<C-a> i - equalizes size of all panes
<C-a> z - hides all but current pane (great for copying)

Some of these -- creating a new pane or renaming a pane -- happen infrequently in the life of a tmux session. Others -- switching panes using their index number -- happen dozens of times a day.

Using <C-a> as the Command Prefix adds a two-key combination to each of the tmux sever commands I issue. Until quite recently this wasn't an issue. However I watched several installments of Learn tmux the other day and one of the first suggestions made was to use the backtic character: ` as the Command Prefix.

The trick to this mapping is to set up your .tmux.conf file to pass along a ` when two are pressed back-to-back. This way you can still utilize the backtic character (as I have creating this posting).

unbind C-b
set -g prefix `
bind ` send-prefix

The unbind C-b removes the default Command Prefix binding (Control-b). The set -g prefix ` make ` the new Command Prefix key. And bind ` send-prefix uses send-prefix to pass ` along to the application. Without the last line above you'll lose the ability to type `.

After having used tmux with <C-a> as my Command Prefix for the better part of two years now, switching to ` will take some effort, but I think the savings in key strokes over time will be well worth it.

Integrating Dash Into Vim and the Command Line

December 26, 2014 | posted in: nerdliness

Dash is an outstanding documentation and API reader for Max OS X. Dash gives you instant, offline, access to over 150 documentation sets -- including cheatsheets, vendor documentation, and user submitted docsets. It's well worth evey penny of it's $19.95 cost. And if that wasn't enough you can easily integrate Dash into any number of editors and/or application launchers.

Here's how I've integrated into my Vim setup and into my command line.

Vim

The dash.vim plugin adds a family of :Dash commands and mappings to your Vim configuration. The mapping that I use is this:

:nmap <silent> <leader>d <Plug>DashSearch

This mapping allows me to seach Dash for the term currently under my cursor. This one mapping has vastly improved my Dash usage, so much so that I haven't bothered to discover what else this plugin adds to Vim.

Command Line

Brett Terpstra regularly creates and writes about wonderful functions, shell scripts, and tool integrations. In Bash and Dash he talks about two shell functions that let you search Dash's docsets from the command line. The first

# Open argument in Dash
function dash() {
  open "dash://$*"
}

searches Dash for the query provided. Each Dash docset is associated with a shortcut, so you can search for things like ruby:FileUtils or chef:service.

The second,

function dman() {
  open "dash://manpages:$*"
}

searches the manpages docset, allowing you to view man pages in Dash rather than in your terminal.

Since I spend most of my working day at the command line or in Vim, these simple configurations make it vastly easier to search documentation.

⇪ UK 24

A stunning visualization of air traffic over the United Kingdom.

UK 24 from NATS on Vimeo.

⇪ IKEA BookBook

Superb.

⇪ Perfect Snooker Game

An amazing run of the snooker table.

How to Rename Chef Roles

July 23, 2014 | posted in: nerdliness

Today I had need to rename several roles used in our Chef configuration. They had been created with dashes rather than underscores in their names and that prevented me from use the role name as a Ruby symbol. For example, while my-role might be a perfectly good name, :my-role as a Ruby symbol doesn't work. Renaming the roles is only part of the problem, finding all occurrences of the role in all the run-lists and updating those occurrences is the real trick.

A word of caution: As with any code you find on the Internet, you should review the script and commands given below to assure yourself you know their impact. Mass changes can have far reaching consequences.

First off you need to download the role from the Chef Server to your workstation.

$ knife download roles

(This command actually gets all your roles.)

Next, rename the file.

$ mv my-role.json my_role.json

Now edit the file and correct the JSON "name": to match the new file name.

$ vim my_role.json

Upload the new role to your Chef Server

$ knife role from file my_role.json

On your Chef server now you should have two roles, one with a dash and one with an underscore that have identical contents.

To update all run-lists I used this script, which I found here: renaming a role

role_from = ARGV[2]  # format role[foo]'
role_to = ARGV[3]

if ARGV.length > 3 and ARGV[4] == 'dry-run'
   dry_run = true
else
   dry_run = false
end
puts role_from
nodes.all do |node|
   if node.run_list.include?(role_from)
     puts "match #{node}, current run_list #{node.run_list}"
     if node.run_list.include?(role_to)
       puts "ERROR #{node} has #{role_from} and #{role_to} skipping"
       next
     end
     pos = -1
     node.run_list.each_with_index do |item, index|
       if item == role_from
         pos = index
         break
       end
     end
     node.run_list.run_list_items[pos] = node.run_list.coerce_to_run_list_item(role_to)
     puts "\tchange  <at> pos=#{pos} new run list #{node.run_list}"
     if dry_run == false
       node.save()
       puts "\tsaving node #{node}"
     end
   end
end

exit 0

To run this, save the script somewhere on your workstation. I put mine in my ~/.chef directory in a new sub-directory called scripts. Once the script is saved run this knife exec command.

$ knife exec ~/.chef/scripts/rename_nodes.rb "role[my-role]" "role[my_role]"

Optionally you can do a dry-run of the script to see its impact before making any changes.

$ knife exec ~/.chef/scripts/rename_nodes.rb "role[my-role]" "role[my_role]" dry-run

You must specify the enclosing role[ ... ] otherwise the matching performed by the rename_nodes script won't work.

All your nodes should now be using the new role name. As a final step you can restart the Chef Client on all your nodes.

$ knife ssh 'name:*' 'sudo /etc/init.d/chef-client restart'

Once everything checks out you can remove the old role.

Using Teamocil To Manage Tmux Sessions

July 02, 2014 | posted in: nerdliness

When I am wearing my operations hat at work I need ready access to a number of servers, 15 or so. I use tmux to create a “console” session that is broken into 5 separate windows, each further divided into panes (one per server). The windows are rough categories: production, alpha, web, ESB, etc. I also normally have a “tools” session in a separate Terminal instance that contains 4 more windows, each to utility servers that I use periodically throughout the day.

Rather than try to create these tmux sessions by hand I’ve been using a tool called teamocil to capture the session::window::pane configuration in YAML files. With a console.yml file I can setup the entire console session with a single command.

Here’s how I set things up.

First you need to install the teamocil Gem.

$ gem install teamocil

Next you’ll need a space to keep the YAML files that describe your tmux sessions. teamocil expects this to be called ~/.teamocil

$ mkdir ~/.teamocil

teamocil comes with a command that will use your $EDITOR to create new session configuration files.

$ teamocil —edit console

The teamocil web site has good documentation on the options and attributes required to setup a configuration. Here is a sanitized version of my console.yml file.

session:
  name: "console"
  windows:
    - name: "prod"
      root: "~"
      options:
        synchronize-panes: true
      layout: even-vertical
      panes:
        - cmd: "ssh prod1"
        - cmd: "ssh prod2"
        - cmd: "ssh prod3"
        - cmd: "ssh prod4"
    - name: "web"
      root: "~"
      options:
        synchronize-panes: true
      layout: even-vertical
      panes:
        - cmd: "ssh web1"
        - cmd: "ssh web2"
        - cmd: "ssh web3"
        - cmd: "ssh web4"
        - cmd: "ssh web5"
    - name: "esb"
      root: "~"
      options:
        synchronize-panes: true
      layout: even-vertical
      panes:
        - cmd: "ssh hub1"
        - cmd: "ssh hub2"
    - name: alpha
      root: "~"
      options:
        synchronize-panes: true
      layout: even-vertical
      panes:
        - cmd: "ssh alpha1"
        - cmd: "ssh alpha2"
        - cmd: "ssh alpha3"
        - cmd: "ssh alpha4"

The name attribute is the name that will appear in my tmux status bar. root sets the directory in which each pane will be created. The synchronize-panes option is particularly nice as it allows me to type commands in one pane and have them run in all panes. The layout attribute determines whether the panes are vertically stacked, horizontally arranges or tiled.

Since I have my terminal set to create or reattach to my base tmux session creating my console session is one command away.

$ teamocil console

In seconds I have 5 windows (the original, prod, alpha, web, and ESB), each subdivided in to as many panes as there are servers. Quick, simple, repeatable.