CodeGoalie

Welcome, Juniors!

Tue, 30 Jun 2026 11:04:53 -0400

I believe one of the reasons I’ve been pretty successful in adopting and utilizing Agents iny engineering work is relying on the decades of hard fought knowledge, experience, and taste. How do folks without that get it in this age of AI?

It’s unacceptable for me to say “AI is only for seniors and above” or other nonsensical takes. Here’s what I’m currently thinking. And as with most everything “rethought” for the AI era, it’s the same as before.

Appropriate scope

It’s unreasonable to expect a junior dev to architect an entire new service, or even feature. Continue to provide appropriate work, but allow for AI to help with even those small tasks. Thorough planning helps even with a small bugfix. You can provide critical feedback to the agent on test case edge cases, etc. It’s training for the big stuff but at a smaller scale. The same as its always been.

Apprenticeship

I read this recently and it really resonated with me. The concept is to have a junior and a senior work closely together for an extended period of time to learn first hand, day to day on how the senior operates.

One reason I find this super compelling is that there are many things that I do throughout the day that I would never think to capture in a training or lunch ‘n learn. Probably a lot of them I don’t even know I do.

Also, I love watching and learning from people more skilled and experienced.

I’m not worrying too much about new people coming up missing some aspects of the job that I know. The good ones will learn what they need and ignore all the extraneous crap that I’m burdened with ;)

Happy learning!

– Chris

Agents for System Design

Sat, 18 Apr 2026 05:54:21 -0400

System Design used to be reserved for the senior folks. They’ll hand down a document showcasing the new architecture or new service. Agents are making it so everyone can do some system design because even the smallest features can benefit from more intentional design.

Over the past couple months, I’ve been tasked with leading a very large cross-team project to essentially replace the core data model in our business. This has involved designing new services, new and novel interactions between services, and adding new capabilities to existing services. I’ve leaned heavily on agents to help produce and communicate my intent to other people (and other agents). Here is advice I’ve picked up along this journey.

Know what you want

This is a general principle for all if not close to all interactions with an agent and LLM. It’s not a magic oracle, even if it genuinely feels like it sometimes. It can’t read your mind. It doesn’t have the context of your company, team, feature, past mistakes, etc. It wasn’t at the last all-hands meeting to absorb some of that new initiative from the leadership team. It wasn’t a part of that ad-hoc conversation last week where we got clarity on an upcoming API interaction.

It doesn’t need to know that. It shouldn’t know that. It’s your job to determine what it should know, and in order to do that, you need to determine what you want it to do first. Every time you open up your agent, you should know what you want from it and when you’ll be able to exit it (ctrl+d ctrl+d).

As a principal engineer at PrizePicks, a lot of my work is designing new systems, adding new capabilities to existing systems, or integrating 2 systems together. The first step that I take is to create a proposal doc. There is a lot of formatting and process around this typically in companies. I ignore most of that here. I want the agent to produce a document which describes at a high level how this new thing is going to work.

That last sentence is pretty much the functional part of my initial prompt. The key here is know what you want before you start:

A document defining something in a 3rd party system (Notion, Linear docs, etc.)
A series of steps for me to execute some task
JSON for a grafana chart
Mermaid diagram of this flow/architecture
etc.

Have an end in mind before you start. Agents are a tool. Waving a shovel around wildly in the air is not the best way to dig a hole.

Provide the load bearing walls

This is a recent insight that I realized I’d been doing naturally, but I articulated it to a colleague recently and it actually inspired this whole post. This 5-second version is that if you’re building a tent with the help of an agent, you need to bring the tent poles. It can build the waterproof and functional covering. It can put in a door with a 2 sided zipper. But if you don’t bring the poles, it’ll just be a pile of fabric on the floor.

Conversely, you don’t need to build the whole tent out. It knows what tents are and how, generally, they work. It’s your job to, first, communicate that you’re building a tent (these are general purpose models). And then articulate what makes your tent yours.

A good recipe for this is to start with what you have. Talk about what you need. Then be explicit about what you want. A while back, I had an agent research the Epcot Flower & Garden festival menus for this year, comparing first a markdown file then a static HTML file. My wife and I went through the menus and picked the items we wanted to be sure to try. Now, I wanted to combine the two – Well, wait. Here’s the exact prompt I used:

❯ I have a full overview of the 2026 Epcot Flower & Garden festival in the
2026.[md|html] files. I've gone through them and noted my favorites in
2026-notes.md. I'd like to come up with a useful and intuitive guide to use
while I'm in Epcot to make sure I don't miss these choices. We'll be going
around the world showcase in one order or another so put the booths in the same
order as the world showcase starting with Mexico. I also would love to be able
to hide stuff I've already tried.

This was in plan mode in Claude Code so it generated a plan.

Meticulously review

Once you have a plan or document, it needs your review. You’ve provided a few sentences and it’s generated a whole document. This upscaling of information is lossy, by definition. In fact, each step of the way, you’re going from lower specificity to higher specificity and review is necessary.

The earlier steps need more review as they’re setting the stage for everything afterwards. A small misalignment in the initial plan will cascade and amplify into the final product.

I once missed that a message topic needed to have a version bump. The agent used v1 in the plan, but we actually wanted v2. Even the copilot during code review after I manually fixed the version numbers was complaining that “it should be v1 here.” 😩

This ties directly into the first point in this post. Know what you want beforehand so that you can evaluate it when the agent produces the artifact.

Cutthroat revisions

Since each phase feeds directly into another increase in specificity, be ruthless in your revisions earlier rather than later. I use the same working model as I do for code reviews: if this isn’t what you would have done, say something. It will only get worse from here. The models are very good at making things sound plausible and correct. That’s exactly what they do, in fact. “The new multiplication function can live in the addition package since they are both math.” Nope. Maybe you want to rename that package to newmath, or put multiplication in its own package. You need to make the call now and update the plan. It’s a one sentence change in the plan, but from here on out it gets more ingrained (maybe AGENTS.md updates to include something like “All math features are implemented in the addition package” 😱)

Functionally, I like to have the agent make the changes usually in the same conversation as the original plan was created. However, since you have an artifact, you can start a new conversation if you want. That’s the beauty of working with artifacts outside of the harness.

I find the system design part of this new Agentic Engineering paradigm to be the most interactive. Some days I like that. Other days I appreciate being able to hit a button and get a PR (more on that later). But the PR button only works when the plan is solid. I used to spend days hand-writing a technical design doc and it was often just OK. With agents, I get really great docs in often less than an hour of focused work after days of noodling.

Happy designing!

– Chris

Agents Calm Down

Mon, 30 Mar 2026 07:21:16 -0400

Agentic coding harnesses are violating the unix philosophy and trying to become the everything app for coding (and work and personal assistant and so on).

Why not build everything?

Many of the articles I’m reading are saying what we’ve been saying all along, “writing the code hasn’t been the bottleneck.” No project was late because engineers didn’t spend enough time sitting at their desks typing code. And now that writing the code is approaching zero, I think it’s more important than ever to carefully, methodically consider every project and initiative.

“This feature is not a good fit for coding agents so we aren’t going to put it into our harness.”

No AI company ever

Non-engineering folks have long enjoyed the luxury of engineering being the bottleneck (not writing code, but review, deploy, integration, etc.) and have been able to change requirements, goals and targets while we were working. When we can take a set of requirements and later that afternoon be dev complete… Just think about what would have happened if we’d implemented verbatim some of those first draft project proposals?

I’m working on a personal project to upgrade my personal daily task list from a plain markdown file to use Remember The Milk (the only task tracking app which gets recurring items right, but I digress). I have a pretty decent idea of what I want. I’ve been working in this flow for over 8 years after all. I’m, of course, having an agent write all the code. But I’m going slow. I’m carefully considering each feature and more precisely how it should work than I ever have ahead of time before. I feel this great responsibility to move intentionally because I have this great power to push a button, fold 5 pieces of clothes, turn back to my laptop and have a working feature.

To me, LLMs and agents and all of it, from the start, has never been about getting faster. It’s about reducing the “manual” effort of willing a (software) thing into existence. As I’ve been gaining experience with agents et al, I keep coming back time and time again to the best practices of the industry: “have a really good and detailed scope doc,” “compare that to the code and form a focused implementation plan,” “break that down into units of work that depend on each other or can be done in parallel.” We always wanted that but it was too time consuming to do and we relied on each other’s existing context. Many tickets don’t have any body description. We just talked about the project scope in the kick off, but didn’t formalize a document. Ironically, the machine is forcing (and enabling) us to do the things that would have been best all along, even for humans.

Do one thing well

Time and again the best software is focused and does one thing really well. That’s the unix philosophy. Composability over batteries included. Make a simple and powerful tool to let those using it combine it with other simple and powerful tools to make something greater than the sum of the tools themselves.

These new releases in the agent harnesses are hard to reason about because many of the new features feel half-baked and not rooted in solving a real problem. I end up feeling obligated to find use for them. It’s FOMO a bit. “Well, if they implemented it and released it, it must be something good. I must be missing something if I can’t find the use case.”

It’s hard to face this down. The company worth tens of billions of dollars is churning out features on the most talked about software of the moment and my gut is telling me to ignore most of it. I got bored and frustrated really quickly babysitting the agent. “Yes” “Approve” over and over. Same prompt. Same slash command. Same skill. Tweak it a bit here and there. Now, I’m automating it and running into some longing for more focus on a simple tool that I can work with not indefinitely inside of.

Do less

I don’t want to leave on a down note so let’s talk solutions. Do less. Make more. Let’s build an ecosystem around this tool. Let’s solve the rest of the SDLC competently. LLMs are a versatile technology and are made even more powerful with agent harnesses. They aren’t a silver bullet either. I love that we’re pushing the envelope of what can these things do. I love that we don’t know yet. It’s exciting to be a part of.

The more I work with agents the more I see how important the “old” best practices are. The more pre-planning matters. The more detailed specs matter. The more nailing down inter-service interacations matter. Start at the boundaries and work inward on both sides. Hand offs. Metrics. Automated end-to-end testing. There’s nothing new here. It’s just compressed and accelerated; and can be exhausting at times. In this age where we can build anything, the taste and discipline to not build something matters. I’d love to see that in the tools themselves.

Happy shipping!

– Chris

Use Your Noodler

Sat, 24 May 2025 06:52:56 -0400

A former coworker of mine over 8 years ago put this image into my head that has never left. He was working as my manager at the time and he had an upcoming large, foundational shifting feature that he wanted to let me know was coming. He said “Don’t do anything right now, but just put this in your noodler and see what comes out.”

He went on to explain that a former boss of his introduced the idea of the noodler to him. Now, I’m not your boss (I can confidently say that because I’m no ones boss as of this writing ;) ), but I’d still like to share this image with you that’s helped me so much and I’ve never been able to shake.

If you haven’t guessed by now, the noodler is the back part of your brain. Back near the subconscious, it’s like a rock tumbler. It’s always spinning and tumbling ideas without conscious effort. You just throw raw stuff into it and in a few days or so, see what pops out. Sometimes it’s just a smoother rock and sometimes it’s a true gem.

I like this metaphor for a few reasons. First, it’s a pretty vivid, concrete concept. I always imagine that red, kid’s science kit version of a rock tumbler that I always wanted. Now I have one and it’s turning half-baked, super hazy concepts into novel, actionable ideas.

Second, it’s an escape hatch. Do this one easy thing, make progress, done for the day. Take something off your plate. You can rest easier knowing that something, somewhere is happening, but you don’t need to spend active energy on it, right now. Wash the dishes and think about the birds outside. Mow the lawn and listen to that podcast. It’s OK. Some other part of you has got this.

Lastly, it works. Not all the time. But more often than not, something comes to me and I can make a small step, or a bigger leap. And if it doesn’t, typically, I’m no worse off.

Happy noodling!!

– Chris

golibnotify 0.2.0 released!

Mon, 22 Jul 2024 05:18:09 -0400

golibnotify is a Go library for sending desktop notifications on Linux. It is a wrapper around the libnotify library.

There are some memory leak fixes in the latest release, v0.2.0.

Big thank you to Stefan van den Akker (@omhaal) for fixing some unfreed memory in the SimpleNotifier. I’m not a c or cGo export so I really appreciate it!

Be sure to update your go.mod files to v0.2.0 and let me know if you run into any other bugs or issues!

Happy notifying!

Format JSON in Nvim with jq

Mon, 12 Feb 2024 16:09:11 -0500

I often get large JSON files which contain no newlines or indentation. I’d like to look at these reasonably in neovim, but it’s been cumbersome until today.

TL;DR: You can replace the contents of the buffer with output formatted by jq with one command: :%!jq .

Other techniques I’ve used, from worse to lest worse:

Replacing every , with ,\r and then using = to auto-indent the lines. This rarely works well.
Using jq externally. cat input.json | jq . > output.json. This kind of sucks because you now have 2 files…

Now, you can open the file in neovim and type :%!jq .. Magically, the buffer will be nicely formatted.

Roughly, this is entering a command, :, on the whole buffer, %, then executing a shell command, !, of jq . the most basic jq invocation. Presumably, you could entry and jq filter command to form some other JSON instead of just formatting the given JSON, but I’ll leave that up to you to play with.

Happy formatting!!

– Chris

AI and I, or You're Holding That Wrong

Mon, 15 May 2023 19:59:01 -0400

How can a tool that is indescernably wrong be anything more than a toy?

You’re holding that wrong. A month or so ago, I would have been the first to laugh you off if you started talking to me about ChatGPT being an experience like the first time you programmed or “saw the internet.” It can’t even answer basic questions like those I ask my smart speaker everyday? (That’s hyperbole. I don’t speak to me smart speaker nearly everyday.) Also, it lies to you; like to your face lies to you. What even is this? A fancy Markov chain?

Well, yes, actually it is. Then I heard Simon Willison on The Changelog and I finally got the mental model.

A calculator for words

– Simon Willison

What makes these seem so terrible at first glance is the interface. Imagine if a calculator has a single text prompt as an interface. No buttons. “Tell me the calculation you desire.” OK, so you as it to do your taxes. Unquestionably, a calculator can’t do that. But if that’s your expectation, you’re terribly disappointed and go back to carrying the one with paper and pencil.

A calculator does not know your intentions or your context. It can’t answer even slightly complex questions. But, if you know how to wield it, it’s incredibly useful. It can help you do your taxes and in record time*. In fact, all the accountants that never learned to use a calculator are unemployed; replaced by those that did.

These new LLMs are terribly persuasive. We’ve become used to a person being at the other end of written words. Text messages, emails, social media posts, news articles. A vast majority of the content we consume is written words. We’ve been conditioned to assume there’s a person at the other end. LLMs flip that assumption on its head. Despite stringing together the words to be quite convincing to the contrary, it has no feeling. It fact, it doesn’t even know what it’s saying.

Briefly, (if you already know technical details about LLMs, save yourself from getting mad at me and skip this paragraph) LLMs are a next word/paragraph predictor. Give some input words or sentences, it can guess, statistically which word or sentence should come next. There’s also randomness built in so you don’t get the same output twice. That’s it. It’s funny only because most text is also funny in that context. It’s bad at facts because you can’t give random answers and be correct all the time.

So, it can’t be right and it doesn’t know what it’s saying. How do I think about this? What is it good at? LLMs are really good at creating a bunch of useful words. They aren’t great out of the box. You can’t copy/paste straight from ChatGPT into your homework and call it a day. But it can give you a list of terrible ideas for something with a few gems sprinkled in. It can generate a first draft for you.

How to think about all this?

First, it’s not the be all, end all. It’s not a silver bullet. But, it can help. It’s a starting point. It’s a some, or most, of the way there. It’s an intern or beginner needing correction. It saves keystrokes. It’s a first draft. It won’t replace you. Someone who uses it will.

How I have used it to date?

I’m just scratching the surface on all of this and things are moving at an extremely fast pace. I don’t claim to be an expert but here are some examples of ways that I’ve found LLMs useful.

First and most significantly, Github Copilot. If you take nothing away from this post, start using Copilot (or one of the open-source alternatives) today. Get it integrated into your editor. The neovim integration is particularly slick. When you pause typing, some low-lighted text will appear after your cursor. It is suggested code from Copilot. It’s especially good at handling errors or filling small utility functions, and filling in function arguments. By accepting the suggestion, you can save a ton of time and frustration from typos. However, it’s not always syntactically or logically correct. You must double check its work. But even so, it’s a game changer for me, constantly, all day long.

Next, generating ideas. But (another gem from Simon) is to ask for tens of ideas; like 30 - 40. Most will be useless, but there will be a few gems in there. OR, more likely there will be a few that spur an actual good idea.

Then, I’ve used LLMs to help draft technical prompts. It’s writing that I don’t really need to be personal or “in my voice.” I really just needed some test. In fact, the LLM added some context and details that I wouldn’t have thought to add.

Help wriring small scripts or programs; especially using an API or package I’m not familiar with or use that often. The other day, I needed to read a CSV file, use it to fetch some related data, generate some SQL queries and write them to a different file. I don’t work with files day to day nor CSV, but I could quickly get a scaffold back from the LLM and fill in the actual “business logic” bits.

How have you been using LLMs?

One last note, I didn’t use any help with this post. :)

– Chris

when compared to those not using a calculator

One-liner to copy a Github PR link from the terminal

Wed, 04 Jan 2023 14:35:12 -0500

Here’s a super quick one-liner to copy the current branch’s PR URL to your clipboard (for posting into Slack, tickets, etc.)

$ gh pr view --json url --jq .url | <pbcopy | xclip -sel c>

Remember to replace the copy command with your platform’s specific command.

This would also make a good shell alias.

Also, also, I might try setting up Clipboard for multi-platform goodness.

Happy PR-ing!

– Chris

Web Scraping 2.0

Wed, 23 Jun 2021 06:21:51 -0400

With the wide adoption of robust front-end frameworks and CSS-in-JS, traditional methods of targeting xpath or CSS selectors for scraping data from web pages are becoming ineffective. However, this new paradigm of front-end development leaves a key opportunity to improve the web scraping experience.

Having data that’s not readily available in an API is something that I’ve kind of become obsessed with over the past year or so. Even with access to public datasets becoming more and more prevalant, I still wanted access to data from websites or companies that I use but have no interest in separating their data from their platform. So, how do you scrape a React or Angular or other front-end framework app/site? While scraping the data from the HTML may not be feasible or break constantly, you can use their API. The same API the front-end uses. However, often these private APIs are resistant to general use through opaque authentication and authorization mechanisms which make them more work than it’s worth to try and write a traditional API client.

Why not both?

Instead of an either/or approach, a hybrid approach can be used by leveraging browser automation and API requests to quickly and reliably get the data we need. Specifically, executing JS fetch requests using browser automation tools (almost all allow arbitrary JS execution in the current “page”). Parse this request back into the native automation language (typically JSON string returned from JS). Bam! You have a pseudo API!

I’ve been very successful in pulling large amounts of data from an actively scraper resistant environment using this approach. One of the really nice things is that authentication/authorization for the private API requests is handled virtually automatically by the browser through cookies.

Here’s a rough outline of the entire process:

Manually go through the web application performing the actions intended to be automated while watching the network dev tools tab.
Note any requests of interest.
Generalize those requests and reverse engineer a subset of the API.
Begin automation.
Automate authentication, if necessary (Type creds into login form).
Evaluate JS on the page to execute the AJAX/XHR/private API requests.
Parse the returned string into datatypes in your native automation language.
???
Profit!!?

My intuition leads me to believe there are some common patterns here which may be surfaceable in a library, but my experience isn’t quite to the point where I feel confident providing the right abstractions. But, stay tuned and we’ll see how this idea unfolds.

Let me know how your experience scraping data from JS heavy web apps goes! I’d love to know if this works for you or if this ends up being more hassle than it’s worth.

Happy scraping!

Magic PR Journeys

Mon, 22 Mar 2021 06:45:22 -0400

I was recently having a conversation about Pull Request reviews with a colleague of mine and I mentioned that when I’m ready to look at code for a PR, I review one commit at a time in order. After a confused pause, she asked how I could get the full context by only looking at one commit at a time. I supposed to her that perhaps the full context is not what facilitates a better review. Instead, we should follow the journey the developer who opened the PR has set out for us through their commits.

I have many more thoughts on this subject, but I’d like to hear yours too. Email me at chris@codegoalie.com and let me know what you think about the above supposition. Possible? Nonsensical? Outright dumb? :)

Here’s to your next magical PR journey!

— Chris

Commit Before Code

Fri, 26 Jun 2020 06:43:29 -0400

I propose a case for writing all of your commit messages for a feature, bug, pull request before writing a single line of code and afterward not adding any additional commits to your branch. Plan and execute. Give clarity and focus to your reviewers. Here’s how.

This is an idea that’s been floating around in my head for a while now: pre-planning commits in your branches. In fact, I checked my older posts because I thought there might be a pretty good chance I had written this post already. There wasn’t. So, here it is.

What am I talking about?

Making all your commits to complete your task before you begin coding. Literally using --allow-empty to outline all the tasks your branch needs to do before you write one line of code. Every commit after that of code changes are all fixup commits to one of the original commits. No new commits allowed.

Why would anyone do that?

I find the exercise of planning my attack in the form of concrete commits beneficial for clarity and focus.

First, I can think abstractly about the work to be done. I don’t know which lines of code will change or even what they will change into, but rather, I can focus solely on what those changes need to achieve.

I can think at a much higher level and coordinate the expected changes together without getting bogged down by the intricacies or specific implementation details. I don’t need to think about error handling, variable naming, design patterns, code smells, testability, etc. (unless any of those are core to the rationale of the branch). It’s almost a pure-er way of processing the task at hand. I focus only on what are the smallest set of steps to reach the goal.

Secondly, and often more importantly, I force myself to remain focused on the task at hand. If I make a change that doesn’t fit into any of the predetermined commits, there’s a very good chance that the change doesn’t belong in this branch and should be made elsewhere.

Most often these are housekeeping tasks; small improvements to the code that I happen to notice while doing something unrelated. Sometimes these are areas that don’t adhere to newly adopted best practices. Or, improvements in readability. These aren’t things that are critical, but help keep code quality up over time. I’ll get into what to do with these kinds of changes later.

Ok, but how can I do this?

Let’s take an example and build out our branch of empty commits. Imagine we’re working on an app to track vacations. We have an entity that holds our check-in and check-out dates. What we want now is to display the number of days and nights of the stay. For example, a stay from July 14th to 20th is 7 days/6 nights.

One thing to notice about this new feature is that the numbers are always one off from each other. We can calculate one and then derive the other.

I like to write my tests first (in one commit) and then get them to pass in another. I’ll also prefix my test-only commits with “Ensure”. Using this pattern, I’d want to write test for and then implement the number of days calculation. Then test and implement displaying to users. Here’s the commands to acheive this:

$ git commit --allow-empty -m "Ensure amount of days can be calculated from trips"
$ git commit --allow-empty -m "Calculate amount of days from trips"
$ git commit --allow-empty -m "Ensure users can see days/nights for a trip"
$ git commit --allow-empty -m "Display days/nights for trips"

Then after writing the tests for the calculation, I would create a fixup commit to autosquash into the first commit from above:

$ git commit --fixup HEAD~~~ # using a relative commit reference
# or with the sha1 (prefix)
$ git commit --fixup 06aa143

Learn more about git references.

When all is said and done (or even before that), you can have git clean things up for you with:

$ git rebase master --autosquash

What about those small, unrelated changes?

These are changes I, personally, love to make. I believe this is where the rubber meets the road in software as a craft. The holy trail of the huge rewrite is often so tempting, but if you can’t put new learnings or best practices into existing code then how can you think that you’ll be able to do it in a brand new system. If you can prove that you are a good enough developer to make the existing system its best, then you know you are ready to tackle a rewrite. However, you also won’t need to. ;)

So, what could be the harm of slipping in some minor improvements with your other work? I’m already in here. Let’s just fix it really quick. Well, this will get you into trouble in at least 2 ways.

First, if the rest of your branch becomes obsolete or a better solution arrives before you merge, the codebase doesn’t get to benefit from your unmerged, unrelated change. Or worse still, if your merged branch needs to be reverted, you will also be reverting your unrelated change.

Secondly, it has a very real chance to hold up your PR during review. There’s no better way to almost guarantee a flame war than think you can squeeze in a small, quick change. “Why is this fix here?” “If you’re fixing this, why didn’t you fix that?” “You should fix it in this way instead.”, etc. Your 2 second “improvement” will undoubedly attract comments and suggestions like moths to a flame. The actual point of the PR will be fine, but you’ll get stuck going around and around on your small addiiton.

If you can’t make these changes in your current branch, where do they go?

TL;DR Move them into another branch and open a separate PR.

There are 2 ways to do this. One uses the stash and one uses commits and cherry picking. I’ll start with stash.

Use the git-stash to move changes across branches. The stash is a temporary holding place for uncommitted changes. You can stash these changes on one branch and apply them to another. I’ll assume you’ve committed all changes with fixups and want to move all uncommitted changes to another branch.

$ git stash save
$ git checkout master
$ git checkout -b a-good-name-for-changes
$ git stash pop
$ git add .
$ git commit
$ git push

I always prefer git stash pop to git stash apply because the former removes the stash entry when applying it and keeps your stash clean. Once it’s applied and committed, you no longer need that stash and there’s no sense in keeping it around.

Now, for the second way. This way is a little less dangerous since you always have your changes committed, but maybe only by a hair. Here you’ll commit to your working branch. Then create a new branch and cherry pick that commit into the new branch. This technique also works will if your “small” fix has multiple commits.

$ git commit
[commit-before-code e4fe22f] Prefer single quoted strings
1 file changed, 3 insertions(+), 3 deletions(-)
$ git checkout master
$ git checkout -b string-style-fixes
$ git cherry-pick e4fe22f
$ git push

ProTip: cherry-pick takes any git commit reference. Utilize the branch name to cherry pick the tip of that branch into your current branch.

The really cool thing about this method is that if your fix branch gets merged before your feature branch, you can simply rebase on master (which you should be doing anyway) and git will detect the duplicate changes and automatically remove the cherry picked commit.

Next time you are starting to work on a new branch, give this method a try and see if it helps keep you clear on your objective and focused on nailing it.

Let me know how it works out for you.

Happy committing!

— Chris

Cleaning Up Docker Images

Mon, 30 Dec 2019 16:13:36 -0500

After a while of using docker locally for development, you may notice that you have a lot of images and containers laying around on your disk that you no longer use. This post shows you an effective way to clean up 50 of GB or more of disk space.

The first, best thing to do is clean up dangling images. These are docker images that are not referenced by a container. These can be orphaned when containers are removed or intermediate steps that don’t get cleaned up. One quick command can remove them:

$ docker image prune

Typically, if you’ve been using docker for a while, this will clear up quite a bit of space. But, there’s more we can do.

The next step you might think to do is remove other images that you know you don’t use. We can list them with the docker images command. However, you won’t be able to delete them directly because they are being referenced by containers (otherwise they would have been removed by the prune command). So, let’s look at containers:

$ docker ps -a

Note: the a flag shows all containers not just those running. You probably don’t want to remove a container that’s currently running anyway.

Wow. That’s a long list. Way down at the bottom I noticed some old and temporary projects and experiments. Here’s an example of some less notables from my list:

68585b76a752 46d00b64e6cb "/bin/sh -c yarn" 13 months ago Exited (1) 13 months ago upbeat_tesla
3cb298f62318 3bc06921fcd0 "/bin/sh -c yarn" 14 months ago Exited (1) 14 months ago charming_benz
31e91aaa9111 3bc06921fcd0 "/bin/sh -c yarn" 14 months ago Exited (1) 14 months ago upbeat_lovelace
79428b687b22 6533ab88e0ff "basj" 16 months ago Created welcomely_web_run_75

Ok, so now copy and paste the container ID (first column) for each one you… J/K.

I also noticed that this output has the created time in the fourth column. This is helpful, but not particularly useful. I do have some long running projects and would prefer not to simply remove containers that I’ve created long ago.

However, the very next column shows the last time the container was executed. I would like to remove containers that I haven’t run in a while. We can grep for rows of the output which exited many months ago:

 $ docker ps -a | grep -E "Exited \([0-9]{1,3}\) 2[0-3] months ago"

This looks scary. But, I promise I actually wrote this by hand and it’s pretty simple. If I’m being honest, it’s probably too simple, but it worked for my case.

First, we use the E flag to enable the regex engine. Then we want to match rows with the string like “Existed () <some number more than 20> months ago”. Most of that is plain strings. Really just for the numbers we want to use the regex. The first is the exit code in the parenthesis. Most containers had 0 or 1 exit codes. But, a few had 137. So I just matched on any up to 3 digit number within the parenthesis: [0-9]{1,3}.

Second, we match a plain 2 then any units digit of months between 0 and 3, inclusively: 2[0-3]. My results had the oldest container stopping 23 months ago. You mileage may vary and I’ll leave it up to you to modify and re-run to suit your situation.

Now that we’ve got a good list of containers to delete, let’s use some command-line fu to save a ton of copy/pasting. Let’s pipe this filtered list into awk to get rid of the rest of the text save for the first hash (field).

awk '{print $1}'

Then, pass each row (which now only contains the ID hash of the container) into docker rm. To do this, we can use xargs:

xargs -i@ docker rm @

Putting it all together:

$ docker ps -a | grep -E "Exited \([0-9]{1,3}\) 2[0-3] months ago" | awk '{print $1}' | xargs -i@ docker rm @

Then, go back to the top and remove dangiling images:

$ docker image prune

There you have it. I removed containers I hadn’t run for more than 8 months or so and freed up almost 50GB. I hope you have similar or better results! Let me know how you fare: chris@codegoalie.com

Happy docker rm-ing!

Buffalo: Render a Template to String

Wed, 06 Mar 2019 15:43:24 -0500

The Golang web app framework, Buffalo, has a very good templating system called Plush. It adds some nice features to the standard library templating specific to web applications such as partials and local context. It’s pretty intuitive if you’re coming from Rails.

While the default setup makes rendering a template as a response to a request super easy, using rendered template content elsewhere isn’t so obvious.

Once you see it though, it’s pretty straightforward to render a template to a string within your action handler functions. Typically, we’d render a template like this:

func homeHandler(c buffalo.Context) error {
return c.Render(200, r.HTML("index.html"))
}

Today I needed to return some HTML in a JSON resposne to a webhook to display some of our content within HelpScout for their dynamic app integration. Originally, I had a simple link to render which I did as:

render(200, r.JSON(
struct {
HTML string `json:"html"`
}{
HTML: fmt.Sprintf(`<a href="%s/users/%s">%s's profile</a>`, HOST, user.ID, user.FullName()),
}),
)

This barely worked for this case, but it did work. Then we wanted to add more info and I wasn’t willing to write HTML within a Sprintf call. There are three steps to render an arbitrary template to a string:

Allocate an io.Writer to hold the rendered template’s contents
Render the template with a Renderer.
Read the string from the Writer.

Here’s the code:

// allocate a buffer to render the template into
respBuf := &bytes.Buffer{}
// set any required values in the context (could be any `render.Data`)
c.Set("user", user)
// create the Renderer with the HTML method and call Render with the buffer and the necessary data
err = r.HTML("helpscout/sidebar.html", "layouts/zero.html").Render(respBuf, c.Data())
if err != nil {
err = errors.WithStack(err)
return err
}
// read the string out of the buffer (and in this case render it into a JSON response)
return c.Render(200, r.JSON(helpscoutResponse{HTML: respBuf.String()}))

That’s it. It’s pretty straightforward once you see it laid out. Hope this helps you (as I’m sure it will help me in 6 months).

– Chris

Service 'Objects' in Buffalo

Wed, 27 Feb 2019 05:09:02 -0500

For many recent projects, I’ve been using the excellent Buffalo web development eco-system. It’s a great collection of tools and packages for building web applications without needing to reinvent the wheel for each app. In this post, I’ll be highlighting a technique I use based on the service object design pattern of abstracting business logic from your applicaton implementation to increase readability and reusability.

But, why?

Here’s a scenario I find myself in with increasing frequency. I’ve just implemented some great new feature or process. It’s good to go and super helpful in its intended usage. All’s good.

Then, you deiscover another, slightly different, use for the same feature. For example, the system can send one-off reminder emails to individual users, but now we want to remind all users. Or, I’d like to manually kick off that background data integrity routine, but only for a single account and not the entire user base. Or, I’d like to queue this process up in a background worker, but only in certain cases.

There are proper extractions here for these exact scenarios. Typically, this is the extract function or method refactoring step. We will talk about that in a minute, but the first thing I want to address is the where does that abstraction live.

A place for everything and everything in its place, until

The universal answer to almost any question in programming and software design is “Sure, it could.” For a certain case, almost any imaginable specific solution approaches the ideal solution. Should this be a method on the model? Sure, it could. Wouldn’t this be better in the controller? Sure, it could. How about we extract this into its own repo and it will live in a completely separate open-source project? Sure, it could.

To combat this, I like to personify my functions, classes, and methods as super lazy beings. They are constantly asking me, “Do I have to do that?” and “What if I don’t want to know about all that?” I find this extremely helpful in following the single responsibility principle and avoiding scope creep within code constructs.

Imagine your functions as super lazy people that don’t want to do more than they absolutely have to.

For example, my models don’t like to send emails. My email sending functions don’t want to know about my user models, just email address strings. Route handlers don’t feel like learning SQL. You get the idea.

But (and there’s always a but), sometimes there isn’t a really good, immediately known place to put some piece of logic. This is where the service “object” pattern comes in. I almost think of it as a catch-all, last resort solution to design questins. Will a service work best for you? Sure, it could.

Ok what is it exactly

Hopefully by now, you’re on board with at least the idea of using a service to contain some kind of cross-cutting, or self-contained unit of logic. Specifically, in go in the Buffalo world, what is a service?

I put services as exported functions in a services package. The directory structure looks like this:

 actions/
mailers/
templates/
+ services/
+ notify_weekly_winner.go

And I implement one like this:

// service/notify_weekly_winner.go
package services
// NotifyWeeklyWinner sends a congratulations notification to a weekly winner
func NotifyWeeklyWinner(email, firstName string, dogsWalked int) error {
giftCardCode, _ := GetGiftCardForWalkingDogs(dogsWalked) // another service ;)

return mailers.SendWeeklyWinnerEmail(email, firstName, giftCardCode)
}

I know what you’re thinking: “Ok, this is a really simple example just to show the concept. A real service would be way more complex, right?” Sure, it could. However, I think even this short example is actually very exemplary. Let’s see an alternative implementation and back into how we ended up here.

Add to route handler (controller)

Probably the first implementation of this process was in a route handler function (or controller) somewhere. Maybe there’s a specific button in the interface to notify a weekly winner which maps to a handler which does some controller-y stuff (validates params, fetches data from DB, etc.), then has the two lines from the service above, then redirects to another path.

// actions/notifications.go

func notifyOfWinningHandler(c buffalo.Context) error {
// load some models

giftCardCode, _ := GetGiftCardForWalkingDogs(dogsWalked)
// actually handle error

err := mailers.SendWeeklyWinnerEmail(email, firstName, giftCardCode)
if err != nil {
// report error
 return errors.WithStack(err)
}
return c.Redirect(302, "/home")
}

Ok, actually seems reasonable. Could this be a better solution? Sure, it could.

So, time for a confession. I just made the above service example to show a super simple example. I gave no thought to the rest of the application around it or what it might even do. Now that we’re stuck with it, let’s take a second to think about the world we’ve created.

We know that every week we send a gift card to someone (with a first name and an email address) who has walked some dogs. The amount of the card may be based on the number of dogs walked. We’ve implemented some handler to send a notification with the gift card code to such a person.

We do have some known unknowns though. We don’t know what criteria constitutes a winner. We don’t know if there are one or multiple weekly winners. We don’t even know if this system has a User model. All that is a great thing! We don’t need to know any of those things to send an email. Our implementation doesn’t need (or want) to know it either.

However, I’m not happy. Based on this handler, it would seem that someone somewhere needs to click a link or button every week to notify the winner. We can do better.

Using a background worker

Let’s say we don’t trust humans and want to send the notification each week on an automated schedule. Let’s also say we have a background task system which does this. Then we’d simply extract the logic into a worker.

// actions/worker.go

var w worker.Worker
func init() {
_ = w.Register("notify_weekly_winner", func(args worker.Args) error {
// load from args, etc.

giftCardCode, _ := GetGiftCardForWalkingDogs(dogsWalked)
return mailers.SendWeeklyWinnerEmail(email, firstName, giftCardCode)
})
}

The nice thing here is that we can also keep our handler, if we need a manual fail-safe.

// actions/notifications.go

func notifyOfWinningHandler(c buffalo.Context) error {
// load some models

_ = w.Perform(worker.Job{
Handler: "notify_weekly_winner",
Args: worker.Args{
"firstName": firstName,
"email": email,
"dogsWalked": len(dogs),
},
})
return c.Redirect(302, "/home")
}

This is fine. It’s not very testable. Could it be justifed extracting to a service just for better tesability? Sure, it could. You might also have a great testing harness for background jobs and this isn’t an issue. However, I’m going to get a little hand wavy here with the specifics, but I find myself in the below type of situation somewhat regulary. I bet you do too.

There’s that random, one-off-that-becomes-semi-weekly, completely unanticipated feature request. The CEO needs wants to send a gift card to his personal dog walker who’s not in the system. We want to back-fill the past 6 months of winners that we tracked in a spreadsheet. This is where the rubber meets the road.

The hallmark of a well designed sytem is its ability to handle unexpected change.

Using our service, we can easily add a grift task to send arbitrary notifications.

// grifts/notify.go

var _ = grift.Namespace("noitfy", func() {
grift.Desc("winner", "Send a gift card for walking dogs")
grit.Add("winner", func(c *grift.Context) error {
email := c.Args[0]
firstName := c.Args[1]
dogsWalked := c.Args[2]
return service.NotifyWeeklyWinner(email, firstName, dogsWalked)
})
})

Or we can upload a CSV to send notifications.

// actions/upload.go

func uploadWinnersHandler(c buffalo.Context) error {
// read from CSV

for _, winner := range winners {
service.NotifyWeeklyWinner(winner.Email, winner.FirstName, winner.DogsWalked)
}
return c.Redirect(302, "/home")
}

Thank you for taking this journey into service “objects” with me. As always, please feel free to contact me with any questions, suggestions, criticisms, or thoughts at chris@codegoalie.com.

– Chris

Ruby refactoring: Open/Closed Principle

Fri, 12 Jan 2018 08:20:15 -0500

Let’s examine a refactoring to move code to follow the open/closed principle. We’ll see how the resulting calling code becomes much easier to read locally, without having to open other files or classes to understand what’s going on.

Open/closed principle

First, let’s do a quick recap of the open/closed principle. Classes should be open to extension, but closed to modification. What does that mean? Let’s take it a half at a time.

“Classes should be open to extension.” This means building your classes in a way that they are simple and single-use enough to be reusable or combine-able. This is the difference between building a Lego brick and a fully built spaceship. The spaceship is more complete, but when you’re tired of playing spaceships, you can’t easily reconfigure it into something else.

Second, “classes should be closed to modification.” Violating this principle will require you to modify far off code to implement your current feature. You should structure your code to avoid this because you may forget to change that far off code, other developers on the team may not know to change that code, and it’s often a mystery why this far off code is even related. More practically, this often manifests in a list of classes, types, “allowed” things, “blacklisted” stuff, etc. If you add a new class and then have to add that class to a list somewhere, you’re probably not following open/closed.

Sending update notifications

I recently had a project where we wanted to send some update notifications to users when entities they were interested in were updated by other users. This Rails application exposes a JSON API and has a bulk update mechanism. The controller action which handles these bulk requests is generic such that many types of models can be updated simultaneously. After successful updates, we schedule a background job to send the update notifications:

if entity.save
SendUpdateNotificationJob.perform_later(entity)
...
end

Initially we only sent updates for new or updated Posts:

class SendUpdateNotificationJob < ApplicationJob
def perform(entity)
return unless entity.class == Post
# send push notifications here
end
end

Adding more notification types

Seems reasonable enough. However, new and updated post notifications were such a hit that when we added responses to the application, people asked to get updates for them as well. Super easy to do that:

- return unless entity.class == Post
+ return unless entity.class == Post || entitty.class == Response

And then bulletins

- return unless entity.class == Post || entitty.class == Response
+ return if entity.class == Post || entitty.class == Response || entitty.class == Bulletin

Your level of pain tolerance may vary, but to me, this is already way out of control. Even though we aren’t expressing it as an array or list explicitly, we are building up a list of classes which we decided earler was a sign of violating open/closed.

Naming conditionals

So, how do we fix this? A really nice refactoring technique I like to use is what I call “naming conditionals” (super trademarked; not to be used without express written consent of Ricky Bobby, inc.). This is pretty easy to do. I take a conditional, extact it to a method and give it a name. Let’s do that here:

def ______?(entity)
entity.class == Post || entitty.class == Response || entitty.class == Bulletin
end

Now the hardest part of programming: naming this method. I like to keep it simple, so how about send_updates?? Now our calling code reads like this:

class SendUpdateNotificationJob < ApplicationJob
def perform(entity)
return unless send_updates?(entity)
# send push notifications here
end
def send_updates?(entity)
entity.class == Post || entitty.class == Response || entitty.class == Bulletin
end
end

The calling code is much easier to read: Don’t send updates unless you should.

But, we haven’t solved the open/closed problem yet.

Reverse dependency injection

One of my favorite discussions in programming is about who doesn’t care about things. In this case, I would say that this job does not care about which classes the entities are that it gets. It only cares about if it can send notifications for them, and then doing so. In this case, it would not check the classes of the entity.

If tho doesn’t know about the classes, then who does? Well, the entities know their classes. What if the job asked them if updates should be sent for themselves?

Then we’d have code like:

class SendUpdateNotificationJob < ApplicationJob
def perform(entity)
return unless entity.send_updates?
# send push notifications here
end

That seems pretty straightforward to me.

Quick aside: It might also be worth removing that conditional from the job and preforming the check before even scheduling the background job. This would allow you to send notifications for “unallowed” types if really necessary but put the gate keeping burden upon the scheduler of the job. Trade-offs…

Set a default, override when necessary

With Rails 5+, the concept of an ApplicationRecord was introduced to give a single parent class for your models within your application. This is a great place to add code or configuration to be shared by all models.

For most models, we don’t want to send updates:

class ApplicationRecord < ActiveRecord::Base
...
def send_updates?
false
end
end

Any class we want to send updates for can say so:

class Post
...
def send_updates?
true
end
end

Now, we don’t ever have to open the send updates job and add more classes to the list. When we add NewsArtlcles next month, it’ll be super straightforward to allow or disallow notifications for them. Further, we can add more logic in the send_updates? method; like only sending updates bulletin created after 2pm on a Sunday.

If you find yourself adding and removing from lists of classes in your application, take a moment to think about the open/closed principle and whether or not it can help you make more readable, modifiable, and exemplary code.

Happy open/closing!

– Chris

Poor Man's React Router

Sat, 03 Jun 2017 12:11:17 -0400

Recently at Videofruit, we had a fun internal exercise of building an app in single workday. That app ended up being an Email Service Provider picker where you answer a series of questions and we recommend an email service provider to meet your needs. In this post, we’ll breakdown a simplified version of that app focusing in on how we built a routing mechanism that is simple but effective and uses no external dependencies.

The ESP picker app essentially consists of 4 pages: the landing page, a question page, an email collection page, and the recommendation page. We’ve implemented each of these pages as their own component and will now demonstrate how we built a sub-system to decide which component to render and then renders it.

Surprise, surprise. In typical React fasion, we’ve implemented this component selection logic wthin yet another component. This PagePicker component exists near the root of the component hierarchy and has its own state which holds the current component to render. PagePicker also defines a set of callbacks to pass into each page in order to update that state and then render new components.

import React, { Component } from "react";
import LandingPage from "./LandingPage";
import QuestionPage from "./QuestionPage";
import RecommendationPage from "./RecommendationPage";
import questions from "./questions.js";
class PagePicker extends Component {
constructor(props) {
super(props);
this.state - {
currentPage: "LandingPage",
currentQuestion: 0
};
thisstartQuestions = this.startQuestions.bind(this);
this.goToRecommendationPage = this.goToRecommendationPage.bind(this);
}
startQuestions() {
this.setState({ currentPage: "QuestionPage", currentQuestion: 0 });
}
nextQuestion() {
this.setState(prevState => {
let nextQuestion= prevState.currentQuestion + 1;
if (nextQuestion >= questions.length) {
return { currentPage: "RecommendationPage" };
} else {
return { currentQuestion: nextQuestion };
}
});
}
calculateRecommendation() {
// Your logic here
return "super-awesome-recommendation";
}
render() {
switch (this.state.currentPage) {
case "LandingPage":
return (
<LandingPage
startQuestions={this.startQuestions}
/>
);
break;
case "QuestionPage":
return (
<QuestionPage
question={questions[this.state.currentQuestion]}
nextQuestion={this.nextQuestion}
/>
);
break;
case "RecommendationPage":
return (
<RecommendationPage
recommendation={this.calculateRecommendation()}
/>
);
break;
}
}
}

As you can see, this approach is pretty straight forward and, most importantly, it requires no third-party dependencies. As with any implementation, there are caveats and trade-offs. Firstly, this approach does not work with an app where you expect the number of pages to grow large. The switch statement can easily get unweildy and hard to maintain.

If your app has pages which need to communicate to eachother directly, this approach could cause problems. Our app is very linear and each page is very separate in both state and function. We take advantage of that in this technique.

Overall, we’re pleased with the declarative layout of this approach and that the current page being shown is clearly defined in the state. Any of the transition callbacks can contain more complex logic which would be fuctionally encapsulated and easily testable. We hope you find some value in this technique too.

Happy routing!

– Chris

How shell scripting saves my side projects

Tue, 19 Jul 2016 17:58:03 -0400

I work on a lot of little side projects. Many of them are short-lived and small scale. I start them to learn a new technology, to learn a new concept, try an idea, or just to see if I can actually build something cool. I should also mention that my memory is terrible. I “remember” things by figuring them out again. Rote memorization is basically non-existent to me. I’ve recently come up with a new best practice for my side projects which has saved my butt several times now.

It’s a deploy.sh script and I put one in every project that I do which needs to be built, deployed, or executed beyond the typical compile step. This very blog has one. There are also a few best practices I’ve developed for making this file useful and not just another headache.

Runable

This seems like a no-brainer, but hear me out. This file should be executable on its own. It should not be deploy.rb. It should not need to be compiled or otherwise prepared to run. You are writing a script. Script away all of that tedium. If you need to deploy with ruby, make your deploy.sh simply contain ruby deploy.rb. The point here is that you can cd into your project and run this script. Period. If you have to remember to initialize your Python virtualenv first, you’ve missed the point. Set it and forget it.

Helpful

Not everything can be scripted away. Sometimes a deploy requires some unique information, such as an environment or version number. Every microservice I create is meant to run in a docker container on kubernetes. In order to automate building, tagging, pushing the image, and then updating the deployment, the deploy script needs to know the version number. Since I’ve deployed 10 times today to “fix” that one bug, I can easily remember to pass the version to the script: ./deploy.sh v1.0.1337. However, I won’t be so hip to that fact tomorrow. Good thing we are writing a script. Just add some checks and helpful error messages to guide your future self down the right path. Here’s an easy one for requiring an argument to your script:

#!/bin/bash
if [ -z "$1" ]
then
echo "Please provide a version number, like v35"
exit 1
fi
# other deploy code here

It may seem like a drag, but you’ll be so happy that you took the time to take care of your own self now.

Complete

Lastly, your deploy script needs to do every step on its own. It’s a grown-up script now and can handle it. If you need to do just one more thing after the deploy finishes, you might as well just rm deploy.sh. Even steps which you might consider “optional” now, need to be included. For example, this blog is hosted on AWS S3 and CDN’ed by CloudFront. It really is enough to only upload (read: sync; save those bits) the files. Eventually the cache will expire and the new content will be shown. However, in just one more line in the deploy.sh, I can invalidate the CloudFront distribution and see this awesome new post right away. If you think it’s optional, it’s required.

Well, those are my three tips for keeping your side project deployments sane.

Happy deploying!!

P.S. I’ve recently had a resurgence of shell scripting in general. In the past, I’ve certainly been one to (ab)use zsh’s history substring search to re-run the same set of commands over and again. See also: “Up 12 times then enter. 6 times.” By committing those mini-scripts to ~/bin/, it really has made me much more efficient. A practical example is streamlining my workflow at work. We use Github for code hosting and reviews. We also use YouTrack for our project management. I wrote a command line client for YouTrack and combined all the commands I was doing to checkout branches by story number or create new branches as well as marking stories In Progress/Under Review/etc. into shell scripts like review_story c-1234 or

$ start_story -t bug -s c-1234 -b this-is-a-heisenbug
Applying "Assignee: me In Progress" to story c-1234
Switched to new branch 'bugs/c-1234-this-is-a-heisenbug'

Get creative and really push yourself to save some time. Again, you’ll thank yourself, and then me. ;)

Git References

Wed, 06 Apr 2016 23:04:17 -0400

Everyone knows how to checkout a branch, git checkout master, or how to look at a specific commit, git show a1c8b6d. But few know that branches, tags, and commit SHAs are typically interchangeable in these and many other git commands. Plus there is a whole set of modifiers to reference commits near to specific commits. In this post, we’ll explore all these techniques to become expert git referencers.

References

Below we will explore the different ways we can express a commit in git. While each technique has its own strengths and weaknesses, being able to wield a lot of git’s power and saving a lot of time comes from using these references interchangeably.

SHA1s

This first and most recognizable way to reference a commit is by its SHA1. This is the full or part of the long random looking string of letters and numbers. These are generated based on the contents of the commit, including current state of the repo’s files, commit message, author and committer, and timestamp. If any of those items change, a new SHA1 will be generated. Here are some examples of git commands using the SHA1.

git show abc123
git cherry-pick 321bca
git reset bbbaa4

Branch Names

Branch names are also something that are a big part of a git workflow. Under the hood, branch names are just aliases for commit SHA1s. Literally, you can see the branch names in the .git/refs/heads directory of your repo. Each file there contains one line which has the SHA1 of the commit it references. Here are some common usages of branch names:

git checkout master
git rebase stable
git merge features/new-hottness
git cherry-pick bugs/broken-email-form

An important aspect of branch names is the commit they reference can change. For example, when you create a new commit the branch which you are currently on automatically gets updated to the new SHA1.

HEAD

The HEAD commit is the current tip of the branch you are on. Similar to the pwd program, HEAD tells you where you are. On its own, the useful command is show which will show you the details of the last commit you made.

Modifiers

git also allows you to specify commits by their relation to other commits. In the git data structure the only explicit relationship between commits is child-parent. That is, the child commit knows its parent, but the parent does not know its child/children. Many of the commands which show or span multiple commits, such as log or diff, actually traverse the git tree to aggregate their output.

First Parent

The ~ modifier follows a references first parent. For example, HEAD~ references the first parent of the current commit. The first parent modifier can be repeated: HEAD~~ references the first parent of the first parent of the current commit. Also, a number can be specified for many generations: HEAD~5 follows ancestors back 5 generations. Here are some other uses of the first parent modifier:

// Move the second to last commit into the current branch
git cherry-pick bugs/broken-email-form~
// Modify some of the last 5 commits on your branch
git rebase HEAD~6
// Mark your current changes to fixup into two commits ago
git commit -a --fixup HEAD~

Other Parents

Through merges, a commit can have multiple parents. The last modifier of the day specifies any parent of a commit. The caret, ^, modifies the reference to follow its first parent; the same as ‘~’. However, specifying a number along with the caret follows subsequent parent commits. For example, reference the last commit of a branch before it was merged into the current branch with HEAD^2.

Bringing it all together

Any of the above references can be combined with any number of modifiers to express the exact commit succinctly. Here are some examples:

// See the 3rd to last commit from the last branch merged to the stable branch
git show stable^2~~~
// Pull second to last commit from version 2 into your current branch
git cherry-pick v2.0.0~
// Create a new branch 2 commits back from a known SHA1
git checkout -b branch-from-history abcd122~~

The above examples might seem extremely specific, but knowing and using these techniques can turn wasted time digging through git logs or copy and pasting SHA1s into a quick one-liner. Impress your friends and co-workers, save your own time, and begin thinking of your git history relatively.

Happy referencing!!

– Chris

Working with Containers: beyond the tutorial

Thu, 04 Feb 2016 15:11:50 -0500

Over the past few months, I’ve been getting my hands dirty with containers, Docker, and Kubernetes in an effort to get some hands on experience working toward microservices. I’ve been building up a small application to generate and serve Sudoku puzzles. My original goal was to see how many puzzles with unique solution I could actually find, but that’s another post entirely. There are already many great tutorials about Docker and Kubernetes on Google Container Engine, so I won’t go into getting started details here. This post is meant to explain some of the stickier points that I had to pickup the hard way. I hope to save you one or two headaches.

Turn Key Kubernetes

Typically, I’ve used AWS for my infrastructure needs, but Google Container Engine runs kubernetes for you and can spin up a new cluster with one command. Again, there are many tutorials about the specifics but here are a few things I didn’t know going in which would have been helpful.

Keeping costs down

By default your cluster is composed of 3 n1-standard-1 instances. When just getting started this is probably more than you need in both resources and cost. Since I was more interested in playing with containers than messing with compute engine instances (that’s the whole point of containers), I didn’t pay any attention to them, but you should. It’s super easy and quick to change both the number of instances and the instance types in your cluster.

Google Container Engine clusters are built on the idea of instance groups, which are just one or more instance defined by the same instance template. The template described the instances you want in your group with attributes like machine type and base image, etc. In the Google Cloud Console, you can design a new template right in the interface (pick a new machine type from a drop down) and use smaller or larger instances.

Scaling the number of instances in your cluster is even easier. Edit the Instance Group and change the “Number of Instances” textbox value. You can even enter 0 to stop getting billed for the instance time. This works great when you don’t need the instances to be up 24/7 while just playing around.

Persistent Storage

When a pod is restarted or rescheduled, it gets recreated from scratch. Anything that was stored on disk is gone. This can cause unexpected issues. For example, I was using redis for storing Sudoku puzzles and if the redis-master pod restarted, I lost all contents of the database. This can be unexpected because when running redis locally, the disk snapshots always remain even when restarting the redis-server process. With containers, this is not the case. If you need to store anything to disk that you care about, you’ll need to store that in a volume backed by a Google Persistent Disk. Once again, there are many tutorials on the specifics.

Stuck in CreatingContainer

When using a Google persistent disks, I was running into pods getting rescheduled onto new nodes (instance), but the disk remained attached to the previous node. Since Google Cloud persistent disks can only be attached to one node at a time, this caused the pod to hang during creation and never start. The quick fix to this is just detach the disk from it’s current node. Kubernetes will attach it to the correct node for you.

gcloud compute instances detach-disk <instance-name> --disk <disk-name>

There’s currently an effort to redesign the volumne mount/dismount system in Kubernetes to address this issue. I also think that as redis was maxing out the memory on the nodes and crashing, it was causing unnecessary havok on my cluster.

“Restarting” a pod

Early on my Sudoku generator was very naive and had exponention performance, so it would get “stuck” trying to come up with new puzzles. Locally, I was just restarting the process. But when I got it into the cluster, it wasn’t always clear if it was broken, or just getting held up (I did fix this and went from a puzzle every few minutes to a few puzzles a second). The great feature of using replication controllers is that it will keep the specified number of pods running. So to “restart” a pod that’s being managed by a replication controller, just delete it. The replication controller will bring up a new one. If you aren’t using a replication controller to manage your pods, do so.

Hopefully these tips will make your container journey a little smoother.

New blog; New identity

Thu, 04 Feb 2016 14:52:18 -0500

You may have noticed a few changes around here. First, a new domain name: codegoalie.com. I am moving away from ChrisMar035 in favor os something, as I said on Twitter, that “better show my love of programming and blocking vulcanized rubber disks with my body.” Also, my last name is not Mar.

Along with Twitter, I’ll be moving other usernames to codegoalie shortly. I already snagged codegoalie@gmail.com. Either way, if you see codegoalie or chrismar035 anywhere, it’s probably me.

Update 2016-07027: I updated my Github username to @codegoalie. If you’ve cloned some public repos or were following me, change those links. I’ve also re-registered @chrismar035 again and will be updating repos with links to the new corresponding ones.

Useful command line expansions

Mon, 25 Aug 2014 00:00:00 +0000

Here’s a quick trick I’ve been using for a while, but have found that not many others know about it. It’s a way to grab the arguments from the last command you executed. Command line expansion allows you to do many things, but today we’ll focus on just argument expansion.

First to simply repeat the last command with !!. This is most helpful when needing to prefix a command; with something like sudo or bundle exec.

$ mv /really/long/path/with/more/dirs/file /another/deeply/nested/dir/structure
mv: cannot move `...' to `...': Permission denied
$ sudo !!

Learn more about bang-bang on episode 32 of sysadmin casts.

Expanding on this concept, how about only part of the previous command? The !$ history expansion expands to the last argument (or token) from the previous command. Working off the previous example, if we now want to edit that freshly moved file:

$ vim !$

which expands to

$ vim /another/deeply/nested/dir/structure

Quick and easy! There are many ways to use history expansion to get parts of the previous (and others from history) command. !$ is by far the one I use the most. You’ll do yourself a huge favor to commit that to muscle memory.

To get an argument other than the last, you can use !:x where x is the (0 based) index of the arguemnt, or a range, or * to get all the arguments.

$ vim /another/deeply/nested/dir/structure
$ git checkout -b features/laser-sharks
!:2 => -b
!:0 => git
!:$ => features/laser-sharks
!:^ => checkout
!:1-2 => checkout -b
!:-2 => git checkout -b
!:2-$ => -b features/laser-sharks
!:* => checkout -b features/laser-sharks

ZSH power tip: Tapping tab with any of these expansions in ZSH will expand them inline to preview before executing.

Happy history-ing!

– Chris

Delete multiple branches with git

Wed, 25 Jun 2014 00:00:00 +0000

When working on projects with multiple developers following a feature branching workflow, you can end up with a lot of branch laying around. I’ve been trying to optimize my branch cleanup workflow for a while now and had a big breakthrough recently.

UPDATE: I now use a shell script which has the meat of the git-sweep functionality. Thanks to @pengwynn for letting me steal that.

Before we get to that, I must mention git-sweep. It has been revolutionary to keeping branches (both locally and remote) clean. If you care at all about the output of git branch, install that now.

However, there’s another simpler trick I just recently discovered. Deleting multiple local branches with one command.

$ git branch -D old-branch-1 old-branch-2

That’s it! Simply pass as many branches into branch -D as you want and git will gladly delete them all.

But why do you need this?

I just said that git-sweep cleans up local and remote branches after they are merged into master. So, why would we ever need to delete local branches? For one, you might not merge branches into master, but still want to delete them. But more likely, you’ll need to do this after pulling down a colleague’s branch to review. Once you give them the green light, it’s best practice for them to rebase onto master (interactively to prevent accidentally merging !fixup commits). However, you will have one state of the branch locally, but they’ve merged a different, rebased state. This is just enough of a change to fool git-sweep, and git herself, into missing the fact that these branches are the same. Luckily, there is a quick way to identify and remove those branches too.

Handily, git-branch has a super verbose mode which can be shown with the “double v” flag (-vv). This displays the branch name, HEAD commit, remote tracking branch, and HEAD commit message.

$ git branch -vv
* gh-pages 123456 [origin/gh-pages] Show new features on the project website
master 654321 [origin/master] Merge features/super-new-feature into master

Additionally, super verbose mode will also show us when a remote tracking branch disappears (is deleted). Immediately following the tracking branch name, we’ll notice a “gone”.

$ git branch -vv
* master 654321 [origin/master] Merge features/super-new-feature into master
old-feature 523621 [origin/old-feature: gone] Users can see the brand new feature

I like to grep for that and filter out the branches which I don’t want to delete. Basically, my whole cleanup procedure goes like this:

$ git branch -vv | grep gone]
old-feature-1 523621 [origin/old-feature-1: gone] Link to the new account page
old-feature-2 856956 [origin/old-feature-2: gone] Toggle turtle visibility with feature flag
$ git branch -D old-feature-1 old-feature-2
Deleted branch old-feature-1 (was 523621).
Deleted branch old-feature-2 (was 856956).

And there you have it, removing multiple branches in git with a bonus of finding branches who’ve lost their remote tracks.

Happy branching!!

– Chris

Go Concurrency by Example

Sat, 14 Jun 2014 00:00:00 +0000

Recently I’ve been dabling in Go. I was lured in by the lore of a purposefully simple language with amazing concurrency features. While that is true, I have to admit that I stayed around for the documentation. Golang.org has a this great Tour which walks you through the basics of the language through interactive examples and exercises. Secondly, the godoc command has a --server flag which spins up essentially a local copy of golang.org but the docs pages also include any locale packages you have pulled down, or that you’ve developed. Ok, enough gushing about Go documentation.

The end of the tour talks about concurrency and has some exercises to help you learn by doing. I found this method very compelling and came up with two different methods for handling concurrency in Go; each suited for slightly different use cases.

Primer

If you aren’t familiar with concurrency in Go, this section is for you. I will touch on some key points which are necessary to understand the examples below. This is not an in-depth guide. I, myself, am still learning the ropes, but this should cover us enough.

The idomatic way to handle concurrency in Go is through something called a goroutine. A goroutine is something like a lightweight thread. They are cheap to create and are ideal for small little jobs. Think of using goroutines as if to say “Someone else go do this real quick.” A thread will go do your bidding and your program can continue to execute.

However, what about results from a goroutine? Many uses of concurrency involve a collection phase where results from the concurrent tasks are reduced down to a final result. This is where channels come in. Think of a channel like a typed Unix pipe. Data written in at one end can be read from the other end.

Say we wanted to add all the digits from one to ten. We could let two goroutines work on the first and half last each, then combine the individual results.

func summer(start int, end int, c chan int) {
sum := 0
for i := start; i <= end; i++ {
sum += i
}
c <- sum
}
func main() {
c := make(chan int)
go summer(1, 5, c)
go summer(6, 10, c)
// read the values from the channel
first_half := <- c
last_half := <- c
fmt.Println(first_half, last_half, first_half + last_half)
}

Worker driven concurrency

Concurrency with a known amount of return values, like the above example, is fairly straightforward, but also quite contrived. A (slightly) more realistic example would include a case where the exact number of values to be sent through a channel is unknown. Reads from channels will wait until a value is available (similarly writes also wait for an empty space in a channel). The range function will wait-read on a channel until the channel is closed. Closing a channel should only be done on the writing end as trying to write to a closed channel will panic. This wait-read cycle allows us to setup a loop waiting for values on a channel and break out of the loop once the channel is closed.

In the example below we are calculating the decimal part of division in a goroutine with the decimal_divide function. Since we won’t know exactly how many digits the decimal result will be we can wait on the digits and print them as we get them. Once decimal_divide completes the division, it signals main by closing the channel.

func decimal_divide(numerator, denominator int, c chan int) {
// do long division "by hand"
for ; numerator != 0; numerator = numerator % denominator {
numerator = numerator * 10
// send the next calculated digit back to main
c <- numerator / denominator
}
// all done; close the channel
close(c)
}
func main() {
numerator := 22
denominator := 7
// print the integer part of the result
fmt.Print(numerator / denominator, ".")
// calculate the decimal part digit by digit
c := make(chan int)
go decimal_divide(numerator, denominator, c)
for digit := range c {
fmt.Print(digit)
}
fmt.Print("\n")
}

Multi-worker concurrency

In the example above, is fairly “linear” since the input of the next step is the output of the previous. It isn’t straightforward to add additional working in that case. However, let’s look at a “fan out” style of problem where we won’t know the number of workers, nor the number of expected values. These kinds of problems can be thought of as graph traversal problems.

Suppose we had a list of cities each with their own list of immediately neighboring cities which were connected by roads.

type cityList map[string][]string
var list = cityList{
"Cleveland":
[]string{"Columbus",
"Toledo",
"Pittsburg"},
"Columbus":
[]string{"Cincinati",
"Cleveland",
"Pittsburg",
"Indinapolis"},
"Toledo":
[]string{"Detroit",
"Columbus"},
"London":
[]string{"Bristol",
"Sheffield"},
"Sheffield":
[]string{"London",
"Bristol",
"Liverpool"}
}

Using this information and a starting city, you could generate a list all the cities which could be visited by car. Each time a city’s neighbors are found, a new goroutine can be used to fetch each neighbor’s neighbors. Since, we won’t know, in advance, how many neighbors each city has or how many neighbors each neighbor will have. We’ll have to do some housekeeping of on our own to make sure everyone is accounted. Further, after we see a city for the first time, there’s no reason to re-visit its neighbors again (and avoid graph loops).

func main() {
startingCity := "Cleveland";
// keep track of cities we've visited to avoid infinite loops
fetchedCities := map[string]bool{startingCity: true}
// pass the neighbors after fetching
c := make(chan []string)
var fetchNeighbors = func(c chan []string, city string, list cityList) {
if neighbors, ok := list[city]; ok {
// if we've found the city on the map, send back the neighbors
c <- neighbors;
} else {
// otherwise send no neighbors back
// could be a good place for an error message, etc.
c <- []string{};
}
}
// track how many cities' neighbors are still to be fetched
// start at one because of the starting city
stillToFetch := 1;
// go fetch the first set of neighbors
go fetchNeighbors(c, startingCity, cityMap);
// while we are still fetching neighbors
for stillToFetch > 0 {
// read the neighbors from the channel
neighbors := <- c
// track that we've fetched a set of neighbors
stillToFetch--;
for _, city := range neighbors {
// if we haven't seen this city before
if !fetchedCities[city] {
// mark it as seen
fetchedCities[city] = true
// now we have one more city to fetch
stillToFetch ++
// fetch
go fetchNeighbors(c, city, cityMap);
// print out the name of the neighbor as visitable
fmt.Println(city);
}
}
}
}

As you can see we’ve had to use the stillToFetch variable to keep track of how many sets of neighbors to read from the channel. Each time we get a new set of neighbors, we can subtract one to say that we’ve fetched neighbors. And on the flip side, each new neighbor that we haven’t seen yet, we add one to say that we have another set of neighbors to fetch.

These examples show some basic techniques to work with concurrency in go and many more techniques exist to create performant, robust applications. For full, working examples please see the gist to accompany this post. With the knowledge above, I hope you are inspired to take advantage of the wonderful power amd simplicity of go concurrency. Please link to any such work in the comments below.

Android Development: first impressions

Fri, 08 Nov 2013 00:00:00 +0000

As a web application developer working in a company full of mobile app developers with a mobile focus and as a full time Android user, there’s a natural gravity in my universe toward mobile apps and the power of the app plus server backend paradigm. Naturally, I wanted to build my own app and really see what it is like on the other side of the API.

{% img left /images/android_first/car_icon.webp 150 150 Car Payment Calculaor %} As with any new platform, language or environment, I read up extensively on the Android ecosystem, poured over countless resources on design princiles and best practices–NOT. I jumped right in. Since I was shopping for a new car at the time, I whipped up a quick and dirty [Car Payment Calculator][6]. I mostly wanted to see the whole process of building an app and deploying it to the Anroid market. I must say the process is extremely simple and straightforward. However, the car payment app doesn’t really do anything…

A few form fields, a button and some quick math. No networking, no server-side, no authorization, no fun. So, I set out on my second app: a push notification service; something like [Boxcar][7] or [Notify My Android][9]. As an Android user with a tablet and a phone, it’s really annoying to get all my notifications twice. I could also scratch my own itch with some home automation notifications as well as external services. But most importantly, I knew this was something that I could actually build. This post isn’t about Push Something (that will come later), so enough about that.

I followed the [Google+ Sign-in for Android tutorial][8] and then the [GCM setup tutorial][5] and finally a great [ListView tutorial][4]. Now I know that each of these tutorials are meant to get you started and intentionally limit the scope to only the topic at hand, but I must admit the app I ended up with, while functional, seemed like a giant pile of spaghetti. I wasn’t sure why certain pieces of code went where, what purpose different arguments provided to many methods (is it the class context variable, getApplicationContext(), or am I passing the Activity with this?). And even at two activities, debugging was becoming a game of pick a file and scroll, scroll, scroll looking for the correct group of statements.

{% img right /images/android_first/uh_oh_android.jpg 400 200 Uh Oh Android %} Now, I am not complaining or blaming the tutorials here. This is exactly what my beloved Rails community does to new develoeprs as well. Dump all business logic into the controller. No No. Move it into the model. NO NO. Service objects or concerns and so on. Having been through that process and been at many stages simultaneously, I am familiar and comfortable with the evolution of a Rails app. However, being new to Android, I must admit I felt a bit lost and even considered the possibility that Android apps were forever doomed to be a mess of deeply nested ifs and try/catch blocks, 200+ character lines of code and methods with a staggaring list of arguments.

Thankfully, I am the only developer on my little side project and I had to take a break from Android land to do a little catchup work on the server side. As a budding Rails app itself, I found myself being able to see further ahead than ever before in my Rails code and allow myself the luxury of writing code which doesn’t make large design decisions too soon. Through experience, I am building a solid and extensible base on which to build the rest of the app. Whoopty doo. That’s my job 5 days a week. Then I thought, why not be able to utilize some basic OOP principles in my Java code too? These tutorials were (as most are) a starting point; a launch pad to get the juices flowing and enable the reader to build something truly awesome.

And that bings me to where I am at today, not rebuilding but refactoring. In the same way I would enfore single responsibility and avoid “just throw that code anywhere” in my Ruby code, I should do so in Java code as well. When I sat down to write this post, I meant to document some frustration with my first Android experiences and document my plan moving forward. But after getting this far in, I think I’m seeing something else and one of the reasons I started loving programming to begin with: there are always other ways to acheive the desired results. [Give 10 developers the same requirements][3] and you’ll get at least 10 different programs. Some good, some not as good. And that is the reality of software development. It’s easy to blame the language, or the documentation, or the framework for your bad code, but at the end of the day it’s YOUR code. Own it. Be proud of it. Don’t let language or framework limitations bring down your work. You can be as awesome as you choose to be. So don’t settle for ‘it works’; rest when ‘it’s aweosme!’ Your future self with thank your current self; instead of swearing at him. ;)

{% img /images/android_first/The-Time-to-Be-Awesome-is-Now-Kid-President1.jpg %}

[2]: [3]: http://exercism.io/ [4]: http://www.vogella.com/articles/AndroidListView/article.html [5]: http://developer.android.com/google/gcm/gs.html [6]: https://play.google.com/store/apps/details?id=com.chrismar035.carpaymentcalculator [7]: http://boxcar.io [8]: https://developers.google.com/+/mobile/android/sign-in [9]: http://www.notifymyandroid.com

Commit Message Starters

Tue, 14 May 2013 00:00:00 +0000

As I’ve been working in git and crafting my commits with rebase working toward creating pull requests which are easy to read, review, and comment, I’ve been noticing myself fall into habits of naming commit messages similarly which contain code to acheive similar ends. Further, when I struggle to find the correct words for a commit message, it’s often easier to being with a familiar word to get things flowing.

{% img right /images/readthesource.jpg 460 276 Read the source %} Over the past few months, I’ve also had the pleasure of working with some great developers. As such, I’ve been afforded the opportunity to review a lot of their code. If there is one thing I recommend to all developers, from novice to expert, it’s read code. Your own, your friend’s, strangers, projects you admire, anything you can get your hands on. Not only will it enlighten you to how others write code, new coding patterns and syntax, but it will empower you to be able to figure out how the code you use works and how to fix it.

As a quick aside, I was able recently to dig into the Ruby source code, even into the C, to discover exaclty what was happening with Range bracket operator access on an empty Array. It felt great to be able to see how Ruby herself was implemented and know exactly why my code worked; to go beyond “well, it worked in irb so it must be correct.”

The great thing about reviewing pull requests is that I am able to open discussions with the authors and inquire as to why they made certain decisions and what their motiviations were for writing certain code. Also, it gave me a chance to see code which was not polished and final. Thereby, recognizing anti-patterns and less than best practices.

Coming back to the topic of this post. These anti-patterns and less than best- practices extend to commit messages as well. Below I’ve listed some of the dos and don’ts which I have started to use in my everyday committing.

What I like

These words I often use to start commit messages which make some common changes in the codebase.

Ensure

Ensure phone numbers strip extraneous characters
Ensure deactivation resets users' api tokens

I use “Ensure” to start a commit which contains tests for my code. These commits are often the first commits in a pull request, especially if I am using TDD. A whole commit just for test? Surely, that’s overkill? There are two reasons I like to encapsulate tests into their own commits.

{% img left /images/ensure.jpg Ensure %} First, this standardization allows others to know which commits will define and test the behavior I aim to implement. We all know that tests can be a great documentation tool and prefixing commit messages which contain “Ensure” allow people to find them easier.

Second, if the tests are in their own commit(s) before the implementation commits, it’s very easy for a reviewer to simulate the red-green test cycle. Either through a checkout or selecting a commit to be ‘editied’ in an interactive rebase, the state of the repo after the tests are written, but before the task is implemented can be recreated. Tests are run at that point and a reviewer can cofirm that the new tests do indeed fail. The rebase is continued and the tests can be run again. This time they succeed. A review has very simply been able to confirm that the tests are working correctly.

Boyscout

Boyscout the user model specs
Boyscout line lengths in the sessions controller

Boyscouting refers to updating the format and/or style of code without changing the behavior. I picked it up from a presentation about code styles which I now cannot find to reference here. However, I did find this written reference online. The term refers to a famous quote by the founder of the Boyscouts, Robert Stephenson Smyth Baden-Powell, “Try and leave this world a little better than you found it”.

Continuing to think of the reviewer when committing, just by looking at the message, expectations are set that this commit does not change functionality, only formatting. Review of a change like that is much easier when you know that nothing is supposed to happen.

Migrate

Migrate a unique index for user email addresses
Migrate timestamp columns onto the comments table

As you might have guessed, I use “Migrate” to start commit messages which contain migrations. As a best practice, we like to isolate migrations to their own commits containing only the migration class definition and the changes to the schema file. It also can be useful, but not 100% reliable, to quickly see if a particular set of commits has migrations; say before a deploy or after pulling down the latest changes. Again, along the theme of setting expectations for others.

What I avoid

These words often start a commit message off on the wrong foot. Being very generic on their own, they often lead to very generic commit messages.

Fixed

Fixed broken specs
Fixed editing a user

{% img right /images/fixedit.jpg FIXED!!! %} First off, past tense should be avoided in commit messages. The rule of thumb is to write a commit message as if you were commanding the code to take an action. Secondly, commit messages should describe the reason for a change, not the change itself. If someone would like a know what the changes were, they need to do no more that a git show or click in github. However, to reason why the change was made or what it is attempting to do is much harder to determine from the code itself. Great commit messages can be the difference between Fixed broken specs and Activate users after registration in user specs.

Added

Added new Order model
Added full_name method to User model

“Added” is not strictly a not so best practice, but I see it go awry more often than I see it used for good. Again, “added” has a great tendnecy to lean toward what happened instead of why did it happen. I, myself, sometimes write a commit starting with “added”. For example, Added jquery javascript library at first glance seems perfectly reasonable. I didn’t fix anything. I didn’t write a great new feature. I just added jQuery. However, when I find myself with a commit message like this, I go through the practice of adding “in order to” and then trying to finish the commit message. Added jquery javascript library in order to provide DOM manipulation methods. Put it together for Provide DOM manipulation methods with jQuery. Now, I’ve given some context to why I added jQuery instead of simply restating the code changes in the commit message.

That’s all I have for now. Please comment below with your commit message best practices and tendencies. I bet you guys have some great tips too!

See ya soon and keep committing!

–Chris

Better pull requests with git --autosquash

Thu, 04 Apr 2013 00:00:00 +0000

UPDATE (Jun 7, 2013): I created an ascii.io video demo of autosquash for your viewing pleasure.

Pull requests are an extremly crucial part of our workflow at Mobile Defense. We pride ourselves on producing quality code and we do not subscribe to the git-blame “I didn’t write it” culture. We all take responsibility for all the code we produce, as a team. As such, being able to review our code efficiently and thoroughly before it gets into master is a goal we are constantly looking to improve upon.

A somewhat paradox in reviewing code arises when the need for reviewers to see the updates based on their feedback quickly and efficiently and for the developer to be able to ‘fixup’ their branch before merging to ensure the code enters master as cleanly as possible. That’s where git-rebase comes in.

Rebase is an extremely powerful tool in the git toolbelt. Combine commits, edit commits in-place, rewrite history, etc. It’s especially great for cleaning up your topic branches before submitting for code review. However, remembering which commits need to be squashed into which can be tricky especially when being the reviewee and the reviewer on multiple pull requests daily. Well, it turns out git already has a really clever, built-in way to organize and automate the branch cleanup process. It is perfectly suited for making changes based on pull request feedback. It’s called autosquash.

Here was a typical pull request workflow:

Code up a feature; commit early and often.
Push to github and open a pull request.
Do a quick self-review of my code.
Ask others to review my code.
Add temporary commits in response to comments.
When the branch is approved for merge, rebase and fixup the temporary commits into the original commits.
Merge the branch.

As you might have experienced or noted in the workflow above, how do you keep organized and remember which temporary commits should be squashed into which original commits? The problem I ran into time and time again was that as soon as I started an interactive rebase I would be unable to remember exactly which commits needed to be changed, squashed, or fixedup. So, I had been developing some commit message conventions for temporary commits.

For example, a commit which I knew was incomplete work or needed to be looked at again would have it’s message prefixed with an excalmation mark.

! Possible logic error in the complex_redirect logic

Further, commits which I knew I wanted to squash into previous commits would be prefixed with four hyphens and have a message which I would be able to make sense of when editing the rebase todo file.

---- into the first set of unit tests

This worked fairly well as long as I left myself good notes. But, surely there must be a better way!

Enter autosquash

The --autosquash flag on git-rebase is exactly what I was looking for. During an interactive rebase, it processes messages to stage squashes and fixups. These messages are setup with flags on git-commit at commit time, which is when you are most likely to know what you ultimately intend to do with commits. Further, it works via the commit message so it’s easy for humans to understand as well!

Say you have a feature branch with three commits: (viewing git logs with the incredibly awesome lg alias)

* cdaa049 (HEAD, features/new_hotness) Add new hotness to UI (66 seconds ago) <Chris Marshall>
* e14aa92 Implement new hotness backend (2 minutes ago) <Chris Marshall>
* 293070e Ensure new hotness works (2 minutes ago) <Chris Marshall>
* de72adc (origin/master, origin/HEAD, master) Add git internals meetup talk slides (6 months ago) <Chris Marshall>

Now let’s say you get a review comment on the commit for the tests (‘Ensure …') which asks to add a test for a missed edge case.

So we will code up our edge case fix and as we commit, we’ll add the flag to specify what we ultimately will do with the commit.

Git-commit has two such flags: --fixup and --squash with each of these we will specify the commit into which we want to fixup or squash. In our example, we want to fixup into the first commit after master: 293070e.

We can do that with:

$ git commit --fixup=293070e

Which will produce a commmit with a message of:

fixup! Ensure new hotness works

A commit message formatted this way tells git-rebase (with --autosquash) to fix the patch in this commit up into the commit with the message “Ensure new hotness works”. The power of using the commit message instead of SHA1 to identify commits is that this will work even after a rebase without --autosquash (e.g. pulling in lastest master changes).

Here’s how it would look in github:

Obviously, don’t have conversations with yourself on github. This is purely educational ;)

Now the fun part! When the pull request is approved, you can rebase with autosquash (and interactive) and watch as git does the work for you:

$ git rebase master -i --autosquash

Produces the rebase todo file:

pick 293070e Ensure new hotness works
fixup 3877fdf fixup! Ensure new hotness works
pick e14aa92 Implement new hotness backend
pick cdaa049 Add new hotness to UI

Simply save it and git rebases the branch as if you had changed the file yourself. You can then merge and have a beer for all your efforts!

Automating autosquash

Adding the long --autosquash flag all the time can get tiresome. Luckily, you can add the rebase.autosquash config value to run autosquash automatically for every interactive rebase.

$ git config --global rebase.autosquash true

If no commits with the fixup! or squash! syntax are found, the rebase continues normally. To override the autosquash default value, simply pass the --no-autosquash flag to rebase.

Happy squashing!!

–Chris

Oh my! What large hot fixes you have

Tue, 26 Feb 2013 00:00:00 +0000

We’ve all been there: just finished a nice new feature, deployed it out to production only to find that you’ve completely overlooked an edge case or use case which is now totally breaking your app. Sometimes you have to push a big hot fix. It has to get in immediately. It can’t wait for writing correct tests or other better abstractions and implementations. What to do? What to do?

Well, there’s the common way of hacking as fast as you can until you’re pretty sure the bug is fixed, commit everything in a big “Avert crisis!!” commit. However, I think we can do better.

First step checkout a new branch at your current ‘production,’ or similar. Call it production_pre_fix. Don’t check it out or work on it, just leave it there as a placeholder.

$ git checkout production
$ git branch production_pre_fix

Now for the fun part: hack away! But, follow your usual development practices of committing early and often. Refactor a method. Commit. Add some if checks for edge cases. Commit. Get the shit fixed and deployed. Go on. I’ll wait.

Ok, alarms are quiet now and you’re done breaking shit? Good. Now let’s get to work on the real fix. I’m glad your hacked together bunch of duct tape fix is holding, but let’s not leave things like that. I know you and I bet you’ve got some good work there. You probably don’t want to just throw it all away. That’s cool. Why don’t we use the duct tape as a basis for the permanent fix? Here’s how:

First, get a clean branch off master

$ git checkout master
$ git checkout -b bugs/prod_fix

Then, write tests to account for your new found bugs.

Next, cherry pick your current fix onto this branch.

$ git cherry-pick production_pre_fix..production

You may have noticed I passed two branch names with two periods in between to cherry-pick. What the fuck, right? Well, as we all know, two commit SHA1s separated by two periods is a range of commits. For exmple, 2e45f..34b79. You are saying that you’d like to use all the commits starting at 34b79 and following parent links back until you reach 2e45f.

But I’ve used brach names here, which are just human friendly shortcuts for SHA1s. Remember branch names are basically pointers to a particular commit. They can be used in any git command which would otherwise take a commit SHA1. Add that to the fact that cherry-pick can take a range and suddenly your doing some pretty powerful stuff with a pretty simple command. Go you!

Now a little housekeeping,

Make sure your fix actually passes your tests.
Refactor and make your fix nice. Rebase, as necessary.
Start pull request and get the feature merged back into master.

Then we need to reset your temp fix in production. Like this:

$ git checkout production
$ git reset --hard production_pre_fix
$ git branch -d production_pre_fix
$ git push -f

Woah woah woah! No, I didn’t. I just force pushed a non-topic branch. Isn’t that a big no-no? Usually, yes and I would expect to be taken out back and beaten until I learned my lesson. However, until I’m told of a good reason otherwise: production or other released branches are not shared among developers. They are an exact representation of what is to be deployed next to the environment they represent.

Why aren’t release branches an accurate representation of what is currently deployed to the environment they represent? Because if you want to know that, look at the environment. There’s no need for duplicate information. A branch like production should be curated and hold what will be deployed next to production. You might not do that curating and updating until moments before the deploy. But for all that time until you deploy, if you had to emergency deploy, whatever is in the branch will be deployed.

Back to business, we’ve reset production back to not include the duct tape. Now it’s time to merge master and our fix up to it.

$ git checkout production
$ git merge master

Congratulations! You’ve not only saved your company from sure collapse by fixing a huge production bug. But you’ve also reconciled your technical debt and prevented other developers behind you from cursing your name at the duct tape-y hacks you introduced to the code base. That deserves a beer!

– Chris

Zeus gem error: "panic: runtime error: invalid memory address or nil pointer dereference"

Tue, 12 Feb 2013 00:00:00 +0000

I was getting this error and the excellent Zeus rails environment loader gem was crashing making it a huge bummer to dev.

Luckily, I found a fix in one of the open issues on the project.

It was easy to fix, just run a bundle install to add the missing gems:

Easy as that. Hope this helps someone else!

– Chris

Help! I lost a commit from days ago!!!

Fri, 08 Feb 2013 00:00:00 +0000

Oh no! I’m screwed! Better try to remember what work I did and recreate it… :( J/K! This is git! Of course there’s a slick way to recover what you lost. Here’s my situation: I was working on a feature branch happily coding away. Finished the feature, rebased it quickly to fix a typo and then merged it into master. All’s well in my world. However, that was days ago and today I just realized that I accidentally removed one of my commits from the feature branch while I was working on it. No worries. Here’s how I got the commit back with relative ease.

First, let me give you a lay of the land. I use the incredible git lg alias.

lg = log --color --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset' --abbrev-commit --

Seriously, steal that now. Use it a thousand times a day. I wish I could attribute it to the original author, but I have since forgotten where I originally saw it. At any rate, every few weeks I see it posted anew on a different site.

Anyway, here’s where I was at before the recovery:

Well…now what? Redo all that work in the lost commit. Nah. Reflog to the rescue! Git’s reflog is a (by default) 90 day record of everywhere your HEAD ref was pointing. Anytime you do a checkout, create branch, commit, rebase; anytime the ‘current’ commit of your working directory changes, the reflog gets a new entry. You can think of the reflog as a record of what you’ve been doing. If you’ve had it in your working directory in the past 30 days, it’s in the reflog.

Remember what the commit message was for my lost commit? Nice! We can easily grep for it:

chrismar035:super_sweet_rails_app(master) $ git reflog | grep "commit message"

What’s that? I didn’t tell you the commit message for the lost commit? Well, I don’t remember either! Now we are screwed.

Gotcha again! We can still find it. We can use the ancient art of looking for what we lost. It’s easy. Simply do a git reflog and you will see a list of everywhere you’ve been:

AHA! Not only did you find the LOST COMMIT!!!, but we can also look at what happened too. Apparently, on a mad dash to get the feature merged before going out and getting my drink on with some chaps, I accidentally rebased my commit right out of the project. Damnit.

Oh well, at least you found it for me. Thanks. Now, we’ve only got one real choice to fix this. We have to cherry-pick the commit out of the reflog back into master. Because I’ve already merged my feature branch into master and pushed it, I cannot go back and rebase or otherwise edit that pushed history of master. I’ve got to bite the bullet and add a commit straight onto master… Everyone screws up once in a while.

In the reflog, there are two ways to reference the commit: the SHA1 and the HEAD@{integer} reference. Either one will work. I like the SHA1 because I can double click in the terminal and it gets correctly highlighted for copying. Copy it and we’ll paste it back into the command line with:

chrismar035:super_sweet_rails_app(master) $ git cherry-pick 9a7de0e

lg that again and see how we look:

Nice! Now, notice that the SHA1 changed from what you copied out of the reflog. Remember, commit SHA1s are based on their content (files, dirs, commit messages and meta data) AND their parent(s). So, if you change the parent of the commit, you change the commit (Same reason all the SHA1s change when you rebase). In reality, the commit you cherry-picked is duplicated on to the branch you cherry-pick it into. However, in our case that’s exactly what we wanted, because I messed up and deleted the commit in the first place.

Nice work! Push and have a beer!! :)

– Chris

SuperBowl Commercial Bingo Card Generator

Sun, 05 Feb 2012 00:00:00 +0000

Everybody loves the SuperBowl commercials! Last year, I hacked up a script to randomly generate items to create your own bingo cards. This year, I improved the script to actually generate the cards as a PDF file based on an excel sheet of bingo items.

Here it is:

Check out the second comment for more info.

– Chris

Ruby on Rails/ActiveSupport: To delegate or not to delegate

Fri, 12 Aug 2011 00:00:00 +0000

This is an account of my adventure with delegation from problem to apparent solution to bug in ActiveSupport to you’re doing it wrong!

TL;DR

Delegation is used when you want to forward a method call from on class/model to an associated class/model. The Ruby on Rails Guides use the example of a User model which has a Profile associated with it. This is a great example with a little subtlety which I overlooked. If you aren’t famaliar with delegation, go ahead and get yourself learned. I won’t wait for you though. This is text just come back when you’re up to speed.

# app/views/drivers/show.html.haml
.truck_number
-if @driver.truck
= @driver.truck.number
-else
Not Driving any Truck

NBD. Here’s our problem:

The Problem

As when any good application which acts as a font-end for a database, we track history on all edits, etc. So, when we update the Driver in a Truck, we want to add history for the Truck and the Driver. If the Driver is already in another Truck we want to show, in his history, that he moved from Truck 1001 into Truck 2002. We’ll say we have a create_history method included into our Entity classes whose declaration looks something like:

def create_history(attribute, prev_value, new_value)

So, we want to call something like:

@driver.create_history('Truck Number', @driver.truck.number, @new_truck.number)

Jackpot! Works perfectly. We get the old Truck number (1001) from the @driver instance and the new Truck number (2002) directly from the @truck instance. We are geniuses. Hold the phone though. What if that Driver didn’t have a previous truck… Spoiler Alert: NoMethodError: undefined method ‘number’ for nil:NilClass I guess we could check that a Truck exists each time we want to access any attributes for Driver.truck. But, really, what do we expect Driver.truck.number to be if the Driver is not in a Truck? Nil would work…

Delegate: for science!

I’ll allow it. Let’s delegate and allow_nil:

# app/models/driver.rb
class Driver < ActiveRecord::Model
delegate :number, :to => :truck, :prefix => true, :allow_nil => true
# ...
end

Works! Now, we’ve got history showing: Driver had no Truck and moved into Truck 2002. Geniusness times 2. Delegation to perferction.

Sometime later…

Instead of the Truck number, maybe we want to store the id of the Truck in the history. Perhaps we know the truck number may change over time and want to link up the correct Truck record. Seems easy:

delegate :id, to: :truck, prefix: true, allow_nil: true

But, when our Driver doesn’t have a ‘previous’ truck:

RuntimeError - Called id for nil, which would mistakenly be 4 -- if you really wanted the id of nil, use object_id

WTF? I allowed nil. I know he doesn’t have a previous truck. I just want my nil back and I’ll go about my business. Let’s get our hard hats on and get digging into the source. I’ll save you some googling and just tell you that the Delegation module is in ActiveSupport in the Core Extensions under Module. Scroll, scroll, scroll. There! Line 136! They(we)(whomever) are only rescuing from NoMethodErrors. Calling id on nil raises a RuntimeError, which isn’t rescued and bubbles up to us. This is me stomping my feet and wining. My knee jerk reaction was to just rescue RuntimeErrors too:

rescue NoMethodError, RuntimeError

We’re gonna be Famous

It works! AND, how cool are we? We get to submit a bug fix to Rails. Uber-cool. Uh-oh, uneasyness…crap… It just doesn’t feel right to just catch any old RuntimeError and possibly, silently fail to nil. However, the urge to try and submit a simple ‘fix’ to Rails was strong. Instead, I started doing more research into delegate and Ruby Exception handling. Here’s what I’ve come up with:

The Moral of the Story

You should delegate when you want to hide an architectural aspect and expose the expected abstraction. In the Ruby on Rails Guides example, we expect a user to have a name through @user.name even through name actually resided in Profile. However, in our example, a Driver isn’t expected to have a truck_id. A truck has an id, but a driver does not have a truck_id.

One of the classic ‘bad smells’ in code for refactoring is using a temporary variable in place a a query. Ruby and its optional parantheses, blurs the lines between variable, attribute and method. It’s easy to forget what you are doing and assume everything is an attribute. Why shouldn’t a Driver have a truck_id? The same reason a Truck doesn’t have a CDL. But, we can query the truck_id for the kTruck associated with a Driver with a method.

After all that, I should have just written a query method in Driver as such:

def truck_id
self.truck.id if self.truck
end

Too easy…

UPDATE 3/16/2013

As I convert this blog to Octopress and reread these posts, I’m see some imporvements. Instead of the code above, the try method is really what we want to use here:

 def truck_id
self.truck.try(:id)
end

On Object, try will attempt to send the parameter as a message to the invoking object. So, it works just like calling the method. However, on nil, try simply returns nil. It effectively exactly replaces the first implementation nicely.

– Chris

Ruby Auto-Responder for Postfix/Vmail

Tue, 15 Feb 2011 00:00:00 +0000

Not knowing perl, I set out to write my own script in Ruby to send auto-responses from e-mail addresses setup in a vmail folder structure. You don’t need to use postfix, just have the ‘new’ folder where new messages are stored.

I recently moved our company’s e-mail server to a new VPS on slicehost and started using postfix. I followed this great tutorial on How-ToForge (except for the squirrelMail part) to setup a mysql database with the vmail folders to store the mail. I couldn’t get the Autoresponder in the tutorial to function correctly. Actually, it was ‘eating’ random mail, which was a weeks worth of headache in itself. So, we rolled out without any auto-response capabilities.

I’ve been half-heartedly looking for another solution every few weeks, but haven’t found anything other than a few Perl scripts (I’m not really famaliar with perl). So, yesterday I decided to write a ruby script to send auto-responses. I wanted to use text files to hold each autoresponse. Then, I’d just move them int and out of a folder to activate and deactivate them. I also took advantage of the ‘new’ folder within the vmail structure to reply to mail newer than the autoresponder config file.

The format of the config files is:

The message can be multiple lines until the end of the file.

For each config file in the directory, the script parses the file for the above information. Then it will check that user’s new mail folder for files. Any new mail file which was modified (sent) after the modification date of the config file is parsed to get the sender’s address(es)*. A new e-mail is composed using benprew’s pony gem and sent to each of the senders. Finally, the script ‘touches’ the config file to move the modification date newer than the messages for which it just responded.

The nifty parts of the script are iterating over a directory listing of files:

and using the stat method of a file object to get the modified dateTime:

and lastly, using the FileUtils module to ‘touch’ a file:

Here’s the full source:

– Chris

Trick out your Vim

Fri, 25 Jun 2010 00:00:00 +0000

Today, I’m going to add some plugins and scripts to make better use of Vim. I have not started this project and I will be updating this post throughout the day with my finds and experiences.

I generally use Vim to code web applications. My site at work run on PHP and at home I’m working with Ruby on Rails. So, I’ll be looking for something to specifically help with that, but who knows what I’ll find.

.vimrc

.vimrc is a file in your home directory. It contains a script which runs each time you start Vim. You can setup your configurations, preferences and plugins in here. The .vimrc is the starting point for fine tuning your Vim to exactly how you want it. If you are a beginner, I would recommend finding some more advanced users' .vimrc’s and, at least, check them out, if not start using them for yourself. I started with Todd Werth’s .vimrc file. He keeps his in a separate folder and uses a symlink to put them in his home folder. I strongly recommend this as well. You can automatically sync your .vimrc (or any other files) across computers with Dropbox. See how in my post on how to Use Dropbox to keep your preferences across Computers.

Here are some useful lines to start you out:

" Set tabs to 2 sapces
set softtabstop=2
set shiftwidth=2
set tabstop=2
set expandtab
" Move your backup and .swp files out of the directory of the file (Helpful to keep from adding them to a repo)
set backup
set backupdir=~/backup
set dir=~/swap
" Highlight the current line
set cursorline
" Keyboard shortcuts (Mappings)
c> " Use mat Insert Mode (instead of Escape)
imap => => " Hit => to insert a "rocket" in insert mode
" Wildmenu - show a tab completion matches list
set wildmenu
set wildmode=list:longest,full

Again, search the google machine and checkout github for more samples and if you find anything cool, let me know in the comments.

Update 3/16/2013

Check out my public vim configs repo for more info!

See ya Soon!

– Chris

Use Dropbox to keep your preferences across Computers

Fri, 25 Jun 2010 00:00:00 +0000

Sometimes it can be kind of painful to discover a cool new setting or mode for on of your favorite programs. For instance, I use TTYtter, a terminal based Twitter client. A few weeks into using it, I discovered it had ReadLine support for tab auto-completion of @usernames, in-line editing of posts and command history (up key). Awesome right? I know. Except, now I need to change my .ttytterrc file on my laptop, home computer, work computer, everywhere. Also, how do I get it there? USB drive, e-mail to myself, browse the network, etc. Being geeks, we don’t want to go through all that.

Then I discovered Dropbox (that link is a referral link, here’s the regular homepage). Not only is Dropbox a free 2GB of cloud storage for backing up your much needed files, but the really cool thing about Dropbox is that it automatically detects when files change and syncs them across all of your computers. Think about configuration files and read that again. Discover a new mode in any program with a config file and BAM! every computer you use has that mode enabled. The sync works in all directions so it doesn’t matter which computer you make the change on, every computer gets it. Seems perfect, right?

If you’re new to Dropbox, check out this post on Cloudwards on getting setup.

Here’s how it all works. When you install the Dropox client software on your computer, it will create a Dropbox folder with all your stuff in it. Add, delete or edit any file in this folder and Dropbox will sync it up to their servers, plus download the changes to any other computer running the Dropbox client with your account. It’s kind of cool to have your desktop and laptop running next to each other and watch them download the changes instantly.

Back to business, I know what you are thinking, “All these files are in one Dropbox folder (and subfolders). My config file needs to be in my home directory. I still have to copy from my Dropbox to my home.” Well, let’s let the computer do the copying. Actually, let’s not even waste time and space copying. Let’s create a symlink to make the config files appear to be in your home folder but actually be stored in the Dropbox folder.

First, open a terminal, navigate to the Dropbox folder and create a new subfolder; I called mine ‘configs.’

$ cd ~/Dropbox
$ mkdir configs
$ cd configs

Now, let’s move the config file from the home folder into the Dropbox/configs folder and navigate to the home folder.

$ mv ~/.ttytterrc ~/Dropbox/configs/.ttytterrc
$ cd ~/

Here’s the magic. We’ll create a symlink (like a shortcut in Windows) which will make it appear that the file is in your home directory, but Dropbox will still keep track of it.

$ ln -s ~/Dropbox/configs/.ttytterrc ~/.ttytterrc

Rough Spot: If you get a file exists error, the link will not overwrite an existing file. Remove the file from where you are creating the link into (the home dir in our case).

Do this on all your computers, then you can edit the file directly from my home directory on any computer and the changes will be automatically propagated to all my other computers instantaneously.

Here are some other ideas for this technique which I haven’t tried, but might be cool.

– Chris

Keep ssh sessions from timing out

Thu, 20 May 2010 00:00:00 +0000

The Ubuntu Blog has a nice lil' article about keeping SSH sessions alive It basically boils down to editing your /etc/ssh/ssh_config file and adding the following:

# /etc/ssh/ssh_config
ServerAliveInterval 5

The number is the number of seconds to send the small keep alive which keeps the connection open. Ubuntu Blog suggests changing it from 5 to 240 or 300 (4 or 5 minutes).

– Chris

Breaking Bad Season 3 Firefox Persona

Fri, 19 Mar 2010 00:00:00 +0000

Update: HURRAY!!! It was approved!! Get it now at http://www.getpersonas.com/en-US/persona/140288

That’s it! Enjoy the Breaking Bad awesomeness!!

UPDATE: I found out that it usually takes about a week for a Persona to get approved which should be around Friday March 26, 2010. I will update this post when I hear word. If you see if before me, let me know in the comments!

Installing PianoBar - Console client for Pandora Radio

Tue, 26 Jan 2010 00:00:00 +0000

PianoBar is a console client for Pandora Internet Radio. For me, this is a huge discovery. If you run Ubuntu, the flash player for Pandora can be a pain in the ass to install. Plus, on my netbook, the flash plugin for Chrome usually eats about 40-55% CPU constantly. A console client for Pandora will resolve all these issues.

However, the install instructions in the readme at the PianoBar Github Repo are very sparse and for someone pretty new to linux (like me) may be a little daunting. In the next few paragraphs, I will try to flesh out the installation instructions for PianoBar on Ubuntu linux.

1. Get the PianoBar source.

I downloaded the .tar.gz from http://6xq.net/html/00/17.html and extracted it with

tar -xzf PromyLOPh-pianobar-e079b45.tar.gz

2. Try to Make the PianoBar source.

I say try because for me it didn’t work. I was missing some of the required dependencies. But, you might not be missing the same ones as me. So, try to make it and then follow the instructions below for the dependencies you are missing.

cd PromyLOPh-pianobar-e079b45/
cmake .

3. Get and install libao

Libao is a cross-platform audio library for playing audio. I followed these instructions from Linux from Scratch to install it. Download the source from that page. Extract the .tar.gz:

tar -xzf libao-0.8.8

Then, from withing the libao-0.8.8 directory, configure the library with:

./configure --prefix=/usr && make

Finally, install it as root:

sudo make install && sudo install -v -m644 README /usr/share/doc/libao-0.8.8

Then, I retried step 2 to see that I didn’t need the LIBAO anymore.

4. Install FAAD2

Download the bootstrapped .tar.gz from AudioCoding.com. Extract the .tar.gz with:

tar -xzf f@d2-2.7

There are installation instructions in the INSTALL file in the extracted folder. I’ll summarize what worked for me. Configure it:

./configure

Make it:

make

Install it:

sudo make install

Then, I tried to make pianobar again. It still said it couldn’t find libmad, but it turned out I didn’t need it.

5. Install PianoBar.

After running the cmake from the extracted PianoBar folder, you should see that the object files were made. Then, run make:

make

And install:

sudo make install

If all goes well, you will be able to run it:

./src/pianobar

Enter your pandora username/email address and password. Choose your station. Hear music!

UPDATE:

I recently used this post to install pianobar on another computer and was getting pianobar: error while loading shared libraries: libf@d.so.2: cannot open shared object file: No such file or directory I needed to update the Shared Libraries (You can read more about Linux Shared Libraries from this page, specifically 3.5). I tried ldd /usr/local.bin/pianobar to see which shared libraries where being used by PianoBar and did see that libf@d/so/2 was not found. linux-gate.so.1 => (0x0044c000) libf@d.so.2 => not found libao.seeo.2 => /usr/lib/libao.so.2 (0x00edb000) libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0x00507000) libm.so.6 => /lib/tls/i686/cmov/libibm.so.6 (0x00794000) libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0x00982000) libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0x00f20000) /lib/0x00f20000ld-linux.so.2 (0x00f4e000) But I could find the libf@d.so.2 file in the /usr/local/lib/directory. Therefore, I needed to run ldconfig: sudo ldconfig Then I ran the ldd again and saw that libf@d.so.2 was now found in /usr/local/lib/ and PianoBar was now running again. Also, I learned how to automatically sign in and change the keyboard short cuts.

6. PianoBar config

You can change the configuration of PianoBar to switch up the keyboard short cuts and automatically login on run. To do this, you need to create and edit a file named ‘config’ in the .config/pianobar/ within your home directory.

vim ~/.config/pianobar/config

To automatically login, add the following two lines

user = pandoraUserName
password = pandoraPassword

I also found that changing the “Loving Song” keyboard short cut from + to = worked out better, since I didn’t have to press Shift+=.

act_songlove = =

– Chris

Notepad++ Color Configurator and styler.xml

Fri, 08 Jan 2010 00:00:00 +0000

Before I start talking about colors, let’s talk font. Consolas. That’s all you need to know. By far the best programming font. Get it. Use it. Love it.

Recently I started learning Ruby on Rails and I found RailsCasts with a ton of Rails focused screencasts. I believe he use TextMate for mac. I hadn’t really thought much about color in my code before watching some of these screencasts. But, it actually helps, a lot! I use Notepad++ in Windows and they have a great styling system. You can use the in program Style Confirurator or you can directly modify the XML style sheet.

Style Configurator

The Style Configurator is a dialog within Notepad++ accessible through the Settings menu. The really great thing about the Configurator is that the styles you choose are updated live in your code. So, open up a file with a lot of code. Then open the Configurator and move it over to another screen. If you don’t have more than one monitor, check the Transparency box in the lower right and you will be able to see through the Configurator. The slider there sets the Transparency level. Now you can see all of your code and pick styles for them.

Start with the very first ‘language,’ Global Styles. These not only include default styles of text but many of the stuff around your code, such as line number margin, inactive and active tab color, and the edge. The edge is great to keep your lines of code to a certain length for printing or displaying in a terminal. I also set the global font to Consolas. Also, be sure to set the current line color and selected text background color.

Now you are ready to style for you language. I usu PHP mostly at work so I’ll go though that one. Hopefully, I’ll be familiar with Ruby soon enough to write a post about Ruby styles.

QUESTION MARK - This is the style of the opening and closing php tags ('')
DEFAULT - Text that doesn’t match any of the other styles will be this.
STRING - Any string enclosed with double quotes
STRING VARIABLE - variables enclosed withing literal strings: “The Count is $count”
SIMPLESTRING - These are string enclosed with single quotes. (I used a very similar green to STRING)
WORD - These are the keywords of PHP. There are a bunch defined. You can also define some of your own in the blank box.
NUMBER - Number literals, including array indices
VARIABLE - Any word started with a $
COMMENT - Text between matching multi-line comments ('/' and ‘/')
COMMENTLINE - Text on a line after and including two forward slashes (//)
OPERATOR - Operators (+,=,-,etc.), matched parenthesis, brackets, curly brackets and logical operators

Stylers.xml

The stylers.xml file contains all the settings from the Configurator used by Notepad++. In addition to using the Configurator, you can manually edit this file. Or, download one from the web. The Notepad++ site has some under Theme Files. Joy Boner (read the about page) has 60 themes. I didn’t check any of these out, but it might be worth a shot if you really don’t want to set your own styles.

I also put my personal stylers.xml as a gist on github. Feel free to fork it.

If you have any other Notepad++ styling tips, comment below!

–Chris

Getting and Parsing E-mail with PHP

Fri, 18 Dec 2009 00:00:00 +0000

Here’s my Problem: My website sends text files to a partner’s site via FTP. Our partner site sends us the results of their processing the file in an e-mail. This e-mail is a the direct output from their processing script. Or, the relevant details are buried in a bunch of garbled text.

My Solution: I first looked at message piping, but my hosting provider doesn’t provide an easy way to do this. Then I found this PHP class from PHP classes, which handles the interactions with the server. You do have to create an account with PHP classes to download the files. If you e-mail me directly, I could send them to you. It’s easy enough to get working quickly.

Edit the test_pop3.php file. All you need to do is change the sever name, user name and password to get rolling. After you see the full output and it’s getting the mail correctly, I recommend setting the debug and html_debug values to 0. This will cut out the unnecessary text of the interaction with the server and just show you the messages and what’s going on.

One hangup of this class, the body of the message is an array and the message was repeated twice in the array.

Luckily, the part boundaries are included in the array which makes it fairly easy to split the array and just get the message once.

I needed to run some regular expressions on the body and imploded it into a single string.

Now I have the body of my e-mail message as a string variable ready for processing. Alternatively, you can change the first parameter in implode( to a line break if you need to display the message not as one long string.

Lastly, once you process your messages, you don’t need them anymore.

The DeleteMessage( method will mark messages for deletion when you close the connection to your mail server. Please note that this method does not immediately delete the messages and messages can still be ‘un-deleted’ until the connection is closed. The ResetDeletedMessages( method is used to un-mark all messages for deletion.

Happy Parsing!!

–Chris

CodeGoalie

Welcome, Juniors!

Appropriate scope

Apprenticeship

Agents for System Design

Know what you want

Provide the load bearing walls

Meticulously review

Cutthroat revisions

Agents Calm Down

Why not build everything?

Do one thing well

Do less

Use Your Noodler

golibnotify 0.2.0 released!

Format JSON in Nvim with jq

AI and I, or You're Holding That Wrong

How to think about all this?

How I have used it to date?

One-liner to copy a Github PR link from the terminal

Web Scraping 2.0

Why not both?

Magic PR Journeys

Commit Before Code

What am I talking about?

Why would anyone do that?

Ok, but how can I do this?

What about those small, unrelated changes?

If you can’t make these changes in your current branch, where do they go?

Cleaning Up Docker Images

Buffalo: Render a Template to String

Service 'Objects' in Buffalo

But, why?

A place for everything and everything in its place, until

Ok what is it exactly

Add to route handler (controller)

Using a background worker

Ruby refactoring: Open/Closed Principle

Open/closed principle

Sending update notifications

Adding more notification types

Naming conditionals

Reverse dependency injection

Set a default, override when necessary

Poor Man's React Router

How shell scripting saves my side projects

Runable

Helpful

Complete

Git References

References

SHA1s

Branch Names

Tags

HEAD

Modifiers

First Parent

Other Parents

Bringing it all together

Working with Containers: beyond the tutorial

Turn Key Kubernetes

Keeping costs down

Persistent Storage

Stuck in CreatingContainer

“Restarting” a pod

New blog; New identity

Useful command line expansions

Delete multiple branches with git

But why do you need this?

Go Concurrency by Example

Primer

Worker driven concurrency

Multi-worker concurrency

Android Development: first impressions

Commit Message Starters

What I like

Ensure

Boyscout

Migrate

What I avoid