Implements Programmer - Posts

Lessons in EKS

2024-05-10T01:13:31-04:00
I took for granted that managed Kubernetes offerings, like Amazon's Elastic Kubernetes Service (EKS), would continue to Just Work (tm). What happens when they break down? Here are some lessons I've learned:</p>
Sometimes, with high-churn workloads (e.g., knative), the de facto CNI of vpc-cni</code> breaks down and refuses to assign more IP addresses. When that happens, you're SOL.</li>
Despite some clever Terraform code lining up the EKS cluster version and grabbing the latest version of the vpc-cni</code> EKS add-on, do not trust that it is stable enough for production use until verified directly.</li>
The EKS control plane (kube api) not permitting custom CNI plugins use definitely limits its utility and makes the entire deployment and verification more complex.</li>
Setting hostNetwork: true</code> or otherwise setting hostPort:</code> on all of the operators' pod templates is both less secure and very difficult for some helm-based workloads that are missing a parameter that would control that feature.</li>
The hostNetwork: true</code> workaround will almost certainly violate any decent Pod Security Standards</a> for the given namespace.</li>
The Kubernetes scheduler will still try to put a pod, which needs to expose a specific port on the host network, on a node where that same port is already in use.</li>
Karpenter or Cluster Autoscaler is a must</em> when using a custom CNI for IPAM. So many more nodes...</li>
Calico talks about eBPF and other higher-order features, but none of them seem to work reliably in EKS. Overlay VXLAN networking barely works.</li>
Just because a Deployment</code>/Daemonset</code>/Statefulset</code> says its pods are healthy does not mean they are. Check the logs. All of the logs.</li>
Just because you've run helm upgrade</code> without --reuse-values</code> does not mean any out-of-band, manual changes to the resources that helm chart manages have been reverted</li> </ul>

Intro to Blogging: Engineer Edition
2023-07-10T04:41:00+00:00
Blogging is supposed to be super easy, low effort, and high reward. compare the effort it takes to produce a video for YouTube. Maybe how long it takes to code an application. Or publish an academic article. All of those make information you have to say publicly available, but how much work is put into it to get the content processed and over the finish line?</p>
For this reason, among many others, blogging is a form of self-promotion, of self-discovery, and of public navel-gazing (you didn't think it's all sunshine and roses, did you?) where thoughts and ideas can be freely shared or ignored. The web is the public forum.</p>

The classic advice🔗</a></h1>
Start writing🔗</a></h2>
First and foremost, collect your thoughts in written form. Bullet lists, long form prose, mind maps that evolve over time. All of these are valid ways of iterating into a document one might publish online. Because this is a document.</p>
Even if you don't have a blog on a fancy website, Wordpress.com</a> is free. Many blog sites are free. Maybe follow in line with Derek Sivers’ advice</a> and write plain text notes for now. Just get started.</p>
It will never be perfect🔗</a></h2>
Given a “blog” (originally from “web log”) is, by definition, an online medium, one can easily update a post after it has been made publicly available. This doesn't mean put a stream of consciousness set of ramblings out there (or do, it might work well for you), but being okay with bullet points for key topics might be sufficient to put out there. It all depends on how formal you aim to be.</p>
I, for one, tried to be too formal out of the gate and it led to sporadic bursts of posts here and there, only some of which survived into the blog seen today. I'm not a professional writer, nor do I am to have that level of polish. I do aim to be understood. If it is understandable, it is good enough. Publish. That's my defining line.</p>
You will be terrible at it, and that is okay🔗</a></h2>
Every writer I have heard of, when asked, indicates how terrible of quality their written works were on the first draft, in their early career, or when they're first organizing thoughts. That's normal. Writing is a skill and, just like a muscle, it gets easier to do impressive things the more you practice in earnest.</p>
Imposter syndrome is real🔗</a></h2>
There is always a non-zero chance that you have something of value to contribute. Even if it's regurgitated facts learned elsewhere, maybe it's phrased in a way that makes sense to others struggling with the topic. Maybe it helps solidify the topic in your mind. Maybe it indicates where your mind was at a certain point in history.</p>
Clients and employers looking at your background may see this and get a better idea about how you think, what you're interested in, and assess talent. With the amount of bluster out there it is refreshing, speaking as someone who has conducted countless interviews over the years, to have a candidate that is demonstrably transparent. Even if what you say is factually incorrect, a blog is your own platform and you can update it. Or even erase it.</p>
Contemporary advice🔗</a></h1>
My audience is not your audience🔗</a></h2>
How do I know that? Because my audience is most often… me.</p>
I write because I have a terrible memory. But because I write it down I get practice. And with practice I get better at writing. For me. Others may not see improvement, but that is because I'm not optimizing for them. They are not me.</p>
Don't mistake me for not caring. If others find benefit in my writing, I get a special kind of giddy in my soul. The off-chance of that happening is one reason I make my ramblings available. I also don't have comments for a very particular reason: my comments to myself are in the form of edits to the original post or follow up posts.</p>
I don't take myself seriously (anymore) and it is incredibly freeing.</p>
Make it accessible🔗</a></h2>
While web accessibility is a core value of mine, I mean to focus on accessibility to write. Don't limit yourself to [only writing when sitting at a desk]({{<relref "./blogging-made-easy.md" >}}), pondering the wonders of life in your amazingly abundant free time. I never have that kind of free time. If I do, I'd rather write code or read technical documents, or maybe even immerse myself in a good fantasy novel or video game. But that rarely happens without me falling asleep midway through.</p>
My writing process looks like composition in Notion, from my phone, writing and revising the meat of the article when and where I have time. 20 minutes while rocking my eldest back to sleep, 5 minutes while waiting for my order to be finished at the coffee shop, whenever I'm available. I don't leave it as the source of truth because I have strong feelings now, in my old age (heh) about having control over my own data. When my revisions get to a certain point I move it to a “queued” state where some automation periodically checks for any posts in that state, publishes the extracted markdown from them to my static site generator’s git repository, and updates the post in Notion to move the post to the next state.</p>
I write anywhere and can push to public when I feel like it.</p>
You have more to say than you realize🔗</a></h2>
When I start enumerating my thoughts into words, I keep tripping over myself with “but first” and “prerequisite knowledge to understand that is…”</p>
To avoid getting caught up in that loop, leave a loose end here and there. One need not explain the world when it is a note to oneself. It is helpful to lean into the online medium by linking to other content. Maybe someone else wrote something already. Maybe it's more eloquent than you could write. Maybe it's more polished or speaks to a different crowd than you intend. Or maybe you wrote it already. Link to it. If your audience already knows about consensus algorithms and how Raft differs from Zookeeper’s approach, they'll skip the link as a given. If the audience is confused, and has the wherewithal to dig deeper, they'll click through to the linked page. Lean on the world wide web being… a web.</p>
Also, keep in mind that, when writing this article, it only started as about 5 bullet points in a list. Now I can't shut up!</p>
I never fully finish my thought🔗</a></h2>
Again, it doesn't need to be perfect. It also doesn't need to be any certain length. Plenty of folks blog within a 500 character limit. It's called micro-blogging, done by applications like Mastodon, Twitter, and the like.</p>
A partial thought is still a thought worth having. Post it. Edit it later. Or don't.</p>
Own your content🔗</a></h2>
Ever think that social media sites own way too much of what you present as your public image? That they can dictate if you're allowed to speak in what is thought of as a public forum? Host it yourself.</p>
Many free services are available to host a static site or small-time blog. I like to host mine on Netlify, though I have tinkered with GitHub pages, Vercel, and Amazon S3 before. Keep a copy of your posts locally and build it as HTML ahead of time. It is much harder to hack a system that doesn't do anything dynamically.</p>
Buy a domain for $10 a year. Point to a set of links under that domain name. Even if you change where it is hosted, others’ links will remain active and your blog will survive. Your information will be more difficult to censor.</p>
Twitter, Facebook, etc. are allowed to monetize you (mostly) or delete everything about you without your consent. Why not take ownership? Check out the Indie Web movement</a> for more on this topic.</p>


Dopamine to Create
2023-01-01T12:38:17-05:00
Looking at the Nivenly Discord, I saw Hachyderm.io was having some HTTP 504 issues indicating some backend service was malfunctioning. This lead to a link of a post-mortem</a> last time this happened.</p>

    
        
            19:06</strong> @dma</code> status.hachyderm.io updated acknowledging the root cause</li>
        </ul>
    </blockquote>
    
        From blog.hachyderm.io</a></cite>
    </figcaption>
</figure>
This small snippet from the timeline sparked an urge. It was an urge to do web development for a running service, not just a static site or two—of which this very blog is one.</p>
I've felt this urge before: create something new, get that dopamine hit, and ride high on that feeling of having accomplished something productive; even if that thing is MySpace for dogs and literally receives zero traffic, even from bots.</p>
Why would it satisfy that need to create something? Would it actually provide some level of utility, or is it polishing the tile in a bathroom that's about to be renovated?</p>
It makes me wonder how much of my need to do "cool tech things" is based almost entirely on getting that next hit of dopamine, imagining I'm creating something or pushing society forward with the progress I make. Is it sweeter if I can observe as it happens and bask in the potential</em> utility? For some hypothetical person? Or is the delayed gratification of seeing a fully working, but far more complex mechanism after a large binge of effort? Also for some hypothetical person?</p>
The urge to create passed and I realized I should channel my inspiration into journaling. At least then I'll have something useful to show for it, years down the road.</p>


Neurodivergence and Remote Work
2022-06-05T16:10:40-04:00
I'm seeing an influx of people talking about "everyone back to the office!" with responses like "lol, I moved. When should I expect the travel itinerary with accommodations you've booked?" I too have moved. I too would respond this way. I too believe if I am able to do my job well remotely, it shouldn't friggin' matter</em> if you've spent millions of dollars on a state of the art office. Your poor investment is not my problem.</p>
This is the prevailing thought for many and, again, I share it, but what isn't discussed too often is the massive benefit neurodivergent people get out of a fully remote work situation. It is much easier to be more productive when we don't have to mask for long periods of time. It drastically improves quality of life when we actually have energy at the end of the day which normally would have been expended white-knuckling through our natural responses to, for example, Terry coming over for some water cooler chit chat while we're focused on something or the air conditioner blowing frigid air on just one part of one shoulder. Or how we can't make any clicks or noises or other more obvious tics that help soothe the growing stress of the environment because... it would fundamentally change others' views of what we are capable of handling.</p>
Let's unpack that.</p>
What is Neurodivergence?🔗</a></h2>
Generally, when people talk about neurodivergence (ND), they contrast it with "neurotypical" behavior. There is no agreed-upon, defining line between what is neurotypical and what is neurodivergent, but what is often epitomized as neurodivergent linked to stereotypical autistic quirks. Things like</p>

Aversion to certain sensory stimuli: sights, sounds, smells, textures, flavors, etc.

Dislike of t-shirts with a seam at the end of the shoulder or only wearing ones without tags</li>
Noise level that neurotypical people find acceptable or don't notice being disruptive (too loud or not loud enough) for neurodivergent people</li>
Gagging at the smell of a bonfire</li>
</ul>
</li>
Readily apparent behavior (ie. "tics") that is done periodically to soothe one's growing mental discomfort.

Twitch of the arm, fingers, legs, neck, facial muscles, etc.</li>
Making of nonsensical sounds or audibly reciting a mantra whose meaning matters less than the repetition itself</li>
Toussling hair or scratching some spot, regarless of if it itches</li>
</ul>
</li>
Fundamentally different perspectives, values, and methods when seeing the same objective circumstances</li>
Hyper-fixation or extreme interest in one specific topic for a given period of time (maybe life-long, maybe for a few hours)</li>
</ul>
These are some of the common patterns neurodivergent people seem to follow, many having their own reasons for each behavior. It means those with Tourette's Syndrome, Autism Spectrum Disorder (ASD), Attention Deficit Disorder (ADD or ADHD), and more are commonly considered neurodivergent. Why does this matter? What does this mean for neurotypical people?</p>
We just think differently. Our brains are built differently. There are cases where this is a massive</em> benefit compared to neurotypical people. Some of us see patterns more easily when others only see chaos. Hyper-fixation especially pays off in knowledge-based work like programming, information security, and similarly technical career paths. You even see some hiring managers (perhaps unknowingly) paying homage to this with the expressed desire of a "T-shaped" engineer. They're referring to someone who is a good generalist, but also knows a metric crap-ton about at least one specific area of expertise.</p>
I am neurodivergent. I was diagnosed with ADHD from an early age. Even now, I'm the textbook definition of a person with ADHD.</p>
What is Masking?🔗</a></h2>
Neurodivergent people may go undiagnosed with ASD, ADHD, or other labels for years. Maybe they never get diagnosed. Until they get that diagnosis and</em> become comfortable publicly having one of those labels, it is often reinforced at an early age to "act normal." For those of us with ADHD, it was to "stop fidgeting in [our] seat." It was to otherwise hide our behaviors that weren't neurotypical, almost like putting on a mask over our true selves so we could go about life uninhibited by public perception. But it's exhausting.</p>
Masking takes energy to suppress what behaviors come naturally. Anecdotally, I see a lot of neurodivergent people also identifying as introverted.</p>

	
		Introversion vs. Extraversion</p>
	</header>
	
		Introversion can be defined as requiring use of energy to engage is social behavior. Some behaviors take more energy than others. Recharging energy is often done through solitary activities such as reading, sleeping, knitting, or playing video games.</p>
Extraversion is just the opposite: social behavior results in recharging energy.</p>
Introversion and extraversion can be considered a spectrum. Rarely is a person purely extraverted or purely introverted.</p>

	</div>
</section>
I, for example, mask to put on a kind face, actively listening, and expressions on my face that empathize with the person speaking. Though I genuinely feel that empathy, kindness, and desire to actively listen, it doesn't come as naturally to me as it might for others. I consciously do it. And it takes energy.</p>
What Does This Have To Do with Remote Work?🔗</a></h2>
When someone like me is masking, it takes effort to maintain that mask just like it does to carry in the 743 bags of groceries mom came home with all in one trip. It's difficult at first, but pretty manageable if you're quick to finish unloading the groceries (or whatever task is required), but as time goes on, you increasingly need a break. It starts to wear on you, marking your arms with the indentations that are painful, and making you irritable if people distract you from just finishing that task. But it's a muscle. The more often you do it, the easier it becomes to maintain it until the task is complete. The less often you need to take a break, get a better grip, and continue on. But it's still friggin' exhausting.</p>
Face-to-face meetings require masking. Masking is like carrying in those groceries. I've gotten fairly good at masking, but it still takes energy that would be better used on absolutely anything else. When working in an office, I need to keep my mask on all day. Basically every single moment unless I find a private spot (e.g., bathroom stall) to take a break and get a grip again. With remote work, it's anytime that I'm not actively on camera in a meeting.</p>
This means that energy throughout the day that would have been spent making myself behaviorally presentable to society could be repurposed. Maybe I have energy at the end of the day to play with my kids, or to put up those curtains I've had on my to-do list for months, or cook dinner instead of get takeout for the 4th time this week. This means I have energy for a deep-dive on a given topic and can lean into my hyper-fixation for the benefit of my team.</p>
Why You Should Care🔗</a></h2>
Companies would not get this much productivity or benefit out of me if it were in-person, regardless of how awesome of a company it is to work for or how acceptable it is for me to go about my day without masking (I'd still wear a mask, regardless). I wouldn't be nearly as mentally and emotionally present for my wife and kids if I had to use that energy to not get on someone's to-fire list.</p>
Remote work has made me that much more successful. So why go into the office if you can do your job better remotely?</p>


Facets of a Good README
2022-01-25T22:11:01-04:00
I see a good project README answering the following 3 questions:</p>

What does this do?</li>
How do I install/use it?</li>
What do I need to do/know to make changes?</li>
</ol>
I tend to answer this with a few key sections of a README with some self-evident headings. Doing so, in my case, generally looks something like this:</p>

# My Project

This is my project. It does cool things and talks to neat people.

## Requirements

- Python (>= 3.7.1)
- [Poetry](https://python-poetry.org/)
- Sanity

## Usage

```
user@localhost $ my-project --do-things fun --often
This is the output. Take actual output and put
it in here, then replace personal details like
usernames and IP addresses with generic placeholders
```

## Build

Run the following:

```
user@localhost $ poetry install
user@localhost $ poetry build
```

### Testing

```
user@localhost $ poetry run pytest
```

</pre>
Some key concepts to keep it brief:</p>

Prefer linking to other docs whenever possible. If you don't understand a thing that is a hyperlink, you're going to click through to find out more info about it. Make your life easy and rely on prior art</li>
Enumerate organization- or project-specific caveats or changes to what is posted in linked content in the README</li>
Show, don't describe. Give example and, where possible, include example output.</li>
Plaintext is better, where possible. For terminal output, paste the output itself instead of embedding a screenshot</li>
</ul>


Nix Is Worth the Complexity
2021-07-04T01:11:27-04:00
Recently I've gotten fed up with the breaking changes in Homebrew package manager</a>. After some research, using Nixpkgs</a> seemed like a far more stable option for GNU/Linux tooling on macOS, albeit with a decent learning curve for configuration.</p>
Without going too much further into it Nix is pretty cool</a>.</p>
Over the following months, I'd been spending what free time I had tinkering with Nix on macOS, specifically with Home Manager</a> and nix-darwin</a>. Nix is cross-platform between Linux and macOS, and, frankly, I found myself maintaining an increasing number of shell scripts for installing important tools I use. I got really</em> good at writing bash scripts. 🤣</p>
Bash scripts are really handy, but there are limits. There is no clean state with them, there's only whatever you're working with right then. You try your best to make them idempotent, but there's no reasonable way to test that they meet that expectation. It can only be reliably tested from a clean state once. Conversely, Nix builds in a clean room every time.</p>
Don't pollute the global state🔗</a></h2>
Nix is a clean state, it's purpose-built for isolation between each program, allowing me to better follow the adage "don't mutate global state" and sandbox each tool I needed. Then I could selectively upgrade and, if the upgrade broke something, roll back to previous state easily.</p>
It has happened more times than I can count, I help my coworkers through a borked python setup when the underlying python version gets upgraded in-place. Thanks brew upgrade</code>... 😑</p>
Instead of digging through all the virtualenvs out there, and rummaging through whether pyenv was set up right for that shell, or any number of other issues, why not decouple it?</p>
Nix offers the best of both dynamic and static linking</a> when building an application. It allows for multiple versions of python 3 at the same time. Or Java. Or Haskell. Or Go. Or glibc. Upgrade one library, then it doesn't need to update them all. Similarly if 5 applications all use the same library, there's no reason to duplicate it 5 times on the disk.</p>
Keep project-specific tools with the project🔗</a></h2>
I tend to use direnv</a> in my workflow for exactly this purpose: I can keep project-specific settings (e.g., tool selection) specific to that base directory. Unfortunately this is typically limited to versions</em> of known programs (e.g., python 3.7.3 instead of python 3.9.1) and workstation-specific environment variables (e.g., path to secret files).</p>
Introducing Nix, this changes my workflow. By using nixify</code> (roughly inspired by this bash function</a>) I am able to install and use postgres, limiting it only to being used in this one project directory. Maybe I'll (re)use postgres in another project. Do I need it installed globally? Absolutely not. This is a development machine, not an application server.</p>
Current state of integration🔗</a></h2>
I've been working with Home Manager</a> for managing my dotfiles and (user-scoped) system configuration. So far it has been difficult translating certain parts of RCM</a>'s framework, such as its overlay approach (having both ~/.dotfiles</code> and ~/.dotfiles-local</code> repos cloned with the latter containing higher priority config files).</p>
Instead of symlinking files into place, thereby ensuring any changes to them in-place are reflected back in the git repo, they're made immutable. The only</em> way to change them is from the git repo.</p>
I've begun ripping out the version managers like pyenv, asdf, chruby, and others to completely replace it with project-specific Nix expressions.</p>


7 principles of a good sysadmin
2021-04-12T12:35:00-04:00
This seems to be a common topic of conversation, so I figure I should put it on paper (so to speak) what I value as a systems administrator, or "sysadmin."</p>

Keep it simple</li>
Ensure it can be reproduced</li>
Keep it close to stock</li>
Magic is bad</li>
No development tools on the server</li>
Prefer complexity at compile time over runtime</li>
Consume artifacts</li>
</ol>
What does this mean?</p>
The fewer moving parts, the easier to diagnose🔗</a></h2>
By keeping things simple, reproduceable, and close to their defaults, this sets a sysadmin up for success when</em> things go wrong. More so, by keeping things close to the default settings you maximize the chance that your setup overlaps someone else's. Bonus to finding info on Stack Overflow or that one forum post</a>!</p>

	
		</a>TIP: Script it out</p>
	</header>
	

		By scripting out everything you do, no matter how small, this ensures you can walk away mid-thought and pick up where you left off later.</p>

Keep scripts in some central location to share with your team of sysadmins</li>
Version control systems (VCS) are best for iterating on these scripts</li>
Make the VCS repo private, lest credentials are mistakenly hardcoded</li>
</ul>
Make your shell script executable documentation</em>. Write it by defining your own shell functions describing each step you're taking.</p>

	</div>
</section>
Know your tools🔗</a></h2>
If you don't understand how a thing works, fix that. Learn about it. Pull back that abstraction layer and look under the hood.</p>
Why is magic bad? When you're troubleshooting some error, how can you logically rule out the tool as a contributing factor?</p>
This principle does not preclude you from using said magical tool, but it does mandate you dispel that magic by working to understand how it is implemented.</p>
Compile time vs. runtime🔗</a></h2>
In a sysadmin context, compile time can mean "the stuff done to configure and setup a service, system, or application before it is immediately needed."</p>
In contrast, runtime means "stuff being done as the application is being put into its 'running' state."</p>

	
		Example: Docker</p>
	</header>
	
		Some container images contain a shell script as an entrypoint (e.g., entrypoint.sh</code>). These shift some compile time tasks to execute at runtime, then defer to running the underlying application.</p>
Reasons for this design might be:</p>

More in-depth configuration changes required to pivot between environments</li>
Rapid action may be required to change credentials, so they're only passed at runtime</li>
</ul>
Actions taken in a Dockerfile</code> when docker build</code> is run are considered "compile time."</p>
Actions taken when running docker exec</code>/docker run</code> are considered "runtime."</p>

	</div>
</section>
Complexity costs at compile time need only be paid down once: when things are being setup.</p>
If at runtime, cost of complexity is paid down every time the application is started.</p>
Keep it simple. Pay down the cost as soon as possible.</p>
Artifacts are like gold🔗</a></h2>
Whether the application uses an interpreted language (e.g., python) or is statically compiled (e.g., golang), an artifact can be built to make rolling forward and reverting simple.</p>
In the case of python, a wheel (my_pkg-0.1.0-py3-any.whl</code>) is a well-formed package holding the python source code. In a venv</a>, install it with pip install ./my_pkg-0.1.1-py3-any.whl</code> to upgrade and pip install ./my_pkg-0.1.0-py3-any.whl</code> to roll back.</p>
With golang it's even easier. Just drop the new binary in place and, if it doesn't work, drop the old binary in that spot instead.</p>
Some VCS providers, like GitHub.com, allow uploading binaries and other assets related to a release to a location that can be accessed later, perhaps even by shell scripts.</p>


Chef goes RedHat
2019-04-04T23:19:01-04:00
Given the recent news</a> that broke about Chef's changes to their licensing model (and the associated FAQ</a>), a lot of chaos has ensued and I feel the need to straighten out a few things since I've been present for most every public development so far.</p>
Here's a quick overview:</p>

Rebranding of products</li>
Open source release of Chef Automate</li>
Change in licensing, affecting monetization</li>
Price increase for commercial licenses</li>
</ul>
Let's break these down into more consumable pieces.</p>
Rebranding of products🔗</a></h2>
This part is a relatively minor, semantics change for clarity and consistent brand. Here's a quick guide:</p>

Chef</code> (config management tool) → Chef Infra</code></li>
Habitat</code> → Chef Habitat</code></li>
InSpec</code> → Chef InSpec</code></li>
Automate</code> → Chef Automate</code></li>
</ul>
As part of this rebranding, Chef (hereafter a term used solely to refer to the company) will be bundling their individual products as part of an "Effortless Infrastructure" and "Enterprise Automation" stack. This becomes important later.</p>
Emphasis on open source🔗</a></h2>
Per their announcement, Chef will be licensing the source code for Chef Automate under the same Apache 2.0 license as the rest of Chef's intellectual property. The release of this source code is to take place on April 16, 2019.</p>
Shocking, right? My immediate question was, "How will they make money? All support contracts and professional services?" That's where the licensing change comes in.</p>
License change🔗</a></h2>
Here's where things get a little dicey. All of the official statements from Chef have indicated the following:</p>

The source code for all Chef products (listed above) are to have their source code released under the Apache 2.0 license.</li>
Binary distributions of the Chef products are to be provided under a different license</a> than the source, specifically targetting commercial use.</li>
Commercial license fees for the previously free products have been updated on their pricing page</a> (more on pricing change below)</li>
Change in licensing is inspired by RedHat's business model</li>
</ol>
A quick list of reported exceptions to the license requirements for binaries, the details of which are to be reported by Chef at some undisclosed, later date:</p>

Personal use</li>
Experimentation for proving viability in a commercial setting (a.k.a., proof-of-concept work)</li>
Non-profit organizations</li>
Open source contributors to the products that meet certain criteria</li>
</ul>
Starting with the following versions, the new EULA for this commercial license will be included:</p>

Chef Habitat — (version not yet disclosed)</li>
Chef Infra — v15.0</li>
Chef InSpec — v4.0</li>
Chef Automate — (first publicly available version)</li>
</ul>
Development of the products is still to take place in the open. In fact, Adam Jacob made a call to the open source community to continue to collaborate</a> on the products and take advantage of the fact that it is still open source. He also made generalizations that this is entirely intentional so Chef can rely less on the "heroes" and "martyrs" who work on the previously closed source applications, inevitably burning themselves out. Instead, that burden can shift to anyone from the community who wishes to undertake contributing to the previously proprietary software in as big or small of a way as they wish.</p>
It is important to emphasize that the source code</em> will be under Apache 2.0 license and the "binaries</em>" (a term Chef has chosen to indicate the result of packaging with Omnibus and Habitat) will have the new commercial EULA.</p>
In other words, it's as if Chef is saying "You can either pay us for this PPA, Yum repo, etc. that makes it super easy to install and update our software, or you can take this tar.gz file and compile the same software yourself with quite a bit of trouble for free."</p>
Price increase🔗</a></h2>
Per the previous licensing change, several products have since changed from their $0 price tag to... well, other non-zero prices. While it's interesting how they've decided to measure units for increasing the price tag (e.g., InSpec is based on number of scan targets), the most interesting part here is for existing customers.</p>
The price tag for Chef Automate seems to have increased from $75/node previously to $150/node currently, according to some community members. That increase is difficult to verify since the pricing is no longer just for Chef Automate, but rather the entire Enterprise Automation stack. (Did I tell you their organizing their products into bundles?)</p>
Conclusion🔗</a></h2>
What does this mean? I'll tell you later, in a separate post. Right now I'm going to keep this page updated with as many facts as I can find as the story unfolds.</p>


InSpec: You're Doing It Wrong
2019-03-05T23:10:31-05:00
I've been using InSpec</a> for over 2 years now and have been writing automated tests for longer than that. What's the one thing that I've seen trip up the most people? Useless tests.</p>
InSpec as a project is based off of RSpec, a behavior driven testing framework. That isn't to say Gherkin like "Given X</em>, When Y</em>, I should see Z</em>", but rather a more generic definition. RSpec promotes writing tests in a feature-oriented manner:</p>
describe </span>MyClass</span> do</span></span>
  it </span>'</span>can instantiate with no arguments</span>'</span> do</span></span>
    expect {</span> MyClass</span>.</span>new</span> }.</span>not_to</span> raise_error</span></span>
  end</span></span>
</span>
  it </span>'</span>has an attribute for its name</span>'</span> do</span></span>
    obj</span> =</span> MyClass</span>.</span>new</span></span>
</span>
    expect</span>(obj.</span>name</span>).</span>to</span> eq </span>'</span>MyClass</span>'</span></span>
  end</span></span>
end</span></span></code></pre>
The net benefit with RSpec comes from the reporting format:</p>
 ✅ MyClass can instantiate with no arguments</span></span>
 ⛔ MyClass has an attribute for its name</span></span>
</span>
       expected: 'MyClass'</span></span>
       actual: 'Lol, just kidding'</span></span></code></pre>
Instead of testing each unit like 1 + 2 = 3</code>, we can have a very tightly scoped integration test that gives us enough info to figure out that</em> something is wrong. The frist battle is figuring out that</em> something is wrong, then you need to know how</em> it's wrong.</p>
Back to the core of the matter, what do I mean by useless tests? As stated earlier, 1 + 2 = 3</code> sorts of tests are useless. We don't care about the unit as much as we care about the gestalt. It doesn't matter if the calculator widget works flawlessly if the entire page doesn't render!</p>
What if, instead of 1 + 2 = 3</code>, we tested some small specific things:</p>

Does the login page present a button to login?</li>
When I send invalid credentials, does the backend refuse to authorize me to act as the given user?</li>
When I send correct credentials, does the backend allow me to act as the given user?</li>
Are there any links on any of the pages that result in a 404 or 500 HTTP status code?</li>
</ul>
Any of these tests would be relatively low effort to implement, but provide much better feedback as to the overall state of the web application. Just like these real life examples from my career as a web developer writing automated tests, the same principals apply to testing infrastructure: we should test the gestalt. Let's consider something basic.</p>
I want to confirm that my web service is operating correctly on my EC2 instance in AWS. How might I be doing that currently?</p>
describe </span>service</span>(</span>'</span>httpd</span>'</span>)</span> do</span></span>
  it { should be_running }</span></span>
end</span></span>
</span>
describe </span>service</span>(</span>'</span>mariadb</span>'</span>)</span> do</span></span>
  it { should be_running }</span></span>
end</span></span>
</span>
describe </span>port</span>(</span>80</span>)</span> do</span></span>
  it { should be_listening }</span></span>
end</span></span>
</span>
describe </span>port</span>(</span>443</span>)</span> do</span></span>
  it { should be_listening }</span></span>
end</span></span>
</span>
describe </span>port</span>(</span>3306</span>)</span> do</span></span>
  it { should be_listening }</span></span>
end</span></span>
</span>
# My application's deployment artifacts exist in their proper locations</span></span>
describe </span>file</span>(</span>'</span>/var/www/public_html/index.php</span>'</span>)</span> do</span></span>
  it { should exist }</span></span>
end</span></span>
</span>
# I have an Apache configuration entry for my service</span></span>
describe </span>file</span>(</span>'</span>/etc/httpd/sites-available/my-site.conf</span>'</span>)</span> do</span></span>
  it { should exist }</span></span>
end</span></span>
</span>
# The prior Apache configuration is currently active</span></span>
describe </span>file</span>(</span>'</span>/etc/httpd/sites-enabled/my-site.conf</span>'</span>)</span> do</span></span>
  it { should exist }</span></span>
end</span></span>
</span>
# I have the correct database credentials written to the necessary configuration files</span></span>
describe </span>file</span>(</span>'</span>/var/www/config.php</span>'</span>)</span> do</span></span>
  it { should exist }</span></span>
  its</span>(</span>'</span>content</span>'</span>) { should </span>match</span>(/</span>db_username="</span>#{</span>Regexp</span>.</span>escape</span>(</span>'</span>1337h4x0r</span>'</span>)</span>}</span>"$</span>/) }</span></span>
  its</span>(</span>'</span>content</span>'</span>) { should </span>match</span>(/</span>db_password="</span>#{</span>Regexp</span>.</span>escape</span>(</span>'</span>hunter2</span>'</span>)</span>}</span>"$</span>/) }</span></span>
end</span></span></code></pre>
This is the thought process one might go through. I'm not willing to go so far as to say these tests are useless, per se, but they certainly miss the forest for the trees if you still get an angry tweet from an end user including a picture of a severely broken page.</p>
Because we're Agile, Lean, or whatever other trendy buzz-words that can be thrown at the problem, we're going to make small, incremental changes. To figure out what changes to make to fix a bug, I ask the following questions in the given order:</p>

How is it broken?</li>
What indicates it is broken?</li>
How can I automate checking if it is broken?</li>
</ol>
The answers may surprise you. From that hypothetical tweet we just got, we might decide that a working definition for "broken" in this context means anything from "I see a blank white screen when the page loads" to "Firefox warns me that something related to the SSL certificate is weird", or even "I found the rainbow unicorn when I tried to load GitHub.com." How can we test these indicators, then?</p>
describe </span>http</span>(</span>'</span>https://svc.example.com/my-page</span>'</span>)</span> do</span></span>
  its</span>(</span>'</span>body</span>'</span>) { should </span>match</span>(/</span>You're visitor number </span>\d</span>+ today</span>/) }</span></span>
end</span></span>
</span>
describe </span>ssl</span>(</span>port</span>:</span> 443</span>).</span>protocols</span>(</span>'</span>ssl3</span>'</span>)</span> do</span></span>
  it { should_not be_enabled }</span></span>
end</span></span></code></pre>
Okay, pause. What functionality does that first test actually check?</p>

The httpd service is running (or else would have I/O failure establishing connection)</li>
The machine is serving HTTP requests over port 443 (or else would have I/O failure establishing connection)</li>
We have TLS certs installed to the proper spot (or else would have a TLS failure)</li>
There aren't any syntax or compile errors in the server-side code (or else would be HTTP status code 500)</li>
The httpd service is configured to serve requests according to the server-side code you deployed (instead of the default Hello World site or a completely different project)</li>
</ul>
Assuming the content you're checking on the page is derived from data gathered from the database (e.g., "You're the 394th visitor today!"), this might also check the following:</p>

Our connection to the database is allowed through the firewall</li>
Our database credentials are up to date</li>
The database schema is setup in the expected format (at least in part)</li>
</ul>
Is this an all-encompassing test? No, but it does test a helluva lot of moving parts and, if any one of them is broken, we'll know because the test will fail. That's one useful canary in this here coal mine!</p>
Now, we find that the test is failing. What now? Here's where the initial set of tests might help. They check the known hiding spots for some of these types of errors. Maybe the source of the problem is in one of these spots, maybe not. At least we can go through by process of elimination. Even better if the tests were already written too!</p>
The moral of this story is that the initial tests on the application server aren't wrong, but they don't have a broad enough scope to determine if</em> there's a problem. You want to be hyper sensitive to if</em> there's a problem, then hone in when</em> there's a problem using investigative tests, helping us diagnose the problem if the canary test is failing.</p>
There are any number of known or unknown reasons why a problem may exist. The most effective way to be successful is to know as early as possible that</em> a problem exists so you can start looking for the source of it.</p>

Let's examine one more concept, best embodied by InSpec's cloud provider resources.</p>
Why would you need to test that a cloud provider's setting has a certain value? Are you testing that Terraform did its job? Are you testing that a bit of automation for building up and tearing down cloud resources is working? Great, but something tells me you're not telling the whole story.</p>
Consider the following InSpec tests:</p>
</span>
ec2_sg</span> =</span> '</span>sg-asdlkjasdflkj321iu21340</span>'</span> # some security group id</span></span>
rds_sg</span> =</span> '</span>sg-23oiwqoj23lqwedasdlkj2</span>'</span> # some other security group id</span></span>
</span>
describe </span>aws_ec2_instance</span>(</span>name</span>:</span> '</span>web-server</span>'</span>)</span> do</span></span>
  it { should exist }</span></span>
  its</span>(</span>'</span>security_group_ids</span>'</span>) { should </span>include</span> ec2_sg }</span></span>
end</span></span>
</span>
describe </span>aws_security_group</span>(rds_sg)</span> do</span></span>
  it { should exist }</span></span>
  it { should </span>allow_in</span>(</span>to_port</span>:</span> 3306</span>,</span> security_group</span>: ec2_sg,</span> protocol</span>:</span> '</span>tcp</span>'</span>) }</span></span>
  it { should </span>allow_in</span>(</span>to_port</span>:</span> 3306</span>,</span> security_group</span>: ec2_sg,</span> protocol</span>:</span> '</span>udp</span>'</span>) }</span></span>
end</span></span></code></pre>
In case you aren't fluent in AWS jargon, this means my virtual machine running my web service is whitelisted in the firewall to allow communication with my database over TCP or UDP on the database's port 3306.</p>
Instead of testing that the firewall rule, what if we just tested that we're functionally able to connect from one host to the other?</p>
db_username</span> =</span> '</span>root</span>'</span></span>
db_password</span> =</span> '</span>hunter2</span>'</span></span>
db_host</span> =</span> '</span>mariadb.example.com</span>'</span></span>
db_port</span> =</span> 3306</span></span>
</span>
sql</span> =</span> mysql_session</span>(db_username, db_password, db_host, db_port)</span></span>
</span>
describe sql.</span>query</span>(</span>'</span>SELECT 1;</span>'</span>)</span> do</span></span>
  it { should_not </span>match</span>(/</span>Can't connect to .* MySQL server</span>/i) }</span></span>
  it { should_not </span>match</span>(/</span>^error </span>/i) }</span></span>
end</span></span></code></pre>
What does this change really accomplish?</p>
Let's say some poor engineer working for Amazon pushes some bad code. This bad code disregards your settings and changes the actual port for a few RDS instances to be listening from 9999 instead. What happens with our tests from earlier? All green! Why? Because we're testing the settings, not the functionality.</p>
The same goes for if AWS has an unexpected outage. Why is my web service suddenly failing, despite all of my infrastructure being configured correctly? Oh because I'm only checking that I've told it the right thing, not that it's doing the right thing.</p>
Not to rip on cloud providers, but the same concept is found in testing, say, an IOS router by Cisco. I might write a test that confirms its config commands output something in particular. Is checking the output helpful? Maybe, but we're not actually testing that packets get routed to their proper destination.</p>
Let's focus on the end goal instead of being so myopic to how we arrive at that goal.</p>


Blogging made easy
2018-01-28T23:10:31-05:00
Since starting this blog, I've encountered any number of topics where I've literally said, "Maybe I'll write a blog post about that..." The biggest hinderance has always been thus:</p>
When I'm at my computer, I want to code. When I'm away from my computer, I'm typically on my phone scrolling through twitter or reddit, wishing I would blog more like the posts I end up reading.</p>

Netlify🔗</a></h2>
I typically use Netlify for deploying any static sites these days, if only because I can easily hook it up to github and auto-deploy using more than just Jekyll, even going so far as to have a custom domain with LetsEncrypt TLS certs. And I have done that before</a>. It only makes sense that I would investigate what else they have to offer.</p>
Netlify offers a full-on CMS</a> in the form of for managing your static site. It claims to work with any static site generator with a little configuration and requires the use of Github for hosting the source code of the site.</p>
Behind the scenes, Netlify CMS</a> creates a new branch off of master</code> or gh-pages</code> (whichever you configure is the main site) and creates a new commit on that branch every time you save with its built-in editor. It opens a pull request so it might be picked up later by the CMS. It does this all through the Github API.</p>
My issues with Netify CMS🔗</a></h3>

NOTE:</strong> Netlify CMS was renamed to Decap CMS in February 2023. (announcement</a>)</p>
</blockquote>
First off, the drafts of your blog posts will always be publicly available. Something so public so early on is not conducive to stress-free brainstorming or just getting started with writing. I get it though, it's super intelligent to store the data on Github's side instead of committing to any data storage by Netlify. It also makes it so you have one less dependency for storing drafts. This is a trade-off they have clearly considered and I just have to deal with the consequences.</p>
My second issue is that, when I last tinkered with it, Netlify CMS</a> had some show-stopping issues in integrating it with a Middleman site. The biggest one had to do with file extensions not being honored in the expected manner. It seems Jekyll was the favored child.</p>
Thirdly, if you prefer Gitlab for source code storage, you're SOL--and reasonably so, given Gitlab was never intended as a drop-in replacement for Github. Again, this is an obvious trade-off that has been considered already and means I fall outside of the target audience.</p>
There is nothing inherently wrong with Netlify CMS</a>, in fact I applaud its existence and would love to see it grow. It just isn't the right tool for my situation.</p>
Evernote🔗</a></h2>
After Netlify's solution not working out, I considered using a proprietary option that I already use to store a metric crap-ton of my data: Evernote</a>. I have a mobile app for it, it has an unofficial Linux client called Nixnote</a> (formerly Nevernote), and I'm already paying for a premium account. Why not?</p>
My issues with Evernote🔗</a></h3>
Okay so Evernote</a> has a much nicer solution than Netlify's web interface that may or may not work based on my internet connectivity. I can just pop open my Blog Posts notebook and go to town, right?</p>
That works up until it comes to transposing them into Jekyll. Evernote</a> does the WYSIWYG</a> approach, which is great when composing but not too great when copying it into a markup language like Markdown or even HTML. Underneath the covers, Evernote</a> notes use a form of XML with a proprietary schema for certain things like TODOs and inter-note linking. This means building a tool to work with Evernote</a>’s APIs is that much more difficult and I would have to build Markdown pages from XML. No way.</p>
What's the other option? Well, I would have to rewrite every single blog post by hand, reading from Evernote</a> and writing into vim in Markdown. The only way I'd be able to preview it with any certainty is to run the Jekyll server locally.</p>
Writing using a WYSIWYG</a> editor is enough of a hinderance, but having to transpose manually afterwards? That's a hard no. Too much overhead to write a simple blog post.</p>
SimpleNote🔗</a></h2>
This brings me to a solution made by WordPress' parent company, Automattic</a>. Yes, the company behind the blogging engine I set out to avoid at all costs could actually help me with blogging. Automattic bought the company</a> behind a note syncing application called SimpleNote</a>, with renowned client applications such as Notational Velocity</a> and the fork, nvAlt</a>. Before acquisition, SimpleNote</a> had been touted as a no-frills note-taking application with incredibly fast</em>, near-instant syncing of plaintext notes to Dropbox and other select cloud storage providers. Various clients, such as nvAlt</a>, supported previewing of markdown in its richer formatting.</p>
By the time I got to SimpleNote</a>, Automattic</a> had long-since bought it and made its own improvements. It removed the ability to send to Dropbox (et. al) and, for a time, broke compatibility with some of the third party clients. What it brought to the table were:</p>

even</em></a> faster</em></a> syncing</li>
creating native clients for Linux, Windows, and Android</li>
adding a web app option</li>
making all of the official clients open source</li>
nixing all paid plans</li>
</ul>
My issues with SimpleNote🔗</a></h3>
As stated previously, it almost feels like I'm working with WordPress again. it doesn't (yet) allow me to link to other notes and transpose them to another platform. It doesn't allow me to organize my notes into ones that are on a particular topic or ready to publish. It also doesn't let me see how long-winded I am in rendered markdown while also editing it.</p>
Honestly, that’s about it. SimpleNote</a> allows me to seamlessly write from my desktop, my phone, or some random computer with an internet connection and a web browser.</p>
Notion🔗</a></h2>
A proprietary platform, Notion</a> allows me to compose my thoughts in a knowledgbase that already is built for organization, displaying data in a way that works with your brain, and optimized for entirely keyboard-driven operation. Until 2022 it did not have a public API upon which to build stable integrations. Now it does. Now others have built them to extract the WYSIWYG elements they call "blocks" and transform them into markdown with good accuracy. With a bit more time and effort, the translation of Notion blocks to markdown could improve to handle Hugo's shortcodes, which help with custom layouts of content.</p>
My issues with Notion🔗</a></h3>
It is a blackbox solution which seems to structure the stored data based on position in the Notion application. Any dynamic field is evaluated by the client instead of the server, which means they've written a language which is used to evaluate those dynamic fields into static values. Any third party converter would need to reimplement the underlying language, without a spec, on a different platform.</p>


What is a Symbol?
2017-10-09T23:02:00-04:00
As I was browsing Twitter this evening, I came across a peculiar tweet:</p>

    
        Ruby friends who've been writing Ruby for ~5 years or less: what sort of things did you find surprising and need someone to explain to you?</p>
    </blockquote>
    
        Tweeted 10:12am - 9 Oct 2017</time></a> by Justin Searls (@searls</a>)</cite>
    </figcaption>
</figure>
The most common concept that remained confusing was symbols, so here’s my explanation of it. To explain symbols, I must first explain datatypes.</p>
Datatypes🔗</a></h2>
Do you know the difference between datatypes?</p>
Primitives🔗</a></h3>
There’s a string</code> (e.g., "Lorem ipsum dolor"</code>), which is a set of characters (char</code>s). There are various numbers ranging from an integer (e.g., 1</code> but not 1.4</code>), a double (a really large integer), a float (e.g., 1.4</code> or 1.0</code>), and any of those can be signed (including negative numbers) or unsigned (absolute values, positive numbers only). There are super simple concepts like booleans, which can be exactly “yes” or “no”.</p>
Why are there all these variations? Isn’t it all just text? Think about this:</p>
Binary🔗</a></h3>
A computer thinks in binary, ones and zeros. Binary can represent any number, such as 000000</code> represents zero and 000001</code> represents one, as one might imagine. The tricky part is 000010</code> represents two. How’s that? Let’s continue.</p>

000011</code> → three</li>
000100</code> → four</li>
000101</code> → five</li>
000110</code> → six</li>
000111</code> → seven</li>
001000</code> → eight</li>
</ul>
Get the idea? If we continue with this pattern, we’ll see that we eventually hit 111111</code> (sixty-three), but that’s not the last number there exists, is it? We need another digit to hold a zero or one, and it represents all the way up to one-hundred twenty-seven. Wow, that’s quite a jump, right? How does all of this have anything to do with data types?</p>
Datatypes in binary🔗</a></h3>
A computer has to store a representation of data in memory. But wait, didn’t we just cover that a computer thinks in binary? Well that’s still true. We have to come up with a shorthand for storing things like 1.3</code> in binary.</p>
Let’s tear this apart. This is a float datatype. We have to come up with some shorthand for noting what is before the decimal point and what is after. How about we say the first four digits are the number before the decimal point, and the last four are the number after. We can extend this beyond eight digits later.</p>
In this case the binary representation of 1.3</code> might be 00010011</code>. What’s the difference between this and the integer 19</code>? Both are represented by the same binary, right? We’ll need to come up with some universal shorthand to note the difference.</p>
In an integer, we know the same value 010011</code> (19) might also be positive or negative. This is called a “signed” integer. Other numeric datatypes can be signed as well, but we’ll keep it simple with integers for now. We’ll keep our shorthand of splitting up the digits, but whether it’s positive or negative is just 2 choices. We can probably keep to the first digit where 1</code> means negative and 0</code> means positive. Therefore 19</code> means 0010011</code> and -19</code> means 1010011</code>.</p>
Data headers🔗</a></h3>
We have all these shorthand tricks we have to remember, but at the end of the day it’s just 1’s and 0’s. How will we differentiate between an unsigned 19</code> and 1.3</code> from before? Let’s come up with one, universal shorthand to determine which data type we will be storing for the next few digits. Three digits should cover it for now. We’ll remember some datatypes by this mapping for now:</p>

001</code> → char</li>
010</code> → unsigned integer</li>
011</code> → signed integer</li>
100</code> → float</li>
110</code> → double</li>
111</code> → boolean</li>
</ul>
To integrate this we’ll tack this onto the starting of each value and just remember to interpret the first 3 digits as the datatype indicators.</p>
Data maximums/minimums🔗</a></h3>
To have all of these shorthands, we have to agree on the next X</em> number of digits, which will signify the value of the datatype. A boolean, true and false, only needs one digit (plus the header) while an unsigned integer can only count between 0 and 127 with 3 digits, which might not be enough for fun things like counting the number of seats in a stadium.</p>
We’ll come up with some arbitrary lengths of digits for these datatypes. Here are some examples:</p>

0000000000</code> → char (10 digits)</li>
0000000000000</code> → float (13 digits)</li>
000000000</code> → unsigned integer (9 digits)</li>
0000000000</code> → signed integer (10 digits)</li>
0</code> → boolean (1 digit)</li>
</ul>
All together, a float will take up 13 digits for the value + 3 digits for the header indicating that it’s a float. 16 digits. If we take a multiple instances of data (datatype header + value in binary), we can string them together.</p>

1000000000010011</code> → float datatype of value 1.3</code></li>
1110</code> → boolean datatype of value false</code></li>
</ul>
Let’s take these two example data instances and put them together like a computer might keep it in memory: 11101000000000010011</code></p>
What?</p>
Okay, let's tear it apart again.</p>
If we saw just this binary without any other context, we can remember our shorthand for the headers by reading left to right. The first 3 digits signify it’s a boolean, which we know has a value length of 1 digit. We read it as boolean false</code>, and we’ve parsed the first 4 digits in the example data.</p>
Since we're done with the first 4, we'll start at digit 5, follow along the same pattern with interpreting the first 3 digits as the datatype (100</code> = float), and parse the next X</em> digits (float value → 13 digits) as the value. The next 13 digits amount to 0000000010011</code>, which we’ll pretend like we already established a new shorthand where floats have the last 3 digits reserved for the decimal number. This makes it easier since we remember this was 1.3</code>.</p>
Doing this again with another example value: 01100000111010010001011100</code></p>
Reading the first 3 digits (011</code>) we see it’s a signed integer. This means we read the next 10 digits (0000011101</code>) as the value. Knowing our shorthand for signed integers, we recall the first digit of the value (0</code>) tells us whether it’s positive or negative, and the remaining digits (000011101</code>) are the number itself. We can then tell it’s +29</code>.</p>
The next three digits after that’s done is 001</code>, which means a character. You know from before that a character datatype reserves the next 10 digits for its value, but we don’t know how to decode it. We haven’t established that shorthand. The important part here, though, is that we can differentiate between the different datatypes when they’re thrown together in one, long, unbreaking string of 1’s and 0’s.</p>
Characters versus Strings🔗</a></h2>
In case you weren’t already aware, a character might represent any one key on your keyboard, including letters, numbers, punctuation (like ?</code> and !</code>), or other bits of written language (like {</code> or )</code>). A character can make up individual letters you might not see on your keyboard, like “é” or “→”. The point is, a character can be numerous possible values taking up the same physical space on your screen as a 1 or 0.</p>
Going back to what was said before, computers think in binary. We could come up with a way to store every character, mapping to a numerical value. These are called character encodings. Like other datatypes this tells us how many digits each character will take up. A character might be ASCII and be only what you might see on a QWERTY keyboard, or it might be UTF-8 and include emoji and other richer styles of a character. For the sake of simplicity, we’ll stick with ASCII.</p>
There are 26 letters in the english alphabet, plus 11 special symbol keys (upper- and lowercase for each, so 22 characters), plus the number key line of 10 keys (so +20 characters), plus the spacebar, which brings us to 69 possible characters. We need enough binary digits, or bits, to handle a maximum number of 69. By my calculations, that’s 7 bits. Not too far off from our 10 bits we reserved earlier for holding character data, right?</p>
To make a string like "Hello world!"</code>, we need to disect it. It’s the character H</code>, then the character e</code>, then the character l</code>, and so on for the word “Hello”.</p>
Wait. We have capital letters here too?! Shoot. Let’s amend our count of 69 characters to add in 26 more, 1 more for each letter of the alphabet in its capital form. That’s a total of 95 possible characters. Good thing our 7 bits can store a representation of any number between 0 and 127.</p>
So we have “Hello” and “world!” separated by a space character, which amounts to 12 characters. Since each character takes up 10 bits (3 header + 7 value), we’re looking at 120 bits to render "Hello world!"</code> as binary data. Here’s where I’m going to stop while you to ponder that for a minute.</p>
Symbol datatype🔗</a></h2>
Let’s assume you have a mapping of characters to bit values, and the maximum bits mapped out perfectly. You have that string that takes up 12 characters for 120 bits just to say hello to the world. What if there’s something you only want to reference internally for your own purposes? We don’t care about the actual value, we just care that the value happens to be unique from any other value. It has semantic meaning only. You know how we have those mappings of letters, base10 numbers, etc. to binary? Those are the same concept of a symbol. That’s part of the char</code> shorthand. What if we took it a step further?</p>
First, after all that binary-talk, I need to take my head out of the theoretical for a moment. I need to go back to the programming I know: Ruby, Python, etc.</p>
Ruby and Python are dynamic languages where you rarely have to know how many bits in memory a variable, holding a particular datatype, will take up. The way I operate, I care about how easily I can keep the inner workings of a program in my head at any one time. I need cues that won’t matter to the computer, but do only to me as the programmer. Take a logging class (in most languages), for example.</p>
We have the concept of a logger, which takes an enum of various error levels: ERROR</code>, WARN</code>, INFO</code>, DEBUG</code>, and possibly more. Do we care about storing the string of characters to represent it? No. We just need some internal representation to reference. Let’s choose a datatype that has a really small memory footprint here.</p>
Representing "ERROR"</code> (as a string) would be 5 characters × 10 bits per character = 50 bits. Other logging levels might be greater or fewer characters in number, so we just need to have a datatype less than 50 bits, or binary digits long to stand in for the one other, more memory intensive value. Let’s choose the unsigned integer 4</code>, and the other logging levels as unsigned integers as well. We don’t care about the value 4</code>, only what it represents, so it could just as easily be 986</code> or a poop emoji. It is meant to differentiate ERROR</code> from WARN</code> and the others.</p>
So the unsigned integer 4</code> is represented with 010000000100</code> in binary. That’s 12 bits long when the string version could be 50 bits. That’s quite the savings! Only 25% of the original memory footprint!</p>
In a language like ruby, a small savings like that might not make too big of a difference, but say it could encode it in binary instead of unsigned integer, with all the wasted space up front reclaimed. We only need the first 3 bits for the header and the 3 bits following for the value of 4</code>. What if our compiler was smart enough to see we only needed a maximum value of 4</code>?</p>
In that case, we could remember a new datatype that says the following X</em> digits represent a symbol, where X</em> is determined once the entire program is analyzed to figure out the max value one might ever see. In our case it’s 4, so we downsize the number of bits used from 12 to 6. What once was 010000000100</code> is now 010100</code>.</p>
But wait, we still have the datatype header for an unsigned integer! That’ll screw everything up! That’s very true. We can't keep breaking our own shorthand conventions with encoding/decode binary, so we’ll have to come up with a new, on-the-fly datatype, which we’ll refer to as a symbol. It’ll go by the header 101</code>, since it hasn’t been used yet.</p>
From the original 50 bits, to 010000000100</code> (12 bits), to 101100</code> (6 bits), that’s quite a bit of downsizing for the same functionality; the same functionality from the programmer's and the end user's perspective.</p>
Recap🔗</a></h2>
Symbols are just what they sound like: an in-memory stand-in value for something else. Their function is to save on memory usage when you don’t need to allocate a ton of bits to have a label that only needs to represent that it's different from other labels. The values don't actually matter. They are a handy representation, shifting work onto the compiler initially, but nets less memory (and equal computation power) at runtime.</p>
Moving forward🔗</a></h2>
These optimizations occur not only for storing data, but referencing binary chunk of data representing logic the computer is supposed to follow. Do you really think your computer remembers to look in that file for the string my_main_function()</code> in order call the logic defined therein? Do you feel the computer cares how you name things? No! It reads the logic into binary and determines a symbol only it remembers in order for it to easily call the functionality. These are more compilation optimizations that happen automatically in the programming languages you use.</p>
Languages like C and Ruby allow you direct access to symbols as a datatype, but languages like PHP and earlier versions of Java require you to declare your preferred datatype and value, leaving the memory optimization to the programmer when defining that a symbol exists.</p>
Are symbols helpful? Sure. How often? That depends on what and how you’re coding the task at hand. Hopefully this will serve as an introduction for what circumstances would be best to use symbols versus other datatypes.</p>


Recruiter quality score
2017-06-20T14:35:00-04:00
Although I know I'm incredibly fickle when it comes to who I'll work with when it comes to recruiting, the fact of the matter is that I value honest effort over the type to throw bodies at the wall. The shotgun approach is just lazy and rarely pans out. Here's my attempt at creating a rubric for my minimum barrier to responding.</p>
Please understand that I am a privileged person. I know that. My reactions based on this rubric will be skewed toward my particular context.</p>
This is coming from someone who:</p>

lives in a well-off area</li>
can decline offers due to savings to fall back on</li>
works in the software engineering industry with an in-demand language</li>
does not have outside obligations, requiring a rigid work-life balance</li>
has a reasonably up-to-date portfolio of work publicly available</li>
speaks english as a first language</li>
is a white male living in the United States</li>
</ul>
Scoring rubric🔗</a></h2>
Points</th> Description</th></tr></thead>

- 7</td> Emailed personal address</td></tr>
- 4</td> Use exceedingly old resume (>1 year old, if newer one is published)</td></tr>
- 6</td> Content is overly wordy (not easy to skim)</td></tr>
- 8</td> Content is overly generalized to be part of a mass email</td></tr>
- 8</td> Salutations are generalized for a mass email, e.g., "applicant" or "developer" (exception: blank "Hey,\n" or jumping straight to the body of the email is perfectly acceptable)</td></tr>
- 50</td> Mass email is sent using the TO or CC field</td></tr>
- 10</td> Mass email is sent using the BCC field</td></tr>
- 6</td> Name of the hiring company is excluded</td></tr>
- 8</td> Location of the position is excluded</td></tr>
- 5</td> No mention of if contract-to-hire, contract, or direct-hire</td></tr>
- 50</td> Require me to fill out onboarding paperwork before position is filled by me</td></tr>
+ [10 to 30]</td> Personalization of the email content to my personal situation (based on degree of personalization)</td></tr>
- 30</td> Unsubscribe link was followed and does not seem to do anything</td></tr>
- 30</td> No unsubscribe link and signs of a mass email</td></tr>
- 15^{n</em></sup></td>} Disregard for direct instructions, e.g., "Not interested" or "Do not contact me again" (where n</em> is the number of times it occurs)</td></tr>
+ [0 to 99]</td> Exactness of fit of the job description to my skills</td></tr>
+ [0 to 30]</td> Signs of remediation for prior score subtractions</td></tr>
+ 40</td> Worked with this recruiter on prior occasion, resulting in at least 1 interview</td></tr>
+ 10</td> Worked with this recruiting firm on prior occasion, with at least 1 submission</td></tr>
</tbody></table>
Reaction (if available for hire)🔗</a></h2>
Total score</th> Response</th></tr></thead>

>80</td> Tailor resume for position, immediately send a response that I'm interested and actively updating my resume for this position</td></tr>
40 to 79</td> Respond to email within 2 hours with more specifically pertinent resume from archive</td></tr>
20 to 39</td> Respond to email within 24 hours with more specifically pertinent resume from archive</td></tr>
10 to 19</td> Respond to email within 36 hours with general resume</td></tr>
0 to 9</td> Respond to email within 3 days, asking questions about the position</td></tr>
-19 to 0</td> Ignore email</td></tr>
-80 to -20</td> (Probation) Monitor future email from that recruiting firm for future quality problems</td></tr>
Below -80</td> Filter future email from the recruiting firm to black hole</td></tr>
</tbody></table>
Reaction (if unavailable)🔗</a></h2>
Total score</th> Response</th></tr></thead>

>80</td> Respond within 24 hours with info that it is a good match but bad timing. Add to contacts as preferred recruiter. Refer out to colleague</td></tr>
40 to 79</td> Respond to email within 36 hours with most up-to-date resume for their records, add to contacts as preferred recruiter</td></tr>
20 to 39</td> Respond to email within 1 week with canned "I'm not looking right now..." response</td></tr>
-19 to 19</td> Ignore email</td></tr>
-80 to -20</td> (Probation) Monitor future email from that recruiting firm for future quality problems</td></tr>
Below -80</td> Filter future email from the recruiting firm to black hole</td></tr>
</tbody></table>


Usernames, not Email Addresses
2017-06-20T11:05:00-04:00
</p>
Yet another email from yet another recruiter, today. This is one more email sent to an address I haven't used since 2014. If their sources of information really 3 years old, why are they pinging me about my work with Ruby on Rails? 3 years ago I wasn't publicly posting any such interest online.</p>
How are they getting this information? I'm not certain, but I suspect it's scraping Github's API. Either way it's getting really fricking annoying. Let me tell you why:</p>
Security🔗</a></h2>
Too many websites out there, these days, accept email addresses as a person's username. While this particular issue isn't the fault of the recruiter sending this email, despite them more than likely pulling my email from GitHub against their terms of use</a>, it is the fault of many login systems.</p>
Without multi-factor authentication enabled on all accounts with an email that can</em> act as the username, a potential attacker already has one piece of the keys to your account. In most cases which involve a cooling-off period between failed login attempts, it's just a waiting game for an attacker to brute-force your account password. If you work on an application with a login system like this, use whatever sway you can to enable a lockout system requiring password reset, not a cooling off period.</p>
For an email account, requiring a password reset wouldn't make sense. Where would you send the reset request, via email? No, in this case a cooling off period makes sense and a user should always, always</em>, ALWAYS</em> have some form of multifactor authentication enabled. For any other service, it doesn't make sense.</p>
As a programmer, I get it. Finding a unique, memorable key for someone to use as one half of their authentication mechanism makes sense. The username/password approach is ingrained in the user experience for most internet goers. An efficiency gain is to use their email address as a string acting as their username, right? Then you can guarantee it's unique (pending validation by clicking a confirmation link in a welcome email) and won't have to validate uniqueness as a database constraint, right? Wrong.</p>
Okay, security aside. Let's assume I have multi-factor authentication on everything that uses the concerned email address. What else might be a problem?</p>
Reply-all🔗</a></h2>
This email keeps coming. Notice the list of recipients? The fact that I can see them means the sender didn't use the BCC field, as is convention. If you haven't been part of a reply-all rage-fest, I do not recommend it. One recipient hits reply-all because maybe the opportunity interests them, maybe they're being a good netizen and replying to every email inquiry they receive, or maybe they have a vacation responder set for every</em> incoming email. If multiple vacation responders do what they're intended? You have an exponentially growing chain of responses. Hopefully it's just 2 auto-responders, in that event.</p>
Okay, no one has vacation responders setup to reply to anyone in this case. What's wrong with the TO or CC field? Do you remember the Github API abuse mentioned earlier? Anyone else who received that email is able to pull in every other address on the list into their own, spammy sales pipeline. And the cycle continues...</p>


On the Importance of Packaging
2017-06-06T10:18:53-04:00
You have an idea and want to turn it into a bit of code to carry it out. What do you do? You open up your IDE/Editor, perhaps you structure it inside of a folder with some default tooling for linting and (if necessary) compiling, and you get the code to work. It's hacky, it's ugly, but it works.</p>
Now that you have something, you want to add one more feature to it or work out a bug. What do you do? You follow the same process. If you're feeling zesty, maybe you init a new git repository at the base of the project, but there's no real point to branching and merging back into master. Backing up the code? Might be an account on Bitbucket, or maybe something with GitLab. They both have free usage tiers with private repositories, and this definitely isn't good enough for the public to see.</p>
This continues with more features hacked on, with no tests and zero documentation (except a comment here and there), all kept 100% private. Maybe you try to make it work on a Digital Ocean server you have stood up as a floating workspace with a persistent internet connection. It was difficult, but it works now. Kinda. Well, some of the time, anyway.</p>
Your code portfolio is still missing a lot. Not a whole lot is visible because not a whole lot is in good enough condition to show anybody. You have a lot of these tiny projects that are riddled with what you know are bad practices, but were so much easier that figuring out the right way to do it. You treat your open source code contributions almost like a stage performance where you have everything perfectly thought out. First you have to write documentation, which means you have to clearly define that snowflake server you have setup to run your script. It also means you have to write usage documentation. Automated tests too, with varying importance depending on the community surrounding the language in which your project is implemented.</p>
This has been my process for years with any number of once-off scripts. Projects that were intended for a recurring, minisculely scoped purpose I would have privately and, when I didn't have need for them anymore or they broke due to changes elsewhere (e.g., web scrapers breaking because website updates), I would just delete the project and call it done. Of course I would only be able to verbally mention "Yeah, I built something like that in my free time" during job interviews, but wouldn't be able to prove it because it was longer than a week ago and I didn't save the work.</p>
What I've found, recently, is that there is a way to ease these growing pains.</p>
Packaging🔗</a></h1>
Every (legitimate) language these days has a package management strategy. Not familiar with the concept of package management? Let's talk about that for a sec.</p>
You have a new project, let's say it's written in Python for now. The standard package format is well defined for installing using pip</code>. It includes dependencies, name of the package, author name and contact info, version constraints for dependencies... a lot of information that is--and should be--standardized. In python, that's consistently defined in setup.py</code> at the base of the repository.</p>
Consider the following repository structure:</p>
├── LICENSE.md</span></span>
├── README.md</span></span>
├── setup.py</span></span>
└── my_project</span></span>
    ├── __init__.py</span></span>
    ├── core</span></span>
    │   ├── admin_commands.py</span></span>
    │   ├── inbox.py</span></span>
    │   ├── initialize.py</span></span>
    │   ├── mentions.py</span></span>
    │   ├── posts.py</span></span>
    │   ├── user_interaction.py</span></span>
    │   └── validation.py</span></span>
    ├── helpers</span></span>
    │   ├── misc.py</span></span>
    │   └── wiki.py</span></span>
    ├── main.py</span></span>
    └── strings</span></span>
        ├── debug.py</span></span>
        ├── posts.py</span></span>
        ├── responses.py</span></span>
        └── urls.py</span></span></code></pre>
We have a project named my_project</code>, which is should be the name of the package in setup.py</code>. Python has the Java-esque convention of import package.name.here</code> to map to package/name/here.py</code> filesystem structure.</p>
Readme🔗</a></h1>
Packages always have a README. If you use a package generator (like bundle gem</code>) and it generates a README, always</em> remove the default generated description (and other TODO-lines) and insert your own. Not sure what your project is about? Note what it currently covers. You can update it later. That's the point of version control.</p>
In the README, include installation instructions that are realistic to your situation. Are there native OS dependencies that you need installed too? Those might not fit well in the setup.py</code> or *.gemspec</code> files. In that case, put it in the README.</p>
A lot of times the boilerplate includes a lot of noise that might not be necessary for publicly publishing a package. Here are the bare minimum requirements for any package I make, public or private. I'm a forgetful person so coming back to a project 2 months later will require some getting up-to-speed again anyway. I like to make it easier for myself with this list of requirements for a README:</p>

Usage</li>
Installation procedure (assume a freshly installed OS)</li>
Description</li>
(if applicable) Assumptions of native platform dependencies</li>
</ul>
Dependencies🔗</a></h1>
Packages provide a very clear definition of what it takes to run a piece of software that was written. I've been hearing for years that successful projects include a very narrowly defined scope and that the best way to do that is through narrowly defined interfaces.</p>
In programming, you might see a junior dev hack away at a magic method that takes inputs in any variety of forms and is able to normalize it. Unless that is the primary function of the method/command/atomic unit they're creating, more experienced developers realize this very easily snowballs into a maintenance nightmare. The same might be said for dependency tracking. Having a clearly defined set of required dependencies (instead of the end user doing trial and error to install everything) will always be better.</p>
Overkill🔗</a></h1>
Python, for example, involves some bootstrapping to setup a package. For me, it has involved looking up prior projects' setup.py</code> file, modifying it to fit the new project, and creating some default structures like my_project/__init__.py</code> containing variables for version (__version__</code>) and author (__author__</code>). It also involves setting up automated testing with unittest and shell scripts to make automated testing easy to execute.</p>
Here's my heuristic when it comes to creating a package:</p>

Are there any</em> dependencies that require a package manager (e.g., pip</code>, npm</code>, gem</code>)?</li>
Does the source code need to be split up into multiple files?</li>
Am I testing my work with any more granularity than full, end-to-end tests?</li>
Do I need to deploy this easily as, e.g., a command line tool?</li>
Do I need to version the releases and have stable, beta, and dev copies of the source code?</li>
Am I tracking code changes with git?</li>
</ul>
If I've answered yes to any one</em> of these questions, I know to create turn the project into a package.</p>
This means I...</p>

create a new git repository (and private remote, for backup purposes)</li>
create a README (in markdown, preferably)</li>
track dependencies I install or remove, as I install or remove them</li>
formulate a testing strategy to make sure everything works as expected (either automated or manual)</li>
create a set of convenience scripts for running repeatable tasks scoped only to the current project (e.g., Makefile, Rakefile, bundler binstubs, shell script)</li>
</ul>


Direnv: for secure coding and kind documentation
2017-06-01T14:43:22-04:00
What environment variables are🔗</a></h2>
Introduction to /usr/bin/env</code>🔗</a></h3>
/usr/bin/env</code> (or just env</code>) echoes out all of the currently set environment variables.</p>
Shebangs at the top of files in Unix-based OS's determine how to execute instead of by file extension.</p>
Might see #!/usr/bin/env python</code> or #!/usr/bin/env ruby</code> as a nicer way of not hardcoding the path to ruby</code> or python</code> binary.</p>
Why not #!ruby</code> or #!python</code>? Environment variables such as GEM_HOME</code>.</p>
Let's take a look into what is set in your terminal environment: (Incoming!</em>)</p>
# echoes out all current environment variables</span></span>
/usr/bin/env</span></span></code></pre>Accessing from the language🔗</a></h2>
Ruby:</p>
puts</span> ENV</span>[</span>'</span>HOME</span>'</span>]</span></span>
# => '/home/myuser'</span></span></code></pre>
Python:</p>
print</span>(os.environ.get(</span>'</span>Home</span>'</span>))</span></span>
# => '/home/myuser'</span></span></code></pre>
PowerShell:</p>
Write-Host $env:HOME</span></span>
# => 'C:\Users\MyUser'</span></span></code></pre>Following good practices🔗</a></h2>
3 basic ways, depending on how framework-y you want to get:</p>

Just-in-time environment variables</li>
Shell script setting environment variables</li>
dotenv</code> / direnv</code> frameworks</li>
</ol>
Just-in-time🔗</a></h3>
Setup: N/a</em></p>
Invoke:</p>
./some_command.sh</span></span>
</span>
# vs</span></span>
</span>
SOME_VAR="derp" ./some_command.sh</span></span></code></pre>Shell script🔗</a></h3>
Setup:</p>
# FILE: environment.sh</span></span>
</span>
export SOME_VAR="derp"</span></span></code></pre>
Invoke:</p>
source ./environment.sh</span></span>
./some_command.sh</span></span></code></pre>Frameworks🔗</a></h3>
Depending on the framework, it auto-loads (a la source</code>) the file when entering the directory. It is hidden by default, hence filename starting with .</code>, so it stays out of your way most times. The file evaluated depends on the framework (.env</code> for dotenv</code>, .envrc</code> for direnv</code>), and evaluates all other files by that name up the directory tree. Might have seen this approach already with other tools (rspec, git, etc.)</p>
Setup:</p>
# FILE: .env</span></span>
</span>
export SOME_VAR="derp"</span></span></code></pre>
Invoke:</p>
cd <project-directory></span></span>
</span>
# if dotenv and no language specific dotenv implementation is loaded program, uncomment the following line:</span></span>
#dotenv</span></span>
</span>
./some_command.sh</span></span></code></pre>Documentation🔗</a></h2>
Document it.</p>
No. Seriously.</p>
When🔗</a></h3>
Always. As soon as it is introduced, even in a branch.</p>
Make the code fail spectacularly (throw unhandled exception with explanation in message) if the environment varaible is not set.</p>
Where🔗</a></h3>
Possible locations:</p>

.env.example</code></li>
.envrc.example</code></li>
within README.md</code></li>
within other file that is versioned with the project</em></li>
</ul>
How (Development Practices)🔗</a></h3>
Documenting environment variables can be done in a README.md</code> (best), but the quickest notes for later reference can be as comments in a .env.example</code> file itself.</p>
JIT environment variable setting🔗</a></h2>
Document example usage in README.md</code>, or other docs</p>
Benefits</strong></p>

Consistent -- Same approach as with /etc/init.d/*</code> scripts</li>
Dependencies -- No additional tooling</li>
Power -- Access to everything the shell has to offer</li>
Transitive -- Switching projects? Variables do not persist to shell session</li>
</ul>
Tradeoffs</strong></p>

Forgetting -- Must remember to include variables on every</em> invocation</li>
Human Error -- Grabbing what variables are needed requires parsing the docs intended for humans (which is less fault-tolerant)</li>
Complicated -- Must be a one-liner, or else it falls under the "sourced files" option by definition</li>
</ul>
Source files manually🔗</a></h2>
Center on the .env</code> convention and include a file (committed to the repository) named .env.example</code>, containing sample values for all variables</em>.</p>
This allows people to optionally use a framework like dotenv</code>. Do not allow yourself to commit .env</code> to the repo, so add it to global gitignore for your workstation.</p>
Benefits</strong></p>

Simplicity -- Includes example, machine-parseable values with the project source code</li>
Dependencies -- No additional tooling</li>
Power -- Access to everything the shell has to offer</li>
</ul>
Tradeoffs</strong></p>

Forgetting -- Must remember to source ./.env</code> in every terminal running the application

Designed well:</em> Blows up spectacularly at earliest possible phase</li>
Designed poorly:</em> Partially executes, then bombs out</li>
</ul>
</li>
Dirty Environment -- Must open new or restart terminal when switching projects</li>
</ul>
Frameworks🔗</a></h2>
I use direnv</code> because I like the set-and-forget approach. For this section, I'm going to focus on direnv</code> implementation.</p>

Create a .envrc</code> file</li>
Globally gitignore .envrc</code></li>
Create a .envrc.example</code> file</li>
Store sanitized, dummy examples in .envrc.example</code></li>
</ol>
Yes, I store credentials on local machine in plaintext. Shame me. If you can think of a way around this, I'm very open to improvement.</p>
Direnv🔗</a></h3>
Found at https://direnv.net/</p>
Benefits</strong></p>

Secure -- Required to manually whitelist versions of .envrc</code> every time it changes (direnv allow</code>)</li>
Automatic -- Sets variables when cd</code> into directory</li>
Self-Contained -- No hooks to additional libraries inside of program (like with dotenv</code>)</li>
Inheiritance -- PWD containing .envrc</code> overwrites ones in parent dirs, on up the chain</li>
</ul>
Tradeoffs</strong></p>

Execution -- Execute from outside of project directory (./project-dir/some_script.sh</code> will not automatically set environment variables in ./project-dir/.envrc</code>)</li>
Lag -- Additional program running on every cd</code> in the shell</li>
Shell -- Limits to one of several popular shells</li>
Variables -- Only handles shell variables (e.g., export SOME_VAR='foo'</code>, not some_var() { echo 'foo' }</code>)</li>
</ul>


First thoughts on Habitat
2017-05-25T13:46:21-04:00
This past week I was at ChefConf going to various panels, intent on learning about Inspec and the best practices therein. While there, half of the panels were on this thing called Habitat, which I originally thought to be Yet Another Docker Alternative ®. First of all, it's not. Like Vagrant's relationship with Virtualbox, it tends to pair nicely with it and use it by default, but it is not the only way to use it by a long shot.</p>
During the first keynote at ChefConf, I was exposed to some of the features of Chef Automate. I had seen a lot of these features a few weeks prior when some Chef people came to my office to demo Automate and some of the features (e.g., Compliance, Delivery), but one thing stood out that I hadn't noticed before: it looked like there was a dependency handling system that, if a dependent package had a version bump or new release, all packages that depend on it are rebuilt with that new version and packages that depend on those newly rebuilt ones are too, rippling through until all dependencies are resolved. This was especially resonant when I was dealing with Apache's mod_ssl and Heartbleed a few years back. You mean if I use this software and a CVE comes out for a low-level dependency, I can patch it and redeploy same day? Easily? Yes.</p>
Wait a second. This isn't what Chef does. Chef is about configuration management, not application dependency resolution nor, hopefully, application deployment. What's this doing in Chef's Automate software then?</p>
I dug in a little more and found out that build system I saw was actually hooked into a project name that tickled at the back of my brain a little bit, like I had seen it before. Its name was Habitat.</p>
I met up with some of my colleagues so we could divvy up this 6-track conference among the three of us, which we would then collaborate on what we learned and report back to our respective teams with this information. I wanted to go to the Habitat sessions, which appeared to be a track all its own. We could really use some better dependency rebuilding triggers</em>, I thought, and I'm not sure how we're going to do it with our Jenkins setup quite yet</em>.</p>
Session 1: Getting Started with Habitat🔗</a></h2>
This first session was lead by the lead engineer of Habitat, Jamie Winsor, and here's where things got a little fuzzy. I hopped into this session, barely making it 2 minutes after it started (thanks to delays surrounding lunch), and Jamie was already on-stage talking about what Habitat actually is. A few weeks prior to this, I had seen some material on the Habitat homepage and it seemed to indicate to my ADHD-ridden mind that it was Yet Another Docker Alternative ®, putting a slightly different interface onto it. The difference was that, in the keynote, it was mentioned it could export to Docker, VM, or bare metal, so that didn't quite fit with my existing mental model.</p>
Jamie carried on about what was involved with habitat while I struggled to piece together what was going on. It was one of those classic moments of an engineer being to deep in the subject to explain things at the level of the audience. That said, the rest of the audience members appeared to be following along just fine, so maybe it was just me.</p>
Here's what I gathered from his talk:</p>

Habitat is written in Rust, in a way that is intended to be entirely self-contained</li>
The intention of Habitat is for sandboxing applications</li>
A prime focus when creating a new bit of software like this was to make it super easy to learn ("Measured in terms of lunch breaks, not weeks to learn")</li>
Habitat is supposed to be 100% platform agnostic</li>
There's this thing called habitat studio (hab studio enter</code>) where you can hop into that super sandboxed environment from your terminal</li>
This supervisor thing appears to aggregate logs and make sure application packages are running correctly (similar to Foreman</a> and its Procfile?)</li>
Somehow, an application package can be scaled up from the supervisor, grouped using a naming convention of [package].[group]</code></li>
Application packages can be started without a supervisor and assigned to an already running one ad-hoc. Why might this be useful? No idea</li>
Building and installing your application's components are done using some hooks, like plan.sh</code> in a specific file in a specific directory</li>
</ul>
This left me with a bunch of questions:</p>

What did I just watch?</li>
What is the purpose of the Habitat Studio?</li>
How is this different from creating a Docker container with the base being Alpine Linux?</li>
Does this run on Windows at all, or are all the hooks built for a Unix-based system?</li>
What would it look like if I switched some of the legacy projects we run over to this orchestration(???) framework today?</li>
</ul>


Intro to rclone
2017-04-19T14:23:19-04:00
The other day, as I was looking for a way to keep Vivaldi in sync across my devices, I stumbled onto a paid product which syncs a local directory with a remote one on one of a few cloud providers. My main ones I like to use are Dropbox, Google Drive, and MediaFire, but I would like to be able to use S3 and other more robust options for when I start scaling up in my storage needs. The problem was that this paid product had a high price tag, showed up in forum spam (and I don't want to encourage that method of marketing), and it only worked with a handful of providers. It did not support Google Drive, for instance.</p>
"Boy, I wish they made something like rsync for these cloud providers...", I remarked.</p>
The cloud storage providers I routinely use all have publicly available APIs, are all targeting the same problem space, and have very similar approaches to solving the problem at hand (albeit with differing free usage tiers). "That shouldn't be too difficult. A weekend hackfest to build a tool like rsync for them!"</p>
I'm going to pause there and just mention that a "weekend hackfest" rarely produces the expected result, as many others have blogged to the same effect.</p>
My first step before starting any project is to get a lay of the land. I googled for "rsync [cloud provider]" (e.g., dropbox) to see if there were any pre-existing solutions I had missed in the first round. If there were none, I could see what came close, figure out what worked for them, and possibly use it as a basis for a new open source tool. Luckily my search stopped there.</p>
Use case🔗</a></h2>
Before we get too far into this, let's establish what I wanted to build (assuming I didn't find a pre-existing solution).</p>
I love Dropbox, but I don't trust their agent if I can't audit their source code. I also don't like that it has sufficient permissions to read all files on my filesystem by default. Seems like a major security hole! I would like to copy individual files or recurse through a directory of files to transfer (a la scp</code>).</p>
I also love Google Drive because it'll follow me wherever my ever-persistent Google account follows me.</p>
Enter rclone</code>🔗</a></h2>
Rclone is an odd tool. Much the same as the first time I used scp</code> and overthinking how to make good use of it, I was afraid of how to get started. Really it's very easy.</p>
Step 1, install rclone per the documentation for your platform. I've got added an option in my dotfiles hooks to install it from homebrew (if I'm on my Mac) and Ubuntu (if I'm on my Chromebook). Mac allows for the super straightforward brew install rclone</code>. Ubuntu is slightly less convenient, but very easily scriptable, so I'll refer you to the installation docs</a> rather than dig any deeper here.</p>
Step 2, run rclone config</code> and follow the prompts. I set it up with my Dropbox (as dropbox</code>) and my Google Drive (as drive</code>) accounts.</p>
Step 3, ???</p>
Step 4, profit!</p>
Examples🔗</a></h2>
Restoring GPG keys🔗</a></h3>
I have my GPG signing keys (password protected, of course) backed up on Dropbox, for this example. I need to grab those locally and import them.</p>
$</span> rclone copy dropbox:gpg_keys/opensource.secret.key ./personal.key</span></span>
$</span> rclone copy dropbox:gpg_keys/opensource.public.key ./personal.pub</span></span>
$</span> gpg2</span> --import --allow-secret-key-import</span> ./personal.key</span></span>
$</span> gpg2</span> --import</span> ./personal.pub</span></span>
$</span> rm ./personal.key ./personal.pub</span></span></code></pre>
If I wanted to be more security-conscious and do this in fewer steps, it looks like gpg</code> allows streaming in the imports via STDIN. Let's try that route!</p>
$</span> rclone cat dropbox:gpg_keys/opensource.secret.key</span> |</span> gpg2</span> --import --allow-secret-key-import</span></span>
$</span> rclone cat dropbox:gpg_keys/opensource.public.key</span> |</span> gpg2</span> --import</span></span></code></pre>Syncing personal documents🔗</a></h3>
Google Drive was originally intended for office-style documents, right? How about we sync it with our Documents directory?</p>
$</span> rclone sync drive:local-documents/ ~/Documents/google-drive</span></span></code></pre>
In this case, any time there's a file or directory that exists solely on Google Drive in the local-documents</code> directory, it syncs it to a subdirectory of our Documents directory called google-drive</code>. If there's a file or directory that exists solely on the local filesystem, it'll do the opposite. When a file exists in both places and it differs, the local copy will overwrite the remote copy.</p>
The bonus for using sync here is you can hook it up to cron for scheduled syncing with one or more of your cloud providers. No more separate Dropbox, Google Drive, and OneDrive agents! I can finally get rid of my Google Drive client on Linu--oh wait... :(</p>
Finding files🔗</a></h3>
What if I don't recall what the directory structure is like on my cloud hosting provider? One of the nuances with rclone ls</code> is that it's like the unix equivalent of ls -R</code>, recursing through each subdirectory to list all files. Only care about going 1 level deep? There's rclone lsd</code> for that.</p>
Here's an example of how I essentially use find</code> (by name) with a remote provider.</p>
$</span> rclone ls drive:</span> |</span> grep</span> '</span>some_file</span>'</span> |</span> sed</span> -e</span> '</span>s/^ *[0-9]* *//g</span>'</span></span>
</span>
MyFiles__local/some_file.notes.txt</span></span>
derp.some_file/fizz_buzz.doc</span></span></code></pre>
Before you puke over resorting to sed</code>, I only use it to trim some leading whitespace and numbers (indicating file size). You can leave that in if you prefer, but I like things that can be easily scripted out. This allows for straight-up copy and paste for later iteration in a shell script using a bash for-each loop.</p>
Suggested improvements🔗</a></h2>
Rclone is far from a perfect tool. As a reasonably well-versed Linux CLI junkie, I am very used to traversing the filesystem, copying/moving/renaming files locally, and transferring them across the network via SSH, CIFS (Samba, a.k.a., Windows share), and SFTP. I would love it if it were just intuitive to use the unix-y commands ls</code>, rm</code>, cp</code>, mv</code>, and so on as a subcommand of rclone. I would also accept it being a mostly drop-in replacement for rsync.</p>
That said, I understand that rclone is cross-platform and intended to have a unified interface for Windows and unix-y systems. Is there any reason we can't make use of command aliases here? Why not have rclone copy</code> be the exact same interface as rclone cp</code>? And for the Windows users, aliasing rclone dir</code> to rclone ls</code>?</p>


Iron Crystals
2017-01-07T21:30:00-05:00
Today, I've begun looking into compiling the Crystal compiler from another
direction, possibly opening up the cross-platform capability of Crystal to all
of the operating systems certain other popular languages already support.</p>
In past weeks, I'd begun digging into porting crystal to run under Alpine Linux,
that way I can have a very small, very specialized docker container to compile
Crystal projects, or even run their binaries. The problems I ran into were that
Crystal is a garbage-collected language. Why is that? Why not make it have zero
garbage collection, if at all possible, similar to how C, C++, or Rust have it?</p>
Crystal has boasted that it is completely self-hosted, meaning it can be
compiled, contributed to, and more using the Crystal language itself. This comes
as a liability when trying to port it to another platform because, in order to
compile crystal for that platform, you must already have a Crystal compiler for
it. See the dilemma? Chicken and the egg.</p>
It's okay though, according to the docs, there is a way to port it to other
platforms by using the --cross-compile</code> and --target</code> compile-time flags. I
recently tried this in my endeavor to get it to work with Crystal, but failed in
that there were segfaults all along the way. The most cryptic? Something about
sh</code> and not being able to access a value since it was out of bounds. In sh</code>.
Yeah.</p>
Okay, I've followed every documented piece of evidence in order to cross-compile
the Crystal compiler using --cross-compile</code>, now what? The docs support the
alternative of stepping through since v0.0.1, when the compiler was written in
Ruby, to create a crystal compiler that works on the other platform. Given the
number of released versions since then, and how many more there will likely be
in the future, this doesn't seem very future-proof either. What else can we do?</p>
Here's a thought:</p>
Crystal has boasted since day zero (day one?) that it is inspired by the Ruby
programming language. Why wouldn't that inspire alternate implementations too?
why can't we create a JRuby or Rubinius or even Opal of Crystal? That might
satisfy some of the pre-requisite of porting the official Crystal compiler to
another platform by using the cross-platform traits of the other language. How
should we proceed?</p>
That's the ultimate question. Over the coming weeks, I'm going to be digging
into Rust, Crystal, and basic, lower-level concepts involved in constructing a
programming language.</p>


Local gems with Bundler
2016-12-28T17:27:06-05:00
Testing an unreleased version of a gem? Want to develop 2 unreleased projects
that are based on each other and not have to worry about the following?</p>
gem </span>'</span>some_gem</span>'</span>,</span> path:</span> '</span>../my-dev-snapshot</span>'</span></span></code></pre>
Leaving the Gemfile</code> as such will screw with your project history, so we want
the version of Gemfile</code> as it will be when the gem is released, keeping that
version in our "git memory". See the following:</p>
$</span> bundle config local.some_gem</span> "</span>$(</span>realpath</span> ../my-dev-snapshot)</span>"</span></span></code></pre>
This will allow your Gemfile</code> to remain pristine without the path: '../foo'</code>
hacks so others can set their own path to the gem source directory.</p>
Caveats:🔗</a></h2>
The gem, in this case some_gem</code>, must be pointed at a git repository. In this
case, it would need to be:</p>
gem </span>'</span>some_gem</span>'</span>,</span> github:</span> '</span>foo/bar</span>'</span>,</span> branch:</span> '</span>master</span>'</span></span></code></pre>
This would allow us not only to optimize network traffic so we don't make calls
out to the git repository all the time, but also point it toward a local working
copy.</p>
There is an additional caveat, though. The given branch -- in the example,
master</code> -- must match the branch of the current working copy at the path
specified with bundle config</code> from earlier.</p>
Implements Programmer - Posts

Lessons in EKS

Intro to Blogging: Engineer Edition

Dopamine to Create

Neurodivergence and Remote Work

Facets of a Good README

Nix Is Worth the Complexity

7 principles of a good sysadmin

Chef goes RedHat

Rebranding of products🔗</a></h2> This part is a relatively minor, semantics change for clarity and consistent brand. Here's a quick guide:</p> Chef</code> (config management tool) → Chef Infra</code></li> Habitat</code> → Chef Habitat</code></li> InSpec</code> → Chef InSpec</code></li>

License change🔗</a></h2> Here's where things get a little dicey. All of the official statements from Chef have indicated the following:</p> The source code for all Chef products (listed above) are to have their source code released under the Apache 2.0 license.</li>

InSpec: You're Doing It Wrong

Blogging made easy

What is a Symbol?

Recruiter quality score

Usernames, not Email Addresses

On the Importance of Packaging

Direnv: for secure coding and kind documentation

Following good practices🔗</a></h2> 3 basic ways, depending on how framework-y you want to get:</p> Just-in-time environment variables</li> Shell script setting environment variables</li>

Where🔗</a></h3> Possible locations:</p> .env.example</code></li> .envrc.example</code></li> within README.md</code></li>

First thoughts on Habitat

Intro to rclone

Iron Crystals

Local gems with Bundler
