+++ /dev/null
----
-title: Tuesday, August 8, 2023
-tags: ["daily"]
-date: 2023-08-08T14:14:33.182-04:00
----
-
-
-After about a month helix is something I dropped. I work on linux, mac's and Windows envs and if my editor can't do the same I drop it. Unfortunately there's a weird bug in helix where some keypresses (like `escape` to get out edit mode) take a second to register. That seriously messes with my workflow where I mash `esc` and then quickly jump to another section.. Enough to make using Helix a frustrating experience.
-
-Instead I've been testing [Lunar Vim](https://www.lunarvim.org/) which is a bunch of plugins pre-configured for neovim. My goal is to be able to get set up on my editing environment quickly and neovim is not that. But Lunar Vim seems to have mostly sensible defaults (except for the nerdfonts requirement)
-
-
-
-
-
-
-
\ No newline at end of file
+++ /dev/null
----
-title: Sunday, June 18, 2023
-tags: ["daily"]
-date: 2023-06-18T06:30:13.680-04:00
----
-
-
-Been a few days since an update here - mostly because I've been quite swamped. The *Reddit Fiasco* has been somewhat freeing in that I've just moved on from the platform. I never really contributed much in my 13+ years outside of a few DMs and responding to help. I haven't actually replaced it with anything. I like kbin/lemmy.. but I'm really not looking for the same thing. I think I'm looking for smaller communities... again the IndieWeb beckons.
-
-I've also been spending a lot of time working on my PBBG [Rising Legends](https://www.risinglegends.net). It's still not even ready for an alpha.. but the bones are there. It's actually been great to work on it and it's finally at the point where I can start adding content. I took a bunch of extra time to integrate with [AirTable](https://airtable.com/) which has been lovely (if a bit laggy) to work with. Things are finally at the point where I can focus on content for a week or two and then start getting some more dedicated testers involved.
-
-I've also been playing around with funding models for the game that stays away from MTX. I've never been a fan of MTX and always been a fan of FREE. I'll probably have to do a longer post on the matter as there's quite a lot to unpack I think.
-
-
-
-
-
-
-
\ No newline at end of file
+++ /dev/null
----
-title: Friday, June 2nd, 2023
-tags: ["daily"]
-date: 2023-06-02T10:36:35.285-04:00
----
-
-
-There was something magical about uploading the wordpress zip file to your shared host and going through `install.php`
-
-
-
-
-
-
-
\ No newline at end of file
+++ /dev/null
----
-title: Wednesday, June 21, 2023
-tags: ["daily"]
-date: 2023-06-21T05:57:49.127-04:00
----
-
-
-I don't do a lot of posts with images.. mostly because I do all my work in markdown and rarely preview as I'm going. However this new outliner writing method has me paranoid and constantly previewing the content as I go and it's made me add images for the first time in a long time...
-
-
-
-
-
-
-
\ No newline at end of file
+++ /dev/null
----
-title: Thursday, June 22, 2023
-tags: ["daily"]
-date: 2023-06-22T10:11:03.546-04:00
----
-
-
-Trying out [Helix](https://helix-editor.com/) and just the ease of getting up and running with all the features that I want out of the box was incredible? Like that alone should be enough for new devs to get started. Vim/NeoVim is great if you're already in that world and have years of configs/settings that you're used to. But if you're just starting out with modal editors, you can't compete with everything just working out of the box?
-
-I literally have a git repo where I track my vim config and all the plugins that I use so that I don't need to remember that. Being able to skip out on all of that is a wonderful idea/change.
-
-
-
-
-
-
-
\ No newline at end of file
+++ /dev/null
----
-title: Saturday, June 3rd, 2023
-tags: ["daily"]
-date: 2023-06-03T06:08:02.255-04:00
----
-
-
-In a strange twist, it appears that I may be more of a morning person than I though. The last few days I've been going to bed by 9:30 (kids...) and then waking up at 4am and starting my day. It's been a strange experience. By about 7am I've managed to get a bunch of work done and had time to walk the dog before the 30 degree weather kicks in and melts her.
-
-Learned about the `hugo -w` flag which allows hugo to constantly watch/build my posts. So the process from outline -> blog is a bit more streamlined.
-
-
-
-
-
-
-
\ No newline at end of file
+++ /dev/null
----
-title: Sunday, June 4th, 2023
-tags: ["daily"]
-date: 2023-06-05T09:47:29.995-04:00
----
-
-
-I've been noodling around in the fediverse for a long time now, entranced by some of the ideas Diaspora had when it first launched.. and mostly just continually disappointed by the various communities that tend to spring up around it. Hoping this foray into the activity-pub based world (kbin, lemmy, pleroma) will be different. But already the main lemmy instance is proving to be a bad one specifically because of the devs. Why can't we just have nice things?
-
-Created the first "Garden" post about my [Outliner](/gardens/outliner) project which will serve as a long running (forever updating) "home page" for the project.
-
-
-
-
-
-
-
\ No newline at end of file
+++ /dev/null
----
-title: Monday, June 5th, 2023
-tags: ["daily"]
-date: 2023-06-05T14:14:27.562-04:00
----
-
-
-Caching is a bad idea until you have some concrete numbers as to why you're caching a thing.
-
-I feel like all reddit has to do is say "Ok, we've changed the pricing to something more reasonable" and then everyone will pat themselves on the back and start paying for their own content. Reddit exists because users devote the time to submit new content and to MODERATE IT all for free. They want you to continue to do that, but then charge users to read it the way they want to read it? Kind of crazy. All this is to say that I'm being drawn back in to the whole fediverse stuff....
-
-
-
-
-
-
-
\ No newline at end of file
+++ /dev/null
----
-title: Tuesday, June 7th, 2023
-tags: ["daily"]
-date: 2023-06-07T06:15:03.434-04:00
----
-
-
-Today's the day I finally start looking into mouse support on the Outliner. It's honestly been a great tool and a daily driver for my notes/blog updates. I don't really see much need in mouse support, but someone saw me using it and was intrigued. And I had to reluctantly tell them it doesn't support a mouse...
-
-With the fires out in Quebec continuing, air quality has continued to deteriorate. For a good chunk of yesterday everything had a bit of an orange hue. You can track air quality forecasts here - https://firesmoke.ca/
-
-
-
-
-
-
-
\ No newline at end of file
+++ /dev/null
----
-title: Friday, June 9th, 2023
-tags: ["daily"]
-date: 2023-06-09T05:49:51.517-04:00
----
-
-
-[Rising Legends](https://www.risinglegends.net) is finally getting to the point where the core game loop is in. It's exciting to be back in this space after so many years. Every couple years I build the mechanics/system for some PBBG - often to test out some new architecture/design patterns. This time I really focused on getting something playable rather than the code behind it. The next step is to start getting some 3rd party testers and early alpha feedback!
-
-
-
-
-
-
-
\ No newline at end of file
+++ /dev/null
----
-title: Thursday, May 24, 2023
-tags: ["daily"]
-date: 2023-05-24T23:11:51.166-04:00
----
-
-
-I'm attempting to adopt [Dave Winer's](https://scripting.com) posting setup of daily updates. I realize there's so much that I want to just say over the course of the day. Some of it might morph into full blown posts.. some of it will just disappear. I originally thought I would try and transition some into "tweets" and "toots" or whatever mastodon calls them.. but for now they'll just live here.
-
-I moved to hugo about a year ago, mostly to gain the easy github pages integration... I'm only now beginning to discover some of the neat details that made it so popular.
-
-
-
-
-
-
-
\ No newline at end of file
+++ /dev/null
----
-title: Thursday, May 25, 2023
-tags: ["daily"]
-date: 2023-05-25T21:20:53.011-04:00
----
-
-
-I have a sneaking suspicion that [Azure DevOps](https://azure.microsoft.com/en-us/products/devops/) is way better than GitHub but no one is willing to admit it.
-
-For whatever reason this year has been really bad on my allergies. I'm trying a variety of over-the-counter medications but will likely have to resort to something a bit stronger before the summer is out.
-
-I just realized that all the "Daily Updates" are already one day out of sync due to the fact that I don't know what day it is. Just did an update to fix this once and for all.
-
-
-
-
-
-
-
\ No newline at end of file
+++ /dev/null
----
-title: Friday, May 26, 2023
-tags: ["daily"]
-date: 2023-05-26T13:33:16.642-04:00
----
-
-
-Everyone wanting their own streaming service is just re-inventing cable and the whole reason that services like Netflix "distrupted". Gabe Newell said it best "[Piracy] is a service issue". And the service here sucks.
-
-
-
-
-
-
-
\ No newline at end of file
+++ /dev/null
----
-title: Tuesday, May 30, 2023
-tags: ["daily"]
-date: 2023-05-30T11:12:44.329-04:00
----
-
-
-I feel like I'm ALMOST there on the process for deployments. I think right now I need to adjust the build process so that after I write a post in the outliner, it gets pushed up to github automatically (the raw data) which builds and deploys it to github pages. That way the "built" info isn't in github.
-
-
-
-
-
-
-
\ No newline at end of file
+++ /dev/null
----
-title: Wednesday, November 22, 2023
-tags: ["daily"]
-date: 2023-11-22T03:07:16.121-05:00
----
-
-
-[Lunar Vim](https://www.lunarvim.org/) has become my new editor configuration. The easy of setup + inter-plugin configuration trumphs the requirement for me to craft my own personal installation. Out of the plugins that I use, everything works well enough. It's been enough that my configuration has become quite minimal.
-
-**.config/lvim/config.lua**
-
-<script src="https://gist.github.com/AngeloR/898d59df9ca86e4807b72f3e28c7164f.js"></script>
-
-To be honest, there's actually very little in here that is required apart from the relative line numbering. The same configuration works across platforms (linux, windows with wsl, mac) and honestly I've run into no real issues with it. There are some minor peculiarities that I've had to re-learn (mostly just which search shortcut based on the scenario). The web-based docs are adequate.. but honestly I think the in-app configuration stuff is a lot clearer to navigate.
-
-
-
-
-
-
-
\ No newline at end of file
+++ /dev/null
----
-title: Tuesday, November 21, 2023
-tags: ["daily"]
-date: 2023-11-22T00:25:43.339-05:00
----
-
-
-[Lunar Vim](https://www.lunarvim.org/) has become my new editor configuration. The easy of setup + inter-plugin configuration trumphs the requirement for me to craft my own personal installation. Out of the plugins that I use, everything works well enough. It's been enough that my configuration has become quite minimal.
-
-**.config/lvim/config.lua**
-
-<script src="https://gist.github.com/AngeloR/898d59df9ca86e4807b72f3e28c7164f.js"></script>
-
-To be honest, there's actually very little in here that is required apart from the relative line numbering.
-
-
-
-
-
-
-
\ No newline at end of file
+++ /dev/null
----
-title: river.xangelo.ca
-tags: ["daily"]
-date: 2024-01-31T21:36:06.372-05:00
----
-
-
-I've been running [https://river.xangelo.ca](https://river.xangelo.ca) for a number of years now - going back to tweak things ever-so-slightly. This is the first time I've been happy about the whole system. This whole "build it for yourself" phase seems to be working...
-
-The river is just a few websites that I like to follow, but mostly sporadically. I'll hop on and check things out when I can because I find the content to be pretty interesting. It reads some RSS feeds, and constantly updates the main page with any new posts that I haven't seen yet.
-
-
-
-
-
-
-
\ No newline at end of file
+++ /dev/null
----
-title: Links!
-tags: ["daily"]
-date: 2024-02-04T23:00:12.253-05:00
----
-
-
-I've added a new [Links](https://xangelo.ca/page/links) section to my blog to highlight individual-owned/operated blogs that I follow. It's kind of a throwback to the old blog-world pre web2.0 where you just maintained a link to other blogs that you wanted to support.
-
-
-
-
-
-
-
\ No newline at end of file
+++ /dev/null
----
-title: Streamlining my deployment process
-tags: ["daily"]
-date: 2024-02-07T22:43:46.372-05:00
----
-
-
-I've been hosting my website over on GH Pages for a long time now. Honestly I don't have too many problems with it or the process. But I was skeptical when I first started.. and as such I wanted to actually commit the build artifacts to a separate repo so that I could inspect them and not rely too much on GH Actions. I feel like I'm finally at the point where I'm ok with things so I took some time to rework my process a bit.
-
-Now when I commit and push to github, I'm utilizing GitHub Actions to build and publish the build directly to GH Pages. Literally just lifted the configuration that [hugo publishes](https://gohugo.io/hosting-and-deployment/hosting-on-github/) because I didn't care to spend time figuring it out myself and did a bit of re-work on my repo.
-
-I know eventually I'll have to change some of this as I'd like to add some more functionality to my site to support some [IndieWeb](https://indieweb.org) functionality like [WebMentions](https://www.w3.org/TR/webmention/). In order to do that I need to run various server components to be able to receive mentions and (Perhaps directly through Caddy) and serve up [WebFinger](https://webfinger.net/).
-
-
-
-
-
-
-
\ No newline at end of file
+++ /dev/null
----
-title: An RSS reader view
-tags: ["daily", "river"]
-date: 2024-02-09T09:29:09.900-05:00
----
-
-
-The [river](https://river.xangelo.ca) is nice.. but there are some blogs that I like to actually keep up with. But I do only remember sporadically.
-
-I was a big user of Google Reader.. which was killed quite suddenly. I tried numerous alternatives but I just never found one I liked. Maybe this newest addition to the river can be it.
-
-There's a second [reader view](https://river.xangelo.ca/reader.html) that displays the feeds, and also read/unread counts for all of it.
-
-> *edit*: It comes to my attention that I've used the words "Reader View" incorrectly. It's not a stripped down view of the content. It's more of an "RSS Reader View" where the view resembles what you'd expect of a more traditional RSS reader as opposed to the "River" view.
-
-
-
-
-
-
-
\ No newline at end of file
+++ /dev/null
----
-title: Outliner [Garden]
-tags: ["project", "outliner", "garden"]
-summary: A Garden (Continually updating) post about the current Outliner Status.
-date: 2023-11-22T01:03:25.836-05:00
----
-
-
-This page will serve as the current state of my Outliner project.
-
-Github - https://github.com/angelor/outliner
-
-## Current Version: 0.0.1-alpha-4
-
-### Known Bugs
-
-- None
-
-### Planned Features
-
-There's no timeline on any of these features, but their current order is rougly the order that I'll be working on them on.
-
-#### Mouse Support
-
-I've been holding off on this a bit, mostly because I don't want to deal with the node-reordering during dragging/dropping. However, I think it'll be one of those things that are really helpful even if you're a keyboard only user.
-
-#### Copy Nodes
-
-Hitting `ctrl`+`c` on a hilighted node should all you to copy the content of that particular node. Pasting the node should happen in one of two ways: If you are in edit mode, it should paste just the content of the node in your current editor. If you are in navigation mode, it should paste a new copy of the node at your cursor, pushing the existing content down.
-
-This lends itself really well to "referential nodes" as well
-
-#### General Publishing
-
-General publish is working in alpha-4. I'm actually quite pleased with it so far. The configuration is abstracted away, but does expect that you're going to be operating out of the `/home` directory specifically. But, apart from that, the actual work to generate posts, inculding drag+drop support for images, and also code is allowing this to become a really all-rounded tool for me.
-
-#### Referential Node Links
-
-Each node in the outliner is saved as its own JSON file. The reason for this was to eventually implement the idea of "Referential Node Links". A single node, that can be referenced in multiple outliners. Updating the node from any of the outlines will update the node in every outline. The idea is to give you the ability to re-use certain templates/bits of code. If you hit `ctrl+shift+c` it should copy the node reference and allow you to paste that in
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+++ /dev/null
----
-title: Rising Legends [Garden]
-tags: ["project", "rising-legends", "garden"]
-summary: A Garden (Continually updating) post about the current status of a PBBG Game I'm working on.
-date: 2023-09-01T13:52:42.461-04:00
----
-
-
-This page will serve as the current state of [Rising Legends](https://www.risinglegends.net), a Persistent Browser-Based Game.
-
-## Known Bugs With Current Release
-
-- When you swap apps on mobile the websocket disconnects... then it reconnects it always throws you back to the Explore screen.
-
-- With the introduction of the Vigor System, the balancing of everything is completely out of whack...
-
----
-
-{{% include "./static/CHANGELOG.md" %}}
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
--- /dev/null
+---
+tags:
+ - publish
+title: index
+date: 2024-12-12T14:52:24.197Z
+lastmod: 2024-12-12T15:31:04.389Z
+---
+# Angelo Rodrigues
+
+I’m Angelo, a seasoned Software Developer with over 20 years of experience spanning various roles and disciplines. My journey began as a “Full Stack” developer/designer in the early 2000s, inspired by projects like [2Advanced Studios v3](https://www.webdesignmuseum.org/exhibitions/2advanced-studios-v3-2001). Over time, I’ve moved progressively deeper into the stack. Since 2012, my technical focus has been on Backend development, DevOps, and Site Reliability Engineering (SRE), but my broader passion lies in “Leveling Up Engineering”—helping teams and organizations reach their next stage of growth.
+
+I specialize in joining teams that are ready to evolve from “We have an MVP” to “We are a company.” My experience includes founding a startup in the Security space, consulting on both technical challenges and engineering management, leading teams as a People Manager, and driving initiatives as a Principal Engineer. While my career spans a variety of industries, I’ve gravitated toward EdTech and Security, with a few ventures into Cryptocurrency.
+
+This blog serves as a living archive of the insights, techniques, and lessons I’ve gathered along the way. Updates are sporadic, reflecting the ebb and flow of life, but my goal is to provide something valuable for every reader—whether you’re an engineer, a manager, or just curious about the craft.
+
+If you have questions or want to connect, feel free to email me—let’s figure it out together.
--- /dev/null
+---
+title: All Posts
+tags:
+ - publish
+date: 2024-12-12T14:45:17.752Z
+lastmod: 2024-12-12T15:38:57.828Z
+---
+# All Posts
+++ /dev/null
----
-title: "About"
-date: 2020-04-29T23:38:37-04:00
----
-
-I'm Angelo. I've been working in Software Development for the last 20 years or so, doing a bit of everything. I started my journey as a "Full Stack" developer/designer back when [2Advanced Studios launched v3](https://www.webdesignmuseum.org/exhibitions/2advanced-studios-v3-2001) and over the years I've moved further and further "backward" in the stack. Since about 2012 my technical focus has been around Backend/DevOps/SRE work and more broadly around "Leveling Up Engineering". I have a history of joining teams that are ready to take the next step from "We have this MVP" and helping them get to "We are a company". I've gotten to do a bit of everything from Founding my own Startup in the Security Space, Consulting (Both technical and Engineering Management), Leading teams as a People Manager, and contributing as a Principal Engineer. My own career seems to have a focus on Ed Tech + Security, with a bit of dabbling in Crypto Currency.
-
-This blog is a way for me to document all of the things I've learned and am still learning about everything I run across. The updates happen sporadically (as life does) and I hope that you'll be able to find something interesting here.
-
-If you have any questions, don't hesitate to drop me an email and we can figure it out.
+++ /dev/null
----
-title: Links
-date: 2024-02-05T09:41:19.161-05:00
----
-
-
-Here's a collection of blogs that I enjoy and try and keep up with. They aren't listed in any particular order. I'm really looking to re-live the nostalgia of early webrings in the pre web2.0 era. Infact, at some point I'm going to have to move this blog off GitHub pages so that I can start experimenting with some of the IndieWeb functionality...
-
----
-
-### Drew Devaults Blog - https://drewdevault.com
-
-Drew’s blog has been something I’ve long followed. He’s the guy behind [SourceHut](https://sourcehut.org) which has quickly become my favourite spot to host projects. He has a long list of projects that he works on outside of that, but almost everything he does is open source in some way. His blog contains a bit of everything from software to politics.
-
-### Rosano - https://utopia.rosano.ca
-
-Rosano is someone I had known a long time ago. We drifted apart for some time.. and then randomly discovered that we had both ended up working in software somehow. Rosano has had the courage to do something I have never been able to - dedicate himself to making things better. We all like to make things better for people as things cross our path - but Rosano seems to made it a lifestyle choice for him. Part digital nomad part philosophy, his blog is something I subscribe to via email because I don’t like to miss a post. He also has a wonderful podcast where he talks to non-celebrities wherever he finds himself.
-
-### Dave Winer - https://scripting.com
-
-Dave has been involved in the creaction of RSS, almost all of the surrounding tooling, and a bunch of uses (OPML, Podcasts, Blogging?). He was also my first introduction to the world of outliners and rivers of news. He’s also really easy to get in touch with if you’re interested in any of those things. I definitely recommend checking out his blog, if only to keep up with the tools and projects that he works on.
-
-### Caleb (Zalaah) - https://cardboardmachete.com/blog
-
-Zalaah is a pretty well known member of the Persistent Browser-based Gaming community. He’s got a bunch of different games out and still manages to find time to be very involved in the community. He has some really interesting posts on his blog about the approach he takes with various components of his PBBGs and the technical minutae around it.
-
-### Gaffer on Games - https://www.gafferongames.com
-
-Glenn’s website is the primer on working on multiplayer networking for gaming. In addition to incredibly detailed posts about things like “Building reliability over UDP” or LERP/SLERPing. When I was first starting out building multiplayer systems in gaming his blog (andeventually the video posts) were absolutely vital to geting going. If you’re ever curious about that stuff I definetely recommend you take a peek at their site and take plenty of notes.
-
-
-
-
-
\ No newline at end of file
--- /dev/null
+---
+tags:
+ - publish
+ - page
+ - feature
+title: about
+date: 2024-12-09T19:14:24.046Z
+lastmod: 2024-12-12T14:34:22.538Z
+---
+# Angelo Rodrigues
+
+I'm Angelo. I've been working in Software Development for the last 20 years or so, doing a bit of everything. I started my journey as a "Full Stack" developer/designer back when [2Advanced Studios launched v3](https://www.webdesignmuseum.org/exhibitions/2advanced-studios-v3-2001) and over the years I've moved further and further "backward" in the stack. Since about 2012 my technical focus has been around Backend/DevOps/SRE work and more broadly around "Leveling Up Engineering". I have a history of joining teams that are ready to take the next step from "We have this MVP" and helping them get to "We are a company". I've gotten to do a bit of everything from Founding my own Startup in the Security Space, Consulting (Both technical and Engineering Management), Leading teams as a People Manager, and contributing as a Principal Engineer. My own career seems to have a focus on Ed Tech + Security, with a bit of dabbling in Crypto Currency.
+
+This blog is a way for me to document all of the things I've learned and am still learning about everything I run across. The updates happen sporadically (as life does) and I hope that you'll be able to find something interesting here.
+
+If you have any questions, don't hesitate to drop me an email and we can figure it out.
--- /dev/null
+---
+tags:
+ - publish
+ - page
+title: links
+date: 2024-12-09T19:01:16.962Z
+lastmod: 2024-12-12T04:36:10.397Z
+---
+# Links
+
+Here's a collection of blogs that I enjoy and try and keep up with. They aren't listed in any particular order. I'm really looking to re-live the nostalgia of early webrings in the pre web2.0 era. In fact, at some point I'm going to have to move this blog off GitHub pages so that I can start experimenting with some of the IndieWeb functionality...
+
+***
+
+### Drew Devaults Blog - https://drewdevault.com
+
+Drew’s blog has been something I’ve long followed. He’s the guy behind [SourceHut](https://sourcehut.org) which has quickly become my favourite spot to host projects. He has a long list of projects that he works on outside of that, but almost everything he does is open source in some way. His blog contains a bit of everything from software to politics.
+
+### Rosano - https://utopia.rosano.ca
+
+Rosano is someone I had known a long time ago. We drifted apart for some time.. and then randomly discovered that we had both ended up working in software somehow. Rosano has had the courage to do something I have never been able to - dedicate himself to making things better. We all like to make things better for people as things cross our path - but Rosano seems to made it a lifestyle choice for him. Part digital nomad part philosophy, his blog is something I subscribe to via email because I don’t like to miss a post. He also has a wonderful podcast where he talks to non-celebrities wherever he finds himself.
+
+### Dave Winer - https://scripting.com
+
+Dave has been involved in the creation of RSS, almost all of the surrounding tooling, and a bunch of uses (OPML, Podcasts, Blogging?). He was also my first introduction to the world of outliners and rivers of news. He’s also really easy to get in touch with if you’re interested in any of those things. I definitely recommend checking out his blog, if only to keep up with the tools and projects that he works on.
+
+### Caleb (Zalaah) - https://cardboardmachete.com/blog
+
+Zalaah is a pretty well known member of the Persistent Browser-based Gaming community. He’s got a bunch of different games out and still manages to find time to be very involved in the community. He has some really interesting posts on his blog about the approach he takes with various components of his PBBGs and the technical minutae around it.
+
+### Gaffer on Games - https://www.gafferongames.com
+
+Glenn’s website is the primer on working on multiplayer networking for gaming. In addition to incredibly detailed posts about things like “Building reliability over UDP” or LERP/SLERPing. When I was first starting out building multiplayer systems in gaming his blog (and eventually the video posts) were absolutely vital to getting going. If you’re ever curious about that stuff I definitely recommend you take a peek at their site and take plenty of notes.
--- /dev/null
+---
+tags:
+ - publish
+ - aws
+ - elasticbeanstalk
+ - guide
+summary: Funky ElasticBeanstalk behaviours
+title: AWS ElasticBeanstalk Gotcha's
+date: 2024-12-09T16:29:52.051Z
+lastmod: 2024-12-12T05:45:27.752Z
+---
+# ElasticBeanstalk Gotchas
+
+I've been lucky and unlucky enough to work with AWS ElasticBeanstalk for a number of years and here's a list of footguns.
+
+## Degraded on 4xx Errors
+
+If your app is doing any kind of authentication you're going to run into an eventual issue where every time someone incorrectly authenticates you run the risk of the environment moving into a "degraded" state. This is because ElasticBeanstalk counts 4xx errors as "application" errors and if you have too many of them it thinks your app is in a degraded state and will attempt to redeploy.
+
+You can fix it two ways:
+
+1. There's an option in the UI for you to check
+2. A custom ConfigDocument
+
+### UI option
+
+You can find this section under Health monitoring rule customization for the environment.\
+
+
+### Config Document
+
+This is displayed as you would for TypeScript based CDK, but I think it's pretty straight forward to convert?
+
+```js
+ {
+ namespace: 'aws:elasticbeanstalk:healthreporting:system',
+ optionName: 'ConfigDocument',
+ value: JSON.stringify({
+ Rules: {
+ Environment: {
+ Application: {
+ ApplicationRequests4xx: {
+ Enabled: false
+ }
+ }
+ }
+ },
+ Version: 1
+ })
+ },
+```
+
+## Deployment Options
+
+EB is great for getting started - but as you start getting users you'll start running into outages as you deploy your application. That's because, by default, ElasticBeanstalk uses "All at Once" as a deployment mode. This means it goes through your list of instances, sets up the new version, then restarts the proxy which means downtime on each instance.
+
+Depending on your load, and application start time, you can see periodic downtime for a good chunk of your deployment. This is no-bueno. Your best bet is to swap to either:\
+\- Rolling with an additional batch\
+\- Immutable
+
+### Rolling with an Additional Batch
+
+In this scenario, EB will launch a new set of instances with the new application, and then cycle through the old instances updating as it goes (similar to the rolling method). By spinning up the new batch of instances first, EB can ensure that your application maintains its availability/throughput during the deployment.
+
+### Immutable
+
+In this scenario, EB will spin up an entirely new set of instances and do a hard swap from the old ones. This takes the longest amount of time, depending on the number of instances you're running simultaneously.
--- /dev/null
+---
+tags:
+ - aws
+ - docker
+ - amplify
+ - publish
+date: 2023-12-05
+summary: Amplify has some odd docker limitations it doesn’t want to fix
+title: Amplify Docker Limitations
+lastmod: 2024-12-10T19:47:15.594Z
+---
+# Amplify Docker Limitations
+
+I've recently had the chance to do some work with Amplify in AWS and I'm surprised how simultaneously feature rich and half baked it is. It seems if you're in to click-ops you'll be fine in Amplify until you hit a problem.
+
+If you're deploying through amplify you'll eventually hit this problem:
+
+`[BUILD_CONTAINER_UNABLE_TO_PULL_IMAGE: Unable to pull customer's container image. CannotPullContainerError: Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit]`
+
+Which seems perfectly reasonable at first glance. Nowhere in your project setup did you really define credentials for your docker account.
+
+But if you dig a bit it gets weird: You actually can't define credentials for a docker account.. and the limits for the free tier are high enough that you probably wouldn't hit them with a small team - 100 pulls every 6 hours per IP.
+
+Turns out you actually share a build pool with other users in amplify - which makes sense. But that in turn means that every so often your build pool will hit the limits for pulling containers from docker hub. And since there's no way to set credentials for your pull.. you're just kinda left hoping that if you retry things will work. There's probably a large enough build pool that it works sometimes, but there are reports of it taking up to an hour or so.
+
+So you think: Oh yeah, no worries -> I'll just set up ECR, and configure Amplify to grab my image from that repo instead. But ECR is a non-starter. It turns out that you can't assign any kind of roles to amplify to allow authentication against an ECR.. so if you're hosting your image in ECR, you have to make the entire repo publicly accessible.
+
+If you're using customized images unfortunately this is really the only way to go. You have to build and deploy to ECR and make the repo publicly available.
+
+If you're using pre-built images, amazon actually has their own public replica of popular docker images <https://gallery.ecr.aws/>. If you can find your image on there then you can reference it in the build params for your Amplify project.
--- /dev/null
+---
+title: Chrome HTTP Request stuck in "Stalled" State
+summary: Chrome reports that a request is stuck in a "stalled" state and I try and figure out why
+tags:
+ - chrome
+ - investigation
+ - explanation
+ - publish
+draft: false
+date: 2023-06-20T12:30:56.105-04:00
+lastmod: 2024-12-10T19:47:36.987Z
+---
+# Chrome HTTP Request stuck in a "Stalled" State
+
+I got the chance to investigate a really odd bug where randomly network requests in Chrome would just hang. This would only occurr in our test environments at work and not in production. The request would hang for some long amount of time.. and then eventually complete successfully. The bug has been occurring for some time, but has been getting worse in Chrome. It got so bad that it was guaranteed that if you were using Chrome it was going to happen to you. Eventually it started happening in Firefox as well.. during an investor demo (what good is a demo if it doesn't go up in flames?). That's when I got roped in.
+
+The first thing I did was attempt to replicate it and capture a `.har` file that I could share with anyone else that was interested. This part was easy - just popping open the network tab, navigating to the app on a test env, and then clicking every link that would trigger a network request. After about 30-40 seconds I had replicated the event
+
+
+
+So we can clearly see here that the request took 2 minutes and the entirety of that time the connection was stuck in the `stalled` state. That indicates one of two things:
+
+1. Chrome never attempted to make the network request at all. Perhaps the priority on the request was dropped, maybe there were too many connections open to that FQDN already.
+
+2. In some situations chrome actually merges the CORS preflight requests into what it reports as `stalled`. So it's possible that there was a problem in the preflight request that caused the delay before the actual request happened.
+
+I did a bit more testing to figure out the scope of the issue. On/Off VPN with multiple browsers would all replicate the problem just fine. In this particular situation the fact that we could replicate it across EVERYTHING made it pretty clear that it was something related to our server.. but what? If the request never actually hits the server what could be going on?
+
+### Chrome Network Log
+
+One tool that chrome has to diagnose networking issues is hidden away at `chrome://net-export`. It generates a very VERY detailed log of everything network related that chrome is aware of.
+
+
+
+I unchecked the `strip private information` option and told it to include cookies + credentials and started logging it to disk. Then I swapped back to my tab and replicated the issue. Waited a few seconds, and then went back and ended the capture session.
+
+Once you get that capture file, you have to head over to https://netlog-viewer.appspot.com and import it. There's a TON of information here, and honestly I didn't even look at half of it. The only two things I cared about were the "Events" and "Timeline" sections. The Timeline really makes no sense until you have a idea of when your actual network event happened, so we can skip that and jump right over to Events
+
+There will likely be a lot of events. The "filter" at the top never worked for me given the sheer size of the events.. but scrolling through them all was just fine and eventually I found the URL request that caused the issue. If you click on the event it will display a bunch of debug information about the request.
+
+
+
+As you can see.. suddenly there's a HUGE jump in time from `66807` to `187631`. We've confirmed now that this is a problem that's occurring within the CORS preflight request specifically, and it's just getting rolled into the `stalled` state. The log viewer makes it trivial to dig down into the events and if you click on the details of the `HTTP_STREAM_JOB_CONTROLLER` event you can see some more details.
+
+
+
+Here again, we see that there is a definitely delay when it attempts to call `HTTP_STREAM_REQUEST_STARTED_JOB`
+
+
+
+And now we can easily see the problem: `SOCKET_POOL_STALLED_MAX_SOCKETS_PER_GROUP`
+
+In HTTP1.1 each tab in your browser is configured to only make a certain number of requests per FQDN at the same time.This is one of the reasons why we load "static assets" on a different subdomain. By loading static assets on a separate FQDN we can increase the objects that are simultaneously loaded in our tab providing a better experience (for some definition of experience) to our user. In HTTP2, this restriction is across every single tab in your browser. For chrome, it can only instantiate 6 concurrent connections to an FQDN. This is because your connections are persistent in http2 and you don't need to deal with the initialization handshakes on every request. The connection, once opened, is continually reused.
+
+<span style="font-family: sans-serif; font-size: 16px; font-style: normal; font-variant-caps: normal;">For some reason, the socket pool dedicated to this particular FQDN gets filled up and so it can't actually make the next request. So it just sits there.. until suddenly a socket is available (2 minutes later) and it is able to complete the rest of the request as expected. The "suddenly" is likely due to the default socket timeout. Once that timeout is hit, Chrome kills the connection and opens a new one and suddenly our request works again.</span><br>
+
+We can dig even further! Since we know that this is happening on an HTTP2 call, we can filter our events to only show us the http2 connections and that paints a more serious picture!
+
+
+
+Every one of our http2 sockets is getting sent a `GOAWAY` frame by the server.. but notice that it says `NO_ERROR`. This generally indicates that the server is telling the client that it will be shutting down this socket and. The `GOAWAY` frame also tells the client what the last stream that it processed was. This is so that the client can resend any data that it needs to on a new connection. What should happen is that after this frame, the connection is ended by both parties and we move on to a new one. In practice, it happens after a following `GOAWAY` frame that indicates the connection is now dead. Except that final disconnect frame is never sent. So as far as chrome is concerned, we're still happily connected so it returns the connection to the pool. But the server has disconnected.
+
+So it just sits there trying to use the connection again, times out, and then closes and opens a new connection! And so we tracked down the mysterious slow-down and also used some cool tools in the process!
+
+***
+
+One thing I do want to note: This seems like a really straight forward problem - but that's just in hindsight. In the moment there's lots of googling and staring off into space trying to remember obscure keywords. I have a really bad memory, and so one of the things I do is memorize keywords/ideas rather than content because there's just too much to remember. In this way I can ensure that I can find the pieces of information I need when I need to. In this case the keys things were:
+
+* chrome has some kind of detailed network log
+
+* browsers like to fold CORS requests into the main request for reporting
+
+* http2 has a max connection limit across your browser
+
+The rest of the information used is all derivable from those keys and a search engine.
--- /dev/null
+---
+title: Code Reviews Are Failure
+date: 2022-02-24T11:05:34-05:00
+tags:
+ - git
+ - development
+ - engineering processes
+ - publish
+summary: Code Reviews are nothing more than a half-hearted attempt to avoid planning
+lastmod: 2024-12-10T19:49:29.025Z
+---
+# Code Reviews are a Failure
+
+As a new startup with one or two engineers on staff, you're very likely not doing code reviews. Engineers at this stage have a very deep understanding of the code - after all, they've probably written most of it. When it's time for a new feature, these initial developers know exactly how they're going to implement it given the architecture of their code base. Chances are, they keep their own work in a branch, and open a Pull Request or Merge Request, but they aren't asking someone to take a look at it. Instead they're making sure their changes work and they're merging it in themselves. Often they'll do this many times a day as they crank out features and bug fixes.
+
+At some point things are going better than they were and this small group of engineers start adding more! Now you have 5 or 6 engineers, all with varying familiarity of your code base. This is generally the first time Code Reviews come about - and normally for good reason. Often someone has pushed some code to production that has broken things and the developers take a step back and realize that maybe before they push code, it's best if they have others review it. Perhaps bugs like this can be caught next time. And so they come up with rules and reasons for why they need Code Reviews. Non technical managers think "Ah, this won't happen again - we're instituting code reviews now!"
+
+We've all seen the reasons for Code Reviews:
+
+* Find bugs further downstream
+* Propagation of Knowledge
+* Team Ownership
+* Double check functionality/architecture
+
+These are nonsense - Code Reviews in isolation almost always end up with the following results:
+
+* Reviews languishing in a "Ready for Review" state
+* Drastic code architecture changes
+* Being "Approved" based on social standing of the developer opening the request
+
+Code Reviews are often seen as some kind of magic bullet to catching errors before they get merged into code bases. The ideal is that a developer gets a ticket, makes some code changes, and then shares those changes with everyone else on the team for feedback. The idea is that other developers, with perhaps more context, can catch potential issues or side-effects in the code that the developer doing the work may not have even known about.
+
+Why the heck are you waiting until things are "Ready for Review" before you share you knowledge and context with the developer doing the task? Why are you telling them to "open a draft request as they go" so that you can share with them little tidbits of knowledge along the way.
+
+Why do Code Reviews fail? Because it moves in-depth planning to the end of the development process once someone is already "done" mentally with a task.
+
+## The Opposite of Code Reviews
+
+The opposite of Code Reviews isn't NO code reviews.
+
+The opposite of Code Reviews is Planning!
+
+Let me regale you with tales of how development works in the Open Source world. I specifically pick the Open Source community because they operate the opposite way of most startups. Both have no budget, but OS communities have much longer (almost.. undefined in some cases) delivery times for new features. But not just any Open Source community - I'm specifically going to target communities that were around before 2008. Old Open Source. I'm doing this specifically to highlight not just the mechanisms that were used, but also how they contributed to this default of Planning over Code Reviews.
+
+When a new feature was proposed - it didn't just appear on an issue board somewhere. Instead someone had to find the mailing list for a piece of software, get ON that mailing list, and then send an email to the development group explaining in detail what your feature was. If you were not clear, chances are you'd simply receive a short email asking for more explanation about a certain part, or possibly just outright dismissal of your idea. In either case, you'd end up needing to explain yourself further.
+
+At some point someone on the development team might agree that your idea should be a feature. At this point, they start digging into the code. They probably aren't making the changes you need for your feature, but they're validating how they would go about implementing it. A followup email you may receive would actually include code snippets, specific files/line-numbers to reference, and links to external documentation.
+
+This all happens in public. Other developers that see that email may realize that they have more context, and will chime in with additional edge cases, more documentation, maybe more code! This will likely go on for a while. Days is obscenely fast. Weeks is more reasonable. Eventually what will happen is that you'll end up with an email that details the goals for the feature, and a flurry of emails outlining the best way to go about it - complete with links to documentation, code, and even code snippets to implement the feature.
+
+The goal for this email activity isn't to code the feature - it's to get it to the point that anyone can pick up this email chain and implement the feature based on the knowledge shared up front.
+
+Once that's done, the developer may still run into cases they didn't think about and may message the group again. But often, things are planned out enough that the developer implementing the feature can solve the problem. This completed code is sent back to the group/maintainer for Integration. The "Code Review" at this stage is simply to make sure that things are happening as expected, to do one last check of the code, and to actually merge the changes in and test things.
+
+Now, there's plenty of problems with this process - but the Code Review is rarely one of them.
+
+## The Advent of Code Reviews
+
+I want to place the blame squarely on GitHub for enabling and promoting Code Reviews, via mostly circumstantial evidence.
+
+Prior to GitHub contributing to an Open Source project required you to get involved in the developer community for the project before submitting code. After GitHub anyone can stumble across a repo and open a Pull Request with code changes - without discussing anything at all.
+
+That mentality has premeated current development processes everywhere. Now when you start work on a task, instead of having all the information/discussion up front the developer is expected to seek it out. They're expected to not just figure out that they may be missing information, but also figure out the best person to ask to find out what they're missing. The developer is immediately set up for failure unless they already know the codebase well. At any sufficiently large enough project, that is very unlikely to be the case.
+
+Instead you end up open a code review that sits in review for days while engineers chime in with little fixes, slightly better ways of going about things, while you scramble to implement them and get your code reviewed again and again. Of course, some places recognize this problem and simply outlaw large scale architecture changes at this stage even if they are completely warranted.
+
+See the problem isn't that the Code Review is bad - it's that the Code Review is the first time anyone has actually looked at the code related to the problem.
+
+## The Solution to Code Reviews
+
+There isnt one.
+
+All planning up front without a deadline isn't helpful. All work without planning is pointless. But where your team draws the line between planning that's "good enough" and the length of time attributed to feature development changes frequently. It changes based on team composition, it changes based on the company state, it changes based on the market your company operates in. The only thing that's certain is that the amount of planning from feature to feature will be different.
+
+You have to be able to make a call at some point that the planning is enough. But if your planning doesn't include pseudo code and links to code in the project, chances are it's not enough planning.
+
+Planning should not be a single person event either. The whole team that would participate in the code review should be participating in the planning process as well.
+
+Once the planning is done and a developer completes the code change, the Code Review needs to happen. At this point generally it's about testing change rather than re-architecting how the change is made. When we're at this stage, there's even more tools at our disposal!
+
+Unit tests, Integration Tests, Synthetic/BlackBox Tests - all of these can help ease the time code spends stuck in code reviews. By minimizing the time spent in code reviews, and maximizing the time spent in planning instead we can achieve things like:
+
+* Actually find bugs further downstream and upstream
+* Propagation of Knowledge throughout the team
+* Team Ownership of a feature
+* Double check functionality/architecture
+
+How fun.
+
+## Notes
+
+* This was originally published on Medium - https://xangelo.medium.com/code-reviews-are-a-failure-36b72a659de4
--- /dev/null
+---
+date: 2022-09-21T22:36:23-04:00
+tags:
+ - engineering
+ - engineering org
+ - engineering processes
+ - publish
+summary: How to implement a successful on-call process for the first time
+title: Designing On Call
+lastmod: 2024-12-10T19:47:03.143Z
+---
+# Designing On-Call
+
+On-call is one of those things that all developers end up doing at some point. My goal isn't to discuss the merit of on-call, but rather what the point of on-call is and how to go about designing what “on-call” means at your company. I'm going to start at the very beginning because chances are you're already doing it wrong. I should also note that I'm looking at this specifically from a SaaS point of view.
+
+## Why do devs go on-call?
+
+The point of on-call is simple: People use your app 24/7 relative to you. They may be using it from Italy or Australia or Thailand while you're tucked up in bed in Wisconsin. But for them, it's working hours and they expect your site to be available. But, of course, that's not always possible. Things happen that will inevitably take your project down. In these situations "on-call" is vital for several reasons: The least of which is solving the problem.
+
+Normally you go on-call because your founders or manager or a customer noticed that your service was offline when it wasn't supposed to be. People start complaining, it makes some rounds internally, and then eventually ends up at the desks of whoever deployed it. Often that ends up being some kind of "infrastructure" team (You probably call them DevOps or SREs - why that's wrong is an issue for another time). These folk hear about it, fix the problem, and then go about setting up some alerts to let them know when the issue happens again. Bam, you've just enabled on-call without any of the good parts. You can all but guarantee that your on-call team is going to end up angry and upset about on-call which will manifest in the worst ways possible.
+
+## Let's walk through the standard on-call process
+
+Let's walk through a very high-level on-call process and then we can break it down.
+
+1. Something breaks
+2. You find out that something is broken
+3. You bash at your keyboard until it works again
+4. You put in some metrics so that if "thing" happens again, you'll know
+
+It's wrong.
+
+Ok, it's not wrong - it's just a subset of what "on-call" really is.
+
+See the thing that you're REALLY missing is that nowhere in this process have you actually figured out what's going on. Sure you solved a symptom - maybe that symptom is synonymous with the cause. But you don't know, because your process doesn't encompass that.
+
+To have a good on-call process you have to understand a fundamental truth of on-call: on-call isn't just about engineering. on-call is the culmination of a bunch of different business, product, and culture decisions.
+
+Here's the truth of each step in your process
+
+### Something breaks
+
+#### Something
+
+The first thing we need to define is what "something" is. The easiest thing to do is to take a look at your code repositories. If you’ve broken things down into services, you likely have one repo/service. That’s an easy way to define the boundaries of “something”. But it’s also probably wrong.
+
+If you have a monolithic codebase in a single repository this is a much harder step for you, and you’ll probably do a better job.
+
+It’s very easy to define “something” by engineering components - but this isn’t always answering “what” broke, but “how” and “why”. A user doesn’t care that your lambda was opening too many connections to RDS causing a spike in memory which caused your API to lag. They care that they weren’t able to send a cat picture.
+
+When defining your “something” start with user flows. Look at your application and define “core” actions that your user can take. From there, simplify until you have a handful of actions that you want a user to be able to do All the Time.
+
+#### Broke
+
+Defining the "something" is relatively easy compared to defining "broken". We know a thing is broken if it's not starting, but:
+
+* What if it works but 1% of the requests are resulting in an error?
+* What if it works but 1% of the time it crashes and restarts?
+* What if it works but is very slow?
+* What if it works, is not slow, and doesn't crash, but your API docs don't match what the endpoint is returning?
+* What if it works, but your database is being crushed by a sudden increase in traffic?
+
+Defining our “something” first is important because it helps us to set bounds on what we consider “broke” and what is “degraded” and what is “fine”.
+
+Perhaps we don't care if 1% of requests fail.. maybe we don't care if 0.1% of requests fail. Defining these thresholds is tough because it requires realistically looking at our application and deciding what is "acceptable" and what is not. Sometimes we have to make difficult choices here. Maybe we want 0.1% to be considered broken.. but actually, 1% is broken. This is where we impact your actual application.
+
+Deciding 0.1% of errors is "broken" where your application is currently sitting at 1% may seem like a good thing. 0.1% is where we want to be and so letting devs know when that isn't the case is good. We can work toward 0.1%. But this involves much larger product considerations.
+
+* When these alerts happen overnight - are devs tasked with a complete resolution during work hours?
+* Does that mean feature work will suffer?
+* How will you buffer your sprint to make time for these interruptions?
+* Do you have the ability to buffer your sprint given product launch dates?
+* What happens when a dev is up all night dealing with issues, do they take the next day off?
+* Do you offload the task to a different team (Ops/Infra/SRE) since feature work is so important?
+* What happens when those teams get burnt out and leave?
+
+On the flip side setting your alerts at 1% is accepting that this is the current state.. but now it becomes a decision on whether or not a 0.1% error rate is more important than the next feature you're supposed to get out.
+
+Defining the state of "broken" is hard because it is the intersection of the current and future state of our application, while simultaneously being the fulcrum that will help tip the scales between "acceptable" and "unacceptable".
+
+There's no right answer here - just an answer for your team, for right now. Whatever you decide today, the most important thing is that you re-evaluate it and allow those that are on-call to provide feedback on if this was a reasonable level of "broken".
+
+#### Service Level Objectives
+
+By defining what broken means you've now set a baseline stat for your systems. You've defined your first Service Level Objective (SLO). Yay!
+
+In the nutshell, SLOs form the baseline for your Service Level Agreements (SLA) with customers. Your SLA can't be better than your SLO or you're going to have to sacrifice on feature development to bring them up to speed. Now we can talk about all kinds of marketing tricks you can do (mainly playing with the measurement period) but the truth is if your SLO defines a particular flow as "broken" it sets the floor for your SLA. You can only promise less than that until you invest in improving that SLO.
+
+Let's move on to the 2<sup>nd</sup> step.
+
+### You find out that something is broken
+
+#### What's on first?
+
+Ok. something is definitely broken. How do you find out?
+
+This is probably the most technically straightforward piece, but the one that has the most impact on people.
+
+When something is broken, an alert is generated you get notified.
+
+The first step is figuring out how you get data out of your system into a place that can perform some analysis and generate these alerts. The two big products here are Datadog and NewRelic. These are all-in-one solutions that allow you to instrument your application and gather metrics+logs+events and put them somewhere you can visualize them. These systems also allow you to create “alerts” that get generated based on the data you’ve been feeding it.
+
+Given that you now know what you consider critical user flows your goal is to go through your application and ensure that everything that supports these flows is properly instrumented. This is where we start looking at the different infrastructure components. This is where we learn how/why our system broke. This is an important step - it's easy to think that devs should just "go through and instrument everything", but that makes things harder to reason about. You're increasing your "noise" and when you're looking at a Signal vs. Noise ratio, you want to keep your noise to a minimum.
+
+Take a look at your SLOs -> and then define your Service Level Indicators (SLI).
+
+#### Service Level Indicators
+
+These are the metrics that you can look at to identify that you're meeting your SLOs. They're often aggregate metrics. If your objective is a 1% error rate, your SLI will likely ensure that you are getting an accurate count of errors vs. successful requests across your system. That will involve you going through various systems and making sure that your code base is sending appropriate logs. It'll also be going through your monitoring tooling and creating dashboards/alerts so you can view your SLI and all the subcomponents that make up your SLI easily.
+
+Now that we have these metrics, and we know what we consider “broken” we can tune our alarms. When one of these components is in a state that impacts our user flow - thats an alert.
+
+That alert is going to be sent to whomever is on-call.
+
+#### Who’s on first?
+
+Everyone in Engineering that is responsible for writing code that could end up affecting a user.
+
+Bam. Easiest decision ever.
+
+But you'll likely face some pushback from teams that don't normally see themselves as "needing to be in rotation". Doesn't matter - at some point you have to make a decision and this is one hill you must die on as a manager.
+
+Traditionally on-call was reserved for infrastructure/devops/sre groups and developers weren't required to be on-call. This is a silly separation of concerns. The people who wrote the code that likely resulted in user impacts are often not from your infrastructure/platform team -> they're likely from the application team. This should be your first tier of on-call when thinking about SLOs from the product.
+
+Often you'll end up with a rotation of engineers, probably daily, with multiple tiers.
+
+1. The primary: They're the first ones to get alerted about an issue
+2. The secondary: If alerts to the primary don't get acknowledged, it goes to this person
+3. The backup: If primary AND secondary are not responding -> this person is here.
+
+You may decide to have different schedules for weekdays/weekends but in the end the goal is to cycle through all developers without making it feel like anyones "job" is to be on-call.
+
+Now that you have your schedule, you must know that
+
+1. Everyone should feel empowered to schedule alternates if they're busy. If I'm scheduled tomorrow night and something comes up - I should feel comfortable addressing the engineering team as a whole and asking someone to trade with me. Likewise, you should feel comfortable saying no.
+2. No one should treat on-call as being stuck at the computer. This does end up personal preference for engineers, but if you're on-call you should be able to take your laptop/phone with you where you go. If your company is not technically ready for this, then you better be compensating your employees well for tying them to their homes outside of work hours.
+
+### You bash at your keyboard until it works again
+
+Alright. It's 3 am, something broke, you were paged. You hop on your computer, bleary-eyed and upset because it's chillier than you expected because you forgot your pants in the rush to deal with this situation.
+
+Relax.
+
+Your job is not to solve the problem - your job is to ensure that your application is performing as expected.
+
+Solving the problem is a task that is best left to those on full nights of sleep. When it's 3 am and you finally manage to log into your laptop, "solving" the problem should be the furthest thing from your mind. The goal is to mitigate the outage and restore acceptable functionality based on your SLOs.
+
+In some cases simply restarting a process is enough to get things back to working. Sometimes you may need to upsize a database.
+
+What you should NOT be required to do, is:
+
+* figure out where your application code is crashing
+* optimize that weird nested join that you're supposed to tackle next sprint
+
+Your job at 3 am is problem mitigation. Maybe nothing can be done except throw on a status message so that users know what's happening.
+
+Perhaps this alert is fine, this level of error is acceptable.. so you can just adjust your alerting and call it a night.
+
+Don't attempt to be smart at 3 am because when you get paged at 6 am you'll be much worse off trying to decipher what you did.
+
+This is often where we run into issues. Engineers like to solve problems. When we get an alert we dig in. We want to fully understand what's happening because we can easily over/under complicate an issue. Oh is CPU on the DB at 100%? Do we need to just upsize the database? Are we locking something? Is there some periodic system running?
+
+Sometimes the problem is obvious and fixing the symptom is the same as fixing the problem. It's much harder when the problem isn't obvious. Rememeber that while you're "digging in" this is impacting users - otherwise it wouldn't be alerting you right? As unnatural and silly as it seems, at 3 am -> solve the symptom for immediate relief. Then when you're not stressed, solve the problem.
+
+Again: Solving the problem at 3 am is not your job.
+
+Once you fix the symptom causing the alert and verify that things are working again - leave notes in a public place. Explain what you did and why. The why is the most important part. This is that when you or whomever else logs in the following day and looks over what happened "on-call" you can easily decipher your thought process and areas that you should begin looking to resolve the underlying problem.
+
+### You put in some metrics so that if "thing" happens again, you'll know
+
+This is the easiest part: Once you identify the "root cause" of your problem, get some metrics/logs in place so that you know. Don't forget to include the metric in any SLI calculations that you might need.
+
+## Compensation
+
+Your propensity for compensation is entirely dependent on your org and the legalities of where you reside. In some situations you will have to pay engineers overtime for on-call. In some you can get away it.
+
+Don't be a jerk.
+
+Whatever you decide to do understand that your team will leave if they think they're getting a bad deal. on-call is requesting something of your team outside of normal working hours. It doesn't matter if you put it in a contract, it doesn't matter if it was on the job description. You are requesting something of your team that is above their work day.
+
+If an on-call night was rough, let them take the following day off. If they've been having a bad week personally, remind them that its' ok if they want to move their on-call.
+
+## Your On-Call Process
+
+Finally, you've ready all that and are ready to implement a real "on-call" process. Here's what it should look like:
+
+1. Identify your SLOs
+2. Identify your SLIs
+3. Work to track metrics/logs that relate to your SLIs
+4. Create an on-call rotation
+5. Something breaks
+6. You find out that something is broken
+7. You mitigate the problem
+8. You put in some metrics so that if "thing" happens again, you'll know
+9. You revisit the alert the morning after to figure out root cause
+10. You resolve the actual problem.
+
+## Your On-Call Process in Practice
+
+The truth is, there's a huge difference from the "ideal" process and what you end up with in practice. So much of this is up to the people involved and the culture of the workplace. Chances are you already have some of this implemented and it kinda works for you mostly.
+
+That's ok.
+
+The goal here is that you understand ideal, and you understand where you are. And now you have the information to be able to make trade-offs. Expending your entire development capital on the first 3 items on this process before getting to the next is silly.
+
+Instead this process is more of a "Flow". If you understand ther flow, you can understand how each thing feeds into the next. When you know that you can better break down your approach.
+
+You probably have Service Level Agreements (SLAs) with your customers. And you likely have some rough SLOs/SLIs that track that. Solidify them. Make them available to the team on demand and make sure that they're accurate.
+
+The on-call process is a larger movement within your engineering organization that pushes the whole team toward excellence: Both technical in being able to understand the inner-workings of your application and all the components, but also of your product. If your SLAs are at 99% uptime, but you're sitting at 99.9% uptime -> that's an excellent way to upsell. Likewise, if your uptime is actually at 98% you now know exactly the area your team will need to focus on next.
+
+## Final Thoughts
+
+Understand that no matter your best efforts, pages will be missed. Bugs will go undetected. Users will be upset. But these are short-term issues. You can fix bugs. You can explain to users. You can tell your devs that this push is temporary.
+
+Do understand the difference between pages being missed and pages being ignored - call your team out on it however you need. Missing a page isn't just letting customers down, it's letting your team down. If it happens frequently enough the team will either think that the alert isn't important or that an individual is above the on-call rotation. It'll foster all kinds of interpersonal problems.
+
+Don't treat on-call as the solution to your stability and reliability issues. Reacting is not being proactive. When alerts happen overnight, ensure that there is SOME mechanism to address the root cause reasonably.
+
+If you are running into issues with this, reach out - I'd love to chat. I've been doing this for a long time and I love to help out. I set aside a few hours a week to donate to help companies and individuals make sense of these kinds of larger scale engineering problems. My email is at the bottom of every page.
--- /dev/null
+---
+tags:
+ - outliner
+ - blog
+ - project
+ - publish
+date: 2023-05-21T23:29:45.732-04:00
+summary: An update to how I publish new posts
+title: Now Powered by Outlines
+lastmod: 2024-12-10T19:45:18.372Z
+---
+# Now Powered by Outlines
+
+One of the things that I do every so often is completely re-write the backend of my blog. I've mostly hit upon a UI that I like, but I've swapped out the backend over the years between various custom iterations, wordpress, ghost, and now finally Hugo. This time, I've swapped out how I write my blog posts - but kept everything else the same.
+
+The current system allows me to write markdown in vim. I'm normally running `hugo serve -w` at the same time so I can watch the rendered version of what I'm doing as I go. It's sort of like a hacked-together live preview. It works well enough.
+
+However, for the last 10 years (maybe more?) I've been a huge fan of outliners. I original started with various projects by [Dave Winer](https://scripting.com) and I used almost everything he's written around them for a number of years. I've also tried tooling like [Workflowy](https://workflowy.com) and almost every other infinite-bullet-list tool that came after them. They were all.. fine? I had no real problems with them except that they never really stuck around for very long. They were in a tab in my browser, and my browser has like 100 tabs open at any given time.
+
+For the last 6 months or so, however, I've been working on my own outliner. It started as an in-browser tool... and I quickly moved it to an offline-first desktop app via [Tauri](https://tauri.app). Having it offline first meant a few big things.
+
+1. All the syncing tools just work. Dropbox, rsync, backblaze, s3 as a filesystem. Whatever. It works. All nodes in the outliner are individual json files on your computer. There's an "outline" file that stitches the nodes together into a tree. <br>
+
+2. Building desktop first allowed me to bypass the need for user accounts and passwords. It allowed me to skip out on the complexities of providing reliable encrypted storage to users. I don't need to run a collection of servers and databases and object stores to power this thing.
+
+Since I use this tool across operating systems all day long it slowly ended up being where most of my thoughts for blog ideas end up. And so what I hope to be the final feature of this project was realized: I'd love to be able to write my whole blog post in here, and then hit a button to have it publish to my hugo deploy.
+
+This first iteration uses a lot of hard-coded stuff.. and I'll probably take some time to iron out some of the edge cases around rendering.. but it honestly came together pretty quickly. Since every node in the outliner is markdown it was trivial to put it together. As of right now, I can write my blog post in my outliner, press `shift+p` and have it write out a markdown file to my local hugo instance.
+
+For now, I do some manual reviewing before officially publishing it. For now there's a few more usability things that need to be added like
+
+* differentiating which posts are published vs. un-published
+
+* being able to 'unpublish' a node
+
+But honestly? I'm kind of enjoying this right now.
--- /dev/null
+---
+date: 2024-12-09
+tags:
+ - publish
+ - blog
+ - hugo
+ - obsidian
+authors:
+ - angelor
+title: Publishing with Obsidian
+summary: Blogging with Obsidian
+lastmod: 2024-12-09T21:19:10.953Z
+---
+# Publishing with Obsidian
+
+For the last couple years I've been been trending toward the idea that tools like https://remotestorage.io and https://5apps.com/ are actually not any better for owning your content than everything they purport to replace. Instead the true bastion of content owners is the lowly file that exists on their computer. While tools like RemoteStorage (and I'm definitely picking on them for no good reason) talk about freedom, really they're just pushing their vision of freedom on users. Freedom is really about making informed choices.
+
+As I came to terms with the idea, I started working on software that.. wrote actual files. My https://github.com/angelor/outliner that I've used for a while now writes everything locally. There's no network requests necessary. That means if you want, you can go in, grab all those files, and just.. put them wherever you want: Dropbox, RemoteStorage, your NAS. That's freedom. That's really giving users control.
+
+I recently stumbled across a blog post by Steph Anjo called [File over app](https://stephango.com/file-over-app) which really resonated with me. As a result, I kind of dropped everything and moved to Obsidian as my primary note taking tool.
+
+It sounds impulsive, but the truth is - the output of Obsidian is just markdown. An open protocol with hundreds of implementations across almost every programming language. Obsidian is a good enough note-taking tool - but one day it too will be gone. Likely replaced by something better with more features I haven't thought of. But that's ok, it'll probably import my Markdown files just fine.
+
+## Setup
+
+My existing deployment process has been:
+
+1. Write my blog posts in my outliner
+2. Export the outline-based notes to markdown (I wrote a small parser that flattened my outlines)
+3. Import those to my hugo-based blog repo locally
+4. Commit the changes and deploy to GitHub
+5. GitHub workflows builds my site and publishes it for me
+
+The only thing that's going to change is that
+
+1. I'm going to utilize Obsidian to write my posts
+2. I'm going to export them, to my hugo-based blog repo locally
+
+## FrontMatter
+
+I'm using Obsidian's built in Template system to create a basic FrontMatter that gets applied whenever I want to convert something into a post.
+
+```
+---
+date: {{date}}
+tags: [publish]
+authors:
+ - angelor
+title: {{title}}
+---
+
+```
+
+For me, specifically, I add the "publish" tag to denote "published" blog posts vs. things I'm still working on. This gets formatted with some Hugo specifics (like the current date and title), and inserted verbatim into the post.
+
+## Compiling
+
+The default vault format for Obsidian would be good enough for me, except I like to keep all my notes in a single vault. Sometimes things start off as memo's to myself, and slowly morph into actual blog posts. Other times as I build up an understanding of a topic, I update my notes, and those slowly become blog posts. As a result, I want a way of differentiating between notes that are "blog posts" and those that are just regular notes.
+
+I use a Community Plugin called [Hugo Publish](https://github.com/kirito41dd/obsidian-hugo-publish) that does this part for me.
+
+
+
+My settings are pretty straight forward - that "publish" tag that I mentioned earlier is what selects a post for "compilation". It doesn't really do too much, just rewrites some links and moves the uploaded images to the right folder in my hugo setup - and then it copies the compiled files over.
+
+## Thoughts
+
+I think overall I'm pretty pleased with this process - one of the things that I've really struggled with is that writing a blog post is a very involved process right now using a custom editor. Using Obsidian this can happen much easier, and I get the added benefit of being able to sync the files across different platforms in a variety of ways. I'll have to play around with this a bit more - but so far I'm liking it.
--- /dev/null
+---
+title: Removing the Default Font
+date: 2022-03-31T00:07:57-04:00
+tags:
+ - upkeep
+ - fonts
+ - publish
+summary: Where I spend too long talking about why I removed the default font temporarily
+lastmod: 2024-12-10T19:49:17.644Z
+---
+# Removing the Default Font
+
+This is a small change that I've made to the site that I've actually been thinking about for quite some time. I've always had a monospaced font configured in my CSS, forcing all text into whatever the default monospace font on your system is.
+
+Personally I like monospaced fonts - but then again, I spend a lot of my time looking at them so it's only inevitable.
+
+However, monospaced fonts arose to solve a technical problem that original typesetters had long since solved.
+
+## Monospaced Font
+
+A monospaced font is a font that has a "fixed with". When you look at a word in a monospaced font, it's not that each character is the same size, but rather the area that contains each character is the same size.
+
+
+
+In this image we can see how a monospace font is "set". Whenever a character appears on the screen, these "boxes" surrounding them denote their bounds. In a monospace'f font, the bounding box for each character is the same size. The size of the character and its placement within the box may vary - but by standardizing on the bounding box size it will appear that each character is the same "width".
+
+This is very cool stuff.
+
+Since each character has a bounding box that's the same size as every other character we actually run into a very specific instance of a cool typography side-effect known as [Rivers](https://en.wikipedia.org/wiki/River_\(typography\)). Each character aligns itself perfectly with the character above and below it, creating a giant grid of characters on your screen.
+
+But this isn't always the best for reading.
+
+## Proportional Font
+
+A proportional font is one that allows the bounding box for any individual character to vary. This has the side-effect of allowing for much narrow kerning (the space between letters) and does 100% lead to improved legibility (see [this study here](https://journals.sagepub.com/doi/pdf/10.1177/001872088302500303)).
+
+
+
+By varying the kerning and allowing each character to use as much space as it needs, font designers can really tweak the legibility of the font.
+
+## Why does any of this matter?
+
+We know that monospaced fonts improve the legibility of text in SOME cases, like reading code or if the reader has a disability like dyslexia. In all other cases, proportional fonts improve legibility. Serifs in generally tend to be chosen by popular type users (newspapers for example) because of their legibility.
+
+All of this means that... me setting the default font to be a monospace font is actually a problem for legibility. By leaving this setting blank and using the browser defaults, I can ensure that you are reading the content of this site how you would best consume it. I've taken a lot of time to ensure that things like dark/light modes respect user settings. I've spent longer than I would have liked getting the right shades of blue/purple (the default unvisited/visited link colors) so that they are the same in dark/light modes. The only thing I didn't do was think about the font - I just set it to monospaced since I like it and went about my day.
+
+I've since decided to remove that setting, allowing your browser/system defaults to set the primary font on the site. After all, what matters on this site is the content and you should be able to view that as you'd like.
+
+## Appendix
+
+The study linked looks specifically at fixed vs. variable letter width (monospace vs. proportional) for televised text. This is particularly relevant since we're talking about screens rather than print. The differences cited are also quite marginal - but that's the point of this font change. To the majority of users it won't make a difference, but to a small few it will matter a lot.
--- /dev/null
+---
+summary: A naive redis job queue implementation
+tags:
+ - redis
+ - architecture
+ - publish
+date: 2023-05-25T00:15:53.362-04:00
+title: Simple Redis Job Queue
+lastmod: 2024-12-10T19:48:22.826Z
+---
+# Simple Redis Job Queue
+
+A common pattern in most software projects is the "queue". You'll throw some stuff on there and eventually get around to sorting it all out. In some cases, you may not even really care about that.
+
+I recent ran into a situation in node.js where I needed to offload some CPU intensive out of the main http request handling process. Originally I thought about just forking the process, but I due to the fact that this endpoint is public facing (albeit authenticated and paid for), I was wary. Instead I figured I'd rope in a quick job queue.
+
+### Caveats:
+
+* The jobs have an easy "deduplication key". An ID that is the same given the same inputs.<br>
+
+* If we don't complete a job for any reason, that's fine, another one will likely show up again in a few minutes
+
+We were also using redis so here's a real simple pattern that can handle this kind of workload.
+
+1. Whenever we get a new job we calculate its deduplication ID. We'll call this the `jobId` moving forward.
+
+2. We call `set job:[jobId] value NX EX 10`. This sets the key `job:[jobId]` to some value we provide only if the key does not already exist. It also sets a 10s expiration time. We're using `NX` to provide some additional deduplication and we're using `EX` to ensure that if for some reason our job doesn't complete in 10s.. just allow another job to be inserted with the same key. You can tune `EX` as necessary, or leave it off entirely if you never want this job re-processed again until you flush your cache.
+
+3. We use `rpush [queueName] [jobId]` to add the job Id to the list of jobs that need to be processed.
+
+4. If we have any additional information about the job we want to pass on, we can use `hset jobDetails:[jobId] k1 v1 k2 v2...` and store it in there.
+
+Your "worker" is just a process that runs `lrange [queueName] 0 0`. That will retrieve the oldest `jobId` in your queue. You can grab any further information from the `jobDetails:[jobId]` hash set and do whatever work you need. When you're done you can call `ltrim [queueName] 0 0` which will remove the job from the queue.
+
+An interesting fact about this setup is that all your calls are `O(1)` and you can pipeline the initial `set`/`rpush`/`hset` calls so that things are even faster.
--- /dev/null
+---
+title: Started from *Free* now we're here
+summary: How FREE has shaped my life
+tags:
+ - game dev
+ - mtx
+ - bio
+ - publish
+date: 2023-06-19T06:12:47.302-04:00
+lastmod: 2024-12-10T19:48:03.260Z
+---
+# Started from *Free* now we're here
+
+Neopets was the first time I'd ever heard about HTML or CSS. I had been playing terrible games for a long time at this point, even digging into basic at one point to try and do something.. but I didn't have the interest at the time. I didn't have anyone in my (or my parents) circle that was interested in computers at the time, so I never even considered what was possible. Neopets was a free game where you took care of a digital pet and then played a ton of games to earn in-game money. They even had a huge "Auction house" that was made up of individual stores that users ran. You could start a store, and set up your own items for sale. It was amazing. It also gave you a little area to enter HTML/CSS snippets so you could customize your store. That was why I bothered learning HTML/CSS at all. To customize my neopets store. They had the same interface for your clan page - customize your entire clan page with HTML and CSS. It was amazing. It was the first time anything related to "programming" clicked and I realized I loved it.
+
+I don't remember a single ad on neopet trying to force me to make a purchase of a premium currency. There probably were.. but I don't remember them at all.
+
+After learning HTML/CSS and offering my services to other neopets players in exchange for ingame goods.. I grew out of what neopets offered. In that time I discovered geocities and it DHTML. I loved it. I grew out of that too when I started digging into PHP and started looking around for free shared hosting. There were a lot of hosting companies at the time.
+
+One in particular, Host Matrix, caught my attention - because I love the Matrix. They were having a promotion where if you wrote long form tutorials for them (around HTML/CSS/PHP) you could get free hosting! And so I did. I wrote a lot of tutorials so that I could get free hosting and continue into this weird world I had stumbled in to.
+
+Around the same time I picked up the guitar. My dad had played the guitar since his teens, and so there was always one around. I had never shown an interest in it.. but one day I picked it up and started learning how to play. Mostly because of tabs (tabulature) - a fingering based musical notation that works beautifully for the guitar. People, on the internet, would listen to songs, figure out what they were playing.. and then write it down and put it on the internet.. for free! I devoured everything I could find. Eventually I got good enough that I could write my own tabs and share with others! It was incredible.
+
+When I started branching out musically.. it was because of free access to it. The first time I had heard Ravenous by [Arch Enemy](https://archenemy.net/en/) my mind was blown - I ran home and downloaded the song. I listened to it every day for a week and then started learning it. When I discovered Children of Bodom and Kalmah it was the same thing. Obscure bands that I would never have had the hope of finding out, accessible to me because it was free.
+
+Don't get me wrong, I understand there were costs to creating this. I've spent the last decade of my life building SaaS software for companies and managing infra budgets. I understand the cost. But also, I understand that I wasn't a customer. Arch Enemy didn't lose money by having me download their album. I didn't have access to purchase their album and I didn't have the money to do it. Arch Enemy gained a fan that would later on buy tickets to shows, and albums, and t-shirts. A fan that would have never bothered if he hadn't heard their albums for free.
+
+Free played a huge role in my life and I've always loved it. I donate a few hours every week to local startups that need technical advice. I hang out in slack groups answering questions. I hang out in Magic discord groups answering questions. I write software and give it away for free. I do it because I owe so much to free.
+
+And now I sit here trying to figure out monetization strategies for my projects.. and I don't like it. I wish I could do it for free, but there are costs associated with it that I know I won't be able to subsidize forever. I know I want as much of it to be free as possible. I don't want people to feel like they need to pay, I want them to want to. So I have to think about the best way to monetize [Rising Legends](https://www.risinglegends.net) that are in line with my philosophies...
+++ /dev/null
----
-title: Amplify docker limitations
-summary: Amplify has some odd docker limitations it doesn't want to fix
-tags: ["aws", "docker", "amplify"]
-date: 2023-12-05T05:16:29.288-05:00
----
-
-
-I've recently had the chance to do some work wtih Amplify in AWS and I'm surprised how simultaneously feature rich and half baked it is. It seems if you're in to click-ops you'll be fine in Amplify until you hit a problem.
-
-If you're deploying through amplify you'll eventually hit this problem:
-
-```[BUILD_CONTAINER_UNABLE_TO_PULL_IMAGE: Unable to pull customer's container image. CannotPullContainerError: Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit]```
-
-Which seems perfectly reasonable at first glance. Nowhere in your project setup did you really define credentials for your docker account.
-
-But if you dig a bit it gets weird: You actually can't define credentials for a docker account.. and the limits for the free tier are high enough that you probably wouldn't hit them with a small team - 100 pulls every 6 hours per IP.
-
-Turns out you actually share a build pool with other users in amplify - which makes sense. But that in turn means that every so often your build pool will hit the limits for pulling containers from docker hub. And since there's no way to set credentials for your pull.. you're just kinda left hoping that if you retry things will work. There's probably a large enough build pool that it works sometimes, but there are reports of it taking up to an hour or so.
-
-So you think: Oh yeah, no worries -> I'll just set up ECR, and configure Amplify to grab my image from that repo instead. But ECR is a non-starter. It turns out that you can't assign any kind of roles to amplify to allow authentication against an ECR.. so if you're hosting your image in ECR, you have to make the entire repo publicly accessible.
-
-If you're using customized images unfortunately this is really the only way to go. You have to build and deploy to ECR and make the repo publicly available.
-
-If you're using pre-built images, amazon actually has their own public replica of popular docker images [https://gallery.ecr.aws/](https://gallery.ecr.aws/). If you can find your image on there then you can reference it in the build params for your Amplify project.
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+++ /dev/null
----
-title: Chrome HTTP Request stuck in "Stalled" State
-summary: Chrome reports that a request is stuck in a "stalled" state and I try and figure out why
-tags: ["chrome", "investigation", "explanation"]
-draft: false
-date: 2023-06-20T12:30:56.105-04:00
----
-
-
-I got the chance to investigate a really odd bug where randomly network requests in Chrome would just hang. This would only occurr in our test environments at work and not in production. The request would hang for some long amount of time.. and then eventually complete successfully. The bug has been occurring for some time, but has been getting worse in Chrome. It got so bad that it was guaranteed that if you were using Chrome it was going to happen to you. Eventually it started happening in Firefox as well.. during an investor demo (what good is a demo if it doesn't go up in flames?). That's when I got roped in.
-
-The first thing I did was attempt to replicate it and capture a `.har` file that I could share with anyone else that was interested. This part was easy - just popping open the network tab, navigating to the app on a test env, and then clicking every link that would trigger a network request. After about 30-40 seconds I had replicated the event
-
-
-
-So we can clearly see here that the request took 2 minutes and the entirety of that time the connection was stuck in the `stalled` state. That indicates one of two things:
-
-1. Chrome never attempted to make the network request at all. Perhaps the priority on the request was dropped, maybe there were too many connections open to that FQDN already.
-
-1. In some situations chrome actually merges the CORS preflight requests into what it reports as `stalled`. So it's possible that there was a problem in the preflight request that caused the delay before the actual request happened.
-
-I did a bit more testing to figure out the scope of the issue. On/Off VPN with multiple browsers would all replicate the problem just fine. In this particular situation the fact that we could replicate it across EVERYTHING made it pretty clear that it was something related to our server.. but what? If the request never actually hits the server what could be going on?
-
-### Chrome Network Log
-
-One tool that chrome has to diagnose networking issues is hidden away at `chrome://net-export`. It generates a very VERY detailed log of everything network related that chrome is aware of.
-
-
-
-I unchecked the `strip private information` option and told it to include cookies + credentials and started logging it to disk. Then I swapped back to my tab and replicated the issue. Waited a few seconds, and then went back and ended the capture session.
-
-Once you get that capture file, you have to head over to https://netlog-viewer.appspot.com and import it. There's a TON of information here, and honestly I didn't even look at half of it. The only two things I cared about were the "Events" and "Timeline" sections. The Timeline really makes no sense until you have a idea of when your actual network event happened, so we can skip that and jump right over to Events
-
-There will likely be a lot of events. The "filter" at the top never worked for me given the sheer size of the events.. but scrolling through them all was just fine and eventually I found the URL request that caused the issue. If you click on the event it will display a bunch of debug information about the request.
-
-
-
-As you can see.. suddenly there's a HUGE jump in time from `66807` to `187631`. We've confirmed now that this is a problem that's occurring within the CORS preflight request specifically, and it's just getting rolled into the `stalled` state. The log viewer makes it trivial to dig down into the events and if you click on the details of the `HTTP_STREAM_JOB_CONTROLLER` event you can see some more details.
-
-
-
-Here again, we see that there is a definitely delay when it attempts to call `HTTP_STREAM_REQUEST_STARTED_JOB`
-
-
-
-And now we can easily see the problem: `SOCKET_POOL_STALLED_MAX_SOCKETS_PER_GROUP`
-
-In HTTP1.1 each tab in your browser is configured to only make a certain number of requests per FQDN at the same time.This is one of the reasons why we load "static assets" on a different subdomain. By loading static assets on a separate FQDN we can increase the objects that are simultaneously loaded in our tab providing a better experience (for some definition of experience) to our user. In HTTP2, this restriction is across every single tab in your browser. For chrome, it can only instantiate 6 concurrent connections to an FQDN. This is because your connections are persistent in http2 and you don't need to deal with the initialization handshakes on every request. The connection, once opened, is continually reused.
-
-<span style="font-family: sans-serif; font-size: 16px; font-style: normal; font-variant-caps: normal;">For some reason, the socket pool dedicated to this particular FQDN gets filled up and so it can't actually make the next request. So it just sits there.. until suddenly a socket is available (2 minutes later) and it is able to complete the rest of the request as expected. The "suddenly" is likely due to the default socket timeout. Once that timeout is hit, Chrome kills the connection and opens a new one and suddenly our request works again.</span><br>
-
-We can dig even further! Since we know that this is happening on an HTTP2 call, we can filter our events to only show us the http2 connections and that paints a more serious picture!
-
-
-
-Every one of our http2 sockets is getting sent a `GOAWAY` frame by the server.. but notice that it says `NO_ERROR`. This generally indicates that the server is telling the client that it will be shutting down this socket and. The `GOAWAY` frame also tells the client what the last stream that it processed was. This is so that the client can resend any data that it needs to on a new connection. What should happen is that after this frame, the connection is ended by both parties and we move on to a new one. In practice, it happens after a following `GOAWAY` frame that indicates the connection is now dead. Except that final disconnect frame is never sent. So as far as chrome is concerned, we're still happily connected so it returns the connection to the pool. But the server has disconnected.
-
-So it just sits there trying to use the connection again, times out, and then closes and opens a new connection! And so we tracked down the mysterious slow-down and also used some cool tools in the process!
-
----
-
-One thing I do want to note: This seems like a really straight forward problem - but that's just in hindsight. In the moment there's lots of googling and staring off into space trying to remember obscure keywords. I have a really bad memory, and so one of the things I do is memorize keywords/ideas rather than content because there's just too much to remember. In this way I can ensure that I can find the pieces of information I need when I need to. In this case the keys things were:
-
-- chrome has some kind of detailed network log
-
-- browsers like to fold CORS requests into the main request for reporting
-
-- http2 has a max connection limit across your browser
-
-The rest of the information used is all derivable from those keys and a search engine.
-
-
-
-
-
-
-
-
-
-
-
-
-
+++ /dev/null
----
-title: "Code Reviews Are Failure"
-date: 2022-02-24T11:05:34-05:00
-tags: ["git", "development", "engineering processes"]
-summary: Code Reviews are nothing more than a half-hearted attempt to avoid planning
----
-
-## Code Reviews are a Failure
-As a new startup with one or two engineers on staff, you're very likely not doing code reviews. Engineers at this stage have a very deep understanding of the code - after all, they've probably written most of it. When it's time for a new feature, these initial developers know exactly how they're going to implement it given the architecture of their code base. Chances are, they keep their own work in a branch, and open a Pull Request or Merge Request, but they aren't asking someone to take a look at it. Instead they're making sure their changes work and they're merging it in themselves. Often they'll do this many times a day as they crank out features and bug fixes.
-
-At some point things are going better than they were and this small group of engineers start adding more! Now you have 5 or 6 engineers, all with varying familiarity of your code base. This is generally the first time Code Reviews come about - and normally for good reason. Often someone has pushed some code to production that has broken things and the developers take a step back and realize that maybe before they push code, it's best if they have others review it. Perhaps bugs like this can be caught next time. And so they come up with rules and reasons for why they need Code Reviews. Non technical managers think "Ah, this won't happen again - we're instituting code reviews now!"
-
-We've all seen the reasons for Code Reviews:
-- Find bugs further downstream
-- Propagation of Knowledge
-- Team Ownership
-- Double check functionality/architecture
-
-These are nonsense - Code Reviews in isolation almost always end up with the following results:
-- Reviews languishing in a "Ready for Review" state
-- Drastic code architecture changes
-- Being "Approved" based on social standing of the developer opening the request
-
-Code Reviews are often seen as some kind of magic bullet to catching errors before they get merged into code bases. The ideal is that a developer gets a ticket, makes some code changes, and then shares those changes with everyone else on the team for feedback. The idea is that other developers, with perhaps more context, can catch potential issues or side-effects in the code that the developer doing the work may not have even known about.
-
-Why the heck are you waiting until things are "Ready for Review" before you share you knowledge and context with the developer doing the task? Why are you telling them to "open a draft request as they go" so that you can share with them little tidbits of knowledge along the way.
-
-Why do Code Reviews fail? Because it moves in-depth planning to the end of the development process once someone is already "done" mentally with a task.
-
-## The Opposite of Code Reviews
-The opposite of Code Reviews isn't NO code reviews.
-
-The opposite of Code Reviews is Planning!
-
-Let me regale you with tales of how development works in the Open Source world. I specifically pick the Open Source community because they operate the opposite way of most startups. Both have no budget, but OS communities have much longer (almost.. undefined in some cases) delivery times for new features. But not just any Open Source community - I'm specifically going to target communities that were around before 2008. Old Open Source. I'm doing this specifically to highlight not just the mechanisms that were used, but also how they contributed to this default of Planning over Code Reviews.
-
-When a new feature was proposed - it didn't just appear on an issue board somewhere. Instead someone had to find the mailing list for a piece of software, get ON that mailing list, and then send an email to the development group explaining in detail what your feature was. If you were not clear, chances are you'd simply receive a short email asking for more explanation about a certain part, or possibly just outright dismissal of your idea. In either case, you'd end up needing to explain yourself further.
-
-At some point someone on the development team might agree that your idea should be a feature. At this point, they start digging into the code. They probably aren't making the changes you need for your feature, but they're validating how they would go about implementing it. A followup email you may receive would actually include code snippets, specific files/line-numbers to reference, and links to external documentation.
-
-This all happens in public. Other developers that see that email may realize that they have more context, and will chime in with additional edge cases, more documentation, maybe more code! This will likely go on for a while. Days is obscenely fast. Weeks is more reasonable. Eventually what will happen is that you'll end up with an email that details the goals for the feature, and a flurry of emails outlining the best way to go about it - complete with links to documentation, code, and even code snippets to implement the feature.
-
-The goal for this email activity isn't to code the feature - it's to get it to the point that anyone can pick up this email chain and implement the feature based on the knowledge shared up front.
-
-Once that's done, the developer may still run into cases they didn't think about and may message the group again. But often, things are planned out enough that the developer implementing the feature can solve the problem. This completed code is sent back to the group/maintainer for Integration. The "Code Review" at this stage is simply to make sure that things are happening as expected, to do one last check of the code, and to actually merge the changes in and test things.
-
-Now, there's plenty of problems with this process - but the Code Review is rarely one of them.
-
-## The Advent of Code Reviews
-I want to place the blame squarely on GitHub for enabling and promoting Code Reviews, via mostly circumstantial evidence.
-
-Prior to GitHub contributing to an Open Source project required you to get involved in the developer community for the project before submitting code. After GitHub anyone can stumble across a repo and open a Pull Request with code changes - without discussing anything at all.
-
-That mentality has premeated current development processes everywhere. Now when you start work on a task, instead of having all the information/discussion up front the developer is expected to seek it out. They're expected to not just figure out that they may be missing information, but also figure out the best person to ask to find out what they're missing. The developer is immediately set up for failure unless they already know the codebase well. At any sufficiently large enough project, that is very unlikely to be the case.
-
-Instead you end up open a code review that sits in review for days while engineers chime in with little fixes, slightly better ways of going about things, while you scramble to implement them and get your code reviewed again and again. Of course, some places recognize this problem and simply outlaw large scale architecture changes at this stage even if they are completely warranted.
-
-See the problem isn't that the Code Review is bad - it's that the Code Review is the first time anyone has actually looked at the code related to the problem.
-
-## The Solution to Code Reviews
-There isnt one.
-
-All planning up front without a deadline isn't helpful. All work without planning is pointless. But where your team draws the line between planning that's "good enough" and the length of time attributed to feature development changes frequently. It changes based on team composition, it changes based on the company state, it changes based on the market your company operates in. The only thing that's certain is that the amount of planning from feature to feature will be different.
-
-You have to be able to make a call at some point that the planning is enough. But if your planning doesn't include pseudo code and links to code in the project, chances are it's not enough planning.
-
-Planning should not be a single person event either. The whole team that would participate in the code review should be participating in the planning process as well.
-
-Once the planning is done and a developer completes the code change, the Code Review needs to happen. At this point generally it's about testing change rather than re-architecting how the change is made. When we're at this stage, there's even more tools at our disposal!
-
-Unit tests, Integration Tests, Synthetic/BlackBox Tests - all of these can help ease the time code spends stuck in code reviews. By minimizing the time spent in code reviews, and maximizing the time spent in planning instead we can achieve things like:
-- Actually find bugs further downstream and upstream
-- Propagation of Knowledge throughout the team
-- Team Ownership of a feature
-- Double check functionality/architecture
-
-How fun.
-
-
-## Notes
-- This was originally published on Medium - https://xangelo.medium.com/code-reviews-are-a-failure-36b72a659de4
+++ /dev/null
----
-title: "Removing the Default Font"
-date: 2022-03-31T00:07:57-04:00
-tags: ["upkeep", "fonts"]
-summary: Where I spend too long talking about why I removed the default font temporarily
----
-
-This is a small change that I've made to the site that I've actually been thinking about for quite some time. I've always had a monospaced font configured in my CSS, forcing all text into whatever the default monospace font on your system is.
-
-Personally I like monospaced fonts - but then again, I spend a lot of my time looking at them so it's only inevitable.
-
-However, monospaced fonts arose to solve a technical problem that original typesetters had long since solved.
-
-## Monospaced Font
-A monospaced font is a font that has a "fixed with". When you look at a word in a monospaced font, it's not that each character is the same size, but rather the area that contains each character is the same size.
-
-
-
-In this image we can see how a monospace font is "set". Whenever a character appears on the screen, these "boxes" surrounding them denote their bounds. In a monospace'f font, the bounding box for each character is the same size. The size of the character and its placement within the box may vary - but by standardizing on the bounding box size it will appear that each character is the same "width".
-
-This is very cool stuff.
-
-Since each character has a bounding box that's the same size as every other character we actually run into a very specific instance of a cool typography side-effect known as [Rivers](https://en.wikipedia.org/wiki/River_(typography)). Each character aligns itself perfectly with the character above and below it, creating a giant grid of characters on your screen.
-
-But this isn't always the best for reading.
-
-## Proportional Font
-A proportional font is one that allows the bounding box for any individual character to vary. This has the side-effect of allowing for much narrow kerning (the space between letters) and does 100% lead to improved legibility (see [this study here](https://journals.sagepub.com/doi/pdf/10.1177/001872088302500303)).
-
-
-
-By varying the kerning and allowing each character to use as much space as it needs, font designers can really tweak the legibility of the font.
-
-## Why does any of this matter?
-We know that monospaced fonts improve the legibility of text in SOME cases, like reading code or if the reader has a disability like dyslexia. In all other cases, proportional fonts improve legibility. Serifs in generally tend to be chosen by popular type users (newspapers for example) because of their legibility.
-
-All of this means that... me setting the default font to be a monospace font is actually a problem for legibility. By leaving this setting blank and using the browser defaults, I can ensure that you are reading the content of this site how you would best consume it. I've taken a lot of time to ensure that things like dark/light modes respect user settings. I've spent longer than I would have liked getting the right shades of blue/purple (the default unvisited/visited link colors) so that they are the same in dark/light modes. The only thing I didn't do was think about the font - I just set it to monospaced since I like it and went about my day.
-
-I've since decided to remove that setting, allowing your browser/system defaults to set the primary font on the site. After all, what matters on this site is the content and you should be able to view that as you'd like.
-
-## Appendix
-The study linked looks specifically at fixed vs. variable letter width (monospace vs. proportional) for televised text. This is particularly relevant since we're talking about screens rather than print. The differences cited are also quite marginal - but that's the point of this font change. To the majority of users it won't make a difference, but to a small few it will matter a lot.
-
+++ /dev/null
----
-title: "DEVLOG: Custom Theme for Hugo"
-date: 2020-06-10T12:00:00-04:00
-draft: false
-tags: ["devlog", "blog", "devlog-blog"]
----
-
-I've started working on a custom theme for Hugo[^1] that I use on my blog. The goal here is to be as simple as possible and takes its inspiration from the IETF .txt file[^2] where possible.
-
-The theme itself contains a light/dark mode as well as a small variation of the main theme for mobile. I actually had some help from a good friend[^3] who helped me adjust the colors for both dark and light modes to make them a bit more legible.
-
-The two complicated components and 1 simple component are outlined below.
-
-## Features
-
-### Numbered Headers
-
-I wanted to ensure that each heading would have a number associated with it that matched the number that it was automatically assigned by Hugo during the Table of Contents generation. I didn't want to have to update numbers as I added/remove header tags and I didn't want to utilize any JavaScript on the site, so I was left with CSS only.
-
-I ended up utilizing counters[^4] to achieve this.
-
-
-
-```css
-body {counter-reset: h2;}
-h2 {counter-reset: h3;}
-h3 {counter-reset: h4;}
-h4 {counter-reset: h5;}
-h5 {counter-reset: h6;}
-
-h2:before {counter-increment: h2; content: counter(h2) ". ";}
-h3:before {counter-increment: h3; content: counter(h2) "."counter(h3) ". ";}
-h4:before {counter-increment: h4; content: counter(h2) "."counter(h3) "."counter(h4) ". ";}
-h5:before {counter-increment: h5; content: counter(h2) "."counter(h3) "."counter(h4) "."counter(h5) ". ";}
-h6:before {counter-increment: h6; content: counter(h2) "."counter(h3) "."counter(h4) "."counter(h5) "."counter(h6) ". "; }
-h2.nocount:before,h3.nocount:before,h4.nocount:before,h5.nocount:before,h6.nocount:before {content: ""; counter-increment: none;}
-```
-
-
-
-I only have a single `<h1>` tag in my document which is for the title of the post, which I don't want numbered. The code above it ensures that every time we run across an `<h2>` tag it will reset the counter for `<h3>` tags to 0, increment the counter for `<h2>` tags, and then display the current value of the counter.
-
-It repeats this for all header tags appending the counter for each subsequent tag level.
-
-I've also included provisions for ensuring that you can skip incrementing the counters if you add a `.nocount` class to any header.
-
-
-
-### Flexible visual line-up
-
-This was probably the hardest part.
-
-
-
-That dashed line that lets you link a post with the corresponding date. Both elements are positioned at the exterior of their boxes, and the titles have a varying width.
-
-In order to accomplish this I ended up needing to introduce a 3rd element, the line itself. However, I can't actually use an `<hr />` tag because I want to work towards supporting screen readers.. and those tags get interpreted as dividers between content.
-
-The article list is an unordered list, so the HTML code looks like this:
-
-```html
-<li>
- <a href="/link/to/post">Post Title</a>
- <span class="divider"></span>
- <span class="pubdate">YYYY-MM-DD</span>
-</li>
-```
-
-I set the `<li>` tag to utilize flex `display: flex` and tell it to align the items to the center
-
-I then set the `.divider` to `flex-grow: 1` [^4] and don't set the property on the `<a>` or `.pubdate` elements. This causes the divider to grow to take up the entirety of space within the list element, subtracting the side of the title/date tags.
-
-```css
-.article-list li {
- list-style: none;
- display: flex;
- align-items: center;
-}
-
-.article-list .divider {
- flex-grow: 1;
- border-bottom: 1px dashed black;
- margin: 0.3rem;
-}
-```
-
-### Dark/Light mode
-One of the things that I ended up implementing, that I'm not 100% sold on is the auto dark/light mode. Utilizing CSS we can adjust things to account for if the user is utilizing a light or dark mode on their browser:
-```css
-@media (prefers-color-scheme: dark) {
- /* your stuff goes here */
-}
-```
-It's a very simple technique and given the minimalist nature of this site it works rather perfectly.
-
-Where I'm not 100% happy is on color choices. Where possible I attmpted to tweak things so that they would look the same in light or dark mode. I tweaked the original colors of links (new/visited) and also settled on a particular code-highlighting style.
-
-Where I'm not 100% happy, is with the default font color. In light mode it's browser defaults, but in dark mode it's a blue/grey and I think it might be a tad too dark. It's something that I'll continue tweaking in small batches until I find something that works well.
-
-I also faded out images just a bit so that they're not so jaring. The idea is if you hover/touch the images they'll fade in to full color, but if you're just scrolling through you won't run into any weirdness.
-
-### Support for Utteranc.es
-This[^5] is something that I recently stumbled across - the ability to utilize GitHub issues to track comments on a site. I really love the idea - especially since I'm hosting my blog on GitHub Pages. By adding some params to your `config.toml` file you'll be able to turn on support for utterances.
-
-```toml
-[params.utterances]
-repo = "AngeloR/angelor.github.io"
-issue_term="pathname"
-label="comment"
-theme="preferred-color-scheme"
-```
-
-The settings match exactly the configuration options provided by utteranc.es, so you should be able to see the mapping quite easiy. The only thing I changed was making it `issue_term` instead of `issue-term`.
-
-If you don't add this configuration, it'll just hide the comment block and you won't even load the utteranc.es client js.
-
-## Download/Install
-If you're interested in this theme, you can grab it here https://github.com/AngeloR/plain-hugo-theme
-
-Up to date installation instructions can be found on the readme, but I've included a snapshot of what they were at this point:
-
-```bash
-git clone https://github.com/AngeloR/plain-hugo-theme /path/to/hugo/themes/plain/
-
-Edit your config.toml to add the following line:
-theme = "plain"
-```
-
-If you do end up using it and come across any bugs, please let me know over on GitHub!
-
-## Footnotes
-
-[^1]: https://github.com/AngeloR/plain-hugo-theme
-[^2]: https://tools.ietf.org/rfc/rfc7993.txt
-[^3]: http://westleysz.com/
-[^4]: https://developer.mozilla.org/en-US/docs/Web/CSS/flex-grow
-[^5]: https://utteranc.es/
\ No newline at end of file
+++ /dev/null
----
-title: "DEVLOG: NewsRiver Updates"
-date: 2020-11-10T09:43:04-05:00
-summary: A River-of-News style RSS Reader
----
-
-## What is News River?
-[News River](https://newsriver.xangelo.ca) is an auto-updating stream of news that doesn't need to clutter up your RSS feeds. It was based off ideas from [Dave Winer](https://scripting.com). There are some sites that you don't necessarily CARE about reading every single update. Sometimes you just want to see the most recent stuff and move on. Sites like [HackerNews](https://news.ycombinator.com) and some subreddits ([/r/devops](https://reddit.com/r/devops) or [/r/aws](https://reddit.com/r/aws)) are things that I'm curious about more recent things that get up there, but I don't necessarily want to read every single item. Normally I'm just skimming for interesting titles.
-
-I used to have these things synced up to my RSS reader, but the sheer number of updates to them caused me to go insane. There's no reasonable way to keep up with all of that stuff. So I had to pull them and then just.. remember to go there. Eventually I got tired of that and wrote the first News River system. You provide a list of sources (RSS or RedditJSON) and it will periodically scan them, download the items, save them, and serve them up for you to see in a simple little auto-updating front-end.
-
-It was a pretty handy tool that was written in Node.js and relied on Redis as the backend storage, with everything hosted on Heroku. It was written at a time when I was really digging into various Redis use cases and the free-form nature was a useful thing when trying to figure out the project. However, due to the nature of Redis I had to jump through quite a few loops to "optimize" queries. It had a strange bucketing system for the keys that allowed me to query all updates in the last 10 minutes by grouping all updates into sets where the key was in 5 minute intervals. It was a crazy system and honestly most of the code was handling the bucketing+tests for it.
-
-## Sqlite3
-
-The latest version of NewsRiver actually rips out Redis and replaces it with Sqlite3. The crazy bucketing system no longer exists since I can just sort by rounded date values. By utilizing sqlite3 instead of Redis I was able to move the entire system over to a small VPS that I ran and eliminate the heroku requirement for running it. The move to sqlite3 was really pushed by the need to simplify the code base.
-
-NewsRiver is not a project I want to keep tinkering with. I'd like it to be complete and to be able to use it daily without making a bunch of tweaks. But the date code had become unwieldy and as a result I spent most of time reviewing it to make sure I wasn't breaking it when fixing things. Moving to sqlite3 allowed me to drop that whole system, delete a ton of code, and just move to something that didn't really require a lot of explanation/documentation.
-
-Eventually we can replace the sqlite3 only setup by utilizing something like `knex` to allow for a swappable data storage layer.
-
+++ /dev/null
----
-title: "Dungeon Generation in Roguelikes"
-date: 2021-04-23T09:44:35-04:00
-tags: ["roguelike", "explanation"]
-summary: A simple dungeon generation algorithm for organic rooms
----
-
-
-## Intro
-As always, I've been working on a terrible browser-based game. This time, it
-will be revolving around gameplay mechanics from roguelikes(or roguelites
-technically)+jrpgs.
-
-The thing I specifically want to look at today is the topic of Dungeon
-Generation - both in and out of code.
-
-Roguelikes all tend to share one mechanic: Procedurally Generated Dungeons. The
-idea is that every play-through is different, every floor of a dungeon is
-completely unique. As a result, it requires the player to learn the mechanics of
-the game over multiple runs as most roguelikes also include perma-death as a
-required feature.
-
-Dungeon Generation in roguelikes are a very interesting topic for two reasons.
-There is a definite technical component to being able to generate "good"
-dungeons quickly. Dungeons that are large enough, with enough rooms/pathways to
-be interesting, but that don't take forever to generate. The goal is each floor
-is generated randomly when the player enters - we don't want load times
-equivalent to stepping into a room in Morrowind. The second reason is one that I
-think roguelikes can miss out on - dungeon generation must fit the environment.
-
-Most roguelikes have a generation alorithm that gets assigned to every single
-floor. But that's not really that helpful. Dungeons should reflect the
-environment the player is in. Having square rooms in a castle dungeon makes
-sense. Having square rooms in a cave, not so much.
-
-A lot of dungeon generation information you find online tend to focus on the
-techtechnical component: HOW do you generate a dungeon. I'm hoping to also cover
-the second bit here.
-
-## Random
-The easiest to explain algorithm is the "Random" one, but it's probably the
-hardest to get right. Random dungeons are exactly what you'd think -> just
-randomly place obstacles and interaction points around the dungeon. As a result
-it's very easy to actually MAKE the dungeon.
-
-It also has the added benefit that the "random" look works really well for open
-fields. Think areas that are strewn with trees or rocks or something like that.
-It could also work really well for large underground caverns since you would be
-peppering the inside with obstances.
-
-But we're not just making a dungeon - we're making a game. And truly-random
-dungeons are hard to tune. How do you ensure that every chest you place is
-actually reachable by the player? How about the one set of stairs? How do you
-ensure that the player doesn't spawn in a box, closed off from the rest of the
-dungeon. How do you ensure that they don't spawn right next to some stairs?
-
-Each of these questions (and many many more) result in you tuning you random
-generation more and more. You'll never get it 100% right. There'll always be
-edge cases reported by your places that you didn't even think about (did a
-monster spawn in deep water so the player didn't even know it was there and is
-now reporting that they didn't get an achievement for killing all the monsters
-on a foor?).
-
-I started with a purly random dungeon generation system myeself. It's definitely
-not a bad call to make - but you just have to be aware of the edge cases. But I
-kept having to tune/adjust things and I'd still end up with play testers saying
-they got spawned in a box, or couldn't reach the stairs. That's frustrating
-enough to probably just stop playing.
-
-The nice thing about random, however, is that you don't NEED to generate the map
-in its entirety. Nor do you need it to be entirely random.
-
-
-### On-demand generation
-So you have the container for your map.. and your character is spawned in a
-particular point `(x,y)`. Given a field of view (`v`) really you only need to
-generate a box defined by the points `(x-v, y-v)`, `(x+v, y-v)`, `(x-v, y+v)`,
-and `(x+v, y+v)`.
-
-If the player steps in a direction, you only need to expand the generated
-portion of the map by 1 tile (or whatever your fov is). In this way, your entire
-dungeon is being generated as the player discovers it. You get a few benefits
-like not needing to store the entirety of the map if the player doesn't visit
-it. You can tune drop rates for everything just by how much of the map is
-discovered vs. isn't. You can also almost guarantee that EVERY point on the map is
-reachable by the player and that they'll discover the stairs exactly when you'd
-like them to.
-
-### Pseudo-random Fabrication
-Now, lets get the pedantics out of the way - nothing we're doing is random, it's
-all pseudo-random. The difference is that in this generation mechanic we're
-actually building the map from pre-defined map parts.
-
-The downside, of course, is that you have to spend a bunch of time generating
-the pieces of map and you do have to have to have some guidelines. But, being
-able to tune each individual section of map gives you a ton of control over the
-actual gameplay. It also allows you to use non-standard map shapes (get outta
-here square rooms) and generate really unique looking maps.
-
-You would need to rotate the location of interactables, but with enough of these
-map pieces being put together in random orders it gives you a lot of variation
-for your players. And if the unlikely event that they end up getting the EXACT
-same map somehow (you know, cause random), the locations of all the interactable
-items will be randomized.
-
-## Standard Dungeons (Connected Squares)
-There are a million of these tutorials as this is the standard look for
-dungeons. I think they work wonderfully when you're actually exploring a dungeon
-in a castle or something - but otherwise they seem out of place
-
-The premise is pretty simplem but it involves iterating over the map numerous
-times to achieve the look you want. The steps themselves are pretty
-straight-foward.
-
-1. First go over the map and generate rooms (squares or rectangles, whatever you
- want).
-2. Then go over the map again and connect the rooms together via pathways.
-
-Normally you'll make multiple passes over the map to generate the appropriate
-room density that you want. The more rooms, the easier it is to connect them
-all.
-
-The connection part CAN be confusing, but the easiest way is to actually look at
-various path-finding algorithms. Since the map doesn't actually exist, any
-pathfinding algorithms will find the straightest line possible between your
-rooms.
-
-Simply iterate over each pair of rooms and connect them using your chosen
-path-finding algorithm. [A\*](https://csis.pace.edu/~benjamin/teaching/cs627/webfiles/Astar.pdf)
-is always a good choice to understand the real basics of the algorithm so that
-you can implement it yourself. Or at least so you understand what's happening
-before using whatever package in your programming language of choice.
-
-The pro's of this technique is that it's a very well documented approach. You
-can adjust the density of the map very easily.
-
-So easily, in fact, that you can ramp up the density to the point that all of
-the rooms overlap. The iterate to remove random sections of obstacles (walls)
-that are inside the room. That will allow you to generate LARGE rooms where
-everything is accessible.
-
-## Drunk Walking
-The Drunkards Walk is a very simple algorithm and one that I stumbled upon
-without really knowing about it. The idea is simple and is based on these requirements:
-
-1. You want to generate organic looking maps
-2. You want to ensure that all sections of the map are reachable
-
-The organic maps requirement is probably the most interesting one to me. The
-ability to random generate maps that don't look like a bunch of pre-defined
-shapes. You could, technically, achieve that same look a myriad of ways, but
-this seems the easiest.
-
-The method is as follows:
-1. Create a "walker" on your map of some size (maybe 4 tiles? maybe 9? whatever
- you feel like)
-2. Start them on one side of the map, with their edges being walls, and their
- interiors being walkable floors.
-3. Make them march to the opposite side, adding some jitter along the other
- axis.
-
-The size of the "walker" dictates the minimum width of the space you're
-generating. If you want something to feel more open, make it larger. But if
-you're making caves or something, just make them smaller.
-
-The "drunken walk" step sounds confusing, but it's pretty straight-forward. If,
-for example, we start the walker on the west side of the map, they will be
-walking to the east. Every step they take to the east should be coupled with
-them randomly shifting north/south by a MAX of the size of the walker.
-
-That's it.
-
-When you're done, you have a single corridor. To build a map, add more walkers
-all moving along the same plane (west->east for example), and then add one or
-two walkers moving from the perpendiclar plane (north->south in this case). This
-ensures that all corridors will be connected, and gives you really neat looking
-maps. To me, they work wonderfully for caves/pathways.
-
-
-But, by playing with the size of the walker you can change the entire look of
-the map. Really wide players give you huge open spaces!
-
-the only thing this DOESN'T do, is actual square looking rooms like if you were
-in a real dungeon..
+++ /dev/null
----
-title: "DEVLOG: Infinite Clicker, with a story"
-date: 2020-06-07T12:09:04-04:00
-draft: true
-tags: ["devlog", "sketchy-heroes", "devlog-sketchy-heroes"]
----
-
-## Introduction
-I've been working on Sketchy Heroes for about 2 years now. Honestly it started as more of a learning
+++ /dev/null
----
-title: "Discovering Problems as a Manager"
-date: 2021-11-01T17:11:57-04:00
-tags: ["management"]
-draft: true
----
-
+++ /dev/null
----
-title: Started from *Free* now we're here
-summary: How FREE has shaped my life
-tags: ["game dev", "mtx", "bio"]
-date: 2023-06-19T06:12:47.302-04:00
----
-
-
-Neopets was the first time I'd ever heard about HTML or CSS. I had been playing terrible games for a long time at this point, even digging into basic at one point to try and do something.. but I didn't have the interest at the time. I didn't have anyone in my (or my parents) circle that was interested in computers at the time, so I never even considered what was possible. Neopets was a free game where you took care of a digital pet and then played a ton of games to earn in-game money. They even had a huge "Auction house" that was made up of individual stores that users ran. You could start a store, and set up your own items for sale. It was amazing. It also gave you a little area to enter HTML/CSS snippets so you could customize your store. That was why I bothered learning HTML/CSS at all. To customize my neopets store. They had the same interface for your clan page - customize your entire clan page with HTML and CSS. It was amazing. It was the first time anything related to "programming" clicked and I realized I loved it.
-
-I don't remember a single ad on neopet trying to force me to make a purchase of a premium currency. There probably were.. but I don't remember them at all.
-
-After learning HTML/CSS and offering my services to other neopets players in exchange for ingame goods.. I grew out of what neopets offered. In that time I discovered geocities and it DHTML. I loved it. I grew out of that too when I started digging into PHP and started looking around for free shared hosting. There were a lot of hosting companies at the time.
-
-One in particular, Host Matrix, caught my attention - because I love the Matrix. They were having a promotion where if you wrote long form tutorials for them (around HTML/CSS/PHP) you could get free hosting! And so I did. I wrote a lot of tutorials so that I could get free hosting and continue into this weird world I had stumbled in to.
-
-Around the same time I picked up the guitar. My dad had played the guitar since his teens, and so there was always one around. I had never shown an interest in it.. but one day I picked it up and started learning how to play. Mostly because of tabs (tabulature) - a fingering based musical notation that works beautifully for the guitar. People, on the internet, would listen to songs, figure out what they were playing.. and then write it down and put it on the internet.. for free! I devoured everything I could find. Eventually I got good enough that I could write my own tabs and share with others! It was incredible.
-
-When I started branching out musically.. it was because of free access to it. The first time I had heard Ravenous by [Arch Enemy](https://archenemy.net/en/) my mind was blown - I ran home and downloaded the song. I listened to it every day for a week and then started learning it. When I discovered Children of Bodom and Kalmah it was the same thing. Obscure bands that I would never have had the hope of finding out, accessible to me because it was free.
-
-Don't get me wrong, I understand there were costs to creating this. I've spent the last decade of my life building SaaS software for companies and managing infra budgets. I understand the cost. But also, I understand that I wasn't a customer. Arch Enemy didn't lose money by having me download their album. I didn't have access to purchase their album and I didn't have the money to do it. Arch Enemy gained a fan that would later on buy tickets to shows, and albums, and t-shirts. A fan that would have never bothered if he hadn't heard their albums for free.
-
-Free played a huge role in my life and I've always loved it. I donate a few hours every week to local startups that need technical advice. I hang out in slack groups answering questions. I hang out in Magic discord groups answering questions. I write software and give it away for free. I do it because I owe so much to free.
-
-And now I sit here trying to figure out monetization strategies for my projects.. and I don't like it. I wish I could do it for free, but there are costs associated with it that I know I won't be able to subsidize forever. I know I want as much of it to be free as possible. I don't want people to feel like they need to pay, I want them to want to. So I have to think about the best way to monetize [Rising Legends](https://www.risinglegends.net) that are in line with my philosophies...
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
---
-title: "Gemini, an HTTP alternative"
+title: Gemini, an HTTP alternative
date: 2021-06-21T17:11:57-04:00
-tags: ["gemini", "upkeep"]
+tags:
+ - gemini
+ - upkeep
+ - publish
summary: An HTTP alternative
+lastmod: 2024-12-12T05:25:23.255Z
---
-
For the last few months now I've started embracing my weird love for text and text-related user interfaces. From email to irc, I'm moving more and more of my social interactions away from traditional websites/apps and over to my terminal. A big part of this move is a step back from 3rd party applications to things that I have a bit more control over - but there are many reasons.
-* Privacy: Text only platforms make it very hard for trackers. Most email nowdays include "tracking pixels" that allow the sender to know when you opened the email and to gather information about the device what opened the email itself.
+* Privacy: Text only platforms make it very hard for trackers. Most email nowdays include "tracking pixels" that allow the sender to know when you opened the email and to gather information about the device what opened the email itself.
* No ads: When everything is text, there's 0 need to include the ability to deal with images. As such advertisements just can't happen in a traditional format. The closest you can get is what you think of when you look at global chat in MMOs - people just spamming whatever they want. As a result it makes it easier to deal with (blocking).
* Faster: As we've added more and more compute power to devices, the tools and applications that we use have just gotten larger and less efficient. It's very common for websites loaded to be 1mb in size - that's kinda terrible. By moving to a text-only interface, you lose the ability to include a bunch of extra things and you end up with what matters - Just The Content.
* Interoperability: When everything is text, it's much easier to wire up multiple tools together. Think of the unix interface philosophy. Every tool does one thing well, and the input/output is text. So you have the ability to wire up a chain of tools that the original authors don't need to know/care about.
But I still found myself looking for an alternative. And I found Gemini.
## What is Gemini
-[Gemini](https://gemini.circumlunar.space) is an alternative to the HTTP protocol. It's definitely not going to replace HTTP, nor does it want to. Instead it aims to operate alongside. It's a very stripped down subset of markdown to enable easy implementations of parsing. A lot of the complexity of modern browser systems is around parsing HTML/CSS effectively and gemini just does away with that whole idea.
+
+[Gemini](https://gemini.circumlunar.space) is an alternative to the HTTP protocol. It's definitely not going to replace HTTP, nor does it want to. Instead it aims to operate alongside. It's a very stripped down subset of markdown to enable easy implementations of parsing. A lot of the complexity of modern browser systems is around parsing HTML/CSS effectively and gemini just does away with that whole idea.
This means that documents in Gemini are very light-weight, and are readable without any parsing. You could just read the raw text that makes up a gemini doc (called gemtext) and be just fine. Any styling choices are entirely up to the renderer (browser mostly) and are irrelevant to the document itself. There is no way to view images in-line, there is no JS, there is nothing but text.
I really recommend you check out the gemini home page and do a bit of reading up on it.
## Accessing Gemini
+
Since gemini is a completely different protocol you can't access it with a regular web-browser that deals in HTTP. Instead you have to find an alternative browser that deals with the gemini:// protocol. There are [numerous](https://gemini.circumlunar.space/software/) owing to the simplicitly of the parser implementation. The one I've been using most is a terminal based one called [amfora](https://github.com/makeworld-the-better-one/amfora), but if you're looking for something graphical I highly recommend [Lagrange](https://gmi.skyjake.fi/lagrange/) which is wonderful.
-Obviously this is just a browser, so you need to go somewhere. I recommend starting at [geminispace.info](gemini://geminispace.info).
+Obviously this is just a browser, so you need to go somewhere. I recommend starting at [geminispace.info](gemini://geminispace.info).
-One thing I should caution you - don't expect a full replacement for what you consider "the internet". Gemini is very simplistic and focuses on ease of content consumption. It doesn't make content creation hard, but because of the focus on content itself, "interactions" are minimal. Some capsules (the gemini equivalent of a website) have "comment" sections, which harken back to the guestbooks of the 90s.
+One thing I should caution you - don't expect a full replacement for what you consider "the internet". Gemini is very simplistic and focuses on ease of content consumption. It doesn't make content creation hard, but because of the focus on content itself, "interactions" are minimal. Some capsules (the gemini equivalent of a website) have "comment" sections, which harken back to the guestbooks of the 90s.
## Why I'm interested in Gemini
+
The thing that attracts me to Gemini is the same thing that attracts me to most projects: putting data privacy/ownership above all else.
Getting technical, Gemini foces TLS connections. Nothing is transmitted un-encrypted. It also brings to the forefront client certificates in SSL. The original spec for certs already supports this and it's crazy that it never took off. It honestly resolves the whole account identification/password problem in a really neat way.
### A sidenote on SSL and Authentication
+
This is, I think, one of my favourite features of gemini. SSL certs have two sides, and most people are only familiar with one. Servers generate certs to identify themselves. But since anyone can just generate a cert, there is really no way of knowing that the server you are connecting to is who they say they are. That is, I can generate a cert for duckduckgo.com, and you would have no way of knowing I'm NOT the actual owner of that site. One way that we currently resolve this problem is "Certificate Authorities".
Certificate Authorities (CA) are just a collection of companies that at SOME POINT were declared trustworthy. You had no say in that process, and despite numerous hacks of the CAs, they're still considered "trustworthy". You pay these companies to certify that you are who you say you are. There are various levels of "validation" that you can do, but the end goal is that your browser trusts a bunch of these CAs. When they see a cert from your website signed by a CA, they trust it.
The "certification generation" part is the tricky part and is why this never caught on for authentication on the web. But in Gemini, which is currentyly a more technical audience, it's a bit more feasible as a solution. What it means is, as a user, I can generate a single "account" and reuse it everywhere.
## Updating Hugo to output Gemini valid content
+
I'm interested enough in Gemini to put some time into modifying my website to output docs in gemtext so that I can serve my site over gemini as well as http. There are a few steps here (including getting off github pages so that both domains can be served over the xangelo.ca root) but that's ok.
The first thing I had to do was figure out how to get my content into gemtext. The easiest way was just to utilize hugo and add a new gemini output. There are a couple resources on this, but the two that were the most helpful were [this article by Sylvain Durand](https://sylvaindurand.org/gemini-and-hugo/) and [this one by Wouter Groenveld](https://brainbaking.com/post/2021/04/using-hugo-to-launch-a-gemini-capsule/). I won't re-hash what these are but the final configurations for my site can be found here:
```bash to start agate
./agate.x86_64-unknown-linux-gnu --content ./gemini/ --addr [::]:1965 --addr 0.0.0.0:1965 --hostname gemini.xangelo.ca --lang en-US
- ```
+```
For now, scp is fine - but I'll probably just set up a gemini submodule/repo and deploy that to my server instead. That way I don't have to worry too much about transferring the entirety of the site.
- ## Conclusion
- Overall, I'm pretty excited about Gemini. The ease of getting into the ecosystem as a consumer and publisher is amazing. There are some awesome projects to really ease that gap. But there's also plenty of room for growth and a small enough community that you can get involved in. The best place to get into it is [gemini space](gemini://gemini.circumlunar.space). I think the thing that excites me most about it is just the fact that there's such a big focus on privacy and small contributors. I think that's something that's missing from the web today, that we used to have at some point.
+## Conclusion
+
+Overall, I'm pretty excited about Gemini. The ease of getting into the ecosystem as a consumer and publisher is amazing. There are some awesome projects to really ease that gap. But there's also plenty of room for growth and a small enough community that you can get involved in. The best place to get into it is [gemini space](gemini://gemini.circumlunar.space). I think the thing that excites me most about it is just the fact that there's such a big focus on privacy and small contributors. I think that's something that's missing from the web today, that we used to have at some point.
---
-title: "Git Branching Strategies"
+title: Git Branching Strategies
date: 2021-07-16T16:14:40-04:00
-tags: ["git", "development", "dev workflow", "engineering processes"]
+tags:
+ - git
+ - development
+ - dev workflow
+ - engineering processes
+ - publish
summary: A topic that every developer in the world has an opinion about
+lastmod: 2024-12-12T05:25:42.443Z
---
+In a [previous](/posts/git-workflows/) post, I talked about git workflows and I outlined the two big problems I see with git workflows:
-In a [previous](/posts/git-workflows/) post, I talked about git workflows and I outlined the two big problems I see with git workflows:
* Pull Requests/Reviews
* Branching Strategies
-I took the easy route at the time and just talked about Pull Requests/Reviews. But I was requested to throw some thoughts out around branching strategies and how they should work.
+I took the easy route at the time and just talked about Pull Requests/Reviews. But I was requested to throw some thoughts out around branching strategies and how they should work.
-The reason I think branching strategies are such a complicated topic is because it touches more than just git. As an industry we've mostly settled on the idea of Continuous Integration and Continuous Deployment (CI/CD). These processes tend to tie directly into git and have deep ties to your branching strategies. In addition branching strategies tend to reflect the testing architecture and experience level of the engineering organization itself. It's not the simple strategy that workflows like GitHub Flow like to pretend it is. Whether you like it or not, your git branching strategy is already embedded in your organiztion and reflected in your development process. It's your job to discover what that is and then decide if it's even worth it to change. Sometimes, unfortunately, even though it may not be what you consider ideal, the strategy the team is currently using is MOSTLY one that works for them. At that point it's just a matter of uncovering bottlenecks (oh don't worry, the engineers will tell you what those are) and mitigating them.
+The reason I think branching strategies are such a complicated topic is because it touches more than just git. As an industry we've mostly settled on the idea of Continuous Integration and Continuous Deployment (CI/CD). These processes tend to tie directly into git and have deep ties to your branching strategies. In addition branching strategies tend to reflect the testing architecture and experience level of the engineering organization itself. It's not the simple strategy that workflows like GitHub Flow like to pretend it is. Whether you like it or not, your git branching strategy is already embedded in your organiztion and reflected in your development process. It's your job to discover what that is and then decide if it's even worth it to change. Sometimes, unfortunately, even though it may not be what you consider ideal, the strategy the team is currently using is MOSTLY one that works for them. At that point it's just a matter of uncovering bottlenecks (oh don't worry, the engineers will tell you what those are) and mitigating them.
The two main branching strategies people tend to advocate for are "Environment Branching" and "Feature Branching" so lets talk about the strategy itself, and then see how it affects your org.
## Environment Branching
-Environment branching models the fact that the software at your org has some kind of "release cycle" where a series of patches (in the git sense) are first in a "development" environment where the developers involved can work on the feature without impacting others. We then merge the dev branch into a "staging"/"testing"/"integration" environment where a patch or series of patches are released for other people in the org to test. Often this involves a QA team looking at the changes and validating that there aren't any regression tests. Finally once everyone is happy we merge staging into production and deploy it.
+
+Environment branching models the fact that the software at your org has some kind of "release cycle" where a series of patches (in the git sense) are first in a "development" environment where the developers involved can work on the feature without impacting others. We then merge the dev branch into a "staging"/"testing"/"integration" environment where a patch or series of patches are released for other people in the org to test. Often this involves a QA team looking at the changes and validating that there aren't any regression tests. Finally once everyone is happy we merge staging into production and deploy it.
```Environment branching example
┌┬──────┐ ┌┬───────┐ ┌┬──────┐
└─┘
```
-Note that not every commit to the dev branch results in an equivalent merge to the staging branch. Infact sometimes there'll be plenty of back/forth on the dev branch. Finally a set of features is selected and deployed to staging. Once it's been debugged, it goes off to production. Of course, this is when everything goes smoothly. Most times you'll have hotfixes that you'll need to make to prod to fix additional bugs.
+Note that not every commit to the dev branch results in an equivalent merge to the staging branch. Infact sometimes there'll be plenty of back/forth on the dev branch. Finally a set of features is selected and deployed to staging. Once it's been debugged, it goes off to production. Of course, this is when everything goes smoothly. Most times you'll have hotfixes that you'll need to make to prod to fix additional bugs.
Different orgs deal with that differently. Sometimes we'll have "hotfixes", which bypass the branching model entirely. It allows us to take whatever is it prod, branch from it to fi xa bug and merge it straight back into prod without going through the other branches. That hotfix is then ported back to development.
This model, of course, has its pros/cons. But if it works for your org then it's right for your org. I've outlined some caveats and things to think about if you decide to go with the environment branching model.
### It forces a release cadence
+
When you have a single team working in this model it's very easy. It also maps so cleanly to traditional engineering pipelines that it doesn't require a lot of explanation or documentation. But this requires that each team working has its own "dev" branch/environment to ensure that they're not impacting others testing. But this is expensive, so there's a tradeoff. Often you'll find "shared" dev environments, which are really just a very broken integration environment where you can't really trust what's happening, and a less broken integration environment. Just the inclusion of an environment that's considering "integration" requires scheduled releases.
-You need to ensure that your features aren't impacting other developers and so you need to coordinate between internal teams. This takes time. It means while your changes can be tested in tandem in "dev", eventually you'll need to settle on what's going out to production adn only those patches get moved into staging. Those patches get tested on for some time and then move into production. There you have a release cadence. Your cadence might be a day, or it might be two weeks. It doesn't matter - that is your cadence. That is the minimal amount of time, on average, that it takes for a change to make it to production.
+You need to ensure that your features aren't impacting other developers and so you need to coordinate between internal teams. This takes time. It means while your changes can be tested in tandem in "dev", eventually you'll need to settle on what's going out to production adn only those patches get moved into staging. Those patches get tested on for some time and then move into production. There you have a release cadence. Your cadence might be a day, or it might be two weeks. It doesn't matter - that is your cadence. That is the minimal amount of time, on average, that it takes for a change to make it to production.
+
+There are always ways to mitigate this problem, but I'd argue that it isn't a real problem. Having a release cadence is perfectly fine once you have a product that has paying users. It ensures that releases can happen during low-volume traffic times and doesn't impact as many users. It also ensures that users know when they can expect new features.
-There are always ways to mitigate this problem, but I'd argue that it isn't a real problem. Having a release cadence is perfectly fine once you have a product that has paying users. It ensures that releases can happen during low-volume traffic times and doesn't impact as many users. It also ensures that users know when they can expect new features.
* Have dedicated dev environments for each team with no shared resources to speed up the development time. Keep the "staging"/"integration" environment the only place where multiple features/bug-fixes are tested in tandem.
* Declare your cadence officially so that everyone knows when the cutoff for a release is. Stick to this. No last minute sneaking in of changes allowed. This allows internal/external expectations for when new features will be rolled out.
* Understand that there is a difference between deploying a change and releasing it to users.
### It's very easy for the branches to get out of sync
+
Since you now have multiple teams working it's easy for features that everyone thinks is ready to make it to integration.. only to discover that they are quite broken. So we need to pull them out so that they're not impacting anything else. But how do you do that? You can't just muck with staging directly since merges from dev->staging will then break. So you need to actually back the change out in dev, then merge dev->staging to remove the change. But are you using squash+merge or rebase functionality? That can affect the method of resolution you decide on. Regardless, you can expect dev/staging/prod branches to get out of sync and you'll need to manually intervene to sort that nonsense out.
### It's easy for broken things to get out
+
The dev environment will probably be mostly broken - with working state only conveyed via some internal communication among devs. Merging that right into staging will cause unexpected bugs or broken features to make it out. The best way to deal with this is to have each team working on a single feature/code-base and having dedicated development environments for each team. This way you ensure that a team is able to test what they are working on in isolation. Once they're sure it's working they'll be able to merge their fixes into the integration environment. This is suspiciously close to the other model of branching.
### Conclusion
+
I'm really not a fan of environment branching, but I do concur that when there are processes/releases cadences already in place that are immovable (imagine you have a 3rd party group that needs to do testing on releases before they go out) then this can be a good way to handle dependencies. Personally I think environment branching is easy for SREs/devops teams to think about and they map very cleanly to how software orgs traditionally worked. It provides the lowest barrier to adoption - but it has enough problems that it can seem like the new process doesn't work as well as the old one.
I would be wary about moving an org TO this model as there's just a lot of things that can go wrong that will result in certain individuals spending time rectifying process problems rather than business ones.
## Feature Branches
+
This is the other model of branching that I think attempts to tackle some of the major problems with environment branching (while offering up a few of its own). In feature branches you don't have a single dev branch.. instead every thing you work on gets its own "dev" branch. It's expected that while you're working on your thing your feature branch is probably in a "broken" state. You keep adding commits to it and eventually when you're happy you merge it to your integration environment branch. Once it's tested, the integration environment gets deployed to production.
Feature branches aim to solve the problem of environment branches by integrating the "staging" and "production" branches. Devs work on feature branches, and then merge their features into "integration" environments. When people are happy with that, the features get moved into production environments. There's a lot less moving parts (on the surface) and so this is normally the process that people go with.
## Branching Strategies Affect Everything
+
These branching strategies sound great - but branching strategies aren't an isolated problem that can be solved without affecting anything else. Environment branches require you to have multiple environments for your code. It requires you to have "development" and "staging" and "production" environments. All branching strategies require you to differentiate between "code" and "build artifacts" which are actually what you should be deploying.
---
-title: "Git Workflows Suck"
+title: Git Workflows Suck
date: 2021-07-15T12:16:45-04:00
-tags: ["git", "development", "engineering processes"]
+tags:
+ - git
+ - development
+ - engineering processes
+ - publish
summary: Common pitfalls with everyones "git process"
+lastmod: 2024-12-12T05:25:48.171Z
---
-
I'm tired of all the git workflows. I'm tired of talking about the best way to craft commit messages. I'm tired of how people pile on the best way to do PRs. I'm sick of people picking apart architectural decisions in a PR. I'm tired of opening PRs only to have no one look at them for days while I continue to harass people into reviews. I'm over the fact that I don't think I've ever seen a single person clone a PR locally and actually verify that it does what it says it does.
And I don't think I'm alone in these problems. Almost every engineering team I've ever worked with has their own slightly customized git workflow that "works" for them. And every one has had the same problems I mentioned above. Honestly I think it's just cause no one uses git properly.
The problem, of course, is not "using git". The problem is from all the interconnected bits around it. Writing a commit message is easy. Writing a commit message on code that no one at your company knows anything about, where you're pretty sure it does what you want, is much harder. In order to even talk about "using git properly" there's so much background that needs to be talked about.
## What is git?
-Git has this as the first sentence on it's site:
-> Git is a free and open source distributed version control system designed to handle everything from small to very large projects with speed and efficiency.
+
+Git has this as the first sentence on it's site:
+
+> Git is a free and open source distributed version control system designed to handle everything from small to very large projects with speed and efficiency.
Git is a free and open source *distributed version control system*.
Traditional Version Control Systems (VCS) tended to be centralized. That is you had a "server" where your code resided. Developers would "check out" the files that they wanted to work on. Do the work locally, and then re-upload them to this central server. git changed that model. EVERYONE has a copy of the code. You make your changes locally, and you can push to anyone else you'd like! You could make a change, and then send that code to a friend directly from git USING git. Push to their repo. They could do the same to you. Instead of having one central server that contains all the code, people work with each other.
## What are Forges?
-Of course, that works really well when you're a small group that's making changes. When things get larger, it can be a bit more difficult. There are inherent problems with having copies of your code scattered around. Who has the most recent version? Who has the authority to merge changes in? How do you communicate these changes to everyone?
-These problems are what propelled tooling like GitHub, GitLab, and BitBucket into the spotlight (known as a Forge). They became these central repositories for your code that allows you to not have to think about that. They allow you a central location where you push/pull your code. They provided an interface for Pull Requests. The ability to track issues and correlate them with code changes. They provided a service that filled a gap in git tooling. They even started publishing general "workflows" on how best to use git that are all slight variations on this:
+Of course, that works really well when you're a small group that's making changes. When things get larger, it can be a bit more difficult. There are inherent problems with having copies of your code scattered around. Who has the most recent version? Who has the authority to merge changes in? How do you communicate these changes to everyone?
+
+These problems are what propelled tooling like GitHub, GitLab, and BitBucket into the spotlight (known as a Forge). They became these central repositories for your code that allows you to not have to think about that. They allow you a central location where you push/pull your code. They provided an interface for Pull Requests. The ability to track issues and correlate them with code changes. They provided a service that filled a gap in git tooling. They even started publishing general "workflows" on how best to use git that are all slight variations on this:
* clone from remote
* create a new branch from master
There are variations, as always. [Git Flow](https://nvie.com/posts/a-successful-git-branching-model/) talks about conventions for branch naming/merging processes. [GitHub Flow](https://guides.github.com/introduction/flow/) and [GitLab Flow](https://docs.gitlab.com/ee/topics/gitlab_flow.html) do the same, but with a focus on their tooling instead of general git level instructions.
## The Big Git Workflow Problem
-The problems people have with git workflows tend to stem from two areas: Pull Requests and Branching Strategies. I'm going to avoid chatting about branching strategies here because there is a lot more to consider than just naming conventions. A lot of companies have a lot of additional tooling that is very integrated into the git branching styles used. Things like CI/CD that use environment branches, merging to master auto-deploying to certain environments - all these things force additional constraints on the branching strategies decided upon. Often it's very hard to change these things and you're left in an in-between state indefinitely that is much worse than the original.
-Instead I'm going to focus on Pull Requests and the processes around it: Commits, Pull Request Reviews, Merging Strategies.
+The problems people have with git workflows tend to stem from two areas: Pull Requests and Branching Strategies. I'm going to avoid chatting about branching strategies here because there is a lot more to consider than just naming conventions. A lot of companies have a lot of additional tooling that is very integrated into the git branching styles used. Things like CI/CD that use environment branches, merging to master auto-deploying to certain environments - all these things force additional constraints on the branching strategies decided upon. Often it's very hard to change these things and you're left in an in-between state indefinitely that is much worse than the original.
+Instead I'm going to focus on Pull Requests and the processes around it: Commits, Pull Request Reviews, Merging Strategies.
### Commits
+
The smallest unit of the PR is the commit. Now, even at this stage, there's lots of argument on how to proceed but there's at least one "agreement": A commit should be one thing.
-Of course, that's much easier said than done. A lot of the time you'll be in your code and then you may see some small bit of logging or metrics that could be improved. It's only a one line change? May as well make it now to improve things for everything. Sometimes the changes you're making end up really being two indepedent changes that COULD work separately, but only together do they make a whole feature.. is that one commit or two?
+Of course, that's much easier said than done. A lot of the time you'll be in your code and then you may see some small bit of logging or metrics that could be improved. It's only a one line change? May as well make it now to improve things for everything. Sometimes the changes you're making end up really being two indepedent changes that COULD work separately, but only together do they make a whole feature.. is that one commit or two?
Once you figure out what's going to be IN your commit, then you get to figure out what message you to leave your future self.
-I used to think that [https://chris.beams.io/posts/git-commit/](https://chris.beams.io/posts/git-commit/) was a great example on how to write commit messages. I've since changed my mind a bit to be less prescriptive about process and more descriptive about outcome.
+I used to think that <https://chris.beams.io/posts/git-commit/> was a great example on how to write commit messages. I've since changed my mind a bit to be less prescriptive about process and more descriptive about outcome.
+
+The goal for a final commit message is that when you look at this message in a year, what's the best way for you to recontruct the context that you had when you came up with your solution. In order to do this you need to have:
-The goal for a final commit message is that when you look at this message in a year, what's the best way for you to recontruct the context that you had when you came up with your solution. In order to do this you need to have:
* A good summary sentence, as short as possible and no shorter (the hardest part, I always write this last)
* A good explanation of what your single change is. Go as in-depth as you want, but if you find yourself writing multiple paragraphs, put that in a separate document and then add it to the references section.
* A References section with links to ticket/issue/tasks related, any documentation you used to come up with your solution (possibly links to other repos if you used a particular pattern that you found elsewhere)
These types of active commits are fine, but once you're DONE and you're happy with what you did, it's time to REBASE your branch. Rebasing seems like a bit of an ambiguous word, but what we are going to be doing is taking a look at the git commit history you've created, and then squashing them all up into a single commit or maybe a couple if you tried to do multiple things. I don't want to go into all the details about this, because there are plenty of instructions for how to accomplish what you want [this website being one of the better ones](https://git-rebase.io/) I've run into.
-Rebase allows you to see all the commits between two different commits, and allows you to decide what to do with them. So if you made 10 commits that were "fix this typo" and one that had the actual implementation, you could choose to squash all those commits in a single one just by telling git "squash this commit" and choose to "pick" the commit that you want to remain as a commit. Git even gives you a nice little interactive interface that allows you to do this (it is a text based so don't get too excited).
+Rebase allows you to see all the commits between two different commits, and allows you to decide what to do with them. So if you made 10 commits that were "fix this typo" and one that had the actual implementation, you could choose to squash all those commits in a single one just by telling git "squash this commit" and choose to "pick" the commit that you want to remain as a commit. Git even gives you a nice little interactive interface that allows you to do this (it is a text based so don't get too excited).
Rebasing is important because it allows us to move from an "active commit" state to a "code complete" state. We are officially declaring that we are done with this bit of work and we don't intend to do anything more with it as it relates to the task that we're working on. When we rebase we get a chance to edit your final commit message, which should ideally follow the rules above.
### Pull Requests
+
Pull requests is where I have most of my problems. PRs are a way for devs to share the changes they've made with other devs, and gather/address feedback. Things you may have missed in your changes, other repos to look at for similar things. In large orgs it's not unlikely that someone might point out to you a repo you've never heard of that does something similar that you can take inspiration from. These are valuable interactions to have - but they happen at the wrong time.
Pull Requests are always at the end of development process on a task. Once you've finished your changes, you submit them for review. I think this is just too late, and I'll explain why with some... history!
Before anyone actually DID the work there was plenty of back and forth in the mailing list about the change that was going to happen. Not just theoritcal "we could do this", but code samples would be sent around, links to relevant code bits would be included so everyone had the same context. The evolution of the change was publicly documented and the subject matter experts were able to weigh in on the how they'd approach the change. This would go back and forth until there was a VERY clear understanding of what the change would look like. Then someone would take that and get it over the line. They'd then send the patches they made back to the mailing list for people to look at. This almost ALWAYS involved taking the patches, applying them locally and doing doing some testing to validate that it was working. At this point, the goal is that changes at this point are not major architectural ones, but smaller ones to tackle edge cases that people might have uncovered during testing.
-This is the process that PRs are attempting to replicate -> except because of how most companies operate the details of the change only happen when you start working on it, and the architectural decisions are often only reviewed once they've been made.
+This is the process that PRs are attempting to replicate -> except because of how most companies operate the details of the change only happen when you start working on it, and the architectural decisions are often only reviewed once they've been made.
I think "Pull Requests" should really be put into multiple categories. There's an initial invesitgation/proposal process where people communicate and collaborate over how to approach the solution to a problem. This should definitely be time-boxed as it's too easy for this to stretch into infinity. The goal is to do those investigations and allow SMEs to contribute to the conversation early. The goal for this document is to have a very clear understanding of the change needed to be made. Schema changes, repos that might need to be adjusted AND WHERE. If there are no links to code within this then you're probably not done. Someone should be able to grab it and move ahead without necessarily being part of the conversation to craft the solution.
Then there's the Review process. After the code is complete sharing it with others and having them RETRIEVE YOUR CHANGES AND TEST THEM LOCALLY. Testing locally is important for many reasons - often the main one is that your test suite isn't robust enough, or it isn't trustworthy (then delete them, but that's another story). But also it highlights pain points in your development process that you may not deal with always. Missing documentation for how to run something, or how to set up some data for a test. Not being able to run your tests locally because they take too long. Having people really test your code is a great final step to getting your code reviewed.
## Conclusion
+
This was a lot of text - but thinking about development workflows/optimizations and really understanding where the problems/bottle-necks are is something I'm really passionate about. This was also a very high-level introduction to these ideas. There will be a follup post on commits and rebasing itself that will hopefully make things a bit easier for people to understand. If you have any questions, don't hesitate to reach out to me.
---
-title: "gitweb - a GitHub/GitLab alternative"
+title: gitweb - a GitHub/GitLab alternative
date: 2022-03-24T01:29:37-04:00
-tags: ["git", "development", "dev tools", "gitweb"]
+tags:
+ - git
+ - development
+ - dev tools
+ - gitweb
+ - publish
summary: Setting up your own git repo browser
+lastmod: 2024-12-10T19:50:13.254Z
---
+# gitweb - a GitHub/GitLab alternative
## Owning Your Digital Space
-Over the last year or so I've slowly pushed further and further into the idea of
-owning your digital space. Part of that has been re-evaluating all of the
-services online that I think of as "necessary". One of these such services has
-been GitHub.
-
-The more I dive into development processes the more I find that they are all
-centered around the idea that in order for you to be a "developer" it mostly
-requires that you buy into the idea of centralized forges like GitHub/GitLab.
-But these very ideas make it harder and harder to actually get work done. All
-development over the last few years has been about dealing with changes that
-GitHub has brought about. Don't get me wrong - GitHub really does have some
-wonderful services and they've done a lot for visibility and getting people
-involved in OS projects.
+
+Over the last year or so I've slowly pushed further and further into the idea of\
+owning your digital space. Part of that has been re-evaluating all of the\
+services online that I think of as "necessary". One of these such services has\
+been GitHub.
+
+The more I dive into development processes the more I find that they are all\
+centered around the idea that in order for you to be a "developer" it mostly\
+requires that you buy into the idea of centralized forges like GitHub/GitLab.\
+But these very ideas make it harder and harder to actually get work done. All\
+development over the last few years has been about dealing with changes that\
+GitHub has brought about. Don't get me wrong - GitHub really does have some\
+wonderful services and they've done a lot for visibility and getting people\
+involved in OS projects.
But they definitely shouldn't be the only game in town.
-In an attempt to take some control back from the major forges, I've been
+In an attempt to take some control back from the major forges, I've been\
experimenting with a small tool called gitweb.
## gitweb
-gitweb is a very simple tool - it allows you to browse all the git repositories
-within a specified folder. You simply install gitweb, point nginx over to it,
-and edit a single configuration file. You immediately get
-- A browser for all local git projects
-- A tree view for your repos with raw file previews
-- Commit history w/ colorized diffs
-- Snapshot downloads
-- RSS feed tracking commit history
-- Search (with regex) throughout your repos
-
-For personal projects, or even for small collaborative projects gitweb provides
+
+gitweb is a very simple tool - it allows you to browse all the git repositories\
+within a specified folder. You simply install gitweb, point nginx over to it,\
+and edit a single configuration file. You immediately get
+
+* A browser for all local git projects
+* A tree view for your repos with raw file previews
+* Commit history w/ colorized diffs
+* Snapshot downloads
+* RSS feed tracking commit history
+* Search (with regex) throughout your repos
+
+For personal projects, or even for small collaborative projects gitweb provides\
more than enough functionality.
-The two features that I think are missing from gitweb are Issue Tracking and
-Merge Requests. I don't think these are necessarily features that have any place
-in gitweb itself, but it means as a replacement for a centralized forge today,
+The two features that I think are missing from gitweb are Issue Tracking and\
+Merge Requests. I don't think these are necessarily features that have any place\
+in gitweb itself, but it means as a replacement for a centralized forge today,\
you probably need to rely on additional tooling.
-
## Setting up gitweb
-Actually setting up gitweb was surprisingly easy.
+
+Actually setting up gitweb was surprisingly easy.
### Installing gitweb
+
```bash
sudo apt install gitweb
```
### Configuring gitweb
-The gitweb configuration file is located at `/etc/gitweb.conf`. Installing
-gitweb automatically populates this file with some of the defaults. It's a very tiny
-file and honestly you don't need to touch most of it to get going. The only
+
+The gitweb configuration file is located at `/etc/gitweb.conf`. Installing\
+gitweb automatically populates this file with some of the defaults. It's a very tiny\
+file and honestly you don't need to touch most of it to get going. The only\
thing that's required is setting the `$projectroot`.
```inf
```
### Configuring nginx
-Most of the tutorials about getting gitweb going seem to be primarily apache
-related. I haven't personally used apache in close to 10 years now - mostly
-living in nginx land. Here's a very short snippet to get your nginx config going
+
+Most of the tutorials about getting gitweb going seem to be primarily apache\
+related. I haven't personally used apache in close to 10 years now - mostly\
+living in nginx land. Here's a very short snippet to get your nginx config going\
to actually serve gitweb.
```nginx
}
}
```
-All the paths included are the default locations of things gitweb configures.
-The entire block should work for you if you just change the `server_name`
+
+All the paths included are the default locations of things gitweb configures.\
+The entire block should work for you if you just change the `server_name`\
directive.
## Further Customizing
-Unfortunately not all the configuration options are specified in the
-configuration file that's generated. Reading the source will get you a list
-pretty quick but if you don't feel like it, here's a few other params I changed
+
+Unfortunately not all the configuration options are specified in the\
+configuration file that's generated. Reading the source will get you a list\
+pretty quick but if you don't feel like it, here's a few other params I changed\
up.
```inf
```
## Resources
-- Git Docs: https://git-scm.com/docs/gitweb.html
-- Gitweb Source: https://repo.or.cz/w/git.git/tree/HEAD:/gitweb/
-- My projects: https://git.xangelo.ca
+
+* Git Docs: https://git-scm.com/docs/gitweb.html
+* Gitweb Source: https://repo.or.cz/w/git.git/tree/HEAD:/gitweb/
+* My projects: https://git.xangelo.ca
+++ /dev/null
----
-title: "Moving to Hugo and GitHub Pages"
-Description: "Moving to Hugo"
-date: 2020-06-03T14:00:00-04:00
-draft: false
-tags: ["general", "hugo", "github"]
----
-
-## Goal
-
-I've been running my blog over on Ghost[^1] for a few years now. I really don't have a lot of issues with it, except for the fact that I have to manage the server itself. I've been running a small Digital Ocean VPS for a number of years where I host my blog. Ghost itself goes through a lot of updates and keeping the application up to date was starting to become a hassle.
-
-I was keen to move to a solution where I could
-
-- Write my posts in markdown
-- Not have to worry about hosting/management
-
-After looking at a few different solutions I ended up settling on a static site that was hosted through GitHub. The benefits of this was that
-
-- I could have all my content stored on GitHub.
-- I could utilize GitHub pages to serve the site with a custom domain name (with SSL)
-- I wouldn't need to keep everything up to date
-- I could re-purpose my VPS for other things
-
-
-
-Of course, not all static sites are created the same. I specifically settled on Hugo to power my static site.
-
-## Hugo
-Unlike more traditional blogging software (WordPress, Ghost, etc.) which saves your posts in a database, Hugo[^2] generates a bunch of static files (HTML) for each post. By doing this you're still able to have the more complex features (tagging, drafts, pagination, themeing, etc.) but because Hugo pre-processes everything, you're just left with a bunch of HTML files that represent your site. Once you're happy, you can just sync the generated files with your server. In my case, I'm simply able to `git add` and `git push` my content to GitHub.
-
-### Dynamic Blogging Platforms
-Lets look at Wordpress to contrast Hugo. WordPress is the most used blogging software in the world and has the following requirements:
-- PHP
-- MySQL
-
-The implicit requirement is also a server that supports these two. That means you are either relying on a 3rd party host that manages the server and just exposes the Wordpress interface... or you are managing that server yourself.
-
-Most blogging systems work this way - Some language requirement, and some database system for storing the data you create.
-
-However, there's lots of additional caveats that you aren't normally aware of, that tend to bite you later:
-- You need to run a web server, either Apache or Nginx
-- You need to keep the server itself up to date (security patches)
-- You need to keep Apache|Nginx/PHP/MySQL up to date (security patches)
-- You need to manage database backups (what happens if your server crashes)
-- You need to keep WordPress up to date.
-
-That's a heck of a lot of stuff for something that's supposed to be easy. And I just picked WordPress because it's popular. Almost every blogging system is the same way.
-
-Except for static site generators.
-
-### Static Site Generators
-Static Site Generators are a different way to approach blogging that merges more recent tooling with traditional delivery methods. That's just a fancy way of saying Static Site Generators give you some tooling to generate a bunch of HTML documents that represent your site.
-
-With Hugo, I install hugo locally.. write some markdown for my posts, and then use Hugo to generate the HTML for my site. With this, we bring our "management" layer down to:
-- Hugo
-- Web server (nginx/apache)
-- The hosting server
-
-Hugo runs locally, so the surface of attack is pretty minimal. There's nothing really wrong with running an outdated version of Hugo except you won't get the most recent features/bug fixes. But there's no security implications of doing so.
-
-Since the output from Static Site Generators is just .. static content it can be hosted anywhere. You can use a hosting provider, a VPS, S3+CloudFront, or.. GitHub Pages!
-
-## GitHub Pages
-GitHub pages is a feature of GitHub that is available to all accounts, regardless of subscription. GitHub pages allows you to serve static content from any repository you'd like. They have some rules around how you need to organize your code and how to configure your site (project vs. main) but it's all relatively straight forward.
-
-I haven't really hosted much on GitHub pages, but it's very easy - They also have a great walkthrough page[^3] that shows you how to configure the kind of Static Site you want to host.
-
-
-
-## Project Configuration
-
-I'm not going to go into too much detail, because both Hugo and GitHub have very good introductions, but this should be enough to get a site running.
-
-
-
-### Installing Hugo
-
-Installing Hugo can be done from a terminal in your chosen operating system. There are alternate installation instructions available if you don't have brew installed or if you don't have a mac[^4].
-
-```bash
-brew install hugo
-```
-
-
-
-### Setup Hugo
-
-Open up your terminal, and navigate to the place you want to store your website. Then you can run the following command:
-
-```bash
-hugo new site mysite
-```
-
-This creates a new folder called `mysite` that contains the raw files for your static site. At this point you don't have anything generated just the files that will eventually be compiled into your static site.
-
-You can install a theme by doing the following:
-
-```bash
-git init
-git submodule add https://github.com/budparr/gohugo-theme-ananke.git themes/ananke
-```
-
-You can actually choose whatever theme you would like from https://themes.gohugo.io/ [^5]and just replace the `git submodule` line with the instructions from your theme.
-
-
-
-Line 1 creates a git repository in the `mysite` directory. We want this because themes in Hugo are installed via `git submodule` which clones a particular repository into the current one. It also allows you to save the raw site (before static generation) to git!
-
-
-
-Now it's time to head over to your `config.toml` file and set up the defaults for your website. My config file looks like this. You only really need the configurations on the first 9 lines. The rest are just further configurations[^6] that I've made.
-
-```toml
-baseURL = "https://xangelo.ca"
-theme = "plain"
-title = "Xangelo.ca"
-author = "Angelo R"
-copyright = "Copyright © 2011 - 2020"
-paginate = 15
-languageCode = "en-us"
-enableInlineShortcodes = true
-footnoteReturnLinkContents = "^"
-
-[params]
-subtitle = "Technical musings and other tidbits"
-
-[markup]
-[markup.highlight]
-lineNos = true
-lineNumbersInTable = true
-style = "vs"
-tabWidth = 2
-[markup.tableOfContents]
-endLevel = 3
-ordered = true
-startLevel = 2
-```
-
-
-
-
-
-### Create and test!
-
-Hugo ships with a built in server that allows you to preview what your site will end up looking like, you can start it up with:
-
-```bash
-hugo server -w
-```
-
-This creates the server and the `-w` flag sets it to "watch". Any changes that are made will automatically reload the server and also reload the site. You can visit this site by navigating to `https://localhost:1313` in your browser.
-
-You can create a new post by using `hugo new posts/welcome.md`. This creates a new Markdown file in the `posts/` directory.
-
-You can go ahead and edit that file - saving it will cause Hugo to live reload the website.
-
-
-
-### Publish your site!
-
-Once you're happy with your stuff, you can turn off the live-reloading server and run the `hugo` command. This parses all your information and generates the static files for your website. It will then put all this content in the `public/` directory. If you followed the instructions from earlier around setting up GitHub Pages, you can `cd` into the `public/` directory and set it up as another git repository.
-
-then running `git push `will publish your static files to whatever GitHub repository you configured to host the site.
-
-
-
-## Wrapping Up
-
-I haven't really dug too much into setting up GitHub pages because it's mostly clicking around the interface and the instructions for that are kept up to date [^3]. I highly recommend GitHub pages because of the ease of setting up
-
-- Custom domain names (if you have your own domain it's trivial to point it to your GitHub pages site)
-- Custom domain name SSL certificafte. Most places only allow you custom SSL certs if you use a subdomain, but GitHub lets you use a custom domain name.
-
-
-
-I also highly suggest you spend some time going through the themes[^5] on Hugo to find one that you really like. There are so many and it's so easy to make new themes that you'll definitely find one you like.
-
-
-
-## Footnotes
-
-[^1]: Ghost, a node.js based blogging engine: https://ghost.org/
-[^2]: Hugo, static site generatror: https://gohugo.io
-[^3]: GitHub Pages setup: https://pages.github.com/
-[^4]: Hugo installation: https://gohugo.io/getting-started/installing
-[^5]: Hugo themes: https://themes.gohugo.io/
-[^6]: Hugo configuration options: https://gohugo.io/getting-started/configuration-markup
\ No newline at end of file
+++ /dev/null
----
-title: "My i3wm Setup"
-date: 2022-03-18T12:16:36-04:00
-draft: true
----
-
-I've recently made the switch from the stock Ubuntu setup over to i3wm. I've
-used xmonad for many years before this, but always found it a bit harder than I
-would have liked to customize it.
-
-Most of what i3 does out of the box works perfectly for me. The only thing I
-didn't like was the fact that navigation was shifted from `hjkl` to `jkl;`. I'm
-also not a big fan of dmenu. Honestly I'm not totally sure why, but I've always
-struggled getting it to do what I wanted.
-
-With i3 I made a few small config changes, and swapped out the launcher and so
-far things have been quite excellent. I'm just posting this here so that I
-don't forget my configs.
-
-##
-
-
-## My Config
-As a vim user, I'm much more comfortable with the default vim bindings, than the "1 ket to the right" bindings that i3 tends to use.
+++ /dev/null
----
-title: "Mongo vs. DynamoDB"
-date: 2022-09-17T02:12:13-04:00
-draft: true
-tags: ["database", "mongo", "documentdb"]
----
-
-## Abstract
-Whenever you get into the conversation of NoSQL, and specifically "Document based" systems, you inevitably seem to end up on either DynamoDB or Mongo. The truth is while DynamoDB is a wonderful tool, Mongo should probably be your first stop.
-
-## Mongo vs. DynamoDB
-Mongo and DynamoDB are two document storage databases in the NoSQL family. Where mongo took the more traditional "server" route, DynamoDB opted for "serverless". As a result, most of the distinction between the two comes from this divide.
-### The infrastructure and Scaling
-### The UX
-### Support
+++ /dev/null
----
-title: "Designing On-Call"
-date: 2022-09-21T22:36:23-04:00
-tags: ["engineering", "engineering org", "engineering processes"]
-summary: How to implement a successful on-call process for the first time
----
-On-call is one of those things that all developers end up doing at some point. My goal isn't to discuss the merit of on-call, but rather what the point of on-call is and how to go about designing what “on-call” means at your company. I'm going to start at the very beginning because chances are you're already doing it wrong. I should also note that I'm looking at this specifically from a SaaS point of view.
-
-## Why do devs go on-call?
-The point of on-call is simple: People use your app 24/7 relative to you. They may be using it from Italy or Australia or Thailand while you're tucked up in bed in Wisconsin. But for them, it's working hours and they expect your site to be available. But, of course, that's not always possible. Things happen that will inevitably take your project down. In these situations "on-call" is vital for several reasons: The least of which is solving the problem.
-
-Normally you go on-call because your founders or manager or a customer noticed that your service was offline when it wasn't supposed to be. People start complaining, it makes some rounds internally, and then eventually ends up at the desks of whoever deployed it. Often that ends up being some kind of "infrastructure" team (You probably call them DevOps or SREs - why that's wrong is an issue for another time). These folk hear about it, fix the problem, and then go about setting up some alerts to let them know when the issue happens again. Bam, you've just enabled on-call without any of the good parts. You can all but guarantee that your on-call team is going to end up angry and upset about on-call which will manifest in the worst ways possible.
-
-## Let's walk through the standard on-call process
-Let's walk through a very high-level on-call process and then we can break it down.
-
-1. Something breaks
-2. You find out that something is broken
-3. You bash at your keyboard until it works again
-4. You put in some metrics so that if "thing" happens again, you'll know
-
-It's wrong.
-
-Ok, it's not wrong - it's just a subset of what "on-call" really is.
-
-See the thing that you're REALLY missing is that nowhere in this process have you actually figured out what's going on. Sure you solved a symptom - maybe that symptom is synonymous with the cause. But you don't know, because your process doesn't encompass that.
-
-To have a good on-call process you have to understand a fundamental truth of on-call: on-call isn't just about engineering. on-call is the culmination of a bunch of different business, product, and culture decisions.
-
-Here's the truth of each step in your process
-
-### Something breaks
-#### Something
-The first thing we need to define is what "something" is. The easiest thing to do is to take a look at your code repositories. If you’ve broken things down into services, you likely have one repo/service. That’s an easy way to define the boundaries of “something”. But it’s also probably wrong.
-
-If you have a monolithic codebase in a single repository this is a much harder step for you, and you’ll probably do a better job.
-
-It’s very easy to define “something” by engineering components - but this isn’t always answering “what” broke, but “how” and “why”. A user doesn’t care that your lambda was opening too many connections to RDS causing a spike in memory which caused your API to lag. They care that they weren’t able to send a cat picture.
-
-When defining your “something” start with user flows. Look at your application and define “core” actions that your user can take. From there, simplify until you have a handful of actions that you want a user to be able to do All the Time.
-
-#### Broke
-Defining the "something" is relatively easy compared to defining "broken". We know a thing is broken if it's not starting, but:
-
-- What if it works but 1% of the requests are resulting in an error?
-- What if it works but 1% of the time it crashes and restarts?
-- What if it works but is very slow?
-- What if it works, is not slow, and doesn't crash, but your API docs don't match what the endpoint is returning?
-- What if it works, but your database is being crushed by a sudden increase in traffic?
-
-Defining our “something” first is important because it helps us to set bounds on what we consider “broke” and what is “degraded” and what is “fine”.
-
-Perhaps we don't care if 1% of requests fail.. maybe we don't care if 0.1% of requests fail. Defining these thresholds is tough because it requires realistically looking at our application and deciding what is "acceptable" and what is not. Sometimes we have to make difficult choices here. Maybe we want 0.1% to be considered broken.. but actually, 1% is broken. This is where we impact your actual application.
-
-Deciding 0.1% of errors is "broken" where your application is currently sitting at 1% may seem like a good thing. 0.1% is where we want to be and so letting devs know when that isn't the case is good. We can work toward 0.1%. But this involves much larger product considerations.
-
-- When these alerts happen overnight - are devs tasked with a complete resolution during work hours?
-- Does that mean feature work will suffer?
-- How will you buffer your sprint to make time for these interruptions?
-- Do you have the ability to buffer your sprint given product launch dates?
-- What happens when a dev is up all night dealing with issues, do they take the next day off?
-- Do you offload the task to a different team (Ops/Infra/SRE) since feature work is so important?
-- What happens when those teams get burnt out and leave?
-
-On the flip side setting your alerts at 1% is accepting that this is the current state.. but now it becomes a decision on whether or not a 0.1% error rate is more important than the next feature you're supposed to get out.
-
-Defining the state of "broken" is hard because it is the intersection of the current and future state of our application, while simultaneously being the fulcrum that will help tip the scales between "acceptable" and "unacceptable".
-
-There's no right answer here - just an answer for your team, for right now. Whatever you decide today, the most important thing is that you re-evaluate it and allow those that are on-call to provide feedback on if this was a reasonable level of "broken".
-
-#### Service Level Objectives
-By defining what broken means you've now set a baseline stat for your systems. You've defined your first Service Level Objective (SLO). Yay!
-
-In the nutshell, SLOs form the baseline for your Service Level Agreements (SLA) with customers. Your SLA can't be better than your SLO or you're going to have to sacrifice on feature development to bring them up to speed. Now we can talk about all kinds of marketing tricks you can do (mainly playing with the measurement period) but the truth is if your SLO defines a particular flow as "broken" it sets the floor for your SLA. You can only promise less than that until you invest in improving that SLO.
-
-Let's move on to the 2<sup>nd</sup> step.
-
-### You find out that something is broken
-#### What's on first?
-Ok. something is definitely broken. How do you find out?
-
-This is probably the most technically straightforward piece, but the one that has the most impact on people.
-
-When something is broken, an alert is generated you get notified.
-
-The first step is figuring out how you get data out of your system into a place that can perform some analysis and generate these alerts. The two big products here are Datadog and NewRelic. These are all-in-one solutions that allow you to instrument your application and gather metrics+logs+events and put them somewhere you can visualize them. These systems also allow you to create “alerts” that get generated based on the data you’ve been feeding it.
-
-Given that you now know what you consider critical user flows your goal is to go through your application and ensure that everything that supports these flows is properly instrumented. This is where we start looking at the different infrastructure components. This is where we learn how/why our system broke. This is an important step - it's easy to think that devs should just "go through and instrument everything", but that makes things harder to reason about. You're increasing your "noise" and when you're looking at a Signal vs. Noise ratio, you want to keep your noise to a minimum.
-
-Take a look at your SLOs -> and then define your Service Level Indicators (SLI).
-
-#### Service Level Indicators
-These are the metrics that you can look at to identify that you're meeting your SLOs. They're often aggregate metrics. If your objective is a 1% error rate, your SLI will likely ensure that you are getting an accurate count of errors vs. successful requests across your system. That will involve you going through various systems and making sure that your code base is sending appropriate logs. It'll also be going through your monitoring tooling and creating dashboards/alerts so you can view your SLI and all the subcomponents that make up your SLI easily.
-
-Now that we have these metrics, and we know what we consider “broken” we can tune our alarms. When one of these components is in a state that impacts our user flow - thats an alert.
-
-That alert is going to be sent to whomever is on-call.
-
-#### Who’s on first?
-Everyone in Engineering that is responsible for writing code that could end up affecting a user.
-
-Bam. Easiest decision ever.
-
-But you'll likely face some pushback from teams that don't normally see themselves as "needing to be in rotation". Doesn't matter - at some point you have to make a decision and this is one hill you must die on as a manager.
-
-Traditionally on-call was reserved for infrastructure/devops/sre groups and developers weren't required to be on-call. This is a silly separation of concerns. The people who wrote the code that likely resulted in user impacts are often not from your infrastructure/platform team -> they're likely from the application team. This should be your first tier of on-call when thinking about SLOs from the product.
-
-Often you'll end up with a rotation of engineers, probably daily, with multiple tiers.
-
-1. The primary: They're the first ones to get alerted about an issue
-2. The secondary: If alerts to the primary don't get acknowledged, it goes to this person
-3. The backup: If primary AND secondary are not responding -> this person is here.
-
-You may decide to have different schedules for weekdays/weekends but in the end the goal is to cycle through all developers without making it feel like anyones "job" is to be on-call.
-
-Now that you have your schedule, you must know that
-1. Everyone should feel empowered to schedule alternates if they're busy. If I'm scheduled tomorrow night and something comes up - I should feel comfortable addressing the engineering team as a whole and asking someone to trade with me. Likewise, you should feel comfortable saying no.
-1. No one should treat on-call as being stuck at the computer. This does end up personal preference for engineers, but if you're on-call you should be able to take your laptop/phone with you where you go. If your company is not technically ready for this, then you better be compensating your employees well for tying them to their homes outside of work hours.
-
-### You bash at your keyboard until it works again
-Alright. It's 3 am, something broke, you were paged. You hop on your computer, bleary-eyed and upset because it's chillier than you expected because you forgot your pants in the rush to deal with this situation.
-
-Relax.
-
-Your job is not to solve the problem - your job is to ensure that your application is performing as expected.
-
-Solving the problem is a task that is best left to those on full nights of sleep. When it's 3 am and you finally manage to log into your laptop, "solving" the problem should be the furthest thing from your mind. The goal is to mitigate the outage and restore acceptable functionality based on your SLOs.
-
-In some cases simply restarting a process is enough to get things back to working. Sometimes you may need to upsize a database.
-
-What you should NOT be required to do, is:
-- figure out where your application code is crashing
-- optimize that weird nested join that you're supposed to tackle next sprint
-
-Your job at 3 am is problem mitigation. Maybe nothing can be done except throw on a status message so that users know what's happening.
-
-Perhaps this alert is fine, this level of error is acceptable.. so you can just adjust your alerting and call it a night.
-
-Don't attempt to be smart at 3 am because when you get paged at 6 am you'll be much worse off trying to decipher what you did.
-
-This is often where we run into issues. Engineers like to solve problems. When we get an alert we dig in. We want to fully understand what's happening because we can easily over/under complicate an issue. Oh is CPU on the DB at 100%? Do we need to just upsize the database? Are we locking something? Is there some periodic system running?
-
-Sometimes the problem is obvious and fixing the symptom is the same as fixing the problem. It's much harder when the problem isn't obvious. Rememeber that while you're "digging in" this is impacting users - otherwise it wouldn't be alerting you right? As unnatural and silly as it seems, at 3 am -> solve the symptom for immediate relief. Then when you're not stressed, solve the problem.
-
-Again: Solving the problem at 3 am is not your job.
-
-Once you fix the symptom causing the alert and verify that things are working again - leave notes in a public place. Explain what you did and why. The why is the most important part. This is that when you or whomever else logs in the following day and looks over what happened "on-call" you can easily decipher your thought process and areas that you should begin looking to resolve the underlying problem.
-
-### You put in some metrics so that if "thing" happens again, you'll know
-This is the easiest part: Once you identify the "root cause" of your problem, get some metrics/logs in place so that you know. Don't forget to include the metric in any SLI calculations that you might need.
-
-## Compensation
-Your propensity for compensation is entirely dependent on your org and the legalities of where you reside. In some situations you will have to pay engineers overtime for on-call. In some you can get away it.
-
-Don't be a jerk.
-
-Whatever you decide to do understand that your team will leave if they think they're getting a bad deal. on-call is requesting something of your team outside of normal working hours. It doesn't matter if you put it in a contract, it doesn't matter if it was on the job description. You are requesting something of your team that is above their work day.
-
-If an on-call night was rough, let them take the following day off. If they've been having a bad week personally, remind them that its' ok if they want to move their on-call.
-
-## Your On-Call Process
-Finally, you've ready all that and are ready to implement a real "on-call" process. Here's what it should look like:
-
-1. Identify your SLOs
-1. Identify your SLIs
-1. Work to track metrics/logs that relate to your SLIs
-1. Create an on-call rotation
-1. Something breaks
-2. You find out that something is broken
-3. You mitigate the problem
-4. You put in some metrics so that if "thing" happens again, you'll know
-1. You revisit the alert the morning after to figure out root cause
-1. You resolve the actual problem.
-
-
-## Your On-Call Process in Practice
-The truth is, there's a huge difference from the "ideal" process and what you end up with in practice. So much of this is up to the people involved and the culture of the workplace. Chances are you already have some of this implemented and it kinda works for you mostly.
-
-That's ok.
-
-The goal here is that you understand ideal, and you understand where you are. And now you have the information to be able to make trade-offs. Expending your entire development capital on the first 3 items on this process before getting to the next is silly.
-
-Instead this process is more of a "Flow". If you understand ther flow, you can understand how each thing feeds into the next. When you know that you can better break down your approach.
-
-You probably have Service Level Agreements (SLAs) with your customers. And you likely have some rough SLOs/SLIs that track that. Solidify them. Make them available to the team on demand and make sure that they're accurate.
-
-The on-call process is a larger movement within your engineering organization that pushes the whole team toward excellence: Both technical in being able to understand the inner-workings of your application and all the components, but also of your product. If your SLAs are at 99% uptime, but you're sitting at 99.9% uptime -> that's an excellent way to upsell. Likewise, if your uptime is actually at 98% you now know exactly the area your team will need to focus on next.
-
-## Final Thoughts
-
-Understand that no matter your best efforts, pages will be missed. Bugs will go undetected. Users will be upset. But these are short-term issues. You can fix bugs. You can explain to users. You can tell your devs that this push is temporary.
-
-Do understand the difference between pages being missed and pages being ignored - call your team out on it however you need. Missing a page isn't just letting customers down, it's letting your team down. If it happens frequently enough the team will either think that the alert isn't important or that an individual is above the on-call rotation. It'll foster all kinds of interpersonal problems.
-
-Don't treat on-call as the solution to your stability and reliability issues. Reacting is not being proactive. When alerts happen overnight, ensure that there is SOME mechanism to address the root cause reasonably.
-
-If you are running into issues with this, reach out - I'd love to chat. I've been doing this for a long time and I love to help out. I set aside a few hours a week to donate to help companies and individuals make sense of these kinds of larger scale engineering problems. My email is at the bottom of every page.
+++ /dev/null
----
-title: Organizing Organizations
-summary: A walk through various organizational structures
-tags: ["engineering org"]
-draft: true
-date: 2024-06-01T23:29:09.966-05:00
----
-
-> A lot of what I'll be waxing poetic on will be focused around the tech industry, and smaller companies in general. But the truth is all small companies think they're large, and large companies get so large that they have small companies within them. The real truth then is that you are stuck in a never ending cycle - a wheel of workplace - to borrow and then bastardize from Buddhism. But rather than be disheartened, the repetition is required because the collective you is always changing and the new change is never quite the same because of it.
-
-I was lucky that early in my career I got the chance to really dig into organizational structures. It happened at a time when I didn't really have a lot of obligations outside of work. The startup that I had co-founded had been doing well, I had naively thought, and so I could devote the little free time I had to figure out how to scale this inevitable behemoth of a company.
-
-## The Tribes model
-A few years after that crashed and burned, I got the opportunity to put my knowledge into practice. Spotify had just published their "tribes" model - which is a matrix org with jazz hands. It took the SaaS world by storm. Cross functional teams centered around a particular system. In spotify's world, for example, search was a "tribe". There was a group of people devoted to just search - they could really focus on it. The tribes model was a "matrix" because it offloaded your functional growth among a lateral management line.
-
-
-In this example, each line of business has its own team needs. There could be any number of these individuals organized around the business need. This allowed them to iterate and become SMEs in their area of focus. This is hugely important to the line of business. Having a long running team that you can take through all stages (Forming, Storming, Norming, Performing) meant that you could really optimize that line of business.
-
-The problem with this, is the exact same as any matrix organization. The person that owns your line of business may have very little understanding of what you do actually do. Imagine a team focused around search. The lead for the team could really be focused on the user experience.. but there's a deep technical aspect to real-time search. If they don't understand it, it can be hard to gauge how well individuals are working.
-
-Another inherent issue is isolation. Front end engineers within a line of business do not get to interact with other front end engineers. Things become inefficient as multiple teams work through similar classes of problems and each of them come away with a slightly different method that works for them.
-
-Traditionally in a matrix org you report to two individuals - your direct manger, who ran the particular team. And then a functional lead, who was the expert in what you did. These two often clash as the functional lead rarely has control over what you do day-to-day, and then business lead has no interest in larger scale functional changes.
-
-This is true of all matrix organizations.
-
-The truth is what Spotify had found is that by providing purpose to a diverse group you could achieve great things.
-
-## It's always Maslow
-
-
-
-This Maslow guy was no slouch it turns out. In the context of a job, they are ideally taking care of the first two levels (Bottom up). They hope that you feel a sense of connection and belonging because then you are "aligned" with the companies vision. Once you have those things, we introduce a performance review and bam - now you know about yourself and where you stand. You can understand your unique skillset and the areas you still need to grow. It helps you identify those areas in others and fosters a natural respect within the company. Everyone knows that one guy who you go to when things explode.
-
-When viewed through this lens, all organizational changes attempt to change three things:
-
- - Your sense of connection to your team
- - Your self esteem and confidence within the team
- - What your team is responsible for.
-
-## Matrix: Reloaded
-
-Armed with this knowledge, we can revisit the tribes model with a little more insight. All re-orgs come with the risk of severing your sense of connection to your team. The tribes model hopes that by providing a shared purpose, the sense of connection to your team will grow.
-
-A standard growth cycle of a team through the Forming, Storming, Norming, and Performance phases takes care of the self-esteem portion. Coupled with performance reviews. And your purpose has already been given to you.
-
-But what actually happens is that it severs your connections with other individuals within your discipline. You do form closer bonds with your team, but you lose sight of the larger actions being taken upon by your function and so your career can stagnate, without looking like it. What can (and does) end up happening is that your team is focused on your immediate manager and the goals of the Business Unit. Often the thing that suffers is your growth within your function. Backend engineers don't really progress in skillset from jr -> mid -> sr, but instead get stuck in a state where they aren't aware they may be falling behind. The truth is there are known knowns, and there are known unknowns. But there are also unknown unknowns (I don't know why I'm obsessed with this phrasing by Rumsfeld). And the scope of your unknown unknowns is limited by your exposure to your function.
-
-Some people can find this growth in communities outside work. Some people can't. It's not wrong one way or another - but it is important that re-orgs that jump to matrix's understand this. You are trading overall growth for specific growth. Now that also isn't wrong - infact, I'd argue this is precisely what a PhD is. The important thing is that we make these decisions explicitly. The only things that bite you in the butt are implicit decisions.
-
-## The two-to-three Legged Stool
-
-A nother model that is favoured because of the way it implicitly arises is the two-to-three legged stool. In this system you have 2 or 3 major divisions in your team:
-
- - Engineering, lead by an engineering manager
- - Product Manager, lead by some kind of VP of Product
- - Designer, lead by some kind of VP of Design
-
-Occasionally the designer is folded into the product org under the guise (maybe appropriately) of "user experience".
-
-This model does have some benefits, but the truth is this is model is never an explicit decision - it arises from natural growth. You have some engineers and a Product Manager working together. As the team grows you add more engineers, eventually you have enough engineers that a "tech lead" isn't enough -> so one of them becomes an Engineering Manager. Bobs your uncle. 2 legged stool.
-
-I dislike this model because it splits things unevenly. The PM is being judged on their ability to deliver, but that is being controlled by a different entity - the engineering manager. Often the two have to work together and you get plenty of arguments like "we are supposed to get a sprint to work on tech debt" or "we said 20% of our time could be focused on these devex tasks."
-
-I find this often ends up happening when you have a large dichotomy higher up in the company that is resolved through collaboration. What happens is that you have two individuals who work well together - one on product, and one on engineering. Each one is comfortable staying out of the others domain and they've hit some kind of equilibrium on how they work together. They're given full control of the down-stream org and they implicitly replicate their relationship without any of the history.
-
-It's not that I don't like this model, it's that I don't like the implicit nature of this model. When it works, it works amazingly well. Engineering managers can carve out time for their team to not just deliver on the product, but also make time to improve the developer experience. But it separates the workload in "things we have to do" and "things we'd like to do" - which is not a good place to be in.
-
-## Random Thoughts
- - I often send around rough drafts of these and one of the questions I got was "what about the STO model". The truth is the STO model is nothing more than a matrix org where the business unit leader is given a few more powers. Depending on how high up the STO is defined really just changes who reports to them - but the org remains matrix'd.
- - At around the same time the [OKR Video](https://www.youtube.com/watch?v=mJB83EZtAjc) at Google started to make the rounds and they seemed to go hand in hand. Taking your cross functional team, and giving them a single area of focus, allows for specialization. The long running team facilitated experimentation which fits perfectly into the realm of OKRs. This worked even better when viewed under the umbrella of agile.
-
+++ /dev/null
----
-title: Now Powered by Outlines
-tags: ["outliner", "blog", "project"]
-date: 2023-05-21T23:29:45.732-04:00
-summary: An update to how I publish new posts
----
-
-
-One of the things that I do every so often is completely re-write the backend of my blog. I've mostly hit upon a UI that I like, but I've swapped out the backend over the years between various custom iterations, wordpress, ghost, and now finally Hugo. This time, I've swapped out how I write my blog posts - but kept everything else the same.
-
-The current system allows me to write markdown in vim. I'm normally running `hugo serve -w` at the same time so I can watch the rendered version of what I'm doing as I go. It's sort of like a hacked-together live preview. It works well enough.
-
-However, for the last 10 years (maybe more?) I've been a huge fan of outliners. I original started with various projects by [Dave Winer](https://scripting.com) and I used almost everything he's written around them for a number of years. I've also tried tooling like [Workflowy](https://workflowy.com) and almost every other infinite-bullet-list tool that came after them. They were all.. fine? I had no real problems with them except that they never really stuck around for very long. They were in a tab in my browser, and my browser has like 100 tabs open at any given time.
-
-For the last 6 months or so, however, I've been working on my own outliner. It started as an in-browser tool... and I quickly moved it to an offline-first desktop app via [Tauri](https://tauri.app). Having it offline first meant a few big things.
-
-1. All the syncing tools just work. Dropbox, rsync, backblaze, s3 as a filesystem. Whatever. It works. All nodes in the outliner are individual json files on your computer. There's an "outline" file that stitches the nodes together into a tree. <br>
-
-2. Building desktop first allowed me to bypass the need for user accounts and passwords. It allowed me to skip out on the complexities of providing reliable encrypted storage to users. I don't need to run a collection of servers and databases and object stores to power this thing.
-
-Since I use this tool across operating systems all day long it slowly ended up being where most of my thoughts for blog ideas end up. And so what I hope to be the final feature of this project was realized: I'd love to be able to write my whole blog post in here, and then hit a button to have it publish to my hugo deploy.
-
-This first iteration uses a lot of hard-coded stuff.. and I'll probably take some time to iron out some of the edge cases around rendering.. but it honestly came together pretty quickly. Since every node in the outliner is markdown it was trivial to put it together. As of right now, I can write my blog post in my outliner, press `shift+p` and have it write out a markdown file to my local hugo instance.
-
-For now, I do some manual reviewing before officially publishing it. For now there's a few more usability things that need to be added like
-
-- differentiating which posts are published vs. un-published
-
-- being able to 'unpublish' a node
-
-But honestly? I'm kind of enjoying this right now.
-
-
-
-
-
-
-
+++ /dev/null
----
-title: "Running a Git Server"
-date: 2022-03-30T11:08:53-04:00
-draft: true
----
-
-# Running your own Git server
-Following this theme of owning your digital space I wanted to do a quick post on what it takes to run your own git server. There are a bunch of services out there like that will host your git repositories for free (I personally recommend [sourcehut](https://sourcehut.org/)), but you can get a most of what they offer for free with [gitweb](https://xangelo.ca/posts/gitweb/).
-
-## Why would you want to do this?
-Honestly it's not as complicated as you would think to run your own git server and get almost all the same functionality from popular forges. Running your own git server doesn't mean you can't use something like GitHub or GitLab - it's not a one or the other choice. You can host all your repos on your own VPS while still pushing to GitHub/GitLab to ensure that you can still take advantage of their social networks.
-
-For me the biggest precursor to hosting my own repos was just to get more familiar with git. I've been using git for a number of years, but it was only in the last year or so that I really started digging into all the functionality that it offers.
-
-In addition to that there's a huge difference in complexity between running your own server, and running a service for millions of others to consume. As GitHub grows (both in size and in feature set) the likelihood of any one piece of it being down also grows. If we look at the [Incident History](https://www.githubstatus.com/history) of GitHub from Jan 2022 to March 2022 we can see that some piece of it has been dozens of incidents and outages. Using git allows you to bypass a lot of those problems, but centralizing on any hosted forge will always come with these inherent risks. It's much better to use hosted forges as secondary locations and just host your own git repos.
-
-## What do you need?
-1. Server - You need somewhere to host your git server. Personally I use [Digital Ocean]() but you are free to use anything that gives you ssh access and an IP address.
-1. You'll need root access to that server
-
-That's it. Those are the minimum requirements to actually host your git repos elsewhere. Everything else on top of that is just extra functionality and we'll talk about those later.
-
-### A Server
-The easiest thing to do would be to spin up a brand new server on a cloud platform like [DigitalOcean](https://digitalocean.com). It costs approx 5$ CAD and is probably one of the cheaper shared hosts you can find. You don't need anything fancy because the git repos actually do nothing the majority of the time. There's only activity on them when you push/pull.
-
-If you spin up a droplet (server) in Digital Ocean it will provide you the ability to set up your root credentials. Personally I'd suggest using an SSH key! It's a heck of a lot easier than typing in your password every time you try and connect.
-
-#### ssh-keygen
+++ /dev/null
----
-title: Simple Redis Job Queue
-summary: A naive redis job queue implementation
-tags: ["redis", "architecture"]
-date: 2023-05-25T00:15:53.362-04:00
----
-
-
-A common pattern in most software projects is the "queue". You'll throw some stuff on there and eventually get around to sorting it all out. In some cases, you may not even really care about that.
-
-I recent ran into a situation in node.js where I needed to offload some CPU intensive out of the main http request handling process. Originally I thought about just forking the process, but I due to the fact that this endpoint is public facing (albeit authenticated and paid for), I was wary. Instead I figured I'd rope in a quick job queue.
-
-### Caveats:
-
-- The jobs have an easy "deduplication key". An ID that is the same given the same inputs.<br>
-
-- If we don't complete a job for any reason, that's fine, another one will likely show up again in a few minutes
-
-We were also using redis so here's a real simple pattern that can handle this kind of workload.
-
-1. Whenever we get a new job we calculate its deduplication ID. We'll call this the `jobId` moving forward.
-
-1. We call `set job:[jobId] value NX EX 10`. This sets the key `job:[jobId]` to some value we provide only if the key does not already exist. It also sets a 10s expiration time. We're using `NX` to provide some additional deduplication and we're using `EX` to ensure that if for some reason our job doesn't complete in 10s.. just allow another job to be inserted with the same key. You can tune `EX` as necessary, or leave it off entirely if you never want this job re-processed again until you flush your cache.
-
-1. We use `rpush [queueName] [jobId]` to add the job Id to the list of jobs that need to be processed.
-
-1. If we have any additional information about the job we want to pass on, we can use `hset jobDetails:[jobId] k1 v1 k2 v2...` and store it in there.
-
-Your "worker" is just a process that runs `lrange [queueName] 0 0`. That will retrieve the oldest `jobId` in your queue. You can grab any further information from the `jobDetails:[jobId]` hash set and do whatever work you need. When you're done you can call `ltrim [queueName] 0 0` which will remove the job from the queue.
-
-An interesting fact about this setup is that all your calls are `O(1)` and you can pipeline the initial `set`/`rpush`/`hset` calls so that things are even faster.
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
---
-title: "The Disk Is Slow"
+title: The Disk Is Slow
date: 2020-04-29T16:21:29-04:00
-draft: false
-tags: ["explanation"]
+tags:
+ - explanation
+ - publish
+summary: A short dive into an old truth
+lastmod: 2024-12-12T05:31:16.755Z
---
-
-"The disk is slow" is one of those things that most programmers take for granted. Yes it is slow given the speed of other components. But rarely have programmers taken the time to dig into WHY the disk is slow and what that actually means. Yet, doing so can lead us down some interesting rabbit holes.
+"The disk is slow" is one of those things that most programmers take for granted. Yes it is slow given the speed of other components. But rarely have programmers taken the time to dig into WHY the disk is slow and what that actually means. Yet, doing so can lead us down some interesting rabbit holes.
-## What is slow?
-For a while now the speed of a hard-drive was measured in RPM or Revolutions Per Minute. This is an indication of how quickly the disk can spin. It is common now-days to see drives advertising 7,200 rpm, or 10,000 rpm or even 15,000 rpm. How fancy.
-Now, the disk itself is split into a couple major components:
-- The disks that data is stored on:
-- The read/write head
+## What is slow?
+
+For a while now the speed of a hard-drive was measured in RPM or Revolutions Per Minute. This is an indication of how quickly the disk can spin. It is common now-days to see drives advertising 7,200 rpm, or 10,000 rpm or even 15,000 rpm. How fancy.
+
+Now, the disk itself is split into a couple major components:
+
+* The disks that data is stored on:
+* The read/write head
These disks are where the data is actually stored, and when you see a number like "7,200 rpm" what you are seeing is how quickly these disks can spin. In a simplified manner, what happens when you "write something to disk", is that the disk spins to an empty point, and the head beings to write. Likewise, when you "read" data from the disk, it spins to a designated point (the "start" of the data) and the head begins to "read" the data until it is done.
Let's walk through a theoretical "disk". Your "disk" can hold 8 units (00000000) of storage. We are going to perform a few actions on this disk.
+
```
You write 'a' twice - aa000000
You write 'b' 3 times - aabbb000
You delete the two 'a' - 00bbb000
You write 'c' 4 to,es - ccbbbcc0
```
+
See your drive is smart enough to know that, even though there isn't enough "contiguous" space, there is still enough space scattered around on the drive to store your 4 units of 'c'. What happens is that your drive will spin to a free location, and let you start writing. When you run out of contiguous space, it will spin to a new location. This results in your data actually spread out all over your drive instead of next to each other. This is a good thing, of course. It means that you can use all the space on your drive without worrying about WHERE things are stored.
But, it also means that instead of the disk spinning just ONCE to get to the start of your data, it actually needs to spin twice.
As we grow our disk size from "8 units" to hundreds of gigabytes as most modern drives today have, we run into a problem - there is no guarantee that the data we need will be next to each other. In actuality, there is a high probability that we will need to keep jumping about on the disk to be able to read ALL the data we want. Our data ends up "fragmented".
## Data Fragmentation
+
The result of all this fragmentation, is that things just get slower over time. Unfortunately, the eventual degradation of data storage efficiencies is never attributed to the hard-drive because users don't actually USE the hard-drive directly - they go through the OS which is supposed to manage these things. As a result, the eventual experience of degrading of performance is chalked up to "my Windows is slow". Operating systems combatted this eventual degradation by shipping with a defragmenter, which does exactly what you'd expect. It takes all these scattered fragments from around your drive, and puts them next to each other. This reduces the overall amount of seeks necessary to retrieve necessary information, thereby making things speedier.
-But that's an expensive (resource wise) thing to do. In order to defragment a system, the program needs to:
-- find an application that has its data fragmented
-- copy the data between the fragmented data to memory or some other free space on the drive
-- move the data closer together.
-- repeat
+But that's an expensive (resource wise) thing to do. In order to defragment a system, the program needs to:
+
+* find an application that has its data fragmented
+* copy the data between the fragmented data to memory or some other free space on the drive
+* move the data closer together.
+* repeat
Lets go back to our previous scenario, and see how defragmentation could work:
+
```
Initial State: ccbbbcc0
Step 4: ccccb0bb
Step 5: cccc0bbb
```
+
Obviously this is not optimized in any way, so there's plenty we could do to speed this up. But this is essentially what your drive is doing. It's like putting a deck of cards back in order after you've been shuffling them. Sure it's possible, but it just takes some time.
A much better idea, would be to try and optimize STORING this data in such a way that would reduce fragmentation. That is, maybe we keep data that is related next to each other on the drive when we WRITE the data the first time. That way things don't get as fragmented as quickly.
## Blocks and Pages
+
The first step in ensuring data is kept close together is the idea of "blocks". Basically the filesystem that actually interacts with the hard-drive will define a "block-size". The block size is basically a measure of how much data will fix in a block, and the filesystem reads/writes in blocks instead of individual bytes. Think of it this way: If your hard-drive was a piece of lined paper, we were originally writing things down one word per line. With blocks, we basically said "well, we'll just write until we reach the horizontal end of this line". So now instead of one word per line, we have a few words per line. Perhaps, we could say, we have have one sentence per line.
Using our previous 8 unit drive example, we could sub-divide that into blocks of 2 units, making it look like this:
+
```
Drive State: [c,c][c,c][b,b][b,0]
```
+
Now when we want to read all the c values, we have two seeks instead of 1 per record. This is already a big improvement (we've reduced seeks by 50%), but we could probably reduce it even more. Since the filesystem has to expose a standard block size to all applications, systems that have to have a high amount of HDD I/O need an alternative. The easiest thing to do, is take the concept of a block containing records and create another abstraction: a page containing blocks.
At its "lowest" level, a relational database deals with "pages". Pages are really just collections of the data that you are storing. Relational databases (non-relational databases might as well, but I haven't really dug into the internals of a lot of them) utilize this concept of a "page" to further decrease IO latency with the disk. Rather than dealing with the storage of individual records or information, it groups records together into a "page" and uses that. It will read/write a whole page. This allows them to capitalize on the assumption that when you are reading/writing data the data you are accessing is probably next to other data that you also require.
They even go so far as to let you customize this via "clustered keys". A clustered key is just a mechanism to allow you, the database administrator, to define HOW the database orders the data within pages. As the administrator, you know the data you are trying to store, and the primary ways that it might be accessed. Databases give you the ability to say "well, group all these records together on the disk by the values in this column". This creates pages that are grouped around a particular value (a userID for example), so that all records with that same value are near each other.
Think of a database where you want to associate a list of items with a user. You have two tabes, `users` and `items`.
+
```
+-------+ +---------+
| users | | items |
Databases are very intricate systems, and I don't want you leaving thinking there isn't a whole lot more to this whole concept. This is a HUGE simplification of what the database is actually doing, but it should provide you with an understanding of why it is doing some of that at a storage level.
## The problem with blocks
+
The block system, however, is not without its own problems. By using "blocks" we've introduced a bit of wasted space into our storage. Lets go back to our block example:
+
```
Drive State: [c,c][c,c][b,b][b,0]
```
That kind of sucks, and is actually a fundamental problem with "blocks". As long as you have data to write, blocks are great, but they will almost always result in the LAST block in a segment not being completely filled. This is natural of course, since whatever application is using that space generally doesn't care to know (nor should it!) about the block size it needs to be using. The result of this is that the more "small files" you have on your drive, the more "slack space" you have on the drive - space that isn't being used for anything, but is still seen as "used".
-So now we come to a decision, either we just leave that space empty and accept it as part of the operating costs, or we try and figure out how to utilize it. Engineers, (un)fortunately, are quite obsessed with performance. These "tails" (the last block) are inefficient, and could probably be removed with a bit of smart thinking. This results in two possible ways to resolve this problem.:
-- We allow the filesystem to support variable block sizes
-- We figure out how to use that tail block for something useful
+So now we come to a decision, either we just leave that space empty and accept it as part of the operating costs, or we try and figure out how to utilize it. Engineers, (un)fortunately, are quite obsessed with performance. These "tails" (the last block) are inefficient, and could probably be removed with a bit of smart thinking. This results in two possible ways to resolve this problem.:
+
+* We allow the filesystem to support variable block sizes
+* We figure out how to use that tail block for something useful
The first way, variable block sizes, is something file systems like ZFS utilize in an attempt to have more efficient storage. Since you know the kind of data you will be using the drive for, ZFS will let you specify your block size. If you know you have a lot of small things that need to be stored, drop the block size, likewise, increase if you have large things. It even has some magic like block level compression to try and use those blocks to their fullest. It is a very simple idea - and as we know, the simple ideas are the hardest to implement!
The second way, is another simple solution to the problem. If we know we have a bunch of tail blocks that are half-filled.. why don't we just combine them? That way we aren't creating a new tail block, but are instead re-using another tail block. This would result in another seek to read/write this data, but it ensures that we are using this disk to its fullest capacity. File systems like BTRFS will combine multiple tail blocks. The reason this is so effective is because the average block size is actually some multiple of 512 bytes. If you think about it, a text file might be a couple bytes? In a traditional file system that's 1 block per couple bytes. That's a heck of a lot of things you can stuff into a single block at that rate!
## Changing the game
+
As you can see, we've put a lot of work into reducing the seek time for hard-drives. They've been such a fundamental component of computing that it was a requirement. But, what if you could just ignore seeking entirely? What if there was a way to almost instantly seek? In computer science we refer to this as O(1). That is the size of the data we are looking through is irrelevant - we can access any section of the data as quickly as any other. Welcome to the world of solid state storage. Solid state storage utilizes electronics instead of mechanical instrumentation. That is, instead of a spinning disk and actuators for the read/write heads it used electrical circuits. By removing the mechanical parts, it eliminated the "seek" time of disks that we find so slow. The only problem was that it was expensive and hard to make ENOUGH storage this way. We could easily make HDDs that were several gigabytes, but were struggling to make solid state drives at megabytes. It just couldn't keep up.
Until it could.
How interesting
-It's crazy to think of all that we've accomplished because of that little mechanical hard-drive. But what's crazier is that we are only able to see this in retrospect. No one was able to see what the result of spinning disk drives would be. No would thought that we would invent so many different file systems to solve the problems. That we would make so many advancements in technology just to store MORE data on the drives. At the time, they were just better than tape. They were simply a step in the chain, that in retrospect, was pretty cool.
-Follow the conversation at HackerNews https://news.ycombinator.com/item?id=13091192
\ No newline at end of file
+It's crazy to think of all that we've accomplished because of that little mechanical hard-drive. But what's crazier is that we are only able to see this in retrospect. No one was able to see what the result of spinning disk drives would be. No would thought that we would invent so many different file systems to solve the problems. That we would make so many advancements in technology just to store MORE data on the drives. At the time, they were just better than tape. They were simply a step in the chain, that in retrospect, was pretty cool.\
+Follow the conversation at HackerNews https://news.ycombinator.com/item?id=13091192
---
-title: "Serverless with CloudFlare Workers"
-Description: Building LootCap with CloudFlare Workers
+title: Serverless with CloudFlare Workers
+summary: Building LootCap with CloudFlare Workers
date: 2020-05-07T17:31:57-04:00
draft: false
-tags: ["architecture", "lootcap"]
+tags:
+ - architecture
+ - lootcap
+ - publish
+lastmod: 2024-12-12T05:30:40.335Z
---
-
## What is LootCap?
-Last week we ([Adam Cochran](https://twitter.com/AdamScochran) and myself) launched [LootCap](https://lootcap.com). The goal is to provide tracking on a new class of tokens called [Loot](https://medium.com/@adamscochran/what-are-loot-tokens-understanding-an-emerging-asset-class-380b0cc38749). It's been a few years since I used to work at Vault of Satoshi and since that time I fell a bit out of Crypto Currencies. I always felt that most of the buzz around them was focused on the tech or the coin itself. It never provided any value. It felt like most coins out there were focused on trying to re-create the trajectory of Bitcoin rather than trying to DO anything. Ethereum was different. It was different enough to force me to pay attention. It allows you to run [Smart Contracts](https://github.com/ethereumbook/ethereumbook/blob/develop/07smart-contracts-solidity.asciidoc#what-is-a-smart-contract). In recent years you've likely seen "Tokens" suddenly gain popularity. Well those are primarily powered by Ethereum. Again, however, it felt like tokens were just trying to recreate the Bitcoin boom.
+Last week we ([Adam Cochran](https://twitter.com/AdamScochran) and myself) launched [LootCap](https://lootcap.com). The goal is to provide tracking on a new class of tokens called [Loot](https://medium.com/@adamscochran/what-are-loot-tokens-understanding-an-emerging-asset-class-380b0cc38749). It's been a few years since I used to work at Vault of Satoshi and since that time I fell a bit out of Crypto Currencies. I always felt that most of the buzz around them was focused on the tech or the coin itself. It never provided any value. It felt like most coins out there were focused on trying to re-create the trajectory of Bitcoin rather than trying to DO anything. Ethereum was different. It was different enough to force me to pay attention. It allows you to run [Smart Contracts](https://github.com/ethereumbook/ethereumbook/blob/develop/07smart-contracts-solidity.asciidoc#what-is-a-smart-contract). In recent years you've likely seen "Tokens" suddenly gain popularity. Well those are primarily powered by Ethereum. Again, however, it felt like tokens were just trying to recreate the Bitcoin boom.
-Recently however, Loot tokens have become a thing. To condense Adam's [article](https://medium.com/@adamscochran/what-are-loot-tokens-understanding-an-emerging-asset-class-380b0cc38749) (which you should read if you're interested in this stuff)
+Recently however, Loot tokens have become a thing. To condense Adam's [article](https://medium.com/@adamscochran/what-are-loot-tokens-understanding-an-emerging-asset-class-380b0cc38749) (which you should read if you're interested in this stuff)
> Loot tokens are earned by contributing to communities.
-What that means is, instead of you buying the token, the community you belong to gives you the token because your contributions to the community have some merit. It's a way to reward content creators for utilizing your community platform.
+What that means is, instead of you buying the token, the community you belong to gives you the token because your contributions to the community have some merit. It's a way to reward content creators for utilizing your community platform.
When Adam first approached me about the idea for [LootCap](https://lootcap.com) I was on board - this was the first timssse I saw a use for Crypto that wasn't being sold as "Well it's crypto so it's just better!". My task was to build a simple Market Cap site targeting a select number of these tokens that actually qualify as "Loot".
The first step in any project is to have a plan. This doesn't mean that I have some giant design document sitting on my computer, or is it scrawled on a napkit. It varies for every project. In this case, a markdown document with some relatively simple diagrams via mermaidjs was more than enough. In fact, the goal of the site could be expressed in a single sentence.
-> Display relevant market-cap information quickly to everyone
+> Display relevant market-cap information quickly to everyone
I'm also a firm believer in starting application development from the data side of things. The data is the only constant in your application. Everything is just abstracting away queries for your users to display the data in ways that make sense. So that's where I begin.
### Database Design
-Whenever I start a project I do the same thing, from the technical side.
+Whenever I start a project I do the same thing, from the technical side.
1. What will be UI look like?
2. What data do I need to make this happen?
I've always believed that the reason that normalization existed is two fold
-1. Resources were expensive. Going from 8GB to 16GB HDDs were a huge financial leap.
-2. Development was segmented. DBAs worked in isolation, and often against, developers.
+1. Resources were expensive. Going from 8GB to 16GB HDDs were a huge financial leap.
+2. Development was segmented. DBAs worked in isolation, and often against, developers.
-As a database designer, you already know all the normalization rules. So take a look at how the application is being designed. What pieces if information are always shown together. What is important and what isn't? These questions can help you better shape your database design.
+As a database designer, you already know all the normalization rules. So take a look at how the application is being designed. What pieces if information are always shown together. What is important and what isn't? These questions can help you better shape your database design.
With these things in mind, I took a look at my UI, drawing inspiration from the many market-cap sites devoted to crypto currencies. After all, while Tokens aren't Coins, they're treated quite similarly. This wasn't something I did in isolation - Adam has been a part of the crypto community for much longer than I have and I heavily relied on his expertise in defining what data we want to showcase.
-Knowing what data I had to display for a first pass, and how my data was organized I opted to skip a database entirely.
+Knowing what data I had to display for a first pass, and how my data was organized I opted to skip a database entirely.
-See it turns out I can assemble most of the information I need from various 3rd parties, cache it in memory, and when that cache expires, grab the information from the sources and assemble it again.
+See it turns out I can assemble most of the information I need from various 3rd parties, cache it in memory, and when that cache expires, grab the information from the sources and assemble it again.
This worked because the scope of the initial launch is small. I only need to display fairly static data (may change every few minutes) that's simultaneously general. I don't need real-time precision at the moment. This means that I had a lot more flexibility in my architecture design, but also it meant that apart from the cache load/reload events, serving this data would be as fast as I can possible make it.
#### API
-When designing any application I always start with my API. It allows me to define my endpoints and figure out how I'm going to breaking up the data so that I can return the most relevant data for an endpoint. That is, now that I've broken down how I'm going to store my data, what's the best way to create my endpoints so that I have the flexibility to compose the UI that I have envisioned, but also allows me to change that front end as necessary.
+When designing any application I always start with my API. It allows me to define my endpoints and figure out how I'm going to breaking up the data so that I can return the most relevant data for an endpoint. That is, now that I've broken down how I'm going to store my data, what's the best way to create my endpoints so that I have the flexibility to compose the UI that I have envisioned, but also allows me to change that front end as necessary.
It's a tough nut to craft. If you pop open your console on [LootCap](https://lootcap.com) you'll see that we make requests to 2 different endpoints:
/api/v1/coin/ethereum/price
```
-For us, that's enough to render the entire first page and also preload some data so that if you click on any Token for more information, that data is already loaded and ready for you. I also made sure to version the end-points. The reason for this is to differentiate between deployment/releases. We can roll out new functionality to the API endpoints and test them to our hearts content before releasing front-end changes and deprecating the old API versions. We can also split test UI designs if we are interested in such a thing. Versioning your API endpoints is vital.
+For us, that's enough to render the entire first page and also preload some data so that if you click on any Token for more information, that data is already loaded and ready for you. I also made sure to version the end-points. The reason for this is to differentiate between deployment/releases. We can roll out new functionality to the API endpoints and test them to our hearts content before releasing front-end changes and deprecating the old API versions. We can also split test UI designs if we are interested in such a thing. Versioning your API endpoints is vital.
In original dev versions, you could actually supply multiple contract addresses, comma separated, and it would return all the data at once. It was a way to reduce 7 or 8 calls to load the page into 2.
-Obviously that's not what happened in production, but we'lll get to that later.
+Obviously that's not what happened in production, but we'lll get to that later.
#### Front end
-The front-end for the app is actually just a SPA. We load some minimal HTML and dynamically populate the data as soon as we can. The idea behind this is that once we hit production, we can cache the static content for much longer periods of time improving response times which would help us achieve our original goal of Displaying relevant market-cap information quickly for everyone.
+The front-end for the app is actually just a SPA. We load some minimal HTML and dynamically populate the data as soon as we can. The idea behind this is that once we hit production, we can cache the static content for much longer periods of time improving response times which would help us achieve our original goal of Displaying relevant market-cap information quickly for everyone.
We are using a two different 3rd party libraries on the front-end.
-- Mvp.css - https://andybrewer.github.io/mvp/ - because we wanted a simple, classless, css framework as a base
-- BigNumber - https://mikemcl.github.io/bignumber.js - because dealing with crypto numbers in JS can be challenging.
+* Mvp.css - https://andybrewer.github.io/mvp/ - because we wanted a simple, classless, css framework as a base
+* BigNumber - https://mikemcl.github.io/bignumber.js - because dealing with crypto numbers in JS can be challenging.
We aren't using React, and so for the times where we have to update the Dom we're using this function and interacting directly with the Element nodes.
We're also relying on `fetch` since all our endpoints are very simple `GET` based requests.
-We are also using a very rudimentary SPA router that relies on css classes to define routes. We're only able to do this because we have 2 routes.
+We are also using a very rudimentary SPA router that relies on css classes to define routes. We're only able to do this because we have 2 routes.
```javascript
function route() {
Once I had a copy working locally it was time to deploy it. [Heroku](https://heroku.com) has long been my dpeloyment strategy of choice for projects that are undergoing rapid development. I always use git, but I don't always use a git hosting provider (GitLab/GitHub). Since heroku supports deploying from git, so I'm able to deploy from my dev env to share.
-If you haven't had a chance to mess with Heroku for yourself, I high recommend you do. It's a wonderful service that allows you virtually unlimited free applications, and you can easily build your PoC without concern.
+If you haven't had a chance to mess with Heroku for yourself, I high recommend you do. It's a wonderful service that allows you virtually unlimited free applications, and you can easily build your PoC without concern.
-They also offer numerous paid plans that you can use to take your service from PoC to live by clicking a few buttons. My initial thoughts on architecture were pretty standard. Perhaps two "dynos" or servers+load-balancer would be more than enough. We have no state, just in-mem caches, I don't care if the caches are repliced for eahc node. Base price like 20$ a month easy.
+They also offer numerous paid plans that you can use to take your service from PoC to live by clicking a few buttons. My initial thoughts on architecture were pretty standard. Perhaps two "dynos" or servers+load-balancer would be more than enough. We have no state, just in-mem caches, I don't care if the caches are repliced for eahc node. Base price like 20\$ a month easy.
### Serverless?
-However this is somethign we're launching for fun. While spending 20$ for infrastructure (bandwidth might cause some overage) is certainly doable.. is it the best option for us?
+However this is somethign we're launching for fun. While spending 20\$ for infrastructure (bandwidth might cause some overage) is certainly doable.. is it the best option for us?
-For one thing, with any cloud provider, if I'm running a container or vm, I'm stuck to a paricular data-center. That's not very helpful if someone in Japan wants to take a look at my site. Remember, our goal was to provide this information quickly for everyone.
+For one thing, with any cloud provider, if I'm running a container or vm, I'm stuck to a paricular data-center. That's not very helpful if someone in Japan wants to take a look at my site. Remember, our goal was to provide this information quickly for everyone.
So maybe we look at **SERVERLESS**
The idea behind server less infrastructure is simply: You write apps that are designed as "functions". These functions are deployed and spun up/down as necessary. Scale becomes pretty straight-forward. The infrastructure is available, it's just a matter of
-- How much do you want to pay?
-- What's the cold-start time on your app?
+* How much do you want to pay?
+* What's the cold-start time on your app?
-The payment is pretty straight foward. Since these serverless tends to be small apps that cater well to "bursty" applications in terms of requests, they tend to be cheaper to run.
+The payment is pretty straight foward. Since these serverless tends to be small apps that cater well to "bursty" applications in terms of requests, they tend to be cheaper to run.
Most of these providers will take your app out of the pool if you don't get any requests for a preset amount of time. This is to free up compute space for other apps. The time to start for an app NOT currently pre-loaded is what's refered to as "cold start". That varies, mostly reliant on the size of the application you're starting, but there is a baseline cost regardless.
### CloudFlare Workers and KV
-Then I ran across CloudFlare Workers. I remember seeing the announcement post some time back, and thought "hey that's neat" but quickly pushed it out of my mind. I returned to CloudFlare workers and did some more investigation.
+Then I ran across CloudFlare Workers. I remember seeing the announcement post some time back, and thought "hey that's neat" but quickly pushed it out of my mind. I returned to CloudFlare workers and did some more investigation.
-Oh My. I think CloudFlare Workers are the only TRUE serverless mechanism out there.
+Oh My. I think CloudFlare Workers are the only TRUE serverless mechanism out there.
CloudFlare workers are essentially serverless functions.. except they run at every CloudFlare Point-of-Presence (PoP). They have milliseconds of cold-start time, and they run globally. They do have way more restrictions than traditional serverless offerings
-- 1MB compressed filesize
-- 50ms CPU time
-- max 30 workers
-- < 50 subrequests per worker (redirects count!)
+* 1MB compressed filesize
+* 50ms CPU time
+* max 30 workers
+* < 50 subrequests per worker (redirects count!)
But I felt that if we COULD work within their limits, it would bring us closest to what we needed. They also offered an eventually consistent KV store that was accessible from the workers. Since our data was cached anyway, we didn't care about the eventual consistency - worst case we served stale data for a minute more than we expected.
-After a couple of days of fiddling with their tooling and documentation, I was able to convert our regular API into a bunch of "serverless" functions that worked with the KV store and were deployed to all CloudFlare PoP.
+After a couple of days of fiddling with their tooling and documentation, I was able to convert our regular API into a bunch of "serverless" functions that worked with the KV store and were deployed to all CloudFlare PoP.
Today, when you load [LootCap](https://lootcap.com) you're loading data from 3 different workers
---
-title: "What the heck is a Database Index?"
+title: What the heck is a Database Index?
+summary: Explaining database indices
date: 2020-04-29T16:21:29-04:00
draft: false
-tags: ["explanation", "database"]
+tags:
+ - explanation
+ - database
+ - publish
+lastmod: 2024-12-12T05:30:56.492Z
---
When you do a standard `select * from table_name where columns=some_value` from a table.. the database has no idea where IN the table that data exists. In order to figure out which rows in the table it needs to return to you (based on your where clause), it looks through the rows in the table. Of course, if you don't set a limit clause, it has to look through EVERY single entry in the table because it doesn't know where/how-many instances of a particular value it might find. This is a pretty common problem. Think of a standard `users` table
+
```
+-------------------+
| users |
+-------------------+
```
-In this scenario there are a couple different uses of this table .
+In this scenario there are a couple different uses of this table .
+
1. We are registering a user (inserting a row)
-1. A user might be changing their password (updating a row)
-1. A user is logging in (reading a row)
+2. A user might be changing their password (updating a row)
+3. A user is logging in (reading a row)
-In a table without any indices when logging in we only know the users username and password. We have to run a `select * from users where username = ?` on the database. Even if we add a `limit 1` to our query, the database still needs to scan through every single row in our database until it finds one (or more) that matches our filter.
+In a table without any indices when logging in we only know the users username and password. We have to run a `select * from users where username = ?` on the database. Even if we add a `limit 1` to our query, the database still needs to scan through every single row in our database until it finds one (or more) that matches our filter.
+
+If you create an index on a particular column in the table, it serves as a lookup. If you look for a row where a particular indexed column matches a value.. the database doesn't need to scan the table, it can look at the index. You can think of an index exactly the same as as you would an index in a book. If you want to know where a word in the book occurs you have two choices.
-If you create an index on a particular column in the table, it serves as a lookup. If you look for a row where a particular indexed column matches a value.. the database doesn't need to scan the table, it can look at the index. You can think of an index exactly the same as as you would an index in a book. If you want to know where a word in the book occurs you have two choices.
1. You can either flip through the entire book until you see what you're looking for
-1. Consult the index, which tells you which page in the book you need to look at.
+2. Consult the index, which tells you which page in the book you need to look at.
Once you tell the database that you want a particular column to have an index, you don't need to do anything else. The database will ensure that the index is kept up to date (as you modify the data in the table). You also don't need to change anything about how you're accessing the data - (IE: you don't need to modify your queries).
-Ok - so it sounds like indexes are pretty awesome right? If it speeds up looking things up, why don't we create an index on every column?
+Ok - so it sounds like indexes are pretty awesome right? If it speeds up looking things up, why don't we create an index on every column?
Well, like everything - there's a caveat. READING data from the table is very quick with an index.. but Creating, Updating, and Deleting data becomes a bit slower. This is because the database needs some additional time to update the index with the new data. So it's a bit of a balancing act. The more indices you have on a table.. the slower it becomes to write data to that table (insert/update/delete). You want to take a look at your table and how you're using the data in your table and create the index carefully.
-Because of this, you can't really have a "rule" on when to create an index - you'll want to monitor your database and create them based on your investigations. Of course, it's never that easy - the more data IN the table when you create the index the longer it will take to create. It doesn't sound too bad - but if you have hundreds of thousands of rows in a table.. it can take a few minutes to create the index, during which your application will need to be down. So you need to be somewhat pre-emptive in creating your indices.. but also you can't go overboard.. but also there isn't really a guide on when to create an index. How wonderful.
+Because of this, you can't really have a "rule" on when to create an index - you'll want to monitor your database and create them based on your investigations. Of course, it's never that easy - the more data IN the table when you create the index the longer it will take to create. It doesn't sound too bad - but if you have hundreds of thousands of rows in a table.. it can take a few minutes to create the index, during which your application will need to be down. So you need to be somewhat pre-emptive in creating your indices.. but also you can't go overboard.. but also there isn't really a guide on when to create an index. How wonderful.
-As you progress in your database tuning/development life you'll find use-cases where it makes sense to have an index from the start. For example, most "user" tables will have an index on the username column. This is something that you can then take and implement on your next project!
+As you progress in your database tuning/development life you'll find use-cases where it makes sense to have an index from the start. For example, most "user" tables will have an index on the username column. This is something that you can then take and implement on your next project!
## Further Reading
+
I really recommend you peruse this documentation from mysql on what an index is. While you may be using Postgres/MSSQL - the core concepts of what an index is remains the same. https://dev.mysql.com/doc/refman/5.7/en/mysql-indexes.html
+++ /dev/null
-# Changelog
-
-All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
-
-### [0.4.1](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.4.1;hp=v0.4.0;ds=sidebyside) (2023-09-29)
-
-
-### Bug Fixes
-
-* add missing migration 34d64f8
-* remove clickhouse f7a8ccf
-
-## [0.4.0](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.4.0;hp=v0.3.6;ds=sidebyside) (2023-09-29)
-
-
-### ⚠ BREAKING CHANGES
-
-* dungeon traversal
-
-### Features
-
-* cleanup chat commands 074d6ad
-* dungeon traversal 2ec43df
-* min level for all locations 2be0160
-* psql based event system 43f0bc3
-
-
-### Bug Fixes
-
-* auto-enter dungeon if you are in it dfa62a7
-* remove log of 0 events being flushed 552c744
-
-### [0.3.6](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.3.6;hp=v0.3.5;ds=sidebyside) (2023-09-06)
-
-
-### Features
-
-* chat command to set player level 9ced477
-* rbac support with admin permission 940079d
-
-### [0.3.5](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.3.5;hp=v0.3.4;ds=sidebyside) (2023-09-06)
-
-
-### Bug Fixes
-
-* reduce specials to 10% 5beba79
-
-### [0.3.4](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.3.4;hp=v0.3.3;ds=sidebyside) (2023-09-06)
-
-
-### Bug Fixes
-
-* add missing migration d7268ba
-
-### [0.3.3](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.3.3;hp=v0.3.2;ds=sidebyside) (2023-09-06)
-
-
-### Features
-
-* add debug/error methods to logger 3238965
-* display equippable hand option only d7569d3
-* migrate to augmenting express.Request interface d820e11
-* monster variants b8bed23
-* remove body-part targeting c6a5f62
-* variable level monsters b5577b4
-
-
-### Bug Fixes
-
-* min/max level definitions for monsters bedec85
-
-### [0.3.2](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.3.2;hp=v0.3.1;ds=sidebyside) (2023-09-01)
-
-
-### Features
-
-* `/online` to list usernames of all online players 3e37d7f
-
-
-### Bug Fixes
-
-* add missing "Back to Town" button 0752f70
-* skills not progressing with use 36b7b7c
-* small chat style tweaks e6b5884
-
-### [0.3.1](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.3.1;hp=v0.3.0;ds=sidebyside) (2023-08-31)
-
-
-### Bug Fixes
-
-* add button that goes back to town from any page 06c0e48
-* spells support durability 2dbb9b8
-* stop z-stacking alert messages 6e756e8
-* tooltip text centered due to media-query cbfebd1
-
-## [0.3.0](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.3.0;hp=v0.2.17;ds=sidebyside) (2023-08-30)
-
-
-### ⚠ BREAKING CHANGES
-
-* vigor mortensen
-
-### Features
-
-* display optimal level range for monsters 5878793
-* move alerts to bottom of main section 9575cfb
-* repairing damaged equipment 161b5bf
-* unequip items if they hit 0 ap in battle bc9e05f
-* vigor mortensen f6aba7a
-
-
-### Bug Fixes
-
-* spacing for stat increase button 61e6d07
-
-### [0.2.17](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.2.17;hp=v0.2.16;ds=sidebyside) (2023-08-25)
-
-
-### Bug Fixes
-
-* xss username on signup a827642
-
-### [0.2.16](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.2.16;hp=v0.2.15;ds=sidebyside) (2023-08-25)
-
-
-### Bug Fixes
-
-* properly increment skill level 24e6b3b
-
-### [0.2.15](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.2.15;hp=v0.2.14;ds=sidebyside) (2023-08-25)
-
-
-### Features
-
-* expoential exp drop-off/gain c8bd4d6
-
-
-### Bug Fixes
-
-* add block timer for casting and clear after fight bffb415
-* make signup collapsible 97b3c28
-* rate limit fights! 235d836
-* standardize blocking timeouts for buttons 01d06ca
-* time displays at 0 for midnight 789380b
-
-### [0.2.14](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.2.14;hp=v0.2.13;ds=sidebyside) (2023-08-21)
-
-
-### Features
-
-* display travel progress d66fcf8
-
-### [0.2.13](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.2.13;hp=v0.2.12;ds=sidebyside) (2023-08-21)
-
-
-### Bug Fixes
-
-* strip all tags in chat 9da5336
-
-### [0.2.12](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.2.12;hp=v0.2.11;ds=sidebyside) (2023-08-21)
-
-
-### Features
-
-* return to town button while travelling fd070de
-
-
-### Bug Fixes
-
-* xss chat input 943cbc3
-
-### [0.2.11](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.2.11;hp=v0.2.10;ds=sidebyside) (2023-08-21)
-
-
-### Bug Fixes
-
-* chat history clearning existing chat on load 8a7161c
-* green button colors b8755db
-* migrate recruiter to htmx 8f42a66
-
-### [0.2.10](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.2.10;hp=v0.2.9;ds=sidebyside) (2023-08-18)
-
-
-### Features
-
-* add icons for beginner equipment 218a9ee
-* increase hp gain rate 1f9aaf6
-
-
-### Bug Fixes
-
-* background not appearing if reload during fight 64a76af
-* cant perform other actions in a fight b1a1999
-* chat timeline to messages show up chronologically b82a2ef
-* dont display death text after fleeing 2a1bffe
-* missing % from player bar a0606a5
-* move purchase button under icon in stores 93aeef5
-* only disable equipping/unequipping in a fight b6e9f9a
-
-### [0.2.9](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.2.9;hp=v0.2.8;ds=sidebyside) (2023-08-15)
-
-
-### Features
-
-* default player to the Explore tab 9b5b9fa
-* new UI c793612
-
-
-### Bug Fixes
-
-* avatar takes up too much space on mobile dced2d0
-
-### [0.2.8](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.2.8;hp=v0.2.7;ds=sidebyside) (2023-08-10)
-
-
-### Bug Fixes
-
-* migrate signup/login form to htmx 26e475c
-
-### [0.2.7](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.2.7;hp=v0.2.6;ds=sidebyside) (2023-08-09)
-
-
-### Bug Fixes
-
-* reduce spam requests on /status 5862d25
-* seed equipment info bac2f63
-* time displays 0pm at noon instead of 12pm 4993906
-
-### [0.2.6](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.2.6;hp=v0.2.5;ds=sidebyside) (2023-08-09)
-
-
-### Bug Fixes
-
-* armour icon support 4f6823d
-* auto-load player object and place in request 7079401
-* chat history calls clearing chat 0d1626c
-* migrate stat increase to htmx 734e42d
-
-### [0.2.5](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.2.5;hp=v0.2.4;ds=sidebyside) (2023-08-05)
-
-
-### Features
-
-* add health-potion functionality 2f13844
-* purchasable items from the store 4d8b8b1
-
-
-### Bug Fixes
-
-* migrate chat to htmx 34c9eae
-* migrate explore fight to htmx d955aa8
-* migrate exploring to htmx 1580d3a
-* migrate healer to htmx 18e87bf
-* migrate inventory to htmx 0a45adb
-* migrate item usage to htmx 6fb15e2
-* migrate shops to htmx 09b3c0d
-* migrate skills page to htmx c0de5fa
-* profile page to html (wip) eeead89
-* remove unnecessary console.log e547728
-* rename shop_items to shop_equipment 68d481a
-* support time display e110480
-
-### [0.2.4](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.2.4;hp=v0.2.3;ds=sidebyside) (2023-07-29)
-
-
-### Bug Fixes
-
-* filtered shop views f160ce2
-* time gradients not scaled properly 2b5a905
-
-### [0.2.3](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.2.3;hp=v0.2.2;ds=sidebyside) (2023-07-28)
-
-
-### Bug Fixes
-
-* spacing of time on mobile 1db9c53
-
-### [0.2.2](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.2.2;hp=v0.2.1;ds=sidebyside) (2023-07-28)
-
-
-### Features
-
-* first profile picture ab0d041
-
-
-### Bug Fixes
-
-* 3s delay when clicking the button to keep walking 8710ea9, closes #11
-* display right background based on travelled distance 5f6b5f9
-* extend in-game day to 2 IRL hours 8e44b72
-
-### [0.2.1](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.2.1;hp=v0.2.0;ds=sidebyside) (2023-07-27)
-
-
-### Bug Fixes
-
-* only run into monsters 20% of the time 84c3282
-
-## [0.2.0](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.2.0;hp=v0.1.1;ds=sidebyside) (2023-07-27)
-
-
-### ⚠ BREAKING CHANGES
-
-* travelling between towns!
-
-### Features
-
-* paths now define a distance between them 53b5ae7
-* return to town button if killed while travelling 9325533
-* travelling between towns! 621c35b
-* when you die your travel plan is discarded 6f21c21
-
-
-### Bug Fixes
-
-* back to town button after healing 7f29b2f
-* disable switching tabs in a fight ea7eafe
-* display stat vals without sign or coloring bf702a6
-* display the image from the destination city at 50% progress 084353a
-* replace city background if you die while travelling 8a15f77
-* travel now takes X steps c4f9b81
-
-### [0.1.1](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.1.1;hp=v0.1.0;ds=sidebyside) (2023-07-13)
-
-
-### Features
-
-* allow users to log out 22d96bc
-* display current deployed version in UI a717a30
-
-
-### Bug Fixes
-
-* casting now uses INT instead of CON 223bc6d
-* check players hp before allowing them to start a battle 4d7a556, closes #2
-* move jquery to a webpack external to reduce bundle size 56baafa
-* prod build process + migration cecab78
-
-## [0.1.0](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.1.0;hp=v0.0.3;ds=sidebyside) (2023-07-12)
-
-
-### ⚠ BREAKING CHANGES
-
-* players get stat points on every level up
-
-### Features
-
-* players get stat points on every level up 1c9c8a5, closes #5
-
-
-### Bug Fixes
-
-* define stats as an enum for easier iteration f113d4a
-* spacing of stats/equipped items section on mobile 9f45e48
-
-### [0.0.3](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.0.3;hp=v0.0.2;ds=sidebyside) (2023-07-10)
-
-
-### Features
-
-* add damage mitigation to armour f8156c8, closes #3
-
-### [0.0.2](https://git.xangelo.ca/?p=risinglegends.git;a=commitdiff;h=v0.0.2;hp=v0.0.1;ds=sidebyside) (2023-07-07)
-
-
-### Bug Fixes
-
-* give users free healing if they have less than 20g 8dbffa6
-* type in temp professional services 5671ff4
-
-### 0.0.1 (2023-07-04)
-
-
-### Bug Fixes
-
-* background gradient doesnt span full height 20d6206
-* cache main map screen 1a5ad09
-* client socket sometimes sends events before server finished auth 6d3944f
-* display error if trying to log in with invalid creds 5bc3c9f
-* display HP+ for restoration spells 39dc648
-* display login message to all players 2bb30ea
-* display when your armour gets destroyed and actual HP damage 067db3b
-* empty skills usage causes fights to freeze 1da6f3b
-* level <= 3 is free healing 23646fd
-* make main page section more legible f85442e
-* missing defined type 7aa155d
-* monster selector autoselects previous monster a563227
-* new players didn't get profession level set ef3cf37
-* new players don't get their profession set 3d591f9
-* switching between tabs doesnt always work 375c2fd
-* users city location not persisting 86d54fb
Home / xangelo.ca.git / commitdiff
