Older Posts
Orient Express: My First Project at the Glen
Orient Express is one of many climbs at the Glen that I have wanted to try for several years now. (along with One Inch Punch and Black Eagle) I finally feel like I am in a position to be able to spend more time driving out to the glen and actually project climbs. In 2020, one of my goals was the climb a V8 outdoors, but of course COVID had other plans. Unfortunately, since then, I have not made any serious goals or attempts to progress my climbing. This was partly due to focusing on my career and partly due to lack of ambition. Now that I am more comfortable with where my career is headed, I decided to take another stab at my goal from 2020. Although I am 6 years late, my modus operandi seems to be better late than never. I have already put in about 4 sessions and can do the climb in two links. I will be returning on Sunday (June 28, 2026) for my 5th session. If I send the climb, I will write more about the process and the micro beta of the climb itself.
May 25, 2026
Logging My Climbs and Blogging My Thoughts
To be honest, I have not really reflected on previous climbs or logged my ascents very accurately. I suppose there is the crag and the mountain project, but I have not really been logging ascents as diligently recently. The reason I wanted to start logging my thoughts was because I realized after trying bonfire rodeo a handful of times now, the climb oddly feels different each time I try it. I doubt that I will go back and read these, but I think it might be interesting to see climbs feel over time and perhaps I can quantify an small progression that I make. I am also realizing that I have been climbing long enough to not really remember what I have already tried. This is especially true for sport routes, as I am simply not as familiar with most sport routes. Specifically looking back at the sport routes that I have worked on, there is only really one climb that I remember, Jug City, which is a 5.9. With that being said, hopefully having a climbing diary will help me to remember what the holds feel like and how hard certain moves are. I think it will also be interesting to see how I progress. Currently, I am probably the strongest I have ever been, despite weighing the most I have in a long time. Hopefully I can find the discipline and motivation to properly diet and train. Anyways, the motivation behind this section of the blog is to track my experience climbing, whether it be achieving goals, making progress on climbs, or finding the motivation train. Maybe I will find the motivation to work on Black Eagle again.
May 01, 2026
A Problem with Taxes
If I may be frank, this is an older ticket, so I am a bit fuzzy on the details (I need to keep better notes on more notable tickets). What I can say is that Verndale had overhauled the entire e-commerce site for this particular client before it was passed on to our team. Naturally, there were a few bugs after the production release, and we were on the hook for them. In hindsight, the client was a little annoyed with how many bugs there were, but in our defence, their products are seasonal, which meant that each quarter they would release entirely new products that were mostly unrelated to the previous quarter. To top things off, they did NOT fully test their products in the QA environment before the start of the season and just released them to production. So from that perspective, there is a case to be made that it was actually surprising how few bugs there were. Siriusware Siriusware is the ticketing point of sale software that runs the whole show. All the products on our site come from querying Siriusware APIs. This includes, prices, availability dates, quantities, products, what variants are available, the ability to pay with payment plans, and of course taxes. What is quite strange is that Siriusware trusts the API caller implicitly unquestionably or validation. Although all the pricing information, tax information, and ticket quantities are all determined by Siriusware, when creating an order, the API caller must send all these values AND Sirius will accept them no matter what. NO MATTER WHAT. The repercussions of this behaviour can be seen here. The Actual Problem One morning, we receive a ticket from our client stating that pricing is incorrect when using the newly released payment plan feature. But of course, it is not on all products, it is only on particular products. This particular product is the family passes. On our end, the "product" with the problem was the VariantItem. VariantItems were sub products of Variants (Which were already sub product of Products). Yes, that was not a typo, in the DB, we literally had a join table called VariantVariantItem. Anyway, the point is that the price is wrong and at this point I do not even know what the payment plan feature even is, so you can forget me even trying to recreate the issue. After playing around with the numbers I see that the discrepancy between what is being charged and what the actual total is the tax. What is strange is that we send the tax amount to Sirius in the payload, and it is correct for everything except for Variant Item Items when using the payment plan (aka separating the payments over multiple charges). In Montreal, there are two chargeable sales taxes: 5% GST + 9.975% QST for a total of 14.975% Some Sirius people might call them TaxA and TaxB (i.e., the people who made Siriusware do). After reading and rereading the documentation and testing what feels like hundreds of payload values to Sirius, it turns out that we need to explicitly state the taxes for each line item and not just the total. For some reason, we need to send TaxA and TaxB as separate values in the payload, but only for payment plans. So now I have to refactor the tax-calculating logic in the cart/checkout flow to split everything, just in case the user wants to select a payment plan at the end of the checkout, even though Siriusware has all the tax information in their own system. We are already sending the total tax that needs to be charged, but now we have to split it up line by line. This tax price and information, however, is data that comes from Sirius in the first place. So at the time of the order, we are sending Siriusware the product we are ordering, the total tax to be paid, TaxA, TaxB, and the price of the product, even though all that information is stored in Sirius. To top things off, this is not mentioned anywhere in the documentation as being necessary (probably because it is a bug). Anyway, it actually took another ticket/edge case for me to completely figure out and resolve this issue, but at the end of the day, it was finally fixed. Lessons Learned Have a better QA environment if you want to catch bugs sooner Don't use Siriusware
Apr 24, 2026
Automating 24/7 Monitoring for a Weekend
This is the story of how I was tasked with monitoring our website all weekend, but I wrote a script to do it instead. I did not realize it at the time, but one Friday morning, a storm was brewing. We received an innocuous ticket from a client that orders appeared to be stuck. At this point, the ticket was assigned to my colleague, and I didn't poke around much further. Introduction At around 3:00 p.m., however, the ticket was transferred to me, as my colleague intended to log off early. Naturally, the ticket was to be prioritized, but there was no cause for alarm. During the knowledge transfer, we noticed that the scheduled job that was supposed to be processing orders appeared to be stuck. The job that usually only takes a few seconds had been running for over an hour and had still not completed. When checking the queue, we noticed that there was an abnormal number of orders. What we later found out was that the client had a special promotion where certain products were free with the sign-up of a subscription. This resulted in an absurd number of new orders coming in. I double-checked to make sure that orders were still being processed, albeit a little slowly. I still wasn't sure that there was a bigger problem at hand, and in addition, there was a senior dev looking into the root cause. One particularly unfortunate behaviour was that when the scheduled job ran, it would first grab all the orders and then process them first in, last out (FILO). Orders that were added after the scheduled job started would not be processed until the job was run again. Since there were thousands of orders coming in, some unlucky orders had to wait longer and longer, especially because the scheduled job had been restarted several times (we were unsure whether it was stuck or not). Come 5:00 p.m., I sync with the senior developer along with the project manager to ensure that we are all on the same page. The job is running and processing as expected, but it is a little slow. Doing the math, it will take over 1.5 days to finish processing, even if there are no more new orders. We are all on the same page, and the PM lets us know that he will talk to the client and let them know what we have found. The senior developer works in a different timezone, so his shift had actually been over for more than four hours. With that in mind, I tell him to log off and reiterate that if there are any further problems, I will handle them. Of course, at this point, there is nothing to do but wait, so I monitored for maybe another 10 minutes before stepping away from my computer to take a break. I also made dinner plans with my significant other, and at around 6:00 p.m., I began getting ready to leave for dinner. I reminded myself that I should check my messages before I leave, but I was not too worried. In addition, I had my phone with me, so they could always reach me that way. Where the Trouble Begins I picked up my significant other and was on my way to the restaurant when I got a message from the Big Boss (my boss's boss). He was trying to figure out what was going on and put out a fire that had been raging. I realized I also had a missed call. I was still driving at this point but ended up pulling into a parking lot to see what was going on. Furthermore, I ended up calling the Big Boss back to see what was going on. It turns out that after I had left, the PM was unable to explain what had happened to the client or provide assurance that the orders were still being processed. I realized at this point that I had failed to remember to check my computer before I left for dinner. Once home, I jumped on a call with the senior developer and the Big Boss and saw that I had missed about a dozen messages. I was told that the client was having a meltdown and was not convinced that orders were going through. They even suggested that they start manually processing orders to get through the whole backlog (this would have taken more than a week and been prone to errors). The Big Boss was barely able to talk the client down and assured them that we would handle the situation. We again confirmed that the orders were being processed, even though it was slow. To ensure that the client was happy, the Big Boss told me that I would have to take turns monitoring the orders to make sure that they were still being processed. We were to give status updates every hour until the orders were completely processed. Long term, we needed to speed up the order processing times and ensure that the queue is FIFO instead of FILO. We never had an issue before because the client would have at most 10 orders in a day. This particular lucky Friday, we had received well in the thousands. In the short term, however, this meant that we would have to take turns staying up all night to ensure that the client was happy. At this point, I was just happy that I was not getting fired (as it was mostly my fault that the issue escalated this far). As such, I offered to take the graveyard shift. The Automation In the middle of the night, I realized that it was partially stupid to stay up all night just to count how many orders were remaining by hand. I noticed that to log in to the back-office/admin section of the website, it only required basic auth. Naturally, I started working on a Puppeteer script that logs in, goes to the right page/tab, counts how many orders are remaining, and logs the number into a Google Sheets document. From there, I used the timestamp and number of orders to graph it in a chart using Google Sheets. To automate the script runs, I created a scheduled task on Windows to run a batch script that, in turn, runs Puppeteer. At the end of the day, I probably should have just checked my messages before I left for dinner. Writing the script honestly took the better part of a working day and was totally not worth it, but at the very least, it was fun. Lessons Learned: Always have your phone with you when you are on call Always double check that an issue is completely resolved from the client's perspective before assuming all work is done
Jan 29, 2026
The Case of the Free Products
This is a post-analysis of an issue that arose with a former client. It wasn't a difficult bug to diagnose as much as it was comical. Introduction It all started one morning when we received a ticket from the client stating that some orders were not totalling correctly. Upon further investigation, we realized that some products were not being charged at all. They were receiving the products completely free. What made it bad was that these items could range from $400 to well over $1,500. Naturally, we tried recreating the issue by adding the same products to our cart, but no matter what we did, we could not reproduce the error. In the old version of the website, to get any prices or information from the store, it would make API calls to the pricing software called Siriusware. This software is quite literally some ancient legacy software that comes to a crawl during peak hours of business. To remedy this, we created a synchronization job that runs once a day during off-peak hours. This would synchronize everything, including products, prices, and availabilities, from Sirius into our own database. What I have come to realize is that a lot of business logic is simply synchronizing data from one system to another. This meant that the prices in our store were stored in our database, even though the source of truth came from Siriusware. This meant that the next step was to verify the prices stored in the database. Were there any prices that were incorrect or missing? To properly understand how the prices are saved in the DB, I should preface that the prices are not stored based on product, but based on variants. The same product could technically have different prices. For example, the same pair of headphones can have different colours, and each colour could have a different price. What was silly was that, in rare cases, a variant could also be broken into variant items, and each variant item would have its own price. For the most part, prices stay the same each day. However, there are times when a product will be on sale for a short period of time. To account for this, we end up storing each product, variant, and variant item by day. Now, that might not seem so bad at first, but if you have 100 products, and each product has five variants, and each variant has another three variant items, and you save a whole year s worth of products, we are looking at a cool 100 5 3 365 = 5.5 million entries in the DB. While 5.5 million entries isn't the worst, it is not the only table that needs to be accounted for. So, to reduce the size of the prices table, we would periodically prune prices from older days. This made sense because we don't need to check the prices of products from three weeks ago if we are making a purchase today, right? RIGHT? Well, that is where we were wrong. When you add an item to the cart, it saves the date as well. This means that if you save an item in your cart, when we do the price lookup, it will request the price of the product, variant, or variant item using the date that the product was added to your cart. This was by design because the site administrators wanted users to be able to add products to their cart and have the price locked in until they make the payment. However, since we were pruning older prices, what ended up happening was that getting the price would fail. Instead of throwing an exception, if the price lookup fails, it simply replaces the price with a null value, which conveniently gets converted to a big, fat zero. This meant that clients who left items in their cart for three weeks or more would automatically be able to check out their items for free. And, of course, Siriusware, being the great software it is, has no guards in place to actually check whether the prices we are sending are in sync with the prices in their system and simply blindly approves whatever we send them. Conclusion Ultimately, the issue was fixed by simply getting the product's price for today if the price lookup fails for that particular day. I recall needing to update the pricing information in the cart logic. Unsurprisingly, although the fix was seemingly simple, the checkout and cart flow was quite difficult to follow. Regardless, we got the job done. Perhaps I am wiser now, but at the time, I do not recall being intimidated or scared, even though I was updating the checkout logic, which could technically adversely affect all purchases made on the site. While this was technically a small change, in hindsight, the issue could have easily become massive if done incorrectly. In addition, I think I was always a little trigger-happy when it came to deploying to production. You could say that I was younger and less experienced. There is nothing quite like the boldness of a junior developer. Lessons Learned: Make sure the fallback for null prices is not 0 (aka free) If you price your products by date, make sure you have a fallback for older carts/dates When changing checkout cart logic, probably spend a bit more time testing (there were no issues this time, but still)
Jan 21, 2026
Zendesk Discord Chatbot using Webhooks
In 2024, the support team at Yaksa (before Verndale had purchased the company) used a Microsoft Teams bot application that was integrated with our Zendesk ticketing platform. The chatbot messages you in real time anytime your ticket is updated. Although we do get emails in near real time, I always find the frequency of the emails to be too noisy, and Outlook/Windows would not properly send notifications when a new email came in. I really appreciated the Zendesk Teams bot, as the instant notifications improved my response time and productivity. In the last year (although unrelated), around the time our company got bought out by Verndale, the Teams bot began misbehaving. It would start double-messaging and phantom-messaging the team about tickets that had not been updated at all. It also sent messages much later than they were updated, defeating the original purpose of the bot. It had gotten so bad that everyone on the support team muted the Zendesk bot and completely stopped using it. Later that year, we eventually migrated our messaging platform from Teams to Slack, and the Zendesk bot had become a memory of the past. Six months ago, there was an in-house AI hackathon that was hosted by Verndale. The support team decided to create a bot that integrated with Zendesk using webhooks. Ultimately, we didn't spend enough time working on the hackathon, which led to a presentation that was less than stellar. It did, however, give us access to various Zendesk APIs and webhooks. Funnily enough, in the back of my mind, I did consider the possibility of creating my own bot that used the webhooks, but the hackathon was over and our access, as well as any enabled webhooks, would soon be disabled or so I thought. Just last month, I began playing with Docker, and one of the new applications that I discovered was Uptime Kuma. It is a site monitoring tool that can integrate into various chat platforms, notifying you when a site becomes unavailable. This was quite useful for me to check the uptime of my Plex and Immich sites. Naturally, I integrated it with Discord, but it also got me thinking again about the possibility of integrating Zendesk with Discord. I checked if the webhooks were still active, and to my excitement, they were! I immediately knew that I wanted to recreate the chatbot, but I also had to ensure that the uptime would be reasonable. I initially thought of hosting the bot using Vercel because of the generous free tier and ease of integrating with Next.js, but I quickly became afraid that it would be violating their terms of service. This bot would technically no longer qualify as a "hobby" project, as it is an internal tool for "commercial use." What I ended up going with was creating a website using Pocketpages on Pockethost. In the end, my solution was quite hacky, as the Pocketpages framework is not very well known, but the documentation was "good enough." The most important part was that it was free and the uptime was not handled by myself. I won't really bore you with the details, but effectively, the webhook that is sent by Zendesk includes all the ticket information, such as ticket number, assignee ID, actor ID, message, ticket type, and custom statuses. With this information, I am able to identify who updated the ticket, who the ticket is assigned to, if the ticket is about to breach, the ticket number, and the client organization. Using that information, I am able to send the ticket to the appropriate Discord user along with essential information. I had a small issue with duplicate messages because multiple webhooks would come in from the same ticket (sometimes within the span of 3 ms), but I finally sorted out the issue by grabbing all the records in the last ~10 seconds and only sending a message if it is the oldest message in the 10-second window. Now, I am able to relax and not worry about whether or not I have missed an update on any of the tickets assigned to me.
Nov 08, 2025{{ blog.title }}
{{ blog.text }}
{{ blog.date }}