Hi everyone,
I’ve returned this week and hope to make this newsletter a more regular occurrence. I apologize if my absence and lack of updates have disappointed any of you.
Since we last caught up, I’ve made several updates. I’ve polished my side project, ec2.shop, which I believe is the best tool for quickly viewing all on-demand, spot, and reserve prices on a single page. Give it a try and let me know what you think.
One significant change in my perspective since last year is my strong belief in the importance of self-hosting more nowadays, rather than relying on free tiers or cloud providers.
I’ve witnessed cases where individuals have lost years of photos after being locked out of their Google accounts, with no means of contacting a real person to resolve the issue. Moving forward, I’ll be adding a self-hosted section so we can started to move into the direction of owning our own content. It’s also the reason my newsletter platform is home-grown, self-made.
Becuase it’s been a while since I last sent out this newsletter, if you nolonger enjoy the content, you can click unsubscribe at the bottom of this email.
As a software engineer, I like to tinker around hardware projects, physical thing that you can touch, and show to our faimily. Radio is fastinating because there are many valueable information being pass around. You can listen to airplane communication, track aircraft via ADS-B, you will write some code to decode the signal and make sense of them. You can even throw a front-end to render them on a map for example.
Understand better how HTTP works in each version.
It just proves that if you want something done right, you have to do it yourself. Encouraged by this optimistic forecast, today we will implement a large language model in SQL.
Given a color image? How can we convert it to black and white, pixel-art-like images? The cool thing about these pixel-art image is they can be easily embed into a README.md and render great. We will learn the binning algorithm, implement purely just with JavaScript.
JavaScript is quick and easy to write. The language is very forgiable so can easily write code that isn’t optimize. In this post we will learn a few way to optimize JS.
I’m honst everytime I get into Figma, I’m frustrated and think how everyone using them? The UI is so clunky, I always have to make sure I don’t accidently move something. But anyway, they operate at massive scale and today they share the story they migrate and shard their database.
Soft delete is the practice of not deleting the row physically but create a flag that we delete a row. It’s very simple to implement because deleting data isn’t easy. Such as deleting a user lead to remove many relevant data, relevant foreign key. Data can also refer to external data, such as user avatar store on S3. because of that, many use soft delete. it does add complexity for operation though, and what can go wrong, we will learn today.
When you have an outage caused by a performance issue, you don’t want to lose precious time just to install the tools needed to diagnose it. Here is a list of “crisis tools” Bredan recommend installing on your Linux servers by default. I would suggest to add 4 mores: nmap, nc, netstat and strace
The One Billion Row Challenge is to write a Java program for retrieving temperature measurement values from a text file and calculating the min, mean, and max temperature per weather station. There’s just one caveat: the file has 1,000,000,000 rows! It has been expand and adopt by many languages or eco system. This one show an optimization to bring up almost 20x performance. There one another similar challenge of moving a billion postgres rows on 100 budget
do you remember years ago you can dial a number on the phone, they play some music. We replicate this setup here with old hardware, open source project like Asterisk to do that. If you have a kid def you would want to look into it. Not an end to end soluition but I show a full setup, how thing tie together, which component you need.
Have you ever wonder what is the process of writing a database library ? This project shows that, implement PostgreSQL wire protocol in Rust
Rusta high performance, arbitrarily scalable, open-source, simple, easily forkable, well-tested, batteries included LLM written in pure Python/Jax and targeting Google Cloud TPUs. MaxText typically achieves 55% to 60% model-flop utilization and scales from single host to very large clusters while staying simple and “optimization-free” thanks to the power of Jax and the XLA compiler.
Pythonfgprof is a sampling Go profiler that allows you to analyze On-CPU as well as Off-CPU (e.g. I/O) time together. I’m always interested in how to implement this CPU profiling tool. The project is less than 1000 LOC so you can grasp the idea behind quickly
GoThere are already tool to help write Postgres extension with Rust such as pgrx. Now we can do with Zig. After the last 2 years working with Rust, I feel like it’s such as great language that is truely innovative compare with other, but there is still room for a low level language that isn’t too strict such as Zig. Using zig to write a postgess extension feel great.
A small JS library for beautiful drawing and handwriting on the HTML Canvas. An use case is you can use to take user signature and have it smoothen out. Check the demo as well.
A Ruby gem to transform HTML into PDFs, PNGs or JPEGs using Google Puppeteer/Chromium. Useful when generating invoice
If you want to generate some kind of activity maps this is a good tool. Example in my email forward app I had a world map on homepage which can probably re-impelemnt with this tool
a persistent message queue Go library built on SQLite and inspired by AWS SQS (but much simpler).
An open source alternative to ChatGPT that runs on the desktop.
A database of vulnerable Ruby Gems. useful to not miss a vulnerable if you are using Ruby
Copy a Postgres database to a target Postgres server (pg_dump | pg_restore on steroids)
highly configurable, distributed, in-memory data store and cache implemented in Go. Think of it like a Redis alternative but speak the same protocol, therefore we can reuse the same client code, use the same our favorite Redis client in our language. With the recent Redis license change, it’s time to look for alternative.
An open source alternative to ChatGPT that runs on the desktop.
Fast, modern and advanced photo management suite. Runs as a Nextcloud app. Can be use as a self-hosted solution instead of google photo.
Incident management framework centered around a ChatOps bot for Slack to allow your teams to easily and effectively identify and manage technical incidents impacting your cloud infrastructure, your products, or your customers’ ability to use your applications and services.
Dashboards for DevOps. Visualize cloud configurations. Assess security posture against a massive library of benchmarks. Build custom dashboards with code. Install it, connect with your account and it will analyze to show any potential issue.
BetterDev Link
Every Monday