Building resilient services at Prime Video with chaos engineering
Large-scale distributed software systems are composed of several individual sub-systems—such as CDNs, load balancers, and databases—and their interactions
Chaos engineering introduce random failure in the system to uncover unforeseen turbulent events and find ways to deal with them.
In this post, Prime Video presents a simple approach for fault injection in systems utilizing EC2 and ECS, and its integration with a load-testing suite to validate the countermeasures put in place to prevent dependency and resource exhaustion failures.
Used Postgres extended statistics to achieve a 3000x speedup
A query took 2 mins to run now can complete in 42ms. A big number, big claim, take it with a grain of salt and put it into its context. The primary action is to hint Postgres Query Planner to take a different path instead of the default generated planner which is slow in this particular query. The lesson learned is that you need to monitor slow queries and see if it has a more efficient path.
SWIM: The scalable membership protocol
In a distributed system we have a group of nodes that need to collaborate and send messages to each other. To achieve that they need to first answer a simple question: Who are my peers?
A membership protocol helps each node in a system to maintain a list of nodes that are alive, notifying them when a new node joins the group, when someone intentionally leaves and when a node dies. SWIM, or Scalable Weakly-consistent Infection-style Process Group Membership Protocol, is one of these protocols.
How NAT traversal works
NAT is used widely in home networks, or Cloud VPC(AWS, GCP, Azure) and many network deployments. Because we cannot have a public reachable IP for every device so all devices share a single IP address and rely on the router to translate the internet package back to the right device. So how do we establish a peer-to-peer connection between two machines that sit behind NATs?
How to contact Google SRE: Dropping a shell in cloud SQL
Using a bunch of GCP Cloud SQL vulnerable to run a reverse shell. Writing file to
/mysql/tmp via a SQL injection issue, create custom plugin blob to run a reverse shell. The interesting fact is how they get escape out of the container that runs reverse shell by sniff metadata endpoint traffic and spoof response to include their SSH keys. This post just convinced me that we should use TLS everywhere no matter how locked down a network is.
How Shopify Reduced Storefront Response Times with a Rewrite
Shopify is huge and they are pushing the edge of technologies, especially Ruby/Rails. They still strongly emphasize on monolith. In this post, they outline the process of rewrite the storefront, which is built based on Liquid theme. As always, the key to performance is handcrafted SQL statement, carefully avoid N+1 queries, caching. The interesting part is before building the new thing, they create a verifier mechanism. This mechanism compares the output of both implementations to make sure both systems behave identically.
A curated list of awesome System Designing articles, videos and resources for distributed computing, AKA Big Data. Whether you’re preparing for an interview or you want to design a distributed/microservice oriented application, this list will definitely help you achieve that.
The Joy of Cryptography
a textbook that the author has been writing for CS427, their undergraduate course in cryptography.
It’s free and is supported by the Oregon State University open textbook initiative. It has 120 exercises.
You can download the draft pdf. It’s a bit math heavy but very close to what we used daily: padding, secret sharing, MAC, RSA, Digital Signatures.
Are CRDTs suitable for shared editing?
CRDTs are often praised as the “holy grail” for building collaborative applications because they don’t require a central authority to resolve sync conflicts.
The answer is yes in this article, but note that common editing platforms like Google Docs, Zoho Writer, CKeditor, ProseMirror, Quill, CodeMirror - all of these work with OTs instead of CRDT for collaborative editing. The hacker news discussion is interesting as well. The xi-editor creator also said CRDT didn’t work out for xe-editor either
Code to read
A pure-Ruby HTML5 parser
API and shell utilities to monitor file system events
A Remote shell into ephemeral environments.
🔩 A relentless key-value store for the browser.
Data saved to ImmortalDB is redundantly stored in Cookies, IndexedDB, and LocalStorage, and relentlessly self heals if any data therein is deleted or corrupted.
provides a program and musical notation that you can use to easily connect and control sounds in a very human way.
Super useful if you want to learn how to do audio programming in Go.
If you want to get some background on audio processing with Go, check out this Audio From Scratch with Go too.
The Art of Code - Dylan Beattie
But what about the code that only exists because somebody wanted to write it? Code created just to make people smile, laugh, maybe even dance? Maybe even code that does nothing at all, created just to see if it was possible?
Microservices for Growth at comparethemarket.com
Kenichi Shibata and Adam Stewart explain their experience with the adoption of microservices in the creation of Comparison as a Service, one of the core pillars of the user journey today. They also go through PaaS and the necessity of building platforms in order to scale microservices efficiently.
An offline tool for querying IP geographic information and CDN provider.
100% user-mode endpoint security agent for macOS 10.15 and above, written in Swift.
a web based data science tool that works on top of your filesystem allowing you to use your editor of choice
The worklflow engine for Kubernetes
tcp connection hijacker, rust rewrite of shijack from 2001.
a simple, fast, website analytics alternative to Google Analytics.
That's it for this round, have a great day! If you like this newsletter, please tell the world, or
tweet about this