Etsy Payment system reached the point where vertical scale won’t work anymore. They used biggest instance on GCP. They migrated this to Vitess. They have to change schema and update data model and roll out in 2 phases and share with us how they do in this article
Every day Refind picks 5 links from around the web that make you smarter, tailored to your interests. Refind is a must-read newsletter loved by 100,000+ curious minds. Sign up for free today
As CPU cores become both faster and more numerous, the limiting factor for most programs is memory access. Hardware designers have come up with ever more sophisticated memory handling and acceleration techniques–such as CPU caches–but these cannot work optimally without some help from the programmer. Unfortunately, neither the structure nor the cost of using the memory subsystem of a computer or the caches on CPUs is well understood by most programmers. This paper explains the structure of memory subsystems in use on modern commodity hardware, illustrating why CPU caches were developed, how they work, and what programs should do to achieve optimal performance by utilizing them.
DataDog handle massive amount of data so they build a system to redesign how they ingest and store log call Husky. Husky’s architecture decouples storage and compute so that they can be scaled independently. In this post we learn how Husky organized its data and establish consensus.
we could build a simple Virtual Machine (VM) with no dynamic linking and a small set of minimalistic APIs that remain stable over time, it would make it a lot easier to build software without worrying about becoming randomly broken by changing APIs or dependencies. The author build such a VM call UVM and share with us here.
Consistent hashing is a technique to associate a resource with a set a nodes evenly, but also support adding new nodes into system without reallocating resource around.
discussed how roles (users and groups) are created and managed in PostgreSQL Depending on your background with permissions, particularly in other database products, some of those nuances in how permissions work may have been surprising.
I always wonder how those walk through, fly through or 3d object model are being build. This is an end to end tutorial to show you the entire process of implement JS, CSS code and build the animation
In JavaScript, its very easy to create object with no extra syntax: just do a {foo: 1}
. But turning out there is another better and fast way: const o = new Map();o.set('foo', 1)
. Map is optimize for adding/removing key frequently
A walk-through of multiple tools that can be very useful when audit and analyse a network system. This is the step call analyze footprint of any system to find relevant components. By running these tools on your own network can yield interesting result sometimes.
A very good way to learn and build reliable distributed system. The famous Kyle of Jepsen team up with fly.io to build a challenge, where you implement the system to communicate with a pre-build server follow a protocol. Your program needs to communicate the right way to pass the challenge.
Skillshare is a platform that offers thousands of classes in a variety of subjects. From creative arts to business and technology, there’s something for everyone. Expert instructors guide students every step of the way and a community of passionate learners provides support and inspiration. Whether you’re looking to learn something new or enhance your existing skills, Skillshare has something for you. Explore the platform today and start your learning journey.
A chrome extension that mount your browser tabs as a file system and you simply interact with part of it like the title, close tab etc. Read the companion post here
JavaScriptFusegenerate a pre-built authentication system into a rails application (web or api-only) that follows both security and rails best practices. By generating code into the user’s application instead of using a library, the user has complete freedom to modify the authentication system so it works best with their app.
Rubya DOM-only, super-fast, uber-tolerant XSS sanitizer for HTML, MathML and SVG. When scraping site or need to display a certain user input such as email or html in browser, this come in handy.
JavaScriptA project that build a queue on Postgres, but expose extra SQL function where you call in to queue/enqueue job. Its API is similar to AWS SQS. I love this kind of code base, I got to learn Rust, learn how to write extension with pgx, a tool to write Postgres extension in Rust, and can also evaluate to use this tool for my lightweight queue in the future
RustA tasty, self-hostable Git server for the command line. You SSH into it and browse it through a TUI (terminal based user interface). You can even click on it too, in a terminal just like a normal remote SSH session.
Web tool for database management
SQLite Foreign Data Wrapper for PostgreSQL. This means that PostgreSQL now can access and join data with another SQLite database.
tream Arrow file data into Postgres. Tools like BigQuery, RedShift already support Arrow format due to their compression, easiser to parse and fast.
Cross-platform client for PostgreSQL databases
a webui to manage kafka jobs, consumers, routing, cluster info
a simple CLI tool that contains everything you need to start creating Alerts. builtin providers (e.g. sentry/datadog or slack/pagerduty), 100% open sourced.
a high-throughput generation engine for running large language models with limited GPU memory (e.g., a 16GB T4 GPU or a 24GB RTX3090 gaming card!). FlexGen allows high-throughput generation by IO-efficient offloading, compression and large effective batch sizes.
BetterDev Link
Every Monday