Read the Docs has been building and hosting documentation for open source projects since 2010. We’re proud to be a part of the open source ecosystem, and today we wanted to take a moment to thank one of our biggest supporters: Amazon Web Services.

We have been a member of the AWS Open Source Credits program since 2021. In that time, they have provided us with almost $500,000 in credits, which covers all of our server-related costs each year for the Community platform.

This post will cover a bit more about how we use AWS, the infrastructure we run on it, and how it helps us support the open source projects that rely on Read the Docs.

Our scale

In the past 30 days, our Community hosting platform has:

• Supported over 100,000 open source projects with documentation hosting
• Run over 350,000 documentation builds
• Handled over 250M documentation pageviews
• Served over 2.2 billion web requests
• Transferred over 45TB of data

Since our creation in 2010, we have been a cornerstone of the Python open source ecosystem, and now host documentation for many other languages and frameworks as well.

Our infrastructure

Read the Docs has a pretty simple architecture, because we have a small team of 5 people running a very large web service. The major AWS services that we rely on are:

• EC2: We run our web servers, web workers, and build servers on EC2 autoscaling groups. This allows us to scale up and down as needed, leading to builders always being available when projects need them.
• S3: We store all of our documentation artifacts in S3. This includes the HTML files, PDFs, and other outputs generated by our build system.
• RDS: Our primary and replica Postgres databases are hosted on RDS. This allows us to scale our database easily, and we rarely have to think about it.

We use a number of other AWS services as well, including Cloudwatch for monitoring, Route 53 for DNS, and many more.

AWS has enabled a better architecture

Before we moved to AWS, we were running on dedicated servers. This meant that every build server and web server was a fixed resource, and we had to manually add and remove servers as needed. This led to a lot of wasted resources during off-peak times, and inability to scale up quickly during load spikes.

To spin up a new web server, we'd have to copy hundreds of gigabytes of data from an existing server to a new one, which could take hours. Moving to AWS has allowed us to migrate this data into S3, and now servers can start up in seconds instead of hours.

Similarly, our build servers now autoscale based on demand. We used to have to sit idly and watch our build queue grow during peak times, but now we know that the load will be handled automatically within a few minutes.

This has led to a much better experience for our users, and allowed us to consider projects like pull request builds which we couldn't have supported before. Having pull request builds has increased our build demand, and requires more flexibility in our hosting infrastructure to expire them 30 days after the pull request is closed.

Read the Docs is a critical component of the open source software ecosystem. And their PR builds are the best and most trustworthy implementation out there.

Matt Fisher

Scientific Software Developer

UC Berkeley Eric and Wendy Schmidt Center for Data Science and Environment

A few of the projects we host

Here are just a few of the big projects whose docs we build and host:

• Godot Engine – the open source 2D/3D game engine behind thousands of indie titles and student projects.
• Jupyter – the open source project that powers Jupyter Notebooks, used by millions of data scientists and educators worldwide.
• Flask – the Python web framework powering everything from hobby APIs to production workloads at scale.
• Python Developers Guide - the official guide for developing Python itself, maintained by the core developers.
• Tens of thousands of other open source libraries. Browse the documentation of the libraries, software, and services you use and it won't take long to find a project whose docs we host.

Read the Docs plays a large part in enabling a small volunteer team to maintain our docs. Without the generous Community plan, we couldn't have afforded anything close to the CI/build resources that we got with Read the Docs.

Max Hilbrunner

Software and Game Developer

Godot Engine

Thanks to AWS for supporting open source

Keeping all this documentation online, building every commit, storing every artifact, and serving content to the entire world—costs a lot of money. We have been given almost $500,000 in credits from AWS since 2021, which covers the lion’s share of our Community infrastructure bill each year.

These credits are important for us in a few major ways:

• They support open source by allowing us to give generous free tiers to any open source project, because we don’t have to worry about the costs of hosting.
• They support our team, because it lets us focus on building features that matter to our users instead of worrying about trying to cobble together funding for our infrastructure.
• They provide a better user experience, because we can use autoscaling builders, instead of a fixed pool of donated builders.

Without partners like AWS, our Community site wouldn't be the reliable home for documentation that it is today.