The blog of , a Ruby on Rails development team

Organizing large Rails projects with namespaces

This post is an excerpt from our e-book Growing Rails Applications in Practice, now available through The Pragmatic Programmers and Leanpub.


As a Rails application grows, so does its app/models folder. We've seen applications grow to hundreds of models. With an app/models directory that big, it becomes increasingly hard to navigate. Also it becomes near-impossible to understand what the application is about by looking at the models folder, where the most important models of your core domain sit next to some support class of low significance.

A good way to not drown in a sea of .rb files is to aggressively namespace models into sub-folders. This doesn't actually reduce the number of files of course, but makes it much easier to browse through your model and highlights the important parts.

Namespacing a model is easy. Let's say we have an Invoice class and each invoice can have multiple invoice items:

class Invoice < ActiveRecord::Base
  has_many :items
end

class Item < ActiveRecord::Base
  belongs_to :invoice
end

Clearly Invoice is a composition of Items and an Item cannot live without a containing Invoice. Other classes will probably interact with Invoice and not with Item. So let's get Item out of the way by nesting it into the Invoice namespace. This involves renaming the class to Invoice::Item and moving the source file to app/models/invoice/item.rb:

 class Invoice::Item < ActiveRecord::Base
   belongs_to :invoice
 end

What might seem like a trivial refactoring has great effects a few weeks down the road. It is a nasty habit of Rails teams to avoid creating many classes, as if adding another file was an expensive thing to do. And in fact making a huge models folder even larger is something that does not feel right.

But since the models/invoice folder already existed, your team felt encouraged to create other invoice-related models and place them into this new namespace:

File Class
app/models/invoice.rb Invoice
app/models/invoice/item.rb Invoice::Item
app/models/invoice/reminder.rb Invoice::Reminder
app/models/invoice/export.rb Invoice::Export

Note how the namespacing strategy encourages the use of service objects in lieu of fat models that contain more functionality than they should.

Real-world example

In order to visualize the effect that heavy namespacing has on a real-world-project, we refactored one of our oldest applications, which was created in a time when we didn't use namespacing.

Here is the models folder before refactoring:

activity.rb
amortization_cost.rb
api_exchange.rb
api_schema.rb
budget_calculator.rb
budget_rate_budget.rb
budget.rb
budget_template_group.rb
budget_template.rb
business_plan_item.rb
business_plan.rb
company.rb
contact.rb
event.rb
fixed_cost.rb
friction_report.rb
internal_working_cost.rb
invoice_approval_mailer.rb
invoice_approval.rb
invoice_item.rb
invoice.rb
invoice_settings.rb
invoice_template.rb
invoice_template_period.rb
listed_activity_coworkers_summary.rb
note.rb
person.rb
planner_view.rb
profit_report_settings.rb
project_filter.rb
project_link.rb
project_profit_report.rb
project_rate.rb
project.rb
project_summary.rb
project_team_member.rb
project_type.rb
rate_group.rb
rate.rb
revenue_report.rb
review_project.rb
review.rb
staff_cost.rb
stopwatch.rb
task.rb
team_member.rb
third_party_cost.rb
third_party_cost_report.rb
topix.rb
user.rb
variable_cost.rb
various_earning.rb
workload_report.rb

Looking at the huge list of files, could you tell what the application is about? Probably not (it's a project management and invoicing tool).

Let's look at the refactored version:

/activity
/api
/contact
/invoice
/planner
/report
/project
activity.rb
contact.rb
planner.rb
invoice.rb
project.rb
user.rb

Note how the app/models folder now gives you an overview of the core domain at one glance. Every single file is still there, but neatly organized into a clear directory structure. If we asked a new developer to change the way invoices work, she would probably find her way through the code more easily.

Use the same structure everywhere

In a typical Rails application there are many places that are (most of the time) structured like the models folder. For instance, you often see helper modules or unit tests named after your models.

When you start using namespaces, make sure that namespacing is also adopted in all the other places that are organized by model. This way you get the benefit of better organization and discoverability in all parts of your application.

Let's say we have a namespaced model Project::Report. We should now namespace helpers, controllers and views in the same fashion:

File Class
app/models/project/report.rb Project::Report
app/helpers/project/report_helper.rb Project::ReportHelper
app/controllers/projects/reports_controller.rb Projects::ReportsController
app/views/projects/reports/show.html.erb View template

Note how we put the controller into a Projects (plural) namespace. While this might feel strange at first, it allows for natural nesting of folders in in app/views:

app/
  views/
    projects/
      index.html.erb
      show.html.erb
      reports/
        show.html.erb

If we put the controller into a Project (singular) namespace, Rails would expect view templates in a structure like this:

app/
  views/
    project/
      reports/
        show.html.erb
    projects/
      index.html.erb
      show.html.erb

Note how two folders project (singular) and projects (plural) sit right next to each other. This doesn't feel right. We feel that the file organization of our views is more important than keeping controller namespace names in singular form.

Organizing test files

When we have tests we nest the test cases and support code like we nest our models. For instance, when you use RSpec and Cucumber, your test files should be organized like this:

File Description
spec/models/project/report_spec.rb Model test
spec/controllers/projects/reports_controller_spec.rb Controller test
features/project/reports.feature Cucumber untegration test
features/step_definitions/project/report_steps.rb Step definitions

Other ways of organizing files

Of course models/controllers/tests don't always map 1:1:1, but often they do. We think it is at the very least a good default with little ambiguity. When you look for a file in a project structured like this, you always know where to look first.

If another way to split up your files feels better, just go ahead and do it. Do not feel forced to be overly consistent, but always have a good default.

Rails LTS: Supported version changes

Rails LTS is a commercially supported fork of Ruby on Rails that provides security patches for legacy Rails releases. Today we're announcing some changes in the versions that we plan to support in 2015 and beyond.

Rails 2.3

We will continue to support Rails 2.3 indefinitely.

Rails 3.0

When we announced support for Rails 3.0 earlier this year, we had hoped to create a stable customer base to fund long-term maintenance of Rails 3.0. Unfortunately we haven't been able to gather much interest for Rails LTS 3.0 despite our best efforts, so with a heavy heart we're announcing plans to sunset support for this version.

In order to give 3.0 users sufficient time to upgrade, we're going to support Ruby on Rails 3.0 for another full year until January 1st, 2016. After that date we will no longer be able to provide security patches for Rails 3.0.

Rails 3.2

We plan to support Rails 3.2 when official maintenance ends entirely, sometime in 2015 (Rails 3.2 still receives limited maintenance for severe security issues). We could already win an enterprise customer for this project, which allowed us to start working on 3.2 support.

Rails 3.2 support will be available to all Rails LTS customers on a Startup, Standard or Enterprise plan.

Now available: Growing Rails Applications in Practice

It's finally done! We just released version 1.0 of our e-book Growing Rails Applications in Practice.

With this book we hope to provide pragmatic advice for scaling large, monolithic Rails codebases. If you've been looking for actionable, low-ceremony techniques to improve your application today, this is your book.

You can pick your own price for this book (there's a minimum price of $10). Let us know if you have questions or feedback!

Swift is not my favorite language

I really wanted to like Swift. In a world that has settled on Javascript, an ambitious language design is a rare treat. And all the money we gave to Apple should buy us a decent language.

So five years after iOS has killed Adobe Flash, Apple releases a language that looks and feels remarkably like ActionScript 3. Just like ActionScript, Swift is locked behind a $1000 paywall. Instead of bindings to proprietary stuff from Adobe it has bindings to proprietary stuff from Apple.

Some observations from my armchair, in no particular order:

Generics: It will be so much fun to watch a generation of Internet hipsters crash into the wall of complexity that is generics. The survivors will learn that you can't have an expressive type system without being Scala. And I'm not sure you want to be Scala.

Explicit null: Language tourists all over the planet are praising Apple for including optional values. This is such a horrible solution to a much deeper problem. Remember how when Java introduced checked exceptions and everyone said this would force people to handle their errors? What it got us was was empty catch clauses and everyone going back to unchecked exceptions. The same fallacy has now brought you optional values.

You can't make this issue go away with language design. You don't have a language problem, you have an API problem and an education problem. Why did anyone feel it was OK to pass or return null? The clueless developer who passed you the null in the first place will now pepper your code with exclamation marks to make the funny compiler errors go away. Enjoy.

Inconsistencies: So a language that prides itself on not doing implicit conversions (because safety) does auto-unwrap arrays during concatenation. This will ruin so many lives.

Name conflict: What really gets my blood boiling is that there already is a programming language that is called Swift. Imagine waking up one morning only to discover that someone with more money took your name and will bully you out of Google forever. I had this happen to me and it sucks. Open Source maintenance is the most thankless job on the planet and this is just a big fuck you in the face of everyone who spends their weekends working on free software.

I don't know what's more infuriating: That Apple couldn't be bothered to run a stupid Google search before deciding on the name or that they did run that search and went ahead anyway. There's still one classy move they can take to fix things, but I doubt that the issue even registers when you're sitting on a mountain of money.

In a nutshell:

  • There are now two languages called Swift
  • One has semicolons
  • Both are better than Objective-C

Our new e-book: Growing Rails Applications in Practice

Find below the opening chapter of our new e-book: Growing Rails Applications in Practice. The complete book will be published this summer. Enjoy!


How we got here

When you started doing Rails some years ago, it all seemed so easy. You saw the blog-in-ten-minutes video. You reproduced the result. ActiveRecord felt great and everything had its place.

Fast forward two years. Your blog is now a full-blown CMS with a hundred models and controllers. Your team has grown to four developers. Every change to the application is a pain. Your code feels like a house of cards.

You turn to the internet for assistance, and many solution seem to be available. You should move away from fat controllers, it says. But do avoid fat models. And use DCI. Or CQRS. Or SOA. As you cycle through patterns, your application is becoming a patchwork of different coding techniques. New team members are having a hard time catching up. And you're beginning to question if all those new techniques actually help, or if you're just adding layers of indirection.

You start missing the early days, when everything had seemed so easy and every new piece of code had its place. You actually liked ActiveRecord before it drowned you in a sea of callbacks. If only there was a way to do things "the Rails way" without having it fall apart as your application grows.

The myth of the infinitely scalable architecture

We'd like to show you one path to write Rails apps that are a joy to understand and change, even as your team and codebase grows. This book describes a complete toolbox that has served us well for all requirements that we have encountered.

But before we do that, we need to let you in on an inconvenient secret: Large applications are large. The optimal implementation of a large application will always be more complex than the optimal representation of a smaller app. We cannot make this go away. What we can do is to organize a codebase in a way that "scales logarithmically". Twice as many models should not mean twice as many problems.

Here is a chart that we made up:

Vanilla Rails vs. structured Rails

In order to walk the green line in the chart above, you don't necessarily need to change the way your application is built. You don't necessarily need to introduce revolutionary architectures to your code. You can probably make it with the tools built into Rails, if you use them in a smarter way.

Compare this to sorting algorithms. When a sorting function is too slow, our first thought is not "install a Hadoop cluster". Instead we simply look for an algorithm that scales better. In a similar fashion this book is not about revolutionary design patterns or magic gems that make all your problems go away. Instead we will show how to use discipline, consistency and organization to make your application grow more gently.

We have three principles that guide our design decisions:

1. Be boring

We have a universally applicable default design that maps any user-faced interaction to CRUD and ActiveModel. By being consistent in your design decisions we can reduce the cognitive overhead required to follow your code. This makes it easier for colleagues and new developers to navigate and change our source.

Read about this principle in the following sections:

  • Why controllers should be boring
  • Using ActiveRecord effectively
  • User-facing models without a database: ActiveModel
  • Taming style sheets

2. Be organized

We organize code in a way that encourages the creation of new classes. By isolating screen-specific concerns from a slim core model we can reduce the side effects that models can have on other parts of your application.

Read about this principle in the following sections:

  • Dealing with fat models
  • Taming style sheets
  • Taming javascripts
  • Organizing large codebases using namespaces
  • Don't let authorization infect your application

3. Be humble

Where we suggest using before/after code comparisons to avoid being sidetracked by the design fad of the day. We will make a case for adopting new technologies and patterns with care, and take full responsibility for those techniques and technologies you do choose to adopt.

  • How to test effectively
  • Hipster Patterns
  • Owning your stack
  • Surviving the upgrade pace of Rails

Are you interested in this book?

We're currently finishing up the final chapters. Growing Rails Applications in Practice will be released this summer. If you're interested in the book, please enter your e-mail address into our notification form. We will let you know as soon as we publish!

Growing Rails Applications in Practice
Check out our new e-book:
Learn to structure large Ruby on Rails codebases with the tools you already know and love.
Our address:
makandra GmbH
Werner-von-Siemens-Str. 6
86159 Augsburg
Germany
Contact us:
+49 821 58866 180
info@makandra.de
Commercial register court:
Augsburg Municipal Court
Register number:
HRB 24202
Sales tax identification number:
DE243555898
Chief executive officers:
Henning Koch
Thomas Eisenbarth