Rails Performance Tips

Richard Huang

@flyerhzm

Where to start

80-90% of the end-user response time is spent on the frontend

High Performance Web Sites by Steve Souders

Frontend Performance Tips

Keith and Mario’s Guide to Fast Websites
From RubyConf Au 2013

1. As less requests as possible

Before

Concatenate CSS & Javascript

Browser
Server
application.html
a.css
b.css
c.css
a.js
b.js
c.js
After

Concatenate CSS & JavaScript

Browser
Server
application.html
application.css
application.js
Before

Use Css Sprite

Browser
Server
application.html
a.jpg
b.jpg
c.jpg
After

Use Css Sprite

Browser
Server
application.html
sprite.jpg

2. As small payloads as possible

Minify and gzip

application.css (original) => application.css (minified) => application.css.gz
application.js (original) => application.js (minified) => application.js.gz

Compress Images

Before

Cookie-less Domains


Host:xinminlabs.com
Referer:https://xinminlabs.com
Cookie:rack.session=BAh7EUkiD3Nlc3Npb25faWQGOgZFVEkiRTJjMjI3NjEzYzQ3
MTZlMmYzZDky%0AMTEzOThkZDdmZjIyYjNiODBlYWI3ZmExYzU1MjRhZjYwN2FhMTBiM
GUwM2YG%0AOwBGSSIJY3NyZgY7AEZJIiUxYzlhMzhlZDBlM2VhNGJhNmFlZDM1MzFjYj
Ri%0AZWYxOQY7AEZJIg10cmFja2luZwY7AEZ7B0kiFEhUVFBfVVNFUl9BR0VOVAY7%0A
AFRJIi1mY2M3ZjA2OGU5N2Q5NzA2YTgwMDdmOTg3MjEwMjJjOThiMjdlYTIw%0ABjsAR
kkiGUhUVFBfQUNDRVBUX0xBTkdVQUdFBjsAVEkiLWRiY2MyMTZiMTBj%0AM2I1MmJkNT
g3ZTBjODIwNTZiMmIyYmE5YjJkNDYGOwBGSSIRYWNjZXNzX3Rv%0Aa2VuBjsAVEkidTA
wRDI4MDAwMDAwZnJFUSFBUU1BUUgxcV9MY2JRbG0yRXh5%0Ad3pqdm1PVjNMZGhNQS5D
SF8xaV94ZDM0X1c5czAwTDgzaHhJV0RaQTBWNW82%0AUFR1clE5cTdUWUtJOTZmUVBoS
2NfUjE3eklsQXp3NHcGOwBUSSIKZW1haWwG%0AOwBUSSIcZmx5ZXJoem0rcHJvZEBnbW
FpbC5jb20GOwBUSSILaWRfdXJsBjsA%0AVEkiSmh0dHBzOi8vbG9naW4uc2FsZXNmb3J
jZS5jb20vaWQvMDBEMjgwMDAw%0AMDBmckVRRUFZLzAwNTI4MDAwMDAxZHlrNkFBQQY7
AFRJIhFpbnN0YW5jZV91%0AcmwGOwBUSSIjaHR0cHM6Ly9saXRhLm15LnNhbGVzZm9yY
2UuY29tBjsAVEki%0ADXVzZXJuYW1lBjsAVEkiHGZseWVyaHptK3Byb2RAZ21haWwuY2
9tBjsAVEki%0ADmlzc3VlZF9hdAY7AFRJIhIxNDU0ODE2NTUxNzE3BjsAVEkiEWRpc3B
sYXlf%0AbmFtZQY7AFRJIhJSaWNoYXJkIEh1YW5nBjsAVEkiEnJlZnJlc2hfdG9rZW4G
%0AOwBUSSJcNUFlcDg2MVRTRVN2V2V1Z193R21UcEcucUlfdDNhbTBpd1JndjFr%0AVD
gyRV8yYjRiQTBMeXFWMzFoVU9Odk1SSEJGMk15V0hzbnZtb2RMVFRYVndr%0AU01UBjs
AVEkiF29yaWdpbmFsX2lzc3VlZF9hdAY7AEZAGw%3D%3D%0A--0e51b9ebf15a845e1f
17b20c0cc947cf8d9b7f60
          
After

Cookie-less Domains


Host:assets.xinminlabs.com
Referer:https://xinminlabs.com
          

3. As fast resources as possible

Use CDN

4. Other

JavaScript at the Bottom

Backend Performance Tips

http://www.speedawarenessmonth.com/when-8020-becomes-2080/

How

  • Don't guess! Don't guess! Don't guess!
  • Find out the bottleneck

Monitor

  • New Relic
  • Skylight

Monitor

Monitor

Monitor

Reproduce

  • Simulate the environment (postgres, memcached, etc.)
  • Simulate the data

Reproduce

  • New Relic (Developer Mode)
  • rack-mini-profiler

Reproduce

Reproduce

Reproduce

Reproduce

Users
Production Database
Production
Production
Production
Production
Production
New Relic Developer Mode

Fix

Benchmark

  • rails-perftest
  • ab / siege

rails-perftest


require 'test_helper'
require 'rails/performance_test_help'

class HomepageTest < ActionDispatch::PerformanceTest
  # Refer to the documentation for all available options
  # self.profile_options = { runs: 5,
  #                          metrics: [:wall_time, :memory],
  #                          output: 'tmp/performance',
  #                          formats: [:flat] }

  test "homepage" do
    get '/'
  end
end
          

rails-perftest


BrowsingTest#test_homepage (58 ms warmup)
        process_time: 63 ms
              memory: 832.13 KB
             objects: 7,882
          

ab / siege

Monitor with New Relic

Summarize

  • Monitor
  • Find out bottleneck
  • Reproduce
  • Fix
  • Benchmark
  • Deploy
  • Monitor

1. As less requests as possible

Before

N+1 Query


# controller
@comments = Comment.limit(10)

# view
<% @comments.each do |comment| %>
  <%= comment.user.username %> Said: <%= comment.body %>
<% end %>
          
Before

N+1 Query

App Server
DB Server
SELECT "comments".* FROM "comments" LIMIT 10
SELECT "users".* FROM "users" WHERE "users"."id" = 6 LIMIT 1
SELECT "users".* FROM "users" WHERE "users"."id" = 64 LIMIT 1
......
SELECT "users".* FROM "users" WHERE "users"."id" = 19 LIMIT 1
Before

N+1 Query

Newrelic response time is 12.3ms

After

N+1 Query


# controller
@comments = Comment.includes(:user).limit(10)

# view
<% @comments.each do |comment| %>
  <%= comment.user.username %> Said: <%= comment.body %>
<% end %>
          
After

N+1 Query

App Server
DB Server
SELECT "comments".* FROM "comments" LIMIT 10
SELECT "users".* FROM "users" WHERE "users"."id" IN (6, 64, 17, 56, 71, 2, 75, 73, 18, 19)
After

N+1 Query

Newrelic response time is 6.81ms

N+1 Query

12.3ms vs 6.81ms-45%
3.88ms vs 0.621ms-84%

N+1 Query

flyerhzm/bullet
Before

Counter Cache


# controller
@posts = Post.limit(10)

# view
<% @posts.each do |post| %>
  <%= post.title %> has <%= post.comment.size %> comments.
<% end %>
          
Before

Counter Cache

App Server
DB Server
SELECT "posts".* FROM "posts" LIMIT 10
SELECT COUNT(*) FROM "comments" WHERE "comments"."post_id" = 1
SELECT COUNT(*) FROM "comments" WHERE "comments"."post_id" = 2
......
SELECT COUNT(*) FROM "comments" WHERE "comments"."post_id" = 10
Before

Counter Cache

Newrelic response time is 12.3ms

After

Counter Cache


# migration
add_column :posts, :comments_count, :integer
Post.all.each do |post|
  Post.reset_counters(post.id, :comments)
end

# model
class Comment < ActiveRecord::Base
  belongs_to :post, counter_cache: true
end
          
After

Counter Cache

App Server
DB Server
SELECT "posts".* FROM "posts" LIMIT 10
After

Counter Cache

Newrelic response time is 7.04ms

Counter Cache

12.3ms vs 7.04ms-43%
6.434ms vs 0.511ms-92%

Counter Cache

flyerhzm/bullet

Bonus

magnusvk/counter_culture
Before

Use Group


# model
class Comment
  belongs_to :post
  scope :approved, -> { where(approved: true) }
end

class Post
  has_many :comments
  def average_rating
    comments.approved.average(:rating)
  end
end
          
Before

Use Group


# controller
@posts = Post.limit(10)

# view
<% @posts.each do |post| %>
  <%= post.title %> average rating is
  <%= post.average_rating %> stars
<% end %>
          
Before

Use Group

App Server
DB Server
SELECT "posts".* FROM "posts" LIMIT 10
SELECT AVG("comments"."rating") FROM "comments" WHERE "comments"."post_id" = 1 AND "comments"."approved" = "t"
SELECT AVG("comments"."rating") FROM "comments" WHERE "comments"."post_id" = 2 AND "comments"."approved" = "t"
......
SELECT AVG("comments"."rating") FROM "comments" WHERE "comments"."post_id" = 10 AND "comments"."approved" = "t"
Before

Use Group

Newrelic response time is 17.4ms

After

Use Group


# Gemfile
gem 'eager_group'

# model
class Post < ActiveRecord::Base
  has_many :comments
  define_eager_group :average_rating, :comments, :average, :rating,
-> { approved }
end

# controller
@posts = Post.eager_group(:average_rating).limit(10)
          
After

Use Group

App Server
DB Server
SELECT "posts".* FROM "posts" LIMIT 10
SELECT AVG("comments"."rating") AS average_rating, post_id AS post_id FROM "comments" WHERE "comments"."approved" = "t" AND "comments"."post_id" IN (1, 2, 3, 4, 5, 6, 7, 8, 9, 10) GROUP BY "comments"."post_id"
After

Use Group

Newrelic response time is 7.48ms

Use Group

17.4ms vs 7.48ms-57%
6.62ms vs 1.55ms-77%

Use Group

xinminlabs/eager_group

Bonus

Postgresql Materialized View
Before

Multi Inserts


# controller
10.times do
  Post.create title: Faker::Lorem.sentence,
              body: Faker::Hipster.paragraph,
              user_id: rand(100) + 1
end
          
Before

Multi Inserts

App Server
DB Server
BEGIN
INSERT INTO "posts" ("title", "body", "user_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id"
COMMIT
......
BEGIN
INSERT INTO "posts" ("title", "body", "user_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id"
COMMIT
Before

Multi Inserts

Newrelic response time is 35.4ms

After

Multi Inserts


# Gemfile
gem 'activerecord-import'

# controller
posts = []
10.times do
  posts << Post.new(title: Faker::Lorem.sentence,
                    body: Faker::Hipster.paragraph,
                    user_id: rand(100) + 1)
end
Post.import posts
          
After

Multi Inserts

App Server
DB Server
Class Create Many Without Validations Or Callbacks (3.6ms) INSERT INTO "posts" ("id","title","body","user_id") VALUES (nextval('public.posts_id_seq'),'title','body',31),(nextval('public.posts_id_seq'),'title','body',80),(nextval('public.posts_id_seq'),'title','body',73),(nextval('public.posts_id_seq'),'title','body',56),(nextval('public.posts_id_seq'),'title','body',66),(nextval('public.posts_id_seq'),'title','body',19),(nextval('public.posts_id_seq'),'title','body',11),(nextval('public.posts_id_seq'),'title','body',90),(nextval('public.posts_id_seq'),'title','body',90),(nextval('public.posts_id_seq'),'title','body',9) RETURNING id
After

Multi Inserts

Newrelic response time is 11.6ms

Multi Inserts

35.4ms vs 11.6ms-67%
19.39ms vs 2.39ms-88%

Multi Inserts

zdennis/activerecord-import

Multi Updates


Post.where("created_at < ?", 10.years.ago).update_all(archive: true)
          

Multi Deletes


Post.where("created_at < ?", 10.years.ago).destroy_all

Post.where("created_at < ?", 10.years.ago).delete_all
          

2. As small payloads as possible

Before

Select Data You Really Need


# controller
@posts = Post.limit(10)

# view
<% @posts.each do |post| %>
  <%= post.title %>
<% end %>
          
Before

Select Data You Really Need

App Server
DB Server
SELECT "posts".* FROM "posts" LIMIT 10
Before

Select Data You Really Need

Newrelic response time is 4.86ms

After

Select Data You Really Need


# controller
@posts = Post.select('title').limit(10)
          
After

Select Data You Really Need

App Server
DB Server
SELECT "posts"."title" FROM "posts" LIMIT 10
After

Select Data You Really Need

Newrelic response time is 4.54ms

Select Data You Really Need

4.86ms vs 4.54ms-7%
0.563ms vs 0.421ms-25%

3. As fast resources as possible

Before

Use Cache


# controller
@posts = Post.limit(10)

# view
<% @posts.each do |post| %>
  <%= post.user.username %> said <%= post.title %>
<% end %>
          
Before

Use Cache

App Server
DB Server
SELECT "posts".* FROM "posts" LIMIT 10
SELECT "users".* FROM "users" WHERE "users"."id" = 50
SELECT "users".* FROM "users" WHERE "users"."id" = 97
......
SELECT "users".* FROM "users" WHERE "users"."id" = 9
Before

Use Cache

Newrelic response time is 12.2ms

After

Use Cache


# Gemfile
gem 'dalli'

# config/application.rb
config.cache_store = :mem_cache_store

# view
<% @posts.each do |post| %>
  <% cache post do %>
    <%= post.user.username %> said <%= post.title %>
  <% end %>
<% end %>
          
After

Use Cache

App Servers
Database
Memcached
SELECT "posts".* FROM "posts" LIMIT 10
Post #50 Cache Page
Post #97 Cache Page
......
Post #9 Cache Page
After

Use Cache

Newrelic response time is 9.97ms

Use Cache

12.2ms vs 9.97ms-18%
8.11ms vs 5.79ms-29%
Bonus
Rails.cache.read_multi
https://github.com/n8/multi_fetch_fragments https://github.com/hooopo/second_level_cache

Use ElasticSearch for search

Use Redis

Use SSD

4. Other

Before

Use Index


# controller
@posts = Post.order('created_at desc').limit(10)

# view
<% @posts.each do |post| %>
  <%= post.title %>
<% end %>
          
Before

Use Index


SELECT  "posts".* FROM "posts"  ORDER BY created_at desc LIMIT 10
          
Before

Use Index

Newrelic response time is 21.5ms

After

Use Index


# migration
def change
  add_index :posts, :created_at
end
          
After

Use Index

Newrelic response time is 4.95ms

Use Index

21.5ms vs 4.95ms-77%
17.3ms vs 0.595ms-97%

Use Index

plentz/lol_dba
Before

Optimize JSON Rendering


# controller
@posts = Post.limit(10)
render json: @posts
          
Before

Optimize JSON Rendering

Newrelic response time is 6.28ms

After

Optimize JSON Rendering


# Gemfile
gem 'oj'
gem 'oj_mimic_json'
          
After

Optimize JSON Rendering

Newrelic response time is 5.76ms

Optimize JSON Rendering

6.28ms vs 5.76ms-8%
3.92ms vs 3.28ms-16%

Optimize JSON Rendering

ohler55/oj

Memory Optimize

https://github.com/rails/rails/pull/20946

shave off 1,114 string objects on every request

https://github.com/rails/rails/pull/21057

shave off 34,299 objects on every request

Find in batch


Person.where("age > 21").each do |person|
  person.party_all_night!
end
          
=>

Person.where("age > 21").find_each do |person|
  person.party_all_night!
end
          

Find in batch

  • Send multiple sql requests
  • But reduce memory usage

No tip is always true

Review

  • Frontend performance tuning does first
  • Backend performance tuning is important

Review

  • As less requests as possible
  • As small payloads as possible
  • As fast resources as possible
  • Other

Review

  • Monitor
  • Find out bottleneck
  • Reproduce
  • Fix
  • Benchmark
  • Deploy
  • Monitor

Demos Code

https://github.com/xinminlabs/rails-performance-tips-code

Thank You

Richard Huang

@flyerhzm