Digests for the book "Scalability Rules: 50 Principles for Scaling Web Sites"

50 Principles for Scaling Web Sites

Martin L. Abbott & Michael T. Fisher

 Reduce the Equation
 Distribute your work
 Design to scale out horizontally
 Use the right tools
 Don't duplication your work
 Use caching aggressively
 Learn from your mistakes
 Database rules
 Design for fault tolerance and graceful failure
 Avoid or distribute state
 Asynchronous communication and message buses
 Miscellaneous rules

 Any big problem, if approached properly, is
really just a collection of smaller problems
waiting to be solved.
 Makes big architectural problems smaller and
doing less work while still achieving the
necessary business results.
 Complexity is the enemy of scalability.

Rule 1 - Don’t Overengineer the Solution

 Systems that are overly complex limit your
ability to scale. Simple systems are more
easily and cost effectively maintained and
scaled.

Reduce the Equation


 What
 Guard against complex solutions during design.
 When
 Can be used for any project and should be used for all
large or complex systems or projects.
 How
 Resist the urge to overengineer solutions by testing
ease of understanding with fellow engineers.
 Why
 Complex solutions are costly to implement and have
excessive long-term costs.
Reduce the Equation


 Products designed and implemented to
exceed the useful requirements of the
product (aka scope creep).
 higher cost to develop (engineer) the solution
 higher cost to implement the solution in hardware
and software
 impact the company by delaying the product
launch
 Products that are made to be overly complex.
Reduce the Equation


 Making something work harder to get a job
done than is necessary
 Making a user work harder to get a job done
than is necessary
 Making an engineer work harder to
understand something than necessary

Reduce the Equation


 Simply wastes money and time
 Wastes processing resources
 Increases the cost of scale
 Limit the overall scalability of the system
(how far that system can be scaled)

Reduce the Equation


 Easy to follow solutions mean that less senior
engineers can more quickly come up to speed
to support systems.
 Easy to understand solutions mean that
problems can be found more quickly during
troubleshooting, and systems can be restored
to their proper working order in a faster
manner.

Reduce the Equation

Rule 2 - Design Scale into the Solution (D-I-D Process)

 Teams can save a lot of money and time by
thinking of how to scale solutions early,
implementing (coding) them a month or so
before they are needed, and implementing
them days before the customer rush or
demand.

Reduce the Equation


 What
 An approach to provide JIT (Just In Time) Scalability.
 When
 On all projects; this approach is the most cost effective
(resources and time) to ensure scalability.
 How
 Design for 20x capacity.
 Implement for 3x capacity.
 Deploy for ~1.5x capacity.
 Why
 D-I-D provides a cost effective, JIT method of scaling your
product.

Reduce the Equation


 You should invest (and deploy) the day before
the solution is needed.
 But let’s face it—timing such an investment
and deployment “just in time” is simply
impossible.
 The next best thing to investing and
deploying “the day before” is AKF Partners’
Design-Implement-Deploy or D-I-D approach
to thinking about scalability.
Reduce the Equation


Design Implement Deploy
Scale Objective 20x to Infinite 3x to 20x 1.5x to 3x

Intellectual Cost High Medium Low to Medium

Engineering Cost Low High Medium

Asset Cost Low Low to Medium High to Very High

Total Cost Low/Medium Medium Medium

Reduce the Equation

Rule 3 - Simplify the Solution 3 Times Over

 Simplification needs to happen during every
aspect of product development.

Reduce the Equation


 What
 Used when designing complex systems, this rule simplifies the scope,
design, and implementation.
 When
 When designing complex systems or products where resources
(engineering or computational) are limited.
 How
 Simplify scope using the Pareto Principle.
 Simplify design by thinking about cost effectiveness and scalability.
 Simplify implementation by leveraging the experience of others.
 Why
 Focusing just on “not being complex” doesn’t address the issues
created in requirements or story and epoch development or the actual
implementation.

Reduce the Equation


 Apply the Pareto Principle (also known as the
80-20 rule) frequently.
 With 4/5ths fewer features, your system will
no doubt have fewer dependencies between
functions and as a result will be able to scale
both more effectively and cost effectively.

Reduce the Equation


 Complexity elimination is about doing less
work, and design simplification is about doing
that work faster and easier.
 Put simply, the step of design simplification
asks us how to get the job done in an easy to
understand, cost-effective, and scalable way.

Reduce the Equation


 Leverage the experiences of others and existing
solutions
 look to find widely adopted open source or third-party
solutions to meet our needs
 look to see if someone within our own organization has
developed a scalable solution to solve the problem
 look externally to see if someone has described a scalable
approach to solve the problem that we can legally copy or
mimic
 Only in the absence of finding one of these three
things should we embark on attempting to solve the
solution ourselves.
Reduce the Equation

Rule 4 - Reduce DNS Lookups

 Reduction of objects, tasks, computation,
and so on is a great way of speeding up page
load time, but division of labor must be
considered as well.

Reduce the Equation


 What
 Reduce the number of DNS lookups from a user
perspective.
 When
 On all Web pages where performance matters.
 How
 Minimize the number of DNS lookups required to
download pages, but balance this with the browser’s
limitation for simultaneous connections.
 Why
 DNS lookups take a great deal of time, and large numbers
of them can amount to a large portion of your user
experience.
Reduce the Equation


 Depending on the freshness of DNS cache in our
browser, operating system, and so on, each DNS
lookup can take essentially no time up to
hundreds of milliseconds.
 As a general rule, the fewer DNS lookups on
your pages the better your page download
performance will be.
 There is a downside to combining all your
objects into a single domain (maximum
simultaneous connects.)
Reduce the Equation

Rule 5 - Reduce Objects Where Possible

 The balance between objects and methods
that serve them is a science that requires
constant measurement and adjustment; it’s a
balance between customer usability,
usefulness, and performance.

Reduce the Equation


 What
 Reduce the number of objects on a page where
possible.
 When
 On all web pages where performance matters.
 How
 Reduce or combine objects but balance this with
maximizing simultaneous connections.
 Test changes to ensure performance improvements.
 Why
 The number of objects impacts page download times.

Reduce the Equation


 Reduce the number of objects on a page.
 The biggest offenders on most pages are
graphical objects such as pictures and images.
 An image sprite is a combination of small
images into one larger image that can be
manipulated with CSS to display any single
individual image.

Reduce the Equation


 The simultaneous connection feature of a
browser is a limit ascribed to each domain
that is serving the objects.
 One technique to really take advantage of
this browser feature is to serve different
objects from different subdomain.

Reduce the Equation


 In summary, the fewer the number of objects
on a page the better for performance, but
this must be balanced with many other
factors
 the amount of content that must be displayed
 how many objects can be combined
 how to maximize the use of simultaneous
connections by adding domains
 the total page weight
 whether penalization can help, and so on.
Reduce the Equation

Rule 6 - Use Homogenous Networks

 Heterogeneous networking gear tends to
cause availability and scalability problems.
Choose a single provider.

Reduce the Equation

Rule 6 - Use Homogenous Networks

 What
 Don’t mix the vendor networking gear.
 When
 When designing or expanding your network.
 How
 Do not mix different vendors’ networking gear
(switches and routers).
 Buy best of breed for other networking gear (firewalls,
load balancers, and so on).
 Why
 Intermittent interoperability and availability issues
simply aren’t worth the potential cost savings.
Reduce the Equation

 Cloning and replication
 Separating functionality or services
 Splitting similar data sets across storage and
application systems

Digests for the book "Scalability Rules: 50 Principles for Scaling Web Sites"

Rule 7 - Design to Clone Things (X Axis)

 X axis splits are fast to implement and can
allow for transaction, but not data scalability.

Distribute Your Work


 What
 Typically called horizontal scale, this is the duplication of
services or databases to spread transaction load.
 When
 Databases with a very high read to write ratio (5:1 or greater—
the higher the better).
 Any system where transaction growth exceeds data growth.
 How
 Simply clone services and implement a load balancer.
 For databases, ensure the accessing code understands the
difference between a read and a write.
 Why
 Allows for fast scale of transactions at the cost of duplicated
data and functionality.


 The hardest part of a system to scale is
almost the database or persistent storage tier.
 Most applications depend on the database to
support and enforce the relationships
because of its ACID properties.
 One technique for scaling databases is to
take advantage of the fact that most
applications and databases perform
significantly more reads than writes.
 read-only copies (replicas)


 Time sensitivity is how fresh or completely
correct the read-copy has to be relative to the
write copy.
 While perfectly in sync data is ideal, it
costs…a lot. Furthermore, it doesn’t always
give you the return that you might expect or
desire for that cost.
 you can/should push back on the idea that all
data has to be kept in sync in real time.


 Use a caching tier in front of the database.
 Replicating the database (such as slaves in
master/slave concept of MySQL).



 The X axis (horizontal duplication) can be
applied to more than just the database.
 Web servers and application servers typically
can be easily cloned.
 Unfortunately, it doesn't really help us when
trying to scale the data we must manipulate
to perform transactions.


Rule 8 - Design to Split Different Things (Y Axis)

 Y axis or data/service-oriented splits, allow
for efficient scaling of transactions, large data
sets, and can help with fault isolation.



 What
 Sometimes referred to as scale through services or resources, this rule
focuses on scaling data sets, transactions, and engineering teams.
 When
 Very large data sets where relations between data are not necessary.
 Large, complex systems where scaling engineering resources requires
specialization.
 How
 Split up actions by using verbs or resources by using nouns or use a
mix.
 Split both the services and the data along the lines defined by the
verb/noun approach.
 Why
 Allows for efficient scaling of not only transactions, but very large data
sets associated with those transactions.



 Both concepts of SOA and ROA force
architects and engineers to think in terms of
separation of responsibilities within their
architectures.
 At a high and simple level, they do this
through the concepts of verbs (services) and
nouns (resources).
 Rule 8 is about scaling through the separation
of distinct and different functions and data
within your site.


 While Y axis splits are most useful in scaling
data sets, they are also useful in scaling code
bases.
 This means that very large engineering teams
developing complex systems can become
experts in subsets of those systems and don’t
need to worry about or become experts on
every other part of the system.


Rule 9 - Design to Split Similar Things (Z Axis)

 Z axis splits are effective at helping you to
scale customer bases but can also be applied
to other very large data sets that can’t be
pulled apart using the Y axis methodology.



 What
 This is very often a split by some unique aspect of the customer
such as customer ID, name, geography, and so on.
 When
 Very large, similar data sets such as large and rap-idly growing
customer bases.
 How
 Identify something you know about the customer, such as
customer ID, last name, geography, or device and split or
partition both data and services based on that attribute.
 Why
 Rapid customer growth exceeds other forms of data growth or
you have the need to perform fault isolation between certain
customer groups as you scale.



 Often referred to as sharding and podding
 Taking one data set or service and
partitioning it into several pieces.
 These pieces are often equal sized but may
be of different sizes if there is value in having
several unequal sized chunks or shards.
 One reason to have unequal sized shards is to
enable application rollouts that limit your risk by
affecting first a small customer segment.


 Often sharding is accomplished by separating
something we know about the requestor or
customer.
 customer ID, last name, geography, or device
 recency, frequency, and monetization of
customers' purchases.
 product catalogs
 transaction ID


 Scaling up is failing up.
 When You hit the limitation of the fastest and
biggest system provided by the most costly
provider of the system in question, you are in
big trouble.
 You should design your systems to scale
horizontally, or out, rather than up.

Rule 10 - Design Your Solution to Scale Out-Not Just Up

 Plan for success and design your systems to
scale out. Don’t get caught in the trap of
expecting to scale up only to find out that
you’ve run out of faster and larger systems to
purchase.

Design to Scale Out Horizontally


 What
 Scaling out is the duplication of services or databases to
spread transaction load and is the alternative to buying
larger hardware, known as scaling up.
 When
 Any system, service, or database expected to grow rapidly.
 How
 Use the AKF Scale Cube to determine the correct split for
your environment. Usually the horizontal split (cloning) is
the easiest.
 Why
 Allows for fast scale of transactions at the cost of
duplicated data and functionality.


 Scale up
 Total capital expenditures increase significantly.
 Eventually you will get to a point that either the
cost becomes uneconomical or there is no bigger
hardware made.
 Scale out
 The costs to solve the problem with engineering
resources will also likely increase, but this cost
should be linear.


 If the application has to keep track of state, a
possible solution is to allow session cookies
from the load balancer to maintain affinity
between a customer’s browser and a
particular application server.


Rule 11 - Use Commodity Systems

 Build your systems to be capable of relying
on commodity hardware and don’t get
caught in the trap of using high-margin, high-
end servers.



 What
 Use small, inexpensive systems where possible.
 When
 Use this approach in your production environment
when going through hyper growth.
 How
 Stay away from very large systems in your
production environment.
 Why
 Allows for fast, cost-effective growth.


 You are paying more on a CPU basis, but
actually doing less per CPU. You are getting
nailed twice!
 It might, in fact, be the case that the larger
system will draw less power and save you
money.
 We can find the right system for us that still
fits the commodity range and maximizes the
intersection of computing power with
minimal power and space requirements


 There are many reasons to virtualize, but
greater system capacity in a larger domained
system as compared to smaller equivalently
sized systems is a fallacy and is not one of
them.


Rule 12 - Scale Out Your Data Centers

 When implementing disaster recovery, lower
your cost of disaster recovery by designing
your systems to leverage three or more live
data centers. Use the spare capacity for spiky
demand when necessary.



 What
 Design your systems to have three or more live data centers to reduce
overall cost, increase availability, and implement disaster recovery.
 When
 Any rapidly growing business that is considering adding a disaster
recovery (cold site) data center.
 How
 Split up your data to spread across data centers and spread
transaction load across those data centers in a “multiple live”
configuration. Use spare capacity for peak periods of the year.
 Why
 The cost of data center failure can be disastrous to your business.
Design to have three or more as the cost is often less than having two
data centers. Make use of idle capacity for peak periods rather than
slowing down your transactions.



 Segmentation
 Y and Z axes
 If we have sliced our data properly along the Z axis,
we can now potentially locate data closer to the
users requesting that data.
 Replication and cloning of data and services
 Stateless



Site Configuration Network Servers Databases Storage Network Site Total Cost
Connections
Single Site 100% 100% 100% 100% 0 100%
2 Site "Hot" and 200% 200% 200% 200% 1 200%
"Cold"
2 Site Live/Live 200% 200% 200% 200% 1 200%
3 Site 150% 150% 150% 200% 3 ~166%
Live/Live/Live



 Higher availability as compared to a hot and cold site
con-figuration
 Lower costs compared to a hot and cold site
configuration
 Faster customer response times if customers are
routed to the closest data center for dynamic calls
 Greater flexibility in rolling out products in a SaaS
environment
 Greater confidence in operations versus a hot and
cold site configuration
 Fast and easy “on-demand” growth for spikes using
spare capacity in each data center

Rule 13 - Design to Leverage the Cloud

 Design to leverage virtualization and the
cloud to meet unexpected spiky demand.



 What
 This is the purposeful utilization of cloud technologies to scale on
demand.
 When
 When demand is temporary, spiky, and inconsistent and when
response time is not a core issue in the product.
 How
 Make use of third-party cloud environments for temporary demands,
such as large batch jobs or QA environments during testing cycles.
 Design your application to service some requests from a third-party
cloud when demand exceeds a certain peak level.
 Why
 Provisioning of hardware in a cloud environment takes a few minutes
as compared to days or weeks for physical servers in your own
collocation facility. When utilized temporarily this is also very cost
effective.


 Customers are allowed to spin up or start
using one of these virtual servers and are
typically charged by how long they use it.
 Pricing is different for each of the vendors
providing these services, but typically the
break-even point for utilizing a virtual server
versus purchasing a physical server is around
12 months.
 Provisioning of the hardware typically takes
minutes.


 When demand is either temporary or
inconsistent.
 batch processing
 testing environment
 surge capacity


 When all you have is a hammer, everything
looks like a nail.

Rule 14 - Use Database Appropriately

 Use the right storage tool for your data. Don't
get lured into ticking everything in a
relational database just because you are
comfortable accessing data in a database.

Use the Right Tools


 What
 Use relational databases when you need ACID properties to
maintain relationships between your data. For other data
storage needs consider more appropriate tools.
 When
 When you are introducing new data or data structures into the
architecture of a system.
 How
 Consider the data volume, amount of storage, response time
requirements, relationships, and other factors to choose the
most appropriate storage tool.
 Why
 An RDBMS provides great transactional integrity but is more
difficult to scale, costs more, and has lower availability than
many other storage options.
Use the Right Tools


File Systems NoSQL RDBMS
Embedded Low Medium High
Relationships
Flexibility Low Medium High
Cost of Scale Low Medium High
Limits to Low Medium High
Scale

Use the Right Tools


 The number of degree of relationships
needed between elements.
 The rate of growth of the solution.
 The ratio of reads to writes of the data (and
potentially whether data is updated).
 How well the data monetizes?

Use the Right Tools


Use the Right Tools


 Monetization value of the data is critical to
understand because as many struggling
startups have experienced, storing terabytes
of user data for free on class A storage is a
quick way to run out of capital. A much better
approach might be using tiers of data storage;
as the data ages in terms of access date,
continue to push it off to cheaper and slower
access storage media.
Use the Right Tools

Rule 15 - Firewalls, Firewalls, Everywhere!

 While firewalls are useful, they are often
overused and represent both an availability
and scalability concern if not designed and
implemented properly.

Use the Right Tools


 What
 Use firewalls only when they significantly reduce risk
and recognize that they cause issues with scalability
and availability.
 When
 Always.
 How
 Employ firewalls for critical PII, PCI compliance, and
so on. Don’t use them for low-value static content.
 Why
 Firewalls can lower availability and cause unnecessary
scalability chokepoints.
Use the Right Tools


 Capital cost to purchase the firewalls.
 An additional item that needs to be scaled.
 an impact to our availability as it is one more
device on the critical route of any transaction.

Use the Right Tools


 Don’t assume that everything deserves the
same level of protection.
 Firewall is not an all or nothing approach to
security.
 There are many other things that you should
be doing for security.

Use the Right Tools


Value to Cost to Examples Firewall
Bad Guy Firewall Decision
Low High css, static images, JavaScript No
Low Medium product catalogs, search services No
Medium Medium critical business functions Maybe
High Low personal identifiable information Yes

Use the Right Tools

Rule 16 - Actively Use Log Files

 Make good use of your log files, and you will
have fewer production issues with your
system.

Use the Right Tools


 What
 Use your application’s log files to diagnose and prevent
problems.
 When
 Put a process in place that monitors log files and forces people
to take action on issues identified.
 How
 Use any number of monitoring tools from custom scripts to
Splunk to watch your application logs for errors. Export these
and assign resources for identifying and solving the issue.
 Why
 The log files are excellent sources of information about how
your application is performing for your users; don’t throw this
resource away without using it.

Use the Right Tools


1. Aggregate your logs
 Sampling
 multi-stage aggregation
2. Monitor the aggregated logs with
automated tools
3. Take action to fix the problems

Use the Right Tools


 We must pay attention to our costs for
logging and make a cost-effective decision of
both how much to log and how much data to
keep.

Use the Right Tools

Rule 17 - Don't Check Your Work

 Never justify reading something you just
wrote for the purposes of validating the data.
Read and act upon errors associated with the
write activity instead. Avoid other types of
reads of recently written data by storing that
data locally.

Don't Duplicate Your Work


 What
 Avoid checking things you just did or reading things you
just wrote within your products.
 When
 Always
 How
 Never read what you just wrote for the purpose of
validation. Store data in a local or distributed cache if it is
required for operations in the near future.
 Why
 The cost of validating your work is high relative to the
unlikely cost of failure. Such activities run counter to cost-
effective scaling.


 Doubles your costs
 Limits your scalability
 Rarely returns value in risk mitigation
commensurate with the costs



 Regulatory/legal requirement
 Think twice before you say 'YES.'
 Competitive differentiation
 Think twice before you say 'YES.'
 Asynchronous completion
 If the failure is in copying the data to a remote
backup for high availability, simply reapply that
record or transaction. Never block the user under
any scenario.

Rule 18 - Stop Redirecting Traffic

 Use redirects correctly and only when
necessary.



 What
 Avoid redirects when possible; use the right method
when they are necessary.
 When
 Use redirects as little as possible.
 How
 If you must have them, consider server configurations
instead of HTML or other code-based solutions.
 Why
 Redirects in general delay the user, consume
computation resources, and are prone to errors.


 Delays the user from getting the resource he
wants
 Takes up computational resources.
 Messed-up redirection hurting user browsing
or search engine ranking.
 Redirect errors
 Redirect loops
 Too long URLs
 Empty directs


 mod_alias
 Alias /image /www/html/image
 Redirect /service http://foo.company.com/service
 mod_rewrite
 RewriteEngine on
RewriteCond %{HTTP_HOST} ^artofscale.com$ [OR]
RewriteCond %{HTTP_HOST} ^www.artofscale.com$
RewriteRule ^/?(.*)$
"http://theartofscalability.com/$1" [R=301,L]



 .htaccess files should be avoided in favor of
the main configuration files primarily because
of performance.
 Apache looks in every directory for .htaccess files
 .htaccess file is loaded every time a document is
requested


Rule 19 - Relax Temporal Constraints

 Carefully consider the need for constraints
such as items being available from the time a
user views them until the user purchases
them. Some possible edge cases where users
are disappointed are much easier to
compensate for than not being able to scale.



 What
 Alleviate temporal constraints in your system whenever
possible.
 When
 Any time you are considering adding a constraint that an
item or object maintains a certain state between a user’s
actions.
 How
 Relax constraints in the business rules.
 Why
 The difficulty in scaling systems with temporal constraints
is significant because of the ACID properties of most
RDBMSs.


 Three core requirements exist when designing
applications in a distributed environment, but it
is impossible to simultaneously satisfy all:
 Consistency
▪ The client perceives that a set of operations has occurred all
at once.
 Availability
▪ Every operation must terminate in an intended response.
 Partition tolerance
▪ Operation will complete, even if individual components are
unavailable.


 Basically Available, Soft State, and Eventually
Consistent.
 Architectures that solve CAP.
 A BASE architecture allows for the databases
to become consistent, eventually.
 By relaxing the ACID properties of
consistency, we have greater flexibility in how
we scale.



 An example of a temporal constraint within a typical
SaaS application would be purchasing an item in stock.
There are time lapses between a user viewing an item,
putting it in his shopping cart, and purchasing it.
 The way we would redesign our system to
accommodate this eventual consistency would be to
relax the temporal constraint. The user just viewing an
item would not guarantee that it was available. The
application would “lock” the data when it was placed
into a shopping cart, and this would be done on the
primary write copy or master database


 Cache is King.
 Caching is important for tuning activity as
well as scaling activity.
 Caching is also about how to minimize the
amount of work your system does. (Don't
Duplicate Your Work)

 Multiple levels of caching can make it more
difficult to troubleshoot problems in your
product.
 Design the caching to be monitored.
 Caching also needs to be engineered to scale
well.

Rule 20 - Leverage Content Delivery Network

 CDNs are a fast and simple way to offset
spikiness of traffic as well as traffic growth in
general. Make sure you perform a cost-
benefit analysis and monitor the CDN usage.

Use Caching Aggressively


 What
 Use CDNs (content delivery networks) to offload traffic
from your site.
 When
 Ensure it is cost justified and then choose which content is
most suitable.
 How
 Most CDNs leverage DNS (Domain Name Services or
Domain Name Servers) to serve content on your site’s
behalf.
 Why
 CDNs help offload traffic spikes and are often economical
ways to scale parts of a site’s traffic.


The CDN can direct requests to nodes that are
optimally suited to respond.


 The analysis of when to convert to a CDN
almost never works on a cost-only basis.
 We’ve found that clients with greater than
10M of avenue revenues are better served by
implementing CDNs than continuing to serve
that traffic themselves.



 Even dynamic pages have static content.
Images, JavaScript, CSS, and so on, are all
usually static, which means they can be
cached in a CDN.
 CDNs are starting to enable dynamic page
support.


Rule 21 - Use Expires Headers

 For each object type (IMAGE, HTML, CSS,
PHP, and so on) consider how long the object
can be cached for and implement the
appropriate header for that timeframe.



 What
 Use Expires headers to reduce requests and improve
the scalability and performance of your system.
 When
 All object types need to be considered.
 How
 Headers can be set on Web servers or through
application code.
 Why
 The reduction of object requests increases the page
performance for the user and decreases the number
of requests your system must handle per user.


HTTP Status Code: HTTP/1.1 200 OK
Date: Thu, 21 Oct 2010 20:03:38 GMT
Server: Apache/2.2.9 (Fedora)
X-Powered-By: PHP/5.2.6
Expires: Mon, 26 Jul 2011 05:00:00 GMT
Last-Modified: Thu, 21 Oct 2010 20:03:38 GMT
Cache-Control: no-cache
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8


 public
 The response may be cached by any cache, shared or
nonshared.
 private
 The response message is intended for a single user and
must not be cached by a shared cache.
 no-cache
 A cache must not use the response to satisfy a subsequent
request without revalidation with the origin server.
 max-age
 The response is stale if its current age is greater than the
value given (in seconds) at the time of request. It overrides
the Expires header.


 mod_expires
 ExpiresActive On
ExpiresByType image/png "access plus 1 day"
ExpiresByType image/gif "modification plus 5
hours"
ExpiresByType text/html "access plus 1 month 15
days 2 hours"
ExpiresDefault "access plus 1 month"



 Unfortunately, meta tags in HTML (such as
Pragma, Expires, or Cache-Control) are
recommendations of how a page should be
treated by the browser, but many browsers
do not pay attention to these tags.
 Even worse, because proxy caches don't
inspect the HTML, they do not abide by these
tags at all.


 What's the difference between max-age=0
and max-age=1?
 If it take 3 seconds to load your pages, what's
the meaning of max-age=0?

Rule 22 - Cache Ajax Calls

 Leverage Ajax and cache Ajax calls as much
as possible to increase user satisfaction and
increase scalability.



 What
 Use appropriate HTTP response headers to ensure
cacheability of Ajax calls.
 When
 Every Ajax call but those absolutely requiring real time
data that are likely to have been recently updated.
 How
 Modify Last-Modified, Cache-Control, and Expires headers
appropriately.
 Why
 Decrease user perceived response time, increase user
satisfaction, and increase the scalability of your platform
or solution.


 If handled properly, Ajax not only makes for
wonderfully interactive user interfaces, it
helps us in our scalability endeavors by
allowing the client to handle and interact with
data and objects without requiring additional
server side work. But if not handled properly,
Ajax can actually create some unique
scalability constraints by significantly
increasing the number of requests our servers
need to handle.


 With a little work, we can cache the results of
Ajax interactions within the client browser
(Rule 21) and potentially with-in our CDNs
(Rule 20), page caches (Rule 23), and
application caches (Rule 24).
 Ajax requests behave (or should behave) the
same as any other HTTP request and
response.



 systems that request specific objects such as
an item for sale in an auction, a message in a
social networking site, or an e-mail system
should use specific message IDs (such as
id=123445) rather than relative offsets when
making requests (such as page=3&item=2).


Rule 23 - Leverage Page Cache

 Page caches are a great way to offload
dynamic requests and to scale cost effectively.



 What
 Deploy page caches in front of your Web services.
 When to use
 Always.
 How
 Choose a caching system and deploy.
 Why
 Decrease load on Web servers by caching and
delivering previously generated dynamic requests and
quickly answering calls for static objects.


Rule 24 - Utilize Application Caches

 Carefully consider the need for constraints
such as items being available from the time a
user views it until they purchase. Some
possible edge cases where users are
disappointed are much easier to compensate
for than not being able to scale.



 What
 Alleviate temporal constraints in your system whenever
possible.
 When
 Any time you are considering adding a constraint that an
item or object maintains a certain state between a user’s
actions.
 How
 Relax constraints in the business rules.
 Why
 The difficulty in scaling systems with temporal constraints
is significant because of the ACID properties of most
RDBMSs.


 You absolutely must employ application level
caching if you want to scale in a cost-effective
manner.
 Such caching must be developed from a
system architecture perspective to effective
long term.



 Pareto's Principle - The 80-20 Rule
 X Axis
▪ Horizontal duplication
 Y Axis
▪ Split by function, service, or resource
 Z Axis
▪ Lookup for formulaic splits
▪ Something you knew about the requester or customer



 The first step is to hypothesize as to likely usage
and determine ways to split to maximize
cacheability. After implementing these splits in
both the application and supporting persistent
data stores, evaluate their effectiveness in
production. Further refine your approach based
on production data and iteratively apply the
Pareto Principle and the AKF Scale Cube (Rules 7,
8, and 9) to refine and increase cache hit rates.
Lather, rinse, repeat.

Rule 25 - Make Use of Object Caches

 Consider implementing an object cache
anywhere computations are performed
repeatedly, but primarily this is done
between the database and application tiers.



 What
 Implement object caches to help your system scale.
 When
 Any time you have repetitive queries or computations.
 How
 Select any one of the many open source or vendor
supported solutions and implement the calls in your
application code.
 Why
 A fairly straightforward object cache implementation
can save a lot of computational resources on
application servers or database servers.


 Object caches are simply in-process data
stores that store a hashed summary of each
item.
 These caches are used primarily for caching
data that may be computationally expensive
to regenerate.
 memcached, Apache' OJB, and NCache



 Between the database and the application to
cache result sets from SQL queries.
 For results of complex application
computations, such as user
recommendations, product prioritization, or
reordering advertisements based on recent
past performance.



1. Choose one that best fits your needs and
implement it.
 Building a home-grown solution should be highly
scrutinized.
2. Actually implement the object cache.
3. Monitor object cache for the cache hit rate.
 the number of times the system requests an
object that is in the cache compared to the total
number of requests.
 Ideally, this ratio is 85% or better.

Rule 26 - Put Object Caches on Their Own "Tier"

 When implementing an object cache it is
simplest to put the service on an existing tier
such as the application servers. Consider
implementing or moving the object cache to
its own tier for better performance and
scalability.



 What
 Use a separate tier in your architecture for object
caches.
 When
 Any time you have implemented object caches.
 How
 Move object caches onto their own servers.
 Why
 The benefits of a separate tier are better utilization of
memory and CPU resources and having the ability to
scale the object cache independently of other tiers.


 The advantage of separating these tiers is
that you can size the servers appropriately.
 object caches typically require a lot of memory
but require relatively low computational
processing power.
 Sizing the server correctly can save quite a bit
of money.


 We can only truly learn from our failures if we
foster an environment of open, honest
communication and fold in lightweight
processes that help us repeatedly learn and
get the most from our mistakes and failures.

Rule 27 - Learn Aggressively

 Be constantly and aggressively learning. The
companies that learn best, fastest, and most
often are the ones that grow the fastest and
are the most scalable.

Learn from Your Mistakes


 What
 Take every opportunity to learn.
 When
 Be constantly learning from your mistakes as well as successes.
 How
 Watch your customers or use A/B testing to determine what
works. Use postmortems to learn from incidents and problems
in production.
 Why
 Doing something without measuring the results or having an
incident without learning from it are wasted opportunities that
your competitors are taking advantage of.



 Customers
 Launch quickly and watch your customers'
reactions.
▪ A/B testing
▪ keeping track of usage and adoption of new features
 Think twice before you bring in focus groups.
 Business/technology operations
 You must not let incidents or problems pass
without learning from them.

Rule 28 - Don't Rely on QA to Find Mistakes

 QA doesn’t increase the quality of your
system, as you can’t test quality into a system.
If used properly, it can increase your
productivity while decreasing cost and most
importantly it can keep you from increasing
defect rates faster than your rate of
organization growth during periods of rapid
hiring.



 What
 Use QA to lower the cost of delivered products, increase
engineering throughput, identify quality trends, and decrease
defects—not to increase quality.
 When
 Whenever you can get greater throughput by hiring someone
focused on testing rather than writing code. Use QA to learn
from past mistakes—always.
 How
 Hire a QA person anytime you get greater than one engineer’s
worth of output with the hiring of a single QA person.
 Why
 Reduce cost, increase delivery volume/velocity, decrease the
number of repeated defects.



 The issue is that you shouldn’t rely solely on
these teams to identify all your defects.
 Testing only identifies issues that you created
during development, and as a result it is an
identification of value that you destroyed and
can recapture.
 It is rare that testing, or the group that
performs it, identifies untapped opportunities
that might create additional value.


 Archives
 Reducing cost
 Increasing defect identification
 Increasing throughput
 The greatest value of OA organizations arises
in the case of hyper growth companies.
 In these situations, standards are hard to enforce.



 A tool to help the organization learn
 what mistakes it is making repeatedly
 where those mistakes lie
 how the organization can keep from making them
in the future
 OA is likely the only team capable of seeing
the recurring problems.
 create an environment in which those problems
are discussed and eliminated.

Rule 29 - Failing to Design for Rollback Is Designing for Failure

 Don’t accept that the application is too
complex or that you release code too often as
excuses that you can’t roll back. No sane pilot
would take off in an airplane without the
ability to land, and no sane engineer would
roll code that they could not pull back off in
an emergency.



 What
 Always have the ability to roll back code.
 When
 Ensure all releases have the ability to roll back, practice it
in a staging or QA environment, and use it in production
when necessary to resolve customer incidents.
 How
 Clean up your code and follow a few simple procedures to
ensure you can roll back your code.
 Why
 If you haven’t experienced the pain of not being able to roll
back, you likely will at some point if you keep playing with
the “fix-forward” fire.


 Code
 Database
 As you’d expect the majority of the problem with
rolling back is in the database.



 Database changes must only be additive
 Columns or tables should only be added, not deleted,
until the next version of code is released that
deprecates the dependency on those columns.
 DLL and DML scripted and tested
 The team needs to test the rollback process in QA or
staging.
 the script needs to be tested under some amount of
load condition to ensure it can be executed while the
application is utilizing the database.


 Restricted SQL queries in the application
 removing all SELECT * queries
 adding column names to all UPDATE statements
 Semantic changes of data
 The development team must not change the
definition of data within a release.
 View on/Wire off
 The application should have a framework added that
allows code paths and features to be accessed by
some users and not by others, based on an external
configuration.

Rule 30 - Discuss and Learn from Failures

 Never let a good failure go to waste. Learn
from every one and identify the technology,
people, and process issues that need to be
corrected.



 What
 Leverage every failure to learn and teach important
les-sons.
 When
 Always.
 How
 Employ a postmortem process and hypothesize
failures in low failure environments.
 Why
 We learn best from our mistakes—not our successes.



 The complexity inherent to modern coupled
systems makes accidents inevitable.
 Appolo 13



 Organizations that fail often have greater
opportunities to learn and grow than those
that do not, assuming of course that they
take an opportunity to learn from them.
 Systems that fail infrequently offer little
opportunity to learn and as a result in the
absence of other approaches the teams and
systems will not grow and improve.



1. Timeline
 Focus on generating a timeline of the events leading
up to the issue or crisis.
2. Issue Identification
 The process facilitator walks through the timeline
and works with the team to identify issues.
3. State Action
 The process facilitator walks down the list of issues
and works with the team to identify an action, an
owner, an expected result, and a time by which it
should be completed.
 SMART principles.


 An organization should attack that issue with
the postmortem process.
 No postmortem should be considered
complete until it has addressed the people,
process, and technology issues responsible
for the failure.
 In our experience, you are never finished
unless you can answer “Why” at least five
times to cover five different potential
problems.


 Preoccupation with failure
 Monitoring our product and our systems and
reporting errors in a timely fashion.
 Reports should be widely distributed and
discussed frequently.
 Reluctance to simplify interpretations
 Take nothing for granted and seek input from
diverse sources.
 Don’t try to box failures into expected behavior
and act with a healthy bit of paranoia.


 Sensitivity to operations
 Look at detail data at the minute level.
 Commitment to resilience
 Build excess capability by rotating positions and
training your people in new skills.
 Deference to expertise
 During crisis events, shift the leadership role to
the person possessing the greatest expertise to
deal with the problem.

Rule 31 - Be Ware of Costly Relationships

 Think ahead and plan the data model
carefully. Consider normalized forms, how
you will likely split the database in the future,
and possible data needs of the application.

Database Rules


 What
 Be aware of relationships in the data model.
 When
 When designing the data model, adding tables/columns, or
writing queries consider how the relationships between entities
will affect performance and scalability in the long run.
 How
 Think about database splits and possible future data needs as
you design the data model.
 Why
 The cost of fixing a broken data model after it has been
implemented is likely 100x as much as fixing it during the design
phase.

Database Rules


 Database relationships are determined by the
data model, which captures the cardinality
and referential integrity rules of the data.
 Foreign keys enforce referential integrity by
completing an association between two
entities (rows) of different entity sets (tables).
 Entity relationship diagrams (ERDs) show the
cardinality between entity sets, one-to-one,
one-to-many, or many-to-many relationships.
Database Rules


 The primary purpose of normalizing a data
model is to ensure the data is stored in a manner
that allows for insert, update, select, and delete
with data integrity.
 Non-normalized data models have a high degree
of data redundancy, which means that the risk of
data integrity problems is greater.
 If a database adheres to at least the third normal
form it is considered normalized.
 Often to increase scale, we look to reduce
normal forms.

Database Rules


 The relationships between entities
dramatically affect how efficiently the data is
stored, retrieved, and updated.
 It also plays a large role in scalability as these
relationships define how we are able to split
or shard our database.
 One last aspect of data relationships that is
important to scalability is how we join tables
in our queries.
Database Rules


 Tune the query.
 Create a view, materialized view, summary
table, and so on that can preprocess the joins.
 Do not join in the query but rather pull the
data sets into the application and join in
memory in the application.
 Push back on the business requirements.

Database Rules

Rule 32 - Use the Right Type of Database Lock

 Understand the types of locks and manage
their usage to maximize database throughput
and concurrency. Change lock types to get
better utilization of databases and look to
split schemas or distribute databases as you
grow. When choosing databases, ensure you
choose one that allows multiple lock types
and granularity to maximize concurrency.

Database Rules


 What
 Be cognizant of the use of explicit locks and monitor implicit
locks.
 When
 Anytime you employ relational databases for your solution.
 How
 Monitor explicit locks in code reviews. Monitor databases for
implicit locks and adjust explicitly as necessary to moderate
throughput. Choose a database and storage engine that allows
flexibility in types and granularity of locking.
 Why
 Maximize concurrency and throughput in databases within your
environment.

Database Rules


 Locks are the way in which databases allow
concurrent users while helping to ensure the
consistency and isolation components of the
ACID properties of a database.
 There are many different types of database
locks, and even different approaches to
implementing them.
 Locks might be either explicit or implicit.

Database Rules


 Row
 Row level locking locks a row in a table of a database that is
being updated, read, or created.
 Page
 Page level locking locks the entire page that contains a row or
group of rows being updated.
 Extent
 Typically, these are locks on groups of pages. They are common
when database space is being added.
 Table
 Locks an entire table (an entity within a database).
 Database
 Locks the entirety of entities and relationships within a
database.
Database Rules


 The size or granularity of lock to employ
ultimately impacts the throughput of
transactions.
 Often a component of the database
(commonly called an optimizer within many
databases) determines what size of element
should be locked in an attempt to allow
maximum concurrency while ensuring
consistency and isolation.
Database Rules


 By monitoring performance statistics
historically in the production environment,
we can identify when the optimizer is
incorrectly identifying the type of lock it
should use and force the database to use an
appropriate type of locking.
 We should determine if we can change the
entity relationships to reduce contention and
increase concurrency (X, Y, Z axis).
Database Rules


 Where we employ databases to do our work
we should try to ensure we are choosing the
best solution.
 MySQL is an example where the selection of
a storage engine such as MyISAM can limit
you to table level locks and, as a result,
potentially limits your transaction throughput.

Database Rules

Rule 33 - Pass on Using Multiphase Commits

 Do not use multiphase commit protocols as a
simple way to extend the life of your
monolithic database. It will likely cause it to
scale even less and result in an even earlier
demise of your system.

Database Rules


 What
 Do not use a multiphase commit protocol to store or
process transactions.
 When
 Always pass or alternatively “never use” multi-phase
commits.
 How
 Don’t use it; split your data storage and processing
systems with Y or Z axis splits.
 Why
 Multiphase commits are blocking protocols that do not
permit other transactions from occurring until it is
complete.
Database Rules


Database Rules


 2PC might seem like a good alternative to
actually splitting your database by a Y or Z
axis split.
 Pull (or separate) database tables apart the
smart way instead of trying to extend the life
of your monolithic database with a
multiphase commit protocol.

Database Rules

Rule 34 - Try Not to Use "Select For Update"

 Cursors are powerful constructs that when
properly used can actually make
programming faster and easier while
speeding up transactions. But FOR UPDATE
cursors may cause long held locks and slow
transactions. Refer to your database
documentation for whether you need to use
the FOR READ ONLY clause to minimize locks.

Database Rules


 What
 Minimize the use of the FOR UPDATE clause in a
SELECT statement when declaring cursors.
 When
 Always.
 How
 Review cursor development and question every
SELECT FOR UPDATE usage.
 Why
 Use of FOR UPDATE causes locks on rows and may
slow down transactions.
Database Rules


 Be aware that some databases default to “for
update” for cursors.
 In fact, the American National Standards
Institute (ANSI) SQL standard indicates that
any cursor should default to FOR UPDATE
unless it includes the clause FOR READ ONLY
on the DECLARE statement.

Database Rules

Rule 35 - Don't Select Everything

 Don’t use wildcards when selecting or
inserting data.

Database Rules


 What
 Don’t use Select *in queries.
 When
 Always use this rule (or put another way, never select
everything).
 How
 Always declare what columns of data you are
selecting or inserting in a query.
 Why
 Selecting everything in a query is prone to break
things when the table structure changes and it
transfers unneeded data.
Database Rules


 Insert
 data mapping problems
 transfer of unnecessary data
 Update
 waste resources
 likely to break or potentially even corrupt data
 prevent you from rolling back

Database Rules

 In our experience, the second most common
scalability related failure behind “Not
designed to scale” is “Not designed to fail.”
 A product that isn’t highly available really
doesn’t need to scale and a site that can’t
scale won’t be highly available when the
demand comes.

Rule 36 - Design Using Fault Isolative "Swimlanes"

 Fault isolation consists of eliminating
synchronous calls between fault isolation
domains, limiting asynchronous calls and
handling synchronous call failure, and
eliminating the sharing of services and data
between swimlanes.

Design for Fault Tolerance and Graceful Failure


 What
 Implement fault isolation or swimlanes in your designs.
 When
 Whenever you are beginning to split up databases to scale.
 How
 Split up databases and services along the Y or Z axis and
disallow synchronous communication or access between
services.
 Why
 Increase availability and scalability and reduce incident
identification and resolution as well as time to market and
cost.


 Pod
 Pods are self-contained sets of functionality containing
app servers, persistent storage (such as a database or
other persistent and shared file system), or both.
 Pods are most often splits along the Z axis
 Cluster
 Clusters are sometimes used for Web and application
servers in the same fashion as a “pool”
 A cluster refers to an X axis scale of similar functionality or
purpose configured such that all nodes or participants are
“active.”
 Clusters might also refer to active/passive configuration


 Pool
 Pools are servers that group similar functionality or
potentially separate groups of customers.
 Pools are typically X axis replicated (cloned) servers that
are demarcated by function (Y axis) or customer (Z axis).
 Shard
 Shards are horizontal partitions of databases or search
engines.
 Shards typically occur along the Z axis of scale (for
instance splitting up customers among shards), but some
companies refer to functional (Y axis) splits as shards as
well.


 Swimlane is a design concept around creating
fault isolation domains.
 A fault isolation domain is an area in which,
should a physical or logical service fail to
work appropriately, whether that failure is in
a slow response time or an absolute failure to
respond, the only services affected are those
within the failure domain.



 Availability
 Incident detection
 Scalability
 Cost
 Time to market



1. Share nothing (almost)
2. No synchronous calls between swimlanes
3. Limit asynchronous calls between
swimlanes
4. Use timeouts and Wire-On/Wire-Off with
asynchronous calls
5. Don't mix virtual servers from different
swimlanes on the same physical device.



1. Duplicate the service that we believe we
need and put it in our swimlane.
 Payment gateways are one example.
 Use read-only replicas when some shared
information is needed
 read-only replica of databases
 cache


Rule 37 - Never Trust Single Points of Failure

 Strive for active/active rather than
active/passive solutions. Use load balancers
to balance traffic across instances of a service.
Use control services with active/passive
instances for patterns that require singletons.



 What
 Never implement and always eliminate single points
of failure.
 When
 During architecture reviews and new designs.
 How to use
 Identify single instances on architectural diagrams.
Strive for active/active configurations.
 Why
 Maximize availability through multiple instances.



 SPOFs can be anywhere in the system from a
single Web server or single network device
but most often the SPOF in a system is the
database.
 The solution to most SPOFs is simply
requisitioning another piece of hardware and
running two or more of every service by
cloning that service as described in the X axis
of scale.
 Eliminate the SPOF as soon as possible.


 Fix the code so that the service can be
implemented on many different hosts.
 quick fixes (when code changes cannot be
made)
 use an active-passive configuration
 use another component in the system to control
the access to data
 load balancer (with sticky sessions)


Rule 38 - Avoid Putting Systems in Series

 Avoid adding components to your system
that are connected in series. When necessary
to do so add multiple versions of that
component so that if one fails others are
available to take its place.



 What
 Reduce the number of components that are
connected in series.
 When
 Anytime you are considering adding components.
 How
 Remove unnecessary components or add multiple
versions of them to minimize the impact.
 Why
 Components in series have a multiplicative effect of
failure.


 Layers are connected in series.
 Adding components in parallel, even when
tiers are connected in series, helps reduce the
risk of total system failure caused by a
component failure.
 On the Web and application tiers most people
readily acknowledge this concept. Where
most people overlook this problem is in the
database and network layers.

Rule 39 - Ensure You Can Wire On And Off Functions

 Implement Wire On/Wire Off Frameworks
whenever the cost of implementation is less
than the risk and associated cost of failure.
Work to develop shared libraries that can be
reused to lower the cost of future
implementation.



 What
 Create a framework to disable and enable features of your
product.
 When
 Risky, very high use, or shared services that might otherwise
cause site failures when slow to respond or unavailable.
 How
 Develop shared libraries to allow automatic or on-demand
enabling and disabling of services.
 Why
 Graceful failure (or handling failures) of transactions can keep
you in business while you recover from the incident and
problem that caused it.



 Help to ensure that your system can either
fail gracefully (in the event of self-diagnosing
frameworks) or can operate with certain
functionality disabled by human intervention.
 The approach to enabling and disabling
services probably depends on the capabilities
of your engineering team, your operations
team, and the business criticality of the
service in question.


 Automatic markdown based on timeouts
 useful for synchronous calls for internal and external
services. Calls are not made upon markdown at all,
and service is considered "offline."
 Stand in service
 replace a service with an auto responder with a
dummy response that indicates service unavailable or
a cached response of "likely good data."
 Synchronous markdown command
 User intervention sends a command to services to
stop using failed or slow services.


 Config file markdown
 Change configuration file variable to indicate the
"wire off " of a service.
 File markdown
 Presence of a file indicates service up.
 Database markdown
 Use of a variable per service in a database to enable or
disable features.
 Runtime variable
 Read at startup as an argument to the program for
daemon-like server processes.


 The cost of implementation is less than the
risk to our business.
 Never for the features that never fail (though
unlikely).
 Any feature that has a high rate of usage (high
throughput) and whose failure will impact other
important features on the site (e.g. shared service).
 Any feature that is undergoing fairly significant
change in a given release.

 The state we hate most is persistent state (and
session information) held with-in the application
of an Internet site.
 We strive to eliminate any approach that will
limit the degree of tenancy on any system. State
and session cost us both in terms of memory and
processing power, and as a result is an enemy to
our cost-effective scale goals.
 If state is necessary, we need to implement it in
a fault-tolerant, highly available, and cost-
effective way.

Rule 40 - Strive for Statelessness

 Always push back on the need for state in any
system. Use business metrics and
multivariate (or A/B) testing to determine
whether state in an application truly results in
the expected user behavior and business
value.

Avoid or Distribute State


 What
 Design and implement stateless systems.
 When
 During design of new systems and redesign of
existing systems.
 How
 Choose stateless implementations whenever possible.
If stateful implementations are warranted for
business reasons, refer to Rules 41 and 42.
 Why
 The implementation of state limits scalability and
increases cost.


 we can sometimes rely on clustering software
with state or session replication to help us
scale, but such an approach only delays the
inevitable.
 If your company is successful you will quickly
grow out of this rather costly session
synchronization method.



 Load balancer with sticky session is an
adequate approach for many products where
growth has slowed or where customers have
more relaxed availability requirements.
 Capacity planning can become troublesome
when several high volume or long running
sessions become bound to a handful of
servers.



 Why do you need it at all?
 There are certain solutions that probably
need state, but more often than not, state is
luxury and costly..
 Never underestimate the power of "simple
and easy."
 Session and state cost money and you should
only implement them where there is a clear
competitive advantage displayed in key
operating metrics

Rule 41 - Maintain Sessions in the Browser When Possible

 Using cookies to store session data is a
common approach and has advantages in
terms of ease of scale but also has some
drawbacks. One of the most critical cons is
that unsecured cookies can easily be
captured and used to log into people’s
accounts.



 What
 Try to avoid session data completely, but when needed,
consider putting the data in users’ browsers.
 When
 Anytime that you need session data for the best user
experience.
 How
 Use cookies to store session data on the users’ browsers.
 Why
 Keeping session data on the users’ browsers allows the
user request to be served by any Web server in the pool
and takes the storage requirement away from your system.


 Pros
 Relieves the system of a large burden of storage and
workload.
 The browser's request can be serviced by any server in
the pool.
 Cons
 The data must be transferred back and forth between
the browser and the servers. It is critical to maintain
control of the size of the cookie data.
 Session data can be easily captured on an open WiFi
network and used to log in to someone else's account.


 Transmit your pages and cookies all in HTTPS.
 Using at least two cookies. One cookie is an
authorization cookie that is requested via
HTTPS on each HTTP page using a JavaScript
call such as the following
<script type="text/javascript"
src="https://verify.akfdemo.com/authenticat
e.php"></script>


Rule 42 - Make Use of Distribute Cache for States

 Many Web servers or languages offer simple
server-based session management, but these
are often fraught with problems such as user
affiliation with specific servers. Implementing
a distributed cache allows you to store
session data in your system and continue to
scale.



 What
 Use a distributed cache when storing session data in your
system.
 When
 Anytime you need to store session data and cannot do so
in users’ browsers.
 How
 Watch for some common mistakes such as a session
management system that requires affinity of a user to a
Web server.
 Why
 Careful consideration of how to store session data can
help ensure your system will continue to scale.


 Don’t implement systems that require affinity
to a server to function properly.
 Don’t use state or session replication to
create duplicates of data on different systems.
 Don’t locate the cache on the system doing
the work (this doesn’t mean you shouldn’t
have a local application cache—just that
session information is best handled in its own
tier of servers).


 Database-only implementations are the most costly
overall, but allow all data to be persisted and handle
conflicts between updates and reads very well in a
distributed environment.
 Nonpersistent object caches are fast and
comparatively inexpensive, but do not allow data to
be recovered upon failure and aren’t good for
implementations with long periods between accesses
by users.
 Hybrid solutions with databases providing persistency
and caches providing cost-effective scale are great
when persistency is required and low relative cost is
preferred.

 When implemented properly, asynchronous
communication is absolutely a valuable rung
in the ladder of near infinite scale.
 When implemented haphazardly, it merely
hides the faults.
 Not only communicates in an asynchronous
fashion but actually develops the application
to be asynchronous in behavior.

Rule 43 - Communicate Asynchronously As Much As Possible

 Use asynchronous communication
techniques to ensure that each service and
tier is as independent as possible. This allows
the system to scale much farther than if all
components are closely coupled together.

Asynchronously Communication and Message Buses


 What
 Use asynchronous instead of synchronous
communication as often as possible.
 When
 Consider for all calls between services and tiers.
 How
 Use language specific calls to ensure the requests are
made and not waited on.
 Why
 Synchronous calls stop the entire program’s execution
waiting for a response, which ties all the services and
tiers together resulting in cascading failures.


 In general, asynchronous calls are much more
difficult to implement than synchronous calls.
 Asynchronous calls often require
coordination to communicate back to the
service that first sent a message that the
request has been completed.



 Callbacks are mechanisms designed to allow
for the coordination and communication
between the parent method and the
asynchronous child method.
 There are many design patterns that use
callbacks, such as the delegate design pattern
and the observer design pattern.



 The real question is which ones should be
made asynchronous.
 Synchronous calls should have timeouts that
allow for gracefully handling errors or
continued processing when a synchronously
called method or service fails.



 External API/third party
 you do not want the health and availability of your
system tied to a system that you can’t control.
 Long running processes
 Often the more problematic issues are with slow
running processes rather than outright failures.
 Error prone/changed frequently methods
 Avoid tying critical code with code that needs to
be changed frequently.
 Temporal constraint

Rule 44 - Ensure Your Message Bus Can Scale

 Treat message buses like any other critical
component of your system. Scale them
ahead of demand using either the Y or Z axes
of scale.



 What
 Message buses can fail from demand like any other
physical or logical system. They need to be scaled.
 When
 Anytime a message bus is part of your architecture.
 How
 Employ the Y and Z AKF Axes of Scale.
 Why
 To ensure your bus scales to demand.



 While it is true that applications designed to be
asynchronous are often more resilient to failure, and
while these apps also tend to scale more efficiently,
they are still prone to high demand bottlenecks and
failure points.
 The principles will as easily resolve the scalability
needs of a message bus as they will solve the needs of
a database.
 The most important point in determining how to scale
a message bus is to ensure that the approach is
consistent with the approach applied to the rest of the
technology architecture.


 Dedicate buses to particular purposes.
 splitting of a service bus reduce flexibility in
communication. But remember that these splits
are to enable the greater good of enabling hyper
growth and staying in business!
 We can look at things we know about the
data such as its temporal qualities.
 Is the data likely to be needed quickly or is it just a
“for your information” piece of datum?


 Temporal
 Service
 Quality of Service
 Resource



 It makes most sense when your
implementation has already employed a Z
axis split.
 Each of the swimlanes or pods can have a
dedicated message bus.
 This would need to be the case if we truly wanted
fault isolation.
 That doesn’t mean that we can’t leverage one or
more message buses to communicate
asynchronously between swimlanes.

Rule 45 - Don't Overcrowding Your Message Bus

 Don’t publish everything. Sample traffic to
ensure alignment between cost and value.



 What
 Limit bus traffic to items of higher value than the cost
to handle them.
 When
 On any message bus.
 How
 Value and cost justify message traffic. Eliminate low
value, high cost traffic. Sample low value/low cost and
high value/high cost traffic to reduce the cost.
 Why
 Message traffic isn’t “free” and presents costly
demand on your system.


 Not all data has equivalent value to your
business.
 Just because you have implemented a
message bus (or a scalable message bus
solution) doesn’t mean that you have to use it
for everything.
 We want the solution to scale within certain
cost constraints.


Rule 46 - Be Wary of Scaling Through Third Parties

 Do not rely on vendor products, services, or
features to scale your system. Keep your
architecture simple, keep your destiny in your
own hands, and keep your costs in control. All
three of these can be violated by using a
vendor’s proprietary scaling solution.

Miscellaneous Rules


 What
 Scale your own system; don’t rely on vendor solutions to
achieve scalability.
 When
 Whenever considering whether to use a new feature or product
from a vendor.
 How
 Rely on the rules of this book for understanding how to scale
and use vendor provided products and services in the most
simplistic manner possible.
 Why
 Three reasons for following this rule: Own your destiny, keep
your architecture simple, and reduce your total cost of
ownership.

Miscellaneous Rules


 Control of your own
 additional complexity
 destiny
 total cost of ownership

Miscellaneous Rules

Rule 47 - Purge, Archive, and Cost-Justify Storage

 It is important to understand and calculate
the value of your data and to match storage
costs to that value. Don’t pay for data that
doesn’t have a stakeholder return.

Miscellaneous Rules


 What
 Match storage cost to data value, including removing data of
value lower than the costs to store it.
 When
 Apply to data and its underlying storage infrastructure during
design discussions and throughout the lifecycle of the data in
question.
 How
 Apply recency, frequency, and monetization analysis to
determine the value of the data. Match storage costs to data
value.
 Why
 Not all data is created equal (that is, of the same value), and in
fact it often changes in value over time. Why then should we
have a single storage solution with equivalent cost for that data?
Miscellaneous Rules


 The storage itself costs money to purchase.
 The space it occupied costs us money (or lost
opportunity relative to higher value services
where we owned the space.)
 The power and HVAC to spin and cool the
drives was increasing rather than decreasing
in cost on a per-unit basis.

Miscellaneous Rules


 Recency
 How recently the data item in question has been
accessed
 Frequency
 How frequently that data is accessed
 Monetization
 The value that a specific piece of data has to your
business in general

Miscellaneous Rules


Miscellaneous Rules


 RFM isn't a one-time analysis but rather a
living process.
 We need to have processes and procedures to
"archive" data or to move it to lower cost
systems.
 In rare cases, data may actually increase in
value with age.

Miscellaneous Rules

Rule 48 - Remove Business Intelligence from Transaction Processing

 Databases and internal corporate systems can
be costly to scale due to license and unique
system characteristics. As such, we want them
dedicated to their specific tasks. In the case of
databases, we want them focused on
transactions rather than product intelligence. In
the case of back office systems (business
intelligence), we do not want our product tied to
their capabilities to scale. Use asynchronous
transfer of data for business systems.
Miscellaneous Rules


 What
 Separate business systems from product systems and product
intelligence from database systems.
 When
 Anytime you are considering internal company needs and data
transfer within, to, or from your product.
 How
 Remove stored procedures from the database and put them in
your application logic. Do not make synchronous calls between
corporate and product systems.
 Why
 Putting application logic in databases is costly and represents
scale challenges. Tying corporate systems and product systems
together is also costly and represents similar scale challenges as
well as availability concerns.
Miscellaneous Rules


 Databases tend to be one of the most
expensive systems or services within your
architecture, even in the case where you are
using an open source database.
 Often, in mature environments, these
systems are tuned to do one thing well-
perform relational operations and commit
transactions to a stable storage engine as
quickly as possible.
Miscellaneous Rules


 Product Intelligence should not be placed on
databases.
 Business intelligence should not be tied to
product transactions.

Miscellaneous Rules

Rule 49 - Design Your Application to Be Monitored

 Adopt as an architectural principle that your
application must be monitored. Additionally,
look at your overall monitoring strategy to
make sure you are first answering the
question of “Is there a problem?” and then
the “Where” and “What.”

Miscellaneous Rules


 What
 Think about how you will need to monitor your application
as you are designing it.
 When
 Anytime you add or change modules of your code base.
 How
 Build hooks into your system to record transaction times.
 Why
 Having insight into how your application is performing will
help answer many questions when there is a problem.

Miscellaneous Rules


 Network traffic/system monitors and
automatic alerting system are just not
enough.
 Just because a server has a high CPU or
memory utilization does not mean that your
customers are having any issue with your site
at all.
 The way is to monitor your system from the
perspective of a business metric.
Miscellaneous Rules


1. Is there a problem?
2. Where is the problem?
3. What is the problem?

Miscellaneous Rules


Miscellaneous Rules


 Your application code should make it easy to
place hooks in for watching the execution of
transactions such as SQL, API, RPC, or
method calls.
 Some of the best monitored systems have
asynchronous calls before and after a
transaction to record the start time,
transaction type, and end time.

Miscellaneous Rules


 Why is there a problem?
 Your organization must integrate learning from
answering "Why" into the next hour's code release.
 Will there be a problem?
 Requires the combination of
business/application/hardware monitoring data
 Using statistical tools or machine learning algorithms
to predict whether a problem is likely to occur.
 Automatic or self-healing systems are a long way off

Miscellaneous Rules

Rule 50 - Be Competent

 Don’t confuse competence with build versus
buy or core versus context decisions. You can
buy solutions and still be competent in their
deployment and maintenance. In fact, your
customers demand that you do so.

Miscellaneous Rules


 What
 Be competent, or buy competency in/for each component
of your architecture.
 When
 For any Internet service or commerce solution.
 How
 For each component of your infrastructure, identify the
team responsible and level of competency with that
component.
 Why
 To a customer, every problem is your problem. You can’t
blame suppliers or providers. You provide a service—not
software.
Miscellaneous Rules


 Your customers expect you to deliver a
service to them.
 The decision that something is “non-core” or
that one should buy versus build should not
be confused with whether your team should
be competent in the technology that you buy.
 If we must be competent in our software to
deliver a service, so must we be competent in
everything else that we employ to deliver
that service.
Miscellaneous Rules


 We need to know that we are using the
components correctly, maintaining it
properly, and restoring it to service promptly
when it fails.
 The larger our team and the more we rely on
the component in question, the more likely it
is that we should have some in-house
expertise.
 The provider of the service needs to have
"skin in the game."
Miscellaneous Rules

Digests for the book "Scalability Rules: 50 Principles for Scaling Web Sites"

More Related Content

What's hot (20)

Similar to Digests for the book "Scalability Rules: 50 Principles for Scaling Web Sites" (20)

More from Cyril Wang (6)

Recently uploaded (20)

Digests for the book "Scalability Rules: 50 Principles for Scaling Web Sites"