Sitepoint PHP

Feliratkozás Sitepoint PHP hírcsatorna csatornájára
Learn CSS | HTML5 | JavaScript | Wordpress | Tutorials-Web Development | Reference | Books and More
Frissítve: 37 perc 29 másodperc

Improving Performance Perception: On-demand Image Resizing

h, 07/02/2018 - 18:00

Over a series of articles, we've been building a sample application --- a multi-image gallery blog --- for performance benchmarking and optimizations. At this point, our application serves the same image regardless of the resolution and screen size it's being served in. In this tutorial, we'll modify it to serve a resized version depending on display size.

Objective

There are two stages to this improvement.

  1. We need to make all images responsive wherever this might be useful. One place is the thumbnails on the home page and in the gallery pages, and another is the full-size image when an individual image is clicked in the gallery.
  2. We need to add resizing-logic to our app. The point is to generate a resized image on the fly as it's demanded. This will keep non-popular images from polluting our hard drive, and it'll make sure the popular ones are, on subsequent requests, served in optimal sizes.
Responsive Images?

As this post explains, images in the modern web are incredibly complex. Instead of just <img src="mypic.jpg"> from the olden days, we now have something crazy like this:

<picture> <source media="(max-width: 700px)" sizes="(max-width: 500px) 50vw, 10vw" srcset="stick-figure-narrow.png 138w, stick-figure-hd-narrow.png 138w"> <source media="(max-width: 1400px)" sizes="(max-width: 1000px) 100vw, 50vw" srcset="stick-figure.png 416w, stick-figure-hd.png 416w"> <img src="stick-original.png" alt="Human"> </picture>

A combination of srcset, picture and sizes is necessary in a scenario where you're doubtful that if you use the same image for a smaller screen size, the primary subject of the image may become too small in size. You want to display a different image (more focused on the primary subject) in a different screen size, but still want to display separate assets of the same image based on device-pixel ratio, and want to customize height and width of the image based on viewport.

Since our images are photos and we always want them to be in their default DOM-specified position filling up the maximum of their parent container, we have no need for picture (which lets us define an alternative source for a different resolution or browser support --- like trying to render SVG, then PNG if SVG is unsupported) or sizes (which lets us define which viewport portion an image should occupy). We can get away with just using srcset, which loads a different size version of the same image depending on the screen size.

Adding srcset

The first location where we encounter images is in home-galleries-lazy-load.html.twig, the partial template that renders the home screen's galleries list.

<a class="gallery__link" href="{{ url('gallery.single-gallery', {id: gallery.id}) }}"> <img src="{{ gallery.images.first|getImageUrl }}" alt="{{ gallery.name }}" class="gallery__leading-image card-img-top"> </a>

We can see here that the image's link is fetched from a Twig filter, which can be found in the src/Twig/ImageRendererExtension.php file. It takes the image's ID and the route's name (defined in the annotation in ImageController's serveImageAction route) and generates a URL based on that formula: /image/{id}/raw -> replacing {id} with the ID given:

public function getImageUrl(Image $image) { return $this->router->generate('image.serve', [ 'id' => $image->getId(), ], RouterInterface::ABSOLUTE_URL); }

Let's change that to the following:

public function getImageUrl(Image $image, $size = null) { return $this->router->generate('image.serve', [ 'id' => $image->getId() . (($size) ? '--' . $size : ''), ], RouterInterface::ABSOLUTE_URL); }

Now, all our image URLs will have --x as a suffix, where x is their size. This is the change we'll apply to our img tag as well, in the form of srcset. Let's change it to:

<a class="gallery__link" href="{{ url('gallery.single-gallery', {id: gallery.id}) }}"> <img src="{{ gallery.images.first|getImageUrl }}" alt="{{ gallery.name }}" srcset=" {{ gallery.images.first|getImageUrl('1120') }} 1120w, {{ gallery.images.first|getImageUrl('720') }} 720w, {{ gallery.images.first|getImageUrl('400') }} 400w" class="gallery__leading-image card-img-top"> </a>

If we refresh the home page now, we'll notice the srcset's new sizes listed:

This isn't going to help us much, though. If our viewport is wide, this will request full-size images, despite them being thumbnails. So instead of srcset, it's better to use a fixed small thumbnail size here:

<a class="gallery__link" href="{{ url('gallery.single-gallery', {id: gallery.id}) }}"> <img src="{{ gallery.images.first|getImageUrl('250') }}" alt="{{ gallery.name }}" class="gallery__leading-image card-img-top"> </a>

We now have thumbnails-on-demand, but which get cached and fetched when they're already generated.

Let's hunt down other srcset locations now.

In templates/gallery/single-gallery.html.twig, we apply the same fix as before. We're dealing with thumbnails, so let's just shrink the file by adding the size parameter into our getImageUrl filter:

<img src="{{ image|getImageUrl(250) }}" alt="{{ image.originalFilename }}" class="single-gallery__item-image card-img-top">

And now for the srcset implementation, finally!

The individual image views are rendered with a JavaScript modal window at the bottom of the same single-gallery view:

{% block javascripts %} {{ parent() }} <script> $(function () { $('.single-gallery__item-image').on('click', function () { var src = $(this).attr('src'); var $modal = $('.single-gallery__modal'); var $modalBody = $modal.find('.modal-body'); $modalBody.html(''); $modalBody.append($('<img src="' + src + '" class="single-gallery__modal-image">')); $modal.modal({}); }); }) </script> {% endblock %}

There's an append call which adds the img element into the modal's body, so that's where our srcset attribute must go. But since our image URLs are dynamically generated, we can't really call the Twig filter from within the script. One alternative is to add the srcset into the thumbnails and then use it in the JS by copying it from the thumb elements, but this would not only make the full-sized images load in the background of the thumbnails (because our viewport is wide), but it would also call the filter 4 times for each thumbnail, slowing things down. Instead, let's create a new Twig filter in src/Twig/ImageRendererExtension.php which will generate the full srcset attribute for each image.

public function getImageSrcset(Image $image) { $id = $image->getId(); $sizes = [1120, 720, 400]; $string = ''; foreach ($sizes as $size) { $string .= $this->router->generate('image.serve', [ 'id' => $image->getId() . '--' . $size, ], RouterInterface::ABSOLUTE_URL).' '.$size.'w, '; } $string = trim($string, ', '); return html_entity_decode($string); }

We mustn't forget to register this filter:

public function getFilters() { return [ new Twig_SimpleFilter('getImageUrl', [$this, 'getImageUrl']), new Twig_SimpleFilter('getImageSrcset', [$this, 'getImageSrcset']), ]; }

We have to add these values into a custom attribute, which we'll call data-srcset on each individual thumbnail:

<img src="{{ image|getImageUrl(250) }}" alt="{{ image.originalFilename }}" data-srcset=" {{ image|getImageSrcset }}" class="single-gallery__item-image card-img-top">

Now each individual thumbnail has a data-srcset attribute with the required srcset values, but this doesn't trigger because it's in a custom attribute, data to be used later.

The final step is updating the JS to take advantage of this:

{% block javascripts %} {{ parent() }} <script> $(function () { $('.single-gallery__item-image').on('click', function () { var src = $(this).attr('src'); var srcset = $(this).attr('data-srcset'); var $modal = $('.single-gallery__modal'); var $modalBody = $modal.find('.modal-body'); $modalBody.html(''); $modalBody.append($('<img src="' + src + '" srcset="" + srcset + '" class="single-gallery__modal-image">')); $modal.modal({}); }); }) </script> {% endblock %}

The post Improving Performance Perception: On-demand Image Resizing appeared first on SitePoint.

Kategóriák: IT Hírek

How to Boost Your Server Performance with Varnish

h, 07/02/2018 - 07:58

Varnish Cache is an HTTP accelerator and reverse proxy developed by Danish consultant and FreeBSD core developer Poul-Henning Kamp, along with other developers at Norwegian Linpro AS. It was released in 2006.

According to Pingdom.com, a company focused on web performance, in 2012 Varnish was already famous among the world's top websites for its capacity to speed up web delivery, and it was being used by sites such as Wired, SlideShare, Zappos, SoundCloud, Weather.com, Business Insider, Answers.com, Urban Dictionary, MacRumors, DynDNS, OpenDNS, Lonely Planet, Technorati, ThinkGeek and Economist.com.

It is licensed under a two-clause BSD license. Varnish has a premium tier, Varnish Plus, focused on enterprise customers, which offers some extra features, modules, and support.

Although there are other solutions that also shine, Varnish is still a go-to solution that can dramatically improve website speed, reduce the strain on the web application server's CPU, and even serve as a protection layer from DDoS attacks. KeyCDN recommends deploying it on the origin server stack.

Varnish can sit on a dedicated machine in case of more demanding websites, and make sure that the origin servers aren't affected by the flood of requests.

At the time of this writing (November 2017), Varnish is at version 5.2.

How it Works

Caching in general works by keeping the pre-computed outputs of an application in memory, or on the disk, so that expensive computations don't have to be computed over and over on every request. Web Cache can be on the client (browser cache), or on the server. Varnish falls into the second category. It is usually configured so that it listens for requests on the standard HTTP port (80), and then serves the requested resource to the website visitor.

The first time a certain URL and path are requested, Varnish has to request it from the origin server in order to serve it to the visitor. This is called a CACHE MISS, which can be read in HTTP response headers, depending on the Varnish setup.

According to the docs,

when an object, any kind of content i.e. an image or a page, is not stored in the cache, then we have what is commonly known as a cache miss, in which case Varnish will go and fetch the content from the web server, store it and deliver a copy to the user and retain it in cache to serve in response to future requests.

When a particular URL or a resource is cached by Varnish and stored in memory, it can be served directly from server RAM; it doesn't need to be computed every time. Varnish will start delivering a CACHE HIT in a matter of microseconds.

This means that neither our origin server or our web application, including its database, are touched by future requests. They won't even be aware of the requests loaded on cached URLs.

The origin server --- or servers, in case we use Varnish as a load balancer --- are configured to listen on some non-standard port, like 8888, and Varnish is made aware of their address and port.

Varnish Features

Varnish is threaded. It's been reported that Varnish was able to handle over 200,000 requests per second on a single instance. If properly configured, the only bottlenecks of your web app will be network throughput and the amount of RAM. (This shouldn't be an unreasonable requirement, because it just needs to keep computed web pages in memory, so for most websites, a couple of gigabytes should be sufficient.)

Varnish is extendable via VMODS. These are modules that can use standard C libraries and extend Varnish functionality. There are community-contributed VMODS listed here. They range from header manipulation to Lua scripting, throttling of requests, authentication, and so on.

Varnish has its own domain-specific language, VCL. VCL provides comprehensive configurability. With a full-page caching server like Varnish, there are a lot of intricacies that need to be solved.

When we cache a dynamic website with dozens or hundreds of pages and paths, with GET query parameters, we'll want to exclude some of them from cache, or set different cache-expiration rules. Sometimes we'll want to cache certain Ajax requests, or exclude them from the cache. This varies from project to project, and can't be tailored in advance.

Sometimes we'll want Varnish to decide what to do with the request depending on request headers. Sometimes we'll want to pass requests directly to the back end with a certain cookie set.

To quote the Varnish book,

VCL provides subroutines that allow you to affect the handling of any single request almost anywhere in the execution chain.

Purging the cache often needs to be done dynamically --- triggered by publishing articles or updating the website. Purging also needs to be done as atomically as possible --- meaning it should target the smallest possible scope, like a single resource or path.

This means that specific rules need to be defined, with their order of priority in mind. Some examples can be found in the Varnish book (which is available to read online or as a downloadable PDF).

Varnish has a set of tools for monitoring and administering the server:

  • There's varnishtop, which lets us monitor requested URLs and their frequency.

  • varnishncsa can be used to print the Varnish Shared memory Log (VSL): it dumps everything pointing to a certain domain and subdomains.

  • varnishhist reads the VSL and presents a live histogram showing the distribution of the last number of requests, giving an overview of server and back-end performance.

  • varnishtest is used to test VCL configuration files and develop VMODS.

  • varnishstat displays statistics about our varnishd instance:

  • varnishlog is used to get data about specific clients and requests.

Varnish Software offers a set of commercial, paid solutions either built on top of Varnish cache, or extending its usage and helping with monitoring and management: Varnish Api Engine, Varnish Extend, Akamai Connector for Varnish, Varnish Administration Console (VAC), and Varnish Custom Statistics (VCS).

The post How to Boost Your Server Performance with Varnish appeared first on SitePoint.

Kategóriák: IT Hírek

Optimization Auditing: A Deep Dive into Chrome’s Dev Console

p, 06/29/2018 - 20:00

Chrome DevTools incorporates many sub-tools for debugging web applications on the client side --- like recording performance profiles and inspecting animations --- most of which you've likely been using since your early days of learning web development, mostly through the DevTools console.

Let's look at some of those tools, focusing particularly on the console and the performance metrics.

To access Chrome's DevTools:

  • right click anywhere on a page, click Inspect from the context menu
  • use the keyboard shortcuts Ctrl + Shift + I on Windows and Linux systems or Alt + Command + I on macOS
  • use the keyboard shortcuts Ctrl + Shift + J on Windows and Linux systems or Alt + Command + J on macOS.
The Snippets Tool

If you're frequently writing JavaScript code right in the console, make sure to use the Snippets feature of DevTools instead, which is similar to a code editor and provides mechanisms to write JavaScript code snippets, run them in the context of the current page and save them for later. It's better than writing multi-line JavaScript code directly in the console.

You can access the Snippets tool from the Sources panel. Once open, the console gets stacked below (if it doesn't, just press Escape) so you can write, run your code and see the console output at the same time.

Using the Chrome DevTools Console

You can use the console to interact with any web page using JavaScript. You can query and change the DOM and query/output different types of performance information.

The console can be opened either as a full-screen dedicated panel or as a drawer next to any other DevTools panel by pressing Escape while DevTools is open and has focus.

When working with the browser's console, if you want to enter multi-line expressions you need to use Shift + Enter, because just Enter will execute what's in the input line at that moment.

The console history

You can clear the console history in different ways:

  • by typing clear() in the console
  • by calling console.clear() method in the console or JavaScript code
  • by clicking on the red circle in the top left corner of the console
  • by pressing CTRL+L in macOS, Windows and Linux
  • by right-clicking in the Console and then pressing Clear console.

You can preserve the log (by enabling the Preserve log checkbox) between page refreshes or changes until you clear the console or close the tab.

You can save the history in the Console as a text file by right-clicking in the console and selecting Save as…, then choosing the location of a log file.

Console variables

The variables you create in the Console are persisted until you do a page refresh, so pay attention to when you're using keywords such as let or const when declaring variables. Running the same code or function for the second time will throw an Uncaught SyntaxError, saying that the identifier has already been declared. You can either use the OR (||) operator to check if the variable is already defined or you can use var to declare variables, since it doesn't complain for previously declared variables.

The post Optimization Auditing: A Deep Dive into Chrome’s Dev Console appeared first on SitePoint.

Kategóriák: IT Hírek

Improving Page Load Performance: Pingdom, YSlow and GTmetrix

cs, 06/28/2018 - 20:00

Optimizing websites for speed is a craft, and each craft requires tools. The most-used website optimization tools are GTmetrix, YSlow and Pingdom Tools.

GTmetrix is a rather advanced tool that offers a lot on its free tier, but it also offers premium tiers. If you sign up, you can compare multiple websites, multiple versions of the same website, tested under different conditions, and save tests for later viewing.

YSlow is still relevant, although its best days were those when Firebug ruled supreme among the browser inspectors. It offers a Chrome app and other implementations --- such as add-ons for Safari and Opera, a bookmarklet, an extension for PhantomJS, and so on.

For advanced users, PhantomJS integration means that one could, for example, automate the testing of many websites --- hundreds or thousands --- and export the results into the database.

YSlow's Ruleset Matrix has for a number of years been a measuring stick for website performance.

Pingdom Tools is a SaaS service that offers monitoring and reporting of website performance, and it has strengthened its market position in recent years. It also offers a DNS health check and website speed testing on its free tier, which is comparable to GTMetrix and YSlow.

For the purposes of this article, we purchased a fitting domain name --- ttfb.review --- and installed Drupal with some demo content on it. We also installed WordPress on wp.ttfb.review, and demo installations of Yii and Symfony on their respective subdomains.

We used the default WordPress starting installation. For Drupal, we used the Devel and Realistic Dummy Content extensions to generate demo content. For Symfony we used the Symfony demo application, and for Yii we used basic application template.

This way, we'll be able to compare these installations side-by-side, and point out the things that deserve attention.

Please be aware that these are development-level installations, used only for demonstration purposes. They aren't optimized for running in production, so our results are likely to be subpar.

The post Improving Page Load Performance: Pingdom, YSlow and GTmetrix appeared first on SitePoint.

Kategóriák: IT Hírek

How to Use Varnish and Cloudflare for Maximum Caching

sze, 06/27/2018 - 20:00

This article is part of a series on building a sample application --- a multi-image gallery blog --- for performance benchmarking and optimizations. (View the repo here.)

As we can see in this report, our site's landing page loads very quickly and generally scores well, but it could use another layer of caching and even a CDN to really do well.

To learn more about GTMetrix and other tools you can use to measure and debug performance, see Improving Page Load Performance: Pingdom, YSlow and GTmetrix

Let's use what we've learned in our previous Varnish post, along with the knowledge gained in the Intro to CDN and Cloudflare posts to really tune up our server's content delivery now.

Varnish

Varnish was created solely for the purpose of being a type of super-cache in front of a regular server.

Note: Given that Nginx itself is a pretty good server already, people usually opt for one or the other, not both. There's no harm in having both, but one does have to be wary of cache-busting problems which can occur. It's important to set them both up properly so that the cache of one of them doesn't remain stale while the other's is fresh. This can lead to different content being shown to different visitors at different times. Setting this up is a bit outside the context of this post, and will be covered in a future guide.

We can install Varnish by executing the following:

curl -L https://packagecloud.io/varnishcache/varnish5/gpgkey | sudo apt-key add - sudo apt-get update sudo apt-get install -y apt-transport-https

The current list of repos for Ubuntu does not have Varnish 5+ available, so additional repositories are required. If the file /etc/apt/sources.list.d/varnishcache_varnish5.list doesn't exist, create it. Add to it the following:

deb https://packagecloud.io/varnishcache/varnish5/ubuntu/ xenial main deb-src https://packagecloud.io/varnishcache/varnish5/ubuntu/ xenial main

Then, run:

sudo apt-get update sudo apt-get install varnish varnishd -V

The result should be something like:

$ varnishd -V varnishd (varnish-5.2.1 revision 67e562482) Copyright (c) 2006 Verdens Gang AS Copyright (c) 2006-2015 Varnish Software AS

We then change the server's default port to 8080. We're doing this because Varnish will be sitting on port 80 instead, and forwarding requests to 8080 as needed. If you're developing locally on Homestead Improved as instructed at the beginning of this series, the file you need to edit will be in /etc/nginx/sites-available/homestead.app. Otherwise, it'll probably be in /etc/nginx/sites-available/default.

server { listen 8080 default_server; listen [::]:8080 default_server ipv6only=on;

Next up, we'll configure Varnish itself by editing /etc/default/varnish and replacing the default port on the first line (6081) with 80:

DAEMON_OPTS="-a :80 \ -T localhost:6082 \ -f /etc/varnish/default.vcl \ -S /etc/varnish/secret \ -s malloc,256m"

The same needs to be done in /lib/systemd/system/varnish.service:

ExecStart=/usr/sbin/varnishd -j unix,user=vcache -F -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m

Finally, we can restart both Varnish and Nginx for the changes to take effect:

sudo service nginx restart sudo /etc/init.d/varnish stop sudo /etc/init.d/varnish start systemctl daemon-reload

The last command is there so that the previously edited varnish.service daemon settings also reload, otherwise it'll only take into account the /etc/default/varnish file's changes. The start + stop procedure is necessary for Varnish because of a current bug which doesn't release ports properly unless done this way.

Comparing the result with the previous one, we can see that the difference is minimal due to the landing page already being extremely optimized.

Sidenote

Both of the low grades are mainly the result of us "not serving from a consistent URL", as GTMetrix would put it:

This happens because we used random images to populate our galleries, and the sample of randomness was small, so some of them are repeated. This is fine and won't be an issue when the site is in production. In fact, this is one of the very rare cases where a site will by default score better in production than it does in development.

The post How to Use Varnish and Cloudflare for Maximum Caching appeared first on SitePoint.

Kategóriák: IT Hírek

Server-side Optimization with Nginx and pm-static

k, 06/26/2018 - 20:00

This article is part of a series on building a sample application --- a multi-image gallery blog --- for performance benchmarking and optimizations. (View the repo here.)

Let's continue optimizing our app. We're starting with on-the-fly thumbnail generation that takes 28 seconds per request, depending on the platform running your demo app (in my case it was a slow filesystem integration between host OS and Vagrant), and bring it down to a pretty acceptable 0.7 seconds.

Admittedly, this 28 seconds should only happen on initial load. After the tuning, we were able to achieve production-ready times:

Troubleshooting

It is assumed that you've gone through the bootstrapping process and have the app running on your machine --- either virtual or real.

Note: if you're hosting the Homestead Improved box on a Windows machine, there might be an issue with shared folders. This can be solved by adding type: "nfs" setting to the folder in Homestead.yaml:

You should also run vagrant up from a shell/powershell interface that has administrative privileges if problems persist (right-click, run as administrator).

In one example before doing this, we got 20 to 30 second load times on every request, and couldn't get a rate faster than one request per second (it was closer to 0.5 per second):

The Process

Let's go through the testing process. We installed Locust on our host, and created a very simple locustfile.py:

from locust import HttpLocust, TaskSet, task class UserBehavior(TaskSet): @task(1) def index(self): self.client.get("/") class WebsiteUser(HttpLocust): task_set = UserBehavior min_wait = 300 max_wait = 1000

Then we downloaded ngrok to our guest machine and tunneled all HTTP connections through it, so that we can test our application over a static URL.

Then we started Locust and swarmed our app with 100 parallel users:

Our server stack consisted of PHP 7.1.10, Nginx 1.13.3 and MySQL 5.7.19, on Ubuntu 16.04.

PHP-FPM and its Process Manager Setting

php-fpm spawns its own processes, independent of the web-server process. Management of the number of these processes is configured in /etc/php/7.1/fpm/pool.d/www.conf (7.1 here can be exchanged for the actual PHP version number currently in use).

In this file, we find the pm setting. This setting can be set to dynamic, ondemand and static. Dynamic is maybe the most common wisdom; it allows the server to juggle the number of spawned PHP processes between several settings:

pm = dynamic ; The number of child processes to be created when pm is set to 'static' and the ; maximum number of child processes when pm is set to 'dynamic' or 'ondemand'. ; This value sets the limit on the number of simultaneous requests that will be ; served. pm.max_children = 6 ; The number of child processes created on startup. ; Note: Used only when pm is set to 'dynamic' ; Default Value: min_spare_servers + (max_spare_servers - min_spare_servers) / 2 pm.start_servers = 3 ; The desired minimum number of idle server processes ; Note: Used only when pm is set to 'dynamic' ; Note: Mandatory when pm is set to 'dynamic' pm.min_spare_servers = 2 ; The desired maximum number of idle server proceses ; Note: Used only when pm is set to 'dynamic' ; Note: Mandatory when pm is set to 'dynamic' pm.max_spare_servers = 4

The meanings of these values are self-explanatory, and the spawning of processes is being done on demand, but constrained by these minimum and maximum values.

After fixing the Windows shared-folders issue with nfs, and testing with Locust, we were able to get approximately five requests per second, with around 17–19% failures, with 100 concurrent users. Once it was swarmed with requests, the server slowed down and each request took over ten seconds to finish.

Then we changed the pm setting to ondemand.

Ondemand means that there are no minimum processes: once the requests stop, all the processes will stop. Some advocate this setting, because it means the server won't be spending any resources in its idle state, but for the dedicated (non-shared) server instances this isn't necessarily the best. Spawning a process includes an overhead, and what is gained in memory is being lost in time needed to spawn processes on-demand. The settings that are relevant here are:

pm.max_children = 6 ; and pm.process_idle_timeout = 20s; ; The number of seconds after which an idle process will be killed. ; Note: Used only when pm is set to 'ondemand' ; Default Value: 10s

When testing, we increased these settings a bit, having to worry about resources less.

There's also pm.max_requests, which can be changed, and which designates the number of requests each child process should execute before respawning.

This setting is a tradeoff between speed and stability, where 0 means unlimited.

ondemand didn't bring much change, except that we noticed more initial waiting time when we started swarming our application with requests, and more initial failures. In other words, there were no big changes: the application was able to serve around four to maximum six requests per second. Waiting time and rate of failures were similar to the dynamic setup.

Then we tried the pm = static setting, allowing our PHP processes to take over the maximum of the server's resources, short of swapping, or driving the CPU to a halt. This setting means we're forcing the maximum out of our system at all times. It also means that --- within our server's constraints --- there won't be any spawning overhead time cost.

What we saw was an improvement of 20%. The rate of failed requests was still significant, though, and the response time was still not very good. The system was far from being ready for production.

However, on Pingdom Tools, we got a bearable 3.48 seconds when the system was not under pressure:

This meant that pm static was an improvement, but in the case of a bigger load, it would still go down.

In one of the previous articles, we explained how Nginx can itself serve as a caching system, both for static and dynamic content. So we reached for the Nginx wizardry, and tried to bring our application to a whole new level of performance.

And we succeeded. Let's see how.

The post Server-side Optimization with Nginx and pm-static appeared first on SitePoint.

Kategóriák: IT Hírek

Using Background Processing to Speed Up Page Load Times

h, 06/25/2018 - 20:00

This article is part of a series on building a sample application --- a multi-image gallery blog --- for performance benchmarking and optimizations. (View the repo here.)

In a previous article, we've added on-demand image resizing. Images are resized on the first request and cached for later use. By doing this, we've added some overhead to the first load; the system has to render thumbnails on the fly and is "blocking" the first user's page render until image rendering is done.

The optimized approach would be to render thumbnails after a gallery is created. You may be thinking, "Okay, but we'll then block the user who is creating the gallery?" Not only would it be a bad user experience, but it also isn't a scalable solution. The user would get confused about long loading times or, even worse, encounter timeouts and/or errors if images are too heavy to be processed. The best solution is to move these heavy tasks into the background.

Background Jobs

Background jobs are the best way of doing any heavy processing. We can immediately notify our user that we've received their request and scheduled it for processing. The same way as YouTube does with uploaded videos: they aren't accessible after the upload. The user needs to wait until the video is processed completely to preview or share it.

Processing or generating files, sending emails or any other non-critical tasks should be done in the background.

How Does Background Processing Work?

There are two key components in the background processing approach: job queue and worker(s). The application creates jobs that should be handled while workers are waiting and taking from the queue one job at a time.

You can create multiple worker instances (processes) to speed up processing, chop a big job up into smaller chunks and process them simultaneously. It's up to you how you want to organize and manage background processing, but note that parallel processing isn't a trivial task: you should take care of potential race conditions and handle failed tasks gracefully.

Our tech stack

We're using the Beanstalkd job queue to store jobs, the Symfony Console component to implement workers as console commands and Supervisor to take care of worker processes.

If you're using Homestead Improved, Beanstalkd and Supervisor are already installed so you can skip the installation instructions below.

Installing Beanstalkd

Beanstalkd is

a fast work queue with a generic interface originally designed for reducing the latency of page views in high-volume web applications by running time-consuming tasks asynchronously.

There are many client libraries available that you can use. In our project, we're using Pheanstalk.

To install Beanstalkd on your Ubuntu or Debian server, simply run sudo apt-get install beanstalkd. Take a look at the official download page to learn how to install Beanstalkd on other OSes.

Once installed, Beanstalkd is started as a daemon, waiting for clients to connect and create (or process) jobs:

/etc/init.d/beanstalkd Usage: /etc/init.d/beanstalkd {start|stop|force-stop|restart|force-reload|status}

Install Pheanstalk as a dependency by running composer require pda/pheanstalk.

The queue will be used for both creating and fetching jobs, so we'll centralize queue creation in a factory service JobQueueFactory:

<?php namespace App\Service; use Pheanstalk\Pheanstalk; class JobQueueFactory { private $host = 'localhost'; private $port = '11300'; const QUEUE_IMAGE_RESIZE = 'resize'; public function createQueue(): Pheanstalk { return new Pheanstalk($this->host, $this->port); } }

Now we can inject the factory service wherever we need to interact with Beanstalkd queues. We are defining the queue name as a constant and referring to it when putting the job into the queue or watching the queue in workers.

Installing Supervisor

According to the official page, Supervisor is a

client/server system that allows its users to monitor and control a number of processes on UNIX-like operating systems.

We'll be using it to start, restart, scale and monitor worker processes.

Install Supervisor on your Ubuntu/Debian server by running
sudo apt-get install supervisor. Once installed, Supervisor will be running in the background as a daemon. Use supervisorctl to control supervisor processes:

$ sudo supervisorctl help default commands (type help <topic>): ===================================== add exit open reload restart start tail avail fg pid remove shutdown status update clear maintail quit reread signal stop version

To control processes with Supervisor, we first have to write a configuration file and describe how we want our processes to be controlled. Configurations are stored in /etc/supervisor/conf.d/. A simple Supervisor configuration for resize workers would look like this:

[program:resize-worker] process_name=%(program_name)s_%(process_num)02d command=php PATH-TO-YOUR-APP/bin/console app:resize-image-worker autostart=true autorestart=true numprocs=5 stderr_logfile = PATH-TO-YOUR-APP/var/log/resize-worker-stderr.log stdout_logfile = PATH-TO-YOUR-APP/var/log/resize-worker-stdout.log

We're telling Supervisor how to name spawned processes, the path to the command that should be run, to automatically start and restart the processes, how many processes we want to have and where to log output. Learn more about Supervisor configurations here.

Resizing images in the background

Once we have our infrastructure set up (i.e., Beanstalkd and Supervisor installed), we can modify our app to resize images in the background after the gallery is created. To do so, we need to:

  • update image serving logic in the ImageController
  • implement resize workers as console commands
  • create Supervisor configuration for our workers
  • update fixtures and resize images in the fixture class.

The post Using Background Processing to Speed Up Page Load Times appeared first on SitePoint.

Kategóriák: IT Hírek

Improving Performance Perception with Pingdom and GTmetrix

p, 06/22/2018 - 20:00

This article is part of a series on building a sample application --- a multi-image gallery blog --- for performance benchmarking and optimizations. (View the repo here.)

In this article, we'll analyze our gallery application using the tools we explained in the previous guide, and we'll look at possible ways to further improve its performance.

As per the previous post, please set up Ngrok and pipe to the locally hosted app through it, or host the app on a demo server of your own. This static URL will enable us to test our app with external tools like GTmetrix and Pingdom Tools.

We went and scanned our website with GTmetrix to see how we can improve it. We see that results, albeit not catastrophically bad, still have room for improvement.

The first tab --- PageSpeed --- contains a list of recommendations by Google. The first item under the PageSpeed tab --- a warning about a consistent URL --- pertains to our application outputting the images randomly, so that is an item we will skip. The next thing we can do something about is browser caching.

Browser Caching

We see that there is a main.css file that needs its Expires headers set, and the images in the gallery need the same thing. Now, the first idea for these static files would be to set this in our Nginx configuration:

location ~* \.(?:ico|css|js|gif|jpe?g|png)$ { expires 14d; }

We can simply put this inside our server block and leave it to Nginx, right?

Well, not really. This will take care of our static files, like CSS, but the /raw images we are being warned about aren't really that static. So this snippet in our Nginx configuration won't exactly fix this issue so easily. For our images, we have an actual controller that creates these on the fly, so it would be ideal if we could set our response headers right there, in the controller. For some reason, these weren't being set properly by Glide.

Maybe we could set our Nginx directive in a way to include the raw resources, but we felt the controller approach to be more future-proof. This is because we aren't sure what other content may end up with an raw suffix eventually --- maybe some videos, or even audio files.

So, we opened /src/ImageController.php in our image gallery app, and dropped these two lines inside of our serveImageAction(), just before the line return $response:

// cache for 2 weeks $response->setSharedMaxAge(1209600); // (optional) set a custom Cache-Control directive $response->headers->addCacheControlDirective('must-revalidate', true);

This will modify our dynamic image responses by adding the proper Cache Control and Expires headers.

Symfony has more comprehensive options for the caching of responses, as documented here.

Having restarted Nginx, we re-tested our app in GTmetrix, and lo and behold:

The post Improving Performance Perception with Pingdom and GTmetrix appeared first on SitePoint.

Kategóriák: IT Hírek

MySQL Performance Boosting with Indexes and Explain

cs, 06/21/2018 - 20:00

Techniques to improve application performance can come from a lot of different places, but normally the first thing we look at --- the most common bottleneck --- is the database. Can it be improved? How can we measure and understand what needs and can be improved?

One very simple yet very useful tool is query profiling. Enabling profiling is a simple way to get a more accurate time estimate of running a query. This is a two-step process. First, we have to enable profiling. Then, we call show profiles to actually get the query running time.

Let's imagine we have the following insert in our database (and let's assume User 1 and Gallery 1 are already created):

INSERT INTO `homestead`.`images` (`id`, `gallery_id`, `original_filename`, `filename`, `description`) VALUES (1, 1, 'me.jpg', 'me.jpg', 'A photo of me walking down the street'), (2, 1, 'dog.jpg', 'dog.jpg', 'A photo of my dog on the street'), (3, 1, 'cat.jpg', 'cat.jpg', 'A photo of my cat walking down the street'), (4, 1, 'purr.jpg', 'purr.jpg', 'A photo of my cat purring');

Obviously, this amount of data will not cause any trouble, but let's use it to do a simple profile. Let's consider the following query:

SELECT * FROM `homestead`.`images` AS i WHERE i.description LIKE '%street%';

This query is a good example of one that can become problematic in the future if we get a lot of photo entries.

To get an accurate running time on this query, we would use the following SQL:

set profiling = 1; SELECT * FROM `homestead`.`images` AS i WHERE i.description LIKE '%street%'; show profiles;

The result would look like the following:

Query_Id Duration Query 1 0.00016950 SHOW WARNINGS 2 0.00039200 SELECT * FROM homestead.images AS i \nWHERE i.description LIKE \'%street%\'\nLIMIT 0, 1000 3 0.00037600 SHOW KEYS FROM homestead.images 4 0.00034625 SHOW DATABASES LIKE \'homestead\ 5 0.00027600 SHOW TABLES FROM homestead LIKE \'images\' 6 0.00024950 SELECT * FROM homestead.images WHERE 0=1 7 0.00104300 SHOW FULL COLUMNS FROM homestead.images LIKE \'id\'

As we can see, the show profiles; command gives us times not only for the original query but also for all the other queries that are made. This way we can accurately profile our queries.

But how can we actually improve them?

We can either rely on our knowledge of SQL and improvise, or we can rely on the MySQL explain command and improve our query performance based on actual information.

Explain is used to obtain a query execution plan, or how MySQL will execute our query. It works with SELECT, DELETE, INSERT, REPLACE, and UPDATE statements, and it displays information from the optimizer about the statement execution plan. The official documentation does a pretty good job of describing how explain can help us:

With the help of EXPLAIN, you can see where you should add indexes to tables so that the statement executes faster by using indexes to find rows. You can also use EXPLAIN to check whether the optimizer joins the tables in an optimal order.

The post MySQL Performance Boosting with Indexes and Explain appeared first on SitePoint.

Kategóriák: IT Hírek

PHP-level Performance Optimization with Blackfire

sze, 06/20/2018 - 20:00

Throughout the past few months, we've introduced Blackfire and ways in which it can be used to detect application performance bottlenecks. In this post, we'll apply it to our freshly started project to try and find the low-points and low-hanging fruit which we can pick to improve our app's performance.

If you're using Homestead Improved (and you should be), Blackfire is already installed. Blackfire should only ever be installed in development, not in production, so it's fine to only have it there.

Note: Blackfire can be installed in production, as it doesn't really trigger for users unless they manually initiate it with the installed Blackfire extension. However, it's worth noting that defining profile triggers on certain actions or users that don't need the extension will incur a performance penalty for the end user. When Blackfire-testing live, make the test sessions short and effective, and avoid doing so under heavy load.

While it's useful to be introduced to Blackfire before diving into this, applying the steps in this post won't require any prior knowledge; we'll start from zero.

Setup

The following are useful terms when evaluating graphs produced by Blackfire.

  • Reference Profile: We usually need to run our first profile as a reference profile. This profile will be the performance baseline of our application. We can compare any profile with the reference, to measure the performance achievements.

  • Exclusive Time: The amount of time spent on a function/method to be executed, without considering the time spent for its external calls.

  • Inclusive Time: The total time spent to execute a function including all the external calls.

  • Hot Paths: Hot Paths are the parts of our application that were most active during the profile. These could be the parts that consumed more memory or took more CPU time.

The first step is registering for an account at Blackfire. The account page will have the tokens and IDs which need to be placed into Homestead.yaml after cloning the project. There's a placeholder for all those values at the bottom:

# blackfire: # - id: foo # token: bar # client-id: foo # client-token: bar

After uncommenting the rows and replacing the values, we need to install the Chrome companion.

The Chrome companion is useful only when needing to trigger profiling manually --- which will be the majority of your use cases. There are other integrations available as well, a full list of which can be found here.

Optimization with Blackfire

We'll test the home page: the landing page is arguably the most important part of any website, and if that takes too long to load, we're guaranteed to lose our visitors. They'll be gone before Google Analytics can kick in to register the bounce! We could test pages on which users add images, but read-only performance is far more important than write performance, so we'll focus on the former.

This version of the app loads all the galleries and sorts them by age.

Testing is simple. We open the page we want to benchmark, click the extension's button in the browser, and select "Profile!".

Here's the resulting graph:

In fact, we can see here that the execution time inclusive to exclusive is 100% on the PDO execution. Specifically, this means that the whole dark pink part is spent inside this function and that this function in particular is not waiting for any other function. This is the function being waited on. Other method calls might have light pink bars far bigger than PDO's, but those light pink parts are a sum of all the smaller light pink parts of depending functions, which means that looked at individually, those functions aren't the problem. The dark ones need to be handled first; they are the priority.

Also, switching to RAM mode reveals that while the whole call used almost a whopping 40MB of RAM, the vast majority is in the Twig rendering, which makes sense: it is showing a lot of data, after all.

In the diagram, hot paths have thick borders and generally indicate bottlenecks. Intensive nodes can be part of the hot path, but also be completely outside it. Intensive nodes are nodes a lot of time is spent in for some reason, and can be indicative of problems just as much.

By looking at the most problematic methods and clicking around on relevant nodes, we can identify that PDOExecute is the most problematic bottleneck, while unserialize uses the most RAM relative to other methods. If we apply some detective work and follow the flow of methods calling each other, we'll notice that both of these problems are caused by the fact that we're loading the whole set of galleries on the home page. PDOExecute takes forever in memory and wall time to find them and sort them, and Doctrine takes ages and endless CPU cycles to turn them into renderable entities with unserialize to loop through them in a twig template. The solution seems simple --- add pagination to the home page!

By adding a PER_PAGE constant into the HomeController and setting it to something like 12, and then using that pagination constant in the fetching procedure, we block the first call to the newest 12 galleries:

$galleries = $this->em->getRepository(Gallery::class)->findBy([], ['createdAt' => 'DESC'], self::PER_PAGE);

We'll trigger a lazy load when the user reaches the end of the page when scrolling, so we need to add some JS to the home view:

{% block javascripts %} {{ parent() }} <script> $(function () { var nextPage = 2; var $galleriesContainer = $('.home__galleries-container'); var $lazyLoadCta = $('.home__lazy-load-cta'); function onScroll() { var y = $(window).scrollTop() + $(window).outerHeight(); if (y >= $('body').innerHeight() - 100) { $(window).off('scroll.lazy-load'); $lazyLoadCta.click(); } } $lazyLoadCta.on('click', function () { var url = "{{ url('home.lazy-load') }}"; $.ajax({ url: url, data: {page: nextPage}, success: function (data) { if (data.success === true) { $galleriesContainer.append(data.data); nextPage++; $(window).on('scroll.lazy-load', onScroll); } } }); }); $(window).on('scroll.lazy-load', onScroll); }); </script> {% endblock %}

Since annotations are being used for routes, it's easy to just add a new method into the HomeController to lazily load our galleries when triggered:

/** * @Route("/galleries-lazy-load", name="home.lazy-load") */ public function homeGalleriesLazyLoadAction(Request $request) { $page = $request->get('page', null); if (empty($page)) { return new JsonResponse([ 'success' => false, 'msg' => 'Page param is required', ]); } $offset = ($page - 1) * self::PER_PAGE; $galleries = $this->em->getRepository(Gallery::class)->findBy([], ['createdAt' => 'DESC'], 12, $offset); $view = $this->twig->render('partials/home-galleries-lazy-load.html.twig', [ 'galleries' => $galleries, ]); return new JsonResponse([ 'success' => true, 'data' => $view, ]); }

The post PHP-level Performance Optimization with Blackfire appeared first on SitePoint.

Kategóriák: IT Hírek

Building an Image Gallery Blog with Symfony Flex: Data Testing

k, 06/19/2018 - 20:00

In the previous article, we demonstrated how to set up a Symfony project from scratch with Flex, and how to create a simple set of fixtures and get the project up and running.

The next step on our journey is to populate the database with a somewhat realistic amount of data to test application performance.

Note: if you did the “Getting started with the app” step in the previous post, you've already followed the steps outlined in this post. If that's the case, use this post as an explainer on how it was done.

As a bonus, we'll demonstrate how to set up a simple PHPUnit test suite with basic smoke tests.

More Fake Data

Once your entities are polished, and you've had your "That's it! I'm done!" moment, it's a perfect time to create a more significant dataset that can be used for further testing and preparing the app for production.

Simple fixtures like the ones we created in the previous article are great for the development phase, where loading ~30 entities is done quickly, and it can often be repeated while changing the DB schema.

Testing app performance, simulating real-world traffic and detecting bottlenecks requires bigger datasets (i.e. a larger amount of database entries and image files for this project). Generating thousands of entries takes some time (and computer resources), so we want to do it only once.

We could try increasing the COUNT constant in our fixture classes and seeing what will happen:

// src/DataFixtures/ORM/LoadUsersData.php class LoadUsersData extends AbstractFixture implements ContainerAwareInterface, OrderedFixtureInterface { const COUNT = 500; ... } // src/DataFixtures/ORM/LoadGalleriesData.php class LoadGalleriesData extends AbstractFixture implements ContainerAwareInterface, OrderedFixtureInterface { const COUNT = 1000; ... }

Now, if we run bin/refreshDb.sh, after some time we'll probably get a not-so-nice message like PHP Fatal error: Allowed memory size of N bytes exhausted.

Apart from slow execution, every error would result in an empty database because EntityManager is flushed only at the very end of the fixture class. Additionally, Faker is downloading a random image for every gallery entry. For 1,000 galleries with 5 to 10 images per gallery that would be 5,000 - 10,000 downloads, which is really slow.

There are excellent resources on optimizing Doctrine and Symfony for batch processing, and we're going to use some of these tips to optimize fixtures loading.

First, we'll define a batch size of 100 galleries. After every batch, we'll flush and clear the EntityManager (i.e., detach persisted entities) and tell the garbage collector to do its job.

To track progress, let's print out some meta information (batch identifier and memory usage).

Note: After calling $manager->clear(), all persisted entities are now unmanaged. The entity manager doesn't know about them anymore, and you'll probably get an "entity-not-persisted" error.

The key is to merge the entity back to the manager $entity = $manager->merge($entity);

Without the optimization, memory usage is increasing while running a LoadGalleriesData fixture class:

> loading [200] App\DataFixtures\ORM\LoadGalleriesData 100 Memory usage (currently) 24MB / (max) 24MB 200 Memory usage (currently) 26MB / (max) 26MB 300 Memory usage (currently) 28MB / (max) 28MB 400 Memory usage (currently) 30MB / (max) 30MB 500 Memory usage (currently) 32MB / (max) 32MB 600 Memory usage (currently) 34MB / (max) 34MB 700 Memory usage (currently) 36MB / (max) 36MB 800 Memory usage (currently) 38MB / (max) 38MB 900 Memory usage (currently) 40MB / (max) 40MB 1000 Memory usage (currently) 42MB / (max) 42MB

Memory usage starts at 24 MB and increases for 2 MB for every batch (100 galleries). If we tried to load 100,000 galleries, we'd need 24 MB + 999 (999 batches of 100 galleries, 99,900 galleries) * 2 MB = ~2 GB of memory.

After adding $manager->flush() and gc_collect_cycles() for every batch, removing SQL logging with $manager->getConnection()->getConfiguration()->setSQLLogger(null) and removing entity references by commenting out $this->addReference('gallery' . $i, $gallery);, memory usage becomes somewhat constant for every batch.

// Define batch size outside of the for loop $batchSize = 100; ... for ($i = 1; $i <= self::COUNT; $i++) { ... // Save the batch at the end of the for loop if (($i % $batchSize) == 0 || $i == self::COUNT) { $currentMemoryUsage = round(memory_get_usage(true) / 1024); $maxMemoryUsage = round(memory_get_peak_usage(true) / 1024); echo sprintf("%s Memory usage (currently) %dKB/ (max) %dKB \n", $i, $currentMemoryUsage, $maxMemoryUsage); $manager->flush(); $manager->clear(); // here you should merge entities you're re-using with the $manager // because they aren't managed anymore after calling $manager->clear(); // e.g. if you've already loaded category or tag entities // $category = $manager->merge($category); gc_collect_cycles(); } }

As expected, memory usage is now stable:

> loading [200] App\DataFixtures\ORM\LoadGalleriesData 100 Memory usage (currently) 24MB / (max) 24MB 200 Memory usage (currently) 26MB / (max) 28MB 300 Memory usage (currently) 26MB / (max) 28MB 400 Memory usage (currently) 26MB / (max) 28MB 500 Memory usage (currently) 26MB / (max) 28MB 600 Memory usage (currently) 26MB / (max) 28MB 700 Memory usage (currently) 26MB / (max) 28MB 800 Memory usage (currently) 26MB / (max) 28MB 900 Memory usage (currently) 26MB / (max) 28MB 1000 Memory usage (currently) 26MB / (max) 28MB

Instead of downloading random images every time, we can prepare 15 random images and update the fixture script to randomly choose one of them instead of using Faker's $faker->image() method.

Let's take 15 images from Unsplash and save them in var/demo-data/sample-images.

Then, update the LoadGalleriesData::generateRandomImage method:

private function generateRandomImage($imageName) { $images = [ 'image1.jpeg', 'image10.jpeg', 'image11.jpeg', 'image12.jpg', 'image13.jpeg', 'image14.jpeg', 'image15.jpeg', 'image2.jpeg', 'image3.jpeg', 'image4.jpeg', 'image5.jpeg', 'image6.jpeg', 'image7.jpeg', 'image8.jpeg', 'image9.jpeg', ]; $sourceDirectory = $this->container->getParameter('kernel.project_dir') . '/var/demo-data/sample-images/'; $targetDirectory = $this->container->getParameter('kernel.project_dir') . '/var/uploads/'; $randomImage = $images[rand(0, count($images) - 1)]; $randomImageSourceFilePath = $sourceDirectory . $randomImage; $randomImageExtension = explode('.', $randomImage)[1]; $targetImageFilename = sha1(microtime() . rand()) . '.' . $randomImageExtension; copy($randomImageSourceFilePath, $targetDirectory . $targetImageFilename); $image = new Image( Uuid::getFactory()->uuid4(), $randomImage, $targetImageFilename ); return $image; }

It's a good idea to remove old files in var/uploads when reloading fixtures, so I'm adding rm var/uploads/* command to bin/refreshDb.sh script, immediately after dropping the DB schema.

Loading 500 users and 1000 galleries now takes ~7 minutes and ~28 MB of memory (peak usage).

Dropping database schema... Database schema dropped successfully! ATTENTION: This operation should not be executed in a production environment. Creating database schema... Database schema created successfully! > purging database > loading [100] App\DataFixtures\ORM\LoadUsersData 300 Memory usage (currently) 10MB / (max) 10MB 500 Memory usage (currently) 12MB / (max) 12MB > loading [200] App\DataFixtures\ORM\LoadGalleriesData 100 Memory usage (currently) 24MB / (max) 26MB 200 Memory usage (currently) 26MB / (max) 28MB 300 Memory usage (currently) 26MB / (max) 28MB 400 Memory usage (currently) 26MB / (max) 28MB 500 Memory usage (currently) 26MB / (max) 28MB 600 Memory usage (currently) 26MB / (max) 28MB 700 Memory usage (currently) 26MB / (max) 28MB 800 Memory usage (currently) 26MB / (max) 28MB 900 Memory usage (currently) 26MB / (max) 28MB 1000 Memory usage (currently) 26MB / (max) 28MB

Take a look at the fixture classes source: LoadUsersData.php and LoadGalleriesData.php.

The post Building an Image Gallery Blog with Symfony Flex: Data Testing appeared first on SitePoint.

Kategóriák: IT Hírek

Building an Image Gallery Blog with Symfony Flex: the Setup

h, 06/18/2018 - 20:00

This post begins our journey into Performance Month's zero-to-hero project. In this part, we'll set our project up so we can fine tune it throughout the next few posts, and bring it to a speedy perfection.

Now and then you have to create a new project repository, run that git init command locally and kick off a new awesome project. I have to admit I like the feeling of starting something new; it's like going on an adventure!

Lao Tzu said:

The journey of a thousand miles begins with one step

We can think about the project setup as the very first step of our thousand miles (users!) journey. We aren't sure where exactly we are going to end up, but it will be fun!

We also should keep in mind the advice from prof. Donald Knuth:

Premature optimization is the root of all evil (or at least most of it) in programming.

Our journey towards a stable, robust, high-performance web app will start with the simple but functional application --- the so-called minimum viable product (MVP). We'll populate the database with random content, do some benchmarks and improve performance incrementally. Every article in this series will be a checkpoint on our journey!

This article will cover the basics of setting up the project and organizing files for our Symfony Flex project. I'll also show you some tips, tricks and helper scripts I'm using for speeding up the development.

What Are We Building?

Before starting any project, you should have a clear vision of the final destination. Where are you headed? Who will be using your app and how? What are the main features you're building? Once you have that knowledge, you can prepare your environment, third-party libraries, and dive into developing the next big thing.

In this series of articles, we'll be building a simple image gallery blog where users can register or log in, upload images, and create simple public image galleries with descriptions written in Markdown format.

We'll be using the new Symfony Flex and Homestead (make sure you've read tutorials on them, as we're not going to cover them here). We picked Flex because Symfony 4 is just about to come out (if it hasn't already, by the time you're reading this), because it's infinitely lighter than the older version and lends itself perfectly to step-by-step optimization, and it's also the natural step in the evolution of the most popular enterprise PHP framework out there.

All the code referenced in this article is available at the GitHub repo.

We're going to use the Twig templating engine, Symfony forms, and Doctrine ORM with UUIDs as primary keys.

Entities and routes will use annotations; we'll have simple email/password based authentication, and we'll prepare data fixtures to populate the database.

Getting Started with the app

To try out the example we've prepared, do the following:

  • Set up an empty database called "blog".
  • Clone the project repository from GitHub.
  • Run composer install.
  • If you now open the app in your browser, you should see an exception regarding missing database tables. That's fine, since we haven't created any tables so far.
  • Update the .env file in your project root directory with valid database connection string (i.e., update credentials).
  • Run the database init script ./bin/refreshDb.sh and wait until it generates some nice image galleries.
  • Open the app in your browser and enjoy!

After executing bin/refreshDb.sh you should be able to see the home page of our site:

You can log in to the app with credentials user1@mailinator.com and password 123456. See LoadUserData fixture class for more details regarding generated users.

Starting from scratch

In this section, we'll describe how to set up a new project from scratch. Feel free to take a look at the sample app codebase and see the details.

After creating a new project based on symfony/skeleton by executing the command

composer create-project "symfony/skeleton:^3.3" multi-user-gallery-blog

… we can first set minimum stability to "dev" because of some cutting edge packages:

composer config minimum-stability dev

… and then require additional packages (some of them are referenced by their aliases, the new feature brought by Flex):

composer req annotations security orm template asset validator ramsey/uuid-doctrine

Dependencies used only in the dev environment are required with the --dev flag:

composer req --dev fzaninotto/faker doctrine/Doctrine-Fixtures-Bundle

Flex is doing some serious work for us behind the scenes, and most of the libraries (or bundles) are already registered and configured with good-enough defaults! Check the config directory. You can check all the dependencies used in this project in the composer.json file.

Routes are defined by annotations, so the following will be automatically added into config/routes.yaml:

controllers: resource: ../src/Controller/ type: annotation Database, Scripts and Fixtures

Configure the DATABASE_URL environment variable (for example, by editing the .env file) to set up a working DB connection. If you're using our own Homestead Improved (recommended), you've got a database set up called homestead with the user / pass homestead / secret. A DB schema can be generated from existing entities by executing:

./bin/console doctrine:schema:create

If this doesn't run, try executing the console by invoking the PHP binary, like so:

php bin/console doctrine:schema:create

If this step executed fine in the "Getting Started with the app" section above, you should be able to see newly created tables in the database (for Gallery, Image and User entities).

If you want to drop the database schema, you can run:

./bin/console doctrine:schema:drop --full-database --force Fake it 'til you make it!

I can't imagine developing an app today without having data fixtures (i.e., scripts for seeding the DB). With a few simple scripts, you can populate your database with realistic content, which is useful when it comes to rapid app development and testing, but it's also a requirement for a healthy CI pipeline.

I find the Doctrine Fixtures Bundle to be an excellent tool for handling data fixtures as it supports ordered fixtures (i.e., you can control the order of execution), sharing objects (via references) between scripts, and accessing the service container.

Default Symfony services configuration doesn't allow public access to services, as best practice is to inject all dependencies. We'll need some services in our fixtures, so I'm going to make all services in App\Services publicly available by adding the following to config/services.yaml:

App\Service\: resource: '../src/Service/*' public: true

I'm also using Faker to get random but realistic data (names, sentences, texts, images, addresses, …).

Take a look at the script for seeding galleries with random images to get a feeling of how cool this combination is.

Usually, I combine commands for dropping the existing DB schema, creating the new DB schema, loading data fixtures, and other repetitive tasks into a single shell script --- for example, bin/refreshDb.sh --- so I can easily regenerate the schema and load dummy data:

# Drop schema ./bin/console doctrine:schema:drop --full-database --force # Create schema ./bin/console doctrine:schema:create # Load fixtures ./bin/console doctrine:fixtures:load -n --fixtures src/DataFixtures/ORM # Install assets ./bin/console assets:install --symlink # Clear cache ./bin/console cache:clear

Make sure you restrict execution of this script on production, or you're going to have some serious fun at one point.

One can argue that randomly generated data can't reproduce different edge cases, so your CI can sometimes fail or pass depending on the data generation. It's true, and you should make sure all edge cases are covered with your fixtures.

Every time you find an edge case causing a bug, make sure you add it to data fixtures. This will help you build a more robust system and prevent similar errors in the future.

The post Building an Image Gallery Blog with Symfony Flex: the Setup appeared first on SitePoint.

Kategóriák: IT Hírek

Apache vs Nginx Performance: Optimization Techniques

sze, 06/13/2018 - 20:00

Some years ago, the Apache Foundation's web server, known simply as "Apache", was so ubiquitous that it became synonymous with the term "web server". Its daemon process on Linux systems has the name httpd (meaning simply http process) --- and comes preinstalled in major Linux distributions.

It was initially released in 1995, and, to quote Wikipedia, "it played a key role in the initial growth of the World Wide Web". It is still the most-used web server software according to W3techs. However, according to those reports which show some trends of the last decade and comparisons to other solutions, its market share is decreasing. The reports given by Netcraft and Builtwith differ a bit, but all agree on a trending decline of Apache's market share and the growth of Nginx.

Nginx --- pronounced engine x --- was released in 2004 by Igor Sysoev, with the explicit intent to outperform Apache. Nginx's website has an article worth reading which compares these two technologies. At first, it was mostly used as a supplement to Apache, mostly for serving static files, but it has been steadily growing, as it has been evolving to deal with the full spectrum of web server tasks.

It is often used as a reverse proxy, load balancer, and for HTTP caching. CDNs and video streaming providers use it to build their content delivery systems where performance is critical.

Apache has been around for a long time, and it has a big choice of modules. Managing Apache servers is known to be user-friendly. Dynamic module loading allows for different modules to be compiled and added to the Apache stack without recompiling the main server binary. Oftentimes, modules will be in Linux distro repositories, and after installing them through system package managers, they can be gracefully added to the stack with commands like a2enmod. This kind of flexibility has yet to be seen with Nginx. When we look at a guide for setting up Nginx for HTTP/2, modules are something Nginx needs to be built with --- configured for at build-time.

One other feature that has contributed to Apache's market rule is the .htaccess file. It is Apache's silver bullet, which made it a go-to solution for the shared hosting environments, as it allows controlling the server configuration on a directory level. Every directory on a server served by Apache can have its own .htaccess file.

Nginx not only has no equivalent solution, but discourages such usage due to performance hits.

Server vendors market share 1995–2005. Data by Netcraft

LiteSpeed, or LSWS, is one server contender that has a level of flexibility that can compare to Apache, while not sacrificing performance. It supports Apache-style .htaccess, mod_security and mod_rewrite, and it's worth considering for shared setups. It was planned as a drop-in replacement for Apache, and it works with cPanel and Plesk. It's been supporting HTTP/2 since 2015.

LiteSpeed has three license tiers, OpenLiteSpeed, LSWS Standard and LSWS Enterprise. Standard and Enterprise come with an optional caching solution comparable to Varnish, LSCache, which is built into the server itself, and can be controlled, with rewrite rules, in .htaccess files (per directory). It also comes with some DDOS-mitigating "batteries" built in. This, along with its event-driven architecture, makes it a solid contender, targeting primarily performance-oriented hosting providers, but it could be worth setting up even for smaller servers or websites.

Hardware Considerations

When optimizing our system, we cannot emphasize enough giving due attention to our hardware setup. Whichever of these solutions we choose for our setup, having enough RAM is critical. When a web server process, or an interpreter like PHP, don't have enough RAM, they start swapping, and swapping effectively means using the hard disk to supplement RAM memory. The effect of this is increased latency every time this memory is accessed. This takes us to the second point --- the hard disk space. Using fast SSD storage is another critical factor of our website speed. We also need to mind the CPU availability, and the physical distance of our server's data centers to our intended audience.

To dive in deeper into the hardware side of performance tuning, Dropbox has a good article.

Monitoring

One practical way to monitor our current server stack performance, per process in detail, is htop, which works on Linux, Unix and macOS, and gives us a colored overview of our processes.

Other monitoring tools are New Relic, a premium solution with a comprehensive set of tools, and Netdata, an open-source solution which offers great extensibility, fine-grained metrics and a customizable web dashboard, suitable for both little VPS systems and monitoring a network of servers. It can send alarms for any application or system process via email, Slack, pushbullet, Telegram, Twilio etc.

Monit is another, headless, open-source tool which can monitor the system, and can be configured to alert us, or restart certain processes, or reboot the system when some conditions are met.

Testing the System

AB --- Apache Benchmark --- is a simple load-testing tool by Apache Foundation, and Siege is another load-testing program. This article explains how to set them both up, and here we have some more advanced tips for AB, while an in-depth look at Siege can be found here.

If you prefer a web interface, there is Locust, a Python-based tool that comes in very handy for testing website performance.

After we install Locust, we need to create a locustfile in the directory from which we will launch it:

from locust import HttpLocust, TaskSet, task class UserBehavior(TaskSet): @task(1) def index(self): self.client.get("/") @task(2) def shop(self): self.client.get("/?page_id=5") @task(3) def page(self): self.client.get("/?page_id=2") class WebsiteUser(HttpLocust): task_set = UserBehavior min_wait = 300 max_wait = 3000

Then we simply launch it from the command line:

locust --host=https://my-website.com

One warning with these load-testing tools: they have the effect of a DDoS attack, so it's recommended you limit testing to your own websites.

Tuning Apache Apache's mpm modules

Apache dates to 1995 and the early days of the internet, when an accepted way for servers to operate was to spawn a new process on each incoming TCP connection and to reply to it. If more connections came in, more worker processes were created to handle them. The costs of spawning new processes were high, and Apache developers devised a prefork mode, with a pre-spawned number of processes. Embedded dynamic language interpreters within each process (like mod_php) were still costly, and server crashes with Apache's default setups became common. Each process was only able to handle a single incoming connection.

This model is known as mpm_prefork_module within Apache's MPM (Multi-Processing Module) system. According to Apache's website, this mode requires little configuration, because it is self-regulating, and most important is that the MaxRequestWorkers directive be big enough to handle as many simultaneous requests as you expect to receive, but small enough to ensure there's enough physical RAM for all processes.

A small Locust load test that shows spawning of huge number of Apache processes to handle the incoming traffic.

We may add that this mode is maybe the biggest cause of Apache's bad name. It can get resource-inefficient.

Version 2 of Apache brought another two MPMs that try to solve the issues that prefork mode has. These are worker module, or mpm_worker_module, and event module.

Worker module is not process-based anymore; it's a hybrid process-thread based mode of operation. Quoting Apache's website,

a single control process (the parent) is responsible for launching child processes. Each child process creates a fixed number of server threads as specified in the ThreadsPerChild directive, as well as a listener thread which listens for connections and passes them to a server thread for processing when they arrive.

This mode is more resource efficient.

2.4 version of Apache brought us the third MPM --- event module. It is based on worker MPM, and added a separate listening thread that manages dormant keepalive connections after the HTTP request has completed. It's a non-blocking, asynchronous mode with a smaller memory footprint. More about version 2.4 improvements here.

We have loaded a testing WooCommerce installation with around 1200 posts on a virtual server and tested it on Apache 2.4 with the default, prefork mode, and mod_php.

First we tested it with libapache2-mod-php7 and mpm_prefork_module at https://tools.pingdom.com:

Then, we went for testing the event MPM module.

We had to add multiverse to our /etc/apt/sources.list:

deb http://archive.ubuntu.com/ubuntu xenial main restricted universe multiverse deb http://archive.ubuntu.com/ubuntu xenial-updates main restricted universe multiverse deb http://security.ubuntu.com/ubuntu xenial-security main restricted universe multiverse deb http://archive.canonical.com/ubuntu xenial partner

Then we did sudo apt-get updateand installed libapache2-mod-fastcgi and php-fpm:

sudo apt-get install libapache2-mod-fastcgi php7.0-fpm

Since php-fpm is a service separate from Apache, it needed a restart:

sudo service start php7.0-fpm

Then we disabled the prefork module, and enabled the event mode and proxy_fcgi:

sudo a2dismod php7.0 mpm_prefork sudo a2enmod mpm_event proxy_fcgi

We added this snippet to our Apache virtual host:

&lt;filesmatch "\.php$"&gt; SetHandler "proxy:fcgi://127.0.0.1:9000/" &lt;/filesmatch&gt;

This port needs to be consistent with php-fpm configuration in /etc/php/7.0/fpm/pool.d/www.conf. More about the php-fpm setup here.

Then we tuned the mpm_event configuration in /etc/apache2/mods-available/mpm_event.conf, keeping in mind that our mini-VPS resources for this test were constrained --- so we merely reduced some default numbers. Details about every directive on Apache's website, and tips specific to the event mpm here. Keep in mind that started servers consume an amount of memory regardless of how busy they are. The MaxRequestWorkers directive sets the limit on the number of simultaneous requests allowed: setting MaxConnectionsPerChild to a value other than zero is important, because it prevents a possible memory leak.

<ifmodule mpm_event_module> StartServers 1 MinSpareThreads 30 MaxSpareThreads 75 ThreadLimit 64 ThreadsPerChild 30 MaxRequestWorkers 80 MaxConnectionsPerChild 80 </ifmodule>

Then we restarted the server with sudo service apache2 restart (if we change some directives, like ThreadLimit, we will need to stop and start the service explicitly, with sudo service apache2 stop; sudo service apache2 start).

Our tests on Pingdom now showed page load time reduced by more than half:

The post Apache vs Nginx Performance: Optimization Techniques appeared first on SitePoint.

Kategóriák: IT Hírek

Making Your Website Faster and Safer with Cloudflare

k, 06/12/2018 - 20:00

Cloudflare is an industry leader in the content-delivery space, reducing load and speeding up millions of websites.

What is peculiar about this provider is that it didn't start as speed-up/performance tool, but was instead born from Project Honeypot, which was conceived as a spam and hacking protection service. To this day, this is one of Cloudflare's major selling points: DDoS detection and protection. Their algorithms take note of visitors' IP addresses, payloads, resources requested, and request frequency to detect malicious visitors.

Because it sits as a proxy between websites and all incoming traffic, Cloudflare is able to reduce strain on servers significantly, so much so that DDoS attacks won't even reach the origin websites, as explained in this introduction. Cloudflare also provides the Always Online option, which caches a version of the user's website and serves a limited version of it in case of origin server outage --- when the original website returns 5xx or 4xx errors. It also features a full-fledged page cache.

These features can be a huge advantage: they can salvage a struggling web server under heavy load, and in case of server errors, can give some breathing room to developers to figure things out.

It's also available free. There are premium tiers, of course, and there are things (like additional page rules) that require paying, but the scope of Cloudflare's free tier alone makes it worthwhile to learn its ins and outs.

Comparison benchmarks put Cloudflare somewhere in the middle in regard to speed, but it would be hard to argue that it is the best value CDN on the market.

Setting Up Cloudflare

Setting a site up with Cloudflare is very straightforward. After registering at (cloudflare.com)[http://www.cloudflare.com], we can add a new website. While the system scans for the given domain's IP and other details, we're offered an introductory video. Upon completion, we're given new nameservers to set up with our registrar.

We need to register these nameservers with our registrar and wait for changes to propagate across the internet. It may take up to 24 hours.

This change means giving all control over our domain to Cloudflare. This also means that, if we have email on this domain (MX records), we need to transfer these records to Cloudflare. If we have any subdomains, they also need to be set up as respective A records in Cloudflare's dashboard.

All existing domain records set up with our domain registrar or hosting provider need to be moved/copied to Cloudflare.

Some managed hosting providers may simplify/automate this transition process even more.

For each of our domain records, we can decide to simply let all the traffic pass through directly to our servers --- which means we can set exceptions for certain subdomains --- or we can turn off all Cloudflare functionality --- for example, while we're making some changes on the website.

Once we've set the domain up, that's basically all the work required outside of Cloudflare's dashboard. There's nothing more to do on the website itself, or the origin server. All further tuning is done on the Cloudflare website.

Setting up Encryption

An SSL certificate is part of the free plan on Cloudflare. There are four options for SSL setup, and we can find them under the Crypto tab in the dashboard.

  • OFF - this is self-explanatory. All traffic will be redirected to unsecured protocol (http)
  • FLEXIBLE - regardless of the protocol of our server, and whether we have an existing SSL certificate on it or not, Cloudflare will serve all our pages to end-visitors over https. Connections from Cloudflare to the origin server will go over an unsecured connection.
  • FULL - Cloudflare will communicate to your server via https, but won't validate certificates on the origin. Traffic from Cloudflare to visitors is served over https.
  • FULL STRICT - Cloudflare will require valid (not self-signed) SSL certificates on the origin server. Traffic from Cloudflare to visitors is served over https.

With these settings, we need to make sure the setup is sensible because we have two layers between our end users and our server content, so omissions here can result in a redirect loop, or too many redirections which can end up slowing the website.

Cloudflare also offers the option to buy a custom certificate, and for premium users who require extra safety or care about their market image, it gives the option of uploading custom/premium certificates. This is a part of premium plans.

Securing the Website

This is one area where Cloudflare shines: it gives unprecedented value for free. Across the hosting landscape, DDoS protection is a premium service, and not always provided, even for paying customers.

Cloudflare offers unmetered DDoS protection on the free tier, together with some other, rather sophisticated tools that protect websites on an infrastructure level before malicious traffic even reaches it. It offers rate limiting --- throttling of visits
according to user-defined, customizable rules. It offers smart firewall rules, country blocks, browser integrity checks, captcha protections, and more.

Today, when botnets rule the internet and freshly installed websites or servers are sometimes drowned in brute-force break-in attempts within minutes of going online, when spammers automatize web comments, and referrer spam is rampant even without any break-ins, POST attacks and slow attacks utilizing unorthodox means are not rare. This kind of protection can make or break smaller- or medium-sized websites.

There's also scraping protection, denying certain resources to certain visitor profiles, or obfuscating emails.

Premium tiers offer even more options.

The post Making Your Website Faster and Safer with Cloudflare appeared first on SitePoint.

Kategóriák: IT Hírek

What Is a CDN and How Does It Work?

h, 06/11/2018 - 20:00

CDN - you keep seeing the acronym. Maybe in URLs, maybe on landing pages, but it never quite clicked - what are Content Delivery Networks, what do they do exactly?

We'll explain in this overview article, and demonstrate on two popular ones in followup posts.

CDN Basics

A CDN is a network of computers that delivers content.

More specifically, it's a bunch of servers geographically positioned between the origin server of some web content, and the user requesting it, all with the purpose of delivering the content faster by reducing latency. This is their primary purpose.

These geographically closer servers, also called PoPs or Points of Presence, also cache the cacheable content which removes a lot of the load from the origin server. There are different types of CDNs offering different kinds of services, and they can have differing network topology: scattered CDNs aim to have as many servers scattered around the world as possible. Akamai is one such CDN. Consolidated CDNs have fewer points, but bigger ones built for network performance, throughput, and DDoS resistance.

Types of CDNs

We said that their primary purpose was to reduce latency and speed up rendering. But in the modern world of 2MB images and 500kb JavaScript libraries that take 3 minutes to boot up on websites, this latency matters little. But there are other purposes to CDNs, too, which evolved over time.

Content-oriented CDNs

Initially, CDNs were just for static content (JS, CSS, HTML). You had to push content to them as you created/uploaded it (they didn't know they needed to update their cache with your content, not even as someone requested it).

Then, they added origin pulling, making things more automatic - this meant that a user requested the CDN's URL, and then the CDN requested the origin website's URL automatically, caching what ever it got back. Additionally, availability became an important factor. Many CDNs now cache a website's "last alive" state so that if the origin goes down, the CDNed content is still accessible to users, creating the illusion of stability until things return to normal.

Additionally, modern CDNs often offer auto-optimization layers which will automagically resize images and save them for future use based on the image size requested. This means what if your site has a 2MB header image and someone requests it on a 300px wide screen, the CDN will make a copy that's 30kb in size and 300px wide and serve that in the future to all mobile users, automatically making the site faster.

Security-oriented CDNs

The final layer of practicality added to CDNs was DDoS and bot protection. CDNs like Incapsula specialize in this.

As the CDN is the outermost layer of a website's infrastructure and the first recipient of traffic, it can detect DDoS attacks early and block them with special DDoS protection servers called scrubbers without them ever reaching the origin server and crashing it.

The post What Is a CDN and How Does It Work? appeared first on SitePoint.

Kategóriák: IT Hírek

HTTP/2: Background, Performance Benefits and Implementations

p, 06/08/2018 - 20:00

On top of the infrastructure of the internet --- or the physical network layers --- sits the Internet Protocol, as part of the TCP/IP, or transport layer. It's the fabric underlying all or most of our internet communications.

A higher level protocol layer that we use on top of this is the application layer. On this level, various applications use different protocols to connect and transfer information. We have SMTP, POP3, and IMAP for sending and receiving emails, IRC and XMPP for chatting, SSH for remote sever access, and so on.

The best-known protocol among these, which has become synonymous with the use of the internet, is HTTP (hypertext transfer protocol). This is what we use to access websites every day. It was devised by Tim Berners-Lee at CERN as early as 1989. The specification for version 1.0 was released in 1996 (RFC 1945), and 1.1 in 1999.

The HTTP specification is maintained by the World Wide Web Consortium, and can be found at http://www.w3.org/standards/techs/HTTP.

The first generation of this protocol --- versions 1 and 1.1 --- dominated the web up until 2015, when HTTP/2 was released and the industry --- web servers and browser vendors --- started adopting it.

HTTP/1

HTTP is a stateless protocol, based on a request-response structure, which means that the client makes requests to the server, and these requests are atomic: any single request isn't aware of the previous requests. (This is why we use cookies --- to bridge the gap between multiple requests in one user session, for example, to be able to serve an authenticated version of the website to logged in users.)

Transfers are typically initiated by the client --- meaning the user's browser --- and the servers usually just respond to these requests.

We could say that the current state of HTTP is pretty "dumb", or better, low-level, with lots of "help" that needs to be given to the browsers and to the servers on how to communicate efficiently. Changes in this arena are not that simple to introduce, with so many existing websites whose functioning depends on backward compatibility with any introduced changes. Anything being done to improve the protocol has to be done in a seamless way that won't disrupt the internet.

In many ways, the current model has become a bottleneck with this strict request-response, atomic, synchronous model, and progress has mostly taken the form of hacks, spearheaded often by the industry leaders like Google, Facebook etc. The usual scenario, which is being improved on in various ways, is for the visitor to request a web page, and when their browser receives it from the server, it parses the HTML and finds other resources necessary to render the page, like CSS, images, and JavaScript. As it encounters these resource links, it stops loading everything else, and requests specified resources from the server. It doesn't move a millimeter until it receives this resource. Then it requests another, and so on.

The number of requests needed to load world's biggest websites is often in couple of hundreds.

This includes a lot of waiting, and a lot of round trips during which our visitor sees only a white screen or a half-rendered website. These are wasted seconds. A lot of available bandwidth is just sitting there unused during these request cycles.

CDNs can alleviate a lot of these problems, but even they are nothing but hacks.

As Daniel Stenberg (one of the people working on HTTP/2 standardization) from Mozilla has pointed out, the first version of the protocol is having a hard time fully leveraging the capacity of the underlying transport layer, TCP.
Users who have been working on optimizing website loading speeds know this often requires some creativity, to put it mildly.

Over time, internet bandwidth speeds have drastically increased, but HTTP/1.1-era infrastructure didn't utilize this fully. It still struggled with issues like HTTP pipelining --- pushing more resources over the same TCP connection. Client-side support in browsers has been dragging the most, with Firefox and Chrome disabling it by default, or not supporting it at all, like IE, Firefox version 54+, etc.
This means that even small resources require opening a new TCP connection, with all the bloat that goes with it --- TCP handshakes, DNS lookups, latency… And due to head-of-line blocking, the loading of one resource results in blocking all other resources from loading.

A synchronous, non-pipelined connection vs a pipelined one, showing possible savings in load time.

Some of the optimization sorcery web developers have to resort to under the HTTP/1 model to optimize their websites include image sprites, CSS and JavaScript concatenation, sharding (distributing visitors' requests for resources over more than one domain or subdomain), and so on.

The improvement was due, and it had to solve these issues in a seamless, backward-compatible way so as not to interrupt the workings of the existing web.

SPDY

In 2009, Google announced a project that would become a draft proposal of a new-generation protocol, SPDY (pronounced speedy), adding support to Chrome, and pushing it to all of its web services in subsequent years. Then followed Twitter and server vendors like Apache, nginx with their support, Node.js, and later came Facebook, WordPress.com, and most CDN providers.

SPDY introduced multiplexing --- sending multiple resources in parallel, over a single TCP connection. Connections are encrypted by default, and data is compressed. First, preliminary tests in the SPDY white paper performed on the top 25 sites showed speed improvements from 27% to over 60%.

After it proved itself in production, SPDY version 3 became basis for the first draft of HTTP/2, made by the Hypertext Transfer Protocol working group httpbis in 2015.

HTTP/2 aims to address the issues ailing the first version of the protocol --- latency issues --- by:

It also aims to solve head-of-line blocking. The data it transfers is in binary format, improving its efficiency, and it requires encryption by default (or at least, this is a requirement imposed by major browsers).

Header compression is performed with the HPACK algorithm, solving the vulnerability in SPDY, and reducing web request sizes by half.

Server push is one of the features that aims to solve wasted waiting time, by serving resources to the visitor's browser before the browser requires it. This reduces the round trip time, which is a big bottleneck in website optimization.

Due to all these improvements, the difference in loading time that HTTP/2 brings to the table can be seen on this example page by imagekit.io.

Savings in loading time become more apparent the more resources a website has.

The post HTTP/2: Background, Performance Benefits and Implementations appeared first on SitePoint.

Kategóriák: IT Hírek

The Complete Guide to WordPress Performance Optimization

cs, 06/07/2018 - 20:00

According to Builtwith.com, WordPress holds close to 50% of the CMS share of the world's top 1,000,000 websites. As for the ecommerce sphere, we're at 33% with WooCommerce. And if we cast a wider net, percentages go higher. Although we may complain that WordPress can get bloated, resource-heavy, and its data model leaves a lot to be desired, there is no denying that WordPress is everywhere.

WordPress can thank its simplicity and a low barrier to entry for this pervasiveness. It's easy to set up, and requires next to no technical knowledge. Hosting for WordPress can be found for as little as a couple of dollars per month, and the basic setup takes just a half hour of clicking. Free themes for WordPress are galore, some with included WYSIWYG page builders.

Many look down on it, but in many ways we can thank WordPress for the growth of the internet and PHP, and many internet professionals have WP's gentle learning curve to thank for their careers.

But this ease of entry comes at a cost. Plenty of websites that proudly wear the WordPress badge were not done by professionals but by the cheapest developers. And often, it shows. Professional look and professional performance were afterthoughts.

One of the main points of feedback the owner of an aspiring high-quality website will get from a grudging professional is that performance and a professional look and feel shouldn't be afterthoughts. You can't easily paint or stick them over a website. Professional websites should be premeditated.

Above, a famous UK used car dealer, Ling's Cars, tried a unique way to make a kitsch marketing punchline. Unless you're REALLY sure about what you're doing, DO NOT try this at home

And this starts with…

Choice of Hosting

Typically, new users will go with products that are on the low-cost side, with most of beginner-friendly bells and whistles. Considering the shady business practices by some big industry players in this arena, and the complaints and demands for site migration professionals coming from clients, this is a part of website setup that requires due attention.

We can divide WordPress hosting vendors into a few tiers.

Premium, WordPress-dedicated vendors like Kinsta whose plans start at $100 per month, or even higher-grade managed hosting like WordPress VIP by Automattic, may be worth their salt, but also may be out of reach for many website owners.

Medium tier Flywheel, A2 hosting, Siteground and Pantheon are among those considered reliable and performance oriented, offering acceptable speed and a managed hosting service for those more price-conscious. Users here may get a bit less hand-holding, but these services usually strike an acceptable balance between a solid setup, price, and options for more advanced users. Not to forget, there is Cloudways, which is a hybrid between VPS and managed hosting. Those with their audience in Europe may look into Pilvia, as it offers a performant server stack and is pretty affordable.

There's an interesting survey of customer satisfaction with more prominent hosting vendors, published by Codeinwp.

For those of us not scared of the command line, there are VPS and dedicated-server vendors like Digital Ocean, Vultr, Linode, Amazon's Lightsail, Hetzner in Europe, and OVH. Hetzner is a German vendor known for its quality physical servers on offer, somewhat above the price of virtual servers, while OVH offers very cost-efficient virtual servers. For the price-conscious, OVH's subsidiary Kimsufi in Europe and Canada also offers bargain physical dedicated servers, and Host US has very affordable virtual servers.

With managed hosting, things to look for are a good server stack, good CDN integration, and of course SSD storage. Guaranteed resources, like with A2, are a big plus. The next thing to look for is SSH-access. Tech-savvy users may profit from WP-CLI availability.

When choosing a VPS, the thing to look for is XEN or KVM virtualization over OpenVZ, because it mitigates the overselling of resources, helping guarantee that the resources you bought are really yours. It also provides better security.

Easy Engine is software that can make your entire VPS/WordPress installation a one-hour job.

Regarding the server stack, Nginx is preferred to Apache if we're pursuing performance, and PHP 7 is a must. If we really need Apache, using Nginx as a reverse proxy is a plus, but this setup can get complex.

Tests performed give PHP 7 a big edge over the previous version. According to fasthosts.co.uk:

WordPress 4.1 executed 95% more requests per second on PHP 7 compared to PHP 5.6.

When choosing your hosting, be aware of negative experiences with some providers that have become notorious.

Software Considerations

Things that usually slow down WordPress websites are bulky, bloated front ends with a lot of static resources and database queries. These issues arise from the choice of theme (and its page builders, huge sliders, etc) --- which not only slow down initial loading due to many requests and overall size, but often slow down the browser due to a lot of JavaScript, and stuff to render, making it unresponsive.

The golden rule here is: don't use it unless there's a good reason to.

This may seem like a rule coming from the mouth of Homer Simpson, but if you can skip any of the bells and whistles, do so. Be conservative. If you must add some shiny functionality or JS eye candy, always prefer those tailored and coded as specifically as possible for your exact need. If you're a skilled coder, and the project justifies the effort, code it yourself with minimalism in mind.

Review all the plugins your website can't live without --- and remove the others.

And most importantly: back up your website before you begin pruning!

Data model

If you have a theme where you use a lot of custom posts or fields, be warned that a lot of these will slow down your database queries. Keep your data model as simple as possible, and if not, consider that WordPress' original intended purpose was as a blogging engine. If you need a lot more than that, you may want to consider some of the MVC web frameworks out there that will give you greater control over your data model and the choice of database.

In WordPress we can build a rich custom data model by using custom post types, custom taxonomies and custom fields, but be conscious of performance and complexity costs.

If you know your way around the code, inspect your theme to find unnecessary database queries. Every individual database trip spends precious milliseconds in your TTFB, and megabytes of your server's memory. Remember that secondary loops can be costly --- so be warned when using widgets and plugins that show extra posts, like in sliders or widget areas. If you must use them, consider fetching all your posts in a single query, where it may otherwise slow down your website. There's a GitHub repo for those not wanting to code from scratch.

Meta queries can be expensive

Using custom fields to fetch posts by some criteria can be a great tool to develop sophisticated things with WP. This is an example of a meta query, and here you can find some elaboration on its costs. Summary: post meta wasn't built for filtering, taxonomies were.

get_post_meta is a function typically called to fetch custom fields, and it can be called with just the post ID as an argument, in which case it fetches all the post's meta fields in an array, or it can have a custom field's name as a second argument, in which case it returns just the specified field.

If using get_post_meta()for a certain post multiple times on a single page or request (for multiple custom fields), be aware that this won't incur extra cost, because the first time this function is called, all the post meta gets cached.

Database hygiene

Installing and deleting various plugins, and changing different themes over the lifetime of your website, often clutters your database with a lot of data that isn't needed. It's completely possible to discover --- upon inspecting why a WordPress website is sluggish, or why it won't even load, due to exhausted server memory --- that the database has grown to hundreds and hundreds of megabytes, or over a gigabyte, with no content that explains it.

wp-options is where a lot of orphaned data usually gets left behind. This includes, but is not limited to, various transients (this post warns of best practices regarding deletion of transients in plugins). Transients are a form of cache, but as with any other caching, if misused, it can do more harm than good. If your server environment provides it, wp-cli has a command set dedicated to transients management, including deletion. If not, there are plugins in the WordPress plugins repo that can delete expired transients, but which offer less control.

If deleting transients still leaves us with a bloated database without any tangible cause, WP-Sweep is an excellent free tool that can do the job of cleaning up the database. Another one to consider is WP Optimize.

Before doing any kind of database cleanup, it's strongly recommended that you back up your database!

One of the plugins that comes in very handy for profiling of the whole WordPress request lifecycle is Debug Objects. It offers an inspection of all the transients, shortcodes, classes, styles and scripts, templates loaded, db queries, and hooks.

After ensuring a sane, performance-oriented setup --- considering our server stack in advance, eliminating the possible bloat created by theme choice and plugins and widgets overload --- we should try to identify bottlenecks.

If we test our website in a tool like Pingdom Speed Test, we'll get a waterfall chart of all the resources loaded in the request:

This gives us details about the request-response lifecycle, which we can analyze for bottlenecks. For instance:

  • If the pink DNS time above is too big, it could mean we should consider caching our DNS records for a longer period. This is done by increasing the TTL setting in our domain management/registrar dashboard.
  • If the SSL part is taking too long, we may want to consider enabling HTTP/2 to benefit from ALPN, adjusting our cache-control headers, and finally switching to a CDN service. “Web Performance in a Nutshell: HTTP/2, CDNs and Browser Caching” is a thorough article on this topic, as is “Analyzing HTTPS Performance Overhead” by KeyCDN.
  • Connect, Send, and Receive parts usually depend on network latency, so these can be improved by hosting close to your intended audience, making sure your host has a fast uplink, and using a CDN. For these items, you may want to consider a ping tool too (not to be confused with the Pingdom tools mentioned above), to make sure your server is responsive.
  • The Wait part --- the yellow part of the waterfall --- is the time your server infrastructure takes to produce or return the requested website. If this part takes too much time, you may want to return to our previous topics of optimizing the server, WordPress installation, and database stack. Or you may consider various layers of caching.

To get a more extensive testing and hand-holding tips on improving the website, there's a little command line utility called webcoach. In an environment with NodeJS and npm installed (like Homestead Improved), installing it is simple:

npm install webcoach -g

After it's been installed, we can get detailed insights and advice on how to improve our website's various aspects, including performance:

The post The Complete Guide to WordPress Performance Optimization appeared first on SitePoint.

Kategóriák: IT Hírek

An Introduction to MongoDB

h, 05/28/2018 - 21:00

MongoDB is an open-source, document-oriented, NoSQL database program. If you’ve been involved with the traditional, relational databases for long, the idea of a document-oriented, NoSQL database might indeed sound peculiar. “How can a database not have tables?”, you might wonder. This tutorial introduces you to some of the basic concepts of MongoDB and should help […]

Continue reading %An Introduction to MongoDB%

Kategóriák: IT Hírek

How to Fix Magento Login Issues with Cookies and Sessions

h, 05/21/2018 - 08:00

This article was created in partnership with Ktree. Thank you for supporting the partners who make SitePoint possible.

In this article are looking at how Magento cookies can create issues with the login functionality of both the customer-facing front-end and admin back-end, the reason it occurs and how it should be resolved.

This is also known as the looping issue, as the screen redirects itself to the same screen, even though the username and password is correct.

A script is provided at the end of the article which can help detect a few of the issues. Feel free to use and modify as per your needs.

What is a Cookie?

A cookie is a piece of text that a web server can store on a user's hard drive, and can also later retrieve it. Magento uses cookies in Cart & Backend Admin functionalities, and they may be the source of a few problems when unable to login to Magento.

What is a Session?

A session is an array variable on the server side, which stores information to be used across multiple pages. For example, items added to the cart are typically saved in sessions, and when the user browses the checkout page they are read from the session.

Sessions are identified by a unique ID. Its name changes depemnding on the programming language — in PHP it is called a 'PHP Session ID'. As you might have guessed, the same PHP Session ID needs to be stored as a cookie in the client browser to relate.

Magento's storage of Sessions

Magento can store sessions via multiple session providers and this can be configured in the Magento config file at app/etc/local.xml. These session providers can be chosen here.

File

<session_save><![CDATA[files]]></session_save> <session_save_path> <![CDATA[/tmp/session]]> </session_save_path>

Database

Allowing sessions to store themselves in the database is done in /app/etc/local.xml by adding <session_save><![CDATA[db]]></session_save>.

Magento applications store sessions in the Core\_session table.

Redis

<session_save>db</session_save> <redis_session> <host>127.0.0.1</host> <port>6379</port> </redis_session>

MemCache

session_save><![CDATA[memcache]]></session_save> <session_save_path> <![CDATA[tcp://localhost:11211?persistent=1&weight=2&timeout=10&retry_interval=10]]> </session_save_path> Magento Usage

Magento uses two different cookies named 'frontend' and 'adminhtml'. The first one is created when any page is browsed. The same cookie is also updated whenever the customer logs in, and the next one is created when a backend user is logged in. You can check whether the cookies have been created by clicking Inspect Element > Application, as in the below picture (from Chrome):

Cookies are configured in Magento via the Configuration admin menu - System > Configuration > General > Web.

Problem: Login Fails & Redirects to Login Page

If you haven't experienced this problem, then you haven't worked with Magento long enough!

This is how it typically happens: when you login by entering your username and password, you will be redirected to the same login page and URL, and your browser is appended with nonce id. This happens for both the customer front-end and the Magento back-end login.

Let's look at a few reasons why this happens, and how we should resolve those issues.

Reason #1: Cookie domain does not match server domain

Let's say your Magento site is example.com and the cookie domain in Magento is configured as xyz.com.

In this scenario both Magento cookies will set Domain Value as xyz.com, but for validating the session Magento will consider the domain through which the site was accessed — in this case example.com. Since it won't be able to find an active session with the example.com domain value, it will redirect the user to the login page even when valid credentials are provided.

app/code/core/Mage/Core/Model/Session/Abstract.php

After login or logout, the Magento system will regenerate the session using the following script:

public function renewSession() { $this->getCookie()->delete($this->getSessionName()); $this->regenerateSessionId(); $sessionHosts = $this->getSessionHosts(); $currentCookieDomain = $this->getCookie()->getDomain(); if (is_array($sessionHosts)) { foreach (array_keys($sessionHosts) as $host) { // Delete cookies with the same name for parent domains if (strpos($currentCookieDomain, $host) > 0) { $this->getCookie()->delete($this->getSessionName(), null, $host); } } } return $this; }

app/code/core/Mage/Core/Model/Session/Abstract/Varien.php

Magento will validate the session for every request with the following method:

public function init($namespace, $sessionName=null) { if (!isset($_SESSION)) { $this->start($sessionName); } if (!isset($_SESSION[$namespace])) { $_SESSION[$namespace] = array(); } $this->_data = &$_SESSION[$namespace]; $this->validate(); $this->revalidateCookie(); return $this; }

You may normally see this when you migrate your Magento instance from one domain to another domain, for example from Production to Staging, and forget to change the cookie domain.

Note: you can run the provided cookieTest.php script, which validates what the server cookie domain is, and what is set in the Magento config.

Solution:

Change the Cookie Domain via the Configuration admin menu. Go to System > Configuration > General > Web, as per the screenshot.

Alternatively you can change this by running these SQL queries.

For validating the cookie domain use this select query to get the configuration:

SELECT * FROM core_config_data WHERE path = 'web/cookie/cookie_domain';

After executing this query, we will get the results. Verify the 'value' column is the same as your domain. Update the value if it is not the same as your domain.

To update the cookie domain, use this query:

UPDATE core_config_data SET VALUE = "domain.com" WHERE path = 'web/cookie/cookie_domain'; Reason #2: Multiple subdomains used and Magento's cookie configuration is incorrect

Let's say your site is example.com. Logging into example.com/admin works fine.

But on your staging/QA site, for example staging.example.com/admin, you are unable to login without deleting all cookies. The system may allow logins to staging.example.com, but when we login again to example.com/admin, your next click on staging.example.com kicks you back to the login page. Similar behavior is experienced for customers using the front-end login as well.

Solution 1

Option A: If your main domain and subdomains are hosted on the same server

Continue reading %How to Fix Magento Login Issues with Cookies and Sessions%

Kategóriák: IT Hírek

8 Tips for Improving Bootstrap Accessibility

h, 02/12/2018 - 17:00

A few years ago, I wrote about my experiences on developing a Bootstrap version 3 project to be fully accessible for people with disabilities. This focussed mostly on how accessible it is in terms of front-end design. (It didn’t cover accessibility in terms of screen readers, as that’s a whole other story.)

While I could see that the developers behind Bootstrap were making an effort, there were a few areas where this popular UI library fell short. I could also see that there were issues raised on the project that showed they were actively improving — which is fantastic, considering how approximately 3.6% of websites use Bootstrap.

Recently, Bootstrap version 4 was released, so let’s take a look and see if any of the issues I had in the past have improved.

What We’re Looking For with Design Accessibility

There are a few things to consider when designing a website with accessibility in mind. I believe these improve the user experience for everyone and will likely cover a lot of points you would consider anyway.

Layout

One way to achieve accessibility is by having a clean, easy-to-use layout that looks good on all devices, as well as looking good at a high zoom level. Up to 200% is a good guide.

Bonus points: having front-end code that matches the layout is also good for users who access the Web with a screen reader or by using a keyboard instead of a mouse.

This allows people to use your website easily irrespective of how they’re viewing it.

Continue reading %8 Tips for Improving Bootstrap Accessibility%

Kategóriák: IT Hírek

Oldalak