Stopping screen tearing

It’s a blight of Linux generally, the dreaded screen tearing, certainly for users who use Nvidia cards/drivers. On some Distro’s i’ve had tearing, others not. On some Nvidia cards i’ve had tearing, on others not. It almost seems like it’s the luck of the draw. I’ve trawled Google for answers, i’ve tried almost everything until, almost by chance, I stumbled on an answer, for me at least, hopefully it will help others or point them in the right direction to solving their tearing issues.

I must admit I like my desktop to look clean, simple but very ‘fancy’. Rather duplicital i’ll admit. I like flat icons and window borders but I like the fancy opacity that compositing brings. I’ll get it out of the way now, I like Gnome 3, I’m running Gnome 3.12 right now. My current system has a nice big fancy Nvidia card coupled to two HD monitors and it’s great, no tearing. The issue I have with tearing is my bog standard Ubuntu/Unity Core2 media machine. All it does is stream video and music so it’s just got a GT218 Nvidia card with HDMI-out, it’s quite capable of supplying full 1080p to my 1080p TV, but here’s the rub, I get tearing in films! That sucks!

So what is ‘screen tearing’ or ‘tearing’ anyway? Basically it’s two (or more) frames being shown at the same time on your screen, this results in imperfections in the playback, or lines horizontally across the entire screen, or the areas of the screen that are changing quickly. It’s caused by the refresh rate (number of times in a given period that the image is updated) of your video card not matching the refresh rate of your screen. To solve this problem there are settings available to us to synchronize the refresh rates, only they don’t always work! EG: Compiz and Nvidia have Vsync settings, it doesn’t matter which one of these is enabled, or not, we can still get tearing.

Lots of threads talk about adding things like “userevents” “true” to our Xorg config (etc/X11/xorg.config), switching from Compiz to Mutter and so on, having tried all of these plus some, none of them work…..

For my system the following did work and it’s rather simple, which I like:

sudo apt-get install mplayer2 && smplayer    — this install Mplayer 2 and smplayer

sudo apt-get install vdpau-va-driver && libvdpau1    — This installs vdpau drivers designed by Nvidia, these help to offset some of the decoding and post-processing (making the image you see nicer than the raw decoded image) to the GPU (Graphical-Processing-Unit) instead of the CPU. Basically we’re telling our hardware that all things to do with graphics is to be done by the GPU, the CPU can get on with keeping the rest of the system working!

sudo gedit /home/user/.mplayer/config    — (user is of course whatever your home folder is called, EG: Fred)

Into that file add:

[default]

ao=pulse

vo=vdpau

Save the file and exit (ao=pulse says to use pulseaudio, vo=vdpau says to use the vdpau-drivers during playback in Mplayer.)

Along with these I also have the following settings:

Nvidia-settings: Vsync=enabled, Allow-flipping=enabled, Compiz-compositor=enabled, Compiz-openGL=disabled.

Now I can play full 1080 films without any tearing at all. Hopefully this will help someone, or point them down the right road to tear-free film enjoyment, feel free to comment.

Advertisements

Creating a SWAP partition

We will start with a quick overview of what exactly SWAP is, and when it’s needed, then we’ll look at creating a SWAP partition on our system.

SWAP is useful when we have limited RAM, under 4gb for current Gnome etc desktop environments and applications. It’s basically an extension of ram on a hard-drive (SSD or HDD (Solid-State Drive or Hard-Disk Drive)).

As we open applications they write information to the RAM, the more we open or the larger the application the more the RAM fills to a point it’s exhausted. If we had no SWAP at this point the OOM-Killer (Out-Of-Memory Killer) would kick in to try and free up space, however this usually leads to a hang or reboot as OOM-Killer can sometimes select a critical system process! By having SWAP we allow the Linux kernel to move inactive pages (sets of information in bytes) about applications we maybe opened three hours ago to the SWAP partition on our hard-drive.

“Why doesn’t RAM just get cleared out by new information?” Calling information from scratch about an application on your disk is slow (for those with spinning HDD’s), calling it from RAM is many times faster so the kernel attempts to keep this information in RAM for as long as possible, either until it’s needed again or until new information needs the space it occupies. The next best thing to RAM is to keep all the relevant information about the old application together (paged) and write it to our SWAP partition, here it can sit until we shutdown incase we re-open that application from three hours ago.

So SWAP is a way of keeping your system working with small amounts of RAM, there are lots of tweaks to tune how the kernel deals with this old information, over time i’ll try and cover most of the ones I have found to actually do something!

Lets say that we installed our Linux OS and we decided that our 4gb of RAM would be plenty, now however we’re using lots of applications simultaneously or a couple of large applications, we keep getting hangs, ‘failed to fork’ notices or reboots and we can’t just put more RAM in, we need a SWAP partition! (SWAP needs to be a continuous block of disk space for best results, hence why we are using a partition. An alternative would be to set up a SWAP file). Below is how we go about creating a SWAP partition:

First we need to know what our HDD’s are called and what the existing partition numbers are. I have used Gnome Disk Utility for a long time, it’s an easy to use nicely laid out application with useful tools and information, like SMART data (Self-monitoring, analysis and Reporting Technology), and the disk and partition names & numbers. We can also erase, format and adjust partitions, it’s a lot faster than using terminal to create a new partition, and if you’re unsure of your disk names/numbers it might be better to see it visually! As a useful quick way of checking disks and partitions we can open terminal and enter:

Sudo parted -l

This shows us the HDD names, EG: disk /dev/sda:128GB (sda= solid-state drive A. for a spinning disk it would be hda), beneath that it lists existing partitions EG:

Number   Start          End          Size         Type         File system   Flags
1              2097kB     44.6GB    44.6GB     primary    ext4               boot
2              44.6GB     127GB     82.8GB     primary    ext4

In the above we have two partitions: /dev/sda2 and /dev/sda1, we can see from the ‘flags’ that sda1 is a bootable partition, where the OS resides.

From within Gnome Disk Utility we select the disk we want to use to create our SWAP partition on. It can be the disk with the OS & /home on or a separate disk. When selected it will highlight, now click on the cogs in the lower left just beneath the partition graphic.

We want ‘Format‘. Three options will now be displayed: Erase, Type and Name.

Erase: Does what it says, it will either overwrite, or overwrite with 0’s first. Overwriting with 0’s is the best as it clears everything, however depending on the partition size it can be slow.

Type: Is what we want the partition to end up as, for this we will need ‘compatible with Linux systems (ext4)’.

Name: We can call it a name but we’ll spot it as it will always have a ‘SWAP’ flag with it.

Then we hit ‘Format‘, this will now create the new partition. Once this has completed highlight the new partition, click on the cogs again but this time we want ‘Edit partition‘.

Select ‘Linux swap (0x82)‘, then hit ‘Change

We have now created a new SWAP partition, take a note of what it’s called eg: sda2. Now we need to tell our OS that it exists and needs to use it. For this job we do need terminal:

sudo blkid /dev/sda2    — This tells us the UUID (Universally Unique Identifier) of the new SWAP partition. (We’ll use 49883f4d-88d9-482f-8a1b-90cbdf123aa8 from now on, replace this with your own UUID!)

sudo swapon -U 49883f4d-88d9-482f-8a1b-90cbdf123aa8    — This mounts and turns on the new SWAP partition immediately.

sudo gedit /etc/fstab    — This will open up fstab (file systems table) which lists all our partitions and HDD’s we want mounted automatically at boot. The first two will be the root and home partitions, beneath those lines we need to add the following line to this file to enable SWAP at every boot:

UUID=49883f4d-88d9-482f-8a1b-90cbdf123aa8     none    swap    sw      0   0    — save the file then:

sudo update-initramfs -u    — to make sure everything updates prior to rebooting.

All done! If you hibernate (I always use suspend) your PC you’ll also need to do the following:

sudo gedit /etc/initramfs-tools/conf.d/resume    — To edit initramfs file and add the line:

RESUME=UUID=49883f4d-88d9-482f-8a1b-90cbdf123aa8    — Save and exit, then:

sudo update-initramfs -u    — To make sure everything updates prior to rebooting.

We now have a SWAP partition set up and running to stop any out-of-memory issues or hangs. There are other tweaks we can do to reduce the amount of writing the OS does to SWAP, i’ll be covering these at a later stage, feel free to comment.

Memory Overcommit Settings

Today I delved into the underworld of Linux memory allocation, in particular into overcommitting memory (RAM).

After a couple of X11 hangs I decided I needed to learn a little more about the various settings that come as stock with the Linux kernel, to try to tame them, or at least reduce or stop these annoying hangs followed by reboots!

Most applications ask for more memory than they might actually need to startup, some of this is down to bad software design, or they expect that you’ll need that much at some point in the future….a sort of “this is my worst case scenario requirement of RAM, and i’ll tell you that now before we start!”

The stock linux kernel settings kind of just agrees to the applications request without checking if the actual resource, or the hardware, could support the total requested memory in that worst case scenario, partly because most applications never need what they ask for. But, what happens when they do……..

To see your memory system now, under ‘default’ settings, enter the following into terminal:

sudo cat /proc/meminfo

We can see lots of lines but the four we’re interested in are:

MemTotal: The total amount of physical RAM available on your system.

MemFree: The total amount of physical RAM not being used for anything.

CommitLimit: The total amount of memory, both RAM and SWAP, available to commit to the running and requested applications (not necessarily directly related to the actual physical RAM amount, we will see why later).

Commited_AS: The total amount of memory required in the worse case scenario right now if all the applications actually used what they asked for at startup!

If the application/s needed what they originally asked for, an out-of-memory or ‘OOM’ would happen. This would mean that the OOM-killer would kick in to try and free up actual memory by killing running processes it thinks might help to free up memory. By then though a kernel-panic (or at best  X11 would hang) might have happened resulting in a frozen system (aka blue-screen in MS terms) or of course OOM-killer killed a critical system process.

To solve the random selections of the OOM-killer potentially killing off a critical system process, or not kicking in prior to a kernel-panic, we can change the following:

vm.overcommit_ratio=100: The percentage of total actual memory resources available to applications. This might be the total of RAM + SWAP, or just RAM if you have no SWAP. (IE: RAM=1gb & SWAP=1gb, overcommit_ratio=100 would mean 2gb could be allocated to applications. overcommit_ratio=50 would mean 1gb could be allocated to applications – this would obviously not be a sensible choice as 1gb would never be used!)

vm.overcommit_memory=2: This tells the kernel to never agree to allocate more than the total percentage of actual memory determined by overcommit_ratio= and disables the OOM-killer daemon.

We can change the above settings by entering the following into terminal:

sudo sync    — this tells any files in cache on RAM to write to disk now

sudo sh -c “sync; echo 3 > /proc/sys/vm/drop_caches”    — this drops all caches from RAM

sudo cat /proc/meminfo    — check that Committed_AS is below CommitLimit

sudo sysctl -w vm.overcommit_ratio=99    — use 99% of physical memory

sudo sysctl -w vm.overcommit_memory=2    — only allow applications to start if there is enough memory determined by the above command

So now when we try to open a memory hungry application, or we have to many applications open already, the new application is refused with a notification that IE: ‘file manager failed to fork’, or failed to start because there isn’t the available memory. Potentially the application could theoretically start with what memory is available now, but it may continue to require memory to a point the system is unusable as a result and hangs or crashes. A web-browser would be a good example, it opens with only one tab, but during the day you open a dozen more, at some point memory would be exhausted.

By using the two above tweaks we end up with a system that cannot agree to give applications more memory allocation than it physically has. This stops hangs or kernel panics that render the entire system useless, potentially losing those last bits of information you were inputting, instead it simply tells you that there is no more memory, you need to go buy more RAM!

We now know our system will just tell us there’s no more memory for that new application to open, and we like it, we want these settings to survive power cycles (rebooting), we do this by adding the above commands into:

sudo gedit /etc/sysctl.conf    — I use gedit, but nano, vi etc all work

Add: sudo sysctl -w vm.overcommit_ratio=99 and sudo sysctl -w vm.overcommit_memory=2 to the bottom of that file on separate lines and save. Mine look like this:

#system tweaks
vm.swappiness=5
vm.vfs_cache_pressure=50
vm.overcommit_ratio=99
vm.overcommit_memory=2

(I use 99% just to give a little allowance).

Of course you could increase the size of your SWAP partition as CommitLimit is a total of RAM+SWAP (remembering that SWAP is disk based so slower than RAM) so you can open all those tabs, or applications without getting ‘failed to fork’ messages, or you could add a SWAP partition if you haven’t got one already.

“But I have an SSD and SWAP is bad”, well yes it is if you are constantly using it because you only have 1gb of RAM! If you have 4+gb of ram, and depending on what you use your system for, SWAP on an SSD would act as a final safety net saving you from kernel panic under stock settings, or by using the above settings it would stop the constant ‘failed to fork’, but if that’s a regular message following these changes i’d suggest you buy more RAM!

NB: Default is: vm.overcommit_memory=0 which means in short that no tabs are kept on actual available memory space, the kernel agrees to all requests for memory from applications and OOM-killer is activated, in my experience followed by hangs and reboots…….

Feel free to contact me, the above is a condensed and simplified explanation for those still learning.

Introduction

This blog is going to consist of the system tweaks I have learnt and made to my various Linux OS’s. Hopefully this will help others as they Google, primarily it will act as future reference for me.

It may well have random ramblings or views over time to do with technology generally, but mostly it will revolve around things Linux related.