Running a Qlik Core License in the Cloud? Then you must read this!

A Core means a single processing unit within a processor or CPU, whether physical or virtual, including a vCPU or virtual core, which is capable of executing a single software thread at a time

That definition means that what you can count in the Task Manager > Performance > CPU > Virtual Processers is what the Qlik license considers the actual number of cores is. When you’re running a physical server, you can buy an Intel® Xeon® Processor E5-2697A v4 and get 16 cores, which means that with HyperThreaded enabled it will dispaly as 32 virtual processors. If you have a 16 core license, then it means you should turn off HyperThreading which then will allow you to address all 16 cores. If HyperThreading is on, then Qlik will only use 16 of 32 virtual processors, which means that you’re in reality only using 8 of your 16 cores! Since Qlik uses the 16 first virtual CPUs, which translate to 8 physical core.

So if you’re using core licenses, you should always turn HyperThreading off.

But then some of us are hosting the Qlik server in the Cloud! Which means that you cannot just turn HyperThreading off in the BIOS. But we can do a different trick, which is very easy to implement on Linux, and more difficult on Windows. AWS has written a nice article about this and I can confirm this also works on other cloud vendors.

What the article describes is basically to disable every second virtual processer. Which means that Qlik can address every second virtual processer, which equals a full core. This however has the drawback, that you lose some performance, since the CPU will no longer be able to manage thread execution. Qlik internal recommendation says to have HyperTreading on.

Turn HyperThreading off with access to BIOS

In order to turn HyperThreading off in the OS layer, do the following: Go to Task Manager > Details > Engine.exe > Set Affinity the settings here enable you to override the default setup for how many core the Engine Process is allowed to use. Caution, you must enable every second, not first half or the second half. Precisly every second CPU. This is because each of these is exactly 1 core that run 2 threads, and we want to ensure that we are running 1 threads on 1 core. So CPU 0 and CPU 1, use the same core. The above is offcourse true for Core’s that support 2 threads pr. Core. If you have 3 Threads pr. Core you would need to disable 2 of 3 CPUs. This metod it okay for testing the effects, but is not usable in production, because the setting don’t persist, which means if you restart the Engine.exe then they default back all cores.

But there are some friendly guys who a while back write a program called WinAFC which enables us to monitor the affinity setting of a process, and change them if they go back to defaults. You need to setup so that the program starts with the server, and you need to edit the configureation file.

The file in the installation folder called affinityinput.txt cotains the settings. My setup for enabling every second of 32 core is:

Basically enable every even numbered Core and Core 0.
Finally when setting up WinAFC and you’re getting some errors like CPU’s not identified, make sure you’re running the 64 bit version. Which is more difficult to find from the website, and the default download is 32 bit.

In our testing using the Qlik Scalabilty tools, and depending on how we measure performance. We saw in increase in performance increase of 60% compared to running on 16 core with HyperThreading on.

This week I’m at Qonnections and I’ve already talked this partners who are not aware of this. There is a huge difference in getting this setup right and essentially ensuring you get you full performance for each core!