GET /eSCL/ScannerStatus HTTP/1.1

Came across an issue with an Epson printer that kept outputting an HTTP request during printing.

Seems this is a common problem, dating back into 2023, for USB attached printers and Windows 10 / Windows 11.

The advice in the Microsoft Community forums is to disable showing information notifications for local printers:

The printer I worked with had this disabled, and the printout still occurred. I completely wiped the drivers, software and printer devices from the system and reinstalled fresh using drivers directly from vendor website, with no luck.

I saw some chatter about it possibly being related to hibernation issue, but the symptoms related to that didn’t seem the same as mine.

Finally, I saw mention of HP Smart software being problematic. I didn’t find the software installed, however, I did find an HP print driver installed. So I deleted all the print drivers except the current necessary drivers. This seemed to work for me temporarily, now it is problematic again. Disabling WiFi and any direct print on the printer and will see if that resolves the issue.

Printers… *grumble*

ActiveSync EAS Broken in M365 Outlook

Update: This issue is fixed in Outlook Desktop Version 2402 Build 17328.20068 and higher

Ran into an issue with an update to M365 Apps where ActiveSync FolderSync breaks with connection resets on Outlook version 16.0.17126.20182.

To find out the build version in Outlook, go to File > Account

To find out the build version from a command line:

reg query HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office\ClickToRun\Configuration /v ClientVersionToReport

Microsoft is investigating and has issued the following workaround: https://support.microsoft.com/en-us/office/outlook-desktop-exchange-activesync-connections-stopped-working-in-version-2401-da2dc0dd-297e-48ba-b6d9-ab18f5d522fc

The command to downgrade the M365 Apps:

cd %programfiles%\Common Files\Microsoft Shared\ClickToRun
officec2rclient.exe /update user updatetoversion=16.0.17126.20132

High CPU Process Troubleshooting

Introduction

High CPU usage by the “System” process can often be caused by a hardware driver issue (bug, old version, incompatility etc).

The System process loads (or hosts) multiple hardware drivers from different vendors that require higher level of memory access. This is why diagnosing the specific culprit can require a bit of detective work as described below.

Diagnosing the issue

To diagnose the CPU usage issues, you should use Event Tracing for Windows (ETW) to capture CPU Sampling data / Profile.

To capture the data, install the Windows Performance Toolkit, which is part of the Windows SDK.

The Windows 10 WPT can be used on Windows 8/Server 2012, Windows 8.1/Server 2012R2 and Windows 10/Server 2016. If you still use Windows 7, use the SDK/WPT with Build 15086.

 (all other entries can be unselected)

Now run WPRUI.exe, select First Level, under Resource select CPU usage and click on start.

enter image description here

Now capture 1 minute of the CPU usage. After 1 minute, click on Save.

Now analyze the generated ETL file with the Windows Performance Analyzer by dragging and dropping the CPU Usage (sampled) graph to the analysis pane and ordering the columns like you see in the picture:

enter image description here

Inside WPA, load the debug symbols and expand Stack of the SYSTEM process. In this demo, the CPU usage comes from the nVIDIA driver.


In the following demo, the CPU usage comes from the Realtek NIC driver:

enter image description here

When you see calls like ntoskrnl.exe!ViKeTrimWorkerThreadRoutine, ntoskrnl.exe!MmVerifierTrimMemory, ntoskrnl.exe!VerifierKeLeaveCriticalRegion, this means you have Driver Verifier enabled. This also hurts performance a lot and causes high SYSTEM usage. Disable Driver Verifier and reboot.

enter image description here

In this demo, the driver iai2ce.sys (Intel Serial IO GPIO Controller driver) causes it:

enter image description here

In this example, the CPU usage comes from the file rtsuvc.sys which seems to be the Realtek UVC webcam Driver

enter image description here

This demo shows that Bitdefender driver ignis.sys

enter image description here

In the following example, the CPU usage is casued by the broadcom network driver bcmwl664.sys

enter image description here

When you see ntoskrnl.exe!MiZeroWorkerPages as cause, it is trickier. This means the function of the kernel which zeros the memory before it can be used again causes the high CPU usage:

enter image description here

There is no real way to detect which process causes it, but I know that Chrome can cause it if you have hardware acceleration enabled in Chrome. So if you see this and use Chrome, turn hardware acceleration in Chrome off.


When you see those ntoskrnl.exe!RtlpGenericRandomPatternWorker, ntoskrnl.exe!RtlpTestMemoryRandomUp calls

enter image description here

the CPU usage comes from the Kernel to test memory for issues (memtest). This usage is triggered via the idle maintenance task of Windows 8.1/10. You can use Task Scheduler to disable the idle task.

enter image description here

In Windows 10, the task is called RunFullMemoryDiagnostics under Microsoft > Windows > MemoryDiagnostic > RunFullMemoryDiagnostic.

enter image description here

In this case, the CPU usage seems to come from the Data Deduplication Feature (dedup.sys!DdpPostCreate) of Windows Server:

enter image description here

In this demo, the CPU usage is caused by the WIFI card driver athrx.sys

enter image description here

Search for a driver update if you see this.


In the following demo, a citrix driver is involved:

enter image description here

So contact your IT for how to solve Citrix issues.


In this demo, the function usbhub.sys!UsbhPortRecycle causes the CPU usage:

enter image description here

Changing USB2.0 ports to 1.1 speed or connecting USB drives to other USB 2.0 ports helped for some users.


In this case, a small amount of SYSTEM usage comes from the Acronis driver tdrpm251.sys:

enter image description here

In this demo, the CPU usage ntoskrnl.exe!KeAcquireSpinLockRaiseToDpc and ntoskrnl.exe!KeReleaseSpinLock.

enter image description here

so a driver is using SpinLocks very heavily. Disable some devices/drivers until you see one which causes it.


In this case, the CPU usage is caused by the driver L1C62x64.sys

enter image description here

This is the qualcomm atheros AR8171/8175 PCI-E gigabit Ethernet driver. So update the driver if you see it in the stack.


Here, the CPU usage comes from scanning the host file (netbt.sys!DelayedScanLmHostFile)

enter image description here

make sure your hosts file is not too large to avoid this usage.


In this case, the CPU usage comes from SRTSP64.SYS from symantec.

enter image description here

Update your used symantec product to the latest version.


Here, the CPU usage comes from the AMD GPU driver (atikmdag.sys)

enter image description here

if you see this, go to AMD site and get the latest driver for your AMD card.


Here, the drivers TMXPFlt.sys and VsapiNt.sys cause the high CPU usage.

enter image description here

From what I see, those files are part of Trend Micro AV suite. Update the tool or remove it.


In this example, the CPU usage comes from the function ntoskrnl.exe!MmGetPageFileInformation

enter image description here

This function gets information about the pagefile.

Routine Description: This routine returns information about the currently active paging files.

Disable the pagefile, reboot and enable it again and see if this fixes it. Also, removing Intel services (e.g Intel Content Protection HECI Service) seems to fixed it for a user.


Here, you can see that the driver Netwtw04.sys (Intel Wifi driver) calls the function flushCompleteAllPendingFlushRequests and this causes a high CPU usage.

enter image description here

Because the debug symbols get loaded the Windows inbox driver is used. Only here we can get debug symbols to see the callstack with the function name flushCompleteAllPendingFlushRequests.

Here, you should install the latest driver from Intel to fix it.


The most complicated case of SYSTEM usage is ACPI.sys usage in the callstack:

Line #, DPC/ISR, Module, Stack, Count, Process, Weight (in view) (ms), TimeStamp (s), % Weight
6, , ,   |    |- ACPI.sys!ACPIWorkerThread, 40246, , 39.992,941063, , 4,13
7, , ,   |    |    ACPI.sys!RestartCtxtPassive, 40246, , 39.992,941063, , 4,13
8, , ,   |    |    ACPI.sys!InsertReadyQueue, 40246, , 39.992,941063, , 4,13
9, , ,   |    |    ACPI.sys!RunContext, 40246, , 39.992,941063, , 4,13
10, , ,   |    |    ntoskrnl.exe!KeReleaseSpinLock, 40246, , 39.992,941063, , 4,13
11, , ,   |    |    ntoskrnl.exe!KiDpcInterrupt, 40246, , 39.992,941063, , 4,13
12, , ,   |    |    ntoskrnl.exe!KiDispatchInterruptContinue, 40246, , 39.992,941063, , 4,13
13, , ,   |    |    ntoskrnl.exe!KxRetireDpcList, 40246, , 39.992,941063, , 4,13
14, , ,   |    |    ntoskrnl.exe!KiRetireDpcList, 40246, , 39.992,941063, , 4,13
15, , ,   |    |    |- ntoskrnl.exe!KiExecuteAllDpcs, 40198, , 39.945,173325, , 4,13
16, , ,   |    |    |    |- ACPI.sys!ACPIInterruptDispatchEventDpc, 27565, , 27.408,930428, , 2,83
17, , ,   |    |    |    |    |- ACPI.sys!ACPIGpeEnableDisableEvents, 24525, , 24.384,921620, , 2,52
18, , ,   |    |    |    |    |    ACPI.sys!ACPIWriteGpeEnableRegister, 24525, , 24.384,921620, , 2,52
19, , ,   |    |    |    |    |    |- hal.dll!HalpAcpiPmRegisterWrite, 24421, , 24.281,015516, , 2,51
20, , ,   |    |    |    |    |    |    |- hal.dll!HalpAcpiPmRegisterWritePort, 24166, , 24.027,316013, , 2,48

this is extremely difficult to debug. In a sysinternals topic, I listed some advice:

  • make sure the CPU doesn’t overheat because of dust in the CPU fan
  • update or re-flash the (same) BIOS/UEFI
  • load default BIOS/UEFI settings
  • make sure the battery is not damaged, remove the battery from the notebook or disable the battery in device manager.
  • change jumper on HDD caddy if you have replaced the DVD/Blue-Ray Drive with a Caddy to install an SSD next to your old HDD
enter image description here

In the following demo, the Intel HD driver igdkmd64.sys in version .4574 for the Intel HD 630 causes the issue:

enter image description here

The solution is to update to driver with version of at least .4590.


In the following case, the CPU usage of the SYSTEM process is caused by the driver stdriverx64.sys

enter image description here

This seems to be an audio streaming driver. So update this software/driver if you see this in WPA.


If you see a driver called risdxc64.sys in callstack of SYSTEM that causes the high CPU usage, update the Ricoh PCIe SDXC/MMC Host Controller driver or disable the SD card reader in device manager if no driver update fixes it.

enter image description here

This SD card reader seems to be built-in to many Lenovo devices.


The user @stevemidgley showed a new issue of higher CPU usage with Wdf01000.sys!FxSystemWorkItem::_WorkItemThunk

enter image description here

Here you can see a driver UDE.sys causing it.

In symbol hub

enter image description here

I can see it belongs to Modem driver and PNP data of the trace shows Fibocom L850-GL (LTE Modem) as possible device:

enter image description here

And the solution is to disable the modem and USB composite device in device manager.


The user @fajar provided the following case:

enter image description here

Here the cpu usage is small, but if you change the view to DPC/ISR usage

enter image description here

you can see that the avgNetHub.sys driver causes a lof of DPC usage

enter image description here

The name indicates that this driver is part of AVG anti virus software. So update the software or remove it if you see this in your trace.

#etw, #event-tracing-for-windows, #performance, #troubleshooting, #windows-performance-toolkit

Server 2019 Error 0x800f0950

Documenting a resolution to the error received when attempting a DISM /restorehealth as well as .NET 3.5 repair.

Uninstall .NET 3.5 via DISM

dism /online /disable-feature /featurename:NetFX3

Add .NET 3.5 package

dism /online /add-package /packagepath:d:\sources\sxs\microsoft-windows-netfx3-ondemand-package~31bf3856ad364e35~amd64~~.cab

Repair Health

dism /online /cleanup-image /restorehealth /source:wim:"d:\sources\install.wim":2 /limitaccess

Server 2019 DC Dying

Running in to all sorts of problems on a 2019 DC.

Here’s my action plan.

  1. dcdiag / repadmin tools to verify health correcting all errors found before starting any further operations.
  2. stand up the new 2019
    • patch it fully
    • license it
    • join existing domain
    • add active directory domain services
    • promote it
    • use dcdiag / repadmin tools to again verify health
  3. when all is good decommission / demote old one.