Holy wall of text dude, just take me to the BLUF

OK.

Prologue

All I wanted was a wired mouse with good tracking, reasonable ergonomics, and a decent number of buttons to replace an ancient MX518 that finally kicked the bucket. I haven’t been in the gaming scene for years, but a gaming mouse on sale for under thirty bucks seemed like a deal. Not a huge fan of gamer bling, but the glowy logo in the product shot didn’t look that bad, and the shape appeared more suited to human hands than the cyborg greeble fetish found in some alternatives.

In which light is seen, and a darker path taken

On plugging in, the logo is revealed to be lit by an RGB LED array, which continually cycles through its color palette at retina scorching brightness.

Because gamer.

USB mice have theoretically been plug-and-play since the last century, but if you want to stop the light show or assign functions to all the buttons, you need to install Razer’s1 software.

OK, whatever. How bad can it be?

The software is almost 400 megabytes2? How does that even work? How do you find 400 MB of stuff for a mouse driver3 to do?

It also wants you to set up an online account. For your mouse4.

OK, whatever, SSDs are 10￠/GB so who really cares. Install the driver, skip the online account, turn the LEDs down to 2% brightness, and forget about it.

… and then the murders began

At first, it was just a few CPU cycles here and there. Build going a little slow, sluggish tabs, huh, there’s that Razer stuff in task manager, whatever, I don’t feel like dealing with it.

Until one day5, it turned into a massacre:

screenshot of task manager showing Razer Central Service using 80% CPU

Why is “Razer Central Service” maxing out 3 CPU cores? A little poking around with Process Monitor suggests an answer:

screenshot of sysinternals Process Monitor showing RazerCentralService.exe doing lots of file access

Files. Lots of files.

C:\ProgramData\Razer\Razer Central\Accounts\RZR_<28 char hex string>\Emily3\AsyncManager has over 100,000 files in it. All named <hex string that looks like a version 4 UUID>.xml. All with byte-identical content:

<AsyncItem>
 <Operation>Delete</Operation>
 <Setting>
<Path>Devices\92\Profiles</Path>
<Name>(different apparent version 4 UUID, identical in all files).xml</Name>
<Encoding>Binary</Encoding>
 </Setting>
</AsyncItem>

New files appear every few seconds. Some are deleted shortly after, most are not.

In which reason is abandoned for science

This is where a reasonable person might order a nice rhymes-with-smogitech, throw the Razer in the trash, uninstall the crapware, and never think of it again.

A reasonable person might do that. So here we are.

Theory: The sheer number of files makes file operations in that directory ridiculously CPU intensive.

Experiment: Move all the files into a subdirectory where the Razer software won’t trip over them when it’s trying to… whatever the hell it’s trying to do.

Result: Dragging and dropping 100K+ files hangs Windows Explorer for the better part of an hour, but eventually completes. CPU load returns to normal. OK, theory checks out, but why?

A needle in a crapstack

This thing has an attack surface that makes the Hindenburg look svelte6, but that in itself presents a conundrum: Where do you even start? Using Process Explorer and checking the usual Windows locations, most of the files appear to be in:

C:\Program Files (x86)\Razer
C:\ProgramData\Razer
C:\Users\username\AppData\Local\Razer

A bit of find and grep-ing nets over 1700 .exe and .dll files in the first two of those directories. Running strings on them produces a > 500 MB text file.

Retreating from that steaming coprolitic edifice, a quick look in TCPView reveals RazerCentralService continually phoning home7 to some AWS EC2 IPs. Uh oh. Is this annoyingly craptastic bloatware or something more sinister?

screenshot of TCPView showing Razer Central Service connections

Since the traffic is https8 I turn to mitmproxy to find out what it’s phoning home about.

In which crapware fails at crap

With a bit of packet wrangling, some pieces start to fit together. Several times per minute, RazerCentralService sends a request like:

POST https://ec.razer.com/1/setting/delete HTTP/1.1
Content-Type: application/xml
User-Agent: RazerNatasha/7.3.32.173 (Windows NT 10.0; Win64; x32) Emily3/3.6.228.21813
Host: ec.razer.com
Content-Length: 1772
<COP>
  <User>
    <ID>RZR_(same hex glop as RZR directory)</ID>
    <Token>(appears to be JWT)</Token>
  </User>
  <Setting>
    <Name>(same UUID as in XML file content).xml</Name>
    <Path>Devices\92\Profiles</Path>
    <Data></Data>
  </Setting>
  <Product>
    <Name>Emily3</Name>
    <Version></Version>
  </Product>
</COP>

Each time, the server responds with something like:

HTTP/1.1 200 OK
Date: Mon, 22 Mar 2021 01:59:27 GMT
Content-Type: text/xml; charset=utf-8
Content-Length: 370
Connection: keep-alive
<?xml version='1.0' standalone='yes'?>
<COP>
  <Status>
    <Errno>-1</Errno>
    <Message>Invalid token</Message>
  </Status>
  <RequestStatus>
    <Errno>-1</Errno>
    <Message>Invalid token</Message>
    <ErrorList>
      <Error>
        <ErrorCode>4114</ErrorCode>
      </Error>
    </ErrorList>
    <Timestamp>1616378367</Timestamp>
    <Server>ODk4OSRYJDE2JFgkTMcPuCA7IS0QOw7FLrp3B8PLqajfDcHwKc+0EN7g7SU=</Server>
  </RequestStatus>
</COP>

Despite the 200 OK, the content appears to indicate an error. The rejected token appears to be a JWT.

Odds and ends, all ending in failure

As one might expect, the delete requests aren’t the only traffic. Aside from chatter apparently related to checking for updates, a few things stand out.

Around Windows login, it sends a request9 like:

PUT https://id.razer.com/api/csr/cert HTTP/2.0
content-length: 2189
accept: application/json, text/plain, */*
origin:	https://id.razer.com
user-agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.79 Safari/537.36
content-type: application/json;charset=UTF-8
referer: https://id.razer.com/account
accept-encoding: gzip, deflate, br
accept-language: en-US,en;q=0.9

{
    "data": {
        "credential": {
            "auth_type": "jwt",
            "service_code": "0280",
            "token": "(jwt like in the delete request)",
            "uuid": "RZR_(same hex glop as directory)"
        },
        "payload": {
            "C": "CERTIFICATERENEW",
            "R": "Q",
            "csr": "-----BEGIN CERTIFICATE REQUEST-----\r\...(csr)...-----END CERTIFICATE REQUEST-----\r\n",
            "digest": "SHA-256",
            "exp": 31536000
        }
    },
    "method": "PUT"
}

The response hints at an emerging pattern:

HTTP/2.0 401
content-type: application/json; charset=utf-8
content-length: 57
x-dns-prefetch-control: off
x-frame-options: SAMEORIGIN
strict-transport-security: max-age=15552000; includeSubDomains
x-download-options: noopen
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
etag: W/"39-XMdFMOYAvKlfuOEWsD9JeogTHjI"
date:	Mon, 22 Mar 2021 01:58:11 GMT
{
    "code": 211,
    "message": "crypto/ecdsa: verification error"
}

Other requests include when the mouse is plugged in (/1/deviceplugin/get), unplugged (/1/deviceplugin/delete) and when the Windows user logs on or logs off (/7/servicelog/put). There’s also a few /1/setting/get and /1/settinglist/get which seem to correlate to interacting with the settings UI.

All receive an “Invalid Token” response similar to the delete request.

A petard, perchance to hoist?

At this point, a little devil taps me on the shoulder and asks, “Could these errors be a product of the MITM-ing, not the underlying problem?”

Thanks for asking. It really shouldn’t, TLS errors would prevent the connection from being made at all, so mitmproxy shouldn’t even see a conversation. Additionally, the match between the AsycManager file content and the requests is essentially perfect, and the files started appearing before the MITM-ing. Finally, the various Razer services record voluminous logs10, which confirm that the errors occur when the proxy isn’t active.

So no. Not today, little devil. It really receives some kind of authentication error for virtually every request, by all appearances due entirely to its own brokenness.

Mystery solved. New mysteries await

Summing up: Despite using the guest option to avoid signing up for an online account, the service phones home, gets an authentication error from the Razer server, and dutifully tries again.

Forever.

The system for making requests apparently involves putting them in XML files in a directory monitored by another process11, which neglects to delete some of them. The files pile up, until just reading the directory reduces the CPU to whimpering terror.

Case closed, but I have questions.

  1. How do I make it stop?
  2. Is it just something gone awry on my system, or is it a general problem?
  3. Given that I declined the online account, why is it phoning home at all?

Have you tried turning it off and back on again?

The obvious first step to answering #1 and #2 is to try re-installing. Going to remove, the Razer uninstaller offers a repair option.

Oh, wise Razer, you acknowledge that your magnificent handiwork might be broken? Yes, let’s repair it.

Repair re-downloads 400 MB of mouse driver, accomplishing exactly nothing. The existing RZR_ directory and AsyncManager files are untouched, and the delete requests continue to be made and rejected.

Moving on, I uninstall, delete all the directories identified earlier, re-download 400 MB of mouse driver, and run the install from scratch. Once again ignoring Razer’s plea that I sign up for a Razer account, I choose the guest option.

Upon completion, the \Razer Central\Accounts\RZR_(hex glop) directory has a different hex glop, the AsyncManager directory no longer accumulates UUID named XML files, and mitmproxy shows the flood of rejected /1/settting/delete has come to an end. The various requests receive non-error responses.

So the answer to question #1 is “re-install”, and #2 is at least “doesn’t affect everyone all the time”

In which guests may check out any time they like

I noted earlier that /1/setting/get appeared to correlate with action in the Razer mouse settings UI. In fact, watching the AsyncManager directory, adjusting the LED brightness triggers the creation of a file, like:

<AsyncItem>
  <Operation>Upload</Operation>
  <Setting>
    <Path>Devices\92\Features\03ae37d5-37db-4879-86d2-30e74f73a02f</Path>
    <Name>(version 4 UUID).xml</Name>
    <Encoding>Binary</Encoding>
    <Value>PD94bWwgdmVyc2lvbj0iMS4wIj8+DQo8TGVkQnJpZ2h0bmVzcyB4bWxuczp4c2k9Imh0dHA6Ly93d3cudzMub3JnLzIwMDEvWE1MU2NoZW1hLWluc3RhbmNlIiB4bWxuczp4c2Q9Imh0dHA6Ly93d3cudzMub3JnLzIwMDEvWE1MU2NoZW1hIj4NCiAgPFN0YXRlPjE8L1N0YXRlPg0KICA8QnJpZ2h0bmVzcz4yPC9CcmlnaHRuZXNzPg0KPC9MZWRCcmlnaHRuZXNzPg==</Value>
  </Setting>
</AsyncItem>

Using base64 to decode the value reveals… more XML:

<?xml version="1.0"?>
<LedBrightness xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <State>1</State>
  <Brightness>2</Brightness>
</LedBrightness>

With the Razer server no longer rejecting requests, the last pieces needed to answer question #3 click into place. Changing a setting results in a /1/setting/get followed by a /1/setting/put:

Content-Type: application/xml
User-Agent: RazerNatasha/7.3.32.173 (Windows NT 10.0; Win64; x32) Emily3/3.6.415.40812
Host: ec.razer.com
Content-Length: 2063
<COP>
  <User>
    <ID>RZR_(hex glop)</ID>
    <Token>(JWT)</Token>
  </User>
  <Setting>
    <Name>(version 4 UUID).xml</Name>
    <Path>Devices\92\Features\dfdc636d-8c78-4a9c-870d-83ceec46fe95</Path>
    <Data>PD94bWwgdmVyc2lvbj0iMS4wIj8+DQo8UG9sbGluZ1JhdGUgeG1sbnM6eHNpPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxL1hNTFNjaGVtYS1pbn
N0YW5jZSIgeG1sbnM6eHNkPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxL1hNTFNjaGVtYSI+DQogIDxWYWx1ZT41MDA8L1ZhbHVlPg0KPC9Qb2xsaW5nUmF0ZT
4=</Data>
    <Encoding>1</Encoding>
  </Setting>
  <Product>
    <Name>Emily3</Name>
    <Version></Version>
  </Product>
</COP>

The response looks like:

Date: Mon, 26 Apr 2021 04:51:40 GMT
Content-Type: text/xml; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
<?xml version='1.0' standalone='yes'?>
<COP>
  <User>
    <ID>RZR_(hex glop)</ID>
  </User>
  <Setting>
    <Name>(version 4 UUID).xml</Name>
    <Path>Devices\92\Features\dfdc636d-8c78-4a9c-870d-83ceec46fe95</Path>
    <Data>(signed AWS S3 URL)</Data>
    <UpdateTimestamp>1619412700</UpdateTimestamp>
    <OriginalSize>0</OriginalSize>
    <Encoding>1</Encoding>
  </Setting>
  <Product>
    <Name>Emily3</Name>
    <Version>1.0.0.0</Version>
  </Product>
  <Status>
    <Errno>1</Errno>
    <Message>settings saved</Message>
  </Status>
  <RequestStatus>
    <Errno>1</Errno>
    <Message>settings saved</Message>
    <Timestamp>1619412700</Timestamp>
    <Server>NzUwOSRYJDE2JFgkIPiX7JAwWQfdEXcxpFOA5ePNE59lguisqhP0JLzyQBM=</Server>
  </RequestStatus>
</COP>

Despite not signing up for an account, every single LED setting and key assignment change is sent to Razer’s servers, through a baroque sequence of services, XML files and REST API calls.

Whether the values are recorded or actually retrievable for guests is unclear12, but apparently, rather than writing a code path that doesn’t send XML around the world for people who don’t want an account, Razer chose to simply use the same flow as for real accounts, generating all the necessary authentication glop without user intervention13.

In my previous install, that authentication glop broke somewhere, and since there’s no actual account, I was never prompted to log in again or anything like that.

When you use the guest account, the Razer Central software displays the oddly worded message “You’re currently using a Guest account where all settings are anonymous to us” which notably does not exclude the settings being sent to them.

Conclusions

If you use Razer stuff, you might want to check whether it’s pilling up a bazillion files in \ProgramData\Razer\Razer Central\Accounts\RZR_…\Emily3\AsyncManager

If bandwidth, power consumption, or having every possible CPU cycle at your disposal14 are things you care about, you might want to avoid Razer software in general.

If you are concerned about privacy, the content phoned home15 doesn’t appear particularly sensitive for normal people, but is not entirely negligible either. Regardless of whether you have a Razer account, the software regularly reports when you log in, what specific serial number Razer devices you use, when you change settings, and the typical manufacturer make, model information reported by Windows16. While your real identity isn’t directly associated with these, it plausibly could be if the Razer data is absorbed into the databroker-industrial complex.

The hardware seems fine, my only complaint is the wheel made plasticky squeaks for quite a while. If it weren’t saddled with atrocious crapware, I’d consider it a quite good product.





Yes, these footnotes are bad and I feel bad. Thanks for checking.

  1. Razer™ is presumably a registered trademark of Razer, inc., and is used here solely because it’s the name of the company that made the thing being discussed. Lord knows, I don’t own it and wouldn’t want to. ↩

  2. Compressed download size. Installed, it’s more like a gig. ↩

  3. In fairness to Razer, in strict technical terms it isn’t all mouse driver. It’s a suite of software that lets you create “profiles” with individual settings per game and control all the lighting of your compatible peripherals together, all integrated with The Cloud. There’s even an SDK for 3rd parties who want to interact with this ecosystem. Yay. I guess. ↩

  4. Technically, it’s so you can have your settings on multiple machines, by simply installing 400 MB of crapware on each of them and logging into a dodgy online service, rather than setting each up manually like some kind of filthy savage. ↩

  5. Based on file timestamps, late December 2020, which IIRC was after a Razer software update. ↩

  6. Razer apparently withdrew from hackerone bug bounty system because they were ”…having difficulty keeping up with the volume of reports.” ↩

  7. It also has a web service that listens on localhost, but aside from duly noting the universally acknowledged truth that no self respecting mouse driver ought be without a built-in http server, I’ll leave that story for another day. ↩

  8. At least I can take comfort in knowing that whatever it may be phoning home isn’t trivially visible to others. Yay, I guess? ↩

  9. Keen observers may note the user agent is a) entirely different from the delete request, and b) an ancient version of Chrome (possibly CefSharp, based on strings and files present.) This may go some way to answering the question “how do you make a 400 MB mouse driver?” ↩

  10. For example, under \ProgramData\Razer

    \GameManager\Logs\GameManagerClient_RazerCentralService.log includes lines with the full XML of the requests and responses.

    \Razer Central\Logs\Razer Central Service.log similarly includes the error responses.

    \Razer Central\Logs\cef_<windows user name>.log contains endless messages like:

    [0425/111701.509:INFO:CONSOLE(1)] "[xxxx] WCRI ERROR [xxxx] {"name":"JsonWebTokenError","message":"invalid signature"} - attempting to retry.", source: https://razerid-assets.razerzone.com/static/js/main.6b4ea6c4.chunk.js (1)

    Unlike the AsyncManager files, these logs do not appear to be at risk of filling up your disk. They seem to be rotated around 5-10MB and limited to 5-10 files in rotation. The volume of writes is nontrivial, perhaps tens to hundreds of MB per day, but probably not a real problem. ↩

  11. Because this is the 21st century and that’s how we do IPC, I suppose. ↩

  12. Determining whether the returned signed URL may be subject to abuse is left as an exercise to the reader. Determining whether everyone’s settings end up on an unsecured S3 bucket somewhere is left to… do you feel lucky, punk? ↩

  13. Determining whether authentication glop produced in this manner serves any purpose beyond padding AWS bills is left as an exercise to the reader. ↩

  14. A thing which, if I recall correctly from my gamer days, gamers never care about. ↩

  15. Separately, the Razer software logs every executable it notices you running, as well as which executable owns the foreground window, presumably in an attempt to know whether you are playing a game. It also periodically scans your computer for games. I did not observe it obviously transmitting any of this information, although Razer’s TOS suggests it might. ↩

  16. To be entered by OEM. ↩