Jayd Saucedo

Blog

Python Keyboard Heatmapper
In my effort to become more familiar with working in Python I have been inventing things for myself to create. About a month ago I made a tool that searches a website for me. Now I've come up with the idea to map keystrokes, like a keylogger, and then use that information to make a visual display of your keyboard usage. Keys that are used more frequently have warmer colors, keys that are used less frequently have cooler colors. Thus, creating a keyboard heatmap.


(Usage from creating this post)

The first thing I did was try and figure out how to track key usage. Or in other words, how to make a keylogger. I decided to get this out of the way as easily (and lazily) as possible. So I didn't even use Python, I used AutoHotkey to map out every single key on the keyboard. Using AutoHotkey i made it so when you pressed a combination of keys it'd save all your key usage data to a text file, which I later used Python to read. So I made the rest of my python program around that, but when I was done with the program I went back and I got rid of the AutoHotkey usage. I didn't have much an idea what libraries to use in Python to capture key usage across the system. So what I ended up doing is downloading a Python keylogger I found in a google search and dissecting it to see how it handles key capturing. Turns out all you really need is PyHook and PyWin32.

From there capturing key usage is as easy as:

import pythoncom, pyHook

def OnKeyboardEvent(event):
	global ctrl_pressed, f9_pressed, keys_pressed
	
	key =  event.Key  
	time = event.Time
	if(key == "Lcontrol" or key == "Rcontrol"):
		ctrl_pressed = time
	if(key == "F9"):
		f9_pressed = time
	if(f9_pressed and ctrl_pressed and (abs(ctrl_pressed - f9_pressed) < 300)):
		print "Built!"
		buildHeatmap()
		
	if(event.Key == "Return" and event.Extended == 1):
		key = "NumpadReturn"
	
	keys_pressed.append(key)
	print len(keys_pressed)

# return True to pass the event to other handlers
	return True

# create a hook manager
hm = pyHook.HookManager()
# watch for all mouse events
hm.KeyDown = OnKeyboardEvent
# set the hook
hm.HookKeyboard()
# wait forever
pythoncom.PumpMessages() 

Now, my Python program has 3 global variables. One is "keys_pressed" which is a set which contains key press information, and f9_pressed and ctrl_pressed which contain the last time the control key and f9 key were pressed. Control plus F9 is my shortcut to compile the heatmap image. Whenever either is pressed I store the time in milliseconds in their respective variable. If they are pressed within 300 milliseconds of eachother I build the heatmap image.

So, now that I got key capture over with my next big challenge was image manipulation. I started out with this base image. Which I found in a quick google image search:

Now I have to figure out how to color each key. I came up with the idea that I wanted there to a lot of variation in color, and that I wanted things to get warmer on a long gradient. So what I did was make a 1000x1 pixel image of a heatmap gradient. My uploader couldn't handle stretching a 1000x1 pixel image. So here's a 500x100 pixel version:


What the program actually does is find the most used key, and set that to the 1000th pixel (the warmest color) and then every number below that is scaled respectively.

Here is nearly my entire "buildHeatmap" function:

def buildHeatmap():
	global keys_pressed
	
	t = {}

	for i in keys_pressed:
		if i in t:
			t[i] = t[i] + 1.0
		else:
			t[i] = 1.0

	biggest = max(v for k, v in t.iteritems())


	heatmap = Image.open("heat_gradient.png")
	im = Image.open("keyboard.png")

	for k, v in t.iteritems():
		
		if(k not in key_location):
			continue
		
		heatbox = ((int((v/biggest)*1000)-1), 0, int((v/biggest)*1000), 1)
		heatRegion = heatmap.crop(heatbox)

		heatColorInfo = heatRegion.getcolors()[0][1]
		
		newImg = Image.new('RGB', (img_w, img_h), heatColorInfo)
		
		box = (key_location[k][0], key_location[k][1], key_location[k][0]+img_w, key_location[k][1]+img_h)
		region = im.crop(box)
		region = Image.blend(region, newImg, .5)

		im.paste(region, box)

	im.save('keyboard_heatmap.jpg')		
	im.show()
	
	return True

So again, let's say I have pressed 3 keys. I pressed "A" 400 times, I presed "B" 1200 times, and I pressed "C" 700 times. That means my color for the "B" key would be at pixel 1000 of my gradient. "A" would be at the 333rd pixel ((400/1200)*1000) and "C" would be at the 583rd pixel ((700/1200)*1000). Then I use whatever color is at those pixels to color a new image the same size as the key and overlap it on my plain keyboard image.

Now, when I say "here is NEARLY my entire function". I mean that I am leaving out the most time consuming and tedious part of this program. That would be mapping my keyboard image. What I literally had to do was find out where each pixel for each key started, and where it ended, and then make a dictionary of all the respective key pixel locations. This took probably about an hour, but it felt like it took days. In the end I had a dictionary that looked like this:

key_location = {"Escape": (13, 10, 0), "F1": (78, 10, 0), "F2": (116, 10, 0), "F3": (154, 10, 0), "F4": (193, 10, 0), "F5": (253, 10, 0), "F6": (291, 10, 0), "F7": (329, 10, 0), "F8": (367, 10, 0), "F9": (428, 10, 0), "F10": (466, 10, 0), "F11": (504, 10, 0), "F12": (542, 10, 0), "Snapshot": (601, 10, 0), "Scroll`": (639, 10, 0), "Pause": (677, 10, 0), "Oem_3": (13, 82, 0), "1": (52, 83, 0), "2": (89, 82, 0), "3": (127, 82, 0), "4": (165, 82, 0), "5": (203, 82, 0), "6": (242, 82, 0), "7": (280, 82, 0), "8": (318, 82, 0), "9": (356, 82, 0), "0": (394, 82, 0), "Oem_Minus": (432, 82, 0), "Oem_Plus": (470, 82, 0), "Back": (508, 82, 1), "Insert": (603, 82, 0), "Home": (641, 82, 0), "Prior": (679, 82, 0), "NumLock": (738, 82, 0), "Divide-": (776, 82, 0), "Multiply*": (814, 82, 0), "Subtract": (852, 82, 0), "Tab": (13, 122, 2), "Q": (69, 122, 0), "W": (107, 122, 0), "E": (146, 122, 0), "R": (184, 122, 0), "T": (222, 122, 0), "Y": (260, 122, 0), "U": (297, 122, 0), "I": (336, 122, 0), "O": (374, 122, 0), "P": (412, 122, 0), "Oem_4": (449, 122, 0), "Oem_6": (487, 122, 0), "Oem_5": (526, 122, 3), "Delete": (602, 122, 0), "End": (640, 122, 0), "Next": (678, 122, 0), "Numpad7": (737, 122, 0), "Numpad8": (775, 122, 0), "Numpad9": (813, 122, 0), "Add": (852, 122, 4), "Capital": (13, 161, 5), "A": (79, 161, 0), "S": (117, 161, 0), "D": (156, 161, 0), "F": (194, 161, 0), "G": (232, 161, 0), "H": (270, 161, 0), "J": (308, 161, 0), "K": (346, 161, 0), "L": (384, 161, 0), "Oem_1": (422, 161, 0), "Oem_7": (461, 161, 0), "Return": (499, 161, 6), "Numpad4": (737, 161, 0), "Numpad5": (776, 161, 0), "Numpad6": (814, 161, 0), "Lshift": (13, 200, 7), "Z": (106, 200, 0), "X": (145, 200, 0), "C": (183, 200, 0), "V": (222, 200, 0), "B": (260, 200, 0), "N": (298, 200, 0), "M": (336, 200, 0), "Oem_Comma": (374, 200, 0), "Oem_Period": (413, 200, 0), "Oem_2": (451, 200, 0), "Rshift": (489, 200, 8), "Up": (641, 200, 0), "Numpad1": (738, 201, 0), "Numpad2": (775, 201, 0), "Numpad3": (814, 201, 0), "NumpadReturn": (851, 201, 9), "Lcontrol": (13, 240, 10), "Lwin": (69, 240, 11), "Lmenu": (119, 240, 11), "Space": (169, 240, 12), "Rmenu": (377, 240, 11), "Rwin": (427, 240, 11), "Apps": (476, 240, 11), "Rcontrol": (525, 240, 10), "Left": (603, 240, 0), "Down": (641, 240, 0), "Right": (679, 240, 0), "Numpad0": (738, 240, 13), "Decimal": (814, 240, 0)}
Yes, that is the location for every single key on a standard keyboard. The first two numbers are the pixel location (like a Cartesian graph) and the third number is what type of key it is. A lot of keys are the same size so I figured it'd be faster just listing the type of key and figuring out the exact size later. Some keys though, such as the caps lock, tab, ctrl, space, numpad zero, numpad enter, backspace, and numpad plus keys are all sorts of wonky sizes though. So yes, I made a 14 case switch that I won't be posting.

All in all it was very educational. Exactly what I was hoping for. Here's a link to the final product.