This blog post describes input handling from the reMarkable 2 on a remote PC using Python and SSH.

Everything you do with your reMarkable tablet outside of its intended use is at your own risk, neither reMarkable nor I can be held liable if you brick your device.

Happy hacking :)

About reMarkable

“Replace your notebooks and printed documents with the only tablet that feels like paper. reMarkable bridges the gap between pen and paper, and your digital devices. Easily access your notes from your laptop or phone, where they’re always readily available to review and reuse. “ - remarkable.com


After receiving the ‘reMarkable 2’ tablet, I was interested in what makes it tick. A quick online search shows it is running Linux!

The following sources were of great help to get more insights in the reMarkable OS:

Getting into the device

While browsing on the tablet itself something caught my eye,

under Menu > Settings > Help > Copytights and licenses I was greeted with a “GPLv3 Compliance” note which states the following:

GPLv3 Compliance

The General Public License version 3 and the Lesser General Public License version 3 also requires you as and end-user to be able to access your device to be able to modify the copyrighted software licenced under these licenses running on it.

To do so, this device acts as an USB ethernet device, and you can connect using the SSH protocol using the username root and the password `«REDACTED»``.

This feels like a dream come true, how easy could have this have been, just run SSH and you’re in! When the reMarkable tablet is connected using USB to a computer it acts as a network interface and is accessible using the IP 10.11.99.1, meanwhile, the tablet is also accessible wirelessly when it is connected to your local WiFi network.

SSH on different platforms

Linux

  open applications, search for term, enter
  ssh [email protected]
  type the password

MacOS

  open Spotlight (Cmd-Space) type Terminal, enter
  ssh [email protected]
  type the password

Windows

  You do you, figure it out yourself

Welcome to: ZERO SUGAR

$ ssh [email protected]
[email protected]'s password: 
reMarkable
╺━┓┏━╸┏━┓┏━┓   ┏━┓╻ ╻┏━╸┏━┓┏━┓
┏━┛┣╸ ┣┳┛┃ ┃   ┗━┓┃ ┃┃╺┓┣━┫┣┳┛
┗━╸┗━╸╹┗╸┗━┛   ┗━┛┗━┛┗━┛╹ ╹╹┗╸
reMarkable: ~/ uname -a
Linux reMarkable 5.4.70-v1.1.5-rm11x #1 SMP PREEMPT Fri Nov 12 14:59:18 UTC 2021 armv7l GNU/Linux
reMarkable: ~/

And now the fun can begin :)

The idea

Both reMarkable 1 and 2 have great touch and pen interfaces that feel just right. This made my drawing tablet lose its purpose, but the reMarkable tablet can’t fully replace a drawing tablet connected to a general-purpose computer. Or can it…

As mentioned before, the reMarkable is powered by Linux and the end-user has root-level access using SSH. Meaning I have the full reMarkable Linux OS right at my fingertips. Then I had the idea

“What if you could use the pen input of the reMarkable tablet as mouse input (or even pen input) for any PC."

Let’s focus on pen first and touch later. Firstly I want the get the remarkable to control the mouse cursor on a different computer. Later that can be changed to a pen device on a remote computer and touch can be used as a mouse.

Steps

  1. Find out how touch & pen input is handled on this device
  2. Access raw touch & pen data over SSH using Python
  3. Parse raw data on a remote computer
  4. Parsing pen input data
  5. Parsing touch input data
  6. Improving the code
  7. Create new uinput device on the host PC and feed the reMarkable event data

Step 1 | Find out how touch & pen input is handled on this device

The first place I look at is /dev/input since that is a common place in Linux for input events.

reMarkable: ~/ ls -l /dev/input 
drwxr-xr-x    2 root     root           100 Jan 11 10:50 by-path
crw-rw----    1 root     input      13,  64 Jan 11 10:50 eve6nt2
lrwxrwxrwx    1 root     root             6 Jan 11 10:50 touchscreen0 -> event1

And it looks like it’s indeed here, but that’s quite easy to test out by watching cat /dev/input/eventX or hexdump /dev/input/eventX

  • /dev/input/event0 Gives no output on touch or pen, so let’s skip that
reMarkable: ~/ hexdump /dev/input/event0
^C
reMarkable: ~/
  • /dev/input/event1 This shows data when the pen is getting near, buth the tip and the eraser is being picked up
reMarkable: ~/ hexdump /dev/input/event1
0000000 32c8 61e3 2cb3 0009 0001 0140 0001 0000
0000010 32c8 61e3 2cb3 0009 0003 0000 1a01 0000
0000020 32c8 61e3 2cb3 0009 0003 0001 2ed8 0000
0000030 32c8 61e3 2cb3 0009 0003 0019 0037 0000
0000040 32c8 61e3 2cb3 0009 0003 001a fa24 ffff
0000050 32c8 61e3 2cb3 0009 0003 001b 0b54 0000
0000060 32c8 61e3 2cb3 0009 0000 0000 0000 0000
0000070 32c8 61e3 33b4 0009 0003 0000 1a02 0000
0000080 32c8 61e3 33b4 0009 0003 0019 0035 0000
0000090 32c8 61e3 33b4 0009 0000 0000 0000 0000^C
reMarkable: ~/
  • /dev/input/event2 Touch data can be found here
reMarkable: ~/ hexdump /dev/input/event2   
0000000 32aa 61e3 f25f 0000 0003 0039 1b9d 0000
0000010 32aa 61e3 f25f 0000 0003 0035 039d 0000
0000020 32aa 61e3 f25f 0000 0003 0036 02e5 0000
0000030 32aa 61e3 f25f 0000 0003 003a 0053 0000
0000040 32aa 61e3 f25f 0000 0003 0030 0008 0000
0000050 32aa 61e3 f25f 0000 0003 0034 0001 0000
0000060 32aa 61e3 f25f 0000 0000 0000 0000 0000
0000070 32aa 61e3 e667 0001 0003 0039 ffff ffff
0000080 32aa 61e3 e667 0001 0000 0000 0000 0000^c
reMarkable: ~/

The above findings are also validated on this Github issue

Of course, this data is not readable as text, but it is usable in Python, the language I use for this experiment.

Now that is known what input event is for touch & pen, let’s move on to the next step where a connection to the tablet will be made using Python and SSH.

Step 2 | Access raw touch & pen data over SSH using Python

In this example I’m using Python 3.9.7

Preparing the Python development environment

To connect to the reMarkable device using Python, a library called paramiko is used, but there are more alternatives available if that is desired.

Later to let the reMarkable tablet control a remote mouse cursor it is necessary to know the reMarkable’s screen size as well as the screen size of the host pc. The reMarkable has a screen resolution of 1872x1404 NOTE: the default orientation of the remarkable isn’t portrait as it’s being used as, but rather landscape with then pen on the bottom To get the host’s screen size a library called screeninfo is used. To control the mouse cursor pyautogui is used. These are the Python libraries being used for this step.

Let’s place the required dependencies into a file:

requirements.txt

paramiko
screeninfo
pyautogui

pip install -r requirements.txt


Before I dive into the code I want one thing out of the way:

  • By no means my code is the “right” way to do this;
  • This is merely an example/proof of concept to show that it works; Let’s move on now to the fun stuff :3

Connecting using SSH

main.py

from datetime import datetime
from enum import Enum
import paramiko
from paramiko import SSHClient

ssh_ip = "10.11.99.1" # Default to USB ip
ssh_port = 22
ssh_username = "root"
ssh_password = "password" # Update with your password

def main():
  print("Attempting to connect...")

  ssh_client = SSHClient()
  try:
    # Since this is an test environment let paramiko automatically set the missing host key
    ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())

    ssh_client.connect(
        hostname=ssh_ip,
        port=ssh_port,
        username=ssh_username,
        password=ssh_password,
    )
  except: # Catch connection error
    print("Connect failed")
    exit(0)

  print("Connect successfully")

if __name__ == "__main__":
  main()

Running the code above should result in the following output:

$ python main.py 
Attempting to connect...
Connect successfully
$

Or the following if the device is not accessible:

$ python main.py 
Attempting to connect...
Connect failed
$

Reading /dev/input/eventX

This connects to the tablet and exits immediately after, which isn’t what we need. Time to move on to reading /dev/input/event1 (pen input) using ssh

First, make some changes to the top of the main file:

main.py

...
from paramiko import SSHClient

linux_event = "/dev/input/event1"

ssh_ip = "10.11.99.1" # Default to USB ip
...

And after connecting, it calls a function to read the input event:

main.py

...
  print("Connect successfully")
  handle_input(ssh_client)

def handle_input(ssh_client):
  ssh_transport = ssh_client.get_transport()
  ssh_channel = ssh_transport.open_session()
  ssh_channel.exec_command(f"cat {linux_event}")

  try:
    while ssh_transport.is_active():
      packet_size = 32
      data = ssh_channel.recv(packet_size).hex()
      for packet in [data[i : i + packet_size] for i in range(0, len(data), packet_size)]: # Sometimes a packet is too large
          print(packet)
  except KeyboardInterrupt:
    print("Interrupted")
    ssh_client.close()
    exit(0)

if __name__ == "__main__":
...

After executing this code, it will show the raw event data in hex format:

$ python main.py 
Attempting to connect...
Connect successfully
3646e361725703000100410101000000
3646e3617257030003000000c9290000
3646e361725703000300010088210000
3646e361725703000300190021000000
3646e3617257030003001a004cebffff
3646e3617257030003001b00a8fdffff
3646e361725703000000000000000000
3646e3615f5e030003000000c8290000
3646...

Let’s explore what this actually means and why packet_size is 32

Using the Linux command cat the input event can be read as streaming output. Every time we request 32 bytes from the running cat command, hence packet_size, then in Python we use .hex() on the data that has been received to convert it to HEX so it can be used more easily.

In the next step (step 3) we’ll explore the data in more detail.

Step 3 | Parse raw data on a remote computer

To get a better understanding, let’s explore the following received packet 3646e361 72570300 0300 0100 88210000

Note: This is LSB (Least Significant Bit first), so every received message part* needs to be reverse

*message part’s are as follows:

Type MSB LSB DEC
seconds 3646e361 61e34636 1642284598
useconds 72570300 00035772 218994
type 0300 0003 3
code 0100 0001 1
value 88210000 00002188 8584

The seconds and useconds can be used to construct date & time information, in this case: 15-01-2022 22:09:58.218994 (DD-MM-YYYY HH:MM:SS.SSSSSS)

Now we have three unknown variables type, code, and value.

While value is quite self-explanatory, let’s focus on the other two.

After consulting kernel.org and linux/input-event-codes.h we can see what type and code mean.

type:

  • EV_SYN: 0x00
  • EV_KEY: 0x01
  • EV_REL: 0x02
  • EV_ABS: 0x03
  • EV_…
  • Full TYPE info

code:

depends on type

Getting back at the message we were exploring we now know that type 3 means EV_ABS (absolute position) and code 1 means ABS_Y (absolute y position), so the pen was at position 8584 (value) on the y-axis.

Let’s move on to the next step (step 4) where the data will be used to move the mouse cursor

Step 4 | Parsing pen input data

Now that event data is received in Python, it can be parsed and used to, for example, move the mouse cursor.

The following type’s and code’s will be used:

  • EV_KEY | To indicate which tool is used
    • BTN_TOOL_PEN
    • BTN_TOOL_RUBBER
  • EV_ABS | To get the absolute tool values
    • ABS_X | Absolute X position as reported by the Wacom driver
    • ABS_Y | Absolute Y position on the screen, ranging from 0 to
    • ABS_TILT_X | Absolute tilt X of the pen
    • ABS_TILT_Y | Absolute tilt Y of the pen
    • ABS_DISTANCE | Absolute distance the pen tip is from the screen

Let’s add some code to call a function with the HEX data:

main.py

...
      for packet in [data[i : i + packet_size] for i in range(0, len(data), packet_size)]: # Sometimes a packet is too large
          date_time, pen_down, eraser_down, posx, posy, tiltx, tilty, distance = parse_packet(packet)
          print(str(date_time) + " - " + str(pen_down) + " - " + str(eraser_down) + " - " + str(posx) + " - " + str(posy) + " - " + str(tiltx) + " - " + str(tilty) + " - " + str(distance))
  except KeyboardInterrupt:
...

Now it’s time to create a function in our Python code that parses the HEX data to usable variables:

main.py

# Store the last know values to send with the new data
last_date_time = None
last_pen_down = False
last_eraser_down = False
last_posx = 0
last_posy = 0
last_tiltx = 0
last_tilty = 0
last_distance = 0

def parse_packet(packet):
  global last_date_time
  global last_pen_down
  global last_eraser_down
  global last_posx
  global last_posy
  global last_tiltx
  global last_tilty
  global last_distance

  raw_data = [packet[j : j + 2] for j in range(0, len(packet), 2)] # Split the raw packet every 2 characters
  if len(raw_data) == 16: # If the packet is complete
    sec = int("".join(raw_data[0:4][::-1]), base=16)
    usec = int("".join(raw_data[4:8][::-1]), base=16)
    type = int("".join(raw_data[8:10][::-1]), base=16)
    code = int("".join(raw_data[10:12][::-1]), base=16)
    value = int("".join(raw_data[12:][::-1]), base=16)

    last_date_time = datetime.fromtimestamp(sec + usec / 1e6) # Convert received sec & usec to Python datetime object

    evtype = EVTYPE(type).name
    evcode = get_code(evtype, code)

    if evtype == "EV_ABS":
      if evcode == "ABS_X":
        last_posx = value
      if evcode == "ABS_Y":
        last_posy = value
      if evcode == "ABS_TILT_X":
        last_tiltx = value
      if evcode == "ABS_TILT_Y":
        last_tilty = value
      if evcode == "ABS_DISTANCE":
        last_distance = value
        
    if evtype == "EV_KEY":
      if evcode == "BTN_TOOL_PEN":
        last_pen_down = True if value == 1 else False
      if evcode == "BTN_TOOL_RUBBER":
        last_eraser_down = True if value == 1 else False

  return (last_date_time, last_pen_down, last_eraser_down, last_posx, last_posy, last_tiltx, last_tilty, last_distance)

def get_code(evtype, raw_code):
  evcode = ""

  if evtype == "EV_SYN":
    evcode = EVMAPSYN(raw_code).name

  if evtype == "EV_KEY":
    evcode = EVMAPKEY(raw_code).name

  if evtype == "EV_ABS":
    evcode = EVMAPABS(raw_code).name
        
  return evcode

class EVTYPE(Enum):
    EV_SYN = 0x00
    EV_KEY = 0x01
    EV_REL = 0x02
    EV_ABS = 0x03
    EV_MSC = 0x04
    EV_SW = 0x05
    EV_LED = 0x11
    EV_SND = 0x12
    EV_REP = 0x14
    EV_FF = 0x15
    EV_PWR = 0x16
    EV_FF_STATUS = 0x17

class EVMAPKEY(Enum):
    BTN_TOOL_PEN = 0x140
    BTN_TOOL_RUBBER = 0x141
    BTN_TOUCH = 0x14A

class EVMAPSYN(Enum):
    SYN_REPORT = 0x00
    SYN_CONFIG = 0x01
    SYN_MT_REPORT = 0x02
    SYN_DROPPED = 0x03

class EVMAPABS(Enum):
    ABS_X = 0x00
    ABS_Y = 0x01
    ABS_PRESSURE = 0x18
    ABS_DISTANCE = 0x19
    ABS_TILT_X = 0x1A
    ABS_TILT_Y = 0x1B

Executing this new code shows the following output:

$ python main.py 
Attempting to connect...
Connect successfully
2022-01-16 01:07:52.237602 - True - False - 0 - 0 - 0 - 0 - 0
2022-01-16 01:07:52.237602 - True - False - 5336 - 0 - 0 - 0 - 0
2022-01-16 01:07:52.237602 - True - False - 5336 - 10383 - 0 - 0 - 0
2022-01-16 01:07:52.237602 - True - False - 5336 - 10383 - 0 - 0 - 22
2022-01-16 01:07:52.237602 - True - False - 5336 - 10383 - 900 - 0 - 22
2022-01-16 01:07:52.239384 - True - False - 5339 - 10383 - 900 - 200 - 22
2022-01-16 01:07:52.241211 - True - False - 5347 - 10381 - 900 - 200 - 19
2022-01-16 01:07:52.241211 - True - False - 5347 - 10381 - 900 - 200 - 17
2022-01-16 01:07:52.254145 - True - False - 5411 - 10387 - 900 - 200 - 7
2022-01-16 01:07:52.254145 - True - False - 5411 - 10390 - 900 - 200 - 7
2022-01-16 01:07:52.254145 - True - False - 5411 - 10390 - 900 - 200 - 8
2022-01-16 01:07:52.328054 - True - False - 5355 - 10284 - 4294965496 - 200 - 0
2022-01-16 01:07:52.330034 - True - False - 5361 - 10291 - 4294965496 - 200 - 12
2022-01-16 01:07:52.372098 - True - False - 5609 - 10466 - 4294965696 - 300 - 81
2022-01-16 01:07:52.373945 - False - False - 5609 - 10479 - 4294965696 - 300 - 81

This output shows every received aspect from touching the pen onto the reMarkable screen. Note: tilt X and tilt Y can be quite unreliable and sometimes reports big numbers.

Step 5 | Parsing touch input data

Inside of the code change from the pen input event (/dev/input/event1) to the touch input event (/dev/input/event2)

main.py

...
# linux_event = "/dev/input/event1"
linux_event = "/dev/input/event2"
...

# inside of def parse_packet(packet):
...
    evcode = get_code(evtype, code)
    print(evcode) # Print raw received EV codes

    if evtype == "EV_ABS":
...

After executing the modified code above we can see that the previously generated pen data no longer works and that it prints different event codes:

$ python main.py
ABS_MT_SLOT
2022-01-16 12:18:16.564759 - False - False - 0 - 0 - 0 - 0 - 0
ABS_MT_POSITION_X
2022-01-16 12:18:16.564759 - False - False - 0 - 0 - 0 - 0 - 0
ABS_MT_PRESSURE
2022-01-16 12:18:16.564759 - False - False - 0 - 0 - 0 - 0 - 0
ABS_MT_TOUCH_MINOR
2022-01-16 12:18:16.564759 - False - False - 0 - 0 - 0 - 0 - 0
ABS_MT_ORIENTATION
2022-01-16 12:18:16.564759 - False - False - 0 - 0 - 0 - 0 - 0
ABS_MT_SLOT
2022-01-16 12:18:16.564759 - False - False - 0 - 0 - 0 - 0 - 0
ABS_MT_POSITION_X
2022-01-16 12:18:16.564759 - False - False - 0 - 0 - 0 - 0 - 0
ABS_MT_POSITION_Y
2022-01-16 12:18:16.564759 - False - False - 0 - 0 - 0 - 0 - 0
ABS_MT_PRESSURE
2022-01-16 12:18:16.564759 - False - False - 0 - 0 - 0 - 0 - 0^C
$

Let’s write down every detected event code (also for multi-touch)

ABS_MT_SLOT
ABS_MT_TRACKING_ID
ABS_MT_POSITION_X
ABS_MT_POSITION_Y
ABS_MT_PRESSURE
ABS_MT_ORIENTATION
ABS_MT_TOUCH_MAJOR
ABS_MT_TOUCH_MINOR

Knowing what event code it sends out, modify the parse_packet(packet) functions as follows:

# Store the last know values to send with the new data
last_date_time = None
last_slot = 0
last_tracking_id = 0
last_posx = 0
last_posy = 0
last_pressure = 0
last_orientation = 0
last_touch_major = 0
last_touch_minor = 0

def parse_packet(packet):
  global last_date_time
  global last_slot
  global last_tracking_id
  global last_posx
  global last_posy
  global last_pressure
  global last_orientation
  global last_touch_major
  global last_touch_minor

  raw_data = [packet[j : j + 2] for j in range(0, len(packet), 2)] # Split the raw packet every 2 characters
  if len(raw_data) == 16: # If the packet is complete
    sec = int("".join(raw_data[0:4][::-1]), base=16)
    usec = int("".join(raw_data[4:8][::-1]), base=16)
    type = int("".join(raw_data[8:10][::-1]), base=16)
    code = int("".join(raw_data[10:12][::-1]), base=16)
    value = int("".join(raw_data[12:][::-1]), base=16)

    last_date_time = datetime.fromtimestamp(sec + usec / 1e6) # Convert received sec & usec to Python datetime object

    evtype = EVTYPE(type).name
    evcode = get_code(evtype, code)

    if evtype == "EV_ABS":
      if evcode == "ABS_MT_SLOT":
        last_slot = value
      if evcode == "ABS_MT_TRACKING_ID":
        last_tracking_id = value
      if evcode == "ABS_MT_POSITION_X":
        last_posx = value
      if evcode == "ABS_MT_POSITION_Y":
        last_posy = value
      if evcode == "ABS_MT_PRESSURE":
        last_pressure = value
      if evcode == "ABS_MT_ORIENTATION":
        last_orientation = value
      if evcode == "ABS_MT_TOUCH_MAJOR":
        last_touch_major = value
      if evcode == "ABS_MT_TOUCH_MINOR":
        last_touch_minor = value
        
  return (last_date_time, last_slot, last_tracking_id, last_posx, last_posy, last_pressure, last_orientation, last_touch_major, last_touch_minor)

Also change the function call to account for the new data

...
for packet in [data[i : i + packet_size] for i in range(0, len(data), packet_size)]: # Sometimes a packet is too large
        date_time, slot, tracking_id, posx, posy, pressure, orientation, touch_major, touch_minor = parse_packet(packet)
        print(str(date_time) + " - " + str(slot) + " - " + str(tracking_id) + " - " + str(posx) + " - " + str(posy) + " - " + str(pressure) + " - " + str(orientation) + " - " + str(touch_major) + " - " + str(touch_minor))
  except KeyboardInterrupt:
...

Now, after executing the modified code, we can look at the results:

$ python main.py
2022-01-16 12:32:56.843194 - 0 - 7189 - 546 - 800 - 98 - 3 - 17 - 17
2022-01-16 12:32:56.843194 - 0 - 7189 - 546 - 800 - 98 - 3 - 17 - 17
2022-01-16 12:32:56.854692 - 0 - 7189 - 546 - 800 - 97 - 3 - 17 - 17
2022-01-16 12:32:56.854692 - 1 - 7189 - 546 - 800 - 97 - 3 - 17 - 17
2022-01-16 12:32:56.854692 - 1 - 7189 - 546 - 800 - 103 - 3 - 17 - 17
2022-01-16 12:32:56.854692 - 1 - 7189 - 546 - 800 - 103 - 3 - 17 - 17
2022-01-16 12:32:56.866617 - 0 - 7189 - 546 - 800 - 103 - 3 - 17 - 17^c
$
Event Description
ABS_MT_SLOT What finger ID the data is from (first touching finger = 0, second = 1, etc…)
ABS_MT_TRACKING_ID Unique ID of initiated touch contact
ABS_MT_POSITION_X The center X position ranging from 0 to screen width where the touch happened
ABS_MT_POSITION_Y The center Y position ranging from 0 to screen height where the touch happened
ABS_MT_PRESSURE Pressure on contact area
ABS_MT_ORIENTATION Ellipse orientation
ABS_MT_TOUCH_MAJOR Major axis of touching ellipse
ABS_MT_TOUCH_MINOR Minor axis (omit if circular)

Step 6 | Improving the code

Before we continue with our journey and control the mouse cursor with the received data. It is important to clean the current code first and split things up into multiple modules and add threads so we’re not limited to reading a single event at the time.

The file structure is as follows:

project
│   main.py
│   input_event_codes.py
|   input_manager.py
|   local_input_manager.py
|   ssh_manager.py
|   utils.py

The input_event_codes.py file is just a conversion from linux/input-event-codes.h to python with some redundant input event codes removed.

input_event_codes.py

from enum import Enum

def get_code(evtype, raw_code):
  evcode = ""

  if evtype == "EV_SYN":
    evcode = EVCODESYN(raw_code).name

  if evtype == "EV_KEY":
    evcode = EVCODEKEY(raw_code).name

  if evtype == "EV_ABS":
    evcode = EVCODEABS(raw_code).name
        
  return evcode

# Event types
class EVTYPE(Enum):
    EV_SYN = 0x00
    EV_KEY = 0x01
    EV_REL = 0x02
    EV_ABS = 0x03
    EV_MSC = 0x04
    EV_SW = 0x05
    EV_LED = 0x11
    EV_SND = 0x12
    EV_REP = 0x14
    EV_FF = 0x15
    EV_PWR = 0x16
    EV_FF_STATUS = 0x17
    EV_MAX = 0x1f

# Synchronization events
class EVCODESYN(Enum):
    SYN_REPORT = 0x00
    SYN_CONFIG = 0x01
    SYN_MT_REPORT = 0x02
    SYN_DROPPED = 0x03
    SYN_MAX = 0x0f

# Key and button events
class EVCODEKEY(Enum):
    BTN_TOOL_PEN = 0x140
    BTN_TOOL_RUBBER = 0x141
    BTN_TOUCH = 0x14a
    KEY_MAX = 0x2ff

# Absolute events
class EVCODEABS(Enum):
    ABS_X = 0x00
    ABS_Y = 0x01
    ABS_PRESSURE = 0x18
    ABS_DISTANCE = 0x19
    ABS_TILT_X = 0x1a
    ABS_TILT_Y = 0x1b
    ABS_TOOL_WIDTH = 0x1c
    ABS_VOLUME = 0x20
    ABS_MISC = 0x28
    ABS_MT_SLOT = 0x2f # MT slot being modified
    ABS_MT_TOUCH_MAJOR = 0x30 # Major axis of touching ellipse
    ABS_MT_TOUCH_MINOR = 0x31 # Minor axis (omit if circular)
    ABS_MT_WIDTH_MAJOR = 0x32 # Major axis of approaching ellipse
    ABS_MT_WIDTH_MINOR = 0x33 # Minor axis (omit if circular)
    ABS_MT_ORIENTATION = 0x34 # Ellipse orientation
    ABS_MT_POSITION_X = 0x35 # Center X touch position
    ABS_MT_POSITION_Y = 0x36 # Center Y touch position
    ABS_MT_TOOL_TYPE = 0x37 # Type of touching device
    ABS_MT_BLOB_ID = 0x38 # Group a set of packets as a blob
    ABS_MT_TRACKING_ID = 0x39 # Unique ID of initiated contact
    ABS_MT_PRESSURE = 0x3a # Pressure on contact area
    ABS_MT_DISTANCE = 0x3b # Contact hover distance
    ABS_MT_TOOL_X = 0x3c # Center X tool position
    ABS_MT_TOOL_Y = 0x3d # Center Y tool position
    ABS_MAX = 0x3f

utils.py

def map(x, in_min, in_max, out_min, out_max):
    return (x - in_min) * (out_max - out_min) / (in_max - in_min) + out_min

ssh_manager.py

import paramiko
from paramiko import SSHClient

class SSH_MANAGER():
    def __init__(self, host, port, username, password):
        self.host = host
        self.port = port
        self.username = username
        self.password = password

        self.ssh_client = SSHClient()
        self.is_client_connected = False

    def connect(self):
        try:
            # Since this is an test environment let paramiko automatically set the missing host key
            self.ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
            self.ssh_client.connect(
                hostname=self.host,
                port=self.port,
                username=self.username,
                password=self.password,
            )
        except: # Catch connection error
            self.is_client_connected = False
            return
        
        self.is_client_connected = True
    
    def disconnect(self):
        if self.ssh_client is not None:
            self.ssh_client.close()
        
        self.is_client_connected = False

    def is_connected(self):
        return self.is_client_connected

    def get_client(self):
        return self.ssh_client

input_manager.py

import threading
from datetime import datetime
import input_event_codes

class INPUT_EVENT_MANAGER():
    def __init__(self, ssh_client):
        self.ssh_client = ssh_client
        self.last_packet_data = {}
        self.new_data_available = False

    def run(self, linux_event):
        self.thread = threading.Thread(target=self._start, args=(linux_event,))
        self.thread.start()

    def stop(self):
        if self.thread is not None:
            self.thread.join()
            self.thread = None

    def _start(self, linux_event):
        ssh_transport = self.ssh_client.get_transport()
        ssh_channel = ssh_transport.open_session()
        ssh_channel.exec_command(f"cat {linux_event}")

        while ssh_transport.is_active():
            packet_size = 32
            data = ssh_channel.recv(packet_size).hex()
            for packet in [data[i: i + packet_size] for i in range(0, len(data), packet_size)]: # Sometimes a packet is too large
                self.parse_raw_packet(packet)
    
    def parse_raw_packet(self, packet):
        raw_data = [packet[j : j + 2] for j in range(0, len(packet), 2)] # Split the raw packet every 2 characters
        if len(raw_data) == 16: # If the packet is complete
            sec = int("".join(raw_data[0:4][::-1]), base=16)
            usec = int("".join(raw_data[4:8][::-1]), base=16)
            type = int("".join(raw_data[8:10][::-1]), base=16)
            code = int("".join(raw_data[10:12][::-1]), base=16)
            value = int("".join(raw_data[12:][::-1]), base=16)

            date_time = datetime.fromtimestamp(sec + usec / 1e6) # Convert received sec & usec to Python datetime object

            evtype = input_event_codes.EVTYPE(type).name
            evcode = input_event_codes.get_code(evtype, code)

            self.last_packet_data["TIME"] = date_time
            self.last_packet_data[evcode] = value
            self.new_data_available = True

    def has_new_data(self):
        new_data_available = self.new_data_available
        self.new_data_available = False
        return new_data_available
    
    def get_last_data(self):
        return self.last_packet_data

local_input_manager.py

class INPUT_PEN():
    def __init__(self):
        pass

    def setup(self):
        pass

    def update(self, data):
        print("[PEN] " + str(data))

class INPUT_TOUCH():
    def __init__(self):
        pass

    def setup(self):
        pass

    def update(self, data):
        print("[TOUCH] " + str(data))

main.py

from input_manager import INPUT_EVENT_MANAGER
from ssh_manager import SSH_MANAGER
from local_input_manager import INPUT_PEN, INPUT_TOUCH
from screeninfo import get_monitors

ssh_ip = "10.11.99.1" # Default to USB ip
ssh_port = 22
ssh_username = "root"
ssh_password = "password" # Update with your password

remarkable_pen_event = "/dev/input/event1"
remarkable_touch_event = "/dev/input/event2"

def main():
    ssh_client = connect_ssh()

    pen_manager = INPUT_EVENT_MANAGER(ssh_client)
    touch_manager = INPUT_EVENT_MANAGER(ssh_client)

    pen_manager.run(remarkable_pen_event)
    touch_manager.run(remarkable_touch_event)

    host_pen_manager = INPUT_PEN()
    host_touch_manager = INPUT_TOUCH()
    host_pen_manager.setup()
    host_touch_manager.setup()

    try:
        while True:
            if pen_manager.has_new_data():
                host_pen_manager.update(pen_manager.get_last_data())

            if touch_manager.has_new_data():
                host_touch_manager.update(touch_manager.get_last_data())
            
    except KeyboardInterrupt:
        pen_manager.stop()
        touch_manager.stop()
        
def connect_ssh():
    print("[SSH] connecting...")
    ssh_manager = SSH_MANAGER(ssh_ip, ssh_port, ssh_username, ssh_password)
    ssh_manager.connect()

    if not ssh_manager.is_connected():
        print("[SSH] failed to connect!")
        exit(0)
    
    print("[SSH] connected!")
    return ssh_manager.get_client()

if __name__ == "__main__":
    main()

Executing the new code will log touch & pen events from the reMarkable to the console.

Both /dev/input/event1 (pen input) and /dev/input/event2 (touch input) on the reMarkable tablet are running inside of their own thread.

$ python main.py 
[SSH] connecting...
[SSH] connected!
[PEN] {'TIME': datetime.datetime(2022, 1, 16, 16, 57, 46, 310436), 'BTN_TOOL_PEN': 1, 'ABS_X': 615, 'ABS_Y': 12537, 'ABS_DISTANCE': 0, 'ABS_TILT_X': 4294961396, 'ABS_TILT_Y': 1000, 'SYN_REPORT': 0, 'BTN_TOUCH': 1, 'ABS_PRESSURE': 163}
[PEN] {'TIME': datetime.datetime(2022, 1, 16, 16, 57, 46, 312280), 'BTN_TOOL_PEN': 1, 'ABS_X': 630, 'ABS_Y': 12523, 'ABS_DISTANCE': 0, 'ABS_TILT_X': 4294961196, 'ABS_TILT_Y': 1000, 'SYN_REPORT': 0, 'BTN_TOUCH': 1, 'ABS_PRESSURE': 3}
[PEN] {'TIME': datetime.datetime(2022, 1, 16, 16, 57, 46, 316092), 'BTN_TOOL_PEN': 1, 'ABS_X': 653, 'ABS_Y': 12494, 'ABS_DISTANCE': 0, 'ABS_TILT_X': 4294961196, 'ABS_TILT_Y': 1000, 'SYN_REPORT': 0, 'BTN_TOUCH': 0, 'ABS_PRESSURE': 0}
[TOUCH] {'TIME': datetime.datetime(2022, 1, 16, 16, 57, 49, 727396), 'ABS_MT_TRACKING_ID': 7306, 'ABS_MT_POSITION_X': 1403, 'ABS_MT_POSITION_Y': 897, 'ABS_MT_PRESSURE': 23, 'ABS_MT_ORIENTATION': 2, 'SYN_REPORT': 0, 'ABS_MT_TOUCH_MAJOR': 17, 'ABS_MT_TOUCH_MINOR': 8}
[TOUCH] {'TIME': datetime.datetime(2022, 1, 16, 16, 57, 49, 739364), 'ABS_MT_TRACKING_ID': 7306, 'ABS_MT_POSITION_X': 1403, 'ABS_MT_POSITION_Y': 901, 'ABS_MT_PRESSURE': 20, 'ABS_MT_ORIENTATION': 2, 'SYN_REPORT': 0, 'ABS_MT_TOUCH_MAJOR': 17, 'ABS_MT_TOUCH_MINOR': 8}
[TOUCH] {'TIME': datetime.datetime(2022, 1, 16, 16, 57, 49, 775144), 'ABS_MT_TRACKING_ID': 7306, 'ABS_MT_POSITION_X': 1403, 'ABS_MT_POSITION_Y': 916, 'ABS_MT_PRESSURE': 14, 'ABS_MT_ORIENTATION': 1, 'SYN_REPORT': 0, 'ABS_MT_TOUCH_MAJOR': 8, 'ABS_MT_TOUCH_MINOR': 8}

After running the code, the logged data is slowly expanding as more input data is received, this must be taken into account for the next step (step 7), since the required variables may not yet be received.

Step 7 | Create new uinput device on the host PC and feed the reMarkable event data

This step focuses on the file local_input_manager.py since all other groundwork has been done in the previous step.

local_input_manager.py consists of two important functions: setup() and update(data). The setup function is called before a single update call can be made and is meant to set up a controllable input device on the host. While the update function uses the input device that has been created in the setup function to stream data into.

TODO