Exploring the reMarkable pen input
Table of Contents
This blog post describes input handling from the reMarkable 2 on a remote PC using Python and SSH.
Everything you do with your reMarkable tablet outside of its intended use is at your own risk, neither reMarkable nor I can be held liable if you brick your device.
Happy hacking :)
About reMarkable
“Replace your notebooks and printed documents with the only tablet that feels like paper. reMarkable bridges the gap between pen and paper, and your digital devices. Easily access your notes from your laptop or phone, where they’re always readily available to review and reuse. “ - remarkable.com
After receiving the ‘reMarkable 2’ tablet, I was interested in what makes it tick. A quick online search shows it is running Linux!
The following sources were of great help to get more insights in the reMarkable OS:
Getting into the device
While browsing on the tablet itself something caught my eye,
under Menu > Settings > Help > Copyrights and licenses
I was greeted with a “GPLv3 Compliance” note which states the following:
GPLv3 Compliance
The General Public License version 3 and the Lesser General Public License version 3 also requires you as and end-user to be able to access your device to be able to modify the copyrighted software licenced under these licenses running on it.
To do so, this device acts as an USB ethernet device, and you can connect using the SSH protocol using the username
root
and the password `«REDACTED»``.
This feels like a dream come true, how easy could have this have been, just run SSH and you’re in!
When the reMarkable tablet is connected using USB to a computer it acts as a network interface and is accessible using the IP 10.11.99.1
, meanwhile, the tablet is also accessible wirelessly when it is connected to your local WiFi network.
SSH on different platforms
Linux
open applications, search for term, enter
ssh [email protected]
type the password
MacOS
open Spotlight (Cmd-Space) type Terminal, enter
ssh [email protected]
type the password
Windows
You do you, figure it out yourself
Welcome to: ZERO SUGAR
$ ssh [email protected]
[email protected]'s password:
reMarkable
╺━┓┏━╸┏━┓┏━┓ ┏━┓╻ ╻┏━╸┏━┓┏━┓
┏━┛┣╸ ┣┳┛┃ ┃ ┗━┓┃ ┃┃╺┓┣━┫┣┳┛
┗━╸┗━╸╹┗╸┗━┛ ┗━┛┗━┛┗━┛╹ ╹╹┗╸
reMarkable: ~/ uname -a
Linux reMarkable 5.4.70-v1.1.5-rm11x #1 SMP PREEMPT Fri Nov 12 14:59:18 UTC 2021 armv7l GNU/Linux
reMarkable: ~/
And now the fun can begin :)
The idea
Both reMarkable 1 and 2 have great touch and pen interfaces that feel just right. This made my drawing tablet lose its purpose, but the reMarkable tablet can’t fully replace a drawing tablet connected to a general-purpose computer. Or can it…
As mentioned before, the reMarkable is powered by Linux and the end-user has root-level access using SSH. Meaning I have the full reMarkable Linux OS right at my fingertips. Then I had the idea
“What if you could use the pen input of the reMarkable tablet as mouse input (or even pen input) for any PC."
Let’s focus on pen first and touch later. Firstly I want the get the remarkable to control the mouse cursor on a different computer. Later that can be changed to a pen device on a remote computer and touch can be used as a mouse.
Steps
- Find out how touch & pen input is handled on this device
- Access raw touch & pen data over SSH using Python
- Parse raw data on a remote computer
- Parsing pen input data
- Parsing touch input data
- Improving the code
- Create new uinput device on the host PC and feed the reMarkable event data
Step 1 | Find out how touch & pen input is handled on this device
The first place I look at is /dev/input
since that is a common place in Linux for input events.
reMarkable: ~/ ls -l /dev/input
drwxr-xr-x 2 root root 100 Jan 11 10:50 by-path
crw-rw---- 1 root input 13, 64 Jan 11 10:50 eve6nt2
lrwxrwxrwx 1 root root 6 Jan 11 10:50 touchscreen0 -> event1
And it looks like it’s indeed here, but that’s quite easy to test out by watching cat /dev/input/eventX
or hexdump /dev/input/eventX
/dev/input/event0
Gives no output on touch or pen, so let’s skip that
reMarkable: ~/ hexdump /dev/input/event0
^C
reMarkable: ~/
/dev/input/event1
This shows data when the pen is getting near, buth the tip and the eraser is being picked up
reMarkable: ~/ hexdump /dev/input/event1
0000000 32c8 61e3 2cb3 0009 0001 0140 0001 0000
0000010 32c8 61e3 2cb3 0009 0003 0000 1a01 0000
0000020 32c8 61e3 2cb3 0009 0003 0001 2ed8 0000
0000030 32c8 61e3 2cb3 0009 0003 0019 0037 0000
0000040 32c8 61e3 2cb3 0009 0003 001a fa24 ffff
0000050 32c8 61e3 2cb3 0009 0003 001b 0b54 0000
0000060 32c8 61e3 2cb3 0009 0000 0000 0000 0000
0000070 32c8 61e3 33b4 0009 0003 0000 1a02 0000
0000080 32c8 61e3 33b4 0009 0003 0019 0035 0000
0000090 32c8 61e3 33b4 0009 0000 0000 0000 0000^C
reMarkable: ~/
/dev/input/event2
Touch data can be found here
reMarkable: ~/ hexdump /dev/input/event2
0000000 32aa 61e3 f25f 0000 0003 0039 1b9d 0000
0000010 32aa 61e3 f25f 0000 0003 0035 039d 0000
0000020 32aa 61e3 f25f 0000 0003 0036 02e5 0000
0000030 32aa 61e3 f25f 0000 0003 003a 0053 0000
0000040 32aa 61e3 f25f 0000 0003 0030 0008 0000
0000050 32aa 61e3 f25f 0000 0003 0034 0001 0000
0000060 32aa 61e3 f25f 0000 0000 0000 0000 0000
0000070 32aa 61e3 e667 0001 0003 0039 ffff ffff
0000080 32aa 61e3 e667 0001 0000 0000 0000 0000^c
reMarkable: ~/
The above findings are also validated on this Github issue
Of course, this data is not readable as text, but it is usable in Python, the language I use for this experiment.
Now that is known what input event is for touch & pen, let’s move on to the next step where a connection to the tablet will be made using Python and SSH.
Step 2 | Access raw touch & pen data over SSH using Python
In this example I’m using Python 3.9.7
Preparing the Python development environment
To connect to the reMarkable device using Python, a library called paramiko is used, but there are more alternatives available if that is desired.
Later to let the reMarkable tablet control a remote mouse cursor it is necessary to know the reMarkable’s screen size as well as the screen size of the host pc. The reMarkable has a screen resolution of 1872x1404 NOTE: the default orientation of the remarkable isn’t portrait as it’s being used as, but rather landscape with then pen on the bottom To get the host’s screen size a library called screeninfo is used. To control the mouse cursor pyautogui is used. These are the Python libraries being used for this step.
Let’s place the required dependencies into a file:
requirements.txt
paramiko
screeninfo
pyautogui
pip install -r requirements.txt
Before I dive into the code I want one thing out of the way:
- By no means my code is the “right” way to do this;
- This is merely an example/proof of concept to show that it works; Let’s move on now to the fun stuff :3
Connecting using SSH
main.py
from datetime import datetime
from enum import Enum
import paramiko
from paramiko import SSHClient
ssh_ip = "10.11.99.1" # Default to USB ip
ssh_port = 22
ssh_username = "root"
ssh_password = "password" # Update with your password
def main():
print("Attempting to connect...")
ssh_client = SSHClient()
try:
# Since this is an test environment let paramiko automatically set the missing host key
ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh_client.connect(
hostname=ssh_ip,
port=ssh_port,
username=ssh_username,
password=ssh_password,
)
except: # Catch connection error
print("Connect failed")
exit(0)
print("Connect successfully")
if __name__ == "__main__":
main()
Running the code above should result in the following output:
$ python main.py
Attempting to connect...
Connect successfully
$
Or the following if the device is not accessible:
$ python main.py
Attempting to connect...
Connect failed
$
Reading /dev/input/eventX
This connects to the tablet and exits immediately after, which isn’t what we need. Time to move on to reading /dev/input/event1
(pen input) using ssh
First, make some changes to the top of the main file:
main.py
...
from paramiko import SSHClient
linux_event = "/dev/input/event1"
ssh_ip = "10.11.99.1" # Default to USB ip
...
And after connecting, it calls a function to read the input event:
main.py
...
print("Connect successfully")
handle_input(ssh_client)
def handle_input(ssh_client):
ssh_transport = ssh_client.get_transport()
ssh_channel = ssh_transport.open_session()
ssh_channel.exec_command(f"cat {linux_event}")
try:
while ssh_transport.is_active():
packet_size = 32
data = ssh_channel.recv(packet_size).hex()
for packet in [data[i : i + packet_size] for i in range(0, len(data), packet_size)]: # Sometimes a packet is too large
print(packet)
except KeyboardInterrupt:
print("Interrupted")
ssh_client.close()
exit(0)
if __name__ == "__main__":
...
After executing this code, it will show the raw event data in hex format:
$ python main.py
Attempting to connect...
Connect successfully
3646e361725703000100410101000000
3646e3617257030003000000c9290000
3646e361725703000300010088210000
3646e361725703000300190021000000
3646e3617257030003001a004cebffff
3646e3617257030003001b00a8fdffff
3646e361725703000000000000000000
3646e3615f5e030003000000c8290000
3646...
Let’s explore what this actually means and why packet_size
is 32
Using the Linux command cat
the input event can be read as streaming output.
Every time we request 32 bytes from the running cat
command, hence packet_size
, then in Python we use .hex()
on the data that has been received to convert it to HEX so it can be used more easily.
In the next step (step 3) we’ll explore the data in more detail.
Step 3 | Parse raw data on a remote computer
To get a better understanding, let’s explore the following received packet 3646e361 72570300 0300 0100 88210000
Note: This is LSB (Least Significant Bit first), so every received message part* needs to be reverse
*message part’s are as follows:
Type | MSB | LSB | DEC |
---|---|---|---|
seconds | 3646e361 | 61e34636 | 1642284598 |
useconds | 72570300 | 00035772 | 218994 |
type | 0300 | 0003 | 3 |
code | 0100 | 0001 | 1 |
value | 88210000 | 00002188 | 8584 |
The seconds and useconds can be used to construct date & time information, in this case: 15-01-2022 22:09:58.218994
(DD-MM-YYYY HH:MM:SS.SSSSSS)
Now we have three unknown variables type
, code
, and value
.
While value
is quite self-explanatory, let’s focus on the other two.
After consulting kernel.org and linux/input-event-codes.h we can see what type
and code
mean.
type:
- EV_SYN: 0x00
- EV_KEY: 0x01
- EV_REL: 0x02
- EV_ABS: 0x03
- EV_…
- Full TYPE info
code:
depends on type
- EV_…
- EV_KEY: 0x01
- KEY_ESC: 0x01
- KEY_1: 0x02
- BTN_TOOL_PEN: 0x140
- BTN_TOOL_RUBBER: 0x141
- KEY_…
- Full KEY_ info
- EV_REL: 0x02
- REL_X: 0x00
- REL_Y: 0x01
- REL_…
- Full REL_ info
- EV_ABS: 0x03
- ABS_X: 0x00
- ABS_Y: 0x01
- ABS_…
- Full ABS_ info
- EV_…
- Full CODE info
Getting back at the message we were exploring we now know that type
3
means EV_ABS
(absolute position) and code
1
means ABS_Y
(absolute y position), so the pen was at position 8584
(value
) on the y-axis.
Let’s move on to the next step (step 4) where the data will be used to move the mouse cursor
Step 4 | Parsing pen input data
Now that event data is received in Python, it can be parsed and used to, for example, move the mouse cursor.
The following type
’s and code
’s will be used:
- EV_KEY | To indicate which tool is used
- BTN_TOOL_PEN
- BTN_TOOL_RUBBER
- EV_ABS | To get the absolute tool values
- ABS_X | Absolute X position as reported by the Wacom driver
- ABS_Y | Absolute Y position on the screen, ranging from 0 to
- ABS_TILT_X | Absolute tilt X of the pen
- ABS_TILT_Y | Absolute tilt Y of the pen
- ABS_DISTANCE | Absolute distance the pen tip is from the screen
Let’s add some code to call a function with the HEX data:
main.py
...
for packet in [data[i : i + packet_size] for i in range(0, len(data), packet_size)]: # Sometimes a packet is too large
date_time, pen_down, eraser_down, posx, posy, tiltx, tilty, distance = parse_packet(packet)
print(str(date_time) + " - " + str(pen_down) + " - " + str(eraser_down) + " - " + str(posx) + " - " + str(posy) + " - " + str(tiltx) + " - " + str(tilty) + " - " + str(distance))
except KeyboardInterrupt:
...
Now it’s time to create a function in our Python code that parses the HEX data to usable variables:
main.py
# Store the last know values to send with the new data
last_date_time = None
last_pen_down = False
last_eraser_down = False
last_posx = 0
last_posy = 0
last_tiltx = 0
last_tilty = 0
last_distance = 0
def parse_packet(packet):
global last_date_time
global last_pen_down
global last_eraser_down
global last_posx
global last_posy
global last_tiltx
global last_tilty
global last_distance
raw_data = [packet[j : j + 2] for j in range(0, len(packet), 2)] # Split the raw packet every 2 characters
if len(raw_data) == 16: # If the packet is complete
sec = int("".join(raw_data[0:4][::-1]), base=16)
usec = int("".join(raw_data[4:8][::-1]), base=16)
type = int("".join(raw_data[8:10][::-1]), base=16)
code = int("".join(raw_data[10:12][::-1]), base=16)
value = int("".join(raw_data[12:][::-1]), base=16)
last_date_time = datetime.fromtimestamp(sec + usec / 1e6) # Convert received sec & usec to Python datetime object
evtype = EVTYPE(type).name
evcode = get_code(evtype, code)
if evtype == "EV_ABS":
if evcode == "ABS_X":
last_posx = value
if evcode == "ABS_Y":
last_posy = value
if evcode == "ABS_TILT_X":
last_tiltx = value
if evcode == "ABS_TILT_Y":
last_tilty = value
if evcode == "ABS_DISTANCE":
last_distance = value
if evtype == "EV_KEY":
if evcode == "BTN_TOOL_PEN":
last_pen_down = True if value == 1 else False
if evcode == "BTN_TOOL_RUBBER":
last_eraser_down = True if value == 1 else False
return (last_date_time, last_pen_down, last_eraser_down, last_posx, last_posy, last_tiltx, last_tilty, last_distance)
def get_code(evtype, raw_code):
evcode = ""
if evtype == "EV_SYN":
evcode = EVMAPSYN(raw_code).name
if evtype == "EV_KEY":
evcode = EVMAPKEY(raw_code).name
if evtype == "EV_ABS":
evcode = EVMAPABS(raw_code).name
return evcode
class EVTYPE(Enum):
EV_SYN = 0x00
EV_KEY = 0x01
EV_REL = 0x02
EV_ABS = 0x03
EV_MSC = 0x04
EV_SW = 0x05
EV_LED = 0x11
EV_SND = 0x12
EV_REP = 0x14
EV_FF = 0x15
EV_PWR = 0x16
EV_FF_STATUS = 0x17
class EVMAPKEY(Enum):
BTN_TOOL_PEN = 0x140
BTN_TOOL_RUBBER = 0x141
BTN_TOUCH = 0x14A
class EVMAPSYN(Enum):
SYN_REPORT = 0x00
SYN_CONFIG = 0x01
SYN_MT_REPORT = 0x02
SYN_DROPPED = 0x03
class EVMAPABS(Enum):
ABS_X = 0x00
ABS_Y = 0x01
ABS_PRESSURE = 0x18
ABS_DISTANCE = 0x19
ABS_TILT_X = 0x1A
ABS_TILT_Y = 0x1B
Executing this new code shows the following output:
$ python main.py
Attempting to connect...
Connect successfully
2022-01-16 01:07:52.237602 - True - False - 0 - 0 - 0 - 0 - 0
2022-01-16 01:07:52.237602 - True - False - 5336 - 0 - 0 - 0 - 0
2022-01-16 01:07:52.237602 - True - False - 5336 - 10383 - 0 - 0 - 0
2022-01-16 01:07:52.237602 - True - False - 5336 - 10383 - 0 - 0 - 22
2022-01-16 01:07:52.237602 - True - False - 5336 - 10383 - 900 - 0 - 22
2022-01-16 01:07:52.239384 - True - False - 5339 - 10383 - 900 - 200 - 22
2022-01-16 01:07:52.241211 - True - False - 5347 - 10381 - 900 - 200 - 19
2022-01-16 01:07:52.241211 - True - False - 5347 - 10381 - 900 - 200 - 17
2022-01-16 01:07:52.254145 - True - False - 5411 - 10387 - 900 - 200 - 7
2022-01-16 01:07:52.254145 - True - False - 5411 - 10390 - 900 - 200 - 7
2022-01-16 01:07:52.254145 - True - False - 5411 - 10390 - 900 - 200 - 8
2022-01-16 01:07:52.328054 - True - False - 5355 - 10284 - 4294965496 - 200 - 0
2022-01-16 01:07:52.330034 - True - False - 5361 - 10291 - 4294965496 - 200 - 12
2022-01-16 01:07:52.372098 - True - False - 5609 - 10466 - 4294965696 - 300 - 81
2022-01-16 01:07:52.373945 - False - False - 5609 - 10479 - 4294965696 - 300 - 81
This output shows every received aspect from touching the pen onto the reMarkable screen. Note: tilt X and tilt Y can be quite unreliable and sometimes reports big numbers.
Step 5 | Parsing touch input data
Inside of the code change from the pen input event (/dev/input/event1
) to the touch input event (/dev/input/event2
)
main.py
...
# linux_event = "/dev/input/event1"
linux_event = "/dev/input/event2"
...
# inside of def parse_packet(packet):
...
evcode = get_code(evtype, code)
print(evcode) # Print raw received EV codes
if evtype == "EV_ABS":
...
After executing the modified code above we can see that the previously generated pen data no longer works and that it prints different event codes:
$ python main.py
ABS_MT_SLOT
2022-01-16 12:18:16.564759 - False - False - 0 - 0 - 0 - 0 - 0
ABS_MT_POSITION_X
2022-01-16 12:18:16.564759 - False - False - 0 - 0 - 0 - 0 - 0
ABS_MT_PRESSURE
2022-01-16 12:18:16.564759 - False - False - 0 - 0 - 0 - 0 - 0
ABS_MT_TOUCH_MINOR
2022-01-16 12:18:16.564759 - False - False - 0 - 0 - 0 - 0 - 0
ABS_MT_ORIENTATION
2022-01-16 12:18:16.564759 - False - False - 0 - 0 - 0 - 0 - 0
ABS_MT_SLOT
2022-01-16 12:18:16.564759 - False - False - 0 - 0 - 0 - 0 - 0
ABS_MT_POSITION_X
2022-01-16 12:18:16.564759 - False - False - 0 - 0 - 0 - 0 - 0
ABS_MT_POSITION_Y
2022-01-16 12:18:16.564759 - False - False - 0 - 0 - 0 - 0 - 0
ABS_MT_PRESSURE
2022-01-16 12:18:16.564759 - False - False - 0 - 0 - 0 - 0 - 0^C
$
Let’s write down every detected event code (also for multi-touch)
ABS_MT_SLOT
ABS_MT_TRACKING_ID
ABS_MT_POSITION_X
ABS_MT_POSITION_Y
ABS_MT_PRESSURE
ABS_MT_ORIENTATION
ABS_MT_TOUCH_MAJOR
ABS_MT_TOUCH_MINOR
Knowing what event code it sends out, modify the parse_packet(packet)
functions as follows:
# Store the last know values to send with the new data
last_date_time = None
last_slot = 0
last_tracking_id = 0
last_posx = 0
last_posy = 0
last_pressure = 0
last_orientation = 0
last_touch_major = 0
last_touch_minor = 0
def parse_packet(packet):
global last_date_time
global last_slot
global last_tracking_id
global last_posx
global last_posy
global last_pressure
global last_orientation
global last_touch_major
global last_touch_minor
raw_data = [packet[j : j + 2] for j in range(0, len(packet), 2)] # Split the raw packet every 2 characters
if len(raw_data) == 16: # If the packet is complete
sec = int("".join(raw_data[0:4][::-1]), base=16)
usec = int("".join(raw_data[4:8][::-1]), base=16)
type = int("".join(raw_data[8:10][::-1]), base=16)
code = int("".join(raw_data[10:12][::-1]), base=16)
value = int("".join(raw_data[12:][::-1]), base=16)
last_date_time = datetime.fromtimestamp(sec + usec / 1e6) # Convert received sec & usec to Python datetime object
evtype = EVTYPE(type).name
evcode = get_code(evtype, code)
if evtype == "EV_ABS":
if evcode == "ABS_MT_SLOT":
last_slot = value
if evcode == "ABS_MT_TRACKING_ID":
last_tracking_id = value
if evcode == "ABS_MT_POSITION_X":
last_posx = value
if evcode == "ABS_MT_POSITION_Y":
last_posy = value
if evcode == "ABS_MT_PRESSURE":
last_pressure = value
if evcode == "ABS_MT_ORIENTATION":
last_orientation = value
if evcode == "ABS_MT_TOUCH_MAJOR":
last_touch_major = value
if evcode == "ABS_MT_TOUCH_MINOR":
last_touch_minor = value
return (last_date_time, last_slot, last_tracking_id, last_posx, last_posy, last_pressure, last_orientation, last_touch_major, last_touch_minor)
Also change the function call to account for the new data
...
for packet in [data[i : i + packet_size] for i in range(0, len(data), packet_size)]: # Sometimes a packet is too large
date_time, slot, tracking_id, posx, posy, pressure, orientation, touch_major, touch_minor = parse_packet(packet)
print(str(date_time) + " - " + str(slot) + " - " + str(tracking_id) + " - " + str(posx) + " - " + str(posy) + " - " + str(pressure) + " - " + str(orientation) + " - " + str(touch_major) + " - " + str(touch_minor))
except KeyboardInterrupt:
...
Now, after executing the modified code, we can look at the results:
$ python main.py
2022-01-16 12:32:56.843194 - 0 - 7189 - 546 - 800 - 98 - 3 - 17 - 17
2022-01-16 12:32:56.843194 - 0 - 7189 - 546 - 800 - 98 - 3 - 17 - 17
2022-01-16 12:32:56.854692 - 0 - 7189 - 546 - 800 - 97 - 3 - 17 - 17
2022-01-16 12:32:56.854692 - 1 - 7189 - 546 - 800 - 97 - 3 - 17 - 17
2022-01-16 12:32:56.854692 - 1 - 7189 - 546 - 800 - 103 - 3 - 17 - 17
2022-01-16 12:32:56.854692 - 1 - 7189 - 546 - 800 - 103 - 3 - 17 - 17
2022-01-16 12:32:56.866617 - 0 - 7189 - 546 - 800 - 103 - 3 - 17 - 17^c
$
Event | Description |
---|---|
ABS_MT_SLOT | What finger ID the data is from (first touching finger = 0, second = 1, etc…) |
ABS_MT_TRACKING_ID | Unique ID of initiated touch contact |
ABS_MT_POSITION_X | The center X position ranging from 0 to screen width where the touch happened |
ABS_MT_POSITION_Y | The center Y position ranging from 0 to screen height where the touch happened |
ABS_MT_PRESSURE | Pressure on contact area |
ABS_MT_ORIENTATION | Ellipse orientation |
ABS_MT_TOUCH_MAJOR | Major axis of touching ellipse |
ABS_MT_TOUCH_MINOR | Minor axis (omit if circular) |
Step 6 | Improving the code
Before we continue with our journey and control the mouse cursor with the received data. It is important to clean the current code first and split things up into multiple modules and add threads so we’re not limited to reading a single event at the time.
The file structure is as follows:
project
│ main.py
│ input_event_codes.py
| input_manager.py
| local_input_manager.py
| ssh_manager.py
| utils.py
The input_event_codes.py
file is just a conversion from linux/input-event-codes.h to python with some redundant input event codes removed.
input_event_codes.py
from enum import Enum
def get_code(evtype, raw_code):
evcode = ""
if evtype == "EV_SYN":
evcode = EVCODESYN(raw_code).name
if evtype == "EV_KEY":
evcode = EVCODEKEY(raw_code).name
if evtype == "EV_ABS":
evcode = EVCODEABS(raw_code).name
return evcode
# Event types
class EVTYPE(Enum):
EV_SYN = 0x00
EV_KEY = 0x01
EV_REL = 0x02
EV_ABS = 0x03
EV_MSC = 0x04
EV_SW = 0x05
EV_LED = 0x11
EV_SND = 0x12
EV_REP = 0x14
EV_FF = 0x15
EV_PWR = 0x16
EV_FF_STATUS = 0x17
EV_MAX = 0x1f
# Synchronization events
class EVCODESYN(Enum):
SYN_REPORT = 0x00
SYN_CONFIG = 0x01
SYN_MT_REPORT = 0x02
SYN_DROPPED = 0x03
SYN_MAX = 0x0f
# Key and button events
class EVCODEKEY(Enum):
BTN_TOOL_PEN = 0x140
BTN_TOOL_RUBBER = 0x141
BTN_TOUCH = 0x14a
KEY_MAX = 0x2ff
# Absolute events
class EVCODEABS(Enum):
ABS_X = 0x00
ABS_Y = 0x01
ABS_PRESSURE = 0x18
ABS_DISTANCE = 0x19
ABS_TILT_X = 0x1a
ABS_TILT_Y = 0x1b
ABS_TOOL_WIDTH = 0x1c
ABS_VOLUME = 0x20
ABS_MISC = 0x28
ABS_MT_SLOT = 0x2f # MT slot being modified
ABS_MT_TOUCH_MAJOR = 0x30 # Major axis of touching ellipse
ABS_MT_TOUCH_MINOR = 0x31 # Minor axis (omit if circular)
ABS_MT_WIDTH_MAJOR = 0x32 # Major axis of approaching ellipse
ABS_MT_WIDTH_MINOR = 0x33 # Minor axis (omit if circular)
ABS_MT_ORIENTATION = 0x34 # Ellipse orientation
ABS_MT_POSITION_X = 0x35 # Center X touch position
ABS_MT_POSITION_Y = 0x36 # Center Y touch position
ABS_MT_TOOL_TYPE = 0x37 # Type of touching device
ABS_MT_BLOB_ID = 0x38 # Group a set of packets as a blob
ABS_MT_TRACKING_ID = 0x39 # Unique ID of initiated contact
ABS_MT_PRESSURE = 0x3a # Pressure on contact area
ABS_MT_DISTANCE = 0x3b # Contact hover distance
ABS_MT_TOOL_X = 0x3c # Center X tool position
ABS_MT_TOOL_Y = 0x3d # Center Y tool position
ABS_MAX = 0x3f
utils.py
def map(x, in_min, in_max, out_min, out_max):
return (x - in_min) * (out_max - out_min) / (in_max - in_min) + out_min
ssh_manager.py
import paramiko
from paramiko import SSHClient
class SSH_MANAGER():
def __init__(self, host, port, username, password):
self.host = host
self.port = port
self.username = username
self.password = password
self.ssh_client = SSHClient()
self.is_client_connected = False
def connect(self):
try:
# Since this is an test environment let paramiko automatically set the missing host key
self.ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
self.ssh_client.connect(
hostname=self.host,
port=self.port,
username=self.username,
password=self.password,
)
except: # Catch connection error
self.is_client_connected = False
return
self.is_client_connected = True
def disconnect(self):
if self.ssh_client is not None:
self.ssh_client.close()
self.is_client_connected = False
def is_connected(self):
return self.is_client_connected
def get_client(self):
return self.ssh_client
input_manager.py
import threading
from datetime import datetime
import input_event_codes
class INPUT_EVENT_MANAGER():
def __init__(self, ssh_client):
self.ssh_client = ssh_client
self.last_packet_data = {}
self.new_data_available = False
def run(self, linux_event):
self.thread = threading.Thread(target=self._start, args=(linux_event,))
self.thread.start()
def stop(self):
if self.thread is not None:
self.thread.join()
self.thread = None
def _start(self, linux_event):
ssh_transport = self.ssh_client.get_transport()
ssh_channel = ssh_transport.open_session()
ssh_channel.exec_command(f"cat {linux_event}")
while ssh_transport.is_active():
packet_size = 32
data = ssh_channel.recv(packet_size).hex()
for packet in [data[i: i + packet_size] for i in range(0, len(data), packet_size)]: # Sometimes a packet is too large
self.parse_raw_packet(packet)
def parse_raw_packet(self, packet):
raw_data = [packet[j : j + 2] for j in range(0, len(packet), 2)] # Split the raw packet every 2 characters
if len(raw_data) == 16: # If the packet is complete
sec = int("".join(raw_data[0:4][::-1]), base=16)
usec = int("".join(raw_data[4:8][::-1]), base=16)
type = int("".join(raw_data[8:10][::-1]), base=16)
code = int("".join(raw_data[10:12][::-1]), base=16)
value = int("".join(raw_data[12:][::-1]), base=16)
date_time = datetime.fromtimestamp(sec + usec / 1e6) # Convert received sec & usec to Python datetime object
evtype = input_event_codes.EVTYPE(type).name
evcode = input_event_codes.get_code(evtype, code)
self.last_packet_data["TIME"] = date_time
self.last_packet_data[evcode] = value
self.new_data_available = True
def has_new_data(self):
new_data_available = self.new_data_available
self.new_data_available = False
return new_data_available
def get_last_data(self):
return self.last_packet_data
local_input_manager.py
class INPUT_PEN():
def __init__(self):
pass
def setup(self):
pass
def update(self, data):
print("[PEN] " + str(data))
class INPUT_TOUCH():
def __init__(self):
pass
def setup(self):
pass
def update(self, data):
print("[TOUCH] " + str(data))
main.py
from input_manager import INPUT_EVENT_MANAGER
from ssh_manager import SSH_MANAGER
from local_input_manager import INPUT_PEN, INPUT_TOUCH
from screeninfo import get_monitors
ssh_ip = "10.11.99.1" # Default to USB ip
ssh_port = 22
ssh_username = "root"
ssh_password = "password" # Update with your password
remarkable_pen_event = "/dev/input/event1"
remarkable_touch_event = "/dev/input/event2"
def main():
ssh_client = connect_ssh()
pen_manager = INPUT_EVENT_MANAGER(ssh_client)
touch_manager = INPUT_EVENT_MANAGER(ssh_client)
pen_manager.run(remarkable_pen_event)
touch_manager.run(remarkable_touch_event)
host_pen_manager = INPUT_PEN()
host_touch_manager = INPUT_TOUCH()
host_pen_manager.setup()
host_touch_manager.setup()
try:
while True:
if pen_manager.has_new_data():
host_pen_manager.update(pen_manager.get_last_data())
if touch_manager.has_new_data():
host_touch_manager.update(touch_manager.get_last_data())
except KeyboardInterrupt:
pen_manager.stop()
touch_manager.stop()
def connect_ssh():
print("[SSH] connecting...")
ssh_manager = SSH_MANAGER(ssh_ip, ssh_port, ssh_username, ssh_password)
ssh_manager.connect()
if not ssh_manager.is_connected():
print("[SSH] failed to connect!")
exit(0)
print("[SSH] connected!")
return ssh_manager.get_client()
if __name__ == "__main__":
main()
Executing the new code will log touch & pen events from the reMarkable to the console.
Both /dev/input/event1
(pen input) and /dev/input/event2
(touch input) on the reMarkable tablet are running inside of their own thread.
$ python main.py
[SSH] connecting...
[SSH] connected!
[PEN] {'TIME': datetime.datetime(2022, 1, 16, 16, 57, 46, 310436), 'BTN_TOOL_PEN': 1, 'ABS_X': 615, 'ABS_Y': 12537, 'ABS_DISTANCE': 0, 'ABS_TILT_X': 4294961396, 'ABS_TILT_Y': 1000, 'SYN_REPORT': 0, 'BTN_TOUCH': 1, 'ABS_PRESSURE': 163}
[PEN] {'TIME': datetime.datetime(2022, 1, 16, 16, 57, 46, 312280), 'BTN_TOOL_PEN': 1, 'ABS_X': 630, 'ABS_Y': 12523, 'ABS_DISTANCE': 0, 'ABS_TILT_X': 4294961196, 'ABS_TILT_Y': 1000, 'SYN_REPORT': 0, 'BTN_TOUCH': 1, 'ABS_PRESSURE': 3}
[PEN] {'TIME': datetime.datetime(2022, 1, 16, 16, 57, 46, 316092), 'BTN_TOOL_PEN': 1, 'ABS_X': 653, 'ABS_Y': 12494, 'ABS_DISTANCE': 0, 'ABS_TILT_X': 4294961196, 'ABS_TILT_Y': 1000, 'SYN_REPORT': 0, 'BTN_TOUCH': 0, 'ABS_PRESSURE': 0}
[TOUCH] {'TIME': datetime.datetime(2022, 1, 16, 16, 57, 49, 727396), 'ABS_MT_TRACKING_ID': 7306, 'ABS_MT_POSITION_X': 1403, 'ABS_MT_POSITION_Y': 897, 'ABS_MT_PRESSURE': 23, 'ABS_MT_ORIENTATION': 2, 'SYN_REPORT': 0, 'ABS_MT_TOUCH_MAJOR': 17, 'ABS_MT_TOUCH_MINOR': 8}
[TOUCH] {'TIME': datetime.datetime(2022, 1, 16, 16, 57, 49, 739364), 'ABS_MT_TRACKING_ID': 7306, 'ABS_MT_POSITION_X': 1403, 'ABS_MT_POSITION_Y': 901, 'ABS_MT_PRESSURE': 20, 'ABS_MT_ORIENTATION': 2, 'SYN_REPORT': 0, 'ABS_MT_TOUCH_MAJOR': 17, 'ABS_MT_TOUCH_MINOR': 8}
[TOUCH] {'TIME': datetime.datetime(2022, 1, 16, 16, 57, 49, 775144), 'ABS_MT_TRACKING_ID': 7306, 'ABS_MT_POSITION_X': 1403, 'ABS_MT_POSITION_Y': 916, 'ABS_MT_PRESSURE': 14, 'ABS_MT_ORIENTATION': 1, 'SYN_REPORT': 0, 'ABS_MT_TOUCH_MAJOR': 8, 'ABS_MT_TOUCH_MINOR': 8}
After running the code, the logged data is slowly expanding as more input data is received, this must be taken into account for the next step (step 7), since the required variables may not yet be received.
Step 7 | Create new uinput device on the host PC and feed the reMarkable event data
This step focuses on the file local_input_manager.py
since all other groundwork has been done in the previous step.
local_input_manager.py
consists of two important functions: setup()
and update(data)
. The setup function is called before a single update call can be made and is meant to set up a controllable input device on the host. While the update function uses the input device that has been created in the setup function to stream data into.