Frame SDK for Python
The Frame SDK for Python is available in a low-level library (frame-ble) for Bluetooth LE connectivity using Bleak with a limited Lua REPL, and an application-level library (frame-msg) for passing rich objects between your host program and Frame, such as images, streamed audio, IMU data and rasterized text.
frame-ble package: low-level connectivity
The FrameBle class allows a host to connect to Frame, initiating pairing if required. A sequence of Lua strings can be sent to Frame for execution, optionally returning their results as strings. Transmitted and received strings must be shorter than the Bluetooth MTU (FrameBle.max_lua_payload()).
Frame-specific Lua APIs can be called to draw on the display, sample the accelerometer, put Frame into firmware update mode, set the tap and user data callbacks, and more.
In addition to Lua strings, FrameBle can also send control commands (break and reset), as well as custom user data messages that can be handled by a user-defined data handler function on Frame. See Talking to Frame Over Bluetooth for more details.
Consult the API Reference for the complete list of functions available in frame-ble, and the Examples repo to see them in action. Don’t see an example you need? Ask in Discord to have one added.
Note: It is expected that most application developers will prefer to use FrameMsg from the frame-msg package to make use of first-class messages and efficient transport for sprites, images, audio clips, and to avoid having to wrangle Bluetooth packet fragmentation and assembly. The frame-ble package, with only a dependency on the bleak library, is still a handy choice for "Hello, World!", simple utilities, or as the basis of a custom application framework in Python.
Example usage
pip install frame-ble
import asyncio
from frame_ble import FrameBle
async def main():
frame = FrameBle()
try:
await frame.connect()
await frame.send_lua("frame.display.text('Hello, Frame!', 1, 1);frame.display.show();print(nil)", await_print=True)
await frame.disconnect()
except Exception as e:
print(f"Not connected to Frame: {e}")
return
if __name__ == "__main__":
asyncio.run(main())
frame-msg package: application-level messaging
frame-msg enables both the hostside and Frameside halves of an application to communicate using richly typed messages - for example transmitting Sprites to Frame with width, height, palette data and pixel data attributes. (See the code below for an example.)
Additionally, frame-msg uses a programming model in which a main event loop is running on Frame. As a result, messages are exchanged asychronously and handled in an event-driven way. This is in contrast to the “Lua REPL” model, in which single statements are sent to Frame to be executed, and the host attempts to block until execution has completed. In a REPL-style model, it would not be practical to create an application that for example continuously streams audio from Frame to host, and simultaneously sends text back for display (a live translation app). In contrast, with a main event loop running on Frame it is possible to send some audio back to the host if some is available, and check for messages indicating text is ready for display, then loop around again.
How it works
In order to serialize and deserialize large objects, frame-msg makes use of FrameBle’s send_message() function that fragments large payloads into multiple Bluetooth packets marked with a matching msg_code, and efficiently reassembles them on Frame. After parsing the reconstructed payload into the desired type, a complete object is made available for the application’s main loop in Lua.
Handling rich message types is done by providing symmetrical packing code and parsing code for standard types.
For “Transmit” (
Tx) message types, the packing (serialization) code is in the Python type (e.g.TxSprite.pack()) and the parsing (deserialization) code is in the corresponding Lua module (sprite.lua/parse_sprite()).For “Receive” (
Rx) message types, e.g.RxPhoto, the serialization code is present implicitly in the Lua modulecamera.luaincapture_and_send(), and the deserialization code is present in theRxPhotoclass.
This approach presents a natural extension point for user-defined types too: a custom application-specific message might contain an icon, a heading and some detailed text - and this can all be sent together and used all at once on Frame to update the display. By implementing the packing and parsing code for user-defined types, custom objects can be used in applications in the same way as the standard types.
Startup sequence
frame-msg applications tend to follow a startup sequence that ensures Frame is in a known state for the application to run, namely:
- Optionally send a break/reset/break signal sequence to ensure that Frame is in Lua REPL mode and its memory is freshly cleared
- Upload standard Lua libraries to Frame:
datafor handling all messaging, plus any others to support specific messages the application uses (e.g.sprite,plain_text, etc.) Unless specified otherwise, minified Lua code files are uploaded, so make sure your Frameside app includessprite.minin this case. - Upload the main Frame application Lua file (e.g.
my_frame_app.lua). This can often be an exact copy of an example main Lua file that handles the same message types. - Start the main application loop on Frame (after which no Lua REPL commands will be accepted by Frame).
After the Frameside application reports that it has started, the hostside and Frameside application halves begin messaging each other asynchronously: passing text, images, audio etc.
Example usage
pip install frame-msg
sprite_display.py
import asyncio
from pathlib import Path
from frame_msg import FrameMsg, TxSprite
async def main():
"""
Displays sample images on the Frame display.
The images are indexed (palette) PNG images, in 2, 4,
and 16 colors (that is, 1-, 2- and 4-bits-per-pixel).
"""
frame = FrameMsg()
try:
await frame.connect()
# Let the user know we're starting
await frame.print_short_text('Loading...')
# send the std lua files to Frame that handle data accumulation and sprite parsing
await frame.upload_stdlua_libs(lib_names=['data', 'sprite'])
# Send the main lua application from this project to Frame that will run the app
await frame.upload_frame_app(local_filename="lua/sprite_frame_app.lua")
# attach the print response handler so we can see stdout from Frame Lua print() statements
frame.attach_print_response_handler()
# "require" the main frame_app lua file to run it, and block until it has started.
await frame.start_frame_app()
# send the 1-bit image to Frame in chunks
sprite = TxSprite.from_indexed_png_bytes(Path("images/logo_1bit.png").read_bytes())
await frame.send_message(0x20, sprite.pack())
# send a 2-bit image
sprite = TxSprite.from_indexed_png_bytes(Path("images/street_2bit.png").read_bytes())
await frame.send_message(0x20, sprite.pack())
# send a 4-bit image
sprite = TxSprite.from_indexed_png_bytes(Path("images/hotdog_4bit.png").read_bytes())
await frame.send_message(0x20, sprite.pack())
await asyncio.sleep(5.0)
# unhook the print handler
frame.detach_print_response_handler()
# break out of the frame app loop and reboot Frame
await frame.stop_frame_app()
except Exception as e:
print(f"An error occurred: {e}")
finally:
# clean disconnection
await frame.disconnect()
if __name__ == "__main__":
asyncio.run(main())
sprite_frame_app.lua
local data = require('data.min')
local sprite = require('sprite.min')
-- Phone to Frame flags
USER_SPRITE = 0x20
-- register the message parsers so they are automatically called when matching data comes in
data.parsers[USER_SPRITE] = sprite.parse_sprite
-- Main app loop
function app_loop()
frame.display.text('Frame App Started', 1, 1)
frame.display.show()
-- tell the host program that the frameside app is ready (waiting on await_print)
print('Frame app is running')
while true do
rc, err = pcall(
function()
-- process any raw data items, if ready
local items_ready = data.process_raw_items()
-- one or more full messages received
if items_ready > 0 then
if data.app_data[USER_SPRITE] ~= nil then
local spr = data.app_data[USER_SPRITE]
-- set the palette in case it's different to the standard palette
sprite.set_palette(spr.num_colors, spr.palette_data)
-- show the sprite
frame.display.bitmap(1, 1, spr.width, 2^spr.bpp, 0, spr.pixel_data)
frame.display.show()
-- clear the object and run the garbage collector right away
data.app_data[USER_SPRITE] = nil
collectgarbage('collect')
end
end
-- can't sleep for long, might be lots of incoming bluetooth data to process
frame.sleep(0.001)
end
)
-- Catch an error (including the break signal) here
if rc == false then
-- send the error back on the stdout stream and clear the display
print(err)
frame.display.text(' ', 1, 1)
frame.display.show()
break
end
end
end
-- run the main app loop
app_loop()
| Selected Examples | Python | Lua |
|---|---|---|
| Display a Sprite | Python | Lua |
| Display Unicode Text | Python | Lua |
| Take a photo (auto exposure) | Python | Lua |
| Take a photo (manual exposure) | Python | Lua |
| Live camera feed | Python | Lua |
| Stream audio | Python | Lua |
Legacy SDKs
Note: Legacy SDK packages are available for frame-sdk and frameutils as previously documented here.
In order to allow richer experiences on Frame including image display and audio streaming, the decision was made to move to an SDK model based on asynchronous message-passing between a main application loop running on the host, and a main event loop running on Frame. This approach is used in both the Python and Flutter SDKs.
While this requires a slight adjustment from the way the legacy SDKs were used, there are many examples provided and a helpful community in Discord to help you build your project.