p5-phone

p5-phone: Mobile Hardware for p5.js

p5-phone is a single-file helper library that gives p5.js sketches access to mobile phone hardware — motion sensors, microphone, sound, speech, camera (with ML5 coordinate mapping), vibration, torch/flashlight, NFC, GPS/geolocation, Bluetooth LE, and multi-user Share rooms — plus mobile gesture locking, browser-permission activation UI, and an on-screen debug console. Current version: 1.15.0.

It works in both p5.js 1.x and 2.x (auto-detected at runtime). Every public function is attached to window (global mode) and mirrored on p5.prototype (instance mode), so you call them as bare globals like lockGestures() and enableSensorTap().

Pair this with the p5js-2x skill. p5-phone only unlocks the hardware and hands you p5's own globals and objects; the p5.js 2.x language patterns around them (async asset loading, the unified pointer/touch model, renamed APIs) live in the p5js-2x skill. Load both when writing phone sketches — and read the next section before writing any hardware code.

When to use this skill

Use it whenever the request involves a p5-phone sketch, mobile p5.js hardware interaction, or an explanation of how p5-phone works: device orientation / accelerometer / gyroscope, touch, microphone / p5.sound, speech recognition, PhoneCamera and ML5 mapping, vibration, torch/flashlight, NFC, GPS/geolocation (geoRead, geoDistance, geoInPolygon), Bluetooth BLE, Share multi-user rooms (shareSetup, shared/me/guests), keeping the screen on (Screen Wake Lock), lockGestures, enablePermissionsTap / enableHardwareTap, arbitrary hardware combinations, mobile browser permissions, or p5.js 2 compatibility.

Quick Start

For generated examples, produce a complete index.html and sketch.js unless the user asks for a single file or snippet.

HTML baseline (p5.js 2-compatible):

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Mobile p5.js App</title>
  <style>
    body { margin: 0; padding: 0; overflow: hidden; }
  </style>
  <script src="https://cdn.jsdelivr.net/npm/[email protected]/lib/p5.js"></script>
  <script src="https://cdn.jsdelivr.net/npm/[email protected]/src/preload.js"></script>
  <script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/p5-phone.min.js"></script>
</head>
<body>
  <script src="sketch.js"></script>
</body>
</html>

Add p5.sound only when the sketch uses microphone levels, oscillators, audio input, sound output, or speech. Place it after p5 and before p5-phone/sketch.js:

<script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/p5.sound.min.js"></script>

Minimal sketch (sketch.js):

function setup() {
  createCanvas(windowWidth, windowHeight);
  lockGestures();
  enableSensorTap('Tap to enable motion sensors');
}

function draw() {
  background(20);
  if (!window.sensorsEnabled) {
    fill(255);
    textAlign(CENTER, CENTER);
    text('Waiting for sensors', width / 2, height / 2);
    return;
  }
  // rotationX / rotationY / rotationZ are p5.js built-ins, live once sensorsEnabled is true.
  fill(0, 180, 255);
  circle(width / 2 + rotationY * 4, height / 2 + rotationX * 4, 80);
}

function mousePressed() {
  return false; // let p5-phone's gesture handling manage the touch
}

Golden Rules

1. Call lockGestures() in every mobile sketch setup(). It blocks pull-to-refresh, swipe-back, pinch-zoom, double-tap zoom, long-press menus, and overscroll so the canvas behaves like an app.
2. Request every permission from a user gesture. iOS grants sensitive APIs only during transient user activation (a tap/click). Never auto-request on page load — use an enable* activation UI.
3. Gate all hardware reads behind the matching window.*Enabled flag (sensorsEnabled, micEnabled, bleConnected, etc.). Reading before permission returns stale/undefined data.
4. Use mousePressed / mouseDragged / mouseReleased, not p5 1.x touchStarted / touchMoved / touchEnded. The mouse callbacks fire for both mouse and touch in p5.js 1.x and 2.x; the touch callbacks are removed/no-ops in p5.js 2.
5. Serve over HTTPS (or localhost). Sensors, mic, camera, NFC, BLE, GPS, and torch all require a secure context on mobile.
6. Need several hardware features from one tap? Use a single combined call — enablePermissionsTap(['sensors', 'torch']) — not several single-permission binds on the same gesture. One call keeps iOS transient activation intact and fires userSetupComplete() once.
7. Use exactly one activation style per permission need unless the user explicitly asks to compare styles.

Use p5.js built-ins — do not reimplement them

The most common failure mode is a model hand-rolling hardware plumbing (a raw DeviceOrientationEvent listener, a touchstart handler, a Web Audio graph, a manual asset loader) instead of reading the values p5-phone and p5.js already provide. p5-phone deliberately surfaces everything through p5's own globals and objects. For each concern below, use the p5 built-in — never a bespoke equivalent:

Two specific traps worth calling out:

• Motion values are p5.js built-ins, not p5-phone APIs. p5-phone only requests the permission and sets window.sensorsEnabled. The data (rotationX, accelerationX, deviceShaken(), …) comes straight from p5.js. Do not invent p5-phone getters for them.
• p5.js has no rotation rate. rotationRateAlpha, rotationRateBeta and rotationRateGamma are not p5 globals, in 1.x or 2.x, and p5-phone does not add them. Reading them throws a ReferenceError on every frame once the sensors are on, so the sketch stops drawing. For how fast the phone is turning, use the change in rotationX/Y/Z from pRotationX/Y/Z each frame (watch the wrap at ±180°), or deviceTurned() and turnAxis for a quarter turn. Only if a sketch truly needs the gyroscope's own rate, read event.rotationRate (it can be null) in a devicemotion listener once window.sensorsEnabled is true. That is the one place a listener is the right call.
• There is no bleValue() getter. Read incoming BLE data from window.bleValues[name] or from the bleReceive(name, value) callback. bleValues is an object keyed by characteristic name.

Permissions model

Every hardware family exposes the same six activation styles. Pick one:

• Tap — enable<Feature>Tap(message) — full-screen tap overlay.
• Button — enable<Feature>Button(buttonText, statusText?) — auto-generated button.
• Canvas — enable<Feature>Canvas(message) — prompt drawn on the p5 canvas.
• Banner — enable<Feature>Banner(message, position?) — animated slide-in banner (position = 'top'/'bottom').
• On (custom element) — enable<Feature>On(selector) — bind activation to any existing HTML element by CSS selector. The element stays on the page; hide it in draw() once the flag is true (select('#start').hide()).
• Minimal — enable<Feature>Minimal(message?) or enable<Feature>Minimal({ color, opacity, icon, iconColor, iconSize, message }) — bare semi-transparent overlay with a pulsing icon, no message box (v1.14.0).

Full matrix:

Content truncated.