DIY AI math pencil

Build an AI Pencil That Solves Your Math Homework!

Meet Q — a pencil-shaped gadget with a tiny hidden camera. Press the button, and Q snaps a photo of your handwritten math problem, sends it off to an AI math-solving service over WiFi, and shows the answer right on its own mini screen.

⏱️ 4–5 hours 🎂 Ages 11+ (with an adult) 📷 ESP32-CAM + WiFi + AI

What Is an AI Math Pencil, Anyway?

Q combines three big ideas: a tiny camera that "sees" your handwriting, a WiFi connection that sends the photo to a smart online math-solving service, and a small screen that shows the answer back to you. The camera and WiFi both live on a single small board called the ESP32-CAM — one of the most powerful boards you can build with as a beginner.

Good to know: recognizing handwriting and solving math is a genuinely hard AI problem, far beyond what a tiny board can do alone. That's why Q sends the photo out to an online math-solving service (an OCR + math API) to do the heavy thinking, then just displays the result — the same "smart glasses" idea used by real AI gadgets.

🧰

What You'll Need

Gather these parts before you start building Q!

ESP32-CAM Module

A tiny board with a built-in camera and WiFi.

FTDI USB-to-Serial Programmer

Needed to upload code, since ESP32-CAM has no USB port of its own.

0.96" I2C OLED Display

Shows the solved answer.

Push Button

Press to snap a photo of your math problem.

3.7V LiPo Battery + Charging Module

Powers Q while it's out of reach of a cable.

Pencil-Shaped Casing

A 3D-printed or cardboard tube to house everything.

Math OCR / Solver API Key

A free or trial account with an online handwritten-math recognition service.

Jumper Wires

Connects the OLED and button to the ESP32-CAM.

🔌

The Circuit Diagram

The ESP32-CAM's camera pins are already built in — you only need to wire the OLED, button, and programmer.

OLED → GPIO 14 (SDA), GPIO 15 (SCL) Button → GPIO 13 FTDI (upload only) → GPIO 1, GPIO 3 Camera uses its own dedicated pins — no wiring needed

🛠️

Step-by-Step Build Instructions

Ask an adult to help with the FTDI programming step — it trips up a lot of beginners the first time!

Sign up for a math OCR API key

Create a free or trial account with an online handwritten math recognition service, and copy your API key — you'll paste it into the code.

Wire the FTDI programmer for uploading

Connect the FTDI's TX to the ESP32-CAM's U0R (GPIO 3), RX to U0T (GPIO 1), and briefly connect GPIO 0 to GND only while uploading code.

💡 Tip: Disconnect GPIO 0 from GND after uploading, or the board will stay stuck in programming mode!

Wire the OLED and button

Connect the OLED's SDA and SCL to GPIO 14 and 15, and wire the push button to GPIO 13, exactly as shown in the circuit diagram.

Install the required libraries

In the Arduino IDE, install support for the ESP32 board, then install ArduinoJson, Adafruit_SSD1306, and Adafruit_GFX.

Update the code and upload

Fill in your WiFi name, password, and API key in the code below, then upload it while GPIO 0 is grounded.

Build the pencil casing

Once everything works on the breadboard, carefully mount the ESP32-CAM, OLED, button, and battery inside your pencil-shaped casing, with the camera lens and screen both facing outward.

Test it on a real math problem

Write a simple equation on paper, hold Q's camera over it, press the button, and watch the answer appear on the screen!

💻

The ESP32-CAM Code

Fill in your WiFi details, API endpoint, and key, then upload with the board set to AI Thinker ESP32-CAM.

// ✏️🤖 Q the AI Math Pencil — ESP32-CAM Robotics Project
// Snaps a photo of a math problem, sends it to an online solver, shows the answer

#include "esp_camera.h"
#include <WiFi.h>
#include <HTTPClient.h>
#include <ArduinoJson.h>
#include <Wire.h>
#include <Adafruit_GFX.h>
#include <Adafruit_SSD1306.h>

// ---- AI-Thinker ESP32-CAM pin map (standard, don't change) ----
#define PWDN_GPIO_NUM  32
#define RESET_GPIO_NUM -1
#define XCLK_GPIO_NUM   0
#define SIOD_GPIO_NUM  26
#define SIOC_GPIO_NUM  27
#define Y9_GPIO_NUM    35
#define Y8_GPIO_NUM    34
#define Y7_GPIO_NUM    39
#define Y6_GPIO_NUM    36
#define Y5_GPIO_NUM    21
#define Y4_GPIO_NUM    19
#define Y3_GPIO_NUM    18
#define Y2_GPIO_NUM     5
#define VSYNC_GPIO_NUM 25
#define HREF_GPIO_NUM  23
#define PCLK_GPIO_NUM  22

// ---- Fill in your own details ----
const char* ssid     = "YOUR_WIFI_NAME";
const char* password = "YOUR_WIFI_PASSWORD";
const char* apiKey   = "YOUR_MATH_SOLVER_API_KEY";
const char* apiURL   = "https://your-chosen-math-ocr-service.com/solve";

const int buttonPin = 13;
const int flashLED  = 4;

Adafruit_SSD1306 display(128, 64, &Wire, -1);

void setup() {
  Serial.begin(115200);
  pinMode(buttonPin, INPUT_PULLUP);
  pinMode(flashLED, OUTPUT);

  Wire.begin(14, 15);   // SDA, SCL for the OLED
  display.begin(SSD1306_SWITCHCAPVCC, 0x3C);
  showMessage("Connecting WiFi...");

  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) delay(300);
  showMessage("Ready! Press button.");

  // ---- Configure and start the camera ----
  camera_config_t config;
  config.pin_pwdn = PWDN_GPIO_NUM;
  config.pin_reset = RESET_GPIO_NUM;
  config.pin_xclk = XCLK_GPIO_NUM;
  config.pin_sscb_sda = SIOD_GPIO_NUM;
  config.pin_sscb_scl = SIOC_GPIO_NUM;
  config.pin_d7 = Y9_GPIO_NUM;  config.pin_d6 = Y8_GPIO_NUM;
  config.pin_d5 = Y7_GPIO_NUM;  config.pin_d4 = Y6_GPIO_NUM;
  config.pin_d3 = Y5_GPIO_NUM;  config.pin_d2 = Y4_GPIO_NUM;
  config.pin_d1 = Y3_GPIO_NUM;  config.pin_d0 = Y2_GPIO_NUM;
  config.pin_vsync = VSYNC_GPIO_NUM;
  config.pin_href = HREF_GPIO_NUM;
  config.pin_pclk = PCLK_GPIO_NUM;
  config.xclk_freq_hz = 20000000;
  config.pixel_format = PIXFORMAT_JPEG;
  config.frame_size = FRAMESIZE_VGA;   // good balance of detail vs upload speed
  config.jpeg_quality = 12;
  config.fb_count = 1;

  esp_camera_init(&config);
}

void loop() {
  if (digitalRead(buttonPin) == LOW) {
    solveMathProblem();
    delay(1000);   // small pause to avoid re-triggering instantly
  }
}

// Takes a photo, sends it to the solver, and displays the result
void solveMathProblem() {
  showMessage("Scanning...");
  digitalWrite(flashLED, HIGH);
  camera_fb_t *fb = esp_camera_fb_get();
  digitalWrite(flashLED, LOW);

  if (!fb) {
    showMessage("Camera error!");
    return;
  }

  showMessage("Solving...");
  String answer = sendImageToSolver(fb);
  esp_camera_fb_return(fb);

  showMessage("Answer: " + answer);
}

// Uploads the photo to the math-solving API and reads back the answer
String sendImageToSolver(camera_fb_t *fb) {
  HTTPClient http;
  http.begin(apiURL);
  http.addHeader("Content-Type", "image/jpeg");
  http.addHeader("Authorization", apiKey);

  int httpCode = http.POST(fb->buf, fb->len);
  String result = "Error";

  if (httpCode == 200) {
    String payload = http.getString();
    StaticJsonDocument<512> doc;
    deserializeJson(doc, payload);
    result = doc["answer"].as<String>();  // adjust to match your API's response format
  }

  http.end();
  return result;
}

// Shows a short message on the OLED
void showMessage(String msg) {
  display.clearDisplay();
  display.setTextSize(1);
  display.setTextColor(SSD1306_WHITE);
  display.setCursor(0, 25);
  display.println(msg);
  display.display();
}

🧠

How Does Q Actually Work?

Here are the big ideas hiding inside this project:

📷

Camera Frame Buffers

esp_camera_fb_get() grabs one photo as raw JPEG data in memory — that data is exactly what gets uploaded to the solver, no separate file needed.

☁️

Offloading Hard Work to the Cloud

Recognizing handwriting and solving equations needs a large AI model — far too big for a tiny board. Sending the photo to a powerful online service is how real small devices "borrow" big AI brainpower.

📦

Reading the Response

The solver sends its answer back as JSON — ArduinoJson unpacks that into a simple value your code can display, just like the weather globe project did with weather data.

🔦

Using the Onboard Flash

The ESP32-CAM's built-in flash LED (GPIO 4) briefly lights up right when the photo is taken, helping the camera get a clearer shot of your handwriting.

🧑‍🔬 Safety First!

Build with an adult, especially during the FTDI programming step and when handling a LiPo battery.
Never puncture, bend, or overcharge a LiPo battery — always use a proper charging module.
Keep your WiFi password and API key private, and don't share screenshots of your code that reveal them.
This is a learning project — always double-check homework answers yourself rather than trusting any single device completely.

❓

Frequently Asked Questions

Which math OCR service should I use?

Several online services specialize in reading handwritten math and returning a solved answer. Search for "math OCR API" or "handwriting equation solver API," compare their free tiers, and adjust the apiURL and JSON field names in the code to match the one you choose.

Why does the ESP32-CAM need a separate FTDI programmer?

The ESP32-CAM board doesn't include a USB-to-serial chip like most Arduino boards do, to keep it small and cheap — the FTDI adapter provides that missing piece just for uploading code.

The upload fails — what should I check?

Make sure GPIO 0 is connected to GND only during upload, that TX/RX aren't swapped, and that you've selected "AI Thinker ESP32-CAM" as your board in the Arduino IDE.

What age group is this project good for?

Because it involves cloud APIs, camera modules, and careful power handling, this is an advanced project best suited for kids around age 11+ working closely with an adult.

ROBOTICS

Search This Blog