Build a Talking AI Dictionary with Raspberry Pi 🐰📖
Turn a Raspberry Pi, a tiny camera, and a touch screen into a gadget that looks at any object, tells you what it is, and reads the answer out loud — like your own robot teacher!
📖 What is an AI Visual Dictionary?
Imagine holding up an apple to a little box, and instead of typing anything, the box looks at the apple, thinks for a second, and then says out loud: "This is an Apple! A round fruit that grows on trees. Fun fact — apples float in water!"
That's exactly what we're building. It uses:
- A camera to see the world (or a microphone to hear a question)
- A smart AI model (Google Gemini) to understand what it sees or hears
- A touch screen to show the answer
- A speaker to say the answer out loud
🧰 Gather Your Materials
Here's everything you need. Most of these plug in — no soldering required!
🔌 Wire It Up
Good news: almost everything on this build is plug-and-play. Here's how each part connects to the Raspberry Pi 5:
Connect the camera
Gently lift the black clip on the Pi's camera port, slide the ribbon cable in (blue side facing the USB ports), then push the clip back down.
Connect the touch display
Plug the display's HDMI cable into the Pi's HDMI port, and its separate USB touch cable into any USB port — this lets your finger taps register as clicks.
Plug in the mic
Plug your USB microphone into any free USB port on the Pi.
Add the push button (optional)
Connect one leg of the button to GPIO 17 and the other leg to any GND (ground) pin.
💻 Set Up the Software
Now let's wake up the "brain." Open a Terminal on your Raspberry Pi and type these commands one at a time.
Update the Pi & install tools
sudo apt update && sudo apt upgrade -y
sudo apt install -y python3-pip python3-venv mpg123 portaudio19-dev python3-picamera2Create your project folder
mkdir ~/AI_Dictionary
cd ~/AI_Dictionary
python3 -m venv venv --system-site-packages
source venv/bin/activateInstall the Python packages
pip install google-genai gtts pillow gpiozero sounddevice scipy faster-whisper numpyAdd your AI key
Get a free Gemini API key from Google AI Studio, then save it:
nano config.pyAPI_KEY = "paste_your_key_here"🐍 Write the Code, Piece by Piece
We'll build seven small files. Each one has a single job — like separate departments in a tiny robot company!
📷 camera.py — takes the photo
from picamera2 import Picamera2
from time import sleep
import os
class Camera:
def __init__(self):
self.picam2 = Picamera2()
config = self.picam2.create_still_configuration()
self.picam2.configure(config)
self.picam2.start()
sleep(2)
os.makedirs("temp", exist_ok=True)
self.filename = "temp/capture.jpg"
def capture(self):
self.picam2.capture_file(self.filename)
return self.filename
def stop(self):
self.picam2.stop()🧠 ai_engine.py — the AI brain
import json
from PIL import Image
from google import genai
from config import API_KEY
class AIEngine:
def __init__(self):
self.client = genai.Client(api_key=API_KEY)
def object_prompt(self):
return """Identify ONLY the main object nearest the
center of the image. Return ONLY JSON:
{"word":"","pronunciation":"","meaning":"",
"example":"","fun_fact":""}
Meaning suitable for Grade 5. Max 30 words."""
def identify_object(self, image_path):
image = Image.open(image_path)
response = self.client.models.generate_content(
model="gemini-2.5-flash",
contents=[image, self.object_prompt()]
)
text = response.text.strip().replace("```json","").replace("```","")
return json.loads(text)🔊 speech.py — makes it talk
import os, threading
from gtts import gTTS
class Speaker:
def speak(self, text):
threading.Thread(target=self._play, args=(text,), daemon=True).start()
def _play(self, text):
tts = gTTS(text=text, lang="en")
tts.save("temp/speech.mp3")
os.system("mpg123 -q temp/speech.mp3")🎙️ voice.py — listens to your questions
import sounddevice as sd
from scipy.io.wavfile import write
from faster_whisper import WhisperModel
import numpy as np, tempfile, os
class VoiceRecognizer:
def __init__(self):
self.model = WhisperModel("tiny", device="cpu", compute_type="int8")
def listen(self):
rec = sd.rec(int(5*16000), samplerate=16000, channels=1, dtype="int16")
sd.wait()
path = os.path.join(tempfile.gettempdir(), "voice.wav")
write(path, 16000, rec)
segments, _ = self.model.transcribe(path, language="en")
return "".join(s.text for s in segments).strip()🗂️ history.py — remembers past words
import json, os
from datetime import datetime
class History:
def __init__(self, filename="history.json"):
self.filename = filename
if not os.path.exists(filename):
json.dump([], open(filename, "w"))
def add(self, data):
entries = json.load(open(self.filename))
entries.insert(0, {
"word": data.get("word",""),
"time": datetime.now().strftime("%Y-%m-%d %H:%M")
})
json.dump(entries[:50], open(self.filename, "w"), indent=2)🖐️ gui.py — the touch screen face
This file builds the on-screen dictionary card with two big buttons: Scan Object and Ask Word. It uses Python's built-in tkinter library to draw the layout full-screen on your 4.3" display.
🚦 main.py — connects everything
from camera import Camera
from ai_engine import AIEngine
from speech import Speaker
from voice import VoiceRecognizer
from history import History
from gui import DictionaryGUI
camera = Camera()
ai = AIEngine()
speaker = Speaker()
voice = VoiceRecognizer()
history = History()
def handle_scan():
photo = camera.capture()
result = ai.identify_object(photo)
history.add(result)
speaker.speak(result["meaning"])
gui.show_result(result)
def handle_ask():
question = voice.listen()
result = ai.ask_word(question)
history.add(result)
speaker.speak(result["meaning"])
gui.show_result(result)
gui = DictionaryGUI(on_scan=handle_scan, on_ask=handle_ask)
gui.run()🚀 Run Your AI Dictionary!
Launch it
cd ~/AI_Dictionary
source venv/bin/activate
python main.pyYour screen should light up full-screen, and you'll hear "AI Dictionary is ready" 🎉
Try it out
- Point the camera at something like an apple, a toy, or a pencil and press Scan Object
- Or press Ask Word and speak a question out loud, like "What is a volcano?"
🔧 Fix Common Hiccups
| No camera found | Check the ribbon cable is fully clipped in, and enable the camera in sudo raspi-config → Interface Options. |
| Mic not detected | Run lsusb to check the Pi actually sees it — try a different USB port if not. |
| No sound from speaker | Re-pair it with bluetoothctl connect [address] and set it as default with pactl set-default-sink. |
| AI gives an error | Check your internet connection and that your API key in config.py is correct and active. |
| Voice hears the wrong words | Add language="en" to the transcribe call, and speak clearly and close to the mic. |
❓ Frequently Asked Questions
Do I need to know how to code already?
Not much! This project teaches you as you go. If you can type and follow instructions carefully, you can build this.
Does this work without internet?
The camera, screen, and speaker all work offline, but the "thinking" part uses Google's Gemini AI, which needs an internet connection.
Can I enter this in a school science fair?
Absolutely! It's a great blend of electronics, coding, and artificial intelligence — a real crowd-pleaser at exhibitions.
How much does this cost to build?
Costs vary by region and where you buy parts, but a Raspberry Pi 5, camera, small touch display, USB mic, and Bluetooth speaker are all affordable, widely available components.
🎉 You built a real talking AI!
Show it off, teach a friend, and keep experimenting — try new objects, funnier fun-facts, or even a second language!
Comments
Post a Comment