---
title: "Conversational Robot"
summary: "A small robot that holds a spoken conversation on its own."
date: 2026-10-14
links:
  - "https://aesv.io/img/nao/squatting.png"
  - "https://developer.softbankrobotics.com/nao6/naoqi-developer-guide"
  - "https://aesv.io/img/nao/down.png"
  - "https://aesv.io/video/nao-longest-word.mp4"
---

<!-- DRAFT: Axel to rewrite -->

## Problem

A NAO is a two-foot humanoid robot. This one could only recite scripted replies. The goal, written before any code: "NAO can be naturally induced into a conversation in which it answers to any statement appropriately in natural time without relying on a laptop."

![NAO, a small white humanoid robot, squatting on a desk](/img/nao/squatting.png)

_The robot at rest, waiting for both feet to be pressed._

The hard part is turn-taking. The robot must detect when a person stops speaking, even while its own voice is on the microphone.

## Approach

Rules handle turn-taking, filtering and session limits. The language model only writes the reply.

```mermaid
flowchart LR
  P([person and the robot's own voice]) -->|sound| MIC[front mic 16 kHz]
  MIC --> VAD{onset 150 ms, endpoint 600 ms}
  VAD --> STT[speech to text]
  STT --> G{blank, echo or goodbye?}
  G -->|text| LLM[chat, streamed]
  LLM -->|sentence by sentence| TTS[robot speaks and gestures]
  TTS --> P
  VAD -->|onset while speaking| STOP[stop speech, cancel stream]
  STOP --> P
```

The first version, in 2024, ran on a laptop. The key constraint was the robot's limited processing: I used a MacBook to run the script and process speech, freeing the robot's thread for rapid speech production through the NAO SDK. The 2026 rebuild runs on the robot.

## Solution

Voice activity detection compares microphone loudness to a noise floor that adapts to the room. While the robot speaks, the threshold rises above its own voice.

```python
# whose turn is it? energy against a floor that tracks the room
start = rms > max(floor * 1.7, 300)            # sustained 150 ms: onset
if speaking:
    start = rms > max(base, echo_level * 1.2)  # louder than its own voice, 250 ms
end = quiet_for_ms(max(floor * 1.3, 150)) >= 600
```

Rules discard utterances under 300 ms, blank transcripts, echoes and goodbyes. The robot speaks each streamed sentence as it arrives.

```mermaid
flowchart LR
  A([armed]) -->|both feet within 1.5 s| I[idle]
  I -->|onset| C[capture]
  C -->|under 300 ms| I
  C -->|600 ms of silence| T[think]
  T -->|first sentence| S[speak]
  S -->|barge-in or head touch| C
  S -->|done| I
  I -->|goodbye, 30 s idle or a cap| A
```

An interruption stops the speech and cancels the stream.

```python
# barge-in: stop, cancel, and record only what was actually said
def interrupt(self):
    self.gen += 1                 # any late write from the old turn is ignored
    self.stream.cancel()
    self.tts.stop_all()
    self.history.append(("robot", self.spoken_so_far + " [interrupted]"))
```

Each session stops after 4 empty turns, 600 seconds or 60 transcription calls. These limits were added after two runaway sessions on 2026-10-02.

SSH access revealed it ran Gentoo with about 8GB to spare. The robot came with a GUI application and simulator, but I stuck with the terminal for speed to a POC. Digging through the binaries led to NAOqi, for which I found deeply nested documentation. NAOqi only loads in Python 2.7, and there is no root access. A Python 2.7 bridge process therefore owns all robot calls.

```mermaid
flowchart LR
  subgraph Robot[NAO, Gentoo, no root]
    AL[NAOqi autoload] --> BR[bridge, Python 2.7 with NAOqi]
    BR <-->|local socket, 5-byte frames| TK[talk, Python 3.5, standard library]
    BR --> SVC[audio, speech, eyes, touch]
  end
  TK -->|HTTPS| GW[model gateway]
```

The [NAOqi developer guide](https://developer.softbankrobotics.com/nao6/naoqi-developer-guide) documents the speech and motion services.

## Outcome

Latency over seven turns in three runs:

| Measure                       | Value                            |
| ----------------------------- | -------------------------------- |
| End of speech to first reply  | median 1.88 s, 1.67 s to 2.17 s  |
| Transcription, with the 0.6 s endpoint wait | 1.19 s, 1.35 s, 1.58 s (min, median, max) |
| Chat, first token             | 0.36 s to 0.50 s, median 0.47 s  |
| Idle hang-up at 30 s          | 30.045 s                         |
| Tests off the robot           | 157                              |

After a reboot, the robot was ready in one minute without a laptop. One failure is still open: the robot once heard its own voice and replied to itself.

![NAO lying face down on the floor](/img/nao/down.png)

_The robot face down on the floor._

![NAO says the longest word it knows and waves](/video/nao-longest-word.mp4)

## Applicability

The design fits any voice interface that takes turns with a person, such as kiosks, phone lines and assistive devices. The bridge pattern fits any SDK that only runs on an old runtime.