speak-core-workflow-a

Speak Core Workflow A: AI Conversation Practice

Overview

Primary workflow for Speak: AI-powered conversation practice with real-time pronunciation feedback and adaptive tutoring. Speak uses GPT-4o for conversation generation and OpenAI's Realtime API for speech processing, delivering sub-second response times.

Prerequisites

• Completed speak-install-auth setup
• Valid API credentials configured
• Audio handling capabilities (microphone or pre-recorded files)

Instructions

Step 1: Start a Conversation Session

import { SpeakClient } from '@speak/language-sdk';

const client = new SpeakClient({
  apiKey: process.env.SPEAK_API_KEY!,
  appId: process.env.SPEAK_APP_ID!,
  language: 'es',
});

// Start a restaurant ordering scenario in Spanish
const session = await client.startConversation({
  scenario: 'ordering-food',
  language: 'es',
  level: 'intermediate',
  nativeLanguage: 'en',
  maxTurns: 10,
  feedbackDetail: 'phoneme', // 'word' or 'phoneme'
});

console.log('Session started:', session.id);
console.log('AI Tutor:', session.firstPrompt.text);
// "Bienvenido al restaurante. Soy tu camarero. Que le gustaria ordenar?"

Step 2: Send Student Responses

// Submit audio for pronunciation scoring
const turn1 = await client.sendTurn(session.id, {
  audioPath: './recordings/student-response-1.wav',
});

console.log('Tutor:', turn1.tutorText);
console.log('Pronunciation:', turn1.pronunciationScore); // 0-100
console.log('Grammar:', turn1.corrections);
// [{original: "yo quiero", suggestion: "quisiera", note: "More polite form for ordering"}]
console.log('Vocabulary:', turn1.vocabularyNotes);
// ["camarero = waiter", "ordenar = to order"]

// Or submit text (skips pronunciation scoring)
const turn2 = await client.sendTurn(session.id, {
  text: 'Quisiera una ensalada y un vaso de agua, por favor.',
});

Step 3: Conversation Loop with Progress Tracking

async function runConversationLesson(
  client: SpeakClient,
  scenario: string,
  language: string,
  level: string,
) {
  const session = await client.startConversation({
    scenario, language, level, nativeLanguage: 'en',
  });

  const turns: TurnResult[] = [];
  let isComplete = false;

  while (!isComplete && turns.length < 10) {
    // Display tutor prompt
    const prompt = turns.length === 0
      ? session.firstPrompt.text
      : turns[turns.length - 1].tutorText;
    console.log(`\nTutor: ${prompt}`);

    // Get student audio (mic input or file)
    const audioPath = await recordStudentAudio();

    // Submit and get feedback
    const turn = await client.sendTurn(session.id, { audioPath });
    turns.push(turn);

    // Show feedback
    if (turn.pronunciationScore < 60) {
      console.log(`Pronunciation needs work: ${turn.pronunciationScore}/100`);
      console.log('Try again with this phrase.');
    }
    if (turn.corrections.length > 0) {
      console.log('Grammar notes:', turn.corrections.map(c => c.note).join('; '));
    }

    isComplete = turn.sessionComplete;
  }

  // End session and get summary
  const summary = await client.endSession(session.id);
  return summary;
}

Step 4: Multi-Topic Session

const topics = ['greetings', 'directions', 'ordering-food', 'shopping'];
const results: SessionSummary[] = [];

for (const topic of topics) {
  console.log(`\n=== ${topic.toUpperCase()} ===`);
  const summary = await runConversationLesson(client, topic, 'es', 'intermediate');
  results.push(summary);
  console.log(`Score: ${summary.avgPronunciationScore}/100`);
}

// Overall progress report
console.log('\n=== Session Report ===');
console.table(results.map(r => ({
  topic: r.scenario,
  pronunciation: r.avgPronunciationScore,
  grammar: r.grammarAccuracy + '%',
  newWords: r.newWords.length,
  duration: r.durationMinutes + 'min',
})));

Topic Categories

Output

• Conversation session with AI tutor
• Real-time pronunciation feedback (0-100 score)
• Grammar corrections and suggestions
• Vocabulary notes for new words
• Session summary with progress metrics

Error Handling

Resources

• Speak Website
• Speak GPT-4 Blog
• OpenAI Realtime API
• Live Roleplays

Next Steps

For pronunciation-focused training, see speak-core-workflow-b.

Examples

Quick test: Start a greetings scenario with level: 'beginner', send 3 text responses, end session, and review the summary scores.

Full lesson: Run 4 topics in sequence, track pronunciation improvement across topics, and generate a progress report.