Как я создал репетитор AI Voice с GPT-4, чтобы ускорить мое обучение
ai machine-learning startups saas ai-voice-generation generative-voice-ai build-in-public voice-ai-technology kinde

Как я создал репетитор AI Voice с GPT-4, чтобы ускорить мое обучение

7 августа 2025 г.

Несколько месяцев назад я заметил что -то разочаровывающее.

Я бы часами пытался выучить новые концепции - смотреть видео, читать статьи или общаться с Chatgpt. Но это все еще чувствовалосьмедленныйПолем Неуклюжий. Пассивный. Набор вопросов казался домашним заданием. Прокрутка для «правильного» объяснения было утомительным.

Итак, я спросил себя:

Что, если обучение больше похоже на разговор?

Этот вопрос превратился вLearnflow AI-Голосовой помощник по обучению, с которым вы можете поговорить, как личный репетитор по требованию.

В этой серии я покажу вам, как именно я его построил-от Zero до приложения GPT-4 с поддержкой в реальном времениVapiВNext.js, иOpenaiПолем

Что такое изучение AI?

Learnflow AIэто голосовой интерфейс обучения-подумайте, но вы не печатаете. ТыразговариватьИ ИИ говорит в режиме реального времени.

Он используетVapi.aiдля потокового голосового взаимодействия иGPT-4для интеллектуальных ответов. Эта комбинация создает невероятно естественный опыт обучения - нет беспорядка пользовательского интерфейса, просто нажмите кнопку и говорите.

Вы можете использовать этот же стек для построения:

  • Наставники ИИ, которые говорят и слушают
  • Голосовые спутники и продуктивность ботов
  • Помощники для обучения без рук

Что мы строим в этой части

Цель:Производственный MVP, который позволяет вам говорить с GPT-4 и получить разговорные ответы в режиме реального времени.

Вот что включено в часть 1:

  • Голосовой помощник построен сVapi.ai
  • GPT-4 для рассуждения
  • Next.js Frontend с использованием маршрутизатора приложения
  • Tailwind, Radix и Shadcn для стиля и компонентов
  • Нет пользовательской аудитории, базы данных, памяти или кредитов (пока - это часть 2)

Почему голос первым?

Набрать медленный. Прокрутка ошеломляет. Голос все меняет.

Когда вы громко задаете вопросы:

  • Тыпроцесс быстрее(нет необходимости структурировать печатные подсказки)
  • Это чувствует большеестественный и интуитивно понятный
  • Это имитируетРеальные учебные разговорыс репетитором

Разговор чувствует себя как обучение - набрать набрать как поиск.

Мой технический стек

Слой

Технический

Почему это было выбрано

Голосовой интерфейс

Vapi.ai

Потоковая потоковая передача звука в реальном времени + готово открыто

LLM -поставщик

Openai GPT-4

Высококачественные ответы, быстрый вывод

Внешний интерфейс

Next.js (маршрутизатор приложения)

Масштабируемая маршрутизация на основе файлов

Стиль

Tailwind CSS

Быстро итерация, отзывчивый

Компоненты

Radix UI + SHADCN

Доступные, низкоуровневые примитивы

Язык

Машинопись

Безопасность типа DX +

Хостинг

Вертел

Мгновенные развертывания для lete.js

Структура файла (Voice MVP)

Эта часть приложения намеренно проста - цель состоит в том, чтобы быстро показать голосового помощника в действии.

learnflow-ai/
├── app/
        └── globals.css
        └── layout.tsx
        └── page.tsx
├── constants/
    └── soundwaves.json
└── lib/
        └── utils.ts
    └── vapi.sdk.ts

Мы фокусируемся исключительно на том, чтобы работать голосовым потоком, прежде чем уловить государство, аут, БД или персонализацию.

Пошаговый: настройка голосового помощника

Эта часть предполагает, что вы уже настроили свой CodeBase Router CodeBase + Shadcn следующего приложения. Вы можете выполнить шаги здесь, чтобы сделать это:

Шаг 1: Настройка макета приложения

import type { Metadata } from "next";
import { Bricolage_Grotesque } from "next/font/google";
import "./globals.css";

const bricolage = Bricolage_Grotesque({
  variable: "--font-bricolage",
  subsets: ["latin"],
});

export const metadata: Metadata = {
  title: "Learnflow AI",
  description: "A voice-only learning platform for developers",
};

export default function RootLayout({
  children,
}: Readonly<{
  children: React.ReactNode;
}>) {
  return (
    <html lang="en">
      <body className={`${bricolage.variable} antialiased`}>
          {children}
      </body>
    </html>
  );
}

Шаг 2: Создайте папку Constants и файл Soundwaves.json, а затем вставьте этот JSON (constants/soundwaves.json)

{"nm":"Render","ddd":0,"h":250,"w":250,"meta":{"g":"LottieFiles AE 3.1.1"},"layers":[{"ty":4,"nm":"Arrow Outlines 4","sr":1,"st":0,"op":300.00001221925,"ip":0,"hd":false,"ddd":0,"bm":0,"hasMask":false,"ao":0,"ks":{"a":{"a":0,"k":[13,15.5,0],"ix":1},"s":{"a":0,"k":[170,170,100],"ix":6},"sk":{"a":0,"k":0},"p":{"a":0,"k":[100.5,126,0],"ix":2},"r":{"a":0,"k":0,"ix":10},"sa":{"a":0,"k":0},"o":{"a":0,"k":100,"ix":11}},"ef":[],"shapes":[{"ty":"sh","bm":0,"hd":false,"mn":"ADBE Vector Shape - Group","nm":"Path 1","ix":1,"d":1,"ks":{"a":1,"k":[{"o":{"x":0.333,"y":0},"i":{"x":0.667,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.471,13.618],[12.471,14.676]]}],"t":0},{"o":{"x":0.333,"y":0},"i":{"x":0.667,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.471,8.103],[12.471,18.868]]}],"t":30},{"o":{"x":0.333,"y":0},"i":{"x":0.667,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.471,8.324],[12.471,19.235]]}],"t":45},{"o":{"x":0.333,"y":0},"i":{"x":0.667,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.515,4.206],[12.515,25.853]]}],"t":70},{"o":{"x":0.333,"y":0},"i":{"x":0.667,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.471,8.985],[12.471,17.912]]}],"t":83},{"o":{"x":0.333,"y":0},"i":{"x":0.667,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.515,5.309],[12.544,24.088]]}],"t":97},{"o":{"x":0.333,"y":0},"i":{"x":1,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.456,7.588],[12.456,20.044]]}],"t":109},{"o":{"x":0.333,"y":0},"i":{"x":1,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.456,6.265],[12.456,21]]}],"t":121},{"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.471,13.618],[12.471,14.676]]}],"t":128.000005213547}],"ix":2}},{"ty":"st","bm":0,"hd":false,"mn":"ADBE Vector Graphic - Stroke","nm":"Stroke 1","lc":2,"lj":1,"ml":4,"o":{"a":0,"k":100,"ix":4},"w":{"a":0,"k":4,"ix":5},"c":{"a":0,"k":[0.9922,0.949,0.9922,1],"ix":3}},{"ty":"sh","bm":0,"hd":false,"mn":"ADBE Vector Shape - Group","nm":"Path 2","ix":3,"d":1,"ks":{"a":0,"k":{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.441,11.118],[12.441,16.735]]},"ix":2}}],"ind":1},{"ty":4,"nm":"Arrow Outlines 3","sr":1,"st":0,"op":300.00001221925,"ip":0,"hd":false,"ddd":0,"bm":0,"hasMask":false,"ao":0,"ks":{"a":{"a":0,"k":[13,15.5,0],"ix":1},"s":{"a":0,"k":[170,170,100],"ix":6},"sk":{"a":0,"k":0},"p":{"a":0,"k":[146.75,126,0],"ix":2},"r":{"a":0,"k":0,"ix":10},"sa":{"a":0,"k":0},"o":{"a":0,"k":100,"ix":11}},"ef":[],"shapes":[{"ty":"sh","bm":0,"hd":false,"mn":"ADBE Vector Shape - Group","nm":"Path 1","ix":1,"d":1,"ks":{"a":1,"k":[{"o":{"x":0.973,"y":0},"i":{"x":0.581,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.471,13.618],[12.471,14.676]]}],"t":0},{"o":{"x":0.973,"y":0},"i":{"x":0.24,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.441,13.765],[12.441,18.353]]}],"t":30},{"o":{"x":0.973,"y":0},"i":{"x":0.24,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.471,13.618],[12.471,13.721]]}],"t":45},{"o":{"x":0.167,"y":0},"i":{"x":0.24,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.529,7.074],[12.5,15.191]]}],"t":70},{"o":{"x":0.973,"y":0},"i":{"x":0.24,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.456,5.529],[12.441,16.735]]}],"t":83},{"o":{"x":0.167,"y":0},"i":{"x":0.24,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.544,7.147],[12.515,15.044]]}],"t":97},{"o":{"x":0.973,"y":0},"i":{"x":0.592,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.456,3.103],[12.471,21.809]]}],"t":109},{"o":{"x":0.973,"y":0},"i":{"x":0.893,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.441,11.118],[12.441,16.735]]}],"t":122},{"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.471,13.618],[12.471,14.676]]}],"t":129.000005254278}],"ix":2}},{"ty":"st","bm":0,"hd":false,"mn":"ADBE Vector Graphic - Stroke","nm":"Stroke 1","lc":2,"lj":1,"ml":4,"o":{"a":0,"k":100,"ix":4},"w":{"a":0,"k":4,"ix":5},"c":{"a":0,"k":[0.9922,0.949,0.9922,1],"ix":3}},{"ty":"sh","bm":0,"hd":false,"mn":"ADBE Vector Shape - Group","nm":"Path 2","ix":3,"d":1,"ks":{"a":0,"k":{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.441,11.118],[12.441,16.735]]},"ix":2}}],"ind":2},{"ty":4,"nm":"Arrow Outlines 2","sr":1,"st":0,"op":300.00001221925,"ip":0,"hd":false,"ddd":0,"bm":0,"hasMask":false,"ao":0,"ks":{"a":{"a":0,"k":[13,15.5,0],"ix":1},"s":{"a":0,"k":[170,170,100],"ix":6},"sk":{"a":0,"k":0},"p":{"a":0,"k":[116.75,126,0],"ix":2},"r":{"a":0,"k":0,"ix":10},"sa":{"a":0,"k":0},"o":{"a":0,"k":100,"ix":11}},"ef":[],"shapes":[{"ty":"sh","bm":0,"hd":false,"mn":"ADBE Vector Shape - Group","nm":"Path 1","ix":1,"d":1,"ks":{"a":1,"k":[{"o":{"x":0.333,"y":0},"i":{"x":0,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.471,13.618],[12.471,14.676]]}],"t":0},{"o":{"x":0.333,"y":0},"i":{"x":0,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.441,11.118],[12.441,16.735]]}],"t":24},{"o":{"x":0.973,"y":0},"i":{"x":0.24,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.441,4.647],[12.441,25.706]]}],"t":41},{"o":{"x":0.973,"y":0},"i":{"x":0.24,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.441,11.118],[12.441,16.735]]}],"t":55},{"o":{"x":0.167,"y":0},"i":{"x":0.24,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.471,6.118],[12.471,20.412]]}],"t":70},{"o":{"x":0.973,"y":0},"i":{"x":0.24,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.456,1.926],[12.456,22.838]]}],"t":87},{"o":{"x":0.973,"y":0},"i":{"x":0,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.471,7.735],[12.471,20.265]]}],"t":101},{"o":{"x":0.333,"y":0},"i":{"x":0.24,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.441,11.118],[12.441,16.735]]}],"t":115},{"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.471,13.618],[12.471,14.676]]}],"t":134.000005457932}],"ix":2}},{"ty":"st","bm":0,"hd":false,"mn":"ADBE Vector Graphic - Stroke","nm":"Stroke 1","lc":2,"lj":1,"ml":4,"o":{"a":0,"k":100,"ix":4},"w":{"a":0,"k":4,"ix":5},"c":{"a":0,"k":[0.9922,0.949,0.9922,1],"ix":3}},{"ty":"sh","bm":0,"hd":false,"mn":"ADBE Vector Shape - Group","nm":"Path 2","ix":3,"d":1,"ks":{"a":0,"k":{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.441,11.118],[12.441,16.735]]},"ix":2}}],"ind":3},{"ty":4,"nm":"Arrow Outlines","sr":1,"st":0,"op":300.00001221925,"ip":0,"hd":false,"ddd":0,"bm":0,"hasMask":false,"ao":0,"ks":{"a":{"a":0,"k":[13,15.5,0],"ix":1},"s":{"a":0,"k":[170,170,100],"ix":6},"sk":{"a":0,"k":0},"p":{"a":0,"k":[131.75,126,0],"ix":2},"r":{"a":0,"k":0,"ix":10},"sa":{"a":0,"k":0},"o":{"a":0,"k":100,"ix":11}},"ef":[],"shapes":[{"ty":"sh","bm":0,"hd":false,"mn":"ADBE Vector Shape - Group","nm":"Path 1","ix":1,"d":1,"ks":{"a":1,"k":[{"o":{"x":0.973,"y":0},"i":{"x":0.581,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.471,13.618],[12.471,14.676]]}],"t":0},{"o":{"x":0.973,"y":0},"i":{"x":0.24,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.5,2],[12.5,28.5]]}],"t":30},{"o":{"x":0.973,"y":0},"i":{"x":0.24,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.441,11.118],[12.441,16.735]]}],"t":45},{"o":{"x":0.167,"y":0},"i":{"x":0.24,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.5,2],[12.5,28.5]]}],"t":70},{"o":{"x":0.973,"y":0},"i":{"x":0.24,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.441,11.118],[12.441,16.735]]}],"t":83},{"o":{"x":0.167,"y":0},"i":{"x":0.24,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.5,2],[12.5,28.5]]}],"t":97},{"o":{"x":0.973,"y":0},"i":{"x":0.24,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.471,5.75],[12.471,21.809]]}],"t":109},{"o":{"x":0.973,"y":0},"i":{"x":0.24,"y":1},"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.441,11.118],[12.441,16.735]]}],"t":125},{"s":[{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.471,13.618],[12.471,14.676]]}],"t":132.00000537647}],"ix":2}},{"ty":"st","bm":0,"hd":false,"mn":"ADBE Vector Graphic - Stroke","nm":"Stroke 1","lc":2,"lj":1,"ml":4,"o":{"a":0,"k":100,"ix":4},"w":{"a":0,"k":4,"ix":5},"c":{"a":0,"k":[0.9922,0.949,0.9922,1],"ix":3}},{"ty":"sh","bm":0,"hd":false,"mn":"ADBE Vector Shape - Group","nm":"Path 2","ix":3,"d":1,"ks":{"a":0,"k":{"c":false,"i":[[0,0],[0,0]],"o":[[0,0],[0,0]],"v":[[12.441,11.118],[12.441,16.735]]},"ix":2}}],"ind":4},{"ty":4,"nm":"cir 1","sr":1,"st":0,"op":300.00001221925,"ip":0,"hd":false,"ddd":0,"bm":0,"hasMask":false,"ao":0,"ks":{"a":{"a":0,"k":[0,0,0],"ix":1},"s":{"a":1,"k":[{"o":{"x":0.775,"y":0},"i":{"x":0.206,"y":1},"s":[111.8,111.8,100],"t":0},{"o":{"x":0.665,"y":0},"i":{"x":0.274,"y":1},"s":[120,120,100],"t":30},{"o":{"x":0.167,"y":0},"i":{"x":0.206,"y":1},"s":[111.8,111.8,100],"t":60},{"o":{"x":0.665,"y":0},"i":{"x":0.274,"y":1},"s":[120,120,100],"t":71.289},{"o":{"x":0.167,"y":0},"i":{"x":0.206,"y":1},"s":[111.8,111.8,100],"t":82.576},{"o":{"x":0.665,"y":0},"i":{"x":0.274,"y":1},"s":[120,120,100],"t":97.628},{"o":{"x":0.167,"y":0},"i":{"x":0.667,"y":1},"s":[111.8,111.8,100],"t":107.661},{"s":[111.8,111.8,100],"t":134.000005457932}],"ix":6},"sk":{"a":0,"k":0},"p":{"a":0,"k":[126.539,126.166,0],"ix":2},"r":{"a":0,"k":0,"ix":10},"sa":{"a":0,"k":0},"o":{"a":0,"k":100,"ix":11}},"ef":[],"shapes":[{"ty":"gr","bm":0,"hd":false,"mn":"ADBE Vector Group","nm":"Ellipse 1","ix":1,"cix":2,"np":3,"it":[{"ty":"el","bm":0,"hd":false,"mn":"ADBE Vector Shape - Ellipse","nm":"Ellipse Path 1","d":1,"p":{"a":0,"k":[0,0],"ix":3},"s":{"a":0,"k":[99.367,99.367],"ix":2}},{"ty":"fl","bm":0,"hd":false,"mn":"ADBE Vector Graphic - Fill","nm":"Fill 1","c":{"a":0,"k":[0.1255,0.1529,0.1725,1],"ix":4},"r":1,"o":{"a":0,"k":100,"ix":5}},{"ty":"tr","a":{"a":0,"k":[0,0],"ix":1},"s":{"a":0,"k":[100,100],"ix":3},"sk":{"a":0,"k":0,"ix":4},"p":{"a":0,"k":[-1.539,-1.166],"ix":2},"r":{"a":0,"k":0,"ix":6},"sa":{"a":0,"k":0,"ix":5},"o":{"a":0,"k":100,"ix":7}}]}],"ind":5},{"ty":4,"nm":"cir 2","sr":1,"st":0,"op":300.00001221925,"ip":0,"hd":false,"ddd":0,"bm":0,"hasMask":false,"ao":0,"ks":{"a":{"a":0,"k":[0,0,0],"ix":1},"s":{"a":1,"k":[{"o":{"x":0.775,"y":0},"i":{"x":0.206,"y":1},"s":[110,110,100],"t":0},{"o":{"x":0.665,"y":0},"i":{"x":0.274,"y":1},"s":[194.775,194.775,100],"t":34},{"o":{"x":0.167,"y":0},"i":{"x":0.206,"y":1},"s":[150,150,100],"t":60},{"o":{"x":0.665,"y":0},"i":{"x":0.274,"y":1},"s":[194.775,194.775,100],"t":71.289},{"o":{"x":0.167,"y":0},"i":{"x":0.206,"y":1},"s":[150,150,100],"t":82.576},{"o":{"x":0.665,"y":0},"i":{"x":0.274,"y":1},"s":[194.775,194.775,100],"t":97.628},{"o":{"x":0.167,"y":0},"i":{"x":0.274,"y":1},"s":[150,150,100],"t":117},{"s":[110,110,100],"t":123.966255049249}],"ix":6},"sk":{"a":0,"k":0},"p":{"a":0,"k":[126.539,126.166,0],"ix":2},"r":{"a":0,"k":0,"ix":10},"sa":{"a":0,"k":0},"o":{"a":0,"k":100,"ix":11}},"ef":[],"shapes":[{"ty":"gr","bm":0,"hd":false,"mn":"ADBE Vector Group","nm":"Ellipse 1","ix":1,"cix":2,"np":3,"it":[{"ty":"el","bm":0,"hd":false,"mn":"ADBE Vector Shape - Ellipse","nm":"Ellipse Path 1","d":1,"p":{"a":0,"k":[0,0],"ix":3},"s":{"a":0,"k":[99.367,99.367],"ix":2}},{"ty":"fl","bm":0,"hd":false,"mn":"ADBE Vector Graphic - Fill","nm":"Fill 1","c":{"a":0,"k":[0.1255,0.1529,0.1725,1],"ix":4},"r":1,"o":{"a":0,"k":10,"ix":5}},{"ty":"tr","a":{"a":0,"k":[0,0],"ix":1},"s":{"a":0,"k":[100,100],"ix":3},"sk":{"a":0,"k":0,"ix":4},"p":{"a":0,"k":[-1.539,-1.166],"ix":2},"r":{"a":0,"k":0,"ix":6},"sa":{"a":0,"k":0,"ix":5},"o":{"a":0,"k":100,"ix":7}}]}],"ind":6},{"ty":4,"nm":"cir 3","sr":1,"st":0,"op":300.00001221925,"ip":0,"hd":false,"ddd":0,"bm":0,"hasMask":false,"ao":0,"ks":{"a":{"a":0,"k":[0,0,0],"ix":1},"s":{"a":1,"k":[{"o":{"x":0.775,"y":0},"i":{"x":0.206,"y":1},"s":[110,110,100],"t":0},{"o":{"x":0.665,"y":0},"i":{"x":0.274,"y":1},"s":[230,230,100],"t":34},{"o":{"x":0.167,"y":0},"i":{"x":0.206,"y":1},"s":[190,190,100],"t":60},{"o":{"x":0.665,"y":0},"i":{"x":0.274,"y":1},"s":[230,230,100],"t":71.289},{"o":{"x":0.167,"y":0},"i":{"x":0.206,"y":1},"s":[190,190,100],"t":82.576},{"o":{"x":0.665,"y":0},"i":{"x":0.274,"y":1},"s":[230,230,100],"t":97.628},{"o":{"x":0.167,"y":0},"i":{"x":0.274,"y":1},"s":[190,190,100],"t":118},{"s":[110,110,100],"t":134.000005457932}],"ix":6},"sk":{"a":0,"k":0},"p":{"a":0,"k":[126.539,126.166,0],"ix":2},"r":{"a":0,"k":0,"ix":10},"sa":{"a":0,"k":0},"o":{"a":0,"k":100,"ix":11}},"ef":[],"shapes":[{"ty":"gr","bm":0,"hd":false,"mn":"ADBE Vector Group","nm":"Ellipse 1","ix":1,"cix":2,"np":3,"it":[{"ty":"el","bm":0,"hd":false,"mn":"ADBE Vector Shape - Ellipse","nm":"Ellipse Path 1","d":1,"p":{"a":0,"k":[0,0],"ix":3},"s":{"a":0,"k":[99.367,99.367],"ix":2}},{"ty":"fl","bm":0,"hd":false,"mn":"ADBE Vector Graphic - Fill","nm":"Fill 1","c":{"a":0,"k":[0.1255,0.1529,0.1725,1],"ix":4},"r":1,"o":{"a":0,"k":5,"ix":5}},{"ty":"tr","a":{"a":0,"k":[0,0],"ix":1},"s":{"a":0,"k":[100,100],"ix":3},"sk":{"a":0,"k":0,"ix":4},"p":{"a":0,"k":[-1.539,-1.166],"ix":2},"r":{"a":0,"k":0,"ix":6},"sa":{"a":0,"k":0,"ix":5},"o":{"a":0,"k":100,"ix":7}}]}],"ind":7}],"v":"4.8.0","fr":29.9700012207031,"op":135.000005498663,"ip":0,"assets":[]}

Шаг 3: Создайте еще один файл в папке Constants (constants/index.ts) и вставьте этот код

export const subjects = [
  "javascript",
  "python",
  "html",
  "css",
  "algorithms",
  "databases",
];

export const subjectsColors = {
  javascript: "#FFD166",
  python: "#9BE7FF",
  html: "#FF9AA2",
  css: "#B5EAD7",
  algorithms: "#CBAACB",
  databases: "#FFDAC1",
};

export const voices = {
  male: { casual: "2BJW5coyhAzSr8STdHbE", formal: "c6SfcYrb2t09NHXiT80T" },
  female: { casual: "ZIlrSGI4jZqobxRKprJz", formal: "sarah" },
};

export const recentSessions = [
  {
    id: "1",
    subject: "javascript",
    name: "Codey the JS Debugger",
    topic: "Understanding Closures",
    duration: 40,
    color: "#FFD166",
  },
  {
    id: "2",
    subject: "python",
    name: "Snakey the Python Guru",
    topic: "List Comprehensions & Lambdas",
    duration: 35,
    color: "#9BE7FF",
  },
  {
    id: "3",
    subject: "html",
    name: "Structo the Markup Architect",
    topic: "Semantic Tags & Accessibility",
    duration: 25,
    color: "#FF9AA2",
  },
  {
    id: "4",
    subject: "css",
    name: "Stylo the Flexbox Wizard",
    topic: "Flexbox vs Grid Layouts",
    duration: 30,
    color: "#B5EAD7",
  },
  {
    id: "5",
    subject: "algorithms",
    name: "Algo the Problem Solver",
    topic: "Binary Search Explained Visually",
    duration: 45,
    color: "#CBAACB",
  },
  {
    id: "6",
    subject: "databases",
    name: "Query the Data Whisperer",
    topic: "SQL Joins: Inner vs Outer",
    duration: 20,
    color: "#FFDAC1",
  },
];

Шаг 4: Установите пакет Lottie-React

npm install lottie-react

Шаг 5: Установите VAPI SDK

npm install @vapi-ai/web

Вам понадобится бесплатноVapi accountЧтобы получить свой ключ API.

Шаг 6: Инициализировать клиент VAPI (lib/vapi.sdk.ts)

Это устанавливает VAPI SDK с вашим ключом API, позволяя вашему приложению подключаться к голосовой инфраструктуре VAPI:

import Vapi from "@vapi-ai/web";

export const vapi = new Vapi(process.env.NEXT_PUBLIC_VAPI_WEB_TOKEN!);

Он инициализирует основной клиент VAPI, который обрабатывает аудио, потоковую передачу и подключение к вашему помощнику искусственного интеллекта в реальном времени. Каждое взаимодействие голоса начинается здесь.

Шаг 7: Создайте (lib/utils.ts) подайте и вставьте эти коды

import { clsx, type ClassValue } from "clsx";
import { twMerge } from "tailwind-merge";
import { subjectsColors, voices } from "@/constants";
import { CreateAssistantDTO } from "@vapi-ai/web/dist/api";

export function cn(...inputs: ClassValue[]) {
  return twMerge(clsx(inputs));
}

export const getSubjectColor = (subject: string) => {
  return subjectsColors[subject as keyof typeof subjectsColors];
};

export const configureAssistant = (voice: string, style: string) => {
  const voiceId = voices[voice as keyof typeof voices][
          style as keyof (typeof voices)[keyof typeof voices]
          ] || "sarah";

  const vapiAssistant: CreateAssistantDTO = {
    name: "Companion",
    firstMessage:
        "Hello, let's start the session. Today we'll be talking about {{topic}}.",
    transcriber: {
      provider: "deepgram",
      model: "nova-3",
      language: "en",
    },
    voice: {
      provider: "11labs",
      voiceId: voiceId,
      stability: 0.4,
      similarityBoost: 0.8,
      speed: 1,
      style: 0.5,
      useSpeakerBoost: true,
    },
    model: {
      provider: "openai",
      model: "gpt-4",
      messages: [
        {
          role: "system",
          content: `You are a highly knowledgeable tutor teaching a real-time voice session with a student. Your goal is to teach the student about the topic and subject.

                    Tutor Guidelines:
                    Stick to the given topic - {{ topic }} and subject - {{ subject }} and teach the student about it.
                    Keep the conversation flowing smoothly while maintaining control.
                    From time to time make sure that the student is following you and understands you.
                    Break down the topic into smaller parts and teach the student one part at a time.
                    Keep your style of conversation {{ style }}.
                    Keep your responses short, like in a real voice conversation.
                    Do not include any special characters in your responses - this is a voice conversation.
              `,
        },
      ],
    },
    clientMessages: [],
    serverMessages: [],
  };
  return vapiAssistant;
};

Шаг 8: Создайте своего помощника

Просто вставьте этот код в свой (app/page.tsx) файл

'use client';

import {useEffect, useRef, useState} from 'react'
import {cn, configureAssistant, getSubjectColor} from "@/lib/utils";
import {vapi} from "@/lib/vapi.sdk";
import Image from "next/image";
import Lottie, {LottieRefCurrentProps} from "lottie-react";
import soundwaves from '@/constants/soundwaves.json'
import { useMutation } from "convex/react";
import { api } from '@/convex/_generated/api';
import { Id } from "@/convex/_generated/dataModel";

enum CallStatus {
    INACTIVE = 'INACTIVE',
    CONNECTING = 'CONNECTING',
    ACTIVE = 'ACTIVE',
    FINISHED = 'FINISHED',
}

const Page = () => {

        //Demo details
        const subject = "javascript"
        const topic = "React and Typescript"
        const name = "Better Call Saul"
        const style = "casual"
        const voice = "male"
        const userName = "Shola - student"
        const userImage = "images/me.png"

    const [callStatus, setCallStatus] = useState<CallStatus>(CallStatus.INACTIVE);
    const [isSpeaking, setIsSpeaking] = useState(false);
    const [isMuted, setIsMuted] = useState(false);
    const [messages, setMessages] = useState<SavedMessage[]>([]);

    const lottieRef = useRef<LottieRefCurrentProps>(null);

    useEffect(() => {
        if(lottieRef) {
            if(isSpeaking) {
                lottieRef.current?.play()
            } else {
                lottieRef.current?.stop()
            }
        }
    }, [isSpeaking, lottieRef])

    useEffect(() => {
        const onCallStart = () => setCallStatus(CallStatus.ACTIVE);

        const onCallEnd = () => {
            setCallStatus(CallStatus.FINISHED);
        }

        const onMessage = (message: Message) => {
            if(message.type === 'transcript' && message.transcriptType === 'final') {
                const newMessage= { role: message.role, content: message.transcript}
                setMessages((prev) => [newMessage, ...prev])
            }
        }

        const onSpeechStart = () => setIsSpeaking(true);
        const onSpeechEnd = () => setIsSpeaking(false);

        const onError = (error: Error) => console.log('Error', error);

        vapi.on('call-start', onCallStart);
        vapi.on('call-end', onCallEnd);
        vapi.on('message', onMessage);
        vapi.on('error', onError);
        vapi.on('speech-start', onSpeechStart);
        vapi.on('speech-end', onSpeechEnd);

        return () => {
            vapi.off('call-start', onCallStart);
            vapi.off('call-end', onCallEnd);
            vapi.off('message', onMessage);
            vapi.off('error', onError);
            vapi.off('speech-start', onSpeechStart);
            vapi.off('speech-end', onSpeechEnd);
        }
    }, []);

    const toggleMicrophone = () => {
        const isMuted = vapi.isMuted();
        vapi.setMuted(!isMuted);
        setIsMuted(!isMuted)
    }

    const handleCall = async () => {
        setCallStatus(CallStatus.CONNECTING)

        const assistantOverrides = {
            variableValues: { subject, topic, style },
            clientMessages: ["transcript"],
            serverMessages: [],
        }

        // @ts-expect-error - The configureAssistant function's return type doesn't match the expected type, but it works at runtime
        vapi.start(configureAssistant(voice, style), assistantOverrides)
    }

    const handleDisconnect = () => {
        setCallStatus(CallStatus.FINISHED)
        vapi.stop()
    }

    return (
        <section className="flex flex-col h-[70vh]">
            <section className="flex gap-8 max-sm:flex-col">
                <div className="companion-section">
                    <div className="companion-avatar" style={{ backgroundColor: getSubjectColor(subject)}}>
                        <div
                            className={
                            cn(
                                'absolute transition-opacity duration-1000', callStatus === CallStatus.FINISHED || callStatus === CallStatus.INACTIVE ? 'opacity-1001' : 'opacity-0', callStatus === CallStatus.CONNECTING && 'opacity-100 animate-pulse'
                            )
                        }>
                            <Image src={`/icons/${subject}.svg`} alt={subject} width={150} height={150} className="max-sm:w-fit" />
                        </div>

                        <div className={cn('absolute transition-opacity duration-1000', callStatus === CallStatus.ACTIVE ? 'opacity-100': 'opacity-0')}>
                            <Lottie
                                lottieRef={lottieRef}
                                animationData={soundwaves}
                                autoplay={false}
                                className="companion-lottie"
                            />
                        </div>
                    </div>
                    <p className="font-bold text-2xl">{name}</p>
                </div>

                <div className="user-section">
                    <div className="user-avatar">
                        <Image src={userImage} alt={userName} width={130} height={130} className="rounded-lg" />
                        <p className="font-bold text-2xl">
                            {userName}
                        </p>
                    </div>
                    <button className="btn-mic" onClick={toggleMicrophone} disabled={callStatus !== CallStatus.ACTIVE}>
                        <Image src={isMuted ? '/icons/mic-off.svg' : '/icons/mic-on.svg'} alt="mic" width={36} height={36} />
                        <p className="max-sm:hidden">
                            {isMuted ? 'Turn on microphone' : 'Turn off microphone'}
                        </p>
                    </button>
                    <button className={cn('rounded-lg py-2 cursor-pointer transition-colors w-full text-white', callStatus ===CallStatus.ACTIVE ? 'bg-red-700' : 'bg-primary', callStatus === CallStatus.CONNECTING && 'animate-pulse')} onClick={callStatus === CallStatus.ACTIVE ? handleDisconnect : handleCall}>
                        {callStatus === CallStatus.ACTIVE
                        ? "End Session"
                        : callStatus === CallStatus.CONNECTING
                            ? 'Connecting'
                        : 'Start Session'
                        }
                    </button>
                </div>
            </section>

            <section className="transcript">
                <div className="transcript-message no-scrollbar">
                    {messages.map((message, index) => {
                        if(message.role === 'assistant') {
                            return (
                                <p key={index} className="max-sm:text-sm">
                                    {
                                        name
                                            .split(' ')[0]
                                            .replace('/[.,]/g, ','')
                                    }: {message.content}
                                </p>
                            )
                        } else {
                           return <p key={index} className="text-primary max-sm:text-sm">
                                {userName}: {message.content}
                            </p>
                        }
                    })}
                </div>

                <div className="transcript-fade" />
            </section>
        </section>
    )
}

export default Page

Почему vapi?

Без VAPI вам нужно управлять:

  • Веб -вагоны
  • STT (речь в текст)
  • TTS (текст в речь)
  • Воспроизведение голоса и синхронизация потока

VAPI обрабатывает все это всего лишь несколькими строками кода. Это как волшебство для приложений AI-Prirst Voice.

Как работает голосовой помощник (шаг за шагом)

  1. Пользователь нажимает кнопку вызова
  2. VAPI открывает живой голосовой поток
  3. Пользователь отвечает на вопрос
  4. VAPI расшифровывает и отправляет его в OpenAI (GPT-4)
  5. Openai возвращает ответ
  6. VAPI превращает ответ в речь
  7. Браузер воспроизводит ответ на пользователя

Локальная установка

Предварительные условия:

  • Node.js 18+
  • VAPI API -ключ

.env.localфайл:

env
NEXT_PUBLIC_VAPI_WEB_TOKEN=your_vapi_web_token
VAPI_SECRET_KEY=your_vapi_secret_ke

Запустить это:

npm install
npm run dev

Открытьhttp://localhost:3000и нажмите кнопку, чтобы начать разговаривать с GPT-4.

Вынос

Учитесь AI AI доказывает одну вещь:

Разговор с ИИ гораздо более плавна, чем набрать к одному.

Комбинация VAPI + GPT-4 позволяет вам создавать мощных помощников с:

  • Разговорные разговоры в реальном времени
  • Zero Friction UI
  • Высокое удержание и понимание

И вы можете построить весь MVP на выходных.

Что происходит во второй части

Далее мы пойдем глубже и сделаем это личным:

  • Авторизм и войти в систему сВроде
  • Защищенные маршруты + мош
  • Выпуклость для обновлений в режиме государства и в реальном времени
  • Ограничения использования при отслеживании кредита

Попробуйте MVP или создайте свой собственный

GitHub:github.com/sholajegede/learnflow_ai

Если вы хотите настроитьКорьВПроверьте этот пост.

Оригинал

Recent Post

Categories

PREVIOUS ARTICLE
NEXT ARTICLE