Introduction.
Developing “Mia,” a talking cat-shaped robot that speaks various dialects.
In the last issue of this newsletter, I described the implementation of the audio recording, playback, and uploading function using flutter_sound, but for some reason, the audio files were downloaded to the ESP32 mia body, but could not be played back.
Since no errors were generated, it was difficult to pinpoint the cause, but the cause was that the recording in m4a format by flutter_sound had been changed to mp3 with only the extension, and the contents were still in m4a format, so when played back on ESP32, it could not be played back.
We could modify the code on the ESP32 side so that it can play m4a as it is, but since machine speech synthesis already outputs mp3, we would like to unify it with mp3 this time.
flutter_sound does not support mp3!
The flutter_sound_platform_interface package seems to support encoding in mp3 format (although it says “shame on you!”), but when I actually used mp3, I got an error and could not encode.
Found someone else encountering the same problem.
https://github.com/Canardoux/flutter_sound/issues/175
As stated in the respondent’s table here, MP3 encoding is not supported by flutter_sound for either iOS or Android.
What about the record + audioplayers package?
As an alternative, we considered using a package other than flutter_sound, and after researching, considered using the record package for audio recording and the audioplayers package for audio playback.
However, it turns out that the record package also mainly supports recording file formats such as m4a and wav, and not mp3 format, so in the end, the same issue remains as in the case of the current flutter_sound package.
Therefore, we changed our policy to convert m4a to mp3 using the ffmpeg_kit_flutter_audio package. This package supports a wide range of formats using external libraries and includes the lame library needed to convert to mp3 format.
Conversion using ffmpeg_kit_flutter_audio
The ffmpeg_kit_flutter package is available here.
https://pub.dev/packages/ffmpeg_kit_flutter
The ffmpeg_kit_flutter package itself does not include an mp3 converter library, but provides eight additional external libraries, and the audio library includes an mp3 converter library.
https://github.com/arthenica/ffmpeg-kit?tab=readme-ov-file
So, install the ffmpeg_kit_flutter_audio package.
dependencies:
ffmpeg_kit_flutter_audio: 6.0.3-LTSThe code to convert m4a to mp3 using the ffmpeg_kit_flutter_audio package is as follows.
Future _convertToMp3() async {
if (_recordedFilePath == null) return;
final directory = await getTemporaryDirectory();
final outputPath =
'${directory.path}/recording_${DateTime.now().millisecondsSinceEpoch}.mp3';
final ffmpegCommand =
'-i $_recordedFilePath -codec:a libmp3lame -qscale:a 2 $outputPath';
await FFmpegKit.execute(ffmpegCommand).then((session) async {
final returnCode = await session.getReturnCode();
if (ReturnCode.isSuccess(returnCode)) {
setState(() {
_convertedFilePath = outputPath;
});
print('MP3への変換が成功しました: $outputPath');
} else {
setState(() {
_errorText = "MP3への変換に失敗しました";
});
print('MP3への変換に失敗しました。エラーメッセージを確認してください。');
}
});
}The converted MP3 audio file path is then sent to the API.
Future _addOrUpdatePhrase() async {
if ((_selectedTime != null && _selectedDays.isEmpty) ||
(_selectedTime == null && _selectedDays.isNotEmpty)) {
setState(() {
_errorText = '再生時間と再生曜日は両方設定するか、両方未設定にしてください。';
});
return;
}
await _convertToMp3(); // MP3に変換
List englishDays = translateDaysToEnglish(_selectedDays);
final userPhraseNotifier = ref.read(userPhraseProvider.notifier);
try {
if (_convertedFilePath != null) {
// 音声ファイルがある場合は必ず非公開に設定
_isPublic = false;
if (widget.phraseWithSchedule != null) {
await userPhraseNotifier.updateVoiceWithSchedule(
widget.phraseWithSchedule!.phrase.id,
_newPhrase,
_convertedFilePath!,
!_isPublic,
_selectedTime,
englishDays,
);
} else {
final newPhrase = await userPhraseNotifier.uploadVoiceWithSchedule(
_newPhrase,
_convertedFilePath!,
!_isPublic,
_selectedTime,
englishDays,
);
// IDを取得して音声ファイルパスを保存
int? newPhraseId = newPhrase.phrase.id;
debugPrint("newphrase id: $newPhraseId");
if (newPhraseId != null) {
await AudioStorageService.saveRecordedFilePath(
newPhraseId,
_convertedFilePath!,
);
}
}
} else {
// 音声ファイルがない場合の処理
if (widget.phraseWithSchedule == null) {
await userPhraseNotifier.addUserPhraseWithSchedule(
_newPhrase,
false,
!_isPublic,
_selectedTime,
englishDays,
);
} else {
await userPhraseNotifier.updateUserPhraseWithSchedule(
widget.phraseWithSchedule!.phrase.id,
_newPhrase,
false,
!_isPublic,
_selectedTime,
englishDays,
);
}
}
Navigator.of(context).pop();
} catch (e) {
setState(() {
_errorText = 'フレーズの保存に失敗しました: $e';
});
}
}operation check
The XCode logs show the following
ffmpeg version n6.0 Copyright (c) 2000-2023 the FFmpeg developers
...
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/recording.m4a':
...
Output #0, mp3, to '/recording.mp3':
...
flutter: MP3への変換が成功しました: /recording.mp3
flutter: 元のM4Aファイル: /recording.m4aIt was successfully converted to MP3 and played on ESP32.
