[ESP32] Implementation of Deep Sleep Mode and Timer Wake-Up Function

Introduction.
Select Deep-sleep mode
Deep Sleep mode implemented
When I checked the operation, the display was still glowing white.
Fill screen to black + Light sleep mode
operation check

Introduction.

Developing “Mia,” a talking cat-shaped robot that speaks dialects.

The talking cat-shaped robot Mia shares sadness and joy with more than 100 different rich expressions. It also speaks in...

After releasing the beta version, several users have actually used it.

I would like the ability to automatically turn off Mia’s eye display.”

The implementation of this function is described in this issue because we received a request for it.

We asked for more details,

It is in the living room, next to the TV, but if the door is left open, it is just visible from the bedroom, so when Mia’s eyes are shining, the children have a hard time sleeping.

When Mia’s eyes are OFF, she can encourage the child to say, “The cat is sleeping, so it’s time for us to go to bed too.

I am sure it is scary when only the eyes are glowing like this. It is indeed scary when only the eyes are glowing like this.

Select Deep-sleep mode

In a previous article here, we described the characteristics and requirements for ESP32’s Deep Sleep mode and Light Sleep mode, respectively.

In light-sleep mode, the power supply to the eye’s LCD display is not turned off, so I will implement this time in deep-sleep mode.

Now, how to implement triggering of deep-sleep mode and release from deep-sleep mode, we are currently setting the time to talk in the application.

End time of talking time = Time when Mia turns off = Start of Deep-sleep mode
Start time of talking time = time when Mia turns on = Deep-sleep mode is released.

I would like to do so.

Deep Sleep mode implemented

Click here for an article on the official website explaining the Sleep mode of ESP32.

https://docs.espressif.com/projects/esp-idf/en/stable/esp32/api-reference/system/sleep_modes.html

To start Deep Sleep mode on the ESP32, use the esp_deep_sleep_start() function.

In addition, since this time we want to return from Deep-sleep mode by a specific elapsed time, the esp_sleep_enable_timer_wakeup the function is called just before the esp_deep_sleep_start() function.

Originally, I implemented a function to determine if it is within the speaking time (start time to end time) as follows, and if it is outside the speaking time, it skips the eye expression and voice playback, so I will implement the deep sleep mode function in this part

C++

tl::expected startTimeAction() {
  // 現在時刻の取得
  time_t now;
  time(&now);
  struct tm *now_tm = localtime(&now);
  auto &syncShadow = SyncShadow::getInstance();
  auto &logger = Logger::getInstance();
  auto &expressionService = ExpressionService::getInstance();

  auto reported = syncShadow.getReported();
  auto currentUnixTime = getNowUnixTime();

  // 設定した時刻の範囲外なら後続のアクションをスキップ
  if (!isWithinTalkTime(reported.config->talk_start_time, reported.config->talk_end_time, now_tm)) {
    // 設定した時刻の範囲外なら後続のアクションをスキップ
    Serial.println("out of talk time");
    return {};
  }
}

Here is the actual added Deep Sleep mode function (enterDeepSleepUntilNextStartTime() function)

The sleep time until the next talk_start_time is calculated and passed as an argument to the esp_sleep_enable_timer_wakeup function. esp_sleep_enable_timer_wakeup function arguments must be passed in microseconds.

C++

#include "esp_sleep.h"

// Deep Sleepから復帰したかどうかを確認
bool isWakeupFromDeepSleep() {
  return esp_sleep_get_wakeup_cause() == ESP_SLEEP_WAKEUP_TIMER;
}

// Deep Sleepに入る前の準備
void enterDeepSleepUntilNextStartTime(const std::optional &talk_start_time, struct tm *now_tm) {
  if (!talk_start_time) {
    return;
  }
  const int currentMinutes = now_tm->tm_hour * 60 + now_tm->tm_min;
  const int startMinutes = talk_start_time->hour.value_or(0) * 60 + talk_start_time->minute.value_or(0);

  int wakeupMinutes = startMinutes;
  if (currentMinutes >= startMinutes) {
    // 次の日の開始時刻までスリープする
    wakeupMinutes += 24 * 60;
  }
  const int sleepMinutes = wakeupMinutes - currentMinutes;
  const int sleepSeconds = sleepMinutes * 60;
  const uint64_t sleepMicroseconds = sleepSeconds * 1000000ULL;

  // Deep Sleepモードの時間を設定
  esp_sleep_enable_timer_wakeup(sleepMicroseconds);
  esp_deep_sleep_start();
}

Call the created enterDeepSleepUntilNextStartTime() function in the startTimeAction() function when outside the speaking time range.

C++

tl::expected startTimeAction() {
  // 現在時刻の取得
  time_t now;
  time(&now);
  struct tm *now_tm = localtime(&now);
  auto &syncShadow = SyncShadow::getInstance();
  auto &logger = Logger::getInstance();
  auto &expressionService = ExpressionService::getInstance();

  auto reported = syncShadow.getReported();
  auto currentUnixTime = getNowUnixTime();

  // 設定した時刻の範囲外なら後続のアクションをスキップ
  if (!isWithinTalkTime(reported.config->talk_start_time, reported.config->talk_end_time, now_tm)) {
    // 設定した時刻の範囲外なら後続のアクションをスキップ
    Serial.println("out of talk time");
    // Deep Sleepに入る
    enterDeepSleepUntilNextStartTime(reported.config->talk_start_time, now_tm);
    return {};
  }
 }

When I checked the operation, the display was still glowing white.

Okay, I thought this was okay, so I checked the operation, and, sure enough, I could confirm that deep sleep mode was called, but the display was still glowing white.

Upon closer examination, the ST7735S LCD display implemented on my current homebrew board has the pin that controls the LED backlight (No. 8 in the figure below) connected directly to 3.3V.

Therefore, even if the ESP32 enters Deep Sleep mode, the power supply to the backlight will continue and the screen will remain glowing white.

To solve this, instead of connecting the backlight pin directly to 3.3V, we need to connect the backlight pin to the GPIO pin of the ESP32 and change the software to control the backlight on/off, but since this involves a board change, we will try to do this at the time of ordering the next lot I will try to do it at the time of ordering the next lot.

So, I was going to describe an article on DEEP SLEEP MODE, but the original policy no longer works, so I will consider an alternative.

Fill screen to black + Light sleep mode

Even if the backlight cannot be physically turned off, it may be possible to make the display appear off by painting the screen black, so try this.

Considering power consumption, it should also be combined with light-sleep mode.

sleep_utils.h

C++

#pragma once

#include "json/device_config.h"
#include "json/device_shadow.h"
#include 
#include "esp_sleep.h"
#include "eye.h"

void enterLightSleepUntilNextStartTime(const std::optional &talk_start_time, struct tm *now_tm);

sleep_utils.cpp

C++

#include "sleep_utils.h"

// Light Sleepに入る前の準備
void enterLightSleepUntilNextStartTime(const std::optional &talk_start_time, struct tm *now_tm) {
  Serial.println("enterLightSleepUntilNextStartTime function called");
  if (!talk_start_time) {
    return;
  }
  const int currentMinutes = now_tm->tm_hour * 60 + now_tm->tm_min;
  const int startMinutes = talk_start_time->hour.value_or(0) * 60 + talk_start_time->minute.value_or(0);

  int wakeupMinutes = startMinutes;
  if (currentMinutes >= startMinutes) {
    // 次の日の開始時刻までスリープする
    wakeupMinutes += 24 * 60;
  }
  const int sleepMinutes = wakeupMinutes - currentMinutes;
  const int sleepSeconds = sleepMinutes * 60;
  const uint64_t sleepMicroseconds = sleepSeconds * 1000000ULL;

  // 画面を黒くする
  displayBlack();
  // 指定された時間（マイクロ秒）でタイマーウェイクアップを有効にする
  // esp_sleep_enable_timer_wakeup(sleepMicroseconds);
  // デバッグ用：15秒
  esp_sleep_enable_timer_wakeup(15 * 1000000ULL);
  esp_light_sleep_start();
}

This time, for debugging purposes, the system wakes up 15 seconds after entering light sleep mode so that the operation can be checked.

In the case of light sleep mode, the stage in which the eyes are constantly moving in normal mode becomes a still picture state that stops on the spot (in the case of deep sleep mode, the screen goes completely white), so just before entering light sleep mode, the displayBlack() function is used to to make the screen completely black.

eye.cpp

C++

void displayBlack() {
  Serial.println("display black function called");
  // 黒画面を表示
  for (uint8_t e = 0; e < NUM_EYES; e++) {
    digitalWrite(eye[e].tft_cs, LOW);
    tft.startWrite();
    tft.fillScreen(TFT_BLACK);
    tft.endWrite();
    digitalWrite(eye[e].tft_cs, HIGH);
  }
}

Call the created function in main.cpp.

When entering light sleep mode, wifi is automatically turned off, so wifi must be reconnected when waking up.

So, I used setupWiFi(ssid, password); to perform the Wi-Fi connection, and the initializeWiFiService() function to initialize the service to be performed after the Wi-Fi connection, specifically the MQTT communication.

main.cpp

C++


tl::expected<void, String> startTimeAction() {
  // 現在時刻の取得
  time_t now;
  time(&now);
  struct tm *now_tm = localtime(&now);
  auto &syncShadow = SyncShadow::getInstance();
  auto &logger = Logger::getInstance();
  auto &expressionService = ExpressionService::getInstance();

  auto reported = syncShadow.getReported();
  auto currentUnixTime = getNowUnixTime();

  // 設定した時刻の範囲外なら後続のアクションをスキップ
  if (!isWithinTalkTime(reported.config->talk_start_time, reported.config->talk_end_time, now_tm)) {
    // 設定した時刻の範囲外なら後続のアクションをスキップ
    Serial.println("out of talk time");
    enterLightSleepUntilNextStartTime(reported.config->talk_start_time, now_tm);
    return {};
  }

  // スリープから復帰した後の処理
  if (esp_sleep_get_wakeup_cause() == ESP_SLEEP_WAKEUP_TIMER) {
    Serial.println("Woke up from sleep");
    // Wi-Fi接続などの再初期化処理
    setupWiFi(ssid, password);
    delay(1000); // 接続後少し待つ
    initializeWiFiServices();
  }
}

operation check

In the application, set the end time of the talking time (in this case 12:37) and check if the screen goes black after that time.

The log was successfully called.

ShellScript

12:38:01.011 > out of talk time
12:38:01.012 > enterLightSleepUntilNextStartTime function called
12:38:01.014 > display black function called

We confirmed that the screen actually went black and 15 seconds later the eyes started moving again. We also confirmed that the Wi-Fi reconnection was working and MQTT communication was possible, as the home screen showed the text “Mia is online”.

Although it was not implemented in Deep Sleep Mode, the desired implementation was achieved by painting the screen black, so I thought the result was okay.

Finally, I have two corrections to make on the application.

A note in the talking time setting that after the end time, the device goes into sleep mode and the eye display turns black.
Error handling so that the end time cannot be set to a time before the start time.

talk_time_setting_frequency.dart

Dart

import 'package:clocky_app/api/user.dart';
import 'package:clocky_app/api/user_notifier.dart';
import 'package:clocky_app/models/hour_minute.dart';
import 'package:clocky_app/utils/picker_utils.dart';
import 'package:clocky_app/widgets/editable_field.dart';
import 'package:flutter/material.dart';
import 'package:flutter_riverpod/flutter_riverpod.dart';

class TalkTimeSettingScreen extends ConsumerStatefulWidget {
  const TalkTimeSettingScreen({super.key});

  @override
  _TalkTimeSettingScreenState createState() => _TalkTimeSettingScreenState();
}

class _TalkTimeSettingScreenState extends ConsumerState {
  HourMinute? startTime;
  HourMinute? endTime;

  @override
  void initState() {
    super.initState();
    final user = ref.read(userProvider);
    setState(() {
      startTime = user?.talkStartTime;
      endTime = user?.talkEndTime;
    });
  }

  void _showErrorDialog(String message) {
    showDialog(
      context: context,
      builder: (BuildContext context) {
        return AlertDialog(
          title: const Text('エラー'),
          content: Text(message),
          actions: [
            TextButton(
              child: const Text('OK'),
              onPressed: () {
                Navigator.of(context).pop();
              },
            ),
          ],
        );
      },
    );
  }

  @override
  Widget build(BuildContext context) {
    final userNotifier = ref.read(userProvider.notifier);
    return Scaffold(
      appBar: AppBar(
        title: const Text('話す時間の設定'),
      ),
      body: ListView(
        children: [
          ListTile(
            title: const Text('開始時刻'),
            trailing: EditableField(
              text: startTime?.toString() ?? '未設定',
              onTap: () {
                showPickerNumber(context, '開始時間', startTime, (selectedTime) {
                  if (endTime != null && selectedTime.isAfter(endTime!)) {
                    _showErrorDialog('開始時刻は終了時刻より前の時間を設定してください。');
                  } else {
                    userNotifier.updateUser(User(talkStartTime: selectedTime));
                    setState(() {
                      startTime = selectedTime;
                    });
                  }
                });
              },
            ),
          ),
          const Divider(color: Colors.grey, thickness: 0.5),
          ListTile(
            title: const Text('終了時刻'),
            trailing: EditableField(
              text: endTime?.toString() ?? '未設定',
              onTap: () {
                showPickerNumber(context, '終了時間', endTime, (selectedTime) {
                  if (startTime != null && selectedTime.isBefore(startTime!)) {
                    _showErrorDialog('終了時刻は開始時刻より後の時間を設定してください。');
                  } else {
                    userNotifier.updateUser(User(talkEndTime: selectedTime));
                    setState(() {
                      endTime = selectedTime;
                    });
                  }
                });
              },
            ),
          ),
          const ListTile(
            subtitle: Text(
              '仕事の時間を設定すると、ミーアは、その時刻に仕事に関連するフレーズをしゃべります。',
            ),
          ),
        ],
      ),
    );
  }
}