Parcourir la source

refactor: random_wake_up_answers played before starting order litenner. add manuall threshold. #359

nico il y a 7 ans
Parent
commit
e869c9d4d4

+ 67 - 18
Docs/settings.md

@@ -4,7 +4,7 @@ This part of the documentation explains the main configuration of Kalliope place
 
 ## Triggers configuration
 
-#### default_trigger
+### default_trigger
 
 The trigger is the module detecting the hotword that will wake up Kalliope.
 Common usage of hotword include Alexa on Amazon Echo, OK Google on some Android devices and Hey Siri on iPhones.
@@ -17,7 +17,7 @@ default_trigger: "trigger_name"
 Available triggers for Kalliope are:
 - snowboy
 
-#### triggers
+### triggers
 The hotword (also called a wake word or trigger word) detector is the engine in charge of waking up Kalliope.
 
 Each Trigger has it own configuration. This configuration is passed as argument following the syntax bellow
@@ -38,7 +38,7 @@ See the complete list of [available triggers here](trigger.md).
 
 ## Players configuration
 
-#### default_player
+### default_player
 
 The player is the module managing the sound in Kalliope.
 
@@ -51,7 +51,7 @@ E.g
 default_player: "mplayer"
 ```
 
-#### players
+### players
 The player is the engine in charge of running sounds in Kalliope.
 
 Each Players has it own configuration. 
@@ -91,7 +91,7 @@ Core players are already packaged with the installation of Kalliope an can be us
 
 ## Speech to text configuration
 
-#### default_speech_to_text
+### default_speech_to_text
 
 A Speech To Text(STT) is an engine used to translate what you say into a text that can be processed by Kalliope core.
 By default, Kalliope uses google STT engine.
@@ -108,7 +108,7 @@ default_speech_to_text: "google"
 
 Get the full list of [SST engine here](stt.md).
 
-#### speech_to_text
+### speech_to_text
 Each STT has it own configuration. This configuration is passed as argument as shown bellow
 ```yml
 speech_to_text:
@@ -126,9 +126,58 @@ speech_to_text:
 
 Some arguments are required, some others are optional, please refer to the [STT documentation](stt.md) to know available parameters for each supported STT.
 
+### stt_options
+
+Represents a collection of speech recognition settings and functionality.
+```yml
+stt_options:
+  option_name: option_value
+  option_name2: option_value2
+```
+
+E.g
+```yml
+stt_options:
+  energy_threshold: 3000
+```
+
+#### energy_threshold
+
+Represents the energy level threshold for sounds. By default set to **4000**.
+Values below this threshold are considered silence, and values above this threshold are considered speech.
+This is adjusted automatically if dynamic thresholds are enabled with `adjust_for_ambient_noise_second` parameter.
+
+This threshold is associated with the perceived loudness of the sound, but it is a nonlinear relationship. 
+The actual energy threshold you will need depends on your microphone sensitivity or audio data. 
+Typical values for a silent room are 0 to 100, and typical values for speaking are between 150 and 3500. 
+Ambient (non-speaking) noise has a significant impact on what values will work best.
+
+If you're having trouble with the recognizer trying to recognize words even when you're not speaking, try tweaking this to a higher value. 
+If you're having trouble with the recognizer not recognizing your words when you are speaking, try tweaking this to a lower value. 
+For example, a sensitive microphone or microphones in louder rooms might have a ambient energy level of up to 4000.
+```yml
+stt_options:
+  energy_threshold: 4000
+```
+
+>**Note:** The default value is 4000 if not set
+
+#### adjust_for_ambient_noise_second
+
+If defined, will adjusts the energy threshold dynamically by capturing the current ambient noise of the room during the number of second set in the parameter.
+When set, the `energy_threshold` parameter is overridden by the returned value of the noise calibration.
+This value should be at least 0.5 in order to get a representative sample of the ambient noise.
+
+```yml
+stt_options:
+  adjust_for_ambient_noise_second: 1
+```
+
+>**Note:** The number of second here represents the time between kalliope's awakening and the moment when you can give her your order.
+
 ## Text to speech configuration
 
-#### default_text_to_speech
+### default_text_to_speech
 A Text To Speech is an engine used to translate written text into a speech, into an audio stream.
 By default, Kalliope use Pico2wave TTS engine.
 
@@ -144,7 +193,7 @@ default_text_to_speech: "pico2wave"
 
 Get the full list of [TTS engine here](tts.md).
 
-#### text_to_speech
+### text_to_speech
 Each TTS has it own configuration. This configuration is passed as argument following the syntax bellow
 ```yml
 text_to_speech:
@@ -165,7 +214,7 @@ Some arguments are required, some other optional, please refer to the [TTS docum
 
 ## Wake up answers configuration
 
-#### random_wake_up_answers
+### random_wake_up_answers
 When Kalliope detects your trigger/hotword/magic word, it lets you know that it's operational and now waiting for order. It's done by answering randomly
 one of the sentences provided in the variable random_wake_up_answers.
 
@@ -187,7 +236,7 @@ random_wake_up_answers:
   - "Yes?"
 ```
 
-#### random_wake_up_sounds
+### random_wake_up_sounds
 You can play a sound when Kalliope detects the hotword/trigger instead of saying something from
 the `random_wake_up_answers`.
 Place here a list of full paths of the sound files you want to use. Otherwise, you can use some default sounds provided by Kalliope which you can find in `/usr/lib/kalliope/sounds`.
@@ -215,7 +264,7 @@ E.g: `# random_wake_up_answers:`
 ## On ready notification
 This section is used to notify the user when Kalliope is waiting for a trigger detection by playing a sound or speak a sentence out loud
 
-#### play_on_ready_notification
+### play_on_ready_notification
 This parameter define if you play the on ready notification:
  - `always`: every time Kalliope is ready to be awaken
  - `never`: never play a sound or sentences when kalliope is ready
@@ -225,7 +274,7 @@ E.g:
 ```yml
 play_on_ready_notification: always
 ```
-#### on_ready_answers
+### on_ready_answers
 The on ready notification can be a sentence. Place here a sentence or a list of sentence. If you set a list, one sentence will be picked up randomly
 
 E.g:
@@ -235,7 +284,7 @@ on_ready_answers:
   - "Waiting for order"
 ```
 
-#### on_ready_sounds
+### on_ready_sounds
 You can play a sound instead of a sentence.
 Remove the `on_ready_answers` parameters by commenting it out and use this one instead.
 Place here the path of the sound file. Files must be .wav or .mp3 format.
@@ -272,22 +321,22 @@ rest_api:
   allowed_cors_origin: "*"
 ```
 
-#### active
+### active
 To enable the rest api server.
 
-#### port
+### port
 The listening port of the web server. Must be an integer in range 1024-65535.
 
-#### password_protected
+### password_protected
 If `True`, the whole api will be password protected.
 
 #### Login
 Login used by the basic HTTP authentication. Must be provided if `password_protected` is `True`
 
-#### Password
+### Password
 Password used by the basic HTTP authentication. Must be provided if `password_protected` is `True`
 
-#### Cors request
+### Cors request
 If you want to allow request from external application, you'll need to enable the CORS requests settings by defining authorized origins.
 To do so, just indicated the origins that are allowed to leverage the API. The authorize values are:
 

+ 11 - 4
Tests/test_models.py

@@ -4,6 +4,7 @@ import mock
 
 from kalliope.core.Models.Player import Player
 from kalliope.core.Models.Signal import Signal
+from kalliope.core.Models.SttOptions import SttOptions
 from kalliope.core.Models.Tts import Tts
 
 from kalliope.core.Models.Trigger import Trigger
@@ -243,6 +244,8 @@ class TestModels(unittest.TestCase):
                                 active=True,
                                 port=5000, allowed_cors_origin="*")
 
+            stt_options = SttOptions()
+
             setting1 = Settings(default_tts_name="pico2wav",
                                 default_stt_name="google",
                                 default_trigger_name="swoyboy",
@@ -260,7 +263,8 @@ class TestModels(unittest.TestCase):
                                 cache_path="/tmp/kalliope",
                                 default_synapse="default_synapse",
                                 resources=None,
-                                variables={"key1": "val1"})
+                                variables={"key1": "val1"},
+                                stt_options=stt_options)
             setting1.kalliope_version = "0.4.5"
 
             setting2 = Settings(default_tts_name="accapela",
@@ -279,7 +283,8 @@ class TestModels(unittest.TestCase):
                                 cache_path="/tmp/kalliope_tmp",
                                 default_synapse="my_default_synapse",
                                 resources=None,
-                                variables={"key1": "val1"})
+                                variables={"key1": "val1"},
+                                stt_options=stt_options)
             setting2.kalliope_version = "0.4.5"
 
             setting3 = Settings(default_tts_name="pico2wav",
@@ -299,7 +304,8 @@ class TestModels(unittest.TestCase):
                                 cache_path="/tmp/kalliope",
                                 default_synapse="default_synapse",
                                 resources=None,
-                                variables={"key1": "val1"})
+                                variables={"key1": "val1"},
+                                stt_options=stt_options)
             setting3.kalliope_version = "0.4.5"
 
             expected_result_serialize = {
@@ -331,7 +337,8 @@ class TestModels(unittest.TestCase):
                 'resources': None,
                 'triggers': ['snowboy'],
                 'rpi_settings': None,
-                'players': ['mplayer']
+                'players': ['mplayer'],
+                'stt_options': {'energy_threshold': 4000, 'adjust_for_ambient_noise_second': 0}
             }
 
             self.assertDictEqual(expected_result_serialize, setting1.serialize())

+ 2 - 0
Tests/test_settings_loader.py

@@ -11,6 +11,7 @@ from kalliope.core.Models.Player import Player
 from kalliope.core.Models.RestAPI import RestAPI
 from kalliope.core.Models.Settings import Settings
 from kalliope.core.Models.Stt import Stt
+from kalliope.core.Models.SttOptions import SttOptions
 from kalliope.core.Models.Trigger import Trigger
 from kalliope.core.Models.Tts import Tts
 
@@ -119,6 +120,7 @@ class TestSettingLoader(unittest.TestCase):
             "test": "kalliope"
         }
         settings_object.machine = platform.machine()
+        settings_object.stt_options = SttOptions()
 
         sl = SettingLoader(file_path=self.settings_file_to_test)
 

+ 30 - 0
kalliope/core/ConfigurationManager/SettingLoader.py

@@ -3,6 +3,7 @@ import os
 from six import with_metaclass
 
 from kalliope.core.Models.RpiSettings import RpiSettings
+from kalliope.core.Models.SttOptions import SttOptions
 from .YAMLLoader import YAMLLoader
 from kalliope.core.Models.Resources import Resources
 from kalliope.core.Utils.Utils import Utils
@@ -118,6 +119,7 @@ class SettingLoader(with_metaclass(Singleton, object)):
         resources = self._get_resources(settings)
         variables = self._get_variables(settings)
         rpi_settings = self._get_rpi_settings(settings)
+        stt_options = self._get_stt_options(settings)
 
         # Load the setting singleton with the parameters
         setting_object.default_tts_name = default_tts_name
@@ -139,6 +141,7 @@ class SettingLoader(with_metaclass(Singleton, object)):
         setting_object.resources = resources
         setting_object.variables = variables
         setting_object.rpi_settings = rpi_settings
+        setting_object.stt_options = stt_options
 
         return setting_object
 
@@ -784,3 +787,30 @@ class SettingLoader(with_metaclass(Singleton, object)):
         except KeyError:
             logger.debug("[SettingsLoader] No Rpi config")
             return None
+
+    @staticmethod
+    def _get_stt_options(settings):
+        """
+        return the value of stt_threshold
+        :param settings: The loaded YAML settings file
+        :return: integer or 1200 by default if not set
+        """
+        stt_options = SttOptions()
+
+        try:
+            stt_options_dict = settings["stt_options"]
+
+            if "energy_threshold" in stt_options_dict:
+                stt_options.energy_threshold = stt_options_dict["energy_threshold"]
+                logger.debug("[SettingsLoader] energy_threshold set to %s" % stt_options.energy_threshold)
+            if "adjust_for_ambient_noise_second" in stt_options_dict:
+                stt_options.adjust_for_ambient_noise_second = stt_options_dict["adjust_for_ambient_noise_second"]
+                logger.debug("[SettingsLoader] adjust_for_ambient_noise_second set to %s"
+                             % stt_options.adjust_for_ambient_noise_second)
+            return stt_options
+
+        except KeyError:
+            logger.debug("[SettingsLoader] no stt_options defined. Set to default")
+
+        logger.debug("[SettingsLoader] stt_options: %s" % str(stt_options))
+        return stt_options

+ 5 - 2
kalliope/core/Models/Settings.py

@@ -27,7 +27,8 @@ class Settings(object):
                  default_synapse=None,
                  resources=None,
                  variables=None,
-                 rpi_settings=None):
+                 rpi_settings=None,
+                 stt_options=None):
 
         self.default_tts_name = default_tts_name
         self.default_stt_name = default_stt_name
@@ -50,6 +51,7 @@ class Settings(object):
         self.machine = platform.machine()   # can be x86_64 or armv7l
         self.kalliope_version = current_kalliope_version
         self.rpi_settings = rpi_settings
+        self.stt_options = stt_options
 
     def serialize(self):
         """
@@ -80,7 +82,8 @@ class Settings(object):
             'variables': self.variables,
             'machine': self.machine,
             'kalliope_version': self.kalliope_version,
-            'rpi_settings': self.rpi_settings.serialize() if self.rpi_settings is not None else None
+            'rpi_settings': self.rpi_settings.serialize() if self.rpi_settings is not None else None,
+            'stt_options': self.stt_options.serialize() if self.stt_options is not None else None,
         }
 
     def __str__(self):

+ 27 - 0
kalliope/core/Models/SttOptions.py

@@ -0,0 +1,27 @@
+class SttOptions(object):
+    """
+    This Class is representing a Speech To Text (STT) element with name and parameters
+
+    .. note:: must be defined in the settings.yml
+    """
+
+    def __init__(self, energy_threshold=4000, adjust_for_ambient_noise_second=0):
+        self.energy_threshold = energy_threshold
+        self.adjust_for_ambient_noise_second = adjust_for_ambient_noise_second
+
+    def __str__(self):
+        return str(self.serialize())
+
+    def serialize(self):
+        return {
+            'energy_threshold': self.energy_threshold,
+            'adjust_for_ambient_noise_second': self.adjust_for_ambient_noise_second
+        }
+
+    def __eq__(self, other):
+        """
+        This is used to compare 2 objects
+        :param other:
+        :return:
+        """
+        return self.__dict__ == other.__dict__

+ 5 - 0
kalliope/settings.yml

@@ -25,6 +25,11 @@ triggers:
 # This is the STT that will be used by default
 default_speech_to_text: "google"
 
+# Speech to text options
+#stt_options:
+#  energy_threshold: 4000
+#  adjust_for_ambient_noise_second: 1
+
 # Speech to Text engines configuration
 # Available engine are:
 # - google (via SpeechRecognition)

+ 3 - 3
kalliope/signals/order/order.py

@@ -29,8 +29,8 @@ class Order(Thread):
               'playing_ready_sound',
               'waiting_for_trigger_callback',
               'stopping_trigger',
-              'start_order_listener',
               'playing_wake_up_answer',
+              'start_order_listener',
               'waiting_for_order_listener_callback',
               'analysing_order']
 
@@ -71,9 +71,9 @@ class Order(Thread):
         self.machine.add_transition('play_ready_sound', 'starting_trigger', 'playing_ready_sound')
         self.machine.add_transition('wait_trigger_callback', 'playing_ready_sound', 'waiting_for_trigger_callback')
         self.machine.add_transition('stop_trigger', 'waiting_for_trigger_callback', 'stopping_trigger')
-        self.machine.add_transition('play_wake_up_answer', 'waiting_for_trigger_callback', 'playing_wake_up_answer')
+        self.machine.add_transition('play_wake_up_answer', 'stopping_trigger', 'playing_wake_up_answer')
         self.machine.add_transition('wait_for_order', 'playing_wake_up_answer', 'waiting_for_order_listener_callback')
-        self.machine.add_transition('analyse_order', 'waiting_for_order_listener_callback', 'analysing_order')
+        self.machine.add_transition('analyse_order', 'playing_wake_up_answer', 'analysing_order')
 
         self.machine.add_ordered_transitions()
 

+ 17 - 2
kalliope/stt/Utils.py

@@ -32,8 +32,23 @@ class SpeechRecognition(Thread):
         if audio_file is None:
             # audio file not set, we need to capture a sample from the microphone
             with self.microphone as source:
-                # we only need to calibrate once, before we start listening
-                self.recognizer.adjust_for_ambient_noise(source)
+                if self.settings.stt_options.adjust_for_ambient_noise_second > 0:
+                    # threshold is calculated from capturing ambient sound
+                    logger.debug("[SpeechRecognition] threshold calculated by "
+                                 "capturing ambient noise during %s seconds" %
+                                 self.settings.stt_options.adjust_for_ambient_noise_second)
+                    Utils.print_info("[SpeechRecognition] capturing ambient sound during %s seconds" %
+                                     self.settings.stt_options.adjust_for_ambient_noise_second)
+                    self.recognizer.adjust_for_ambient_noise(source,
+                                                             duration=self.settings.
+                                                             stt_options.adjust_for_ambient_noise_second)
+                else:
+                    # threshold is defined manually
+                    logger.debug("[SpeechRecognition] threshold defined by settings: %s" %
+                                 self.settings.stt_options.energy_threshold)
+                    self.recognizer.energy_threshold = self.settings.stt_options.energy_threshold
+
+                Utils.print_info("Threshold set to: %s" % self.recognizer.energy_threshold)
         else:
             # audio file provided
             with sr.AudioFile(audio_file) as source: