Browse Source

Merge pull request #216 from kalliope-project/audio_api

Rest API Review
Monf 8 years ago
parent
commit
40b9e55cb3

+ 6 - 5
Docs/contributing/contribute_stt.md

@@ -37,14 +37,15 @@ The constructor has a __**kwargs argument__ which is corresponding to the Dict o
 1. Attach the incoming callback to the self.main_controller_callback attribute. This callback come from the main controller and will receive the text at the end of the process
 1. Obtain audio from the microphone in the constructor. (Note : we mostly use the [speech_recognition library](https://pypi.python.org/pypi/SpeechRecognition/))
 1. Set your callback method into the mother class with `self.set_callback(self.google_callback)`. This callback is the one which will process the audio into a text.
-1. Use self.start_listening() from the mother class to get an audio. Once caught, the mother class will give the audio stream to the callback you've set before.
-1. The callback method must implement two arguments: recognizer and audio. The audio argument contains the stream caught by the microphone
+1. Use self.start_processing() from the mother class to get an audio from the microphone or read the audio file path if provided. The mother class will give the audio stream to the callback you've set before.
+1. The callback method must implement two arguments: recognizer and audio. The audio argument contains the stream caught by the microphone or read from an audio file path
 1. Do magic stuff with the audio in order to get a string that contains the translated text
 1. Once you get the text, let give it to the main_controller_callback method received in the constructor by calling it with the text string as argument `self.main_controller_callback(audio_to_text)`
 
     ```python
     def __init__(self, callback=None, **kwargs):
-        OrderListener.__init__(self)
+        # give the audio file path to process directly to the mother class if exist
+        SpeechRecognition.__init__(self, kwargs.get('audio_file_path', None))
         # here is the main controller callback. We will return the text at the end of the process
         self.main_controller_callback = callback
         
@@ -52,8 +53,8 @@ The constructor has a __**kwargs argument__ which is corresponding to the Dict o
         
         #  give our callback   
         self.set_callback(self.my_callback)
-        # start the microphone to capture an audio
-        self.start_listening()
+        # start processing, record a sample from the microphone if no audio file path provided, else read the file
+        self.start_processing()
         
         def my_callback((self, recognizer, audio):
             # ---------------------------------------------

+ 71 - 11
Docs/rest_api.md

@@ -4,24 +4,41 @@ Kalliope provides the REST API to manage the synapses. For configuring the API r
 
 ## Synapse API
 
-| Method | URL                      | Action                      |
-|--------|--------------------------|-----------------------------|
-| GET    | /synapses                | List synapses               |
-| GET    | /synapses/<synapse_name> | Show synapse details        |
-| POST   | /synapses/<synapse_name> | Run a synapse by its name   |
-| POST   | /order                   | Run a synapse from an order |
+| Method | URL                               | Action                             |
+|--------|-----------------------------------|------------------------------------|
+| GET    | /                                 | Get kaliope version                |
+| GET    | /synapses                         | List synapses                      |
+| GET    | /synapses/<synapse_name>          | Get synapse details by name        |
+| POST   | /synapses/start/id/<synapse_name> | Run a synapse by its name          |
+| POST   | /synapses/start/order             | Run a synapse from a text order    |
+| POST   | /synapses/start/audio             | Run a synapse from an audio sample |
 
 ## Curl examples
 
 >**Note:** --user is only needed if `password_protected` is True
 
+### Get Kalliope's version
+
+Normal response codes: 200
+Error response codes: unauthorized(401)
+Curl command:
+```bash
+curl -i --user admin:secret -X GET  http://localhost:5000/
+```
+Output example:
+```JSON
+{
+  "Kalliope version": "0.4.2"
+}
+```
+
 ### List synapses
 
 Normal response codes: 200
 Error response codes: unauthorized(401), itemNotFound(404)
 Curl command:
 ```bash
-curl -i --user admin:secret -X GET  http://localhost:5000/synapses/
+curl -i --user admin:secret -X GET  http://localhost:5000/synapses
 ```
 
 Output example:
@@ -107,7 +124,7 @@ Normal response codes: 201
 Error response codes: unauthorized(401), itemNotFound(404)
 Curl command:
 ```bash
-curl -i --user admin:secret -X POST  http://localhost:5000/synapses/say-hello
+curl -i --user admin:secret -X POST  http://localhost:5000/synapses/start/id/say-hello
 ```
 
 Output example:
@@ -141,7 +158,7 @@ Error response codes: unauthorized(401), itemNotFound(404)
 
 Curl command:
 ```bash
-curl -i --user admin:secret -H "Content-Type: application/json" -X POST -d '{"order":"my order"}' http://localhost:5000/order/
+curl -i --user admin:secret -H "Content-Type: application/json" -X POST -d '{"order":"my order"}' http://localhost:5000/synapses/start/order
 ```
 
 If the order contains accent or quotes, use a file for testing with curl
@@ -151,7 +168,7 @@ cat post.json
 ```
 Then
 ```bash
-curl -i --user admin:secret -H "Content-Type: application/json" -X POST --data @post.json http://localhost:5000/order/
+curl -i --user admin:secret -H "Content-Type: application/json" -X POST --data @post.json http://localhost:5000/synapses/start/order
 ```
 
 Output example if the order have matched and so launched synapses:
@@ -176,7 +193,50 @@ Output example if the order have matched and so launched synapses:
 }
 ```
 
-If the order haven't match ny synapses:
+If the order haven't match any synapses:
+```JSON
+{
+  "error": {
+    "error": "The given order doesn't match any synapses"
+  }
+}
+```
+
+### Run a synapse from an audio file
+
+Normal response codes: 201
+Error response codes: unauthorized(401), itemNotFound(404)
+
+The audio file must use WAV or MP3 extension.
+
+Curl command:
+```bash
+curl -i --user admin:secret -X POST  http://localhost:5000/synapses/start/audio -F "file=@/home/nico/Desktop/input.wav"
+```
+
+Output example if the order inside the audio have matched and so launched synapses:
+```JSON
+{
+  "synapses": [
+    {
+      "name": "Say-hello", 
+      "neurons": [
+        {
+          "name": "say", 
+          "parameters": "{'message': ['Hello sir']}"
+        }
+      ], 
+      "signals": [
+        {
+          "order": "hello"
+        }
+      ]
+    }
+  ]
+}
+```
+
+If the order haven't match any synapses:
 ```JSON
 {
   "error": {

+ 19 - 0
Tests/brains/brain_test_api.yml

@@ -0,0 +1,19 @@
+---
+  - name: "test"
+    signals:
+      - order: "test_order"
+    neurons:
+      - say:
+          message:
+            - "test message"
+
+  - name: "test2"
+    signals:
+      - order: "bonjour"
+    neurons:
+      - say:
+          message:
+            - "test message"
+
+  - includes:
+    - included_brain_test.yml

BIN
Tests/files/bonjour.wav


+ 244 - 183
Tests/test_rest_api.py

@@ -1,11 +1,14 @@
+import json
 import os
 import unittest
 
+from werkzeug.datastructures import FileStorage
+
 from flask import Flask
 from flask_testing import LiveServerTestCase
 
 from kalliope.core.Models import Singleton
-
+from kalliope._version import version_str
 from kalliope.core.ConfigurationManager import BrainLoader
 from kalliope.core.ConfigurationManager import SettingLoader
 from kalliope.core.RestAPI.FlaskAPI import FlaskAPI
@@ -20,8 +23,12 @@ class TestRestAPI(LiveServerTestCase):
         # be sure that the singleton haven't been loaded before
         Singleton._instances = {}
         current_path = os.getcwd()
-        full_path_brain_to_test = current_path + os.sep + "Tests/brains/brain_test.yml"
-        print full_path_brain_to_test
+        if "/Tests" in os.getcwd():
+            full_path_brain_to_test = current_path + os.sep + "brains/brain_test_api.yml"
+            self.audio_file = "files/bonjour.wav"
+        else:
+            full_path_brain_to_test = current_path + os.sep + "Tests/brains/brain_test_api.yml"
+            self.audio_file = "Tests/files/bonjour.wav"
 
         # rest api config
         sl = SettingLoader()
@@ -36,196 +43,250 @@ class TestRestAPI(LiveServerTestCase):
         brain = brain_loader.brain
 
         self.app = Flask(__name__)
+        self.app.config['TESTING'] = True
         self.flask_api = FlaskAPI(self.app, port=5000, brain=brain)
-        self.flask_api.app.config['TESTING'] = True
+        self.client = self.app.test_client()
         return self.flask_api.app
 
-    # TODO all following test passes with 'python -m unittest Tests.TestRestAPI' but not with discover
-    # def test_get_all_synapses(self):
-    #     url = "http://127.0.0.1:5000/synapses"
-    #
-    #     result = requests.get(url=url)
-    #     expected_content = {
-    #         "synapses": [
-    #             {
-    #                 "name": "test",
-    #                 "neurons": [
-    #                     {
-    #                         "say": {
-    #                             "message": [
-    #                                 "test message"
-    #                             ]
-    #                         }
-    #                     }
-    #                 ],
-    #                 "signals": [
-    #                     {
-    #                         "order": "test_order"
-    #                     }
-    #                 ]
-    #             },
-    #             {
-    #                 "name": "test2",
-    #                 "neurons": [
-    #                     {
-    #                         "say": {
-    #                             "message": [
-    #                                 "test message"
-    #                             ]
-    #                         }
-    #                     }
-    #                 ],
-    #                 "signals": [
-    #                     {
-    #                         "order": "test_order_2"
-    #                     }
-    #                 ]
-    #             },
-    #             {
-    #                 "includes": [
-    #                     "included_brain_test.yml"
-    #                 ]
-    #             },
-    #             {
-    #                 "name": "test3",
-    #                 "neurons": [
-    #                     {
-    #                         "say": {
-    #                             "message": [
-    #                                 "test message"
-    #                             ]
-    #                         }
-    #                     }
-    #                 ],
-    #                 "signals": [
-    #                     {
-    #                         "order": "test_order_3"
-    #                     }
-    #                 ]
-    #             }
-    #         ]
-    #     }
-    #
-    #     self.assertEqual(result.status_code, 200)
-    #     self.assertEqual(expected_content, json.loads(result.content))
-    #
-    # def test_get_one_synapse(self):
-    #     url = "http://127.0.0.1:5000/synapses/test"
-    #     result = requests.get(url=url)
-    #
-    #     expected_content = {
-    #         "synapses": {
-    #             "name": "test",
-    #             "neurons": [
-    #                 {
-    #                     "say": {
-    #                         "message": [
-    #                             "test message"
-    #                         ]
-    #                     }
-    #                 }
-    #             ],
-    #             "signals": [
-    #                 {
-    #                     "order": "test_order"
-    #                 }
-    #             ]
-    #         }
-    #     }
-    #
-    #     self.assertEqual(expected_content, json.loads(result.content))
-    #
-    # def test_get_synapse_not_found(self):
-    #     url = "http://127.0.0.1:5000/synapses/test-none"
-    #     result = requests.get(url=url)
-    #
-    #     expected_content = {
-    #         "error": {
-    #             "synapse name not found": "test-none"
+    def test_server_is_up_and_running(self):
+        # response = urllib2.urlopen(self.get_server_url())
+        response = self.client.get(self.get_server_url())
+        self.assertEqual(response.status_code, 200)
+
+    def test_get_main_page(self):
+        url = self.get_server_url() + "/"
+        response = self.client.get(url)
+        expected_content = {
+            "Kalliope version": "%s" % version_str
+        }
+        self.assertEqual(json.dumps(expected_content), json.dumps(json.loads(response.get_data())))
+
+    def test_get_all_synapses(self):
+        url = self.get_server_url()+"/synapses"
+
+        response = self.client.get(url)
+        expected_content = {
+            "synapses": [
+                {
+                    "name": "test",
+                    "neurons": [
+                        {
+                            "name": "say",
+                            "parameters": {
+                                "message": [
+                                    "test message"
+                                ]
+                            }
+                        }
+                    ],
+                    "signals": [
+                        {
+                            "order": "test_order"
+                        }
+                    ]
+                },
+                {
+                    "name": "test2",
+                    "neurons": [
+                        {
+                            "name": "say",
+                            "parameters": {
+                                "message": [
+                                    "test message"
+                                ]
+                            }
+                        }
+                    ],
+                    "signals": [
+                        {
+                            "order": "bonjour"
+                        }
+                    ]
+                },
+                {
+                    "name": "test3",
+                    "neurons": [
+                        {
+                            "name": "say",
+                            "parameters": {
+                                "message": [
+                                    "test message"
+                                ]
+                            }
+                        }
+                    ],
+                    "signals": [
+                        {
+                            "order": "test_order_3"
+                        }
+                    ]
+                }
+            ]
+        }
+        # a lot of char ti process
+        self.maxDiff = None
+        self.assertEqual(response.status_code, 200)
+        # print response.get_data()
+        # print json.dumps(expected_content)
+        self.assertEqual(json.dumps(expected_content), json.dumps(json.loads(response.get_data())))
+
+    def test_get_one_synapse(self):
+        url = self.get_server_url() + "/synapses/test"
+        response = self.client.get(url)
+
+        expected_content = {
+            "synapses": {
+                "name": "test",
+                "neurons": [
+                    {
+                        "name": "say",
+                        "parameters": {
+                            "message": [
+                                "test message"
+                            ]
+                        }
+                    }
+                ],
+                "signals": [
+                    {
+                        "order": "test_order"
+                    }
+                ]
+            }
+        }
+        self.assertEqual(json.dumps(expected_content), json.dumps(json.loads(response.get_data())))
+
+    def test_get_synapse_not_found(self):
+        url = self.get_server_url() + "/synapses/test-none"
+        result = self.client.get(url)
+
+        expected_content = {
+            "error": {
+                "synapse name not found": "test-none"
+            }
+        }
+
+        self.assertEqual(expected_content, json.loads(result.get_data()))
+        self.assertEqual(result.status_code, 404)
+
+    def test_run_synapse_by_name(self):
+        url = self.get_server_url() + "/synapses/start/id/test"
+        result = self.client.post(url)
+
+        expected_content = {
+            "synapses": {
+                "name": "test",
+                "neurons": [
+                    {
+                        "name": "say",
+                        "parameters": {
+                            "message": [
+                                "test message"
+                            ]
+                        }
+                    }
+                ],
+                "signals": [
+                    {
+                        "order": "test_order"
+                    }
+                ]
+            }
+        }
+        self.assertEqual(json.dumps(expected_content), json.dumps(json.loads(result.get_data())))
+        self.assertEqual(result.status_code, 201)
+
+    def test_post_synapse_not_found(self):
+        url = self.get_server_url() + "/synapses/start/id/test-none"
+        result = self.client.post(url)
+
+        expected_content = {
+            "error": {
+                "synapse name not found": "test-none"
+            }
+        }
+
+        self.assertEqual(json.dumps(expected_content), json.dumps(json.loads(result.get_data())))
+        self.assertEqual(result.status_code, 404)
+
+    def test_run_synapse_with_order(self):
+        url = self.get_server_url() + "/synapses/start/order"
+        headers = {"Content-Type": "application/json"}
+        data = {"order": "test_order"}
+        result = self.client.post(url, headers=headers, data=json.dumps(data))
+
+        expected_content = {
+            "synapses": [
+                {
+                    "name": "test",
+                    "neurons": [
+                        {
+                            "name": "say",
+                            "parameters": {
+                                "message": [
+                                    "test message"
+                                ]
+                            }
+                        }
+                    ],
+                    "signals": [
+                        {
+                            "order": "test_order"
+                        }
+                    ]
+                }
+            ]
+        }
+        print result.get_data()
+        self.assertEqual(json.dumps(expected_content), json.dumps(json.loads(result.get_data())))
+        self.assertEqual(result.status_code, 201)
+
+    def test_post_synapse_by_order_not_found(self):
+        url = self.get_server_url() + "/synapses/start/order"
+        data = {"order": "non existing order"}
+        headers = {"Content-Type": "application/json"}
+        result = self.client.post(url, headers=headers, data=json.dumps(data))
+
+        expected_content = {'error': {'error': "The given order doesn't match any synapses"}}
+
+        self.assertEqual(json.dumps(expected_content), json.dumps(json.loads(result.get_data())))
+        self.assertEqual(result.status_code, 400)
+
+    # TODO this doesn't work on travis but works locally with python -m unittest discover
+    # def test_post_synapse_by_audio(self):
+    #     url = self.get_server_url() + "/synapses/start/audio"
+    #     with open(os.path.join(self.audio_file), 'rb') as fp:
+    #         file = FileStorage(fp)
+    #         data = {
+    #             'file': file
     #         }
-    #     }
+    #         result = self.client.post(url, data=data, content_type='multipart/form-data')
     #
-    #     self.assertEqual(expected_content, json.loads(result.content))
-    #     self.assertEqual(result.status_code, 404)
-    #
-    # def test_run_synapse_by_name(self):
-    #     url = "http://127.0.0.1:5000/synapses/test"
-    #     result = requests.post(url=url)
-    #
-    #     expected_content = {
-    #         "synapses": {
-    #             "name": "test",
-    #             "neurons": [
-    #                 {
-    #                     "say": {
-    #                         "message": [
-    #                             "test message"
-    #                         ]
-    #                     }
-    #                 }
-    #             ],
-    #             "signals": [
+    #         expected_content = {
+    #             "synapses": [
     #                 {
-    #                     "order": "test_order"
+    #                     "name": "test2",
+    #                     "neurons": [
+    #                         {
+    #                             "name": "say",
+    #                             "parameters": {
+    #                                 "message": [
+    #                                     "test message"
+    #                                 ]
+    #                             }
+    #                         }
+    #                     ],
+    #                     "signals": [
+    #                         {
+    #                             "order": "bonjour"
+    #                         }
+    #                     ]
     #                 }
     #             ]
     #         }
-    #     }
-    #
-    #     self.assertEqual(expected_content, json.loads(result.content))
-    #     self.assertEqual(result.status_code, 201)
-    #
-    # def test_post_synapse_not_found(self):
-    #     url = "http://127.0.0.1:5000/synapses/test-none"
-    #     result = requests.post(url=url)
-    #
-    #     expected_content = {
-    #         "error": {
-    #             "synapse name not found": "test-none"
-    #         }
-    #     }
-    #
-    #     self.assertEqual(expected_content, json.loads(result.content))
-    #     self.assertEqual(result.status_code, 404)
-    #
-    # def test_run_synapse_with_order(self):
-    #     url = "http://127.0.0.1:5000/order/"
-    #     headers = {"Content-Type": "application/json"}
-    #     data = {"order": "test_order"}
-    #     result = requests.post(url=url, headers=headers, json=data)
-    #
-    #     expected_content = {
-    #         "synapses": [
-    #             {
-    #                 "name": "test",
-    #                 "neurons": [
-    #                     {
-    #                         "name": "say",
-    #                         "parameters": "{'message': ['test message']}"
-    #                     }
-    #                 ],
-    #                 "signals": [
-    #                     {
-    #                         "order": "test_order"
-    #                     }
-    #                 ]
-    #             }
-    #         ]
-    #     }
-    #
-    #     self.assertEqual(expected_content, json.loads(result.content))
-    #     self.assertEqual(result.status_code, 201)
-    #
-    # def test_post_synapse_by_order_not_found(self):
-    #     url = "http://127.0.0.1:5000/order/"
-    #     data = {"order": "non existing order"}
-    #     headers = {"Content-Type": "application/json"}
-    #     result = requests.post(url=url, headers=headers, json=data)
-    #
-    #     expected_content = {'error': {'error': "The given order doesn't match any synapses"}}
     #
-    #     self.assertEqual(expected_content, json.loads(result.content))
-    #     self.assertEqual(result.status_code, 400)
+    #         self.assertEqual(json.dumps(expected_content), json.dumps(json.loads(result.get_data())))
+    #         self.assertEqual(result.status_code, 201)
 
 if __name__ == '__main__':
     unittest.main()

+ 1 - 1
install/files/python_requirements.txt

@@ -14,7 +14,7 @@ flask_cors==3.0.2
 requests==2.12.4
 httpretty==0.8.14
 mock==2.0.0
-Flask-Testing==0.6.1
+Flask-Testing>=0.6.1
 apscheduler==3.3.0
 GitPython==2.1.1
 packaging>=16.8

+ 5 - 1
kalliope/core/OrderListener.py

@@ -19,7 +19,7 @@ class OrderListener(Thread):
         starting to listen the incoming order. Basically it avoids delays.
     """
 
-    def __init__(self, callback=None, stt=None):
+    def __init__(self, callback=None, stt=None, audio_file_path=None):
         """
         This class is called after we catch the hotword that has woken up Kalliope.
         We now wait for an order spoken out loud by the user, translate the order into a text and run the action
@@ -42,6 +42,7 @@ class OrderListener(Thread):
         sl = SettingLoader()
         self.settings = sl.settings
         self.stt_instance = None
+        self.audio_file_path = audio_file_path
 
     def run(self):
         """
@@ -56,6 +57,9 @@ class OrderListener(Thread):
         for stt_object in self.settings.stts:
             if stt_object.name == self.stt_module_name:
                 stt_object.parameters["callback"] = self.callback
+                # add the audio file path to the list of parameter if set
+                if self.audio_file_path is not None:
+                    stt_object.parameters["audio_file_path"] = self.audio_file_path
 
                 stt_folder = None
                 if self.settings.resources:

+ 123 - 8
kalliope/core/RestAPI/FlaskAPI.py

@@ -1,6 +1,15 @@
 import logging
+import os
 import threading
 
+import time
+
+from kalliope.core.Utils.FileManager import FileManager
+
+from kalliope.core.ConfigurationManager import SettingLoader
+from kalliope.core.OrderListener import OrderListener
+from werkzeug.utils import secure_filename
+
 from flask import jsonify
 from flask import request
 from flask_restful import abort
@@ -9,10 +18,15 @@ from flask_cors import CORS, cross_origin
 from kalliope.core import OrderAnalyser
 from kalliope.core.RestAPI.utils import requires_auth
 from kalliope.core.SynapseLauncher import SynapseLauncher
+from kalliope._version import version_str
 
 logging.basicConfig()
 logger = logging.getLogger("kalliope")
 
+UPLOAD_FOLDER = '/tmp/kalliope/tmp_uploaded_audio'
+ALLOWED_EXTENSIONS = {'mp3', 'wav'}
+
+
 class FlaskAPI(threading.Thread):
     def __init__(self, app, port=5000, brain=None, allowed_cors_origin=False):
         """
@@ -28,6 +42,20 @@ class FlaskAPI(threading.Thread):
         self.brain = brain
         self.allowed_cors_origin = allowed_cors_origin
 
+        # get current settings
+        sl = SettingLoader()
+        self.settings = sl.settings
+
+        # list of launched synapse by the Order Analyser when using the /synapses/start/audio URL
+        self.launched_synapses = None
+        # boolean used to notify the main process that we get the list of returned synapse
+        self.order_analyser_return = False
+
+        # configure the upload folder
+        app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDER
+        # create the temp folder
+        FileManager.create_directory(UPLOAD_FOLDER)
+
         # Flask configuration remove default Flask behaviour to encode to ASCII
         self.app.url_map.strict_slashes = False
         self.app.config['JSON_AS_ASCII'] = False
@@ -36,16 +64,28 @@ class FlaskAPI(threading.Thread):
             cors = CORS(app, resources={r"/*": {"origins": allowed_cors_origin}}, supports_credentials=True)
 
         # Add routing rules
+        self.app.add_url_rule('/', view_func=self.get_main_page, methods=['GET'])
         self.app.add_url_rule('/synapses', view_func=self.get_synapses, methods=['GET'])
         self.app.add_url_rule('/synapses/<synapse_name>', view_func=self.get_synapse, methods=['GET'])
-        self.app.add_url_rule('/synapses/<synapse_name>', view_func=self.run_synapse, methods=['POST'])
-        self.app.add_url_rule('/order/', view_func=self.run_order, methods=['POST'])
+        self.app.add_url_rule('/synapses/start/id/<synapse_name>', view_func=self.run_synapse_by_name, methods=['POST'])
+        self.app.add_url_rule('/synapses/start/order', view_func=self.run_synapse_by_order, methods=['POST'])
+        self.app.add_url_rule('/synapses/start/audio', view_func=self.run_synapse_by_audio, methods=['POST'])
         self.app.add_url_rule('/shutdown/', view_func=self.shutdown_server, methods=['POST'])
 
-
     def run(self):
         self.app.run(host='0.0.0.0', port="%s" % int(self.port), debug=True, threaded=True, use_reloader=False)
 
+    def get_main_page(self):
+        data = {
+            "Kalliope version": "%s" % version_str
+        }
+        return jsonify(data), 200
+
+    @staticmethod
+    def allowed_file(filename):
+        return '.' in filename and \
+               filename.rsplit('.', 1)[1].lower() in ALLOWED_EXTENSIONS
+
     def _get_synapse_by_name(self, synapse_name):
         """
         Find a synapse in the brain by its name
@@ -65,6 +105,8 @@ class FlaskAPI(threading.Thread):
     def get_synapses(self):
         """
         get all synapses.
+        test with curl:
+        curl -i --user admin:secret  -X GET  http://127.0.0.1:5000/synapses
         """
         data = jsonify(synapses=[e.serialize() for e in self.brain.synapses])
         return data, 200
@@ -73,6 +115,8 @@ class FlaskAPI(threading.Thread):
     def get_synapse(self, synapse_name):
         """
         get a synapse by its name
+        test with curl:
+        curl --user admin:secret -i -X GET  http://127.0.0.1:5000/synapses/say-hello-en
         """
         synapse_target = self._get_synapse_by_name(synapse_name)
         if synapse_target is not None:
@@ -85,11 +129,11 @@ class FlaskAPI(threading.Thread):
         return jsonify(error=data), 404
 
     @requires_auth
-    def run_synapse(self, synapse_name):
+    def run_synapse_by_name(self, synapse_name):
         """
         Run a synapse by its name
         test with curl:
-        curl -i --user admin:secret -X POST  http://localhost:5000/synapses/say-hello
+        curl -i --user admin:secret -X POST  http://127.0.0.1:5000/synapses/start/id/say-hello-fr
         :param synapse_name:
         :return:
         """
@@ -103,15 +147,15 @@ class FlaskAPI(threading.Thread):
 
         # run the synapse
         SynapseLauncher.start_synapse(synapse_name, brain=self.brain)
-        data = jsonify(synapses=synapse_target)
+        data = jsonify(synapses=synapse_target.serialize())
         return data, 201
 
     @requires_auth
-    def run_order(self):
+    def run_synapse_by_order(self):
         """
         Give an order to Kalliope via API like it was from a spoken one
         Test with curl
-        curl -i --user admin:secret -H "Content-Type: application/json" -X POST -d '{"order":"my order"}' http://localhost:5000/order
+        curl -i --user admin:secret -H "Content-Type: application/json" -X POST -d '{"order":"my order"}' http://localhost:5000/synapses/start/order
         In case of quotes in the order or accents, use a file
         cat post.json:
         {"order":"j'aime"}
@@ -144,6 +188,53 @@ class FlaskAPI(threading.Thread):
             }
             return jsonify(error=data), 400
 
+    def run_synapse_by_audio(self):
+        """
+        Give an order to Kalliope with an audio file
+        Test with curl
+        curl -i --user admin:secret -X POST  http://localhost:5000/synapses/start/audio -F "file=@/path/to/input.wav"
+        :return:
+        """
+        # check if the post request has the file part
+        if 'file' not in request.files:
+            data = {
+                "error": "No file provided"
+            }
+            return jsonify(error=data), 400
+
+        file = request.files['file']
+        # if user does not select file, browser also
+        # submit a empty part without filename
+        if file.filename == '':
+            data = {
+                "error": "No file provided"
+            }
+            return jsonify(error=data), 400
+        if file and self.allowed_file(file.filename):
+            # save the file
+            filename = secure_filename(file.filename)
+            base_path = os.path.join(self.app.config['UPLOAD_FOLDER'])
+            file.save(os.path.join(base_path, filename))
+
+            # now start analyse the audio with STT engine
+            audio_path = base_path + os.sep + filename
+            ol = OrderListener(callback=self.audio_analyser_callback, audio_file_path=audio_path)
+            ol.start()
+            ol.join()
+            # wait the Order Analyser processing. We need to wait in this thread to keep the context
+            while not self.order_analyser_return:
+                time.sleep(0.1)
+            self.order_analyser_return = False
+            if self.launched_synapses is not None and self.launched_synapses:
+                data = jsonify(synapses=[e.serialize() for e in self.launched_synapses])
+                self.launched_synapses = None
+                return data, 201
+            else:
+                data = {
+                    "error": "The given order doesn't match any synapses"
+                }
+                return jsonify(error=data), 400
+
     @requires_auth
     def shutdown_server(self):
         func = request.environ.get('werkzeug.server.shutdown')
@@ -151,3 +242,27 @@ class FlaskAPI(threading.Thread):
             raise RuntimeError('Not running with the Werkzeug Server')
         func()
         return "Shutting down..."
+
+    def audio_analyser_callback(self, order):
+        """
+        Callback of the OrderListener. Called after the processing of the audio file
+        This method will
+        - call the Order Analyser to analyse the  order and launch corresponding synapse as usual.
+        - get a list of launched synapse.
+        - give the list to the main process via self.launched_synapses
+        - notify that the processing is over via order_analyser_return
+        :param order: string order to analyse
+        :return:
+        """
+        logger.debug("order to process %s" % order)
+        if order is not None:  # maybe we have received a null audio from STT engine
+            order_analyser = OrderAnalyser(order, brain=self.brain)
+            synapses_launched = order_analyser.start()
+            self.launched_synapses = synapses_launched
+        else:
+            if self.settings.default_synapse is not None:
+                SynapseLauncher.start_synapse(name=self.settings.default_synapse, brain=self.brain)
+                self.launched_synapses = self.brain.get_synapse_by_name(synapse_name=self.settings.default_synapse)
+
+        # this boolean will notify the main process that the order have been processed
+        self.order_analyser_return = True

+ 22 - 11
kalliope/stt/Utils.py

@@ -12,7 +12,7 @@ logger = logging.getLogger("kalliope")
 
 class SpeechRecognition(Thread):
 
-    def __init__(self):
+    def __init__(self, audio_file=None):
         """
         Thread used to caught n audio from the microphone and pass it to a callback method
         """
@@ -22,22 +22,33 @@ class SpeechRecognition(Thread):
         self.callback = None
         self.stop_thread = None
         self.kill_yourself = False
-        with self.microphone as source:
-            # we only need to calibrate once, before we start listening
-            self.recognizer.adjust_for_ambient_noise(source)
+        self.audio_stream = None
+
+        if audio_file is None:
+            # audio file not set, we need to capture a sample from the microphone
+            with self.microphone as source:
+                # we only need to calibrate once, before we start listening
+                self.recognizer.adjust_for_ambient_noise(source)
+        else:
+            # audio file provided
+            with sr.AudioFile(audio_file) as source:
+                self.audio_stream = self.recognizer.record(source)  # read the entire audio file
 
     def run(self):
         """
         Start the thread that listen the microphone and then give the audio to the callback method
         """
-        Utils.print_info("Say something!")
-        self.stop_thread = self.recognizer.listen_in_background(self.microphone, self.callback)
-        while not self.kill_yourself:
-            sleep(0.1)
-        logger.debug("kill the speech recognition process")
-        self.stop_thread()
+        if self.audio_stream is None:
+            Utils.print_info("Say something!")
+            self.stop_thread = self.recognizer.listen_in_background(self.microphone, self.callback)
+            while not self.kill_yourself:
+                sleep(0.1)
+            logger.debug("kill the speech recognition process")
+            self.stop_thread()
+        else:
+            self.callback(self.recognizer, self.audio_stream)
 
-    def start_listening(self):
+    def start_processing(self):
         """
         A method to start the thread
         """

+ 4 - 2
kalliope/stt/apiai/apiai.py

@@ -12,7 +12,8 @@ class Apiai(SpeechRecognition):
         :param callback: The callback function to call to send the text
         :param kwargs:
         """
-        SpeechRecognition.__init__(self)
+        # give the audio file path to process directly to the mother class if exist
+        SpeechRecognition.__init__(self, kwargs.get('audio_file_path', None))
 
         # callback function to call after the translation speech/tex
         self.main_controller_callback = callback
@@ -23,7 +24,8 @@ class Apiai(SpeechRecognition):
 
         # start listening in the background
         self.set_callback(self.apiai_callback)
-        self.start_listening()
+        # start processing, record a sample from the microphone if no audio file path provided, else read the file
+        self.start_processing()
 
     def apiai_callback(self, recognizer, audio):
         """

+ 4 - 2
kalliope/stt/bing/bing.py

@@ -12,7 +12,8 @@ class Bing(SpeechRecognition):
         :param callback: The callback function to call to send the text
         :param kwargs:
         """
-        SpeechRecognition.__init__(self)
+        # give the audio file path to process directly to the mother class if exist
+        SpeechRecognition.__init__(self, kwargs.get('audio_file_path', None))
 
         # callback function to call after the translation speech/tex
         self.main_controller_callback = callback
@@ -22,7 +23,8 @@ class Bing(SpeechRecognition):
 
         # start listening in the background
         self.set_callback(self.bing_callback)
-        self.start_listening()
+        # start processing, record a sample from the microphone if no audio file path provided, else read the file
+        self.start_processing()
 
     def bing_callback(self, recognizer, audio):
         """

+ 4 - 2
kalliope/stt/cmusphinx/cmusphinx.py

@@ -12,14 +12,16 @@ class Cmusphinx(SpeechRecognition):
         :param callback: The callback function to call to send the text
         :param kwargs:
         """
-        SpeechRecognition.__init__(self)
+        # give the audio file path to process directly to the mother class if exist
+        SpeechRecognition.__init__(self, kwargs.get('audio_file_path', None))
 
         # callback function to call after the translation speech/tex
         self.main_controller_callback = callback
 
         # start listening in the background
         self.set_callback(self.sphinx_callback)
-        self.start_listening()
+        # start processing, record a sample from the microphone if no audio file path provided, else read the file
+        self.start_processing()
 
     def sphinx_callback(self, recognizer, audio):
         """

+ 5 - 3
kalliope/stt/google/google.py

@@ -14,7 +14,8 @@ class Google(SpeechRecognition):
         :param callback: The callback function to call to send the text
         :param kwargs:
         """
-        SpeechRecognition.__init__(self)
+        # give the audio file path to process directly to the mother class if exist
+        SpeechRecognition.__init__(self, kwargs.get('audio_file_path', None))
 
         # callback function to call after the translation speech/tex
         self.main_controller_callback = callback
@@ -22,9 +23,10 @@ class Google(SpeechRecognition):
         self.language = kwargs.get('language', "en-US")
         self.show_all = kwargs.get('show_all', False)
 
-        # start listening in the background
+        # set the callback that will process the audio stream
         self.set_callback(self.google_callback)
-        self.start_listening()
+        # start processing, record a sample from the microphone if no audio file path provided, else read the file
+        self.start_processing()
 
     def google_callback(self, recognizer, audio):
         """

+ 4 - 2
kalliope/stt/houndify/houndify.py

@@ -12,7 +12,8 @@ class Houndify(SpeechRecognition):
         :param callback: The callback function to call to send the text
         :param kwargs:
         """
-        SpeechRecognition.__init__(self)
+        # give the audio file path to process directly to the mother class if exist
+        SpeechRecognition.__init__(self, kwargs.get('audio_file_path', None))
 
         # callback function to call after the translation speech/tex
         self.main_controller_callback = callback
@@ -24,7 +25,8 @@ class Houndify(SpeechRecognition):
 
         # start listening in the background
         self.set_callback(self.houndify_callback)
-        self.start_listening()
+        # start processing, record a sample from the microphone if no audio file path provided, else read the file
+        self.start_processing()
 
     def houndify_callback(self, recognizer, audio):
         """

+ 4 - 2
kalliope/stt/wit/wit.py

@@ -12,7 +12,8 @@ class Wit(SpeechRecognition):
         :param callback: The callback function to call to send the text
         :param kwargs:
         """
-        SpeechRecognition.__init__(self)
+        # give the audio file path to process directly to the mother class if exist
+        SpeechRecognition.__init__(self, kwargs.get('audio_file_path', None))
 
         # callback function to call after the translation speech/tex
         self.main_controller_callback = callback
@@ -21,7 +22,8 @@ class Wit(SpeechRecognition):
 
         # start listening in the background
         self.set_callback(self.wit_callback)
-        self.start_listening()
+        # start processing, record a sample from the microphone if no audio file path provided, else read the file
+        self.start_processing()
 
     def wit_callback(self, recognizer, audio):
         try:

+ 2 - 2
setup.py

@@ -73,11 +73,11 @@ setup(
         'requests>=2.12.4',
         'httpretty==0.8.14',
         'mock==2.0.0',
-        'Flask-Testing==0.6.1',
+        'Flask-Testing>=0.6.1',
         'apscheduler==3.3.0',
         'GitPython==2.1.1',
         'packaging>=16.8',
-        'transitions>=0.4.3'
+        'transitions>=0.4.3',
     ],