Преглед на файлове

Merge pull request #148 from kalliope-project/dev_nico

kalliope neuron-install
Monf преди 8 години
родител
ревизия
7e654b2616

+ 28 - 18
Docs/neuron_list.md

@@ -2,22 +2,32 @@
 
 A neuron is a module that will perform some actions attached to an order. You can use it in your synapses. See the [complete neuron documentation](neurons.md) for more information.
 
-| Name                                                          | Description                                                                             |
-|---------------------------------------------------------------|-----------------------------------------------------------------------------------------|
-| [ansible_playbook](../kalliope/neurons/ansible_playbook/)     | Run an ansible playbook                                                                 |
-| [gmail_checker](../kalliope/neurons/gmail_checker/)           | Get the number of unread email and their subjects from a gmail account                  |
-| [kill_switch](../kalliope/neurons/kill_switch/)               | Stop Kalliope process                                                                   |
-| [neurotransmitter](../kalliope/neurons/neurotransmitter/)     | Link synapse together                                                                   |
-| [push_message](../kalliope/neurons/push_message/)             | Send a push message to a remote device like Android/iOS/Windows Phone or Chrome browser |
-| [rss_reader](../kalliope/neurons/rss_reader/)                 | get rss feed from website                                                               |
-| [say](../kalliope/neurons/say/)                               | Make Kalliope talk by using TTS                                                         |
-| [script](../kalliope/neurons/script/)                         | Run an executable script                                                                |
-| [shell](../kalliope/neurons/shell/)                           | Run a shell command                                                                     |
-| [sleep](../kalliope/neurons/sleep/)                           | Make Kalliope sleep for a while before continuing                                       |
-| [systemdate](../kalliope/neurons/systemdate/)                 | Give the local system date and time                                                     |
-| [tasker_autoremote](../kalliope/neurons/tasker_autoremote/)   | Send a message to Android tasker app                                                    |
-| [twitter](../kalliope/neurons/twitter/)                       | Send a Twit from kalliope                                                               |
-| [uri](../kalliope/neurons/uri/)                               | Interacts with HTTP and HTTPS web services.                                             |
-| [wake_on_lan](../kalliope/neurons/wake_on_lan/)               | Wake on lan a computer                                                                  |
-| [wikipedia_searcher](../kalliope/neurons/wikipedia_searcher/) | Search for a page on Wikipedia                                                          |
+## Core neuron
 
+| Name                                                      | Description                                       |
+|-----------------------------------------------------------|---------------------------------------------------|
+| [ansible_playbook](../kalliope/neurons/ansible_playbook/) | Run an ansible playbook                           |
+| [kill_switch](../kalliope/neurons/kill_switch/)           | Stop Kalliope process                             |
+| [neurotransmitter](../kalliope/neurons/neurotransmitter/) | Link synapse together                             |
+| [say](../kalliope/neurons/say/)                           | Make Kalliope talk by using TTS                   |
+| [script](../kalliope/neurons/script/)                     | Run an executable script                          |
+| [shell](../kalliope/neurons/shell/)                       | Run a shell command                               |
+| [sleep](../kalliope/neurons/sleep/)                       | Make Kalliope sleep for a while before continuing |
+| [systemdate](../kalliope/neurons/systemdate/)             | Give the local system date and time               |
+| [uri](../kalliope/neurons/uri/)                           | Interacts with HTTP and HTTPS web services.       |
+
+## Community neuron
+
+| Name                                                                           | Description                                                                             |
+|--------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------|
+| [gmail_checker](https://github.com/kalliope-project/kalliope_neuron_gmail)     | Get the number of unread email and their subjects from a gmail account                  |
+| [pushetta](https://github.com/kalliope-project/kalliope_neuron_pushetta)       | Send a push message to a remote device like Android/iOS/Windows Phone or Chrome browser |
+| [rss_reader](https://github.com/kalliope-project/kalliope_neuron_rss_reader)   | get rss feed from website                                                               |
+| [tasker](https://github.com/kalliope-project/kalliope_neuron_tasker)           | Send a message to Android tasker app                                                    |
+| [twitter](https://github.com/kalliope-project/kalliope_neuron_twitter)         | Send a Twit from kalliope                                                               |
+| [wake_on_lan](https://github.com/kalliope-project/kalliope_neuron_wake_on_lan) | Wake on lan a computer                                                                  |
+| [wikipedia](https://github.com/kalliope-project/kalliope_neuron_wikipedia)     | Search for a page on Wikipedia                                                          |
+
+Wanna add your neuron in the list? Open [an issue](../../issues) with the link of your neuron or send a pull request to update the list directly.
+
+To know how to install a community neuron, read the "Installation" section of the [neuron documentation](neurons.md).

+ 5 - 0
Docs/neuron_template.md

@@ -4,6 +4,11 @@
 
 Little description of what the neuron does.
 
+Installation
+```
+kalliope neuron-install --git-url "https://github.com/my_user/my_neuron.git"
+```
+
 ## Options
 
 (usage of a [table generator](http://www.tablesgenerator.com/markdown_tables) is recommended)

+ 18 - 0
Docs/neurons.md

@@ -3,6 +3,22 @@
 A neuron is a plugin that performs a specific action. You use it to create a synapse.
 You can add as many neurons as you want to a synapse. The neurons are executed one by one when the input order is triggered.
 
+## Installation
+
+Core neurons are already packaged with the installation of kalliope an can be used out of the box. Community neuron need to be installed manually.
+
+Use the CLI
+```
+kalliope neuron-install --git-url "<git_url>"
+```
+
+E.g:
+```
+kalliope neuron-install --git-url "https://github.com/kalliope-project/kalliope_neuron_wikipedia.git"
+```
+
+You may be prompted to type your `sudo` password during the process. You can see the list of [available neuron here](neuron_list.md)
+
 ## Usage
 Neurons are declared in the `neurons` section of a synapse in your brain file.
 The `neurons` section is a list (because it starts with a "-") which contains neuron modules names
@@ -22,6 +38,8 @@ neurons:
 ```
 > **note:** parameters are indented with two spaces bellow the neuron's name following the YAML syntax.
 
+> **note:** Kalliope will try to load the neuron from your resources directory, then from core neuron packages.
+
 To know the list of required parameters, check of documentation of the neuron.
 Full list of [available neuron here](neuron_list.md)
 

+ 19 - 0
Docs/settings.md

@@ -225,5 +225,24 @@ E.g
 default_synapse: "Default-response"
 ```
 
+## Resources directory
+
+The resources directory is the path where Kalliope will try to load community modules like Neurons, STTs or TTSs.
+Set a valid path is required if you want to install community neuron. The path can be relative or absolute.
+
+```
+resource_directory:
+  resource_name: "path"
+```
+
+E.g
+```
+resource_directory:
+  neuron: "resources/neurons"
+  stt: "resources/stt"
+  tts: "resources/tts"
+  trigger: "/full/path/to/trigger"
+```
+
 ## Next: configure the brain of Kalliope
 Now your settings are ok, you can start creating the [brain](brain.md) of your assistant.

+ 5 - 2
kalliope/__init__.py

@@ -42,7 +42,7 @@ def main():
     parser.add_argument("--run-synapse", help="Name of a synapse to load surrounded by quote")
     parser.add_argument("--brain-file", help="Full path of a brain file")
     parser.add_argument("--debug", action='store_true', help="Show debug output")
-    parser.add_argument("--git-url", action='store_true', help="Git URL of the neuron to install")
+    parser.add_argument("--git-url", help="Git URL of the neuron to install")
 
     # parse arguments from script parameters
     args = parser.parse_args()
@@ -95,7 +95,10 @@ def main():
         if not args.git_url:
             Utils.print_danger("You must specify the git url")
         else:
-            ResourcesManager(args.git_url)
+            parameters = {
+                "git_url": args.git_url
+            }
+            ResourcesManager("install", **parameters)
 
 
 def configure_logging(debug=None):

+ 1 - 1
kalliope/_version.py

@@ -1,2 +1,2 @@
 # https://www.python.org/dev/peps/pep-0440/
-version_str = "0.3.0"
+version_str = "0.4.0"

+ 1 - 0
kalliope/core/ConfigurationManager/SettingLoader.py

@@ -562,6 +562,7 @@ class SettingLoader(object):
                                         tts_folder=tts_folder,
                                         trigger_folder=trigger_folder)
         except KeyError:
+            logger.debug("Resource directory not found in settings")
             resource_object = None
 
         return resource_object

+ 107 - 31
kalliope/core/ResourcesManager.py

@@ -1,6 +1,9 @@
+import getpass
 import os
 
 import logging
+
+import shutil
 from git import Repo
 from kalliope.core.Models import Neuron
 
@@ -30,6 +33,11 @@ class ResourcesManager(object):
         # in case of update or install, url where
         self.git_url = kwargs.get('git_url', None)
 
+        # temp path where we install the new module
+        self.tmp_path = self.settings.resources.neuron_folder + os.sep + TMP_GIT_FOLDER
+        self.dna_file_path = self.tmp_path + os.sep + DNA_FILE_NAME
+        self.dna_file = None
+
         if self.action == "install":
             self.install()
 
@@ -38,37 +46,105 @@ class ResourcesManager(object):
         Neuron installation method
         :return:
         """
+        if self.is_settings_ok():
+            # first, we clone the repo
+            self._clone_repo()
+
+            # check the content of the cloned repo
+            if self.is_neuron_ok():
+                self.install_neuron()
+
+    def is_settings_ok(self):
+        """
+        To be able to install a neuron, the user must has configured his settings
+        :return: True if settings are ok
+        """
+        if self.settings.resources is None:
+            message = "Resources folder not set in settings, cannot install a community neuron"
+            logger.debug(message)
+            Utils.print_danger(message)
+            return False
+
+        if self.settings.resources.neuron_folder is None:
+            message = "No neuron folder set in settings, cannot install a community neuron"
+            logger.debug(message)
+            Utils.print_danger(message)
+            return False
+
+        return True
+
+    def is_neuron_ok(self):
+        """
+        Check if the git cloned repo is fine to be installed
+        :return:
+        """
+        Utils.print_info("Checking repository...")
+        if not os.path.exists(self.dna_file_path):
+            Utils.print_danger("Missing %s file" % DNA_FILE_NAME)
+            return False
+
+        # get the content of the DNA file
+        self.dna_file = YAMLLoader().get_config(self.dna_file_path)
+        logger.debug("[ResourcesManager] DNA file content: " + str(self.dna_file))
+        if "neuron_name" not in self.dna_file:
+            Utils.print_danger("The DNA of the neuron does not contains a \"neuron_name\" tag")
+            os.remove(self.tmp_path)
+            return False
+
+        # check that a install.yml file is present
+        install_file_path = self.tmp_path + os.sep + INSTALL_FILE_NAME
+        if not os.path.exists(install_file_path):
+            Utils.print_danger("Missing %s file" % DNA_FILE_NAME)
+            return False
+
+        return True
+
+    def _clone_repo(self):
+        """
+        Use git to clone locally the neuron in a temp folder
+        :return:
+        """
         # clone the repo
-        tmp_path = self.settings.resources.neuron_folder + os.sep + TMP_GIT_FOLDER
+        logger.debug("GIT clone into folder: %s" % self.tmp_path)
         Utils.print_info("Cloning repository...")
-        Repo.clone_from(self.git_url, tmp_path)
-
-        # get the dna file
-        Utils.print_info("Checking DNA")
-        dna_file_path = tmp_path + os.sep + DNA_FILE_NAME
-        if os.path.exists(dna_file_path):
-            dna_file = YAMLLoader().get_config(dna_file_path)
-            logger.debug("[ResourcesManager] DNA file content: " + str(dna_file))
-
-            if "neuron_name" not in dna_file:
-                Utils.print_danger("The DNA of the neuron does not contains a \"neuron_name\"")
-                os.remove(tmp_path)
-            else:
-                # rename the folder
-                new_absolute_neuron_path = self.settings.resources.neuron_folder + os.sep + dna_file["neuron_name"].lower()
-                os.rename(tmp_path, new_absolute_neuron_path)
-
-                # check install file exists
-                install_file_path = new_absolute_neuron_path + os.sep + INSTALL_FILE_NAME
-                if os.path.exists(install_file_path):
-                    ansible_neuron_parameters = {
-                        "task_file": install_file_path
-                    }
-                    neuron = Neuron(name="ansible_playbook", parameters=ansible_neuron_parameters)
-                    NeuronLauncher.start_neuron(neuron)
-                else:
-                    Utils.print_danger("Missing %s file in %s" % (INSTALL_FILE_NAME, install_file_path))
-
-        else:
-            Utils.print_danger("Missing %s file" % DNA_FILE_NAME)
+        # if the folder already exist we remove it
+        if os.path.exists(self.tmp_path):
+            shutil.rmtree(self.tmp_path)
+        Repo.clone_from(self.git_url, self.tmp_path)
+
+    def _rename_temp_neuron_folder(self):
+        """
+        Rename the temp folder of the cloned neuron
+        Return the name of the path of the neuron to install
+        :return: path of the neuron
+        """
+        neuron_name = self.dna_file["neuron_name"].lower()
+        new_absolute_neuron_path = self.settings.resources.neuron_folder + os.sep + neuron_name
+        try:
+            os.rename(self.tmp_path, new_absolute_neuron_path)
+        except OSError:
+            # the folder already exist
+            Utils.print_warning("The neuron %s already exist in the resource directory" % neuron_name)
+            # remove the cloned repo
+            shutil.rmtree(self.tmp_path)
+            new_absolute_neuron_path = None
+        finally:
+            return new_absolute_neuron_path
 
+    def install_neuron(self):
+        # rename the folder
+        new_neuron_path = self._rename_temp_neuron_folder()
+        if new_neuron_path is not None:
+            install_file_path = new_neuron_path + os.sep + INSTALL_FILE_NAME
+            Utils.print_info("Starting neuron installation")
+            # ask the sudo password
+            pswd = getpass.getpass('Sudo password:')
+            ansible_neuron_parameters = {
+                "task_file": install_file_path,
+                "sudo": True,
+                "sudo_user": "root",
+                "sudo_password": pswd
+            }
+            neuron = Neuron(name="ansible_playbook", parameters=ansible_neuron_parameters)
+            NeuronLauncher.start_neuron(neuron)
+            Utils.print_success("Neuron %s installed" % self.dna_file["neuron_name"])

+ 39 - 15
kalliope/neurons/ansible_playbook/README.md

@@ -11,14 +11,19 @@ This neuron can be used to perform complex operation with all [modules available
 
 ## Options
 
-| parameter | required | default | choices | comment                                      |
-|-----------|----------|---------|---------|----------------------------------------------|
-| task_file | YES      |         |         | path to the Playbook file that contain tasks |
+| parameter     | required | default | choices      | comment                                                                                                                          |
+|---------------|----------|---------|--------------|----------------------------------------------------------------------------------------------------------------------------------|
+| task_file     | YES      |         |              | path to the Playbook file that contains tasks                                                                                    |
+| sudo          | NO       | FALSE   | True | False | If the playbook will require root privileges (become=true) , this must be set to True and sudo_user and password set accordingly |
+| sudo_user     | NO       |         |              | The target user with admin privileges. In most of case "root"                                                                    |
+| sudo_password | NO       |         |              | The password of the sudo_user                                                                                                    |
 
 
 
 ## Synapses example
 
+### Playbook without admin privileges
+
 Call the playbook named playbook.yml
 ```
   - name: "Ansible-test"
@@ -28,7 +33,7 @@ Call the playbook named playbook.yml
       - ansible_playbook: 
           task_file: "playbook.yml"
       - say:
-          message: "Tache terminée"    
+          message: "The task is done"
 ```
 
 Content of the playbook. This playbook will use the [URI module](http://docs.ansible.com/ansible/uri_module.html) to interact with a webservice on a remote server.
@@ -54,23 +59,42 @@ Content of the playbook. This playbook will use the [URI module](http://docs.ans
             {"app_name": "music", "state": "start"}
 ```
 
+### Playbook with admin privileges
+
+In some cases, a playbook requires sudo right to perform admin operations like installing a package.
+In this case, you must give to the neuron the login and password of the user which has admin privileges.
+```
+  - name: "Ansible-root"
+    signals:
+      - order: "playbook"
+    neurons:
+      - ansible_playbook:
+          task_file: "playbook-root.yml"
+          sudo: true,
+          sudo_user: "root"
+          sudo_password: "secret"
+```
+
+And the playbook would be. Notice that we use `become: True`
+```
+- hosts: localhost
+  gather_facts: no
+  connection: local
+  become: True
+
+  tasks:
+    - name: "Install a useful train package"
+      apt:
+        name: sl
+        state: present
+```
 
 ## Note
 
-Ansible contain a lot of modules that can be useful for Kalliope
+Ansible contains a lot of modules that can be useful for Kalliope
 
 - [Notification](http://docs.ansible.com/ansible/list_of_notification_modules.html): can be used to send a message to Pushbullet, IRC channel, Rocket Chat and a lot of other notification services
 - [Files](http://docs.ansible.com/ansible/list_of_files_modules.html): can be used to perform a backup or synchronize two file path
 - [Windows](http://docs.ansible.com/ansible/list_of_windows_modules.html): Can be used to control a Windows Desktop
 
 Shell neuron or script neuron can perform same actions. Ansible is just a way to simplify some execution or enjoy some [already made plugin](http://docs.ansible.com/ansible/modules_by_category.html). 
-
-Here is the example of synapse you would use to perform a call to a web service without Ansible:
-```
-- name: "start-music"
-    signals:
-      - order: "start music rock"
-    neurons:
-      - shell:
-          cmd: "curl -i --user admin:secret -H \"Content-Type: application/json\" -X POST -d '{\"app_name\":\"music\",\"state\":\"start\"}' http://192.168.0.17:8000/app"      
-```

+ 40 - 8
kalliope/neurons/ansible_playbook/ansible_playbook.py

@@ -1,4 +1,6 @@
 from collections import namedtuple
+
+import logging
 from ansible.parsing.dataloader import DataLoader
 from ansible.vars import VariableManager
 from ansible.inventory import Inventory
@@ -6,27 +8,28 @@ from ansible.executor.playbook_executor import PlaybookExecutor
 
 from kalliope.core.NeuronModule import NeuronModule, MissingParameterException
 
+logging.basicConfig()
+logger = logging.getLogger("kalliope")
+
 
 class Ansible_playbook(NeuronModule):
     def __init__(self, **kwargs):
         super(Ansible_playbook, self).__init__(**kwargs)
 
         self.task_file = kwargs.get('task_file', None)
+        self.sudo = kwargs.get('sudo', False)
+        self.sudo_user = kwargs.get('sudo_user', False)
+        self.sudo_password = kwargs.get('sudo_password', False)
 
         # check if parameters have been provided
         if self._is_parameters_ok():
 
-            Options = namedtuple('Options',
-                                 ['connection', 'forks', 'become', 'become_method', 'become_user', 'check', 'listhosts',
-                                  'listtasks', 'listtags', 'syntax', 'module_path'])
-
             variable_manager = VariableManager()
             loader = DataLoader()
-            options = Options(connection='local', forks=100, become=None, become_method=None, become_user=None, check=False,
-                              listhosts=False, listtasks=False, listtags=False, syntax=False, module_path="")
-            passwords = dict(vault_pass='secret')
+            options = self._get_options()
+            passwords = {'become_pass': self.sudo_password}
 
-            inventory = Inventory(loader=loader, variable_manager=variable_manager, host_list='localhost')
+            inventory = Inventory(loader=loader, variable_manager=variable_manager, host_list="localhost")
             variable_manager.set_inventory(inventory)
             playbooks = [self.task_file]
 
@@ -43,4 +46,33 @@ class Ansible_playbook(NeuronModule):
     def _is_parameters_ok(self):
         if self.task_file is None:
             raise MissingParameterException("task_file parameter required")
+
+        # check if the user want to use sudo for root privileges
+        if self.sudo:
+            # the user must set a login and password
+            if not self.sudo_user:
+                raise MissingParameterException("sudo_user parameter required with sudo True")
+            if not self.sudo_password:
+                raise MissingParameterException("sudo_password parameter required with sudo True")
+
         return True
+
+    def _get_options(self):
+        """
+        Return a valid dict of option usable by Ansible depending on the sudo value if set
+        :return: dict of option
+        """
+        Options = namedtuple('Options',
+                             ['connection', 'forks', 'become', 'become_method', 'become_user', 'check', 'listhosts',
+                              'listtasks', 'listtags', 'syntax', 'module_path'])
+        if self.sudo:
+            options = Options(connection='local', forks=100, become=True, become_method="sudo",
+                              become_user=self.sudo_user, check=False, listhosts=False, listtasks=False, listtags=False,
+                              syntax=False, module_path="")
+        else:
+            options = Options(connection='local', forks=100, become=None, become_method=None, become_user=None,
+                              check=False, listhosts=False, listtasks=False, listtags=False, syntax=False,
+                              module_path="")
+
+        logger.debug("Ansible options: %s" % str(options))
+        return options

+ 4 - 4
kalliope/settings.yml

@@ -118,7 +118,7 @@ default_synapse: "Default-synapse"
 # resource directory path
 # ---------------------------
 #resource_directory:
-#  neuron: "/home/me/kalliope_resources_dir/neurons"
-#  stt: "/home/me/kalliope_resources_dir/stt"
-#  tts: "/home/me/kalliope_resources_dir/tts"
-#  trigger: "/home/me/kalliope_resources_dir/trigger"
+#  neuron: "resources/neurons"
+#  stt: "resources/stt"
+#  tts: "resources/tts"
+#  trigger: "resources/trigger"