浏览代码

Merge pull request #124 from kalliope-project/tests_nico

refactor event manager
Monf 8 年之前
父节点
当前提交
a14ca2cf3d

+ 12 - 0
CHANGELOG.md

@@ -1,3 +1,15 @@
+v0.3.0 / 2016-12-7
+=================
+- add unit tests for core & neurons
+- add CI (Travis)
+- refactor Event manager
+- support installation with setup.py
+- support pip installation
+- fix ansible_playbook neuron
+- add rss_reader neuron
+- review settings and brain file loading
+
+
 v0.2 / 2016-11-21
 =================
 

+ 68 - 22
Docs/signals.md

@@ -36,35 +36,62 @@ will be started by Kalliope. So keep in mind that the best practice is to use re
 
 An event is a way to schedule the launching of a synapse periodically at fixed times, dates, or intervals.
 
-The event system is based on [Linux crontab](https://en.wikipedia.org/wiki/Cron). A crontab is file that specifies shell commands to run periodically
- on a given schedule.
-When you declare an event in the signal, Kalliope will load the crontab file to schedule the launching of the target synapse.
+The event system is based on [APScheduler](http://apscheduler.readthedocs.io/en/latest/modules/triggers/cron.html) which it is itself based on [Linux crontab](https://en.wikipedia.org/wiki/Cron). 
+When you declare an event in the signal, Kalliope will schedule the launching of the target synapse.
 
 The syntax of an event declaration in a synapse is the following
 ```
 signals:
-    - event: "<contab period>"
+  - event:
+      parameter1: "value1"
+      parameter2: "value2"
 ```
 
-Where a crontab period follows the syntax bellow:
+For example, if we want Kalliope to run the synapse every day a 8:30, the event will be declared like this:
 ```
- ┌───────────── min (0 - 59)
- │ ┌────────────── hour (0 - 23)
- │ │ ┌─────────────── day of month (1 - 31)
- │ │ │ ┌──────────────── month (1 - 12)
- │ │ │ │ ┌───────────────── day of week (0 - 6) (0 to 6 are Sunday to
- │ │ │ │ │                  Saturday, or use names; 7 is also Sunday)
- │ │ │ │ │
- │ │ │ │ │
- * * * * *  
+- event:
+    hour: "8"
+    minute: "30"
 ```
 
-For example, if we want Kalliope to run the synapse every day a 8 PM, the event will be declared like this:
-```
-- event: "0 20 * * *"
-```
+### Parameters
+Parameters are keyword you can use to build your event
+
+List of available parameter:
+
+| parameter   | required | default | choices                                                         | comment   |
+|-------------|----------|---------|-----------------------------------------------------------------|-----------|
+| year        | no       | *       | 4 digit                                                         | E.g: 2016 |
+| month       | no       | *       | month (1-12)                                                    |           |
+| day         | no       | *       | day of the (1-31)                                               |           |
+| week        | no       | *       | ISO week (1-53)                                                 |           |
+| day_of_week | no       | *       | number or name of weekday  (0-6 or mon,tue,wed,thu,fri,sat,sun) | 6=Sunday  |
+| hour        | no       | *       | hour (0-23)                                                     |           |
+| minute      | no       | *       | minute (0-59)                                                   |           |
+| second      | no       | *       | second (0-59)                                                   |           |
+
+> **Note:** You must set at least one parameter from the list of parameter
+
+### Expression 
+Expressions can be used in value of each parameter. Multiple expression can be given in a single field, separated by commas.
+
+| Expression | Field | Description                                                                             |
+|------------|-------|-----------------------------------------------------------------------------------------|
+| *          | any   | Fire on every value                                                                     |
+| */a        | any   | Fire every `a` values, starting from the minimum                                        |
+| a-b        | any   | Fire on any value within the `a-b` range (a must be smaller than b)                     |
+| a-b/c      | any   | Fire every c values within the `a-b` range                                              |
+| xrd y      | day   | Fire on the `x` -rd occurrence of weekday `y` within the month                          |
+| last x     | day   | Fire on the last occurrence of weekday `x` within the month                             |
+| last x     | day   | Fire on the last day within the month                                                   |
+| x,y,z      | day   | Fire on any matching expression; can combine any number of any of the above expressions |
 
-Let's make a complete example. We want Kalliope to wake us up each morning of working day (Monday to friday) at 7 AM and:
+
+### Examples
+
+#### Web clock radio
+
+Let's make a complete example. We want Kalliope to wake us up each morning of working day (Monday to friday) at 7:30 AM and:
 - Wish us good morning
 - Give us the time
 - Play our favourite web radio
@@ -73,7 +100,10 @@ The synapse in the brain would be
 ```
   - name: "wake-up"
     signals:
-      - event: "0 7 * * 1,2,3,4,5"
+      - event:
+          hour: "7"
+          minute: "30"
+          day_of_week: "1,2,3,4,5"
     neurons:
       - say:
           message:
@@ -83,6 +113,7 @@ The synapse in the brain would be
             - "It is {{ hours }} hours and {{ minutes }} minutes"
       - shell: 
           cmd: "mplayer http://192.99.17.12:6410/"
+          async: True
 ```
 
 After setting up an event, you must restart Kalliope
@@ -92,8 +123,23 @@ python kalliope.py start
 
 If the syntax is ok, Kalliope will show you each synapse that it has loaded in the crontab
 ```
-Synapse "wake up" added to the crontab
-Event loaded in crontab
+Add synapse name "wake-up" to the scheduler: cron[day_of_week='1,2,3,4,5', hour='7', minute='30']
+Event loaded
 ```
 
 That's it, the synapse is now scheduled and will be started automatically.
+
+
+####  Make Kalliope say something on the third Friday of June, July, August, November and December at 00:00, 01:00, 02:00 and 03:00
+```
+- name: "wake-up"
+  signals:
+    - event:
+        day: "3rd fri"        
+        month: "6-8,11-12"
+        hour: "0-3"        
+  neurons:
+    - say:
+        message:
+          - "This is a schedulled sentence"
+```

+ 2 - 2
Tests/test_brain_loader.py

@@ -85,10 +85,10 @@ class TestBrainLoader(unittest.TestCase):
     def test_get_event_or_order_from_dict(self):
 
         order_object = Order(sentence="test_order")
-        event_object = Event(period="0 7 * * *")
+        event_object = Event(hour="7")
 
         dict_order = {'order': 'test_order'}
-        dict_event = {'event': '0 7 * * *'}
+        dict_event = {'event': {'hour': '7'}}
 
         bl = BrainLoader(file_path=self.brain_to_test)
         order_from_bl = bl._get_event_or_order_from_dict(dict_order)

+ 9 - 1
Tests/test_configuration_checker.py

@@ -68,9 +68,15 @@ class TestConfigurationChecker(unittest.TestCase):
             ConfigurationChecker.check_signal_dict(invalid_signal)
 
     def test_check_event_dict(self):
-        valid_event = '0 * * * *'
+        valid_event = {
+            "hour": "18",
+            "minute": "16"
+          }
         invalid_event = None
         invalid_event2 = ""
+        invalid_event3 = {
+            "notexisting": "12"
+        }
 
         self.assertTrue(ConfigurationChecker.check_event_dict(valid_event))
 
@@ -78,6 +84,8 @@ class TestConfigurationChecker(unittest.TestCase):
             ConfigurationChecker.check_event_dict(invalid_event)
         with self.assertRaises(NoEventPeriod):
             ConfigurationChecker.check_event_dict(invalid_event2)
+        with self.assertRaises(NoEventPeriod):
+            ConfigurationChecker.check_event_dict(invalid_event3)
 
     def test_check_order_dict(self):
         valid_order = 'test_order'

+ 4 - 3
install/files/python_requirements.txt

@@ -1,15 +1,15 @@
 SpeechRecognition==3.4.6
 markupsafe==0.23
 pyaudio==0.2.9
-ansible==2.1.1.0
+ansible==2.2.0.0
 python2-pythondialog==3.4.0
 jinja==1.2
 python-crontab==2.1.1
-cffi==1.8.3
+cffi==1.9.1
 pygmail==0.0.5.4
 pushetta==1.0.15
 wakeonlan==0.2.2
-ipaddress==1.0.16
+ipaddress==1.0.17
 pyowm==2.5.0
 python-twitter==3.1
 flask==0.11.1
@@ -20,3 +20,4 @@ httpretty==0.8.14
 mock==2.0.0
 feedparser==5.2.1
 Flask-Testing==0.6.1
+apscheduler==3.3.0

+ 4 - 5
kalliope/__init__.py

@@ -6,7 +6,7 @@ import logging
 from kalliope.core import ShellGui
 from kalliope.core import Utils
 from kalliope.core.ConfigurationManager.BrainLoader import BrainLoader
-from kalliope.core.CrontabManager import CrontabManager
+from kalliope.core.EventManager import EventManager
 from kalliope.core.MainController import MainController
 import signal
 import sys
@@ -75,10 +75,9 @@ def main():
             SynapseLauncher.start_synapse(args.run_synapse, brain=brain)
 
         if args.run_synapse is None:
-            # first, load events in crontab
-            crontab_manager = CrontabManager(brain=brain)
-            crontab_manager.load_events_in_crontab()
-            Utils.print_success("Events loaded in crontab")
+            # first, load events in event manager
+            EventManager(brain.synapses)
+            Utils.print_success("Events loaded")
             # then start kalliope
             Utils.print_success("Starting Kalliope")
             Utils.print_info("Press Ctrl+C for stopping")

+ 1 - 1
kalliope/_version.py

@@ -1,2 +1,2 @@
 # https://www.python.org/dev/peps/pep-0440/
-version_str = "0.3"
+version_str = "0.3.0"

+ 10 - 0
kalliope/brain-test.yml

@@ -0,0 +1,10 @@
+---
+  - name: "say-hello-fr"
+    signals:
+      - event:
+          hour: "18"
+          minute: "08"
+    neurons:
+      - say:
+          message:
+            - "Bonjour monsieur"

+ 1 - 1
kalliope/brain.yml

@@ -15,4 +15,4 @@
     neurons:
       - say:
           message:
-            - "Hello sir"
+            - "Hello sir"

+ 23 - 3
kalliope/core/ConfigurationManager/BrainLoader.py

@@ -165,8 +165,8 @@ class BrainLoader(object):
 
         return signals
 
-    @staticmethod
-    def _get_event_or_order_from_dict(signal_or_event_dict):
+    @classmethod
+    def _get_event_or_order_from_dict(cls, signal_or_event_dict):
         """
         The signal is either an Event or an Order
 
@@ -186,7 +186,7 @@ class BrainLoader(object):
             # print "is event"
             event = signal_or_event_dict["event"]
             if ConfigurationChecker.check_event_dict(event):
-                return Event(period=event)
+                return cls._get_event_object(event)
 
         if 'order' in signal_or_event_dict:
             order = signal_or_event_dict["order"]
@@ -215,3 +215,23 @@ class BrainLoader(object):
         if os.path.isfile(brain_path):
             return brain_path
         raise IOError("Default brain.yml file not found")
+
+    @classmethod
+    def _get_event_object(cls, event_dict):
+        def get_key(key_name):
+            try:
+                return event_dict[key_name]
+            except KeyError:
+                return None
+
+        year = get_key("year")
+        month = get_key("month")
+        day = get_key("day")
+        week = get_key("week")
+        day_of_week = get_key("day_of_week")
+        hour = get_key("hour")
+        minute = get_key("minute")
+        second = get_key("second")
+
+        return Event(year=year, month=month, day=day, week=week,
+                     day_of_week=day_of_week, hour=hour, minute=minute, second=second)

+ 26 - 1
kalliope/core/ConfigurationManager/ConfigurationChecker.py

@@ -196,8 +196,33 @@ class ConfigurationChecker:
         .. raises:: NoEventPeriod
         .. warnings:: Static and Public
         """
+        def get_key(key_name):
+            try:
+                return event_dict[key_name]
+            except KeyError:
+                return None
+
         if event_dict is None or event_dict == "":
-            raise NoEventPeriod("Event must contain a period: %s" % event_dict)
+            raise NoEventPeriod("Event must contain at least one of those elements: "
+                                "year, month, day, week, day_of_week, hour, minute, second")
+
+        # check content as at least on key
+        year = get_key("year")
+        month = get_key("month")
+        day = get_key("day")
+        week = get_key("week")
+        day_of_week = get_key("day_of_week")
+        hour = get_key("hour")
+        minute = get_key("minute")
+        second = get_key("second")
+
+        list_to_check = [year, month, day, week, day_of_week, hour, minute, second]
+        number_of_none_object = list_to_check.count(None)
+        list_size = len(list_to_check)
+        if number_of_none_object >= list_size:
+            raise NoEventPeriod("Event must contain at least one of those elements: "
+                                "year, month, day, week, day_of_week, hour, minute, second")
+
         return True
 
     @staticmethod

+ 0 - 118
kalliope/core/CrontabManager.py

@@ -1,118 +0,0 @@
-import logging
-
-from crontab import CronSlices
-from crontab import CronTab
-
-from kalliope.core import Utils
-from kalliope.core.Models import Event
-
-logging.basicConfig()
-logger = logging.getLogger("kalliope")
-
-
-class InvalidCrontabPeriod(Exception):
-    """
-    Event are based on the Crontab. The Period must be corresponding to the Crontab format
-    .. seealso:: Event
-    """
-    pass
-
-CRONTAB_COMMENT = "KALLIOPE"
-KALLIOPE_ENTRY_POINT_SCRIPT = "__init__.py"
-
-
-class CrontabManager:
-
-    def __init__(self, brain=None):
-        self.my_user_cron = CronTab(user=True)
-        self.brain = brain
-        self.base_command = self._get_base_command()
-
-    def load_events_in_crontab(self):
-        """
-        Remove all line in crontab with the CRONTAB_COMMENT
-        Then add back line from event in the brain.yml
-        """
-        # clean the current crontab from all Kalliope event
-        self._remove_all_job()
-        # load the brain file
-        for synapse in self.brain.synapses:
-            for signal in synapse.signals:
-                # print signal
-                # if the signal is an event we add it to the crontab
-                if type(signal) == Event:
-                    # for all synapse with an event, we add the task id to the crontab
-                    self._add_event(period_string=signal.period, event_id=synapse.name)
-
-    def _add_event(self, period_string, event_id):
-        """
-        Add a single event in the crontab.
-        Will add a line like:
-        <period_string> python /path/to/kalliope.py start --brain-file /path/to/brain.yml --run-synapse "<event_id>"
-
-        E.g:
-        30 7 * * * python /home/me/kalliope/kalliope.py start --brain-file /home/me/brain.yml --run-synapse  "Say-hello"
-        :param period_string: crontab period
-        :type period_string: str
-        :param event_id:
-        :type event_id: str
-        :return:
-        """
-        my_user_cron = CronTab(user=True)
-        job = my_user_cron.new(command=self.base_command+" "+str("\"" + event_id + "\""), comment=CRONTAB_COMMENT)
-        if CronSlices.is_valid(period_string):
-            job.setall(period_string)
-            job.enable()
-        else:
-            raise InvalidCrontabPeriod("The crontab period %s is not valid" % period_string)
-        # write the file
-        my_user_cron.write()
-        Utils.print_info("Synapse \"%s\" added to the crontab" % event_id)
-
-    def get_jobs(self):
-        """
-        Return all current jobs in the crontab
-        :return:
-        """
-        return self.my_user_cron.find_comment(CRONTAB_COMMENT)
-
-    def _remove_all_job(self):
-        """
-        Remove all line in crontab that are attached to Kalliope
-        """
-        iter_item = self.my_user_cron.find_comment(CRONTAB_COMMENT)
-        for job in iter_item:
-            logger.debug("remove job %s from crontab" % job)
-            self.my_user_cron.remove(job)
-        # write the file
-        self.my_user_cron.write()
-
-        # this is a fix for the CronTab lib
-        # see https://github.com/peak6/python-crontab/issues/1
-        new_iter = self.my_user_cron.find_comment(CRONTAB_COMMENT)
-        sum_job = sum(1 for _ in new_iter)
-        while sum_job > 0:
-            self._remove_all_job()
-
-    def _get_base_command(self):
-        """
-        Return the path of the entry point of Kalliope
-        Example: /home/user/kalliope/kalliope.py
-        :return: The path of the entry point script kalliope.py
-        :rtype: str
-        """
-        import inspect
-        import os
-        # get current script directory path. We are in /an/unknown/path/kalliope/core
-        cur_script_directory = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
-        # get parent dir. Now we are in /an/unknown/path/kalliope
-        parent_dir = os.path.normpath(cur_script_directory + os.sep + os.pardir)
-        # we add the kalliope.py file name
-        real_entry_point_path = parent_dir + os.sep + KALLIOPE_ENTRY_POINT_SCRIPT
-        # We test that the file exist before return it
-        logger.debug("Real Kalliope.py path: %s" % real_entry_point_path)
-        if os.path.isfile(real_entry_point_path):
-            crontab_cmd = "python %s start --brain-file %s --run-synapse " % (real_entry_point_path,
-                                                                              self.brain.brain_file)
-            return crontab_cmd
-        raise IOError("kalliope.py file not found")

+ 49 - 0
kalliope/core/EventManager.py

@@ -0,0 +1,49 @@
+from apscheduler.schedulers.background import BackgroundScheduler
+from apscheduler.triggers.cron import CronTrigger
+
+from kalliope.core.ConfigurationManager import BrainLoader
+from kalliope.core.SynapseLauncher import SynapseLauncher
+from kalliope.core import Utils
+from kalliope.core.Models import Event
+
+
+class EventManager(object):
+
+    def __init__(self, synapses):
+        Utils.print_info('Starting event manager')
+        self.scheduler = BackgroundScheduler()
+        self.synapses = synapses
+        self.load_events()
+        self.scheduler.start()
+
+    def load_events(self):
+        """
+        For each received synapse that have an event as signal, we add a new job scheduled
+        to launch the synapse
+        :return:
+        """
+        for synapse in self.synapses:
+            for signal in synapse.signals:
+                # if the signal is an event we add it to the task list
+                if type(signal) == Event:
+                    my_cron = CronTrigger(year=signal.year,
+                                          month=signal.month,
+                                          day=signal.day,
+                                          week=signal.week,
+                                          day_of_week=signal.day_of_week,
+                                          hour=signal.hour,
+                                          minute=signal.minute,
+                                          second=signal.second)
+                    Utils.print_info("Add synapse name \"%s\" to the scheduler: %s" % (synapse.name, my_cron))
+                    self.scheduler.add_job(self.run_synapse_by_name, my_cron, args=[synapse.name])
+
+    @staticmethod
+    def run_synapse_by_name(synapse_name):
+        """
+        This method will run the synapse
+        """
+        Utils.print_info("Event triggered, running synapse: %s" % synapse_name)
+        # get a brain
+        brain_loader = BrainLoader()
+        brain = brain_loader.brain
+        SynapseLauncher.start_synapse(synapse_name, brain=brain)

+ 23 - 5
kalliope/core/Models/Event.py

@@ -5,12 +5,21 @@ class Event(object):
     .. note:: Events are based on the system crontab
     """
 
-    def __init__(self, period):
-        self.period = period
+    def __init__(self, year=None, month=None, day=None, week=None, day_of_week=None,
+                 hour=None, minute=None, second=None):
+        self.year = year
+        self.month = month
+        self.day = day
+        self.week = week
+        self.day_of_week = day_of_week
+        self.hour = hour
+        self.minute = minute
+        self.second = second
 
     def __str__(self):
-        return "%s: period: %s" % (self.__class__.__name__,
-                                   self.period)
+        return "%s:  year: %s, month: %s, day: %s, week: %s, day_of_week: %s, hour: %s, minute: %s, second: %s" \
+               % (self.__class__.__name__, self.year, self.month, self.day, self.week,
+                  self.day_of_week, self.hour, self.minute, self.second)
 
     def serialize(self):
         """
@@ -21,7 +30,16 @@ class Event(object):
         """
 
         return {
-            'event': self.period
+            'event': {
+                "year": self.year,
+                "month": self.month,
+                "day": self.day,
+                "week": self.week,
+                "day_of_week": self.day_of_week,
+                "hour": self.hour,
+                "minute": self.minute,
+                "second": self.second,
+            }
         }
 
     def __eq__(self, other):

+ 5 - 4
setup.py

@@ -60,15 +60,15 @@ setup(
         'SpeechRecognition==3.4.6',
         'markupsafe==0.23',
         'pyaudio==0.2.9',
-        'ansible==2.1.1.0',
+        'ansible==2.2.0.0',
         'python2-pythondialog==3.4.0',
         'jinja==1.2',
         'python-crontab==2.1.1',
-        'cffi==1.8.3',
+        'cffi==1.9.1',
         'pygmail==0.0.5.4',
         'pushetta==1.0.15',
         'wakeonlan==0.2.2',
-        'ipaddress==1.0.16',
+        'ipaddress==1.0.17',
         'pyowm==2.5.0',
         'python-twitter==3.1',
         'flask==0.11.1',
@@ -78,7 +78,8 @@ setup(
         'httpretty==0.8.14',
         'mock==2.0.0',
         'feedparser==5.2.1',
-        'Flask-Testing==0.6.1'
+        'Flask-Testing==0.6.1',
+        'apscheduler==3.3.0'
     ],