Plugin binary part

TODO : “helpers” for special pages

Purpose

First, as already said in the chapter about creating the plugin repository on GitHub, there are some rules about a plugin name : don’t forget to respect them.

The binary part of a plugin will in all cases be a single file. Assuming your plugin name is myplugin, the file name will be bin/myplugin.py.

This file is the gateway between the library and Domogik. Basically it will:

  • import the needed libraries
  • check if the plugin is configured
  • get the plugin configuration values
  • instantiate the library class
  • use the library and thanks to callback functions communicate wih Domogik
  • set the plugin as ready when it is fully started

Depending on your plugin, more actions may be done:

  • after getting the configuration, get the devices list from the database (over the MQ). This is needed when the plugin can’t know the devices addresses, for example a weather plugin don’t know in which town you live, so it will check the weather for already created devices.
  • register a newly detected device. For example, if a new sensor is plugged on your onewire network or if you add a new temperature sensor compliant with RFXCOM devices, the related plugins (onewire and rfxcom) will be able to detect the new devices, and if the device is not known, it will be registered, so a notification will be send to the user interfaces over MQ.

XplPlugin and its useful functions

..todo ::
overview

Helper function

  • register_helper(action, help_string, callback)
  • publish_helper(key, data)

Hidden but useful features

..todo ::
check plugin not laucnhed, status, …

On demande features

..todo ::
  • ready, config, library, packages, resources, data getter functions
  • logging, force_leave (and return code),
  • MQ usage

Template of the binary part

Import the needed libraries

Then, you need to import some libraries needed by the plugin. Example:

from domogik.xpl.common.plugin import XplPlugin
from domogik.xpl.common.xplmessage import XplMessage

from domogik_packages.plugin_diskfree.lib.diskfree import Disk
import threading
import traceback

Some import lines are mandatory:

from domogik.xpl.common.plugin import XplPlugin # this one is needed as all the xPL plugins extends from the class XplPlugin. This class will provide you some useful functions to get the configuration parameters values, ...
from domogik.xpl.common.xplmessage import XplMessage # this one is needed to create a xPL message.

Then, you have to import your plugin library objects (here for the plugin diskfree we import a class):

from domogik_packages.plugin_diskfree.lib.diskfree import Disk

To finish, depending on the plugin needs, you may need to import some various librairies. Example:

import threading
import traceback

Create the main class

Now, you can create your main class. Here is an empty class template and the final part of the plugin bin file which will allow to instantiate this class when you execute the bin file:

class MypluginManager(XplPlugin):
    """ A description about the class
    """

    def __init__(self):
        """ The constructor of your class. This function  will be called when the class is instantiated.
        """
        XplPlugin.__init__(self, name='myplugin')
        # do some other actions

if __name__ == "__main__":
    MypluginManager()

Of course, rename MypluginManager and myplugin with your plugin name!

What should be done in the main class ?

Well, it depends on your plugin!

First, you should handle the configuration part:

  • get the global configuration elements for your plugin (see the configuration part of the json file)
  • eventually, get the created devices list and for each device get its configuration values
  • if needed, do some checks about the configuration elements and the devices

Then, start the features:

  • if the plugin manage some sensors, launch some threads to listen for the sensors (the threads should launch functions from the plugin library)
  • if the plugin manage some actuators, create some listeners for xPL commands messages. These listeners will call some callback functions (which should be defined in the library part)

Finally, tell Domogik that the plugin is ready:

  • when all is ready, call the ready() function

Focus on the start process

The plugin global configuration

If you except the mandatory auto_startup key, a plugin can have no configuration elements and another one can have multiple configuration elements. A plugin without configuration elements can be started without any configuration from the user (but maybe the user will need to create some devices, we will see this later). But for plugins with some configuration elements, the plugin developer may want to check that the plugin has been configured before the plugin can start! This can be done by calling the function ``self.check_configured()`` and check its return value:

# check if the plugin is configured. If not, this will stop the plugin and log an error
if not self.check_configured():
    return

How this functions works ? It is quite simple: when the user save the plugin configuration on a user interface, a configured key is inserted in database for the plugin and the host on which it is installed. If this key is not set to true in database, when you call this function, it will return False.

To retrieve a configuration parameter value, you just need to do this:

self.interval = self.get_config("interval")

This function will check in database if a value is set for the key interval and the host where the plugin is installed. If there is such a value, it will be cast returned and casted to the data type configured in the json file. If there is no value, the default value configured in the json file will be returned. So, you don’t need to cast the value and you don’t need to handle some default value as it is already done in the json file.

Notice that the auto_startup configuration key which is dynamically added to the configuration parameters set in the info.json file is not to handle in the plugin. This is a configuration key used only by the manager.

Get the created devices list for a plugin

Some plugins may need to know the devices on which they need to interact. To get the devices list, just do:

# get the devices list
self.devices = self.get_device_list(quit_if_no_device = True)

If your plugin needs some devices to be created to be run, set the parameter quit_if_no_device to True. If you do so and if no device exists when the plugin starts, the plugin will stop itself and log an error about this.

Then, you can make a look on the device list and for each device get its parameters and do something (launch a thread, create a listener, …). Example for the loop:

# loop on all found devices
for a_device in self.devices:
    try:
        # get the configuration values for the key 'the_key' for the sensor 'the_sensor' of the device
        the_value = self.get_parameter_for_feature(a_device, "xpl_stats", "the_sensor", "the_key")

        # do something
    except:
        # if there is an error, log it
        self.log.error(traceback.format_exc())
        # if the error is something blocking (a hardware gateway unavailable for example), you may want the plugin to stop.
        # if so, uncomment the following lines
        #self.force_leave()
        #return

Here is an example from the diskfree plugin:

# get the devices list
self.devices = self.get_device_list(quit_if_no_device = True)

# instantiate the class of the library
# notice that we sent as parameters some callbacks :
# * the logger object (self.log)
# * the function to send a xPL messages
# * a function to help to stop the plugin
disk_manager = Disk(self.log, self.send_xpl, self.get_stop())

# loop on all found devices
threads = {}
for a_device in self.devices:
    try:
        ### feature get_total_space
        # get the path to check for the sensor 'get_total_space' of the device. The configuration key is 'device'
        path = self.get_parameter_for_feature(a_device, "xpl_stats", "get_total_space", "device")
        # get the interval between each check for the sensor 'get_total_space' of the device. The configuration key is 'interval'
        interval = self.get_parameter_for_feature(a_device, "xpl_stats", "get_total_space", "interval")

        self.log.info("Start monitoring total space for '%s'" % path)
        # start a thread
        # the thread must be named to be explicit in the logs
        thr_name = "{0}-{1}".format(a_device['name'], "get_total_space")
        # create the thread
        threads[thr_name] = threading.Thread(None,
                                       disk_manager.get_total_space,
                                      thr_name,
                                      (path, interval,),
                                      {})
        # start the thread
        threads[thr_name].start()
        self.register_thread(threads[thr_name])


        # [...] 3 other features are managed like this in the plugin

    except:
        # if there is an error, log it
        self.log.error(traceback.format_exc())
        # we don't quit plugin if an error occurred
        # a disk can have been unmounted for a while
        #self.force_leave()
        #return

Notice in this example that we get the devices list, then instantiate the class of the library and finally, for each device, call a method (or more if needed) from the library class in a thread.

Perform updating devices process

After declaring an instance of an handling update method of devices, you can register it as callback

# register a callback device.update MQ message.
self.register_cb_update_devices(myHandleDeviceUpdate)

Your myHandleDeviceUpdate method will be called each device.update event comming from MQ pub message. That method must have devices list as parameters. Example of refreshing method to register

# A methode to handle updated devices by callback, in Class declaration :
def myHandleDeviceUpdate(self, devices):
    for hardDevice in self._myHardDevices:
        hardDevice.refreshAllDmgDevice(devices)
    self.log.info(u"All hard devives are updated from domogik devices")

Updating global parameters of device

# @param paramId: db id of global parameters getting from dict device['parameters']['myGlobalParam']['id']
# @param value : New parameter value.
# @return : True if success else False.
self.udpate_device_param(param_id, value)

Note

Overwrite Plugin class method on_message needed to call the parent method at first

# Your new overwrite methode
def on_message(self, msgid, content):
    Plugin.on_message(self, msgid, content)
    ....

Focus on the end : plugin ready

At the end of your __init__ function, just add these 2 lines:

self.ready()
self.log.info("Plugin ready :)")

Focus on devices tools

Get a data types specifications

If you need a data type detail definition to handle a sensor/command. To get it use Plugin class method

@param name :  the name of DT_Type
@return : dict DT_Type himself, empty dict if not find.
myDataType = self.get_data_type(name)

Get json product information

Get product informations from json plugin can helpfull for detection devices. To get it use Plugin class method

#    @param productId : part of productId set in json plugin. Search it in lower case if your <myProductId> is contained in product['id'] string.
#    @return : dict of product defined in json plugin with picture file name added. If not find empty dict.
#        {
#            "name" : The name of product,
#            "id" : product id, this is base name of picture,
#            "documentation" : Link to manufacturer documentation, manual, specification,
#            "type": Device_type linked to this product,
#            "picture" : if exist, file name of picture representing product without full path, else None
#       }
self.get_product_by_id(self, myProductId):

Get json sensor definition

Get sensor defined in json plugin, helpfull for detection devices

  • Search by Id

    #    @param id : sensor id to find set in json plugin.
    #    @return : dict of sensor if find else empty dict.
    sensor = self.get_sensor_by_id(mySensorId):
    
  • Search by name

    #    @param name : value of key name set in json plugin, Search in lower case.
    #        sensors" : {
    #            "my_sensor" : {
    #                "name" : "mySensorName", <== key compared to param <name>
    #                ...
    #    @return : dict of all sensors with same key name. If not find return empty dict.
    #        {
    #        "my_sensor1" : {
    #            "name" : "mySensorName",
    #            ...
    #            },
    #        "my_sensor2" : {
    #            "name" : "mySensorName",
    #            ...
    #            },
    #        ...
    #        }
    sensor = self.get_sensors_by_name(mySensorName)
    

You could have several returned sensors if sensors json plugin have same name

Get json command definition

Get command defined in json plugin, helpfull for detection devices - Search by Id

#    @param id : command id to find set in json plugin.
#    @return : dict of commands if find else empty dict.
command = self.get_command_by_id(myCommandId):
  • Search by key value

    #    @param key : value of one key parameter set in json plugin. Search in lower case.
    #        commands" : {
    #            "my_command" : {
    #                "name" : "foo name",
    #                "parameters" : [{
    #                        "key" : "myKeyValue", <== value of key compared to param <key>
    #                        ...
    #                        }]
    #    @return : dict of all command include same key. If not find return empty dict.
    #        {
    #        "my_command1" : {
    #            "name" : "foo name2",
    #                "parameters" : [{
    #                        "key" : "myKeyValue",
    #                        ...
    #                        }]
    #            },
    #        "my_command2" : {
    #            "name" : "foo name2",
    #                "parameters" : [{
    #                        "key" : "myKeyValue",
    #                        ...
    #                        }]
    #            },
    #        ...
    #        }
    
    commands = self.get_commands_by_key(myKeyValue)
    

You could have several returned commands if commands json plugin have same key value

Focus on the logs

There are 4 methods that enable you to log informations in the log files. The default logfiles are located in the directory configured in /etc/domogik/domogik.cfg as log_dir_path (default is /var/log/domogik/). The filename is myplugin.log.

Here are the 4 methods:

  • self.log.error("a message"): log an error message
  • self.log.warning("a message"): log a warning message
  • self.log.info("a message"): log an information message
  • self.log.debug("a message"): log a debug message

Error

The error messages are to be used to log all critical or blocking points. You must use this :

  • each time the plugin can’t open a resource (hardware, file, url)
  • each time the plugin catch some important exception, as a connexion lost to a hardware, url, …
  • if something is wrong in the plugin configuration

Warning

The warning messages are to be used to log the small errors (with no real impact : for example an incorrect value is read from a sensor but it may happen as the technology is not 100% reliable) or some strange behaviour that are not blocking points

Info

The info messages must be used for important informations:

  • some informations about the plugin startup
  • any information that may help someone who reads the log to check the plugin started successfully
  • give some informations about the hardware (if the plugin uses some hardware) : firmware version, configuration modes, …

You must not use this log level for debug messages, for example:

  • avoid to use this level in eternal loops (to avoir big log files). Use the debug level in this case

Debug

The debug log level is not activated with Domogik official packages, so these messages won’t be written in the log files. The development version of Domogik has this log level activated. The debug messages are to be used for: * logging any step in the plugin processing * log actions in the eternal loops * log important variable status * log any connection to a service or a hardware. For example, with a serial device you must log all that is sent to the device or all that is received from the device. These are very important informations for debugging! These informations can also be used later to create some automated tests or fix bugs and test them thanks to the user logs. * …

You can and you must use the debug messages! They are evry important when a bug is discovered as they allow anybody to check what is the issue.

Using the logs in the library part

See the library part documentation

Focus on xPL : send xPL messages and listen for xPL messages

Send xPL messages

To allow the plugin to send some xPL messages, you must create such a function in the binary part. The function will not be the same for all the plugins and depends on the plugin features and on the xPL schema used.

Here is a template:

def send_xpl(self, arg1, arg2, ...):
    """ Send xPL message on network
    """
    self.log.debug("Sending xPL message for arg1={0}, arg2={1}...".format(arg1, arg2))
    msg = XplMessage()
    msg.set_type("xpl-stat")
    msg.set_schema("aschema.basic")
    msg.add_data({"key1" : arg1})
    msg.add_data({"key2" : arg2})
    # ...
    self.myxpl.send(msg)

This function should be called from the library part. To allow the library part to call this function, you must sends this function as a callback parameter to the library.

Here is an example of the function for the sensor.basic xPL message:

def send_xpl(self, address, type, value):
    """ Send xPL message on network
    """
    self.log.debug("Values for {0} on {1} : {2}".format(type, address, value))
    msg = XplMessage()
    msg.set_type("xpl-stat")
    msg.set_schema("sensor.basic")
    msg.add_data({"device" : address})
    msg.add_data({"type" : type})
    msg.add_data({"current" : value})
    self.myxpl.send(msg)

Let’s see how to use it with the library…

First, when you instantiate the class of the library, set the function as a parameter (this is call a callback):

my_object = MyLibraryClass(self.log, self.send_xpl, self.get_stop())

As you can see here, the self.send_xpl function is set as a parameter. We will see in the library documentation how to use it.

Listen for xPL messages

Todo

TODO !

Focus on launching threads

When you want to launch an infinite task, you can use a thread. In this thread you can have an infinite loop with a timer in it or an infinite reading on something.

To be sure that the thread could be stopped when the plugin is requested to stop, you will need to use a stop flag in the library. So you need to give this stop flag to the library, for example when instantiating the library class. Example:

my_object = MyLibraryClass(self.log, self.send_xpl, self.get_stop())

The function self.get_stop() will get the stop flag of the plugin and so give it to my_object. Then, in the methods of the library class, you will be able to handle this flag in the loops. You can read the library documentation to see how to handle it.

Then, you can create your thread. Example:

import threading
# [...]

class MypluginManager(XplPlugin):
    # [...]

    def a_function(self):
        # [...]

        # instantiate the class of the library
        my_object = MyLibraryClass(self.log, self.send_xpl, self.get_stop())

        # set the thread name
        thr_name = "a_name_for_my_thread"
        # create the thread
        my_thread = threading.Thread(None,
                                      my_object.the_function,
                                      thr_name,
                                      (arg1, arg2,),
                                      {})
        # start the thread
        my_thread.start()
        self.register_thread(my_thread)

Notice that a function self.register_thread() has been called. This function is important is it register the thread in a list. When the plugin is requested to stop, this list is used to help killing the existing threads! If your plugin doesn’t stop, please check that all your threads are registered.

Focus on the json file

If you need, for some reasons to access the json file content, please notice that:

  • you should really ask yourself if the data you are looking for is not available somewhere else (over MQ or over REST).
  • the json content is available in self.json_data. Don’t write in it!!!