Viewing entries in
Smart Home

EmonCMS and multiple OpenTRV nodes

1 Comment

EmonCMS and multiple OpenTRV nodes

As a follow up on my initial experiments with getting data from OpenTRV to Open Energy Monitor this post is going to build on that and add support for parsing data received from the other OpenTRV nodes other than the one directly connected via the USB serial.

The Hardware

The hardware I am using is a Rev 2 board as the receiver as described previously and the transmitters are two Rev 11 boards. Due to the design of the firmware however this should work the same on any of the devices.

Firmware

To get these to talk to each other we need to ensure the firmware for on the receiver device has the ENABLE_STATS_RX build option defined and on the transmitting end ENABLE_STATS_TX needs to be defined.

Testing

With the new firmware built and flashed it is time to power up and see what we get. Load up the Node-RED UI with the work sheet we created last time. Turn off all the debug nodes apart from the last one of the switch node. This will now only show the console output we are not parsing, (you may need to wait about 10 mins to see anything).

You should see some JSON output that looks suspiciously like the end of the status lines from last time. It is probably of no surprise that in fact in is the same format as the JSON part of the status text. So we already have the code to parse this text.

Parsing The Output

First things first though, we need to separate out the JSON lines from the rest of the unhandled Lines. Double click on the switch node and add a new entry above the otherwise entry. Set this to look for a regex and enter the following;  

^{

This will now pull out the JSON by detecting the opening  { . 

Now connect a new function node to a new switch output and grab the JSON code we wrote before; 

var obj = JSON.parse('{'+parts[16]+'}');
for(var i in obj) 
{
    switch(i)
    {
        case '@':
            newMsg.nodeid = obj['@'];
            break;
        case 'T|C16':
            var temp16 = parseInt(obj[i]);
            var temp = ((temp16 & ~0xf) >> 4) + 
                       ((temp16 & 0xf) / 16);
            newMsg.payload.temp = temp;
            break;
        default:
            var mapping = {
                'B|mV': 'battery',
                'occ|%': 'occupancy',
                'v|%': 'valve',
                'O': 'occupancyState',
                'vac|h': 'vacancyHours',
                'L': 'ambientLight',
                'H|%': 'humidity',
                'vC|%': 'cumulativeValveMovement',
                'tT|C': 'nominalTarget'
            }
            
            if(mapping[i]) {
                newMsg.payload[mapping[i]] = obj[i];
            }
    }
}

We have to make a few changes to make this stand alone. First the JSON is coming from msg.payload  rather than parts[16] and we are going to write the parsed output back directly to msg.payload. The reason for this will become clear soon.

var json = msg.payload;
msg.payload = {};

var obj = JSON.parse(json);
for(var i in obj) 
{
    switch(i)
    {
        case '@':
            msg.nodeid = obj['@'];
            break;
        case 'T|C16':
            var temp16 = parseInt(obj[i]);
            var temp = ((temp16 & ~0xf) >> 4) + 
                       ((temp16 & 0xf) / 16);
            msg.payload.temp = temp;
            break;
        default:
            var mapping = {
                'B|mV': 'battery',
                'occ|%': 'occupancy',
                'v|%': 'valve',
                'O': 'occupancyState',
                'vac|h': 'vacancyHours',
                'L': 'ambientLight',
                'H|%': 'humidity',
                'vC|%': 'cumulativeValveMovement',
                'tT|C': 'nominalTarget'
            };
            
            if(mapping[i]) {
                msg.payload[mapping[i]] = obj[i];
            }
    }
}

return msg;

Now let's insert a debug node and test see if this works.

Sending to EmonCMS

Now all that is left to do disconnect the function output to the EmonCMS node, or is it? You may have noticed I the previous example that we entered the EmonCMS node ID in the settings of the EmonCMS node in Node-RED. Now we are receiving from more than one OpenTRV device we need to translate the OpenTRV ID to an EmonCMS node ID. So we need to write a function to map the IDs from one system to another. 

Place a function node after the two parsing nodes. and before the EmonCMS node;

Our parsing code is writing the OpenTRV ID in to msg.nodeid and if the Node setting is left blank the EmonCMS node will usemsg.nodegroup for the Node. This can be done with the following;

var nodeMapping = {
    'f9ea': 26,
    'a7de': 27
};

if(msg.nodeid && nodeMapping[msg.nodeid]) {
    msg.nodegroup = nodeMapping[msg.nodeid];
}

return msg;

You will need to alter the content of nodeMapping for your environment.

As we are dealing with more than just msg.payload it is handy to debug the whole msg object. To do this you can open a debug node's settings (double click it) and change the Output to complete msg object.

 
 

The other change we need to make is to clear the fixed Node setting in the EmonCMS node. Open the EmonCMS node settings and delete anything in the Node setting. Don't worry if the setting highlights red, this can be ignored as we are passing in the setting for this value. A warning may also be given when deploying, this too can be ignored.

 
 

We can now deploy the sheet and it should look something like this;

If all is working you should now see all the OpenTRV nodes showing data in your EmonCMS.

Making Improvements

We could leave it there, but there is a bit of tidy up we can do to help reduce the maintenance. For example the newer versions of the firmware send the battery level using 'B|cV'. To add support for this we would have to edit two similar pieces of code. We can easily update the code to put all the JSON parsing in a single node.

Instead of parsing the JSON in the OpenTRV Parse Status we can pass the JSON along to the OpenTRV Parse JSON node so it can be parse there. Lets disconnect the output of the OpenTRV Parse Status from the EmonCMS node and to the inout of theOpenTRV Parse JSON so we have something that looks like this;

Now first lets stop OpenTRV Parse Status from parsing JSON and pass it on. We do not want the JSON to be part of themsg.payload as we have not parsed it, only extracted it from the status line so we are going to pass it on as msg.json. Delete the JSON parsing code and replace with the following;

// JSON block, 16
if(null !== parts[16]) 
{
    newMsg.json = '{'+parts[16]+'}';
}

Now in the OpenTRV Parse JSON node we need to pick up any JSON data in msg.json and parse it while also keeping the ability to parse the JSON coming directly from the serial. Replace the first two lines with the following;

var json = "";
if(msg.json) 
{
    json = msg.json
}
else
{
    json = msg.payload;
    msg.payload = {};
}

We can now deploy and test that we are getting the same data uploaded to EmonCMS.

Now we have the JSON parsing in one location lets fix the previously mentioned issue with not parsing the new battery voltage. The easiest fix for this is to change the B|mV to B|cV and if you just setting up the system this is absolutely fine. In my case however I have some historical data so I am going to scale the voltage to mV before passing on, add the following to the switch statement;

        case 'B|cV':
            msg.payload.battery = parseInt(obj[i]) * 10;
            break;

While we are updating the actual JSON parsing code another change I am going to make is to not drop the unknown JSON values we do not lose any data. Be warned however if you are using an older version of EmonCMS this may not be supported. Change the following;

            if(mapping[i]) {
                msg.payload[mapping[i]] = obj[i];
            }

to;

            if(mapping[i]) {
                msg.payload[mapping[i]] = obj[i];
            } else {
                msg.payload[i] = obj[i];
            }

Finally one last change we need to make. The JSON frames sent from our remote OpenTRV devices include the '+' property, an incrementing count to help with missing frame detection so we can just give it a name in our mapping;

            var mapping = {
                '+': 'frame',
                'B|mV': 'battery',
                'occ|%': 'occupancy',
                'v|%': 'valve',
                'O': 'occupancyState',
                'vac|h': 'vacancyHours',
                'L': 'ambientLight',
                'H|%': 'humidity',
                'vC|%': 'cumulativeValveMovement',
                'tT|C': 'nominalTarget'
            };

So in the end you should end up with something like;

var json = "";
if(msg.json) 
{
    json = msg.json
}
else
{
    json = msg.payload;
    msg.payload = {};
}

var obj = JSON.parse(json);
for(var i in obj) 
{
    switch(i)
    {
        case '@':
            msg.nodeid = obj['@'];
            break;
        case 'T|C16':
            var temp16 = parseInt(obj[i]);
            var temp = ((temp16 & ~0xf) >> 4) + 
                       ((temp16 & 0xf) / 16);
            msg.payload.temp = temp;
            break;
        case 'B|cV':
            msg.payload.battery = parseInt(obj[i]) * 10;
            break;
        default:
            var mapping = {
                '+': 'frame',
                'B|mV': 'battery',
                'occ|%': 'occupancy',
                'v|%': 'valve',
                'O': 'occupancyState',
                'vac|h': 'vacancyHours',
                'L': 'ambientLight',
                'H|%': 'humidity',
                'vC|%': 'cumulativeValveMovement',
                'tT|C': 'nominalTarget'
            };
            
            if(mapping[i]) {
                msg.payload[mapping[i]] = obj[i];
            } else {
                msg.payload[i] = obj[i];
            }
    }
}

return msg;

So now we can deploy and check the results;

The Quick Way

Finally just if you get stuck here is the flow to import in to Node-RED;

[{"id":"c6a48855.395b78","type":"emoncms-server","z":"","server":"http://localhost/emoncms","name":"EmonCMS"},{"id":"e914c08d.16eb4","type":"serial-port","z":"1cd21157.e32def","serialport":"/dev/ttyUSB1","serialbaud":"4800","databits":"8","parity":"none","stopbits":"1","newline":"\\n","bin":"false","out":"char","addchar":false},{"id":"9fccc574.603338","type":"serial in","z":"1cd21157.e32def","name":"","serial":"e914c08d.16eb4","x":171,"y":327,"wires":[["f5140166.0aec"]]},{"id":"f5140166.0aec","type":"function","z":"1cd21157.e32def","name":"Trim","func":"msg.payload = msg.payload.trim();\nreturn msg;","outputs":1,"noerr":0,"x":371,"y":327,"wires":[["895c112b.76a3f","6b5f8388.94a07c"]]},{"id":"895c112b.76a3f","type":"debug","z":"1cd21157.e32def","name":"","active":true,"console":"false","complete":"true","x":551,"y":267,"wires":[]},{"id":"6b5f8388.94a07c","type":"switch","z":"1cd21157.e32def","name":"","property":"payload","propertyType":"msg","rules":[{"t":"regex","v":"^$","vt":"str","case":false},{"t":"eq","v":">","vt":"str"},{"t":"regex","v":"^=","vt":"str","case":false},{"t":"regex","v":"^{","vt":"str","case":false},{"t":"else"}],"checkall":"false","outputs":5,"x":551,"y":327,"wires":[["429c7c77.bd6384"],["64924701.9b6db8"],["350ff59f.caf00a","4983e01b.b67c2"],["21d5644.fde2a9c","bfa41a66.e84468"],["b10edb47.a92cd8"]]},{"id":"429c7c77.bd6384","type":"debug","z":"1cd21157.e32def","name":"","active":false,"console":"false","complete":"false","x":770,"y":180,"wires":[]},{"id":"64924701.9b6db8","type":"debug","z":"1cd21157.e32def","name":"","active":false,"console":"false","complete":"false","x":770,"y":220,"wires":[]},{"id":"350ff59f.caf00a","type":"debug","z":"1cd21157.e32def","name":"","active":false,"console":"false","complete":"false","x":770,"y":260,"wires":[]},{"id":"21d5644.fde2a9c","type":"debug","z":"1cd21157.e32def","name":"","active":false,"console":"false","complete":"false","x":770,"y":380,"wires":[]},{"id":"4983e01b.b67c2","type":"function","z":"1cd21157.e32def","name":"OpenTRV Parse Status","func":"var newMsg = {  };\n\nvar parts = msg.payload.match(/^=(F|W|B)([0-9]+)%@([0-9]+)C([0-9A-F]+)(;X([0-9]+))?(;T([0-9]+ [0-9]+) ([^;]*))?(;S([0-9]+) ([0-9]+) ([0-9]+))?(;H([0-9]+ [0-9]+)*)?;?.*{(.*)}$/);\nif(null === parts) {\n    node.error(\"Failed to parse input\", msg);\n    return null;\n}\n\nnewMsg.payload = {\n    //'raw': parts,\n    'frost': 'F' == parts[1] ? 1 : 0,\n    'warm': 'W' == parts[1] ? 1 : 0,\n    'bake': 'B' == parts[1] ? 1 : 0,\n    'valve': parts[2],\n    'temp': parseInt(parts[3]) + (parseInt(\"0x\"+parts[4]) / 16)\n};\n\n// Security, 5-6\nif(null !== parts[5]) {\n}\n\n// Time, 7-8\nif(null !== parts[7]) {\n}\n\n// Program, 9\nif(null !== parts[9]) {\n}\n\n// Target temps, 10-13\nif(null !== parts[10]) \n{\n    newMsg.payload.nominalTarget = parseInt(parts[11]);\n    newMsg.payload.frostTarget = parseInt(parts[12]);\n    newMsg.payload.warmTarget = parseInt(parts[13]);\n}\n\n// House code, 14-15\nif(null !== parts[14]) {\n}\n\n// JSON block, 16\nif(null !== parts[16]) \n{\n    newMsg.json = '{'+parts[16]+'}';\n}\n\nreturn newMsg;","outputs":1,"noerr":0,"x":810,"y":300,"wires":[["bfa41a66.e84468","9b8ae703.987188"]]},{"id":"297b83ac.d6847c","type":"debug","z":"1cd21157.e32def","name":"","active":true,"console":"false","complete":"true","x":1550,"y":300,"wires":[]},{"id":"d742c271.28bd4","type":"emoncms","z":"1cd21157.e32def","name":"Emoncms","emonServer":"c6a48855.395b78","nodegroup":"","x":1560,"y":340,"wires":[]},{"id":"bfa41a66.e84468","type":"function","z":"1cd21157.e32def","name":"OpenTRV Parse JSON","func":"var json = \"\";\nif(msg.json) \n{\n    json = msg.json\n}\nelse\n{\n    json = msg.payload;\n    msg.payload = {};\n}\n\nvar obj = JSON.parse(json);\nfor(var i in obj) \n{\n\tswitch(i)\n\t{\n\t\tcase '@':\n\t\t\tmsg.nodeid = obj['@'];\n\t\t\tbreak;\n\t\tcase 'T|C16':\n\t\t\tvar temp16 = parseInt(obj[i]);\n\t\t\tvar temp = ((temp16 & ~0xf) >> 4) + \n\t\t\t\t\t   ((temp16 & 0xf) / 16);\n\t\t\tmsg.payload.temp = temp;\n\t\t\tbreak;\n        case 'B|cV':\n            msg.payload.battery = parseInt(obj[i]) * 10;\n            break;\n\t\tdefault:\n\t\t\tvar mapping = {\n\t\t\t    '+': 'frame',\n\t\t\t\t'B|mV': 'battery',\n\t\t\t\t'occ|%': 'occupancy',\n\t\t\t\t'v|%': 'valve',\n\t\t\t\t'O': 'occupancyState',\n\t\t\t\t'vac|h': 'vacancyHours',\n\t\t\t\t'L': 'ambientLight',\n\t\t\t\t'H|%': 'humidity',\n\t\t\t\t'vC|%': 'cumulativeValveMovement',\n\t\t\t\t'tT|C': 'nominalTarget'\n\t\t\t};\n\t\t\t\n\t\t\tif(mapping[i]) {\n\t\t\t\tmsg.payload[mapping[i]] = obj[i];\n\t\t\t} else {\n\t\t\t\tmsg.payload[i] = obj[i];\n\t\t\t}\n\t}\n}\n\nreturn msg;","outputs":1,"noerr":0,"x":1070,"y":340,"wires":[["8f1b296d.4a5cb8","251ad4fd.73893c"]]},{"id":"b10edb47.a92cd8","type":"debug","z":"1cd21157.e32def","name":"","active":false,"console":"false","complete":"false","x":770,"y":420,"wires":[]},{"id":"8f1b296d.4a5cb8","type":"function","z":"1cd21157.e32def","name":"Map ID for EmonCMS","func":"var nodeMapping = {\n    'f9ea': 26,\n    'a7de': 27\n};\n\nif(msg.nodeid && nodeMapping[msg.nodeid]) {\n    msg.nodegroup = nodeMapping[msg.nodeid];\n}\n\nreturn msg;\n","outputs":1,"noerr":0,"x":1340,"y":340,"wires":[["d742c271.28bd4","297b83ac.d6847c"]]},{"id":"9b8ae703.987188","type":"debug","z":"1cd21157.e32def","name":"","active":false,"console":"false","complete":"true","x":1010,"y":300,"wires":[]},{"id":"251ad4fd.73893c","type":"debug","z":"1cd21157.e32def","name":"","active":false,"console":"false","complete":"true","x":1290,"y":300,"wires":[]}]

1 Comment

Top