HTTPMOD
| HTTPMOD | |
|---|---|
| Zweck / Funktion | |
| Extract information from devices with an HTTP interface (or, more generic, from any URL) | |
| Allgemein | |
| Typ | Gerätemodul |
| Details | |
| Dokumentation | EN / DE |
| Support (Forum) | Sonstiges |
| Modulname | 98_HTTPMOD.pm |
| Ersteller | StefanStrobel (Forum / Wiki) |
| Wichtig: sofern vorhanden, gilt im Zweifel immer die (englische) Beschreibung in der commandref! | |
The module HTTPMOD provides a generic way to retrieve information from devices with an HTTP interface and store it in readings. It queries a given URL with headers and data defined by attributes.
From the HTTP response it extracts readings named in attributes using regular expressions also defined by attributes.
Availability
The module has been checked in to the FHEM SVN
Prerequisites
This module uses the non blocking HTTP function HttpUtils_NonblockingGet provided by FHEM's HttpUtils in a new version published in December 2013.
If not already installed in your environment, please update FHEM or install it manually using appropriate commands from your environment.
Define
define <name> HTTPMOD <URL> <Interval>
The module connects to the given URL every Interval seconds, sends optional headers and data and then parses the response.
Example:
define PM HTTPMOD http://MyPoolManager/cgi-bin/webgui.fcgi 60
Set-Commands
None implemented so far
Get-Commands
None implemented so far
Attributes
- do_not_notify
- readingFnAttributes
- requestHeader.*
- Define an additional HTTP Header to set in the HTTP request
- requestData
- POST Data to be sent in the request. If not defined, it will be a GET request as defined in HttpUtils used by this module
- readingsName.*
- the name of a reading to extract with the corresponding readingRegex
- readingsRegex.*
- defines the regex to be used for extracting the reading. The value to extract should be in a sub expression e.g. ([\d\.]+) in the above example
- readingsExpr.*
- defines an expression that is used in an eval to compute the readings value. The raw value will be in the variable $val.
Configuration of HTTP Devices
If your device expects special HTTP-headers then specify them as attr requestHeader1 to attr requestHeaderX.
If your Device expects an HTTP POST instead of HTTP GET then the POST-data can be specified as attr requestData.
To get the readings, specify pairs of attr readingNameX and attr readingRegexX to define which readings you want to extract from the HTTP response and how to extract them. The actual values to be extracted have to be sub expressions within () in the regex (see example below)
Example for a PoolManager 5:
The PoolManager Web GUI can be queried with HTTP POST Requests like this one:
POST /cgi-bin/webgui.fcgi HTTP/1.1
Host: 192.168.70.90
Accept: */*
Content-Type: application/json;charset=UTF-8
Content-Length: 60
{"get" :["34.4001.value" ,"34.4008.value" ,"34.4033.value"]}
The resulting HTTP Response would look like this:
HTTP/1.1 200 OK
Content-type: application/json; charset=UTF-8
Expires: 0
Cache-Control: no-cache
Date: Sun, 12 Jan 2014 12:23:11 GMT
Server: lighttpd/1.4.26
Content-Length: 179
{
"data": {
"34.4001.value": "7.00",
"34.4008.value": "0.52",
"34.4033.value": "24.8"
},
"status": {
"code": 0
},
"event": {
"type": 1,
"data": "48.30000.0"
}
}
To configure HTTPMOD for a PoolManager one would first define a PoolManager device with e.g. the name PM, the URL and an interval of e.g. 60 seconds.
Then the data to be sent in the request needs to be defined because in this example the device expects a POST request so the query is not contained in the URL but in the request data.
Also as seen above the device expects special HTTP headers in the request so these headers also need to be defined as attr PM requestHeader1 and attr PM requestHeader2
Then the names of the readings to be extracted would be set with attributes
Then for each reading value to be extracted a regular expression needs to be set that will match the value in question within ().
Example:
define PM HTTPMOD http://MyPoolManager/cgi-bin/webgui.fcgi 60
attr PM requestData {"get" :["34.4001.value" ,"34.4008.value" ,"34.4033.value", "14.16601.value", "14.16602.value"]}
attr PM requestHeader1 Content-Type: application/json
attr PM requestHeader2 Accept: */*
attr PM readingsName1 PH
attr PM readingsName2 CL
attr PM readingsName3 TEMP
attr PM readingsRegex1 34.4001.value":[ \t]+"([\d\.]+)"
attr PM readingsRegex2 34.4008.value":[ \t]+"([\d\.]+)"
attr PM readingsRegex3 34.4033.value":[ \t]+"([\d\.]+)"
attr PM stateFormat {sprintf("%.1f Grad, PH %.1f, %.1f mg/l Chlor", ReadingsVal($name,"TEMP",0), ReadingsVal($name,"PH",0), ReadingsVal($name,"CL",0))}
Example for AmbientMonitor
AmbientMonitor is a webbased visualisation for sensors connected to an Arduino device. Its web interface can also be queried with HTTMOD to grab the data into readings.
This example was provided by locutus. The hardware configuration is an Arduino + Ethercard with ENC28J60 Controller + DHT22 Sensor and software can be downloaded from https://github.com/lucadentella/AmbientMonitor
In this example an HTTP GET is sufficent, so no requestData is needed. The device provides temperature and humidity readings in an HTTP response that looks like:
HTTP/1.0 200 OK
Content-Type: text/html
myCB({'temperature':22.00,'humidity':46.00})
the definition could be:
define AmbientMonitor HTTPMOD http://192.168.1.221/?callback=? 300
attr AmbientMonitor requestHeader Content-Type: application/json
attr AmbientMonitor readingsName1 Temperatur
attr AmbientMonitor readingsName2 Feuchtigkeit
attr AmbientMonitor readingsRegex1 temperature':([\d\.]+)
attr AmbientMonitor readingsRegex2 humidity':([\d\.]+)
attr AmbientMonitor stateFormat {sprintf("Temperatur %.1f C, Feuchtigkeit %.1f %", ReadingsVal($name,"Temperatur",0), ReadingsVal($name,"Feuchtigkeit",0))}
Some help with Regular Expressions
If HTTPMOD seems not to work and the FHEM Logfile contains a message like
HTTPMOD: Response didn't match Reading ...
then you should check if the value you want to extract is read into the internal with the name buf. Internals are visible when you click on the defined HTTPMOD Device. buf is an internal variable that contains the HTTP Response read. If the value is there and you get the mentioned message then probably something is wrong with your regular expression. If you are new to regular expressions then the introduction at http://perldoc.perl.org/perlretut.html might be helpful.
For a typical HTTPMOD use case where you want to extract a number out of a HTTP-Response you can use something like [\d\.]+ to match the number itself. The expression matches the number characters (\d) or a . if one of these characters occurs at least once.
To tell HTTPMOD that the number is what you want to use for the reading, you have to put the expression in between (). A ([\d\.]+) alone would match the longest number in the HTTP Response which is very likely not the number you are looking for so you need to add something to the expression to give it a context and define how to find the number that you are looking for.
if there is a title text before the number or a special text after the number you can put this in the regex. In one of the examples above humidity':([\d\.]+) is looking for the number that immediately follows the text humidity': without any blanks in between.
Additional notes
If you don't know which URLs, headers or POST data your web GUI uses, you might try a local proxy like BurpSuite BurpSuite to track requests and responses
Future extensions might include attributes to define set commands in a generic way. If a device allows setting of values via HTTP then for each set a name and a corresponding URL with headers and data might be a possible way forward.
Links
- Thread in Fhem Forum that discusses this module
- Introduction to regular expressions
- BurpSuite: Tool (local proxy) to help analyze http traffic