Read a DHT sensor on Raspberry Pi and mqtt the results

In an earlier post I showed how one could address the raspberry GPIO pins, in fact that is quite straightforward given the fact they are digital I/O pins that either are HIGH or LOW.
But what if one wants to read a sensor like a DHT11 attached to one of the GPIO pins? That is also not so difficult, but to make it more all round and easy to use in OpenHAB, I wanted to read a sensor and then MQTT the result to my MQTT broker.
In my case, my MQTT broker is on the same raspberry that I also aimed to attach the DHT11 to, so if I wanted I could read it directly, but just in case I wanted to read it from another raspberry, MQTT is a better solution (for me).

As it is no use to re-invent the wheel, I fortunately found someone already had done most of the work, albeit for a DHT22, but that was easy to alter.

For the next steps I presume you already have python installed. Jessie already should have python 2.7 installed, and should you want to upgrade that, check here.
The Python program calls a few libraries that you most likely already have, maybe with exception of the paho.mqtt.client and the adafruit DHT library.
the mqtt client can be installed as follows:*)
pip install paho-mqtt
if for whatever reason this does not work for you, try:
git clone https://github.com/eclipse/paho.mqtt.python.git => download the client
cd paho.mqtt.python => go to proper directory
python setup.py install => install the client
depending on how you are logged in you may need to add ‘sudo’ before the install commands

The mqtt client is now installed.

For the DHT library do the following:
git clone https://github.com/adafruit/Adafruit_Python_DHT.git => download the library
cd Adafruit_Python_DHT => go to proper directory

Before you install the library, you need some dependencies that may or may not be on your system:
sudo apt-get update
sudo apt-get install build-essential python-dev python-openssl
If you already had modules installed those will be skipped.
Then install the library:
sudo python setup.py install => install the library

At this moment it is a good idea to restart your raspberry.
If you want -when your raspberry is back up and running- you could now try one of the example programs in the Adafruit DHT library, just to make sure everything is working.
Connect your DHT11 to e.g. GPIO4 of your raspberry.
Make sure you are still in  directory: Adafruit_Python_DHT

do:
cd examples
sudo ./AdafruitDHT.py 11 4

You will get the temperature and Humidity reported in your terminal

Now download the DHT_sensor program.
If like me you want to use a DHT11 instead of a DHT22, change line 45:
DHT_TYPE = Adafruit_DHT.DHT22
into
DHT_TYPE = Adafruit_DHT.DHT11

and save the program on your raspberry

I saved it directly in my home/openhabian directory, i.e. the directory that is availble directly after logon

Now have a look at the usage instructions in the program (lines 36-42), but beware:
Presuming you saved the program as “mqtt.dht.sensor.py”, that is the name you need to use instead of “mqtt.channel.py” (the name mentioned in the “Usage”).
also.
I started my program as follows:

./mqtt.dhtsensor.py 'cupboard/temperature1' 'cupboard/humidity1' 4

This says that as my mqtt topics I have cupboard/temperature1 and cupboard/humidity1 and my DHT11 sensor is attached to GPIO pin 4 while the refresh rate is 300 seconds (the default).

To make it a bit easier I created a file called “dht.sh” that contained the single line:
./mqtt.dhtsensor.py 'cupboard/temperature1' 'cupboard/humidity1' 4 50
That does the same as the earlier line, but now I set the refresh rate to 50 seconds. I put it in one file because that is easier to type.
If you want you can make the dht.sh file to automatically start at boot as I describe below.

The MQTT messages are subsequently sent out over the network

And one could use definitions like below in your OpenHAB Items file to use them in OpenHAB

Number Cupboard_temp "Riser temperatuur [%.1f °C]" <temperature> (GF_Corridor) {mqtt="<[mosquitto:cupboard/temperature1:state:default]"}
Number Cupboard_humidity "Riser humidity [%.0f %%]" <line> (GF_Corridor) {mqtt="<[mosquitto:cupboard/humidity1:state:default]"}
Starting the script at boot

The script will run perfectly when started manually, but it will run in an instance of the SSH window and the moment you close that window, the script will stop. It is therefore better to have the script starting at boot. This is simple to do, but you have to be aware of the full PATH you are using.
Change the content of the dht.s file as follows:
Then in your preferred editor open the etc/rc.local file and add a line (the line at the red arrow):
Save the file and reboot. You will now no longer see the temperature and humidity in an ssh window, but the MQTT commands are being sent.

As this program sends its results by MQTT it can be used on other raspberry’s as well, noit just the one you have your openhabian and or MQTT broker installed on. Be sure though that if you install it on another raspberry, you alter the ‘servername’  variable into the name or ip address of the  MQTT broker

A word of warning:
If you are eager to test the program, before you  connected a sensor, it will just skip the sensor and you will get no useful feedback.
Furthermore, Python is quite sensitive to using the proper indentation in its programs as these have a meaning  in how the program is being interpreted, so if you go edit the program, be sure to keep the proper indentation

Added: Errors
If you install the above on an openhabian system you are unlikely to get any errors. However, I tested it on a freshly installed raspian Stretch system (not sure anymore if it was ‘Lite’) and I came upon the following Errors that were easy to fix:
Error
-bash: git: command not found
Solution
sudo apt-get install git
Error
File “setup.py”, line 5, in
from setuptools import setup, find_packages
ImportError: No module named setuptools
Solution
sudo apt-get install python-setuptools
Error
no module named requests
Solution
sudo pip install requests
Error
No pip
Solution
apt install python-pip #python 2
Error
Append error, logging in again: [Errno -2] Name or service not known
Solution
Most likely you are running the file from another raspbery where your MQTT server is not located. Make sure you alter the variable ‘servername’ in the mqtt.dhtsensor.py file into the servername or ip number where your mqtt broker is situated. Also make sure you do not have two clients running with the same name.

Error
“no module named Adafruit_DHT”
Solution
Go to the Adafruit_Python_DHT folder and do:
sudo python setup.py install --force

 

_________________________________

*) I could have used the mosquitto client that was on my raspberry that was already functioning as a broker, but as I also wanted to use it on other raspberries, it was better to use the lightweight paho.mqtt.client

IFTTT Webhooks on Arduino

IFTTT is a real handy service that allows you to make applets doing a variety of things, like email you the weather update every day.
Also, most home automation services like ‘Openhab’, ‘NodeRed’ or ‘HomeAssistent’ have a link with IFTTT so you can use your own data to trigger events or have have IFTTT take action in your home automation application, like switch on lights.

But, what if you dont have a homeautomation system and you just want your individual lonely arduino with ethernetshield to trigger something in IFTTT?

Fortunately IFTTT has a service for that called ‘Webhooks’ (it used to be called ‘Maker’). I have used that service in the past out of curiosity, but recently I needed it again and sadly had to scratch my head on ‘how did I do that again?’
So after getting myself re-aquainted with it, it might be good to share that with people who are interested in using it.

Therefore, let’s make a program that uses IFTTT to Tweet data every time the garagedoor is opened. Yes I know, you can tweet directly from an Arduinoprogram as well, but that’s not the point.

First ofcourse you need to register for IFTTT if you haven’t yet done so already.
Then you create a new applet and for the “IF” service, you choose “webhook”. If you never have used webmaker before, it will ask you to connect that service and provide you with an API key. Store that one, you will need it.
In Step 2 it will ask you for an “event name”.
this is a free to choose name that we will use later to trigger the event. Let’s make that ‘garage_door’.

 

 

 

In step 3 and 4 we chose the Twitter action. If you never used twitter before with IFTTT, you will now be asked for authorization from Twitter.

In Step 5 we format the message that we want to Tweet. The picture shows how I have done that.
Save the applet

We now have finished the IFTTT part, but before we go on to the Arduino part, we will test it:
Go to your browser and type:

https://maker.ifttt.com/trigger/garage_deur/with/key/d56e34gf756/?value1=8&value2=5&value3=good%20morning

in which “d56e34gf756” should be replaced by YOUR apikey.

Within moments you should see your tweet appear in your timeline.

So all we need to do now is to have an Arduino make a POST or GET webrequest akin to the one above.
However, in this case I will not add the values as arguments but as a JSON, but it gives the same result

You will find the code belonging to this action on github.
Now obviously this code needs a trigger as well in the shape of a doorcontact (NC) on digital Pin 4, pulling that down to ground

OpenHab GPIO Example

I know this might not directly be Arduino or ESP8266 related, but I thought I’d publish it anyway as it may come in handy with openhab

First of all, install the GPIO binding in the PaperUI. After one has done that, there isnt really much else to do on that binding. There is no services/gpio.cfg although some of the documentation mentions that file.

Before I go further it is good to note that I used the openhabian distro and I presume a lot of things are already OK in that, but if you did a manual install, it might be good to check if the user ‘openhab’ is member of ‘gpio’ and if not, set that with

sudo adduser openhab gpio

That’s basically all one needs to do.
Let’s setup an ‘input’ which in openhab is a ‘Contact’ and an ‘output’, which in openhab is a Switch. In fact, those are the only states the binding allows.

There is no pressing need to define anything in your sitemap file if e.g. you add the items to a group in your itemsfile, but if you prefer a sitemap then add this to your sitemap file:

Switch item=GPIO_LAMP
Text item=GPIO_BUTTON

Let’s pick some pins to use. It is easiest to use pins on the Raspberry header that are close to a ground pin.

Looking at the pin out of the raspi, we see that for instance GPIO pins 21,13,5, 7, 10, 27,22,24,25,12,16,26, 3, 4, 8 and 17 are next to a ground pin, but some of those have reserved functions so i stayed away from those.
I picked GPIO23 and GPIO24 as that one is also close to a 3V3 pin and when using that as input one can always decide later to make that have a HIGH or a LOW input.The configuration string for the binding is defined as follows:

gpio="pin:PIN_NUMBER [debounce:DEBOUNCE_INTERVAL] [activelow:yes|no] [force:yes|no] [initialValue:high|low]"

for those who consider this as double dutch, it will become clear in the items definition:

Add this to your items file:

Switch GPIO_LAMP "Lamp" (LivingRoom) { gpio="pin:23 force:yes" }
Contact GPIO_BUTTON "Button [%s]" (LivingRoom) { gpio="pin:24 activelow:yes" }

The group ‘LivingRoom’ is not mandatory if you have defined your sitemap and of course you may use another group that suits you more.

The Switch that I set up is quite simple. In fact it already would be enough if it would have been:

Switch GPIO_LAMP "Lamp" {gpio="pin:23"}.

But with ‘force:yes’ we force the use of the pin even if it would have been in use by another process. Obviously that might not be the most elegant way to do it, but as i am pretty sure I am not using it for another process, I just wanted to make sure Openhab would have the pin available.

The current of a GPIO pin is far to small to drive any significant load as it can deliver some 60mA I think.
But the output can drive a small LED to test it… but use a 330 ohm series resistor, no matter what other websites tell you: you dont want to fry your GPIO pin. Eventually it would be more useful to drive a small relay module as long as it has a separate transistor to drive the relay (I know, I am cautious)

The configuration allows for some other parameters.
i already mentioned ‘force:yes|no

activelow yes|no  this means that when the switch is off, there is no voltage on the pin
initialValue:high:low determines the state the pin will take during initialisation.
debounce:interval is used to set the debounce value for the switch in milisecs

If we look at the output, I have not added an activelow because I thought that was the default value (but correct me if I am wrong)
If we look at our input, I set that to activelow, because the way the button works is it applies voltage to trigger the pin and I added it because I was not sure about the default
I did not set a debounce for the contact, but you could state:

Contact GPIO_BUTTON "Button [%s]" (LivingRoom) { gpio="pin:24 debounce:10 activelow:yes" }

 

Arduino Watchdog timer

I will not go too much into the benefits of a Watchdog timer as I presume those are known, but in short a Watchdog timer resets a processor  when it hasn’t seen any activity of that processor for a while. This is  specifically handy for processors  that are in a less accessible space, where otherwise you’d have to go to, get access and  press reset.

Basically there are 2 types of  Watchdog timers: Hardware and Software. About Software WDT’s I can be brief, they work, but  as the very goal of a WDT is to reset in case of software failure, using a software WDT seems a bit counterintuitive. If however you opt for a software timer, you will find a good article here.

Many of the Atmel chips, such as theAtmega328 in the Arduino UNO have an internal hardware Watchdog timer. In reality that watchdog timer is just a simple counter that gives a pulse when it counts up. This pulse can be used either to generate interrupt or simply reset MCU (or both). The watchdog timer can be reset to zero at any time with the WDR command.

The Internal Watchdog timer of the Atmeag328 is set by by programming WDTON fuse when flashing the MCU. You will find more information here.

Other than having to set fuses when flashing your chip, this internal hardware timer has a drawback similar to the software WDT: If you make a mistake, your chip could get into near continuous reset which makes it hard to re-use it again.

From that point of view an External Hardware WDT might be a better option. If you dont need it anymore, or made a mistake, you simply disconnect it.
There are specific  WDT chips available, but the good old NE555 can be used as well.

Watchdog timer
Watchdog timer

The workings are as follows:
In this circuit the 555 is configured as an astable oscillator. When free running it will charge C2 till about 2/3rds of Vcc (so about 3.33V) and then generate a negative pulse on pin 3 that is connected to the RST of the Arduino (or other chip). Diode D1 is not essential, but it does protect the Arduino.
The frequency, and thus period of the oscillator is determined by R2, R3 and C2. In its current config the time period til reset is about 69 seconds. At the end of that period the  circuit will generate a LOW pulse that  will reset the attached microprocessor. As the dutycycle of the circuit is far from symmetrical (fortunately) the LOW pulse will take some 15 mSec which is a useful time. Not too short, not too long.

However, we do not want the processor to restart every 69 seconds, we just want it ro  do its thing unless it hangs and therefore we let the processor restart the  555 oscillator continuously to signal that it is still alive and kicking. We do that by discharging C2 via R1 and we do that simply by using an output pin that we make LOW. So if during each loop of the firmware in the processor we pull that Output pin LOW, the Watchdog knows the processor is still doing its work and only when the program hangs, the watchdog will do a reset after 69 seconds.

It is not sufficient to just make the Output pin LOW to restart the timer, but afterwards it needs to be set HIGH again. However, even if an output pin is HIGH, there still is an internal impedance that could drain C2. Therefore after setting it as OUTPUT and LOW, we set it back to INPUT mode again, because as input the I/O pins of the Arduino have a fairly high impedance.

R1 limits the current that can flow through the sinking pin to abt 9mA (in reality some 6mA). Ofcourse R1 adds to the  time needed to discharge C2, but that does not pose a problem here. (If not given enough time though, the Watchdog will need less than 69 seconds before it is able to send a reset signal again.)

int HeartbeatPin = 8;

void heartbeat() {
// Sink current to drain C2
pinMode(HeartbeatPin, OUTPUT);
digitalWrite(HeartbeatPin, LOW);
delay(300);
// Set pin back to High Impedance
pinMode(HeartbeatPin, INPUT);
}

void setup() {
// Send an initial heartbeat.
heartbeat();
}

void loop() {
//your code
heartbeat();
}

The above is just a very simple example. Ofcourse it is also possible to check specific sections of the program such as say succeeding an internetconnection or  attaching to a serial port. That way you could avoid  that only part of the program (e.g. connecting to internet)  failed, while the processor is still happily kicking the dog on every loop
The end result however will always be a total reset

Reverse polarity protection with a P-MOSFET

Reverse voltage protection with P-FET
Reverse voltage protection with P-FET

Reverse polarity protection can easily be done with a single diode. The problem with that however is the voltage drop (usually 600mV) that with  large currents can mean a substantial powerdissipation. Using a Skottky diode for its smaller voltage drop is risky as the reverse current through a Skottky is not negligable.
A better way is to use a P-MOSFET as shown in the circuit above. This method is explained very well  in an Afroman youtube video.
Though the direction of the P-Fet may seem a bit counterintuitive, but realise that when the Vgs is in conductive range, the FET will  basically operate as a Switch or in fact a  resistor with very low resistive value, allowing current in both directions.
However you need to pick a FET with low VGS(threshold) so that the device is still offering a tiny volt drop at low battery voltages AND you’ll need a FET with low RDS(on) so that at  your peak current  the FET is dropping less than say 100mV. Other than the FQP4706 the DMG3415U may be a good pick. With 1.8V gate drive it has 0.071Ω resistance.

Switching a humidity sensor off and on

Gardeners who use a resistive moisture sensor to measure the moisture of the soil, sooner or later get faced with corrosion of their metal sensor. Though practically all metals corrode in the soil, in this case it is sped up by the fact that a current flows through the sensor, causing electrolysis.

Several solutions are suggested to solve that like: ‘you have to feed them with AC’ but those solutions are not always practical.

There is however a fairly simple method to minimize the electrolysis and that is to switch off current through the sensor when you do not need it.
Moist soils do not need constant measuring, maybe 3-4 times a day is enough, so if you switch off the sensor except for the say 3x a milisecond or so that makes a big difference.

Easiest might be to use an I/O pin to feed the sensor, but not everybody feels comfortable doing that and not all microcontrollers are suitable for that (although the arduino probably is with 40mA per pin).

But it is in fact quite easy to use a transistor -controlled by an I/O pin- to switch the sensor on and off. Figure 1 gives an example of how to do that with either low side switching or high side switching.

Figure 1a is a very basic transistor switch: Pull the base of the transistor HIGH and current will flow through the sensor, pull it LOW and the current will be switched off. Remember there will be a slight voltage drop (say 250-450 mVolt) over the transistor.

Fig 1b is also a rather basic transistor configuration, it is the emitter-follower or common collector that is often used in voltage buffering. Here also a LOW on the base will switch the transistor OFF and a HIGH will switch it on. Also here one has to take a voltage drop into account (basically the voltage on the I/O pin minus 0.6 Volt). Mind you that Fig 1&2 just give a basic circuit. You’ll need to add a base resistor and possibly a pull-down or pull-up  resistor, depending on  what state you want the circuit to start in.

Figure 2 shows two other circuits you may be considering to try, but that’s not necessarily a good idea.
Fig 2A is basically identical to Fig 1A, but with the sensor and series resistor switched position in the voltage divider.
That does not work because even when the transistor is switched off, there still could be a current flowing through the sensor via the internal impedance of the microcontroller input.
Figure 2B shows a PNP transistor used. That should work right? After all it is basically figure 1A but then reversed in polarity. Well, it is not that simple. The I/O pin is tied to a microcontroller running at say 5V, so its voltage level will vary between 0 and a bit less than 5 Volt. At best it might be 4.2 Volt.*

To keep the PNP transistor switched off, we need to keep the base voltage at nearly the same level as that “+” line and that is hard  in this configuration. Therefore opening the transistor is not a problem, but closing it might, although it will ‘probably’ work

Mind you that Fig 1 and 2 just give a basic circuit. You’ll need to add a base resistor and possibly a pull-down or pull-up  resistor, depending on  what state you want the circuit to start in.

*A pFET on the other hand is quite suitable for high side switching

 

 

The LM335 and it’s serieresistor

alibrated mode
Fig 1: LM335 in Calibrated mode

The LM335 belongs to a range of a fairly accurate and simple to use temperature sensors.

The LM335 operates from -40°C to 100°C. It’s output is lineair with 10mV/°Kelvin (¶ 6.6 in the datasheet). It can be used in two ways: Calibrated and uncalibrated mode. The difference is a  10kΩ potentiometer that is used to regulate the output voltage to 2.98 Volt at 25°C (25°Celsius is 298.15°Kelvin) . With more and more devices running off of 3.3 Volt, the question is sometimes raised: will the LM335 run off of 3.3 Volt as well. Going through the datasheet you will not see a min or max value stated for the voltage to be used as all depends on the current through the LM335 and that current is determined by a series resistor (R1 in Fig. 1)

If we want to use the calibrated configuration of the LM335 then this is how to calculate the series resistor for the LM335:
According to the datasheet, the potentiometer needs to set the voltage over the LM335 at 2.98 Volts at  25°C (¶ 6.4 in the datasheet). The LM335 needs between 400uA and 5 mA (¶ 6.2 in the datasheet). Suppose that the max temperature we want to measure is 30 C (=303.15 °K). The output then will be 3.0315 Volt (let’s say 3.03 V).
At 3.03V the current through the 10k potentiometer will be 3.03/10k=303uA. The minimum current that needs to flow through the LM335 is 400uA, so R1 needs to supply 703uA over a voltage of Vcc-3.03V.

So at 5 Volt that would be 1.97/0.703 = 2.8kΩ
At 3.3 Volt it would be 0.27/0.703 = 0.384kOhm =384Ω
(Note 703uA= 0.703mA = 0.000703A. I chose to divide by mA so I get outcome in kΩ).
Now we still have to check if the current will not be too high at the lowest temperature we want to measure:
Let’s say our minimum temperature is 0°C that is 273.15. At 10mV/°K that means an output voltage of 2.73 Volt.
At 5 Volt the resistor current will be (5-2.73)/2.8kΩ = 810uA. We still need to subtract 2.73V/10kOhm=273uA which flows through the potentiometer so we have 810-273=537uA, which is well below the 5mA max.

If we calculate that for 3.3Volt, we find the following:
The resistor current will be (3.3-2.73)/384=1.5mA. Subtract 273uA that flows through the potentiometer and that gives roughly 1.23mA, well within limits.

If you want to measure temperatures higher than 30°C then R1 has to be smaller. When using 3.3 Volt the  theoretical maximum temperature measurable is 330°K which is about 57°C. In practice however that does not work as you need a drop voltage over R1 so the value of R1 cannot be zero, or, in other words: you cannot connect the LM335 directly to Vcc and still expect to get an output lower than Vcc.

Bet let’s see how that would be at 50°C (323.15°K) and 3.3V Vcc :
Well again you need 703uA but this time over 0.07V.
That gives a value  for R1 of 100Ω.
But then at 0 degrees the current is 5.7mA. Still need to subtract 2.73uA which gives 5.4mA which is out of spec.

A program would be as follows:

//A0 is used to read the LM335
void setup()
{
Serial.begin(9600);
}
void loop()
{
int rawvoltage= analogRead(A0);
float millivolts= (rawvoltage/1024.0) * 5000;
float kelvin= (millivolts/10);
Serial.print(kelvin);
Serial.println(" degrees Kelvin");

float celsius= kelvin - 273.15;
Serial.print(celsius);
Serial.println(" degrees Celsius");

float fahrenheit= ((celsius * 9)/5 +32);
Serial.print(fahrenheit);
Serial.println(" degrees Fahrenheit");

delay(2000);
}

Beware though that the ESP8266, in spite of it being a 3V3 device only wants a maximum of 1 Volt on its ADC

Calculating Sunrise and Sunset on Arduino (or other microcontroller)

Knowing the hours of sunset and sunrise may be handy in a variety of situations, an automated chicken coop door might be only one example.

There are several ways to get the proper times: a lookup table in EEPROM, the Timelord library or one of its successors, the Dusk2Dawn library, a rather complicated calculation including the Julian calender, or a fairly simple approximation that I will discuss here.

This method uses the average of the earliest and latest sunset and then for any given day  adds or subtracts a certain amount of time with a maximum of half  of the difference between the earliest and latest sunrise.

Rob Tillaert discusses the method here. It presumes that the sunrise time follows a (co)sinoidal function. I will try to visualize it with a simple example:

Say that on June 23 the earliest sunrise of the year occurs at 4am, and that the latest sunrise of the year occurs at 23 December at 6 am.

Then you know that on any other day the sunrise is between 4 and 6 am. If you take the average that is 5 am, then you know that every other sunrise that year is either 0-1 hr later than 5 am or 0-1 hr earlier than 5 am.

It is the latter that is captured in the formula:

t=avg+0.5Δ*cos((doy+8)/58.09)

  • avg is average sunrise time, in minutes since midnight
  • Δ the difference between the earliest and latest sunrise time
  • doy is the day of the year
  • the 8 is there because we start on the wintersolstice: 23 December is 8 days before jan 1
  • 58.09 is 365/2π. That is necessary because the cosinusfunction has max 2π as input.

If you live in a DST zone, the earliest sunrise wil be under DST, however you need the non-DST corrected time: the sun knows no DST. Calculate firstm then add DST later

For my location the earliest and latest sunrise are:

earliest sunrise is at 4.19 am
latest sunrise is at 8.51
in order to use them in our equation, we have to calculate them in minutes past midnight:
4.19= 4×60+19=259
8.51= 8*60+51=531
The average is (259+531)/2=395
the difference or delta is 531-259=272. We need half of that which is 137.
The equation then becomes:

395+137*cos((doy+8)/58.09)

To check the accuracy of the approximation, I plotted the actual sunrise times (blue curve) against the calculated sunrise time (red curve).

As it shows, the first half of the year is a perfect fit, the second half of the year seems to follow a more linear curve with the max deviation being 20 minutes, that may or may not be accurate enough for your project. With the aid of this curve though I could opt for a linear approximation for the 2nd half of the year.

For sunset we can practically use the same formula, be it that we now have to subtract the variable part rather than add it.
For my location the sunset is as follows:
latest: 22:07 =1327
Earliest: 16:27= 987
avg=2287/2=1144
delta=1327-987=340 ->170
sunset=1144-170*cos((doy+8)/58.09)

That gives the following graph:

This time I didnt bother to enter all the  real sunset times, but it is clearly visible that there is a reasonable fit that could maybe be enhanced a bit by shifting it slightly more to left or decreasing the delta a bit. Again Red graph is the calculated sunset, the blue is the actual sunset. None of the graphs has been corrected for DST.

A procedure for the Arduino would look as follows:
Where DST is a byte indicating whether DST is active (1)  or not active (0).
The day of the year I pull from my RTC library but it can also be calculated as follows:
int(((month-1)*30.5)+dayOfMonth)  (that is an approximation though)

The sunrise and sunset are both given in minutes after midnight. The hour and minutes of the sunrise (and sunset) can be calculated by:
hour=sunrise/60
minute=sunrise%60

MQTT with the W5100 Ethernetshield

New code added

In a previous post I discussed the use of an old ENC28J60 ethernetshield for an MQTT node. The biggest setback of the ENC28J60 shield is that it uses a lot of memory. When using the newer W5100 Ethernetshield, there is a bit more room to play with. So here is a simple framework  for an MQTT node that doesnt’t only publish, but also reacts to simple commands that come in with a subscription.

MQTT-Spy output
MQTT-Spy output
/*
          Arduino UNO with W5100 Ethernetshield or  W5100 Ethernet module, used as MQTT client
          It will connect over Wifi to the MQTT broker and controls a digital output (LED, relay)
          and gives the Temperature and Humidity, as well as the state of some switches
          The topics have the format "home/br/sb" for southbound messages and  "home/nb" for northbound messages
          Southbound are messages going to the client, northbound are messages coming from the client
          As the available memory of a UNO  with Ethernetcard is limited, I have kept the topics short
          Also, the payloads  are kept short
          The Northbound topics are
          home/br/nb/temp  for temperature
          home/br/nb/humid  for humidity
          home/br/nb/deur  for a door switch
          home/br/nb/l for  the lightintensity
          home/br/nb/pr  for the status of a PIR sensor
          home/br/nb/ip showing the IP number of the client
          home/br/nb/relay showing the relaystate

          There is only one southbound topic:
          home/br/sb
          The payload here determines the action:
          0 -Switch the relay off
          1-Switch the  relay on
          2-Publish the IP number of the client
          3 Ask for the relaystate REMOVED

          On Startup, the Client publishes the IP number

*/
#include "Ethernet.h"
#include "PubSubClient.h"
#include "DHT.h"
//kennelijk  geeft update van DHT sensor library boven 1.2.1 een fout
#define CLIENT_ID       "Hal"
//#define TOPIC           "temp"
#define PUBLISH_DELAY   3000
#define PUB_TOPIC "my_username/f/temperature" //Adafruit dashboard
#define PUB_TOPIC_h "my_username/f/humidity" //Adafruit dashboard
#define DHTPIN          3
#define DHTTYPE         DHT11
#define ledPin 13
#define relayPin 8
String ip = "";
bool statusKD = HIGH;
bool statusBD = HIGH;
bool statusGD = HIGH;
bool relaystate = LOW;
bool pir = LOW;
bool startsend = HIGH;
int lichtstatus;
uint8_t mac[6] = {0x00, 0x01, 0x02, 0x03, 0x04, 0x06};

EthernetClient ethClient;
PubSubClient mqttClient;
DHT dht(DHTPIN, DHTTYPE);

long previousMillis;

void setup() {
  pinMode(4, INPUT_PULLUP);
  pinMode(5, INPUT_PULLUP);
  pinMode(6, INPUT_PULLUP);
  pinMode(7, INPUT);
  pinMode(LED_BUILTIN, OUTPUT);
  pinMode(relayPin, OUTPUT);

  // setup serial communication

  Serial.begin(9600);
  while (!Serial) {};
  Serial.println(F("MQTT Arduino Demo"));
  Serial.println();

  // setup ethernet communication using DHCP
  if (Ethernet.begin(mac) == 0) {
    //Serial.println(F("Unable to configure Ethernet using DHCP"));
    for (;;);
  }

  Serial.println(F("Ethernet configured via DHCP"));
  Serial.print("IP address: ");
  Serial.println(Ethernet.localIP());
  Serial.println();
  /*
    Serial.println(Ethernet.localIP()[0]);
    Serial.println(Ethernet.localIP()[1]);
    Serial.println(Ethernet.localIP()[2]);
    Serial.println(Ethernet.localIP()[3]);
  */
  ip = String (Ethernet.localIP()[0]);
  ip = ip + ".";
  ip = ip + String (Ethernet.localIP()[1]);
  ip = ip + ".";
  ip = ip + String (Ethernet.localIP()[2]);
  ip = ip + ".";
  ip = ip + String (Ethernet.localIP()[3]);
  //Serial.println(ip);

  // setup mqtt client
  mqttClient.setClient(ethClient);
  //mqttClient.setServer(mqttServer, 1883);
  // mqttClient.setServer("test.mosquitto.org", 1883);
  //  mqttClient.setServer( "raspberrypi.local",1883);
  //mqttClient.setServer("io.adafruit.com",1883);
  mqttClient.setServer( "192.168.1.102", 1883);
  //Serial.println(F("MQTT client configured"));
  mqttClient.setCallback(callback);
  // setup DHT sensor
  dht.begin();
  Serial.println(F("DHT sensor initialized"));

  Serial.println();
  Serial.println(F("Ready to send data"));
  previousMillis = millis();
  mqttClient.publish("home/br/nb/ip", ip.c_str());
}

void loop() {

  statusBD = digitalRead(4);// FrontdoorSwitch
  statusGD = digitalRead(5);// Garagedoor Switch
  statusKD = (digitalRead(6));//LivingRoom Switch

  lichtstatus = analogRead(A0);//Reads an LDR
  pir = digitalRead(7);//Reads a PIR sensor
  relaystate = digitalRead(relayPin);// Reads the state of a relay

  // it's time to send new data?
  if (millis() - previousMillis > PUBLISH_DELAY) {
    sendData();
    previousMillis = millis();

  }

  mqttClient.loop();
}

void sendData() {

  char msgBuffer[20];
  float h = dht.readHumidity();
  float t = dht.readTemperature();
  Serial.print("Temperature: ");
  Serial.print(t);
  Serial.println("oC");
  Serial.print("Humidity: ");
  Serial.print(h);
  Serial.println("%");
  Serial.print("Relay is: ");
  Serial.println((relaystate == LOW) ? "OPEN" : "CLOSED");
  if (mqttClient.connect(CLIENT_ID)) {
    mqttClient.publish("home/br/nb/temp", dtostrf(t, 6, 2, msgBuffer));
    mqttClient.publish("home/br/nb/humid", dtostrf(h, 6, 2, msgBuffer));
    mqttClient.publish("home/br/nb/deur", (statusBD == HIGH) ? "OPEN" : "CLOSED");
    mqttClient.publish("home/br/nb/garage", (statusGD == HIGH) ? "OPEN" : "DICHT");
    mqttClient.publish("home/br/nb/bel", (statusKD == HIGH) ? "OPEN" : "CLOSED");
    mqttClient.publish("home/br/nb/l", dtostrf(lichtstatus, 4, 0, msgBuffer));
    mqttClient.publish("home/br/nb/p", (pir == HIGH) ? "OPEN" : "CLOSED");
    mqttClient.publish("home/br/nb/relay", (relaystate == LOW) ? "OPEN" : "CLOSED");
    mqttClient.subscribe("home/br/sb");
    if (startsend) {
     // mqttClient.publish("home/br/nb/relay", (relaystate == LOW) ? "OPEN" : "CLOSED");
      mqttClient.publish("home/br/nb/ip", ip.c_str());
      startsend = LOW;
    }
  }
}

void callback(char* topic, byte* payload, unsigned int length) {
  char msgBuffer[20];
  // I am only using one ascii character as command, so do not need to take an entire word as payload
  // However, if you want to send full word commands, uncomment the next line and use for string comparison
   //payload[length] = '\0';            // terminate string with '0'
  //String strPayload = String((char*)payload);  // convert to string
  // Serial.println(strPayload); //can use this if using longer southbound topics
  Serial.print("Message arrived [");
  Serial.print(topic);
  Serial.print("] ");//MQTT_BROKER
  for (int i = 0; i < length; i++) {
    Serial.print((char)payload[i]);
  }
  Serial.println();
  Serial.println(payload[0]);

  // Examine only the first character of the message
  if (payload[0] == 49)             // Message "1" in ASCII (turn output ON)
  {
    digitalWrite(LED_BUILTIN, HIGH);    //
    digitalWrite(relayPin, HIGH);
  } else if (payload[0] == 48)      // Message "0" in ASCII (turn output OFF)
  {
    digitalWrite(relayPin, LOW);     //
    digitalWrite(LED_BUILTIN, LOW);
  } else if (payload[0] == 50)
  {
    mqttClient.publish("home/br/nb/ip", ip.c_str());// publish IP nr
  } else {
    Serial.println("Unknown value");
    mqttClient.publish("home/br/nb", "Syntax Error");
  }

}

You will find the full code for download here.

Edit 
I altered the above code a bit so it takes ON/OFF/READ commands rather than just numbers.

It now looks like this:

/*
Arduino UNO with W5100 Ethernetshield or W5100 Ethernet module, used as MQTT client
It will connect over Wifi to the MQTT broker and controls a digital output (LED, relay)
and gives the Temperature and Humidity, as well as the state of some switches
The topics have the format "home/br/sb" for southbound messages and "home/nb" for northbound messages
Southbound are messages going tomq the client, northbound are messages coming from the client
As the available memory of a UNO with Ethernetcard is limited, I have kept the topics short
Also, the payloads are kept short
The Northbound topics are
home/br/nb/temp for temperature
home/br/nb/humid for humidity
home/br/nb/deur for a door switch
home/br/nb/l for the lightintensity
home/br/nb/pr for the status of a PIR sensor
home/br/nb/ip showing the IP number of the client
home/br/nb/relay showing the relaystate

There is only one southbound topic:
home/br/sb
The payload here determines the action:
OFF -Switch the relay off
ON-Switch the relay on
READ Ask for the relaystate

*/
#include "Ethernet.h"
#include "PubSubClient.h"
#include "DHT.h"
#define CLIENT_ID "Hal"
#define PUBLISH_DELAY 30000
#define DHTPIN 3
#define DHTTYPE DHT11
#define ledPin 13
#define relayPin 8
String ip = "";
bool statusKD = HIGH;
bool statusBD = HIGH;
bool statusGD = HIGH;
bool relaystate = LOW;
bool pir = LOW;
bool startsend = HIGH;
int lichtstatus;
uint8_t mac[6] = {0x00, 0x01, 0x02, 0x03, 0x04, 0x06};

EthernetClient ethClient;
PubSubClient mqttClient;
DHT dht(DHTPIN, DHTTYPE);

long previousMillis;

void setup() {
pinMode(4, INPUT_PULLUP);
pinMode(5, INPUT_PULLUP);
pinMode(6, INPUT_PULLUP);
pinMode(7, INPUT);
pinMode(LED_BUILTIN, OUTPUT);
pinMode(relayPin, OUTPUT);
digitalWrite(relayPin, LOW);
digitalWrite(LED_BUILTIN,HIGH);

// setup serial communication

Serial.begin(9600);
while (!Serial) {};
Serial.println(F("MQTT Arduino Demo"));
Serial.println();

// setup ethernet communication using DHCP
if (Ethernet.begin(mac) == 0) {
//Serial.println(F("Unable to configure Ethernet using DHCP"));
for (;;);
}

Serial.println(F("Ethernet configured via DHCP"));
Serial.print("IP address: ");
Serial.println(Ethernet.localIP());
Serial.println();

ip = String (Ethernet.localIP()[0]);
ip = ip + ".";
ip = ip + String (Ethernet.localIP()[1]);
ip = ip + ".";
ip = ip + String (Ethernet.localIP()[2]);
ip = ip + ".";
ip = ip + String (Ethernet.localIP()[3]);
//Serial.println(ip);

// setup mqtt client
mqttClient.setClient(ethClient);
mqttClient.setServer( "192.168.1.103", 1883);
//Serial.println(F("MQTT client configured"));
mqttClient.setCallback(callback);
// setup DHT sensor
dht.begin();
Serial.println(F("DHT sensor initialized"));

Serial.println();
Serial.println(F("Ready to send data"));
previousMillis = millis();
mqttClient.publish("home/br/nb/ip", ip.c_str());
}

void loop() {

statusBD = digitalRead(4);// FrontdoorSwitch
statusGD = digitalRead(5);// Garagedoor Switch
statusKD = (digitalRead(6));//LivingRoom Switch

lichtstatus = analogRead(A0);//Reads an LDR
pir = digitalRead(7);//Reads a PIR sensor
relaystate = digitalRead(relayPin);// Reads the state of a relay

// it's time to send new data?
if (millis() - previousMillis > PUBLISH_DELAY) {
sendData();
previousMillis = millis();
}

mqttClient.loop();
}

void sendData() {
char msgBuffer[20];
float h = dht.readHumidity();
float t = dht.readTemperature();
Serial.print("Temperature: ");
Serial.print(t);
Serial.println("oC");
Serial.print("Humidity: ");
Serial.print(h);
Serial.println("%");
Serial.print("Relay is: ");
Serial.println((relaystate == LOW) ? "OPEN" : "CLOSED");
if (mqttClient.connect(CLIENT_ID)) {
mqttClient.publish("home/br/nb/temp", dtostrf(t, 6, 2, msgBuffer));
mqttClient.publish("home/br/nb/humid", dtostrf(h, 6, 2, msgBuffer));
mqttClient.publish("home/br/nb/deur", (statusBD == HIGH) ? "OPEN" : "CLOSED");
mqttClient.publish("home/br/nb/garage", (statusGD == HIGH) ? "OPEN" : "DICHT");
mqttClient.publish("home/br/nb/bel", (statusKD == HIGH) ? "OPEN" : "CLOSED");
mqttClient.publish("home/br/nb/l", dtostrf(lichtstatus, 4, 0, msgBuffer));
mqttClient.publish("home/br/nb/p", (pir == HIGH) ? "OPEN" : "CLOSED");
mqttClient.publish("home/br/nb/relay", (relaystate == LOW) ? "OPEN" : "CLOSED");
mqttClient.publish("home/br/nb/ip", ip.c_str());
mqttClient.subscribe("home/br/sb/relay");
if (startsend) {
// mqttClient.publish("home/br/nb/relay", (relaystate == LOW) ? "OPEN" : "CLOSED");
mqttClient.publish("home/br/nb/ip", ip.c_str());
startsend = LOW;
}
}
}

void callback(char* topic, byte* payload, unsigned int length) {
Serial.print("Message arrived [");
Serial.print(topic);
Serial.print("] ");//MQTT_BROKER

for (int i = 0; i < length; i++) {
Serial.print((char)payload[i]);
}
Serial.println();
Serial.println(payload[0]);

if (strncmp((const char*)payload, "ON", 2) == 0) {
digitalWrite(ledPin, HIGH); //
digitalWrite(relayPin, HIGH);
}
if (strncmp((const char*)payload, "OFF", 3) == 0) {
digitalWrite(relayPin, LOW); //
digitalWrite(ledPin, LOW);
}
if (strncmp((const char*)payload, "READ", 4) == 0) {
mqttClient.publish("home/br/nb/relay", (relaystate == LOW) ? "OPEN" : "CLOSED");
}
}

It can be found here.
Apparently the 1st code simple I had published sometimes didn’t keep the relay state. New code does not have that problem.
The code has a number of print statements. Once it is hooked up to the network it might not be very useful to have those anymore. Commenting them out will make more space available

Setting up a local MQTT broker on the Raspberry Pi

On the web there are a ton of instructions and how to’s on how to do this, but I found  it easiest to just  issue two commands on Raspian Jessie:
sudo apt-get update
sudo apt-get install mosquitto mosquitto-clients python-mosquitto

then to test if it was all working, I opened 2 ssh terminals to my raspberry with:
ssh pi@192.168.1.102  (the latter being the ip nr from my raspberry)
and then in one of those terminals I entered:
mosquitto_sub -d -t hello/world
this made the terminal I will call Terminal 1 subscribe to the topic “hello/world”
in the other ssh terminal window (Terminal 2) I entered
mosquitto_pub -d -t hello/world -m "Greetings from Terminal2"
and immediately I got that message published in Terminal 1
(actually you don’t need the ‘-d’ option. That is just for diagnostics)
So I knew my local Mosquitto broker was working, at least within one machine, the raspberry on which it was installed. Time to check if it also worked on other machines. To do that I opened a third terminal on my desktop that didnt ssh to the raspberry but just gave a command line to the machine I was working on (a regular Linux desktop).

Before I could have that machine subscribe to an MQTT topic, I first had to install the MQTT client with:
sudo apt-get install mosquitto-clients

when that was done I entered:
mosquitto_sub -h 192.168.1.102 -t hello/world

mosquitto-terminalNow when I published a message on terminal2 (thats the raspberry) it got published on terminal1 (that same raspberry) and on terminal3 (a regular desktop), so now I knew everything was hunky dory.
However, having to use terminal command lines to check your MQTT broker is not the most convenient. I can advise everybody who works with MQTT to install MQTT-spy. That is a jar file that basically is a publisher and a client that keeps track of MQTT messages on your network.

It comes pre-configured with 3 public brokers but adding your local broker is very simple:
mqtt-localAfter starting MQTT-spy go to  ‘Connections-New Connection” Fill out a name for your connection and under the ‘connectivity tab’ add (in my case): 192.168.1.102:1883  with 1883 being the  standard MQTT port.
Then go to the ‘other’tab and check ‘auto subscribe when opened’.
Now in the main window add a subscription tab called ‘hello/world’ and republish  the messagemqttspy-3 from Terminal2: it should appear in the receive window of mqtt-spy. Vice versa, if  you publish a message from within mqtt spy, it will appear on terminal 1 also