Cold frame monitoring with an ESP8266 WebServer

In this post I will describe a monitor for a Coldframe (that is sort of a small unheated greenhouse).

It has the following functions/uses the following sensors:

  • A HTU21D to measure humidity and temperature inside the coldframe. The HTU21D is a very stable easy to use sensor….but after it has been exposed to 100% humidity for a while, it starts to behave funny, it may give you readings of over 100% or even negative readings. Not sure if this will leave longterm damage but I understand it can be ‘cured’ by  putting it in a 120 degree celsius oven for about an hour.
  • Four DS18B20’s to check temperature in the soil, the air outside the coldframe, the electronics as well as a reservoir.
  • A soil moisture sensor, consisting of  2 carbon rods (those are resistant against electrolysis). Though I could have used the A0 pin of the ESP8266 I am using, I found that one to be unreliable, so I am using a PCF8591 ADC for that. The PCF8591 is a reltively slow 8 bit ADC, but it suffices for the goal. If you want a fast one, consider the ADS1015.
  • A BH1750 to measure the incoming light.
  • A leaf wetness sensor (actually it is cheap chinese webshop ‘rainsensor‘). When used upside down it measures condensation, which is a measure for leaf wetness. I use the pcf8591 ADC to read it.
  • A solar panel as a backup power source. I use one channel of the pcf8591 to measure the voltage.
  • A rain meter (outside the cold frame of course) that I read with yet another channel of the pcf8591.
  • Time is set by a time server and after that kept by the ESP8266
  • A Switch, checking if the lid is open.

There are also three outputs:

  • A fan for ventilation. I have the fan blowing out air from the coldframe -in case it got too hot- but I probably will just put it in another position and use it simply for circulation. The fan is driven through a transistor driver.
  • A fan circulating hot air through an underground heat sink. This fan is autonomous, so there is no switch on the webserver-interface. It does two things: if the coldframe is getting too hot, it will blow that air through the heatsink, thus storing it. If the coldframe gets too cold and the  temperature of the heatsink is higher, it will push the warmth of the heatsink into the coldframe. The heatsink is an underground space filled with stones, throughwhich I have led a pipe.
  • A motor to open the lid of the cold frame. The motor is driven through a MOSFET
    Credit where credit is due, the code around the outputs ‘I stole’ from randomnerdtutorials and modified it for my own use

The reporting and control is done as follows:

  • A webserver connected to the home network (ESP8266 in STA mode)
  • A stand-alone webserver (ESP8266 in AP mode)
  • Thingspeak
  • A MySQL/MariaDB database on a separate computer (e.g. a raspberry). This is also  based on some articles by randomerdtutorials.
  • Googlesheet.
  • Telegram messages: I use Telegram to report the min and max temperature every day at 6 a.m.  It also sends me a message once the temperature goes over a set value. In future I may start to use it to control the Outputs described above

The processor used is a Wemos D1 mini, It might be wise though to get a Wemos D1 mini Pro or an ESP8266-07 board as that has an antenna. Could come in handy, depending where in the garden you place the coldframe.
One more remark about the HTU21D, both the sparkfun library and the adafruit library are rather ‘spartan’ in the use of the HTU21D sensor. It has the possibility of heating up the sensor to prevent condensation. I have added some code to switch that on when the sensor is close to 100% humidity, but only during a few hours in the night. When that internal heating is switched on the temperature is shown in red, like shown below

HTU21D Heated
HTU21D Heated

Construction.
it is not my intention to fully describe how to make a coldframe (I may in future on ‘instructables’), as I presume everybody can make a box with a lid from glass. The one I made is not perfect as it only lets in light from above. I built it during the corona lockdown and had to make do with what I had. But i do want to point out a few details: I put all the electronics in a wooden box with plastic lid, that I just hung in the Coldframe. It has the BH1750 (cover temporarily removed for the picture) on top of it

For the back and one side I used polystyrene, covered with duct tape. In my experience that holds up quite long against the elements.
The construction to open the lid took a lot of pondering. Most of the linear actuator solutions required me to attach the push rod  to the lid…which would impede manual opening. In the end I decide for an egg shaped disc that is eccentrically connected to a motor so when it turns it opens and lowers the lid. make sure to apply some grease on the turning part.

For those not familiar with cold frames: The main ‘danger’  of a cold frame is that it gets too hot during the day. Opening the lid is essential then. It is easy to automate that: insert a line that checks the temperature against a set value and start the motor. As I have chosen to let the motor turn one direction that would mean let it turn for a few seconds and then stop it. I could fine-tune that by adding a position switch,  or even use an H-bridge (requiring an extra pin), but as I have not been on holiday much during the lockdown, I have chosen to keep it manual.

To measure the outside temperature I put a DS18B20 in a piece of conduit pipe sprayed black  on the top half. the idea behind that is that the top will heat more than the bottom, causing an updraft, so the DS18B20 that is located in the bottom will always get fresh outside air

The code:
The core of the code is an Asynchronous webserver that presents the sensordata via server events, every 10 seconds. To prevent the ‘on-start’ values being empty I do an initial replace of the placeholders with the freshly read sensordata. and to make sure I also start with a negative number for the timer so the first 10sec update comes almost immediately.

If one wants to fill a webpage with data there are several approaches: 1. build the webpage, then read the sensors and fill them in; 2. Read sensors while building the webpage and fill them in step by step; 3 read all the sensors first.
I have chosen for the latter.

The webserver contains several icons. I have used 3 ways to put those icons there: 1. from the fontawesome library; or of not available there: 2. directly inserted with base64 codes; or 3. from SPIFFS.
I did that mainly to play around with different techniques, but to use SPIFFS just for 1 picture might be a bit silly and you may consider to not do that. (Edit: I did away with the SPIFFS picture, just picked another one from fontawesome)

The code keeps track of max and min temperature during a 6 a.m to 6 a.m. 24 hr timeslot. Should you lose power or update the code in between via OTA, or if for whatever reason the processor resets, the kept min and mac temperatures are lost. It is easy to prevent that by storing them in SPIFFS (but you will rapidly wear  out the EEPROM), or RTC memory (in a 4 byte per value block). The RTC memory survives reset but not a power outage. I just did not find it important enough to bother with and it introduces some other problems. Mind you that rtcData stored in the first 32 blocks will be lost after performing an OTA update, because they are used by the core internals.

Thingspeak
The thingspeak code is very straightforward. You will need to open a Thingspeak account though and note the channel number and write API code to insert in the program.

coldframe-thingspeak

When using Thingspeak, one can install one of the many Thingspeak apps (such as Thingshow, Thingview or Thingviewer)on one’s phone for easy acces to the data.

Thingshow

MySQL
In order to use this you will need to setup MySQL or MariaDB on a server somewhere. A raspberry Pi would be suitable for that. Earlier I referenced to two articles on randomnerd tutorials that describe how to set up a MySQL  database, no need to reinvent the wheel here, although I did adapt the code a bit.
coldframe-mysql

Googlesheet
You wil need to setup Googlesheet for that as I have described here. You will find an example here too.

Telegram
You will need to setup a Telegram account and create a bot. I describe the process here.

InfluxDB
I did not put my data in influxDB, but if you want it is rather easy to do that with a POST statement.
Add code like this:

#include <ESP8266HTTPClient.h>
HTTPClient ifhttp;
const char* influxUrl = "http://localhost:8086/write?db=mydb";
const char* influxUser = "username";
const char* influxPassword = "password";
String influxData = "";

void (influxDB()
{
influxData ="........" ---> add your fields and data here


// Create POST message
ifhttp.begin(influxUrl);
ifhttp.addHeader("Content-Type", "application/x-www-form-urlencoded");
ifhttp.setAuthorization(influxUser, influxPassword);

// The first attempt to send returns -1
int httpCode = -1;
while(httpCode == -1){
httpCode = ifhttp.POST(influxData);
http.writeToStream(&Serial);
}
ifhttp.end();
}

There is also a library for InfluxDB and when opening the InfluxDB server an example for the ArduinoIDE can be found (in the server go to “Data-Arduino”).

Carriots
Carriots, now called Altair Smartworks is an IoT data collection and processing site. It allows you to store your data and have it processed and attach specific actions to it. It might be useful and is fairly easy to add. Check here.

The code can be updated via website OTA. For development serial OTA is the easiest, so if that is what you prefer: I left the original serial OTA statements in so it is easy to convert back.

Download
You can download the code here. A remark though: it is my ‘as is’ working code and while developing it, I may have put in some unnecessary things and comments and also my variables might not have been grouped logically. Normally i clean up code before publishing it….but that means I have to retest it again and i just did not have the time for that right now. I may in future. But as said, it is a working code. You do need to add your own data such as passwords and API’s though. I am in the process of rewriting the code from the ground up,based on experience gained with the using the  ColdFrame for some time

————————————————————

Experience after a few weeks:
In general the system functions well. The cooling/heating system was able to keep the temperature above 0 degrees celsius when the temperature outside dropped to minus 6 at night for a few hours. It had a bit more trouble when it was freezing minus 6 the entire day. Mind you though that a relatively small object is harder to keep warm because of the unfavorable volume vs. surface ratio.

The HTU21D was a bit of a disappointment. According to the datasheet, the humidity operating range is 0-100% and I have no reason to doubt that. However, after continued exposure to 100% RH, condensation within the sensor can play havoc on the results. RH values above 100% or below 0% are possible then. Robert Smith in fact mentions this in his excellent comparison of various humidity sensors. He though sees it as a possibility to “do your own calibration”. I think though that if the HTU21D jumps from say 114% to -3% within 10 secs, there is more at large than the calibration not being fine tuned.

The question though is whether the condensation also influences the temperature readings. I do not think so, but I might add another DS18B20 just to make sure. I doubt if another humidity sensor would do better. Measuring near the upper limit of the range for a prolonged period seems to be problematic for all. The problem is the condensation in the sensor and no doubt other sensors, like the BME280 would suffer the same fate (eventhough the BME280 seems to use different type of sensor (resistive vs capacitive)). Also here, the relatively small size of the box may play a role as a relatively low amount of water will easily raise the potential humidity.
Ventilation can help, but it is a 2 edged sword: ventilation will generally also drop the temperature, which will make the relative humidity go up. Also the 100% humidity would normally occur when the temperature would go down to 6-7 degrees or lower, i.e. usually at night, and that is the moment you do not want the temperature to drop any lower.
Alternatives: BME280, Sensirion SHT4x or SHT71, HDC1080, AHT20 (datasheet mentions in fact a drift if subject to prolonged >80%RH). The Si7021 is practically identical to the HTU21D (in fact you may get the one if you order the other at less than reputable shops). They are only distinguishable by the serialnumber that supposedly is 60 for the HTU21D and 57 for the Si7021.

The BH1750 functioned well, it provided good information on what hours were most sunny on the spot the coldframe was standing.. It had one quirk though, occasionally (though very rarely), it would display the default value of 56412 Lux for a while and then it would be OK again ( did not find any fault in the wiring). A full reset would normally ‘cure’ it as well.

Obviously not a matter of just ‘finetuning’

The reporting through Telegram worked fine. Only 2 little ‘annoyances’ When the temperature ‘hoovers around 0 degrees or 25 degrees celcius (the lower and upper limit warning levels), I could get a couple of warnings in a row: I get a warning when the temparature dropped below zero, then when it would briefly rise above zero again the warning flag would be reset and subsequently if the temperature would drop below half a minute later I would get a warning again. It is basically the system doing what it was told to do. I could ‘solve’ that but not allowing a second warning within say 5 minutes, but I really did not find it important enough.

The leaf wetness sensor functioned ‘moderately’. Somehow the surface was not as appealing to condensation as the glass top (or the HTU21D sensor for that matter). As I still consider it important info, I will try to find another solution for it.

The combination of the webserver info that is basically à la minute (every 10 secs to be precise) and Thingspeak/MySQL/GoogleSheet that gives an historic overview worked very well.

Improvements
Apart from the HTU21D and leaf wetness sensor I have to look at/reconsider, there are a few things I may add: currently experimenting with a TCS34725 sensor to see if I can monitor ‘green development’.
I am considering if I should add a moveable setpoint for the cooling fan to set in, but it may not be important enough.
Currently I have 4 ADC channels through the PCF8591. I use one of those channels for soil moisture. If I want to use the system in a larger coldframe, or a greenhouse, I may need to add some ADC’s.
The current soil sensor I use (carbon rods) works fine, albeit that the thickness and bluntness of the rods can make it hard to stick it in a pot, so I may decide for a slightly leaner and sharper capacitive sensor, or just use metal nails. The latter though would require a transistor to switch off the current when not used (in order to minimize electrolyses), which takes an extra pin of the ESP8266.
I may expand the function of the lid opener to automatically stop when the lid is open or the lid is closed.
Also considering to add Tweets for notification next to Telegram


If you have a Thingspeak account, tweets can be sent either from your ESP via “thingtweet” or directly from Thingspeak.

I did add an emergency heating in the mean time. It is set to kick in if the temperature drops below 0 degrees, in spite of the underground heatsink providing heat. It is basically a relay that can switch on a heater. In my cold frame it is a resistor heater, if used in a greenhouse it could be a dedicated electrical heater. With seemingly only 2 projected frost days remaining till mid May, I will need to wait with proper testing.

Coldframe Aircon

When the Coldframe is getting too hot, the fan will pull hot air through the underground heatsink that has a rather steady temperature 0f 8-12 degrees Centigrade. When it is getting too cold, the same fan will pull cold air through the underground heatsink which then will warm the air. If that is not enough to prevent coldframe temperature to stay above zero, a heater above the fan wil kick in. I made the heater from 24 half-Watt resistors that are pushed to their limit, producing 12 Watt in heat. The main aim is to keep the temperature above freezing, not to heat the coldframe to say 20 degrees. As said, this is still experimental stage.
It happened to me once that the 6 a.m. telegram message was not received. Checking my SQL database and Thingspeak showed the connection was briefly dropped around 6 a.m. As I do not do any error checking on the connection with Telegram (or MySQL for that matter), the program is not aware of the message not being received.
If it is important to you, the result of bot.sendMessage(chat_id, bottext, "HTML"); is “0” if the transmission failed and “1” if it succeeded. You could do a check on that and resend if necessary. I am using an insecure connection with Telegram by specifying “setInsecure” in the setup. That’s easier and I am not sending top secret info. If you want the connection to be secure, you can do that by stating:

X509List cert(TELEGRAM_CERTIFICATE_ROOT);
WiFiClientSecure botclient;
UniversalTelegramBot bot(BOT_TOKEN, botclient);

Experience after a few months
When summer came, I had trouble loading the coldframe’s website on my network. As I still could load the the AP website when I was close to the coldframe, I was suspecting something was blocking the connection to the router. The most likely suspect was a row of potato filled containers between the coldframe and my house. When those grew into firm plants it isn’t unlikely to think these walls of moisture blocked the signal. Yet the connection to Thingspeak as well as my mySQL server seemed not to suffer.

Granted, I do not really use the coldframe in the summer, yet it might be wise to use a Wemos Pro (or ESP8266-7) with external antenna and place a router or WiFi extender in a room directly facing the garden (currently it connects to a router on the opposite side of the house, 1 floor up).

Also, when I will build another coldframe, I will try to find a way to remove the electronics from the box in an easier way, e.g.by making external plug connections with the ventilators, motors and some of the sensors. and I will start streamlining the code a bit more.

10 thoughts on “Cold frame monitoring with an ESP8266 WebServer”

  1. May I congratulate you on an excellent project and write up. I particularly like the application of physics for the outside DS18B20 and the ingenious underground heat store. I have used similar sensors / ADC devices for my greenhouse and propagator units. I gave up on the HT21U as I found them unreliable, I now use the BME280 and find those much better than the HT21U. I plan to try some capacitive soil moisture sensors but I have been unable to find any information on their long term reliability. I plan to try your web server and Telegram techniques as I haven’t done any of that.

    1. Bob, thank you for your kind words.
      My underground heat store is just some pipe that I buried about 30 cm deep. It works quite OK in keeping everything above zero..
      i have hesitated a bit about the humidity/temperature sensor. Initially I considered the DHT12 (i2C) as it has a wider humidity and temperature range than the ubiquitous DHT11 (which is quite crap). but settled for the HTU21 as that is supposed to be able to measure to 100% humidity. Well, it does but it is having problems at that level. I know the BME280 very well, have one in use. Supposedly it is the most accurate of the various sensors but i am afraid it will have the same issues at 100% humidity as the HTU21D has.
      The ‘chinese webstore’ Capacitive humidity sensors are quite OK….but they have a problem. They are coated before being cut from the main board and thus have uncoated sides and eventually the moisture will creep in. But they do well once you coat them entirely again in PlastiDip (but getting a bottle of that might be quite expensive if just for one sensor). Some people advice using nailpolish for it…but that better be very sturdy nailpolish.
      I am using carbon rods, and those are OK, but still want to (re) try a capacitive sensor, once I can get it covered.
      The commercial ones are just not worth it for my growing.
      i am looking into expanding the telegram interface to control some stuff as well. not too hard as I have done that before, just need to get myself to do it….and then test again.
      Anyway, i wish you success building everything and if you have more questions, do not hesitate to ask.
      Cheers

      1. Thanks for the additional feedback. We do not get 100% humidity in the UK, although I guess one could in a moist enclosure, so I have never concerned myself too much with that. I was not aware of the coating method on the CN moisture sensors but it makes sense. I wonder if coating the edges in a 5 minute epoxy might help or possibly the whole sensor. As I have a few sensors PlastiDip may not be too costly either. I have about 20 capacitive moisture sensors, some from China and some from a European supplier, Catnip electronics. I would expect the latter to be of reasonable quality and the Chinese ones to be plagiarised from the Catnip device.
        I have been contemplating how it might be possible to collect from from the ground on a small DIY basis. The ground temperature at approx. 1200 mm deep varies from about 9 – 12 °C. I know that because I have a pit in my garage and I have a DS18B20 embedded in the bottom and, as a heating engineer, I have studied and installed heat pump systems. I think the issue would be getting a heat collector of sufficient capacity, tightly would finned copper coil maybe, to be worthwhile to heat a small enclose for frost protection

  2. I expect that in open air or a greenhouse, 100% humidity would be rare, but in a smallish coldframe it is no exception of the temperature goes below 6-7 celsius. Epoxying of the edges probably will help but it is a bit fussy, if you have a couple of sensors, Plastidip is not that costly per sensor.
    there are some issues with some of the chinese boards if they use an NE555 rather than a TLC555 and also in some boards the 1M ohm output resister has a faul up. I discuss some of those issues here: https://arduinodiy.wordpress.com/2020/08/24/soil-moisture-sensors/
    With regard to draw heating from the soil: on my scale just a 2 meter buried pipe already was enough to sink heat during the day and draw heat at night…sufficient to keep it above frostlevel. If you plan to use it as (extra) heat source you are going to need longer piping. That itself might be functioning sufficiently as a collector already. It might be easier to pump water through it than air. I use a simple fan, I am pretty sure that is not maximum efficient, so for a longer system water might be the way to go.
    there are systems that use earth warmth to heat entire houses, maybe that can give you a few pointers

    1. Aha, I knew I had read a good article on soil moisture sensors, it was yours!, I couldn’t remember where I saw it. I was thinking of using water plus glycol in my DIY heat collector. I have a national qualification in ground source heat pumps so I know a bit about the domestic systems but there are somewhat expensive for a greenhouse! Thanks ago for the pointers to information.

  3. In my small experience with humidity sensors I had a BME280 that reached 100% many times but was able to revovery for about 3 years then it suddenly broke.
    Maybe a teflon cover like the ones used in weather stations can be useful because I believe that water condensing should be a problem.
    Very professional sensors use a metal mirror plate that is cool down by a peltier and a optic control the condensation over the plate. I don’t remember the name of this type of sensor.
    For soil moisture I read some years ago in the vinduino project blog about a sensor made from steel but inserted in a gypsum block.

    1. Tnx for yr feedback. Mine has a teflon cover already. I think the very essence is to let humidity in. Nevertheless i will switch it with my BME280. Sensors sometimes can give up for unclear reasons. Had a BMP180 that worked perfectly and from one moment to another just gave up.
      I do know the moisture sensors that are cast in gypsum. Even still have some. They tend to lag behind as the gypsum needs to be penetrated first. Very hard for automatic watering. All in all i am reasonably happy with the carbon rods in spite of them being a bit more mechanically vulnerable. Maybe i just need to file a sharp point on them.
      Anyway, i will put in the BME290 and report back in a while

      1. Remember that temperature in BME280 is usually overestimated of about 1°C (datasheet speak of pressure & humidity sensor) instead BMP180 does not. Don’t know why.
        Steel rod in gypsum was used by the Vinduino project for automatic watering of a vineyard in California for years but I don’t know if they are still in use today. The guy started as a project for himself but today he sell modules, but I believe they are still open source. But a cold frame is much smaller.
        Anyway humidity sensors usually have few years of life in high humidity & condensing air, I believe.

      2. Thanks. The drift of the BME280 in my experience only happens in continuous mode when the sensor self heats. I always use it in forced mode with sleep in between. You might very well be right about the limited life span of humidity sensors. DHT11 one of the worst in that aspect. They all die sooner in humid climates than in arid climates it seems.
        The gypsum encased sensors i think were a short hype. Could well be that the vinduino project was the source but as said I tried them and was not impressed. Tbh the easiest i still think are two metal rods that just have the current switched off most of the time.
        Anyway, i ordered a few more sensors and will see how they fare.
        I checked vinduino. They now seem to sell ‘total irrigation and fertilization solutions,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: