Sending messages through Signal or Whatsapp

Earlier I described how to send messages from your ESP8266 (or ESP32 for that matter) via the Telegram message app.
Signal is a slightly less known app that is getting more and more popular and it is also possible to send messages from your microcontroller through that app. It is even simpler (in my humble opinion than via Telegram).
Here is what to do:
install the messenger app on your phone and register.
Add the number “+34 603 21 25 97” to your contact list and give it a name (let’s say ‘Signal Bot’)

Yes, the SignaBot has Whatsapp too


Send the following message to that contact:
“I allow callmebot to send me messages”

You should get a reply “New user registered”, followed by an API key

It is now possible to send messages to your phone with https:/api.callmebot.com/signal/send.php?phone=<yourphonenumber>&apikey=xxxxxx&text=This+is+a+test

So suppose your international phonenumber =+31612345678 and your Api=123456, and your message is “Hello World”, then the call will become:

https:/api.callmebot.com/signal/send.php?phone=+31612345678&apikey=123456&text=Hello+World

Sending pictures is possible like this

https://api.callmebot.com/signal/send.php?phone=+31612345678&apikey=123456&image=https://arduinodiy.files.wordpress.com/2021/07/signal.jpg

Currently it is not possible yet to send messages to Signal, but that supposedly is in the making.

Program
A https connection is necessary, which the ESP8266/ ESP32 can do. Unles you plan to send highly classified material, it is easiest to do away with certificates or fingerprint and use ‘client.setInsecure();’
You can download the program here.

Just a few remarks: My first version of the program worked immediately. However, it did send one message, and somehow it would never send another message. Tried several solutions, but none brought it back to life. Then I picked a basic example of Ivan Gorotkov that I adapted to use with setInsecure() rather than with fingerprint.
As Ivan’s core code was virtually akin to my first program, albeit with some checks on the http request….it did not work. Curious about what the request reply was (hoping to find an indication of the problem), I started fiddling a bit with the cut-off character. Normally this was a “\n”, but that is not very helpful if the first character is in fact a return. To my surprise, using another character, suddenly brought the program back to life and it has been working reliable since (albeit that one time a message took a bit long to arrive). Now it should not really matter in what way i read the reply string in order for the program to function properly, but it does, so I suspect there is a bit of a timing issue before the connection is closed. If anybody can give some insight in this i would be glad to hear it.

Whatsapp

The same approach works for Whatsapp as well:

Note that the https request string is slightly different from the one used in Signal. (It is also possible to send messages from your ESP8266 through WhatApp via Twilio and Webhooks)

Telegram & messenger
The ‘callmebot’ supposedly works with telegram and facebook messenger as well.
As far as telegram goes, the approach there is a bit different.
Make sure you have a usename set in Telegram (in ‘settings’. I found this is easiest through a browser)
Authorize callmebot through this link. Or, you can start the bot sending /start to @CallMeBot_txtbot.
The https request to send is:
https://api.callmebot.com/text.php?user=@yourusername&text=This+is+a+test+from+CallMeBot

Then send messages with this call

https://api.callmebot.com/text.php?user=@yourusername&text=This+is+a+test+from+CallMeBot

mind you though that telegram als has its own bot service I refer to that one at the beginning of this article.
As i do not use facebook, I cannot really try that out, but the procedure is described here.

Parametric 3D Box and Lid

Often, when doing a project, it stays i sort of a ‘experimental’ state coz of lack of a proper casing. I designed a universal box and lid in Freecad. It is parameter driven, which means you can just fill in the required measures in a spreadsheet and you are good to go. Mind you that both the lid and the case need to have the parameters set.
You will find a parameter called ‘shim’. That is just to create a tiny bit of space between the lid extrusion and the casing. For a very tight fit set it to ‘0’

The files are available here.

Spreadsheet for the lid
Spreadsheet for the case

Te above will create a box with outer dimensions of 75x55x30 mm. The wall thickness will be 2 mm. two flaps of 5x5mm are added with a screw hole of 3mm (i advise to make the flaps and the hole a bit bigger, say 8xb8 mm and a 4-5mm hole.) It should fit a standard 5×7 cm veroboard.

The casing has 2 holes, one for a 3mm LED and one for cables going in and out. They can b easily removed if not necessary

How to modify the AsyncElegantOTA library…..if you really need to

The AyncElegantOTA library is an easy way to add WebOTA to your projects with well, uh an ‘Elegant’ interface.

A recent discussion an facebook was about ‘how can I modify the AsyncElegantOTA?’

Ofcourse it is not so hard to do that if you know something about JavaScript and/or HTML, but one who dives into the library files will soon see that a large part of it seems to be encrypted, which no doubt is a hurdle for many, but it can be taken.

It might be worthwhile though to ask yourself why you would want to change the AsyncElegantOTA library, why not just use the native ESP8266HTTPUpdateServer library and make any desired modifications in that one (coz let’s admitt it, that one has a boring interface). No need to decode any encrypted files. You will find that library in C:\Users\<name>\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.7.3\libraries\ESP8266HTTPUpdateServer.src. There is a great example of this library being modified to include password protection and to look a bit more jazzy.

But if you really want to tackle the AsyncElegantOTA library here is how to do it:

Go to where you installed the AsyncElegantOTA files and open the elegantWebpage.h file. It looks like this:

The part we are interested in is de PROGMEM content, but we can’t do anything with it as long as it is still encoded. To decode it copy the PROGMEM content in its entity, without the brackets. So in fact just the numbers with the comma’s in between

Then go to the Cyberchef webpage, and paste the PROGMEM content in the input field. We need to do 2 decoding actions on the inputfield.
First: Under operations choose “From Decimal”, double click it so it is moved under “Recipe” and pick ‘Comma’ as the delimiter. (if you follow the link I gave, both operations should already be added)

Before we add the second operation, we will first do a check:

If you check ‘length’ in the ‘OUTPUT heading, you will see that number is equal to the ELEGANT_HTML_SIZE constant that is mentioned in the elegantWebpage.h. if you are using version 2.2.5 of the library, that number most likely is “53715”. Just take a mental note of where that number popped up because we need it later to calculate a new value Output’
The second decoding we need to add is an unzip. So under operations we now choose “Gunzip”, and in the output file an HTML structure should appear.

If you scroll through the HTML, you will see that it has JavaScript embedded in it the HTML code. This now is the code you can modify. What you exactly want to modify is up to you.

Once you have modified the HTML /Javascript file, there are several ways you could embed it in the library again; You could leave it uncrypted and put it back in your elegantWebpage.h file as a ‘Raw literal’, like so:

const char ELEGANT_HTML[] PROGMEM = R"---(
<html>
<head>
...
)---";



but then you would need to make a change in the AsyncElegantOTA.h file as well:

The above line would need to be changed to something like:

AsyncWebServerResponse *response = request->beginResponse_P(200, "text/html", ELEGANT_HTML);

provided that “ELEGANT_HTML” now is the name you gave to the raw literal. This is not the method I followed though, as I found it simpler to just encode the HTML file again. In order to do this, we go back to the Cyberchef website but now paste the HTML file in the input field. Either remove or pause the two previous operations and now add “Gzip” and “To Decimal”, like so:

If you have done right you now should get a new decimal code in your output field. Copy that and paste it over the previous PROGMEM content in your ‘elegantWebpage.h’ file.

You are now almost done. you need to update the size constant in your elegantWebpage.h file.

const uint32_t ELEGANT_HTML_SIZE = 53715;

Well, remember how I asked you to make a mental note earlier?
After you copied your “altered and recrypted to decimal again” file from the output field and pasted it in the elegantWebpage.h file, you now paste it in the input field of the Cyberchef website again. Make sure the only operation you choose is “From Decimal” with ‘Comma’ set as delimiter. Now in the Output field header you will see the new size: use that one to update the ELEGANT_HTML_SIZE constant.

Your modifcations should work now

50 Shades of Blink

Well, not really 50, but I will dive into various ways to make an LED blink, after all, if you can control an LED you can control a relay and if you can control a relay you can control the world. This article is mainly aimed at showing beginners various approaches to a specific functionality, but perhaps there will be some new stuff for the more seasoned user as well, as I will also touch upon assembly language.

Classic Blink

Everybody who is using an Arduino started with the blink sketch. if not loaded by themselves, it was probably already loaded on the arduino they got.
In all it’s simplicity it looks like this:

void setup() {
  // initialize digital pin LED_BUILTIN as an output.
  pinMode(LED_BUILTIN, OUTPUT);
}

void loop() {
  digitalWrite(LED_BUILTIN, HIGH);   // turn the LED on (HIGH is the voltage level)
  delay(500);                       // wait for a second
  digitalWrite(LED_BUILTIN, LOW);    // turn the LED off by making the voltage LOW
  delay(500);                       // wait for a second
}

When compiled it uses 924 bytes or 3% of the available storage space. That is a lot for just a blinking LED. Can we do better?
Yes we can. We can use something that is called ‘port manipulation’.

Blink with Portmanipulation

If we look at the pin-out of the Arduino, we see that digital pin13 is also called ‘PB5’. That means that it is controlled by the 5th bit of a register that is called ‘PORT B’. Instead of using statements like ‘digitalWrite(13, HIGH);’ we can directly manipulate the status of that 5th bit.


There is something similar of the ‘pinMode’ command: Input and output for the arduino is controlled by another register that is called the ‘Data Direction Register’. For PORTB (that is the one we need for pin13/PB5) we use ‘DataDirectionRegisterB’ or DDRB and as we want to set it to output, we need to write a ‘1’ in that bit. We could do that by writing DDRB=”0B11111111″, but then we set D8-D13 to output (PB0-PB5) all to output and we (uselessly) write to the bits that control the bits controlling the 16Mhz crystal (PB6-PB7).
So we have to make sure we only write to PB5. There are several ways to do that, but I have chosen to ‘bitwise OR’ the DDRB register. With a bitwise OR, we leave the original content of the register intact and only set the bit HIGH that we want to be HIGH.

The bitwise OR of two bits is 1 if either or both of the input bits is 1, otherwise it is 0. In other words:

    0 | 0 == 0
    0 | 1 == 1
    1 | 0 == 1
    1 | 1 == 1


So suppose the content of the DDRB= 00000000 and we OR that with 00100000, it becomes 00100000. If originally it was 00000100 and we OR it with 00100000, it becomes: 001000100. The sign for the OR command is ‘|’.

So our pinMode(13,1); statement becomes DDRB=DDRB | 0B00100000;

We do something similar setting bit 5 of PORTB HIGH, so our digitalWrite(13,HIGH); becomes PORTB=PORTB | B00100000;

In order to set set bit5 of PORT B low again (the equivalent of digitalWrite(13,LOW);), we cannot just ‘OR’ the PORTB register with ‘0’ as that would have no effect: it leaves all ‘1”s to ‘1’. We could just write a ‘0’ to the register, that would work, but then we set all bits to ‘0’ and we might not want to do that.
For this we use another bitwise operator, the ‘bitwise AND’ (symbol &). The truth table looks like:

    0 & 0 == 0
    0 & 1 == 0
    1 & 0 == 0
    1 & 1 == 1

so suppose our PORTB register has bit 5 and bit 1 HIGH and we only want bit 5 to be set LOW, we have to ‘AND’ the PORTB register with 0B11011111. Therefore digitalWrite(13,LOW); becomes PORTB=PORTB & 0B11011111;
Now there are some ‘easier’ ways to deal with having to count zero’s and one’s. It is obvious that 0B11011111 is the inverse of 0B00100000 and as such it can be written as ~(0B00100000). it still can be ‘simpler’ or at least with less ‘counting’ we can just state that we want a ‘1’ on the position of PB5. That is written as (1<<PB5). So our statement becomes PORTB= PORTB& ~(1<<PORTB5); and that can be written in good C++ fashion like PORTB &=~(1<<PORTB5);
The program then looks like this:

void setup() {
  // initialize digital pin 13 as an output
DDRB = DDRB | 0B00100000 ; //set bit 5 to output
}

void loop() {
PORTB = PORTB | B00100000;
delay(500);
PORTB &= ~(1<<PORTB5);
delay(500);
}

//644 bytes
//gewone blink: 924 bytes (3%)
//assembler blink 488 bytes (1%)

We see that we now saved some memory as we only need 640 bytes

Can we do better? yes we can: the ‘delay(500)’ statement is one of those high order statements that might be consuming more bytes than necessary. With some caveats, we can use the _delay_ms(); function

//depending on your IDE version and setup you may need to include the following system libraries
//#include <avr/io.h>
//#include <avr/delay.h>
void setup() {
  // initialize digital pin 13 as an output
DDRB = DDRB | 0B00100000 ; //set bit 5 to output
}

void loop() {
PORTB = PORTB | B00100000;
_delay_ms(1000);
PORTB &= ~(1<<PORTB5);
_delay_ms(500);
}

Again some savings, we now only need 486 bytes

That is quite a saving compared to the original 924 bytes. The downside of using registers is that your code becomes less portable. What may work on an Atmega328 will likely not work on an Atmega8 or Atmega32.
Can we do better? Going to be hard, but let’s see if we can. How about if we forego C++ and use assembly language?

Blink in Assembly

It is possible to write assembly in the arduino IDE. You can do it in the main ino file with the inline ‘asm’ command, but I prefer to do it in a separate asm file.
For that, open the IDE and create two tabs. One called “blinkasm.ino” (or any other name you prefer) and a second one called “blink.S” (or any other name as long as the extension is a capital “S”). Like this:

Then in the ino file we only have to give some basic instructions to go to the asm file.
We start with

extern "C" {
  void start();
  void blink();
}

This basically says that we define two external procedures, called ‘start’ and ‘blink’.
Then in the startup() we just call ‘start()’ and in the loop we call ‘blink()’
The entire file then looks like this:

extern "C" {
  void start();
  void blink();
}
void setup() {
  start();
}

void loop() {
  blink();
}

we then go to the ‘blink.S’ tab and past the following code:

#define __SFR_OFFSET 0
#include "avr/io.h"

.global start
.global blink

start:
sbi DDRB,5    ;PB5as output
ret

blink:
ldi r20,250 ;  set delay in msec. Max is 255
call delay_n_ms
sbi PORTB,5
ldi r20,250
call delay_n_ms
cbi PORTB,5
ret

delay_n_ms:
ldi 31, 6400>>8    ;here we divide 6400 ovet 2 registers
ldi 30, 6400&255
delaylp:
sbiw r30,1  ;subtract immediate word -> subtract a <63 value from a 16 bit pointer register
brne delaylp
subi r20,1  ;subtract "1" from r20 and store in r20
brne delay_n_ms
ret

It is not my intention to give an entire course in AVR assembly so i will discuss the ASM program very superficially: We see 3 procedures: ‘start’ that you could compare with the ‘setup’ in arduino programming, there is ‘blink’ that is repeatedly called from the ‘loop’ in the ino tab,and there is ‘delay_n_ms’ (delay number of microsecs).

In ‘start’, the only thing we do is to set bit 5 in the DataDirectionregisterB (DDRB) and by now we know that means ‘set D13 as output’

In the blink procedure we load 250 in register 20. register 20 is a common register in the ATMEGA 328 that is available to the user,however it can only contain one byte, so we cannot put more in it than 255. For now we put in 250 and use that as a base for our 500ms delay. We then set bit 5 of PORTB, switching the LED on PB5 (=D13) on. We then call the delay function (explained later). We then load r20 again with 250 again and call the delay function again.
The delay function does the following: it creates a loop that takes 5 cycles. On a 16MHz Arduino 1ms takes 1600 cycles, so if we go through that loop 16000/5 =3200 times we have delayed 1ms and if we do it 6400 times, we have a delay of 2ms (stating the obvious). If we then repeat that entire proces 250 times (being the value in r20) we come to 500ms.

Surprisingly, the ASM code takes up a bit more space than the code in which we bitwise manipulated the registers

It uses 488 bytes as opposed to 486 bytes in the previous program.
I am fairly sure though that it is not the most optimal way to write Assembler code and we might still be able to shave a few bytes off using a proper editor, but we are probably very close to what is possible.

The downside of using Assembly is that your code becomes less portable. What may work on an Atmega328 will likely not work on an Atmega8 or Atmega32.

Non-Blocking Blink

There is one thing though that all these programs have in common that virtually makes them worthless: They are ‘blocking’ codes. that eans that as long as the LED is blinking, it is hard to have the code do anything else, so we have to come up with a non blocking way to blink the LED’s. I know that is cut and dry to anybody who spent a bit of time with the Arduino, but for completeness sake I will discuss it here. After all, novices have to learn somewhere. The only thing you ahve to do to avoid blocking code is to get rid of the delay statements and make the LED blink in another way.We can do that with timers, for which e.g. the ‘micros’ command is very handy.

#define BlinkPeriod 500  // this is in milliseconds

boolean ledState = LOW;

// here we store the last time the LED blinked
unsigned long lastTime = 0;

void setup() {
  pinMode(LED_BUILTIN, OUTPUT);
}

void loop() {
  unsigned long now = millis();
  if (now - lastTime >= BlinkPeriod) // this will be true every 500 milliseconds
  {
    lastTime = now;
    if (ledState == LOW)
    {
      ledState = HIGH;
    }
    else
    {
      ledState = LOW;
    }
    digitalWrite(LED_BUILTIN, ledState);
  }

  //can do other stuff here
}

What this program factually does is that it continuously compares the time the loop has been running since the LastTime the LED blinked (now-LastTime) against a preset time (‘BlinkPeriod’) and if that period is surpassed, it ‘toggles’ the LED. Toggling meaning that it just checks what th elast state of the LED was (On or OFF) and then it switches it to the other state.
Interestingly this method is more efficient -using 846 bytes- than the classic blinksketch, eventhough it has improved functionality

Ofcourse we can combine this non blocking code with some of the techniques we learned earlier, like using direct portmanipulation.

Such a sketch would look like this:

#define BlinkPeriod 500  // this is in milliseconds

boolean ledState = LOW;

// here we store the last time the LED blinked
unsigned long lastTime = 0;

void setup() {
  DDRB = DDRB | 0B00100000;
}

void loop() {
  unsigned long now = millis();
  if (now - lastTime >= BlinkPeriod) // this will be true every 500 milliseconds
  {
    lastTime = now;
    if (ledState == LOW)
    {
      ledState = HIGH;
      PORTB = PORTB | B00100000;
    }
    else
    {
      ledState = LOW;
      PORTB=PORTB& ~(1<<PORTB5);
    }
    //digitalWrite(LED_BUILTIN, ledState);
    
  }

  //can do other stuff here
}

Then we get a sketch using only 550 bytes

Classes

What else can we do if we want an LED to blink. Suppose we have a code in which we want to blink one or more LED’s from time to time. Do we have to put in that piece of blink code every time we want to blink the LED? Sure we could define a procedure that takes various parameters such as blinkperiod and LED number, that works, but actually this is a good time to use a ‘class’. A class basically is a collection of commands and variables that belong to eachother. Below I present a class called ‘Blinker’. It allows you to use several different LEDs, all with different on and off periods.

class  Blinker
{
    byte ledNr;
    long OnTime;
    long OffTime;
    boolean lampState;
    unsigned long previousMillis;
  public:
    Blinker(char nr, long on, long off)
    {
      ledNr = nr;
      OnTime = on;
      OffTime = off;
      lampState = true;
      previousMillis = 0;

    }
    void Update()
    {
      unsigned long currentMillis = millis();
      if ((lampState == true) && (currentMillis - previousMillis >= OffTime))
      {
        lampState = false;
        previousMillis = currentMillis;
        digitalWrite(ledNr,HIGH);
      }
      else if ((lampState == false) && (currentMillis - previousMillis >= OnTime))
      {
        lampState = true;
        previousMillis = currentMillis;
        digitalWrite(ledNr,LOW);
      }
    }
    void Stop()
    {
      lampState = false;
    }
};
Blinker led1(13, 1000, 400);
Blinker led2(12, 500,300);

void setup() {
  // put your setup code here, to run once:
}

void loop() {
  // put your main code here, to run repeatedly:
  
  led1.Update();
  led2.Update();
 //here your other code
}



This sketch takes 922 bytes.

So, as one can see, there are various ways to skin a cat when it comes to blinking LEDs. Keep in mind tough that if one uses any ‘delays’ in the rest of the code, that can influence the blink.

Cold frame monitoring with an ESP8266 WebServer

In this post I will describe a monitor for a Coldframe (that is sort of a small unheated greenhouse).

It has the following functions/uses the following sensors:

  • A HTU21D to measure humidity and temperature inside the coldframe. The HTU21D is a very stable easy to use sensor….but after it has been exposed to 100% humidity for a while, it starts to behave funny, it may give you readings of over 100% or even negative readings. Not sure if this will leave longterm damage but I understand it can be ‘cured’ by  putting it in a 120 degree celsius oven for about an hour.
  • Four DS18B20’s to check temperature in the soil, the air outside the coldframe, the electronics as well as a reservoir.
  • A soil moisture sensor, consisting of  2 carbon rods (those are resistant against electrolysis). Though I could have used the A0 pin of the ESP8266 I am using, I found that one to be unreliable, so I am using a PCF8591 ADC for that. The PCF8591 is a reltively slow 8 bit ADC, but it suffices for the goal. If you want a fast one, consider the ADS1015.
  • A BH1750 to measure the incoming light.
  • A leaf wetness sensor (actually it is cheap chinese webshop ‘rainsensor‘). When used upside down it measures condensation, which is a measure for leaf wetness. I use the pcf8591 ADC to read it.
  • A solar panel as a backup power source. I use one channel of the pcf8591 to measure the voltage.
  • A rain meter (outside the cold frame of course) that I read with yet another channel of the pcf8591.
  • Time is set by a time server and after that kept by the ESP8266
  • A Switch, checking if the lid is open.

There are also three outputs:

  • A fan for ventilation. I have the fan blowing out air from the coldframe -in case it got too hot- but I probably will just put it in another position and use it simply for circulation. The fan is driven through a transistor driver.
  • A fan circulating hot air through an underground heat sink. This fan is autonomous, so there is no switch on the webserver-interface. It does two things: if the coldframe is getting too hot, it will blow that air through the heatsink, thus storing it. If the coldframe gets too cold and the  temperature of the heatsink is higher, it will push the warmth of the heatsink into the coldframe. The heatsink is an underground space filled with stones, throughwhich I have led a pipe.
  • A motor to open the lid of the cold frame. The motor is driven through a MOSFET
    Credit where credit is due, the code around the outputs ‘I stole’ from randomnerdtutorials and modified it for my own use

The reporting and control is done as follows:

  • A webserver connected to the home network (ESP8266 in STA mode)
  • A stand-alone webserver (ESP8266 in AP mode)
  • Thingspeak
  • A MySQL/MariaDB database on a separate computer (e.g. a raspberry). This is also  based on some articles by randomerdtutorials.
  • Googlesheet.
  • Telegram messages: I use Telegram to report the min and max temperature every day at 6 a.m.  It also sends me a message once the temperature goes over a set value. In future I may start to use it to control the Outputs described above

The processor used is a Wemos D1 mini, It might be wise though to get a Wemos D1 mini Pro or an ESP8266-07 board as that has an antenna. Could come in handy, depending where in the garden you place the coldframe.
One more remark about the HTU21D, both the sparkfun library and the adafruit library are rather ‘spartan’ in the use of the HTU21D sensor. It has the possibility of heating up the sensor to prevent condensation. I have added some code to switch that on when the sensor is close to 100% humidity, but only during a few hours in the night. When that internal heating is switched on the temperature is shown in red, like shown below

HTU21D Heated
HTU21D Heated

Construction.
it is not my intention to fully describe how to make a coldframe (I may in future on ‘instructables’), as I presume everybody can make a box with a lid from glass. The one I made is not perfect as it only lets in light from above. I built it during the corona lockdown and had to make do with what I had. But i do want to point out a few details: I put all the electronics in a wooden box with plastic lid, that I just hung in the Coldframe. It has the BH1750 (cover temporarily removed for the picture) on top of it

For the back and one side I used polystyrene, covered with duct tape. In my experience that holds up quite long against the elements.
The construction to open the lid took a lot of pondering. Most of the linear actuator solutions required me to attach the push rod  to the lid…which would impede manual opening. In the end I decide for an egg shaped disc that is eccentrically connected to a motor so when it turns it opens and lowers the lid. make sure to apply some grease on the turning part.

For those not familiar with cold frames: The main ‘danger’  of a cold frame is that it gets too hot during the day. Opening the lid is essential then. It is easy to automate that: insert a line that checks the temperature against a set value and start the motor. As I have chosen to let the motor turn one direction that would mean let it turn for a few seconds and then stop it. I could fine-tune that by adding a position switch,  or even use an H-bridge (requiring an extra pin), but as I have not been on holiday much during the lockdown, I have chosen to keep it manual.

To measure the outside temperature I put a DS18B20 in a piece of conduit pipe sprayed black  on the top half. the idea behind that is that the top will heat more than the bottom, causing an updraft, so the DS18B20 that is located in the bottom will always get fresh outside air

The code:
The core of the code is an Asynchronous webserver that presents the sensordata via server events, every 10 seconds. To prevent the ‘on-start’ values being empty I do an initial replace of the placeholders with the freshly read sensordata. and to make sure I also start with a negative number for the timer so the first 10sec update comes almost immediately.

If one wants to fill a webpage with data there are several approaches: 1. build the webpage, then read the sensors and fill them in; 2. Read sensors while building the webpage and fill them in step by step; 3 read all the sensors first.
I have chosen for the latter.

The webserver contains several icons. I have used 3 ways to put those icons there: 1. from the fontawesome library; or of not available there: 2. directly inserted with base64 codes; or 3. from SPIFFS.
I did that mainly to play around with different techniques, but to use SPIFFS just for 1 picture might be a bit silly and you may consider to not do that. (Edit: I did away with the SPIFFS picture, just picked another one from fontawesome)

The code keeps track of max and min temperature during a 6 a.m to 6 a.m. 24 hr timeslot. Should you lose power or update the code in between via OTA, or if for whatever reason the processor resets, the kept min and mac temperatures are lost. It is easy to prevent that by storing them in SPIFFS (but you will rapidly wear  out the EEPROM), or RTC memory (in a 4 byte per value block). The RTC memory survives reset but not a power outage. I just did not find it important enough to bother with and it introduces some other problems. Mind you that rtcData stored in the first 32 blocks will be lost after performing an OTA update, because they are used by the core internals.

Thingspeak
The thingspeak code is very straightforward. You will need to open a Thingspeak account though and note the channel number and write API code to insert in the program.

coldframe-thingspeak

When using Thingspeak, one can install one of the many Thingspeak apps (such as Thingshow, Thingview or Thingviewer)on one’s phone for easy acces to the data.

Thingshow

MySQL
In order to use this you will need to setup MySQL or MariaDB on a server somewhere. A raspberry Pi would be suitable for that. Earlier I referenced to two articles on randomnerd tutorials that describe how to set up a MySQL  database, no need to reinvent the wheel here, although I did adapt the code a bit.
coldframe-mysql

Googlesheet
You wil need to setup Googlesheet for that as I have described here. You will find an example here too.

Telegram
You will need to setup a Telegram account and create a bot. I describe the process here.

InfluxDB
I did not put my data in influxDB, but if you want it is rather easy to do that with a POST statement.
Add code like this:

#include <ESP8266HTTPClient.h>
HTTPClient ifhttp;
const char* influxUrl = "http://localhost:8086/write?db=mydb";
const char* influxUser = "username";
const char* influxPassword = "password";
String influxData = "";

void (influxDB()
{
influxData ="........" ---> add your fields and data here


// Create POST message
ifhttp.begin(influxUrl);
ifhttp.addHeader("Content-Type", "application/x-www-form-urlencoded");
ifhttp.setAuthorization(influxUser, influxPassword);

// The first attempt to send returns -1
int httpCode = -1;
while(httpCode == -1){
httpCode = ifhttp.POST(influxData);
http.writeToStream(&Serial);
}
ifhttp.end();
}

There is also a library for InfluxDB and when opening the InfluxDB server an example for the ArduinoIDE can be found (in the server go to “Data-Arduino”).

Carriots
Carriots, now called Altair Smartworks is an IoT data collection and processing site. It allows you to store your data and have it processed and attach specific actions to it. It might be useful and is fairly easy to add. Check here.

The code can be updated via website OTA. For development serial OTA is the easiest, so if that is what you prefer: I left the original serial OTA statements in so it is easy to convert back.

Download
You can download the code here. A remark though: it is my ‘as is’ working code and while developing it, I may have put in some unnecessary things and comments and also my variables might not have been grouped logically. Normally i clean up code before publishing it….but that means I have to retest it again and i just did not have the time for that right now. I may in future. But as said, it is a working code. You do need to add your own data such as passwords and API’s though. I am in the process of rewriting the code from the ground up,based on experience gained with the using the  ColdFrame for some time

————————————————————

 

Experience after a few weeks:
In general the system functions well. The cooling/heating system was able to keep the temperature above 0 degrees celsius when the temperature outside dropped to minus 6 at night for a few hours. It had a bit more trouble when it was freezing minus 6 the entire day. Mind you though that a relatively small object is harder to keep warm because of the unfavorable volume vs. surface ratio.

The HTU21D was a bit of a disappointment. According to the datasheet, the humidity operating range is 0-100% and I have no reason to doubt that. However, after continued exposure to 100% RH, condensation within the sensor can play havoc on the results. RH values above 100% or below 0% are possible then. Robert Smith in fact mentions this in his excellent comparison of various humidity sensors. He though sees it as a possibility to “do your own calibration”. I think though that if the HTU21D jumps from say 114% to -3% within 10 secs, there is more at large than the calibration not being fine tuned.

The question though is whether the condensation also influences the temperature readings. I do not think so, but I might add another DS18B20 just to make sure. I doubt if another humidity sensor would do better. Measuring near the upper limit of the range for a prolonged period seems to be problematic for all. The problem is the condensation in the sensor and no doubt other sensors, like the BME280 would suffer the same fate (eventhough the BME280 seems to use different type of sensor (resistive vs capacitive)). Also here, the relatively small size of the box may play a role as a relatively low amount of water will easily raise the potential humidity.
Ventilation can help, but it is a 2 edged sword: ventilation will generally also drop the temperature, which will make the relative humidity go up. Also the 100% humidity would normally occur when the temperature would go down to 6-7 degrees or lower, i.e. usually at night, and that is the moment you do not want the temperature to drop any lower.
Alternatives: BME280, Sensirion SHT4x or SHT71, HDC1080, AHT20 (datasheet mentions in fact a drift if subject to prolonged >80%RH). The Si7021 is practically identical to the HTU21D (in fact you may get the one if you order the other at less than reputable shops). They are only distinguishable by the serialnumber that supposedly is 60 for the HTU21D and 57 for the Si7021.

The BH1750 functioned well, it provided good information on what hours were most sunny on the spot the coldframe was standing.. It had one quirk though, occasionally (though very rarely), it would display the default value of 56412 Lux for a while and then it would be OK again ( did not find any fault in the wiring). A full reset would normally ‘cure’ it as well.

Obviously not a matter of just ‘finetuning’

The reporting through Telegram worked fine. Only 2 little ‘annoyances’ When the temperature ‘hoovers around 0 degrees or 25 degrees celcius (the lower and upper limit warning levels), I could get a couple of warnings in a row: I get a warning when the temparature dropped below zero, then when it would briefly rise above zero again the warning flag would be reset and subsequently if the temperature would drop below half a minute later I would get a warning again. It is basically the system doing what it was told to do. I could ‘solve’ that but not allowing a second warning within say 5 minutes, but I really did not find it important enough.

The leaf wetness sensor functioned ‘moderately’. Somehow the surface was not as appealing to condensation as the glass top (or the HTU21D sensor for that matter). As I still consider it important info, I will try to find another solution for it.

The combination of the webserver info that is basically à la minute (every 10 secs to be precise) and Thingspeak/MySQL/GoogleSheet that gives an historic overview worked very well.

Improvements
Apart from the HTU21D and leaf wetness sensor I have to look at/reconsider, there are a few things I may add: currently experimenting with a TCS34725 sensor to see if I can monitor ‘green development’.
I am considering if I should add a moveable setpoint for the cooling fan to set in, but it may not be important enough.
Currently I have 4 ADC channels through the PCF8591. I use one of those channels for soil moisture. If I want to use the system in a larger coldframe, or a greenhouse, I may need to add some ADC’s.
The current soil sensor I use (carbon rods) works fine, albeit that the thickness and bluntness of the rods can make it hard to stick it in a pot, so I may decide for a slightly leaner and sharper capacitive sensor, or just use metal nails. The latter though would require a transistor to switch off the current when not used (in order to minimize electrolyses), which takes an extra pin of the ESP8266.
I may expand the function of the lid opener to automatically stop when the lid is open or the lid is closed.
Also considering to add Tweets for notification next to Telegram


If you have a Thingspeak account, tweets can be sent either from your ESP via “thingtweet” or directly from Thingspeak.

I did add an emergency heating in the mean time. It is set to kick in if the temperature drops below 0 degrees, in spite of the underground heatsink providing heat. It is basically a relay that can switch on a heater. In my cold frame it is a resistor heater, if used in a greenhouse it could be a dedicated electrical heater. With seemingly only 2 projected frost days remaining till mid May, I will need to wait with proper testing.

Coldframe Aircon

When the Coldframe is getting too hot, the fan will pull hot air through the underground heatsink that has a rather steady temperature 0f 8-12 degrees Centigrade. When it is getting too cold, the same fan will pull cold air through the underground heatsink which then will warm the air. If that is not enough to prevent coldframe temperature to stay above zero, a heater above the fan wil kick in. I made the heater from 24 half-Watt resistors that are pushed to their limit, producing 12 Watt in heat. The main aim is to keep the temperature above freezing, not to heat the coldframe to say 20 degrees. As said, this is still experimental stage.
It happened to me once that the 6 a.m. telegram message was not received. Checking my SQL database and Thingspeak showed the connection was briefly dropped around 6 a.m. As I do not do any error checking on the connection with Telegram (or MySQL for that matter), the program is not aware of the message not being received.
If it is important to you, the result of bot.sendMessage(chat_id, bottext, "HTML"); is “0” if the transmission failed and “1” if it succeeded. You could do a check on that and resend if necessary. I am using an insecure connection with Telegram by specifying “setInsecure” in the setup. That’s easier and I am not sending top secret info. If you want the connection to be secure, you can do that by stating:

X509List cert(TELEGRAM_CERTIFICATE_ROOT);
WiFiClientSecure botclient;
UniversalTelegramBot bot(BOT_TOKEN, botclient);

Experience after a few months
When summer came, I had trouble loading the coldframe’s website on my network. As I still could load the the AP website when I was close to the coldframe, I was suspecting something was blocking the connection to the router. The most likely suspect was a row of potato filled containers between the coldframe and my house. When those grew into firm plants it isn’t unlikely to think these walls of moisture blocked the signal. Yet the connection to Thingspeak as well as my mySQL server seemed not to suffer.

Granted, I do not really use the coldframe in the summer, yet it might be wise to use a Wemos Pro (or ESP8266-7) with external antenna and place a router or WiFi extender in a room directly facing the garden (currently it connects to a router on the opposite side of the house, 1 floor up).

Also, when I will build another coldframe, I will try to find a way to remove the electronics from the box in an easier way, e.g.by making external plug connections with the ventilators, motors and some of the sensors. and I will start streamlining the code a bit more.

Strange behaviour in ESP8266 AccessPoint was password related

In my previous article, I made a skeleton program for a tutorial about the various ways of WiFi connection of an ESP8266, I ran into an odd problem that turned ourt to be related to the password.
the code aimed to include both STA and AP connection and I set up the credentials like this.

#define WIFI_SSID "YOURSSID"
#define WIFI_PASS "YOURPASSWORD"

// Set AP credentials
#define AP_SSID "YOUR_DESIRED_AP_NAME"
#define AP_PASS "YOUR_DESIRED_AP_PW"

After uploading the acces point ofcourse identified itself as ‘YOUR_DESIRED_AP’ and I quickly set to correct my mistake of not filling out my credentials and I changed it to:

// Set AP credentials
#define AP_SSID "Pond"
#define AP_PASS "1234"

After uploading again, I was surprised to see that instead of the expected AP SSID ‘Pond’, I now saw ESP-859C2F, with the numbers being the last 3 bytes of my ESP’s MAC-number. Chosing that accespoint made me connect to the ESP without asking for a password.
what the heck was going on? I knew the code was allright as on myinitial upload I had seen the SSID of the AP when it was still called “YOUR_DESIRED_AP_NAME”.
To make a long story short: I found out the problem was with the password: apparently there is a problem with a password that is too short, or contains numbers.
1234–> no proper AP_SSID visible
abcd–>works well
abcd1234 –>no proper AP SSID visible
abcd123–> works well.
So initially I thought that perhaps the amount of letters needed to be bigger than the amount of numbers in th epassword.
but ‘1234567’ worked fine….till I tried ‘123456’ which did not work, and then ‘1234567’ did not work either anymore and i had to go to ‘12345678’ to make it work.
Note: I uploaded the sketches with the “Tools-erase all flash content set”.
So I am still not entirely sure the exact problem, but I do know that numbers in the password apparently can cause problems with the AP

ESP8266 WiFiConnections

This article is probably cut and dry for those who have been working with the ESP8266 for more than a week, but every now and then I do get questions on how to connect an ESP8266 to the internet.

The code below shows how to do that in a combined STA and AP mode. It shows various options, including a static IP address. It includes a simple webserver, just to show the AP and STA connections are made. You can also download the code here.

#include <ESP8266WiFi.h> //for the WiFi connection
#include "ESP8266WebServer.h" //for the Server
ESP8266WebServer server(80);

// Set WiFi credentials
#define WIFI_SSID "YOURSSID"
#define WIFI_PASS "YOURPASSWORD"

// Set AP credentials
#define AP_SSID "YOUR_DESIRED_AP_NAME"
#define AP_PASS "YOUR_DESIRED_AP_PW"

//Extra's. these are optional
#define AP_CHANNEL 1   //choose the channel
#define AP_HIDDEN false //SSID broadcasted or not
#define AP_MAX_CON 4 // max nr of connections to the AP

uint8_t macAddr[6];

//for static IP (all not necessary if you choose DHCP)
// Set IP addresses
IPAddress local_IP(192, 168, 1, 70);
IPAddress gateway(192, 168, 1, 1);
IPAddress subnet(255, 255, 255, 0);
//change the default IP address for the AP (192.168.4.1)
IPAddress AP_IP(192,168,4,5);

void setup() {
  // Setup serial port
  Serial.begin(115200);

  //begin Acces Point
  WiFi.mode(WIFI_AP_STA);
  //WiFi.mode(WIFI_STA);// for just Station connection
  //WiFi.mode(WIFI_AP);//for just AP connection

  //WiFi.softAP(AP_SSID, AP_PASS); //this makes your Acces point connection
  WiFi.softAPConfig(AP_IP, AP_IP, subnet);// this is optional
  WiFi.softAP(AP_SSID, AP_PASS, AP_CHANNEL, AP_HIDDEN, AP_MAX_CON);

  // Begin WiFi
  WiFi.config(local_IP, gateway, subnet);// optional, for  static connection
  WiFi.begin(WIFI_SSID, WIFI_PASS);// this makes your STA connection

  // Loop continuously while WiFi is not connected
  while (WiFi.status() != WL_CONNECTED)
  {
    delay(100);
    Serial.print(".");
  }

  // Connected to WiFi
  WiFi.setAutoReconnect(true); //For automatic reconnect
  WiFi.persistent(true); // ditto
  Serial.println("Connected");
  Serial.print("Network IP address: ");
  Serial.println(WiFi.localIP());
  Serial.print("IP address for network ");
  Serial.print(WIFI_SSID);
  Serial.print(" : ");
  Serial.println(WiFi.localIP());
  Serial.print("IP address for network ");
  Serial.print(AP_SSID);
  Serial.print(" : ");
  Serial.println(WiFi.softAPIP());
  WiFi.softAPmacAddress(macAddr);
  Serial.printf("MAC address = %02x:%02x:%02x:%02x:%02x:%02x\n", macAddr[0], macAddr[1], macAddr[2], macAddr[3], macAddr[4], macAddr[5]);
  Serial.println("");
  //----------------
  server.on("/other", []() {   //Define the handling function for the path

    server.send(200, "text / plain", "Other URL");

  });

  server.on("/", handleRootPath);    //Associate the handler function to the path
  server.begin();                    //Start the server
  Serial.println("Server listening");
}

void loop() {
  server.handleClient();         //Handling of incoming requests
  // put your main code here, to run repeatedly:
  Serial.print("number of clients connected:");
  Serial.println(WiFi.softAPgetStationNum());
delay(3000);
  
}

// for a simple proof of context it would have been enough to just do server.send(200,"text/plain", "Hello World");
// but I added some html to make the content adapt to the size of the viewing device, so it would be properly visible on a phone as well
void handleRootPath() {            //Handler for the rooth path
  server.send(200, "text/html", "<!DOCTYPE HTML><html><head><meta name='viewport' content='width=device-width,initial-scale=1'></head><body>Hello world</body></html>");
}

Also check this post for some strange behaviour of the AccessPointI came across. You may also find this post interesting.

Smoothing

Smoothing of e.g. sensor readings to be displayed in a graph can be done in several ways. A rolling average, sometimes called a moving average, is one way of doing that. The arduino pages contain a program by David Mellis, modified by Tom Igoe that illustrates this: it stores values in a 10 value array and divides the sum by 10. Whenever the array is full it removes one value and adds a new one.

// Define the number of samples to keep track of. The higher the number, the
// more the readings will be smoothed, but the slower the output will respond to
// the input. Using a constant rather than a normal variable lets us use this
// value to determine the size of the readings array.
const int numReadings = 10;

int readings[numReadings]; // the readings from the analog input
int readIndex = 0; // the index of the current reading
int total = 0; // the running total
int average = 0; // the average

int inputPin = A0;

void setup() {
// initialize serial communication with computer:
Serial.begin(9600);
// initialize all the readings to 0:
for (int thisReading = 0; thisReading < numReadings; thisReading++) {
readings[thisReading] = 0;
}
}

void loop() {
// subtract the last reading:
total = total - readings[readIndex];
// read from the sensor:
readings[readIndex] = analogRead(inputPin);
// add the reading to the total:
total = total + readings[readIndex];
// advance to the next position in the array:
readIndex = readIndex + 1;

// if we're at the end of the array...
if (readIndex >= numReadings) {
// ...wrap around to the beginning:
readIndex = 0;
}

// calculate the average:
average = total / numReadings;
// send it to the computer as ASCII digits
Serial.println(average);
delay(1); // delay in between reads for stability
}

Sadly there is a fundamental flaw in this program:

For the initial 10 readings, the result is simply wrong: when the program starts, the array is empty. A value is read into it, and the sum of the array is then divided by 10. Suppose the analog source is quite stable and always gives a reading of 5. Then the first 10 results of the program will be: 5×9*0=5  ->/10=0.5 (rounded t0 0)
5+5+8*0=10 ->/10=1
5+5+5+7*0=15 ->/10=1.5  (rounded to 1)
etc. etc.
Given the fact that the readings are taken with a 1 microsecond delay, that might not be a huge problem, but in that case there are simpler ways, without the need for an array to smooth the readings. For instance like this:

int ReadSensor(){
  int i;
  int sval = 0;

  for (i = 0; i < 10; i++){
    sval = sval + analogRead(0);
  }

  sval = sval / 10;    // average
 
  return sval;
}

strictly speaking this only gives a moving average during this loop and then 10 completely new values are averaged, but at the 1 microsecond interval of the original program that would virtually be the same.

The results in the original program do become a nuisance when you take readings at say 1 minute intervals and you do not want to wait 10 minutes before you get correct data. Fortunately the program can be simply modified to correct for the mistake: for the first 10 readings we divide by the active indexnumber and only after that we divide by the total number of readings (10). We can simply do that by setting a flag:

 // if we're at the end of the array...
if (readIndex >= numReadings) {
// ...wrap around to the beginning:
readIndex = 0;
flag=1;
}

and then calculate results like so:

// calculate the average:
if (flag==0){
average = total / readIndex;
}else{
average=total/numReadings;
}

Now if you really want to smooth things over you could combine the two techniques presented here: take the average of say 50 readings, store the result in an array and calculate the moving average of the array:

// read from the sensor:
readings[readIndex] = ReadSensor();

and ofcourse add the function “ReadSensor”, given bove, to your program

Graphical presentation of Sensordata with ESP8266

In an earlier post I described the use of a graph, showing results of a BME280 and DS18B20 sensor. This was based on an article from siblings Rui anď Sara Santos from randomnerdtutorials, but tailored to my needs. The biggest difference with the original article is that i used a JSON for data transfer in order to display more than one line per graph. The original code as well as my adaptation had some drawbacks, one of them being that after opening the chart, one had to wait the full interval (set to 30 secs) before the first value appeared. That is now fixed (thanks JB): the first value will appear immediately on opening the chart.

As an example I am using an ESP8266-01 that addresses a PCF8591 ADC/DAC. The 4 ADC’s read 4 soil humidity sensors and plot those in a graph. The battery voltage is also plotted. As i use a LiFePO4 battery, there is no need for a regulator and i can simply read the Vcc in order to know the battery voltage. In order to save the soilsensors from electrolysis, i switch them on when i take a measurement and switch them off in between. Rather than using the free gpio1 and gpio3 (Tx and Rx) for that I use the DAC output.

One could argue (and i would agree) that it is quite useless to check soilmoisture every 30 secs where maybe once every 4-6 hrs suffices, but just see it as example of how you could use it say for a slow cooker or your oven or CV.

*These pins hve internal pullup on the ESP01S, but not on the ESP01

You will find the code here.

Another ‘issue’ was that since an external library (highcharts) is called, one needs active connection with internet. This is not always desirable or possible. There are 2 ways around this: download the highcharts library and put it on your own server, or download it and put it in the esp8266 FS and make some small adaptations to your code (solution provided by J. Bredahl). You will find that code here.

Graphical and Table presentation of database info using Googlecharts and PHP

In an earlier post I discussed storing sensor data in a MySQL or MariaDB database (in combo with DeepSleep, which was the focus of the article).
The database part was based on an article by randomnerdtutorials.
Output of the gathered MySQL data was in shape of a table*).

Sometimes though a graph is a more practical way to look at data over a period of time. There are 3 values in the table: temperature, Humidity and Airpressure.
As I wanted one graph, I decided to leave out the Airpressure, as it’s high values  would skew the  Y-axis a bit, so i will be looking at  2 values only: temperature and humidity. The Airpressure is easy to add though.

There are various possibilities of  presenting data in a graph. ‘Highcharts ‘ for example, or jpGraph. This time though I decided for Googlegraphs. Fortunately there is a lot of info available and i found some frameworks at Techjunkgigs that I could rework.

That turned out something like this:

or, with more data:
You will find the program here. There are a few things you need to check. The program expects a database file that is called ‘esp-data’. It checks a table called ‘SensorData’. It looks for fields called ‘value1’, ‘value2’ and ‘reading_time ‘. If you are using different names, you need to adpt tghe program. You also need to insert your database username and password.

You will notice that when time passes by the graph will be condensed more and more.
That is why I added a ‘WHERE’ statement in the query:
$query = “SELECT * from SensorData WHERE id >=1”;

where you can enter a higher ID number (or insert other criteria such as date, or temperatures above a certain value).
However, if you would only be interested in say only the last 100 records, it is better to replace the query by:

$query = "SELECT * FROM(SELECT * from SensorData WHERE id >=1 ORDER BY id DESC LIMIT 100) AS `table` ORDER by id ASC";
_______________

*) The presented table is a bit basic. There are more interactive ways to present data in a table.
espdata-search

You will find the code here.