The W5100 bug(s) – and how to fix it (them)

Bug 1- The SPI bug

On a recent project in which I shared the SPI bus of a 3V3 8MHz ProMini, with a W5100 Ethernet module and another SPI slave, I experienced some problems: sometimes it would work perfectly, sometimes not.
I had hooked up the ethernetmodule with Pin D10 as Chipselect and the other slave (an RFM69) with Pin D8 as Chipselect. Pretty standard and it should work, shouldnt it?
Well as it is, it didnt. Oddly enough, I had build it before with another Ethernet module and that worked fine.
When I connected my EthernetShield in the same way, the issue was gone. Time to throw suspicion on my Funduino W5100 Ethernetmodule. Well on inspection it is immediately clear that eventhough the functionality is supposed to be the same, the ethernetshield and also my previous module have more chips than my Funduino module. Why for instance does my shield have a 74LVC14 and my Funduino module just the W5100 chip and nothing more?

When looking at the circuit of the Ethernetshield it is clear that only one gate of the 74LVC14 (a SchmittTrigger inverter) is used: it takes the SS signal (the ‘chipselect’) inverts that and sends that inverted signal to Pin 31 of the W5100. Pin 31 is the SEN pin.

In the Funduino module that was different. The ‘SEN’ pin is just tied high to 3V3 with a 10k resistor. My shield has that pull-up resister as well, but still is controlled by the inverted chipselect signal.

Time for the datasheet of the W5100

Hmm.. that doesnt say much, other than that in my module the SPI mode is enabled, however, judging from the 74LVC14, it seems that this pin needs to be driven to LOW when the Chip is not selected. In other words, it is apparently necessary to disable the SPI mode in order to release the bus to another slave.

Time for some soldering: apart from the Vcc and ground, one needs two signals: the SS pin and the SEN pin. As the W5100 is an 80 LQTF chip, at my age (eyes) I was not even going to try soldering on the chip, but fortunately the Pullup resistor gives an entry point to the SEND signal and the connector is where we find the SS signal

Pullup resistor on SEN pin
Pullup resistor on SEN pin
Pullup resistor on SEN pin, close-up
Pullup resistor on SEN pin, close-up

I used a 74LCV14  (Low Voltage Inverting SchmittTrigger) for my modification, but I am sure an 74LC04 or 74LVT04 (an inverter) would do just as well. If you vcannot find the 74LCV14, try a 74LVT14 or 74HC(T)14 (it accepts  2 Volt Vcc). John Crouchley who also describes this problem uses a CMOS CD4011. He feeds that from 5Volt, which I did not want to do. It should be posible though, given the fact that the W5100 pins apear to be 5 V tolerant. Perhaps even a simple 1 transistor inverter is possible. Ideal would be an 74AHC1G04 as that is only a single inverter chip that measures only 2x3mm and can easily find a plce on the module.
Currently my modification is breadboarded. I will think of a neat solution and then add some more pictures.

Bug 2- The ‘510’ bug

There is another problem with some of the ‘asian webstore’ W5100 based Ethernet shields which isnt really a bug with the W5100 but more a problem with bad sourcing.
Some of those shields refuse to make contact with the internet because of some wrong components (resistors)In the picture here of the top of an ethernet shield, directly right of the Ethernet jack there is a “spider” resistor that is labelled with “49R9” that are actually 4 resistors in one  package with each a value of 49.9 Ohm. Apparently that was hrd to source at some time and  clone manufacturers decided to use 51 Ohm resitors. That is not a problem if indeed they had used 51 Ohm, but by mistake 510 ohm resistors were used (labelled ‘511’). Resistors with a value of 51 ohm should have been labelled with ‘510’.
So if you have such a board you need to replace the 510 ohm resistors. If you can’t find the proper replacement resistorpack then apparently it is also OK to solder  a 100 ohm resistor between pin 1-2 (Tx+/Tx-) and a 100 ohm resistor between pin 3 and 6 (Rx+/Rx-) as explained here. (So 2×100 Ohm resistors in total).

rj45
The RJ45 Jack, seen from component side

Bug 3 – The Funduino Reset Bug

Screenshot_2017-04-19_22-53-51The “Funduino” W5100 module as is a cheap, but qualitatively good W5100 board. However, apart from the ‘problem’ of not being able to share the SPI bus in unmodified state (adressed above), some  people have a lot of trouble getting it to work in the first place. The problem is then most likely in the Reset of the module. Modern versions of the Ethernetshield have a seperate Reset controller, that is triggered by the RST of the Arduino. The Funduino Module, just has its RESET tied to 3.3V via a 10 k resistor. As a result the Module’s reset state is a bit unpredictable. Some people get it to work by powering the module up and down a few times, but that remains random luck. What I found that works immediately is to add a slight delay in the setup, before the Ethernet connection is initialized. For me 250mS was enough, but you may need a bit more or a bit less.
So my Setup looks as follows:

void setup() {
// setup ethernet communication using DHCP
delay(250);//

Also, when you use the shield, the Reset pins of the Arduino and the shield are connected, so the Arduino can reset the shield. With this module you cannot do that: if you would connect both resets, you will notice that you cannot upload sketches. Perhaps it will work if you only connect the RST after you upload a sketch, I did not try that, too much hassle. I found the 250mS delay to work for me.
This ‘bug’ does not happen with all Funduino W5100 modules. Supposedly some batches are ‘OK’. There are 2 identification numbers: one on the PCB and one on the RJ45 jack. Supposedly if that says “15/10”  it is OK, “14/10” is not. Mine said “16/38” and needed the delay.
The only conenctions (other than Vcc and ground)  that I make are:
D10  -> nss
D11  -> miso
D12  -> mosi
D13  -> sck


		
Advertisements

Upload Data to Thingspeak through MQTT with an ESP8266

Sending  values to Thingspeak via the Thingspeak API is well known. There is another way as well: through MQTT. Thingspeak has recently (5 dec 2016) added a (one way) MQTT broker for this at mqtt.thingspeak.com:1883.

There are two topics one can use:
To upload more than 1 field in one session use:
channels/<channelID/publish/<channelAPI>

To upload an individual channel use:
channels/<channelID>/publish/fields/field1/<channelAPI> (just using field1 as example)

In the first case, the payload string is as follows:
field1=<value1>&field2=<value2>&status=MQTTPUBLISH

In the second case the payload string is just <value1>

In the program below I am using the PubSubClient from Knolleary. The “credentials.h” file is a file that defines my WiFi credentials, you can either create such a file yourself or just insert your wificredentials.

I am using an ESP8266 to make the connection but ofcourse it is also possible to use an Arduino with Ethernet connection when you make the proper changes  in this file in order to connect to Ethernet.

To avoid using again a DHT11 as an example, I show uploading variables by using micros() and a counter

#include "PubSubClient.h" //Knolleary
#include  <ESP8266WiFi.h> //ESP8266WiFi.h
#include   <credentials.h> //This is a personal file containing web credentials

const char* ssid = WAN_SSID;// this constant is defined in my credentials file
const char* password = WAN_PW;// ditto
//char* topic="channels/<channelID/publish/<channelAPI>
char* topic = "channels/123456/publish/T8I9IO457BAJE386"; 
char* server = "mqtt.thingspeak.com";

WiFiClient wifiClient;
PubSubClient client(server, 1883, wifiClient);

void callback(char* topic, byte* payload, unsigned int length) {
  // handle message arrived
}

void setup() {
  Serial.begin(115200);
  delay(10);
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);
  
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.println("WiFi connected");  
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());

String clientName="ESP-Thingspeak";
  Serial.print("Connecting to ");
  Serial.print(server);
  Serial.print(" as ");
  Serial.println(clientName);
  
  if (client.connect((char*) clientName.c_str())) {
    Serial.println("Connected to MQTT broker");
    Serial.print("Topic is: ");
    Serial.println(topic);
    
    if (client.publish(topic, "hello from ESP8266")) {
      Serial.println("Publish ok");
    }
    else {
      Serial.println("Publish failed");
    }
  }
  else {
    Serial.println("MQTT connect failed");
    Serial.println("Will reset and try again...");
    abort();
  }
}

void loop() {
  static int counter = 0;
  String payload="field1=";
  payload+=micros();
  payload+="&field2=";
  payload+=counter;
  payload+="&status=MQTTPUBLISH";
  
  if (client.connected()){
    Serial.print("Sending payload: ");
    Serial.println(payload);
    
    if (client.publish(topic, (char*) payload.c_str())) {
      Serial.println("Publish ok");
    }
    else {
      Serial.println("Publish failed");
    }
  }
  ++counter;
  delay(20000);
}

The file is available for download here. Whether this is a better method than with the api remains to be seen.
Currently the connection time is limited because of the limited number of sockets on Thingspeak so it is ‘connect->upload->disconnect’. Thingspeak currently cannot be used as a ‘broker’. The traffic is one way only. If your client is already connected to an MQTT network on your own private or public broker, then this method cannot be used without ‘bridging’ the two ‘networks’

Adding an MCP23017 16 port IO expander to Arduino or Esp8266 or Attiny85 or……..


Update:
After I made this expander module, a ready made module
with this chip has become available. So I actually would advice anybody needing a 16 bit expander, to buy that one rather than build it. The module will cost you abt 1.50 euro, while the individual chip may set you back a euro or so.

I am not claiming that what I am describing here is earth shattering or trailblazing, because in fact it is very simple and no doubt has been done by many already. But sometimes what is simple for the one, is still a question mark for the other, so here is quick ‘how-to’ of adding 16 I/O ports to your microprocessor. This is especially handy when working with a chip like the ESP8266 that has only limited I/O
The MCP23017 is an I2C enabled 16 I/O port chip. That means that you only need 2 pins (yes with Vcc and ground it makes 4) to control the chip and the added advantage is that you can share I2C with various other devices as well.

The 16 I/O lines are divided into an 8 I/O PORT A and an 8 I/O PORT B. Both can be used as input as well as output. The chip also has 2 configurable interrupts (that I will not be using). The physical layout of the chip makes it quite easy to use it on a piece of strip board.

The circuit (at right) is rather simple. At a last moment I decided to leave out the pull up resistors so it would be more flexible to use together with other I/O devices. The 3 Address pins A0-A2 determine the I2C address that ranges from 0x20 (all pins on ground) to 0x27 (all pins on Vcc).
The chip  can take a Vcc from 2.7V to 5V and this is perfect for 3.3 Volt devices as  the modern arduino’s and the ESP8266 range of boards.

Using the chip in a program is fairly easy. There are good libraries available, but it might help if you know how to program the chip without a library.
In my case I have all  address lines tied to ground and therefore my I2C address is 0x20. Suppose I want to use all PORT A lines as outputs. I do that  as follows:

Wire.beginTransmission(0x20);
Wire.write(0x00); // IODIRA register
Wire.write(0x00); // set entire PORT A to output
Wire.endTransmission();

For PORT B that  is rather similar:

Wire.beginTransmission(0x20);
Wire.write(0x01); // IODIRB register
Wire.write(0x00); // set entire PORT B to output
Wire.endTransmission();

If we then want to send a specific value ‘X’ to that PORT A, we do that as follows

Wire.beginTransmission(0x20);
Wire.write(0x12); // address port A
Wire.write(X);  // value to send
Wire.endTransmission();

‘X’ ofcourse is a byte value that determines whether we set a specific port HIGH or LOW.
If for instance ‘X’is ‘0’ that means we write a LOW to all PORT A outputs. If it is 255 that means we write a HIGH to all PORT A outputs.
To determine what value to send, consider the 8 I/O lines of PORT A as a byte in which the individual bits determine HIGH or LOW.
So if we only want to make PORTA.0 HIGH and the rest LOW, we write a binary value of 0b00000001 =1 to the A register. If we want to make PORTA.0 and PORTA.2 HIGH and the rest LOW we write a binary value of 0b00000101 = 5.
For PORT B it is similar:

Wire.beginTransmission(0x20);
Wire.write(0x13); // address PORT B
Wire.write(X);  // value to send
Wire.endTransmission();

If we want to use PORT B (or PORT A for that matter) as input, we do that as follows:

Wire.beginTransmission(0x20);
Wire.write(0x13); // address PORT B
Wire.endTransmission();
Wire.requestFrom(0x20, 1); // request one byte of data
byte input=Wire.read(); // store incoming byte into "input"

The byte “input” will vary between 0 and 255, in which the individual bits determine the input on the corresponding IO line. So if ‘input’  reads ‘3’  which in binary is 0b00000011, that means that both IO line 0 and 1  were HIGH and the rest LOW

#include <Wire.h> // Wire.h
byte input=0;
void setup()
{
  Serial.begin(9600);
  Wire.begin(); // wake up I2C bus
  Wire.beginTransmission(0x20);
  Wire.write(0x00); // IODIRA register
  Wire.write(0x00); // set entire PORT A as output
  Wire.endTransmission();
}
 
void loop()
{
  // read the inputs of bank B
  Wire.beginTransmission(0x20);
  Wire.write(0x13);
  Wire.endTransmission();
  Wire.requestFrom(0x20, 1);
  input=Wire.read();
 
  // now send the input data to bank A
  Wire.beginTransmission(0x20);
  Wire.write(0x12); // address PORT A
  Wire.write(input);    // PORT A
  Wire.endTransmission();
  delay(100); // for debounce
}

That’s basically it if you want to do the adressing yourself. Using a library, such as the one from Adafruit, makes it much easier though as it has commands to write and read from individual IO lines. One of the example programs to read a single button, looks  for instance like this:

#include <Wire.h> // Wire.h
#include "Adafruit_MCP23017.h"

// Basic pin reading and pullup test for the MCP23017 I/O expander
// public domain!
// Connect pin #12 of the expander to Analog 5 (i2c clock)
// Connect pin #13 of the expander to Analog 4 (i2c data)
// Connect pins #15, 16 and 17 of the expander to ground (address selection)
// Connect pin #9 of the expander to 5V (power)
// Connect pin #10 of the expander to ground (common ground)
// Connect pin #18 through a ~10kohm resistor to 5V (reset pin, active low)
// Input #0 is on pin 21 so connect a button or switch from there to ground

Adafruit_MCP23017 mcp;

void setup() 
{
mcp.begin();      // use default address 0
mcp.pinMode(0, INPUT);
mcp.pullUp(0, HIGH);  // turn on a 100K pullup internally
pinMode(13, OUTPUT);  // use the p13 LED as debugging
}

void loop() {
// The LED will 'echo' the button
digitalWrite(13, mcp.digitalRead(0));
}

If you want to use more than one MCP23017 do that as follows:

#define addr1 0 //addr1 =A2 low , A1 low , A0 low =000
#define addr2 1 //addr 2 = A2 low , A1 low , A0 high =001 
setup(){
    mcp1.begin(addr1);
    mcp2.begin(addr2);
}

Mind you that “0” is in fact 0x20 and ‘1’ is in fact 0x21

If you are using the Adafruit library with the ESP8266, you will encounter a compilation error signaling it cannot find <avr/pgmspace.h>. The solution for this is easy:

Open the cpp file of the library.

replace

#include <avr/pgmspace.h>

with

#if (defined(__AVR__))
#include <avr/pgmspace.h>
#else
#include <pgmspace.h>
#endif

Calculating Sunrise and Sunset on Arduino (or other microcontroller)

Knowing the hours of sunset and sunrise may be handy in a variety of situations, an automated chicken coop door might be only one example.

There are several ways to get the proper times: a lookup table in EEPROM, the Timelord library or one of its successors, the Dusk2Dawn library, a rather complicated calculation including the Julian calender, or a fairly simple approximation that I will discuss here.

This method uses the average of the earliest and latest sunset and then for any given day  adds or subtracts a certain amount of time with a maximum of half  of the difference between the earliest and latest sunrise.

Rob Tillaert discusses the method here. It presumes that the sunrise time follows a (co)sinoidal function. I will try to visualize it with a simple example:

Say that on June 23 the earliest sunrise of the year occurs at 4am, and that the latest sunrise of the year occurs at 23 December at 6 am.

Then you know that on any other day the sunrise is between 4 and 6 am. If you take the average that is 5 am, then you know that every other sunrise that year is either 0-1 hr later than 5 am or 0-1 hr earlier than 5 am.

It is the latter that is captured in the formula:

t=avg+0.5Δ*cos((doy+8)/58.09)

  • avg is average sunrise time, in minutes since midnight
  • Δ the difference between the earliest and latest sunrise time
  • doy is the day of the year
  • the 8 is there because we start on the wintersolstice: 23 December is 8 days before jan 1
  • 58.09 is 365/2π. That is necessary because the cosinusfunction has max 2π as input.

If you live in a DST zone, the earliest sunrise wil be under DST, however you need the non-DST corrected time: the sun knows no DST. Calculate firstm then add DST later

For my location the earliest and latest sunrise are:

earliest sunrise is at 4.19 am
latest sunrise is at 8.51
in order to use them in our equation, we have to calculate them in minutes past midnight:
4.19= 4×60+19=259
8.51= 8*60+51=531
The average is (259+531)/2=395
the difference or delta is 531-259=272. We need half of that which is 137.
The equation then becomes:

395+137*cos((doy+8)/58.09)

To check the accuracy of the approximation, I plotted the actual sunrise times (blue curve) against the calculated sunrise time (red curve).

As it shows, the first half of the year is a perfect fit, the second half of the year seems to follow a more linear curve with the max deviation being 20 minutes, that may or may not be accurate enough for your project. With the aid of this curve though I could opt for a linear approximation for the 2nd half of the year.

For sunset we can practically use the same formula, be it that we now have to subtract the variable part rather than add it.
For my location the sunset is as follows:
latest: 22:07 =1327
Earliest: 16:27= 987
avg=2287/2=1144
delta=240 ->170
sunset=1144-170*cos((doy+8)/58.09)

That gives the following graph:

This time I didnt bother to enter all the  real sunset times, but it is clearly visible that there is a reasonable fit that could maybe be enhanced a bit by shifting it slightly more to left or decreasing the delta a bit. Again Red graph is the calculated sunset, the blue is the actual sunset. None of the graphs has been corrected for DST.

A procedure for the Arduino would look as follows:
Where DST is a byte indicating whether DST is active (1)  or not active (0).
The day of the year I pull from my RTC library but it can also be calculated as follows:
int(((month-1)*30.5)+dayOfMonth)  (that is an approximation though)

The sunrise and sunset are both given in minutes after midnight. The hour and minutes of the sunrise (and sunset) can be calculated by:
hour=sunrise/60
minute=sunrise%60

MQTT with the W5100 Ethernetshield

In a previous post I discussed the use of an old ENC28J60 ethernetshield for an MQTT node. The biggest setback of the ENC28J60 shield is that it uses a lot of memory. When using the newer W5100 Ethernetshield, there is a bit more room to play with. So here is a simple framework  for an MQTT node that doesnt’t only publish, but also reacts to simple commands that come in with a subscription.

MQTT-Spy output
MQTT-Spy output
/*
          Arduino UNO with W5100 Ethernetshield or  W5100 Ethernet module, used as MQTT client
          It will connect over Wifi to the MQTT broker and controls a digital output (LED, relay)
          and gives the Temperature and Humidity, as well as the state of some switches
          The topics have the format "home/br/sb" for southbound messages and  "home/nb" for northbound messages
          Southbound are messages going to the client, northbound are messages coming from the client
          As the available memory of a UNO  with Ethernetcard is limited, I have kept the topics short
          Also, the payloads  are kept short
          The Northbound topics are
          home/br/nb/temp  for temperature
          home/br/nb/humid  for humidity
          home/br/nb/deur  for a door switch
          home/br/nb/l for  the lightintensity
          home/br/nb/pr  for the status of a PIR sensor
          home/br/nb/ip showing the IP number of the client
          home/br/nb/relay showing the relaystate

          There is only one southbound topic:
          home/br/sb
          The payload here determines the action:
          0 -Switch the relay off
          1-Switch the  relay on
          2-Publish the IP number of the client
          3 Ask for the relaystate REMOVED

          On Startup, the Client publishes the IP number

*/
#include "Ethernet.h"
#include "PubSubClient.h"
#include "DHT.h"
//kennelijk  geeft update van DHT sensor library boven 1.2.1 een fout
#define CLIENT_ID       "Hal"
//#define TOPIC           "temp"
#define PUBLISH_DELAY   3000
#define PUB_TOPIC "my_username/f/temperature" //Adafruit dashboard
#define PUB_TOPIC_h "my_username/f/humidity" //Adafruit dashboard
#define DHTPIN          3
#define DHTTYPE         DHT11
#define ledPin 13
#define relayPin 8
String ip = "";
bool statusKD = HIGH;
bool statusBD = HIGH;
bool statusGD = HIGH;
bool relaystate = LOW;
bool pir = LOW;
bool startsend = HIGH;
int lichtstatus;
uint8_t mac[6] = {0x00, 0x01, 0x02, 0x03, 0x04, 0x06};

EthernetClient ethClient;
PubSubClient mqttClient;
DHT dht(DHTPIN, DHTTYPE);

long previousMillis;

void setup() {
  pinMode(4, INPUT_PULLUP);
  pinMode(5, INPUT_PULLUP);
  pinMode(6, INPUT_PULLUP);
  pinMode(7, INPUT);
  pinMode(LED_BUILTIN, OUTPUT);
  pinMode(relayPin, OUTPUT);

  // setup serial communication

  Serial.begin(9600);
  while (!Serial) {};
  Serial.println(F("MQTT Arduino Demo"));
  Serial.println();

  // setup ethernet communication using DHCP
  if (Ethernet.begin(mac) == 0) {
    //Serial.println(F("Unable to configure Ethernet using DHCP"));
    for (;;);
  }

  Serial.println(F("Ethernet configured via DHCP"));
  Serial.print("IP address: ");
  Serial.println(Ethernet.localIP());
  Serial.println();
  /*
    Serial.println(Ethernet.localIP()[0]);
    Serial.println(Ethernet.localIP()[1]);
    Serial.println(Ethernet.localIP()[2]);
    Serial.println(Ethernet.localIP()[3]);
  */
  ip = String (Ethernet.localIP()[0]);
  ip = ip + ".";
  ip = ip + String (Ethernet.localIP()[1]);
  ip = ip + ".";
  ip = ip + String (Ethernet.localIP()[2]);
  ip = ip + ".";
  ip = ip + String (Ethernet.localIP()[3]);
  //Serial.println(ip);

  // setup mqtt client
  mqttClient.setClient(ethClient);
  //mqttClient.setServer(mqttServer, 1883);
  // mqttClient.setServer("test.mosquitto.org", 1883);
  //  mqttClient.setServer( "raspberrypi.local",1883);
  //mqttClient.setServer("io.adafruit.com",1883);
  mqttClient.setServer( "192.168.1.102", 1883);
  //Serial.println(F("MQTT client configured"));
  mqttClient.setCallback(callback);
  // setup DHT sensor
  dht.begin();
  Serial.println(F("DHT sensor initialized"));

  Serial.println();
  Serial.println(F("Ready to send data"));
  previousMillis = millis();
  mqttClient.publish("home/br/nb/ip", ip.c_str());
}

void loop() {

  statusBD = digitalRead(4);// FrontdoorSwitch
  statusGD = digitalRead(5);// Garagedoor Switch
  statusKD = (digitalRead(6));//LivingRoom Switch

  lichtstatus = analogRead(A0);//Reads an LDR
  pir = digitalRead(7);//Reads a PIR sensor
  relaystate = digitalRead(relayPin);// Reads the state of a relay

  // it's time to send new data?
  if (millis() - previousMillis > PUBLISH_DELAY) {
    sendData();
    previousMillis = millis();

  }

  mqttClient.loop();
}

void sendData() {

  char msgBuffer[20];
  float h = dht.readHumidity();
  float t = dht.readTemperature();
  Serial.print("Temperature: ");
  Serial.print(t);
  Serial.println("oC");
  Serial.print("Humidity: ");
  Serial.print(h);
  Serial.println("%");
  Serial.print("Relay is: ");
  Serial.println((relaystate == LOW) ? "OPEN" : "CLOSED");
  if (mqttClient.connect(CLIENT_ID)) {
    mqttClient.publish("home/br/nb/temp", dtostrf(t, 6, 2, msgBuffer));
    mqttClient.publish("home/br/nb/humid", dtostrf(h, 6, 2, msgBuffer));
    mqttClient.publish("home/br/nb/deur", (statusBD == HIGH) ? "OPEN" : "CLOSED");
    mqttClient.publish("home/br/nb/garage", (statusGD == HIGH) ? "OPEN" : "DICHT");
    mqttClient.publish("home/br/nb/bel", (statusKD == HIGH) ? "OPEN" : "CLOSED");
    mqttClient.publish("home/br/nb/l", dtostrf(lichtstatus, 4, 0, msgBuffer));
    mqttClient.publish("home/br/nb/p", (pir == HIGH) ? "OPEN" : "CLOSED");
    mqttClient.publish("home/br/nb/relay", (relaystate == LOW) ? "OPEN" : "CLOSED");
    mqttClient.subscribe("home/br/sb");
    if (startsend) {
     // mqttClient.publish("home/br/nb/relay", (relaystate == LOW) ? "OPEN" : "CLOSED");
      mqttClient.publish("home/br/nb/ip", ip.c_str());
      startsend = LOW;
    }
  }
}

void callback(char* topic, byte* payload, unsigned int length) {
  char msgBuffer[20];
  // I am only using one ascii character as command, so do not need to take an entire word as payload
  // However, if you want to send full word commands, uncomment the next line and use for string comparison
   //payload[length] = '\0';            // terminate string with '0'
  //String strPayload = String((char*)payload);  // convert to string
  // Serial.println(strPayload); //can use this if using longer southbound topics
  Serial.print("Message arrived [");
  Serial.print(topic);
  Serial.print("] ");//MQTT_BROKER
  for (int i = 0; i < length; i++) {
    Serial.print((char)payload[i]);
  }
  Serial.println();
  Serial.println(payload[0]);

  // Examine only the first character of the message
  if (payload[0] == 49)             // Message "1" in ASCII (turn output ON)
  {
    digitalWrite(LED_BUILTIN, HIGH);    //
    digitalWrite(relayPin, HIGH);
  } else if (payload[0] == 48)      // Message "0" in ASCII (turn output OFF)
  {
    digitalWrite(relayPin, LOW);     //
    digitalWrite(LED_BUILTIN, LOW);
  } else if (payload[0] == 50)
  {
    mqttClient.publish("home/br/nb/ip", ip.c_str());// publish IP nr
  } else {
    Serial.println("Unknown value");
    mqttClient.publish("home/br/nb", "Syntax Error");
  }

}

You will find the full code for download here.

Setting up a local MQTT broker on the Raspberry Pi

On the web there are a ton of instructions and how to’s on how to do this, but I found  it easiest to just  issue two commands on Raspian Jessie:
sudo apt-get update
sudo apt-get install mosquitto mosquitto-clients python-mosquitto

then to test if it was all working, I opened 2 ssh terminals to my raspberry with:
ssh pi@192.168.1.102  (the latter being the ip nr from my raspberry)
and then in one of those terminals I entered:
mosquitto_sub -d -t hello/world
this made the terminal I will call Terminal 1 subscribe to the topic “hello/world”
in the other ssh terminal window (Terminal 2) I entered
mosquitto_pub -d -t hello/world -m "Greetings from Terminal2"
and immediately I got that message published in Terminal 1
(actually you don’t need the ‘-d’ option. That is just for diagnostics)
So I knew my local Mosquitto broker was working, at least within one machine, the raspberry on which it was installed. Time to check if it also worked on other machines. To do that I opened a third terminal on my desktop that didnt ssh to the raspberry but just gave a command line to the machine I was working on (a regular Linux desktop).

Before I could have that machine subscribe to an MQTT topic, I first had to install the MQTT client with:
sudo apt-get install mosquitto-clients

when that was done I entered:
mosquitto_sub -h 192.168.1.102 -t hello/world

mosquitto-terminalNow when I published a message on terminal2 (thats the raspberry) it got published on terminal1 (that same raspberry) and on terminal3 (a regular desktop), so now I knew everything was hunky dory.
However, having to use terminal command lines to check your MQTT broker is not the most convenient. I can advise everybody who works with MQTT to install MQTT-spy. That is a jar file that basically is a publisher and a client that keeps track of MQTT messages on your network.

It comes pre-configured with 3 public brokers but adding your local broker is very simple:
mqtt-localAfter starting MQTT-spy go to  ‘Connections-New Connection” Fill out a name for your connection and under the ‘connectivity tab’ add (in my case): 192.168.1.102:1883  with 1883 being the  standard MQTT port.
Then go to the ‘other’tab and check ‘auto subscribe when opened’.
Now in the main window add a subscription tab called ‘hello/world’ and republish  the messagemqttspy-3 from Terminal2: it should appear in the receive window of mqtt-spy. Vice versa, if  you publish a message from within mqtt spy, it will appear on terminal 1 also

MQTT with the ENC28J60 Ethershield

My old ENC28J60 based shield surely has gathered some dust
My old ENC28J60 based shield surely has gathered some dust

While playing with MQTT on various ESP8266’s, I started to wonder if maybe I could do something with an old ENC28j60 shield and module I still had laying around and actually hardly ever used.

For the youngsters: The ENC28J60 Ethernetshield was the first shield to connect the Arduino with the internet. The major drawback of the chip was that it was lacking a stack, that subsequently had to be constructed in software. As a result it was kinda memory hungry. The initial version was without an SD card slot, the later version had an SD card slot. There were two libraries available: The Ethercard and the Ethershield library. The UIPEthernet library came later.
As far as I could figure out, the PubSubClient library that is needed for MQTT doesnt work with the original Ethershield/EtherCard libraries. It does work with the UIPEthernet library though (extended fork here). The UIPEthernet library is a smart piece of coding that made programs written for the W5100 Ethernetshield suitable to be used with the ENC28J60 Shield, simply by changing the included library. Ofcourse there is a price to pay for this, namely more memory consumption. There is also an MQTT client for the ENC28J60/Atmega328 based Nanode. The Ethercard seems to work with the EthercardMQTT library.

Anyway, there still is enough memory to read a DHT11 sensor an analog port and some switches. In my case those were 3 door contacts. The PubSubClient is the original from Knolleary, though normally I am more a fan of the fork by Imroy. For this example I have used the Mosquitto public broker but ofcourse any broker can be used. I have installed the Mosquitto broker on a local raspberry and I found that a call to (“raspberrypi.local”,1883) does not work, using the  local 192.168.1.xxx ip number does work though

MQTT-Output
MQTT-Output ‘Dicht’ means ‘Closed’

The program is rather ‘spartan’, to save space. Print statements have been removed or commented out after initial testing.

#include <UIPEthernet.h>
#include "PubSubClient.h"
#include "DHT.h"

#define CLIENT_ID       "UnoMQTT"
#define INTERVAL        3000 // 3 sec delay between publishing
#define DHTPIN          3
#define DHTTYPE         DHT11
bool statusKD=HIGH;//living room door
bool statusBD=HIGH;//front door
bool statusGD=HIGH;//garage door
int lichtstatus;
uint8_t mac[6] = {0x00,0x01,0x02,0x03,0x04,0x05};

EthernetClient ethClient;
PubSubClient mqttClient;
DHT dht(DHTPIN, DHTTYPE);

long previousMillis;

void setup() {
pinMode(4,INPUT_PULLUP);
pinMode(5,INPUT_PULLUP);
pinMode(6,INPUT_PULLUP);
  // setup serial communication
  //Serial.begin(9600);
  // setup ethernet communication using DHCP
  if(Ethernet.begin(mac) == 0) {
    //Serial.println(F("Ethernet configuration using DHCP failed"));
    for(;;);
  }
  // setup mqtt client
  mqttClient.setClient(ethClient);
  mqttClient.setServer("test.mosquitto.org",1883);
  //mqttClient.setServer("192.168.1.xxx",1883); //for using local broker
  //mqttClient.setServer("broker.hivemq.com",1883);
  //Serial.println(F("MQTT client configured"));

  // setup DHT sensor
  dht.begin();
  previousMillis = millis();
}

void loop() {
  statusBD=digitalRead(4);
  statusGD=digitalRead(5);
  statusKD=digitalRead(6);
  lichtstatus = analogRead(A0);
  // check interval
  if(millis() - previousMillis > INTERVAL) {
    sendData();
    previousMillis = millis();
  }
  mqttClient.loop();
}

void sendData() {
  char msgBuffer[20];
  float h=dht.readHumidity();
  float t = dht.readTemperature();
  if(mqttClient.connect(CLIENT_ID)) {
   mqttClient.publish("hal/temp", dtostrf(t, 6, 2, msgBuffer));
   mqttClient.publish("hal/humid", dtostrf(h, 6, 2, msgBuffer));
   mqttClient.publish("hal/door", (statusBD == HIGH) ? "OPEN" : "DICHT");
   mqttClient.publish("hal/garage",(statusGD == HIGH) ? "OPEN" : "DICHT");
   mqttClient.publish("hal/kamer",(statusKD == HIGH) ? "OPEN" : "DICHT");
   mqttClient.publish("hal/licht", dtostrf(lichtstatus, 4, 0, msgBuffer));
 //hal=hallway, DICHT=Closed, kamer=room, licht=light
 }
}
The old ENC28J60 shield at work
The old ENC28J60 shield at work

Just for completeness sake, this sketch takes about 77% of memory. This same sketch for the WIZ5100 based Ethernet shield, with the  Ethernet.h library takes about 52% of memory. If you decide to adapt the sketch, be careful with altering the character strings. For instance, using the string “OPEN” 3 times is likely less memory consuming than having 3 different strings of the same or even shorter length. Obviously the ‘topic’ strings take a lot of space as well and if you were to shorten them to less meaningful names, you could add more sensors such as for instance a PIR sensor.

Should you copy the program from this  blog page, you may copy ‘stray characters’  that you have to delete. You can also download it here.

Freeing up memory

Should you really be pressed for memory, there is a way to free up about 5K of Flash: Go to your /../sketchfolder/libraries/UIPEthernet-master/utility/uipethernet-conf.h  and open the uipethernet-conf.h file.
in that file you will see the following section:
udp

If you set UIP_CONF_UDP to ‘0’ you will save 5kB flash, by disabling UDP. However, if you use DHCP to connect to your router, you cannot disable UDP as the DHCP connection requires UDP. In that case you still can gain a bit of memory by reducing the UIP_UDP_CONNS.
An example of using a fixed address is this:

#include <UIPEthernet.h>
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED }; 
//the IP address for the shield:
byte ip[] = { 192, 168, 1, 120 };

void setup()
{
 Ethernet.begin(mac, ip);
}
void loop() {}

A final warning… after I updated my libraries, including the Adafruit DHT library, I received an error on compiling this sketch. That disappeared when I returned it to version 1.2.1. I thought I was not using the Adafruit library but the Tillaert library, but DHT libraries come a dime a dozen so maybe my compiler linked in another than I thought.
(The main sketch is my adaptation of work I found on internet, but I think the original source is from Luca Dentella)