Connecting to Thingspeak with Ethernetshield, EtherCard, Ethershield or ESP8266

I wanted to collect some weather-data with my Arduino and have that available as nice graphs, on a webpage, so I could also monitor it from afar
One could of course just use an Arduino as a webserver, but if you want to do anything more than send numbers to a webpage, the Arduino will soon run out of memory.
There are services that will do that for you. Pachube, later called Xively is a known one but currently they have a waiting list for their free accounts. Truthfully, I have a Pachube account that became a Xively account, but just never got any results on it.

There’re few alternatives for Xively:
http://2lemetry.com
http://exosite.com
https://www.carriots.com
https://www.grovestreams.com
https://thingspeak.com
http://openenergymonitor.org

I picked ‘Thingspeak’
As the signup and creating a channel etcetera is quite easy and well explained I will not go into too much detail of that. Basically after you sign up you create a channel to which you add fields where later sensors will send their data.
Under the API tab you will find an API that you later should put in your program.

I describe a simple connection with an ethernet cable and a connection via WiFi

Hurdle 1
Now where to find a program? a working example would be nice right? In the right top corner there is a button ‘Support’ that will go to ‘Tutorials’.
Under ‘Tutorials’ you will find:

“Using an Arduino + Ethernet Shield to Update a ThingSpeak Channel”
Sounds great, so you download that program into your IDE, add the API -key and then compile it.
Darn…. it doesnt compile, you try to fix it (and that is certainly possible) until you come to a point you have no idea what is required.
Apparently the program still expects everybody to use the 022 or 023 IDE.
There is a link to a Github page but that will give u a program to tweet and that is not what you want, at least not for now.
Solution
You ill find a better program to start from right here:

https://github.com/iobridge/ThingSpeak-Arduino-Exa…

That program takes a reading from the A0 port and sends that to “Field1” in your data stream

Ok so you try that, you hang a variable resistor like an LDR or NTC on port A0, add your API in the program and run it.
That works fine, but I didn’t only want to read a value from an Analog port, I had a DHT11 Moisture&Temperature sensor as well as a BMP180 Pressure & temperature sensor. I figured it shouldn’t be too hard.

Hurdle 2
I added the necessary libraries to the Thingspeak, added the objects and read the sensors into a variable.
The sensors however have floats as outcome and Thingspeak wants you to send strings.
With most variables it is rather easy to turn them into a string with the simple ‘string’ function, but it isnt that easy for floats. With floats you have to use the “dtostrf” command (which I guess stands for ‘double-to-string-function’

Trying to find info on that function on internet quickly led me to endless discussions on ‘how stupid’ it was and people asking questions were often told “why would you need that, Serial.print will do that for you” Yeah, true, but I don’t want to print, I need it because Thingspeak wants it.

Solution
To use the dtostrf command you need to set up a buffer space where the string will be stored. It works like this:

char t_buffer[10];
t=(ReadSensor);
String temp=dtostrf(t,0,5,t_buffer);
That bufferspace is important. I had it working with ‘7’ or even ‘5’, but when I added a second sensor that needs this function, my data-stream would crash and or I got the weirdest results. I also figured that I could use the same bufferspace alternating for each sensor, but that also didn’t really work, so now i have a bufferspace for each sensor.
Now I am no crack in C, so maybe there is a better way to do this, if so i would love to hear it, but this worked for me.

Hurdle3
Once I had the string conversions, I could add the data to the data stream.
The Thingspeak example program shows that for one field only, but it becomes clear pretty fast that you have to add the strings.

Solution
So for say 4 different fields it becomes like this:

updateThingSpeak("field1="+temp+"&field2="+humid+"&field3="+pres+"&field4="+temp2);

The Program

Below you will find the full code.
Just a few remarks:
The BMP180 is an updated version of the BMP085. The BMP085 libraries are compatible with the BMP180.
AdaFruit has 2 versions of the library. I picked version 1 as i found it easier to work with. Version 2 also requires installation of the ‘Sensor’ library.
In the code I also present an extra float: ‘m’. that gives the Pressure in “mmHg” as I haven’t used it for now there is no string conversion yet and it is not added to the data stream

/*
Arduino --> ThingSpeak Channel via Ethernet
The ThingSpeak Client sketch is designed for the Arduino and Ethernet.
This sketch updates a channel feed with an analog input reading via the
ThingSpeak API (http://community.thingspeak.com/documentation/)
using HTTP POST. The Arduino uses DHCP and DNS for a simpler network setup.
The sketch also includes a Watchdog / Reset function to make sure the
Arduino stays connected and/or regains connectivity after a network outage.
Use the Serial Monitor on the Arduino IDE to see verbose network feedback
and ThingSpeak connectivity status.
Getting Started with ThingSpeak:
* Sign Up for New User Account -  https://www.thingspeak.com/users/new

* Register your Arduino by selecting Devices, Add New Device
* Once the Arduino is registered, click Generate Unique MAC Address
* Enter the new MAC Address in this sketch under "Local Network Settings"
* Create a new Channel by selecting Channels and then Create New Channel
* Enter the Write API Key in this sketch under "ThingSpeak Settings"
Arduino Requirements:
* Arduino with Ethernet Shield or Arduino Ethernet
* Arduino 1.0 IDE
Network Requirements:
* Ethernet port on Router
* DHCP enabled on Router
* Unique MAC Address for Arduino
Created: October 17, 2011 by Hans Scharler (http://www.iamshadowlord.com)
Additional Credits:
Example sketches from Arduino team, Ethernet by Adrian McEwen
Added dht11/BMP180 showed dtostrf function by diy_bloke 22/11/2014
*/
#include <SPI.h>
#include <Ethernet.h>
#include <dht11.h>
#include <Wire.h>
#include <Adafruit_BMP085.h> // This is the version 1 library
#define DHT11PIN 4  // The Temperature/Humidity sensor
Adafruit_BMP085 bmp;
dht11 DHT11;

// Local Network Settings
byte mac[] = { 0xD4, 0x28, 0xB2, 0xFF, 0xA0, 0xA1 }; // Must be unique on local network
// ThingSpeak Settings
char thingSpeakAddress[] = "api.thingspeak.com";
String writeAPIKey = "REPLACE_THIS_BY_YOUR_API_BUT_KEEP_THE_QUOTES";
const int updateThingSpeakInterval = 16 * 1000; // Time interval in milliseconds to update ThingSpeak (number of seconds * 1000 = interval)
// Variable Setup
long lastConnectionTime = 0;
boolean lastConnected = false;
int failedCounter = 0;
// Initialize Arduino Ethernet Client
EthernetClient client;
void setup()
{
// Start Serial for debugging on the Serial Monitor
Serial.begin(9600);
// Start Ethernet on Arduino
startEthernet();
}
void loop()
{
// Read value from Analog Input Pin 0
String analogPin0 = String(analogRead(A0), DEC);
// Print Update Response to Serial Monitor
if (client.available())
{
char c = client.read();
Serial.print(c);
}
//------DHT11--------
int chk = DHT11.read(DHT11PIN);
char t_buffer[10];
char h_buffer[10];
float t=(DHT11.temperature);
String temp=dtostrf(t,0,5,t_buffer);
//Serial.print(temp);
//Serial.print(" ");
float h=(DHT11.humidity);
String humid=dtostrf(h,0,5,h_buffer);
//Serial.println(humid);

//-----BMP180-----------
bmp.begin();
            float p=(bmp.readPressure()/100.0);//this is for pressure in hectoPascal
            float m=(bmp.readPressure()/133.3);//  this is for pressure in mmHG
            float t2=(bmp.readTemperature());
            char p_buffer[15];
            char t2_buffer[10];
            String pres=dtostrf(p,0,5,p_buffer);
            String temp2=dtostrf(t2,0,5,t2_buffer);
            Serial.println(pres);
 //         }
//----------------

// Disconnect from ThingSpeak



if (!client.connected() && lastConnected)
{
Serial.println("...disconnected");
Serial.println();
client.stop();
}
// Update ThingSpeak
if(!client.connected() && (millis() - lastConnectionTime > updateThingSpeakInterval))
{
updateThingSpeak("field1="+temp+"&field2="+humid+"&field3="+pres+"&field4="+temp2);
}
// Check if Arduino Ethernet needs to be restarted
if (failedCounter > 3 ) {startEthernet();}
lastConnected = client.connected();
}
void updateThingSpeak(String tsData)
{
if (client.connect(thingSpeakAddress, 80))
{
client.print("POST /update HTTP/1.1\n");
client.print("Host: api.thingspeak.com\n");
client.print("Connection: close\n");
client.print("X-THINGSPEAKAPIKEY: "+writeAPIKey+"\n");
client.print("Content-Type: application/x-www-form-urlencoded\n");
client.print("Content-Length: ");
client.print(tsData.length());
client.print("\n\n");
client.print(tsData);
lastConnectionTime = millis();
if (client.connected())
{
Serial.println("Connecting to ThingSpeak...");
Serial.println();
failedCounter = 0;
}
else
{
failedCounter++;
Serial.println("Connection to ThingSpeak failed ("+String(failedCounter, DEC)+")");
Serial.println();
}
}
else
{
failedCounter++;
Serial.println("Connection to ThingSpeak Failed ("+String(failedCounter, DEC)+")");
Serial.println();
lastConnectionTime = millis();
}
}
void startEthernet()
{
client.stop();
Serial.println("Connecting Arduino to network...");
Serial.println();
delay(1000);
// Connect to network amd obtain an IP address using DHCP
if (Ethernet.begin(mac) == 0)
{
Serial.println("DHCP Failed, reset Arduino to try again");
Serial.println();
}
else
{
Serial.println("Arduino connected to network using DHCP");
Serial.println();
}
delay(1000);
}

Using an Ethercard/Ethershield

In case you do not have a WS5100 baset ethernetshield but an ENC28J60 based Ethershield or Ethercard, use the program below:

// Simple demo for feeding some random data to Pachube.
// 2011-07-08 <jc@wippler.nl> http://opensource.org/licenses/mit-license.php

// Handle returning code and reset ethernet module if needed
// 2013-10-22 hneiraf@gmail.com

// Modifing so that it works on my setup for www.thingspeak.com.
// Arduino pro-mini 5V/16MHz, ETH modul on SPI with CS on pin 10.
// Also added a few changes found on various forums. Do not know what the 
// res variable is for, tweaked it so it works faster for my application
// 2015-11-09 dani.lomajhenic@gmail.com

// added dht11/bmp108. tweaked it to work with new IDE
// 2016-06-01  june 1, 2016

#include  <Ethercard.h>//Ethercard.h
#include   <Wire.h>//Wire.h
#include  <Adafruit_BMP085.h>//Adafruit_BMP085.h
#include  <dht11.h>//dht11.h
#define DHT11PIN 2
Adafruit_BMP085 bmp;
dht11 DHT11;
#define APIKEY "QTRR45T7UVW44" // put your key here
#define ethCSpin 10 // put your CS/SS pin here.
// ethernet interface mac address, must be unique on the LAN
static byte mymac[] = { 0x75,0x68,0x68,0x68,0x68,0x68 };
const char website[] PROGMEM = "api.thingspeak.com";
byte Ethernet::buffer[700];
uint32_t timer;
Stash stash;
byte session;
//timing variable
int res = 100; // was 0

void setup () {
  Serial.begin(9600);
  Serial.println("\n[ThingSpeak example]");

  //Initialize Ethernet
  initialize_ethernet();
}

void loop () { 
  //------DHT11--------
int chk = DHT11.read(DHT11PIN);
int t=(DHT11.temperature);
int h=(DHT11.humidity);

//-----BMP180-----------
bmp.begin();
            float p=(bmp.readPressure()/100.0);//this is for pressure in hectoPascal
            float m=(bmp.readPressure()/133.3);//  this is for pressure in mmHG
            float t2=(bmp.readTemperature());
          
 //------ENC28J60----------
  //if correct answer is not received then re-initialize ethernet module
  if (res > 220){
    initialize_ethernet(); 
  }
  
  res = res + 1;
  
  ether.packetLoop(ether.packetReceive());
  
  //200 res = 10 seconds (50ms each res)
  if (res == 200) {

    
    // field1=(Field 1 Data)&field2=(Field 2 Data)&field3=(Field 3 Data)&field4=(Field 4 Data)&field5=(Field 5 Data)&field6=(Field 6 Data)&field7=(Field 7 Data)&field8=(Field 8 Data)&lat=(Latitude in Decimal Degrees)&long=(Longitude in Decimal Degrees)&elevation=(Elevation in meters)&status=(140 Character Message)
    byte sd = stash.create();
    stash.print("field1=");
    stash.print(t);
    stash.print("&field2=");
    stash.print(h);
    stash.print("&field3=");
    stash.print(p);
    stash.print("&field4=");
    stash.print(t2);
    stash.print("&field5=");
    stash.print(t);
    stash.print("&field6=");
    stash.print(h);
    stash.print("&field7=");
    stash.print(p);
    stash.print("&field8=");
    stash.print(t2);
    stash.save();

      // generate the header with payload - note that the stash size is used,
    // and that a "stash descriptor" is passed in as argument using "$H"
    Stash::prepare(PSTR("POST /update HTTP/1.0" "\r\n"
      "Host: $F" "\r\n"
      "Connection: close" "\r\n"
      "X-THINGSPEAKAPIKEY: $F" "\r\n"
      "Content-Type: application/x-www-form-urlencoded" "\r\n"
      "Content-Length: $D" "\r\n"
      "\r\n"
      "$H"),
    website, PSTR(APIKEY), stash.size(), sd);

    // send the packet - this also releases all stash buffers once done
    session = ether.tcpSend(); 

 // added from: http://jeelabs.net/boards/7/topics/2241
 int freeCount = stash.freeCount();
    if (freeCount <= 3) { Stash::initMap(56); } } const char* reply = ether.tcpReply(session); if (reply != 0) { res = 0; // Serial.println(F(" >>>REPLY recieved...."));
    // Serial.println(reply);
   }
   delay(300);
}

void initialize_ethernet(void){  
  for(;;){ // keep trying until you succeed 
    //Reinitialize ethernet module
    //Serial.println("Reseting Ethernet...");
    //digitalWrite(5, LOW);
    //delay(1000);
    //digitalWrite(5, HIGH);
    //delay(500);

    if (ether.begin(sizeof Ethernet::buffer, mymac, ethCSpin) == 0){ 
      Serial.println( F("Failed to access Ethernet controller"));
      continue;
    }
    
    if (!ether.dhcpSetup()){
      Serial.println(F("DHCP failed"));
      continue;
    }

    ether.printIp("IP:  ", ether.myip);
    ether.printIp("GW:  ", ether.gwip);  
    ether.printIp("DNS: ", ether.dnsip);  

    if (!ether.dnsLookup(website))
      Serial.println(F("DNS failed"));

    ether.printIp("SRV: ", ether.hisip);

    //reset init value
    res = 180;
    break;
  }
}

Using a ESP8266

Edit: This ofcourse is now a totally obsolete method. One would just use the ESP8266 by itself -without an Arduino- to send data to Thingspeak. I leave it here for ‘historical purposes’

The previous presented internet connection was made via a cable. However, there is a cheap WiFi module that is available to attach to the Arduino: The ESP 8266. Bear in mind that it needs 3.3 Volt. Some models however claim to be 5 Volt tolerant. I added 2 circuits that could be used as voltage converter. The ESP8266 really needs its own 3.3 V source as the current coming from the Arduino just doesn’t cut it. Also, if you have any problems with it, Connect a decoupling/buffer capacitor  (50uF) between ground and Vcc close to the board There is plenty of information on how to connect the module, I want to focus on the software to make a Thingspeak connection The SoftSerial library is added for debugging. it is not really necessary once the program is working. The same goes for all the print statements to the Software serial port I use 3 analogue values rather than an example with the BMP108 and DHT11 as that requires libraries, so the implementation of theESP8266 is easier to follow. Once you get that, it is easy to implement other sensors. Just make sure you turn all output into a string Below a program to Connect 3 analogue sensors to Thingspeak via an ESP8166 module:

The program

// <a href="https://nurdspace.nl/ESP8266" rel="nofollow"> https://nurdspace.nl/ESP8266
</a>//http://www.instructables.com/id/Using-the-ESP8266-module/
//https://www.zybuluo.com/kfihihc/note/31135
//http://tminusarduino.blogspot.nl/2014/09/experimenting-with-esp8266-5-wifi-module.html
//http://www.cse.dmu.ac.uk/~sexton/ESP8266/
//https://github.com/aabella/ESP8266-Arduino-library/blob/master/ESP8266abella/ESP8266aabella.h
//http://contractorwolf.com/esp8266-wifi-arduino-micro/
//**********************************************************
#include <SoftwareSerial.h>
int sensor_temp = A0;
int value_temp;
int sensor_light = A1;
int value_light;
int sensor_humid = A2;
int value_humid;


#define DEBUG FALSE //comment out to remove debug msgs

//*-- Hardware Serial
#define _baudrate 9600

//*-- Software Serial
//
#define _rxpin      2
#define _txpin      3
SoftwareSerial debug( _rxpin, _txpin ); // RX, TX

//*-- IoT Information
#define SSID "[YOURSSID]"
#define PASS "[YOURPASSWORD]"
#define IP "184.106.153.149" // ThingSpeak IP Address: 184.106.153.149

// GET /update?key=[THINGSPEAK_KEY]&field1=[data 1]&filed2=[data 2]...;
String GET = "GET /update?key=[ThingSpeak_(Write)API_KEY]";

void setup() {
   Serial.begin( _baudrate );
  debug.begin( _baudrate );

  sendDebug("AT");
  delay(5000);
  if(Serial.find("OK"))
  {
    debug.println("RECEIVED: OK\nData ready to sent!");
    connectWiFi();
  }

}

void loop() {
  value_temp = analogRead(sensor_temp);
  value_light = analogRead(sensor_light);
  value_humid = analogRead(sensor_humid);
  String temp =String(value_temp);// turn integer to string
  String light= String(value_light);// turn integer to string
  String humid=String(value_humid);// turn integer to string
  updateTS(temp,light, humid);
  delay(3000); //
}
//----- update the  Thingspeak string with 3 values
void updateTS( String T, String L , String H)
{
  // ESP8266 Client
  String cmd = "AT+CIPSTART=\"TCP\",\"";// Setup TCP connection
  cmd += IP;
  cmd += "\",80";
  sendDebug(cmd);
  delay(2000);
  if( Serial.find( "Error" ) )
  {
    debug.print( "RECEIVED: Error\nExit1" );
    return;
  }

  cmd = GET + "&field1=" + T +"&field2="+ L + "&field3=" + H +"\r\n";
  Serial.print( "AT+CIPSEND=" );
  Serial.println( cmd.length() );
  if(Serial.find( ">" ) )
  {
    debug.print(">");
    debug.print(cmd);
    Serial.print(cmd);
  }
  else
  {
    sendDebug( "AT+CIPCLOSE" );//close TCP connection
  }
  if( Serial.find("OK") )
  {
    debug.println( "RECEIVED: OK" );
  }
  else
  {
    debug.println( "RECEIVED: Error\nExit2" );
  }
}

void sendDebug(String cmd)
{
  debug.print("SEND: ");
  debug.println(cmd);
  Serial.println(cmd);
}

boolean connectWiFi()
{
  Serial.println("AT+CWMODE=1");//WiFi STA mode - if '3' it is both client and AP
  delay(2000);
  //Connect to Router with AT+CWJAP="SSID","Password";
  // Check if connected with AT+CWJAP?
  String cmd="AT+CWJAP=\""; // Join accespoint
  cmd+=SSID;
  cmd+="\",\"";
  cmd+=PASS;
  cmd+="\"";
  sendDebug(cmd);
  delay(5000);
  if(Serial.find("OK"))
  {
    debug.println("RECEIVED: OK");
    return true;
  }
  else
  {
    debug.println("RECEIVED: Error");
    return false;
  }

  cmd = "AT+CIPMUX=0";// Set Single connection
  sendDebug( cmd );
  if( Serial.find( "Error") )
  {
    debug.print( "RECEIVED: Error" );
    return false;
  }
}