HTTP-OTA for Self-updating the Filing system

HTTP OTA of the firmware for an ESP8266 is quite common and it is even possible to have the ESP8266 do that automatically when necessary (self-updating), by updating a version number on the upload server.
It is less known that this is also possible for the file system, which maybe necessary if you have files there, such as an html or JS file, or images that need updating. The below code shows how to do that.

#include <ESP8266WiFi.h>
#include <ESP8266HTTPClient.h>
#include <FS.h>
#include <LittleFS.h> // Use SPIFFS.h if you're using SPIFFS

const char* ssid = "your_SSID";
const char* password = "your_PASSWORD";
const char* fileUrl = "http://your-server.com/filesystem.bin"; // URL of the filesystem image
const char* versionUrl = "http://your-server.com/version.txt"; // URL of the version file
const char* currentVersionFile = "/version.txt"; // File to store the current version

void setup() {
  Serial.begin(115200);
  WiFi.begin(ssid, password);
  
  while (WiFi.status() != WL_CONNECTED) {
    delay(1000);
    Serial.println("Connecting to WiFi...");
  }
  Serial.println("Connected to WiFi");

  // Initialize LittleFS
  if (!LittleFS.begin()) {
    Serial.println("Failed to mount file system");
    return;
  }

  // Check for updates
  if (isUpdateAvailable()) {
    if (updateFilesystem()) {
      Serial.println("Filesystem updated successfully");
    } else {
      Serial.println("Failed to update filesystem");
    }
  } else {
    Serial.println("No update available");
  }
}

bool isUpdateAvailable() {
  String currentVersion = readCurrentVersion();
  String newVersion = fetchServerVersion();

  Serial.println("Current version: " + currentVersion);
  Serial.println("Server version: " + newVersion);

  return newVersion > currentVersion;
}

String readCurrentVersion() {
  if (!LittleFS.exists(currentVersionFile)) {
    return "0.0.0"; // Default version if no version file is found
  }

  File file = LittleFS.open(currentVersionFile, "r");
  if (!file) {
    Serial.println("Failed to open version file");
    return "0.0.0";
  }

  String version = file.readStringUntil('\n');
  file.close();
  return version;
}

String fetchServerVersion() {
  HTTPClient http;
  http.begin(versionUrl);
  int httpCode = http.GET();

  if (httpCode == HTTP_CODE_OK) {
    String version = http.getString();
    version.trim();
    return version;
  } else {
    Serial.printf("Failed to fetch server version, error: %s\n", http.errorToString(httpCode).c_str());
    return "0.0.0";
  }
}

bool updateFilesystem() {
  HTTPClient http;
  http.begin(fileUrl);
  int httpCode = http.GET();

  if (httpCode == HTTP_CODE_OK) {
    WiFiClient* stream = http.getStreamPtr();
    if (!LittleFS.format()) {
      Serial.println("Failed to format filesystem");
      return false;
    }
    
    File file = LittleFS.open("/filesystem.bin", "w");
    if (!file) {
      Serial.println("Failed to open file for writing");
      return false;
    }

    uint8_t buffer[128] = { 0 };
    int len = stream->available();
    while (len > 0) {
      int bytesRead = stream->readBytes(buffer, sizeof(buffer));
      file.write(buffer, bytesRead);
      len -= bytesRead;
    }
    file.close();

    // Update the current version file
    updateCurrentVersion(fetchServerVersion());

    return true;
  } else {
    Serial.printf("HTTP request failed, error: %s\n", http.errorToString(httpCode).c_str());
    return false;
  }
}

void updateCurrentVersion(String version) {
  File file = LittleFS.open(currentVersionFile, "w");
  if (!file) {
    Serial.println("Failed to open version file for writing");
    return;
  }

  file.println(version);
  file.close();
}

void loop() {
  // Your code here
}

Explanation:

  1. Version Check:
    • isUpdateAvailable() function compares the current version with the server version.
    • readCurrentVersion() reads the current version from the filesystem.
    • fetchServerVersion() downloads the version file from the server.
  2. Filesystem Update:
    • The update is only performed if the server version is newer.
    • updateFilesystem() handles the downloading and writing of the new filesystem image.
    • updateCurrentVersion() updates the current version file after a successful update.

in order to get a ‘filesystem.bin’. You go to ‘Tools-ESP8266LittleFSDataUpload’and click that. You will get the following in your monitor:

That tells you where you can find the filesystem bin file. Copy that file to your uploadserver and give it the name your firmware expects (in this example that is “filesystem.bin”).
Make sure your filesystem contained a file ‘version.txt’ with the new version number. after you put the bin file on your upload server. update the version number in the ‘version.txt’ file that is on your upload server.
In case you are confused: your update server should have 2 files:

  • “version.bin”- this is your compiled filesystem that also should have a (compiled) ‘version.text’ in it with a number of the new version.
  • “version.txt” – a simple text file that contains the new version

Time info from a remote ESP8266 (thus not connected to serial port)

Recently I have been working on some deepsleep projects and I needed to get some feedback on the actual sleeptime compared to the programmed sleeptime. As this was in an ESP8266 that was not connected to any terminal, i needed another way to check on how long the sleeptime actually was. Now there are various ways to do it: I could for instance send a value to thingspeak on waking up and see the time difference between two values, but that would require adding quite some code that I did not really need.
As, my project on wake up does already a check on possible new software for an OTA, it already makes an HTTP call, so i just wanted to add another http call that just stores the time. so I simply added:

httpClient.begin(client,"http://192.168.1.128/store_time.php"); // Specify the URL
int httpResponseCode = httpClient.GET(); // Make the GET request

and on my server I added a piece of php script:

<?php
// File path to store the CSV
define('CSV_FILE_PATH', 'timestamps.csv');
// Function to capture the current time and store it in a CSV file
function storeCurrentTimeInCSV() {
    // Capture the current date and time
    $currentTime = date('Y-m-d H:i:s');
    
    // Open the CSV file in append mode
    $fileHandle = fopen(CSV_FILE_PATH, 'a');
    
    // Check if the file is writable
    if ($fileHandle !== false) {
        // Write the current time to the CSV file
        fputcsv($fileHandle, [$currentTime]);
        
        // Close the file
        fclose($fileHandle);
        
        // Return a success response
        return json_encode(['status' => 'success', 'message' => 'Time stored successfully']);
    } else {
        // Return an error response if the file is not writable
        return json_encode(['status' => 'error', 'message' => 'Unable to write to the file']);
    }
}
// Call the function and output the result as a JSON response
header('Content-Type: application/json');
echo storeCurrentTimeInCSV();
?>

So, now every time my ESP8266 wakes up, before it checks for updates, it first makes a call to the “store_time.php” routine. the results look like this:

"2024-05-31 22:59:34"
"2024-05-31 23:32:10"
"2024-06-01 00:05:51"
"2024-06-01 00:39:30"
"2024-06-01 01:13:16"
"2024-06-01 01:47:01"
"2024-06-01 02:20:43"
"2024-06-01 02:54:29"
"2024-06-01 03:28:09"
"2024-06-01 04:01:43"
"2024-06-01 04:35:26"
"2024-06-01 05:09:04"
"2024-06-01 05:42:47"
"2024-06-01 06:16:31"
"2024-06-01 06:50:17"
"2024-06-01 07:23:57"
"2024-06-01 07:57:35"
"2024-06-01 08:31:17"
"2024-06-01 09:05:11"
"2024-06-01 09:38:58"
"2024-06-01 09:39:45"

Anyway, it allows me to try several different settings and once I am happy with what i have it is a matter of commenting out 2 lines and put that new program ready for an automated OTA