measure heating oil usage with a Vl53L0X time of flight sensor and display on an energy dashboard

how to make a heating oil tank level sensor that’s more useful than anything you can buy.

This sensor…

• can measure oil level in any unit – litres, kWh, gallons or days until I need to refill the tank
• displays readings on a time graph (uses Home Assistant history graph).
• receive readings or alerts as an email or phone notification or flashing smart bulb in the house.
• help calculate the date when you’ll need to order or tell you how much free space there is in the tank
• show your oil usage on the Home Assistant wnergy dashboard
• uses an infrared based Vl53L0X time of flight sensor (~£5 ebay)
• uses any ESP32 or ESP8266 development board with wifi. I used a nodemcu v3 ESP8266 (~ £6).
• could work outdoors and on battery power with a waterproof electrician’s housing. If you need to use battery power an ESP8266 might be more appropriate. The battery power might be 6 x 18650 cells based on this power pack I made
• this sensor could instead monitor the salt level in a water softener or the water level in a tank (ref Dr Zzz – youtube)

Background: homes like mine use diesel oil as a fuel to keep warm in winter. Over the course of a winter I might need two or three deliveries of oil but I’ve no way to predict when I’ll need more oil or know how much to order. And I really don’t ever want to run out of oil. The oil is burned in a heating boiler in the same way that many homes in the UK burn methane gas. Although we call the liquid ‘heating oil’ it’s more runny than lubricating oil. You may call it diesel or white spirits or turps or paraffin or kerosine.

I asked local firm Shelford Heating to quote a price for an oil level sensor. Their solution involved drilling the steel tank to fit an Apollo wireless kit – but £220 seemed excessive for this need. I could also have bought the kit, which sends an approximate oil level to a display but I really wanted something that could alert me better.

what’s needed for this project

• Vl53L0X time of flight sensor
• coloured Dupont wires or coloured 18 gauge stranded wires if the wires need to be long
• if your tank is outdoors you’ll need waterproofing (eg Kafuter K-705 gel plus a housing) and get a 5v supply to the oil tank
• an ESP8266 or ESP32 development board
• Home Assistant already set up on a Raspberry Pi
• heat shrink tubing and soldering skills if you want to make reliable connections (you do)
• if you want notifications you need your email account server settings (optional)
• if you want to keep long term records you’d need to send the data to Google Sheets (optional)

there are other posts on this topic

• an earlier oil tank level sensor using the ultrasonic JSN-SR04T
• how I mounted and calibrated the sensor for my oil tank
• the final step where I get Home Assistant to calculate when I need to order oil and display that
• the trick to send sensor data from Home Assistant to a spreadsheet

how to connect up an oil level sensor to a ESP32 / ESP8266 development board

This is the current version of my original oil tank level sensor using the ultrasonic JSN-SR04T that has been working reliably for years. It runs in parallel with the older sensor. The limitation of the older ultrasonic sensor is that it cannot measure at short range so it records the same oil level whether the tall is full or 200mm below full. When the level drops by 200mm the readings are still very useful at alerting me to order more oil.

The Vl53L0X time of flight sensor is an inexpensive alternative that works by sending infrared blips to the liquid surface and then measuring the ‘time of flight’ to estimate how far away it is. It can send correct values at short range ie even if the tank is full. You can find versions of this sensor (VL53L0X, TOF10120, VL53L1X, VL6180X) which work best at different distances (eg below). HOWEVER, do check that ESPHome supports it. I chose the Vl53L0X because it’s supported and made for the range 30mm-1200m as suits my tank. (A post at instructables states that the VL53L1X is a better and more expensive sensor – but one year on I’m very happy with the project as follows. But check: is the sensor supported by your software).

The ESPhome software we use can also set the sensor to a long distance remote range of 1500 -2000mm. Wonderfully, when the sensor is hooked up it sends a distance value (in metres) and I can multiply that by a number to get readings in mm or litres or kW or even money. The calculation is simpler if your oil tank has a regular shape but anything is doable.

The Vl53L0X module has six pin connectors – you only need to use four of them. I used Dupont male to female wires because my ESP8266 nodemcu already had pins. I soldered the male pins to the Vl53L0X module and extended the wires with coloured cables (silicone 18 AWG 0.75mm) and covered the join in heat-shrink. GND and VIN connect to GND and 3.3v on an ESP8266 module. Connect SCL to D1 (or GPI05). Connect SDA to D2 (or GPI04). For connections to a ESP32 see the image below

a housing to protect the VL53L0X oil level sensor and ESP

This is where my system will surely be different. A 3D printed part holds the sensor in place with the help of some hot glue. I don’t yet know if the vapour from the oil will damage the VL53L0X so I’m looking to improve the housing.

Getting dirt in the tiny emitter and sensor holes is to be avoided as the emerging pulse or photon can be deflected randomly / cause crosstalk. So I cut a 30mm circle of acetate film (from a phone case) and glued it to the cap I’d made for the sensor. I might have used a piece of phone screen protector and it would wise to test if the cover affects the distance reading. I cover this in my post about mounting and calibration here.

You can protect the sensor with glass, perspex or polycarbonate. The cover should be as thin, flat, parallel, clear and fitted close to the sensor as possible. ST, the chip maker have a video explaining how a more fussy two-piece glass cover pretty much reduces the crosstalk due to dust. I so wish they’ll create a similarly protected chip. The sensor notes are here.

The sensor needs to be fixed so that it points down at the surface of the oil in the tank (photo). I used a tank hole that had housed the dipstick. A suitable cap, as shown below, was made on a 3D printer. A container cap might have worked … as long as it doesn’t wobble and thus affect the readings. You might instead drill a hole for it but I’ve no advice on this.

how to write the program and install it on the ESP 32 / ESP8266 development board

My oil sensor is a ‘connected’ oil sensor. It takes readings, sends them to Home Assistant which will present them in various ways. To gain this connectivity you need a working Home Assistant installation on a Raspberry Pi (shown in an earlier project).

What you do is to assemble a series of instructions (a program) and get the instructions turned into a file (=compiled into a ‘bin file’) that you upload to a ESP dev board via a USB cable. The file is otherwise called firmware. The process of installing the file to the ESP is called flashing the firmware although what you’re doing is akin to installing the program.

• use your browser to interact with Home Assistant on 192.168.x.x:8123
• in Home Assistant go to the Esphome add-on and add a ‘new device’. Follow the process eg it’ll ask you for a name (eg oil level) and you may be asked to fill in wifi details.
• Still in Esphome look for ‘Edit’ next to your new device. Add the following code making changes to suit your setup.
• choose Install to upload this code to your development board.

# THIS IS FOR AN ESP8266 development board
# see final version further down this page if your trace is noisy
# we produce two entities, one for litres one for kWh
esphome:
  name: oil-tof
substitutions:
  devicename: oil-tof
esp8266:
  board: nodemcuv2
# enable logging
logger:
# enable Home Assistant API
api:
  encryption: 
    key: "FtKhT5Sf4cwe5bh/N0Kzy6JMM5LG79H38rfceMlBHrE"
ota:
  password: ""
wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  manual_ip:
    # set this to the IP of the ESP
    static_ip: 192.168.1.15
    # set this to the IP address of the router
    gateway: 192.168.1.1
    # the subnet of the network
    subnet: 255.255.255.0
  # hotspot in case wifi connection fails
  ap:
    ssid: "oil-tof fail hotspot"
    password: !secret wifi_password
captive_portal:

i2c:
  sda: D2 #GPI04 orange or green (data) pin 4
  scl: D1 #GPI05 yellow or blue (clock) pin 5
  scan: true

# use this sensor section - it works but to remove noise 
# see the complicated version further down this page

sensor:
  - platform: vl53l0x
    name: "oil kW"
    address: 0x29
    id: myoillevel
    update_interval: 10s
    long_range: false
    accuracy_decimals: 0
    filters:
    - lambda: return (x - 0.034) * 1000 * 10;   
    state_class: total
    device_class: energy
    unit_of_measurement: "kWh"

# post comments below explain my calibration
# return (x - 0.034) * 1000 * 10 - you could instead use
# return x * 1000 * 10 to get readings as kWh



  - platform: template
    name: "oil_level"
#    update_interval: 60s
    accuracy_decimals: 0
    unit_of_measurement: "L"
    lambda: |-
      return -0.1 * id(myoillevel).state ;  
# negative to get a decreasing oil level 
 
switch:
  - platform: restart
    name: ${devicename} restart   

---

# THIS IS FOR AN ESP32 development board
# see final version further down this page if your trace is noisy
# we produce two entities, one for litres one for kWh

esphome:
  name: oil-tof

substitutions:
  devicename: oil-tof

esp32:
  board: esp32dev
  framework:
    type: arduino

# enable logging
logger:
# enable Home Assistant API
api:
  encryption: 
    key: "FtKhT5Sf4cwe5bh/N0Kzy6JMM5LG79H38rfceMlBHrE"
ota:
  password: ""
wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  manual_ip:
    # set this to the IP of the ESP
    static_ip: 192.168.1.15
    # set this to the IP address of the router
    gateway: 192.168.1.1
    # the subnet of the network
    subnet: 255.255.255.0
  # Enable hotspot if case wifi connection fails
  ap:
    ssid: "oil-tof fail hotspot"
    password: !secret wifi_password
captive_portal:

i2c:
  sda: 21 # orange or green
  scl: 22 # yellow or blue
  scan: true

# use this sensor section - it works but to remove noise 
# see the complicated version further down this page

sensor:
  - platform: vl53l0x
    name: "oil kW"
    address: 0x29
    id: myoillevel
    update_interval: 10s
    long_range: false
    accuracy_decimals: 0
    filters:
    - lambda: return (x - 0.034) * 1000 * 10;   
    state_class: total
    device_class: energy
    unit_of_measurement: "kWh"

#  use negative below to get negative oil level

  - platform: template
    name: "oil_level"
#    update_interval: 60s
    accuracy_decimals: 0
    unit_of_measurement: "L"
    lambda: |-
      return -0.1 * id(myoillevel).state ;  


# see user comments below for an explanation of 
# return (x - 0.034) * 1000 * 10 - you could instead use
# return x * 1000 * 10 to get readings as kWh
  
switch:
  - platform: restart
    name: ${devicename} restart 
 

add your sensor data to the Home Assistant overview

• when the code is installed, power-up the ESP board with a USB power source and go to Home Assistant > Integrations > + > ESPHome. You might have to enter the IP address you chose in the code. You’ll then see the oil level device that has been added to Home Assistant. The device will have an entity named like sensor.oil_level
• go to the Home Assistant Overview where your stuff is normally displayed. Choose Edit dashboard in the top corner.
• add a new card called history graph. On the entity line select your sensor.oil_level
• add a new card called gauge to display your oil level. On the entity line select your sensor.oil_level

*refine the oil level sensor code

handle a few issues:

After several months use I needed to address some issues. Because of these points I modified the code as shown below. You may want to substitute the above ‘sensor:’ section with the following ‘sensor:’ section (explained below)

• maxing out: at low oil levels the sensor is bouncing off a metal strut inside the tank so I need to relocate the sensor somewhere else. My readings plateau when the tank drops to 20% full.
• calibration: the sensor measures in metres so I have to calculate the litres. My tank has a perfectly regular shape from top to bottom – so by measuring dipstick depths, I found that 100cm of tank depth contains 105.3 litres. In the code below the last two lines (filter and lamda) do this calibration. The two lines can be edited or entirely omitted to suit your tank.
• the readings were a bit noisy. To counteract this I calculate the sliding average of an oil level reading. In my final version I replaced the sliding average with a Kalman calculation.
• it was suggested that the ‘oil used’ graph should fall, not rise, so I made the reading negative. This also makes it easier to know how much oil to order. I should rename the sensor ‘oil used’ because the number it returns is the amount of oil I need to buy to fill the tank. Next I really don’t need an oil level reading every 10 seconds so I increased the update interval. The commercially available FoxRadar takes only a few readings per day.

# add a filter to the sensor section to calibrate the sensor.
  
# add a negative sign to produce a falling oil level:
#   filters: 
#   - lambda: return x * -1053; 

# note: 1053 is 1.053 (fiddle factor for tank) * 1000 (convert to litres)

# add a sensor filter to remove noise:

 filters: 
  - lambda: return x * 1053;
  - sliding_window_moving_average: 
    window_size: 20 
    send_every: 10

# if your tank shape is irregular you can put your calibration 
# graph of distance and volume in the code. SEE THE POST
# FROM 'Kev' below where he improved the filter section so:

#  – calibrate_linear:
#  – 110 -> 0
#  – 95 -> 204
#  – 75 -> 544
#  – 55 -> 900
#  – 35 -> 1320
#  – 15 -> 1660
#  – 0 -> 1800

when do I need to order more oil?

The first nice consequence is that I now know how much oil I need to order and how much I use. We might now need the following:

• to predict the date when we need to order more oil and get notified – see how I calculate when I need to order more oil using Home Assistant
• to store our oil use in a spreadsheet as well as record the outside temperature – see send sensor data from Home Assistant to a spreadsheet

the trace is very noisy – how to remove noise from the trace

The code below includes ways to remove noise from the readings. The quantile filter removes ‘outliers’ (the readings that are wild). Having removed the outliers we use a kalman calculation to smooth the graph. The results are good however occasionally the oil kWh wavers, goes negative and makes a mess of the Energy Dashboard. We therefore create a sensor that doesn’t go negative. This code produces four entities – only two are needed.

• “oil kw-raw” – the raw reading from the sensor as kWh. Includes a tank fiddle factor (1000) to turn mm into litres
• “oil kalman” – we do a ‘Kalman’ calculation on the raw kW to smooth it
• “oil_level” – we take the smoothed “oil kalman” and convert kWh to litres by dividing it by 10
• “oil_kalkw” – we take the smoothed “oil kalman” and prevent it from producing a lower value than the last reading.

IN ESPHOME - revised code for ESP32 and ESP8266 December 2023
# replace the entire sensor section and add globals 

sensor:
  - platform: vl53l0x
    name: "oil kw-raw"
    address: 0x29
    id: myoilkw # UNFILTEREDKW
    update_interval: 10s
    long_range: false
    accuracy_decimals: 0
    unit_of_measurement: "kWh"
    state_class: total
    device_class: energy
    filters:
    - lambda: return x * 1000 * 10;
#   - lambda: return x * your-tank-factor * kW/L

  - platform: template
    name: "oil_level"
    id: myoillevel # unFILTEREDL
    accuracy_decimals: 0
    unit_of_measurement: "L"
    state_class: total
    lambda: |-
      return -0.1 * id(mykalmankw).state ;

**************** REPLACE THIS ************
  - platform: kalman_combinator
    name: "oil kalman"
    id: mykalmankw
    accuracy_decimals: 0
    unit_of_measurement: "kWh"
    state_class: total
    device_class: energy
    process_std_dev: 0.001
    sources:
      - source: myoilkw # UNFILTERED
        error: 1
**************** WITH THIS *********************
  - platform: combination
    type: kalman
    name: "oil kalman"
    id: mykalmankw
    accuracy_decimals: 0
    unit_of_measurement: "kWh"
    state_class: total
    device_class: energy
    process_std_dev: 0.001
    sources:
       - source: myoilkw # UNFILTERED
         error: 1
***********************************************

  - platform: copy
    source_id: mykalmankw
    name: "oil_kalkw"
    accuracy_decimals: 0
    # filters: to reject a lower value
    filters:
      - skip_initial: 50
      - quantile:
          window_size: 6
          send_every: 6
          send_first_at: 6
          quantile: 0.95
      - lambda: |-
          if (abs(x < id(last_good_value))) return {};
          id(last_good_value) = x;
          return x;


globals:
- id: last_good_value
  type: int
  restore_value: no
  initial_value: '100'
- id: counter
  type: int
  restore_value: no
  initial_value: '0'

# add the code above in place of the sensor code earlier. 

edit the Home Assistant energy dashboard

Go to Settings > Dashboards > Gas Consumption > Add Gas Source and find the entity for oil usage in kWh. In my example above this would be “oil_kalkw” or “oil kalman”. To track the costs enter a number e.g. 0.085 GPB/kWh or create an input number (in Helpers) such as input_number.oil_now_cost with a value of 0.085.

Example: if heating oil costs £0.85 per litre which ideally burns to produce 9 kWh of energy. For simpler maths I call this £0.85 for 10 kWh or £0.085 for 1 kWh. You could tweak this to allow for inefficient burning. If I had a home that didn’t use and leak so much energy I might see the value of pretending to be accurate.

If you’re more interested in water tank levels not oil tank levels: