heating oil tank level – Vl53L0X time of flight sensor version

make a heating oil tank level sensor that’s more useful than anything you can buy

• can measure oil level in any unit – litres, kWh, gallons or days until I need to refill the tank
• displays readings on a gauge or time graph (uses Home Assistant gauge & history graph).
• receive readings or alerts as an email or phone notification or flashing smart bulb in the house.
• uses an infrared based Vl53L0X time of flight sensor (~£5 ebay)
• uses any ESP32 or ESP8266 development board with wifi. I used a nodemcu v3 ESP8266 (~ £6).
• with a waterproof electrician’s housing this could work outdoors and on battery power. if you need to use battery power an ESP8266 might be more appropriate. The battery power might be 6 x 18650 cells based on this one I made
• with a little modification this project would monitor the salt level in a water softener or the water in a water barrel (ref Dr Zzz – youtube)

Some homes use oil as a fuel to keep warm in winter. Over the course of a winter I might need to order two or three deliveries of oil but till now I’ve no way to predict the day when I’ll need to top it up or know how much to order. And I really don’t ever want to run out of oil. The oil is burned in a central heating boiler in the same way that many homes in the UK burn methane gas. Although we call the liquid ‘heating oil’ it’s more runny than lubricating oil. It’s more like diesel or white spirits or turps or paraffin or kerosine.

I asked local firm Shelford Heating to quote a price for an oil level sensor. Their solution involved drilling the steel tank to fit an Apollo wireless kit – but £220 seemed excessive for this need. I could also have bought the kit, which sends an approximate oil level to a display but I really wanted something that could alert me better.

how to connect up an oil level sensor to a ESP development board

This is a newer version of my oil tank level sensor using the ultrasonic JSN-SR04T that has been working reliably for years. I have lived with the limitation that the ultrasonic sensor cannot measure at short range so it records the same oil level whether the tall is full or 200mm below full. Thereafter the readings are very useful at alerting me to order more oil.

The Vl53L0X time of flight sensor is an inexpensive alternative that works by sending infrared blips to the liquid surface and then measuring the ‘time of flight’ to estimate how far away it is. It can send correct values even if the tank is full. You can find variations of this sensor (VL53L0X, TOF10120, VL53L1X, VL6180X) which work best at different distances (eg below). I chose the Vl53L0X because it’s made for the range 30mm-1200m as suits my tank. (A project listed at instructables states that the VL53L1X is a much better and more expensive sensor – I may switch to this if and when my sensor fails). The software can also set it in a long distance remote range of 1500 -2000mm. Wonderfully, when the sensor is hooked up it sends a distance value (in metres) and I can multiply that by a number to get readings in mm or litres or kW or even money. The calculation is simple if your oil tank has a regular shape but anything is doable.

The Vl53L0X module has six pin connectors – you only need to use four of them. I use Dupont male to female wires because my ESP8266 nodemcu already had pins. I soldered the male pins to the Vl53L0X module and extended the wires with coloured cables (silicone 18 AWG 0.75mm) and covered the join in heat-shrink. GND and VIN connect to GND and 3.3v on an ESP module. Connect SCL to D1 (or GPI05). Connect SDA to D2 (or GPI04).

a housing to protect the VL53L0X oil level sensor and ESP

This is where my system will be different to others. My 3D printed part holds the sensor in place with the help of some hot glue. I don’t yet know if the vapour from the oil will damage the VL53L0X so I’m looking to improve the housing.

Getting dirt in the tiny emitter and sensor holes is to be avoided as the emerging pulse or photon can be deflected randomly / cause crosstalk. So I cut a 30mm circle of acetate film (from a phone case) and glued it to the cap I’d made for the sensor. I might have used a piece of phone screen protector and it would wise to test if the cover affects the distance reading. I cover this in my post about mounting and calibration here.

You can protect the sensor with glass, perspex or polycarbonate. The cover should be as thin, flat, parallel, clear and fitted close to the sensor as possible. ST, the chip maker have a video explaining how a more fussy two-piece glass cover pretty much reduces the crosstalk due to dust. I so wish they’ll create a similarly protected chip. The sensor notes are here.

The sensor needs to be fixed so that it points down at the surface of the oil in the tank (photo). I used a tank hole that had housed the dipstick. A suitable cap, as shown below, was made on a 3D printer. A container cap might have worked … as long as it doesn’t wobble and thus affect the readings. You might instead drill a hole for it but I’ve no advice on this.

how to write the program and install the code on the ESP 32 / ESP8266 board

My oil sensor is a ‘connected’ oil sensor. It takes readings, sends them to Home Assistant which will present them in various ways. To gain this connectivity you need a working Home Assistant installation on a Raspberry Pi (shown in an earlier project).

What you do now is best seen demonstrated, but you’ll first assemble a series of instructions (a program) and check the code for errors. You then get the instructions turned into a file (you compile a ‘bin file’) that you’ll install on a ESP dev board. The bin or binary file is otherwise called firmware. The process of installing the file into the ESP, via a USB cable is called flashing the ESP32 with firmware.

• use your browser to interact with Home Assistant on 192.168.x.x:8123
• in Home Assistant go to the Esphome add-on and add a ‘new device’ (aka new node).
• a wizard asks you for a name (eg oil) and then whether you have an ESP32 or ESP8266. There will be a choice of branded boards – using a branded board helps a little with pin labelling eg my nodemcuv3 has a pin labelled D1 which is connected to GPI05 of the ESP8266. (If you don’t know your board brand, choose a generic one and refer to pins in your code as ‘GPIOx’ instead of D1, D2 etc).
• you may be asked to fill in wifi details. Below you’ll see I’ve (optionally) entered mine replacing the actual password with !secret wifi_password Look for the SECRETS button in the ESPhome panel and enter here a list such as
  • wifi_ssid: “homewifi”
  • wifi_password: “pass”
  • ha_password: “APIpassword”

• at this point ESPHome may try to connect to your ESP32 but you may skip this connection and do that manually later.
• choose ‘edit’ in the device box you just now created. add the code below but change the items in bold to suit your board, your wifi, your network and the GPIO pins you want to connect to.
• in the ESPhome section of Home Assistant VALIDATE your code. Errors are often due to duplicate sections or incomplete sections or in my case, senior moments.
• in the ESPhome section of Home Assistant choose INSTALL (=compile) and Manual download. This will inspect your code and compile it for the ESP chip and add ‘libraries’ for each feature you put in the code. After a few minutes ESPhome creates a ‘.bin’ file and saves it on your computer.
• using either ESPHome or ESPHome flasher tool, connect your ESP board with a USB cable. Choose the bin file, click flash and view the logs as the bin file is ‘uploaded to the ESP32.
• notice the line below “ota:” which in future allows you to make changes to your code and upload it to the ESP wirelessly.

# SOME SECTIONS BELOW ARE CREATED BY THE ESPHOME WIZARD SO DON'T DUPLICATE THESE

esphome:
  name: oil-tof

esp8266:
  board: nodemcuv2

# Enable logging
logger:

# Enable Home Assistant API
api:

ota:
  password: ""

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  manual_ip:
    # Set this to the IP of the ESP
    static_ip: 192.168.1.15  
    # Set this to the IP address of the router. Often ends with .1
    gateway: 192.168.1.1
    # The subnet of the network. 255.255.255.0 works for most home networks.
    subnet: 255.255.255.0


  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "oil-tof fail hotspot"
    password: !secret wifi_password
captive_portal:


# this section defines how the sensor will work. Read on* to see how I've improved this over time

i2c:
  sda: D2 #GPI04 orange or green # SDA connects I2C SDA (data) pin 4 D2
  scl: D1 #GPI05 yellow or blue  # SCL connects I2C SCL (clock) pin 5 D1
# scan: true

# Comments: 
# sda pinID: the pin for the I²C bus data line. Board default GPIO21 - ESP32 / GPIO4 - ESP8266.
# scl pinID: the pin for the I²C bus clock line. Board default GPIO22 - ESP32 / GPIO5 - ESP8266.
# scan: false # would stop ESPHome searching the I²C address space on startup

# the section below works to deliver readings of both oil level as litres used and oil as kW 

sensor:
  - platform: vl53l0x
    name: "oil kW"
    address: 0x29
    id: myoillevel
    update_interval: 30s
    long_range: false
    accuracy_decimals: 0
    unit_of_measurement: "kW"
    filters:
    - lambda: return x * 1018 * 10;  
    - sliding_window_moving_average:
        window_size: 60
        send_every: 10    
    state_class: total_increasing  
    device_class: 'power'

#  negative multiplier to get oil level below in the template 
  - platform: template
    name: "oil_level"
    update_interval: 60s
    accuracy_decimals: 0
    unit_of_measurement: "L"
    lambda: |-
      return -0.1 * id(myoillevel).state ;  

# this optional section controls the built-in LED light on the nodemcu 8266. 
# Look up the LED GPIO for your ESP dev board and adjust below. On my board Home Assistant reads an on light as off.   
output:
  - platform: gpio
    pin: GPIO16
    id: gpio_16


light:
  - platform: binary
    output: gpio_16
    name: oil-tof light 


# see the para below for how I improved the sensor section for my tank

# battery power configuration entry - read more at ESPHome Deep sleep
deep_sleep:
  run_duration: 10s
  sleep_duration: 60min

add your sensor data to the Home Assistant overview

• when the flashing is done, power-up the ESP32 with any USB charger and go to Home Assistant > Integrations > + > ESPHome. You might have to enter the IP address you chose in the code. You’ll then see the oil level device that has been added to Home Assistant. The device will have an entity named something like sensor.oil_level
• go to the Home Assistant Overview where you stuff is normally displayed. Choose Edit dashboard in the top corner.
• add a new card called history graph. On the entity line select your sensor.oil_level
• add a new card called gauge to display your oil level. On the entity line select your sensor.oil_level

*refine ‘the oil sensor code to correctly calibrate it – the mk II code

After several months use I needed to address some points. Because of these points I modified the code as shown below. You may want to substitute the above ‘sensor:’ section with the following (explained below)

# You can add a filter to the sensor section to calibrate the sensor.  
# To produce a falling oil level simply add a negative sign as here:
#   filters: 
#   - lambda: return x * -1053; 
# Alternative sensor filter section to remove noise:

 filters: 
  - lambda: return x * 1053;
  - sliding_window_moving_average: 
    window_size: 20 
    send_every: 10

# When tank is irregular in shape calibrate the distance into the tank
# SEE THE POST FROM 'Kev' below where he improved the filter section
#  – calibrate_linear:
#  – 110 -> 0
#  – 95 -> 204
#  – 75 -> 544
#  – 55 -> 900
#  – 35 -> 1320
#  – 15 -> 1660
#  – 0 -> 1800

A few issues:

• maxing out: The sensor is bouncing off a metal strut inside the tank so I need to relocate the sensor somewhere else. Because of this I keep my daily eye on the history graph, looking for a plateau that occurs when the sensor ‘sees’ the metal strut.
• calibration: the sensor measures in metres so I have to calculate the litres. My tank has a perfectly regular shape from top to bottom – so by measuring dipstick depths, I found that 100cm of tank depth contains 105.3 litres. In the ‘MkII’ code below the last two lines (filter and lamda) do this calibration. The two lines can be edited or entirely omitted to suit your tank.
• the readings were a bit noisy. To counteract this I calculate the sliding average of an oil level reading.
• it was suggested that the ‘oil used’ graph should fall, not rise, so I made the reading negative. This also makes knowing how much oil to order easier. I should name the sensor ‘oil used’ because the number it returns is the amount of oil I need to buy to fill the tank. Next I really don’t need an oil level reading every 10 seconds so I increased the update interval.

so when do I need to order some more oil?

The first nice consequence is that I now know how much oil I need to order and how much I use. In a cold spell two households used 30 litres a day – I’ll be posting my metrics in a future post. And I can see I would need to order an extra 80 litres as it will take say, 9 days to arrive. As I write I’m thinking about a way to predict the date when I’ll need to order more oil. For the present I’ve put the data in a spreadsheet and it looks like I’ll order in mid January – unless I start to use oil faster.

notify me when more oil needs to be ordered

I order 900 litres of oil and I use 100 litres of oil as I wait for it to be delivered. Therefore I need an alert when the tank has 850 – 100 or 750 litres of oil. Home Assistant offers numerous ways to alert you:

• if you use SMTP email (ie your own non-gmail email) this is the easiest notification to set up. It can work with Gmail after a fiddle. I get Home Assistant to send me an email (‘SMTP platform’) for all sorts of alerts. I prefer these less-naggy alerts. I use a filter on incoming mail so that this mail avoids my inbox.
• you may prefer to see a message in your browser or see a pop-up a ‘toast’ message on a phone. You can set a notification message to persist so that you can’t easily swipe it away. You can also set a notification to be urgent enough to wake up a phone to tell you, or set one that waits quietly until you next pick up the phone.

Here is the automation code. I found the easiest notification method to get working is the email notification. The next easiest may be via the Home Assistant mobile app.

 # TIME TO ORDER OIL AUTOMATION - HOME ASSISTANT

 alias: heating oil level alert
 trigger:
 above: '750'
 entity_id: sensor.oil_level
 platform: numeric_state
 condition:
 after: 07:00
 condition: time
 action:
 data:
   message: at {{ states ('sensor.time') }}
   title: '{{ states (''sensor.oil_level'') }} L now. 850L oil to be ordered'
 service: notify.notify_html5
 data:
   message: at {{ states ('sensor.time') }}
   title: '{{ states (''sensor.oil_level'') }} L now. 850L oil to be ordered'
 service: notify.email 

record your daily readings in a Google spreadsheet using IFTTT

Improvement 1

For a while it was OK to manually record the daily oil level in a spreadsheet. I soon saw that I was using 80 – 100 litres of oil a week during December. I therefore set up a connection between Home Assistant and IFTTT so that the daily level could be recorded in a spreadsheet. This has been happening daily all through to 2022 and hasn’t broken yet. The code for the automation routine that takes the reading is below. My procedure to link Home Assistant and IFTTT is on this page: send sensor data from Home Assistant to a spreadsheet

# THE FOLLOWING IS A TEXT FILE CALLED oil_level.yaml in the folder AUTOMATIONS 
# that is found at /config/automations/oil_level.yaml
id: '12399999'
alias: oil_level_spreadsheet
description: daily readings in oil_level.yaml
trigger:
at: "23:55:00"
platform: time
condition: []
action:
data_template: { "event": "oil_level", "value1": "{{ states('sensor.oil_level')}}", "value2": "{{ states('sensor.temperature_weather')}}", "value3": "{{ states('sensor.power_today')}}" }
service: ifttt.trigger

Improvement 2 June 2022 and December 2022

The sensor is a great help for ordering oil. But it may also be useful to see your oil readings as kW (power) and kWh (energy). This needs some changes to the code to add some entity properties. I’ve guessed we can get 10kW from 1 litre of oil (that’s generous on boiler efficiency.

Notice that the code now includes two filters to remove noise from the readings. We take 20 readings over 5 minutes. The quantile filter is said to remove ‘outliers’ (the readings that are wild). Having removed the outliers we use a rolling/sliding average of those 20 readings. Results shown below.

IN ESPHOME - revised code for ESP32
# the sensor
sensor:
  - platform: vl53l0x
    name: "oil kW"
    address: 0x29
    id: myoillevel
    update_interval: 15s
    long_range: false
    accuracy_decimals: 0
    unit_of_measurement: "kW"
    filters:
  # ALTERNATIVE CALCULATION: - lambda: return x * 1018 * 10; # change 10 to 9 for 90% efficiency however this may affect the level calculation 
 
    - lambda: return (x - 0.034) * 1018 * 10;

    - quantile:
        window_size: 20
        send_every: 5
        send_first_at: 2
        quantile: .95    
    - sliding_window_moving_average:
        window_size: 20
        send_every: 5    
    state_class: total_increasing
    device_class: 'power'

# no negative above to get KW of oil. negative below to get oil level below in the template .
  - platform: template
    name: "oil_level"
#    update_interval: 60s
    accuracy_decimals: 0
    unit_of_measurement: "L"
    lambda: |-
      return -0.1 * id(myoillevel).state ;

Compile and install the code above in place of the code earlier in this post.

# TRY WITHOUT BUT YOU MIGHT ADD THIS TO THE END OF CUSTOMIZE.YAML

sensor.oil_energy_24h:
  last_reset: 2022-07-16T23:00:00.023678+00:00

add your oil meter to the Home Assistant energy dashboard

The result of the above is to give you entities sensor.oil kW (which tells you little) and sensor.oil_energy_24h which tells how much oil is used each day.