heating oil tank level – Vl53L0X time of flight sensor version

make a heating oil tank level sensor that’s more useful than anything you can buy

• can measure oil level in any unit – litres, kWh, gallons or days until I need to refill the tank
• displays readings on a gauge or time graph (uses Home Assistant gauge & history graph).
• receive readings or alerts as an email or phone notification or flashing smart bulb in the house.
• uses an infrared based Vl53L0X time of flight sensor (~£5 ebay)
• uses any ESP32 or ESP8266 development board with wifi. I used a nodemcu v3 ESP8266 (~ £6).
• with a waterproof electrician’s housing this could work outdoors and on battery power. if you need to use battery power an ESP8266 might be more appropriate. The battery power might be 6 x 18650 cells based on this one I made
• with a little modification this project would monitor the salt level in a water softener or the water in a water barrel (ref Dr Zzz – youtube)

Background: some homes use diesel oil as a fuel to keep warm in winter. Over the course of a winter I might need two or three deliveries of oil but till now I’ve no way to predict when I’ll need more oil or know how much to order. And I really don’t ever want to run out of oil. The oil is burned in a heating boiler in the same way that many homes in the UK burn methane gas. Although we call the liquid ‘heating oil’ it’s more runny than lubricating oil. You may call it diesel or white spirits or turps or paraffin or kerosine.

I asked local firm Shelford Heating to quote a price for an oil level sensor. Their solution involved drilling the steel tank to fit an Apollo wireless kit – but £220 seemed excessive for this need. I could also have bought the kit, which sends an approximate oil level to a display but I really wanted something that could alert me better.

what’s needed for this project

• Vl53L0X time of flight sensor
• coloured Dupont wires or coloured 18 gauge stranded wires if the wires need to be long
• If this is outdoors you’ll need waterproofing as well as a way to get a 5v supply to the oil tank
• An ESP8266 or ESP32 development board
• Home Assistant already set up on a Raspberry Pi
• heat shrink tubing and soldering skills if you want to make reliable connections (you do)

how to connect up an oil level sensor to a ESP32 / ESP8266 development board

This is a new version of my original oil tank level sensor using the ultrasonic JSN-SR04T that had been working reliably for years. I have lived with its limitation that the ultrasonic sensor cannot measure at short range so it records the same oil level whether the tall is full or 200mm below full. Thereafter the readings are very useful at alerting me to order more oil.

The Vl53L0X time of flight sensor is an inexpensive alternative that works by sending infrared blips to the liquid surface and then measuring the ‘time of flight’ to estimate how far away it is. It can send correct values at short range ie even if the tank is full. You can find versions of this sensor (VL53L0X, TOF10120, VL53L1X, VL6180X) which work best at different distances (eg below). I chose the Vl53L0X because it’s made for the range 30mm-1200m as suits my tank. (A project listed at instructables states that the VL53L1X is a much better and more expensive sensor – I may switch to this if and when my sensor fails). The software can also set it in a long distance remote range of 1500 -2000mm. Wonderfully, when the sensor is hooked up it sends a distance value (in metres) and I can multiply that by a number to get readings in mm or litres or kW or even money. The calculation is simple if your oil tank has a regular shape but anything is doable.

The Vl53L0X module has six pin connectors – you only need to use four of them. I use Dupont male to female wires because my ESP8266 nodemcu already had pins. I soldered the male pins to the Vl53L0X module and extended the wires with coloured cables (silicone 18 AWG 0.75mm) and covered the join in heat-shrink. GND and VIN connect to GND and 3.3v on an ESP8266 module. Connect SCL to D1 (or GPI05). Connect SDA to D2 (or GPI04). For connections to a ESP32 see the image below

a housing to protect the VL53L0X oil level sensor and ESP

This is where my system will surely be different. A 3D printed part holds the sensor in place with the help of some hot glue. I don’t yet know if the vapour from the oil will damage the VL53L0X so I’m looking to improve the housing.

Getting dirt in the tiny emitter and sensor holes is to be avoided as the emerging pulse or photon can be deflected randomly / cause crosstalk. So I cut a 30mm circle of acetate film (from a phone case) and glued it to the cap I’d made for the sensor. I might have used a piece of phone screen protector and it would wise to test if the cover affects the distance reading. I cover this in my post about mounting and calibration here.

You can protect the sensor with glass, perspex or polycarbonate. The cover should be as thin, flat, parallel, clear and fitted close to the sensor as possible. ST, the chip maker have a video explaining how a more fussy two-piece glass cover pretty much reduces the crosstalk due to dust. I so wish they’ll create a similarly protected chip. The sensor notes are here.

The sensor needs to be fixed so that it points down at the surface of the oil in the tank (photo). I used a tank hole that had housed the dipstick. A suitable cap, as shown below, was made on a 3D printer. A container cap might have worked … as long as it doesn’t wobble and thus affect the readings. You might instead drill a hole for it but I’ve no advice on this.

how to write the program and install the code on the ESP 32 / ESP8266 development board

My oil sensor is a ‘connected’ oil sensor. It takes readings, sends them to Home Assistant which will present them in various ways. To gain this connectivity you need a working Home Assistant installation on a Raspberry Pi (shown in an earlier project).

What you do is to assemble a series of instructions (a program) and get the instructions turned into a file (=compiled into a ‘bin file’) that you upload to a ESP dev board via a USB cable. The file is otherwise called firmware. The process of installing the file to the ESP is called flashing the firmware.

• use your browser to interact with Home Assistant on 192.168.x.x:8123
• in Home Assistant go to the Esphome add-on and add a ‘new device’. Follow the process eg it’ll ask you for a name (eg oil level) and you may be asked to fill in wifi details.
• Still in Esphome look for ‘Edit’ next to your new device. Add the following code making changes to suit your setup.
• choose Install to upload this code to your development board

# THIS IS FOR AN ESP8266 development board
esphome:
  name: oil-tof
substitutions:
  devicename: oil-tof
esp8266:
  board: nodemcuv2
# enable logging
logger:
# enable Home Assistant API
api:
  encryption: 
    key: "FtKhT5Sf4cwe5bh/N0Kzy6JMM5LG79H38rfceMlBHrE"
ota:
  password: ""
wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  manual_ip:
    # set this to the IP of the ESP
    static_ip: 192.168.1.15
    # set this to the IP address of the router
    gateway: 192.168.1.1
    # the subnet of the network
    subnet: 255.255.255.0
  # hotspot in case wifi connection fails
  ap:
    ssid: "oil-tof fail hotspot"
    password: !secret wifi_password
captive_portal:

i2c:
  sda: D2 #GPI04 orange or green (data) pin 4
  scl: D1 #GPI05 yellow or blue (clock) pin 5
  scan: true

sensor:
  - platform: vl53l0x
    name: "oil kW"
    address: 0x29
    id: myoillevel
    update_interval: 10s
    long_range: false
    accuracy_decimals: 0
    unit_of_measurement: "kW"
    filters:
    - lambda: return (x - 0.034) * 999 * 10;
    - quantile:
        window_size: 60
        send_every: 5
        send_first_at: 2
        quantile: .95    
    - sliding_window_moving_average:
        window_size: 60
        send_every: 5    
    state_class: total_increasing
    device_class: 'power'
#  negative to get oil level below
  - platform: template
    name: "oil_level"
#    update_interval: 60s
    accuracy_decimals: 0
    unit_of_measurement: "L"
    lambda: |-
      return -0.1 * id(myoillevel).state ;  
  
switch:
  - platform: restart
    name: ${devicename} restart   

---

# THIS IS FOR AN ESP32 development board
# esphome:
  name: oil-tof

substitutions:
  devicename: oil-tof

esp32:
  board: esp32dev
  framework:
    type: arduino

# enable logging
logger:
# enable Home Assistant API
api:
  encryption: 
    key: "FtKhT5Sf4cwe5bh/N0Kzy6JMM5LG79H38rfceMlBHrE"
ota:
  password: ""
wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  manual_ip:
    # set this to the IP of the ESP
    static_ip: 192.168.1.15
    # set this to the IP address of the router
    gateway: 192.168.1.1
    # the subnet of the network
    subnet: 255.255.255.0
  # Enable hotspot if case wifi connection fails
  ap:
    ssid: "oil-tof fail hotspot"
    password: !secret wifi_password
captive_portal:

i2c:
  sda: 21 # orange or green
  scl: 22 # yellow or blue
  scan: true

sensor:
  - platform: vl53l0x
    name: "oil kW"
    address: 0x29
    id: myoillevel
    update_interval: 10s
    long_range: false
    accuracy_decimals: 0
    unit_of_measurement: "kW"
    filters:
    - lambda: return (x - 0.034) * 999 * 10;
    - quantile:
        window_size: 60
        send_every: 5
        send_first_at: 2
        quantile: .95    
    - sliding_window_moving_average:
        window_size: 60
        send_every: 5    
    state_class: total_increasing
    device_class: 'power'
#  negative to get oil level below
  - platform: template
    name: "oil_level"
#    update_interval: 60s
    accuracy_decimals: 0
    unit_of_measurement: "L"
    lambda: |-
      return -0.1 * id(myoillevel).state ;  
  
switch:
  - platform: restart
    name: ${devicename} restart   

add your sensor data to the Home Assistant overview

• when the code is installed, power-up the ESP board with an USB charger and go to Home Assistant > Integrations > + > ESPHome. You might have to enter the IP address you chose in the code. You’ll then see the oil level device that has been added to Home Assistant. The device will have an entity named like sensor.oil_level
• go to the Home Assistant Overview where you stuff is normally displayed. Choose Edit dashboard in the top corner.
• add a new card called history graph. On the entity line select your sensor.oil_level
• add a new card called gauge to display your oil level. On the entity line select your sensor.oil_level

*refine ‘the oil sensor code to correctly calibrate it – the mk II code

After several months use I needed to address some points. Because of these points I modified the code as shown below. You may want to substitute the above ‘sensor:’ section with the following (explained below)

# You can add a filter to the sensor section to calibrate the sensor.  
# To produce a falling oil level simply add a negative sign as here:
#   filters: 
#   - lambda: return x * -1053; 
# Alternative sensor filter section to remove noise:

 filters: 
  - lambda: return x * 1053;
  - sliding_window_moving_average: 
    window_size: 20 
    send_every: 10

# When tank is irregular in shape calibrate the distance into the tank
# SEE THE POST FROM 'Kev' below where he improved the filter section
#  – calibrate_linear:
#  – 110 -> 0
#  – 95 -> 204
#  – 75 -> 544
#  – 55 -> 900
#  – 35 -> 1320
#  – 15 -> 1660
#  – 0 -> 1800

A few issues:

• maxing out: The sensor is bouncing off a metal strut inside the tank so I need to relocate the sensor somewhere else. Because of this I keep my daily eye on the history graph, looking for a plateau that occurs when the sensor ‘sees’ the metal strut.
• calibration: the sensor measures in metres so I have to calculate the litres. My tank has a perfectly regular shape from top to bottom – so by measuring dipstick depths, I found that 100cm of tank depth contains 105.3 litres. In the ‘MkII’ code below the last two lines (filter and lamda) do this calibration. The two lines can be edited or entirely omitted to suit your tank.
• the readings were a bit noisy. To counteract this I calculate the sliding average of an oil level reading.
• it was suggested that the ‘oil used’ graph should fall, not rise, so I made the reading negative. This also makes knowing how much oil to order easier. I should name the sensor ‘oil used’ because the number it returns is the amount of oil I need to buy to fill the tank. Next I really don’t need an oil level reading every 10 seconds so I increased the update interval.

so when do I need to order some more oil?

The first nice consequence is that I now know how much oil I need to order and how much I use. In a cold spell two households used 30 litres a day – I’ll be posting my metrics in a future post. And I can see I would need to order an extra 80 litres as it will take say, 9 days to arrive. As I write I’m thinking about a way to predict the date when I’ll need to order more oil. For the present I’ve put the data in a spreadsheet and it looks like I’ll order in mid January – unless I start to use oil faster.

notify me when more oil needs to be ordered

I order 900 litres of oil and I use 100 litres of oil as I wait for it to be delivered. Therefore I need an alert when the tank has 850 – 100 or 750 litres of oil. Home Assistant offers numerous ways to alert you:

• if you use SMTP email (ie your own non-gmail email) this is the easiest notification to set up. It can work with Gmail after a fiddle. I get Home Assistant to send me an email (‘SMTP platform’) for all sorts of alerts. I prefer these less-naggy alerts. I use a filter on incoming mail so that this mail avoids my inbox.
• you may prefer to see a message in your browser or see a pop-up a ‘toast’ message on a phone. You can set a notification message to persist so that you can’t easily swipe it away. You can also set a notification to be urgent enough to wake up a phone to tell you, or set one that waits quietly until you next pick up the phone.

Here is the automation code. I found the easiest notification method to get working is the email notification. The next easiest may be via the Home Assistant mobile app.

 # TIME TO ORDER OIL AUTOMATION - HOME ASSISTANT

 alias: heating oil level alert
 trigger:
 above: '750'
 entity_id: sensor.oil_level
 platform: numeric_state
 condition:
 after: 07:00
 condition: time
 action:
 data:
   message: at {{ states ('sensor.time') }}
   title: '{{ states (''sensor.oil_level'') }} L now. 850L oil to be ordered'
 service: notify.notify_html5
 data:
   message: at {{ states ('sensor.time') }}
   title: '{{ states (''sensor.oil_level'') }} L now. 850L oil to be ordered'
 service: notify.email 

record your daily readings in a Google spreadsheet using IFTTT

Improvement 1

For a while it was OK to manually record the daily oil level in a spreadsheet. I soon saw that I was using 80 – 100 litres of oil a week during December. I therefore set up a connection between Home Assistant and IFTTT so that the daily level could be recorded in a spreadsheet. This has been happening daily all through to 2022 and hasn’t broken yet. The code for the automation routine that takes the reading is below. My procedure to link Home Assistant and IFTTT is on this page: send sensor data from Home Assistant to a spreadsheet

# THE FOLLOWING IS A TEXT FILE CALLED oil_level.yaml in the folder AUTOMATIONS 
# that is found at /config/automations/oil_level.yaml
id: '12399999'
alias: oil_level_spreadsheet
description: daily readings in oil_level.yaml
trigger:
at: "23:55:00"
platform: time
condition: []
action:
data_template: { "event": "oil_level", "value1": "{{ states('sensor.oil_level')}}", "value2": "{{ states('sensor.temperature_weather')}}", "value3": "{{ states('sensor.power_today')}}" }
service: ifttt.trigger

Improvement 2 June 2022 and December 2022

The sensor is a great help for ordering oil. But it may also be useful to see your oil readings as kW (power) and kWh (energy). This needs some changes to the code to add some entity properties. I’ve guessed we can get 10kW from 1 litre of oil (that’s generous on boiler efficiency.

Notice that the code now includes two filters to remove noise from the readings. We take 20 readings over 5 minutes. The quantile filter is said to remove ‘outliers’ (the readings that are wild). Having removed the outliers we use a rolling/sliding average of those 20 readings. Results shown below.

IN ESPHOME - revised code for ESP32
# the sensor
sensor:
  - platform: vl53l0x
    name: "oil kW"
    address: 0x29
    id: myoillevel
    update_interval: 15s
    long_range: false
    accuracy_decimals: 0
    unit_of_measurement: "kW"
    filters:
  # ALTERNATIVE CALCULATION: - lambda: return x * 1018 * 10; # change 10 to 9 for 90% efficiency however this may affect the level calculation 
 
    - lambda: return (x - 0.034) * 1018 * 10;
    - quantile:
        window_size: 20
        send_every: 5
        send_first_at: 2
        quantile: .95    
    - sliding_window_moving_average:
        window_size: 20
        send_every: 5    
    state_class: total_increasing
    device_class: 'power'

# no negative above to get KW of oil. negative below to get oil level below in the template .
  - platform: template
    name: "oil_level"
#    update_interval: 60s
    accuracy_decimals: 0
    unit_of_measurement: "L"
    lambda: |-
      return -0.1 * id(myoillevel).state ;

Compile and install the code above in place of the code earlier in this post.

# YOU MIGHT ADD THIS TO THE END OF CUSTOMIZE.YAML

sensor.oil_energy_24h:
  last_reset: 2022-07-16T23:00:00.023678+00:00

add your oil meter to the Home Assistant energy dashboard

The result of the above is to give you entities sensor.oil kW (which tells you little) and sensor.oil_energy_24h which tells how much oil is used each day.