heating oil tank level – Vl53L0X time of flight sensor version

make a heating oil tank level sensor that’s more useful than anything you can buy

monitor oil or water tank liquid level with a time of flight sensor
  • can measure oil level in any unit – litres, kWh, gallons or days until I need to refill the tank
  • displays readings on a gauge or time graph (uses Home Assistant gauge & history graph).
  • receive readings or alerts as an email or phone notification or flashing smart bulb in the house.
  • uses an infrared based Vl53L0X time of flight sensor (~£5 ebay)
  • uses any ESP32 or ESP8266 development board with wifi. I used a nodemcu v3 ESP8266 (~ £6).
  • with a waterproof electrician’s housing this could work outdoors and on battery power. if you need to use battery power an ESP8266 might be more appropriate. The battery power might be 6 x 18650 cells based on this one I made
  • with a little modification this project would monitor the salt level in a water softener or the water in a water barrel (ref Dr Zzz – youtube)

Some homes use oil as a fuel to keep warm in winter. Over the course of a winter I might need to order two or three deliveries of oil but till now I’ve no way to predict the day when I’ll need to top it up or know how much to order. And I really don’t ever want to run out of oil. The oil is burned in a central heating boiler in the same way that many homes in the UK burn methane gas. Although we call the liquid ‘heating oil’ it’s more runny than lubricating oil. It’s more like diesel or white spirits or turps or paraffin or kerosine.

I asked local firm Shelford Heating to quote a price for an oil level sensor. Their solution involved drilling the steel tank to fit an Apollo wireless kit – but £220 seemed excessive for this need. I could also have bought the kit, which sends an approximate oil level to a display but I really wanted something that could alert me better.

how to connect up an oil level sensor to a ESP development board

This is a newer version of my oil tank level sensor using the ultrasonic JSN-SR04T that has been working reliably for years. I have lived with the limitation that the ultrasonic sensor cannot measure at short range so it records the same oil level whether the tall is full or 200mm below full. Thereafter the readings are very useful at alerting me to order more oil.

The Vl53L0X time of flight sensor is an inexpensive alternative that works by sending infrared blips to the liquid surface and then measuring the ‘time of flight’ to estimate how far away it is. It can send correct values even if the tank is full. You can find variations of this sensor (VL53L0X, TOF10120, VL53L1X, VL6180X) which work best at different distances (eg below). I chose the Vl53L0X because it’s made for the range 30mm-1200m as suits my tank. (A project listed at instructables states that the VL53L1X is a much better and more expensive sensor – I may switch to this if and when my sensor fails). The software can also set it in a long distance remote range of 1500 -2000mm. Wonderfully, when the sensor is hooked up it sends a distance value (in metres) and I can multiply that by a number to get readings in mm or litres or kW or even money. The calculation is simple if your oil tank has a regular shape but anything is doable.

This sensor can be had with a dust cover (thanks to ‘Tyeth’ – see comments section at the bottom). We aren’t sure if this can withstand the oil vapour over time. Remember to check the spec sheet to find a sensor with the distance range that you need to measure.

The Vl53L0X module has six pin connectors – you only need to use four of them. I use Dupont male to female wires because my ESP8266 nodemcu already had pins. I soldered the male pins to the Vl53L0X module and extended the wires with coloured cables (silicone 18 AWG 0.75mm) and covered the join in heat-shrink. GND and VIN connect to GND and 3.3v on an ESP module. Connect SCL to D1 (or GPI05). Connect SDA to D2 (or GPI04).

a housing to protect the VL53L0X oil level sensor and ESP

This is where my system will be different to others. My 3D printed part holds the sensor in place with the help of some hot glue. I don’t yet know if the vapour from the oil will damage the VL53L0X so I’m looking to improve the housing.

Getting dirt in the tiny emitter and sensor holes is to be avoided as the emerging pulse or photon can be deflected randomly / cause crosstalk. So I cut a 30mm circle of acetate film (from a phone case) and glued it to the cap I’d made for the sensor. I might have used a piece of phone screen protector and it would wise to test if the cover affects the distance reading. I cover this in my post about mounting and calibration here.

You can protect the sensor with glass, perspex or polycarbonate. The cover should be as thin, flat, parallel, clear and fitted close to the sensor as possible. ST, the chip maker have a video explaining how a more fussy two-piece glass cover pretty much reduces the crosstalk due to dust. I so wish they’ll create a similarly protected chip. The sensor notes are here.

Link to STL file for 3D printed holder file – oil-sensor-holder.stl

The sensor needs to be fixed so that it points down at the surface of the oil in the tank (photo). I used a tank hole that had housed the dipstick. A suitable cap, as shown below, was made on a 3D printer. A container cap might have worked … as long as it doesn’t wobble and thus affect the readings. You might instead drill a hole for it but I’ve no advice on this.

The sensor installed on a messy indoor all-metal oil tank! I lined the sensor holder with sticky felt to ensure it couldn’t slip or shift easily

how to write the program and install the code on the ESP 32 / ESP8266 board

My oil sensor is a ‘connected’ oil sensor. It takes readings, sends them to Home Assistant which will present them in various ways. To gain this connectivity you need a working Home Assistant installation on a Raspberry Pi (shown in an earlier project).

What you do now is best seen demonstrated, but you’ll first assemble a series of instructions (a program) and check the code for errors. You then get the instructions turned into a file (you compile a ‘bin file’) that you’ll install on a ESP dev board. The bin or binary file is otherwise called firmware. The process of installing the file into the ESP, via a USB cable is called flashing the ESP32 with firmware.

This is where you create the code, validate the code and download the binary file for the ESP32
  • use your browser to interact with Home Assistant on 192.168.x.x:8123
  • in Home Assistant go to the Esphome add-on and add a ‘new device’ (aka new node).
  • a wizard asks you for a name (eg oil) and then whether you have an ESP32 or ESP8266. There will be a choice of branded boards – using a branded board helps a little with pin labelling eg my nodemcuv3 has a pin labelled D1 which is connected to GPI05 of the ESP8266. (If you don’t know your board brand, choose a generic one and refer to pins in your code as ‘GPIOx’ instead of D1, D2 etc).
  • you may be asked to fill in wifi details. Below you’ll see I’ve (optionally) entered mine replacing the actual password with !secret wifi_password Look for the SECRETS button in the ESPhome panel and enter here a list such as
    • wifi_ssid: “homewifi”
    • wifi_password: “pass”
    • ha_password: “APIpassword”
  • at this point ESPHome may try to connect to your ESP32 but you may skip this connection and do that manually later.
  • choose ‘edit’ in the device box you just now created. add the code below but change the items in bold to suit your board, your wifi, your network and the GPIO pins you want to connect to.
  • in the ESPhome section of Home Assistant VALIDATE your code. Errors are often due to duplicate sections or incomplete sections or in my case, senior moments.
  • in the ESPhome section of Home Assistant choose INSTALL (=compile) and Manual download. This will inspect your code and compile it for the ESP chip and add ‘libraries’ for each feature you put in the code. After a few minutes ESPhome creates a ‘.bin’ file and saves it on your computer.
  • using either ESPHome or ESPHome flasher tool, connect your ESP board with a USB cable. Choose the bin file, click flash and view the logs as the bin file is ‘uploaded to the ESP32.
  • notice the line below “ota:” which in future allows you to make changes to your code and upload it to the ESP wirelessly.
# SOME SECTIONS BELOW ARE CREATED BY THE ESPHOME WIZARD SO DON'T DUPLICATE THESE

esphome:
  name: oil-tof

esp8266:
  board: nodemcuv2

# Enable logging
logger:

# Enable Home Assistant API
api:

ota:
  password: ""

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  manual_ip:
    # Set this to the IP of the ESP
    static_ip: 192.168.1.15  
    # Set this to the IP address of the router. Often ends with .1
    gateway: 192.168.1.1
    # The subnet of the network. 255.255.255.0 works for most home networks.
    subnet: 255.255.255.0


  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "oil-tof fail hotspot"
    password: !secret wifi_password
captive_portal:


# this section defines how the sensor will work. Read on* to see how I've improved this over time

i2c:
  sda: D2 #GPI04 orange or green # SDA connects I2C SDA (data) pin 4 D2
  scl: D1 #GPI05 yellow or blue  # SCL connects I2C SCL (clock) pin 5 D1
# scan: true

# Comments: 
# sda pinID: the pin for the I²C bus data line. Board default GPIO21 - ESP32 / GPIO4 - ESP8266.
# scl pinID: the pin for the I²C bus clock line. Board default GPIO22 - ESP32 / GPIO5 - ESP8266.
# scan: false # would stop ESPHome searching the I²C address space on startup

# the section below works to deliver readings of both oil level as litres used and oil as kW 

sensor:
  - platform: vl53l0x
    name: "oil kW"
    address: 0x29
    id: myoillevel
    update_interval: 30s
    long_range: false
    accuracy_decimals: 0
    unit_of_measurement: "kW"
    filters:
    - lambda: return x * 1018 * 10;  
    - sliding_window_moving_average:
        window_size: 60
        send_every: 10    
    state_class: total_increasing  
    device_class: 'power'

#  negative multiplier to get oil level below in the template 
  - platform: template
    name: "oil_level"
    update_interval: 60s
    accuracy_decimals: 0
    unit_of_measurement: "L"
    lambda: |-
      return -0.1 * id(myoillevel).state ;  

# this optional section controls the built-in LED light on the nodemcu 8266. 
# Look up the LED GPIO for your ESP dev board and adjust below. On my board Home Assistant reads an on light as off.   
output:
  - platform: gpio
    pin: GPIO16
    id: gpio_16


light:
  - platform: binary
    output: gpio_16
    name: oil-tof light 


# see the para below for how I improved the sensor section for my tank

# battery power configuration entry - read more at ESPHome Deep sleep
deep_sleep:
  run_duration: 10s
  sleep_duration: 60min

add your sensor data to the Home Assistant overview

  • when the flashing is done, power-up the ESP32 with any USB charger and go to Home Assistant > Integrations > + > ESPHome. You might have to enter the IP address you chose in the code. You’ll then see the oil level device that has been added to Home Assistant. The device will have an entity named something like sensor.oil_level
  • go to the Home Assistant Overview where you stuff is normally displayed. Choose Edit dashboard in the top corner.
  • add a new card called history graph. On the entity line select your sensor.oil_level
  • add a new card called gauge to display your oil level. On the entity line select your sensor.oil_level
a sensor card; a history graph card and a gauge card. notice that in this first version of the code, ‘oil level’ is INCREASING. That’s because the number shown is the amount of oil needed to fill the tank!

*refine ‘the oil sensor code to correctly calibrate it – the mk II code

the oil level graph now falls and is correctly calibrated to suit my tank

After several months use I needed to address some points. Because of these points I modified the code as shown below. You may want to substitute the above ‘sensor:’ section with the following (explained below)

# You can add a filter to the sensor section to calibrate the sensor.  
# To produce a falling oil level simply add a negative sign as here:
#   filters: 
#   - lambda: return x * -1053; 
# Alternative sensor filter section to remove noise:

 filters: 
  - lambda: return x * 1053;
  - sliding_window_moving_average: 
    window_size: 20 
    send_every: 10

# When tank is irregular in shape calibrate the distance into the tank
# SEE THE POST FROM 'Kev' below where he improved the filter section
#  – calibrate_linear:
#  – 110 -> 0
#  – 95 -> 204
#  – 75 -> 544
#  – 55 -> 900
#  – 35 -> 1320
#  – 15 -> 1660
#  – 0 -> 1800

A few issues:

  • maxing out: The sensor is bouncing off a metal strut inside the tank so I need to relocate the sensor somewhere else. Because of this I keep my daily eye on the history graph, looking for a plateau that occurs when the sensor ‘sees’ the metal strut.
  • calibration: the sensor measures in metres so I have to calculate the litres. My tank has a perfectly regular shape from top to bottom – so by measuring dipstick depths, I found that 100cm of tank depth contains 105.3 litres. In the ‘MkII’ code below the last two lines (filter and lamda) do this calibration. The two lines can be edited or entirely omitted to suit your tank.
  • the readings were a bit noisy. To counteract this I calculate the sliding average of an oil level reading.
  • it was suggested that the ‘oil used’ graph should fall, not rise, so I made the reading negative. This also makes knowing how much oil to order easier. I should name the sensor ‘oil used’ because the number it returns is the amount of oil I need to buy to fill the tank. Next I really don’t need an oil level reading every 10 seconds so I increased the update interval.

so when do I need to order some more oil?

The first nice consequence is that I now know how much oil I need to order and how much I use. In a cold spell two households used 30 litres a day – I’ll be posting my metrics in a future post. And I can see I would need to order an extra 80 litres as it will take say, 9 days to arrive. As I write I’m thinking about a way to predict the date when I’ll need to order more oil. For the present I’ve put the data in a spreadsheet and it looks like I’ll order in mid January – unless I start to use oil faster.

notify me when more oil needs to be ordered

I order 900 litres of oil and I use 100 litres of oil as I wait for it to be delivered. Therefore I need an alert when the tank has 850 – 100 or 750 litres of oil. Home Assistant offers numerous ways to alert you:

  • if you use SMTP email (ie your own non-gmail email) this is the easiest notification to set up. It can work with Gmail after a fiddle. I get Home Assistant to send me an email (‘SMTP platform’) for all sorts of alerts. I prefer these less-naggy alerts. I use a filter on incoming mail so that this mail avoids my inbox.
  • you may prefer to see a message in your browser or see a pop-up a ‘toast’ message on a phone. You can set a notification message to persist so that you can’t easily swipe it away. You can also set a notification to be urgent enough to wake up a phone to tell you, or set one that waits quietly until you next pick up the phone.

Here is the automation code. I found the easiest notification method to get working is the email notification. The next easiest may be via the Home Assistant mobile app.

 # TIME TO ORDER OIL AUTOMATION - HOME ASSISTANT

 alias: heating oil level alert
 trigger:
 above: '750'
 entity_id: sensor.oil_level
 platform: numeric_state
 condition:
 after: 07:00
 condition: time
 action:
 data:
   message: at {{ states ('sensor.time') }}
   title: '{{ states (''sensor.oil_level'') }} L now. 850L oil to be ordered'
 service: notify.notify_html5
 data:
   message: at {{ states ('sensor.time') }}
   title: '{{ states (''sensor.oil_level'') }} L now. 850L oil to be ordered'
 service: notify.email 

record your daily readings in a Google spreadsheet using IFTTT

Improvement 1

For a while it was OK to manually record the daily oil level in a spreadsheet. I soon saw that I was using 80 – 100 litres of oil a week during December. I therefore set up a connection between Home Assistant and IFTTT so that the daily level could be recorded in a spreadsheet. This has been happening daily all through to 2022 and hasn’t broken yet. The code for the automation routine that takes the reading is below. My procedure to link Home Assistant and IFTTT is on this page: send sensor data from Home Assistant to a spreadsheet

# THE FOLLOWING IS A TEXT FILE CALLED oil_level.yaml in the folder AUTOMATIONS 
# that is found at /config/automations/oil_level.yaml
id: '12399999'
alias: oil_level_spreadsheet
description: daily readings in oil_level.yaml
trigger:
at: "23:55:00"
platform: time
condition: []
action:
data_template: { "event": "oil_level", "value1": "{{ states('sensor.oil_level')}}", "value2": "{{ states('sensor.temperature_weather')}}", "value3": "{{ states('sensor.power_today')}}" }
service: ifttt.trigger

Improvement 2 June 2022 and December 2022

The sensor is a great help for ordering oil. But it may also be useful to see your oil readings as kW (power) and kWh (energy). This needs some changes to the code to add some entity properties. I’ve guessed we can get 10kW from 1 litre of oil (that’s generous on boiler efficiency.

Notice that the code now includes two filters to remove noise from the readings. We take 20 readings over 5 minutes. The quantile filter is said to remove ‘outliers’ (the readings that are wild). Having removed the outliers we use a rolling/sliding average of those 20 readings. Results shown below.

IN ESPHOME - revised code for ESP32
# the sensor
sensor:
  - platform: vl53l0x
    name: "oil kW"
    address: 0x29
    id: myoillevel
    update_interval: 15s
    long_range: false
    accuracy_decimals: 0
    unit_of_measurement: "kW"
    filters:
  # ALTERNATIVE CALCULATION: - lambda: return x * 1018 * 10; # change 10 to 9 for 90% efficiency however this may affect the level calculation 
 
    - lambda: return (x - 0.034) * 1018 * 10;

    - quantile:
        window_size: 20
        send_every: 5
        send_first_at: 2
        quantile: .95    
    - sliding_window_moving_average:
        window_size: 20
        send_every: 5    
    state_class: total_increasing
    device_class: 'power'

# no negative above to get KW of oil. negative below to get oil level below in the template .
  - platform: template
    name: "oil_level"
#    update_interval: 60s
    accuracy_decimals: 0
    unit_of_measurement: "L"
    lambda: |-
      return -0.1 * id(myoillevel).state ;


Compile and install the code above in place of the code earlier in this post.

before and after adding the sliding average filter – see the code section above.
# TRY WITHOUT BUT YOU MIGHT ADD THIS TO THE END OF CUSTOMIZE.YAML

sensor.oil_energy_24h:
  last_reset: 2022-07-16T23:00:00.023678+00:00

add your oil meter to the Home Assistant energy dashboard

The result of the above is to give you entities sensor.oil kW (which tells you little) and sensor.oil_energy_24h which tells how much oil is used each day.

water level graphs. Noisey time of flight sensor
this is the setup on my WATER (not oil) tank

11 Responses

  1. Dennis says:

    Do you find that this sensor is Noisy for you? I’ve extended out the number of reads to once a minute, average for an hour and it still a little up and down.

    • roger says:

      Thanks for writing in. YES! I’ve added a picture of the output to the post. It is noisy but I’ll return to fix this when my urgent (water level) situation has passed! Let’s know id your config improves on mine.

      • Dennis says:

        Thanks for the Reply. I’m currently tweaking window size and the sample frequency.

        I’ll have a few tests over a few days and will come back and update this, tonights test is

        # to stop reading for a predefined interval
        deep_sleep:
        run_duration: 10s
        sleep_duration: 60min

        and this is in the sensor field
        – sliding_window_moving_average:
        window_size: 10
        send_every: 10

        I want it to give me an average of the 10 seconds it is awake; but the last log test i just did indicates that it is keeping track of that last 10 checks somewhere else.

        • roger says:

          Hello Dennis

          Thank you for adding more truth to the idea that this is a noisy sensor. We can’t exclude the idea that the platform / ESP32 might be part of it but no matter, you’ve found a solution for a still very useful sensor.

          GRAPH SMOOTHING / NOISE
          Well done for finding ‘quantile’ as it seems to more suit what I’m looking for by omitting outliers. I’ll note that you’re taking a fantastic number of readings and doing the maths over 15 minutes – a duration which suits our slow use of heating oil.

          In Home Assistant I used a Statistics graph which helped – but by averaging.

          CALIBRATION
          My tank is old-style metal and rectangular. I will tweak the fiddle-factor when it gets topped up for the first time. Just now the figures are good enough to know how much heating oil is used daily and how much to order.

          INDENTING YAML CODE
          This web page messes up indents – go to esphome.io to see how to do it: https://esphome.io/components/sensor/index.html

          • roger says:

            UPDATE:
            Cheers for the idea to use a quantile filter and sliding_window_moving_average on the oil tank ‘tof’ sensor.

            In an update of my post, near the bottom, you’ll see that I’ve stacked one sensor after the other and the effect is pretty good … or much improved.
            The Esphome quantile code removes outliers and this is followed by your second filter: sliding_window_moving_average.
            Thank you for the pointers that led to this.

        • roger says:

          SLEEP
          I recall a report somewhere that mentioned what happens during sleep – in any case good luck there Dennis.

          BATTERY POWER
          If you’ve switched over to battery power pls let us all know if battery life is reasonable.

  2. Dennis says:

    Changed this over to use quantile which looks like it really smooths out the curve. Still a bit of noise but looking much better.
    I have the device on usb power so i take a lot more samples.
    Also, had a different take on measurement. Using the tank’s maximum capacity and a simple formula tells you how much oil is in the tank

    sensor:
    – platform: vl53l0x
    name: “oil_level”
    address: 0x29
    id: myoillevel
    update_interval: 1s
    long_range: false
    accuracy_decimals: 1
    unit_of_measurement: “Gallons Remaining”
    filters:
    – lambda: return 330.0 – x/1.10*330.0;
    # – lambda: return x * 1053;
    – quantile:
    window_size: 900
    send_every: 900
    send_first_at: 900
    state_class: measurement

  3. Tyeth says:

    Was creating similar for my mum, then found she had an old Watchman ultrasonic, but I’d planed to use the VL53L0X/1X. Thought I’d combine with the lilygo T-QT Pro, but only for the visual feedback during setup. Found some sensors that had a nice cover on aliexpress, doubt they’re tested for oil fumes but what I ended up with: https://www.aliexpress.com/item/1005003091941068.html (2nd and 3rd variants)

    • roger says:

      Thank you Tyeth for the link! I’ve incorporated your find in the post. I agree that this cover, or its mount, needs long term testing against oil fumes – in an earlier project using a sonar distance sensor all of the ABS plastic of that sensor has dissolved. One ray of hope is that the PLA plastic used to print a holder mount HAS not perished. I continue to look for a glass/metal casing.

      And the use of a Lilygo T-display does indeed make it mum-friendly! That’s good. Those displays are excellent and these days there’s code to have a slideshow of a graph and another output.
      (My alert is an email that sends daily / when level is lower – I’m hoping for an automation to put an oil order date in the Google calendar but the maths and coding waits for another lockdown).

  4. Stuart says:

    Hi Roger

    Great write up !

    with regards to Current / power reduction / optimisation when on battery.

    Does the Vl53L0X board uses standby current when the ESP is in deep sleep? looking at this website “https://www.instructables.com/WiFi-Oil-Tank-Monitor” they use a transistor connected to one of the outputs, thus perhaps an adaptation of this for ESPhome which turns on a dedicated pin to the base of the transistor, hopefully when in deep sleep the pin will be off and transistor not charged?

    Do you think this will work?

    esphome:
    name: oil-sensor
    on_boot:
    then:
    – delay: 5s
    – output.turn_on: sample_pin

    output:
    – platform: gpio
    pin: 5
    id: sample_pin

    deep_sleep:
    run_duration: 30s
    sleep_duration: 2min

Leave a Reply

Your email address will not be published. Required fields are marked *