measure heating oil usage with a Vl53L0X time of flight sensor and display on an energy dashboard

25 Responses

  1. John says:

    What do the values 1018 or 330 stand for?
    – lambda: return (x – 0.034) * 1018 * 10;
    Can you explain this line, please?
    x = the distance the sensor measures
    0.034 = some offset like sensor packaging ???
    1018 = ??
    10 = ??
    Where do you input the tank dimensions for the calculation?

    • roger says:

      Thanks for writing in with a question. I’ve a few posts on this topic and the explanation* for the factors hasn’t carried over to each post.
      x = the distance the sensor measures
      0.034 = some offset because values do not reach zero on a calibration graph – this is the intercept
      1018 = a tiny fiddle factor specific to my tank based on a few tank top-up refills. It’s 1.018 and because I want litres it’s 1.018 * 1000
      10 = convert litres to kW. It should be 9kW per litre but I used 10kW.

      * https://www.rogerfrost.com/water-or-oil-tank-level-mounting-a-vl53l0x-time-of-flight-sensor/

      There’s a new post coming in the next few days on my success to add Oil consumption to a HA energy dashboard

  2. Markus says:

    Is this example actually measuring oil? or not
    Did you show an example of how oil can be used with water?

    • roger says:

      Is this example actually measuring oil? Yes, oil. In litres.
      Did you show an example of how oil can be used with water? No. I use the same equipment to monitor the level of a water tank.

  3. Stuart says:

    Hi Roger

    Great write up !

    with regards to Current / power reduction / optimisation when on battery.

    Does the Vl53L0X board uses standby current when the ESP is in deep sleep? looking at this website “https://www.instructables.com/WiFi-Oil-Tank-Monitor” they use a transistor connected to one of the outputs, thus perhaps an adaptation of this for ESPhome which turns on a dedicated pin to the base of the transistor, hopefully when in deep sleep the pin will be off and transistor not charged?

    Do you think this will work?

    esphome:
    name: oil-sensor
    on_boot:
    then:
    – delay: 5s
    – output.turn_on: sample_pin

    output:
    – platform: gpio
    pin: 5
    id: sample_pin

    deep_sleep:
    run_duration: 30s
    sleep_duration: 2min

  4. Tyeth says:

    Was creating similar for my mum, then found she had an old Watchman ultrasonic, but I’d planed to use the VL53L0X/1X. Thought I’d combine with the lilygo T-QT Pro, but only for the visual feedback during setup. Found some sensors that had a nice cover on aliexpress, doubt they’re tested for oil fumes but what I ended up with: https://www.aliexpress.com/item/1005003091941068.html (2nd and 3rd variants)

    • roger says:

      Thank you Tyeth for the link! I’ve incorporated your find in the post. I agree that this cover, or its mount, needs long term testing against oil fumes – in an earlier project using a sonar distance sensor all of the ABS plastic of that sensor has dissolved. One ray of hope is that the PLA plastic used to print a holder mount HAS not perished. I continue to look for a glass/metal casing.

      And the use of a Lilygo T-display does indeed make it mum-friendly! That’s good. Those displays are excellent and these days there’s code to have a slideshow of a graph and another output.
      (My alert is an email that sends daily / when level is lower – I’m hoping for an automation to put an oil order date in the Google calendar but the maths and coding waits for another lockdown).

  5. Dennis says:

    Changed this over to use quantile which looks like it really smooths out the curve. Still a bit of noise but looking much better.
    I have the device on usb power so i take a lot more samples.
    Also, had a different take on measurement. Using the tank’s maximum capacity and a simple formula tells you how much oil is in the tank

    sensor:
    – platform: vl53l0x
    name: “oil_level”
    address: 0x29
    id: myoillevel
    update_interval: 1s
    long_range: false
    accuracy_decimals: 1
    unit_of_measurement: “Gallons Remaining”
    filters:
    – lambda: return 330.0 – x/1.10*330.0;
    # – lambda: return x * 1053;
    – quantile:
    window_size: 900
    send_every: 900
    send_first_at: 900
    state_class: measurement

  6. Dennis says:

    Do you find that this sensor is Noisy for you? I’ve extended out the number of reads to once a minute, average for an hour and it still a little up and down.

    • roger says:

      Thanks for writing in. YES! I’ve added a picture of the output to the post. It is noisy but I’ll return to fix this when my urgent (water level) situation has passed! Let’s know id your config improves on mine.

      • Dennis says:

        Thanks for the Reply. I’m currently tweaking window size and the sample frequency.

        I’ll have a few tests over a few days and will come back and update this, tonights test is

        # to stop reading for a predefined interval
        deep_sleep:
        run_duration: 10s
        sleep_duration: 60min

        and this is in the sensor field
        – sliding_window_moving_average:
        window_size: 10
        send_every: 10

        I want it to give me an average of the 10 seconds it is awake; but the last log test i just did indicates that it is keeping track of that last 10 checks somewhere else.

        • roger says:

          SLEEP
          I recall a report somewhere that mentioned what happens during sleep – in any case good luck there Dennis.

          BATTERY POWER
          If you’ve switched over to battery power pls let us all know if battery life is reasonable.

        • roger says:

          Hello Dennis

          Thank you for adding more truth to the idea that this is a noisy sensor. We can’t exclude the idea that the platform / ESP32 might be part of it but no matter, you’ve found a solution for a still very useful sensor.

          GRAPH SMOOTHING / NOISE
          Well done for finding ‘quantile’ as it seems to more suit what I’m looking for by omitting outliers. I’ll note that you’re taking a fantastic number of readings and doing the maths over 15 minutes – a duration which suits our slow use of heating oil.

          In Home Assistant I used a Statistics graph which helped – but by averaging.

          CALIBRATION
          My tank is old-style metal and rectangular. I will tweak the fiddle-factor when it gets topped up for the first time. Just now the figures are good enough to know how much heating oil is used daily and how much to order.

          INDENTING YAML CODE
          This web page messes up indents – go to esphome.io to see how to do it: https://esphome.io/components/sensor/index.html

          • roger says:

            UPDATE:
            Cheers for the idea to use a quantile filter and sliding_window_moving_average on the oil tank ‘tof’ sensor.

            In an update of my post, near the bottom, you’ll see that I’ve stacked one sensor after the other and the effect is pretty good … or much improved.
            The Esphome quantile code removes outliers and this is followed by your second filter: sliding_window_moving_average.
            Thank you for the pointers that led to this.

  7. Gareth says:

    Fantastic stuff!

    Please can you share your code for the entity ‘Oil used since 00’ or is this just created in the statistics graph card?

    After a lot of headache – I couldn’t get ESPHome Add on to flash my device on my Rpi3B, so had to learn how to install it on a PC via the command line, but nearly there!

    Also tried to get a VL53L1X sensor working – unfortunately ESPHome doesn’t support it natively. The instructions for adding custom/external repositories are pretty incomprehensible without prior programming knowledge! So I’ve given up for now. A 0x with long range enabled is fine for me 1.4m tank.

    Many thanks

    • roger says:

      Well done for finding a way to flash it Gareth. I’ve flashed many and conclude that the process is fickle eg some boards need a retry or button press. The upload of a bin file using ESPHome web on a PC is my first thing to try.
      ***
      A link from my post above shows how complicated my thing has become over some years – in fact this blog is my only way to recall what I have done!
      ***
      Pity about your sensor ..1X and yes I don’t go near external libraries without a tutorial. But don’t give up as you need a tank monitor. The following is a tempting polished version that may motivate you to plough on with ESPHome ‘WATCHMAN SENSIT DOMESTIC OIL TANK MONITOR/ GAUGE’

      ***
      When the HA energy dashboard appeared I converted my litres into kWh (multiply times 9) and this made the ‘oil used since midnight’ redundant. However you might create a ‘template sensor’ with this name as below. Get your head around the fact that you use todays midnight level but store this just before midnight as ‘yesterday’s oil level’.

      IN BOLD HERE IS A GUESS AT HOW TO CALCULATE USE SINCE MIDNIGHT. ADD THIS TO THE MAIN configuration.yaml
      >>>
      template:
      – sensor:
      name: oil_used_since00
      unique_id: oil_used_since00
      state: >-
      {{ states(‘input_number.midnight_oil_level’) | int – states(‘sensor.oil_level’) | int }}
      >>>>

      ***
      AS OF NOW THIS IS MY FINAL CODE for an AUTOMATION that runs around midnight. Shout if I can help more.

      alias: OIL – oilrecorder
      description: save midnight level – all based on input number and current level
      trigger:
      – platform: time
      at: “23:55:05”
      action:
      – service: input_number.set_value
      data_template:
      entity_id: input_number.oilusedyesterday
      value: >-
      {{ states(‘input_number.midnight_oil_level’) | int –
      states(‘sensor.oil_level’) | int }}
      – service: input_number.set_value
      data_template:
      entity_id: input_number.oil_level_yesterday
      value: “{{ states(‘input_number.midnight_oil_level’) | int }} ”
      – data:
      event: oil_level
      value1: “{{ states(‘sensor.oil_level’)}}”
      value2: “{{ states(‘sensor.temperature_weath’)}}”
      value3: “{{ states(‘input_number.oilusedyesterday’)}}”
      service: ifttt.trigger
      – service: input_number.set_value
      data_template:
      entity_id: input_number.midnight_oil_level
      value: “{{ states(‘sensor.oil_level’) | int }} ”
      – service: notify.mail_roger
      data:
      message: >-
      oil report at {{ states (‘sensor.time’) }} from boiler room. Order oil
      in {{ states (‘sensor.how_many_days_till_i_order’) }} days on date {{
      (as_timestamp(now()) +
      (states(‘sensor.how_many_days_till_i_order’)|float*24*3600)) |
      timestamp_custom(“%d /%m”, True) }}. Presently you can order {{ states
      (‘sensor.how_much_oil_do_i_order_today’) }} litre DIFFERENCE in oil
      readings {{ states (‘sensor.oil_level’) |int – states
      (‘sensor.oil_levelx’) |int }} oil level TOF {{ states
      (‘sensor.oil_level’) }} oil level JSR {{ states (‘sensor.oil_levelx’)
      }}
      title: oil {{ states (‘sensor.oil_level’) }} level
      mode: single

      • Gareth says:

        Many thanks Roger, I think I’ve got my head around that!

        Thankfully the VL53L0X is working great and incorporated into HA with some of your sensors and gauges but currently pointing at the kitchen cupboard! The adapters are currently being printed this minute (In PETG) – I need to print an extra 1.5″ BSP adapter which yours will be glued on top as there is already a hole in the tank to this specification.

        I’ve got an ESP32 Feather which has a built in battery connector. Once it’s all working (currently testing with a USB power bank) I’ll turn down the sensor refresh rate and order a 2000mAh battery which will also be housed in the waterproof case.

        I may also do the Kalman thing if needed.

        Nearly there, but still some work to do!

        • roger says:

          Great to hear you’re working some magic there. Look forward to hearing .. luckily no hurry for this season!

  8. Andy says:

    I have done exactly the same, starting a couple of years ago. Works. I filled the tank with ping-pong balls to ensure there was a reflective surface. Yes, I did check in advance that they didn’t dissolve. I also isolated the sensor from the tank with a microscope slip cover. I had a previous sensor and I think the sensor got killed by the atmosphere in the tank – not sure.

    The problem I have is that it appears the sensor locks up. Still testing. A reset solves the problem, maybe for a couple of days. I’m running an ESP8266 with battery and solar so only take a reading every four hours and then go to sleep. It ftps readings to another ESP8266 via wifi that runs as a web server. Currently looking at XSHUT to see if that solves the problem, but that means modifying the hardware (:

    For others who try similar.

    • roger says:

      Thank you for your report and fair warnings. I’m very impressed with your FTP and sleeping device solutions. And of course the ping pong balls – i need something floaty like this to sharpen up my noisy readings – instead of cleaning up the noise as I explained. I have two points that aim to help.
      * As shown I put my sensor in a project box and I can vouch for this.
      * Two years on, recently one of three sensors (in three tanks) shows an error in the ESPHome logs ‘Component vl53l0x.sensor is marked FAILED’ and I’ve tried several suggestions I read. I unfairly pointed the blame at the ESP8266. I eventually got to see this one – a water tank in the loft – to find condensation in the down-facing project box housing the sensor.
      I tediously re-made this sensor using an ESP32 and a more waterproof ‘a02yyuw’ sensor. One week on it’s promising (not perfect). My project write-up is here measure heating oil use (or water tank level) with a A02YYUW ultrasonic sensor
      All the best there – with thanks and cheers.

      • I have had issues in the past with the i2c bus locking up on a weather station with a BME280. Found some code on the web to reset it which is incorporated in my code, but as far as I can see that has not been my problem. Before I take a reading I check to see if the the VL53LOX is there. The issue is that it will lock up. Log (which it sends to my NAS):

        [1725825624], Sun 08 Sep 2024, 20:00 – Oil tank readings taken
        [1725825639], Sun 08 Sep 2024, 20:00 – Failed reading=65535
        [1725825640], Sun 08 Sep 2024, 20:00 – Failed reading=65535
        [1725825641], Sun 08 Sep 2024, 20:00 – Failed reading=65535
        [1725825642], Sun 08 Sep 2024, 20:00 – Failed reading=65535

        I do have a version with an esp32, which is still in the garage – haven’t had the time or inclination to take it further yet. It’s better on power consumption, could not get the esp8266 sleep modes working reliably so it uses modem sleep = 25mA, the esp32 with light sleep is 10mA. It might be because I’m cycling it in and out of sleep that is causing the problem.

        FTP – there are a number of libraries you can download, but try finding an FTP client (that works), I ended up writing my own. As well as what I previously said, it also emails me – restart, low oil level or rapid change in level. No credit there, something I found.

        I have OTA installed so can update if needed, okay but when you find you need to modify the hardware…

        While I enjoy messing around with arduino projects, the reliability of components and trying to anticipate and trap all the failure modes is a PITA.

        You need a LOT of ping pong balls.

        Happy to talk further via email if desired.

        Andy

        • roger says:

          Impressed with your expertise there. I’m coming back to this ahead of the heating coming on in October. I’m stalling because the sensor is made never to fall apart or be repaired. Such is the optimism with which we build these things.
          Love them ping pong balls.

          • Yep! I don’t have the luxury of a 3D printer so the sensor is a length of 50mm pipe and pipe flanges, waterproof JB on top. This bolted to the top of the tank. The VL53LOX is araldited to the bottom of the pipe, 4 wires coming up to a connection block. Didn’t think at the time I’d need access to the XSHUT line. This in the hope that toggling that line can reset the sensor and stop lockup. So I’m going to have to build a new sensor unit. Few other jobs first. I’m also going to switch from an ESP8266 to an ESP32.

  9. Roger says:

    From: Robert Cargill
    xcitonbio.com
    Seeking dTOF reference design help
    Hello Roger,
    I’m looking for help finding a time-of-flight sensor solution. Given your interests, I thought you might be able to help. My ideal part is the ST Micro VL53L8CX, but I need to have a separate laser so that I can customize it. I could also solve my problem if I could remove and replace optical filters on the receiver. Do you have any thoughts?
    Thank you,
    Robert

    REPLY
    I hope someone here can help Robert – post a comment below and it will be sent on.

    Hello Robert
    Good luck with your quest. I normally stumble upon sensors by luck so your query is outside my expertise. When I found the TOF sensor I noticed that ST Micro had lots of variants of a TOF sensor. The electronics forums / reddit / stackexchange are good at finding what you ask.

    FYI I’ve been replacing my TOF sensors with the A02YYUW ultrasonic sensor. It’s used on an oil tank and a water tank. This sensor works at close range and while it’s not perfect, is not so noisy. Incidentally the larger transmitter and receiver parts of this sensor could possibly be separated

    Roger F
    http://www.rogerfrost.com

    Roger F

Leave a Reply

Your email address will not be published. Required fields are marked *