Infinite color and LIFX bulbs

[chcore]

Hi @awawa-dev,

I am currently using HyperHDR with some LIFX bulbs, even though they are not technically supported. I do this by having HyperHDR write the RGB values to a named pipe, and use an external program to continuously read from it. This program converts the RGB values to HSBK, and then generates and sends UDP packets to the bulbs using the LAN protocol to update their colors: https://lan.developer.lifx.com/docs/packet-contents

My question is, will I need to modify this setup when I update to the next version that includes the new Infinite Color Engine? Would you have any clue as to whether these bulbs support an extended colorspace, similar to the Hue bulbs? There is some documentation on their color settings here: https://lan.developer.lifx.com/docs/representing-color-with-hsbk

Thank you for this excellent project

[awawa-dev]

Hi
If you've gotten this far, why not create your own driver for these lamps in HyperHDR? As a base, you can take "udpraw," driver which already communicates via UDP, sending raw colors (24bit RGB), and add the necessary header + converted colors. Instead of converting 24-bit colors to the palette supported by these lamps, use float3 for greater precision as an input. Our LED driver currently allows you to select the path (24-bit or float3) for particular device. If you need help with implementation, I'd be happy to help.

[chcore]

If you've gotten this far, why not create your own driver for these lamps in HyperHDR?

I certainly would, if I felt I had the skills to do so!

I was able to, with the help of chatgpt, slap together a python script and use a library (https://github.com/mclarkk/lifxlan) to generate the packets, but sadly I have very little coding ability otherwise

[chcore]

I think that even my color conversion process is flawed and possibly inaccurate, as I'm unsure about the "K" in "HSBK". I use the python colorsys module to convert the RGB to HSV and then just use a constant value for the K, but I'm sure that's not correct. I make up for it a little with color calibration in HyperHDR.

Considering the great care that I know you put into producing accurate colors in HyperHDR, any code I write would be woefully inadequate!

[awawa-dev]

It's a pretty extensive API, so maybe show me what you currently have in your script. Are the lamp addresses hard-coded, or do you do some network discovery each time?

[chcore]

Posted below.

I think probably only SetPower and SetColor are needed. No ACKs or responses needed for any SetColor packets.

The "duration" of SetColor may need to be configurable. I'd probably need to test more extensively to see whether it makes things smoother (especially with the new Infinite Color Engine).

[chcore]

I have a config.json that contains the IP and MAC addresses of each device. These do not change.

Apologies for the ugly code. It could be a lot simpler by just using the lifxlan library's methods to generate each packet in one go, but I was running into performance issues on my Pi 3. So I ended up storing re-usable parts of the packets in variables (the COMPUTED_* ones) and just inserting the color and duration information each time (though I'm undecided if I want to use the duration value, or just have it always set to zero for instant changes. I think the bulbs are limited to around 20 requests/sec).

import os
import sys
import pathlib
import socket
import colorsys
import json
import lifxlan
import time
import bitstring
import struct
import re
import functools

NUMBER_REGEX = re.compile("\d+")
START_COLOR = [43690.0, 65535.0, 65535, 9000]

counter = 0
saved = 0

def get_config():
    result = None
    config_file_path = "config/{}.json".format(sys.argv[1].strip())
    with open(config_file_path, "r", encoding = "utf-8") as f:
        result = json.loads(f.read())
    return result

def calc_min_rgb_total(config):
    return int(float(config["min_rgb_threshold"]) * (255 * 3))

def calc_set_color_zero_duration_threshold(config):
    return int(float(config["set_color_zero_duration_threshold"]) * 65535.0)

def create_bulbs(config):
    return [lifxlan.Light(bulb["mac"], bulb["ip"]) for bulb in config["bulbs"]]

def check_create_fifo(config):
    fifo_name = ".hyperion-{}".format(config["fifo"])
    fifo_path = pathlib.Path(fifo_name)
    if not fifo_path.exists():
        os.mkfifo(fifo_path)
    return fifo_path

def parse_hyperion_data(fifo_line):
    return tuple(map(int, re.findall(NUMBER_REGEX, fifo_line)))

def extract_rgb(hyperion_rgbs, idx):
    offset = idx * 3
    return (hyperion_rgbs[offset], hyperion_rgbs[offset + 1], hyperion_rgbs[offset + 2])

@functools.lru_cache(maxsize = 10000000)
def to_hsvk(rgb):
    hsvk = None
    if sum(rgb) < MIN_RGB_TOTAL:
        rgb = [0, 0, 0]

    try:
        hsv = colorsys.rgb_to_hsv(rgb[0], rgb[1], rgb[2])
        hsvk = (int(hsv[0] * 65535), int(hsv[1] * 65535), hsv[2] * 257, 6500)
    except IndexError:
        pass
    return hsvk

def little_endian(bs):
    shifts = [i*8 for i in range(int(len(bs)/8))]
    int_bytes_little_endian = [int(bs.uintbe >> i & 0xff) for i in shifts]
    packed_message_little_endian = b""
    for b in int_bytes_little_endian:
        packed_message_little_endian += struct.pack("B", b)
    return packed_message_little_endian

@functools.lru_cache(maxsize = 10000000)
def construct_payload(hsvk):
    color = b"".join(little_endian(bitstring.pack("16", field)) for field in hsvk)
    duration_part = COMPUTED_PAYLOAD_ZERO_DURATION if (hsvk[-2] < SET_COLOR_ZERO_DURATION_THRESHOLD) else COMPUTED_PAYLOAD_DURATION
    return COMPUTED_PAYLOAD_RESERVED + color + duration_part

def create_payload_part_reserved():
    return little_endian(bitstring.pack("8", 0))

def create_payload_part_duration(duration_ms):
    return little_endian(bitstring.pack("32", duration_ms))

def create_socket():
    sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    sock.setblocking(False)
    return sock

def close_socket(sock):
    sock.close()

def create_set_color_msg(bulb):
    return lifxlan.msgtypes.LightSetColor(bulb.mac_addr, bulb.source_id, seq_num=0, payload={"color": START_COLOR, "duration": 0}, ack_requested=False, response_requested=False)

def set_color(sock, bulbs, hyperion_rgbs, idx):
    rgb = extract_rgb(hyperion_rgbs, idx)
    if rgb != BULB_PREV_RGBS[idx]:
        hsvk = to_hsvk(rgb)
        if hsvk:
            payload = construct_payload(hsvk)
            sock.sendto(COMPUTED_HEADERS[idx] + payload, (bulbs[idx].ip_addr, bulbs[idx].port))
        BULB_PREV_RGBS[idx] = rgb


if __name__ == '__main__':
    global COMPUTED_HEADERS
    global COMPUTED_PAYLOAD_RESERVED
    global COMPUTED_PAYLOAD_DURATION
    global BULB_PREV_RGBS
    global MIN_RGB_TOTAL
    global SET_COLOR_ZERO_DURATION_THRESHOLD

    config = get_config()
    bulbs = create_bulbs(config)
    bulb_template_msgs = tuple([create_set_color_msg(bulb) for bulb in bulbs])
    BULB_PREV_RGBS = [None for bulb in bulbs]
    COMPUTED_HEADERS = tuple([msg.header for msg in bulb_template_msgs])
    COMPUTED_PAYLOAD_DURATION = create_payload_part_duration(int(config["set_color_duration_ms"]))
    COMPUTED_PAYLOAD_ZERO_DURATION = create_payload_part_duration(0)
    COMPUTED_PAYLOAD_RESERVED = create_payload_part_reserved()
    MIN_RGB_TOTAL = calc_min_rgb_total(config)
    SET_COLOR_ZERO_DURATION_THRESHOLD = calc_set_color_zero_duration_threshold(config)
    fifo = check_create_fifo(config)

    first_read = True
    bulb_idxes = list(range(len(bulbs)))
    print("Started with min RGB total: {}".format(MIN_RGB_TOTAL))
    while True:
        with open(fifo, "r") as f:
            sock = create_socket()
            eof_reached = False
            bulbs_prev_hsvk = [None for bulb in bulbs]

            while not eof_reached:
                data = f.readline()

                if first_read:
                    first_read = False
                    continue
                elif len(data) == 0:
                    eof_reached = True
                    continue

                hyperion_rgbs = parse_hyperion_data(data)
                _ = [set_color(sock, bulbs, hyperion_rgbs, idx) for idx in bulb_idxes]
            close_socket(sock)

[awawa-dev]

Could you test these installers? https://github.com/awawa-dev/HyperHDR/actions/runs/18266108329
make sure that the ip-mac pair in the HyperHDR configuration corresponds to the valid lamp. Also verify HyperHDR logs after you configure it. Currently there is no auto-discovery so you must put them manually.

[chcore]

I can add power off/on, that should be very easy compared to auto-discovery. But I think I can also blindly add auto-discovery if you are willing to test it.

Yes, will gladly test it.

[chcore]

Would it also be possible to add the SetColor "duration" value as a setting please? I think some specific smaller models of bulbs have problems keeping up with the number of requests per second, and this will allow for smooth transitions in those cases.

[chcore]

Would it also be possible to add the SetColor "duration" value as a setting please? I think some specific smaller models of bulbs have problems keeping up with the number of requests per second, and this will allow for smooth transitions in those cases.

I think we might also need to allow the user to change the "Refresh Time" as well. What I would do is set it to some value that the specific type of bulb can handle, then adjust the duration to match. Example: 10Hz refresh -> 100ms Duration.

[awawa-dev]

You set that timer via smoothing. Sorry but 10Hz is below the minimal limit I want to HyperHDR be used with.

[chcore]

You set that timer via smoothing. Sorry but 10Hz is below the minimal limit I want to HyperHDR be used with.

Oh for sure, 10Hz is not a good experience, so that is understandable (even 20Hz isn't amazing to my eyes). Being able to set the duration value to the amount of time between requests can improve this though.

[chcore]

Could you test these installers? https://github.com/awawa-dev/HyperHDR/actions/runs/18266108329

Was tossing up whether to raise another issue for this, but I think I ran into a possibly unrelated bug while testing. I have an APA102 strip with 300 LEDs, but I only use around 260 of them. Every time this version of HyperHDR starts up, it seems to light up at least some of the LEDs beyond the 260th one (I can't quite tell because it's behind the TV and I cannot easily take it down). These are set to white, and stay lit. I can only turn them off by temporarily increasing the LED count, then clearing all colors/effects manually.

[awawa-dev]

It's hard for me to imagine how precise SPI could cause this (it's more likely with Neopixel, noise can do that). So please double-check. Besides, you're not mentioning the driver.

[chcore]

It's hard for me to imagine how precise SPI could cause this (it's more likely with Neopixel, noise can do that). So please double-check. Besides, you're not mentioning the driver.

OK I may have jumped the gun. I cannot reproduce the problem with a clean config, so please disregard. (though I'm not sure how to answer your question about which driver, I would have said APA102 is the driver?)

[awawa-dev]

Could you test these installers: https://github.com/awawa-dev/HyperHDR/actions/runs/18435661148
https://github.com/awawa-dev/HyperHDR/actions/runs/18449012223
It will reset your current LIFX config so back up it first using export to json (general tab), but in theory it should support auto-discovery and power on/off.

[chcore]

Could you test these installers: https://github.com/awawa-dev/HyperHDR/actions/runs/18435661148 https://github.com/awawa-dev/HyperHDR/actions/runs/18449012223 It will reset your current LIFX config so back up it first using export to json (general tab), but in theory it should support auto-discovery and power on/off.

Thank you for the update. I can confirm that Power On and Power Off work great. I was not able to get discovery working. I thought there might be an issue with my network or device configuration, and there appears to be, as discovery with the lifxlan library didn't work for me either. It looks like there is at least one other person in #1309 who is able to test this one, though.

I will be able to play with it some more on the weekend.

[awawa-dev]

Hi
I captured the packet sent by the original LIFX mobile app. It seems they're using some undocumented header properties, or their documentation hasn't been updated. I coded our package identically, and the installers are here: https://github.com/awawa-dev/HyperHDR/actions/runs/18523552674

[chcore]

No devices found. The log contains the following:

[WEBSOCKET] (HyperAPI.cpp:1097) message: [{"command":"leddevice","ledDeviceType":"lifx","subcommand":"discover","tan":621}]
[LEDDEVICE_LIFX] (DriverNetLifx.cpp:283) devicesDiscovered: [{"devices":[],"ledDeviceType":"lifx"}]
[WEBSOCKET] (HyperAPI.cpp:1121) response: [{"devices":[],"ledDeviceType":"lifx"}]

But as stated in my other comment, this could just be an issue with my network.

[awawa-dev]

Does the mobile LIFX app can find the lamps?

[chcore]

Does the mobile LIFX app can find the lamps?

After playing with my router's settings, the LIFX app and HyperHDR can now discover both of my lamps. Works great!

[chcore]

Does the Lifx driver hold back on sending packets, or maybe throttle sending them, if the color has not changed since the last packet was sent? If not, would it be possible to do this and potentially save some CPU cycles and network traffic?

[awawa-dev]

It would be possible but at higher level for all drivers. Before that I want to make sure the Lifx driver works fine and changes can be merged with the master.

[chcore]

Great to hear, as this would be useful for less-powerful devices such as my pi 3.

I've got 4 Lifx lamps working now, with no issues.

[awawa-dev]

OK, thanks for the feedback and testing. I'll merge it this week then. Regarding this new/old functionality (it was there before the migration to Ininite Engine and is waiting to be moved there), it works at the instance and smoothing level: if smoothing reaches its target and the color no longer changes for all lamps, it stops sending new frames unless Continues Output is enabled. Although this is very rare in dynamic ambient lighting systems.