BLUF: Is there a “datadog-for-home”?
I’ve a ton of stuff running at home ranging from a 5-node PI cluster with various containers running on them including things like self-written python scripts doing “super important” stuff, node-red running everything about my energy setup, pfsense, TrueNAS etc.
Logging is painful and I’ve just lost about 4 hours trying to find a fault which stopped car charging. Lots of rabbit holes were entered as I currently don’t have an end-to-end logging solution.
What does everyone else use?
You must log in or register to comment.
I tried some tools and the one that I am currently using is OpenObserve, it’s light has a very good compression and is simple to manage, as an observability platform I think that open observe has some features that can be used instead of datadog like log injestion and open telemetry traces
Also using Graylog, dead simple to set up with rsyslog and at work we even use the sidecar for window logs.
Try BlackESK https://github.com/amitn322/blackesk
Splunk
Anyone else looking at openobserve. Looks OK for homelab, but not really stable
What do you mean by not stable? It’s in use in production by hundreds of organizations.
They state in their documentation that the software is alpha (https://openobserve.ai/docs/ OpenObserve is currently in alpha, but don’t let that stop you from trying it out.) . To be honest I didn’t bother to investigate why ingesting data stops working after a few days, might be my installation then.
I’m very curious which organisation uses alpha software in production
Just write your logs as files on a centralized syslog server with good file structure and you’ll be good.
You may really underestimate how fast and convenient grep+less combo is in comparison to webui-based solutions.
s/grep/ripgrep 😉
It’s a hybrid solution but I prefer putting my logs with an S3 provider, it’s just cheap storage that I don’t have to care about. And there are a lot of tools to do it with, like loki for example.
Another vote for graylog, runs well with opensearch instead of elastic.
I use the ELK stack (Elasticsearch, Kibana, Beats)
https://github.com/openobserve/openobserve . Built in rust - No JVM. Much lighter than the alternatives mentioned here and with extremely good UI. Beautiful dashboards. Could even run on raspberry pi.
Graylog
A simple syslog server is ready.
Look into ELK stack. I personally hate it but if you can learn it there’s a chance you can build a career on it.
Anyone here got a solution working in Unraid? I looked into this recently but got a bit stuck.
Depending on your existing monitoring stack, some options might be:
- Grafana Loki
- Sentry can be self-hosted for application logging
- Logstash is self-explanatory, use with other parts of Elastic’s software like Kibana for visualization.
ELK