macOS Caching Server monitoring - Is it working, and how well?

[chmeisch]

Just thought I'd loop you guys on a nice tool. If you're here, you're running an MDM with probably a high amount of Apple devices. Your enterprise is then too probably running macOS caching servers to save on bandwidth etc.

Currently there's a couple widely used tools for monitoring your caching servers like ErikNG's Cacher (https://github.com/erikng/Cacher) and krypted's precache (https://github.com/krypted/precache). Just wanted to let you guys know about one more that I found particularly helpful.

Netdata (https://github.com/firehol/netdata) by ktsaou at Firehol is a free/great tool for monitoring all types of Linux/Unix type devices. It also can monitor some more basic functions of macOS machines. Recently he assisted me with writing a plugin to also help with monitoring the caching function on our caching servers. (https://github.com/firehol/netdata/issues/2766)

This in conjunction with prometheus and grafana, has allowed me to monitor historicals, create alerts when something's wrong, etc. I attached some images. If you are a sysadmin responsible for monitoring/managing some caching servers, some of this may be of great help to you.

[Taylor_Armstron]

That's beautiful....

[brandon_fisher]

I have 12 cache servers in 5-6 different physical locations, would it support that many?

[chmeisch · Accepted solution]

@brandon.fisher I'd look into creating a netdata master instance if I were you, then install netdata on each of the caching servers and have them stream their metric's back to the master. As long as you have a clear path for streaming w/o a firewall blocking you somewhere, you can definitely do it.

Check the new code here on github. - Also, I learned python and with some growing pains updated the plugin to run on that instead. It's using AssetCacheManagerUtil on 10.13 to get the info and barely touches the CPU for usage.

[lee_smith]

Being a novice, how does one go about installing this?

[chmeisch]

https://github.com/firehol/netdata/wiki/Installation#macos

If you're comfortable in command line, it should be a piece of cake. Netdata's wiki has pretty much all the info you'd need to get things going, especially coupled with the info from the above links.

[bluebox]

Hi All,

I just fired up netdata on our High Sierra caching server and it's indeed beautiful and really useful already out of the box. I'm wondering though if anyone updated the plugin already to High Sierra or if this is something yet to be done?

https://github.com/firehol/netdata/issues/2766

refers in all code to macOS Server and the serveradmin command not present in High Sierra's caching services...

Alex

[bluebox]

Ha, forgive me, overread chmeisch's reply... all good and kudos to chmeisch for updating the plugin for High Sierra...

Alex

[chmeisch]

@bluebox I could probably do better if I had time to revisit it.. it was my first foray into python and now I handle everything with it! Glad it's been useful for you!

[mhegge]

I am going through this and trying to install. Seems to have an issue with OpenSSL, which I did install:

In file included from libnetdata/simple_pattern/../libnetdata.h:304:
libnetdata/socket/security.h:20:12: fatal error: 'openssl/ssl.h' file not found

include <openssl/ssl.h>

^~~~~~~
In file included from In file included from web/api/formatters/charts2json.cweb/api/formatters/json_wrapper.c::33:
:
In file included from In file included from web/api/formatters/charts2json.hweb/api/formatters/json_wrapper.h::66:
:
In file included from In file included from web/api/formatters/rrd2json.hweb/api/formatters/rrd2json.h::66:
:
In file included from In file included from ./web/api/web_api_v1.h./web/api/web_api_v1.h::66:
:
In file included from In file included from ./daemon/common.h./daemon/common.h::66:
:
In file included from In file included from ./daemon/../libnetdata/simple_pattern/../libnetdata.h./daemon/../libnetdata/simple_pattern/../libnetdata.h::304304:
:
./daemon/../libnetdata/socket/security.h./daemon/../libnetdata/socket/security.h::2020::1212:: fatal errorfatal error: : 'openssl/ssl.h''openssl/ssl.h' filefile notnot

foundfound

include <openssl/ssl.h># include <openssl/ssl.h>

^~~~~ ^~~~~
In file included from web/api/formatters/rrdset2json.c:3:
In file included from web/api/formatters/rrdset2json.h:6:
In file included from web/api/formatters/rrd2json.h:6:
In file included from ./web/api/web_api_v1.h:6:
In file included from ./daemon/common.h:6:
In file included from ./daemon/../libnetdata/simple_pattern/../libnetdata.h:304:
./daemon/../libnetdata/socket/security.h:20:12: fatal error: 'openssl/ssl.h' file not found

include <openssl/ssl.h>

^~~~~~~
1 error generated.
make[2]: [libnetdata/os.o] Error 1
make[2]: Waiting for unfinished jobs....
1 error generated.
1 error generated.
1 error generated.
make[2]: [web/api/formatters/rrdset2json.o] Error 1
make[2]: [web/api/formatters/json_wrapper.o] Error 1
make[2]: [web/api/formatters/charts2json.o] Error 1
make[1]: [all-recursive] Error 1
make: * [all] Error 2 FAILED

FAILED

ABORTED netdata-installer.sh exited with error

---

References:
https://franz.com/support/openssl-mac.lhtml

https://docs.netdata.cloud/packaging/installer/#one-line-installation

[atomczynski]

Does this function with Mojave?

[mhegge]

Just seems to not like OpenSSL, even though I installed it. High Sierra. :(

```In file included from libnetdata/config/appconfig.c:3:
In file included from libnetdata/config/../simple_pattern/../libnetdata.h:304:
libnetdata/config/../socket/security.h:20:12: fatal error: 'openssl/ssl.h' file not found

include <openssl/ssl.h>

^~~~~~~
In file included from libnetdata/adaptive_resortable_list/adaptive_resortable_list.c:3:
In file included from libnetdata/adaptive_resortable_list/../simple_pattern/../libnetdata.h:304:
libnetdata/adaptive_resortable_list/../socket/security.h:20:12: fatal error: 'openssl/ssl.h' file not found

include <openssl/ssl.h>

^~~~~~~
1 error generated.
In file included from collectors/cups.plugin/cups_plugin.c:9:
In file included from collectors/cups.plugin/../../libnetdata/simple_pattern/../libnetdata.h:304:
collectors/cups.plugin/../../libnetdata/socket/security.h:20:12: fatal error: 'openssl/ssl.h' file not found

include <openssl/ssl.h>

^~~~~~~
In file included from libnetdata/avl/avl.c:3:
In file included from libnetdata/avl/../simple_pattern/../libnetdata.h:304:
libnetdata/avl/../socket/security.h:20:12: fatal error: 'openssl/ssl.h' file not found

include <openssl/ssl.h>

^~~~~~~
make[2]: [libnetdata/adaptive_resortable_list/adaptive_resortable_list.o] Error 1
make[2]: Waiting for unfinished jobs....
1 error generated.
make[2]: [libnetdata/config/appconfig.o] Error 1
1 error generated.
make[2]: [libnetdata/avl/avl.o] Error 1
1 error generated.
make[2]: [collectors/cups.plugin/cups_plugin.o] Error 1
make[1]: [all-recursive] Error 1
make: * [all] Error 2 FAILED

bash-3.2$ ```

[sdagley]

@mhegge As an FYI if you're pasting in a script or terminal output you should use the ``` (triple backtick) markup code before and after the text so the block doesn't get mangled by the forum software.

[mhegge]

```bash-3.2$ sudo /opt/local/bin/port -N install openssl +universal
Password:
---> Computing dependencies for openssl
---> Cleaning openssl
---> Scanning binaries for linking errors
---> No broken files found.
---> No broken ports found.
bash-3.2$ ```

[mhegge]

Truly not understanding this install....

[malarocca]

Anyone having luck getting this running?

[ChrisTech]

I got it running malarocca took me a few hours. I used homebrew to get netdata installed and then followed the directions based upon where homebrew installs them

[ChrisTech]

Install homebrew
Install netdata
Create caching.conf in /usr/local/etc/netdata/python.d/
Create caching.chart.py in /usr/local/opt/netdata/libexec/netdata/python.d/
Edit python.d.conf in /usr/local/opt/netdata/lib/netdata/conf.d/ as root, add caching: yes
Also to edit the netdata.conf in /usr/local/etc/netdata to bind to the caching server's IP instead of localhost