macOS Caching Server monitoring - Is it working, and how well?

chmeisch
New Contributor III

Just thought I'd loop you guys on a nice tool. If you're here, you're running an MDM with probably a high amount of Apple devices. Your enterprise is then too probably running macOS caching servers to save on bandwidth etc.

Currently there's a couple widely used tools for monitoring your caching servers like ErikNG's Cacher (https://github.com/erikng/Cacher) and krypted's precache (https://github.com/krypted/precache). Just wanted to let you guys know about one more that I found particularly helpful.

Netdata (https://github.com/firehol/netdata) by ktsaou at Firehol is a free/great tool for monitoring all types of Linux/Unix type devices. It also can monitor some more basic functions of macOS machines. Recently he assisted me with writing a plugin to also help with monitoring the caching function on our caching servers. (https://github.com/firehol/netdata/issues/2766)

This in conjunction with prometheus and grafana, has allowed me to monitor historicals, create alerts when something's wrong, etc. I attached some images. If you are a sysadmin responsible for monitoring/managing some caching servers, some of this may be of great help to you.

1a43a2d5e85a4e7c98c1ef7cdefd6abe
f9121a8dadad40a8adc6be261783b5ce
fd5a28a0f7fe4da591efb94d71af6e9c
92d40126d2704fdfa1fb0e8a105f1aad

1 ACCEPTED SOLUTION

chmeisch
New Contributor III

@brandon.fisher I'd look into creating a netdata master instance if I were you, then install netdata on each of the caching servers and have them stream their metric's back to the master. As long as you have a clear path for streaming w/o a firewall blocking you somewhere, you can definitely do it.

Check the new code here on github. - Also, I learned python and with some growing pains updated the plugin to run on that instead. It's using AssetCacheManagerUtil on 10.13 to get the info and barely touches the CPU for usage.

View solution in original post

17 REPLIES 17

Taylor_Armstron
Valued Contributor

That's beautiful....

brandon_fisher
New Contributor

I have 12 cache servers in 5-6 different physical locations, would it support that many?

chmeisch
New Contributor III

@brandon.fisher I'd look into creating a netdata master instance if I were you, then install netdata on each of the caching servers and have them stream their metric's back to the master. As long as you have a clear path for streaming w/o a firewall blocking you somewhere, you can definitely do it.

Check the new code here on github. - Also, I learned python and with some growing pains updated the plugin to run on that instead. It's using AssetCacheManagerUtil on 10.13 to get the info and barely touches the CPU for usage.

lee_smith
Contributor

Being a novice, how does one go about installing this?

chmeisch
New Contributor III

https://github.com/firehol/netdata/wiki/Installation#macos

If you're comfortable in command line, it should be a piece of cake. Netdata's wiki has pretty much all the info you'd need to get things going, especially coupled with the info from the above links.

bluebox
New Contributor III

Hi All,

I just fired up netdata on our High Sierra caching server and it's indeed beautiful and really useful already out of the box. I'm wondering though if anyone updated the plugin already to High Sierra or if this is something yet to be done?

https://github.com/firehol/netdata/issues/2766

refers in all code to macOS Server and the serveradmin command not present in High Sierra's caching services...

Alex

bluebox
New Contributor III

Ha, forgive me, overread chmeisch's reply... all good and kudos to chmeisch for updating the plugin for High Sierra...

Alex

chmeisch
New Contributor III

@bluebox I could probably do better if I had time to revisit it.. it was my first foray into python and now I handle everything with it! Glad it's been useful for you!

mhegge
Contributor III

I am going through this and trying to install. Seems to have an issue with OpenSSL, which I did install:

In file included from libnetdata/simple_pattern/../libnetdata.h:304:
libnetdata/socket/security.h:20:12: fatal error: 'openssl/ssl.h' file not found

include <openssl/ssl.h>

^~~~~~~
In file included from In file included from web/api/formatters/charts2json.cweb/api/formatters/json_wrapper.c::33:
:
In file included from In file included from web/api/formatters/charts2json.hweb/api/formatters/json_wrapper.h::66:
:
In file included from In file included from web/api/formatters/rrd2json.hweb/api/formatters/rrd2json.h::66:
:
In file included from In file included from ./web/api/web_api_v1.h./web/api/web_api_v1.h::66:
:
In file included from In file included from ./daemon/common.h./daemon/common.h::66:
:
In file included from In file included from ./daemon/../libnetdata/simple_pattern/../libnetdata.h./daemon/../libnetdata/simple_pattern/../libnetdata.h::304304:
:
./daemon/../libnetdata/socket/security.h./daemon/../libnetdata/socket/security.h::2020::1212:: fatal errorfatal error: : 'openssl/ssl.h''openssl/ssl.h' filefile notnot

foundfound

include <openssl/ssl.h># include <openssl/ssl.h>

^~~~~ ^~~~~
In file included from web/api/formatters/rrdset2json.c:3:
In file included from web/api/formatters/rrdset2json.h:6:
In file included from web/api/formatters/rrd2json.h:6:
In file included from ./web/api/web_api_v1.h:6:
In file included from ./daemon/common.h:6:
In file included from ./daemon/../libnetdata/simple_pattern/../libnetdata.h:304:
./daemon/../libnetdata/socket/security.h:20:12: fatal error: 'openssl/ssl.h' file not found

include <openssl/ssl.h>

^~~~~~~
1 error generated.
make[2]: [libnetdata/os.o] Error 1
make[2]:
Waiting for unfinished jobs....
1 error generated.
1 error generated.
1 error generated.
make[2]: [web/api/formatters/rrdset2json.o] Error 1
make[2]:
[web/api/formatters/json_wrapper.o] Error 1
make[2]: [web/api/formatters/charts2json.o] Error 1
make[1]:
[all-recursive] Error 1
make: * [all] Error 2 FAILED

FAILED

ABORTED netdata-installer.sh exited with error


References:
https://franz.com/support/openssl-mac.lhtml

https://docs.netdata.cloud/packaging/installer/#one-line-installation

atomczynski
Valued Contributor

Does this function with Mojave?

mhegge
Contributor III

Just seems to not like OpenSSL, even though I installed it. High Sierra. :(

```In file included from libnetdata/config/appconfig.c:3:
In file included from libnetdata/config/../simple_pattern/../libnetdata.h:304:
libnetdata/config/../socket/security.h:20:12: fatal error: 'openssl/ssl.h' file not found

include <openssl/ssl.h>

^~~~~~~
In file included from libnetdata/adaptive_resortable_list/adaptive_resortable_list.c:3:
In file included from libnetdata/adaptive_resortable_list/../simple_pattern/../libnetdata.h:304:
libnetdata/adaptive_resortable_list/../socket/security.h:20:12: fatal error: 'openssl/ssl.h' file not found

include <openssl/ssl.h>

^~~~~~~
1 error generated.
In file included from collectors/cups.plugin/cups_plugin.c:9:
In file included from collectors/cups.plugin/../../libnetdata/simple_pattern/../libnetdata.h:304:
collectors/cups.plugin/../../libnetdata/socket/security.h:20:12: fatal error: 'openssl/ssl.h' file not found

include <openssl/ssl.h>

^~~~~~~
In file included from libnetdata/avl/avl.c:3:
In file included from libnetdata/avl/../simple_pattern/../libnetdata.h:304:
libnetdata/avl/../socket/security.h:20:12: fatal error: 'openssl/ssl.h' file not found

include <openssl/ssl.h>

^~~~~~~
make[2]: [libnetdata/adaptive_resortable_list/adaptive_resortable_list.o] Error 1
make[2]:
Waiting for unfinished jobs....
1 error generated.
make[2]: [libnetdata/config/appconfig.o] Error 1
1 error generated.
make[2]:
[libnetdata/avl/avl.o] Error 1
1 error generated.
make[2]: [collectors/cups.plugin/cups_plugin.o] Error 1
make[1]:
[all-recursive] Error 1
make: * [all] Error 2 FAILED

bash-3.2$ ```

sdagley
Esteemed Contributor II

@mhegge As an FYI if you're pasting in a script or terminal output you should use the ``` (triple backtick) markup code before and after the text so the block doesn't get mangled by the forum software.

mhegge
Contributor III

```bash-3.2$ sudo /opt/local/bin/port -N install openssl +universal
Password:
---> Computing dependencies for openssl
---> Cleaning openssl
---> Scanning binaries for linking errors
---> No broken files found.
---> No broken ports found.
bash-3.2$ ```

mhegge
Contributor III

Truly not understanding this install....

malarocca
New Contributor

Anyone having luck getting this running?

ChrisTech
Contributor

I got it running malarocca took me a few hours. I used homebrew to get netdata installed and then followed the directions based upon where homebrew installs them

ChrisTech
Contributor

Install homebrew
Install netdata
Create caching.conf in /usr/local/etc/netdata/python.d/
Create caching.chart.py in /usr/local/opt/netdata/libexec/netdata/python.d/
Edit python.d.conf in /usr/local/opt/netdata/lib/netdata/conf.d/ as root, add caching: yes
Also to edit the netdata.conf in /usr/local/etc/netdata to bind to the caching server's IP instead of localhost
93ae267244db4bf88eafa616b77821c9