Zabbix and Pure Storage monitoring V2

At the end of 2017, I’ve shared a post talking about Zabbix and Pure Storage monitoring. At this time we were using it as a POC. A few weeks later, we were so happy with the product that we decided to use it in production.

So it was an opportunity to rewrite my monitoring script from scratch.
It’s now time to share it with you.
If you want more information about the purpose, the way I’ve made the integration, etc. don’t hesitate to read my previous post as the only modifications are the code and the type of information the script can gather from the array.

Instead of writing a script for each part of the monitoring (volumes, array, disks,…), I’ve put everything in one file called “PureStorageMonitoring.py”.
The dependencies are still the same : purestorage and urllib3 libraries from Python and Python 3.x

A complete list of datas and triggers that will be gathered by the script is available at the end of this post.

Important note : This script has been tested with Zabbix 3.4.x and 4.0.x and Pure Storage 4.10.x and 5.1.x. I cannot guarantee that it will work fine with any other version even if I made my best regarding this point. The template available is exported from Zabbix 4.0.x so you will have to delete some settings to import it in Zabbix 3.4.x don’t hesitate to contact me in case of trouble.

Setup the proxy or server

To gather data, we need to make some works on the Zabbix Proxy or Zabbix Server.

  • Be sure that you have Python 3.x installed on the server
  • Install the pip packages
pip3 install purestorage urllib3 py-zabbix

Get API Token

To be able to use the REST API of Pure Storage, you have to find your API Token on the web interface.
Go on “SYSTEM”, then “Users” and under the Users menu, you will find “API Tokens”. Click on the three dots on the right of the screens and use “Show API Token” to get your API Token.

Download the scripts

The script is available on my Github repository.
You can download it on the server using the following steps :

  • Find the directory where you have to download it, by looking at the value of “ExternalScripts” setting of your proxy or server (by default it’s “/usr/local/share/zabbix/externalscripts” but if like me you are using the official Debian packages from Zabbix, it will be “/usr/lib/zabbix/externalscritps”
  • Go to this directory
cd /usr/lib/zabbix/externalscripts
  • Download the scripts
wget https://raw.githubusercontent.com/jeremyverda/zabbix-public/master/Pure%20Storage/PureStorageMonitoring.py
  • Set the rights
chmod +x PureStorageMonitoring.py
chown zabbix:zabbix PureStorageMonitoring.py

Import the Zabbix template

You can find the Zabbix template on the Github repository.
You have to import it in Zabbix before you continue.

Configure the Zabbix host

In the Zabbix frontend, add a new host and set the information as follows :

  • Agent interface IP : Fill in with the IP address of your Pure Storage
  • Macros
    • {$PURE_TOKEN_API} : Fill in with the API Token that you have copy previously
    • {$PURE_ZBX} : WIll be the IP of your Zabbix Proxy or server that will be used by the Zabbix Sender. By default it will be 127.0.0.1
  • Monitored by proxy : Set this field if you are going to use a Zabbix proxy
  • Template : select the previously imported template : “Template Pure Storage”

Wait for the data

Once you have completed all the previous steps, you have to wait until the data are gathered by the script.

Data and triggers

Here is the list of data and triggers the script will gather from your Flash Array :

Data

  • Global
    • latencies
    • read and write
    • data reduction ratio
    • array capacity, volume size, used space and shared space
    • version
    • remote assist
    • phone home
    • hostname
    • etc.
  • Volumes
    • size and used size
    • read and write
    • latencies
    • data reduction
    • etc.
  • Hosts (value corresponding to VMware or any other hosts connected to the array)
    • latencies
    • read and write
  • Disks
    • capacity
    • last failure (currently this value won’t be empty but it will be corrected on a future API and Purity version)
    • protocol
    • status
    • type

Triggers

  • Global
    • Phone home is disabled
    • Remote assist is enabled
  • Volume
    • Volume size has changed
  • Disks
    • Error on disk status

This list will be updated for each data or trigger that will be added in the template. Don’t hesitate to ask me to add something if you need it.

Update

  • 01/11/2018 :
    • Replace the Zabbix Sender by a Python Library
    • Increase the delay between each check
    • Get more value at each check instead of requiring more check to get the same amount of values

33 thoughts on “Zabbix and Pure Storage monitoring V2”

  1. Is there a way to test that the python script is working correctly. I ran it manually, passing the values it would have gotten from the template, and it just returns 0. A debug mode would be nice so that we can confirm that it is connecting and getting data.

    1. Hi Andrew,

      I will check what I can do to allow a debug mode.
      By the way, which version of Purity are you running on?

  2. Hi, good job for this zabbix template….We use it but we notice that the python script consume the maximum cpu…
    It discover, all disks, all volumes and then test these volumes and it consume too much CPU….

    How can we calm down the execution of the script ?

    1. Hello,

      I think I could optimize the way I use the Zabbix sender by calling it one time in each function instead of multiples times for each.

      I have to work on that, give me some time and I will let you know.

      If you want, you can send me an e-mail using the contact form and I will contact you to test the fix.

      Before I can fix it, you can extend the time between each check on the template, it won’t reduce the CPU usage but the regularity of the spike.

    2. Hello,

      Could you contact me using the contact form?
      I’ve made a fix that I would like to send you to confirm if it’s working in your case.
      I tried to contact you using the mail you let on the comment form but the e-mail server tells me that your address couldn’t be found.

      Thanks.

  3. I’ve made an update to the script and publish it.
    I’ve made some change like removing the Zabbix Sender by using a Python Library that does the same job.

  4. Hello, before thanks a lot for your script, it’s perfect !!!!
    I tried the new one but I’ve got a lot of error in the zabbix-server.log :
    error reason for “xxxxxxx:PureStorageMonitoring.py[“hostMonitoring”,{HOST.IP},{$PURE_TOKEN_API},XXXXX,{HOST.HOST},{$PURE_ZBX}]” changed: Value “Traceback (most recent call last):
    File “/app/zabbix/externalscripts/PureStorageMonitoring.py”, line 434, in pure_host_monitoring
    File “/app/zabbix/externalscripts/PureStorageMonitoring.py”, line 558, in
    File “/app/zabbix/externalscripts/PureStorageMonitoring.py”, line 445, in pure_host_monitoring
    item “xxxxxxx:PureStorageMonitoring.py[“volumeMonitoring”,{HOST.IP},{$PURE_TOKEN_API},XXXVOLUMEXXX,{HOST.HOST},{$PURE_ZBX}]” became not supported: Timeout while executing a shell script.

    the right are good, old version work perfect but not this one.
    thanks for your help and again for your job!

  5. Hello,

    Thanks for your feedback. Glad to see it’s usefull for you.

    I see that I’ve missed something in the how-to. It’s the instruction to set the Timeout value on the server or proxy side.
    I think that the scripts takes more than 3 seconds to run on your side. Do you have a lot of hosts and datastores?

    You should increase the value of the Timeout variable on your proxy or server configuration file and restart it.
    You can find more information here : https://www.zabbix.com/documentation/4.0/manual/appendix/config/zabbix_proxy

    Let me know if it solve your issue.

  6. When I try to use this script I haven’t been able to get it to do anything.

    I see this error in the discovery page “Value should be a JSON object.”

    This shows up in the zabbix_server.log
    4143:20181129:143227.441 error reason for “[HOSTNAME]:PureStorageMonitoring.py[“arrayMonitoring”,{HOST.IP},{$PURE_TOKEN_API},{HOST.HOST},{$PURE_ZBX}]” changed: Value “/bin/env: python3: No such file or directory” of type “string” is not suitable for value type “Numeric (unsigned)”
    4159:20181129:143228.767 error reason for “[HOSTNAME]:PureStorageMonitoring.py[“diskMonitoring”,{HOST.IP},{$PURE_TOKEN_API},{HOST.HOST},{$PURE_ZBX}]” changed: Value “/bin/env: python3: No such file or directory” of type “string” is not suitable for value type “Numeric (unsigned)”
    4147:20181129:143229.893 error reason for “[HOSTNAME]:PureStorageMonitoring.py[“hostMonitoring”,{HOST.IP},{$PURE_TOKEN_API},{HOST.HOST},{$PURE_ZBX}]” changed: Value “/bin/env: python3: No such file or directory” of type “string” is not suitable for value type “Numeric (unsigned)”
    4147:20181129:143230.899 error reason for “[HOSTNAME]:PureStorageMonitoring.py[“volumeMonitoring”,{HOST.IP},{$PURE_TOKEN_API},{HOST.HOST},{$PURE_ZBX}]” changed: Value “/bin/env: python3: No such file or directory” of type “string” is not suitable for value type “Numeric (unsigned)”
    4158:20181129:143250.224 error reason for “[HOSTNAME]:PureStorageMonitoring.py[“arrayInfo”,{HOST.IP},{$PURE_TOKEN_API},{HOST.HOST},{$PURE_ZBX}]” changed: Value “/bin/env: python3: No such file or directory” of type “string” is not suitable for value type “Numeric (unsigned)”

    1. Hello,

      It seems there is an issue with the shebang at the start of the script.

      Do you have python3 installed?
      Which OS are you using?

      Thanks.

  7. Hello,

    Can you check the first line of the script and check if it’s : #!/usr/bin/env python3

    If it’s the case, can you try to run the following command on your proxy and give me the result : /usr/bin/env python3

    Thanks.

    1. Hello,

      I’m not really sure about that but I don’t think so.
      But as we are not using LDAP integration on the Pure that we are installing I could not give you an accurate response.

      I’m not calling any endpoint in the API that create configuration change so I think that a standard user that can see the graph on the webui could use the API.

      If you have any issue, let me know, I will try to help you.

  8. Getting -bash: ./PureStorageMonitoring.py[diskMonitoring,{10.10.10.10},{xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxxx},{Pure02-xxxxxx},{127.0.0.1}]: No such file or directory
    I have checked the script but I can’t find where there is a reference to a file except for the #!/usr/bin/env python3
    Python3 lib is present in /usr/bin/

    /usr/bin/env python3
    Python 3.4.9 (default, Feb 5 2019, 14:36:09)
    [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux
    Type “help”, “copyright”, “credits” or “license” for more information.

    >

    1. Hello,

      I can see that you are running on Red Hat.
      Could you check that the file is stored on the directory set in your proxy or server config under the variable : ExternalScripts

      I think that the value on Red Hat is not the same as Debian/Ubuntu.

      At the first look, it’s the only things I can think of.

      Let me know if it solve your issue.

      Thanks.

  9. We keep running into the same issue as David. We extended our timeout to the max 30 seconds in the Zabbix server config. Ball park 7 data stores, 5 hosts, Ubuntu 18.04, Zabbix 4.0.7

    1. Hello,

      I did not made my test with this kind of setup, so it’s possible that the timeout is reached.
      I’ve plan to make some change to use a cron or supervisor setup instead of an ExternalScript.
      It will allow to resolve some of those performance and timeout issue.

      I don’t know when it will be done but I will let you know.

  10. I was having this same problem. I was able to solve by performing the following steps:

    python setup.py install

    pip install purestorage

    Edited file PureStorageMonitoring.py

    leaving the first line as follows: #! / usr / bin / env python

  11. Hello,

    I have followed your steps and appear to have array information but do not see any volume, host or disk information yet.

    Any assistance in troubleshooting this would be appreciated.

    Patrick

  12. Please disregard my previous comment. The data is now showing correctly.
    What I would like to know is if you can assist me in creating an item and trigger for the percentage of volume capacity.

    Thanks,

    Patrick

    1. Hello,

      Which kind of triggers do you need?
      For a specific volume of for all volumes?
      Identical percentage for all volumes or not?

      Thanks.

  13. I wanted to create an item and trigger showing the percent free on the volume. I was able to create these manually but had hoped to be able to integrate it into the discovery.

  14. Hi Jérémy

    I would like to thank you for your contribution, I was able to successfully monitor my Pure environment.

    I inform you that my environment is with Zabbix 4.2.5 and Pure 5.1.9, on a server with Cent OS 7.

    For the future, would it be possible to monitor the network controllers?

    1. Hello,

      You are welcome!

      Can you explain me (you can use the contact form if you prefer to discuss by e-mail) what you mean by “Network controllers” on the Pure Storage side?
      I’ve planned a rewrite of the code to use a background system instead of calls made by Zabbix. It will allow me to improve some part of the code and allow me to add some modules if needed.

      Thanks.

  15. Hi,

    I have added your script and template to our zabbix. But we are getting these errors;

    error reason for “xxxx:PureStorageMonitoring.py[“diskMonitoring”,{HOST.IP},{$PURE_TOKEN_API},{HOST.HOST},{$PURE_ZBX}]” changed: Value of type “string” is not suitable for value type “Numeric (unsigned)”. Value “Traceback (most recent call last):
    File “/usr/lib/zabbix/externalscripts/PureStorageMonitoring.py”, line 531, in pure_disk_monitoring
    File “/usr/lib/zabbix/externalscripts/PureStorageMonitoring.py”, line 562, in
    File “/usr/lib/zabbix/externalscripts/PureStorageMonitoring.py”, line 542, in pure_disk_monitoring

    Any help would be great.

    Thanks

    1. Hello,
      Could you try to run the command directly from your proxy/server as the traceback from Zabbix seems to be truncated?
      You should run the following command: python3 PureStorageMonitoring.py diskMonitoring PURE_IP PURE_TOKEN ZABBIX_HOST_HOSTNMAE ZABBIX_PROXY_OR_SERVER_IP
      By example: python3 PureStorageMonitoring.py diskMonitoring 192.168.1.80 bafp4aCOZRDXAfwVsVzU PURE01 127.0.0.1

      You can send me a private message using the contact form if needed.

      Thanks.

  16. Hi,

    Sorry for the delay in responding to your message. I have ran the command a couple of times and double checking the information.

    The box is Pure Storage FlashArrary Purity FA with version 5.3.5.

    root@ldn-zabbix:/usr/lib/zabbix/externalscripts# python3 PureStorageMonitoring.py diskMonitoring ef8462be-a5bb-90ef-xxxx-xxxxxxxxxxxx ldn-t1
    Traceback (most recent call last):
    File “PureStorageMonitoring.py”, line 531, in pure_disk_monitoring
    data.send(metrics)
    File “/usr/local/lib/python3.8/dist-packages/pyzabbix/sender.py”, line 443, in send
    result.parse(self._chunk_send(metrics[m:m + self.chunk_size]))
    File “/usr/local/lib/python3.8/dist-packages/pyzabbix/sender.py”, line 420, in _chunk_send
    raise err
    File “/usr/local/lib/python3.8/dist-packages/pyzabbix/sender.py”, line 408, in _chunk_send
    connection.connect(host_addr)
    ConnectionRefusedError: [Errno 111] Connection refused

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
    File “PureStorageMonitoring.py”, line 562, in
    pure_disk_monitoring()
    File “PureStorageMonitoring.py”, line 542, in pure_disk_monitoring
    data.send(metrics)
    File “/usr/local/lib/python3.8/dist-packages/pyzabbix/sender.py”, line 443, in send
    result.parse(self._chunk_send(metrics[m:m + self.chunk_size]))
    File “/usr/local/lib/python3.8/dist-packages/pyzabbix/sender.py”, line 420, in _chunk_send
    raise err
    File “/usr/local/lib/python3.8/dist-packages/pyzabbix/sender.py”, line 408, in _chunk_send
    connection.connect(host_addr)
    ConnectionRefusedError: [Errno 111] Connection refused
    root@ldn-zabbix:/usr/lib/zabbix/externalscripts#

    I did check our firewall logs and there were blocks between the proxy and the pure storage box. But we do HTTPS on the pure box. If that makes any difference.

    Any more advice, will be great.

    Thanks

  17. Hi,

    Just an update.

    I have managed to get it to work. The cause the problem was the zabbix sender port in pyzabbix. Once i changed it to the correct port it is working brilliantly now.

    Thanks for your help.

  18. Thanks for the script and template, everything worked with Pure Storage FlashArrary Purity FA with version 5.3.10, CentOS 8.3 and Zabbix 5.2.

    In the first attempt I had created the Token API via https://pure1.purestorage.com/ but it was not valid for this purpose, the script always returned value 0 and in zabbix the error “PureHTTPError status code 400 returned by REST version 1.18 at 10.XXX: BAD REQUEST {“msg”: “Invalid api_token”}” . I generated the token via Pure on premise and everything worked.

  19. Hello Jeremy,
    I search for a trigger to prevent us when a FC port was down
    Have you develop this functionality ?
    Thank you very much !

    1. Hello Jullien,

      It’s not possible with the current template.
      Maybe you should try to use something like the generic SNMP network template from Zabbix and customize it : https://git.zabbix.com/projects/ZBX/repos/zabbix/browse/templates/net/generic_snmp

      I plan to rewrite my integration and I will check if it’s something I can do using the REST API. But as we are using only the ISCSI version of the array I hope it could work also for FC.
      But for the moment I don’t have any ETA regarding this rewrite.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.