Build from source #71

gerhard · 2019-02-20T18:14:19Z

Add make ezs so that we can build .ez files for all deps & plugin with a single command.

The resulting files will be placed in ./tmp:

ls -l tmp/
total 992
-rw-r--r--  1 gerhard  staff   12548 20 Feb 18:02 accept-0.3.4.ez
-rw-r--r--  1 gerhard  staff  211232 20 Feb 18:02 prometheus-4.2.2.ez
-rw-r--r--  1 gerhard  staff   14053 20 Feb 18:02 prometheus_cowboy-0.1.7.ez
-rw-r--r--  1 gerhard  staff   21638 20 Feb 18:02 prometheus_httpd-2.1.10.ez
-rw-r--r--  1 gerhard  staff   10708 20 Feb 18:02 prometheus_process_collector-1.4.0.ez
-rw-r--r--  1 gerhard  staff  223943 20 Feb 18:02 prometheus_rabbitmq_exporter-3.7.20190220.ez

I've changed the version of this plugin to use the date for the patch part, feel free to change back if you disagree.

Notice that I've deleted the prometheus_rabbitmq_exporter.app.src, this is now generated by erlang.mk and dropped in ebin/prometheus_rabbitmq_exporter.app - does it look correct @deadtrickster?

Want to integrate with Docker & testing before merging.

This started as #63, see the discussion for more context.

@deadtrickster

The resulting files will be placed in ./tmp, e.g. ls -l tmp/ total 992 -rw-r--r-- 1 gerhard staff 12548 20 Feb 18:02 accept-0.3.4.ez -rw-r--r-- 1 gerhard staff 211232 20 Feb 18:02 prometheus-4.2.2.ez -rw-r--r-- 1 gerhard staff 14053 20 Feb 18:02 prometheus_cowboy-0.1.7.ez -rw-r--r-- 1 gerhard staff 21638 20 Feb 18:02 prometheus_httpd-2.1.10.ez -rw-r--r-- 1 gerhard staff 10708 20 Feb 18:02 prometheus_process_collector-1.4.0.ez -rw-r--r-- 1 gerhard staff 223943 20 Feb 18:02 prometheus_rabbitmq_exporter-3.7.20190220.ez I've changed the version of this plugin to use the date for the patch part, feel free to change back if you disagree. Notice that I've deleted the prometheus_rabbitmq_exporter.app.src, this is now generated by erlang.mk and dropped in ebin/prometheus_rabbitmq_exporter.app - does it look correct @deadtrickster? Want to integrate with Docker & testing before merging. This started as #63, see the discussion for more context.

@deadtrickster

This is what happens when I use `./rebar3 archive` instead of zip. Let me know if you can figure it out @deadtrickster - I'm happy to stick with zip.

This reverts commit 72ad2f2.

gerhard · 2019-02-20T18:15:57Z

cc @dcorbacho

tsloughter · 2019-02-20T19:49:09Z

We prefer it if rebar3 is not committed to repos.

deadtrickster · 2019-02-20T19:55:25Z

I'm ok with the date, wondering why, though.

Does patched rebar3 archive work for you?

@tsloughter

Thanks @tsloughter! You might want to check the following repos for rebar3: * https://github.com/deadtrickster/accept * https://github.com/deadtrickster/prometheus.erl * https://github.com/deadtrickster/prometheus-cowboy * https://github.com/deadtrickster/prometheus_process_collector

gerhard · 2019-02-21T17:19:24Z

I'm ok with the date, wondering why, though.

The last release, prometheus_rabbitmq_exporter v3.7.2.4, indicates to me that it requires RabbitMQ v3.7.2. But does it work with RabbitMQ v3.7.12? What about RabbitMQ v3.7.1? By using a date, the indication is that it works with any RabbitMQ version published on or after this date. Is that a fair assumption to make? Can you think of a better approach?

Maybe @michaelklishin can suggest a better versioning schema. @dumbbell might also be able to help.

Does patched rebar3 archive work for you?

No, it doesn't. 72ad2f2 uses rebar3_archive_plugin v0.0.2 as captured in rebar.config & fails as captured in the log file.

deadtrickster · 2019-02-21T17:46:13Z

But does it work with RabbitMQ v3.7.12? What about RabbitMQ v3.7.1?

Hmm, I explained that in the README and issues, guess not clear, but - it works up to most recent. Really simple.

I just realized you tried to use rebar to build the plugin. Never tried that yet.

@deadtrickster

The resulting Docker images are only tagged with the RabbitMQ minor version, since prometheus_rabbitmq_exporter is expected to work with the latest stable RabbitMQ, regardless which that is at time of building. When these images are built, since we are using the `--pull` flag, we will always get the latest FROM image version at time of building. Without any automation, manually updating this version to the latest upstream feels inefficient and quiet boring - that would explain the version gaps in https://hub.docker.com/r/deadtrickster/rabbitmq_prometheus/tags, so I'm thinking to not use the exact RabbitMQ version in the tag for this image. I am on-board with using the PROJECT_VERSION (e.g. 3.7.2.4) if you think that is better @deadtrickster. Deleted the pure variant, it's just a duplicate of Dockerfile since #54, prometheus_process_collector ends up in both image variants.

gerhard · 2019-02-25T17:06:47Z

@deadtrickster this is now ready for you. I've tested both deadtrickster/rabbitmq_prometheus:3.7 & deadtrickster/rabbitmq_prometheus:3.7-alpine Docker image, prometheus_rabbitmq_exporter works as expected:

gmake docker_run

curl -svL -o /dev/null localhost:15672/api/metrics
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 15672 (#0)
> GET /api/metrics HTTP/1.1
> Host: localhost:15672
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< content-encoding: identity
< content-length: 144853
< content-type: text/plain; version=0.0.4
< date: Mon, 25 Feb 2019 17:05:48 GMT
< server: Cowboy
<
{ [16219 bytes data]
* Connection #0 to host localhost left intact

deadtrickster · 2019-02-26T07:59:32Z

Thanks, that's amazing. do you think

gmake docker_run

curl -svL -o /dev/null localhost:15672/api/metrics

is a good candidate for Travis/Circle?

deadtrickster · 2019-02-26T08:10:53Z

hmm, ln -h is bsd specific?

gerhard · 2019-02-26T08:31:38Z

gmake docker_run

curl -svL -o /dev/null localhost:15672/api/metrics
is a good candidate for Travis/Circle?

I almost added a make target that runs an acceptance test on both containers, but then didn't want to extend the PR too far. Yes, adding it to your CI of choice is a great idea. I would use curl's --fail --fail-early flags.

hmm, ln -h is bsd specific?

Yes, it is. -n has the same effect and is cross-platform:

LN(1)                     BSD General Commands Manual                    LN(1)

NAME
     link, ln -- make links

SYNOPSIS
     ln [-Ffhinsv] source_file [target_file]
     ln [-Ffhinsv] source_file ... target_dir

...
     -h    If the target_file or target_dir is a symbolic link, do not follow it.  This is most useful with the -f option, to replace a symlink which may point to a directory.

...

     -n    Same as -h, for compatibility with other ln implementations.

deadtrickster · 2019-02-26T08:52:00Z

so the sequence is make ezs && make docker_build && make docker_push/run. Or I'm missing the shortcut?

gerhard · 2019-02-26T09:35:01Z

Single command, multiple targets: make distclean up app ezs docker_build docker_run

gerhard · 2019-02-26T09:39:15Z

Will try on the Linux workstation later on today and add a few improvements. I've only made it work. Will make it right next & eventually make it fast 😉

gerhard · 2019-02-27T09:43:32Z

This should be easy to reproduce and guaranteed to work:

docker run --interactive --tty --volume prometheus_rabbitmq_exporter:/prometheus_rabbitmq_exporter --workdir /prometheus_rabbitmq_exporter --name prometheus_rabbitmq_exporter elixir:1.8.1 bash
cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
NAME="Debian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

elixir -v
Erlang/OTP 21 [erts-10.2.4] [source] [64-bit] [smp:10:10] [ds:10:10:10] [async-threads:1] [hipe]
Elixir 1.8.1 (compiled with Erlang/OTP 21)

git clone https://github.com/deadtrickster/prometheus_rabbitmq_exporter.git
cd prometheus_rabbitmq_exporter
apt update && apt install -y zip # required to build prometheus_rabbitmq_exporter*.ez

time make ezs
real	3m6.491s
user	1m50.850s
sys	0m38.550s
# most of the time is spent downloading the rabbit repository

ls -lh tmp/*.ez
-rw-r--r-- 1 root root  13K Feb 27 12:57 tmp/accept-0.3.5.ez
-rw-r--r-- 1 root root 205K Feb 27 12:57 tmp/prometheus-4.2.2.ez
-rw-r--r-- 1 root root  14K Feb 27 12:57 tmp/prometheus_cowboy-0.1.7.ez
-rw-r--r-- 1 root root  21K Feb 27 12:57 tmp/prometheus_httpd-2.1.10.ez
-rw-r--r-- 1 root root  18K Feb 27 12:57 tmp/prometheus_process_collector-1.4.3.ez
-rw-r--r-- 1 root root 219K Feb 27 12:58 tmp/prometheus_rabbitmq_exporter-3.7.2.4.ez

Based on the above, to build all ezs on a Linux system with Erlang 21.2 & Elixir 1.8 installed (wget & zip are also required), all ezs can be built with a single command: make ezs

I've tested the ezs produced:

docker run --detach --volumes-from prometheus_rabbitmq_exporter --name test_prometheus_rabbitmq_exporter rabbitmq:3.7-management

docker exec --interactive --tty test_prometheus_rabbitmq_exporter bash

cp /prometheus_rabbitmq_exporter/prometheus_rabbitmq_exporter/tmp/*.ez /plugins/
chown -fR rabbitmq:rabbitmq /plugins/*
rabbitmq-plugins list
Listing plugins with pattern ".*" ...
 Configured: E = explicitly enabled; e = implicitly enabled
 | Status: * = running on rabbit@3340a91c770c
 |/
[  ] prometheus_rabbitmq_exporter      3.7.2.4
[  ] rabbitmq_amqp1_0                  3.7.12
[  ] rabbitmq_auth_backend_cache       3.7.12
[  ] rabbitmq_auth_backend_http        3.7.12
[  ] rabbitmq_auth_backend_ldap        3.7.12
[  ] rabbitmq_auth_mechanism_ssl       3.7.12
[  ] rabbitmq_consistent_hash_exchange 3.7.12
[  ] rabbitmq_event_exchange           3.7.12
[  ] rabbitmq_federation               3.7.12
[  ] rabbitmq_federation_management    3.7.12
[  ] rabbitmq_jms_topic_exchange       3.7.12
[E*] rabbitmq_management               3.7.12
[e*] rabbitmq_management_agent         3.7.12
[  ] rabbitmq_mqtt                     3.7.12
[  ] rabbitmq_peer_discovery_aws       3.7.12
[  ] rabbitmq_peer_discovery_common    3.7.12
[  ] rabbitmq_peer_discovery_consul    3.7.12
[  ] rabbitmq_peer_discovery_etcd      3.7.12
[  ] rabbitmq_peer_discovery_k8s       3.7.12
[  ] rabbitmq_random_exchange          3.7.12
[  ] rabbitmq_recent_history_exchange  3.7.12
[  ] rabbitmq_sharding                 3.7.12
[  ] rabbitmq_shovel                   3.7.12
[  ] rabbitmq_shovel_management        3.7.12
[  ] rabbitmq_stomp                    3.7.12
[  ] rabbitmq_top                      3.7.12
[  ] rabbitmq_tracing                  3.7.12
[  ] rabbitmq_trust_store              3.7.12
[e*] rabbitmq_web_dispatch             3.7.12
[  ] rabbitmq_web_mqtt                 3.7.12
[  ] rabbitmq_web_mqtt_examples        3.7.12
[  ] rabbitmq_web_stomp                3.7.12
[  ] rabbitmq_web_stomp_examples       3.7.12

rabbitmq-plugins enable prometheus_rabbitmq_exporter
Enabling plugins on node rabbit@3340a91c770c:
prometheus_rabbitmq_exporter
The following plugins have been configured:
  prometheus_rabbitmq_exporter
  rabbitmq_management
  rabbitmq_management_agent
  rabbitmq_web_dispatch
Applying plugin configuration to rabbit@3340a91c770c...
The following plugins have been enabled:
  prometheus_rabbitmq_exporter

started 1 plugins.

apt update && apt install -y curl
curl -sv -o /dev/null localhost:15672/api/metrics
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 15672 (#0)
> GET /api/metrics HTTP/1.1
> Host: localhost:15672
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< content-encoding: identity
< content-length: 144892
< content-type: text/plain; version=0.0.4
< date: Wed, 27 Feb 2019 13:02:25 GMT
< server: Cowboy
<
{ [102235 bytes data]
* Connection #0 to host localhost left intact

deadtrickster · 2019-02-27T10:53:29Z

I added make test. It work, but I'm not sure I manager test container in the best possible way. Also, despite @ it still produces some noise :-)

gerhard added 4 commits February 20, 2019 18:03

Capture rebar archive failure

72ad2f2

This is what happens when I use `./rebar3 archive` instead of zip. Let me know if you can figure it out @deadtrickster - I'm happy to stick with zip.

Revert "Capture rebar archive failure"

7b7980b

This reverts commit 72ad2f2.

Remove redundant comments from Makefile

7cf2cd7

gerhard added 2 commits February 21, 2019 18:25

Preserve the original versioning schema

e4adbbf

gerhard marked this pull request as ready for review February 25, 2019 17:01

deadtrickster merged commit e37018b into prometheus-erl:master Feb 26, 2019

gerhard deleted the build_from_source branch February 26, 2019 08:32

gerhard mentioned this pull request Feb 27, 2019

Improve make test target #72

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Build from source #71

Build from source #71

Uh oh!

gerhard commented Feb 20, 2019 •

edited

Loading

Uh oh!

gerhard commented Feb 20, 2019

Uh oh!

tsloughter commented Feb 20, 2019

Uh oh!

deadtrickster commented Feb 20, 2019

Uh oh!

gerhard commented Feb 21, 2019 •

edited

Loading

Uh oh!

deadtrickster commented Feb 21, 2019

Uh oh!

gerhard commented Feb 25, 2019

Uh oh!

deadtrickster commented Feb 26, 2019

Uh oh!

deadtrickster commented Feb 26, 2019

Uh oh!

gerhard commented Feb 26, 2019 •

edited

Loading

Uh oh!

deadtrickster commented Feb 26, 2019

Uh oh!

gerhard commented Feb 26, 2019 •

edited

Loading

Uh oh!

gerhard commented Feb 26, 2019

Uh oh!

gerhard commented Feb 27, 2019 •

edited

Loading

Uh oh!

deadtrickster commented Feb 27, 2019

Uh oh!

Uh oh!

Build from source #71

Build from source #71

Uh oh!

Conversation

gerhard commented Feb 20, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gerhard commented Feb 20, 2019

Uh oh!

tsloughter commented Feb 20, 2019

Uh oh!

deadtrickster commented Feb 20, 2019

Uh oh!

gerhard commented Feb 21, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

deadtrickster commented Feb 21, 2019

Uh oh!

gerhard commented Feb 25, 2019

Uh oh!

deadtrickster commented Feb 26, 2019

Uh oh!

deadtrickster commented Feb 26, 2019

Uh oh!

gerhard commented Feb 26, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

deadtrickster commented Feb 26, 2019

Uh oh!

gerhard commented Feb 26, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gerhard commented Feb 26, 2019

Uh oh!

gerhard commented Feb 27, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

deadtrickster commented Feb 27, 2019

Uh oh!

Uh oh!

gerhard commented Feb 20, 2019 •

edited

Loading

gerhard commented Feb 21, 2019 •

edited

Loading

gerhard commented Feb 26, 2019 •

edited

Loading

gerhard commented Feb 26, 2019 •

edited

Loading

gerhard commented Feb 27, 2019 •

edited

Loading