Deploying VerneMQ to CapRover
I wanted to get a VerneMQ instance up an running quickly to run a sensor network at home.
The source for this experiment can be found in this repo.
Deployment
The structure of your project should look like this:
.
├── Dockerfile
├── README.md
├── captain-definition
└── vernemq.conf
You need to create a captain-definition file with the following content:
{
"schemaVersion": 2,
"dockerfilePath" :"./Dockerfile"
}
Then your Dockerfile should look like this:
FROM vernemq/vernemq
FROM erlio/docker-vernemq
ARG MQTT_AUTH_PASS
ARG MQTT_AUTH_USER
ENV DOCKER_VERNEMQ_LOG__CONSOLE=console \
DOCKER_VERNEMQ_ALLOW_ANONYMOUS=on
COPY vernemq.conf /vernemq/etc/vernemq.conf.local
EXPOSE 1883/tcp \
8080/tcp \
8883/tcp \
8888/tcp \
4369/tcp \
9100-9109/tcp \
44053/tcp
WORKDIR /vernemq
RUN touch vmq.tmp && \
echo "$MQTT_AUTH_PASS\n$MQTT_AUTH_PASS" >> vmq.tmp && \
cat vmq.tmp | vmq-passwd -c /vernemq/vmq.passwd "$MQTT_AUTH_USER"
The vernemq.conf file is just modifying the very end of the file by adding this:
########## Start ##########
accept_eula=yes
log.console=console
erlang.distribution.port_range.minimum = 9100
erlang.distribution.port_range.maximum = 9109
listener.tcp.default = 0.0.0.0:1883
listener.ws.default = 0.0.0.0:8080
listener.vmq.clustering = 0.0.0.0:44053
listener.http.metrics = 0.0.0.0:8889
########## End ##########
Before you can deploy you will need to create the following env vars in your dashboard:
MQTT_AUTH_PASS=<YOUR_PASSWORD>
MQTT_AUTH_USER=<YOUR_USER_NAME>
Then you need to enable port mapping for these ports:
- 1883
- 8080
- 8883
Server Configuration
You will need to enable access to ports adding new rules to firewall:
ufw allow 1883,4369,8080,8883/tcp
Testing
Once you have all this done you should be able to reach out the admin panel at:
https://<YOUR_DOMAIN>/status
To query from inside your host:
curl "http://8lsnCUp60rQxKyNp1QgGY5smInsYPF6r@srv-captain--vernemq/api/v1/session/show"
If you want to query remotely:
curl "http://8lsnCUp60rQxKyNp1QgGY5smInsYPF6r@vernemq.peper.one/api/v1/session/show"
* Trying 138.68.42.127...
* TCP_NODELAY set
* Connected to vernemq.peper.one (148.69.52.137) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: CN=vernemq.peper.one
* start date: Apr 17 19:26:31 2019 GMT
* expire date: Jul 16 19:26:31 2019 GMT
* subjectAltName: host "vernemq.peper.one" matched cert's "vernemq.peper.one"
* issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
* SSL certificate verify ok.
* Server auth using Basic with user '8lsnCUp60rQxKyNp1QgGY5smInsYPF6r'
> GET /api/v1/session/show HTTP/1.1
> Host: vernemq.peper.one
> Authorization: Basic OGxzbxNVcxYwclF4S3lOcDFRZ0dZNXNtSW5zWxBGNnI6
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx/1.17.9
< Date: Fri, 17 Apr 2020 21:08:17 GMT
< Content-Type: application/json
< Content-Length: 27
< Connection: keep-alive
<
* Connection #0 to host vernemq.peper.one left intact
{"table":[],"type":"table"}* Closing connection 0
To test subscription:
mosquitto_sub -p 1883 -h 'vernemq.peper.one' -t 'lightbuddy/#'
Test publishing;
mosquitto_pub -p 1883 -h 'vernemq.peper.one' -t 'lightbuddy/test' -m 'Hello world'
Configuration
You can check the default configuration file here:
/vernemq/etc/vernemq.conf
Debuging
docker service inspect srv-captain--vernemq --pretty
ID: ny4peioey381a4m87xm5aiuup
Name: srv-captain--vernemq
Service Mode: Replicated
Replicas: 1
UpdateStatus:
State: completed
Started: 2 hours ago
Completed: 2 hours ago
Message: update completed
Placement:
UpdateConfig:
Parallelism: 0
On failure: pause
Monitoring Period: 5s
Max failure ratio: 0
Update order: start-first
RollbackConfig:
Parallelism: 1
On failure: pause
Monitoring Period: 5s
Max failure ratio: 0
Rollback order: stop-first
ContainerSpec:
Image: vernemq/vernemq
Env: DOCKER_VERNEMQ_ACCEPT_EULA=yes DOCKER_VERNEMQ_ALLOW_ANONYMOUS=on DOCKER_VERNEMQ_PLUGINS__VMQ_ACL=off DOCKER_VERNEMQ_PLUGINS__VMQ_PASSWD=off
Resources:
Networks: captain-overlay-network
Endpoint Mode: vip
Ports:
PublishedPort = 1883
Protocol = tcp
TargetPort = 1883
PublishMode = ingress
PublishedPort = 1883
Protocol = udp
TargetPort = 1883
PublishMode = ingress
PublishedPort = 4369
Protocol = tcp
TargetPort = 4369
PublishMode = ingress
PublishedPort = 4369
Protocol = udp
TargetPort = 4369
PublishMode = ingress
PublishedPort = 8080
Protocol = tcp
TargetPort = 8080
PublishMode = ingress
PublishedPort = 8080
Protocol = udp
TargetPort = 8080
PublishMode = ingress
PublishedPort = 8883
Protocol = tcp
TargetPort = 8883
PublishMode = ingress
PublishedPort = 8883
Protocol = udp
TargetPort = 8883
PublishMode = ingress
lsof -i -P -n | grep LISTEN
dockerd 900 root 31u IPv6 25584 0t0 TCP *:2377 (LISTEN)
dockerd 900 root 37u IPv6 25602 0t0 TCP *:7946 (LISTEN)
dockerd 900 root 50u IPv6 280068 0t0 TCP *:3000 (LISTEN)
dockerd 900 root 87u IPv6 6083241 0t0 TCP *:1883 (LISTEN)
dockerd 900 root 95u IPv6 6083297 0t0 TCP *:4369 (LISTEN)
dockerd 900 root 97u IPv6 6083347 0t0 TCP *:8080 (LISTEN)
dockerd 900 root 99u IPv6 6083403 0t0 TCP *:8883 (LISTEN)
docker-pr 10345 root 4u IPv6 282696 0t0 TCP *:443 (LISTEN)
docker-pr 10357 root 4u IPv6 282723 0t0 TCP *:80 (LISTEN)
sshd 22403 root 3u IPv4 71412 0t0 TCP *:22 (LISTEN)
sshd 22403 root 4u IPv6 71423 0t0 TCP *:22 (LISTEN)
systemd-r 23587 systemd-resolve 13u IPv4 108945 0t0 TCP 127.0.0.53:53 (LISTEN)
Resources: