Difference between revisions of "Resource usage"
Jump to navigation
Jump to search
(Basic documentation of MQTT communication IO-rates and packet size statistic as well as packet content) |
|||
| (13 intermediate revisions by one other user not shown) | |||
| Line 1: | Line 1: | ||
| − | + | == Hardware Resources == | |
| + | === CPU === | ||
| + | The majority of CPU utilization associated with the muondetector-daemon is due to the signal transfer to and from the MuonPi HAT via the GPIO-pins. As of [https://github.com/MuonPi/muondetector/releases/tag/v2.0.2 v2.0.2], this is handled by the "pigpiod" process, which takes about 20% of the processor time according to [[wikipedia:top (software)|top]]. The muondetector-daemon itself, uses only ~2%. | ||
| + | == Network Resources == | ||
| + | If you wish to operate the MuonPi within a network, that is a company's or other organization's network, the admins might be interested what data this device gathers and transmits or receives. | ||
| + | Here we elaborate on how the MuonPi uses the network and how a typical data packet looks like. | ||
| − | |||
| − | |||
| − | |||
=== Data rate === | === Data rate === | ||
The MuonPi detector board will send measurement data to our data evaluation servers, the rate of which greatly depends on the exact event rate. | The MuonPi detector board will send measurement data to our data evaluation servers, the rate of which greatly depends on the exact event rate. | ||
In a proper setup these messages will average out to roughly 600 messages per minute with an average size of 110 bytes. | In a proper setup these messages will average out to roughly 600 messages per minute with an average size of 110 bytes. | ||
This however includes the acknowledgement packets which account for two thirds of the number of packets but they are only small in size (>150 bytes). | This however includes the acknowledgement packets which account for two thirds of the number of packets but they are only small in size (>150 bytes). | ||
| − | [[File:Muonpi-iorate.png |thumb|Purple:All messages send between the MuonPi unit and muonpi.org <br> | + | [[File:Muonpi-iorate.png |thumb|Purple: All messages send between the MuonPi unit and muonpi.org <br> |
Green: Only MQTT publish messages, excluding acknowledgement messages]] | Green: Only MQTT publish messages, excluding acknowledgement messages]] | ||
| − | [[File:Muonpi-packetsize.png|thumb|Histogram of | + | [[File:Muonpi-packetsize.png|thumb|Histogram of packet sizes over 45minutes]] |
| + | |||
| + | === Outgoing Ports === | ||
| + | {| class="wikitable" | ||
| + | |- | ||
| + | ! Source Port !! Purpose !! Destination | ||
| + | |- | ||
| + | | muonpi:51508 || GUI connection to other machine on network || othermachine:39830 | ||
| + | |- | ||
| + | | muonpi:41930 || MQTT message publishing to muonpi.org || muonpi.org:1883 | ||
| + | |} | ||
| − | + | (Connections to localhost are excluded here) | |
| − | + | ||
| − | + | As indicated by the table, the firewall has to allow outgoing connections to our servers on port 1883. | |
| + | Otherwise the muondetector-login command will fail to establish the data link, and no event data will be sent to contribute to the detector network. | ||
=== MQTT Packet === | === MQTT Packet === | ||
| − | Detector telemetry and | + | Detector telemetry and measurement data is send to our servers via the MQTT protocol. |
| − | Upon receiving a MQTT message from the client our servers respond with an MQTT acknowledgment packet which is in turn acknowledged by the client | + | Upon receiving a MQTT message from the client our servers respond with an MQTT acknowledgment packet which is in turn acknowledged by the client. |
| − | Generally two types of MQTT messages are | + | Generally two types of MQTT messages are sent: the event message and the log message. |
| + | ====Event message==== | ||
The event messages consist of the following: | The event messages consist of the following: | ||
{| class="wikitable" | {| class="wikitable" | ||
| Line 36: | Line 50: | ||
| MQTT message || 84 byte | | MQTT message || 84 byte | ||
|- | |- | ||
| − | | Total || 150 byte | + | | '''Total''' || '''150 byte''' |
|} | |} | ||
| − | |||
| − | Example data of multiple MQTT packets being send to our | + | The only privacy sensitive information contained in these messages are the geolocation and ambient temperature. |
| + | |||
| + | Example data of multiple MQTT packets being send to our servers: | ||
{| class="wikitable" | {| class="wikitable" | ||
|- | |- | ||
| − | ! # !!MQTT topic !! timestamp (rising edge) !! timestamp (falling edge) !! | + | ! # !!MQTT topic !! timestamp (rising edge) !! timestamp (falling edge) !! - !! Event counter !! - !! - !! - |
|- | |- | ||
| − | | 1 || muonpi/data/ | + | | 1 || muonpi/data/<user>/<detectorID> || 1611698307.895514697 || 1611698307.895515885 || 70 || 35109 || 1 || 2 || 1 |
|- | |- | ||
| − | | 2 || muonpi/data/ | + | | 2 || muonpi/data/<user>/<detectorID> || 1611698307.912153060 || 1611698307.912154247 || 70 || 35110 || 1 || 2 || 1 |
|- | |- | ||
| − | | 3 || muonpi/data/ | + | | 3 || muonpi/data/<user>/<detectorID> || 1611698308.237550951 || 1611698308.237552160 || 70 || 35111 || 1 || 2 || 1 |
|- | |- | ||
| − | | 4 || muonpi/data/ | + | | 4 || muonpi/data/<user>/<detectorID> || 1611698308.404480192 || 1611698308.404481380 || 70 || 35112 || 1 || 2 || 1 |
|- | |- | ||
| − | | 5 || muonpi/data/ | + | | 5 || muonpi/data/<user>/<detectorID> || 1611698308.699026112 || 1611698308.699027299 || 70 || 35113 || 1 || 2 || 1 |
|- | |- | ||
| − | | 6 || muonpi/data/ | + | | 6 || muonpi/data/<user>/<detectorID> || 1611698309.048519516 || 1611698309.048520724 || 70 || 35114 || 1 || 2 || 1 |
|- | |- | ||
| − | | 7 || muonpi/data/ | + | | 7 || muonpi/data/<user>/<detectorID> || 1611698309.133318449 || 1611698309.133319657 || 70 || 35115 || 1 || 2 || 1 |
|} | |} | ||
| + | |||
| + | These event messages are sent every time the unit detects a signal above the threshold. | ||
| + | It contains the UNIX timestamp of the rising and the falling edge of the signal and the number of the event. | ||
| + | The MQTT topic includes the username and the detectorID as registered with us, and allows us to associate the incoming data on our servers to your detector. | ||
==== Log message ==== | ==== Log message ==== | ||
| − | The log message contains various | + | The log message contains various metadata about your MuonPi setup. |
| + | Most of the time, it consist of multiple MQTT messages packed into a single TCP packets, but it can sometimes also be separated into two packets. | ||
| − | + | An exemplary log message along with a description of each entry: | |
{| class="wikitable" | {| class="wikitable" | ||
|- | |- | ||
| Line 89: | Line 109: | ||
| muonpi/log/user/0 || 2021-01-26_22-08-43 || freqAccuracy || 265.0000000 || ps/s || | | muonpi/log/user/0 || 2021-01-26_22-08-43 || freqAccuracy || 265.0000000 || ps/s || | ||
|- | |- | ||
| − | | muonpi/log/user/0 || 2021-01-26_22-08-43 || geoHash || u0ypy122vb || || GeoHash | + | | muonpi/log/user/0 || 2021-01-26_22-08-43 || geoHash || u0ypy122vb || || [[wikipedia:GeoHash|GeoHash]] |
|- | |- | ||
| muonpi/log/user/0 || 2021-01-26_22-08-43 || geoHeightMSL || 177.8400000 || m || Height above mean sea level | | muonpi/log/user/0 || 2021-01-26_22-08-43 || geoHeightMSL || 177.8400000 || m || Height above mean sea level | ||
| Line 145: | Line 165: | ||
| muonpi/log/v/0 || 2021-01-26_22-08-43 || vbias || 31.6119833 || V || SiPM bias voltage | | muonpi/log/v/0 || 2021-01-26_22-08-43 || vbias || 31.6119833 || V || SiPM bias voltage | ||
|- | |- | ||
| − | | muonpi/log/user/0 || 2021-01-26_22-08-43 || vsense || 0.2405174 || V || Voltage | + | | muonpi/log/user/0 || 2021-01-26_22-08-43 || vsense || 0.2405174 || V || Voltage across sensing resistor |
|} | |} | ||
| − | |||
| − | |||
| − | |||
Latest revision as of 15:38, 14 November 2024
Contents
Hardware Resources
CPU
The majority of CPU utilization associated with the muondetector-daemon is due to the signal transfer to and from the MuonPi HAT via the GPIO-pins. As of v2.0.2, this is handled by the "pigpiod" process, which takes about 20% of the processor time according to top. The muondetector-daemon itself, uses only ~2%.
Network Resources
If you wish to operate the MuonPi within a network, that is a company's or other organization's network, the admins might be interested what data this device gathers and transmits or receives. Here we elaborate on how the MuonPi uses the network and how a typical data packet looks like.
Data rate
The MuonPi detector board will send measurement data to our data evaluation servers, the rate of which greatly depends on the exact event rate. In a proper setup these messages will average out to roughly 600 messages per minute with an average size of 110 bytes. This however includes the acknowledgement packets which account for two thirds of the number of packets but they are only small in size (>150 bytes).
Outgoing Ports
| Source Port | Purpose | Destination |
|---|---|---|
| muonpi:51508 | GUI connection to other machine on network | othermachine:39830 |
| muonpi:41930 | MQTT message publishing to muonpi.org | muonpi.org:1883 |
(Connections to localhost are excluded here)
As indicated by the table, the firewall has to allow outgoing connections to our servers on port 1883. Otherwise the muondetector-login command will fail to establish the data link, and no event data will be sent to contribute to the detector network.
MQTT Packet
Detector telemetry and measurement data is send to our servers via the MQTT protocol. Upon receiving a MQTT message from the client our servers respond with an MQTT acknowledgment packet which is in turn acknowledged by the client.
Generally two types of MQTT messages are sent: the event message and the log message.
Event message
The event messages consist of the following:
| Type | Size |
|---|---|
| Ethernet frame | 14 byte |
| IP4 Header | 20 byte |
| TCP Header | 32 byte |
| MQTT message | 84 byte |
| Total | 150 byte |
The only privacy sensitive information contained in these messages are the geolocation and ambient temperature.
Example data of multiple MQTT packets being send to our servers:
| # | MQTT topic | timestamp (rising edge) | timestamp (falling edge) | - | Event counter | - | - | - |
|---|---|---|---|---|---|---|---|---|
| 1 | muonpi/data/<user>/<detectorID> | 1611698307.895514697 | 1611698307.895515885 | 70 | 35109 | 1 | 2 | 1 |
| 2 | muonpi/data/<user>/<detectorID> | 1611698307.912153060 | 1611698307.912154247 | 70 | 35110 | 1 | 2 | 1 |
| 3 | muonpi/data/<user>/<detectorID> | 1611698308.237550951 | 1611698308.237552160 | 70 | 35111 | 1 | 2 | 1 |
| 4 | muonpi/data/<user>/<detectorID> | 1611698308.404480192 | 1611698308.404481380 | 70 | 35112 | 1 | 2 | 1 |
| 5 | muonpi/data/<user>/<detectorID> | 1611698308.699026112 | 1611698308.699027299 | 70 | 35113 | 1 | 2 | 1 |
| 6 | muonpi/data/<user>/<detectorID> | 1611698309.048519516 | 1611698309.048520724 | 70 | 35114 | 1 | 2 | 1 |
| 7 | muonpi/data/<user>/<detectorID> | 1611698309.133318449 | 1611698309.133319657 | 70 | 35115 | 1 | 2 | 1 |
These event messages are sent every time the unit detects a signal above the threshold. It contains the UNIX timestamp of the rising and the falling edge of the signal and the number of the event. The MQTT topic includes the username and the detectorID as registered with us, and allows us to associate the incoming data on our servers to your detector.
Log message
The log message contains various metadata about your MuonPi setup. Most of the time, it consist of multiple MQTT messages packed into a single TCP packets, but it can sometimes also be separated into two packets.
An exemplary log message along with a description of each entry:
| MQTT topic | Date and Time | Variable | Value | Unit | Description |
|---|---|---|---|---|---|
| muonpi/log/user/0 | 2021-01-26_22-08-43 | RXBufUsage | 0.0000000 | % | Buffer usage incoming messages |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | TXBufUsage | 0.0000000 | % | Buffer usage outgoing messages |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | adcSamplingTime | 2.4608333 | ms | |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | antennaPower | 1.0000000 | ||
| muonpi/log/user/0 | 2021-01-26_22-08-43 | antennaStatus | 2 | ||
| muonpi/log/user/0 | 2021-01-26_22-08-43 | clockBias | -319713.3333333 | ns | Offset of GNSS clock |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | clockDrift | -440.3333333 | ns/s | Drift of GNSS clock |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | fixStatus | 3 | GNSS fix status | |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | fixStatusString | 3D-Fix | ms | GNSS fix status |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | freqAccuracy | 265.0000000 | ps/s | |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | geoHash | u0ypy122vb | GeoHash | |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | geoHeightMSL | 177.8400000 | m | Height above mean sea level |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | geoHorAccuracy | 1.2220000 | m | Horizontal GNSS accuracy |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | geoLatitude | 50.5000000 | deg | Location latitude |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | geoLongitude | 8.7000000 | deg | Location longitude |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | geoVertAccuracy | 0.8540000 | m | Vertical GPS accuracy |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | ibias | 10.2492675 | uA | Bias current for SiPMs |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | jammingLevel | 9.0000000 | % | Jamming level of GPS antenna |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | maxCNR | 43.0000000 | dB | Carrier to noise ratio |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | maxRXBufUsage | 25 | % | maximum buffer usage for incoming messages |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | maxTXBufUsage | 4 | % | maximum buffer usage for outgoing messages |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | meanGeoHeightMSL | 185.35 | m | mean height above sea level |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | positionDOP | 1.7333333 | position dilution of precision | |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | preampAGC | 1482.0000000 | ||
| muonpi/log/user/0 | 2021-01-26_22-08-43 | preampNoise | -87.0000000 | dBHz | noise level of SiPM preamp |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | rateAND | 0.0000000 | Hz | event rate (AND) |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | rateXOR | 3.0525173 | Hz | event rate (XOR) |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | sats | 20.7777778 | number of GNSS satellites | |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | systemFreeMem | 815.3660000 | Mb | free memory space in host system |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | systemFreeSwap | 104.8540000 | Mb | free swap space in host system |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | systemLoadAvg | 0.2758790 | average CPU load | |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | systemUptime | 2.6875 | h | uptime |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | temperature | 41.5104167 | degC | Temperature (on PCB) |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | timeAccuracy | 31.5000000 | ns | event timing accuracy |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | timeDOP | 0.9233333 | timing dilution of precision | |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | ubloxUptime | 9771 | s | UBlox uptime in seconds |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | usedSats | 11.6666667 | number of satellites in use | |
| muonpi/log/v/0 | 2021-01-26_22-08-43 | vbias | 31.6119833 | V | SiPM bias voltage |
| muonpi/log/user/0 | 2021-01-26_22-08-43 | vsense | 0.2405174 | V | Voltage across sensing resistor |
