Difference between revisions of "Resource usage"

From MuonPi-Wiki
Jump to navigation Jump to search
(Restructuring for more comprehensive info on Resources in the future)
 
(7 intermediate revisions by one other user not shown)
Line 1: Line 1:
 
== Hardware Resources ==
 
== Hardware Resources ==
 
=== CPU ===
 
=== CPU ===
−
=== RAM ===
+
The majority of CPU utilization associated with the muondetector-daemon is due to the signal transfer to and from the MuonPi HAT via the GPIO-pins. As of [https://github.com/MuonPi/muondetector/releases/tag/v2.0.2 v2.0.2], this is handled by the "pigpiod" process, which takes about 20% of the processor time according to [[wikipedia:top (software)|top]]. The muondetector-daemon itself, uses only ~2%.
−
=== Disk I/O ===
 
  
 
== Network Resources ==
 
== Network Resources ==
 +
If you wish to operate the MuonPi within a network, that is a company's or other organization's network, the admins might be interested what data this device gathers and transmits or receives.
 +
Here we elaborate on how the MuonPi uses the network and how a typical data packet looks like.
 +
 
=== Data rate ===
 
=== Data rate ===
 
The MuonPi detector board will send measurement data to our data evaluation servers, the rate of which greatly depends on the exact event rate.
 
The MuonPi detector board will send measurement data to our data evaluation servers, the rate of which greatly depends on the exact event rate.
 
In a proper setup these messages will average out to roughly 600 messages per minute with an average size of 110 bytes.
 
In a proper setup these messages will average out to roughly 600 messages per minute with an average size of 110 bytes.
 
This however includes the acknowledgement packets which account for two thirds of the number of packets but they are only small in size (>150 bytes).
 
This however includes the acknowledgement packets which account for two thirds of the number of packets but they are only small in size (>150 bytes).
−
[[File:Muonpi-iorate.png |thumb|Purple:All messages send between the MuonPi unit and muonpi.org <br>
+
[[File:Muonpi-iorate.png |thumb|Purple: All messages send between the MuonPi unit and muonpi.org <br>
 
Green: Only MQTT publish messages, excluding acknowledgement messages]]
 
Green: Only MQTT publish messages, excluding acknowledgement messages]]
−
[[File:Muonpi-packetsize.png|thumb|Histogram of packetsizes over 45minutes]]
+
[[File:Muonpi-packetsize.png|thumb|Histogram of packet sizes over 45minutes]]
  
−
=== Used Ports ===
+
=== Outgoing Ports ===
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
−
! Port !! Application
+
! Source Port !! Purpose !! Destination
−
|-
 
−
| 51508 || GUI connection
 
 
|-
 
|-
−
| 8888 || pigpiod
+
| muonpi:51508 || GUI connection to other machine on network || othermachine:39830
 
|-
 
|-
−
| 1883 (out) || MQTT message publish
+
| muonpi:41930 || MQTT message publishing to muonpi.org || muonpi.org:1883
 
|}
 
|}
 +
 +
(Connections to localhost are excluded here)
 +
 +
As indicated by the table, the firewall has to allow outgoing connections to our servers on port 1883.
 +
Otherwise the muondetector-login command will fail to establish the data link, and no event data will be sent to contribute to the detector network.
  
 
=== MQTT Packet ===
 
=== MQTT Packet ===
−
Detector telemetry and mesurement data is send to our servers via the MQTT protocol.
+
Detector telemetry and measurement data is send to our servers via the MQTT protocol.
−
Upon receiving a MQTT message from the client our servers respond with an MQTT acknowledgment packet which is in turn acknowledged by the client with a short TCP message.
+
Upon receiving a MQTT message from the client our servers respond with an MQTT acknowledgment packet which is in turn acknowledged by the client.
  
−
Generally two types of MQTT messages are send, the event message and the log message.
+
Generally two types of MQTT messages are sent: the event message and the log message.
  
 +
====Event message====
 
The event messages consist of the following:
 
The event messages consist of the following:
 
{| class="wikitable"
 
{| class="wikitable"
Line 47: Line 53:
 
|}
 
|}
  
−
The log messages consist of multiple MQTT messages packed into a single (sometimes two seperate) TCP packets
 
  
−
Example data of multiple MQTT packets being send to our data evaluation servers:
+
The only privacy sensitive information contained in these messages are the geolocation and ambient temperature.
 +
 
 +
Example data of multiple MQTT packets being send to our servers:
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
−
! # !!MQTT topic !! timestamp (rising edge) !! timestamp (falling edge) !! Header text !! Event counter !! Header text !! Header text !! Header text
+
! # !!MQTT topic !! timestamp (rising edge) !! timestamp (falling edge) !! - !! Event counter !! - !! - !! -
 
|-
 
|-
−
| 1 || muonpi/data/user/0 || 1611698307.895514697 || 1611698307.895515885 || 70 || 35109 || 1 || 2 || 1
+
| 1 || muonpi/data/<user>/<detectorID> || 1611698307.895514697 || 1611698307.895515885 || 70 || 35109 || 1 || 2 || 1
 
|-
 
|-
−
| 2 || muonpi/data/user/0 || 1611698307.912153060 || 1611698307.912154247 || 70 || 35110 || 1 || 2 || 1
+
| 2 || muonpi/data/<user>/<detectorID> || 1611698307.912153060 || 1611698307.912154247 || 70 || 35110 || 1 || 2 || 1
 
|-
 
|-
−
| 3 || muonpi/data/user/0 || 1611698308.237550951 || 1611698308.237552160 || 70 || 35111 || 1 || 2 || 1
+
| 3 || muonpi/data/<user>/<detectorID> || 1611698308.237550951 || 1611698308.237552160 || 70 || 35111 || 1 || 2 || 1
 
|-
 
|-
−
| 4 || muonpi/data/user/0 || 1611698308.404480192 || 1611698308.404481380 || 70 || 35112 || 1 || 2 || 1
+
| 4 || muonpi/data/<user>/<detectorID> || 1611698308.404480192 || 1611698308.404481380 || 70 || 35112 || 1 || 2 || 1
 
|-
 
|-
−
| 5 || muonpi/data/user/0 || 1611698308.699026112 || 1611698308.699027299 || 70 || 35113 || 1 || 2 || 1
+
| 5 || muonpi/data/<user>/<detectorID> || 1611698308.699026112 || 1611698308.699027299 || 70 || 35113 || 1 || 2 || 1
 
|-
 
|-
−
| 6 || muonpi/data/user/0 || 1611698309.048519516 || 1611698309.048520724 || 70 || 35114 || 1 || 2 || 1
+
| 6 || muonpi/data/<user>/<detectorID> || 1611698309.048519516 || 1611698309.048520724 || 70 || 35114 || 1 || 2 || 1
 
|-
 
|-
−
| 7 || muonpi/data/user/0 || 1611698309.133318449 || 1611698309.133319657 || 70 || 35115 || 1 || 2 || 1
+
| 7 || muonpi/data/<user>/<detectorID> || 1611698309.133318449 || 1611698309.133319657 || 70 || 35115 || 1 || 2 || 1
 
|}
 
|}
 +
 +
These event messages are sent every time the unit detects a signal above the threshold.
 +
It contains the UNIX timestamp of the rising and the falling edge of the signal and the number of the event.
 +
The MQTT topic includes the username and the detectorID as registered with us, and allows us to associate the incoming data on our servers to your detector.
  
 
==== Log message ====
 
==== Log message ====
−
The log message contains various meta data about your MuonPi setup.
+
The log message contains various metadata about your MuonPi setup.
 +
Most of the time, it consist of multiple MQTT messages packed into a single TCP packets, but it can sometimes also be separated into two packets.
  
−
Example log message:
+
An exemplary log message along with a description of each entry:
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
Line 97: Line 109:
 
| muonpi/log/user/0 || 2021-01-26_22-08-43 || freqAccuracy            || 265.0000000  ||  ps/s              ||
 
| muonpi/log/user/0 || 2021-01-26_22-08-43 || freqAccuracy            || 265.0000000  ||  ps/s              ||
 
|-
 
|-
−
| muonpi/log/user/0 || 2021-01-26_22-08-43 || geoHash                || u0ypy122vb    ||                    || GeoHash
+
| muonpi/log/user/0 || 2021-01-26_22-08-43 || geoHash                || u0ypy122vb    ||                    || [[wikipedia:GeoHash|GeoHash]]
 
|-
 
|-
 
| muonpi/log/user/0 || 2021-01-26_22-08-43 || geoHeightMSL            || 177.8400000  || m      || Height above mean sea level
 
| muonpi/log/user/0 || 2021-01-26_22-08-43 || geoHeightMSL            || 177.8400000  || m      || Height above mean sea level
Line 153: Line 165:
 
| muonpi/log/v/0 || 2021-01-26_22-08-43 || vbias                  || 31.6119833    || V      || SiPM bias voltage
 
| muonpi/log/v/0 || 2021-01-26_22-08-43 || vbias                  || 31.6119833    || V      || SiPM bias voltage
 
|-
 
|-
−
| muonpi/log/user/0 || 2021-01-26_22-08-43 || vsense                  || 0.2405174    || V      || Voltage accross sensing resistor
+
| muonpi/log/user/0 || 2021-01-26_22-08-43 || vsense                  || 0.2405174    || V      || Voltage across sensing resistor
 
|}
 
|}
−
 
−
 
−
The only privacy sensitive information contained in these messages are the geolocation and ambient temperature. (ellaborate in [[Privacy]])
 

Latest revision as of 15:38, 14 November 2024

Hardware Resources

CPU

The majority of CPU utilization associated with the muondetector-daemon is due to the signal transfer to and from the MuonPi HAT via the GPIO-pins. As of v2.0.2, this is handled by the "pigpiod" process, which takes about 20% of the processor time according to top. The muondetector-daemon itself, uses only ~2%.

Network Resources

If you wish to operate the MuonPi within a network, that is a company's or other organization's network, the admins might be interested what data this device gathers and transmits or receives. Here we elaborate on how the MuonPi uses the network and how a typical data packet looks like.

Data rate

The MuonPi detector board will send measurement data to our data evaluation servers, the rate of which greatly depends on the exact event rate. In a proper setup these messages will average out to roughly 600 messages per minute with an average size of 110 bytes. This however includes the acknowledgement packets which account for two thirds of the number of packets but they are only small in size (>150 bytes).

Purple: All messages send between the MuonPi unit and muonpi.org
Green: Only MQTT publish messages, excluding acknowledgement messages
Histogram of packet sizes over 45minutes

Outgoing Ports

Source Port Purpose Destination
muonpi:51508 GUI connection to other machine on network othermachine:39830
muonpi:41930 MQTT message publishing to muonpi.org muonpi.org:1883

(Connections to localhost are excluded here)

As indicated by the table, the firewall has to allow outgoing connections to our servers on port 1883. Otherwise the muondetector-login command will fail to establish the data link, and no event data will be sent to contribute to the detector network.

MQTT Packet

Detector telemetry and measurement data is send to our servers via the MQTT protocol. Upon receiving a MQTT message from the client our servers respond with an MQTT acknowledgment packet which is in turn acknowledged by the client.

Generally two types of MQTT messages are sent: the event message and the log message.

Event message

The event messages consist of the following:

Type Size
Ethernet frame 14 byte
IP4 Header 20 byte
TCP Header 32 byte
MQTT message 84 byte
Total 150 byte


The only privacy sensitive information contained in these messages are the geolocation and ambient temperature.

Example data of multiple MQTT packets being send to our servers:

# MQTT topic timestamp (rising edge) timestamp (falling edge) - Event counter - - -
1 muonpi/data/<user>/<detectorID> 1611698307.895514697 1611698307.895515885 70 35109 1 2 1
2 muonpi/data/<user>/<detectorID> 1611698307.912153060 1611698307.912154247 70 35110 1 2 1
3 muonpi/data/<user>/<detectorID> 1611698308.237550951 1611698308.237552160 70 35111 1 2 1
4 muonpi/data/<user>/<detectorID> 1611698308.404480192 1611698308.404481380 70 35112 1 2 1
5 muonpi/data/<user>/<detectorID> 1611698308.699026112 1611698308.699027299 70 35113 1 2 1
6 muonpi/data/<user>/<detectorID> 1611698309.048519516 1611698309.048520724 70 35114 1 2 1
7 muonpi/data/<user>/<detectorID> 1611698309.133318449 1611698309.133319657 70 35115 1 2 1

These event messages are sent every time the unit detects a signal above the threshold. It contains the UNIX timestamp of the rising and the falling edge of the signal and the number of the event. The MQTT topic includes the username and the detectorID as registered with us, and allows us to associate the incoming data on our servers to your detector.

Log message

The log message contains various metadata about your MuonPi setup. Most of the time, it consist of multiple MQTT messages packed into a single TCP packets, but it can sometimes also be separated into two packets.

An exemplary log message along with a description of each entry:

MQTT topic Date and Time Variable Value Unit Description
muonpi/log/user/0 2021-01-26_22-08-43 RXBufUsage 0.0000000 % Buffer usage incoming messages
muonpi/log/user/0 2021-01-26_22-08-43 TXBufUsage 0.0000000 % Buffer usage outgoing messages
muonpi/log/user/0 2021-01-26_22-08-43 adcSamplingTime 2.4608333 ms
muonpi/log/user/0 2021-01-26_22-08-43 antennaPower 1.0000000
muonpi/log/user/0 2021-01-26_22-08-43 antennaStatus 2
muonpi/log/user/0 2021-01-26_22-08-43 clockBias -319713.3333333 ns Offset of GNSS clock
muonpi/log/user/0 2021-01-26_22-08-43 clockDrift -440.3333333 ns/s Drift of GNSS clock
muonpi/log/user/0 2021-01-26_22-08-43 fixStatus 3 GNSS fix status
muonpi/log/user/0 2021-01-26_22-08-43 fixStatusString 3D-Fix ms GNSS fix status
muonpi/log/user/0 2021-01-26_22-08-43 freqAccuracy 265.0000000 ps/s
muonpi/log/user/0 2021-01-26_22-08-43 geoHash u0ypy122vb GeoHash
muonpi/log/user/0 2021-01-26_22-08-43 geoHeightMSL 177.8400000 m Height above mean sea level
muonpi/log/user/0 2021-01-26_22-08-43 geoHorAccuracy 1.2220000 m Horizontal GNSS accuracy
muonpi/log/user/0 2021-01-26_22-08-43 geoLatitude 50.5000000 deg Location latitude
muonpi/log/user/0 2021-01-26_22-08-43 geoLongitude 8.7000000 deg Location longitude
muonpi/log/user/0 2021-01-26_22-08-43 geoVertAccuracy 0.8540000 m Vertical GPS accuracy
muonpi/log/user/0 2021-01-26_22-08-43 ibias 10.2492675 uA Bias current for SiPMs
muonpi/log/user/0 2021-01-26_22-08-43 jammingLevel 9.0000000 % Jamming level of GPS antenna
muonpi/log/user/0 2021-01-26_22-08-43 maxCNR 43.0000000 dB Carrier to noise ratio
muonpi/log/user/0 2021-01-26_22-08-43 maxRXBufUsage 25 % maximum buffer usage for incoming messages
muonpi/log/user/0 2021-01-26_22-08-43 maxTXBufUsage 4 % maximum buffer usage for outgoing messages
muonpi/log/user/0 2021-01-26_22-08-43 meanGeoHeightMSL 185.35 m mean height above sea level
muonpi/log/user/0 2021-01-26_22-08-43 positionDOP 1.7333333 position dilution of precision
muonpi/log/user/0 2021-01-26_22-08-43 preampAGC 1482.0000000
muonpi/log/user/0 2021-01-26_22-08-43 preampNoise -87.0000000 dBHz noise level of SiPM preamp
muonpi/log/user/0 2021-01-26_22-08-43 rateAND 0.0000000 Hz event rate (AND)
muonpi/log/user/0 2021-01-26_22-08-43 rateXOR 3.0525173 Hz event rate (XOR)
muonpi/log/user/0 2021-01-26_22-08-43 sats 20.7777778 number of GNSS satellites
muonpi/log/user/0 2021-01-26_22-08-43 systemFreeMem 815.3660000 Mb free memory space in host system
muonpi/log/user/0 2021-01-26_22-08-43 systemFreeSwap 104.8540000 Mb free swap space in host system
muonpi/log/user/0 2021-01-26_22-08-43 systemLoadAvg 0.2758790 average CPU load
muonpi/log/user/0 2021-01-26_22-08-43 systemUptime 2.6875 h uptime
muonpi/log/user/0 2021-01-26_22-08-43 temperature 41.5104167 degC Temperature (on PCB)
muonpi/log/user/0 2021-01-26_22-08-43 timeAccuracy 31.5000000 ns event timing accuracy
muonpi/log/user/0 2021-01-26_22-08-43 timeDOP 0.9233333 timing dilution of precision
muonpi/log/user/0 2021-01-26_22-08-43 ubloxUptime 9771 s UBlox uptime in seconds
muonpi/log/user/0 2021-01-26_22-08-43 usedSats 11.6666667 number of satellites in use
muonpi/log/v/0 2021-01-26_22-08-43 vbias 31.6119833 V SiPM bias voltage
muonpi/log/user/0 2021-01-26_22-08-43 vsense 0.2405174 V Voltage across sensing resistor
Retrieved from ‘https://wiki.muonpi.org/index.php?title=Resource_usage&oldid=813’